Merge branch 'for-linus' of git://git.kernel.dk/linux-block
Pull block layer fixes from Jens Axboe:
"Final small batch of fixes to be included before -rc1. Some general
cleanups in here as well, but some of the blk-mq fixes we need for the
NVMe conversion and/or scsi-mq. The pull request contains:
- Support for not merging across a specified "chunk size", if set by
the driver. Some NVMe devices perform poorly for IO that crosses
such a chunk, so we need to support it generically as part of
request merging avoid having to do complicated split logic. From
me.
- Bump max tag depth to 10Ki tags. Some scsi devices have a huge
shared tag space. Before we failed with EINVAL if a too large tag
depth was specified, now we truncate it and pass back the actual
value. From me.
- Various blk-mq rq init fixes from me and others.
- A fix for enter on a dying queue for blk-mq from Keith. This is
needed to prevent oopsing on hot device removal.
- Fixup for blk-mq timer addition from Ming Lei.
- Small round of performance fixes for mtip32xx from Sam Bradshaw.
- Minor stack leak fix from Rickard Strandqvist.
- Two __init annotations from Fabian Frederick"
* 'for-linus' of git://git.kernel.dk/linux-block:
block: add __init to blkcg_policy_register
block: add __init to elv_register
block: ensure that bio_add_page() always accepts a page for an empty bio
blk-mq: add timer in blk_mq_start_request
blk-mq: always initialize request->start_time
block: blk-exec.c: Cleaning up local variable address returnd
mtip32xx: minor performance enhancements
blk-mq: ->timeout should be cleared in blk_mq_rq_ctx_init()
blk-mq: don't allow queue entering for a dying queue
blk-mq: bump max tag depth to 10K tags
block: add blk_rq_set_block_pc()
block: add notion of a chunk size for request merging
diff --git a/Documentation/ABI/stable/sysfs-devices-system-cpu b/Documentation/ABI/stable/sysfs-devices-system-cpu
new file mode 100644
index 0000000..33c133e
--- /dev/null
+++ b/Documentation/ABI/stable/sysfs-devices-system-cpu
@@ -0,0 +1,25 @@
+What: /sys/devices/system/cpu/dscr_default
+Date: 13-May-2014
+KernelVersion: v3.15.0
+Contact:
+Description: Writes are equivalent to writing to
+ /sys/devices/system/cpu/cpuN/dscr on all CPUs.
+ Reads return the last written value or 0.
+ This value is not a global default: it is a way to set
+ all per-CPU defaults at the same time.
+Values: 64 bit unsigned integer (bit field)
+
+What: /sys/devices/system/cpu/cpu[0-9]+/dscr
+Date: 13-May-2014
+KernelVersion: v3.15.0
+Contact:
+Description: Default value for the Data Stream Control Register (DSCR) on
+ a CPU.
+ This default value is used when the kernel is executing and
+ for any process that has not set the DSCR itself.
+ If a process ever sets the DSCR (via direct access to the
+ SPR) that value will be persisted for that process and used
+ on any CPU where it executes (overriding the value described
+ here).
+ If set by a process it will be inherited by child processes.
+Values: 64 bit unsigned integer (bit field)
diff --git a/Documentation/ABI/testing/ima_policy b/Documentation/ABI/testing/ima_policy
index f1c5cc9..4c3efe4 100644
--- a/Documentation/ABI/testing/ima_policy
+++ b/Documentation/ABI/testing/ima_policy
@@ -23,7 +23,7 @@
[fowner]]
lsm: [[subj_user=] [subj_role=] [subj_type=]
[obj_user=] [obj_role=] [obj_type=]]
- option: [[appraise_type=]]
+ option: [[appraise_type=]] [permit_directio]
base: func:= [BPRM_CHECK][MMAP_CHECK][FILE_CHECK][MODULE_CHECK]
mask:= [MAY_READ] [MAY_WRITE] [MAY_APPEND] [MAY_EXEC]
diff --git a/Documentation/SubmittingPatches b/Documentation/SubmittingPatches
index 2a8e89e..7e9abb8 100644
--- a/Documentation/SubmittingPatches
+++ b/Documentation/SubmittingPatches
@@ -132,6 +132,20 @@
platform_set_drvdata(), but left the variable "dev" unused,
delete it.
+If your patch fixes a bug in a specific commit, e.g. you found an issue using
+git-bisect, please use the 'Fixes:' tag with the first 12 characters of the
+SHA-1 ID, and the one line summary.
+Example:
+
+ Fixes: e21d2170f366 ("video: remove unnecessary platform_set_drvdata()")
+
+The following git-config settings can be used to add a pretty format for
+outputting the above style in the git log or git show commands
+
+ [core]
+ abbrev = 12
+ [pretty]
+ fixes = Fixes: %h (\"%s\")
3) Separate your changes.
@@ -443,7 +457,7 @@
have been included in the discussion
-14) Using Reported-by:, Tested-by:, Reviewed-by: and Suggested-by:
+14) Using Reported-by:, Tested-by:, Reviewed-by:, Suggested-by: and Fixes:
If this patch fixes a problem reported by somebody else, consider adding a
Reported-by: tag to credit the reporter for their contribution. Please
@@ -498,6 +512,12 @@
idea reporters, they will, hopefully, be inspired to help us again in the
future.
+A Fixes: tag indicates that the patch fixes an issue in a previous commit. It
+is used to make it easy to determine where a bug originated, which can help
+review a bug fix. This tag also assists the stable kernel team in determining
+which stable kernel versions should receive your fix. This is the preferred
+method for indicating a bug fixed by the patch. See #2 above for more details.
+
15) The canonical patch format
diff --git a/Documentation/arm/00-INDEX b/Documentation/arm/00-INDEX
index a94090c..3b08bc2 100644
--- a/Documentation/arm/00-INDEX
+++ b/Documentation/arm/00-INDEX
@@ -46,5 +46,7 @@
- SWP/SWPB emulation handler/logging description
tcm.txt
- ARM Tightly Coupled Memory
+uefi.txt
+ - [U]EFI configuration and runtime services documentation
vlocks.txt
- Voting locks, low-level mechanism relying on memory system atomic writes.
diff --git a/Documentation/arm/memory.txt b/Documentation/arm/memory.txt
index 4bfb9ff..38dc06d 100644
--- a/Documentation/arm/memory.txt
+++ b/Documentation/arm/memory.txt
@@ -41,16 +41,9 @@
fffe0000 fffe7fff ITCM mapping area for platforms with
ITCM mounted inside the CPU.
-fff00000 fffdffff Fixmap mapping region. Addresses provided
+ffc00000 ffdfffff Fixmap mapping region. Addresses provided
by fix_to_virt() will be located here.
-ffc00000 ffefffff DMA memory mapping region. Memory returned
- by the dma_alloc_xxx functions will be
- dynamically mapped here.
-
-ff000000 ffbfffff Reserved for future expansion of DMA
- mapping region.
-
fee00000 feffffff Mapping of PCI I/O space. This is a static
mapping within the vmalloc space.
diff --git a/Documentation/arm/uefi.txt b/Documentation/arm/uefi.txt
new file mode 100644
index 0000000..d60030a
--- /dev/null
+++ b/Documentation/arm/uefi.txt
@@ -0,0 +1,64 @@
+UEFI, the Unified Extensible Firmware Interface, is a specification
+governing the behaviours of compatible firmware interfaces. It is
+maintained by the UEFI Forum - http://www.uefi.org/.
+
+UEFI is an evolution of its predecessor 'EFI', so the terms EFI and
+UEFI are used somewhat interchangeably in this document and associated
+source code. As a rule, anything new uses 'UEFI', whereas 'EFI' refers
+to legacy code or specifications.
+
+UEFI support in Linux
+=====================
+Booting on a platform with firmware compliant with the UEFI specification
+makes it possible for the kernel to support additional features:
+- UEFI Runtime Services
+- Retrieving various configuration information through the standardised
+ interface of UEFI configuration tables. (ACPI, SMBIOS, ...)
+
+For actually enabling [U]EFI support, enable:
+- CONFIG_EFI=y
+- CONFIG_EFI_VARS=y or m
+
+The implementation depends on receiving information about the UEFI environment
+in a Flattened Device Tree (FDT) - so is only available with CONFIG_OF.
+
+UEFI stub
+=========
+The "stub" is a feature that extends the Image/zImage into a valid UEFI
+PE/COFF executable, including a loader application that makes it possible to
+load the kernel directly from the UEFI shell, boot menu, or one of the
+lightweight bootloaders like Gummiboot or rEFInd.
+
+The kernel image built with stub support remains a valid kernel image for
+booting in non-UEFI environments.
+
+UEFI kernel support on ARM
+==========================
+UEFI kernel support on the ARM architectures (arm and arm64) is only available
+when boot is performed through the stub.
+
+When booting in UEFI mode, the stub deletes any memory nodes from a provided DT.
+Instead, the kernel reads the UEFI memory map.
+
+The stub populates the FDT /chosen node with (and the kernel scans for) the
+following parameters:
+________________________________________________________________________________
+Name | Size | Description
+================================================================================
+linux,uefi-system-table | 64-bit | Physical address of the UEFI System Table.
+--------------------------------------------------------------------------------
+linux,uefi-mmap-start | 64-bit | Physical address of the UEFI memory map,
+ | | populated by the UEFI GetMemoryMap() call.
+--------------------------------------------------------------------------------
+linux,uefi-mmap-size | 32-bit | Size in bytes of the UEFI memory map
+ | | pointed to in previous entry.
+--------------------------------------------------------------------------------
+linux,uefi-mmap-desc-size | 32-bit | Size in bytes of each entry in the UEFI
+ | | memory map.
+--------------------------------------------------------------------------------
+linux,uefi-mmap-desc-ver | 32-bit | Version of the mmap descriptor format.
+--------------------------------------------------------------------------------
+linux,uefi-stub-kern-ver | string | Copy of linux_banner from build.
+--------------------------------------------------------------------------------
+
+For verbose debug messages, specify 'uefi_debug' on the kernel command line.
diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
index beb754e8..37fc4f6 100644
--- a/Documentation/arm64/booting.txt
+++ b/Documentation/arm64/booting.txt
@@ -85,6 +85,10 @@
Header notes:
- code0/code1 are responsible for branching to stext.
+- when booting through EFI, code0/code1 are initially skipped.
+ res5 is an offset to the PE header and the PE header has the EFI
+ entry point (efi_stub_entry). When the stub has done its work, it
+ jumps to code0 to resume the normal boot process.
The image must be placed at the specified offset (currently 0x80000)
from the start of the system RAM and called there. The start of the
diff --git a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt
index 4937e6f..02ab997 100644
--- a/Documentation/cgroups/memory.txt
+++ b/Documentation/cgroups/memory.txt
@@ -458,15 +458,11 @@
5.1 force_empty
memory.force_empty interface is provided to make cgroup's memory usage empty.
- You can use this interface only when the cgroup has no tasks.
When writing anything to this
# echo 0 > memory.force_empty
- Almost all pages tracked by this memory cgroup will be unmapped and freed.
- Some pages cannot be freed because they are locked or in-use. Such pages are
- moved to parent (if use_hierarchy==1) or root (if use_hierarchy==0) and this
- cgroup will be empty.
+ the cgroup will be reclaimed and as many pages reclaimed as possible.
The typical use case for this interface is before calling rmdir().
Because rmdir() moves all pages to parent, some out-of-use page caches can be
@@ -540,14 +536,13 @@
5.3 swappiness
-Similar to /proc/sys/vm/swappiness, but only affecting reclaim that is
-triggered by this cgroup's hard limit. The tunable in the root cgroup
-corresponds to the global swappiness setting.
+Overrides /proc/sys/vm/swappiness for the particular group. The tunable
+in the root cgroup corresponds to the global swappiness setting.
-Please note that unlike the global swappiness, memcg knob set to 0
-really prevents from any swapping even if there is a swap storage
-available. This might lead to memcg OOM killer if there are no file
-pages to reclaim.
+Please note that unlike during the global reclaim, limit reclaim
+enforces that 0 swappiness really prevents from any swapping even if
+there is a swap storage available. This might lead to memcg OOM killer
+if there are no file pages to reclaim.
5.4 failcnt
diff --git a/Documentation/cgroups/unified-hierarchy.txt b/Documentation/cgroups/unified-hierarchy.txt
new file mode 100644
index 0000000..324b182
--- /dev/null
+++ b/Documentation/cgroups/unified-hierarchy.txt
@@ -0,0 +1,359 @@
+
+Cgroup unified hierarchy
+
+April, 2014 Tejun Heo <tj@kernel.org>
+
+This document describes the changes made by unified hierarchy and
+their rationales. It will eventually be merged into the main cgroup
+documentation.
+
+CONTENTS
+
+1. Background
+2. Basic Operation
+ 2-1. Mounting
+ 2-2. cgroup.subtree_control
+ 2-3. cgroup.controllers
+3. Structural Constraints
+ 3-1. Top-down
+ 3-2. No internal tasks
+4. Other Changes
+ 4-1. [Un]populated Notification
+ 4-2. Other Core Changes
+ 4-3. Per-Controller Changes
+ 4-3-1. blkio
+ 4-3-2. cpuset
+ 4-3-3. memory
+5. Planned Changes
+ 5-1. CAP for resource control
+
+
+1. Background
+
+cgroup allows an arbitrary number of hierarchies and each hierarchy
+can host any number of controllers. While this seems to provide a
+high level of flexibility, it isn't quite useful in practice.
+
+For example, as there is only one instance of each controller, utility
+type controllers such as freezer which can be useful in all
+hierarchies can only be used in one. The issue is exacerbated by the
+fact that controllers can't be moved around once hierarchies are
+populated. Another issue is that all controllers bound to a hierarchy
+are forced to have exactly the same view of the hierarchy. It isn't
+possible to vary the granularity depending on the specific controller.
+
+In practice, these issues heavily limit which controllers can be put
+on the same hierarchy and most configurations resort to putting each
+controller on its own hierarchy. Only closely related ones, such as
+the cpu and cpuacct controllers, make sense to put on the same
+hierarchy. This often means that userland ends up managing multiple
+similar hierarchies repeating the same steps on each hierarchy
+whenever a hierarchy management operation is necessary.
+
+Unfortunately, support for multiple hierarchies comes at a steep cost.
+Internal implementation in cgroup core proper is dazzlingly
+complicated but more importantly the support for multiple hierarchies
+restricts how cgroup is used in general and what controllers can do.
+
+There's no limit on how many hierarchies there may be, which means
+that a task's cgroup membership can't be described in finite length.
+The key may contain any varying number of entries and is unlimited in
+length, which makes it highly awkward to handle and leads to addition
+of controllers which exist only to identify membership, which in turn
+exacerbates the original problem.
+
+Also, as a controller can't have any expectation regarding what shape
+of hierarchies other controllers would be on, each controller has to
+assume that all other controllers are operating on completely
+orthogonal hierarchies. This makes it impossible, or at least very
+cumbersome, for controllers to cooperate with each other.
+
+In most use cases, putting controllers on hierarchies which are
+completely orthogonal to each other isn't necessary. What usually is
+called for is the ability to have differing levels of granularity
+depending on the specific controller. In other words, hierarchy may
+be collapsed from leaf towards root when viewed from specific
+controllers. For example, a given configuration might not care about
+how memory is distributed beyond a certain level while still wanting
+to control how CPU cycles are distributed.
+
+Unified hierarchy is the next version of cgroup interface. It aims to
+address the aforementioned issues by having more structure while
+retaining enough flexibility for most use cases. Various other
+general and controller-specific interface issues are also addressed in
+the process.
+
+
+2. Basic Operation
+
+2-1. Mounting
+
+Currently, unified hierarchy can be mounted with the following mount
+command. Note that this is still under development and scheduled to
+change soon.
+
+ mount -t cgroup -o __DEVEL__sane_behavior cgroup $MOUNT_POINT
+
+All controllers which are not bound to other hierarchies are
+automatically bound to unified hierarchy and show up at the root of
+it. Controllers which are enabled only in the root of unified
+hierarchy can be bound to other hierarchies at any time. This allows
+mixing unified hierarchy with the traditional multiple hierarchies in
+a fully backward compatible way.
+
+
+2-2. cgroup.subtree_control
+
+All cgroups on unified hierarchy have a "cgroup.subtree_control" file
+which governs which controllers are enabled on the children of the
+cgroup. Let's assume a hierarchy like the following.
+
+ root - A - B - C
+ \ D
+
+root's "cgroup.subtree_control" file determines which controllers are
+enabled on A. A's on B. B's on C and D. This coincides with the
+fact that controllers on the immediate sub-level are used to
+distribute the resources of the parent. In fact, it's natural to
+assume that resource control knobs of a child belong to its parent.
+Enabling a controller in a "cgroup.subtree_control" file declares that
+distribution of the respective resources of the cgroup will be
+controlled. Note that this means that controller enable states are
+shared among siblings.
+
+When read, the file contains a space-separated list of currently
+enabled controllers. A write to the file should contain a
+space-separated list of controllers with '+' or '-' prefixed (without
+the quotes). Controllers prefixed with '+' are enabled and '-'
+disabled. If a controller is listed multiple times, the last entry
+wins. The specific operations are executed atomically - either all
+succeed or fail.
+
+
+2-3. cgroup.controllers
+
+Read-only "cgroup.controllers" file contains a space-separated list of
+controllers which can be enabled in the cgroup's
+"cgroup.subtree_control" file.
+
+In the root cgroup, this lists controllers which are not bound to
+other hierarchies and the content changes as controllers are bound to
+and unbound from other hierarchies.
+
+In non-root cgroups, the content of this file equals that of the
+parent's "cgroup.subtree_control" file as only controllers enabled
+from the parent can be used in its children.
+
+
+3. Structural Constraints
+
+3-1. Top-down
+
+As it doesn't make sense to nest control of an uncontrolled resource,
+all non-root "cgroup.subtree_control" files can only contain
+controllers which are enabled in the parent's "cgroup.subtree_control"
+file. A controller can be enabled only if the parent has the
+controller enabled and a controller can't be disabled if one or more
+children have it enabled.
+
+
+3-2. No internal tasks
+
+One long-standing issue that cgroup faces is the competition between
+tasks belonging to the parent cgroup and its children cgroups. This
+is inherently nasty as two different types of entities compete and
+there is no agreed-upon obvious way to handle it. Different
+controllers are doing different things.
+
+The cpu controller considers tasks and cgroups as equivalents and maps
+nice levels to cgroup weights. This works for some cases but falls
+flat when children should be allocated specific ratios of CPU cycles
+and the number of internal tasks fluctuates - the ratios constantly
+change as the number of competing entities fluctuates. There also are
+other issues. The mapping from nice level to weight isn't obvious or
+universal, and there are various other knobs which simply aren't
+available for tasks.
+
+The blkio controller implicitly creates a hidden leaf node for each
+cgroup to host the tasks. The hidden leaf has its own copies of all
+the knobs with "leaf_" prefixed. While this allows equivalent control
+over internal tasks, it's with serious drawbacks. It always adds an
+extra layer of nesting which may not be necessary, makes the interface
+messy and significantly complicates the implementation.
+
+The memory controller currently doesn't have a way to control what
+happens between internal tasks and child cgroups and the behavior is
+not clearly defined. There have been attempts to add ad-hoc behaviors
+and knobs to tailor the behavior to specific workloads. Continuing
+this direction will lead to problems which will be extremely difficult
+to resolve in the long term.
+
+Multiple controllers struggle with internal tasks and came up with
+different ways to deal with it; unfortunately, all the approaches in
+use now are severely flawed and, furthermore, the widely different
+behaviors make cgroup as whole highly inconsistent.
+
+It is clear that this is something which needs to be addressed from
+cgroup core proper in a uniform way so that controllers don't need to
+worry about it and cgroup as a whole shows a consistent and logical
+behavior. To achieve that, unified hierarchy enforces the following
+structural constraint:
+
+ Except for the root, only cgroups which don't contain any task may
+ have controllers enabled in their "cgroup.subtree_control" files.
+
+Combined with other properties, this guarantees that, when a
+controller is looking at the part of the hierarchy which has it
+enabled, tasks are always only on the leaves. This rules out
+situations where child cgroups compete against internal tasks of the
+parent.
+
+There are two things to note. Firstly, the root cgroup is exempt from
+the restriction. Root contains tasks and anonymous resource
+consumption which can't be associated with any other cgroup and
+requires special treatment from most controllers. How resource
+consumption in the root cgroup is governed is up to each controller.
+
+Secondly, the restriction doesn't take effect if there is no enabled
+controller in the cgroup's "cgroup.subtree_control" file. This is
+important as otherwise it wouldn't be possible to create children of a
+populated cgroup. To control resource distribution of a cgroup, the
+cgroup must create children and transfer all its tasks to the children
+before enabling controllers in its "cgroup.subtree_control" file.
+
+
+4. Other Changes
+
+4-1. [Un]populated Notification
+
+cgroup users often need a way to determine when a cgroup's
+subhierarchy becomes empty so that it can be cleaned up. cgroup
+currently provides release_agent for it; unfortunately, this mechanism
+is riddled with issues.
+
+- It delivers events by forking and execing a userland binary
+ specified as the release_agent. This is a long deprecated method of
+ notification delivery. It's extremely heavy, slow and cumbersome to
+ integrate with larger infrastructure.
+
+- There is single monitoring point at the root. There's no way to
+ delegate management of a subtree.
+
+- The event isn't recursive. It triggers when a cgroup doesn't have
+ any tasks or child cgroups. Events for internal nodes trigger only
+ after all children are removed. This again makes it impossible to
+ delegate management of a subtree.
+
+- Events are filtered from the kernel side. A "notify_on_release"
+ file is used to subscribe to or suppress release events. This is
+ unnecessarily complicated and probably done this way because event
+ delivery itself was expensive.
+
+Unified hierarchy implements an interface file "cgroup.populated"
+which can be used to monitor whether the cgroup's subhierarchy has
+tasks in it or not. Its value is 0 if there is no task in the cgroup
+and its descendants; otherwise, 1. poll and [id]notify events are
+triggered when the value changes.
+
+This is significantly lighter and simpler and trivially allows
+delegating management of subhierarchy - subhierarchy monitoring can
+block further propagation simply by putting itself or another process
+in the subhierarchy and monitor events that it's interested in from
+there without interfering with monitoring higher in the tree.
+
+In unified hierarchy, the release_agent mechanism is no longer
+supported and the interface files "release_agent" and
+"notify_on_release" do not exist.
+
+
+4-2. Other Core Changes
+
+- None of the mount options is allowed.
+
+- remount is disallowed.
+
+- rename(2) is disallowed.
+
+- The "tasks" file is removed. Everything should at process
+ granularity. Use the "cgroup.procs" file instead.
+
+- The "cgroup.procs" file is not sorted. pids will be unique unless
+ they got recycled in-between reads.
+
+- The "cgroup.clone_children" file is removed.
+
+
+4-3. Per-Controller Changes
+
+4-3-1. blkio
+
+- blk-throttle becomes properly hierarchical.
+
+
+4-3-2. cpuset
+
+- Tasks are kept in empty cpusets after hotplug and take on the masks
+ of the nearest non-empty ancestor, instead of being moved to it.
+
+- A task can be moved into an empty cpuset, and again it takes on the
+ masks of the nearest non-empty ancestor.
+
+
+4-3-3. memory
+
+- use_hierarchy is on by default and the cgroup file for the flag is
+ not created.
+
+
+5. Planned Changes
+
+5-1. CAP for resource control
+
+Unified hierarchy will require one of the capabilities(7), which is
+yet to be decided, for all resource control related knobs. Process
+organization operations - creation of sub-cgroups and migration of
+processes in sub-hierarchies may be delegated by changing the
+ownership and/or permissions on the cgroup directory and
+"cgroup.procs" interface file; however, all operations which affect
+resource control - writes to a "cgroup.subtree_control" file or any
+controller-specific knobs - will require an explicit CAP privilege.
+
+This, in part, is to prevent the cgroup interface from being
+inadvertently promoted to programmable API used by non-privileged
+binaries. cgroup exposes various aspects of the system in ways which
+aren't properly abstracted for direct consumption by regular programs.
+This is an administration interface much closer to sysctl knobs than
+system calls. Even the basic access model, being filesystem path
+based, isn't suitable for direct consumption. There's no way to
+access "my cgroup" in a race-free way or make multiple operations
+atomic against migration to another cgroup.
+
+Another aspect is that, for better or for worse, the cgroup interface
+goes through far less scrutiny than regular interfaces for
+unprivileged userland. The upside is that cgroup is able to expose
+useful features which may not be suitable for general consumption in a
+reasonable time frame. It provides a relatively short path between
+internal details and userland-visible interface. Of course, this
+shortcut comes with high risk. We go through what we go through for
+general kernel APIs for good reasons. It may end up leaking internal
+details in a way which can exert significant pain by locking the
+kernel into a contract that can't be maintained in a reasonable
+manner.
+
+Also, due to the specific nature, cgroup and its controllers don't
+tend to attract attention from a wide scope of developers. cgroup's
+short history is already fraught with severely mis-designed
+interfaces, unnecessary commitments to and exposing of internal
+details, broken and dangerous implementations of various features.
+
+Keeping cgroup as an administration interface is both advantageous for
+its role and imperative given its nature. Some of the cgroup features
+may make sense for unprivileged access. If deemed justified, those
+must be further abstracted and implemented as a different interface,
+be it a system call or process-private filesystem, and survive through
+the scrutiny that any interface for general consumption is required to
+go through.
+
+Requiring CAP is not a complete solution but should serve as a
+significant deterrent against spraying cgroup usages in non-privileged
+programs.
diff --git a/Documentation/clk.txt b/Documentation/clk.txt
index c9c399a..1fee72f 100644
--- a/Documentation/clk.txt
+++ b/Documentation/clk.txt
@@ -68,21 +68,27 @@
int (*is_enabled)(struct clk_hw *hw);
unsigned long (*recalc_rate)(struct clk_hw *hw,
unsigned long parent_rate);
- long (*round_rate)(struct clk_hw *hw, unsigned long,
- unsigned long *);
+ long (*round_rate)(struct clk_hw *hw,
+ unsigned long rate,
+ unsigned long *parent_rate);
long (*determine_rate)(struct clk_hw *hw,
unsigned long rate,
unsigned long *best_parent_rate,
struct clk **best_parent_clk);
int (*set_parent)(struct clk_hw *hw, u8 index);
u8 (*get_parent)(struct clk_hw *hw);
- int (*set_rate)(struct clk_hw *hw, unsigned long);
+ int (*set_rate)(struct clk_hw *hw,
+ unsigned long rate,
+ unsigned long parent_rate);
int (*set_rate_and_parent)(struct clk_hw *hw,
unsigned long rate,
- unsigned long parent_rate, u8 index);
+ unsigned long parent_rate,
+ u8 index);
unsigned long (*recalc_accuracy)(struct clk_hw *hw,
- unsigned long parent_accuracy);
+ unsigned long parent_accuracy);
void (*init)(struct clk_hw *hw);
+ int (*debug_init)(struct clk_hw *hw,
+ struct dentry *dentry);
};
Part 3 - hardware clk implementations
diff --git a/Documentation/devicetree/bindings/arm/pmu.txt b/Documentation/devicetree/bindings/arm/pmu.txt
index fe5cef8..75ef91d0 100644
--- a/Documentation/devicetree/bindings/arm/pmu.txt
+++ b/Documentation/devicetree/bindings/arm/pmu.txt
@@ -8,6 +8,7 @@
- compatible : should be one of
"arm,armv8-pmuv3"
+ "arm,cortex-a17-pmu"
"arm,cortex-a15-pmu"
"arm,cortex-a12-pmu"
"arm,cortex-a9-pmu"
diff --git a/Documentation/devicetree/bindings/ata/ahci-platform.txt b/Documentation/devicetree/bindings/ata/ahci-platform.txt
index 48b285f..c96d8dc 100644
--- a/Documentation/devicetree/bindings/ata/ahci-platform.txt
+++ b/Documentation/devicetree/bindings/ata/ahci-platform.txt
@@ -4,10 +4,16 @@
Each SATA controller should have its own node.
Required properties:
-- compatible : compatible list, one of "snps,spear-ahci",
- "snps,exynos5440-ahci", "ibm,476gtr-ahci",
- "allwinner,sun4i-a10-ahci", "fsl,imx53-ahci"
- "fsl,imx6q-ahci" or "snps,dwc-ahci"
+- compatible : compatible string, one of:
+ - "allwinner,sun4i-a10-ahci"
+ - "fsl,imx53-ahci"
+ - "fsl,imx6q-ahci"
+ - "hisilicon,hisi-ahci"
+ - "ibm,476gtr-ahci"
+ - "marvell,armada-380-ahci"
+ - "snps,dwc-ahci"
+ - "snps,exynos5440-ahci"
+ - "snps,spear-ahci"
- interrupts : <interrupt mapping for SATA IRQ>
- reg : <registers mapping>
diff --git a/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt b/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt
index 56d1f49..5286e26 100644
--- a/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt
+++ b/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt
@@ -10,12 +10,12 @@
Required properties:
- compatible
- Shall have one of the following values:
- - "brcm,bcm11351-root-ccu"
- - "brcm,bcm11351-aon-ccu"
- - "brcm,bcm11351-hub-ccu"
- - "brcm,bcm11351-master-ccu"
- - "brcm,bcm11351-slave-ccu"
+ Shall have a value of the form "brcm,<model>-<which>-ccu",
+ where <model> is a Broadcom SoC model number and <which> is
+ the name of a defined CCU. For example:
+ "brcm,bcm11351-root-ccu"
+ The compatible strings used for each supported SoC family
+ are defined below.
- reg
Shall define the base and range of the address space
containing clock control registers
@@ -26,12 +26,48 @@
Shall be an ordered list of strings defining the names of
the clocks provided by the CCU.
+Device tree example:
-BCM281XX family SoCs use Kona CCUs. The following table defines
-the set of CCUs and clock specifiers for BCM281XX clocks. When
-a clock consumer references a clocks, its symbolic specifier
-(rather than its numeric index value) should be used. These
-specifiers are defined in "include/dt-bindings/clock/bcm281xx.h".
+ slave_ccu: slave_ccu {
+ compatible = "brcm,bcm11351-slave-ccu";
+ reg = <0x3e011000 0x0f00>;
+ #clock-cells = <1>;
+ clock-output-names = "uartb",
+ "uartb2",
+ "uartb3",
+ "uartb4";
+ };
+
+ ref_crystal_clk: ref_crystal {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <26000000>;
+ };
+
+ uart@3e002000 {
+ compatible = "brcm,bcm11351-dw-apb-uart", "snps,dw-apb-uart";
+ status = "disabled";
+ reg = <0x3e002000 0x1000>;
+ clocks = <&slave_ccu BCM281XX_SLAVE_CCU_UARTB3>;
+ interrupts = <GIC_SPI 65 IRQ_TYPE_LEVEL_HIGH>;
+ reg-shift = <2>;
+ reg-io-width = <4>;
+ };
+
+BCM281XX family
+---------------
+CCU compatible string values for SoCs in the BCM281XX family are:
+ "brcm,bcm11351-root-ccu"
+ "brcm,bcm11351-aon-ccu"
+ "brcm,bcm11351-hub-ccu"
+ "brcm,bcm11351-master-ccu"
+ "brcm,bcm11351-slave-ccu"
+
+The following table defines the set of CCUs and clock specifiers for
+BCM281XX family clocks. When a clock consumer references a clocks,
+its symbolic specifier (rather than its numeric index value) should
+be used. These specifiers are defined in:
+ "include/dt-bindings/clock/bcm281xx.h"
CCU Clock Type Index Specifier
--- ----- ---- ----- ---------
@@ -64,30 +100,40 @@
slave pwm peri 9 BCM281XX_SLAVE_CCU_PWM
-Device tree example:
+BCM21664 family
+---------------
+CCU compatible string values for SoCs in the BCM21664 family are:
+ "brcm,bcm21664-root-ccu"
+ "brcm,bcm21664-aon-ccu"
+ "brcm,bcm21664-master-ccu"
+ "brcm,bcm21664-slave-ccu"
- slave_ccu: slave_ccu {
- compatible = "brcm,bcm11351-slave-ccu";
- reg = <0x3e011000 0x0f00>;
- #clock-cells = <1>;
- clock-output-names = "uartb",
- "uartb2",
- "uartb3",
- "uartb4";
- };
+The following table defines the set of CCUs and clock specifiers for
+BCM21664 family clocks. When a clock consumer references a clocks,
+its symbolic specifier (rather than its numeric index value) should
+be used. These specifiers are defined in:
+ "include/dt-bindings/clock/bcm21664.h"
- ref_crystal_clk: ref_crystal {
- #clock-cells = <0>;
- compatible = "fixed-clock";
- clock-frequency = <26000000>;
- };
+ CCU Clock Type Index Specifier
+ --- ----- ---- ----- ---------
+ root frac_1m peri 0 BCM21664_ROOT_CCU_FRAC_1M
- uart@3e002000 {
- compatible = "brcm,bcm11351-dw-apb-uart", "snps,dw-apb-uart";
- status = "disabled";
- reg = <0x3e002000 0x1000>;
- clocks = <&slave_ccu BCM281XX_SLAVE_CCU_UARTB3>;
- interrupts = <GIC_SPI 65 IRQ_TYPE_LEVEL_HIGH>;
- reg-shift = <2>;
- reg-io-width = <4>;
- };
+ aon hub_timer peri 0 BCM21664_AON_CCU_HUB_TIMER
+
+ master sdio1 peri 0 BCM21664_MASTER_CCU_SDIO1
+ master sdio2 peri 1 BCM21664_MASTER_CCU_SDIO2
+ master sdio3 peri 2 BCM21664_MASTER_CCU_SDIO3
+ master sdio4 peri 3 BCM21664_MASTER_CCU_SDIO4
+ master sdio1_sleep peri 4 BCM21664_MASTER_CCU_SDIO1_SLEEP
+ master sdio2_sleep peri 5 BCM21664_MASTER_CCU_SDIO2_SLEEP
+ master sdio3_sleep peri 6 BCM21664_MASTER_CCU_SDIO3_SLEEP
+ master sdio4_sleep peri 7 BCM21664_MASTER_CCU_SDIO4_SLEEP
+
+ slave uartb peri 0 BCM21664_SLAVE_CCU_UARTB
+ slave uartb2 peri 1 BCM21664_SLAVE_CCU_UARTB2
+ slave uartb3 peri 2 BCM21664_SLAVE_CCU_UARTB3
+ slave uartb4 peri 3 BCM21664_SLAVE_CCU_UARTB4
+ slave bsc1 peri 4 BCM21664_SLAVE_CCU_BSC1
+ slave bsc2 peri 5 BCM21664_SLAVE_CCU_BSC2
+ slave bsc3 peri 6 BCM21664_SLAVE_CCU_BSC3
+ slave bsc4 peri 7 BCM21664_SLAVE_CCU_BSC4
diff --git a/Documentation/devicetree/bindings/clock/clock-bindings.txt b/Documentation/devicetree/bindings/clock/clock-bindings.txt
index 700e7aa..f157878 100644
--- a/Documentation/devicetree/bindings/clock/clock-bindings.txt
+++ b/Documentation/devicetree/bindings/clock/clock-bindings.txt
@@ -44,10 +44,9 @@
clocks by index. The names should reflect the clock output signal
names for the device.
-clock-indices: If the identifyng number for the clocks in the node
- is not linear from zero, then the this mapping allows
- the mapping of identifiers into the clock-output-names
- array.
+clock-indices: If the identifying number for the clocks in the node
+ is not linear from zero, then this allows the mapping of
+ identifiers into the clock-output-names array.
For example, if we have two clocks <&oscillator 1> and <&oscillator 3>:
@@ -58,7 +57,7 @@
clock-output-names = "clka", "clkb";
}
- This ensures we do not have any empty nodes in clock-output-names
+ This ensures we do not have any empty strings in clock-output-names
==Clock consumers==
diff --git a/Documentation/devicetree/bindings/clock/fixed-clock.txt b/Documentation/devicetree/bindings/clock/fixed-clock.txt
index 48ea0ad..0641a663 100644
--- a/Documentation/devicetree/bindings/clock/fixed-clock.txt
+++ b/Documentation/devicetree/bindings/clock/fixed-clock.txt
@@ -12,7 +12,6 @@
Optional properties:
- clock-accuracy : accuracy of clock in ppb (parts per billion).
Should be a single cell.
-- gpios : From common gpio binding; gpio connection to clock enable pin.
- clock-output-names : From common clock binding.
Example:
diff --git a/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt b/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt
new file mode 100644
index 0000000..7894a64
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt
@@ -0,0 +1,31 @@
+* Hisilicon Hix5hd2 Clock Controller
+
+The hix5hd2 clock controller generates and supplies clock to various
+controllers within the hix5hd2 SoC.
+
+Required Properties:
+
+- compatible: should be "hisilicon,hix5hd2-clock"
+- reg: Address and length of the register set
+- #clock-cells: Should be <1>
+
+Each clock is assigned an identifier and client nodes use this identifier
+to specify the clock which they consume.
+
+All these identifier could be found in <dt-bindings/clock/hix5hd2-clock.h>.
+
+Examples:
+ clock: clock@f8a22000 {
+ compatible = "hisilicon,hix5hd2-clock";
+ reg = <0xf8a22000 0x1000>;
+ #clock-cells = <1>;
+ };
+
+ uart0: uart@f8b00000 {
+ compatible = "arm,pl011", "arm,primecell";
+ reg = <0xf8b00000 0x1000>;
+ interrupts = <0 49 4>;
+ clocks = <&clock HIX5HD2_FIXED_83M>;
+ clock-names = "apb_pclk";
+ status = "disabled";
+ };
diff --git a/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt b/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt
new file mode 100644
index 0000000..3ce97cf
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt
@@ -0,0 +1,29 @@
+AXM5516 clock driver bindings
+-----------------------------
+
+Required properties :
+- compatible : shall contain "lsi,axm5516-clks"
+- reg : shall contain base register location and length
+- #clock-cells : shall contain 1
+
+The consumer specifies the desired clock by having the clock ID in its "clocks"
+phandle cell. See <dt-bindings/clock/lsi,axxia-clock.h> for the list of
+supported clock IDs.
+
+Example:
+
+ clks: clock-controller@2010020000 {
+ compatible = "lsi,axm5516-clks";
+ #clock-cells = <1>;
+ reg = <0x20 0x10020000 0 0x20000>;
+ };
+
+ serial0: uart@2010080000 {
+ compatible = "arm,pl011", "arm,primecell";
+ reg = <0x20 0x10080000 0 0x1000>;
+ interrupts = <GIC_SPI 56 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clks AXXIA_CLK_PER>;
+ clock-names = "apb_pclk";
+ };
+ };
+
diff --git a/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt b/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt
index 307a503c..dc5ea5b 100644
--- a/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt
+++ b/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt
@@ -29,6 +29,11 @@
2 = l2clk (L2 Cache clock derived from CPU0 clock)
3 = ddrclk (DDR controller clock derived from CPU0 clock)
+The following is a list of provided IDs and clock names on Orion5x:
+ 0 = tclk (Internal Bus clock)
+ 1 = cpuclk (CPU0 clock)
+ 2 = ddrclk (DDR controller clock derived from CPU0 clock)
+
Required properties:
- compatible : shall be one of the following:
"marvell,armada-370-core-clock" - For Armada 370 SoC core clocks
@@ -38,6 +43,9 @@
"marvell,dove-core-clock" - for Dove SoC core clocks
"marvell,kirkwood-core-clock" - for Kirkwood SoC (except mv88f6180)
"marvell,mv88f6180-core-clock" - for Kirkwood MV88f6180 SoC
+ "marvell,mv88f5182-core-clock" - for Orion MV88F5182 SoC
+ "marvell,mv88f5281-core-clock" - for Orion MV88F5281 SoC
+ "marvell,mv88f6183-core-clock" - for Orion MV88F6183 SoC
- reg : shall be the register address of the Sample-At-Reset (SAR) register
- #clock-cells : from common clock binding; shall be set to 1
diff --git a/Documentation/devicetree/bindings/clock/qcom,gcc.txt b/Documentation/devicetree/bindings/clock/qcom,gcc.txt
index 767401f..9cfcb4f 100644
--- a/Documentation/devicetree/bindings/clock/qcom,gcc.txt
+++ b/Documentation/devicetree/bindings/clock/qcom,gcc.txt
@@ -4,9 +4,12 @@
Required properties :
- compatible : shall contain only one of the following:
+ "qcom,gcc-apq8064"
"qcom,gcc-msm8660"
"qcom,gcc-msm8960"
"qcom,gcc-msm8974"
+ "qcom,gcc-msm8974pro"
+ "qcom,gcc-msm8974pro-ac"
- reg : shall contain base register location and length
- #clock-cells : shall contain 1
diff --git a/Documentation/devicetree/bindings/clock/corenet-clock.txt b/Documentation/devicetree/bindings/clock/qoriq-clock.txt
similarity index 95%
rename from Documentation/devicetree/bindings/clock/corenet-clock.txt
rename to Documentation/devicetree/bindings/clock/qoriq-clock.txt
index 24711af..5666812 100644
--- a/Documentation/devicetree/bindings/clock/corenet-clock.txt
+++ b/Documentation/devicetree/bindings/clock/qoriq-clock.txt
@@ -7,6 +7,14 @@
cores and peripheral IP blocks.
Please refer to the Reference Manual for details.
+All references to "1.0" and "2.0" refer to the QorIQ chassis version to
+which the chip complies.
+
+Chassis Version Example Chips
+--------------- -------------
+1.0 p4080, p5020, p5040
+2.0 t4240, b4860, t1040
+
1. Clock Block Binding
Required properties:
@@ -85,7 +93,7 @@
#clock-cells = <0>;
compatible = "fsl,qoriq-sysclk-1.0";
clock-output-names = "sysclk";
- }
+ };
pll0: pll0@800 {
#clock-cells = <1>;
diff --git a/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt b/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt
index 6c3c084..8a92b5f 100644
--- a/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt
+++ b/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt
@@ -11,6 +11,7 @@
- compatible: Must be one of the following
- "renesas,r7s72100-mstp-clocks" for R7S72100 (RZ) MSTP gate clocks
+ - "renesas,r8a7779-mstp-clocks" for R8A7779 (R-Car H1) MSTP gate clocks
- "renesas,r8a7790-mstp-clocks" for R8A7790 (R-Car H2) MSTP gate clocks
- "renesas,r8a7791-mstp-clocks" for R8A7791 (R-Car M2) MSTP gate clocks
- "renesas,cpg-mstp-clock" for generic MSTP gate clocks
diff --git a/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt b/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt
new file mode 100644
index 0000000..2c03302
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt
@@ -0,0 +1,41 @@
+These bindings should be considered EXPERIMENTAL for now.
+
+* Renesas R8A7740 Clock Pulse Generator (CPG)
+
+The CPG generates core clocks for the R8A7740 SoC. It includes three PLLs
+and several fixed ratio and variable ratio dividers.
+
+Required Properties:
+
+ - compatible: Must be "renesas,r8a7740-cpg-clocks"
+
+ - reg: Base address and length of the memory resource used by the CPG
+
+ - clocks: Reference to the three parent clocks
+ - #clock-cells: Must be 1
+ - clock-output-names: The names of the clocks. Supported clocks are
+ "system", "pllc0", "pllc1", "pllc2", "r", "usb24s", "i", "zg", "b",
+ "m1", "hp", "hpp", "usbp", "s", "zb", "m3", and "cp".
+
+ - renesas,mode: board-specific settings of the MD_CK* bits
+
+
+Example
+-------
+
+cpg_clocks: cpg_clocks@e6150000 {
+ compatible = "renesas,r8a7740-cpg-clocks";
+ reg = <0xe6150000 0x10000>;
+ clocks = <&extal1_clk>, <&extal2_clk>, <&extalr_clk>;
+ #clock-cells = <1>;
+ clock-output-names = "system", "pllc0", "pllc1",
+ "pllc2", "r",
+ "usb24s",
+ "i", "zg", "b", "m1", "hp",
+ "hpp", "usbp", "s", "zb", "m3",
+ "cp";
+};
+
+&cpg_clocks {
+ renesas,mode = <0x05>;
+};
diff --git a/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt b/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt
new file mode 100644
index 0000000..ed3c8cb
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt
@@ -0,0 +1,27 @@
+* Renesas R8A7779 Clock Pulse Generator (CPG)
+
+The CPG generates core clocks for the R8A7779. It includes one PLL and
+several fixed ratio dividers
+
+Required Properties:
+
+ - compatible: Must be "renesas,r8a7779-cpg-clocks"
+ - reg: Base address and length of the memory resource used by the CPG
+
+ - clocks: Reference to the parent clock
+ - #clock-cells: Must be 1
+ - clock-output-names: The names of the clocks. Supported clocks are "plla",
+ "z", "zs", "s", "s1", "p", "b", "out".
+
+
+Example
+-------
+
+ cpg_clocks: cpg_clocks@ffc80000 {
+ compatible = "renesas,r8a7779-cpg-clocks";
+ reg = <0 0xffc80000 0 0x30>;
+ clocks = <&extal_clk>;
+ #clock-cells = <1>;
+ clock-output-names = "plla", "z", "zs", "s", "s1", "p",
+ "b", "out";
+ };
diff --git a/Documentation/devicetree/bindings/crypto/samsung-sss.txt b/Documentation/devicetree/bindings/crypto/samsung-sss.txt
new file mode 100644
index 0000000..a6dafa8
--- /dev/null
+++ b/Documentation/devicetree/bindings/crypto/samsung-sss.txt
@@ -0,0 +1,34 @@
+Samsung SoC SSS (Security SubSystem) module
+
+The SSS module in S5PV210 SoC supports the following:
+-- Feeder (FeedCtrl)
+-- Advanced Encryption Standard (AES)
+-- Data Encryption Standard (DES)/3DES
+-- Public Key Accelerator (PKA)
+-- SHA-1/SHA-256/MD5/HMAC (SHA-1/SHA-256/MD5)/PRNG
+-- PRNG: Pseudo Random Number Generator
+
+The SSS module in Exynos4 (Exynos4210) and
+Exynos5 (Exynos5420 and Exynos5250) SoCs
+supports the following also:
+-- ARCFOUR (ARC4)
+-- True Random Number Generator (TRNG)
+-- Secure Key Manager
+
+Required properties:
+
+- compatible : Should contain entries for this and backward compatible
+ SSS versions:
+ - "samsung,s5pv210-secss" for S5PV210 SoC.
+ - "samsung,exynos4210-secss" for Exynos4210, Exynos4212, Exynos4412, Exynos5250,
+ Exynos5260 and Exynos5420 SoCs.
+- reg : Offset and length of the register set for the module
+- interrupts : interrupt specifiers of SSS module interrupts, should contain
+ following entries:
+ - first : feed control interrupt (required for all variants),
+ - second : hash interrupt (required only for samsung,s5pv210-secss).
+
+- clocks : list of clock phandle and specifier pairs for all clocks listed in
+ clock-names property.
+- clock-names : list of device clock input names; should contain one entry
+ "secss".
diff --git a/Documentation/devicetree/bindings/dma/mmp-dma.txt b/Documentation/devicetree/bindings/dma/mmp-dma.txt
index a4fa4ef..7a802f6 100644
--- a/Documentation/devicetree/bindings/dma/mmp-dma.txt
+++ b/Documentation/devicetree/bindings/dma/mmp-dma.txt
@@ -1,17 +1,20 @@
* MARVELL MMP DMA controller
Marvell Peripheral DMA Controller
-Used platfroms: pxa688, pxa910, pxa3xx, etc
+Used platforms: pxa688, pxa910, pxa3xx, etc
Required properties:
- compatible: Should be "marvell,pdma-1.0"
- reg: Should contain DMA registers location and length.
- interrupts: Either contain all of the per-channel DMA interrupts
or one irq for pdma device
-- #dma-channels: Number of DMA channels supported by the controller.
+
+Optional properties:
+- #dma-channels: Number of DMA channels supported by the controller (defaults
+ to 32 when not specified)
"marvell,pdma-1.0"
-Used platfroms: pxa25x, pxa27x, pxa3xx, pxa93x, pxa168, pxa910, pxa688.
+Used platforms: pxa25x, pxa27x, pxa3xx, pxa93x, pxa168, pxa910, pxa688.
Examples:
@@ -45,7 +48,7 @@
Marvell Two Channel DMA Controller used specifically for audio
-Used platfroms: pxa688, pxa910
+Used platforms: pxa688, pxa910
Required properties:
- compatible: Should be "marvell,adma-1.0" or "marvell,pxa910-squ"
diff --git a/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt b/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt
new file mode 100644
index 0000000..1405ed07
--- /dev/null
+++ b/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt
@@ -0,0 +1,75 @@
+Xilinx AXI VDMA engine, it does transfers between memory and video devices.
+It can be configured to have one channel or two channels. If configured
+as two channels, one is to transmit to the video device and another is
+to receive from the video device.
+
+Required properties:
+- compatible: Should be "xlnx,axi-vdma-1.00.a"
+- #dma-cells: Should be <1>, see "dmas" property below
+- reg: Should contain VDMA registers location and length.
+- xlnx,num-fstores: Should be the number of framebuffers as configured in h/w.
+- dma-channel child node: Should have at least one channel and can have up to
+ two channels per device. This node specifies the properties of each
+ DMA channel (see child node properties below).
+
+Optional properties:
+- xlnx,include-sg: Tells configured for Scatter-mode in
+ the hardware.
+- xlnx,flush-fsync: Tells which channel to Flush on Frame sync.
+ It takes following values:
+ {1}, flush both channels
+ {2}, flush mm2s channel
+ {3}, flush s2mm channel
+
+Required child node properties:
+- compatible: It should be either "xlnx,axi-vdma-mm2s-channel" or
+ "xlnx,axi-vdma-s2mm-channel".
+- interrupts: Should contain per channel VDMA interrupts.
+- xlnx,data-width: Should contain the stream data width, take values
+ {32,64...1024}.
+
+Optional child node properties:
+- xlnx,include-dre: Tells hardware is configured for Data
+ Realignment Engine.
+- xlnx,genlock-mode: Tells Genlock synchronization is
+ enabled/disabled in hardware.
+
+Example:
+++++++++
+
+axi_vdma_0: axivdma@40030000 {
+ compatible = "xlnx,axi-vdma-1.00.a";
+ #dma_cells = <1>;
+ reg = < 0x40030000 0x10000 >;
+ xlnx,num-fstores = <0x8>;
+ xlnx,flush-fsync = <0x1>;
+ dma-channel@40030000 {
+ compatible = "xlnx,axi-vdma-mm2s-channel";
+ interrupts = < 0 54 4 >;
+ xlnx,datawidth = <0x40>;
+ } ;
+ dma-channel@40030030 {
+ compatible = "xlnx,axi-vdma-s2mm-channel";
+ interrupts = < 0 53 4 >;
+ xlnx,datawidth = <0x40>;
+ } ;
+} ;
+
+
+* DMA client
+
+Required properties:
+- dmas: a list of <[Video DMA device phandle] [Channel ID]> pairs,
+ where Channel ID is '0' for write/tx and '1' for read/rx
+ channel.
+- dma-names: a list of DMA channel names, one per "dmas" entry
+
+Example:
+++++++++
+
+vdmatest_0: vdmatest@0 {
+ compatible ="xlnx,axi-vdma-test-1.00.a";
+ dmas = <&axi_vdma_0 0
+ &axi_vdma_0 1>;
+ dma-names = "vdma0", "vdma1";
+} ;
diff --git a/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt b/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt
index 1ac8ea8..bfeabb8 100644
--- a/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt
+++ b/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt
@@ -8,6 +8,12 @@
the case where there is a device on the bus that has errata and/or bugs
that makes standard multimaster mode not feasible.
+Note that this scheme works well enough but has some downsides:
+* It is nonstandard (not using standard I2C multimaster)
+* Having two masters on a bus in general makes it relatively hard to debug
+ problems (hard to tell if i2c issues were caused by one master, another, or
+ some device on the bus).
+
Algorithm:
diff --git a/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt b/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt
new file mode 100644
index 0000000..898f030
--- /dev/null
+++ b/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt
@@ -0,0 +1,39 @@
+I2C bus that tunnels through the ChromeOS EC (cros-ec)
+======================================================
+On some ChromeOS board designs we've got a connection to the EC (embedded
+controller) but no direct connection to some devices on the other side of
+the EC (like a battery and PMIC). To get access to those devices we need
+to tunnel our i2c commands through the EC.
+
+The node for this device should be under a cros-ec node like google,cros-ec-spi
+or google,cros-ec-i2c.
+
+
+Required properties:
+- compatible: google,cros-ec-i2c-tunnel
+- google,remote-bus: The EC bus we'd like to talk to.
+
+Optional child nodes:
+- One node per I2C device connected to the tunnelled I2C bus.
+
+
+Example:
+ cros-ec@0 {
+ compatible = "google,cros-ec-spi";
+
+ ...
+
+ i2c-tunnel {
+ compatible = "google,cros-ec-i2c-tunnel";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ google,remote-bus = <0>;
+
+ battery: sbs-battery@b {
+ compatible = "sbs,sbs-battery";
+ reg = <0xb>;
+ sbs,poll-retry-count = <1>;
+ };
+ };
+ }
diff --git a/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt b/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt
index 056732c..d4745e3 100644
--- a/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt
+++ b/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt
@@ -5,7 +5,14 @@
Required properties:
- compatible: value should be.
- -> "samsung,exynos5-hsi2c", for i2c compatible with exynos5 hsi2c.
+ -> "samsung,exynos5-hsi2c", (DEPRECATED)
+ for i2c compatible with HSI2C available
+ on Exynos5250 and Exynos5420 SoCs.
+ -> "samsung,exynos5250-hsi2c", for i2c compatible with HSI2C available
+ on Exynos5250 and Exynos5420 SoCs.
+ -> "samsung,exynos5260-hsi2c", for i2c compatible with HSI2C available
+ on Exynos5260 SoCs.
+
- reg: physical base address of the controller and length of memory mapped
region.
- interrupts: interrupt number to the cpu.
@@ -26,7 +33,7 @@
Example:
hsi2c@12ca0000 {
- compatible = "samsung,exynos5-hsi2c";
+ compatible = "samsung,exynos5250-hsi2c";
reg = <0x12ca0000 0x100>;
interrupts = <56>;
clock-frequency = <100000>;
diff --git a/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt b/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt
index befd4fb..5c30026 100644
--- a/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt
+++ b/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt
@@ -5,7 +5,7 @@
- reg : Offset and length of the register set for the device
- compatible : Should be either:
- - "allwinner,sun4i-i2c"
+ - "allwinner,sun4i-a10-i2c"
- "allwinner,sun6i-a31-i2c"
- "marvell,mv64xxx-i2c"
- "marvell,mv78230-i2c"
diff --git a/Documentation/devicetree/bindings/i2c/i2c-rcar.txt b/Documentation/devicetree/bindings/i2c/i2c-rcar.txt
index dd8b2dd1..16b3e07 100644
--- a/Documentation/devicetree/bindings/i2c/i2c-rcar.txt
+++ b/Documentation/devicetree/bindings/i2c/i2c-rcar.txt
@@ -7,6 +7,9 @@
"renesas,i2c-r8a7779"
"renesas,i2c-r8a7790"
"renesas,i2c-r8a7791"
+ "renesas,i2c-r8a7792"
+ "renesas,i2c-r8a7793"
+ "renesas,i2c-r8a7794"
- reg: physical base address of the controller and length of memory mapped
region.
- interrupts: interrupt specifier.
diff --git a/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt b/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt
new file mode 100644
index 0000000..d2153ce
--- /dev/null
+++ b/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt
@@ -0,0 +1,26 @@
+Device tree configuration for Renesas IIC (sh_mobile) driver
+
+Required properties:
+- compatible : "renesas,iic-<soctype>". "renesas,rmobile-iic" as fallback
+- reg : address start and address range size of device
+- interrupts : interrupt of device
+- clocks : clock for device
+- #address-cells : should be <1>
+- #size-cells : should be <0>
+
+Optional properties:
+- clock-frequency : frequency of bus clock in Hz. Default 100kHz if unset.
+
+Pinctrl properties might be needed, too. See there.
+
+Example:
+
+ iic0: i2c@e6500000 {
+ compatible = "renesas,iic-r8a7790", "renesas,rmobile-iic";
+ reg = <0 0xe6500000 0 0x425>;
+ interrupts = <0 174 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&mstp3_clks R8A7790_CLK_IIC0>;
+ clock-frequency = <400000>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
diff --git a/Documentation/devicetree/bindings/gpio/gpio_keys.txt b/Documentation/devicetree/bindings/input/gpio-keys.txt
similarity index 100%
rename from Documentation/devicetree/bindings/gpio/gpio_keys.txt
rename to Documentation/devicetree/bindings/input/gpio-keys.txt
diff --git a/Documentation/devicetree/bindings/input/st-keyscan.txt b/Documentation/devicetree/bindings/input/st-keyscan.txt
new file mode 100644
index 0000000..51eb428
--- /dev/null
+++ b/Documentation/devicetree/bindings/input/st-keyscan.txt
@@ -0,0 +1,60 @@
+* ST Keyscan controller Device Tree bindings
+
+The ST keyscan controller Device Tree binding is based on the
+matrix-keymap.
+
+Required properties:
+- compatible: "st,sti-keyscan"
+
+- reg: Register base address and size of st-keyscan controller.
+
+- interrupts: Interrupt number for the st-keyscan controller.
+
+- clocks: Must contain one entry, for the module clock.
+ See ../clocks/clock-bindings.txt for details.
+
+- pinctrl: Should specify pin control groups used for this controller.
+ See ../pinctrl/pinctrl-bindings.txt for details.
+
+- linux,keymap: The keymap for keys as described in the binding document
+ devicetree/bindings/input/matrix-keymap.txt.
+
+- keypad,num-rows: Number of row lines connected to the keypad controller.
+
+- keypad,num-columns: Number of column lines connected to the keypad
+ controller.
+
+Optional property:
+- st,debounce_us: Debouncing interval time in microseconds
+
+Example:
+
+keyscan: keyscan@fe4b0000 {
+ compatible = "st,sti-keyscan";
+ reg = <0xfe4b0000 0x2000>;
+ interrupts = <GIC_SPI 212 IRQ_TYPE_NONE>;
+ clocks = <&CLK_SYSIN>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_keyscan>;
+
+ keypad,num-rows = <4>;
+ keypad,num-columns = <4>;
+ st,debounce_us = <5000>;
+
+ linux,keymap = < MATRIX_KEY(0x00, 0x00, KEY_F13)
+ MATRIX_KEY(0x00, 0x01, KEY_F9)
+ MATRIX_KEY(0x00, 0x02, KEY_F5)
+ MATRIX_KEY(0x00, 0x03, KEY_F1)
+ MATRIX_KEY(0x01, 0x00, KEY_F14)
+ MATRIX_KEY(0x01, 0x01, KEY_F10)
+ MATRIX_KEY(0x01, 0x02, KEY_F6)
+ MATRIX_KEY(0x01, 0x03, KEY_F2)
+ MATRIX_KEY(0x02, 0x00, KEY_F15)
+ MATRIX_KEY(0x02, 0x01, KEY_F11)
+ MATRIX_KEY(0x02, 0x02, KEY_F7)
+ MATRIX_KEY(0x02, 0x03, KEY_F3)
+ MATRIX_KEY(0x03, 0x00, KEY_F16)
+ MATRIX_KEY(0x03, 0x01, KEY_F12)
+ MATRIX_KEY(0x03, 0x02, KEY_F8)
+ MATRIX_KEY(0x03, 0x03, KEY_F4) >;
+ };
diff --git a/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt b/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt
new file mode 100644
index 0000000..aef5779
--- /dev/null
+++ b/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt
@@ -0,0 +1,20 @@
+sun4i resistive touchscreen controller
+--------------------------------------
+
+Required properties:
+ - compatible: "allwinner,sun4i-a10-ts"
+ - reg: mmio address range of the chip
+ - interrupts: interrupt to which the chip is connected
+
+Optional properties:
+ - allwinner,ts-attached: boolean indicating that an actual touchscreen is
+ attached to the controller
+
+Example:
+
+ rtp: rtp@01c25000 {
+ compatible = "allwinner,sun4i-a10-ts";
+ reg = <0x01c25000 0x100>;
+ interrupts = <29>;
+ allwinner,ts-attached;
+ };
diff --git a/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt b/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt
new file mode 100644
index 0000000..d8e06163
--- /dev/null
+++ b/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt
@@ -0,0 +1,27 @@
+General Touchscreen Properties:
+
+Optional properties for Touchscreens:
+ - touchscreen-size-x : horizontal resolution of touchscreen
+ (in pixels)
+ - touchscreen-size-y : vertical resolution of touchscreen
+ (in pixels)
+ - touchscreen-max-pressure : maximum reported pressure (arbitrary range
+ dependent on the controller)
+ - touchscreen-fuzz-x : horizontal noise value of the absolute input
+ device (in pixels)
+ - touchscreen-fuzz-y : vertical noise value of the absolute input
+ device (in pixels)
+ - touchscreen-fuzz-pressure : pressure noise value of the absolute input
+ device (arbitrary range dependent on the
+ controller)
+ - touchscreen-inverted-x : X axis is inverted (boolean)
+ - touchscreen-inverted-y : Y axis is inverted (boolean)
+
+Deprecated properties for Touchscreens:
+ - x-size : deprecated name for touchscreen-size-x
+ - y-size : deprecated name for touchscreen-size-y
+ - moving-threshold : deprecated name for a combination of
+ touchscreen-fuzz-x and touchscreen-fuzz-y
+ - contact-threshold : deprecated name for touchscreen-fuzz-pressure
+ - x-invert : deprecated name for touchscreen-inverted-x
+ - y-invert : deprecated name for touchscreen-inverted-y
diff --git a/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt b/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt
new file mode 100644
index 0000000..4b641c7
--- /dev/null
+++ b/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt
@@ -0,0 +1,42 @@
+* Texas Instruments tsc2005 touchscreen controller
+
+Required properties:
+ - compatible : "ti,tsc2005"
+ - reg : SPI device address
+ - spi-max-frequency : Maximal SPI speed
+ - interrupts : IRQ specifier
+ - reset-gpios : GPIO specifier
+ - vio-supply : Regulator specifier
+
+Optional properties:
+ - ti,x-plate-ohms : integer, resistance of the touchscreen's X plates
+ in ohm (defaults to 280)
+ - ti,esd-recovery-timeout-ms : integer, if the touchscreen does not respond after
+ the configured time (in milli seconds), the driver
+ will reset it. This is disabled by default.
+ - properties defined in touchscreen.txt
+
+Example:
+
+&mcspi1 {
+ tsc2005@0 {
+ compatible = "ti,tsc2005";
+ spi-max-frequency = <6000000>;
+ reg = <0>;
+
+ vio-supply = <&vio>;
+
+ reset-gpios = <&gpio4 8 GPIO_ACTIVE_HIGH>; /* 104 */
+ interrupts-extended = <&gpio4 4 IRQ_TYPE_EDGE_RISING>; /* 100 */
+
+ touchscreen-fuzz-x = <4>;
+ touchscreen-fuzz-y = <7>;
+ touchscreen-fuzz-pressure = <2>;
+ touchscreen-max-x = <4096>;
+ touchscreen-max-y = <4096>;
+ touchscreen-max-pressure = <2048>;
+
+ ti,x-plate-ohms = <280>;
+ ti,esd-recovery-timeout-ms = <8000>;
+ };
+}
diff --git a/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt b/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt
new file mode 100644
index 0000000..6fa4c73
--- /dev/null
+++ b/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt
@@ -0,0 +1,70 @@
+Samsung Exynos IOMMU H/W, System MMU (System Memory Management Unit)
+
+Samsung's Exynos architecture contains System MMUs that enables scattered
+physical memory chunks visible as a contiguous region to DMA-capable peripheral
+devices like MFC, FIMC, FIMD, GScaler, FIMC-IS and so forth.
+
+System MMU is an IOMMU and supports identical translation table format to
+ARMv7 translation tables with minimum set of page properties including access
+permissions, shareability and security protection. In addition, System MMU has
+another capabilities like L2 TLB or block-fetch buffers to minimize translation
+latency.
+
+System MMUs are in many to one relation with peripheral devices, i.e. single
+peripheral device might have multiple System MMUs (usually one for each bus
+master), but one System MMU can handle transactions from only one peripheral
+device. The relation between a System MMU and the peripheral device needs to be
+defined in device node of the peripheral device.
+
+MFC in all Exynos SoCs and FIMD, M2M Scalers and G2D in Exynos5420 has 2 System
+MMUs.
+* MFC has one System MMU on its left and right bus.
+* FIMD in Exynos5420 has one System MMU for window 0 and 4, the other system MMU
+ for window 1, 2 and 3.
+* M2M Scalers and G2D in Exynos5420 has one System MMU on the read channel and
+ the other System MMU on the write channel.
+The drivers must consider how to handle those System MMUs. One of the idea is
+to implement child devices or sub-devices which are the client devices of the
+System MMU.
+
+Note:
+The current DT binding for the Exynos System MMU is incomplete.
+The following properties can be removed or changed, if found incompatible with
+the "Generic IOMMU Binding" support for attaching devices to the IOMMU.
+
+Required properties:
+- compatible: Should be "samsung,exynos-sysmmu"
+- reg: A tuple of base address and size of System MMU registers.
+- interrupt-parent: The phandle of the interrupt controller of System MMU
+- interrupts: An interrupt specifier for interrupt signal of System MMU,
+ according to the format defined by a particular interrupt
+ controller.
+- clock-names: Should be "sysmmu" if the System MMU is needed to gate its clock.
+ Optional "master" if the clock to the System MMU is gated by
+ another gate clock other than "sysmmu".
+ Exynos4 SoCs, there needs no "master" clock.
+ Exynos5 SoCs, some System MMUs must have "master" clocks.
+- clocks: Required if the System MMU is needed to gate its clock.
+- samsung,power-domain: Required if the System MMU is needed to gate its power.
+ Please refer to the following document:
+ Documentation/devicetree/bindings/arm/exynos/power_domain.txt
+
+Examples:
+ gsc_0: gsc@13e00000 {
+ compatible = "samsung,exynos5-gsc";
+ reg = <0x13e00000 0x1000>;
+ interrupts = <0 85 0>;
+ samsung,power-domain = <&pd_gsc>;
+ clocks = <&clock CLK_GSCL0>;
+ clock-names = "gscl";
+ };
+
+ sysmmu_gsc0: sysmmu@13E80000 {
+ compatible = "samsung,exynos-sysmmu";
+ reg = <0x13E80000 0x1000>;
+ interrupt-parent = <&combiner>;
+ interrupts = <2 0>;
+ clock-names = "sysmmu", "master";
+ clocks = <&clock CLK_SMMU_GSCL0>, <&clock CLK_GSCL0>;
+ samsung,power-domain = <&pd_gsc>;
+ };
diff --git a/Documentation/devicetree/bindings/media/i2c/adv7604.txt b/Documentation/devicetree/bindings/media/i2c/adv7604.txt
new file mode 100644
index 0000000..c27cede
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/i2c/adv7604.txt
@@ -0,0 +1,70 @@
+* Analog Devices ADV7604/11 video decoder with HDMI receiver
+
+The ADV7604 and ADV7611 are multiformat video decoders with an integrated HDMI
+receiver. The ADV7604 has four multiplexed HDMI inputs and one analog input,
+and the ADV7611 has one HDMI input and no analog input.
+
+These device tree bindings support the ADV7611 only at the moment.
+
+Required Properties:
+
+ - compatible: Must contain one of the following
+ - "adi,adv7611" for the ADV7611
+
+ - reg: I2C slave address
+
+ - hpd-gpios: References to the GPIOs that control the HDMI hot-plug
+ detection pins, one per HDMI input. The active flag indicates the GPIO
+ level that enables hot-plug detection.
+
+The device node must contain one 'port' child node per device input and output
+port, in accordance with the video interface bindings defined in
+Documentation/devicetree/bindings/media/video-interfaces.txt. The port nodes
+are numbered as follows.
+
+ Port ADV7611
+------------------------------------------------------------
+ HDMI 0
+ Digital output 1
+
+The digital output port node must contain at least one endpoint.
+
+Optional Properties:
+
+ - reset-gpios: Reference to the GPIO connected to the device's reset pin.
+
+Optional Endpoint Properties:
+
+ The following three properties are defined in video-interfaces.txt and are
+ valid for source endpoints only.
+
+ - hsync-active: Horizontal synchronization polarity. Defaults to active low.
+ - vsync-active: Vertical synchronization polarity. Defaults to active low.
+ - pclk-sample: Pixel clock polarity. Defaults to output on the falling edge.
+
+ If none of hsync-active, vsync-active and pclk-sample is specified the
+ endpoint will use embedded BT.656 synchronization.
+
+
+Example:
+
+ hdmi_receiver@4c {
+ compatible = "adi,adv7611";
+ reg = <0x4c>;
+
+ reset-gpios = <&ioexp 0 GPIO_ACTIVE_LOW>;
+ hpd-gpios = <&ioexp 2 GPIO_ACTIVE_HIGH>;
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ };
+ port@1 {
+ reg = <1>;
+ hdmi_in: endpoint {
+ remote-endpoint = <&ccdc_in>;
+ };
+ };
+ };
diff --git a/Documentation/devicetree/bindings/media/renesas,vsp1.txt b/Documentation/devicetree/bindings/media/renesas,vsp1.txt
new file mode 100644
index 0000000..87fe08a
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/renesas,vsp1.txt
@@ -0,0 +1,43 @@
+* Renesas VSP1 Video Processing Engine
+
+The VSP1 is a video processing engine that supports up-/down-scaling, alpha
+blending, color space conversion and various other image processing features.
+It can be found in the Renesas R-Car second generation SoCs.
+
+Required properties:
+
+ - compatible: Must contain "renesas,vsp1"
+
+ - reg: Base address and length of the registers block for the VSP1.
+ - interrupts: VSP1 interrupt specifier.
+ - clocks: A phandle + clock-specifier pair for the VSP1 functional clock.
+
+ - renesas,#rpf: Number of Read Pixel Formatter (RPF) modules in the VSP1.
+ - renesas,#uds: Number of Up Down Scaler (UDS) modules in the VSP1.
+ - renesas,#wpf: Number of Write Pixel Formatter (WPF) modules in the VSP1.
+
+
+Optional properties:
+
+ - renesas,has-lif: Boolean, indicates that the LCD Interface (LIF) module is
+ available.
+ - renesas,has-lut: Boolean, indicates that the Look Up Table (LUT) module is
+ available.
+ - renesas,has-sru: Boolean, indicates that the Super Resolution Unit (SRU)
+ module is available.
+
+
+Example: R8A7790 (R-Car H2) VSP1-S node
+
+ vsp1@fe928000 {
+ compatible = "renesas,vsp1";
+ reg = <0 0xfe928000 0 0x8000>;
+ interrupts = <0 267 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&mstp1_clks R8A7790_CLK_VSP1_S>;
+
+ renesas,has-lut;
+ renesas,has-sru;
+ renesas,#rpf = <5>;
+ renesas,#uds = <3>;
+ renesas,#wpf = <4>;
+ };
diff --git a/Documentation/devicetree/bindings/mfd/bfticu.txt b/Documentation/devicetree/bindings/mfd/bfticu.txt
new file mode 100644
index 0000000..65c9077
--- /dev/null
+++ b/Documentation/devicetree/bindings/mfd/bfticu.txt
@@ -0,0 +1,25 @@
+KEYMILE bfticu Chassis Management FPGA
+
+The bfticu is a multifunction device that manages the whole chassis.
+Its main functionality is to collect IRQs from the whole chassis and signals
+them to a single controller.
+
+Required properties:
+- compatible: "keymile,bfticu"
+- interrupt-controller: the bfticu FPGA is an interrupt controller
+- interrupts: the main IRQ line to signal the collected IRQs
+- #interrupt-cells : is 2 and their usage is compliant to the 2 cells variant
+ of Documentation/devicetree/bindings/interrupt-controller/interrupts.txt
+- interrupt-parent: the parent IRQ ctrl the main IRQ is connected to
+- reg: access on the parent local bus (chip select, offset in chip select, size)
+
+Example:
+
+ chassis-mgmt@3,0 {
+ compatible = "keymile,bfticu";
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ reg = <3 0 0x100>;
+ interrupt-parent = <&mpic>;
+ interrupts = <6 1 0 0>;
+ };
diff --git a/Documentation/devicetree/bindings/mfd/qriox.txt b/Documentation/devicetree/bindings/mfd/qriox.txt
new file mode 100644
index 0000000..f301e2d
--- /dev/null
+++ b/Documentation/devicetree/bindings/mfd/qriox.txt
@@ -0,0 +1,17 @@
+KEYMILE qrio Board Control CPLD
+
+The qrio is a multifunction device that controls the KEYMILE boards based on
+the kmp204x design.
+It is consists of a reset controller, watchdog timer, LEDs, and 2 IRQ capable
+GPIO blocks.
+
+Required properties:
+- compatible: "keymile,qriox"
+- reg: access on the parent local bus (chip select, offset in chip select, size)
+
+Example:
+
+ board-control@1,0 {
+ compatible = "keymile,qriox";
+ reg = <1 0 0x80>;
+ };
diff --git a/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt b/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt
new file mode 100644
index 0000000..1f5a31fe
--- /dev/null
+++ b/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt
@@ -0,0 +1,59 @@
+* Allwinner PRCM (Power/Reset/Clock Management) Multi-Functional Device
+
+PRCM is an MFD device exposing several Power Management related devices
+(like clks and reset controllers).
+
+Required properties:
+ - compatible: "allwinner,sun6i-a31-prcm"
+ - reg: The PRCM registers range
+
+The prcm node may contain several subdevices definitions:
+ - see Documentation/devicetree/clk/sunxi.txt for clock devices
+ - see Documentation/devicetree/reset/allwinner,sunxi-clock-reset.txt for reset
+ controller devices
+
+
+Example:
+
+ prcm: prcm@01f01400 {
+ compatible = "allwinner,sun6i-a31-prcm";
+ reg = <0x01f01400 0x200>;
+
+ /* Put subdevices here */
+ ar100: ar100_clk {
+ compatible = "allwinner,sun6i-a31-ar100-clk";
+ #clock-cells = <0>;
+ clocks = <&osc32k>, <&osc24M>, <&pll6>, <&pll6>;
+ };
+
+ ahb0: ahb0_clk {
+ compatible = "fixed-factor-clock";
+ #clock-cells = <0>;
+ clock-div = <1>;
+ clock-mult = <1>;
+ clocks = <&ar100_div>;
+ clock-output-names = "ahb0";
+ };
+
+ apb0: apb0_clk {
+ compatible = "allwinner,sun6i-a31-apb0-clk";
+ #clock-cells = <0>;
+ clocks = <&ahb0>;
+ clock-output-names = "apb0";
+ };
+
+ apb0_gates: apb0_gates_clk {
+ compatible = "allwinner,sun6i-a31-apb0-gates-clk";
+ #clock-cells = <1>;
+ clocks = <&apb0>;
+ clock-output-names = "apb0_pio", "apb0_ir",
+ "apb0_timer01", "apb0_p2wi",
+ "apb0_uart", "apb0_1wire",
+ "apb0_i2c";
+ };
+
+ apb0_rst: apb0_rst {
+ compatible = "allwinner,sun6i-a31-clock-reset";
+ #reset-cells = <1>;
+ };
+ };
diff --git a/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt b/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt
new file mode 100644
index 0000000..20963c7
--- /dev/null
+++ b/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt
@@ -0,0 +1,19 @@
+* Device tree bindings for Texas Instruments keystone device state control
+
+The Keystone II devices have a set of registers that are used to control
+the status of its peripherals. This node is intended to allow access to
+this functionality.
+
+Required properties:
+
+- compatible: "ti,keystone-devctrl", "syscon"
+
+- reg: contains offset/length value for device state control
+ registers space.
+
+Example:
+
+devctrl: device-state-control@0x02620000 {
+ compatible = "ti,keystone-devctrl", "syscon";
+ reg = <0x02620000 0x1000>;
+};
diff --git a/Documentation/devicetree/bindings/mfd/twl6040.txt b/Documentation/devicetree/bindings/mfd/twl6040.txt
index 0f5dd70..a41157b 100644
--- a/Documentation/devicetree/bindings/mfd/twl6040.txt
+++ b/Documentation/devicetree/bindings/mfd/twl6040.txt
@@ -19,6 +19,8 @@
Optional properties, nodes:
- enable-active-high: To power on the twl6040 during boot.
+- clocks: phandle to the clk32k clock provider
+- clock-names: Must be "clk32k"
Vibra functionality
Required properties:
diff --git a/Documentation/devicetree/bindings/mmc/mmc.txt b/Documentation/devicetree/bindings/mmc/mmc.txt
index 9dce540..3c18001 100644
--- a/Documentation/devicetree/bindings/mmc/mmc.txt
+++ b/Documentation/devicetree/bindings/mmc/mmc.txt
@@ -38,6 +38,8 @@
- mmc-highspeed-ddr-1_2v: eMMC high-speed DDR mode(1.2V I/O) is supported
- mmc-hs200-1_8v: eMMC HS200 mode(1.8V I/O) is supported
- mmc-hs200-1_2v: eMMC HS200 mode(1.2V I/O) is supported
+- mmc-hs400-1_8v: eMMC HS400 mode(1.8V I/O) is supported
+- mmc-hs400-1_2v: eMMC HS400 mode(1.2V I/O) is supported
*NOTE* on CD and WP polarity. To use common for all SD/MMC host controllers line
polarity properties, we have to fix the meaning of the "normal" and "inverted"
diff --git a/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt b/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt
new file mode 100644
index 0000000..b638191
--- /dev/null
+++ b/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt
@@ -0,0 +1,30 @@
+MOXA ART MMC Host Controller Interface
+
+ Inherits from mmc binding[1].
+
+ [1] Documentation/devicetree/bindings/mmc/mmc.txt
+
+Required properties:
+
+- compatible : Must be "moxa,moxart-mmc" or "faraday,ftsdc010"
+- reg : Should contain registers location and length
+- interrupts : Should contain the interrupt number
+- clocks : Should contain phandle for the clock feeding the MMC controller
+
+Optional properties:
+
+- dmas : Should contain two DMA channels, line request number must be 5 for
+ both channels
+- dma-names : Must be "tx", "rx"
+
+Example:
+
+ mmc: mmc@98e00000 {
+ compatible = "moxa,moxart-mmc";
+ reg = <0x98e00000 0x5C>;
+ interrupts = <5 0>;
+ clocks = <&clk_apb>;
+ dmas = <&dma 5>,
+ <&dma 5>;
+ dma-names = "tx", "rx";
+ };
diff --git a/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt b/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt
new file mode 100644
index 0000000..91b3a34
--- /dev/null
+++ b/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt
@@ -0,0 +1,43 @@
+* Allwinner sunxi MMC controller
+
+The highspeed MMC host controller on Allwinner SoCs provides an interface
+for MMC, SD and SDIO types of memory cards.
+
+Supported maximum speeds are the ones of the eMMC standard 4.5 as well
+as the speed of SD standard 3.0.
+Absolute maximum transfer rate is 200MB/s
+
+Required properties:
+ - compatible : "allwinner,sun4i-a10-mmc" or "allwinner,sun5i-a13-mmc"
+ - reg : mmc controller base registers
+ - clocks : a list with 2 phandle + clock specifier pairs
+ - clock-names : must contain "ahb" and "mmc"
+ - interrupts : mmc controller interrupt
+
+Optional properties:
+ - resets : phandle + reset specifier pair
+ - reset-names : must contain "ahb"
+ - for cd, bus-width and additional generic mmc parameters
+ please refer to mmc.txt within this directory
+
+Examples:
+ - Within .dtsi:
+ mmc0: mmc@01c0f000 {
+ compatible = "allwinner,sun5i-a13-mmc";
+ reg = <0x01c0f000 0x1000>;
+ clocks = <&ahb_gates 8>, <&mmc0_clk>;
+ clock-names = "ahb", "mod";
+ interrupts = <0 32 4>;
+ status = "disabled";
+ };
+
+ - Within dts:
+ mmc0: mmc@01c0f000 {
+ pinctrl-names = "default", "default";
+ pinctrl-0 = <&mmc0_pins_a>;
+ pinctrl-1 = <&mmc0_cd_pin_reference_design>;
+ bus-width = <4>;
+ cd-gpios = <&pio 7 1 0>; /* PH1 */
+ cd-inverted;
+ status = "okay";
+ };
diff --git a/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt b/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt
index 8f3f133..2d4a725 100644
--- a/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt
+++ b/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt
@@ -69,10 +69,6 @@
* supports-highspeed: Enables support for high speed cards (up to 50MHz)
-* caps2-mmc-hs200-1_8v: Supports mmc HS200 SDR 1.8V mode
-
-* caps2-mmc-hs200-1_2v: Supports mmc HS200 SDR 1.2V mode
-
* broken-cd: as documented in mmc core bindings.
* vmmc-supply: The phandle to the regulator to use for vmmc. If this is
@@ -103,7 +99,6 @@
clock-freq-min-max = <400000 200000000>;
num-slots = <1>;
supports-highspeed;
- caps2-mmc-hs200-1_8v;
broken-cd;
fifo-depth = <0x80>;
card-detect-delay = <200>;
diff --git a/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt b/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt
new file mode 100644
index 0000000..8babdaa
--- /dev/null
+++ b/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt
@@ -0,0 +1,33 @@
+* Renesas usdhi6rol0 SD/SDIO host controller
+
+Required properties:
+
+- compatible: must be
+ "renesas,usdhi6rol0"
+- interrupts: 3 interrupts, named "card detect", "data" and "SDIO" must be
+ specified
+- clocks: a clock binding for the IMCLK input
+
+Optional properties:
+
+- vmmc-supply: a phandle of a regulator, supplying Vcc to the card
+- vqmmc-supply: a phandle of a regulator, supplying VccQ to the card
+
+Additionally any standard mmc bindings from mmc.txt can be used.
+
+Example:
+
+sd0: sd@ab000000 {
+ compatible = "renesas,usdhi6rol0";
+ reg = <0xab000000 0x200>;
+ interrupts = <0 23 0x4
+ 0 24 0x4
+ 0 25 0x4>;
+ interrupt-names = "card detect", "data", "SDIO";
+ bus-width = <4>;
+ max-frequency = <50000000>;
+ cap-power-off-card;
+ clocks = <&imclk>;
+ vmmc-supply = <&vcc_sd0>;
+ vqmmc-supply = <&vccq_sd0>;
+};
diff --git a/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt b/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt
new file mode 100644
index 0000000..823d134
--- /dev/null
+++ b/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt
@@ -0,0 +1,35 @@
+* Freescale Quad Serial Peripheral Interface(QuadSPI)
+
+Required properties:
+ - compatible : Should be "fsl,vf610-qspi"
+ - reg : the first contains the register location and length,
+ the second contains the memory mapping address and length
+ - reg-names: Should contain the reg names "QuadSPI" and "QuadSPI-memory"
+ - interrupts : Should contain the interrupt for the device
+ - clocks : The clocks needed by the QuadSPI controller
+ - clock-names : the name of the clocks
+
+Optional properties:
+ - fsl,qspi-has-second-chip: The controller has two buses, bus A and bus B.
+ Each bus can be connected with two NOR flashes.
+ Most of the time, each bus only has one NOR flash
+ connected, this is the default case.
+ But if there are two NOR flashes connected to the
+ bus, you should enable this property.
+ (Please check the board's schematic.)
+
+Example:
+
+qspi0: quadspi@40044000 {
+ compatible = "fsl,vf610-qspi";
+ reg = <0x40044000 0x1000>, <0x20000000 0x10000000>;
+ reg-names = "QuadSPI", "QuadSPI-memory";
+ interrupts = <0 24 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clks VF610_CLK_QSPI0_EN>,
+ <&clks VF610_CLK_QSPI0>;
+ clock-names = "qspi_en", "qspi";
+
+ flash0: s25fl128s@0 {
+ ....
+ };
+};
diff --git a/Documentation/devicetree/bindings/mtd/gpmc-nand.txt b/Documentation/devicetree/bindings/mtd/gpmc-nand.txt
index eb05255..65f4f7c 100644
--- a/Documentation/devicetree/bindings/mtd/gpmc-nand.txt
+++ b/Documentation/devicetree/bindings/mtd/gpmc-nand.txt
@@ -28,6 +28,8 @@
"ham1" 1-bit Hamming ecc code
"bch4" 4-bit BCH ecc code
"bch8" 8-bit BCH ecc code
+ "bch16" 16-bit BCH ECC code
+ Refer below "How to select correct ECC scheme for your device ?"
- ti,nand-xfer-type: A string setting the data transfer type. One of:
@@ -90,3 +92,46 @@
};
};
+How to select correct ECC scheme for your device ?
+--------------------------------------------------
+Higher ECC scheme usually means better protection against bit-flips and
+increased system lifetime. However, selection of ECC scheme is dependent
+on various other factors also like;
+
+(1) support of built in hardware engines.
+ Some legacy OMAP SoC do not have ELM harware engine, so those SoC cannot
+ support ecc-schemes with hardware error-correction (BCHx_HW). However
+ such SoC can use ecc-schemes with software library for error-correction
+ (BCHx_HW_DETECTION_SW). The error correction capability with software
+ library remains equivalent to their hardware counter-part, but there is
+ slight CPU penalty when too many bit-flips are detected during reads.
+
+(2) Device parameters like OOBSIZE.
+ Other factor which governs the selection of ecc-scheme is oob-size.
+ Higher ECC schemes require more OOB/Spare area to store ECC syndrome,
+ so the device should have enough free bytes available its OOB/Spare
+ area to accomodate ECC for entire page. In general following expression
+ helps in determining if given device can accomodate ECC syndrome:
+ "2 + (PAGESIZE / 512) * ECC_BYTES" >= OOBSIZE"
+ where
+ OOBSIZE number of bytes in OOB/spare area
+ PAGESIZE number of bytes in main-area of device page
+ ECC_BYTES number of ECC bytes generated to protect
+ 512 bytes of data, which is:
+ '3' for HAM1_xx ecc schemes
+ '7' for BCH4_xx ecc schemes
+ '14' for BCH8_xx ecc schemes
+ '26' for BCH16_xx ecc schemes
+
+ Example(a): For a device with PAGESIZE = 2048 and OOBSIZE = 64 and
+ trying to use BCH16 (ECC_BYTES=26) ecc-scheme.
+ Number of ECC bytes per page = (2 + (2048 / 512) * 26) = 106 B
+ which is greater than capacity of NAND device (OOBSIZE=64)
+ Hence, BCH16 cannot be supported on given device. But it can
+ probably use lower ecc-schemes like BCH8.
+
+ Example(b): For a device with PAGESIZE = 2048 and OOBSIZE = 128 and
+ trying to use BCH16 (ECC_BYTES=26) ecc-scheme.
+ Number of ECC bytes per page = (2 + (2048 / 512) * 26) = 106 B
+ which can be accomodate in the OOB/Spare area of this device
+ (OOBSIZE=128). So this device can use BCH16 ecc-scheme.
diff --git a/Documentation/devicetree/bindings/mtd/m25p80.txt b/Documentation/devicetree/bindings/mtd/m25p80.txt
index 6d3d576..4611aa8 100644
--- a/Documentation/devicetree/bindings/mtd/m25p80.txt
+++ b/Documentation/devicetree/bindings/mtd/m25p80.txt
@@ -5,8 +5,8 @@
representing partitions.
- compatible : Should be the manufacturer and the name of the chip. Bear in mind
the DT binding is not Linux-only, but in case of Linux, see the
- "m25p_ids" table in drivers/mtd/devices/m25p80.c for the list of
- supported chips.
+ "spi_nor_ids" table in drivers/mtd/spi-nor/spi-nor.c for the list
+ of supported chips.
- reg : Chip-Select number
- spi-max-frequency : Maximum frequency of the SPI bus the chip can operate at
diff --git a/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt b/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt
index 86e0a56..de8b517 100644
--- a/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt
+++ b/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt
@@ -17,6 +17,14 @@
- num-cs: Number of chipselect lines to usw
- nand-on-flash-bbt: boolean to enable on flash bbt option if
not present false
+ - nand-ecc-strength: number of bits to correct per ECC step
+ - nand-ecc-step-size: number of data bytes covered by a single ECC step
+
+The following ECC strength and step size are currently supported:
+
+ - nand-ecc-strength = <1>, nand-ecc-step-size = <512>
+ - nand-ecc-strength = <4>, nand-ecc-step-size = <512>
+ - nand-ecc-strength = <8>, nand-ecc-step-size = <512>
Example:
diff --git a/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt b/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt
new file mode 100644
index 0000000..db93921
--- /dev/null
+++ b/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt
@@ -0,0 +1,54 @@
+
+IBM Akebono board device tree
+=============================
+
+The IBM Akebono board is a development board for the PPC476GTR SoC.
+
+0) The root node
+
+ Required properties:
+
+ - model : "ibm,akebono".
+ - compatible : "ibm,akebono" , "ibm,476gtr".
+
+1.a) The Secure Digital Host Controller Interface (SDHCI) node
+
+ Represent the Secure Digital Host Controller Interfaces.
+
+ Required properties:
+
+ - compatible : should be "ibm,476gtr-sdhci","generic-sdhci".
+ - reg : should contain the SDHCI registers location and length.
+ - interrupt-parent : a phandle for the interrupt controller.
+ - interrupts : should contain the SDHCI interrupt.
+
+1.b) The Advanced Host Controller Interface (AHCI) SATA node
+
+ Represents the advanced host controller SATA interface.
+
+ Required properties:
+
+ - compatible : should be "ibm,476gtr-ahci".
+ - reg : should contain the AHCI registers location and length.
+ - interrupt-parent : a phandle for the interrupt controller.
+ - interrupts : should contain the AHCI interrupt.
+
+1.c) The FPGA node
+
+ The Akebono board stores some board information such as the revision
+ number in an FPGA which is represented by this node.
+
+ Required properties:
+
+ - compatible : should be "ibm,akebono-fpga".
+ - reg : should contain the FPGA registers location and length.
+
+1.d) The AVR node
+
+ The Akebono board has an Atmel AVR microprocessor attached to the I2C
+ bus as a power controller for the board.
+
+ Required properties:
+
+ - compatible : should be "ibm,akebono-avr".
+ - reg : should contain the I2C bus address for the AVR.
diff --git a/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt b/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt
new file mode 100644
index 0000000..c737c833
--- /dev/null
+++ b/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt
@@ -0,0 +1,19 @@
+
+ppc476gtr High Speed Serial Assist (HSTA) node
+==============================================
+
+The 476gtr SoC contains a high speed serial assist module attached
+between the plb4 and plb6 system buses to provide high speed data
+transfer between memory and system peripherals as well as support for
+PCI message signalled interrupts.
+
+Currently only the MSI support is used by Linux using the following
+device tree entries:
+
+Require properties:
+- compatible : "ibm,476gtr-hsta-msi", "ibm,hsta-msi"
+- reg : register mapping for the HSTA MSI space
+- interrupt-parent : parent controller for mapping interrupts
+- interrupts : ordered interrupt mapping for each MSI in the register
+ space. The first interrupt should be associated with a
+ register offset of 0x00, the second to 0x10, etc.
diff --git a/Documentation/devicetree/bindings/powerpc/fsl/board.txt b/Documentation/devicetree/bindings/powerpc/fsl/board.txt
index 380914e..700dec4 100644
--- a/Documentation/devicetree/bindings/powerpc/fsl/board.txt
+++ b/Documentation/devicetree/bindings/powerpc/fsl/board.txt
@@ -67,3 +67,20 @@
gpio-controller;
};
};
+
+* Freescale on-board FPGA connected on I2C bus
+
+Some Freescale boards like BSC9132QDS have on board FPGA connected on
+the i2c bus.
+
+Required properties:
+- compatible: Should be a board-specific string followed by a string
+ indicating the type of FPGA. Example:
+ "fsl,<board>-fpga", "fsl,fpga-qixis-i2c"
+- reg: Should contain the address of the FPGA
+
+Example:
+ fpga: fpga@66 {
+ compatible = "fsl,bsc9132qds-fpga", "fsl,fpga-qixis-i2c";
+ reg = <0x66>;
+ };
diff --git a/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt b/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt
new file mode 100644
index 0000000..454da7e
--- /dev/null
+++ b/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt
@@ -0,0 +1,46 @@
+Freescale CoreNet Coherency Fabric(CCF) Device Tree Binding
+
+DESCRIPTION
+
+The CoreNet coherency fabric is a fabric-oriented, connectivity infrastructure
+that enables the implementation of coherent, multicore systems.
+
+Required properties:
+
+- compatible: <string list>
+ fsl,corenet1-cf - CoreNet coherency fabric version 1.
+ Example chips: T4240, B4860
+
+ fsl,corenet2-cf - CoreNet coherency fabric version 2.
+ Example chips: P5040, P5020, P4080, P3041, P2041
+
+ fsl,corenet-cf - Used to represent the common registers
+ between CCF version 1 and CCF version 2. This compatible
+ is retained for compatibility reasons, as it was already
+ used for both CCF version 1 chips and CCF version 2
+ chips. It should be specified after either
+ "fsl,corenet1-cf" or "fsl,corenet2-cf".
+
+- reg: <prop-encoded-array>
+ A standard property. Represents the CCF registers.
+
+- interrupts: <prop-encoded-array>
+ Interrupt mapping for CCF error interrupt.
+
+- fsl,ccf-num-csdids: <u32>
+ Specifies the number of Coherency Subdomain ID Port Mapping
+ Registers that are supported by the CCF.
+
+- fsl,ccf-num-snoopids: <u32>
+ Specifies the number of Snoop ID Port Mapping Registers that
+ are supported by CCF.
+
+Example:
+
+ corenet-cf@18000 {
+ compatible = "fsl,corenet2-cf", "fsl,corenet-cf";
+ reg = <0x18000 0x1000>;
+ interrupts = <16 2 1 31>;
+ fsl,ccf-num-csdids = <32>;
+ fsl,ccf-num-snoopids = <32>;
+ };
diff --git a/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt b/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt
index 922c30a..f8cd239 100644
--- a/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt
+++ b/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt
@@ -20,3 +20,14 @@
a property named fsl,eref-[CAT], where [CAT] is the abbreviated category
name with all uppercase letters converted to lowercase, indicates that
the category is supported by the implementation.
+
+ - fsl,portid-mapping
+ Usage: optional
+ Value type: <u32>
+ Definition: The Coherency Subdomain ID Port Mapping Registers and
+ Snoop ID Port Mapping registers, which are part of the CoreNet
+ Coherency fabric (CCF), provide a CoreNet Coherency Subdomain
+ ID/CoreNet Snoop ID to cpu mapping functions. Certain bits from
+ these registers should be set if the coresponding CPU should be
+ snooped. This property defines a bitmask which selects the bit
+ that should be set if this cpu should be snooped.
diff --git a/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt b/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt
index 1f5e329..c2b2899 100644
--- a/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt
+++ b/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt
@@ -34,6 +34,15 @@
for legacy drivers.
- interrupt-parent : <phandle>
Phandle to interrupt controller
+- fsl,portid-mapping : <u32>
+ The Coherency Subdomain ID Port Mapping Registers and
+ Snoop ID Port Mapping registers, which are part of the
+ CoreNet Coherency fabric (CCF), provide a CoreNet
+ Coherency Subdomain ID/CoreNet Snoop ID to pamu mapping
+ functions. Certain bits from these registers should be
+ set if PAMUs should be snooped. This property defines
+ a bitmask which selects the bits that should be set if
+ PAMUs should be snooped.
Child nodes:
@@ -88,6 +97,7 @@
compatible = "fsl,pamu-v1.0", "fsl,pamu";
reg = <0x20000 0x5000>;
ranges = <0 0x20000 0x5000>;
+ fsl,portid-mapping = <0xf80000>;
#address-cells = <1>;
#size-cells = <1>;
interrupts = <
diff --git a/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt b/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt
index 31406fd..5c199ee0 100644
--- a/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt
+++ b/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt
@@ -9,6 +9,9 @@
- interrupts: rtc alarm/event interrupt
- #clock-cells: the value should be 0
+Optional properties:
+- clock-output-names: From common clock binding
+
Example:
hym8563: hym8563@51 {
diff --git a/Documentation/devicetree/bindings/rtc/xgene-rtc.txt b/Documentation/devicetree/bindings/rtc/xgene-rtc.txt
new file mode 100644
index 0000000..fd195c3
--- /dev/null
+++ b/Documentation/devicetree/bindings/rtc/xgene-rtc.txt
@@ -0,0 +1,28 @@
+* APM X-Gene Real Time Clock
+
+RTC controller for the APM X-Gene Real Time Clock
+
+Required properties:
+- compatible : Should be "apm,xgene-rtc"
+- reg: physical base address of the controller and length of memory mapped
+ region.
+- interrupts: IRQ line for the RTC.
+- #clock-cells: Should be 1.
+- clocks: Reference to the clock entry.
+
+Example:
+
+rtcclk: rtcclk {
+ compatible = "fixed-clock";
+ #clock-cells = <1>;
+ clock-frequency = <100000000>;
+ clock-output-names = "rtcclk";
+};
+
+rtc: rtc@10510000 {
+ compatible = "apm,xgene-rtc";
+ reg = <0x0 0x10510000 0x0 0x400>;
+ interrupts = <0x0 0x46 0x4>;
+ #clock-cells = <1>;
+ clocks = <&rtcclk 0>;
+};
diff --git a/Documentation/devicetree/bindings/vendor-prefixes.txt b/Documentation/devicetree/bindings/vendor-prefixes.txt
index 5261271..4d7f375 100644
--- a/Documentation/devicetree/bindings/vendor-prefixes.txt
+++ b/Documentation/devicetree/bindings/vendor-prefixes.txt
@@ -142,3 +142,4 @@
xes Extreme Engineering Solutions (X-ES)
xlnx Xilinx
zyxel ZyXEL Communications Corp.
+zarlink Zarlink Semiconductor
diff --git a/Documentation/devicetree/bindings/watchdog/marvel.txt b/Documentation/devicetree/bindings/watchdog/marvel.txt
index de11eb4..97223fd 100644
--- a/Documentation/devicetree/bindings/watchdog/marvel.txt
+++ b/Documentation/devicetree/bindings/watchdog/marvel.txt
@@ -5,11 +5,18 @@
- Compatibility : "marvell,orion-wdt"
"marvell,armada-370-wdt"
"marvell,armada-xp-wdt"
+ "marvell,armada-375-wdt"
+ "marvell,armada-380-wdt"
- reg : Should contain two entries: first one with the
timer control address, second one with the
rstout enable address.
+For "marvell,armada-375-wdt" and "marvell,armada-380-wdt":
+
+- reg : A third entry is mandatory and should contain the
+ shared mask/unmask RSTOUT address.
+
Optional properties:
- interrupts : Contains the IRQ for watchdog expiration
diff --git a/Documentation/efi-stub.txt b/Documentation/efi-stub.txt
index c628788..7747024 100644
--- a/Documentation/efi-stub.txt
+++ b/Documentation/efi-stub.txt
@@ -1,13 +1,21 @@
The EFI Boot Stub
---------------------------
-On the x86 platform, a bzImage can masquerade as a PE/COFF image,
-thereby convincing EFI firmware loaders to load it as an EFI
-executable. The code that modifies the bzImage header, along with the
-EFI-specific entry point that the firmware loader jumps to are
-collectively known as the "EFI boot stub", and live in
+On the x86 and ARM platforms, a kernel zImage/bzImage can masquerade
+as a PE/COFF image, thereby convincing EFI firmware loaders to load
+it as an EFI executable. The code that modifies the bzImage header,
+along with the EFI-specific entry point that the firmware loader
+jumps to are collectively known as the "EFI boot stub", and live in
arch/x86/boot/header.S and arch/x86/boot/compressed/eboot.c,
-respectively.
+respectively. For ARM the EFI stub is implemented in
+arch/arm/boot/compressed/efi-header.S and
+arch/arm/boot/compressed/efi-stub.c. EFI stub code that is shared
+between architectures is in drivers/firmware/efi/efi-stub-helper.c.
+
+For arm64, there is no compressed kernel support, so the Image itself
+masquerades as a PE/COFF image and the EFI stub is linked into the
+kernel. The arm64 EFI stub lives in arch/arm64/kernel/efi-entry.S
+and arch/arm64/kernel/efi-stub.c.
By using the EFI boot stub it's possible to boot a Linux kernel
without the use of a conventional EFI boot loader, such as grub or
@@ -23,7 +31,10 @@
System Partition (ESP) and renamed with the extension ".efi". Without
the extension the EFI firmware loader will refuse to execute it. It's
not possible to execute bzImage.efi from the usual Linux file systems
-because EFI firmware doesn't have support for them.
+because EFI firmware doesn't have support for them. For ARM the
+arch/arm/boot/zImage should be copied to the system partition, and it
+may not need to be renamed. Similarly for arm64, arch/arm64/boot/Image
+should be copied but not necessarily renamed.
**** Passing kernel parameters from the EFI shell
@@ -63,3 +74,11 @@
because the image we're executing is interpreted by the EFI shell,
which understands relative paths, whereas the rest of the command line
is passed to bzImage.efi.
+
+
+**** The "dtb=" option
+
+For the ARM and arm64 architectures, we also need to be able to provide a
+device tree to the kernel. This is done with the "dtb=" command line option,
+and is processed in the same manner as the "initrd=" option that is
+described above.
diff --git a/Documentation/filesystems/f2fs.txt b/Documentation/filesystems/f2fs.txt
index 25311e11..51afba1 100644
--- a/Documentation/filesystems/f2fs.txt
+++ b/Documentation/filesystems/f2fs.txt
@@ -461,11 +461,11 @@
# of blocks in level #n = |
`- 4, Otherwise
- ,- 2^ (n + dir_level),
- | if n < MAX_DIR_HASH_DEPTH / 2,
+ ,- 2^(n + dir_level),
+ | if n + dir_level < MAX_DIR_HASH_DEPTH / 2,
# of buckets in level #n = |
- `- 2^((MAX_DIR_HASH_DEPTH / 2 + dir_level) - 1),
- Otherwise
+ `- 2^((MAX_DIR_HASH_DEPTH / 2) - 1),
+ Otherwise
When F2FS finds a file name in a directory, at first a hash value of the file
name is calculated. Then, F2FS scans the hash table in level #0 to find the
diff --git a/Documentation/filesystems/nfs/nfs41-server.txt b/Documentation/filesystems/nfs/nfs41-server.txt
index b930ad0..c49cd7e 100644
--- a/Documentation/filesystems/nfs/nfs41-server.txt
+++ b/Documentation/filesystems/nfs/nfs41-server.txt
@@ -176,7 +176,5 @@
ca_maxrequestsize request and a ca_maxresponsesize reply, so we may
fail to live up to the promise we made in CREATE_SESSION fore channel
negotiation.
-* No more than one read-like operation allowed per compound; encoding
- replies that cross page boundaries (except for read data) not handled.
See also http://wiki.linux-nfs.org/wiki/index.php/Server_4.0_and_4.1_issues.
diff --git a/Documentation/filesystems/seq_file.txt b/Documentation/filesystems/seq_file.txt
index a1e2e0d..1fe0ccb 100644
--- a/Documentation/filesystems/seq_file.txt
+++ b/Documentation/filesystems/seq_file.txt
@@ -54,6 +54,15 @@
wanting to see the full source for this module can find it at
http://lwn.net/Articles/22359/).
+Deprecated create_proc_entry
+
+Note that the above article uses create_proc_entry which was removed in
+kernel 3.10. Current versions require the following update
+
+- entry = create_proc_entry("sequence", 0, NULL);
+- if (entry)
+- entry->proc_fops = &ct_file_ops;
++ entry = proc_create("sequence", 0, NULL, &ct_file_ops);
The iterator interface
diff --git a/Documentation/filesystems/vfat.txt b/Documentation/filesystems/vfat.txt
index 4a93e98..ce1126a 100644
--- a/Documentation/filesystems/vfat.txt
+++ b/Documentation/filesystems/vfat.txt
@@ -172,6 +172,11 @@
To maintain backward compatibility, '-o nfs' is also accepted,
defaulting to stale_rw
+dos1xfloppy -- If set, use a fallback default BIOS Parameter Block
+ configuration, determined by backing device size. These static
+ parameters match defaults assumed by DOS 1.x for 160 kiB,
+ 180 kiB, 320 kiB, and 360 kiB floppies and floppy images.
+
<bool>: 0,1,yes,no,true,false
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index 9973a7e..b9f6778 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -2361,6 +2361,14 @@
timeout < 0: reboot immediately
Format: <timeout>
+ crash_kexec_post_notifiers
+ Run kdump after running panic-notifiers and dumping
+ kmsg. This only for the users who doubt kdump always
+ succeeds in any situation.
+ Note that this also increases risks of kdump failure,
+ because some panic notifiers can make the crashed
+ kernel more unstable.
+
parkbd.port= [HW] Parallel port number the keyboard adapter is
connected to, default is 0.
Format: <parport#>
diff --git a/Documentation/kmemleak.txt b/Documentation/kmemleak.txt
index a7563ec..b772418 100644
--- a/Documentation/kmemleak.txt
+++ b/Documentation/kmemleak.txt
@@ -142,6 +142,7 @@
kmemleak_free - notify of a memory block freeing
kmemleak_free_part - notify of a partial memory block freeing
kmemleak_free_percpu - notify of a percpu memory block freeing
+kmemleak_update_trace - update object allocation stack trace
kmemleak_not_leak - mark an object as not a leak
kmemleak_ignore - do not scan or report an object as leak
kmemleak_scan_area - add scan areas inside a memory block
diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index 46412bd..f1dc4a2 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -115,8 +115,8 @@
CPU 1 CPU 2
=============== ===============
{ A == 1; B == 2 }
- A = 3; x = A;
- B = 4; y = B;
+ A = 3; x = B;
+ B = 4; y = A;
The set of accesses as seen by the memory system in the middle can be arranged
in 24 different combinations:
diff --git a/Documentation/mtd/spi-nor.txt b/Documentation/mtd/spi-nor.txt
new file mode 100644
index 0000000..548d630
--- /dev/null
+++ b/Documentation/mtd/spi-nor.txt
@@ -0,0 +1,62 @@
+ SPI NOR framework
+ ============================================
+
+Part I - Why do we need this framework?
+---------------------------------------
+
+SPI bus controllers (drivers/spi/) only deal with streams of bytes; the bus
+controller operates agnostic of the specific device attached. However, some
+controllers (such as Freescale's QuadSPI controller) cannot easily handle
+arbitrary streams of bytes, but rather are designed specifically for SPI NOR.
+
+In particular, Freescale's QuadSPI controller must know the NOR commands to
+find the right LUT sequence. Unfortunately, the SPI subsystem has no notion of
+opcodes, addresses, or data payloads; a SPI controller simply knows to send or
+receive bytes (Tx and Rx). Therefore, we must define a new layering scheme under
+which the controller driver is aware of the opcodes, addressing, and other
+details of the SPI NOR protocol.
+
+Part II - How does the framework work?
+--------------------------------------
+
+This framework just adds a new layer between the MTD and the SPI bus driver.
+With this new layer, the SPI NOR controller driver does not depend on the
+m25p80 code anymore.
+
+ Before this framework, the layer is like:
+
+ MTD
+ ------------------------
+ m25p80
+ ------------------------
+ SPI bus driver
+ ------------------------
+ SPI NOR chip
+
+ After this framework, the layer is like:
+ MTD
+ ------------------------
+ SPI NOR framework
+ ------------------------
+ m25p80
+ ------------------------
+ SPI bus driver
+ ------------------------
+ SPI NOR chip
+
+ With the SPI NOR controller driver (Freescale QuadSPI), it looks like:
+ MTD
+ ------------------------
+ SPI NOR framework
+ ------------------------
+ fsl-quadSPI
+ ------------------------
+ SPI NOR chip
+
+Part III - How can drivers use the framework?
+---------------------------------------------
+
+The main API is spi_nor_scan(). Before you call the hook, a driver should
+initialize the necessary fields for spi_nor{}. Please see
+drivers/mtd/spi-nor/spi-nor.c for detail. Please also refer to fsl-quadspi.c
+when you want to write a new driver for a SPI NOR controller.
diff --git a/Documentation/platform/x86-laptop-drivers.txt b/Documentation/platform/x86-laptop-drivers.txt
new file mode 100644
index 0000000..01facd2
--- /dev/null
+++ b/Documentation/platform/x86-laptop-drivers.txt
@@ -0,0 +1,18 @@
+compal-laptop
+=============
+List of supported hardware:
+
+by Compal:
+ Compal FL90/IFL90
+ Compal FL91/IFL91
+ Compal FL92/JFL92
+ Compal FT00/IFT00
+
+by Dell:
+ Dell Vostro 1200
+ Dell Mini 9 (Inspiron 910)
+ Dell Mini 10 (Inspiron 1010)
+ Dell Mini 10v (Inspiron 1011)
+ Dell Mini 1012 (Inspiron 1012)
+ Dell Inspiron 11z (Inspiron 1110)
+ Dell Mini 12 (Inspiron 1210)
diff --git a/Documentation/powerpc/cpu_families.txt b/Documentation/powerpc/cpu_families.txt
new file mode 100644
index 0000000..fc08e22
--- /dev/null
+++ b/Documentation/powerpc/cpu_families.txt
@@ -0,0 +1,221 @@
+CPU Families
+============
+
+This document tries to summarise some of the different cpu families that exist
+and are supported by arch/powerpc.
+
+
+Book3S (aka sPAPR)
+------------------
+
+ - Hash MMU
+ - Mix of 32 & 64 bit
+
+ +--------------+ +----------------+
+ | Old POWER | --------------> | RS64 (threads) |
+ +--------------+ +----------------+
+ |
+ |
+ v
+ +--------------+ +----------------+ +------+
+ | 601 | --------------> | 603 | ---> | e300 |
+ +--------------+ +----------------+ +------+
+ | |
+ | |
+ v v
+ +--------------+ +----------------+ +-------+
+ | 604 | | 750 (G3) | ---> | 750CX |
+ +--------------+ +----------------+ +-------+
+ | | |
+ | | |
+ v v v
+ +--------------+ +----------------+ +-------+
+ | 620 (64 bit) | | 7400 | | 750CL |
+ +--------------+ +----------------+ +-------+
+ | | |
+ | | |
+ v v v
+ +--------------+ +----------------+ +-------+
+ | POWER3/630 | | 7410 | | 750FX |
+ +--------------+ +----------------+ +-------+
+ | |
+ | |
+ v v
+ +--------------+ +----------------+
+ | POWER3+ | | 7450 |
+ +--------------+ +----------------+
+ | |
+ | |
+ v v
+ +--------------+ +----------------+
+ | POWER4 | | 7455 |
+ +--------------+ +----------------+
+ | |
+ | |
+ v v
+ +--------------+ +-------+ +----------------+
+ | POWER4+ | --> | 970 | | 7447 |
+ +--------------+ +-------+ +----------------+
+ | | |
+ | | |
+ v v v
+ +--------------+ +-------+ +----------------+
+ | POWER5 | | 970FX | | 7448 |
+ +--------------+ +-------+ +----------------+
+ | | |
+ | | |
+ v v v
+ +--------------+ +-------+ +----------------+
+ | POWER5+ | | 970MP | | e600 |
+ +--------------+ +-------+ +----------------+
+ |
+ |
+ v
+ +--------------+
+ | POWER5++ |
+ +--------------+
+ |
+ |
+ v
+ +--------------+ +-------+
+ | POWER6 | <-?-> | Cell |
+ +--------------+ +-------+
+ |
+ |
+ v
+ +--------------+
+ | POWER7 |
+ +--------------+
+ |
+ |
+ v
+ +--------------+
+ | POWER7+ |
+ +--------------+
+ |
+ |
+ v
+ +--------------+
+ | POWER8 |
+ +--------------+
+
+
+ +---------------+
+ | PA6T (64 bit) |
+ +---------------+
+
+
+IBM BookE
+---------
+
+ - Software loaded TLB.
+ - All 32 bit
+
+ +--------------+
+ | 401 |
+ +--------------+
+ |
+ |
+ v
+ +--------------+
+ | 403 |
+ +--------------+
+ |
+ |
+ v
+ +--------------+
+ | 405 |
+ +--------------+
+ |
+ |
+ v
+ +--------------+
+ | 440 |
+ +--------------+
+ |
+ |
+ v
+ +--------------+ +----------------+
+ | 450 | --> | BG/P |
+ +--------------+ +----------------+
+ |
+ |
+ v
+ +--------------+
+ | 460 |
+ +--------------+
+ |
+ |
+ v
+ +--------------+
+ | 476 |
+ +--------------+
+
+
+Motorola/Freescale 8xx
+----------------------
+
+ - Software loaded with hardware assist.
+ - All 32 bit
+
+ +-------------+
+ | MPC8xx Core |
+ +-------------+
+
+
+Freescale BookE
+---------------
+
+ - Software loaded TLB.
+ - e6500 adds HW loaded indirect TLB entries.
+ - Mix of 32 & 64 bit
+
+ +--------------+
+ | e200 |
+ +--------------+
+
+
+ +--------------------------------+
+ | e500 |
+ +--------------------------------+
+ |
+ |
+ v
+ +--------------------------------+
+ | e500v2 |
+ +--------------------------------+
+ |
+ |
+ v
+ +--------------------------------+
+ | e500mc (Book3e) |
+ +--------------------------------+
+ |
+ |
+ v
+ +--------------------------------+
+ | e5500 (64 bit) |
+ +--------------------------------+
+ |
+ |
+ v
+ +--------------------------------+
+ | e6500 (HW TLB) (Multithreaded) |
+ +--------------------------------+
+
+
+IBM A2 core
+-----------
+
+ - Book3E, software loaded TLB + HW loaded indirect TLB entries.
+ - 64 bit
+
+ +--------------+ +----------------+
+ | A2 core | --> | WSP |
+ +--------------+ +----------------+
+ |
+ |
+ v
+ +--------------+
+ | BG/Q |
+ +--------------+
diff --git a/Documentation/scsi/LICENSE.qla2xxx b/Documentation/scsi/LICENSE.qla2xxx
index 5020b7b..52f0b43 100644
--- a/Documentation/scsi/LICENSE.qla2xxx
+++ b/Documentation/scsi/LICENSE.qla2xxx
@@ -1,4 +1,4 @@
-Copyright (c) 2003-2013 QLogic Corporation
+Copyright (c) 2003-2014 QLogic Corporation
QLogic Linux FC-FCoE Driver
This program includes a device driver for Linux 3.x.
diff --git a/Documentation/security/Smack.txt b/Documentation/security/Smack.txt
index 5ea996f..b6ef7e9 100644
--- a/Documentation/security/Smack.txt
+++ b/Documentation/security/Smack.txt
@@ -204,6 +204,16 @@
these capabilities are effective at for processes with any
label. The value is set by writing the desired label to the
file or cleared by writing "-" to the file.
+ptrace
+ This is used to define the current ptrace policy
+ 0 - default: this is the policy that relies on smack access rules.
+ For the PTRACE_READ a subject needs to have a read access on
+ object. For the PTRACE_ATTACH a read-write access is required.
+ 1 - exact: this is the policy that limits PTRACE_ATTACH. Attach is
+ only allowed when subject's and object's labels are equal.
+ PTRACE_READ is not affected. Can be overriden with CAP_SYS_PTRACE.
+ 2 - draconian: this policy behaves like the 'exact' above with an
+ exception that it can't be overriden with CAP_SYS_PTRACE.
revoke-subject
Writing a Smack label here sets the access to '-' for all access
rules with that subject label.
diff --git a/Documentation/sysctl/kernel.txt b/Documentation/sysctl/kernel.txt
index 9886c3d..708bb7f 100644
--- a/Documentation/sysctl/kernel.txt
+++ b/Documentation/sysctl/kernel.txt
@@ -77,6 +77,7 @@
- shmmni
- stop-a [ SPARC only ]
- sysrq ==> Documentation/sysrq.txt
+- sysctl_writes_strict
- tainted
- threads-max
- unknown_nmi_panic
@@ -762,6 +763,26 @@
==============================================================
+sysctl_writes_strict:
+
+Control how file position affects the behavior of updating sysctl values
+via the /proc/sys interface:
+
+ -1 - Legacy per-write sysctl value handling, with no printk warnings.
+ Each write syscall must fully contain the sysctl value to be
+ written, and multiple writes on the same sysctl file descriptor
+ will rewrite the sysctl value, regardless of file position.
+ 0 - (default) Same behavior as above, but warn about processes that
+ perform writes to a sysctl file descriptor when the file position
+ is not 0.
+ 1 - Respect file position when writing sysctl strings. Multiple writes
+ will append to the sysctl value buffer. Anything past the max length
+ of the sysctl value buffer will be ignored. Writes to numeric sysctl
+ entries must always be at file position 0 and the value must be
+ fully contained in the buffer sent in the write syscall.
+
+==============================================================
+
tainted:
Non-zero if the kernel has been tainted. Numeric values, which
diff --git a/Documentation/trace/ftrace.txt b/Documentation/trace/ftrace.txt
index bd36598..2479b2a 100644
--- a/Documentation/trace/ftrace.txt
+++ b/Documentation/trace/ftrace.txt
@@ -2003,6 +2003,32 @@
360.774530 | 1) 0.594 us | __phys_addr();
+The function name is always displayed after the closing bracket
+for a function if the start of that function is not in the
+trace buffer.
+
+Display of the function name after the closing bracket may be
+enabled for functions whose start is in the trace buffer,
+allowing easier searching with grep for function durations.
+It is default disabled.
+
+ hide: echo nofuncgraph-tail > trace_options
+ show: echo funcgraph-tail > trace_options
+
+ Example with nofuncgraph-tail (default):
+ 0) | putname() {
+ 0) | kmem_cache_free() {
+ 0) 0.518 us | __phys_addr();
+ 0) 1.757 us | }
+ 0) 2.861 us | }
+
+ Example with funcgraph-tail:
+ 0) | putname() {
+ 0) | kmem_cache_free() {
+ 0) 0.518 us | __phys_addr();
+ 0) 1.757 us | } /* kmem_cache_free() */
+ 0) 2.861 us | } /* putname() */
+
You can put some comments on specific functions by using
trace_printk() For example, if you want to put a comment inside
the __might_sleep() function, you just have to include
diff --git a/Documentation/trace/tracepoints.txt b/Documentation/trace/tracepoints.txt
index 6b018b5..a3efac6 100644
--- a/Documentation/trace/tracepoints.txt
+++ b/Documentation/trace/tracepoints.txt
@@ -115,6 +115,30 @@
EXPORT_TRACEPOINT_SYMBOL_GPL() or EXPORT_TRACEPOINT_SYMBOL() can be
used to export the defined tracepoints.
+If you need to do a bit of work for a tracepoint parameter, and
+that work is only used for the tracepoint, that work can be encapsulated
+within an if statement with the following:
+
+ if (trace_foo_bar_enabled()) {
+ int i;
+ int tot = 0;
+
+ for (i = 0; i < count; i++)
+ tot += calculate_nuggets();
+
+ trace_foo_bar(tot);
+ }
+
+All trace_<tracepoint>() calls have a matching trace_<tracepoint>_enabled()
+function defined that returns true if the tracepoint is enabled and
+false otherwise. The trace_<tracepoint>() should always be within the
+block of the if (trace_<tracepoint>_enabled()) to prevent races between
+the tracepoint being enabled and the check being seen.
+
+The advantage of using the trace_<tracepoint>_enabled() is that it uses
+the static_key of the tracepoint to allow the if statement to be implemented
+with jump labels and avoid conditional branches.
+
Note: The convenience macro TRACE_EVENT provides an alternative way to
define tracepoints. Check http://lwn.net/Articles/379903,
http://lwn.net/Articles/381064 and http://lwn.net/Articles/383362
diff --git a/Documentation/vm/remap_file_pages.txt b/Documentation/vm/remap_file_pages.txt
new file mode 100644
index 0000000..560e436
--- /dev/null
+++ b/Documentation/vm/remap_file_pages.txt
@@ -0,0 +1,28 @@
+The remap_file_pages() system call is used to create a nonlinear mapping,
+that is, a mapping in which the pages of the file are mapped into a
+nonsequential order in memory. The advantage of using remap_file_pages()
+over using repeated calls to mmap(2) is that the former approach does not
+require the kernel to create additional VMA (Virtual Memory Area) data
+structures.
+
+Supporting of nonlinear mapping requires significant amount of non-trivial
+code in kernel virtual memory subsystem including hot paths. Also to get
+nonlinear mapping work kernel need a way to distinguish normal page table
+entries from entries with file offset (pte_file). Kernel reserves flag in
+PTE for this purpose. PTE flags are scarce resource especially on some CPU
+architectures. It would be nice to free up the flag for other usage.
+
+Fortunately, there are not many users of remap_file_pages() in the wild.
+It's only known that one enterprise RDBMS implementation uses the syscall
+on 32-bit systems to map files bigger than can linearly fit into 32-bit
+virtual address space. This use-case is not critical anymore since 64-bit
+systems are widely available.
+
+The plan is to deprecate the syscall and replace it with an emulation.
+The emulation will create new VMAs instead of nonlinear mappings. It's
+going to work slower for rare users of remap_file_pages() but ABI is
+preserved.
+
+One side effect of emulation (apart from performance) is that user can hit
+vm.max_map_count limit more easily due to additional VMAs. See comment for
+DEFAULT_MAX_MAP_COUNT for more details on the limit.
diff --git a/MAINTAINERS b/MAINTAINERS
index 7d101d5..b4a66b9 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -210,6 +210,13 @@
F: Documentation/scsi/aacraid.txt
F: drivers/scsi/aacraid/
+ABI/API
+L: linux-api@vger.kernel.org
+F: Documentation/ABI/
+F: include/linux/syscalls.h
+F: include/uapi/
+F: kernel/sys_ni.c
+
ABIT UGURU 1,2 HARDWARE MONITOR DRIVER
M: Hans de Goede <hdegoede@redhat.com>
L: lm-sensors@lm-sensors.org
@@ -647,7 +654,7 @@
F: sound/soc/codecs/sigmadsp.*
ANALOG DEVICES INC ASOC DRIVERS
-L: adi-buildroot-devel@lists.sourceforge.net
+L: adi-buildroot-devel@lists.sourceforge.net (moderated for non-subscribers)
L: alsa-devel@alsa-project.org (moderated for non-subscribers)
W: http://blackfin.uclinux.org/
S: Supported
@@ -808,6 +815,11 @@
F: arch/arm/boot/dts/sama*.dts
F: arch/arm/boot/dts/sama*.dtsi
+ARM/ATMEL AT91 Clock Support
+M: Boris Brezillon <boris.brezillon@free-electrons.com>
+S: Maintained
+F: drivers/clk/at91
+
ARM/CALXEDA HIGHBANK ARCHITECTURE
M: Rob Herring <robh@kernel.org>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
@@ -1758,54 +1770,54 @@
BLACKFIN ARCHITECTURE
M: Steven Miao <realmz6@gmail.com>
-L: adi-buildroot-devel@lists.sourceforge.net
+L: adi-buildroot-devel@lists.sourceforge.net (moderated for non-subscribers)
T: git git://git.code.sf.net/p/adi-linux/code
W: http://blackfin.uclinux.org
S: Supported
F: arch/blackfin/
BLACKFIN EMAC DRIVER
-L: adi-buildroot-devel@lists.sourceforge.net
+L: adi-buildroot-devel@lists.sourceforge.net (moderated for non-subscribers)
W: http://blackfin.uclinux.org
S: Supported
F: drivers/net/ethernet/adi/
BLACKFIN RTC DRIVER
-L: adi-buildroot-devel@lists.sourceforge.net
+L: adi-buildroot-devel@lists.sourceforge.net (moderated for non-subscribers)
W: http://blackfin.uclinux.org
S: Supported
F: drivers/rtc/rtc-bfin.c
BLACKFIN SDH DRIVER
M: Sonic Zhang <sonic.zhang@analog.com>
-L: adi-buildroot-devel@lists.sourceforge.net
+L: adi-buildroot-devel@lists.sourceforge.net (moderated for non-subscribers)
W: http://blackfin.uclinux.org
S: Supported
F: drivers/mmc/host/bfin_sdh.c
BLACKFIN SERIAL DRIVER
M: Sonic Zhang <sonic.zhang@analog.com>
-L: adi-buildroot-devel@lists.sourceforge.net
+L: adi-buildroot-devel@lists.sourceforge.net (moderated for non-subscribers)
W: http://blackfin.uclinux.org
S: Supported
F: drivers/tty/serial/bfin_uart.c
BLACKFIN WATCHDOG DRIVER
-L: adi-buildroot-devel@lists.sourceforge.net
+L: adi-buildroot-devel@lists.sourceforge.net (moderated for non-subscribers)
W: http://blackfin.uclinux.org
S: Supported
F: drivers/watchdog/bfin_wdt.c
BLACKFIN I2C TWI DRIVER
M: Sonic Zhang <sonic.zhang@analog.com>
-L: adi-buildroot-devel@lists.sourceforge.net
+L: adi-buildroot-devel@lists.sourceforge.net (moderated for non-subscribers)
W: http://blackfin.uclinux.org/
S: Supported
F: drivers/i2c/busses/i2c-bfin-twi.c
BLACKFIN MEDIA DRIVER
M: Scott Jiang <scott.jiang.linux@gmail.com>
-L: adi-buildroot-devel@lists.sourceforge.net
+L: adi-buildroot-devel@lists.sourceforge.net (moderated for non-subscribers)
W: http://blackfin.uclinux.org/
S: Supported
F: drivers/media/platform/blackfin/
@@ -2372,16 +2384,35 @@
S: Maintained
F: drivers/connector/
-CONTROL GROUPS (CGROUPS)
+CONTROL GROUP (CGROUP)
M: Tejun Heo <tj@kernel.org>
M: Li Zefan <lizefan@huawei.com>
-L: containers@lists.linux-foundation.org
L: cgroups@vger.kernel.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git
S: Maintained
+F: Documentation/cgroups/
F: include/linux/cgroup*
F: kernel/cgroup*
-F: mm/*cgroup*
+
+CONTROL GROUP - CPUSET
+M: Li Zefan <lizefan@huawei.com>
+L: cgroups@vger.kernel.org
+W: http://www.bullopensource.org/cpuset/
+W: http://oss.sgi.com/projects/cpusets/
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git
+S: Maintained
+F: Documentation/cgroups/cpusets.txt
+F: include/linux/cpuset.h
+F: kernel/cpuset.c
+
+CONTROL GROUP - MEMORY RESOURCE CONTROLLER (MEMCG)
+M: Johannes Weiner <hannes@cmpxchg.org>
+M: Michal Hocko <mhocko@suse.cz>
+L: cgroups@vger.kernel.org
+L: linux-mm@kvack.org
+S: Maintained
+F: mm/memcontrol.c
+F: mm/page_cgroup.c
CORETEMP HARDWARE MONITORING DRIVER
M: Fenghua Yu <fenghua.yu@intel.com>
@@ -2452,17 +2483,6 @@
S: Maintained
F: tools/power/cpupower/
-CPUSETS
-M: Li Zefan <lizefan@huawei.com>
-L: cgroups@vger.kernel.org
-W: http://www.bullopensource.org/cpuset/
-W: http://oss.sgi.com/projects/cpusets/
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git
-S: Maintained
-F: Documentation/cgroups/cpusets.txt
-F: include/linux/cpuset.h
-F: kernel/cpuset.c
-
CRAMFS FILESYSTEM
W: http://sourceforge.net/projects/cramfs/
S: Orphan / Obsolete
@@ -2680,6 +2700,15 @@
F: Documentation/networking/decnet.txt
F: net/decnet/
+DECSTATION PLATFORM SUPPORT
+M: "Maciej W. Rozycki" <macro@linux-mips.org>
+L: linux-mips@linux-mips.org
+W: http://www.linux-mips.org/wiki/DECstation
+S: Maintained
+F: arch/mips/dec/
+F: arch/mips/include/asm/dec/
+F: arch/mips/include/asm/mach-dec/
+
DEFXX FDDI NETWORK DRIVER
M: "Maciej W. Rozycki" <macro@linux-mips.org>
S: Maintained
@@ -3148,10 +3177,9 @@
F: drivers/scsi/eata_pio.*
EBTABLES
-M: Bart De Schuymer <bart.de.schuymer@pandora.be>
L: netfilter-devel@vger.kernel.org
W: http://ebtables.sourceforge.net/
-S: Maintained
+S: Orphan
F: include/linux/netfilter_bridge/ebt_*.h
F: include/uapi/linux/netfilter_bridge/ebt_*.h
F: net/bridge/netfilter/ebt*.c
@@ -3764,7 +3792,8 @@
F: include/linux/fscache*.h
F2FS FILE SYSTEM
-M: Jaegeuk Kim <jaegeuk.kim@samsung.com>
+M: Jaegeuk Kim <jaegeuk@kernel.org>
+M: Changman Lee <cm224.lee@samsung.com>
L: linux-f2fs-devel@lists.sourceforge.net
W: http://en.wikipedia.org/wiki/F2FS
T: git git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git
@@ -4362,7 +4391,7 @@
F: drivers/crypto/nx/
IBM Power 842 compression accelerator
-M: Robert Jennings <rcj@linux.vnet.ibm.com>
+M: Nathan Fontenot <nfont@linux.vnet.ibm.com>
S: Supported
F: drivers/crypto/nx/nx-842.c
F: include/linux/nx842.h
@@ -4378,12 +4407,18 @@
S: Supported
F: drivers/net/ethernet/ibm/ibmveth.*
-IBM Power Virtual SCSI/FC Device Drivers
-M: Robert Jennings <rcj@linux.vnet.ibm.com>
+IBM Power Virtual SCSI Device Drivers
+M: Nathan Fontenot <nfont@linux.vnet.ibm.com>
L: linux-scsi@vger.kernel.org
S: Supported
-F: drivers/scsi/ibmvscsi/
-X: drivers/scsi/ibmvscsi/ibmvstgt.c
+F: drivers/scsi/ibmvscsi/ibmvscsi*
+F: drivers/scsi/ibmvscsi/viosrp.h
+
+IBM Power Virtual FC Device Drivers
+M: Brian King <brking@linux.vnet.ibm.com>
+L: linux-scsi@vger.kernel.org
+S: Supported
+F: drivers/scsi/ibmvscsi/ibmvfc*
IBM ServeRAID RAID DRIVER
P: Jack Hammer
@@ -5746,17 +5781,6 @@
F: include/linux/vmalloc.h
F: mm/
-MEMORY RESOURCE CONTROLLER
-M: Johannes Weiner <hannes@cmpxchg.org>
-M: Michal Hocko <mhocko@suse.cz>
-M: Balbir Singh <bsingharora@gmail.com>
-M: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
-L: cgroups@vger.kernel.org
-L: linux-mm@kvack.org
-S: Maintained
-F: mm/memcontrol.c
-F: mm/page_cgroup.c
-
MEMORY TECHNOLOGY DEVICES (MTD)
M: David Woodhouse <dwmw2@infradead.org>
M: Brian Norris <computersforpeace@gmail.com>
@@ -5950,6 +5974,7 @@
M: Ulf Hansson <ulf.hansson@linaro.org>
L: linux-mmc@vger.kernel.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/cjb/mmc.git
+T: git git://git.linaro.org/people/ulf.hansson/mmc.git
S: Maintained
F: drivers/mmc/
F: include/linux/mmc/
@@ -6011,6 +6036,28 @@
S: Odd Fixes
F: fs/ncpfs/
+NCR 5380 SCSI DRIVERS
+M: Finn Thain <fthain@telegraphics.com.au>
+M: Michael Schmitz <schmitzmic@gmail.com>
+L: linux-scsi@vger.kernel.org
+S: Maintained
+F: Documentation/scsi/g_NCR5380.txt
+F: drivers/scsi/NCR5380.*
+F: drivers/scsi/arm/cumana_1.c
+F: drivers/scsi/arm/oak.c
+F: drivers/scsi/atari_NCR5380.c
+F: drivers/scsi/atari_scsi.*
+F: drivers/scsi/dmx3191d.c
+F: drivers/scsi/dtc.*
+F: drivers/scsi/g_NCR5380.*
+F: drivers/scsi/g_NCR5380_mmio.c
+F: drivers/scsi/mac_scsi.*
+F: drivers/scsi/pas16.*
+F: drivers/scsi/sun3_NCR5380.c
+F: drivers/scsi/sun3_scsi.*
+F: drivers/scsi/sun3_scsi_vme.c
+F: drivers/scsi/t128.*
+
NCR DUAL 700 SCSI DRIVER (MICROCHANNEL)
M: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
L: linux-scsi@vger.kernel.org
@@ -9057,7 +9104,7 @@
F: include/uapi/linux/toshiba.h
TMIO MMC DRIVER
-M: Ian Molton <ian@mnementh.co.uk>
+M: Ian Molton <ian.molton@codethink.co.uk>
L: linux-mmc@vger.kernel.org
S: Maintained
F: drivers/mmc/host/tmio_mmc*
@@ -9098,7 +9145,6 @@
TRACING
M: Steven Rostedt <rostedt@goodmis.org>
-M: Frederic Weisbecker <fweisbec@gmail.com>
M: Ingo Molnar <mingo@redhat.com>
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git perf/core
S: Maintained
diff --git a/Makefile b/Makefile
index cdaa5b6..c761fb1 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
VERSION = 3
PATCHLEVEL = 15
SUBLEVEL = 0
-EXTRAVERSION = -rc8
+EXTRAVERSION =
NAME = Shuffling Zombie Juror
# *DOCUMENTATION*
@@ -671,7 +671,7 @@
ifdef CONFIG_DEBUG_INFO
KBUILD_CFLAGS += -g
-KBUILD_AFLAGS += -Wa,--gdwarf-2
+KBUILD_AFLAGS += -Wa,-gdwarf-2
endif
ifdef CONFIG_DEBUG_INFO_REDUCED
diff --git a/arch/arc/boot/dts/angel4.dts b/arch/arc/boot/dts/angel4.dts
index bcf662d..5bb2fda 100644
--- a/arch/arc/boot/dts/angel4.dts
+++ b/arch/arc/boot/dts/angel4.dts
@@ -17,7 +17,7 @@
interrupt-parent = <&intc>;
chosen {
- bootargs = "console=ttyARC0,115200n8";
+ bootargs = "console=ttyARC0,115200n8 earlyprintk=ttyARC0";
};
aliases {
diff --git a/arch/arc/include/asm/cache.h b/arch/arc/include/asm/cache.h
index 2fd3162..c1d3d2d 100644
--- a/arch/arc/include/asm/cache.h
+++ b/arch/arc/include/asm/cache.h
@@ -55,4 +55,31 @@
#endif /* !__ASSEMBLY__ */
+/* Instruction cache related Auxiliary registers */
+#define ARC_REG_IC_BCR 0x77 /* Build Config reg */
+#define ARC_REG_IC_IVIC 0x10
+#define ARC_REG_IC_CTRL 0x11
+#define ARC_REG_IC_IVIL 0x19
+#if defined(CONFIG_ARC_MMU_V3) || defined (CONFIG_ARC_MMU_V4)
+#define ARC_REG_IC_PTAG 0x1E
+#endif
+
+/* Bit val in IC_CTRL */
+#define IC_CTRL_CACHE_DISABLE 0x1
+
+/* Data cache related Auxiliary registers */
+#define ARC_REG_DC_BCR 0x72 /* Build Config reg */
+#define ARC_REG_DC_IVDC 0x47
+#define ARC_REG_DC_CTRL 0x48
+#define ARC_REG_DC_IVDL 0x4A
+#define ARC_REG_DC_FLSH 0x4B
+#define ARC_REG_DC_FLDL 0x4C
+#if defined(CONFIG_ARC_MMU_V3) || defined (CONFIG_ARC_MMU_V4)
+#define ARC_REG_DC_PTAG 0x5C
+#endif
+
+/* Bit val in DC_CTRL */
+#define DC_CTRL_INV_MODE_FLUSH 0x40
+#define DC_CTRL_FLUSH_STATUS 0x100
+
#endif /* _ASM_CACHE_H */
diff --git a/arch/arc/include/asm/irq.h b/arch/arc/include/asm/irq.h
index 291a70d..fb4efb6 100644
--- a/arch/arc/include/asm/irq.h
+++ b/arch/arc/include/asm/irq.h
@@ -19,8 +19,6 @@
#include <asm-generic/irq.h>
extern void arc_init_IRQ(void);
-extern int get_hw_config_num_irq(void);
-
-void arc_local_timer_setup(unsigned int cpu);
+void arc_local_timer_setup(void);
#endif
diff --git a/arch/arc/include/asm/processor.h b/arch/arc/include/asm/processor.h
index 15334ab..d99f9b37 100644
--- a/arch/arc/include/asm/processor.h
+++ b/arch/arc/include/asm/processor.h
@@ -18,7 +18,6 @@
#ifndef __ASSEMBLY__
-#include <asm/arcregs.h> /* for STATUS_E1_MASK et all */
#include <asm/ptrace.h>
/* Arch specific stuff which needs to be saved per task.
@@ -41,15 +40,13 @@
/* Forward declaration, a strange C thing */
struct task_struct;
-/*
- * Return saved PC of a blocked thread.
- */
+/* Return saved PC of a blocked thread */
unsigned long thread_saved_pc(struct task_struct *t);
#define task_pt_regs(p) \
((struct pt_regs *)(THREAD_SIZE + (void *)task_stack_page(p)) - 1)
-/* Free all resources held by a thread. */
+/* Free all resources held by a thread */
#define release_thread(thread) do { } while (0)
/* Prepare to copy thread state - unlazy all lazy status */
@@ -82,26 +79,8 @@
#define KSTK_BLINK(tsk) KSTK_REG(tsk, 4)
#define KSTK_FP(tsk) KSTK_REG(tsk, 0)
-/*
- * Do necessary setup to start up a newly executed thread.
- *
- * E1,E2 so that Interrupts are enabled in user mode
- * L set, so Loop inhibited to begin with
- * lp_start and lp_end seeded with bogus non-zero values so to easily catch
- * the ARC700 sr to lp_start hardware bug
- */
-#define start_thread(_regs, _pc, _usp) \
-do { \
- set_fs(USER_DS); /* reads from user space */ \
- (_regs)->ret = (_pc); \
- /* Interrupts enabled in User Mode */ \
- (_regs)->status32 = STATUS_U_MASK | STATUS_L_MASK \
- | STATUS_E1_MASK | STATUS_E2_MASK; \
- (_regs)->sp = (_usp); \
- /* bogus seed values for debugging */ \
- (_regs)->lp_start = 0x10; \
- (_regs)->lp_end = 0x80; \
-} while (0)
+extern void start_thread(struct pt_regs * regs, unsigned long pc,
+ unsigned long usp);
extern unsigned int get_wchan(struct task_struct *p);
diff --git a/arch/arc/include/uapi/asm/Kbuild b/arch/arc/include/uapi/asm/Kbuild
index 18fefae..f50d02d 100644
--- a/arch/arc/include/uapi/asm/Kbuild
+++ b/arch/arc/include/uapi/asm/Kbuild
@@ -2,11 +2,4 @@
include include/uapi/asm-generic/Kbuild.asm
header-y += elf.h
header-y += page.h
-header-y += setup.h
-header-y += byteorder.h
header-y += cachectl.h
-header-y += ptrace.h
-header-y += sigcontext.h
-header-y += signal.h
-header-y += swab.h
-header-y += unistd.h
diff --git a/arch/arc/kernel/entry.S b/arch/arc/kernel/entry.S
index 29b82ad..83a046a 100644
--- a/arch/arc/kernel/entry.S
+++ b/arch/arc/kernel/entry.S
@@ -156,7 +156,7 @@
int1_saved_reg:
.zero 4
-/* Each Interrupt level needs it's own scratch */
+/* Each Interrupt level needs its own scratch */
#ifdef CONFIG_ARC_COMPACT_IRQ_LEVELS
ARCFP_DATA int2_saved_reg
@@ -473,7 +473,7 @@
lr r0, [efa]
mov r1, sp
- ; Now that we have read EFA, its safe to do "fake" rtie
+ ; Now that we have read EFA, it is safe to do "fake" rtie
; and get out of CPU exception mode
FAKE_RET_FROM_EXCPN r11
@@ -678,9 +678,9 @@
brne r9, event_IRQ2, 149f
;------------------------------------------------------------------
- ; if L2 IRQ interrupted a L1 ISR, we'd disbaled preemption earlier
- ; so that sched doesnt move to new task, causing L1 to be delayed
- ; undeterministically. Now that we've achieved that, lets reset
+ ; if L2 IRQ interrupted an L1 ISR, we'd disabled preemption earlier
+ ; so that sched doesn't move to new task, causing L1 to be delayed
+ ; undeterministically. Now that we've achieved that, let's reset
; things to what they were, before returning from L2 context
;----------------------------------------------------------------
@@ -736,7 +736,7 @@
; put last task in scheduler queue
bl @schedule_tail
- ; If kernel thread, jump to it's entry-point
+ ; If kernel thread, jump to its entry-point
ld r9, [sp, PT_status32]
brne r9, 0, 1f
diff --git a/arch/arc/kernel/head.S b/arch/arc/kernel/head.S
index 4ad0491..07a58f2 100644
--- a/arch/arc/kernel/head.S
+++ b/arch/arc/kernel/head.S
@@ -12,10 +12,42 @@
* to skip certain things during boot on simulator
*/
+#include <linux/linkage.h>
#include <asm/asm-offsets.h>
#include <asm/entry.h>
-#include <linux/linkage.h>
#include <asm/arcregs.h>
+#include <asm/cache.h>
+
+.macro CPU_EARLY_SETUP
+
+ ; Setting up Vectror Table (in case exception happens in early boot
+ sr @_int_vec_base_lds, [AUX_INTR_VEC_BASE]
+
+ ; Disable I-cache/D-cache if kernel so configured
+ lr r5, [ARC_REG_IC_BCR]
+ breq r5, 0, 1f ; I$ doesn't exist
+ lr r5, [ARC_REG_IC_CTRL]
+#ifdef CONFIG_ARC_HAS_ICACHE
+ bclr r5, r5, 0 ; 0 - Enable, 1 is Disable
+#else
+ bset r5, r5, 0 ; I$ exists, but is not used
+#endif
+ sr r5, [ARC_REG_IC_CTRL]
+
+1:
+ lr r5, [ARC_REG_DC_BCR]
+ breq r5, 0, 1f ; D$ doesn't exist
+ lr r5, [ARC_REG_DC_CTRL]
+ bclr r5, r5, 6 ; Invalidate (discard w/o wback)
+#ifdef CONFIG_ARC_HAS_DCACHE
+ bclr r5, r5, 0 ; Enable (+Inv)
+#else
+ bset r5, r5, 0 ; Disable (+Inv)
+#endif
+ sr r5, [ARC_REG_DC_CTRL]
+
+1:
+.endm
.cpu A7
@@ -27,7 +59,7 @@
; Don't clobber r0-r2 yet. It might have bootloader provided info
;-------------------------------------------------------------------
- sr @_int_vec_base_lds, [AUX_INTR_VEC_BASE]
+ CPU_EARLY_SETUP
#ifdef CONFIG_SMP
; Ensure Boot (Master) proceeds. Others wait in platform dependent way
@@ -90,7 +122,7 @@
first_lines_of_secondary:
- sr @_int_vec_base_lds, [AUX_INTR_VEC_BASE]
+ CPU_EARLY_SETUP
; setup per-cpu idle task as "current" on this CPU
ld r0, [@secondary_idle_tsk]
diff --git a/arch/arc/kernel/irq.c b/arch/arc/kernel/irq.c
index a4b141e..7d653c0 100644
--- a/arch/arc/kernel/irq.c
+++ b/arch/arc/kernel/irq.c
@@ -150,24 +150,6 @@
set_irq_regs(old_regs);
}
-int get_hw_config_num_irq(void)
-{
- uint32_t val = read_aux_reg(ARC_REG_VECBASE_BCR);
-
- switch (val & 0x03) {
- case 0:
- return 16;
- case 1:
- return 32;
- case 2:
- return 8;
- default:
- return 0;
- }
-
- return 0;
-}
-
/*
* arch_local_irq_enable - Enable interrupts.
*
diff --git a/arch/arc/kernel/process.c b/arch/arc/kernel/process.c
index 07a3a96..fdd89715 100644
--- a/arch/arc/kernel/process.c
+++ b/arch/arc/kernel/process.c
@@ -151,6 +151,29 @@
}
/*
+ * Do necessary setup to start up a new user task
+ */
+void start_thread(struct pt_regs * regs, unsigned long pc, unsigned long usp)
+{
+ set_fs(USER_DS); /* user space */
+
+ regs->sp = usp;
+ regs->ret = pc;
+
+ /*
+ * [U]ser Mode bit set
+ * [L] ZOL loop inhibited to begin with - cleared by a LP insn
+ * Interrupts enabled
+ */
+ regs->status32 = STATUS_U_MASK | STATUS_L_MASK |
+ STATUS_E1_MASK | STATUS_E2_MASK;
+
+ /* bogus seed values for debugging */
+ regs->lp_start = 0x10;
+ regs->lp_end = 0x80;
+}
+
+/*
* Some archs flush debug and FPU info here
*/
void flush_thread(void)
diff --git a/arch/arc/kernel/smp.c b/arch/arc/kernel/smp.c
index 40859e5..cf90b6f 100644
--- a/arch/arc/kernel/smp.c
+++ b/arch/arc/kernel/smp.c
@@ -138,7 +138,7 @@
if (machine_desc->init_smp)
machine_desc->init_smp(smp_processor_id());
- arc_local_timer_setup(cpu);
+ arc_local_timer_setup();
local_irq_enable();
preempt_disable();
diff --git a/arch/arc/kernel/time.c b/arch/arc/kernel/time.c
index 71c4252..36c2aa9 100644
--- a/arch/arc/kernel/time.c
+++ b/arch/arc/kernel/time.c
@@ -219,12 +219,13 @@
/*
* Setup the local event timer for @cpu
*/
-void arc_local_timer_setup(unsigned int cpu)
+void arc_local_timer_setup()
{
- struct clock_event_device *clk = &per_cpu(arc_clockevent_device, cpu);
+ struct clock_event_device *evt = this_cpu_ptr(&arc_clockevent_device);
+ int cpu = smp_processor_id();
- clk->cpumask = cpumask_of(cpu);
- clockevents_config_and_register(clk, arc_get_core_freq(),
+ evt->cpumask = cpumask_of(cpu);
+ clockevents_config_and_register(evt, arc_get_core_freq(),
0, ARC_TIMER_MAX);
/*
@@ -261,7 +262,7 @@
clocksource_register_hz(&arc_counter, arc_get_core_freq());
/* sets up the periodic event timer */
- arc_local_timer_setup(smp_processor_id());
+ arc_local_timer_setup();
if (machine_desc->init_time)
machine_desc->init_time();
diff --git a/arch/arc/mm/cache_arc700.c b/arch/arc/mm/cache_arc700.c
index 89edf79..1f676c4 100644
--- a/arch/arc/mm/cache_arc700.c
+++ b/arch/arc/mm/cache_arc700.c
@@ -73,33 +73,6 @@
#include <asm/cachectl.h>
#include <asm/setup.h>
-/* Instruction cache related Auxiliary registers */
-#define ARC_REG_IC_BCR 0x77 /* Build Config reg */
-#define ARC_REG_IC_IVIC 0x10
-#define ARC_REG_IC_CTRL 0x11
-#define ARC_REG_IC_IVIL 0x19
-#if (CONFIG_ARC_MMU_VER > 2)
-#define ARC_REG_IC_PTAG 0x1E
-#endif
-
-/* Bit val in IC_CTRL */
-#define IC_CTRL_CACHE_DISABLE 0x1
-
-/* Data cache related Auxiliary registers */
-#define ARC_REG_DC_BCR 0x72 /* Build Config reg */
-#define ARC_REG_DC_IVDC 0x47
-#define ARC_REG_DC_CTRL 0x48
-#define ARC_REG_DC_IVDL 0x4A
-#define ARC_REG_DC_FLSH 0x4B
-#define ARC_REG_DC_FLDL 0x4C
-#if (CONFIG_ARC_MMU_VER > 2)
-#define ARC_REG_DC_PTAG 0x5C
-#endif
-
-/* Bit val in DC_CTRL */
-#define DC_CTRL_INV_MODE_FLUSH 0x40
-#define DC_CTRL_FLUSH_STATUS 0x100
-
char *arc_cache_mumbojumbo(int c, char *buf, int len)
{
int n = 0;
@@ -168,72 +141,43 @@
*/
void arc_cache_init(void)
{
- unsigned int cpu = smp_processor_id();
- struct cpuinfo_arc_cache *ic = &cpuinfo_arc700[cpu].icache;
- struct cpuinfo_arc_cache *dc = &cpuinfo_arc700[cpu].dcache;
- unsigned int dcache_does_alias, temp;
+ unsigned int __maybe_unused cpu = smp_processor_id();
+ struct cpuinfo_arc_cache __maybe_unused *ic, __maybe_unused *dc;
char str[256];
printk(arc_cache_mumbojumbo(0, str, sizeof(str)));
- if (!ic->ver)
- goto chk_dc;
-
#ifdef CONFIG_ARC_HAS_ICACHE
- /* 1. Confirm some of I-cache params which Linux assumes */
- if (ic->line_len != L1_CACHE_BYTES)
- panic("Cache H/W doesn't match kernel Config");
+ ic = &cpuinfo_arc700[cpu].icache;
+ if (ic->ver) {
+ if (ic->line_len != L1_CACHE_BYTES)
+ panic("ICache line [%d] != kernel Config [%d]",
+ ic->line_len, L1_CACHE_BYTES);
- if (ic->ver != CONFIG_ARC_MMU_VER)
- panic("Cache ver doesn't match MMU ver\n");
+ if (ic->ver != CONFIG_ARC_MMU_VER)
+ panic("Cache ver [%d] doesn't match MMU ver [%d]\n",
+ ic->ver, CONFIG_ARC_MMU_VER);
+ }
#endif
- /* Enable/disable I-Cache */
- temp = read_aux_reg(ARC_REG_IC_CTRL);
-
-#ifdef CONFIG_ARC_HAS_ICACHE
- temp &= ~IC_CTRL_CACHE_DISABLE;
-#else
- temp |= IC_CTRL_CACHE_DISABLE;
-#endif
-
- write_aux_reg(ARC_REG_IC_CTRL, temp);
-
-chk_dc:
- if (!dc->ver)
- return;
-
#ifdef CONFIG_ARC_HAS_DCACHE
- if (dc->line_len != L1_CACHE_BYTES)
- panic("Cache H/W doesn't match kernel Config");
+ dc = &cpuinfo_arc700[cpu].dcache;
+ if (dc->ver) {
+ unsigned int dcache_does_alias;
- /* check for D-Cache aliasing */
- dcache_does_alias = (dc->sz / dc->assoc) > PAGE_SIZE;
+ if (dc->line_len != L1_CACHE_BYTES)
+ panic("DCache line [%d] != kernel Config [%d]",
+ dc->line_len, L1_CACHE_BYTES);
- if (dcache_does_alias && !cache_is_vipt_aliasing())
- panic("Enable CONFIG_ARC_CACHE_VIPT_ALIASING\n");
- else if (!dcache_does_alias && cache_is_vipt_aliasing())
- panic("Don't need CONFIG_ARC_CACHE_VIPT_ALIASING\n");
+ /* check for D-Cache aliasing */
+ dcache_does_alias = (dc->sz / dc->assoc) > PAGE_SIZE;
+
+ if (dcache_does_alias && !cache_is_vipt_aliasing())
+ panic("Enable CONFIG_ARC_CACHE_VIPT_ALIASING\n");
+ else if (!dcache_does_alias && cache_is_vipt_aliasing())
+ panic("Don't need CONFIG_ARC_CACHE_VIPT_ALIASING\n");
+ }
#endif
-
- /* Set the default Invalidate Mode to "simpy discard dirty lines"
- * as this is more frequent then flush before invalidate
- * Ofcourse we toggle this default behviour when desired
- */
- temp = read_aux_reg(ARC_REG_DC_CTRL);
- temp &= ~DC_CTRL_INV_MODE_FLUSH;
-
-#ifdef CONFIG_ARC_HAS_DCACHE
- /* Enable D-Cache: Clear Bit 0 */
- write_aux_reg(ARC_REG_DC_CTRL, temp & ~IC_CTRL_CACHE_DISABLE);
-#else
- /* Flush D cache */
- write_aux_reg(ARC_REG_DC_FLSH, 0x1);
- /* Disable D cache */
- write_aux_reg(ARC_REG_DC_CTRL, temp | IC_CTRL_CACHE_DISABLE);
-#endif
-
- return;
}
#define OP_INV 0x1
@@ -253,12 +197,16 @@
if (cacheop == OP_INV_IC) {
aux_cmd = ARC_REG_IC_IVIL;
+#if (CONFIG_ARC_MMU_VER > 2)
aux_tag = ARC_REG_IC_PTAG;
+#endif
}
else {
/* d$ cmd: INV (discard or wback-n-discard) OR FLUSH (wback) */
aux_cmd = cacheop & OP_INV ? ARC_REG_DC_IVDL : ARC_REG_DC_FLDL;
+#if (CONFIG_ARC_MMU_VER > 2)
aux_tag = ARC_REG_DC_PTAG;
+#endif
}
/* Ensure we properly floor/ceil the non-line aligned/sized requests
diff --git a/arch/arc/plat-arcfpga/Kconfig b/arch/arc/plat-arcfpga/Kconfig
index 33058aa..e27bb5c 100644
--- a/arch/arc/plat-arcfpga/Kconfig
+++ b/arch/arc/plat-arcfpga/Kconfig
@@ -48,36 +48,4 @@
help
Baud rate for the ARC UART
-menuconfig ARC_HAS_BVCI_LAT_UNIT
- bool "BVCI Bus Latency Unit"
- depends on ARC_BOARD_ML509 || ARC_BOARD_ANGEL4
- help
- IP to add artificial latency to BVCI Bus Based FPGA builds.
- The default latency (even worst case) for FPGA is non-realistic
- (~10 SDRAM, ~5 SSRAM).
-
-config BVCI_LAT_UNITS
- hex "Latency Unit(s) Bitmap"
- default "0x0"
- depends on ARC_HAS_BVCI_LAT_UNIT
- help
- There are multiple Latency Units corresponding to the many
- interfaces of the system bus arbiter (both CPU side as well as
- the peripheral side).
- To add latency to ALL memory transaction, choose Unit 0, otherwise
- for finer grainer - interface wise latency, specify a bitmap (1 bit
- per unit) of all units. e.g. 1,2,12 will be 0x1003
-
- Unit 0 - System Arb and Mem Controller
- Unit 1 - I$ and System Bus
- Unit 2 - D$ and System Bus
- ..
- Unit 12 - IDE Disk controller and System Bus
-
-config BVCI_LAT_CYCLES
- int "Latency Value in cycles"
- range 0 63
- default "30"
- depends on ARC_HAS_BVCI_LAT_UNIT
-
endif
diff --git a/arch/arc/plat-arcfpga/Makefile b/arch/arc/plat-arcfpga/Makefile
index a44e22e..4d1bddc 100644
--- a/arch/arc/plat-arcfpga/Makefile
+++ b/arch/arc/plat-arcfpga/Makefile
@@ -9,4 +9,4 @@
KBUILD_CFLAGS += -Iarch/arc/plat-arcfpga/include
obj-y := platform.o irq.o
-obj-$(CONFIG_SMP) += smp.o
+obj-$(CONFIG_ISS_SMP_EXTN) += smp.o
diff --git a/arch/arc/plat-arcfpga/platform.c b/arch/arc/plat-arcfpga/platform.c
index 19b76b6..61c7e59 100644
--- a/arch/arc/plat-arcfpga/platform.c
+++ b/arch/arc/plat-arcfpga/platform.c
@@ -22,59 +22,6 @@
#include <plat/smp.h>
#include <plat/irq.h>
-/*-----------------------BVCI Latency Unit -----------------------------*/
-
-#ifdef CONFIG_ARC_HAS_BVCI_LAT_UNIT
-
-int lat_cycles = CONFIG_BVCI_LAT_CYCLES;
-
-/* BVCI Bus Profiler: Latency Unit */
-static void __init setup_bvci_lat_unit(void)
-{
-#define MAX_BVCI_UNITS 12
-
- unsigned int i;
- unsigned int *base = (unsigned int *)BVCI_LAT_UNIT_BASE;
- const unsigned long units_req = CONFIG_BVCI_LAT_UNITS;
- const unsigned int REG_UNIT = 21;
- const unsigned int REG_VAL = 22;
-
- /*
- * There are multiple Latency Units corresponding to the many
- * interfaces of the system bus arbiter (both CPU side as well as
- * the peripheral side).
- *
- * Unit 0 - System Arb and Mem Controller - adds latency to all
- * memory trasactions
- * Unit 1 - I$ and System Bus
- * Unit 2 - D$ and System Bus
- * ..
- * Unit 12 - IDE Disk controller and System Bus
- *
- * The programmers model requires writing to lat_unit reg first
- * and then the latency value (cycles) to lat_value reg
- */
-
- if (CONFIG_BVCI_LAT_UNITS == 0) {
- writel(0, base + REG_UNIT);
- writel(lat_cycles, base + REG_VAL);
- pr_info("BVCI Latency for all Memory Transactions %d cycles\n",
- lat_cycles);
- } else {
- for_each_set_bit(i, &units_req, MAX_BVCI_UNITS) {
- writel(i + 1, base + REG_UNIT); /* loop is 0 based */
- writel(lat_cycles, base + REG_VAL);
- pr_info("BVCI Latency for Unit[%d] = %d cycles\n",
- (i + 1), lat_cycles);
- }
- }
-}
-#else
-static void __init setup_bvci_lat_unit(void)
-{
-}
-#endif
-
/*----------------------- Platform Devices -----------------------------*/
#if IS_ENABLED(CONFIG_SERIAL_ARC)
@@ -132,16 +79,11 @@
ARRAY_SIZE(fpga_early_devs));
/*
- * ARC console driver registers itself as an early platform driver
- * of class "earlyprintk".
- * Install it here, followed by probe of devices.
- * The installation here doesn't require earlyprintk in command line
- * To do so however, replace the lines below with
- * parse_early_param();
- * early_platform_driver_probe("earlyprintk", 1, 1);
- * ^^
+ * ARC console driver registers (build time) as an early platform driver
+ * of class "earlyprintk". However it needs explicit cmdline toggle
+ * "earlyprintk=ttyARC0" to be successfuly runtime registered.
+ * Otherwise the early probe below fails to find the driver
*/
- early_platform_driver_register_all("earlyprintk");
early_platform_driver_probe("earlyprintk", 1, 0);
/*
@@ -165,11 +107,9 @@
{
pr_info("[plat-arcfpga]: registering early dev resources\n");
- setup_bvci_lat_unit();
-
arc_fpga_serial_init();
-#ifdef CONFIG_SMP
+#ifdef CONFIG_ISS_SMP_EXTN
iss_model_init_early_smp();
#endif
}
@@ -211,7 +151,7 @@
.init_early = plat_fpga_early_init,
.init_machine = plat_fpga_populate_dev,
.init_irq = plat_fpga_init_IRQ,
-#ifdef CONFIG_SMP
+#ifdef CONFIG_ISS_SMP_EXTN
.init_smp = iss_model_init_smp,
#endif
MACHINE_END
diff --git a/arch/arc/plat-arcfpga/smp.c b/arch/arc/plat-arcfpga/smp.c
index 8a12741..92bad91 100644
--- a/arch/arc/plat-arcfpga/smp.c
+++ b/arch/arc/plat-arcfpga/smp.c
@@ -42,6 +42,24 @@
}
+static inline int get_hw_config_num_irq(void)
+{
+ uint32_t val = read_aux_reg(ARC_REG_VECBASE_BCR);
+
+ switch (val & 0x03) {
+ case 0:
+ return 16;
+ case 1:
+ return 32;
+ case 2:
+ return 8;
+ default:
+ return 0;
+ }
+
+ return 0;
+}
+
/*
* Any SMP specific init any CPU does when it comes up.
* Here we setup the CPU to enable Inter-Processor-Interrupts
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index ad89a03..87b63fd 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -165,12 +165,9 @@
bool
default y
-config RWSEM_GENERIC_SPINLOCK
- bool
- default y
-
config RWSEM_XCHGADD_ALGORITHM
bool
+ default y
config ARCH_HAS_ILOG2_U32
bool
@@ -1089,11 +1086,6 @@
source arch/arm/mm/Kconfig
-config ARM_NR_BANKS
- int
- default 16 if ARCH_EP93XX
- default 8
-
config IWMMXT
bool "Enable iWMMXt support"
depends on CPU_XSCALE || CPU_XSC3 || CPU_MOHAWK || CPU_PJ4 || CPU_PJ4B
@@ -1214,19 +1206,6 @@
register of the Cortex-A9 which reduces the linefill issuing
capabilities of the processor.
-config PL310_ERRATA_588369
- bool "PL310 errata: Clean & Invalidate maintenance operations do not invalidate clean lines"
- depends on CACHE_L2X0
- help
- The PL310 L2 cache controller implements three types of Clean &
- Invalidate maintenance operations: by Physical Address
- (offset 0x7F0), by Index/Way (0x7F8) and by Way (0x7FC).
- They are architecturally defined to behave as the execution of a
- clean operation followed immediately by an invalidate operation,
- both performing to the same memory location. This functionality
- is not correctly implemented in PL310 as clean lines are not
- invalidated as a result of these operations.
-
config ARM_ERRATA_643719
bool "ARM errata: LoUIS bit field in CLIDR register is incorrect"
depends on CPU_V7 && SMP
@@ -1249,17 +1228,6 @@
tables. The workaround changes the TLB flushing routines to invalidate
entries regardless of the ASID.
-config PL310_ERRATA_727915
- bool "PL310 errata: Background Clean & Invalidate by Way operation can cause data corruption"
- depends on CACHE_L2X0
- help
- PL310 implements the Clean & Invalidate by Way L2 cache maintenance
- operation (offset 0x7FC). This operation runs in background so that
- PL310 can handle normal accesses while it is in progress. Under very
- rare circumstances, due to this erratum, write data can be lost when
- PL310 treats a cacheable write transaction during a Clean &
- Invalidate by Way operation.
-
config ARM_ERRATA_743622
bool "ARM errata: Faulty hazard checking in the Store Buffer may lead to data corruption"
depends on CPU_V7
@@ -1285,21 +1253,6 @@
operation is received by a CPU before the ICIALLUIS has completed,
potentially leading to corrupted entries in the cache or TLB.
-config PL310_ERRATA_753970
- bool "PL310 errata: cache sync operation may be faulty"
- depends on CACHE_PL310
- help
- This option enables the workaround for the 753970 PL310 (r3p0) erratum.
-
- Under some condition the effect of cache sync operation on
- the store buffer still remains when the operation completes.
- This means that the store buffer is always asked to drain and
- this prevents it from merging any further writes. The workaround
- is to replace the normal offset of cache sync operation (0x730)
- by another offset targeting an unmapped PL310 register 0x740.
- This has the same effect as the cache sync operation: store buffer
- drain and waiting for all buffers empty.
-
config ARM_ERRATA_754322
bool "ARM errata: possible faulty MMU translations following an ASID switch"
depends on CPU_V7
@@ -1348,18 +1301,6 @@
relevant cache maintenance functions and sets a specific bit
in the diagnostic control register of the SCU.
-config PL310_ERRATA_769419
- bool "PL310 errata: no automatic Store Buffer drain"
- depends on CACHE_L2X0
- help
- On revisions of the PL310 prior to r3p2, the Store Buffer does
- not automatically drain. This can cause normal, non-cacheable
- writes to be retained when the memory system is idle, leading
- to suboptimal I/O performance for drivers using coherent DMA.
- This option adds a write barrier to the cpu_idle loop so that,
- on systems with an outer cache, the store buffer is drained
- explicitly.
-
config ARM_ERRATA_775420
bool "ARM errata: A data cache maintenance operation which aborts, might lead to deadlock"
depends on CPU_V7
@@ -2279,6 +2220,11 @@
config ARM_CPU_SUSPEND
def_bool PM_SLEEP
+config ARCH_HIBERNATION_POSSIBLE
+ bool
+ depends on MMU
+ default y if ARCH_SUSPEND_POSSIBLE
+
endmenu
source "net/Kconfig"
diff --git a/arch/arm/boot/compressed/atags_to_fdt.c b/arch/arm/boot/compressed/atags_to_fdt.c
index d1153c8..9448aa0 100644
--- a/arch/arm/boot/compressed/atags_to_fdt.c
+++ b/arch/arm/boot/compressed/atags_to_fdt.c
@@ -7,6 +7,8 @@
#define do_extend_cmdline 0
#endif
+#define NR_BANKS 16
+
static int node_offset(void *fdt, const char *node_path)
{
int offset = fdt_path_offset(fdt, node_path);
diff --git a/arch/arm/boot/dts/am4372.dtsi b/arch/arm/boot/dts/am4372.dtsi
index 794c73e..db464d7 100644
--- a/arch/arm/boot/dts/am4372.dtsi
+++ b/arch/arm/boot/dts/am4372.dtsi
@@ -857,6 +857,35 @@
ti,hwmods = "hdq1w";
status = "disabled";
};
+
+ dss: dss@4832a000 {
+ compatible = "ti,omap3-dss";
+ reg = <0x4832a000 0x200>;
+ status = "disabled";
+ ti,hwmods = "dss_core";
+ clocks = <&disp_clk>;
+ clock-names = "fck";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ dispc@4832a400 {
+ compatible = "ti,omap3-dispc";
+ reg = <0x4832a400 0x400>;
+ interrupts = <GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>;
+ ti,hwmods = "dss_dispc";
+ clocks = <&disp_clk>;
+ clock-names = "fck";
+ };
+
+ rfbi: rfbi@4832a800 {
+ compatible = "ti,omap3-rfbi";
+ reg = <0x4832a800 0x100>;
+ ti,hwmods = "dss_rfbi";
+ clocks = <&disp_clk>;
+ clock-names = "fck";
+ };
+ };
};
};
diff --git a/arch/arm/boot/dts/am437x-gp-evm.dts b/arch/arm/boot/dts/am437x-gp-evm.dts
index c25d158..003766c 100644
--- a/arch/arm/boot/dts/am437x-gp-evm.dts
+++ b/arch/arm/boot/dts/am437x-gp-evm.dts
@@ -19,6 +19,10 @@
model = "TI AM437x GP EVM";
compatible = "ti,am437x-gp-evm","ti,am4372","ti,am43";
+ aliases {
+ display0 = &lcd0;
+ };
+
vmmcsd_fixed: fixedregulator-sd {
compatible = "regulator-fixed";
regulator-name = "vmmcsd_fixed";
@@ -64,6 +68,44 @@
0x02000069 /* LEFT */
0x0201006c>; /* DOWN */
};
+
+ lcd0: display {
+ compatible = "osddisplays,osd057T0559-34ts", "panel-dpi";
+ label = "lcd";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&lcd_pins>;
+
+ /*
+ * SelLCDorHDMI, LOW to select HDMI. This is not really the
+ * panel's enable GPIO, but we don't have HDMI driver support nor
+ * support to switch between two displays, so using this gpio as
+ * panel's enable should be safe.
+ */
+ enable-gpios = <&gpio5 8 GPIO_ACTIVE_HIGH>;
+
+ panel-timing {
+ clock-frequency = <33000000>;
+ hactive = <800>;
+ vactive = <480>;
+ hfront-porch = <210>;
+ hback-porch = <16>;
+ hsync-len = <30>;
+ vback-porch = <10>;
+ vfront-porch = <22>;
+ vsync-len = <13>;
+ hsync-active = <0>;
+ vsync-active = <0>;
+ de-active = <1>;
+ pixelclk-active = <1>;
+ };
+
+ port {
+ lcd_in: endpoint {
+ remote-endpoint = <&dpi_out>;
+ };
+ };
+ };
};
&am43xx_pinmux {
@@ -171,6 +213,47 @@
0x9c (PIN_OUTPUT | MUX_MODE0) /* gpmc_be0n_cle.gpmc_be0n_cle */
>;
};
+
+ dss_pins: dss_pins {
+ pinctrl-single,pins = <
+ 0x020 (PIN_OUTPUT_PULLUP | MUX_MODE1) /*gpmc ad 8 -> DSS DATA 23 */
+ 0x024 (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x028 (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x02c (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x030 (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x034 (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x038 (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x03c (PIN_OUTPUT_PULLUP | MUX_MODE1) /*gpmc ad 15 -> DSS DATA 16 */
+ 0x0a0 (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS DATA 0 */
+ 0x0a4 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0a8 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0ac (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0b0 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0b4 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0b8 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0bc (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0c0 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0c4 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0c8 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0cc (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0d0 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0d4 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0d8 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0dc (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS DATA 15 */
+ 0x0e0 (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS VSYNC */
+ 0x0e4 (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS HSYNC */
+ 0x0e8 (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS PCLK */
+ 0x0ec (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS AC BIAS EN */
+
+ >;
+ };
+
+ lcd_pins: lcd_pins {
+ pinctrl-single,pins = <
+ /* GPIO 5_8 to select LCD / HDMI */
+ 0x238 (PIN_OUTPUT_PULLUP | MUX_MODE7)
+ >;
+ };
};
&i2c0 {
@@ -359,3 +442,17 @@
};
};
};
+
+&dss {
+ status = "ok";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&dss_pins>;
+
+ port {
+ dpi_out: endpoint@0 {
+ remote-endpoint = <&lcd_in>;
+ data-lines = <24>;
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/am43x-epos-evm.dts b/arch/arm/boot/dts/am43x-epos-evm.dts
index ad362c5..19f1f7e 100644
--- a/arch/arm/boot/dts/am43x-epos-evm.dts
+++ b/arch/arm/boot/dts/am43x-epos-evm.dts
@@ -19,6 +19,10 @@
model = "TI AM43x EPOS EVM";
compatible = "ti,am43x-epos-evm","ti,am4372","ti,am43";
+ aliases {
+ display0 = &lcd0;
+ };
+
vmmcsd_fixed: fixedregulator-sd {
compatible = "regulator-fixed";
regulator-name = "vmmcsd_fixed";
@@ -27,6 +31,44 @@
enable-active-high;
};
+ lcd0: display {
+ compatible = "osddisplays,osd057T0559-34ts", "panel-dpi";
+ label = "lcd";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&lcd_pins>;
+
+ /*
+ * SelLCDorHDMI, LOW to select HDMI. This is not really the
+ * panel's enable GPIO, but we don't have HDMI driver support nor
+ * support to switch between two displays, so using this gpio as
+ * panel's enable should be safe.
+ */
+ enable-gpios = <&gpio2 1 GPIO_ACTIVE_HIGH>;
+
+ panel-timing {
+ clock-frequency = <33000000>;
+ hactive = <800>;
+ vactive = <480>;
+ hfront-porch = <210>;
+ hback-porch = <16>;
+ hsync-len = <30>;
+ vback-porch = <10>;
+ vfront-porch = <22>;
+ vsync-len = <13>;
+ hsync-active = <0>;
+ vsync-active = <0>;
+ de-active = <1>;
+ pixelclk-active = <1>;
+ };
+
+ port {
+ lcd_in: endpoint {
+ remote-endpoint = <&dpi_out>;
+ };
+ };
+ };
+
am43xx_pinmux: pinmux@44e10800 {
cpsw_default: cpsw_default {
pinctrl-single,pins = <
@@ -161,6 +203,46 @@
0x234 (PIN_INPUT_PULLUP | MUX_MODE1) /* cam1_wen.hdq_gpio */
>;
};
+
+ dss_pins: dss_pins {
+ pinctrl-single,pins = <
+ 0x020 (PIN_OUTPUT_PULLUP | MUX_MODE1) /*gpmc ad 8 -> DSS DATA 23 */
+ 0x024 (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x028 (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x02C (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x030 (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x034 (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x038 (PIN_OUTPUT_PULLUP | MUX_MODE1)
+ 0x03C (PIN_OUTPUT_PULLUP | MUX_MODE1) /*gpmc ad 15 -> DSS DATA 16 */
+ 0x0A0 (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS DATA 0 */
+ 0x0A4 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0A8 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0AC (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0B0 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0B4 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0B8 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0BC (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0C0 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0C4 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0C8 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0CC (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0D0 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0D4 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0D8 (PIN_OUTPUT_PULLUP | MUX_MODE0)
+ 0x0DC (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS DATA 15 */
+ 0x0E0 (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS VSYNC */
+ 0x0E4 (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS HSYNC */
+ 0x0E8 (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS PCLK */
+ 0x0EC (PIN_OUTPUT_PULLUP | MUX_MODE0) /* DSS AC BIAS EN */
+ >;
+ };
+
+ lcd_pins: lcd_pins {
+ pinctrl-single,pins = <
+ /* GPMC CLK -> GPIO 2_1 to select LCD / HDMI */
+ 0x08C (PIN_OUTPUT_PULLUP | MUX_MODE7)
+ >;
+ };
};
matrix_keypad: matrix_keypad@0 {
@@ -468,3 +550,17 @@
pinctrl-names = "default";
pinctrl-0 = <&hdq_pins>;
};
+
+&dss {
+ status = "ok";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&dss_pins>;
+
+ port {
+ dpi_out: endpoint@0 {
+ remote-endpoint = <&lcd_in>;
+ data-lines = <24>;
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/bcm21664.dtsi b/arch/arm/boot/dts/bcm21664.dtsi
index 08a44d4..8b36682 100644
--- a/arch/arm/boot/dts/bcm21664.dtsi
+++ b/arch/arm/boot/dts/bcm21664.dtsi
@@ -14,6 +14,8 @@
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/interrupt-controller/irq.h>
+#include "dt-bindings/clock/bcm21664.h"
+
#include "skeleton.dtsi"
/ {
@@ -43,7 +45,7 @@
compatible = "brcm,bcm21664-dw-apb-uart", "snps,dw-apb-uart";
status = "disabled";
reg = <0x3e000000 0x118>;
- clocks = <&uartb_clk>;
+ clocks = <&slave_ccu BCM21664_SLAVE_CCU_UARTB>;
interrupts = <GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>;
reg-shift = <2>;
reg-io-width = <4>;
@@ -53,7 +55,7 @@
compatible = "brcm,bcm21664-dw-apb-uart", "snps,dw-apb-uart";
status = "disabled";
reg = <0x3e001000 0x118>;
- clocks = <&uartb2_clk>;
+ clocks = <&slave_ccu BCM21664_SLAVE_CCU_UARTB2>;
interrupts = <GIC_SPI 66 IRQ_TYPE_LEVEL_HIGH>;
reg-shift = <2>;
reg-io-width = <4>;
@@ -63,7 +65,7 @@
compatible = "brcm,bcm21664-dw-apb-uart", "snps,dw-apb-uart";
status = "disabled";
reg = <0x3e002000 0x118>;
- clocks = <&uartb3_clk>;
+ clocks = <&slave_ccu BCM21664_SLAVE_CCU_UARTB3>;
interrupts = <GIC_SPI 65 IRQ_TYPE_LEVEL_HIGH>;
reg-shift = <2>;
reg-io-width = <4>;
@@ -85,7 +87,7 @@
compatible = "brcm,kona-timer";
reg = <0x35006000 0x1c>;
interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&hub_timer_clk>;
+ clocks = <&aon_ccu BCM21664_AON_CCU_HUB_TIMER>;
};
gpio: gpio@35003000 {
@@ -106,7 +108,7 @@
compatible = "brcm,kona-sdhci";
reg = <0x3f180000 0x801c>;
interrupts = <GIC_SPI 77 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&sdio1_clk>;
+ clocks = <&master_ccu BCM21664_MASTER_CCU_SDIO1>;
status = "disabled";
};
@@ -114,7 +116,7 @@
compatible = "brcm,kona-sdhci";
reg = <0x3f190000 0x801c>;
interrupts = <GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&sdio2_clk>;
+ clocks = <&master_ccu BCM21664_MASTER_CCU_SDIO2>;
status = "disabled";
};
@@ -122,7 +124,7 @@
compatible = "brcm,kona-sdhci";
reg = <0x3f1a0000 0x801c>;
interrupts = <GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&sdio3_clk>;
+ clocks = <&master_ccu BCM21664_MASTER_CCU_SDIO3>;
status = "disabled";
};
@@ -130,7 +132,7 @@
compatible = "brcm,kona-sdhci";
reg = <0x3f1b0000 0x801c>;
interrupts = <GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&sdio4_clk>;
+ clocks = <&master_ccu BCM21664_MASTER_CCU_SDIO4>;
status = "disabled";
};
@@ -140,7 +142,7 @@
interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
#size-cells = <0>;
- clocks = <&bsc1_clk>;
+ clocks = <&slave_ccu BCM21664_SLAVE_CCU_BSC1>;
status = "disabled";
};
@@ -150,7 +152,7 @@
interrupts = <GIC_SPI 102 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
#size-cells = <0>;
- clocks = <&bsc2_clk>;
+ clocks = <&slave_ccu BCM21664_SLAVE_CCU_BSC2>;
status = "disabled";
};
@@ -160,7 +162,7 @@
interrupts = <GIC_SPI 169 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
#size-cells = <0>;
- clocks = <&bsc3_clk>;
+ clocks = <&slave_ccu BCM21664_SLAVE_CCU_BSC3>;
status = "disabled";
};
@@ -170,105 +172,149 @@
interrupts = <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
#size-cells = <0>;
- clocks = <&bsc4_clk>;
+ clocks = <&slave_ccu BCM21664_SLAVE_CCU_BSC4>;
status = "disabled";
};
clocks {
- bsc1_clk: bsc1 {
- compatible = "fixed-clock";
- clock-frequency = <13000000>;
- #clock-cells = <0>;
- };
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
- bsc2_clk: bsc2 {
- compatible = "fixed-clock";
- clock-frequency = <13000000>;
- #clock-cells = <0>;
- };
+ /*
+ * Fixed clocks are defined before CCUs whose
+ * clocks may depend on them.
+ */
- bsc3_clk: bsc3 {
- compatible = "fixed-clock";
- clock-frequency = <13000000>;
+ ref_32k_clk: ref_32k {
#clock-cells = <0>;
- };
-
- bsc4_clk: bsc4 {
- compatible = "fixed-clock";
- clock-frequency = <13000000>;
- #clock-cells = <0>;
- };
-
- pmu_bsc_clk: pmu_bsc {
- compatible = "fixed-clock";
- clock-frequency = <13000000>;
- #clock-cells = <0>;
- };
-
- hub_timer_clk: hub_timer {
compatible = "fixed-clock";
clock-frequency = <32768>;
- #clock-cells = <0>;
};
- pwm_clk: pwm {
+ bbl_32k_clk: bbl_32k {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <32768>;
+ };
+
+ ref_13m_clk: ref_13m {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <13000000>;
+ };
+
+ var_13m_clk: var_13m {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <13000000>;
+ };
+
+ dft_19_5m_clk: dft_19_5m {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <19500000>;
+ };
+
+ ref_crystal_clk: ref_crystal {
+ #clock-cells = <0>;
compatible = "fixed-clock";
clock-frequency = <26000000>;
- #clock-cells = <0>;
};
- sdio1_clk: sdio1 {
- compatible = "fixed-clock";
- clock-frequency = <48000000>;
+ ref_52m_clk: ref_52m {
#clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <52000000>;
};
- sdio2_clk: sdio2 {
- compatible = "fixed-clock";
- clock-frequency = <48000000>;
+ var_52m_clk: var_52m {
#clock-cells = <0>;
- };
-
- sdio3_clk: sdio3 {
compatible = "fixed-clock";
- clock-frequency = <48000000>;
- #clock-cells = <0>;
- };
-
- sdio4_clk: sdio4 {
- compatible = "fixed-clock";
- clock-frequency = <48000000>;
- #clock-cells = <0>;
- };
-
- tmon_1m_clk: tmon_1m {
- compatible = "fixed-clock";
- clock-frequency = <1000000>;
- #clock-cells = <0>;
- };
-
- uartb_clk: uartb {
- compatible = "fixed-clock";
- clock-frequency = <13000000>;
- #clock-cells = <0>;
- };
-
- uartb2_clk: uartb2 {
- compatible = "fixed-clock";
- clock-frequency = <13000000>;
- #clock-cells = <0>;
- };
-
- uartb3_clk: uartb3 {
- compatible = "fixed-clock";
- clock-frequency = <13000000>;
- #clock-cells = <0>;
+ clock-frequency = <52000000>;
};
usb_otg_ahb_clk: usb_otg_ahb {
+ #clock-cells = <0>;
compatible = "fixed-clock";
clock-frequency = <52000000>;
+ };
+
+ ref_96m_clk: ref_96m {
#clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <96000000>;
+ };
+
+ var_96m_clk: var_96m {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <96000000>;
+ };
+
+ ref_104m_clk: ref_104m {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <104000000>;
+ };
+
+ var_104m_clk: var_104m {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <104000000>;
+ };
+
+ ref_156m_clk: ref_156m {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <156000000>;
+ };
+
+ var_156m_clk: var_156m {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <156000000>;
+ };
+
+ root_ccu: root_ccu {
+ compatible = BCM21664_DT_ROOT_CCU_COMPAT;
+ reg = <0x35001000 0x0f00>;
+ #clock-cells = <1>;
+ clock-output-names = "frac_1m";
+ };
+
+ aon_ccu: aon_ccu {
+ compatible = BCM21664_DT_AON_CCU_COMPAT;
+ reg = <0x35002000 0x0f00>;
+ #clock-cells = <1>;
+ clock-output-names = "hub_timer";
+ };
+
+ master_ccu: master_ccu {
+ compatible = BCM21664_DT_MASTER_CCU_COMPAT;
+ reg = <0x3f001000 0x0f00>;
+ #clock-cells = <1>;
+ clock-output-names = "sdio1",
+ "sdio2",
+ "sdio3",
+ "sdio4",
+ "sdio1_sleep",
+ "sdio2_sleep",
+ "sdio3_sleep",
+ "sdio4_sleep";
+ };
+
+ slave_ccu: slave_ccu {
+ compatible = BCM21664_DT_SLAVE_CCU_COMPAT;
+ reg = <0x3e011000 0x0f00>;
+ #clock-cells = <1>;
+ clock-output-names = "uartb",
+ "uartb2",
+ "uartb3",
+ "bsc1",
+ "bsc2",
+ "bsc3",
+ "bsc4";
};
};
diff --git a/arch/arm/boot/dts/marco.dtsi b/arch/arm/boot/dts/marco.dtsi
index 0c9647d..fb35422 100644
--- a/arch/arm/boot/dts/marco.dtsi
+++ b/arch/arm/boot/dts/marco.dtsi
@@ -36,7 +36,7 @@
ranges = <0x40000000 0x40000000 0xa0000000>;
l2-cache-controller@c0030000 {
- compatible = "sirf,marco-pl310-cache", "arm,pl310-cache";
+ compatible = "arm,pl310-cache";
reg = <0xc0030000 0x1000>;
interrupts = <0 59 0>;
arm,tag-latency = <1 1 1>;
diff --git a/arch/arm/boot/dts/omap3-evm-37xx.dts b/arch/arm/boot/dts/omap3-evm-37xx.dts
index 9cba94be..a8bd434 100644
--- a/arch/arm/boot/dts/omap3-evm-37xx.dts
+++ b/arch/arm/boot/dts/omap3-evm-37xx.dts
@@ -26,7 +26,44 @@
};
};
+&dss {
+ pinctrl-names = "default";
+ pinctrl-0 = <
+ &dss_dpi_pins1
+ &dss_dpi_pins2
+ >;
+};
+
&omap3_pmx_core {
+ dss_dpi_pins1: pinmux_dss_dpi_pins2 {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x20d4, PIN_OUTPUT | MUX_MODE0) /* dss_pclk.dss_pclk */
+ OMAP3_CORE1_IOPAD(0x20d6, PIN_OUTPUT | MUX_MODE0) /* dss_hsync.dss_hsync */
+ OMAP3_CORE1_IOPAD(0x20d8, PIN_OUTPUT | MUX_MODE0) /* dss_vsync.dss_vsync */
+ OMAP3_CORE1_IOPAD(0x20da, PIN_OUTPUT | MUX_MODE0) /* dss_acbias.dss_acbias */
+
+ OMAP3_CORE1_IOPAD(0x20e8, PIN_OUTPUT | MUX_MODE0) /* dss_data6.dss_data6 */
+ OMAP3_CORE1_IOPAD(0x20ea, PIN_OUTPUT | MUX_MODE0) /* dss_data7.dss_data7 */
+ OMAP3_CORE1_IOPAD(0x20ec, PIN_OUTPUT | MUX_MODE0) /* dss_data8.dss_data8 */
+ OMAP3_CORE1_IOPAD(0x20ee, PIN_OUTPUT | MUX_MODE0) /* dss_data9.dss_data9 */
+ OMAP3_CORE1_IOPAD(0x20f0, PIN_OUTPUT | MUX_MODE0) /* dss_data10.dss_data10 */
+ OMAP3_CORE1_IOPAD(0x20f2, PIN_OUTPUT | MUX_MODE0) /* dss_data11.dss_data11 */
+ OMAP3_CORE1_IOPAD(0x20f4, PIN_OUTPUT | MUX_MODE0) /* dss_data12.dss_data12 */
+ OMAP3_CORE1_IOPAD(0x20f6, PIN_OUTPUT | MUX_MODE0) /* dss_data13.dss_data13 */
+ OMAP3_CORE1_IOPAD(0x20f8, PIN_OUTPUT | MUX_MODE0) /* dss_data14.dss_data14 */
+ OMAP3_CORE1_IOPAD(0x20fa, PIN_OUTPUT | MUX_MODE0) /* dss_data15.dss_data15 */
+ OMAP3_CORE1_IOPAD(0x20fc, PIN_OUTPUT | MUX_MODE0) /* dss_data16.dss_data16 */
+ OMAP3_CORE1_IOPAD(0x20fe, PIN_OUTPUT | MUX_MODE0) /* dss_data17.dss_data17 */
+
+ OMAP3_CORE1_IOPAD(0x2100, PIN_OUTPUT | MUX_MODE3) /* dss_data18.dss_data0 */
+ OMAP3_CORE1_IOPAD(0x2102, PIN_OUTPUT | MUX_MODE3) /* dss_data19.dss_data1 */
+ OMAP3_CORE1_IOPAD(0x2104, PIN_OUTPUT | MUX_MODE3) /* dss_data20.dss_data2 */
+ OMAP3_CORE1_IOPAD(0x2106, PIN_OUTPUT | MUX_MODE3) /* dss_data21.dss_data3 */
+ OMAP3_CORE1_IOPAD(0x2108, PIN_OUTPUT | MUX_MODE3) /* dss_data22.dss_data4 */
+ OMAP3_CORE1_IOPAD(0x210a, PIN_OUTPUT | MUX_MODE3) /* dss_data23.dss_data5 */
+ >;
+ };
+
mmc1_pins: pinmux_mmc1_pins {
pinctrl-single,pins = <
0x114 (PIN_OUTPUT_PULLUP | MUX_MODE0) /* sdmmc1_clk.sdmmc1_clk */
@@ -75,6 +112,19 @@
};
};
+&omap3_pmx_wkup {
+ dss_dpi_pins2: pinmux_dss_dpi_pins1 {
+ pinctrl-single,pins = <
+ 0x0a (PIN_OUTPUT | MUX_MODE3) /* sys_boot0.dss_data18 */
+ 0x0c (PIN_OUTPUT | MUX_MODE3) /* sys_boot1.dss_data19 */
+ 0x10 (PIN_OUTPUT | MUX_MODE3) /* sys_boot3.dss_data20 */
+ 0x12 (PIN_OUTPUT | MUX_MODE3) /* sys_boot4.dss_data21 */
+ 0x14 (PIN_OUTPUT | MUX_MODE3) /* sys_boot5.dss_data22 */
+ 0x16 (PIN_OUTPUT | MUX_MODE3) /* sys_boot6.dss_data23 */
+ >;
+ };
+};
+
&mmc1 {
pinctrl-names = "default";
pinctrl-0 = <&mmc1_pins>;
diff --git a/arch/arm/boot/dts/omap3-evm-common.dtsi b/arch/arm/boot/dts/omap3-evm-common.dtsi
index 3007e79..8ae8f00 100644
--- a/arch/arm/boot/dts/omap3-evm-common.dtsi
+++ b/arch/arm/boot/dts/omap3-evm-common.dtsi
@@ -44,6 +44,11 @@
#include "twl4030.dtsi"
#include "twl4030_omap3.dtsi"
+#include "omap3-panel-sharp-ls037v7dw01.dtsi"
+
+&backlight0 {
+ gpios = <&twl_gpio 18 GPIO_ACTIVE_LOW>;
+};
&i2c2 {
clock-frequency = <400000>;
@@ -61,6 +66,27 @@
};
};
+&lcd_3v3 {
+ gpio = <&gpio5 25 GPIO_ACTIVE_LOW>; /* gpio153 */
+ enable-active-low;
+};
+
+&lcd0 {
+ enable-gpios = <&gpio5 24 GPIO_ACTIVE_HIGH>; /* gpio152, lcd INI */
+ reset-gpios = <&gpio5 27 GPIO_ACTIVE_HIGH>; /* gpio155, lcd RESB */
+ mode-gpios = <&gpio5 26 GPIO_ACTIVE_HIGH /* gpio154, lcd MO */
+ &gpio1 2 GPIO_ACTIVE_HIGH /* gpio2, lcd LR */
+ &gpio1 3 GPIO_ACTIVE_HIGH>; /* gpio3, lcd UD */
+};
+
+&mcspi1 {
+ tsc2046@0 {
+ interrupt-parent = <&gpio6>;
+ interrupts = <15 0>; /* gpio175 */
+ pendown-gpio = <&gpio6 15 0>;
+ };
+};
+
&mmc1 {
vmmc-supply = <&vmmc1>;
vmmc_aux-supply = <&vsim>;
diff --git a/arch/arm/boot/dts/omap3-gta04.dts b/arch/arm/boot/dts/omap3-gta04.dts
index f8ad125..021311f 100644
--- a/arch/arm/boot/dts/omap3-gta04.dts
+++ b/arch/arm/boot/dts/omap3-gta04.dts
@@ -44,6 +44,36 @@
ti,mcbsp = <&mcbsp2>;
ti,codec = <&twl_audio>;
};
+
+ spi_lcd {
+ compatible = "spi-gpio";
+ #address-cells = <0x1>;
+ #size-cells = <0x0>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&spi_gpio_pins>;
+
+ gpio-sck = <&gpio1 12 0>;
+ gpio-miso = <&gpio1 18 0>;
+ gpio-mosi = <&gpio1 20 0>;
+ cs-gpios = <&gpio1 19 0>;
+ num-chipselects = <1>;
+
+ /* lcd panel */
+ lcd: td028ttec1@0 {
+ compatible = "toppoly,td028ttec1";
+ reg = <0>;
+ spi-max-frequency = <100000>;
+ spi-cpol;
+ spi-cpha;
+
+ label = "lcd";
+ port {
+ lcd_in: endpoint {
+ remote-endpoint = <&dpi_out>;
+ };
+ };
+ };
+ };
};
&omap3_pmx_core {
@@ -78,6 +108,47 @@
0x11e (PIN_INPUT_PULLUP | MUX_MODE0) /* sdmmc1_dat3.sdmmc1_dat3 */
>;
};
+
+ dss_dpi_pins: pinmux_dss_dpi_pins {
+ pinctrl-single,pins = <
+ 0x0a4 (PIN_OUTPUT | MUX_MODE0) /* dss_pclk.dss_pclk */
+ 0x0a6 (PIN_OUTPUT | MUX_MODE0) /* dss_hsync.dss_hsync */
+ 0x0a8 (PIN_OUTPUT | MUX_MODE0) /* dss_vsync.dss_vsync */
+ 0x0aa (PIN_OUTPUT | MUX_MODE0) /* dss_acbias.dss_acbias */
+ 0x0ac (PIN_OUTPUT | MUX_MODE0) /* dss_data0.dss_data0 */
+ 0x0ae (PIN_OUTPUT | MUX_MODE0) /* dss_data1.dss_data1 */
+ 0x0b0 (PIN_OUTPUT | MUX_MODE0) /* dss_data2.dss_data2 */
+ 0x0b2 (PIN_OUTPUT | MUX_MODE0) /* dss_data3.dss_data3 */
+ 0x0b4 (PIN_OUTPUT | MUX_MODE0) /* dss_data4.dss_data4 */
+ 0x0b6 (PIN_OUTPUT | MUX_MODE0) /* dss_data5.dss_data5 */
+ 0x0b8 (PIN_OUTPUT | MUX_MODE0) /* dss_data6.dss_data6 */
+ 0x0ba (PIN_OUTPUT | MUX_MODE0) /* dss_data7.dss_data7 */
+ 0x0bc (PIN_OUTPUT | MUX_MODE0) /* dss_data8.dss_data8 */
+ 0x0be (PIN_OUTPUT | MUX_MODE0) /* dss_data9.dss_data9 */
+ 0x0c0 (PIN_OUTPUT | MUX_MODE0) /* dss_data10.dss_data10 */
+ 0x0c2 (PIN_OUTPUT | MUX_MODE0) /* dss_data11.dss_data11 */
+ 0x0c4 (PIN_OUTPUT | MUX_MODE0) /* dss_data12.dss_data12 */
+ 0x0c6 (PIN_OUTPUT | MUX_MODE0) /* dss_data13.dss_data13 */
+ 0x0c8 (PIN_OUTPUT | MUX_MODE0) /* dss_data14.dss_data14 */
+ 0x0ca (PIN_OUTPUT | MUX_MODE0) /* dss_data15.dss_data15 */
+ 0x0cc (PIN_OUTPUT | MUX_MODE0) /* dss_data16.dss_data16 */
+ 0x0ce (PIN_OUTPUT | MUX_MODE0) /* dss_data17.dss_data17 */
+ 0x0d0 (PIN_OUTPUT | MUX_MODE0) /* dss_data18.dss_data18 */
+ 0x0d2 (PIN_OUTPUT | MUX_MODE0) /* dss_data19.dss_data19 */
+ 0x0d4 (PIN_OUTPUT | MUX_MODE0) /* dss_data20.dss_data20 */
+ 0x0d6 (PIN_OUTPUT | MUX_MODE0) /* dss_data21.dss_data21 */
+ 0x0d8 (PIN_OUTPUT | MUX_MODE0) /* dss_data22.dss_data22 */
+ 0x0da (PIN_OUTPUT | MUX_MODE0) /* dss_data23.dss_data23 */
+ >;
+ };
+
+ spi_gpio_pins: spi_gpio_pinmux {
+ pinctrl-single,pins = <0x5a8 (PIN_OUTPUT | MUX_MODE4) /* clk */
+ 0x5b6 (PIN_OUTPUT | MUX_MODE4) /* cs */
+ 0x5b8 (PIN_OUTPUT | MUX_MODE4) /* tx */
+ 0x5b4 (PIN_INPUT | MUX_MODE4) /* rx */
+ >;
+ };
};
&i2c1 {
@@ -219,3 +290,22 @@
regulator-min-microvolt = <2800000>;
regulator-max-microvolt = <3150000>;
};
+
+/* Needed to power the DPI pins */
+&vpll2 {
+ regulator-always-on;
+};
+
+&dss {
+ pinctrl-names = "default";
+ pinctrl-0 = < &dss_dpi_pins >;
+
+ status = "okay";
+
+ port {
+ dpi_out: endpoint {
+ remote-endpoint = <&lcd_in>;
+ data-lines = <24>;
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/omap3-ldp.dts b/arch/arm/boot/dts/omap3-ldp.dts
index 476ff15..af272c1 100644
--- a/arch/arm/boot/dts/omap3-ldp.dts
+++ b/arch/arm/boot/dts/omap3-ldp.dts
@@ -164,6 +164,11 @@
#include "twl4030.dtsi"
#include "twl4030_omap3.dtsi"
+#include "omap3-panel-sharp-ls037v7dw01.dtsi"
+
+&backlight0 {
+ gpios = <&twl_gpio 7 GPIO_ACTIVE_HIGH>;
+};
&i2c2 {
clock-frequency = <400000>;
@@ -173,6 +178,25 @@
clock-frequency = <400000>;
};
+/* tps61130rsa enabled by twl4030 regen */
+&lcd_3v3 {
+ regulator-always-on;
+};
+
+&lcd0 {
+ enable-gpios = <&twl_gpio 15 GPIO_ACTIVE_HIGH>; /* lcd INI */
+ reset-gpios = <&gpio2 23 GPIO_ACTIVE_HIGH>; /* gpio55, lcd RESB */
+ mode-gpios = <&gpio2 24 GPIO_ACTIVE_HIGH>; /* gpio56, lcd MO */
+};
+
+&mcspi1 {
+ tsc2046@0 {
+ interrupt-parent = <&gpio2>;
+ interrupts = <22 0>; /* gpio54 */
+ pendown-gpio = <&gpio2 22 0>;
+ };
+};
+
&mmc1 {
/* See 35xx errata 2.1.1.128 in SPRZ278F */
compatible = "ti,omap3-pre-es3-hsmmc";
@@ -251,8 +275,3 @@
/* Needed for ads7846 */
regulator-name = "vcc";
};
-
-&vpll2 {
- /* Needed for DSS */
- regulator-name = "vdds_dsi";
-};
diff --git a/arch/arm/boot/dts/omap3-n900.dts b/arch/arm/boot/dts/omap3-n900.dts
index 059a8ff1..ae8ae3f 100644
--- a/arch/arm/boot/dts/omap3-n900.dts
+++ b/arch/arm/boot/dts/omap3-n900.dts
@@ -651,9 +651,24 @@
* Also... order in the device tree actually matters here.
*/
tsc2005@0 {
- compatible = "tsc2005";
+ compatible = "ti,tsc2005";
spi-max-frequency = <6000000>;
reg = <0>;
+
+ vio-supply = <&vio>;
+
+ reset-gpios = <&gpio4 8 GPIO_ACTIVE_HIGH>; /* 104 */
+ interrupts-extended = <&gpio4 4 IRQ_TYPE_EDGE_RISING>; /* 100 */
+
+ touchscreen-fuzz-x = <4>;
+ touchscreen-fuzz-y = <7>;
+ touchscreen-fuzz-pressure = <2>;
+ touchscreen-max-x = <4096>;
+ touchscreen-max-y = <4096>;
+ touchscreen-max-pressure = <2048>;
+
+ ti,x-plate-ohms = <280>;
+ ti,esd-recovery-timeout-ms = <8000>;
};
acx565akm@2 {
diff --git a/arch/arm/boot/dts/omap3-overo-alto35-common.dtsi b/arch/arm/boot/dts/omap3-overo-alto35-common.dtsi
index 19d6486..7aae8fb 100644
--- a/arch/arm/boot/dts/omap3-overo-alto35-common.dtsi
+++ b/arch/arm/boot/dts/omap3-overo-alto35-common.dtsi
@@ -11,6 +11,7 @@
*/
#include "omap3-overo-common-peripherals.dtsi"
+#include "omap3-overo-common-lcd35.dtsi"
#include <dt-bindings/input/input.h>
diff --git a/arch/arm/boot/dts/omap3-overo-chestnut43-common.dtsi b/arch/arm/boot/dts/omap3-overo-chestnut43-common.dtsi
index 19de6ff..17b82f8 100644
--- a/arch/arm/boot/dts/omap3-overo-chestnut43-common.dtsi
+++ b/arch/arm/boot/dts/omap3-overo-chestnut43-common.dtsi
@@ -11,6 +11,7 @@
*/
#include "omap3-overo-common-peripherals.dtsi"
+#include "omap3-overo-common-lcd43.dtsi"
#include <dt-bindings/input/input.h>
diff --git a/arch/arm/boot/dts/omap3-overo-common-dvi.dtsi b/arch/arm/boot/dts/omap3-overo-common-dvi.dtsi
new file mode 100644
index 0000000..802f704f
--- /dev/null
+++ b/arch/arm/boot/dts/omap3-overo-common-dvi.dtsi
@@ -0,0 +1,111 @@
+/*
+ * Copyright (C) 2014 Florian Vaussard, EPFL Mobots group
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+/*
+ * DVI output for some Gumstix Overo boards (Tobi and Summit)
+ */
+
+&omap3_pmx_core {
+ dss_dpi_pins: pinmux_dss_dpi_pins {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x20d4, PIN_OUTPUT | MUX_MODE0) /* dss_pclk.dss_pclk */
+ OMAP3_CORE1_IOPAD(0x20d6, PIN_OUTPUT | MUX_MODE0) /* dss_hsync.dss_hsync */
+ OMAP3_CORE1_IOPAD(0x20d8, PIN_OUTPUT | MUX_MODE0) /* dss_vsync.dss_vsync */
+ OMAP3_CORE1_IOPAD(0x20da, PIN_OUTPUT | MUX_MODE0) /* dss_acbias.dss_acbias */
+ OMAP3_CORE1_IOPAD(0x20dc, PIN_OUTPUT | MUX_MODE0) /* dss_data0.dss_data0 */
+ OMAP3_CORE1_IOPAD(0x20de, PIN_OUTPUT | MUX_MODE0) /* dss_data1.dss_data1 */
+ OMAP3_CORE1_IOPAD(0x20e0, PIN_OUTPUT | MUX_MODE0) /* dss_data2.dss_data2 */
+ OMAP3_CORE1_IOPAD(0x20e2, PIN_OUTPUT | MUX_MODE0) /* dss_data3.dss_data3 */
+ OMAP3_CORE1_IOPAD(0x20e4, PIN_OUTPUT | MUX_MODE0) /* dss_data4.dss_data4 */
+ OMAP3_CORE1_IOPAD(0x20e6, PIN_OUTPUT | MUX_MODE0) /* dss_data5.dss_data5 */
+ OMAP3_CORE1_IOPAD(0x20e8, PIN_OUTPUT | MUX_MODE0) /* dss_data6.dss_data6 */
+ OMAP3_CORE1_IOPAD(0x20ea, PIN_OUTPUT | MUX_MODE0) /* dss_data7.dss_data7 */
+ OMAP3_CORE1_IOPAD(0x20ec, PIN_OUTPUT | MUX_MODE0) /* dss_data8.dss_data8 */
+ OMAP3_CORE1_IOPAD(0x20ee, PIN_OUTPUT | MUX_MODE0) /* dss_data9.dss_data9 */
+ OMAP3_CORE1_IOPAD(0x20f0, PIN_OUTPUT | MUX_MODE0) /* dss_data10.dss_data10 */
+ OMAP3_CORE1_IOPAD(0x20f2, PIN_OUTPUT | MUX_MODE0) /* dss_data11.dss_data11 */
+ OMAP3_CORE1_IOPAD(0x20f4, PIN_OUTPUT | MUX_MODE0) /* dss_data12.dss_data12 */
+ OMAP3_CORE1_IOPAD(0x20f6, PIN_OUTPUT | MUX_MODE0) /* dss_data13.dss_data13 */
+ OMAP3_CORE1_IOPAD(0x20f8, PIN_OUTPUT | MUX_MODE0) /* dss_data14.dss_data14 */
+ OMAP3_CORE1_IOPAD(0x20fa, PIN_OUTPUT | MUX_MODE0) /* dss_data15.dss_data15 */
+ OMAP3_CORE1_IOPAD(0x20fc, PIN_OUTPUT | MUX_MODE0) /* dss_data16.dss_data16 */
+ OMAP3_CORE1_IOPAD(0x20fe, PIN_OUTPUT | MUX_MODE0) /* dss_data17.dss_data17 */
+ OMAP3_CORE1_IOPAD(0x2100, PIN_OUTPUT | MUX_MODE0) /* dss_data18.dss_data18 */
+ OMAP3_CORE1_IOPAD(0x2102, PIN_OUTPUT | MUX_MODE0) /* dss_data19.dss_data19 */
+ OMAP3_CORE1_IOPAD(0x2104, PIN_OUTPUT | MUX_MODE0) /* dss_data20.dss_data20 */
+ OMAP3_CORE1_IOPAD(0x2106, PIN_OUTPUT | MUX_MODE0) /* dss_data21.dss_data21 */
+ OMAP3_CORE1_IOPAD(0x2108, PIN_OUTPUT | MUX_MODE0) /* dss_data22.dss_data22 */
+ OMAP3_CORE1_IOPAD(0x210a, PIN_OUTPUT | MUX_MODE0) /* dss_data23.dss_data23 */
+ >;
+ };
+};
+
+/* Needed to power the DPI pins */
+&vpll2 {
+ regulator-always-on;
+};
+
+&dss {
+ status = "ok";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&dss_dpi_pins>;
+
+ port {
+ dpi_out: endpoint {
+ remote-endpoint = <&tfp410_in>;
+ data-lines = <24>;
+ };
+ };
+};
+
+/ {
+ aliases {
+ display0 = &dvi0;
+ };
+
+ tfp410: encoder@0 {
+ compatible = "ti,tfp410";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ tfp410_in: endpoint@0 {
+ remote-endpoint = <&dpi_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ tfp410_out: endpoint@0 {
+ remote-endpoint = <&dvi_connector_in>;
+ };
+ };
+ };
+ };
+
+ dvi0: connector@0 {
+ compatible = "dvi-connector";
+ label = "dvi";
+
+ digital;
+ ddc-i2c-bus = <&i2c3>;
+
+ port {
+ dvi_connector_in: endpoint {
+ remote-endpoint = <&tfp410_out>;
+ };
+ };
+ };
+};
+
diff --git a/arch/arm/boot/dts/omap3-overo-common-lcd35.dtsi b/arch/arm/boot/dts/omap3-overo-common-lcd35.dtsi
new file mode 100644
index 0000000..233c69e
--- /dev/null
+++ b/arch/arm/boot/dts/omap3-overo-common-lcd35.dtsi
@@ -0,0 +1,165 @@
+/*
+ * Copyright (C) 2014 Florian Vaussard, EPFL Mobots group
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+/*
+ * 4.3'' LCD panel output for some Gumstix Overo boards (Gallop43, Chestnut43)
+ */
+
+&omap3_pmx_core {
+ dss_dpi_pins: pinmux_dss_dpi_pins {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x20d4, PIN_OUTPUT | MUX_MODE0) /* dss_pclk.dss_pclk */
+ OMAP3_CORE1_IOPAD(0x20d6, PIN_OUTPUT | MUX_MODE0) /* dss_hsync.dss_hsync */
+ OMAP3_CORE1_IOPAD(0x20d8, PIN_OUTPUT | MUX_MODE0) /* dss_vsync.dss_vsync */
+ OMAP3_CORE1_IOPAD(0x20da, PIN_OUTPUT | MUX_MODE0) /* dss_acbias.dss_acbias */
+ OMAP3_CORE1_IOPAD(0x20dc, PIN_OUTPUT | MUX_MODE0) /* dss_data0.dss_data0 */
+ OMAP3_CORE1_IOPAD(0x20de, PIN_OUTPUT | MUX_MODE0) /* dss_data1.dss_data1 */
+ OMAP3_CORE1_IOPAD(0x20e0, PIN_OUTPUT | MUX_MODE0) /* dss_data2.dss_data2 */
+ OMAP3_CORE1_IOPAD(0x20e2, PIN_OUTPUT | MUX_MODE0) /* dss_data3.dss_data3 */
+ OMAP3_CORE1_IOPAD(0x20e4, PIN_OUTPUT | MUX_MODE0) /* dss_data4.dss_data4 */
+ OMAP3_CORE1_IOPAD(0x20e6, PIN_OUTPUT | MUX_MODE0) /* dss_data5.dss_data5 */
+ OMAP3_CORE1_IOPAD(0x20e8, PIN_OUTPUT | MUX_MODE0) /* dss_data6.dss_data6 */
+ OMAP3_CORE1_IOPAD(0x20ea, PIN_OUTPUT | MUX_MODE0) /* dss_data7.dss_data7 */
+ OMAP3_CORE1_IOPAD(0x20ec, PIN_OUTPUT | MUX_MODE0) /* dss_data8.dss_data8 */
+ OMAP3_CORE1_IOPAD(0x20ee, PIN_OUTPUT | MUX_MODE0) /* dss_data9.dss_data9 */
+ OMAP3_CORE1_IOPAD(0x20f0, PIN_OUTPUT | MUX_MODE0) /* dss_data10.dss_data10 */
+ OMAP3_CORE1_IOPAD(0x20f2, PIN_OUTPUT | MUX_MODE0) /* dss_data11.dss_data11 */
+ OMAP3_CORE1_IOPAD(0x20f4, PIN_OUTPUT | MUX_MODE0) /* dss_data12.dss_data12 */
+ OMAP3_CORE1_IOPAD(0x20f6, PIN_OUTPUT | MUX_MODE0) /* dss_data13.dss_data13 */
+ OMAP3_CORE1_IOPAD(0x20f8, PIN_OUTPUT | MUX_MODE0) /* dss_data14.dss_data14 */
+ OMAP3_CORE1_IOPAD(0x20fa, PIN_OUTPUT | MUX_MODE0) /* dss_data15.dss_data15 */
+ OMAP3_CORE1_IOPAD(0x20fc, PIN_OUTPUT | MUX_MODE0) /* dss_data16.dss_data16 */
+ OMAP3_CORE1_IOPAD(0x20fe, PIN_OUTPUT | MUX_MODE0) /* dss_data17.dss_data17 */
+ OMAP3_CORE1_IOPAD(0x2100, PIN_OUTPUT | MUX_MODE0) /* dss_data18.dss_data18 */
+ OMAP3_CORE1_IOPAD(0x2102, PIN_OUTPUT | MUX_MODE0) /* dss_data19.dss_data19 */
+ OMAP3_CORE1_IOPAD(0x2104, PIN_OUTPUT | MUX_MODE0) /* dss_data20.dss_data20 */
+ OMAP3_CORE1_IOPAD(0x2106, PIN_OUTPUT | MUX_MODE0) /* dss_data21.dss_data21 */
+ OMAP3_CORE1_IOPAD(0x2108, PIN_OUTPUT | MUX_MODE0) /* dss_data22.dss_data22 */
+ OMAP3_CORE1_IOPAD(0x210a, PIN_OUTPUT | MUX_MODE0) /* dss_data23.dss_data23 */
+ >;
+ };
+
+ lb035_pins: pinmux_lb035_pins {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x2174, PIN_OUTPUT | MUX_MODE4) /* uart2_cts.gpio_144 */
+ >;
+ };
+
+ backlight_pins: pinmux_backlight_pins {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x2176, PIN_OUTPUT | MUX_MODE4) /* uart2_rts.gpio_145 */
+ >;
+ };
+
+ mcspi1_pins: pinmux_mcspi1_pins {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x21c8, PIN_INPUT | MUX_MODE0) /* mcspi1_clk.mcspi1_clk */
+ OMAP3_CORE1_IOPAD(0x21ca, PIN_INPUT | MUX_MODE0) /* mcspi1_simo.mcspi1_simo */
+ OMAP3_CORE1_IOPAD(0x21cc, PIN_INPUT | MUX_MODE0) /* mcspi1_somi.mcspi1_somi */
+ OMAP3_CORE1_IOPAD(0x21ce, PIN_INPUT | MUX_MODE0) /* mcspi1_cs0.mcspi1_cs0 */
+ >;
+ };
+
+ ads7846_pins: pinmux_ads7846_pins {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x2138, PIN_INPUT_PULLDOWN | MUX_MODE4) /* csi2_dx1.gpio_114 */
+ >;
+ };
+};
+
+/* Needed to power the DPI pins */
+&vpll2 {
+ regulator-always-on;
+};
+
+&dss {
+ status = "ok";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&dss_dpi_pins>;
+
+ port {
+ dpi_out: endpoint {
+ remote-endpoint = <&lcd_in>;
+ data-lines = <24>;
+ };
+ };
+};
+
+/ {
+ aliases {
+ display0 = &lcd0;
+ };
+
+ ads7846reg: ads7846-reg {
+ compatible = "regulator-fixed";
+ regulator-name = "ads7846-reg";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ backlight {
+ compatible = "gpio-backlight";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&backlight_pins>;
+ gpios = <&gpio5 17 GPIO_ACTIVE_HIGH>; /* gpio_145 */
+
+ default-on;
+ };
+};
+
+&mcspi1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcspi1_pins>;
+
+ lcd0: display@0 {
+ compatible = "lgphilips,lb035q02";
+ label = "lcd";
+
+ reg = <1>; /* CS1 */
+ spi-max-frequency = <10000000>;
+ spi-cpol;
+ spi-cpha;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&lb035_pins>;
+ enable-gpios = <&gpio5 16 GPIO_ACTIVE_HIGH>; /* gpio_144 */
+
+ port {
+ lcd_in: endpoint {
+ remote-endpoint = <&dpi_out>;
+ };
+ };
+ };
+
+ /* touch controller */
+ ads7846@0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&ads7846_pins>;
+
+ compatible = "ti,ads7846";
+ vcc-supply = <&ads7846reg>;
+
+ reg = <0>; /* CS0 */
+ spi-max-frequency = <1500000>;
+
+ interrupt-parent = <&gpio4>;
+ interrupts = <18 0>; /* gpio_114 */
+ pendown-gpio = <&gpio4 18 0>;
+
+ ti,x-min = /bits/ 16 <0x0>;
+ ti,x-max = /bits/ 16 <0x0fff>;
+ ti,y-min = /bits/ 16 <0x0>;
+ ti,y-max = /bits/ 16 <0x0fff>;
+ ti,x-plate-ohms = /bits/ 16 <180>;
+ ti,pressure-max = /bits/ 16 <255>;
+
+ linux,wakeup;
+ };
+};
diff --git a/arch/arm/boot/dts/omap3-overo-common-lcd43.dtsi b/arch/arm/boot/dts/omap3-overo-common-lcd43.dtsi
new file mode 100644
index 0000000..f5395b7
--- /dev/null
+++ b/arch/arm/boot/dts/omap3-overo-common-lcd43.dtsi
@@ -0,0 +1,178 @@
+/*
+ * Copyright (C) 2014 Florian Vaussard, EPFL Mobots group
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+/*
+ * 4.3'' LCD panel output for some Gumstix Overo boards (Gallop43, Chestnut43)
+ */
+
+&omap3_pmx_core {
+ dss_dpi_pins: pinmux_dss_dpi_pins {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x20d4, PIN_OUTPUT | MUX_MODE0) /* dss_pclk.dss_pclk */
+ OMAP3_CORE1_IOPAD(0x20d6, PIN_OUTPUT | MUX_MODE0) /* dss_hsync.dss_hsync */
+ OMAP3_CORE1_IOPAD(0x20d8, PIN_OUTPUT | MUX_MODE0) /* dss_vsync.dss_vsync */
+ OMAP3_CORE1_IOPAD(0x20da, PIN_OUTPUT | MUX_MODE0) /* dss_acbias.dss_acbias */
+ OMAP3_CORE1_IOPAD(0x20dc, PIN_OUTPUT | MUX_MODE0) /* dss_data0.dss_data0 */
+ OMAP3_CORE1_IOPAD(0x20de, PIN_OUTPUT | MUX_MODE0) /* dss_data1.dss_data1 */
+ OMAP3_CORE1_IOPAD(0x20e0, PIN_OUTPUT | MUX_MODE0) /* dss_data2.dss_data2 */
+ OMAP3_CORE1_IOPAD(0x20e2, PIN_OUTPUT | MUX_MODE0) /* dss_data3.dss_data3 */
+ OMAP3_CORE1_IOPAD(0x20e4, PIN_OUTPUT | MUX_MODE0) /* dss_data4.dss_data4 */
+ OMAP3_CORE1_IOPAD(0x20e6, PIN_OUTPUT | MUX_MODE0) /* dss_data5.dss_data5 */
+ OMAP3_CORE1_IOPAD(0x20e8, PIN_OUTPUT | MUX_MODE0) /* dss_data6.dss_data6 */
+ OMAP3_CORE1_IOPAD(0x20ea, PIN_OUTPUT | MUX_MODE0) /* dss_data7.dss_data7 */
+ OMAP3_CORE1_IOPAD(0x20ec, PIN_OUTPUT | MUX_MODE0) /* dss_data8.dss_data8 */
+ OMAP3_CORE1_IOPAD(0x20ee, PIN_OUTPUT | MUX_MODE0) /* dss_data9.dss_data9 */
+ OMAP3_CORE1_IOPAD(0x20f0, PIN_OUTPUT | MUX_MODE0) /* dss_data10.dss_data10 */
+ OMAP3_CORE1_IOPAD(0x20f2, PIN_OUTPUT | MUX_MODE0) /* dss_data11.dss_data11 */
+ OMAP3_CORE1_IOPAD(0x20f4, PIN_OUTPUT | MUX_MODE0) /* dss_data12.dss_data12 */
+ OMAP3_CORE1_IOPAD(0x20f6, PIN_OUTPUT | MUX_MODE0) /* dss_data13.dss_data13 */
+ OMAP3_CORE1_IOPAD(0x20f8, PIN_OUTPUT | MUX_MODE0) /* dss_data14.dss_data14 */
+ OMAP3_CORE1_IOPAD(0x20fa, PIN_OUTPUT | MUX_MODE0) /* dss_data15.dss_data15 */
+ OMAP3_CORE1_IOPAD(0x20fc, PIN_OUTPUT | MUX_MODE0) /* dss_data16.dss_data16 */
+ OMAP3_CORE1_IOPAD(0x20fe, PIN_OUTPUT | MUX_MODE0) /* dss_data17.dss_data17 */
+ OMAP3_CORE1_IOPAD(0x2100, PIN_OUTPUT | MUX_MODE0) /* dss_data18.dss_data18 */
+ OMAP3_CORE1_IOPAD(0x2102, PIN_OUTPUT | MUX_MODE0) /* dss_data19.dss_data19 */
+ OMAP3_CORE1_IOPAD(0x2104, PIN_OUTPUT | MUX_MODE0) /* dss_data20.dss_data20 */
+ OMAP3_CORE1_IOPAD(0x2106, PIN_OUTPUT | MUX_MODE0) /* dss_data21.dss_data21 */
+ OMAP3_CORE1_IOPAD(0x2108, PIN_OUTPUT | MUX_MODE0) /* dss_data22.dss_data22 */
+ OMAP3_CORE1_IOPAD(0x210a, PIN_OUTPUT | MUX_MODE0) /* dss_data23.dss_data23 */
+ >;
+ };
+
+ lte430_pins: pinmux_lte430_pins {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x2174, PIN_OUTPUT | MUX_MODE4) /* uart2_cts.gpio_144 */
+ >;
+ };
+
+ backlight_pins: pinmux_backlight_pins {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x2176, PIN_OUTPUT | MUX_MODE4) /* uart2_rts.gpio_145 */
+ >;
+ };
+
+ mcspi1_pins: pinmux_mcspi1_pins {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x21c8, PIN_INPUT | MUX_MODE0) /* mcspi1_clk.mcspi1_clk */
+ OMAP3_CORE1_IOPAD(0x21ca, PIN_INPUT | MUX_MODE0) /* mcspi1_simo.mcspi1_simo */
+ OMAP3_CORE1_IOPAD(0x21cc, PIN_INPUT | MUX_MODE0) /* mcspi1_somi.mcspi1_somi */
+ OMAP3_CORE1_IOPAD(0x21ce, PIN_INPUT | MUX_MODE0) /* mcspi1_cs0.mcspi1_cs0 */
+ >;
+ };
+
+ ads7846_pins: pinmux_ads7846_pins {
+ pinctrl-single,pins = <
+ OMAP3_CORE1_IOPAD(0x2138, PIN_INPUT_PULLDOWN | MUX_MODE4) /* csi2_dx1.gpio_114 */
+ >;
+ };
+};
+
+/* Needed to power the DPI pins */
+&vpll2 {
+ regulator-always-on;
+};
+
+&dss {
+ status = "ok";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&dss_dpi_pins>;
+
+ port {
+ dpi_out: endpoint {
+ remote-endpoint = <&lcd_in>;
+ data-lines = <24>;
+ };
+ };
+};
+
+/ {
+ aliases {
+ display0 = &lcd0;
+ };
+
+ lcd0: display@0 {
+ compatible = "samsung,lte430wq-f0c", "panel-dpi";
+ label = "lcd";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <<e430_pins>;
+ enable-gpios = <&gpio5 16 GPIO_ACTIVE_HIGH>; /* gpio_144 */
+
+ port {
+ lcd_in: endpoint {
+ remote-endpoint = <&dpi_out>;
+ };
+ };
+
+ panel-timing {
+ clock-frequency = <9200000>;
+ hactive = <480>;
+ vactive = <272>;
+ hfront-porch = <8>;
+ hback-porch = <4>;
+ hsync-len = <41>;
+ vback-porch = <2>;
+ vfront-porch = <4>;
+ vsync-len = <10>;
+
+ hsync-active = <0>;
+ vsync-active = <0>;
+ de-active = <1>;
+ pixelclk-active = <1>;
+ };
+ };
+
+ ads7846reg: ads7846-reg {
+ compatible = "regulator-fixed";
+ regulator-name = "ads7846-reg";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ backlight {
+ compatible = "gpio-backlight";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&backlight_pins>;
+ gpios = <&gpio5 17 GPIO_ACTIVE_HIGH>; /* gpio_145 */
+
+ default-on;
+ };
+};
+
+&mcspi1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcspi1_pins>;
+
+ /* touch controller */
+ ads7846@0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&ads7846_pins>;
+
+ compatible = "ti,ads7846";
+ vcc-supply = <&ads7846reg>;
+
+ reg = <0>; /* CS0 */
+ spi-max-frequency = <1500000>;
+
+ interrupt-parent = <&gpio4>;
+ interrupts = <18 0>; /* gpio_114 */
+ pendown-gpio = <&gpio4 18 0>;
+
+ ti,x-min = /bits/ 16 <0x0>;
+ ti,x-max = /bits/ 16 <0x0fff>;
+ ti,y-min = /bits/ 16 <0x0>;
+ ti,y-max = /bits/ 16 <0x0fff>;
+ ti,x-plate-ohms = /bits/ 16 <180>;
+ ti,pressure-max = /bits/ 16 <255>;
+
+ linux,wakeup;
+ };
+};
+
diff --git a/arch/arm/boot/dts/omap3-overo-gallop43-common.dtsi b/arch/arm/boot/dts/omap3-overo-gallop43-common.dtsi
index 5e848c2..49d2254 100644
--- a/arch/arm/boot/dts/omap3-overo-gallop43-common.dtsi
+++ b/arch/arm/boot/dts/omap3-overo-gallop43-common.dtsi
@@ -11,6 +11,7 @@
*/
#include "omap3-overo-common-peripherals.dtsi"
+#include "omap3-overo-common-lcd43.dtsi"
#include <dt-bindings/input/input.h>
diff --git a/arch/arm/boot/dts/omap3-overo-palo43-common.dtsi b/arch/arm/boot/dts/omap3-overo-palo43-common.dtsi
index abea232..087aedf 100644
--- a/arch/arm/boot/dts/omap3-overo-palo43-common.dtsi
+++ b/arch/arm/boot/dts/omap3-overo-palo43-common.dtsi
@@ -11,6 +11,7 @@
*/
#include "omap3-overo-common-peripherals.dtsi"
+#include "omap3-overo-common-lcd43.dtsi"
#include <dt-bindings/input/input.h>
diff --git a/arch/arm/boot/dts/omap3-overo-summit-common.dtsi b/arch/arm/boot/dts/omap3-overo-summit-common.dtsi
index 999d1cd..0ac97ba 100644
--- a/arch/arm/boot/dts/omap3-overo-summit-common.dtsi
+++ b/arch/arm/boot/dts/omap3-overo-summit-common.dtsi
@@ -11,6 +11,7 @@
*/
#include "omap3-overo-common-peripherals.dtsi"
+#include "omap3-overo-common-dvi.dtsi"
/ {
leds {
diff --git a/arch/arm/boot/dts/omap3-overo-tobi-common.dtsi b/arch/arm/boot/dts/omap3-overo-tobi-common.dtsi
index 13df50b..9e24b6a 100644
--- a/arch/arm/boot/dts/omap3-overo-tobi-common.dtsi
+++ b/arch/arm/boot/dts/omap3-overo-tobi-common.dtsi
@@ -11,6 +11,7 @@
*/
#include "omap3-overo-common-peripherals.dtsi"
+#include "omap3-overo-common-dvi.dtsi"
/ {
leds {
diff --git a/arch/arm/boot/dts/omap3-panel-sharp-ls037v7dw01.dtsi b/arch/arm/boot/dts/omap3-panel-sharp-ls037v7dw01.dtsi
new file mode 100644
index 0000000..f4b1a61
--- /dev/null
+++ b/arch/arm/boot/dts/omap3-panel-sharp-ls037v7dw01.dtsi
@@ -0,0 +1,71 @@
+/*
+ * Common file for omap dpi panels with QVGA and reset pins
+ *
+ * Note that the board specifc DTS file needs to specify
+ * at minimum the GPIO enable-gpios for display, and
+ * gpios for gpio-backlight.
+ */
+
+/ {
+ aliases {
+ display0 = &lcd0;
+ };
+
+ backlight0: backlight {
+ compatible = "gpio-backlight";
+ default-on;
+ };
+
+ /* 3.3V GPIO controlled regulator for LCD_ENVDD */
+ lcd_3v3: regulator-lcd-3v3 {
+ compatible = "regulator-fixed";
+ regulator-name = "lcd_3v3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ startup-delay-us = <70000>;
+ };
+
+ lcd0: display {
+ compatible = "sharp,ls037v7dw01";
+ label = "lcd";
+ power-supply = <&lcd_3v3>;
+
+ port {
+ lcd_in: endpoint {
+ remote-endpoint = <&dpi_out>;
+ };
+ };
+ };
+};
+
+/* Needed to power the DPI pins */
+&vpll2 {
+ regulator-always-on;
+};
+
+&dss {
+ status = "ok";
+ port {
+ dpi_out: endpoint {
+ remote-endpoint = <&lcd_in>;
+ data-lines = <18>;
+ };
+ };
+};
+
+&mcspi1 {
+ tsc2046@0 {
+ reg = <0>; /* CS0 */
+ compatible = "ti,tsc2046";
+ spi-max-frequency = <1000000>;
+ vcc-supply = <&lcd_3v3>;
+ ti,x-min = /bits/ 16 <0>;
+ ti,x-max = /bits/ 16 <8000>;
+ ti,y-min = /bits/ 16 <0>;
+ ti,y-max = /bits/ 16 <4800>;
+ ti,x-plate-ohms = /bits/ 16 <40>;
+ ti,pressure-max = /bits/ 16 <255>;
+ ti,swap-xy;
+ linux,wakeup;
+ };
+};
diff --git a/arch/arm/boot/dts/omap4-duovero-parlor.dts b/arch/arm/boot/dts/omap4-duovero-parlor.dts
index cd53a64..6dc84d9 100644
--- a/arch/arm/boot/dts/omap4-duovero-parlor.dts
+++ b/arch/arm/boot/dts/omap4-duovero-parlor.dts
@@ -15,6 +15,10 @@
model = "OMAP4430 Gumstix Duovero on Parlor";
compatible = "gumstix,omap4-duovero-parlor", "gumstix,omap4-duovero", "ti,omap4430", "ti,omap4";
+ aliases {
+ display0 = &hdmi0;
+ };
+
leds {
compatible = "gpio-leds";
led0 {
@@ -35,6 +39,21 @@
gpio-key,wakeup;
};
};
+
+ hdmi0: connector@0 {
+ compatible = "hdmi-connector";
+ label = "hdmi";
+
+ type = "d";
+
+ hpd-gpios = <&gpio2 31 GPIO_ACTIVE_HIGH>; /* gpio_63 */
+
+ port {
+ hdmi_connector_in: endpoint {
+ remote-endpoint = <&hdmi_out>;
+ };
+ };
+ };
};
&omap4_pmx_core {
@@ -77,6 +96,15 @@
OMAP4_IOPAD(0x070, PIN_INPUT_PULLUP | MUX_MODE3) /* gpmc_a24.gpio_48: amdix enabled */
>;
};
+
+ dss_hdmi_pins: pinmux_dss_hdmi_pins {
+ pinctrl-single,pins = <
+ OMAP4_IOPAD(0x098, PIN_INPUT | MUX_MODE3) /* hdmi_hpd.gpio_63 */
+ OMAP4_IOPAD(0x09a, PIN_INPUT_PULLUP | MUX_MODE0) /* hdmi_cec.hdmi_cec */
+ OMAP4_IOPAD(0x09c, PIN_INPUT_PULLUP | MUX_MODE0) /* hdmi_ddc_scl.hdmi_ddc_scl */
+ OMAP4_IOPAD(0x09e, PIN_INPUT_PULLUP | MUX_MODE0) /* hdmi_ddc_sda.hdmi_ddc_sda */
+ >;
+ };
};
&i2c2 {
@@ -143,4 +171,20 @@
};
};
+&dss {
+ status = "ok";
+};
+
+&hdmi {
+ status = "ok";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&dss_hdmi_pins>;
+
+ port {
+ hdmi_out: endpoint {
+ remote-endpoint = <&hdmi_connector_in>;
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/omap4.dtsi b/arch/arm/boot/dts/omap4.dtsi
index 43a587e..7e26d22 100644
--- a/arch/arm/boot/dts/omap4.dtsi
+++ b/arch/arm/boot/dts/omap4.dtsi
@@ -922,6 +922,8 @@
ti,hwmods = "dss_hdmi";
clocks = <&dss_48mhz_clk>, <&dss_sys_clk>;
clock-names = "fck", "sys_clk";
+ dmas = <&sdma 76>;
+ dma-names = "audio_tx";
};
};
};
diff --git a/arch/arm/boot/dts/omap5-uevm.dts b/arch/arm/boot/dts/omap5-uevm.dts
index 3b99ec2..1e1b057 100644
--- a/arch/arm/boot/dts/omap5-uevm.dts
+++ b/arch/arm/boot/dts/omap5-uevm.dts
@@ -20,6 +20,10 @@
reg = <0x80000000 0x7F000000>; /* 2032 MB */
};
+ aliases {
+ display0 = &hdmi0;
+ };
+
vmmcsd_fixed: fixedregulator-mmcsd {
compatible = "regulator-fixed";
regulator-name = "vmmcsd_fixed";
@@ -51,6 +55,51 @@
default-state = "off";
};
};
+
+ tpd12s015: encoder@0 {
+ compatible = "ti,tpd12s015";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&tpd12s015_pins>;
+
+ gpios = <&gpio9 0 GPIO_ACTIVE_HIGH>, /* TCA6424A P01, CT CP HPD */
+ <&gpio9 1 GPIO_ACTIVE_HIGH>, /* TCA6424A P00, LS OE */
+ <&gpio7 1 GPIO_ACTIVE_HIGH>; /* GPIO 193, HPD */
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ tpd12s015_in: endpoint@0 {
+ remote-endpoint = <&hdmi_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ tpd12s015_out: endpoint@0 {
+ remote-endpoint = <&hdmi_connector_in>;
+ };
+ };
+ };
+ };
+
+ hdmi0: connector@0 {
+ compatible = "hdmi-connector";
+ label = "hdmi";
+
+ type = "b";
+
+ port {
+ hdmi_connector_in: endpoint {
+ remote-endpoint = <&tpd12s015_out>;
+ };
+ };
+ };
};
&omap5_pmx_core {
@@ -183,6 +232,19 @@
>;
};
+ dss_hdmi_pins: pinmux_dss_hdmi_pins {
+ pinctrl-single,pins = <
+ 0x0fc (PIN_INPUT_PULLUP | MUX_MODE0) /* hdmi_cec.hdmi_cec */
+ 0x100 (PIN_INPUT | MUX_MODE0) /* hdmi_ddc_scl.hdmi_ddc_scl */
+ 0x102 (PIN_INPUT | MUX_MODE0) /* hdmi_ddc_sda.hdmi_ddc_sda */
+ >;
+ };
+
+ tpd12s015_pins: pinmux_tpd12s015_pins {
+ pinctrl-single,pins = <
+ 0x0fe (PIN_INPUT_PULLDOWN | MUX_MODE6) /* hdmi_hpd.gpio7_193 */
+ >;
+ };
};
&omap5_pmx_wkup {
@@ -434,6 +496,13 @@
pinctrl-0 = <&i2c5_pins>;
clock-frequency = <400000>;
+
+ gpio9: gpio@22 {
+ compatible = "ti,tca6424";
+ reg = <0x22>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ };
};
&mcbsp3 {
@@ -491,3 +560,21 @@
&cpu0 {
cpu0-supply = <&smps123_reg>;
};
+
+&dss {
+ status = "ok";
+};
+
+&hdmi {
+ status = "ok";
+ vdda-supply = <&ldo4_reg>;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&dss_hdmi_pins>;
+
+ port {
+ hdmi_out: endpoint {
+ remote-endpoint = <&tpd12s015_in>;
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/omap5.dtsi b/arch/arm/boot/dts/omap5.dtsi
index e58be57..3bfda16 100644
--- a/arch/arm/boot/dts/omap5.dtsi
+++ b/arch/arm/boot/dts/omap5.dtsi
@@ -924,6 +924,68 @@
ti,hwmods = "sata";
};
+ dss: dss@58000000 {
+ compatible = "ti,omap5-dss";
+ reg = <0x58000000 0x80>;
+ status = "disabled";
+ ti,hwmods = "dss_core";
+ clocks = <&dss_dss_clk>;
+ clock-names = "fck";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ dispc@58001000 {
+ compatible = "ti,omap5-dispc";
+ reg = <0x58001000 0x1000>;
+ interrupts = <GIC_SPI 25 IRQ_TYPE_LEVEL_HIGH>;
+ ti,hwmods = "dss_dispc";
+ clocks = <&dss_dss_clk>;
+ clock-names = "fck";
+ };
+
+ dsi1: encoder@58004000 {
+ compatible = "ti,omap5-dsi";
+ reg = <0x58004000 0x200>,
+ <0x58004200 0x40>,
+ <0x58004300 0x40>;
+ reg-names = "proto", "phy", "pll";
+ interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
+ status = "disabled";
+ ti,hwmods = "dss_dsi1";
+ clocks = <&dss_dss_clk>, <&dss_sys_clk>;
+ clock-names = "fck", "sys_clk";
+ };
+
+ dsi2: encoder@58005000 {
+ compatible = "ti,omap5-dsi";
+ reg = <0x58009000 0x200>,
+ <0x58009200 0x40>,
+ <0x58009300 0x40>;
+ reg-names = "proto", "phy", "pll";
+ interrupts = <GIC_SPI 55 IRQ_TYPE_LEVEL_HIGH>;
+ status = "disabled";
+ ti,hwmods = "dss_dsi2";
+ clocks = <&dss_dss_clk>, <&dss_sys_clk>;
+ clock-names = "fck", "sys_clk";
+ };
+
+ hdmi: encoder@58060000 {
+ compatible = "ti,omap5-hdmi";
+ reg = <0x58040000 0x200>,
+ <0x58040200 0x80>,
+ <0x58040300 0x80>,
+ <0x58060000 0x19000>;
+ reg-names = "wp", "pll", "phy", "core";
+ interrupts = <GIC_SPI 101 IRQ_TYPE_LEVEL_HIGH>;
+ status = "disabled";
+ ti,hwmods = "dss_hdmi";
+ clocks = <&dss_48mhz_clk>, <&dss_sys_clk>;
+ clock-names = "fck", "sys_clk";
+ dmas = <&sdma 76>;
+ dma-names = "audio_tx";
+ };
+ };
};
};
diff --git a/arch/arm/boot/dts/prima2.dtsi b/arch/arm/boot/dts/prima2.dtsi
index 3df7ba8..963b7e5 100644
--- a/arch/arm/boot/dts/prima2.dtsi
+++ b/arch/arm/boot/dts/prima2.dtsi
@@ -48,7 +48,7 @@
ranges = <0x40000000 0x40000000 0x80000000>;
l2-cache-controller@80040000 {
- compatible = "arm,pl310-cache", "sirf,prima2-pl310-cache";
+ compatible = "arm,pl310-cache";
reg = <0x80040000 0x1000>;
interrupts = <59>;
arm,tag-latency = <1 1 1>;
diff --git a/arch/arm/boot/dts/sun4i-a10.dtsi b/arch/arm/boot/dts/sun4i-a10.dtsi
index c7b794e..d96e179 100644
--- a/arch/arm/boot/dts/sun4i-a10.dtsi
+++ b/arch/arm/boot/dts/sun4i-a10.dtsi
@@ -713,7 +713,7 @@
};
i2c0: i2c@01c2ac00 {
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun4i-a10-i2c";
reg = <0x01c2ac00 0x400>;
interrupts = <7>;
clocks = <&apb1_gates 0>;
@@ -724,7 +724,7 @@
};
i2c1: i2c@01c2b000 {
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun4i-a10-i2c";
reg = <0x01c2b000 0x400>;
interrupts = <8>;
clocks = <&apb1_gates 1>;
@@ -735,7 +735,7 @@
};
i2c2: i2c@01c2b400 {
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun4i-a10-i2c";
reg = <0x01c2b400 0x400>;
interrupts = <9>;
clocks = <&apb1_gates 2>;
diff --git a/arch/arm/boot/dts/sun5i-a10s.dtsi b/arch/arm/boot/dts/sun5i-a10s.dtsi
index aa1dd59..b64f705 100644
--- a/arch/arm/boot/dts/sun5i-a10s.dtsi
+++ b/arch/arm/boot/dts/sun5i-a10s.dtsi
@@ -560,7 +560,7 @@
i2c0: i2c@01c2ac00 {
#address-cells = <1>;
#size-cells = <0>;
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun5i-a10s-i2c", "allwinner,sun4i-a10-i2c";
reg = <0x01c2ac00 0x400>;
interrupts = <7>;
clocks = <&apb1_gates 0>;
@@ -571,7 +571,7 @@
i2c1: i2c@01c2b000 {
#address-cells = <1>;
#size-cells = <0>;
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun5i-a10s-i2c", "allwinner,sun4i-a10-i2c";
reg = <0x01c2b000 0x400>;
interrupts = <8>;
clocks = <&apb1_gates 1>;
@@ -582,7 +582,7 @@
i2c2: i2c@01c2b400 {
#address-cells = <1>;
#size-cells = <0>;
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun5i-a10s-i2c", "allwinner,sun4i-a10-i2c";
reg = <0x01c2b400 0x400>;
interrupts = <9>;
clocks = <&apb1_gates 2>;
diff --git a/arch/arm/boot/dts/sun5i-a13.dtsi b/arch/arm/boot/dts/sun5i-a13.dtsi
index c9fdb7b..3b2a94c 100644
--- a/arch/arm/boot/dts/sun5i-a13.dtsi
+++ b/arch/arm/boot/dts/sun5i-a13.dtsi
@@ -486,7 +486,7 @@
};
i2c0: i2c@01c2ac00 {
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun5i-a13-i2c", "allwinner,sun4i-a10-i2c";
reg = <0x01c2ac00 0x400>;
interrupts = <7>;
clocks = <&apb1_gates 0>;
@@ -497,7 +497,7 @@
};
i2c1: i2c@01c2b000 {
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun5i-a13-i2c", "allwinner,sun4i-a10-i2c";
reg = <0x01c2b000 0x400>;
interrupts = <8>;
clocks = <&apb1_gates 1>;
@@ -508,7 +508,7 @@
};
i2c2: i2c@01c2b400 {
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun5i-a13-i2c", "allwinner,sun4i-a10-i2c";
reg = <0x01c2b400 0x400>;
interrupts = <9>;
clocks = <&apb1_gates 2>;
diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
index 385933b..01e9466 100644
--- a/arch/arm/boot/dts/sun7i-a20.dtsi
+++ b/arch/arm/boot/dts/sun7i-a20.dtsi
@@ -863,7 +863,7 @@
};
i2c0: i2c@01c2ac00 {
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun7i-a20-i2c", "allwinner,sun4i-a10-i2c";
reg = <0x01c2ac00 0x400>;
interrupts = <0 7 4>;
clocks = <&apb1_gates 0>;
@@ -874,7 +874,7 @@
};
i2c1: i2c@01c2b000 {
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun7i-a20-i2c", "allwinner,sun4i-a10-i2c";
reg = <0x01c2b000 0x400>;
interrupts = <0 8 4>;
clocks = <&apb1_gates 1>;
@@ -885,7 +885,7 @@
};
i2c2: i2c@01c2b400 {
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun7i-a20-i2c", "allwinner,sun4i-a10-i2c";
reg = <0x01c2b400 0x400>;
interrupts = <0 9 4>;
clocks = <&apb1_gates 2>;
@@ -896,7 +896,7 @@
};
i2c3: i2c@01c2b800 {
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun7i-a20-i2c", "allwinner,sun4i-a10-i2c";
reg = <0x01c2b800 0x400>;
interrupts = <0 88 4>;
clocks = <&apb1_gates 3>;
@@ -907,7 +907,7 @@
};
i2c4: i2c@01c2c000 {
- compatible = "allwinner,sun4i-i2c";
+ compatible = "allwinner,sun7i-a20-i2c", "allwinner,sun4i-a10-i2c";
reg = <0x01c2c000 0x400>;
interrupts = <0 89 4>;
clocks = <&apb1_gates 15>;
diff --git a/arch/arm/common/mcpm_entry.c b/arch/arm/common/mcpm_entry.c
index 86fd60f..f91136a 100644
--- a/arch/arm/common/mcpm_entry.c
+++ b/arch/arm/common/mcpm_entry.c
@@ -106,14 +106,14 @@
BUG();
}
-int mcpm_cpu_power_down_finish(unsigned int cpu, unsigned int cluster)
+int mcpm_wait_for_cpu_powerdown(unsigned int cpu, unsigned int cluster)
{
int ret;
- if (WARN_ON_ONCE(!platform_ops || !platform_ops->power_down_finish))
+ if (WARN_ON_ONCE(!platform_ops || !platform_ops->wait_for_powerdown))
return -EUNATCH;
- ret = platform_ops->power_down_finish(cpu, cluster);
+ ret = platform_ops->wait_for_powerdown(cpu, cluster);
if (ret)
pr_warn("%s: cpu %u, cluster %u failed to power down (%d)\n",
__func__, cpu, cluster, ret);
diff --git a/arch/arm/common/mcpm_platsmp.c b/arch/arm/common/mcpm_platsmp.c
index 177251a..92e54d7 100644
--- a/arch/arm/common/mcpm_platsmp.c
+++ b/arch/arm/common/mcpm_platsmp.c
@@ -62,7 +62,7 @@
cpu_to_pcpu(cpu, &pcpu, &pcluster);
- return !mcpm_cpu_power_down_finish(pcpu, pcluster);
+ return !mcpm_wait_for_cpu_powerdown(pcpu, pcluster);
}
static int mcpm_cpu_disable(unsigned int cpu)
diff --git a/arch/arm/include/asm/Kbuild b/arch/arm/include/asm/Kbuild
index 23e728e..f5a3576 100644
--- a/arch/arm/include/asm/Kbuild
+++ b/arch/arm/include/asm/Kbuild
@@ -21,6 +21,7 @@
generic-y += poll.h
generic-y += preempt.h
generic-y += resource.h
+generic-y += rwsem.h
generic-y += sections.h
generic-y += segment.h
generic-y += sembuf.h
diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index b974184..57f0584 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -312,7 +312,7 @@
* you cannot return to the original mode.
*/
.macro safe_svcmode_maskall reg:req
-#if __LINUX_ARM_ARCH__ >= 6
+#if __LINUX_ARM_ARCH__ >= 6 && !defined(CONFIG_CPU_V7M)
mrs \reg , cpsr
eor \reg, \reg, #HYP_MODE
tst \reg, #MODE_MASK
diff --git a/arch/arm/include/asm/cacheflush.h b/arch/arm/include/asm/cacheflush.h
index 8b8b616..fd43f7f 100644
--- a/arch/arm/include/asm/cacheflush.h
+++ b/arch/arm/include/asm/cacheflush.h
@@ -212,7 +212,7 @@
static inline void __flush_icache_all(void)
{
__flush_icache_preferred();
- dsb();
+ dsb(ishst);
}
/*
@@ -487,4 +487,6 @@
int set_memory_x(unsigned long addr, int numpages);
int set_memory_nx(unsigned long addr, int numpages);
+void flush_uprobe_xol_access(struct page *page, unsigned long uaddr,
+ void *kaddr, unsigned long len);
#endif
diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
index 6493802..c3f1152 100644
--- a/arch/arm/include/asm/cp15.h
+++ b/arch/arm/include/asm/cp15.h
@@ -42,24 +42,23 @@
#ifndef __ASSEMBLY__
#if __LINUX_ARM_ARCH__ >= 4
-#define vectors_high() (cr_alignment & CR_V)
+#define vectors_high() (get_cr() & CR_V)
#else
#define vectors_high() (0)
#endif
#ifdef CONFIG_CPU_CP15
-extern unsigned long cr_no_alignment; /* defined in entry-armv.S */
extern unsigned long cr_alignment; /* defined in entry-armv.S */
-static inline unsigned int get_cr(void)
+static inline unsigned long get_cr(void)
{
- unsigned int val;
+ unsigned long val;
asm("mrc p15, 0, %0, c1, c0, 0 @ get CR" : "=r" (val) : : "cc");
return val;
}
-static inline void set_cr(unsigned int val)
+static inline void set_cr(unsigned long val)
{
asm volatile("mcr p15, 0, %0, c1, c0, 0 @ set CR"
: : "r" (val) : "cc");
@@ -80,10 +79,6 @@
isb();
}
-#ifndef CONFIG_SMP
-extern void adjust_cr(unsigned long mask, unsigned long set);
-#endif
-
#define CPACC_FULL(n) (3 << (n * 2))
#define CPACC_SVC(n) (1 << (n * 2))
#define CPACC_DISABLE(n) (0 << (n * 2))
@@ -106,13 +101,17 @@
#else /* ifdef CONFIG_CPU_CP15 */
/*
- * cr_alignment and cr_no_alignment are tightly coupled to cp15 (at least in the
- * minds of the developers). Yielding 0 for machines without a cp15 (and making
- * it read-only) is fine for most cases and saves quite some #ifdeffery.
+ * cr_alignment is tightly coupled to cp15 (at least in the minds of the
+ * developers). Yielding 0 for machines without a cp15 (and making it
+ * read-only) is fine for most cases and saves quite some #ifdeffery.
*/
-#define cr_no_alignment UL(0)
#define cr_alignment UL(0)
+static inline unsigned long get_cr(void)
+{
+ return 0;
+}
+
#endif /* ifdef CONFIG_CPU_CP15 / else */
#endif /* ifndef __ASSEMBLY__ */
diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
index 4764344..8c2b732 100644
--- a/arch/arm/include/asm/cputype.h
+++ b/arch/arm/include/asm/cputype.h
@@ -72,6 +72,7 @@
#define ARM_CPU_PART_CORTEX_A15 0xC0F0
#define ARM_CPU_PART_CORTEX_A7 0xC070
#define ARM_CPU_PART_CORTEX_A12 0xC0D0
+#define ARM_CPU_PART_CORTEX_A17 0xC0E0
#define ARM_CPU_XSCALE_ARCH_MASK 0xe000
#define ARM_CPU_XSCALE_ARCH_V1 0x2000
diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index e701a4d..c45b61a 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -58,21 +58,37 @@
#ifndef __arch_pfn_to_dma
static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
{
+ if (dev)
+ pfn -= dev->dma_pfn_offset;
return (dma_addr_t)__pfn_to_bus(pfn);
}
static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
{
- return __bus_to_pfn(addr);
+ unsigned long pfn = __bus_to_pfn(addr);
+
+ if (dev)
+ pfn += dev->dma_pfn_offset;
+
+ return pfn;
}
static inline void *dma_to_virt(struct device *dev, dma_addr_t addr)
{
+ if (dev) {
+ unsigned long pfn = dma_to_pfn(dev, addr);
+
+ return phys_to_virt(__pfn_to_phys(pfn));
+ }
+
return (void *)__bus_to_virt((unsigned long)addr);
}
static inline dma_addr_t virt_to_dma(struct device *dev, void *addr)
{
+ if (dev)
+ return pfn_to_dma(dev, virt_to_pfn(addr));
+
return (dma_addr_t)__virt_to_bus((unsigned long)(addr));
}
@@ -105,6 +121,13 @@
}
#define dma_max_pfn(dev) dma_max_pfn(dev)
+static inline int set_arch_dma_coherent_ops(struct device *dev)
+{
+ set_dma_ops(dev, &arm_coherent_dma_ops);
+ return 0;
+}
+#define set_arch_dma_coherent_ops(dev) set_arch_dma_coherent_ops(dev)
+
static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
{
unsigned int offset = paddr & ~PAGE_MASK;
diff --git a/arch/arm/include/asm/fixmap.h b/arch/arm/include/asm/fixmap.h
index bbae919..74124b0 100644
--- a/arch/arm/include/asm/fixmap.h
+++ b/arch/arm/include/asm/fixmap.h
@@ -1,24 +1,11 @@
#ifndef _ASM_FIXMAP_H
#define _ASM_FIXMAP_H
-/*
- * Nothing too fancy for now.
- *
- * On ARM we already have well known fixed virtual addresses imposed by
- * the architecture such as the vector page which is located at 0xffff0000,
- * therefore a second level page table is already allocated covering
- * 0xfff00000 upwards.
- *
- * The cache flushing code in proc-xscale.S uses the virtual area between
- * 0xfffe0000 and 0xfffeffff.
- */
-
-#define FIXADDR_START 0xfff00000UL
-#define FIXADDR_TOP 0xfffe0000UL
+#define FIXADDR_START 0xffc00000UL
+#define FIXADDR_TOP 0xffe00000UL
#define FIXADDR_SIZE (FIXADDR_TOP - FIXADDR_START)
-#define FIX_KMAP_BEGIN 0
-#define FIX_KMAP_END (FIXADDR_SIZE >> PAGE_SHIFT)
+#define FIX_KMAP_NR_PTES (FIXADDR_SIZE >> PAGE_SHIFT)
#define __fix_to_virt(x) (FIXADDR_START + ((x) << PAGE_SHIFT))
#define __virt_to_fix(x) (((x) - FIXADDR_START) >> PAGE_SHIFT)
@@ -27,7 +14,7 @@
static inline unsigned long fix_to_virt(const unsigned int idx)
{
- if (idx >= FIX_KMAP_END)
+ if (idx >= FIX_KMAP_NR_PTES)
__this_fixmap_does_not_exist();
return __fix_to_virt(idx);
}
diff --git a/arch/arm/include/asm/ftrace.h b/arch/arm/include/asm/ftrace.h
index f89515a..eb577f4 100644
--- a/arch/arm/include/asm/ftrace.h
+++ b/arch/arm/include/asm/ftrace.h
@@ -52,15 +52,7 @@
#endif
-#define HAVE_ARCH_CALLER_ADDR
-
-#define CALLER_ADDR0 ((unsigned long)__builtin_return_address(0))
-#define CALLER_ADDR1 ((unsigned long)return_address(1))
-#define CALLER_ADDR2 ((unsigned long)return_address(2))
-#define CALLER_ADDR3 ((unsigned long)return_address(3))
-#define CALLER_ADDR4 ((unsigned long)return_address(4))
-#define CALLER_ADDR5 ((unsigned long)return_address(5))
-#define CALLER_ADDR6 ((unsigned long)return_address(6))
+#define ftrace_return_addr(n) return_address(n)
#endif /* ifndef __ASSEMBLY__ */
diff --git a/arch/arm/include/asm/glue-cache.h b/arch/arm/include/asm/glue-cache.h
index c81adc0..a3c24cd 100644
--- a/arch/arm/include/asm/glue-cache.h
+++ b/arch/arm/include/asm/glue-cache.h
@@ -130,22 +130,22 @@
#endif
#ifndef __ASSEMBLER__
-extern inline void nop_flush_icache_all(void) { }
-extern inline void nop_flush_kern_cache_all(void) { }
-extern inline void nop_flush_kern_cache_louis(void) { }
-extern inline void nop_flush_user_cache_all(void) { }
-extern inline void nop_flush_user_cache_range(unsigned long a,
+static inline void nop_flush_icache_all(void) { }
+static inline void nop_flush_kern_cache_all(void) { }
+static inline void nop_flush_kern_cache_louis(void) { }
+static inline void nop_flush_user_cache_all(void) { }
+static inline void nop_flush_user_cache_range(unsigned long a,
unsigned long b, unsigned int c) { }
-extern inline void nop_coherent_kern_range(unsigned long a, unsigned long b) { }
-extern inline int nop_coherent_user_range(unsigned long a,
+static inline void nop_coherent_kern_range(unsigned long a, unsigned long b) { }
+static inline int nop_coherent_user_range(unsigned long a,
unsigned long b) { return 0; }
-extern inline void nop_flush_kern_dcache_area(void *a, size_t s) { }
+static inline void nop_flush_kern_dcache_area(void *a, size_t s) { }
-extern inline void nop_dma_flush_range(const void *a, const void *b) { }
+static inline void nop_dma_flush_range(const void *a, const void *b) { }
-extern inline void nop_dma_map_area(const void *s, size_t l, int f) { }
-extern inline void nop_dma_unmap_area(const void *s, size_t l, int f) { }
+static inline void nop_dma_map_area(const void *s, size_t l, int f) { }
+static inline void nop_dma_unmap_area(const void *s, size_t l, int f) { }
#endif
#ifndef MULTI_CACHE
diff --git a/arch/arm/include/asm/glue-df.h b/arch/arm/include/asm/glue-df.h
index 6b70f1b..04e18b6 100644
--- a/arch/arm/include/asm/glue-df.h
+++ b/arch/arm/include/asm/glue-df.h
@@ -31,14 +31,6 @@
#undef CPU_DABORT_HANDLER
#undef MULTI_DABORT
-#if defined(CONFIG_CPU_ARM710)
-# ifdef CPU_DABORT_HANDLER
-# define MULTI_DABORT 1
-# else
-# define CPU_DABORT_HANDLER cpu_arm7_data_abort
-# endif
-#endif
-
#ifdef CONFIG_CPU_ABRT_EV4
# ifdef CPU_DABORT_HANDLER
# define MULTI_DABORT 1
diff --git a/arch/arm/include/asm/hardware/cache-l2x0.h b/arch/arm/include/asm/hardware/cache-l2x0.h
index 6795ff7..3a5ec1c 100644
--- a/arch/arm/include/asm/hardware/cache-l2x0.h
+++ b/arch/arm/include/asm/hardware/cache-l2x0.h
@@ -26,8 +26,8 @@
#define L2X0_CACHE_TYPE 0x004
#define L2X0_CTRL 0x100
#define L2X0_AUX_CTRL 0x104
-#define L2X0_TAG_LATENCY_CTRL 0x108
-#define L2X0_DATA_LATENCY_CTRL 0x10C
+#define L310_TAG_LATENCY_CTRL 0x108
+#define L310_DATA_LATENCY_CTRL 0x10C
#define L2X0_EVENT_CNT_CTRL 0x200
#define L2X0_EVENT_CNT1_CFG 0x204
#define L2X0_EVENT_CNT0_CFG 0x208
@@ -54,53 +54,93 @@
#define L2X0_LOCKDOWN_WAY_D_BASE 0x900
#define L2X0_LOCKDOWN_WAY_I_BASE 0x904
#define L2X0_LOCKDOWN_STRIDE 0x08
-#define L2X0_ADDR_FILTER_START 0xC00
-#define L2X0_ADDR_FILTER_END 0xC04
+#define L310_ADDR_FILTER_START 0xC00
+#define L310_ADDR_FILTER_END 0xC04
#define L2X0_TEST_OPERATION 0xF00
#define L2X0_LINE_DATA 0xF10
#define L2X0_LINE_TAG 0xF30
#define L2X0_DEBUG_CTRL 0xF40
-#define L2X0_PREFETCH_CTRL 0xF60
-#define L2X0_POWER_CTRL 0xF80
-#define L2X0_DYNAMIC_CLK_GATING_EN (1 << 1)
-#define L2X0_STNDBY_MODE_EN (1 << 0)
+#define L310_PREFETCH_CTRL 0xF60
+#define L310_POWER_CTRL 0xF80
+#define L310_DYNAMIC_CLK_GATING_EN (1 << 1)
+#define L310_STNDBY_MODE_EN (1 << 0)
/* Registers shifts and masks */
#define L2X0_CACHE_ID_PART_MASK (0xf << 6)
#define L2X0_CACHE_ID_PART_L210 (1 << 6)
+#define L2X0_CACHE_ID_PART_L220 (2 << 6)
#define L2X0_CACHE_ID_PART_L310 (3 << 6)
#define L2X0_CACHE_ID_RTL_MASK 0x3f
-#define L2X0_CACHE_ID_RTL_R0P0 0x0
-#define L2X0_CACHE_ID_RTL_R1P0 0x2
-#define L2X0_CACHE_ID_RTL_R2P0 0x4
-#define L2X0_CACHE_ID_RTL_R3P0 0x5
-#define L2X0_CACHE_ID_RTL_R3P1 0x6
-#define L2X0_CACHE_ID_RTL_R3P2 0x8
+#define L210_CACHE_ID_RTL_R0P2_02 0x00
+#define L210_CACHE_ID_RTL_R0P1 0x01
+#define L210_CACHE_ID_RTL_R0P2_01 0x02
+#define L210_CACHE_ID_RTL_R0P3 0x03
+#define L210_CACHE_ID_RTL_R0P4 0x0b
+#define L210_CACHE_ID_RTL_R0P5 0x0f
+#define L220_CACHE_ID_RTL_R1P7_01REL0 0x06
+#define L310_CACHE_ID_RTL_R0P0 0x00
+#define L310_CACHE_ID_RTL_R1P0 0x02
+#define L310_CACHE_ID_RTL_R2P0 0x04
+#define L310_CACHE_ID_RTL_R3P0 0x05
+#define L310_CACHE_ID_RTL_R3P1 0x06
+#define L310_CACHE_ID_RTL_R3P1_50REL0 0x07
+#define L310_CACHE_ID_RTL_R3P2 0x08
+#define L310_CACHE_ID_RTL_R3P3 0x09
-#define L2X0_AUX_CTRL_MASK 0xc0000fff
+/* L2C auxiliary control register - bits common to L2C-210/220/310 */
+#define L2C_AUX_CTRL_WAY_SIZE_SHIFT 17
+#define L2C_AUX_CTRL_WAY_SIZE_MASK (7 << 17)
+#define L2C_AUX_CTRL_WAY_SIZE(n) ((n) << 17)
+#define L2C_AUX_CTRL_EVTMON_ENABLE BIT(20)
+#define L2C_AUX_CTRL_PARITY_ENABLE BIT(21)
+#define L2C_AUX_CTRL_SHARED_OVERRIDE BIT(22)
+/* L2C-210/220 common bits */
#define L2X0_AUX_CTRL_DATA_RD_LATENCY_SHIFT 0
-#define L2X0_AUX_CTRL_DATA_RD_LATENCY_MASK 0x7
+#define L2X0_AUX_CTRL_DATA_RD_LATENCY_MASK (7 << 0)
#define L2X0_AUX_CTRL_DATA_WR_LATENCY_SHIFT 3
-#define L2X0_AUX_CTRL_DATA_WR_LATENCY_MASK (0x7 << 3)
+#define L2X0_AUX_CTRL_DATA_WR_LATENCY_MASK (7 << 3)
#define L2X0_AUX_CTRL_TAG_LATENCY_SHIFT 6
-#define L2X0_AUX_CTRL_TAG_LATENCY_MASK (0x7 << 6)
+#define L2X0_AUX_CTRL_TAG_LATENCY_MASK (7 << 6)
#define L2X0_AUX_CTRL_DIRTY_LATENCY_SHIFT 9
-#define L2X0_AUX_CTRL_DIRTY_LATENCY_MASK (0x7 << 9)
-#define L2X0_AUX_CTRL_ASSOCIATIVITY_SHIFT 16
-#define L2X0_AUX_CTRL_WAY_SIZE_SHIFT 17
-#define L2X0_AUX_CTRL_WAY_SIZE_MASK (0x7 << 17)
-#define L2X0_AUX_CTRL_SHARE_OVERRIDE_SHIFT 22
-#define L2X0_AUX_CTRL_NS_LOCKDOWN_SHIFT 26
-#define L2X0_AUX_CTRL_NS_INT_CTRL_SHIFT 27
-#define L2X0_AUX_CTRL_DATA_PREFETCH_SHIFT 28
-#define L2X0_AUX_CTRL_INSTR_PREFETCH_SHIFT 29
-#define L2X0_AUX_CTRL_EARLY_BRESP_SHIFT 30
+#define L2X0_AUX_CTRL_DIRTY_LATENCY_MASK (7 << 9)
+#define L2X0_AUX_CTRL_ASSOC_SHIFT 13
+#define L2X0_AUX_CTRL_ASSOC_MASK (15 << 13)
+/* L2C-210 specific bits */
+#define L210_AUX_CTRL_WRAP_DISABLE BIT(12)
+#define L210_AUX_CTRL_WA_OVERRIDE BIT(23)
+#define L210_AUX_CTRL_EXCLUSIVE_ABORT BIT(24)
+/* L2C-220 specific bits */
+#define L220_AUX_CTRL_EXCLUSIVE_CACHE BIT(12)
+#define L220_AUX_CTRL_FWA_SHIFT 23
+#define L220_AUX_CTRL_FWA_MASK (3 << 23)
+#define L220_AUX_CTRL_NS_LOCKDOWN BIT(26)
+#define L220_AUX_CTRL_NS_INT_CTRL BIT(27)
+/* L2C-310 specific bits */
+#define L310_AUX_CTRL_FULL_LINE_ZERO BIT(0) /* R2P0+ */
+#define L310_AUX_CTRL_HIGHPRIO_SO_DEV BIT(10) /* R2P0+ */
+#define L310_AUX_CTRL_STORE_LIMITATION BIT(11) /* R2P0+ */
+#define L310_AUX_CTRL_EXCLUSIVE_CACHE BIT(12)
+#define L310_AUX_CTRL_ASSOCIATIVITY_16 BIT(16)
+#define L310_AUX_CTRL_CACHE_REPLACE_RR BIT(25) /* R2P0+ */
+#define L310_AUX_CTRL_NS_LOCKDOWN BIT(26)
+#define L310_AUX_CTRL_NS_INT_CTRL BIT(27)
+#define L310_AUX_CTRL_DATA_PREFETCH BIT(28)
+#define L310_AUX_CTRL_INSTR_PREFETCH BIT(29)
+#define L310_AUX_CTRL_EARLY_BRESP BIT(30) /* R2P0+ */
-#define L2X0_LATENCY_CTRL_SETUP_SHIFT 0
-#define L2X0_LATENCY_CTRL_RD_SHIFT 4
-#define L2X0_LATENCY_CTRL_WR_SHIFT 8
+#define L310_LATENCY_CTRL_SETUP(n) ((n) << 0)
+#define L310_LATENCY_CTRL_RD(n) ((n) << 4)
+#define L310_LATENCY_CTRL_WR(n) ((n) << 8)
-#define L2X0_ADDR_FILTER_EN 1
+#define L310_ADDR_FILTER_EN 1
+
+#define L310_PREFETCH_CTRL_OFFSET_MASK 0x1f
+#define L310_PREFETCH_CTRL_DBL_LINEFILL_INCR BIT(23)
+#define L310_PREFETCH_CTRL_PREFETCH_DROP BIT(24)
+#define L310_PREFETCH_CTRL_DBL_LINEFILL_WRAP BIT(27)
+#define L310_PREFETCH_CTRL_DATA_PREFETCH BIT(28)
+#define L310_PREFETCH_CTRL_INSTR_PREFETCH BIT(29)
+#define L310_PREFETCH_CTRL_DBL_LINEFILL BIT(30)
#define L2X0_CTRL_EN 1
diff --git a/arch/arm/include/asm/highmem.h b/arch/arm/include/asm/highmem.h
index 91b99ab..5355795 100644
--- a/arch/arm/include/asm/highmem.h
+++ b/arch/arm/include/asm/highmem.h
@@ -18,6 +18,7 @@
} while (0)
extern pte_t *pkmap_page_table;
+extern pte_t *fixmap_page_table;
extern void *kmap_high(struct page *page);
extern void kunmap_high(struct page *page);
diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h
index 8aa4cca..3d23418 100644
--- a/arch/arm/include/asm/io.h
+++ b/arch/arm/include/asm/io.h
@@ -179,6 +179,12 @@
/* PCI fixed i/o mapping */
#define PCI_IO_VIRT_BASE 0xfee00000
+#if defined(CONFIG_PCI)
+void pci_ioremap_set_mem_type(int mem_type);
+#else
+static inline void pci_ioremap_set_mem_type(int mem_type) {}
+#endif
+
extern int pci_ioremap_io(unsigned int offset, phys_addr_t phys_addr);
/*
diff --git a/arch/arm/include/asm/mach/arch.h b/arch/arm/include/asm/mach/arch.h
index 17a3fa2..060a75e 100644
--- a/arch/arm/include/asm/mach/arch.h
+++ b/arch/arm/include/asm/mach/arch.h
@@ -14,7 +14,6 @@
#include <linux/reboot.h>
struct tag;
-struct meminfo;
struct pt_regs;
struct smp_operations;
#ifdef CONFIG_SMP
@@ -45,10 +44,12 @@
unsigned char reserve_lp1 :1; /* never has lp1 */
unsigned char reserve_lp2 :1; /* never has lp2 */
enum reboot_mode reboot_mode; /* default restart mode */
+ unsigned l2c_aux_val; /* L2 cache aux value */
+ unsigned l2c_aux_mask; /* L2 cache aux mask */
+ void (*l2c_write_sec)(unsigned long, unsigned);
struct smp_operations *smp; /* SMP operations */
bool (*smp_init)(void);
- void (*fixup)(struct tag *, char **,
- struct meminfo *);
+ void (*fixup)(struct tag *, char **);
void (*init_meminfo)(void);
void (*reserve)(void);/* reserve mem blocks */
void (*map_io)(void);/* IO mapping function */
diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
index a5ff410..d9702eb 100644
--- a/arch/arm/include/asm/mcpm.h
+++ b/arch/arm/include/asm/mcpm.h
@@ -98,14 +98,14 @@
* previously in which case the caller should take appropriate action.
*
* On success, the CPU is not guaranteed to be truly halted until
- * mcpm_cpu_power_down_finish() subsequently returns non-zero for the
+ * mcpm_wait_for_cpu_powerdown() subsequently returns non-zero for the
* specified cpu. Until then, other CPUs should make sure they do not
* trash memory the target CPU might be executing/accessing.
*/
void mcpm_cpu_power_down(void);
/**
- * mcpm_cpu_power_down_finish - wait for a specified CPU to halt, and
+ * mcpm_wait_for_cpu_powerdown - wait for a specified CPU to halt, and
* make sure it is powered off
*
* @cpu: CPU number within given cluster
@@ -127,7 +127,7 @@
* - zero if the CPU is in a safely parked state
* - nonzero otherwise (e.g., timeout)
*/
-int mcpm_cpu_power_down_finish(unsigned int cpu, unsigned int cluster);
+int mcpm_wait_for_cpu_powerdown(unsigned int cpu, unsigned int cluster);
/**
* mcpm_cpu_suspend - bring the calling CPU in a suspended state
@@ -171,7 +171,7 @@
struct mcpm_platform_ops {
int (*power_up)(unsigned int cpu, unsigned int cluster);
void (*power_down)(void);
- int (*power_down_finish)(unsigned int cpu, unsigned int cluster);
+ int (*wait_for_powerdown)(unsigned int cpu, unsigned int cluster);
void (*suspend)(u64);
void (*powered_up)(void);
};
diff --git a/arch/arm/include/asm/memblock.h b/arch/arm/include/asm/memblock.h
index c2f5102..bf47a6c 100644
--- a/arch/arm/include/asm/memblock.h
+++ b/arch/arm/include/asm/memblock.h
@@ -1,10 +1,9 @@
#ifndef _ASM_ARM_MEMBLOCK_H
#define _ASM_ARM_MEMBLOCK_H
-struct meminfo;
struct machine_desc;
-void arm_memblock_init(struct meminfo *, const struct machine_desc *);
+void arm_memblock_init(const struct machine_desc *);
phys_addr_t arm_memblock_steal(phys_addr_t size, phys_addr_t align);
#endif
diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
index 02fa255..2b75146 100644
--- a/arch/arm/include/asm/memory.h
+++ b/arch/arm/include/asm/memory.h
@@ -83,8 +83,6 @@
*/
#define IOREMAP_MAX_ORDER 24
-#define CONSISTENT_END (0xffe00000UL)
-
#else /* CONFIG_MMU */
/*
diff --git a/arch/arm/include/asm/outercache.h b/arch/arm/include/asm/outercache.h
index f94784f..891a56b 100644
--- a/arch/arm/include/asm/outercache.h
+++ b/arch/arm/include/asm/outercache.h
@@ -28,53 +28,84 @@
void (*clean_range)(unsigned long, unsigned long);
void (*flush_range)(unsigned long, unsigned long);
void (*flush_all)(void);
- void (*inv_all)(void);
void (*disable)(void);
#ifdef CONFIG_OUTER_CACHE_SYNC
void (*sync)(void);
#endif
- void (*set_debug)(unsigned long);
void (*resume)(void);
+
+ /* This is an ARM L2C thing */
+ void (*write_sec)(unsigned long, unsigned);
};
extern struct outer_cache_fns outer_cache;
#ifdef CONFIG_OUTER_CACHE
-
+/**
+ * outer_inv_range - invalidate range of outer cache lines
+ * @start: starting physical address, inclusive
+ * @end: end physical address, exclusive
+ */
static inline void outer_inv_range(phys_addr_t start, phys_addr_t end)
{
if (outer_cache.inv_range)
outer_cache.inv_range(start, end);
}
+
+/**
+ * outer_clean_range - clean dirty outer cache lines
+ * @start: starting physical address, inclusive
+ * @end: end physical address, exclusive
+ */
static inline void outer_clean_range(phys_addr_t start, phys_addr_t end)
{
if (outer_cache.clean_range)
outer_cache.clean_range(start, end);
}
+
+/**
+ * outer_flush_range - clean and invalidate outer cache lines
+ * @start: starting physical address, inclusive
+ * @end: end physical address, exclusive
+ */
static inline void outer_flush_range(phys_addr_t start, phys_addr_t end)
{
if (outer_cache.flush_range)
outer_cache.flush_range(start, end);
}
+/**
+ * outer_flush_all - clean and invalidate all cache lines in the outer cache
+ *
+ * Note: depending on implementation, this may not be atomic - it must
+ * only be called with interrupts disabled and no other active outer
+ * cache masters.
+ *
+ * It is intended that this function is only used by implementations
+ * needing to override the outer_cache.disable() method due to security.
+ * (Some implementations perform this as a clean followed by an invalidate.)
+ */
static inline void outer_flush_all(void)
{
if (outer_cache.flush_all)
outer_cache.flush_all();
}
-static inline void outer_inv_all(void)
-{
- if (outer_cache.inv_all)
- outer_cache.inv_all();
-}
+/**
+ * outer_disable - clean, invalidate and disable the outer cache
+ *
+ * Disable the outer cache, ensuring that any data contained in the outer
+ * cache is pushed out to lower levels of system memory. The note and
+ * conditions above concerning outer_flush_all() applies here.
+ */
+extern void outer_disable(void);
-static inline void outer_disable(void)
-{
- if (outer_cache.disable)
- outer_cache.disable();
-}
-
+/**
+ * outer_resume - restore the cache configuration and re-enable outer cache
+ *
+ * Restore any configuration that the cache had when previously enabled,
+ * and re-enable the outer cache.
+ */
static inline void outer_resume(void)
{
if (outer_cache.resume)
@@ -90,13 +121,18 @@
static inline void outer_flush_range(phys_addr_t start, phys_addr_t end)
{ }
static inline void outer_flush_all(void) { }
-static inline void outer_inv_all(void) { }
static inline void outer_disable(void) { }
static inline void outer_resume(void) { }
#endif
#ifdef CONFIG_OUTER_CACHE_SYNC
+/**
+ * outer_sync - perform a sync point for outer cache
+ *
+ * Ensure that all outer cache operations are complete and any store
+ * buffers are drained.
+ */
static inline void outer_sync(void)
{
if (outer_cache.sync)
diff --git a/arch/arm/include/asm/setup.h b/arch/arm/include/asm/setup.h
index 8d6a089..e0adb9f 100644
--- a/arch/arm/include/asm/setup.h
+++ b/arch/arm/include/asm/setup.h
@@ -21,34 +21,6 @@
#define __tagtable(tag, fn) \
static const struct tagtable __tagtable_##fn __tag = { tag, fn }
-/*
- * Memory map description
- */
-#define NR_BANKS CONFIG_ARM_NR_BANKS
-
-struct membank {
- phys_addr_t start;
- phys_addr_t size;
- unsigned int highmem;
-};
-
-struct meminfo {
- int nr_banks;
- struct membank bank[NR_BANKS];
-};
-
-extern struct meminfo meminfo;
-
-#define for_each_bank(iter,mi) \
- for (iter = 0; iter < (mi)->nr_banks; iter++)
-
-#define bank_pfn_start(bank) __phys_to_pfn((bank)->start)
-#define bank_pfn_end(bank) __phys_to_pfn((bank)->start + (bank)->size)
-#define bank_pfn_size(bank) ((bank)->size >> PAGE_SHIFT)
-#define bank_phys_start(bank) (bank)->start
-#define bank_phys_end(bank) ((bank)->start + (bank)->size)
-#define bank_phys_size(bank) (bank)->size
-
extern int arm_add_memory(u64 start, u64 size);
extern void early_print(const char *str, ...);
extern void dump_machine_table(void);
diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile
index 040619c..38ddd9f 100644
--- a/arch/arm/kernel/Makefile
+++ b/arch/arm/kernel/Makefile
@@ -39,6 +39,7 @@
obj-$(CONFIG_ISA_DMA) += dma-isa.o
obj-$(CONFIG_PCI) += bios32.o isa.o
obj-$(CONFIG_ARM_CPU_SUSPEND) += sleep.o suspend.o
+obj-$(CONFIG_HIBERNATION) += hibernate.o
obj-$(CONFIG_SMP) += smp.o
ifdef CONFIG_MMU
obj-$(CONFIG_SMP) += smp_tlb.o
diff --git a/arch/arm/kernel/atags_parse.c b/arch/arm/kernel/atags_parse.c
index 8c14de8..7807ef5 100644
--- a/arch/arm/kernel/atags_parse.c
+++ b/arch/arm/kernel/atags_parse.c
@@ -22,6 +22,7 @@
#include <linux/fs.h>
#include <linux/root_dev.h>
#include <linux/screen_info.h>
+#include <linux/memblock.h>
#include <asm/setup.h>
#include <asm/system_info.h>
@@ -222,10 +223,10 @@
}
if (mdesc->fixup)
- mdesc->fixup(tags, &from, &meminfo);
+ mdesc->fixup(tags, &from);
if (tags->hdr.tag == ATAG_CORE) {
- if (meminfo.nr_banks != 0)
+ if (memblock_phys_mem_size())
squash_mem_tags(tags);
save_atags(tags);
parse_tags(tags);
diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
index ea9ce92..e94a157 100644
--- a/arch/arm/kernel/devtree.c
+++ b/arch/arm/kernel/devtree.c
@@ -27,10 +27,6 @@
#include <asm/mach/arch.h>
#include <asm/mach-types.h>
-void __init early_init_dt_add_memory_arch(u64 base, u64 size)
-{
- arm_add_memory(base, size);
-}
#ifdef CONFIG_SMP
extern struct of_cpu_method __cpu_method_of_table[];
diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S
index 1879e8d..52a949a 100644
--- a/arch/arm/kernel/entry-armv.S
+++ b/arch/arm/kernel/entry-armv.S
@@ -344,7 +344,7 @@
@
@ Enable the alignment trap while in kernel mode
@
- alignment_trap r0
+ alignment_trap r0, .LCcralign
@
@ Clear FP to mark the first stack frame
@@ -413,6 +413,11 @@
@
adr r9, BSYM(ret_from_exception)
+ @ IRQs must be enabled before attempting to read the instruction from
+ @ user space since that could cause a page/translation fault if the
+ @ page table was modified by another CPU.
+ enable_irq
+
tst r3, #PSR_T_BIT @ Thumb mode?
bne __und_usr_thumb
sub r4, r2, #4 @ ARM instr at LR - 4
@@ -484,7 +489,8 @@
*/
.pushsection .fixup, "ax"
.align 2
-4: mov pc, r9
+4: str r4, [sp, #S_PC] @ retry current instruction
+ mov pc, r9
.popsection
.pushsection __ex_table,"a"
.long 1b, 4b
@@ -517,7 +523,7 @@
* r9 = normal "successful" return address
* r10 = this threads thread_info structure
* lr = unrecognised instruction return address
- * IRQs disabled, FIQs enabled.
+ * IRQs enabled, FIQs enabled.
*/
@
@ Fall-through from Thumb-2 __und_usr
@@ -624,7 +630,6 @@
#endif
do_fpe:
- enable_irq
ldr r4, .LCfp
add r10, r10, #TI_FPSTATE @ r10 = workspace
ldr pc, [r4] @ Call FP module USR entry point
@@ -652,8 +657,7 @@
b 1f
__und_usr_fault_16:
mov r1, #2
-1: enable_irq
- mov r0, sp
+1: mov r0, sp
adr lr, BSYM(ret_from_exception)
b __und_fault
ENDPROC(__und_usr_fault_32)
@@ -1143,11 +1147,8 @@
.data
.globl cr_alignment
- .globl cr_no_alignment
cr_alignment:
.space 4
-cr_no_alignment:
- .space 4
#ifdef CONFIG_MULTI_IRQ_HANDLER
.globl handle_arch_irq
diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S
index a2dcafd..7139d4a 100644
--- a/arch/arm/kernel/entry-common.S
+++ b/arch/arm/kernel/entry-common.S
@@ -365,13 +365,7 @@
str r0, [sp, #S_OLD_R0] @ Save OLD_R0
#endif
zero_fp
-
-#ifdef CONFIG_ALIGNMENT_TRAP
- ldr ip, __cr_alignment
- ldr ip, [ip]
- mcr p15, 0, ip, c1, c0 @ update control register
-#endif
-
+ alignment_trap ip, __cr_alignment
enable_irq
ct_user_exit
get_thread_info tsk
diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S
index efb208d..5d702f89 100644
--- a/arch/arm/kernel/entry-header.S
+++ b/arch/arm/kernel/entry-header.S
@@ -37,9 +37,9 @@
#endif
.endm
- .macro alignment_trap, rtemp
+ .macro alignment_trap, rtemp, label
#ifdef CONFIG_ALIGNMENT_TRAP
- ldr \rtemp, .LCcralign
+ ldr \rtemp, \label
ldr \rtemp, [\rtemp]
mcr p15, 0, \rtemp, c1, c0
#endif
diff --git a/arch/arm/kernel/ftrace.c b/arch/arm/kernel/ftrace.c
index c108ddc..af9a8a9 100644
--- a/arch/arm/kernel/ftrace.c
+++ b/arch/arm/kernel/ftrace.c
@@ -14,6 +14,7 @@
#include <linux/ftrace.h>
#include <linux/uaccess.h>
+#include <linux/module.h>
#include <asm/cacheflush.h>
#include <asm/opcodes.h>
@@ -63,6 +64,18 @@
}
#endif
+int ftrace_arch_code_modify_prepare(void)
+{
+ set_all_modules_text_rw();
+ return 0;
+}
+
+int ftrace_arch_code_modify_post_process(void)
+{
+ set_all_modules_text_ro();
+ return 0;
+}
+
static unsigned long ftrace_call_replace(unsigned long pc, unsigned long addr)
{
return arm_gen_branch_link(pc, addr);
diff --git a/arch/arm/kernel/head-common.S b/arch/arm/kernel/head-common.S
index c96ecac..572a383 100644
--- a/arch/arm/kernel/head-common.S
+++ b/arch/arm/kernel/head-common.S
@@ -99,8 +99,7 @@
str r1, [r5] @ Save machine type
str r2, [r6] @ Save atags pointer
cmp r7, #0
- bicne r4, r0, #CR_A @ Clear 'A' bit
- stmneia r7, {r0, r4} @ Save control register values
+ strne r0, [r7] @ Save control register values
b start_kernel
ENDPROC(__mmap_switched)
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 591d6e4..2c35f0f 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -475,7 +475,7 @@
#ifdef CONFIG_SMP_ON_UP
- __INIT
+ __HEAD
__fixup_smp:
and r3, r9, #0x000f0000 @ architecture version
teq r3, #0x000f0000 @ CPU ID supported?
diff --git a/arch/arm/kernel/hibernate.c b/arch/arm/kernel/hibernate.c
new file mode 100644
index 0000000..bb8b796
--- /dev/null
+++ b/arch/arm/kernel/hibernate.c
@@ -0,0 +1,107 @@
+/*
+ * Hibernation support specific for ARM
+ *
+ * Derived from work on ARM hibernation support by:
+ *
+ * Ubuntu project, hibernation support for mach-dove
+ * Copyright (C) 2010 Nokia Corporation (Hiroshi Doyu)
+ * Copyright (C) 2010 Texas Instruments, Inc. (Teerth Reddy et al.)
+ * https://lkml.org/lkml/2010/6/18/4
+ * https://lists.linux-foundation.org/pipermail/linux-pm/2010-June/027422.html
+ * https://patchwork.kernel.org/patch/96442/
+ *
+ * Copyright (C) 2006 Rafael J. Wysocki <rjw@sisk.pl>
+ *
+ * License terms: GNU General Public License (GPL) version 2
+ */
+
+#include <linux/mm.h>
+#include <linux/suspend.h>
+#include <asm/system_misc.h>
+#include <asm/idmap.h>
+#include <asm/suspend.h>
+#include <asm/memory.h>
+
+extern const void __nosave_begin, __nosave_end;
+
+int pfn_is_nosave(unsigned long pfn)
+{
+ unsigned long nosave_begin_pfn = virt_to_pfn(&__nosave_begin);
+ unsigned long nosave_end_pfn = virt_to_pfn(&__nosave_end - 1);
+
+ return (pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn);
+}
+
+void notrace save_processor_state(void)
+{
+ WARN_ON(num_online_cpus() != 1);
+ local_fiq_disable();
+}
+
+void notrace restore_processor_state(void)
+{
+ local_fiq_enable();
+}
+
+/*
+ * Snapshot kernel memory and reset the system.
+ *
+ * swsusp_save() is executed in the suspend finisher so that the CPU
+ * context pointer and memory are part of the saved image, which is
+ * required by the resume kernel image to restart execution from
+ * swsusp_arch_suspend().
+ *
+ * soft_restart is not technically needed, but is used to get success
+ * returned from cpu_suspend.
+ *
+ * When soft reboot completes, the hibernation snapshot is written out.
+ */
+static int notrace arch_save_image(unsigned long unused)
+{
+ int ret;
+
+ ret = swsusp_save();
+ if (ret == 0)
+ soft_restart(virt_to_phys(cpu_resume));
+ return ret;
+}
+
+/*
+ * Save the current CPU state before suspend / poweroff.
+ */
+int notrace swsusp_arch_suspend(void)
+{
+ return cpu_suspend(0, arch_save_image);
+}
+
+/*
+ * Restore page contents for physical pages that were in use during loading
+ * hibernation image. Switch to idmap_pgd so the physical page tables
+ * are overwritten with the same contents.
+ */
+static void notrace arch_restore_image(void *unused)
+{
+ struct pbe *pbe;
+
+ cpu_switch_mm(idmap_pgd, &init_mm);
+ for (pbe = restore_pblist; pbe; pbe = pbe->next)
+ copy_page(pbe->orig_address, pbe->address);
+
+ soft_restart(virt_to_phys(cpu_resume));
+}
+
+static u64 resume_stack[PAGE_SIZE/2/sizeof(u64)] __nosavedata;
+
+/*
+ * Resume from the hibernation image.
+ * Due to the kernel heap / data restore, stack contents change underneath
+ * and that would make function calls impossible; switch to a temporary
+ * stack within the nosave region to avoid that problem.
+ */
+int swsusp_arch_resume(void)
+{
+ extern void call_with_stack(void (*fn)(void *), void *arg, void *sp);
+ call_with_stack(arch_restore_image, 0,
+ resume_stack + ARRAY_SIZE(resume_stack));
+ return 0;
+}
diff --git a/arch/arm/kernel/irq.c b/arch/arm/kernel/irq.c
index 9723d17..2c42576 100644
--- a/arch/arm/kernel/irq.c
+++ b/arch/arm/kernel/irq.c
@@ -37,6 +37,7 @@
#include <linux/proc_fs.h>
#include <linux/export.h>
+#include <asm/hardware/cache-l2x0.h>
#include <asm/exception.h>
#include <asm/mach/arch.h>
#include <asm/mach/irq.h>
@@ -115,10 +116,21 @@
void __init init_IRQ(void)
{
+ int ret;
+
if (IS_ENABLED(CONFIG_OF) && !machine_desc->init_irq)
irqchip_init();
else
machine_desc->init_irq();
+
+ if (IS_ENABLED(CONFIG_OF) && IS_ENABLED(CONFIG_CACHE_L2X0) &&
+ (machine_desc->l2c_aux_mask || machine_desc->l2c_aux_val)) {
+ outer_cache.write_sec = machine_desc->l2c_write_sec;
+ ret = l2x0_of_init(machine_desc->l2c_aux_val,
+ machine_desc->l2c_aux_mask);
+ if (ret)
+ pr_err("L2C: failed to init: %d\n", ret);
+ }
}
#ifdef CONFIG_MULTI_IRQ_HANDLER
diff --git a/arch/arm/kernel/isa.c b/arch/arm/kernel/isa.c
index 3464859..9d1cf71 100644
--- a/arch/arm/kernel/isa.c
+++ b/arch/arm/kernel/isa.c
@@ -20,7 +20,7 @@
static unsigned int isa_membase, isa_portbase, isa_portshift;
-static ctl_table ctl_isa_vars[4] = {
+static struct ctl_table ctl_isa_vars[4] = {
{
.procname = "membase",
.data = &isa_membase,
@@ -44,7 +44,7 @@
static struct ctl_table_header *isa_sysctl_header;
-static ctl_table ctl_isa[2] = {
+static struct ctl_table ctl_isa[2] = {
{
.procname = "isa",
.mode = 0555,
@@ -52,7 +52,7 @@
}, {}
};
-static ctl_table ctl_bus[2] = {
+static struct ctl_table ctl_bus[2] = {
{
.procname = "bus",
.mode = 0555,
diff --git a/arch/arm/kernel/iwmmxt.S b/arch/arm/kernel/iwmmxt.S
index 2452dd1..a5599cf 100644
--- a/arch/arm/kernel/iwmmxt.S
+++ b/arch/arm/kernel/iwmmxt.S
@@ -18,6 +18,7 @@
#include <asm/ptrace.h>
#include <asm/thread_info.h>
#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
#if defined(CONFIG_CPU_PJ4) || defined(CONFIG_CPU_PJ4B)
#define PJ4(code...) code
@@ -65,17 +66,18 @@
* r9 = ret_from_exception
* lr = undefined instr exit
*
- * called from prefetch exception handler with interrupts disabled
+ * called from prefetch exception handler with interrupts enabled
*/
ENTRY(iwmmxt_task_enable)
+ inc_preempt_count r10, r3
XSC(mrc p15, 0, r2, c15, c1, 0)
PJ4(mrc p15, 0, r2, c1, c0, 2)
@ CP0 and CP1 accessible?
XSC(tst r2, #0x3)
PJ4(tst r2, #0xf)
- movne pc, lr @ if so no business here
+ bne 4f @ if so no business here
@ enable access to CP0 and CP1
XSC(orr r2, r2, #0x3)
XSC(mcr p15, 0, r2, c15, c1, 0)
@@ -136,7 +138,7 @@
wstrd wR15, [r1, #MMX_WR15]
2: teq r0, #0 @ anything to load?
- moveq pc, lr
+ beq 3f
concan_load:
@@ -169,8 +171,14 @@
@ clear CUP/MUP (only if r1 != 0)
teq r1, #0
mov r2, #0
- moveq pc, lr
+ beq 3f
tmcr wCon, r2
+
+3:
+#ifdef CONFIG_PREEMPT_COUNT
+ get_thread_info r10
+#endif
+4: dec_preempt_count r10, r3
mov pc, lr
/*
diff --git a/arch/arm/kernel/perf_event_cpu.c b/arch/arm/kernel/perf_event_cpu.c
index 51798d7..a71ae15 100644
--- a/arch/arm/kernel/perf_event_cpu.c
+++ b/arch/arm/kernel/perf_event_cpu.c
@@ -221,6 +221,7 @@
* PMU platform driver and devicetree bindings.
*/
static struct of_device_id cpu_pmu_of_device_ids[] = {
+ {.compatible = "arm,cortex-a17-pmu", .data = armv7_a17_pmu_init},
{.compatible = "arm,cortex-a15-pmu", .data = armv7_a15_pmu_init},
{.compatible = "arm,cortex-a12-pmu", .data = armv7_a12_pmu_init},
{.compatible = "arm,cortex-a9-pmu", .data = armv7_a9_pmu_init},
diff --git a/arch/arm/kernel/perf_event_v7.c b/arch/arm/kernel/perf_event_v7.c
index f4ef398..2037f72 100644
--- a/arch/arm/kernel/perf_event_v7.c
+++ b/arch/arm/kernel/perf_event_v7.c
@@ -1599,6 +1599,13 @@
return 0;
}
+static int armv7_a17_pmu_init(struct arm_pmu *cpu_pmu)
+{
+ armv7_a12_pmu_init(cpu_pmu);
+ cpu_pmu->name = "ARMv7 Cortex-A17";
+ return 0;
+}
+
/*
* Krait Performance Monitor Region Event Selection Register (PMRESRn)
*
@@ -2021,6 +2028,11 @@
return -ENODEV;
}
+static inline int armv7_a17_pmu_init(struct arm_pmu *cpu_pmu)
+{
+ return -ENODEV;
+}
+
static inline int krait_pmu_init(struct arm_pmu *cpu_pmu)
{
return -ENODEV;
diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 50e198c..8a16ee5 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -72,6 +72,7 @@
__setup("fpe=", fpe_setup);
#endif
+extern void init_default_cache_policy(unsigned long);
extern void paging_init(const struct machine_desc *desc);
extern void early_paging_init(const struct machine_desc *,
struct proc_info_list *);
@@ -590,7 +591,7 @@
pr_info("CPU: %s [%08x] revision %d (ARMv%s), cr=%08lx\n",
cpu_name, read_cpuid_id(), read_cpuid_id() & 15,
- proc_arch[cpu_architecture()], cr_alignment);
+ proc_arch[cpu_architecture()], get_cr());
snprintf(init_utsname()->machine, __NEW_UTS_LEN + 1, "%s%c",
list->arch_name, ENDIANNESS);
@@ -603,7 +604,9 @@
#ifndef CONFIG_ARM_THUMB
elf_hwcap &= ~(HWCAP_THUMB | HWCAP_IDIVT);
#endif
-
+#ifdef CONFIG_MMU
+ init_default_cache_policy(list->__cpu_mm_mmu_flags);
+#endif
erratum_a15_798181_init();
feat_v6_fixup();
@@ -628,15 +631,8 @@
int __init arm_add_memory(u64 start, u64 size)
{
- struct membank *bank = &meminfo.bank[meminfo.nr_banks];
u64 aligned_start;
- if (meminfo.nr_banks >= NR_BANKS) {
- pr_crit("NR_BANKS too low, ignoring memory at 0x%08llx\n",
- (long long)start);
- return -EINVAL;
- }
-
/*
* Ensure that start/size are aligned to a page boundary.
* Size is appropriately rounded down, start is rounded up.
@@ -677,17 +673,17 @@
aligned_start = PHYS_OFFSET;
}
- bank->start = aligned_start;
- bank->size = size & ~(phys_addr_t)(PAGE_SIZE - 1);
+ start = aligned_start;
+ size = size & ~(phys_addr_t)(PAGE_SIZE - 1);
/*
* Check whether this memory region has non-zero size or
* invalid node number.
*/
- if (bank->size == 0)
+ if (size == 0)
return -EINVAL;
- meminfo.nr_banks++;
+ memblock_add(start, size);
return 0;
}
@@ -695,6 +691,7 @@
* Pick out the memory size. We look for mem=size@start,
* where start and size are "size[KkMm]"
*/
+
static int __init early_mem(char *p)
{
static int usermem __initdata = 0;
@@ -709,7 +706,8 @@
*/
if (usermem == 0) {
usermem = 1;
- meminfo.nr_banks = 0;
+ memblock_remove(memblock_start_of_DRAM(),
+ memblock_end_of_DRAM() - memblock_start_of_DRAM());
}
start = PHYS_OFFSET;
@@ -854,13 +852,6 @@
static inline void reserve_crashkernel(void) {}
#endif /* CONFIG_KEXEC */
-static int __init meminfo_cmp(const void *_a, const void *_b)
-{
- const struct membank *a = _a, *b = _b;
- long cmp = bank_pfn_start(a) - bank_pfn_start(b);
- return cmp < 0 ? -1 : cmp > 0 ? 1 : 0;
-}
-
void __init hyp_mode_check(void)
{
#ifdef CONFIG_ARM_VIRT_EXT
@@ -903,12 +894,10 @@
parse_early_param();
- sort(&meminfo.bank, meminfo.nr_banks, sizeof(meminfo.bank[0]), meminfo_cmp, NULL);
-
early_paging_init(mdesc, lookup_processor_type(read_cpuid_id()));
setup_dma_zone(mdesc);
sanity_check_meminfo();
- arm_memblock_init(&meminfo, mdesc);
+ arm_memblock_init(mdesc);
paging_init(mdesc);
request_standard_resources(mdesc);
diff --git a/arch/arm/kernel/sleep.S b/arch/arm/kernel/sleep.S
index b907d9b..1b880db 100644
--- a/arch/arm/kernel/sleep.S
+++ b/arch/arm/kernel/sleep.S
@@ -127,6 +127,10 @@
.align
ENTRY(cpu_resume)
ARM_BE8(setend be) @ ensure we are in BE mode
+#ifdef CONFIG_ARM_VIRT_EXT
+ bl __hyp_stub_install_secondary
+#endif
+ safe_svcmode_maskall r1
mov r1, #0
ALT_SMP(mrc p15, 0, r0, c0, c0, 5)
ALT_UP_B(1f)
@@ -144,7 +148,6 @@
ldr r0, [r0, #SLEEP_SAVE_SP_PHYS]
ldr r0, [r0, r1, lsl #2]
- setmode PSR_I_BIT | PSR_F_BIT | SVC_MODE, r1 @ set SVC, irqs off
@ load phys pgd, stack, resume fn
ARM( ldmia r0!, {r1, sp, pc} )
THUMB( ldmia r0!, {r1, r2, r3} )
diff --git a/arch/arm/kernel/stacktrace.c b/arch/arm/kernel/stacktrace.c
index af4e8c8..f065eb05 100644
--- a/arch/arm/kernel/stacktrace.c
+++ b/arch/arm/kernel/stacktrace.c
@@ -3,6 +3,7 @@
#include <linux/stacktrace.h>
#include <asm/stacktrace.h>
+#include <asm/traps.h>
#if defined(CONFIG_FRAME_POINTER) && !defined(CONFIG_ARM_UNWIND)
/*
@@ -61,6 +62,7 @@
#ifdef CONFIG_STACKTRACE
struct stack_trace_data {
struct stack_trace *trace;
+ unsigned long last_pc;
unsigned int no_sched_functions;
unsigned int skip;
};
@@ -69,6 +71,7 @@
{
struct stack_trace_data *data = d;
struct stack_trace *trace = data->trace;
+ struct pt_regs *regs;
unsigned long addr = frame->pc;
if (data->no_sched_functions && in_sched_functions(addr))
@@ -80,16 +83,39 @@
trace->entries[trace->nr_entries++] = addr;
+ if (trace->nr_entries >= trace->max_entries)
+ return 1;
+
+ /*
+ * in_exception_text() is designed to test if the PC is one of
+ * the functions which has an exception stack above it, but
+ * unfortunately what is in frame->pc is the return LR value,
+ * not the saved PC value. So, we need to track the previous
+ * frame PC value when doing this.
+ */
+ addr = data->last_pc;
+ data->last_pc = frame->pc;
+ if (!in_exception_text(addr))
+ return 0;
+
+ regs = (struct pt_regs *)frame->sp;
+
+ trace->entries[trace->nr_entries++] = regs->ARM_pc;
+
return trace->nr_entries >= trace->max_entries;
}
-void save_stack_trace_tsk(struct task_struct *tsk, struct stack_trace *trace)
+/* This must be noinline to so that our skip calculation works correctly */
+static noinline void __save_stack_trace(struct task_struct *tsk,
+ struct stack_trace *trace, unsigned int nosched)
{
struct stack_trace_data data;
struct stackframe frame;
data.trace = trace;
+ data.last_pc = ULONG_MAX;
data.skip = trace->skip;
+ data.no_sched_functions = nosched;
if (tsk != current) {
#ifdef CONFIG_SMP
@@ -102,7 +128,6 @@
trace->entries[trace->nr_entries++] = ULONG_MAX;
return;
#else
- data.no_sched_functions = 1;
frame.fp = thread_saved_fp(tsk);
frame.sp = thread_saved_sp(tsk);
frame.lr = 0; /* recovered from the stack */
@@ -111,11 +136,12 @@
} else {
register unsigned long current_sp asm ("sp");
- data.no_sched_functions = 0;
+ /* We don't want this function nor the caller */
+ data.skip += 2;
frame.fp = (unsigned long)__builtin_frame_address(0);
frame.sp = current_sp;
frame.lr = (unsigned long)__builtin_return_address(0);
- frame.pc = (unsigned long)save_stack_trace_tsk;
+ frame.pc = (unsigned long)__save_stack_trace;
}
walk_stackframe(&frame, save_trace, &data);
@@ -123,9 +149,33 @@
trace->entries[trace->nr_entries++] = ULONG_MAX;
}
+void save_stack_trace_regs(struct pt_regs *regs, struct stack_trace *trace)
+{
+ struct stack_trace_data data;
+ struct stackframe frame;
+
+ data.trace = trace;
+ data.skip = trace->skip;
+ data.no_sched_functions = 0;
+
+ frame.fp = regs->ARM_fp;
+ frame.sp = regs->ARM_sp;
+ frame.lr = regs->ARM_lr;
+ frame.pc = regs->ARM_pc;
+
+ walk_stackframe(&frame, save_trace, &data);
+ if (trace->nr_entries < trace->max_entries)
+ trace->entries[trace->nr_entries++] = ULONG_MAX;
+}
+
+void save_stack_trace_tsk(struct task_struct *tsk, struct stack_trace *trace)
+{
+ __save_stack_trace(tsk, trace, 1);
+}
+
void save_stack_trace(struct stack_trace *trace)
{
- save_stack_trace_tsk(current, trace);
+ __save_stack_trace(current, trace, 0);
}
EXPORT_SYMBOL_GPL(save_stack_trace);
#endif
diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
index 71e1fec..3997c41 100644
--- a/arch/arm/kernel/topology.c
+++ b/arch/arm/kernel/topology.c
@@ -91,13 +91,13 @@
{
const struct cpu_efficiency *cpu_eff;
struct device_node *cn = NULL;
- unsigned long min_capacity = (unsigned long)(-1);
+ unsigned long min_capacity = ULONG_MAX;
unsigned long max_capacity = 0;
unsigned long capacity = 0;
- int alloc_size, cpu = 0;
+ int cpu = 0;
- alloc_size = nr_cpu_ids * sizeof(*__cpu_capacity);
- __cpu_capacity = kzalloc(alloc_size, GFP_NOWAIT);
+ __cpu_capacity = kcalloc(nr_cpu_ids, sizeof(*__cpu_capacity),
+ GFP_NOWAIT);
for_each_possible_cpu(cpu) {
const u32 *rate;
diff --git a/arch/arm/kernel/unwind.c b/arch/arm/kernel/unwind.c
index cb791ac..e67682f 100644
--- a/arch/arm/kernel/unwind.c
+++ b/arch/arm/kernel/unwind.c
@@ -31,7 +31,7 @@
#warning Your compiler does not have EABI support.
#warning ARM unwind is known to compile only with EABI compilers.
#warning Change compiler or disable ARM_UNWIND option.
-#elif (__GNUC__ == 4 && __GNUC_MINOR__ <= 2)
+#elif (__GNUC__ == 4 && __GNUC_MINOR__ <= 2) && !defined(__clang__)
#warning Your compiler is too buggy; it is known to not compile ARM unwind support.
#warning Change compiler or disable ARM_UNWIND option.
#endif
diff --git a/arch/arm/kernel/uprobes.c b/arch/arm/kernel/uprobes.c
index f9bacee..56adf9c1 100644
--- a/arch/arm/kernel/uprobes.c
+++ b/arch/arm/kernel/uprobes.c
@@ -113,6 +113,26 @@
return 0;
}
+void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
+ void *src, unsigned long len)
+{
+ void *xol_page_kaddr = kmap_atomic(page);
+ void *dst = xol_page_kaddr + (vaddr & ~PAGE_MASK);
+
+ preempt_disable();
+
+ /* Initialize the slot */
+ memcpy(dst, src, len);
+
+ /* flush caches (dcache/icache) */
+ flush_uprobe_xol_access(page, vaddr, dst, len);
+
+ preempt_enable();
+
+ kunmap_atomic(xol_page_kaddr);
+}
+
+
int arch_uprobe_pre_xol(struct arch_uprobe *auprobe, struct pt_regs *regs)
{
struct uprobe_task *utask = current->utask;
diff --git a/arch/arm/mach-at91/sysirq_mask.c b/arch/arm/mach-at91/sysirq_mask.c
index 2ba694f..f8bc351 100644
--- a/arch/arm/mach-at91/sysirq_mask.c
+++ b/arch/arm/mach-at91/sysirq_mask.c
@@ -25,24 +25,28 @@
#include "generic.h"
-#define AT91_RTC_IDR 0x24 /* Interrupt Disable Register */
-#define AT91_RTC_IMR 0x28 /* Interrupt Mask Register */
+#define AT91_RTC_IDR 0x24 /* Interrupt Disable Register */
+#define AT91_RTC_IMR 0x28 /* Interrupt Mask Register */
+#define AT91_RTC_IRQ_MASK 0x1f /* Available IRQs mask */
void __init at91_sysirq_mask_rtc(u32 rtc_base)
{
void __iomem *base;
- u32 mask;
base = ioremap(rtc_base, 64);
if (!base)
return;
- mask = readl_relaxed(base + AT91_RTC_IMR);
- if (mask) {
- pr_info("AT91: Disabling rtc irq\n");
- writel_relaxed(mask, base + AT91_RTC_IDR);
- (void)readl_relaxed(base + AT91_RTC_IMR); /* flush */
- }
+ /*
+ * sam9x5 SoCs have the following errata:
+ * "RTC: Interrupt Mask Register cannot be used
+ * Interrupt Mask Register read always returns 0."
+ *
+ * Hence we're not relying on IMR values to disable
+ * interrupts.
+ */
+ writel_relaxed(AT91_RTC_IRQ_MASK, base + AT91_RTC_IDR);
+ (void)readl_relaxed(base + AT91_RTC_IMR); /* flush */
iounmap(base);
}
diff --git a/arch/arm/mach-bcm/bcm_5301x.c b/arch/arm/mach-bcm/bcm_5301x.c
index edff6976..e9bcbdb 100644
--- a/arch/arm/mach-bcm/bcm_5301x.c
+++ b/arch/arm/mach-bcm/bcm_5301x.c
@@ -43,19 +43,14 @@
"imprecise external abort");
}
-static void __init bcm5301x_dt_init(void)
-{
- l2x0_of_init(0, ~0UL);
- of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
-}
-
static const char __initconst *bcm5301x_dt_compat[] = {
"brcm,bcm4708",
NULL,
};
DT_MACHINE_START(BCM5301X, "BCM5301X")
+ .l2c_aux_val = 0,
+ .l2c_aux_mask = ~0,
.init_early = bcm5301x_init_early,
- .init_machine = bcm5301x_dt_init,
.dt_compat = bcm5301x_dt_compat,
MACHINE_END
diff --git a/arch/arm/mach-berlin/berlin.c b/arch/arm/mach-berlin/berlin.c
index 025bcb5..ac181c6 100644
--- a/arch/arm/mach-berlin/berlin.c
+++ b/arch/arm/mach-berlin/berlin.c
@@ -18,16 +18,6 @@
#include <asm/hardware/cache-l2x0.h>
#include <asm/mach/arch.h>
-static void __init berlin_init_machine(void)
-{
- /*
- * with DT probing for L2CCs, berlin_init_machine can be removed.
- * Note: 88DE3005 (Armada 1500-mini) uses pl310 l2cc
- */
- l2x0_of_init(0x70c00000, 0xfeffffff);
- of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
-}
-
static const char * const berlin_dt_compat[] = {
"marvell,berlin",
NULL,
@@ -35,5 +25,10 @@
DT_MACHINE_START(BERLIN_DT, "Marvell Berlin")
.dt_compat = berlin_dt_compat,
- .init_machine = berlin_init_machine,
+ /*
+ * with DT probing for L2CCs, berlin_init_machine can be removed.
+ * Note: 88DE3005 (Armada 1500-mini) uses pl310 l2cc
+ */
+ .l2c_aux_val = 0x30c00000,
+ .l2c_aux_mask = 0xfeffffff,
MACHINE_END
diff --git a/arch/arm/mach-clps711x/board-clep7312.c b/arch/arm/mach-clps711x/board-clep7312.c
index 221b9de..94a7add 100644
--- a/arch/arm/mach-clps711x/board-clep7312.c
+++ b/arch/arm/mach-clps711x/board-clep7312.c
@@ -18,6 +18,7 @@
#include <linux/init.h>
#include <linux/types.h>
#include <linux/string.h>
+#include <linux/memblock.h>
#include <asm/setup.h>
#include <asm/mach-types.h>
@@ -26,11 +27,9 @@
#include "common.h"
static void __init
-fixup_clep7312(struct tag *tags, char **cmdline, struct meminfo *mi)
+fixup_clep7312(struct tag *tags, char **cmdline)
{
- mi->nr_banks=1;
- mi->bank[0].start = 0xc0000000;
- mi->bank[0].size = 0x01000000;
+ memblock_add(0xc0000000, 0x01000000);
}
MACHINE_START(CLEP7212, "Cirrus Logic 7212/7312")
diff --git a/arch/arm/mach-clps711x/board-edb7211.c b/arch/arm/mach-clps711x/board-edb7211.c
index 0776098..f9828f8 100644
--- a/arch/arm/mach-clps711x/board-edb7211.c
+++ b/arch/arm/mach-clps711x/board-edb7211.c
@@ -16,6 +16,7 @@
#include <linux/interrupt.h>
#include <linux/backlight.h>
#include <linux/platform_device.h>
+#include <linux/memblock.h>
#include <linux/mtd/physmap.h>
#include <linux/mtd/partitions.h>
@@ -133,7 +134,7 @@
}
static void __init
-fixup_edb7211(struct tag *tags, char **cmdline, struct meminfo *mi)
+fixup_edb7211(struct tag *tags, char **cmdline)
{
/*
* Bank start addresses are not present in the information
@@ -143,11 +144,8 @@
* Banks sizes _are_ present in the param block, but we're
* not using that information yet.
*/
- mi->bank[0].start = 0xc0000000;
- mi->bank[0].size = SZ_8M;
- mi->bank[1].start = 0xc1000000;
- mi->bank[1].size = SZ_8M;
- mi->nr_banks = 2;
+ memblock_add(0xc0000000, SZ_8M);
+ memblock_add(0xc1000000, SZ_8M);
}
static void __init edb7211_init(void)
diff --git a/arch/arm/mach-clps711x/board-p720t.c b/arch/arm/mach-clps711x/board-p720t.c
index 67b7337..0cf0e51 100644
--- a/arch/arm/mach-clps711x/board-p720t.c
+++ b/arch/arm/mach-clps711x/board-p720t.c
@@ -295,7 +295,7 @@
};
static void __init
-fixup_p720t(struct tag *tag, char **cmdline, struct meminfo *mi)
+fixup_p720t(struct tag *tag, char **cmdline)
{
/*
* Our bootloader doesn't setup any tags (yet).
diff --git a/arch/arm/mach-cns3xxx/core.c b/arch/arm/mach-cns3xxx/core.c
index 2ae28a6..f85449a 100644
--- a/arch/arm/mach-cns3xxx/core.c
+++ b/arch/arm/mach-cns3xxx/core.c
@@ -272,9 +272,9 @@
*
* 1 cycle of latency for setup, read and write accesses
*/
- val = readl(base + L2X0_TAG_LATENCY_CTRL);
+ val = readl(base + L310_TAG_LATENCY_CTRL);
val &= 0xfffff888;
- writel(val, base + L2X0_TAG_LATENCY_CTRL);
+ writel(val, base + L310_TAG_LATENCY_CTRL);
/*
* Data RAM Control register
@@ -285,12 +285,12 @@
*
* 1 cycle of latency for setup, read and write accesses
*/
- val = readl(base + L2X0_DATA_LATENCY_CTRL);
+ val = readl(base + L310_DATA_LATENCY_CTRL);
val &= 0xfffff888;
- writel(val, base + L2X0_DATA_LATENCY_CTRL);
+ writel(val, base + L310_DATA_LATENCY_CTRL);
/* 32 KiB, 8-way, parity disable */
- l2x0_init(base, 0x00540000, 0xfe000fff);
+ l2x0_init(base, 0x00500000, 0xfe0f0fff);
}
#endif /* CONFIG_CACHE_L2X0 */
diff --git a/arch/arm/mach-ep93xx/crunch-bits.S b/arch/arm/mach-ep93xx/crunch-bits.S
index 0ec9bb4..e96923a 100644
--- a/arch/arm/mach-ep93xx/crunch-bits.S
+++ b/arch/arm/mach-ep93xx/crunch-bits.S
@@ -16,6 +16,7 @@
#include <asm/ptrace.h>
#include <asm/thread_info.h>
#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
#include <mach/ep93xx-regs.h>
/*
@@ -62,14 +63,16 @@
* r9 = ret_from_exception
* lr = undefined instr exit
*
- * called from prefetch exception handler with interrupts disabled
+ * called from prefetch exception handler with interrupts enabled
*/
ENTRY(crunch_task_enable)
+ inc_preempt_count r10, r3
+
ldr r8, =(EP93XX_APB_VIRT_BASE + 0x00130000) @ syscon addr
ldr r1, [r8, #0x80]
tst r1, #0x00800000 @ access to crunch enabled?
- movne pc, lr @ if so no business here
+ bne 2f @ if so no business here
mov r3, #0xaa @ unlock syscon swlock
str r3, [r8, #0xc0]
orr r1, r1, #0x00800000 @ enable access to crunch
@@ -142,7 +145,7 @@
teq r0, #0 @ anything to load?
cfldr64eq mvdx0, [r1, #CRUNCH_MVDX0] @ mvdx0 was clobbered
- moveq pc, lr
+ beq 1f
crunch_load:
cfldr64 mvdx0, [r0, #CRUNCH_DSPSC] @ load status word
@@ -190,6 +193,11 @@
cfldr64 mvdx14, [r0, #CRUNCH_MVDX14]
cfldr64 mvdx15, [r0, #CRUNCH_MVDX15]
+1:
+#ifdef CONFIG_PREEMPT_COUNT
+ get_thread_info r10
+#endif
+2: dec_preempt_count r10, r3
mov pc, lr
/*
diff --git a/arch/arm/mach-exynos/common.h b/arch/arm/mach-exynos/common.h
index 80b90e3..16617bd 100644
--- a/arch/arm/mach-exynos/common.h
+++ b/arch/arm/mach-exynos/common.h
@@ -153,7 +153,6 @@
NUM_SYS_POWERDOWN,
};
-extern unsigned long l2x0_regs_phys;
struct exynos_pmu_conf {
void __iomem *reg;
unsigned int val[NUM_SYS_POWERDOWN];
diff --git a/arch/arm/mach-exynos/exynos.c b/arch/arm/mach-exynos/exynos.c
index a56ce45..90aab4d 100644
--- a/arch/arm/mach-exynos/exynos.c
+++ b/arch/arm/mach-exynos/exynos.c
@@ -30,9 +30,6 @@
#include "mfc.h"
#include "regs-pmu.h"
-#define L2_AUX_VAL 0x7C470001
-#define L2_AUX_MASK 0xC200ffff
-
static struct map_desc exynos4_iodesc[] __initdata = {
{
.virtual = (unsigned long)S3C_VA_SYS,
@@ -246,25 +243,6 @@
exynos_map_io();
}
-static int __init exynos4_l2x0_cache_init(void)
-{
- int ret;
-
- if (!soc_is_exynos4())
- return 0;
-
- ret = l2x0_of_init(L2_AUX_VAL, L2_AUX_MASK);
- if (ret)
- return ret;
-
- if (IS_ENABLED(CONFIG_S5P_SLEEP)) {
- l2x0_regs_phys = virt_to_phys(&l2x0_saved_regs);
- clean_dcache_area(&l2x0_regs_phys, sizeof(unsigned long));
- }
- return 0;
-}
-early_initcall(exynos4_l2x0_cache_init);
-
static void __init exynos_dt_machine_init(void)
{
struct device_node *i2c_np;
@@ -333,6 +311,8 @@
DT_MACHINE_START(EXYNOS_DT, "SAMSUNG EXYNOS (Flattened Device Tree)")
/* Maintainer: Thomas Abraham <thomas.abraham@linaro.org> */
/* Maintainer: Kukjin Kim <kgene.kim@samsung.com> */
+ .l2c_aux_val = 0x3c400001,
+ .l2c_aux_mask = 0xc20fffff,
.smp = smp_ops(exynos_smp_ops),
.map_io = exynos_init_io,
.init_early = exynos_firmware_init,
diff --git a/arch/arm/mach-exynos/sleep.S b/arch/arm/mach-exynos/sleep.S
index a2613e9..108a45f 100644
--- a/arch/arm/mach-exynos/sleep.S
+++ b/arch/arm/mach-exynos/sleep.S
@@ -16,8 +16,6 @@
*/
#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
-#include <asm/hardware/cache-l2x0.h>
#define CPU_MASK 0xff0ffff0
#define CPU_CORTEX_A9 0x410fc090
@@ -53,33 +51,7 @@
and r0, r0, r1
ldr r1, =CPU_CORTEX_A9
cmp r0, r1
- bne skip_l2_resume
- adr r0, l2x0_regs_phys
- ldr r0, [r0]
- cmp r0, #0
- beq skip_l2_resume
- ldr r1, [r0, #L2X0_R_PHY_BASE]
- ldr r2, [r1, #L2X0_CTRL]
- tst r2, #0x1
- bne skip_l2_resume
- ldr r2, [r0, #L2X0_R_AUX_CTRL]
- str r2, [r1, #L2X0_AUX_CTRL]
- ldr r2, [r0, #L2X0_R_TAG_LATENCY]
- str r2, [r1, #L2X0_TAG_LATENCY_CTRL]
- ldr r2, [r0, #L2X0_R_DATA_LATENCY]
- str r2, [r1, #L2X0_DATA_LATENCY_CTRL]
- ldr r2, [r0, #L2X0_R_PREFETCH_CTRL]
- str r2, [r1, #L2X0_PREFETCH_CTRL]
- ldr r2, [r0, #L2X0_R_PWR_CTRL]
- str r2, [r1, #L2X0_POWER_CTRL]
- mov r2, #1
- str r2, [r1, #L2X0_CTRL]
-skip_l2_resume:
+ bleq l2c310_early_resume
#endif
b cpu_resume
ENDPROC(exynos_cpu_resume)
-#ifdef CONFIG_CACHE_L2X0
- .globl l2x0_regs_phys
-l2x0_regs_phys:
- .long 0
-#endif
diff --git a/arch/arm/mach-footbridge/cats-hw.c b/arch/arm/mach-footbridge/cats-hw.c
index da04150..8f05489 100644
--- a/arch/arm/mach-footbridge/cats-hw.c
+++ b/arch/arm/mach-footbridge/cats-hw.c
@@ -76,7 +76,7 @@
* hard reboots fail on early boards.
*/
static void __init
-fixup_cats(struct tag *tags, char **cmdline, struct meminfo *mi)
+fixup_cats(struct tag *tags, char **cmdline)
{
#if defined(CONFIG_VGA_CONSOLE) || defined(CONFIG_DUMMY_CONSOLE)
screen_info.orig_video_lines = 25;
diff --git a/arch/arm/mach-footbridge/netwinder-hw.c b/arch/arm/mach-footbridge/netwinder-hw.c
index eb1fa5c..cdee08c 100644
--- a/arch/arm/mach-footbridge/netwinder-hw.c
+++ b/arch/arm/mach-footbridge/netwinder-hw.c
@@ -620,7 +620,7 @@
* the parameter page.
*/
static void __init
-fixup_netwinder(struct tag *tags, char **cmdline, struct meminfo *mi)
+fixup_netwinder(struct tag *tags, char **cmdline)
{
#ifdef CONFIG_ISAPNP
extern int isapnp_disable;
diff --git a/arch/arm/mach-highbank/highbank.c b/arch/arm/mach-highbank/highbank.c
index c7de89b..8c35ae4 100644
--- a/arch/arm/mach-highbank/highbank.c
+++ b/arch/arm/mach-highbank/highbank.c
@@ -51,11 +51,13 @@
}
-static void highbank_l2x0_disable(void)
+static void highbank_l2c310_write_sec(unsigned long val, unsigned reg)
{
- outer_flush_all();
- /* Disable PL310 L2 Cache controller */
- highbank_smc1(0x102, 0x0);
+ if (reg == L2X0_CTRL)
+ highbank_smc1(0x102, val);
+ else
+ WARN_ONCE(1, "Highbank L2C310: ignoring write to reg 0x%x\n",
+ reg);
}
static void __init highbank_init_irq(void)
@@ -64,14 +66,6 @@
if (of_find_compatible_node(NULL, NULL, "arm,cortex-a9"))
highbank_scu_map_io();
-
- /* Enable PL310 L2 Cache controller */
- if (IS_ENABLED(CONFIG_CACHE_L2X0) &&
- of_find_compatible_node(NULL, NULL, "arm,pl310-cache")) {
- highbank_smc1(0x102, 0x1);
- l2x0_of_init(0, ~0UL);
- outer_cache.disable = highbank_l2x0_disable;
- }
}
static void highbank_power_off(void)
@@ -185,6 +179,9 @@
#if defined(CONFIG_ZONE_DMA) && defined(CONFIG_ARM_LPAE)
.dma_zone_size = (4ULL * SZ_1G),
#endif
+ .l2c_aux_val = 0,
+ .l2c_aux_mask = ~0,
+ .l2c_write_sec = highbank_l2c310_write_sec,
.init_irq = highbank_init_irq,
.init_machine = highbank_init,
.dt_compat = highbank_match,
diff --git a/arch/arm/mach-imx/mach-vf610.c b/arch/arm/mach-imx/mach-vf610.c
index 2d8aef5..c446027 100644
--- a/arch/arm/mach-imx/mach-vf610.c
+++ b/arch/arm/mach-imx/mach-vf610.c
@@ -20,19 +20,14 @@
of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
}
-static void __init vf610_init_irq(void)
-{
- l2x0_of_init(0, ~0UL);
- irqchip_init();
-}
-
static const char *vf610_dt_compat[] __initconst = {
"fsl,vf610",
NULL,
};
DT_MACHINE_START(VYBRID_VF610, "Freescale Vybrid VF610 (Device Tree)")
- .init_irq = vf610_init_irq,
+ .l2c_aux_val = 0,
+ .l2c_aux_mask = ~0,
.init_machine = vf610_init_machine,
.dt_compat = vf610_dt_compat,
.restart = mxc_restart,
diff --git a/arch/arm/mach-imx/suspend-imx6.S b/arch/arm/mach-imx/suspend-imx6.S
index 20048ff..fe123b0 100644
--- a/arch/arm/mach-imx/suspend-imx6.S
+++ b/arch/arm/mach-imx/suspend-imx6.S
@@ -334,28 +334,10 @@
* turned into relative ones.
*/
-#ifdef CONFIG_CACHE_L2X0
- .macro pl310_resume
- adr r0, l2x0_saved_regs_offset
- ldr r2, [r0]
- add r2, r2, r0
- ldr r0, [r2, #L2X0_R_PHY_BASE] @ get physical base of l2x0
- ldr r1, [r2, #L2X0_R_AUX_CTRL] @ get aux_ctrl value
- str r1, [r0, #L2X0_AUX_CTRL] @ restore aux_ctrl
- mov r1, #0x1
- str r1, [r0, #L2X0_CTRL] @ re-enable L2
- .endm
-
-l2x0_saved_regs_offset:
- .word l2x0_saved_regs - .
-
-#else
- .macro pl310_resume
- .endm
-#endif
-
ENTRY(v7_cpu_resume)
bl v7_invalidate_l1
- pl310_resume
+#ifdef CONFIG_CACHE_L2X0
+ bl l2c310_early_resume
+#endif
b cpu_resume
ENDPROC(v7_cpu_resume)
diff --git a/arch/arm/mach-imx/system.c b/arch/arm/mach-imx/system.c
index 5e3027d..3b0733e 100644
--- a/arch/arm/mach-imx/system.c
+++ b/arch/arm/mach-imx/system.c
@@ -124,7 +124,7 @@
}
/* Configure the L2 PREFETCH and POWER registers */
- val = readl_relaxed(l2x0_base + L2X0_PREFETCH_CTRL);
+ val = readl_relaxed(l2x0_base + L310_PREFETCH_CTRL);
val |= 0x70800000;
/*
* The L2 cache controller(PL310) version on the i.MX6D/Q is r3p1-50rel0
@@ -137,14 +137,12 @@
*/
if (cpu_is_imx6q())
val &= ~(1 << 30 | 1 << 23);
- writel_relaxed(val, l2x0_base + L2X0_PREFETCH_CTRL);
- val = L2X0_DYNAMIC_CLK_GATING_EN | L2X0_STNDBY_MODE_EN;
- writel_relaxed(val, l2x0_base + L2X0_POWER_CTRL);
+ writel_relaxed(val, l2x0_base + L310_PREFETCH_CTRL);
iounmap(l2x0_base);
of_node_put(np);
out:
- l2x0_of_init(0, ~0UL);
+ l2x0_of_init(0, ~0);
}
#endif
diff --git a/arch/arm/mach-msm/board-halibut.c b/arch/arm/mach-msm/board-halibut.c
index a775298..61bfe58 100644
--- a/arch/arm/mach-msm/board-halibut.c
+++ b/arch/arm/mach-msm/board-halibut.c
@@ -83,11 +83,6 @@
platform_add_devices(devices, ARRAY_SIZE(devices));
}
-static void __init halibut_fixup(struct tag *tags, char **cmdline,
- struct meminfo *mi)
-{
-}
-
static void __init halibut_map_io(void)
{
msm_map_common_io();
@@ -100,7 +95,6 @@
MACHINE_START(HALIBUT, "Halibut Board (QCT SURF7200A)")
.atag_offset = 0x100,
- .fixup = halibut_fixup,
.map_io = halibut_map_io,
.init_early = halibut_init_early,
.init_irq = halibut_init_irq,
diff --git a/arch/arm/mach-msm/board-mahimahi.c b/arch/arm/mach-msm/board-mahimahi.c
index 7d9981c..873c3ca 100644
--- a/arch/arm/mach-msm/board-mahimahi.c
+++ b/arch/arm/mach-msm/board-mahimahi.c
@@ -22,6 +22,7 @@
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
+#include <linux/memblock.h>
#include <asm/mach-types.h>
#include <asm/mach/arch.h>
@@ -52,16 +53,10 @@
platform_add_devices(devices, ARRAY_SIZE(devices));
}
-static void __init mahimahi_fixup(struct tag *tags, char **cmdline,
- struct meminfo *mi)
+static void __init mahimahi_fixup(struct tag *tags, char **cmdline)
{
- mi->nr_banks = 2;
- mi->bank[0].start = PHYS_OFFSET;
- mi->bank[0].node = PHYS_TO_NID(PHYS_OFFSET);
- mi->bank[0].size = (219*1024*1024);
- mi->bank[1].start = MSM_HIGHMEM_BASE;
- mi->bank[1].node = PHYS_TO_NID(MSM_HIGHMEM_BASE);
- mi->bank[1].size = MSM_HIGHMEM_SIZE;
+ memblock_add(PHYS_OFFSET, 219*SZ_1M);
+ memblock_add(MSM_HIGHMEM_BASE, MSM_HIGHMEM_SIZE);
}
static void __init mahimahi_map_io(void)
diff --git a/arch/arm/mach-msm/board-msm7x30.c b/arch/arm/mach-msm/board-msm7x30.c
index 0c4c200..2458843 100644
--- a/arch/arm/mach-msm/board-msm7x30.c
+++ b/arch/arm/mach-msm/board-msm7x30.c
@@ -40,8 +40,7 @@
#include "proc_comm.h"
#include "common.h"
-static void __init msm7x30_fixup(struct tag *tag, char **cmdline,
- struct meminfo *mi)
+static void __init msm7x30_fixup(struct tag *tag, char **cmdline)
{
for (; tag->hdr.size; tag = tag_next(tag))
if (tag->hdr.tag == ATAG_MEM && tag->u.mem.start == 0x200000) {
diff --git a/arch/arm/mach-msm/board-sapphire.c b/arch/arm/mach-msm/board-sapphire.c
index 3276051..e509679 100644
--- a/arch/arm/mach-msm/board-sapphire.c
+++ b/arch/arm/mach-msm/board-sapphire.c
@@ -35,6 +35,7 @@
#include <linux/mtd/nand.h>
#include <linux/mtd/partitions.h>
+#include <linux/memblock.h>
#include "gpio_chip.h"
#include "board-sapphire.h"
@@ -74,22 +75,18 @@
}
};
-static void __init sapphire_fixup(struct tag *tags, char **cmdline,
- struct meminfo *mi)
+static void __init sapphire_fixup(struct tag *tags, char **cmdline)
{
int smi_sz = parse_tag_smi((const struct tag *)tags);
- mi->nr_banks = 1;
- mi->bank[0].start = PHYS_OFFSET;
- mi->bank[0].node = PHYS_TO_NID(PHYS_OFFSET);
if (smi_sz == 32) {
- mi->bank[0].size = (84*1024*1024);
+ memblock_add(PHYS_OFFSET, 84*SZ_1M);
} else if (smi_sz == 64) {
- mi->bank[0].size = (101*1024*1024);
+ memblock_add(PHYS_OFFSET, 101*SZ_1M);
} else {
+ memblock_add(PHYS_OFFSET, 101*SZ_1M);
/* Give a default value when not get smi size */
smi_sz = 64;
- mi->bank[0].size = (101*1024*1024);
}
}
diff --git a/arch/arm/mach-msm/board-trout.c b/arch/arm/mach-msm/board-trout.c
index 5edfbd9..f72b07de 100644
--- a/arch/arm/mach-msm/board-trout.c
+++ b/arch/arm/mach-msm/board-trout.c
@@ -19,6 +19,7 @@
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/clkdev.h>
+#include <linux/memblock.h>
#include <asm/system_info.h>
#include <asm/mach-types.h>
@@ -55,12 +56,9 @@
msm_init_irq();
}
-static void __init trout_fixup(struct tag *tags, char **cmdline,
- struct meminfo *mi)
+static void __init trout_fixup(struct tag *tags, char **cmdline)
{
- mi->nr_banks = 1;
- mi->bank[0].start = PHYS_OFFSET;
- mi->bank[0].size = (101*1024*1024);
+ memblock_add(PHYS_OFFSET, 101*SZ_1M);
}
static void __init trout_init(void)
diff --git a/arch/arm/mach-mvebu/board-v7.c b/arch/arm/mach-mvebu/board-v7.c
index 01cfce6..8bb742f 100644
--- a/arch/arm/mach-mvebu/board-v7.c
+++ b/arch/arm/mach-mvebu/board-v7.c
@@ -78,7 +78,6 @@
mvebu_scu_enable();
coherency_init();
BUG_ON(mvebu_mbus_dt_init(coherency_available()));
- l2x0_of_init(0, ~0UL);
if (of_machine_is_compatible("marvell,armada375"))
hook_fault_code(16 + 6, armada_375_external_abort_wa, SIGBUS, 0,
@@ -182,6 +181,8 @@
};
DT_MACHINE_START(ARMADA_370_XP_DT, "Marvell Armada 370/XP (Device Tree)")
+ .l2c_aux_val = 0,
+ .l2c_aux_mask = ~0,
.smp = smp_ops(armada_xp_smp_ops),
.init_machine = mvebu_dt_init,
.init_time = mvebu_timer_and_clk_init,
@@ -195,6 +196,8 @@
};
DT_MACHINE_START(ARMADA_375_DT, "Marvell Armada 375 (Device Tree)")
+ .l2c_aux_val = 0,
+ .l2c_aux_mask = ~0,
.init_time = mvebu_timer_and_clk_init,
.init_machine = mvebu_dt_init,
.restart = mvebu_restart,
@@ -208,6 +211,8 @@
};
DT_MACHINE_START(ARMADA_38X_DT, "Marvell Armada 380/385 (Device Tree)")
+ .l2c_aux_val = 0,
+ .l2c_aux_mask = ~0,
.init_time = mvebu_timer_and_clk_init,
.restart = mvebu_restart,
.dt_compat = armada_38x_dt_compat,
diff --git a/arch/arm/mach-nomadik/cpu-8815.c b/arch/arm/mach-nomadik/cpu-8815.c
index 4a1065e..9116ca4 100644
--- a/arch/arm/mach-nomadik/cpu-8815.c
+++ b/arch/arm/mach-nomadik/cpu-8815.c
@@ -143,23 +143,16 @@
}
device_initcall(cpu8815_mmcsd_init);
-static void __init cpu8815_init_of(void)
-{
-#ifdef CONFIG_CACHE_L2X0
- /* At full speed latency must be >=2, so 0x249 in low bits */
- l2x0_of_init(0x00730249, 0xfe000fff);
-#endif
- of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
-}
-
static const char * cpu8815_board_compat[] = {
"calaosystems,usb-s8815",
NULL,
};
DT_MACHINE_START(NOMADIK_DT, "Nomadik STn8815")
+ /* At full speed latency must be >=2, so 0x249 in low bits */
+ .l2c_aux_val = 0x00700249,
+ .l2c_aux_mask = 0xfe0fefff,
.map_io = cpu8815_map_io,
- .init_machine = cpu8815_init_of,
.restart = cpu8815_restart,
.dt_compat = cpu8815_board_compat,
MACHINE_END
diff --git a/arch/arm/mach-omap2/Kconfig b/arch/arm/mach-omap2/Kconfig
index cb31d43..0ba4826 100644
--- a/arch/arm/mach-omap2/Kconfig
+++ b/arch/arm/mach-omap2/Kconfig
@@ -65,6 +65,7 @@
select ARCH_HAS_OPP
select ARM_GIC
select MACH_OMAP_GENERIC
+ select MIGHT_HAVE_CACHE_L2X0
config SOC_DRA7XX
bool "TI DRA7XX"
diff --git a/arch/arm/mach-omap2/common.h b/arch/arm/mach-omap2/common.h
index d88aff7..ff02973 100644
--- a/arch/arm/mach-omap2/common.h
+++ b/arch/arm/mach-omap2/common.h
@@ -91,6 +91,7 @@
extern void omap3_secure_sync32k_timer_init(void);
extern void omap3_gptimer_timer_init(void);
extern void omap4_local_timer_init(void);
+int omap_l2_cache_init(void);
extern void omap5_realtime_timer_init(void);
void omap2420_init_early(void);
diff --git a/arch/arm/mach-omap2/devices.c b/arch/arm/mach-omap2/devices.c
index e58609b..592ba0a 100644
--- a/arch/arm/mach-omap2/devices.c
+++ b/arch/arm/mach-omap2/devices.c
@@ -18,7 +18,6 @@
#include <linux/slab.h>
#include <linux/of.h>
#include <linux/pinctrl/machine.h>
-#include <linux/platform_data/omap4-keypad.h>
#include <linux/platform_data/mailbox-omap.h>
#include <asm/mach-types.h>
@@ -29,7 +28,6 @@
#include "iomap.h"
#include "omap_hwmod.h"
#include "omap_device.h"
-#include "omap4-keypad.h"
#include "soc.h"
#include "common.h"
@@ -255,37 +253,6 @@
#endif
}
-int __init omap4_keyboard_init(struct omap4_keypad_platform_data
- *sdp4430_keypad_data, struct omap_board_data *bdata)
-{
- struct platform_device *pdev;
- struct omap_hwmod *oh;
- struct omap4_keypad_platform_data *keypad_data;
- unsigned int id = -1;
- char *oh_name = "kbd";
- char *name = "omap4-keypad";
-
- oh = omap_hwmod_lookup(oh_name);
- if (!oh) {
- pr_err("Could not look up %s\n", oh_name);
- return -ENODEV;
- }
-
- keypad_data = sdp4430_keypad_data;
-
- pdev = omap_device_build(name, id, oh, keypad_data,
- sizeof(struct omap4_keypad_platform_data));
-
- if (IS_ERR(pdev)) {
- WARN(1, "Can't build omap_device for %s:%s.\n",
- name, oh->name);
- return PTR_ERR(pdev);
- }
- oh->mux = omap_hwmod_mux_init(bdata->pads, bdata->pads_cnt);
-
- return 0;
-}
-
#if defined(CONFIG_OMAP2PLUS_MBOX) || defined(CONFIG_OMAP2PLUS_MBOX_MODULE)
static inline void __init omap_init_mbox(void)
{
diff --git a/arch/arm/mach-omap2/gpmc.c b/arch/arm/mach-omap2/gpmc.c
index 852b19a..2c0c281 100644
--- a/arch/arm/mach-omap2/gpmc.c
+++ b/arch/arm/mach-omap2/gpmc.c
@@ -68,6 +68,9 @@
#define GPMC_ECC_BCH_RESULT_1 0x244 /* not available on OMAP2 */
#define GPMC_ECC_BCH_RESULT_2 0x248 /* not available on OMAP2 */
#define GPMC_ECC_BCH_RESULT_3 0x24c /* not available on OMAP2 */
+#define GPMC_ECC_BCH_RESULT_4 0x300 /* not available on OMAP2 */
+#define GPMC_ECC_BCH_RESULT_5 0x304 /* not available on OMAP2 */
+#define GPMC_ECC_BCH_RESULT_6 0x308 /* not available on OMAP2 */
/* GPMC ECC control settings */
#define GPMC_ECC_CTRL_ECCCLEAR 0x100
@@ -677,6 +680,12 @@
GPMC_BCH_SIZE * i;
reg->gpmc_bch_result3[i] = gpmc_base + GPMC_ECC_BCH_RESULT_3 +
GPMC_BCH_SIZE * i;
+ reg->gpmc_bch_result4[i] = gpmc_base + GPMC_ECC_BCH_RESULT_4 +
+ i * GPMC_BCH_SIZE;
+ reg->gpmc_bch_result5[i] = gpmc_base + GPMC_ECC_BCH_RESULT_5 +
+ i * GPMC_BCH_SIZE;
+ reg->gpmc_bch_result6[i] = gpmc_base + GPMC_ECC_BCH_RESULT_6 +
+ i * GPMC_BCH_SIZE;
}
}
@@ -1412,6 +1421,12 @@
else
gpmc_nand_data->ecc_opt =
OMAP_ECC_BCH8_CODE_HW_DETECTION_SW;
+ else if (!strcmp(s, "bch16"))
+ if (gpmc_nand_data->elm_of_node)
+ gpmc_nand_data->ecc_opt =
+ OMAP_ECC_BCH16_CODE_HW;
+ else
+ pr_err("%s: BCH16 requires ELM support\n", __func__);
else
pr_err("%s: ti,nand-ecc-opt invalid value\n", __func__);
diff --git a/arch/arm/mach-omap2/io.c b/arch/arm/mach-omap2/io.c
index 4ec3b4a9..8f55945 100644
--- a/arch/arm/mach-omap2/io.c
+++ b/arch/arm/mach-omap2/io.c
@@ -609,6 +609,7 @@
am43xx_clockdomains_init();
am43xx_hwmod_init();
omap_hwmod_init_postsetup();
+ omap_l2_cache_init();
omap_clk_soc_init = am43xx_dt_clk_init;
}
@@ -640,6 +641,7 @@
omap44xx_clockdomains_init();
omap44xx_hwmod_init();
omap_hwmod_init_postsetup();
+ omap_l2_cache_init();
omap_clk_soc_init = omap4xxx_dt_clk_init;
}
diff --git a/arch/arm/mach-omap2/omap-mpuss-lowpower.c b/arch/arm/mach-omap2/omap-mpuss-lowpower.c
index eb76e47..4001325 100644
--- a/arch/arm/mach-omap2/omap-mpuss-lowpower.c
+++ b/arch/arm/mach-omap2/omap-mpuss-lowpower.c
@@ -187,19 +187,15 @@
* in every restore MPUSS OFF path.
*/
#ifdef CONFIG_CACHE_L2X0
-static void save_l2x0_context(void)
+static void __init save_l2x0_context(void)
{
- u32 val;
- void __iomem *l2x0_base = omap4_get_l2cache_base();
- if (l2x0_base) {
- val = readl_relaxed(l2x0_base + L2X0_AUX_CTRL);
- writel_relaxed(val, sar_base + L2X0_AUXCTRL_OFFSET);
- val = readl_relaxed(l2x0_base + L2X0_PREFETCH_CTRL);
- writel_relaxed(val, sar_base + L2X0_PREFETCH_CTRL_OFFSET);
- }
+ writel_relaxed(l2x0_saved_regs.aux_ctrl,
+ sar_base + L2X0_AUXCTRL_OFFSET);
+ writel_relaxed(l2x0_saved_regs.prefetch_ctrl,
+ sar_base + L2X0_PREFETCH_CTRL_OFFSET);
}
#else
-static void save_l2x0_context(void)
+static void __init save_l2x0_context(void)
{}
#endif
diff --git a/arch/arm/mach-omap2/omap4-common.c b/arch/arm/mach-omap2/omap4-common.c
index 99b0154..326cd98 100644
--- a/arch/arm/mach-omap2/omap4-common.c
+++ b/arch/arm/mach-omap2/omap4-common.c
@@ -167,75 +167,57 @@
return l2cache_base;
}
-static void omap4_l2x0_disable(void)
+static void omap4_l2c310_write_sec(unsigned long val, unsigned reg)
{
- outer_flush_all();
- /* Disable PL310 L2 Cache controller */
- omap_smc1(0x102, 0x0);
+ unsigned smc_op;
+
+ switch (reg) {
+ case L2X0_CTRL:
+ smc_op = OMAP4_MON_L2X0_CTRL_INDEX;
+ break;
+
+ case L2X0_AUX_CTRL:
+ smc_op = OMAP4_MON_L2X0_AUXCTRL_INDEX;
+ break;
+
+ case L2X0_DEBUG_CTRL:
+ smc_op = OMAP4_MON_L2X0_DBG_CTRL_INDEX;
+ break;
+
+ case L310_PREFETCH_CTRL:
+ smc_op = OMAP4_MON_L2X0_PREFETCH_INDEX;
+ break;
+
+ default:
+ WARN_ONCE(1, "OMAP L2C310: ignoring write to reg 0x%x\n", reg);
+ return;
+ }
+
+ omap_smc1(smc_op, val);
}
-static void omap4_l2x0_set_debug(unsigned long val)
+int __init omap_l2_cache_init(void)
{
- /* Program PL310 L2 Cache controller debug register */
- omap_smc1(0x100, val);
-}
-
-static int __init omap_l2_cache_init(void)
-{
- u32 aux_ctrl = 0;
-
- /*
- * To avoid code running on other OMAPs in
- * multi-omap builds
- */
- if (!cpu_is_omap44xx())
- return -ENODEV;
+ u32 aux_ctrl;
/* Static mapping, never released */
l2cache_base = ioremap(OMAP44XX_L2CACHE_BASE, SZ_4K);
if (WARN_ON(!l2cache_base))
return -ENOMEM;
- /*
- * 16-way associativity, parity disabled
- * Way size - 32KB (es1.0)
- * Way size - 64KB (es2.0 +)
- */
- aux_ctrl = ((1 << L2X0_AUX_CTRL_ASSOCIATIVITY_SHIFT) |
- (0x1 << 25) |
- (0x1 << L2X0_AUX_CTRL_NS_LOCKDOWN_SHIFT) |
- (0x1 << L2X0_AUX_CTRL_NS_INT_CTRL_SHIFT));
+ /* 16-way associativity, parity disabled, way size - 64KB (es2.0 +) */
+ aux_ctrl = L2C_AUX_CTRL_SHARED_OVERRIDE |
+ L310_AUX_CTRL_DATA_PREFETCH |
+ L310_AUX_CTRL_INSTR_PREFETCH;
- if (omap_rev() == OMAP4430_REV_ES1_0) {
- aux_ctrl |= 0x2 << L2X0_AUX_CTRL_WAY_SIZE_SHIFT;
- } else {
- aux_ctrl |= ((0x3 << L2X0_AUX_CTRL_WAY_SIZE_SHIFT) |
- (1 << L2X0_AUX_CTRL_SHARE_OVERRIDE_SHIFT) |
- (1 << L2X0_AUX_CTRL_DATA_PREFETCH_SHIFT) |
- (1 << L2X0_AUX_CTRL_INSTR_PREFETCH_SHIFT) |
- (1 << L2X0_AUX_CTRL_EARLY_BRESP_SHIFT));
- }
- if (omap_rev() != OMAP4430_REV_ES1_0)
- omap_smc1(0x109, aux_ctrl);
-
- /* Enable PL310 L2 Cache controller */
- omap_smc1(0x102, 0x1);
-
+ outer_cache.write_sec = omap4_l2c310_write_sec;
if (of_have_populated_dt())
- l2x0_of_init(aux_ctrl, L2X0_AUX_CTRL_MASK);
+ l2x0_of_init(aux_ctrl, 0xcf9fffff);
else
- l2x0_init(l2cache_base, aux_ctrl, L2X0_AUX_CTRL_MASK);
-
- /*
- * Override default outer_cache.disable with a OMAP4
- * specific one
- */
- outer_cache.disable = omap4_l2x0_disable;
- outer_cache.set_debug = omap4_l2x0_set_debug;
+ l2x0_init(l2cache_base, aux_ctrl, 0xcf9fffff);
return 0;
}
-omap_early_initcall(omap_l2_cache_init);
#endif
void __iomem *omap4_get_sar_ram_base(void)
diff --git a/arch/arm/mach-omap2/omap4-keypad.h b/arch/arm/mach-omap2/omap4-keypad.h
deleted file mode 100644
index 20de0d5..0000000
--- a/arch/arm/mach-omap2/omap4-keypad.h
+++ /dev/null
@@ -1,8 +0,0 @@
-#ifndef ARCH_ARM_PLAT_OMAP4_KEYPAD_H
-#define ARCH_ARM_PLAT_OMAP4_KEYPAD_H
-
-struct omap_board_data;
-
-extern int omap4_keyboard_init(struct omap4_keypad_platform_data *,
- struct omap_board_data *);
-#endif
diff --git a/arch/arm/mach-orion5x/common.c b/arch/arm/mach-orion5x/common.c
index 3f1de11..6bbb7b5 100644
--- a/arch/arm/mach-orion5x/common.c
+++ b/arch/arm/mach-orion5x/common.c
@@ -365,8 +365,7 @@
* Many orion-based systems have buggy bootloader implementations.
* This is a common fixup for bogus memory tags.
*/
-void __init tag_fixup_mem32(struct tag *t, char **from,
- struct meminfo *meminfo)
+void __init tag_fixup_mem32(struct tag *t, char **from)
{
for (; t->hdr.size; t = tag_next(t))
if (t->hdr.tag == ATAG_MEM &&
diff --git a/arch/arm/mach-orion5x/common.h b/arch/arm/mach-orion5x/common.h
index 26d6f34..cd0389c 100644
--- a/arch/arm/mach-orion5x/common.h
+++ b/arch/arm/mach-orion5x/common.h
@@ -64,9 +64,8 @@
struct pci_bus *orion5x_pci_sys_scan_bus(int nr, struct pci_sys_data *sys);
int orion5x_pci_map_irq(const struct pci_dev *dev, u8 slot, u8 pin);
-struct meminfo;
struct tag;
-extern void __init tag_fixup_mem32(struct tag *, char **, struct meminfo *);
+extern void __init tag_fixup_mem32(struct tag *, char **);
#ifdef CONFIG_MACH_MSS2_DT
extern void mss2_init(void);
diff --git a/arch/arm/mach-prima2/Makefile b/arch/arm/mach-prima2/Makefile
index 7a6b4a3..8846e7d 100644
--- a/arch/arm/mach-prima2/Makefile
+++ b/arch/arm/mach-prima2/Makefile
@@ -2,7 +2,6 @@
obj-y += common.o
obj-y += rtciobrg.o
obj-$(CONFIG_DEBUG_LL) += lluart.o
-obj-$(CONFIG_CACHE_L2X0) += l2x0.o
obj-$(CONFIG_SUSPEND) += pm.o sleep.o
obj-$(CONFIG_SMP) += platsmp.o headsmp.o
obj-$(CONFIG_HOTPLUG_CPU) += hotplug.o
diff --git a/arch/arm/mach-prima2/common.c b/arch/arm/mach-prima2/common.c
index 47c7819..a860ea2 100644
--- a/arch/arm/mach-prima2/common.c
+++ b/arch/arm/mach-prima2/common.c
@@ -34,6 +34,8 @@
DT_MACHINE_START(ATLAS6_DT, "Generic ATLAS6 (Flattened Device Tree)")
/* Maintainer: Barry Song <baohua.song@csr.com> */
+ .l2c_aux_val = 0,
+ .l2c_aux_mask = ~0,
.map_io = sirfsoc_map_io,
.init_late = sirfsoc_init_late,
.dt_compat = atlas6_dt_match,
@@ -48,6 +50,8 @@
DT_MACHINE_START(PRIMA2_DT, "Generic PRIMA2 (Flattened Device Tree)")
/* Maintainer: Barry Song <baohua.song@csr.com> */
+ .l2c_aux_val = 0,
+ .l2c_aux_mask = ~0,
.map_io = sirfsoc_map_io,
.dma_zone_size = SZ_256M,
.init_late = sirfsoc_init_late,
@@ -63,6 +67,8 @@
DT_MACHINE_START(MARCO_DT, "Generic MARCO (Flattened Device Tree)")
/* Maintainer: Barry Song <baohua.song@csr.com> */
+ .l2c_aux_val = 0,
+ .l2c_aux_mask = ~0,
.smp = smp_ops(sirfsoc_smp_ops),
.map_io = sirfsoc_map_io,
.init_late = sirfsoc_init_late,
diff --git a/arch/arm/mach-prima2/l2x0.c b/arch/arm/mach-prima2/l2x0.c
deleted file mode 100644
index c710253..0000000
--- a/arch/arm/mach-prima2/l2x0.c
+++ /dev/null
@@ -1,49 +0,0 @@
-/*
- * l2 cache initialization for CSR SiRFprimaII
- *
- * Copyright (c) 2011 Cambridge Silicon Radio Limited, a CSR plc group company.
- *
- * Licensed under GPLv2 or later.
- */
-
-#include <linux/init.h>
-#include <linux/kernel.h>
-#include <linux/of.h>
-#include <asm/hardware/cache-l2x0.h>
-
-struct l2x0_aux {
- u32 val;
- u32 mask;
-};
-
-static const struct l2x0_aux prima2_l2x0_aux __initconst = {
- .val = 2 << L2X0_AUX_CTRL_WAY_SIZE_SHIFT,
- .mask = 0,
-};
-
-static const struct l2x0_aux marco_l2x0_aux __initconst = {
- .val = (2 << L2X0_AUX_CTRL_WAY_SIZE_SHIFT) |
- (1 << L2X0_AUX_CTRL_ASSOCIATIVITY_SHIFT),
- .mask = L2X0_AUX_CTRL_MASK,
-};
-
-static const struct of_device_id sirf_l2x0_ids[] __initconst = {
- { .compatible = "sirf,prima2-pl310-cache", .data = &prima2_l2x0_aux, },
- { .compatible = "sirf,marco-pl310-cache", .data = &marco_l2x0_aux, },
- {},
-};
-
-static int __init sirfsoc_l2x0_init(void)
-{
- struct device_node *np;
- const struct l2x0_aux *aux;
-
- np = of_find_matching_node(NULL, sirf_l2x0_ids);
- if (np) {
- aux = of_match_node(sirf_l2x0_ids, np)->data;
- return l2x0_of_init(aux->val, aux->mask);
- }
-
- return 0;
-}
-early_initcall(sirfsoc_l2x0_init);
diff --git a/arch/arm/mach-prima2/pm.c b/arch/arm/mach-prima2/pm.c
index c4525a8..96e9bc1 100644
--- a/arch/arm/mach-prima2/pm.c
+++ b/arch/arm/mach-prima2/pm.c
@@ -71,7 +71,6 @@
case PM_SUSPEND_MEM:
sirfsoc_pre_suspend_power_off();
- outer_flush_all();
outer_disable();
/* go zzz */
cpu_suspend(0, sirfsoc_finish_suspend);
diff --git a/arch/arm/mach-pxa/cm-x300.c b/arch/arm/mach-pxa/cm-x300.c
index 584439b..4d3588d 100644
--- a/arch/arm/mach-pxa/cm-x300.c
+++ b/arch/arm/mach-pxa/cm-x300.c
@@ -837,8 +837,7 @@
cm_x300_init_bl();
}
-static void __init cm_x300_fixup(struct tag *tags, char **cmdline,
- struct meminfo *mi)
+static void __init cm_x300_fixup(struct tag *tags, char **cmdline)
{
/* Make sure that mi->bank[0].start = PHYS_ADDR */
for (; tags->hdr.size; tags = tag_next(tags))
diff --git a/arch/arm/mach-pxa/corgi.c b/arch/arm/mach-pxa/corgi.c
index 57d6054..91dd1c7 100644
--- a/arch/arm/mach-pxa/corgi.c
+++ b/arch/arm/mach-pxa/corgi.c
@@ -34,6 +34,7 @@
#include <linux/input/matrix_keypad.h>
#include <linux/gpio_keys.h>
#include <linux/module.h>
+#include <linux/memblock.h>
#include <video/w100fb.h>
#include <asm/setup.h>
@@ -753,16 +754,13 @@
platform_add_devices(devices, ARRAY_SIZE(devices));
}
-static void __init fixup_corgi(struct tag *tags, char **cmdline,
- struct meminfo *mi)
+static void __init fixup_corgi(struct tag *tags, char **cmdline)
{
sharpsl_save_param();
- mi->nr_banks=1;
- mi->bank[0].start = 0xa0000000;
if (machine_is_corgi())
- mi->bank[0].size = (32*1024*1024);
+ memblock_add(0xa0000000, SZ_32M);
else
- mi->bank[0].size = (64*1024*1024);
+ memblock_add(0xa0000000, SZ_64M);
}
#ifdef CONFIG_MACH_CORGI
diff --git a/arch/arm/mach-pxa/eseries.c b/arch/arm/mach-pxa/eseries.c
index 8280ebca..cfb8641 100644
--- a/arch/arm/mach-pxa/eseries.c
+++ b/arch/arm/mach-pxa/eseries.c
@@ -21,6 +21,7 @@
#include <linux/mtd/nand.h>
#include <linux/mtd/partitions.h>
#include <linux/usb/gpio_vbus.h>
+#include <linux/memblock.h>
#include <video/w100fb.h>
@@ -41,14 +42,12 @@
#include "clock.h"
/* Only e800 has 128MB RAM */
-void __init eseries_fixup(struct tag *tags, char **cmdline, struct meminfo *mi)
+void __init eseries_fixup(struct tag *tags, char **cmdline)
{
- mi->nr_banks=1;
- mi->bank[0].start = 0xa0000000;
if (machine_is_e800())
- mi->bank[0].size = (128*1024*1024);
+ memblock_add(0xa0000000, SZ_128M);
else
- mi->bank[0].size = (64*1024*1024);
+ memblock_add(0xa0000000, SZ_64M);
}
struct gpio_vbus_mach_info e7xx_udc_info = {
diff --git a/arch/arm/mach-pxa/poodle.c b/arch/arm/mach-pxa/poodle.c
index aedf053..131991629 100644
--- a/arch/arm/mach-pxa/poodle.c
+++ b/arch/arm/mach-pxa/poodle.c
@@ -29,6 +29,7 @@
#include <linux/spi/ads7846.h>
#include <linux/spi/pxa2xx_spi.h>
#include <linux/mtd/sharpsl.h>
+#include <linux/memblock.h>
#include <mach/hardware.h>
#include <asm/mach-types.h>
@@ -456,13 +457,10 @@
poodle_init_spi();
}
-static void __init fixup_poodle(struct tag *tags, char **cmdline,
- struct meminfo *mi)
+static void __init fixup_poodle(struct tag *tags, char **cmdline)
{
sharpsl_save_param();
- mi->nr_banks=1;
- mi->bank[0].start = 0xa0000000;
- mi->bank[0].size = (32*1024*1024);
+ memblock_add(0xa0000000, SZ_32M);
}
MACHINE_START(POODLE, "SHARP Poodle")
diff --git a/arch/arm/mach-pxa/spitz.c b/arch/arm/mach-pxa/spitz.c
index 0b11c1af..840c3a4 100644
--- a/arch/arm/mach-pxa/spitz.c
+++ b/arch/arm/mach-pxa/spitz.c
@@ -32,6 +32,7 @@
#include <linux/io.h>
#include <linux/module.h>
#include <linux/reboot.h>
+#include <linux/memblock.h>
#include <asm/setup.h>
#include <asm/mach-types.h>
@@ -971,13 +972,10 @@
spitz_i2c_init();
}
-static void __init spitz_fixup(struct tag *tags, char **cmdline,
- struct meminfo *mi)
+static void __init spitz_fixup(struct tag *tags, char **cmdline)
{
sharpsl_save_param();
- mi->nr_banks = 1;
- mi->bank[0].start = 0xa0000000;
- mi->bank[0].size = (64*1024*1024);
+ memblock_add(0xa0000000, SZ_64M);
}
#ifdef CONFIG_MACH_SPITZ
diff --git a/arch/arm/mach-pxa/tosa.c b/arch/arm/mach-pxa/tosa.c
index ef5557b..c158a6e 100644
--- a/arch/arm/mach-pxa/tosa.c
+++ b/arch/arm/mach-pxa/tosa.c
@@ -37,6 +37,7 @@
#include <linux/i2c/pxa-i2c.h>
#include <linux/usb/gpio_vbus.h>
#include <linux/reboot.h>
+#include <linux/memblock.h>
#include <asm/setup.h>
#include <asm/mach-types.h>
@@ -960,13 +961,10 @@
platform_add_devices(devices, ARRAY_SIZE(devices));
}
-static void __init fixup_tosa(struct tag *tags, char **cmdline,
- struct meminfo *mi)
+static void __init fixup_tosa(struct tag *tags, char **cmdline)
{
sharpsl_save_param();
- mi->nr_banks=1;
- mi->bank[0].start = 0xa0000000;
- mi->bank[0].size = (64*1024*1024);
+ memblock_add(0xa0000000, SZ_64M);
}
MACHINE_START(TOSA, "SHARP Tosa")
diff --git a/arch/arm/mach-realview/core.c b/arch/arm/mach-realview/core.c
index 960b8dd..8c1b39a 100644
--- a/arch/arm/mach-realview/core.c
+++ b/arch/arm/mach-realview/core.c
@@ -31,6 +31,7 @@
#include <linux/amba/mmci.h>
#include <linux/gfp.h>
#include <linux/mtd/physmap.h>
+#include <linux/memblock.h>
#include <mach/hardware.h>
#include <asm/irq.h>
@@ -385,19 +386,15 @@
/*
* Setup the memory banks.
*/
-void realview_fixup(struct tag *tags, char **from, struct meminfo *meminfo)
+void realview_fixup(struct tag *tags, char **from)
{
/*
* Most RealView platforms have 512MB contiguous RAM at 0x70000000.
* Half of this is mirrored at 0.
*/
#ifdef CONFIG_REALVIEW_HIGH_PHYS_OFFSET
- meminfo->bank[0].start = 0x70000000;
- meminfo->bank[0].size = SZ_512M;
- meminfo->nr_banks = 1;
+ memblock_add(0x70000000, SZ_512M);
#else
- meminfo->bank[0].start = 0;
- meminfo->bank[0].size = SZ_256M;
- meminfo->nr_banks = 1;
+ memblock_add(0, SZ_256M);
#endif
}
diff --git a/arch/arm/mach-realview/core.h b/arch/arm/mach-realview/core.h
index 13dc830..868ece2 100644
--- a/arch/arm/mach-realview/core.h
+++ b/arch/arm/mach-realview/core.h
@@ -52,8 +52,7 @@
extern int realview_eth_register(const char *name, struct resource *res);
extern int realview_usb_register(struct resource *res);
extern void realview_init_early(void);
-extern void realview_fixup(struct tag *tags, char **from,
- struct meminfo *meminfo);
+extern void realview_fixup(struct tag *tags, char **from);
extern struct smp_operations realview_smp_ops;
extern void realview_cpu_die(unsigned int cpu);
diff --git a/arch/arm/mach-realview/realview_eb.c b/arch/arm/mach-realview/realview_eb.c
index 6bb070e8..739d4f1 100644
--- a/arch/arm/mach-realview/realview_eb.c
+++ b/arch/arm/mach-realview/realview_eb.c
@@ -442,8 +442,13 @@
realview_eb11mp_fixup();
#ifdef CONFIG_CACHE_L2X0
- /* 1MB (128KB/way), 8-way associativity, evmon/parity/share enabled
- * Bits: .... ...0 0111 1001 0000 .... .... .... */
+ /*
+ * The PL220 needs to be manually configured as the hardware
+ * doesn't report the correct sizes.
+ * 1MB (128KB/way), 8-way associativity, event monitor and
+ * parity enabled, ignore share bit, no force write allocate
+ * Bits: .... ...0 0111 1001 0000 .... .... ....
+ */
l2x0_init(__io_address(REALVIEW_EB11MP_L220_BASE), 0x00790000, 0xfe000fff);
#endif
platform_device_register(&pmu_device);
diff --git a/arch/arm/mach-realview/realview_pb1176.c b/arch/arm/mach-realview/realview_pb1176.c
index 173f2c1..b0e0dca 100644
--- a/arch/arm/mach-realview/realview_pb1176.c
+++ b/arch/arm/mach-realview/realview_pb1176.c
@@ -32,6 +32,7 @@
#include <linux/irqchip/arm-gic.h>
#include <linux/platform_data/clk-realview.h>
#include <linux/reboot.h>
+#include <linux/memblock.h>
#include <mach/hardware.h>
#include <asm/irq.h>
@@ -339,15 +340,12 @@
dsb();
}
-static void realview_pb1176_fixup(struct tag *tags, char **from,
- struct meminfo *meminfo)
+static void realview_pb1176_fixup(struct tag *tags, char **from)
{
/*
* RealView PB1176 only has 128MB of RAM mapped at 0.
*/
- meminfo->bank[0].start = 0;
- meminfo->bank[0].size = SZ_128M;
- meminfo->nr_banks = 1;
+ memblock_add(0, SZ_128M);
}
static void __init realview_pb1176_init(void)
@@ -355,7 +353,13 @@
int i;
#ifdef CONFIG_CACHE_L2X0
- /* 128Kb (16Kb/way) 8-way associativity. evmon/parity/share enabled. */
+ /*
+ * The PL220 needs to be manually configured as the hardware
+ * doesn't report the correct sizes.
+ * 128kB (16kB/way), 8-way associativity, event monitor and
+ * parity enabled, ignore share bit, no force write allocate
+ * Bits: .... ...0 0111 0011 0000 .... .... ....
+ */
l2x0_init(__io_address(REALVIEW_PB1176_L220_BASE), 0x00730000, 0xfe000fff);
#endif
diff --git a/arch/arm/mach-realview/realview_pb11mp.c b/arch/arm/mach-realview/realview_pb11mp.c
index bde7e6b..47bf55f 100644
--- a/arch/arm/mach-realview/realview_pb11mp.c
+++ b/arch/arm/mach-realview/realview_pb11mp.c
@@ -337,8 +337,13 @@
int i;
#ifdef CONFIG_CACHE_L2X0
- /* 1MB (128KB/way), 8-way associativity, evmon/parity/share enabled
- * Bits: .... ...0 0111 1001 0000 .... .... .... */
+ /*
+ * The PL220 needs to be manually configured as the hardware
+ * doesn't report the correct sizes.
+ * 1MB (128KB/way), 8-way associativity, event monitor and
+ * parity enabled, ignore share bit, no force write allocate
+ * Bits: .... ...0 0111 1001 0000 .... .... ....
+ */
l2x0_init(__io_address(REALVIEW_TC11MP_L220_BASE), 0x00790000, 0xfe000fff);
#endif
diff --git a/arch/arm/mach-realview/realview_pbx.c b/arch/arm/mach-realview/realview_pbx.c
index 72c96ca..d89eb40 100644
--- a/arch/arm/mach-realview/realview_pbx.c
+++ b/arch/arm/mach-realview/realview_pbx.c
@@ -29,6 +29,7 @@
#include <linux/irqchip/arm-gic.h>
#include <linux/platform_data/clk-realview.h>
#include <linux/reboot.h>
+#include <linux/memblock.h>
#include <asm/irq.h>
#include <asm/mach-types.h>
@@ -325,23 +326,19 @@
realview_pbx_twd_init();
}
-static void realview_pbx_fixup(struct tag *tags, char **from,
- struct meminfo *meminfo)
+static void realview_pbx_fixup(struct tag *tags, char **from)
{
#ifdef CONFIG_SPARSEMEM
/*
* Memory configuration with SPARSEMEM enabled on RealView PBX (see
* asm/mach/memory.h for more information).
*/
- meminfo->bank[0].start = 0;
- meminfo->bank[0].size = SZ_256M;
- meminfo->bank[1].start = 0x20000000;
- meminfo->bank[1].size = SZ_512M;
- meminfo->bank[2].start = 0x80000000;
- meminfo->bank[2].size = SZ_256M;
- meminfo->nr_banks = 3;
+
+ memblock_add(0, SZ_256M);
+ memblock_add(0x20000000, SZ_512M);
+ memblock_add(0x80000000, SZ_256M);
#else
- realview_fixup(tags, from, meminfo);
+ realview_fixup(tags, from);
#endif
}
@@ -370,8 +367,8 @@
__io_address(REALVIEW_PBX_TILE_L220_BASE);
/* set RAM latencies to 1 cycle for eASIC */
- writel(0, l2x0_base + L2X0_TAG_LATENCY_CTRL);
- writel(0, l2x0_base + L2X0_DATA_LATENCY_CTRL);
+ writel(0, l2x0_base + L310_TAG_LATENCY_CTRL);
+ writel(0, l2x0_base + L310_DATA_LATENCY_CTRL);
/* 16KB way size, 8-way associativity, parity disabled
* Bits: .. 0 0 0 0 1 00 1 0 1 001 0 000 0 .... .... .... */
diff --git a/arch/arm/mach-rockchip/rockchip.c b/arch/arm/mach-rockchip/rockchip.c
index 4499b0a..968cc34 100644
--- a/arch/arm/mach-rockchip/rockchip.c
+++ b/arch/arm/mach-rockchip/rockchip.c
@@ -24,12 +24,6 @@
#include <asm/hardware/cache-l2x0.h>
#include "core.h"
-static void __init rockchip_dt_init(void)
-{
- l2x0_of_init(0, ~0UL);
- of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
-}
-
static const char * const rockchip_board_dt_compat[] = {
"rockchip,rk2928",
"rockchip,rk3066a",
@@ -39,6 +33,7 @@
};
DT_MACHINE_START(ROCKCHIP_DT, "Rockchip Cortex-A9 (Device Tree)")
- .init_machine = rockchip_dt_init,
+ .l2c_aux_val = 0,
+ .l2c_aux_mask = ~0,
.dt_compat = rockchip_board_dt_compat,
MACHINE_END
diff --git a/arch/arm/mach-s3c24xx/mach-smdk2413.c b/arch/arm/mach-s3c24xx/mach-smdk2413.c
index a38f8a0..fb3b80e 100644
--- a/arch/arm/mach-s3c24xx/mach-smdk2413.c
+++ b/arch/arm/mach-s3c24xx/mach-smdk2413.c
@@ -22,6 +22,7 @@
#include <linux/serial_s3c.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/memblock.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
@@ -93,13 +94,10 @@
&s3c2412_device_dma,
};
-static void __init smdk2413_fixup(struct tag *tags, char **cmdline,
- struct meminfo *mi)
+static void __init smdk2413_fixup(struct tag *tags, char **cmdline)
{
if (tags != phys_to_virt(S3C2410_SDRAM_PA + 0x100)) {
- mi->nr_banks=1;
- mi->bank[0].start = 0x30000000;
- mi->bank[0].size = SZ_64M;
+ memblock_add(0x30000000, SZ_64M);
}
}
diff --git a/arch/arm/mach-s3c24xx/mach-vstms.c b/arch/arm/mach-s3c24xx/mach-vstms.c
index 6b706c9..9104c2b 100644
--- a/arch/arm/mach-s3c24xx/mach-vstms.c
+++ b/arch/arm/mach-s3c24xx/mach-vstms.c
@@ -23,6 +23,7 @@
#include <linux/mtd/nand.h>
#include <linux/mtd/nand_ecc.h>
#include <linux/mtd/partitions.h>
+#include <linux/memblock.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
@@ -129,13 +130,10 @@
&s3c2412_device_dma,
};
-static void __init vstms_fixup(struct tag *tags, char **cmdline,
- struct meminfo *mi)
+static void __init vstms_fixup(struct tag *tags, char **cmdline)
{
if (tags != phys_to_virt(S3C2410_SDRAM_PA + 0x100)) {
- mi->nr_banks=1;
- mi->bank[0].start = 0x30000000;
- mi->bank[0].size = SZ_64M;
+ memblock_add(0x30000000, SZ_64M);
}
}
diff --git a/arch/arm/mach-s5pv210/mach-goni.c b/arch/arm/mach-s5pv210/mach-goni.c
index 6c719ec..c1ce921 100644
--- a/arch/arm/mach-s5pv210/mach-goni.c
+++ b/arch/arm/mach-s5pv210/mach-goni.c
@@ -234,14 +234,6 @@
/* TSP */
static struct mxt_platform_data qt602240_platform_data = {
- .x_line = 17,
- .y_line = 11,
- .x_size = 800,
- .y_size = 480,
- .blen = 0x21,
- .threshold = 0x28,
- .voltage = 2800000, /* 2.8V */
- .orient = MXT_DIAGONAL,
.irqflags = IRQF_TRIGGER_FALLING,
};
diff --git a/arch/arm/mach-sa1100/assabet.c b/arch/arm/mach-sa1100/assabet.c
index 8443a27..7dd894e 100644
--- a/arch/arm/mach-sa1100/assabet.c
+++ b/arch/arm/mach-sa1100/assabet.c
@@ -531,7 +531,7 @@
}
static void __init
-fixup_assabet(struct tag *tags, char **cmdline, struct meminfo *mi)
+fixup_assabet(struct tag *tags, char **cmdline)
{
/* This must be done before any call to machine_has_neponset() */
map_sa1100_gpio_regs();
diff --git a/arch/arm/mach-shmobile/board-armadillo800eva-reference.c b/arch/arm/mach-shmobile/board-armadillo800eva-reference.c
index 57d246e..f660fbb 100644
--- a/arch/arm/mach-shmobile/board-armadillo800eva-reference.c
+++ b/arch/arm/mach-shmobile/board-armadillo800eva-reference.c
@@ -164,8 +164,8 @@
r8a7740_meram_workaround();
#ifdef CONFIG_CACHE_L2X0
- /* Early BRESP enable, Shared attribute override enable, 32K*8way */
- l2x0_init(IOMEM(0xf0002000), 0x40440000, 0x82000fff);
+ /* Shared attribute override enable, 32K*8way */
+ l2x0_init(IOMEM(0xf0002000), 0x00400000, 0xc20f0fff);
#endif
r8a7740_add_standard_devices_dt();
diff --git a/arch/arm/mach-shmobile/board-armadillo800eva.c b/arch/arm/mach-shmobile/board-armadillo800eva.c
index bc2cf7a..01f8110 100644
--- a/arch/arm/mach-shmobile/board-armadillo800eva.c
+++ b/arch/arm/mach-shmobile/board-armadillo800eva.c
@@ -1271,8 +1271,8 @@
#ifdef CONFIG_CACHE_L2X0
- /* Early BRESP enable, Shared attribute override enable, 32K*8way */
- l2x0_init(IOMEM(0xf0002000), 0x40440000, 0x82000fff);
+ /* Shared attribute override enable, 32K*8way */
+ l2x0_init(IOMEM(0xf0002000), 0x00400000, 0xc20f0fff);
#endif
i2c_register_board_info(0, i2c0_devices, ARRAY_SIZE(i2c0_devices));
diff --git a/arch/arm/mach-shmobile/board-kzm9g-reference.c b/arch/arm/mach-shmobile/board-kzm9g-reference.c
index 598e324..a735a1d 100644
--- a/arch/arm/mach-shmobile/board-kzm9g-reference.c
+++ b/arch/arm/mach-shmobile/board-kzm9g-reference.c
@@ -36,8 +36,8 @@
sh73a0_add_standard_devices_dt();
#ifdef CONFIG_CACHE_L2X0
- /* Early BRESP enable, Shared attribute override enable, 64K*8way */
- l2x0_init(IOMEM(0xf0100000), 0x40460000, 0x82000fff);
+ /* Shared attribute override enable, 64K*8way */
+ l2x0_init(IOMEM(0xf0100000), 0x00400000, 0xc20f0fff);
#endif
}
diff --git a/arch/arm/mach-shmobile/board-kzm9g.c b/arch/arm/mach-shmobile/board-kzm9g.c
index 03dc3ac..f94ec8c 100644
--- a/arch/arm/mach-shmobile/board-kzm9g.c
+++ b/arch/arm/mach-shmobile/board-kzm9g.c
@@ -876,8 +876,8 @@
gpio_request_one(223, GPIOF_IN, NULL); /* IRQ8 */
#ifdef CONFIG_CACHE_L2X0
- /* Early BRESP enable, Shared attribute override enable, 64K*8way */
- l2x0_init(IOMEM(0xf0100000), 0x40460000, 0x82000fff);
+ /* Shared attribute override enable, 64K*8way */
+ l2x0_init(IOMEM(0xf0100000), 0x00400000, 0xc20f0fff);
#endif
i2c_register_board_info(0, i2c0_devices, ARRAY_SIZE(i2c0_devices));
diff --git a/arch/arm/mach-shmobile/setup-r8a7778.c b/arch/arm/mach-shmobile/setup-r8a7778.c
index 8c02e24..d311ef9 100644
--- a/arch/arm/mach-shmobile/setup-r8a7778.c
+++ b/arch/arm/mach-shmobile/setup-r8a7778.c
@@ -285,10 +285,10 @@
void __iomem *base = ioremap_nocache(0xf0100000, 0x1000);
if (base) {
/*
- * Early BRESP enable, Shared attribute override enable, 64K*16way
+ * Shared attribute override enable, 64K*16way
* don't call iounmap(base)
*/
- l2x0_init(base, 0x40470000, 0x82000fff);
+ l2x0_init(base, 0x00400000, 0xc20f0fff);
}
#endif
diff --git a/arch/arm/mach-shmobile/setup-r8a7779.c b/arch/arm/mach-shmobile/setup-r8a7779.c
index d197b5a..aba4ed6 100644
--- a/arch/arm/mach-shmobile/setup-r8a7779.c
+++ b/arch/arm/mach-shmobile/setup-r8a7779.c
@@ -660,8 +660,8 @@
void __init r8a7779_add_standard_devices(void)
{
#ifdef CONFIG_CACHE_L2X0
- /* Early BRESP enable, Shared attribute override enable, 64K*16way */
- l2x0_init(IOMEM(0xf0100000), 0x40470000, 0x82000fff);
+ /* Shared attribute override enable, 64K*16way */
+ l2x0_init(IOMEM(0xf0100000), 0x00400000, 0xc20f0fff);
#endif
r8a7779_pm_init();
diff --git a/arch/arm/mach-socfpga/socfpga.c b/arch/arm/mach-socfpga/socfpga.c
index d86231e..adbf383 100644
--- a/arch/arm/mach-socfpga/socfpga.c
+++ b/arch/arm/mach-socfpga/socfpga.c
@@ -98,22 +98,17 @@
writel(temp, rst_manager_base_addr + SOCFPGA_RSTMGR_CTRL);
}
-static void __init socfpga_cyclone5_init(void)
-{
- l2x0_of_init(0, ~0UL);
- of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
-}
-
static const char *altera_dt_match[] = {
"altr,socfpga",
NULL
};
DT_MACHINE_START(SOCFPGA, "Altera SOCFPGA")
+ .l2c_aux_val = 0,
+ .l2c_aux_mask = ~0,
.smp = smp_ops(socfpga_smp_ops),
.map_io = socfpga_map_io,
.init_irq = socfpga_init_irq,
- .init_machine = socfpga_cyclone5_init,
.restart = socfpga_cyclone5_restart,
.dt_compat = altera_dt_match,
MACHINE_END
diff --git a/arch/arm/mach-spear/platsmp.c b/arch/arm/mach-spear/platsmp.c
index c19751ff..fd42977 100644
--- a/arch/arm/mach-spear/platsmp.c
+++ b/arch/arm/mach-spear/platsmp.c
@@ -20,6 +20,18 @@
#include <mach/spear.h>
#include "generic.h"
+/*
+ * Write pen_release in a way that is guaranteed to be visible to all
+ * observers, irrespective of whether they're taking part in coherency
+ * or not. This is necessary for the hotplug code to work reliably.
+ */
+static void write_pen_release(int val)
+{
+ pen_release = val;
+ smp_wmb();
+ sync_cache_w(&pen_release);
+}
+
static DEFINE_SPINLOCK(boot_lock);
static void __iomem *scu_base = IOMEM(VA_SCU_BASE);
@@ -30,8 +42,7 @@
* let the primary processor know we're out of the
* pen, then head off into the C entry point
*/
- pen_release = -1;
- smp_wmb();
+ write_pen_release(-1);
/*
* Synchronise with the boot thread.
@@ -58,9 +69,7 @@
* Note that "pen_release" is the hardware CPU ID, whereas
* "cpu" is Linux's internal ID.
*/
- pen_release = cpu;
- flush_cache_all();
- outer_flush_all();
+ write_pen_release(cpu);
timeout = jiffies + (1 * HZ);
while (time_before(jiffies, timeout)) {
diff --git a/arch/arm/mach-spear/spear13xx.c b/arch/arm/mach-spear/spear13xx.c
index 7aa6e8cf..c9897ea 100644
--- a/arch/arm/mach-spear/spear13xx.c
+++ b/arch/arm/mach-spear/spear13xx.c
@@ -38,15 +38,15 @@
if (!IS_ENABLED(CONFIG_CACHE_L2X0))
return;
- writel_relaxed(0x06, VA_L2CC_BASE + L2X0_PREFETCH_CTRL);
+ writel_relaxed(0x06, VA_L2CC_BASE + L310_PREFETCH_CTRL);
/*
* Program following latencies in order to make
* SPEAr1340 work at 600 MHz
*/
- writel_relaxed(0x221, VA_L2CC_BASE + L2X0_TAG_LATENCY_CTRL);
- writel_relaxed(0x441, VA_L2CC_BASE + L2X0_DATA_LATENCY_CTRL);
- l2x0_init(VA_L2CC_BASE, 0x70A60001, 0xfe00ffff);
+ writel_relaxed(0x221, VA_L2CC_BASE + L310_TAG_LATENCY_CTRL);
+ writel_relaxed(0x441, VA_L2CC_BASE + L310_DATA_LATENCY_CTRL);
+ l2x0_init(VA_L2CC_BASE, 0x30a00001, 0xfe0fffff);
}
/*
diff --git a/arch/arm/mach-sti/board-dt.c b/arch/arm/mach-sti/board-dt.c
index df731f2..3cf6ef8 100644
--- a/arch/arm/mach-sti/board-dt.c
+++ b/arch/arm/mach-sti/board-dt.c
@@ -14,25 +14,6 @@
#include "smp.h"
-void __init stih41x_l2x0_init(void)
-{
- u32 way_size = 0x4;
- u32 aux_ctrl;
- /* may be this can be encoded in macros like BIT*() */
- aux_ctrl = (0x1 << L2X0_AUX_CTRL_SHARE_OVERRIDE_SHIFT) |
- (0x1 << L2X0_AUX_CTRL_DATA_PREFETCH_SHIFT) |
- (0x1 << L2X0_AUX_CTRL_INSTR_PREFETCH_SHIFT) |
- (way_size << L2X0_AUX_CTRL_WAY_SIZE_SHIFT);
-
- l2x0_of_init(aux_ctrl, L2X0_AUX_CTRL_MASK);
-}
-
-static void __init stih41x_machine_init(void)
-{
- stih41x_l2x0_init();
- of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
-}
-
static const char *stih41x_dt_match[] __initdata = {
"st,stih415",
"st,stih416",
@@ -41,7 +22,11 @@
};
DT_MACHINE_START(STM, "STiH415/416 SoC with Flattened Device Tree")
- .init_machine = stih41x_machine_init,
- .smp = smp_ops(sti_smp_ops),
.dt_compat = stih41x_dt_match,
+ .l2c_aux_val = L2C_AUX_CTRL_SHARED_OVERRIDE |
+ L310_AUX_CTRL_DATA_PREFETCH |
+ L310_AUX_CTRL_INSTR_PREFETCH |
+ L2C_AUX_CTRL_WAY_SIZE(4),
+ .l2c_aux_mask = 0xc0000fff,
+ .smp = smp_ops(sti_smp_ops),
MACHINE_END
diff --git a/arch/arm/mach-tegra/pm.h b/arch/arm/mach-tegra/pm.h
index 6e92a7c..f4a8969 100644
--- a/arch/arm/mach-tegra/pm.h
+++ b/arch/arm/mach-tegra/pm.h
@@ -35,8 +35,6 @@
void tegra30_lp1_iram_hook(void);
void tegra30_sleep_core_init(void);
-extern unsigned long l2x0_saved_regs_addr;
-
void tegra_clear_cpu_in_lp2(void);
bool tegra_set_cpu_in_lp2(void);
diff --git a/arch/arm/mach-tegra/reset-handler.S b/arch/arm/mach-tegra/reset-handler.S
index 8c1ba4fe..578d4d1a 100644
--- a/arch/arm/mach-tegra/reset-handler.S
+++ b/arch/arm/mach-tegra/reset-handler.S
@@ -19,7 +19,6 @@
#include <asm/cache.h>
#include <asm/asm-offsets.h>
-#include <asm/hardware/cache-l2x0.h>
#include "flowctrl.h"
#include "fuse.h"
@@ -78,8 +77,10 @@
str r1, [r0]
#endif
+#ifdef CONFIG_CACHE_L2X0
/* L2 cache resume & re-enable */
- l2_cache_resume r0, r1, r2, l2x0_saved_regs_addr
+ bl l2c310_early_resume
+#endif
end_ca9_scu_l2_resume:
mov32 r9, 0xc0f
cmp r8, r9
@@ -89,12 +90,6 @@
ENDPROC(tegra_resume)
#endif
-#ifdef CONFIG_CACHE_L2X0
- .globl l2x0_saved_regs_addr
-l2x0_saved_regs_addr:
- .long 0
-#endif
-
.align L1_CACHE_SHIFT
ENTRY(__tegra_cpu_reset_handler_start)
diff --git a/arch/arm/mach-tegra/sleep.h b/arch/arm/mach-tegra/sleep.h
index a4edbb3..339fe42 100644
--- a/arch/arm/mach-tegra/sleep.h
+++ b/arch/arm/mach-tegra/sleep.h
@@ -120,37 +120,6 @@
mov \tmp1, \tmp1, lsr #8
.endm
-/* Macro to resume & re-enable L2 cache */
-#ifndef L2X0_CTRL_EN
-#define L2X0_CTRL_EN 1
-#endif
-
-#ifdef CONFIG_CACHE_L2X0
-.macro l2_cache_resume, tmp1, tmp2, tmp3, phys_l2x0_saved_regs
- W(adr) \tmp1, \phys_l2x0_saved_regs
- ldr \tmp1, [\tmp1]
- ldr \tmp2, [\tmp1, #L2X0_R_PHY_BASE]
- ldr \tmp3, [\tmp2, #L2X0_CTRL]
- tst \tmp3, #L2X0_CTRL_EN
- bne exit_l2_resume
- ldr \tmp3, [\tmp1, #L2X0_R_TAG_LATENCY]
- str \tmp3, [\tmp2, #L2X0_TAG_LATENCY_CTRL]
- ldr \tmp3, [\tmp1, #L2X0_R_DATA_LATENCY]
- str \tmp3, [\tmp2, #L2X0_DATA_LATENCY_CTRL]
- ldr \tmp3, [\tmp1, #L2X0_R_PREFETCH_CTRL]
- str \tmp3, [\tmp2, #L2X0_PREFETCH_CTRL]
- ldr \tmp3, [\tmp1, #L2X0_R_PWR_CTRL]
- str \tmp3, [\tmp2, #L2X0_POWER_CTRL]
- ldr \tmp3, [\tmp1, #L2X0_R_AUX_CTRL]
- str \tmp3, [\tmp2, #L2X0_AUX_CTRL]
- mov \tmp3, #L2X0_CTRL_EN
- str \tmp3, [\tmp2, #L2X0_CTRL]
-exit_l2_resume:
-.endm
-#else /* CONFIG_CACHE_L2X0 */
-.macro l2_cache_resume, tmp1, tmp2, tmp3, phys_l2x0_saved_regs
-.endm
-#endif /* CONFIG_CACHE_L2X0 */
#else
void tegra_pen_lock(void);
void tegra_pen_unlock(void);
diff --git a/arch/arm/mach-tegra/tegra.c b/arch/arm/mach-tegra/tegra.c
index 6191603..15ac9fc 100644
--- a/arch/arm/mach-tegra/tegra.c
+++ b/arch/arm/mach-tegra/tegra.c
@@ -70,40 +70,12 @@
0,
};
-static void __init tegra_init_cache(void)
-{
-#ifdef CONFIG_CACHE_L2X0
- static const struct of_device_id pl310_ids[] __initconst = {
- { .compatible = "arm,pl310-cache", },
- {}
- };
-
- struct device_node *np;
- int ret;
- void __iomem *p = IO_ADDRESS(TEGRA_ARM_PERIF_BASE) + 0x3000;
- u32 aux_ctrl, cache_type;
-
- np = of_find_matching_node(NULL, pl310_ids);
- if (!np)
- return;
-
- cache_type = readl(p + L2X0_CACHE_TYPE);
- aux_ctrl = (cache_type & 0x700) << (17-8);
- aux_ctrl |= 0x7C400001;
-
- ret = l2x0_of_init(aux_ctrl, 0x8200c3fe);
- if (!ret)
- l2x0_saved_regs_addr = virt_to_phys(&l2x0_saved_regs);
-#endif
-}
-
static void __init tegra_init_early(void)
{
of_register_trusted_foundations();
tegra_apb_io_init();
tegra_init_fuse();
tegra_cpu_reset_handler_init();
- tegra_init_cache();
tegra_powergate_init();
tegra_hotplug_init();
}
@@ -191,8 +163,10 @@
};
DT_MACHINE_START(TEGRA_DT, "NVIDIA Tegra SoC (Flattened Device Tree)")
- .map_io = tegra_map_common_io,
+ .l2c_aux_val = 0x3c400001,
+ .l2c_aux_mask = 0xc20fc3fe,
.smp = smp_ops(tegra_smp_ops),
+ .map_io = tegra_map_common_io,
.init_early = tegra_init_early,
.init_irq = tegra_dt_init_irq,
.init_machine = tegra_dt_init,
diff --git a/arch/arm/mach-ux500/cache-l2x0.c b/arch/arm/mach-ux500/cache-l2x0.c
index 264f894..842ebed 100644
--- a/arch/arm/mach-ux500/cache-l2x0.c
+++ b/arch/arm/mach-ux500/cache-l2x0.c
@@ -35,10 +35,16 @@
return 0;
}
+static void ux500_l2c310_write_sec(unsigned long val, unsigned reg)
+{
+ /*
+ * We can't write to secure registers as we are in non-secure
+ * mode, until we have some SMI service available.
+ */
+}
+
static int __init ux500_l2x0_init(void)
{
- u32 aux_val = 0x3e000000;
-
if (cpu_is_u8500_family() || cpu_is_ux540_family())
l2x0_base = __io_address(U8500_L2CC_BASE);
else
@@ -48,28 +54,12 @@
/* Unlock before init */
ux500_l2x0_unlock();
- /* DBx540's L2 has 128KB way size */
- if (cpu_is_ux540_family())
- /* 128KB way size */
- aux_val |= (0x4 << L2X0_AUX_CTRL_WAY_SIZE_SHIFT);
- else
- /* 64KB way size */
- aux_val |= (0x3 << L2X0_AUX_CTRL_WAY_SIZE_SHIFT);
+ outer_cache.write_sec = ux500_l2c310_write_sec;
- /* 64KB way size, 8 way associativity, force WA */
if (of_have_populated_dt())
- l2x0_of_init(aux_val, 0xc0000fff);
+ l2x0_of_init(0, ~0);
else
- l2x0_init(l2x0_base, aux_val, 0xc0000fff);
-
- /*
- * We can't disable l2 as we are in non secure mode, currently
- * this seems be called only during kexec path. So let's
- * override outer.disable with nasty assignment until we have
- * some SMI service available.
- */
- outer_cache.disable = NULL;
- outer_cache.set_debug = NULL;
+ l2x0_init(l2x0_base, 0, ~0);
return 0;
}
diff --git a/arch/arm/mach-vexpress/ct-ca9x4.c b/arch/arm/mach-vexpress/ct-ca9x4.c
index 494d70b..86150d7 100644
--- a/arch/arm/mach-vexpress/ct-ca9x4.c
+++ b/arch/arm/mach-vexpress/ct-ca9x4.c
@@ -45,6 +45,23 @@
iotable_init(ct_ca9x4_io_desc, ARRAY_SIZE(ct_ca9x4_io_desc));
}
+static void __init ca9x4_l2_init(void)
+{
+#ifdef CONFIG_CACHE_L2X0
+ void __iomem *l2x0_base = ioremap(CT_CA9X4_L2CC, SZ_4K);
+
+ if (l2x0_base) {
+ /* set RAM latencies to 1 cycle for this core tile. */
+ writel(0, l2x0_base + L310_TAG_LATENCY_CTRL);
+ writel(0, l2x0_base + L310_DATA_LATENCY_CTRL);
+
+ l2x0_init(l2x0_base, 0x00400000, 0xfe0fffff);
+ } else {
+ pr_err("L2C: unable to map L2 cache controller\n");
+ }
+#endif
+}
+
#ifdef CONFIG_HAVE_ARM_TWD
static DEFINE_TWD_LOCAL_TIMER(twd_local_timer, A9_MPCORE_TWD, IRQ_LOCALTIMER);
@@ -63,6 +80,7 @@
gic_init(0, 29, ioremap(A9_MPCORE_GIC_DIST, SZ_4K),
ioremap(A9_MPCORE_GIC_CPU, SZ_256));
ca9x4_twd_init();
+ ca9x4_l2_init();
}
static int ct_ca9x4_clcd_setup(struct clcd_fb *fb)
@@ -146,16 +164,6 @@
{
int i;
-#ifdef CONFIG_CACHE_L2X0
- void __iomem *l2x0_base = ioremap(CT_CA9X4_L2CC, SZ_4K);
-
- /* set RAM latencies to 1 cycle for this core tile. */
- writel(0, l2x0_base + L2X0_TAG_LATENCY_CTRL);
- writel(0, l2x0_base + L2X0_DATA_LATENCY_CTRL);
-
- l2x0_init(l2x0_base, 0x00400000, 0xfe0fffff);
-#endif
-
for (i = 0; i < ARRAY_SIZE(ct_ca9x4_amba_devs); i++)
amba_device_register(ct_ca9x4_amba_devs[i], &iomem_resource);
diff --git a/arch/arm/mach-vexpress/tc2_pm.c b/arch/arm/mach-vexpress/tc2_pm.c
index 29e7785..b743a0a 100644
--- a/arch/arm/mach-vexpress/tc2_pm.c
+++ b/arch/arm/mach-vexpress/tc2_pm.c
@@ -209,7 +209,7 @@
#define POLL_MSEC 10
#define TIMEOUT_MSEC 1000
-static int tc2_pm_power_down_finish(unsigned int cpu, unsigned int cluster)
+static int tc2_pm_wait_for_powerdown(unsigned int cpu, unsigned int cluster)
{
unsigned tries;
@@ -290,7 +290,7 @@
static const struct mcpm_platform_ops tc2_pm_power_ops = {
.power_up = tc2_pm_power_up,
.power_down = tc2_pm_power_down,
- .power_down_finish = tc2_pm_power_down_finish,
+ .wait_for_powerdown = tc2_pm_wait_for_powerdown,
.suspend = tc2_pm_suspend,
.powered_up = tc2_pm_powered_up,
};
diff --git a/arch/arm/mach-vexpress/v2m.c b/arch/arm/mach-vexpress/v2m.c
index 38f4f6f..6ff681a 100644
--- a/arch/arm/mach-vexpress/v2m.c
+++ b/arch/arm/mach-vexpress/v2m.c
@@ -372,7 +372,6 @@
static void __init v2m_dt_init(void)
{
- l2x0_of_init(0x00400000, 0xfe0fffff);
of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
}
@@ -383,6 +382,8 @@
DT_MACHINE_START(VEXPRESS_DT, "ARM-Versatile Express")
.dt_compat = v2m_dt_match,
+ .l2c_aux_val = 0x00400000,
+ .l2c_aux_mask = 0xfe0fffff,
.smp = smp_ops(vexpress_smp_dt_ops),
.smp_init = smp_init_ops(vexpress_smp_init_ops),
.init_machine = v2m_dt_init,
diff --git a/arch/arm/mach-zynq/common.c b/arch/arm/mach-zynq/common.c
index edbd9d8..31a6fa4 100644
--- a/arch/arm/mach-zynq/common.c
+++ b/arch/arm/mach-zynq/common.c
@@ -109,11 +109,6 @@
struct soc_device *soc_dev;
struct device *parent = NULL;
- /*
- * 64KB way size, 8-way associativity, parity disabled
- */
- l2x0_of_init(0x02060000, 0xF0F0FFFF);
-
soc_dev_attr = kzalloc(sizeof(*soc_dev_attr), GFP_KERNEL);
if (!soc_dev_attr)
goto out;
@@ -202,6 +197,9 @@
};
DT_MACHINE_START(XILINX_EP107, "Xilinx Zynq Platform")
+ /* 64KB way size, 8-way associativity, parity disabled */
+ .l2c_aux_val = 0x02000000,
+ .l2c_aux_mask = 0xf0ffffff,
.smp = smp_ops(zynq_smp_ops),
.map_io = zynq_map_io,
.init_irq = zynq_irq_init,
diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig
index 5bf7c3c..eda0dd0 100644
--- a/arch/arm/mm/Kconfig
+++ b/arch/arm/mm/Kconfig
@@ -897,6 +897,57 @@
This option enables optimisations for the PL310 cache
controller.
+config PL310_ERRATA_588369
+ bool "PL310 errata: Clean & Invalidate maintenance operations do not invalidate clean lines"
+ depends on CACHE_L2X0
+ help
+ The PL310 L2 cache controller implements three types of Clean &
+ Invalidate maintenance operations: by Physical Address
+ (offset 0x7F0), by Index/Way (0x7F8) and by Way (0x7FC).
+ They are architecturally defined to behave as the execution of a
+ clean operation followed immediately by an invalidate operation,
+ both performing to the same memory location. This functionality
+ is not correctly implemented in PL310 as clean lines are not
+ invalidated as a result of these operations.
+
+config PL310_ERRATA_727915
+ bool "PL310 errata: Background Clean & Invalidate by Way operation can cause data corruption"
+ depends on CACHE_L2X0
+ help
+ PL310 implements the Clean & Invalidate by Way L2 cache maintenance
+ operation (offset 0x7FC). This operation runs in background so that
+ PL310 can handle normal accesses while it is in progress. Under very
+ rare circumstances, due to this erratum, write data can be lost when
+ PL310 treats a cacheable write transaction during a Clean &
+ Invalidate by Way operation.
+
+config PL310_ERRATA_753970
+ bool "PL310 errata: cache sync operation may be faulty"
+ depends on CACHE_PL310
+ help
+ This option enables the workaround for the 753970 PL310 (r3p0) erratum.
+
+ Under some condition the effect of cache sync operation on
+ the store buffer still remains when the operation completes.
+ This means that the store buffer is always asked to drain and
+ this prevents it from merging any further writes. The workaround
+ is to replace the normal offset of cache sync operation (0x730)
+ by another offset targeting an unmapped PL310 register 0x740.
+ This has the same effect as the cache sync operation: store buffer
+ drain and waiting for all buffers empty.
+
+config PL310_ERRATA_769419
+ bool "PL310 errata: no automatic Store Buffer drain"
+ depends on CACHE_L2X0
+ help
+ On revisions of the PL310 prior to r3p2, the Store Buffer does
+ not automatically drain. This can cause normal, non-cacheable
+ writes to be retained when the memory system is idle, leading
+ to suboptimal I/O performance for drivers using coherent DMA.
+ This option adds a write barrier to the cpu_idle loop so that,
+ on systems with an outer cache, the store buffer is drained
+ explicitly.
+
config CACHE_TAUROS2
bool "Enable the Tauros2 L2 cache controller"
depends on (ARCH_DOVE || ARCH_MMP || CPU_PJ4)
diff --git a/arch/arm/mm/Makefile b/arch/arm/mm/Makefile
index 7f39ce2..91da64d 100644
--- a/arch/arm/mm/Makefile
+++ b/arch/arm/mm/Makefile
@@ -95,7 +95,8 @@
AFLAGS_proc-v6.o :=-Wa,-march=armv6
AFLAGS_proc-v7.o :=-Wa,-march=armv7-a
+obj-$(CONFIG_OUTER_CACHE) += l2c-common.o
obj-$(CONFIG_CACHE_FEROCEON_L2) += cache-feroceon-l2.o
-obj-$(CONFIG_CACHE_L2X0) += cache-l2x0.o
+obj-$(CONFIG_CACHE_L2X0) += cache-l2x0.o l2c-l2x0-resume.o
obj-$(CONFIG_CACHE_XSC3L2) += cache-xsc3l2.o
obj-$(CONFIG_CACHE_TAUROS2) += cache-tauros2.o
diff --git a/arch/arm/mm/alignment.c b/arch/arm/mm/alignment.c
index 9240364..b8cb1a2 100644
--- a/arch/arm/mm/alignment.c
+++ b/arch/arm/mm/alignment.c
@@ -28,6 +28,7 @@
#include <asm/opcodes.h>
#include "fault.h"
+#include "mm.h"
/*
* 32-bit misaligned trap handler (c) 1998 San Mehat (CCC) -July 1998
@@ -81,6 +82,7 @@
static unsigned long ai_dword;
static unsigned long ai_multi;
static int ai_usermode;
+static unsigned long cr_no_alignment;
core_param(alignment, ai_usermode, int, 0600);
@@ -91,7 +93,7 @@
/* Return true if and only if the ARMv6 unaligned access model is in use. */
static bool cpu_is_v6_unaligned(void)
{
- return cpu_architecture() >= CPU_ARCH_ARMv6 && (cr_alignment & CR_U);
+ return cpu_architecture() >= CPU_ARCH_ARMv6 && get_cr() & CR_U;
}
static int safe_usermode(int new_usermode, bool warn)
@@ -949,6 +951,13 @@
return 0;
}
+static int __init noalign_setup(char *__unused)
+{
+ set_cr(__clear_cr(CR_A));
+ return 1;
+}
+__setup("noalign", noalign_setup);
+
/*
* This needs to be done after sysctl_init, otherwise sys/ will be
* overwritten. Actually, this shouldn't be in sys/ at all since
@@ -966,14 +975,12 @@
return -ENOMEM;
#endif
-#ifdef CONFIG_CPU_CP15
if (cpu_is_v6_unaligned()) {
- cr_alignment &= ~CR_A;
- cr_no_alignment &= ~CR_A;
- set_cr(cr_alignment);
+ set_cr(__clear_cr(CR_A));
ai_usermode = safe_usermode(ai_usermode, false);
}
-#endif
+
+ cr_no_alignment = get_cr() & ~CR_A;
hook_fault_code(FAULT_CODE_ALIGNMENT, do_alignment, SIGBUS, BUS_ADRALN,
"alignment exception");
diff --git a/arch/arm/mm/cache-feroceon-l2.c b/arch/arm/mm/cache-feroceon-l2.c
index dc814a5..e028a7f 100644
--- a/arch/arm/mm/cache-feroceon-l2.c
+++ b/arch/arm/mm/cache-feroceon-l2.c
@@ -350,7 +350,6 @@
outer_cache.inv_range = feroceon_l2_inv_range;
outer_cache.clean_range = feroceon_l2_clean_range;
outer_cache.flush_range = feroceon_l2_flush_range;
- outer_cache.inv_all = l2_inv_all;
enable_l2();
diff --git a/arch/arm/mm/cache-l2x0.c b/arch/arm/mm/cache-l2x0.c
index 7abde2c..efc5cab 100644
--- a/arch/arm/mm/cache-l2x0.c
+++ b/arch/arm/mm/cache-l2x0.c
@@ -16,18 +16,33 @@
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
+#include <linux/cpu.h>
#include <linux/err.h>
#include <linux/init.h>
+#include <linux/smp.h>
#include <linux/spinlock.h>
#include <linux/io.h>
#include <linux/of.h>
#include <linux/of_address.h>
#include <asm/cacheflush.h>
+#include <asm/cp15.h>
+#include <asm/cputype.h>
#include <asm/hardware/cache-l2x0.h>
#include "cache-tauros3.h"
#include "cache-aurora-l2.h"
+struct l2c_init_data {
+ const char *type;
+ unsigned way_size_0;
+ unsigned num_lock;
+ void (*of_parse)(const struct device_node *, u32 *, u32 *);
+ void (*enable)(void __iomem *, u32, unsigned);
+ void (*fixup)(void __iomem *, u32, struct outer_cache_fns *);
+ void (*save)(void __iomem *);
+ struct outer_cache_fns outer_cache;
+};
+
#define CACHE_LINE_SIZE 32
static void __iomem *l2x0_base;
@@ -36,34 +51,97 @@
static u32 l2x0_size;
static unsigned long sync_reg_offset = L2X0_CACHE_SYNC;
-/* Aurora don't have the cache ID register available, so we have to
- * pass it though the device tree */
-static u32 cache_id_part_number_from_dt;
-
struct l2x0_regs l2x0_saved_regs;
-struct l2x0_of_data {
- void (*setup)(const struct device_node *, u32 *, u32 *);
- void (*save)(void);
- struct outer_cache_fns outer_cache;
-};
-
-static bool of_init = false;
-
-static inline void cache_wait_way(void __iomem *reg, unsigned long mask)
+/*
+ * Common code for all cache controllers.
+ */
+static inline void l2c_wait_mask(void __iomem *reg, unsigned long mask)
{
/* wait for cache operation by line or way to complete */
while (readl_relaxed(reg) & mask)
cpu_relax();
}
+/*
+ * By default, we write directly to secure registers. Platforms must
+ * override this if they are running non-secure.
+ */
+static void l2c_write_sec(unsigned long val, void __iomem *base, unsigned reg)
+{
+ if (val == readl_relaxed(base + reg))
+ return;
+ if (outer_cache.write_sec)
+ outer_cache.write_sec(val, reg);
+ else
+ writel_relaxed(val, base + reg);
+}
+
+/*
+ * This should only be called when we have a requirement that the
+ * register be written due to a work-around, as platforms running
+ * in non-secure mode may not be able to access this register.
+ */
+static inline void l2c_set_debug(void __iomem *base, unsigned long val)
+{
+ l2c_write_sec(val, base, L2X0_DEBUG_CTRL);
+}
+
+static void __l2c_op_way(void __iomem *reg)
+{
+ writel_relaxed(l2x0_way_mask, reg);
+ l2c_wait_mask(reg, l2x0_way_mask);
+}
+
+static inline void l2c_unlock(void __iomem *base, unsigned num)
+{
+ unsigned i;
+
+ for (i = 0; i < num; i++) {
+ writel_relaxed(0, base + L2X0_LOCKDOWN_WAY_D_BASE +
+ i * L2X0_LOCKDOWN_STRIDE);
+ writel_relaxed(0, base + L2X0_LOCKDOWN_WAY_I_BASE +
+ i * L2X0_LOCKDOWN_STRIDE);
+ }
+}
+
+/*
+ * Enable the L2 cache controller. This function must only be
+ * called when the cache controller is known to be disabled.
+ */
+static void l2c_enable(void __iomem *base, u32 aux, unsigned num_lock)
+{
+ unsigned long flags;
+
+ l2c_write_sec(aux, base, L2X0_AUX_CTRL);
+
+ l2c_unlock(base, num_lock);
+
+ local_irq_save(flags);
+ __l2c_op_way(base + L2X0_INV_WAY);
+ writel_relaxed(0, base + sync_reg_offset);
+ l2c_wait_mask(base + sync_reg_offset, 1);
+ local_irq_restore(flags);
+
+ l2c_write_sec(L2X0_CTRL_EN, base, L2X0_CTRL);
+}
+
+static void l2c_disable(void)
+{
+ void __iomem *base = l2x0_base;
+
+ outer_cache.flush_all();
+ l2c_write_sec(0, base, L2X0_CTRL);
+ dsb(st);
+}
+
#ifdef CONFIG_CACHE_PL310
static inline void cache_wait(void __iomem *reg, unsigned long mask)
{
/* cache operations by line are atomic on PL310 */
}
#else
-#define cache_wait cache_wait_way
+#define cache_wait l2c_wait_mask
#endif
static inline void cache_sync(void)
@@ -74,59 +152,16 @@
cache_wait(base + L2X0_CACHE_SYNC, 1);
}
-static inline void l2x0_clean_line(unsigned long addr)
-{
- void __iomem *base = l2x0_base;
- cache_wait(base + L2X0_CLEAN_LINE_PA, 1);
- writel_relaxed(addr, base + L2X0_CLEAN_LINE_PA);
-}
-
-static inline void l2x0_inv_line(unsigned long addr)
-{
- void __iomem *base = l2x0_base;
- cache_wait(base + L2X0_INV_LINE_PA, 1);
- writel_relaxed(addr, base + L2X0_INV_LINE_PA);
-}
-
#if defined(CONFIG_PL310_ERRATA_588369) || defined(CONFIG_PL310_ERRATA_727915)
static inline void debug_writel(unsigned long val)
{
- if (outer_cache.set_debug)
- outer_cache.set_debug(val);
-}
-
-static void pl310_set_debug(unsigned long val)
-{
- writel_relaxed(val, l2x0_base + L2X0_DEBUG_CTRL);
+ l2c_set_debug(l2x0_base, val);
}
#else
/* Optimised out for non-errata case */
static inline void debug_writel(unsigned long val)
{
}
-
-#define pl310_set_debug NULL
-#endif
-
-#ifdef CONFIG_PL310_ERRATA_588369
-static inline void l2x0_flush_line(unsigned long addr)
-{
- void __iomem *base = l2x0_base;
-
- /* Clean by PA followed by Invalidate by PA */
- cache_wait(base + L2X0_CLEAN_LINE_PA, 1);
- writel_relaxed(addr, base + L2X0_CLEAN_LINE_PA);
- cache_wait(base + L2X0_INV_LINE_PA, 1);
- writel_relaxed(addr, base + L2X0_INV_LINE_PA);
-}
-#else
-
-static inline void l2x0_flush_line(unsigned long addr)
-{
- void __iomem *base = l2x0_base;
- cache_wait(base + L2X0_CLEAN_INV_LINE_PA, 1);
- writel_relaxed(addr, base + L2X0_CLEAN_INV_LINE_PA);
-}
#endif
static void l2x0_cache_sync(void)
@@ -141,8 +176,7 @@
static void __l2x0_flush_all(void)
{
debug_writel(0x03);
- writel_relaxed(l2x0_way_mask, l2x0_base + L2X0_CLEAN_INV_WAY);
- cache_wait_way(l2x0_base + L2X0_CLEAN_INV_WAY, l2x0_way_mask);
+ __l2c_op_way(l2x0_base + L2X0_CLEAN_INV_WAY);
cache_sync();
debug_writel(0x00);
}
@@ -157,275 +191,883 @@
raw_spin_unlock_irqrestore(&l2x0_lock, flags);
}
-static void l2x0_clean_all(void)
-{
- unsigned long flags;
-
- /* clean all ways */
- raw_spin_lock_irqsave(&l2x0_lock, flags);
- writel_relaxed(l2x0_way_mask, l2x0_base + L2X0_CLEAN_WAY);
- cache_wait_way(l2x0_base + L2X0_CLEAN_WAY, l2x0_way_mask);
- cache_sync();
- raw_spin_unlock_irqrestore(&l2x0_lock, flags);
-}
-
-static void l2x0_inv_all(void)
-{
- unsigned long flags;
-
- /* invalidate all ways */
- raw_spin_lock_irqsave(&l2x0_lock, flags);
- /* Invalidating when L2 is enabled is a nono */
- BUG_ON(readl(l2x0_base + L2X0_CTRL) & L2X0_CTRL_EN);
- writel_relaxed(l2x0_way_mask, l2x0_base + L2X0_INV_WAY);
- cache_wait_way(l2x0_base + L2X0_INV_WAY, l2x0_way_mask);
- cache_sync();
- raw_spin_unlock_irqrestore(&l2x0_lock, flags);
-}
-
-static void l2x0_inv_range(unsigned long start, unsigned long end)
-{
- void __iomem *base = l2x0_base;
- unsigned long flags;
-
- raw_spin_lock_irqsave(&l2x0_lock, flags);
- if (start & (CACHE_LINE_SIZE - 1)) {
- start &= ~(CACHE_LINE_SIZE - 1);
- debug_writel(0x03);
- l2x0_flush_line(start);
- debug_writel(0x00);
- start += CACHE_LINE_SIZE;
- }
-
- if (end & (CACHE_LINE_SIZE - 1)) {
- end &= ~(CACHE_LINE_SIZE - 1);
- debug_writel(0x03);
- l2x0_flush_line(end);
- debug_writel(0x00);
- }
-
- while (start < end) {
- unsigned long blk_end = start + min(end - start, 4096UL);
-
- while (start < blk_end) {
- l2x0_inv_line(start);
- start += CACHE_LINE_SIZE;
- }
-
- if (blk_end < end) {
- raw_spin_unlock_irqrestore(&l2x0_lock, flags);
- raw_spin_lock_irqsave(&l2x0_lock, flags);
- }
- }
- cache_wait(base + L2X0_INV_LINE_PA, 1);
- cache_sync();
- raw_spin_unlock_irqrestore(&l2x0_lock, flags);
-}
-
-static void l2x0_clean_range(unsigned long start, unsigned long end)
-{
- void __iomem *base = l2x0_base;
- unsigned long flags;
-
- if ((end - start) >= l2x0_size) {
- l2x0_clean_all();
- return;
- }
-
- raw_spin_lock_irqsave(&l2x0_lock, flags);
- start &= ~(CACHE_LINE_SIZE - 1);
- while (start < end) {
- unsigned long blk_end = start + min(end - start, 4096UL);
-
- while (start < blk_end) {
- l2x0_clean_line(start);
- start += CACHE_LINE_SIZE;
- }
-
- if (blk_end < end) {
- raw_spin_unlock_irqrestore(&l2x0_lock, flags);
- raw_spin_lock_irqsave(&l2x0_lock, flags);
- }
- }
- cache_wait(base + L2X0_CLEAN_LINE_PA, 1);
- cache_sync();
- raw_spin_unlock_irqrestore(&l2x0_lock, flags);
-}
-
-static void l2x0_flush_range(unsigned long start, unsigned long end)
-{
- void __iomem *base = l2x0_base;
- unsigned long flags;
-
- if ((end - start) >= l2x0_size) {
- l2x0_flush_all();
- return;
- }
-
- raw_spin_lock_irqsave(&l2x0_lock, flags);
- start &= ~(CACHE_LINE_SIZE - 1);
- while (start < end) {
- unsigned long blk_end = start + min(end - start, 4096UL);
-
- debug_writel(0x03);
- while (start < blk_end) {
- l2x0_flush_line(start);
- start += CACHE_LINE_SIZE;
- }
- debug_writel(0x00);
-
- if (blk_end < end) {
- raw_spin_unlock_irqrestore(&l2x0_lock, flags);
- raw_spin_lock_irqsave(&l2x0_lock, flags);
- }
- }
- cache_wait(base + L2X0_CLEAN_INV_LINE_PA, 1);
- cache_sync();
- raw_spin_unlock_irqrestore(&l2x0_lock, flags);
-}
-
static void l2x0_disable(void)
{
unsigned long flags;
raw_spin_lock_irqsave(&l2x0_lock, flags);
__l2x0_flush_all();
- writel_relaxed(0, l2x0_base + L2X0_CTRL);
+ l2c_write_sec(0, l2x0_base, L2X0_CTRL);
dsb(st);
raw_spin_unlock_irqrestore(&l2x0_lock, flags);
}
-static void l2x0_unlock(u32 cache_id)
+static void l2c_save(void __iomem *base)
{
- int lockregs;
- int i;
+ l2x0_saved_regs.aux_ctrl = readl_relaxed(l2x0_base + L2X0_AUX_CTRL);
+}
- switch (cache_id & L2X0_CACHE_ID_PART_MASK) {
- case L2X0_CACHE_ID_PART_L310:
- lockregs = 8;
- break;
- case AURORA_CACHE_ID:
- lockregs = 4;
- break;
- default:
- /* L210 and unknown types */
- lockregs = 1;
- break;
- }
+/*
+ * L2C-210 specific code.
+ *
+ * The L2C-2x0 PA, set/way and sync operations are atomic, but we must
+ * ensure that no background operation is running. The way operations
+ * are all background tasks.
+ *
+ * While a background operation is in progress, any new operation is
+ * ignored (unspecified whether this causes an error.) Thankfully, not
+ * used on SMP.
+ *
+ * Never has a different sync register other than L2X0_CACHE_SYNC, but
+ * we use sync_reg_offset here so we can share some of this with L2C-310.
+ */
+static void __l2c210_cache_sync(void __iomem *base)
+{
+ writel_relaxed(0, base + sync_reg_offset);
+}
- for (i = 0; i < lockregs; i++) {
- writel_relaxed(0x0, l2x0_base + L2X0_LOCKDOWN_WAY_D_BASE +
- i * L2X0_LOCKDOWN_STRIDE);
- writel_relaxed(0x0, l2x0_base + L2X0_LOCKDOWN_WAY_I_BASE +
- i * L2X0_LOCKDOWN_STRIDE);
+static void __l2c210_op_pa_range(void __iomem *reg, unsigned long start,
+ unsigned long end)
+{
+ while (start < end) {
+ writel_relaxed(start, reg);
+ start += CACHE_LINE_SIZE;
}
}
-void __init l2x0_init(void __iomem *base, u32 aux_val, u32 aux_mask)
+static void l2c210_inv_range(unsigned long start, unsigned long end)
{
- u32 aux;
- u32 cache_id;
- u32 way_size = 0;
- int ways;
- int way_size_shift = L2X0_WAY_SIZE_SHIFT;
- const char *type;
+ void __iomem *base = l2x0_base;
- l2x0_base = base;
- if (cache_id_part_number_from_dt)
- cache_id = cache_id_part_number_from_dt;
- else
- cache_id = readl_relaxed(l2x0_base + L2X0_CACHE_ID);
- aux = readl_relaxed(l2x0_base + L2X0_AUX_CTRL);
+ if (start & (CACHE_LINE_SIZE - 1)) {
+ start &= ~(CACHE_LINE_SIZE - 1);
+ writel_relaxed(start, base + L2X0_CLEAN_INV_LINE_PA);
+ start += CACHE_LINE_SIZE;
+ }
+ if (end & (CACHE_LINE_SIZE - 1)) {
+ end &= ~(CACHE_LINE_SIZE - 1);
+ writel_relaxed(end, base + L2X0_CLEAN_INV_LINE_PA);
+ }
+
+ __l2c210_op_pa_range(base + L2X0_INV_LINE_PA, start, end);
+ __l2c210_cache_sync(base);
+}
+
+static void l2c210_clean_range(unsigned long start, unsigned long end)
+{
+ void __iomem *base = l2x0_base;
+
+ start &= ~(CACHE_LINE_SIZE - 1);
+ __l2c210_op_pa_range(base + L2X0_CLEAN_LINE_PA, start, end);
+ __l2c210_cache_sync(base);
+}
+
+static void l2c210_flush_range(unsigned long start, unsigned long end)
+{
+ void __iomem *base = l2x0_base;
+
+ start &= ~(CACHE_LINE_SIZE - 1);
+ __l2c210_op_pa_range(base + L2X0_CLEAN_INV_LINE_PA, start, end);
+ __l2c210_cache_sync(base);
+}
+
+static void l2c210_flush_all(void)
+{
+ void __iomem *base = l2x0_base;
+
+ BUG_ON(!irqs_disabled());
+
+ __l2c_op_way(base + L2X0_CLEAN_INV_WAY);
+ __l2c210_cache_sync(base);
+}
+
+static void l2c210_sync(void)
+{
+ __l2c210_cache_sync(l2x0_base);
+}
+
+static void l2c210_resume(void)
+{
+ void __iomem *base = l2x0_base;
+
+ if (!(readl_relaxed(base + L2X0_CTRL) & L2X0_CTRL_EN))
+ l2c_enable(base, l2x0_saved_regs.aux_ctrl, 1);
+}
+
+static const struct l2c_init_data l2c210_data __initconst = {
+ .type = "L2C-210",
+ .way_size_0 = SZ_8K,
+ .num_lock = 1,
+ .enable = l2c_enable,
+ .save = l2c_save,
+ .outer_cache = {
+ .inv_range = l2c210_inv_range,
+ .clean_range = l2c210_clean_range,
+ .flush_range = l2c210_flush_range,
+ .flush_all = l2c210_flush_all,
+ .disable = l2c_disable,
+ .sync = l2c210_sync,
+ .resume = l2c210_resume,
+ },
+};
+
+/*
+ * L2C-220 specific code.
+ *
+ * All operations are background operations: they have to be waited for.
+ * Conflicting requests generate a slave error (which will cause an
+ * imprecise abort.) Never uses sync_reg_offset, so we hard-code the
+ * sync register here.
+ *
+ * However, we can re-use the l2c210_resume call.
+ */
+static inline void __l2c220_cache_sync(void __iomem *base)
+{
+ writel_relaxed(0, base + L2X0_CACHE_SYNC);
+ l2c_wait_mask(base + L2X0_CACHE_SYNC, 1);
+}
+
+static void l2c220_op_way(void __iomem *base, unsigned reg)
+{
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&l2x0_lock, flags);
+ __l2c_op_way(base + reg);
+ __l2c220_cache_sync(base);
+ raw_spin_unlock_irqrestore(&l2x0_lock, flags);
+}
+
+static unsigned long l2c220_op_pa_range(void __iomem *reg, unsigned long start,
+ unsigned long end, unsigned long flags)
+{
+ raw_spinlock_t *lock = &l2x0_lock;
+
+ while (start < end) {
+ unsigned long blk_end = start + min(end - start, 4096UL);
+
+ while (start < blk_end) {
+ l2c_wait_mask(reg, 1);
+ writel_relaxed(start, reg);
+ start += CACHE_LINE_SIZE;
+ }
+
+ if (blk_end < end) {
+ raw_spin_unlock_irqrestore(lock, flags);
+ raw_spin_lock_irqsave(lock, flags);
+ }
+ }
+
+ return flags;
+}
+
+static void l2c220_inv_range(unsigned long start, unsigned long end)
+{
+ void __iomem *base = l2x0_base;
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&l2x0_lock, flags);
+ if ((start | end) & (CACHE_LINE_SIZE - 1)) {
+ if (start & (CACHE_LINE_SIZE - 1)) {
+ start &= ~(CACHE_LINE_SIZE - 1);
+ writel_relaxed(start, base + L2X0_CLEAN_INV_LINE_PA);
+ start += CACHE_LINE_SIZE;
+ }
+
+ if (end & (CACHE_LINE_SIZE - 1)) {
+ end &= ~(CACHE_LINE_SIZE - 1);
+ l2c_wait_mask(base + L2X0_CLEAN_INV_LINE_PA, 1);
+ writel_relaxed(end, base + L2X0_CLEAN_INV_LINE_PA);
+ }
+ }
+
+ flags = l2c220_op_pa_range(base + L2X0_INV_LINE_PA,
+ start, end, flags);
+ l2c_wait_mask(base + L2X0_INV_LINE_PA, 1);
+ __l2c220_cache_sync(base);
+ raw_spin_unlock_irqrestore(&l2x0_lock, flags);
+}
+
+static void l2c220_clean_range(unsigned long start, unsigned long end)
+{
+ void __iomem *base = l2x0_base;
+ unsigned long flags;
+
+ start &= ~(CACHE_LINE_SIZE - 1);
+ if ((end - start) >= l2x0_size) {
+ l2c220_op_way(base, L2X0_CLEAN_WAY);
+ return;
+ }
+
+ raw_spin_lock_irqsave(&l2x0_lock, flags);
+ flags = l2c220_op_pa_range(base + L2X0_CLEAN_LINE_PA,
+ start, end, flags);
+ l2c_wait_mask(base + L2X0_CLEAN_INV_LINE_PA, 1);
+ __l2c220_cache_sync(base);
+ raw_spin_unlock_irqrestore(&l2x0_lock, flags);
+}
+
+static void l2c220_flush_range(unsigned long start, unsigned long end)
+{
+ void __iomem *base = l2x0_base;
+ unsigned long flags;
+
+ start &= ~(CACHE_LINE_SIZE - 1);
+ if ((end - start) >= l2x0_size) {
+ l2c220_op_way(base, L2X0_CLEAN_INV_WAY);
+ return;
+ }
+
+ raw_spin_lock_irqsave(&l2x0_lock, flags);
+ flags = l2c220_op_pa_range(base + L2X0_CLEAN_INV_LINE_PA,
+ start, end, flags);
+ l2c_wait_mask(base + L2X0_CLEAN_INV_LINE_PA, 1);
+ __l2c220_cache_sync(base);
+ raw_spin_unlock_irqrestore(&l2x0_lock, flags);
+}
+
+static void l2c220_flush_all(void)
+{
+ l2c220_op_way(l2x0_base, L2X0_CLEAN_INV_WAY);
+}
+
+static void l2c220_sync(void)
+{
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&l2x0_lock, flags);
+ __l2c220_cache_sync(l2x0_base);
+ raw_spin_unlock_irqrestore(&l2x0_lock, flags);
+}
+
+static void l2c220_enable(void __iomem *base, u32 aux, unsigned num_lock)
+{
+ /*
+ * Always enable non-secure access to the lockdown registers -
+ * we write to them as part of the L2C enable sequence so they
+ * need to be accessible.
+ */
+ aux |= L220_AUX_CTRL_NS_LOCKDOWN;
+
+ l2c_enable(base, aux, num_lock);
+}
+
+static const struct l2c_init_data l2c220_data = {
+ .type = "L2C-220",
+ .way_size_0 = SZ_8K,
+ .num_lock = 1,
+ .enable = l2c220_enable,
+ .save = l2c_save,
+ .outer_cache = {
+ .inv_range = l2c220_inv_range,
+ .clean_range = l2c220_clean_range,
+ .flush_range = l2c220_flush_range,
+ .flush_all = l2c220_flush_all,
+ .disable = l2c_disable,
+ .sync = l2c220_sync,
+ .resume = l2c210_resume,
+ },
+};
+
+/*
+ * L2C-310 specific code.
+ *
+ * Very similar to L2C-210, the PA, set/way and sync operations are atomic,
+ * and the way operations are all background tasks. However, issuing an
+ * operation while a background operation is in progress results in a
+ * SLVERR response. We can reuse:
+ *
+ * __l2c210_cache_sync (using sync_reg_offset)
+ * l2c210_sync
+ * l2c210_inv_range (if 588369 is not applicable)
+ * l2c210_clean_range
+ * l2c210_flush_range (if 588369 is not applicable)
+ * l2c210_flush_all (if 727915 is not applicable)
+ *
+ * Errata:
+ * 588369: PL310 R0P0->R1P0, fixed R2P0.
+ * Affects: all clean+invalidate operations
+ * clean and invalidate skips the invalidate step, so we need to issue
+ * separate operations. We also require the above debug workaround
+ * enclosing this code fragment on affected parts. On unaffected parts,
+ * we must not use this workaround without the debug register writes
+ * to avoid exposing a problem similar to 727915.
+ *
+ * 727915: PL310 R2P0->R3P0, fixed R3P1.
+ * Affects: clean+invalidate by way
+ * clean and invalidate by way runs in the background, and a store can
+ * hit the line between the clean operation and invalidate operation,
+ * resulting in the store being lost.
+ *
+ * 752271: PL310 R3P0->R3P1-50REL0, fixed R3P2.
+ * Affects: 8x64-bit (double fill) line fetches
+ * double fill line fetches can fail to cause dirty data to be evicted
+ * from the cache before the new data overwrites the second line.
+ *
+ * 753970: PL310 R3P0, fixed R3P1.
+ * Affects: sync
+ * prevents merging writes after the sync operation, until another L2C
+ * operation is performed (or a number of other conditions.)
+ *
+ * 769419: PL310 R0P0->R3P1, fixed R3P2.
+ * Affects: store buffer
+ * store buffer is not automatically drained.
+ */
+static void l2c310_inv_range_erratum(unsigned long start, unsigned long end)
+{
+ void __iomem *base = l2x0_base;
+
+ if ((start | end) & (CACHE_LINE_SIZE - 1)) {
+ unsigned long flags;
+
+ /* Erratum 588369 for both clean+invalidate operations */
+ raw_spin_lock_irqsave(&l2x0_lock, flags);
+ l2c_set_debug(base, 0x03);
+
+ if (start & (CACHE_LINE_SIZE - 1)) {
+ start &= ~(CACHE_LINE_SIZE - 1);
+ writel_relaxed(start, base + L2X0_CLEAN_LINE_PA);
+ writel_relaxed(start, base + L2X0_INV_LINE_PA);
+ start += CACHE_LINE_SIZE;
+ }
+
+ if (end & (CACHE_LINE_SIZE - 1)) {
+ end &= ~(CACHE_LINE_SIZE - 1);
+ writel_relaxed(end, base + L2X0_CLEAN_LINE_PA);
+ writel_relaxed(end, base + L2X0_INV_LINE_PA);
+ }
+
+ l2c_set_debug(base, 0x00);
+ raw_spin_unlock_irqrestore(&l2x0_lock, flags);
+ }
+
+ __l2c210_op_pa_range(base + L2X0_INV_LINE_PA, start, end);
+ __l2c210_cache_sync(base);
+}
+
+static void l2c310_flush_range_erratum(unsigned long start, unsigned long end)
+{
+ raw_spinlock_t *lock = &l2x0_lock;
+ unsigned long flags;
+ void __iomem *base = l2x0_base;
+
+ raw_spin_lock_irqsave(lock, flags);
+ while (start < end) {
+ unsigned long blk_end = start + min(end - start, 4096UL);
+
+ l2c_set_debug(base, 0x03);
+ while (start < blk_end) {
+ writel_relaxed(start, base + L2X0_CLEAN_LINE_PA);
+ writel_relaxed(start, base + L2X0_INV_LINE_PA);
+ start += CACHE_LINE_SIZE;
+ }
+ l2c_set_debug(base, 0x00);
+
+ if (blk_end < end) {
+ raw_spin_unlock_irqrestore(lock, flags);
+ raw_spin_lock_irqsave(lock, flags);
+ }
+ }
+ raw_spin_unlock_irqrestore(lock, flags);
+ __l2c210_cache_sync(base);
+}
+
+static void l2c310_flush_all_erratum(void)
+{
+ void __iomem *base = l2x0_base;
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&l2x0_lock, flags);
+ l2c_set_debug(base, 0x03);
+ __l2c_op_way(base + L2X0_CLEAN_INV_WAY);
+ l2c_set_debug(base, 0x00);
+ __l2c210_cache_sync(base);
+ raw_spin_unlock_irqrestore(&l2x0_lock, flags);
+}
+
+static void __init l2c310_save(void __iomem *base)
+{
+ unsigned revision;
+
+ l2c_save(base);
+
+ l2x0_saved_regs.tag_latency = readl_relaxed(base +
+ L310_TAG_LATENCY_CTRL);
+ l2x0_saved_regs.data_latency = readl_relaxed(base +
+ L310_DATA_LATENCY_CTRL);
+ l2x0_saved_regs.filter_end = readl_relaxed(base +
+ L310_ADDR_FILTER_END);
+ l2x0_saved_regs.filter_start = readl_relaxed(base +
+ L310_ADDR_FILTER_START);
+
+ revision = readl_relaxed(base + L2X0_CACHE_ID) &
+ L2X0_CACHE_ID_RTL_MASK;
+
+ /* From r2p0, there is Prefetch offset/control register */
+ if (revision >= L310_CACHE_ID_RTL_R2P0)
+ l2x0_saved_regs.prefetch_ctrl = readl_relaxed(base +
+ L310_PREFETCH_CTRL);
+
+ /* From r3p0, there is Power control register */
+ if (revision >= L310_CACHE_ID_RTL_R3P0)
+ l2x0_saved_regs.pwr_ctrl = readl_relaxed(base +
+ L310_POWER_CTRL);
+}
+
+static void l2c310_resume(void)
+{
+ void __iomem *base = l2x0_base;
+
+ if (!(readl_relaxed(base + L2X0_CTRL) & L2X0_CTRL_EN)) {
+ unsigned revision;
+
+ /* restore pl310 setup */
+ writel_relaxed(l2x0_saved_regs.tag_latency,
+ base + L310_TAG_LATENCY_CTRL);
+ writel_relaxed(l2x0_saved_regs.data_latency,
+ base + L310_DATA_LATENCY_CTRL);
+ writel_relaxed(l2x0_saved_regs.filter_end,
+ base + L310_ADDR_FILTER_END);
+ writel_relaxed(l2x0_saved_regs.filter_start,
+ base + L310_ADDR_FILTER_START);
+
+ revision = readl_relaxed(base + L2X0_CACHE_ID) &
+ L2X0_CACHE_ID_RTL_MASK;
+
+ if (revision >= L310_CACHE_ID_RTL_R2P0)
+ l2c_write_sec(l2x0_saved_regs.prefetch_ctrl, base,
+ L310_PREFETCH_CTRL);
+ if (revision >= L310_CACHE_ID_RTL_R3P0)
+ l2c_write_sec(l2x0_saved_regs.pwr_ctrl, base,
+ L310_POWER_CTRL);
+
+ l2c_enable(base, l2x0_saved_regs.aux_ctrl, 8);
+
+ /* Re-enable full-line-of-zeros for Cortex-A9 */
+ if (l2x0_saved_regs.aux_ctrl & L310_AUX_CTRL_FULL_LINE_ZERO)
+ set_auxcr(get_auxcr() | BIT(3) | BIT(2) | BIT(1));
+ }
+}
+
+static int l2c310_cpu_enable_flz(struct notifier_block *nb, unsigned long act, void *data)
+{
+ switch (act & ~CPU_TASKS_FROZEN) {
+ case CPU_STARTING:
+ set_auxcr(get_auxcr() | BIT(3) | BIT(2) | BIT(1));
+ break;
+ case CPU_DYING:
+ set_auxcr(get_auxcr() & ~(BIT(3) | BIT(2) | BIT(1)));
+ break;
+ }
+ return NOTIFY_OK;
+}
+
+static void __init l2c310_enable(void __iomem *base, u32 aux, unsigned num_lock)
+{
+ unsigned rev = readl_relaxed(base + L2X0_CACHE_ID) & L2X0_CACHE_ID_PART_MASK;
+ bool cortex_a9 = read_cpuid_part_number() == ARM_CPU_PART_CORTEX_A9;
+
+ if (rev >= L310_CACHE_ID_RTL_R2P0) {
+ if (cortex_a9) {
+ aux |= L310_AUX_CTRL_EARLY_BRESP;
+ pr_info("L2C-310 enabling early BRESP for Cortex-A9\n");
+ } else if (aux & L310_AUX_CTRL_EARLY_BRESP) {
+ pr_warn("L2C-310 early BRESP only supported with Cortex-A9\n");
+ aux &= ~L310_AUX_CTRL_EARLY_BRESP;
+ }
+ }
+
+ if (cortex_a9) {
+ u32 aux_cur = readl_relaxed(base + L2X0_AUX_CTRL);
+ u32 acr = get_auxcr();
+
+ pr_debug("Cortex-A9 ACR=0x%08x\n", acr);
+
+ if (acr & BIT(3) && !(aux_cur & L310_AUX_CTRL_FULL_LINE_ZERO))
+ pr_err("L2C-310: full line of zeros enabled in Cortex-A9 but not L2C-310 - invalid\n");
+
+ if (aux & L310_AUX_CTRL_FULL_LINE_ZERO && !(acr & BIT(3)))
+ pr_err("L2C-310: enabling full line of zeros but not enabled in Cortex-A9\n");
+
+ if (!(aux & L310_AUX_CTRL_FULL_LINE_ZERO) && !outer_cache.write_sec) {
+ aux |= L310_AUX_CTRL_FULL_LINE_ZERO;
+ pr_info("L2C-310 full line of zeros enabled for Cortex-A9\n");
+ }
+ } else if (aux & (L310_AUX_CTRL_FULL_LINE_ZERO | L310_AUX_CTRL_EARLY_BRESP)) {
+ pr_err("L2C-310: disabling Cortex-A9 specific feature bits\n");
+ aux &= ~(L310_AUX_CTRL_FULL_LINE_ZERO | L310_AUX_CTRL_EARLY_BRESP);
+ }
+
+ if (aux & (L310_AUX_CTRL_DATA_PREFETCH | L310_AUX_CTRL_INSTR_PREFETCH)) {
+ u32 prefetch = readl_relaxed(base + L310_PREFETCH_CTRL);
+
+ pr_info("L2C-310 %s%s prefetch enabled, offset %u lines\n",
+ aux & L310_AUX_CTRL_INSTR_PREFETCH ? "I" : "",
+ aux & L310_AUX_CTRL_DATA_PREFETCH ? "D" : "",
+ 1 + (prefetch & L310_PREFETCH_CTRL_OFFSET_MASK));
+ }
+
+ /* r3p0 or later has power control register */
+ if (rev >= L310_CACHE_ID_RTL_R3P0) {
+ u32 power_ctrl;
+
+ l2c_write_sec(L310_DYNAMIC_CLK_GATING_EN | L310_STNDBY_MODE_EN,
+ base, L310_POWER_CTRL);
+ power_ctrl = readl_relaxed(base + L310_POWER_CTRL);
+ pr_info("L2C-310 dynamic clock gating %sabled, standby mode %sabled\n",
+ power_ctrl & L310_DYNAMIC_CLK_GATING_EN ? "en" : "dis",
+ power_ctrl & L310_STNDBY_MODE_EN ? "en" : "dis");
+ }
+
+ /*
+ * Always enable non-secure access to the lockdown registers -
+ * we write to them as part of the L2C enable sequence so they
+ * need to be accessible.
+ */
+ aux |= L310_AUX_CTRL_NS_LOCKDOWN;
+
+ l2c_enable(base, aux, num_lock);
+
+ if (aux & L310_AUX_CTRL_FULL_LINE_ZERO) {
+ set_auxcr(get_auxcr() | BIT(3) | BIT(2) | BIT(1));
+ cpu_notifier(l2c310_cpu_enable_flz, 0);
+ }
+}
+
+static void __init l2c310_fixup(void __iomem *base, u32 cache_id,
+ struct outer_cache_fns *fns)
+{
+ unsigned revision = cache_id & L2X0_CACHE_ID_RTL_MASK;
+ const char *errata[8];
+ unsigned n = 0;
+
+ if (IS_ENABLED(CONFIG_PL310_ERRATA_588369) &&
+ revision < L310_CACHE_ID_RTL_R2P0 &&
+ /* For bcm compatibility */
+ fns->inv_range == l2c210_inv_range) {
+ fns->inv_range = l2c310_inv_range_erratum;
+ fns->flush_range = l2c310_flush_range_erratum;
+ errata[n++] = "588369";
+ }
+
+ if (IS_ENABLED(CONFIG_PL310_ERRATA_727915) &&
+ revision >= L310_CACHE_ID_RTL_R2P0 &&
+ revision < L310_CACHE_ID_RTL_R3P1) {
+ fns->flush_all = l2c310_flush_all_erratum;
+ errata[n++] = "727915";
+ }
+
+ if (revision >= L310_CACHE_ID_RTL_R3P0 &&
+ revision < L310_CACHE_ID_RTL_R3P2) {
+ u32 val = readl_relaxed(base + L310_PREFETCH_CTRL);
+ /* I don't think bit23 is required here... but iMX6 does so */
+ if (val & (BIT(30) | BIT(23))) {
+ val &= ~(BIT(30) | BIT(23));
+ l2c_write_sec(val, base, L310_PREFETCH_CTRL);
+ errata[n++] = "752271";
+ }
+ }
+
+ if (IS_ENABLED(CONFIG_PL310_ERRATA_753970) &&
+ revision == L310_CACHE_ID_RTL_R3P0) {
+ sync_reg_offset = L2X0_DUMMY_REG;
+ errata[n++] = "753970";
+ }
+
+ if (IS_ENABLED(CONFIG_PL310_ERRATA_769419))
+ errata[n++] = "769419";
+
+ if (n) {
+ unsigned i;
+
+ pr_info("L2C-310 errat%s", n > 1 ? "a" : "um");
+ for (i = 0; i < n; i++)
+ pr_cont(" %s", errata[i]);
+ pr_cont(" enabled\n");
+ }
+}
+
+static void l2c310_disable(void)
+{
+ /*
+ * If full-line-of-zeros is enabled, we must first disable it in the
+ * Cortex-A9 auxiliary control register before disabling the L2 cache.
+ */
+ if (l2x0_saved_regs.aux_ctrl & L310_AUX_CTRL_FULL_LINE_ZERO)
+ set_auxcr(get_auxcr() & ~(BIT(3) | BIT(2) | BIT(1)));
+
+ l2c_disable();
+}
+
+static const struct l2c_init_data l2c310_init_fns __initconst = {
+ .type = "L2C-310",
+ .way_size_0 = SZ_8K,
+ .num_lock = 8,
+ .enable = l2c310_enable,
+ .fixup = l2c310_fixup,
+ .save = l2c310_save,
+ .outer_cache = {
+ .inv_range = l2c210_inv_range,
+ .clean_range = l2c210_clean_range,
+ .flush_range = l2c210_flush_range,
+ .flush_all = l2c210_flush_all,
+ .disable = l2c310_disable,
+ .sync = l2c210_sync,
+ .resume = l2c310_resume,
+ },
+};
+
+static void __init __l2c_init(const struct l2c_init_data *data,
+ u32 aux_val, u32 aux_mask, u32 cache_id)
+{
+ struct outer_cache_fns fns;
+ unsigned way_size_bits, ways;
+ u32 aux, old_aux;
+
+ /*
+ * Sanity check the aux values. aux_mask is the bits we preserve
+ * from reading the hardware register, and aux_val is the bits we
+ * set.
+ */
+ if (aux_val & aux_mask)
+ pr_alert("L2C: platform provided aux values permit register corruption.\n");
+
+ old_aux = aux = readl_relaxed(l2x0_base + L2X0_AUX_CTRL);
aux &= aux_mask;
aux |= aux_val;
+ if (old_aux != aux)
+ pr_warn("L2C: DT/platform modifies aux control register: 0x%08x -> 0x%08x\n",
+ old_aux, aux);
+
/* Determine the number of ways */
switch (cache_id & L2X0_CACHE_ID_PART_MASK) {
case L2X0_CACHE_ID_PART_L310:
+ if ((aux_val | ~aux_mask) & (L2C_AUX_CTRL_WAY_SIZE_MASK | L310_AUX_CTRL_ASSOCIATIVITY_16))
+ pr_warn("L2C: DT/platform tries to modify or specify cache size\n");
if (aux & (1 << 16))
ways = 16;
else
ways = 8;
- type = "L310";
-#ifdef CONFIG_PL310_ERRATA_753970
- /* Unmapped register. */
- sync_reg_offset = L2X0_DUMMY_REG;
-#endif
- if ((cache_id & L2X0_CACHE_ID_RTL_MASK) <= L2X0_CACHE_ID_RTL_R3P0)
- outer_cache.set_debug = pl310_set_debug;
break;
+
case L2X0_CACHE_ID_PART_L210:
+ case L2X0_CACHE_ID_PART_L220:
ways = (aux >> 13) & 0xf;
- type = "L210";
break;
case AURORA_CACHE_ID:
- sync_reg_offset = AURORA_SYNC_REG;
ways = (aux >> 13) & 0xf;
ways = 2 << ((ways + 1) >> 2);
- way_size_shift = AURORA_WAY_SIZE_SHIFT;
- type = "Aurora";
break;
+
default:
/* Assume unknown chips have 8 ways */
ways = 8;
- type = "L2x0 series";
break;
}
l2x0_way_mask = (1 << ways) - 1;
/*
- * L2 cache Size = Way size * Number of ways
+ * way_size_0 is the size that a way_size value of zero would be
+ * given the calculation: way_size = way_size_0 << way_size_bits.
+ * So, if way_size_bits=0 is reserved, but way_size_bits=1 is 16k,
+ * then way_size_0 would be 8k.
+ *
+ * L2 cache size = number of ways * way size.
*/
- way_size = (aux & L2X0_AUX_CTRL_WAY_SIZE_MASK) >> 17;
- way_size = 1 << (way_size + way_size_shift);
+ way_size_bits = (aux & L2C_AUX_CTRL_WAY_SIZE_MASK) >>
+ L2C_AUX_CTRL_WAY_SIZE_SHIFT;
+ l2x0_size = ways * (data->way_size_0 << way_size_bits);
- l2x0_size = ways * way_size * SZ_1K;
+ fns = data->outer_cache;
+ fns.write_sec = outer_cache.write_sec;
+ if (data->fixup)
+ data->fixup(l2x0_base, cache_id, &fns);
/*
- * Check if l2x0 controller is already enabled.
- * If you are booting from non-secure mode
- * accessing the below registers will fault.
+ * Check if l2x0 controller is already enabled. If we are booting
+ * in non-secure mode accessing the below registers will fault.
*/
- if (!(readl_relaxed(l2x0_base + L2X0_CTRL) & L2X0_CTRL_EN)) {
- /* Make sure that I&D is not locked down when starting */
- l2x0_unlock(cache_id);
+ if (!(readl_relaxed(l2x0_base + L2X0_CTRL) & L2X0_CTRL_EN))
+ data->enable(l2x0_base, aux, data->num_lock);
- /* l2x0 controller is disabled */
- writel_relaxed(aux, l2x0_base + L2X0_AUX_CTRL);
+ outer_cache = fns;
- l2x0_inv_all();
-
- /* enable L2X0 */
- writel_relaxed(L2X0_CTRL_EN, l2x0_base + L2X0_CTRL);
- }
+ /*
+ * It is strange to save the register state before initialisation,
+ * but hey, this is what the DT implementations decided to do.
+ */
+ if (data->save)
+ data->save(l2x0_base);
/* Re-read it in case some bits are reserved. */
aux = readl_relaxed(l2x0_base + L2X0_AUX_CTRL);
- /* Save the value for resuming. */
- l2x0_saved_regs.aux_ctrl = aux;
+ pr_info("%s cache controller enabled, %d ways, %d kB\n",
+ data->type, ways, l2x0_size >> 10);
+ pr_info("%s: CACHE_ID 0x%08x, AUX_CTRL 0x%08x\n",
+ data->type, cache_id, aux);
+}
- if (!of_init) {
- outer_cache.inv_range = l2x0_inv_range;
- outer_cache.clean_range = l2x0_clean_range;
- outer_cache.flush_range = l2x0_flush_range;
- outer_cache.sync = l2x0_cache_sync;
- outer_cache.flush_all = l2x0_flush_all;
- outer_cache.inv_all = l2x0_inv_all;
- outer_cache.disable = l2x0_disable;
+void __init l2x0_init(void __iomem *base, u32 aux_val, u32 aux_mask)
+{
+ const struct l2c_init_data *data;
+ u32 cache_id;
+
+ l2x0_base = base;
+
+ cache_id = readl_relaxed(base + L2X0_CACHE_ID);
+
+ switch (cache_id & L2X0_CACHE_ID_PART_MASK) {
+ default:
+ case L2X0_CACHE_ID_PART_L210:
+ data = &l2c210_data;
+ break;
+
+ case L2X0_CACHE_ID_PART_L220:
+ data = &l2c220_data;
+ break;
+
+ case L2X0_CACHE_ID_PART_L310:
+ data = &l2c310_init_fns;
+ break;
}
- pr_info("%s cache controller enabled\n", type);
- pr_info("l2x0: %d ways, CACHE_ID 0x%08x, AUX_CTRL 0x%08x, Cache size: %d kB\n",
- ways, cache_id, aux, l2x0_size >> 10);
+ __l2c_init(data, aux_val, aux_mask, cache_id);
}
#ifdef CONFIG_OF
static int l2_wt_override;
+/* Aurora don't have the cache ID register available, so we have to
+ * pass it though the device tree */
+static u32 cache_id_part_number_from_dt;
+
+static void __init l2x0_of_parse(const struct device_node *np,
+ u32 *aux_val, u32 *aux_mask)
+{
+ u32 data[2] = { 0, 0 };
+ u32 tag = 0;
+ u32 dirty = 0;
+ u32 val = 0, mask = 0;
+
+ of_property_read_u32(np, "arm,tag-latency", &tag);
+ if (tag) {
+ mask |= L2X0_AUX_CTRL_TAG_LATENCY_MASK;
+ val |= (tag - 1) << L2X0_AUX_CTRL_TAG_LATENCY_SHIFT;
+ }
+
+ of_property_read_u32_array(np, "arm,data-latency",
+ data, ARRAY_SIZE(data));
+ if (data[0] && data[1]) {
+ mask |= L2X0_AUX_CTRL_DATA_RD_LATENCY_MASK |
+ L2X0_AUX_CTRL_DATA_WR_LATENCY_MASK;
+ val |= ((data[0] - 1) << L2X0_AUX_CTRL_DATA_RD_LATENCY_SHIFT) |
+ ((data[1] - 1) << L2X0_AUX_CTRL_DATA_WR_LATENCY_SHIFT);
+ }
+
+ of_property_read_u32(np, "arm,dirty-latency", &dirty);
+ if (dirty) {
+ mask |= L2X0_AUX_CTRL_DIRTY_LATENCY_MASK;
+ val |= (dirty - 1) << L2X0_AUX_CTRL_DIRTY_LATENCY_SHIFT;
+ }
+
+ *aux_val &= ~mask;
+ *aux_val |= val;
+ *aux_mask &= ~mask;
+}
+
+static const struct l2c_init_data of_l2c210_data __initconst = {
+ .type = "L2C-210",
+ .way_size_0 = SZ_8K,
+ .num_lock = 1,
+ .of_parse = l2x0_of_parse,
+ .enable = l2c_enable,
+ .save = l2c_save,
+ .outer_cache = {
+ .inv_range = l2c210_inv_range,
+ .clean_range = l2c210_clean_range,
+ .flush_range = l2c210_flush_range,
+ .flush_all = l2c210_flush_all,
+ .disable = l2c_disable,
+ .sync = l2c210_sync,
+ .resume = l2c210_resume,
+ },
+};
+
+static const struct l2c_init_data of_l2c220_data __initconst = {
+ .type = "L2C-220",
+ .way_size_0 = SZ_8K,
+ .num_lock = 1,
+ .of_parse = l2x0_of_parse,
+ .enable = l2c220_enable,
+ .save = l2c_save,
+ .outer_cache = {
+ .inv_range = l2c220_inv_range,
+ .clean_range = l2c220_clean_range,
+ .flush_range = l2c220_flush_range,
+ .flush_all = l2c220_flush_all,
+ .disable = l2c_disable,
+ .sync = l2c220_sync,
+ .resume = l2c210_resume,
+ },
+};
+
+static void __init l2c310_of_parse(const struct device_node *np,
+ u32 *aux_val, u32 *aux_mask)
+{
+ u32 data[3] = { 0, 0, 0 };
+ u32 tag[3] = { 0, 0, 0 };
+ u32 filter[2] = { 0, 0 };
+
+ of_property_read_u32_array(np, "arm,tag-latency", tag, ARRAY_SIZE(tag));
+ if (tag[0] && tag[1] && tag[2])
+ writel_relaxed(
+ L310_LATENCY_CTRL_RD(tag[0] - 1) |
+ L310_LATENCY_CTRL_WR(tag[1] - 1) |
+ L310_LATENCY_CTRL_SETUP(tag[2] - 1),
+ l2x0_base + L310_TAG_LATENCY_CTRL);
+
+ of_property_read_u32_array(np, "arm,data-latency",
+ data, ARRAY_SIZE(data));
+ if (data[0] && data[1] && data[2])
+ writel_relaxed(
+ L310_LATENCY_CTRL_RD(data[0] - 1) |
+ L310_LATENCY_CTRL_WR(data[1] - 1) |
+ L310_LATENCY_CTRL_SETUP(data[2] - 1),
+ l2x0_base + L310_DATA_LATENCY_CTRL);
+
+ of_property_read_u32_array(np, "arm,filter-ranges",
+ filter, ARRAY_SIZE(filter));
+ if (filter[1]) {
+ writel_relaxed(ALIGN(filter[0] + filter[1], SZ_1M),
+ l2x0_base + L310_ADDR_FILTER_END);
+ writel_relaxed((filter[0] & ~(SZ_1M - 1)) | L310_ADDR_FILTER_EN,
+ l2x0_base + L310_ADDR_FILTER_START);
+ }
+}
+
+static const struct l2c_init_data of_l2c310_data __initconst = {
+ .type = "L2C-310",
+ .way_size_0 = SZ_8K,
+ .num_lock = 8,
+ .of_parse = l2c310_of_parse,
+ .enable = l2c310_enable,
+ .fixup = l2c310_fixup,
+ .save = l2c310_save,
+ .outer_cache = {
+ .inv_range = l2c210_inv_range,
+ .clean_range = l2c210_clean_range,
+ .flush_range = l2c210_flush_range,
+ .flush_all = l2c210_flush_all,
+ .disable = l2c310_disable,
+ .sync = l2c210_sync,
+ .resume = l2c310_resume,
+ },
+};
+
/*
* Note that the end addresses passed to Linux primitives are
* noninclusive, while the hardware cache range operations use
@@ -524,6 +1166,100 @@
}
}
+static void aurora_save(void __iomem *base)
+{
+ l2x0_saved_regs.ctrl = readl_relaxed(base + L2X0_CTRL);
+ l2x0_saved_regs.aux_ctrl = readl_relaxed(base + L2X0_AUX_CTRL);
+}
+
+static void aurora_resume(void)
+{
+ void __iomem *base = l2x0_base;
+
+ if (!(readl(base + L2X0_CTRL) & L2X0_CTRL_EN)) {
+ writel_relaxed(l2x0_saved_regs.aux_ctrl, base + L2X0_AUX_CTRL);
+ writel_relaxed(l2x0_saved_regs.ctrl, base + L2X0_CTRL);
+ }
+}
+
+/*
+ * For Aurora cache in no outer mode, enable via the CP15 coprocessor
+ * broadcasting of cache commands to L2.
+ */
+static void __init aurora_enable_no_outer(void __iomem *base, u32 aux,
+ unsigned num_lock)
+{
+ u32 u;
+
+ asm volatile("mrc p15, 1, %0, c15, c2, 0" : "=r" (u));
+ u |= AURORA_CTRL_FW; /* Set the FW bit */
+ asm volatile("mcr p15, 1, %0, c15, c2, 0" : : "r" (u));
+
+ isb();
+
+ l2c_enable(base, aux, num_lock);
+}
+
+static void __init aurora_fixup(void __iomem *base, u32 cache_id,
+ struct outer_cache_fns *fns)
+{
+ sync_reg_offset = AURORA_SYNC_REG;
+}
+
+static void __init aurora_of_parse(const struct device_node *np,
+ u32 *aux_val, u32 *aux_mask)
+{
+ u32 val = AURORA_ACR_REPLACEMENT_TYPE_SEMIPLRU;
+ u32 mask = AURORA_ACR_REPLACEMENT_MASK;
+
+ of_property_read_u32(np, "cache-id-part",
+ &cache_id_part_number_from_dt);
+
+ /* Determine and save the write policy */
+ l2_wt_override = of_property_read_bool(np, "wt-override");
+
+ if (l2_wt_override) {
+ val |= AURORA_ACR_FORCE_WRITE_THRO_POLICY;
+ mask |= AURORA_ACR_FORCE_WRITE_POLICY_MASK;
+ }
+
+ *aux_val &= ~mask;
+ *aux_val |= val;
+ *aux_mask &= ~mask;
+}
+
+static const struct l2c_init_data of_aurora_with_outer_data __initconst = {
+ .type = "Aurora",
+ .way_size_0 = SZ_4K,
+ .num_lock = 4,
+ .of_parse = aurora_of_parse,
+ .enable = l2c_enable,
+ .fixup = aurora_fixup,
+ .save = aurora_save,
+ .outer_cache = {
+ .inv_range = aurora_inv_range,
+ .clean_range = aurora_clean_range,
+ .flush_range = aurora_flush_range,
+ .flush_all = l2x0_flush_all,
+ .disable = l2x0_disable,
+ .sync = l2x0_cache_sync,
+ .resume = aurora_resume,
+ },
+};
+
+static const struct l2c_init_data of_aurora_no_outer_data __initconst = {
+ .type = "Aurora",
+ .way_size_0 = SZ_4K,
+ .num_lock = 4,
+ .of_parse = aurora_of_parse,
+ .enable = aurora_enable_no_outer,
+ .fixup = aurora_fixup,
+ .save = aurora_save,
+ .outer_cache = {
+ .resume = aurora_resume,
+ },
+};
+
/*
* For certain Broadcom SoCs, depending on the address range, different offsets
* need to be added to the address before passing it to L2 for
@@ -588,16 +1324,16 @@
/* normal case, no cross section between start and end */
if (likely(bcm_addr_is_sys_emi(end) || !bcm_addr_is_sys_emi(start))) {
- l2x0_inv_range(new_start, new_end);
+ l2c210_inv_range(new_start, new_end);
return;
}
/* They cross sections, so it can only be a cross from section
* 2 to section 3
*/
- l2x0_inv_range(new_start,
+ l2c210_inv_range(new_start,
bcm_l2_phys_addr(BCM_VC_EMI_SEC3_START_ADDR-1));
- l2x0_inv_range(bcm_l2_phys_addr(BCM_VC_EMI_SEC3_START_ADDR),
+ l2c210_inv_range(bcm_l2_phys_addr(BCM_VC_EMI_SEC3_START_ADDR),
new_end);
}
@@ -610,26 +1346,21 @@
if (unlikely(end <= start))
return;
- if ((end - start) >= l2x0_size) {
- l2x0_clean_all();
- return;
- }
-
new_start = bcm_l2_phys_addr(start);
new_end = bcm_l2_phys_addr(end);
/* normal case, no cross section between start and end */
if (likely(bcm_addr_is_sys_emi(end) || !bcm_addr_is_sys_emi(start))) {
- l2x0_clean_range(new_start, new_end);
+ l2c210_clean_range(new_start, new_end);
return;
}
/* They cross sections, so it can only be a cross from section
* 2 to section 3
*/
- l2x0_clean_range(new_start,
+ l2c210_clean_range(new_start,
bcm_l2_phys_addr(BCM_VC_EMI_SEC3_START_ADDR-1));
- l2x0_clean_range(bcm_l2_phys_addr(BCM_VC_EMI_SEC3_START_ADDR),
+ l2c210_clean_range(bcm_l2_phys_addr(BCM_VC_EMI_SEC3_START_ADDR),
new_end);
}
@@ -643,7 +1374,7 @@
return;
if ((end - start) >= l2x0_size) {
- l2x0_flush_all();
+ outer_cache.flush_all();
return;
}
@@ -652,283 +1383,67 @@
/* normal case, no cross section between start and end */
if (likely(bcm_addr_is_sys_emi(end) || !bcm_addr_is_sys_emi(start))) {
- l2x0_flush_range(new_start, new_end);
+ l2c210_flush_range(new_start, new_end);
return;
}
/* They cross sections, so it can only be a cross from section
* 2 to section 3
*/
- l2x0_flush_range(new_start,
+ l2c210_flush_range(new_start,
bcm_l2_phys_addr(BCM_VC_EMI_SEC3_START_ADDR-1));
- l2x0_flush_range(bcm_l2_phys_addr(BCM_VC_EMI_SEC3_START_ADDR),
+ l2c210_flush_range(bcm_l2_phys_addr(BCM_VC_EMI_SEC3_START_ADDR),
new_end);
}
-static void __init l2x0_of_setup(const struct device_node *np,
- u32 *aux_val, u32 *aux_mask)
+/* Broadcom L2C-310 start from ARMs R3P2 or later, and require no fixups */
+static const struct l2c_init_data of_bcm_l2x0_data __initconst = {
+ .type = "BCM-L2C-310",
+ .way_size_0 = SZ_8K,
+ .num_lock = 8,
+ .of_parse = l2c310_of_parse,
+ .enable = l2c310_enable,
+ .save = l2c310_save,
+ .outer_cache = {
+ .inv_range = bcm_inv_range,
+ .clean_range = bcm_clean_range,
+ .flush_range = bcm_flush_range,
+ .flush_all = l2c210_flush_all,
+ .disable = l2c310_disable,
+ .sync = l2c210_sync,
+ .resume = l2c310_resume,
+ },
+};
+
+static void __init tauros3_save(void __iomem *base)
{
- u32 data[2] = { 0, 0 };
- u32 tag = 0;
- u32 dirty = 0;
- u32 val = 0, mask = 0;
+ l2c_save(base);
- of_property_read_u32(np, "arm,tag-latency", &tag);
- if (tag) {
- mask |= L2X0_AUX_CTRL_TAG_LATENCY_MASK;
- val |= (tag - 1) << L2X0_AUX_CTRL_TAG_LATENCY_SHIFT;
- }
-
- of_property_read_u32_array(np, "arm,data-latency",
- data, ARRAY_SIZE(data));
- if (data[0] && data[1]) {
- mask |= L2X0_AUX_CTRL_DATA_RD_LATENCY_MASK |
- L2X0_AUX_CTRL_DATA_WR_LATENCY_MASK;
- val |= ((data[0] - 1) << L2X0_AUX_CTRL_DATA_RD_LATENCY_SHIFT) |
- ((data[1] - 1) << L2X0_AUX_CTRL_DATA_WR_LATENCY_SHIFT);
- }
-
- of_property_read_u32(np, "arm,dirty-latency", &dirty);
- if (dirty) {
- mask |= L2X0_AUX_CTRL_DIRTY_LATENCY_MASK;
- val |= (dirty - 1) << L2X0_AUX_CTRL_DIRTY_LATENCY_SHIFT;
- }
-
- *aux_val &= ~mask;
- *aux_val |= val;
- *aux_mask &= ~mask;
-}
-
-static void __init pl310_of_setup(const struct device_node *np,
- u32 *aux_val, u32 *aux_mask)
-{
- u32 data[3] = { 0, 0, 0 };
- u32 tag[3] = { 0, 0, 0 };
- u32 filter[2] = { 0, 0 };
-
- of_property_read_u32_array(np, "arm,tag-latency", tag, ARRAY_SIZE(tag));
- if (tag[0] && tag[1] && tag[2])
- writel_relaxed(
- ((tag[0] - 1) << L2X0_LATENCY_CTRL_RD_SHIFT) |
- ((tag[1] - 1) << L2X0_LATENCY_CTRL_WR_SHIFT) |
- ((tag[2] - 1) << L2X0_LATENCY_CTRL_SETUP_SHIFT),
- l2x0_base + L2X0_TAG_LATENCY_CTRL);
-
- of_property_read_u32_array(np, "arm,data-latency",
- data, ARRAY_SIZE(data));
- if (data[0] && data[1] && data[2])
- writel_relaxed(
- ((data[0] - 1) << L2X0_LATENCY_CTRL_RD_SHIFT) |
- ((data[1] - 1) << L2X0_LATENCY_CTRL_WR_SHIFT) |
- ((data[2] - 1) << L2X0_LATENCY_CTRL_SETUP_SHIFT),
- l2x0_base + L2X0_DATA_LATENCY_CTRL);
-
- of_property_read_u32_array(np, "arm,filter-ranges",
- filter, ARRAY_SIZE(filter));
- if (filter[1]) {
- writel_relaxed(ALIGN(filter[0] + filter[1], SZ_1M),
- l2x0_base + L2X0_ADDR_FILTER_END);
- writel_relaxed((filter[0] & ~(SZ_1M - 1)) | L2X0_ADDR_FILTER_EN,
- l2x0_base + L2X0_ADDR_FILTER_START);
- }
-}
-
-static void __init pl310_save(void)
-{
- u32 l2x0_revision = readl_relaxed(l2x0_base + L2X0_CACHE_ID) &
- L2X0_CACHE_ID_RTL_MASK;
-
- l2x0_saved_regs.tag_latency = readl_relaxed(l2x0_base +
- L2X0_TAG_LATENCY_CTRL);
- l2x0_saved_regs.data_latency = readl_relaxed(l2x0_base +
- L2X0_DATA_LATENCY_CTRL);
- l2x0_saved_regs.filter_end = readl_relaxed(l2x0_base +
- L2X0_ADDR_FILTER_END);
- l2x0_saved_regs.filter_start = readl_relaxed(l2x0_base +
- L2X0_ADDR_FILTER_START);
-
- if (l2x0_revision >= L2X0_CACHE_ID_RTL_R2P0) {
- /*
- * From r2p0, there is Prefetch offset/control register
- */
- l2x0_saved_regs.prefetch_ctrl = readl_relaxed(l2x0_base +
- L2X0_PREFETCH_CTRL);
- /*
- * From r3p0, there is Power control register
- */
- if (l2x0_revision >= L2X0_CACHE_ID_RTL_R3P0)
- l2x0_saved_regs.pwr_ctrl = readl_relaxed(l2x0_base +
- L2X0_POWER_CTRL);
- }
-}
-
-static void aurora_save(void)
-{
- l2x0_saved_regs.ctrl = readl_relaxed(l2x0_base + L2X0_CTRL);
- l2x0_saved_regs.aux_ctrl = readl_relaxed(l2x0_base + L2X0_AUX_CTRL);
-}
-
-static void __init tauros3_save(void)
-{
l2x0_saved_regs.aux2_ctrl =
- readl_relaxed(l2x0_base + TAUROS3_AUX2_CTRL);
+ readl_relaxed(base + TAUROS3_AUX2_CTRL);
l2x0_saved_regs.prefetch_ctrl =
- readl_relaxed(l2x0_base + L2X0_PREFETCH_CTRL);
-}
-
-static void l2x0_resume(void)
-{
- if (!(readl_relaxed(l2x0_base + L2X0_CTRL) & L2X0_CTRL_EN)) {
- /* restore aux ctrl and enable l2 */
- l2x0_unlock(readl_relaxed(l2x0_base + L2X0_CACHE_ID));
-
- writel_relaxed(l2x0_saved_regs.aux_ctrl, l2x0_base +
- L2X0_AUX_CTRL);
-
- l2x0_inv_all();
-
- writel_relaxed(L2X0_CTRL_EN, l2x0_base + L2X0_CTRL);
- }
-}
-
-static void pl310_resume(void)
-{
- u32 l2x0_revision;
-
- if (!(readl_relaxed(l2x0_base + L2X0_CTRL) & L2X0_CTRL_EN)) {
- /* restore pl310 setup */
- writel_relaxed(l2x0_saved_regs.tag_latency,
- l2x0_base + L2X0_TAG_LATENCY_CTRL);
- writel_relaxed(l2x0_saved_regs.data_latency,
- l2x0_base + L2X0_DATA_LATENCY_CTRL);
- writel_relaxed(l2x0_saved_regs.filter_end,
- l2x0_base + L2X0_ADDR_FILTER_END);
- writel_relaxed(l2x0_saved_regs.filter_start,
- l2x0_base + L2X0_ADDR_FILTER_START);
-
- l2x0_revision = readl_relaxed(l2x0_base + L2X0_CACHE_ID) &
- L2X0_CACHE_ID_RTL_MASK;
-
- if (l2x0_revision >= L2X0_CACHE_ID_RTL_R2P0) {
- writel_relaxed(l2x0_saved_regs.prefetch_ctrl,
- l2x0_base + L2X0_PREFETCH_CTRL);
- if (l2x0_revision >= L2X0_CACHE_ID_RTL_R3P0)
- writel_relaxed(l2x0_saved_regs.pwr_ctrl,
- l2x0_base + L2X0_POWER_CTRL);
- }
- }
-
- l2x0_resume();
-}
-
-static void aurora_resume(void)
-{
- if (!(readl(l2x0_base + L2X0_CTRL) & L2X0_CTRL_EN)) {
- writel_relaxed(l2x0_saved_regs.aux_ctrl,
- l2x0_base + L2X0_AUX_CTRL);
- writel_relaxed(l2x0_saved_regs.ctrl, l2x0_base + L2X0_CTRL);
- }
+ readl_relaxed(base + L310_PREFETCH_CTRL);
}
static void tauros3_resume(void)
{
- if (!(readl_relaxed(l2x0_base + L2X0_CTRL) & L2X0_CTRL_EN)) {
+ void __iomem *base = l2x0_base;
+
+ if (!(readl_relaxed(base + L2X0_CTRL) & L2X0_CTRL_EN)) {
writel_relaxed(l2x0_saved_regs.aux2_ctrl,
- l2x0_base + TAUROS3_AUX2_CTRL);
+ base + TAUROS3_AUX2_CTRL);
writel_relaxed(l2x0_saved_regs.prefetch_ctrl,
- l2x0_base + L2X0_PREFETCH_CTRL);
+ base + L310_PREFETCH_CTRL);
+
+ l2c_enable(base, l2x0_saved_regs.aux_ctrl, 8);
}
-
- l2x0_resume();
}
-static void __init aurora_broadcast_l2_commands(void)
-{
- __u32 u;
- /* Enable Broadcasting of cache commands to L2*/
- __asm__ __volatile__("mrc p15, 1, %0, c15, c2, 0" : "=r"(u));
- u |= AURORA_CTRL_FW; /* Set the FW bit */
- __asm__ __volatile__("mcr p15, 1, %0, c15, c2, 0\n" : : "r"(u));
- isb();
-}
-
-static void __init aurora_of_setup(const struct device_node *np,
- u32 *aux_val, u32 *aux_mask)
-{
- u32 val = AURORA_ACR_REPLACEMENT_TYPE_SEMIPLRU;
- u32 mask = AURORA_ACR_REPLACEMENT_MASK;
-
- of_property_read_u32(np, "cache-id-part",
- &cache_id_part_number_from_dt);
-
- /* Determine and save the write policy */
- l2_wt_override = of_property_read_bool(np, "wt-override");
-
- if (l2_wt_override) {
- val |= AURORA_ACR_FORCE_WRITE_THRO_POLICY;
- mask |= AURORA_ACR_FORCE_WRITE_POLICY_MASK;
- }
-
- *aux_val &= ~mask;
- *aux_val |= val;
- *aux_mask &= ~mask;
-}
-
-static const struct l2x0_of_data pl310_data = {
- .setup = pl310_of_setup,
- .save = pl310_save,
- .outer_cache = {
- .resume = pl310_resume,
- .inv_range = l2x0_inv_range,
- .clean_range = l2x0_clean_range,
- .flush_range = l2x0_flush_range,
- .sync = l2x0_cache_sync,
- .flush_all = l2x0_flush_all,
- .inv_all = l2x0_inv_all,
- .disable = l2x0_disable,
- },
-};
-
-static const struct l2x0_of_data l2x0_data = {
- .setup = l2x0_of_setup,
- .save = NULL,
- .outer_cache = {
- .resume = l2x0_resume,
- .inv_range = l2x0_inv_range,
- .clean_range = l2x0_clean_range,
- .flush_range = l2x0_flush_range,
- .sync = l2x0_cache_sync,
- .flush_all = l2x0_flush_all,
- .inv_all = l2x0_inv_all,
- .disable = l2x0_disable,
- },
-};
-
-static const struct l2x0_of_data aurora_with_outer_data = {
- .setup = aurora_of_setup,
- .save = aurora_save,
- .outer_cache = {
- .resume = aurora_resume,
- .inv_range = aurora_inv_range,
- .clean_range = aurora_clean_range,
- .flush_range = aurora_flush_range,
- .sync = l2x0_cache_sync,
- .flush_all = l2x0_flush_all,
- .inv_all = l2x0_inv_all,
- .disable = l2x0_disable,
- },
-};
-
-static const struct l2x0_of_data aurora_no_outer_data = {
- .setup = aurora_of_setup,
- .save = aurora_save,
- .outer_cache = {
- .resume = aurora_resume,
- },
-};
-
-static const struct l2x0_of_data tauros3_data = {
- .setup = NULL,
+static const struct l2c_init_data of_tauros3_data __initconst = {
+ .type = "Tauros3",
+ .way_size_0 = SZ_8K,
+ .num_lock = 8,
+ .enable = l2c_enable,
.save = tauros3_save,
/* Tauros3 broadcasts L1 cache operations to L2 */
.outer_cache = {
@@ -936,43 +1451,26 @@
},
};
-static const struct l2x0_of_data bcm_l2x0_data = {
- .setup = pl310_of_setup,
- .save = pl310_save,
- .outer_cache = {
- .resume = pl310_resume,
- .inv_range = bcm_inv_range,
- .clean_range = bcm_clean_range,
- .flush_range = bcm_flush_range,
- .sync = l2x0_cache_sync,
- .flush_all = l2x0_flush_all,
- .inv_all = l2x0_inv_all,
- .disable = l2x0_disable,
- },
-};
-
+#define L2C_ID(name, fns) { .compatible = name, .data = (void *)&fns }
static const struct of_device_id l2x0_ids[] __initconst = {
- { .compatible = "arm,l210-cache", .data = (void *)&l2x0_data },
- { .compatible = "arm,l220-cache", .data = (void *)&l2x0_data },
- { .compatible = "arm,pl310-cache", .data = (void *)&pl310_data },
- { .compatible = "bcm,bcm11351-a2-pl310-cache", /* deprecated name */
- .data = (void *)&bcm_l2x0_data},
- { .compatible = "brcm,bcm11351-a2-pl310-cache",
- .data = (void *)&bcm_l2x0_data},
- { .compatible = "marvell,aurora-outer-cache",
- .data = (void *)&aurora_with_outer_data},
- { .compatible = "marvell,aurora-system-cache",
- .data = (void *)&aurora_no_outer_data},
- { .compatible = "marvell,tauros3-cache",
- .data = (void *)&tauros3_data },
+ L2C_ID("arm,l210-cache", of_l2c210_data),
+ L2C_ID("arm,l220-cache", of_l2c220_data),
+ L2C_ID("arm,pl310-cache", of_l2c310_data),
+ L2C_ID("brcm,bcm11351-a2-pl310-cache", of_bcm_l2x0_data),
+ L2C_ID("marvell,aurora-outer-cache", of_aurora_with_outer_data),
+ L2C_ID("marvell,aurora-system-cache", of_aurora_no_outer_data),
+ L2C_ID("marvell,tauros3-cache", of_tauros3_data),
+ /* Deprecated IDs */
+ L2C_ID("bcm,bcm11351-a2-pl310-cache", of_bcm_l2x0_data),
{}
};
int __init l2x0_of_init(u32 aux_val, u32 aux_mask)
{
+ const struct l2c_init_data *data;
struct device_node *np;
- const struct l2x0_of_data *data;
struct resource res;
+ u32 cache_id, old_aux;
np = of_find_matching_node(NULL, l2x0_ids);
if (!np)
@@ -989,23 +1487,29 @@
data = of_match_node(l2x0_ids, np)->data;
- /* L2 configuration can only be changed if the cache is disabled */
- if (!(readl_relaxed(l2x0_base + L2X0_CTRL) & L2X0_CTRL_EN)) {
- if (data->setup)
- data->setup(np, &aux_val, &aux_mask);
-
- /* For aurora cache in no outer mode select the
- * correct mode using the coprocessor*/
- if (data == &aurora_no_outer_data)
- aurora_broadcast_l2_commands();
+ old_aux = readl_relaxed(l2x0_base + L2X0_AUX_CTRL);
+ if (old_aux != ((old_aux & aux_mask) | aux_val)) {
+ pr_warn("L2C: platform modifies aux control register: 0x%08x -> 0x%08x\n",
+ old_aux, (old_aux & aux_mask) | aux_val);
+ } else if (aux_mask != ~0U && aux_val != 0) {
+ pr_alert("L2C: platform provided aux values match the hardware, so have no effect. Please remove them.\n");
}
- if (data->save)
- data->save();
+ /* All L2 caches are unified, so this property should be specified */
+ if (!of_property_read_bool(np, "cache-unified"))
+ pr_err("L2C: device tree omits to specify unified cache\n");
- of_init = true;
- memcpy(&outer_cache, &data->outer_cache, sizeof(outer_cache));
- l2x0_init(l2x0_base, aux_val, aux_mask);
+ /* L2 configuration can only be changed if the cache is disabled */
+ if (!(readl_relaxed(l2x0_base + L2X0_CTRL) & L2X0_CTRL_EN))
+ if (data->of_parse)
+ data->of_parse(np, &aux_val, &aux_mask);
+
+ if (cache_id_part_number_from_dt)
+ cache_id = cache_id_part_number_from_dt;
+ else
+ cache_id = readl_relaxed(l2x0_base + L2X0_CACHE_ID);
+
+ __l2c_init(data, aux_val, aux_mask, cache_id);
return 0;
}
diff --git a/arch/arm/mm/cache-v7.S b/arch/arm/mm/cache-v7.S
index 778bcf8..615c99e 100644
--- a/arch/arm/mm/cache-v7.S
+++ b/arch/arm/mm/cache-v7.S
@@ -59,7 +59,7 @@
bgt 2b
cmp r2, #0
bgt 1b
- dsb
+ dsb st
isb
mov pc, lr
ENDPROC(v7_invalidate_l1)
@@ -166,7 +166,7 @@
finished:
mov r10, #0 @ swith back to cache level 0
mcr p15, 2, r10, c0, c0, 0 @ select current cache level in cssr
- dsb
+ dsb st
isb
mov pc, lr
ENDPROC(v7_flush_dcache_all)
@@ -335,7 +335,7 @@
add r0, r0, r2
cmp r0, r1
blo 1b
- dsb
+ dsb st
mov pc, lr
ENDPROC(v7_flush_kern_dcache_area)
@@ -368,7 +368,7 @@
add r0, r0, r2
cmp r0, r1
blo 1b
- dsb
+ dsb st
mov pc, lr
ENDPROC(v7_dma_inv_range)
@@ -390,7 +390,7 @@
add r0, r0, r2
cmp r0, r1
blo 1b
- dsb
+ dsb st
mov pc, lr
ENDPROC(v7_dma_clean_range)
@@ -412,7 +412,7 @@
add r0, r0, r2
cmp r0, r1
blo 1b
- dsb
+ dsb st
mov pc, lr
ENDPROC(v7_dma_flush_range)
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 5bef8585..4c88935 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -885,7 +885,7 @@
static void __dma_page_cpu_to_dev(struct page *page, unsigned long off,
size_t size, enum dma_data_direction dir)
{
- unsigned long paddr;
+ phys_addr_t paddr;
dma_cache_maint_page(page, off, size, dir, dmac_map_area);
@@ -901,14 +901,15 @@
static void __dma_page_dev_to_cpu(struct page *page, unsigned long off,
size_t size, enum dma_data_direction dir)
{
- unsigned long paddr = page_to_phys(page) + off;
+ phys_addr_t paddr = page_to_phys(page) + off;
/* FIXME: non-speculating: not required */
- /* don't bother invalidating if DMA to device */
- if (dir != DMA_TO_DEVICE)
+ /* in any case, don't bother invalidating if DMA to device */
+ if (dir != DMA_TO_DEVICE) {
outer_inv_range(paddr, paddr + size);
- dma_cache_maint_page(page, off, size, dir, dmac_unmap_area);
+ dma_cache_maint_page(page, off, size, dir, dmac_unmap_area);
+ }
/*
* Mark the D-cache clean for these pages to avoid extra flushing.
diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c
index 3387e60..43d54f5b 100644
--- a/arch/arm/mm/flush.c
+++ b/arch/arm/mm/flush.c
@@ -104,17 +104,20 @@
#define flush_icache_alias(pfn,vaddr,len) do { } while (0)
#endif
+#define FLAG_PA_IS_EXEC 1
+#define FLAG_PA_CORE_IN_MM 2
+
static void flush_ptrace_access_other(void *args)
{
__flush_icache_all();
}
-static
-void flush_ptrace_access(struct vm_area_struct *vma, struct page *page,
- unsigned long uaddr, void *kaddr, unsigned long len)
+static inline
+void __flush_ptrace_access(struct page *page, unsigned long uaddr, void *kaddr,
+ unsigned long len, unsigned int flags)
{
if (cache_is_vivt()) {
- if (cpumask_test_cpu(smp_processor_id(), mm_cpumask(vma->vm_mm))) {
+ if (flags & FLAG_PA_CORE_IN_MM) {
unsigned long addr = (unsigned long)kaddr;
__cpuc_coherent_kern_range(addr, addr + len);
}
@@ -128,7 +131,7 @@
}
/* VIPT non-aliasing D-cache */
- if (vma->vm_flags & VM_EXEC) {
+ if (flags & FLAG_PA_IS_EXEC) {
unsigned long addr = (unsigned long)kaddr;
if (icache_is_vipt_aliasing())
flush_icache_alias(page_to_pfn(page), uaddr, len);
@@ -140,6 +143,26 @@
}
}
+static
+void flush_ptrace_access(struct vm_area_struct *vma, struct page *page,
+ unsigned long uaddr, void *kaddr, unsigned long len)
+{
+ unsigned int flags = 0;
+ if (cpumask_test_cpu(smp_processor_id(), mm_cpumask(vma->vm_mm)))
+ flags |= FLAG_PA_CORE_IN_MM;
+ if (vma->vm_flags & VM_EXEC)
+ flags |= FLAG_PA_IS_EXEC;
+ __flush_ptrace_access(page, uaddr, kaddr, len, flags);
+}
+
+void flush_uprobe_xol_access(struct page *page, unsigned long uaddr,
+ void *kaddr, unsigned long len)
+{
+ unsigned int flags = FLAG_PA_CORE_IN_MM|FLAG_PA_IS_EXEC;
+
+ __flush_ptrace_access(page, uaddr, kaddr, len, flags);
+}
+
/*
* Copy user data from/to a page which is mapped into a different
* processes address space. Really, we want to allow our "user
diff --git a/arch/arm/mm/highmem.c b/arch/arm/mm/highmem.c
index 21b9e1b..45aeaac 100644
--- a/arch/arm/mm/highmem.c
+++ b/arch/arm/mm/highmem.c
@@ -18,6 +18,21 @@
#include <asm/tlbflush.h>
#include "mm.h"
+pte_t *fixmap_page_table;
+
+static inline void set_fixmap_pte(int idx, pte_t pte)
+{
+ unsigned long vaddr = __fix_to_virt(idx);
+ set_pte_ext(fixmap_page_table + idx, pte, 0);
+ local_flush_tlb_kernel_page(vaddr);
+}
+
+static inline pte_t get_fixmap_pte(unsigned long vaddr)
+{
+ unsigned long idx = __virt_to_fix(vaddr);
+ return *(fixmap_page_table + idx);
+}
+
void *kmap(struct page *page)
{
might_sleep();
@@ -63,20 +78,20 @@
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR * smp_processor_id();
- vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
+ vaddr = __fix_to_virt(idx);
#ifdef CONFIG_DEBUG_HIGHMEM
/*
* With debugging enabled, kunmap_atomic forces that entry to 0.
* Make sure it was indeed properly unmapped.
*/
- BUG_ON(!pte_none(get_top_pte(vaddr)));
+ BUG_ON(!pte_none(*(fixmap_page_table + idx)));
#endif
/*
* When debugging is off, kunmap_atomic leaves the previous mapping
* in place, so the contained TLB flush ensures the TLB is updated
* with the new mapping.
*/
- set_top_pte(vaddr, mk_pte(page, kmap_prot));
+ set_fixmap_pte(idx, mk_pte(page, kmap_prot));
return (void *)vaddr;
}
@@ -94,8 +109,8 @@
if (cache_is_vivt())
__cpuc_flush_dcache_area((void *)vaddr, PAGE_SIZE);
#ifdef CONFIG_DEBUG_HIGHMEM
- BUG_ON(vaddr != __fix_to_virt(FIX_KMAP_BEGIN + idx));
- set_top_pte(vaddr, __pte(0));
+ BUG_ON(vaddr != __fix_to_virt(idx));
+ set_fixmap_pte(idx, __pte(0));
#else
(void) idx; /* to kill a warning */
#endif
@@ -117,11 +132,11 @@
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR * smp_processor_id();
- vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
+ vaddr = __fix_to_virt(idx);
#ifdef CONFIG_DEBUG_HIGHMEM
- BUG_ON(!pte_none(get_top_pte(vaddr)));
+ BUG_ON(!pte_none(*(fixmap_page_table + idx)));
#endif
- set_top_pte(vaddr, pfn_pte(pfn, kmap_prot));
+ set_fixmap_pte(idx, pfn_pte(pfn, kmap_prot));
return (void *)vaddr;
}
@@ -133,5 +148,5 @@
if (vaddr < FIXADDR_START)
return virt_to_page(ptr);
- return pte_page(get_top_pte(vaddr));
+ return pte_page(get_fixmap_pte(vaddr));
}
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 928d596..659c75d 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -23,6 +23,7 @@
#include <linux/dma-contiguous.h>
#include <linux/sizes.h>
+#include <asm/cp15.h>
#include <asm/mach-types.h>
#include <asm/memblock.h>
#include <asm/prom.h>
@@ -36,6 +37,14 @@
#include "mm.h"
+#ifdef CONFIG_CPU_CP15_MMU
+unsigned long __init __clear_cr(unsigned long mask)
+{
+ cr_alignment = cr_alignment & ~mask;
+ return cr_alignment;
+}
+#endif
+
static phys_addr_t phys_initrd_start __initdata = 0;
static unsigned long phys_initrd_size __initdata = 0;
@@ -81,24 +90,21 @@
* initialization functions, as well as show_mem() for the skipping
* of holes in the memory map. It is populated by arm_add_memory().
*/
-struct meminfo meminfo;
-
void show_mem(unsigned int filter)
{
int free = 0, total = 0, reserved = 0;
- int shared = 0, cached = 0, slab = 0, i;
- struct meminfo * mi = &meminfo;
+ int shared = 0, cached = 0, slab = 0;
+ struct memblock_region *reg;
printk("Mem-info:\n");
show_free_areas(filter);
- for_each_bank (i, mi) {
- struct membank *bank = &mi->bank[i];
+ for_each_memblock (memory, reg) {
unsigned int pfn1, pfn2;
struct page *page, *end;
- pfn1 = bank_pfn_start(bank);
- pfn2 = bank_pfn_end(bank);
+ pfn1 = memblock_region_memory_base_pfn(reg);
+ pfn2 = memblock_region_memory_end_pfn(reg);
page = pfn_to_page(pfn1);
end = pfn_to_page(pfn2 - 1) + 1;
@@ -115,8 +121,9 @@
free++;
else
shared += page_count(page) - 1;
- page++;
- } while (page < end);
+ pfn1++;
+ page = pfn_to_page(pfn1);
+ } while (pfn1 < pfn2);
}
printk("%d pages of RAM\n", total);
@@ -130,16 +137,9 @@
static void __init find_limits(unsigned long *min, unsigned long *max_low,
unsigned long *max_high)
{
- struct meminfo *mi = &meminfo;
- int i;
-
- /* This assumes the meminfo array is properly sorted */
- *min = bank_pfn_start(&mi->bank[0]);
- for_each_bank (i, mi)
- if (mi->bank[i].highmem)
- break;
- *max_low = bank_pfn_end(&mi->bank[i - 1]);
- *max_high = bank_pfn_end(&mi->bank[mi->nr_banks - 1]);
+ *max_low = PFN_DOWN(memblock_get_current_limit());
+ *min = PFN_UP(memblock_start_of_DRAM());
+ *max_high = PFN_DOWN(memblock_end_of_DRAM());
}
#ifdef CONFIG_ZONE_DMA
@@ -274,14 +274,8 @@
return phys;
}
-void __init arm_memblock_init(struct meminfo *mi,
- const struct machine_desc *mdesc)
+void __init arm_memblock_init(const struct machine_desc *mdesc)
{
- int i;
-
- for (i = 0; i < mi->nr_banks; i++)
- memblock_add(mi->bank[i].start, mi->bank[i].size);
-
/* Register the kernel text, kernel data and initrd with memblock. */
#ifdef CONFIG_XIP_KERNEL
memblock_reserve(__pa(_sdata), _end - _sdata);
@@ -412,54 +406,53 @@
/*
* The mem_map array can get very big. Free the unused area of the memory map.
*/
-static void __init free_unused_memmap(struct meminfo *mi)
+static void __init free_unused_memmap(void)
{
- unsigned long bank_start, prev_bank_end = 0;
- unsigned int i;
+ unsigned long start, prev_end = 0;
+ struct memblock_region *reg;
/*
* This relies on each bank being in address order.
* The banks are sorted previously in bootmem_init().
*/
- for_each_bank(i, mi) {
- struct membank *bank = &mi->bank[i];
-
- bank_start = bank_pfn_start(bank);
+ for_each_memblock(memory, reg) {
+ start = memblock_region_memory_base_pfn(reg);
#ifdef CONFIG_SPARSEMEM
/*
* Take care not to free memmap entries that don't exist
* due to SPARSEMEM sections which aren't present.
*/
- bank_start = min(bank_start,
- ALIGN(prev_bank_end, PAGES_PER_SECTION));
+ start = min(start,
+ ALIGN(prev_end, PAGES_PER_SECTION));
#else
/*
* Align down here since the VM subsystem insists that the
* memmap entries are valid from the bank start aligned to
* MAX_ORDER_NR_PAGES.
*/
- bank_start = round_down(bank_start, MAX_ORDER_NR_PAGES);
+ start = round_down(start, MAX_ORDER_NR_PAGES);
#endif
/*
* If we had a previous bank, and there is a space
* between the current bank and the previous, free it.
*/
- if (prev_bank_end && prev_bank_end < bank_start)
- free_memmap(prev_bank_end, bank_start);
+ if (prev_end && prev_end < start)
+ free_memmap(prev_end, start);
/*
* Align up here since the VM subsystem insists that the
* memmap entries are valid from the bank end aligned to
* MAX_ORDER_NR_PAGES.
*/
- prev_bank_end = ALIGN(bank_pfn_end(bank), MAX_ORDER_NR_PAGES);
+ prev_end = ALIGN(memblock_region_memory_end_pfn(reg),
+ MAX_ORDER_NR_PAGES);
}
#ifdef CONFIG_SPARSEMEM
- if (!IS_ALIGNED(prev_bank_end, PAGES_PER_SECTION))
- free_memmap(prev_bank_end,
- ALIGN(prev_bank_end, PAGES_PER_SECTION));
+ if (!IS_ALIGNED(prev_end, PAGES_PER_SECTION))
+ free_memmap(prev_end,
+ ALIGN(prev_end, PAGES_PER_SECTION));
#endif
}
@@ -535,7 +528,7 @@
set_max_mapnr(pfn_to_page(max_pfn) - mem_map);
/* this will put all unused low memory onto the freelists */
- free_unused_memmap(&meminfo);
+ free_unused_memmap();
free_all_bootmem();
#ifdef CONFIG_SA1111
diff --git a/arch/arm/mm/ioremap.c b/arch/arm/mm/ioremap.c
index f9c32ba..d1e5ad7 100644
--- a/arch/arm/mm/ioremap.c
+++ b/arch/arm/mm/ioremap.c
@@ -438,6 +438,13 @@
EXPORT_SYMBOL(__arm_iounmap);
#ifdef CONFIG_PCI
+static int pci_ioremap_mem_type = MT_DEVICE;
+
+void pci_ioremap_set_mem_type(int mem_type)
+{
+ pci_ioremap_mem_type = mem_type;
+}
+
int pci_ioremap_io(unsigned int offset, phys_addr_t phys_addr)
{
BUG_ON(offset + SZ_64K > IO_SPACE_LIMIT);
@@ -445,7 +452,7 @@
return ioremap_page_range(PCI_IO_VIRT_BASE + offset,
PCI_IO_VIRT_BASE + offset + SZ_64K,
phys_addr,
- __pgprot(get_mem_type(MT_DEVICE)->prot_pte));
+ __pgprot(get_mem_type(pci_ioremap_mem_type)->prot_pte));
}
EXPORT_SYMBOL_GPL(pci_ioremap_io);
#endif
diff --git a/arch/arm/mm/l2c-common.c b/arch/arm/mm/l2c-common.c
new file mode 100644
index 0000000..10a3cf2
--- /dev/null
+++ b/arch/arm/mm/l2c-common.c
@@ -0,0 +1,20 @@
+/*
+ * Copyright (C) 2010 ARM Ltd.
+ * Written by Catalin Marinas <catalin.marinas@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#include <linux/bug.h>
+#include <linux/smp.h>
+#include <asm/outercache.h>
+
+void outer_disable(void)
+{
+ WARN_ON(!irqs_disabled());
+ WARN_ON(num_online_cpus() > 1);
+
+ if (outer_cache.disable)
+ outer_cache.disable();
+}
diff --git a/arch/arm/mm/l2c-l2x0-resume.S b/arch/arm/mm/l2c-l2x0-resume.S
new file mode 100644
index 0000000..99b05f2
--- /dev/null
+++ b/arch/arm/mm/l2c-l2x0-resume.S
@@ -0,0 +1,58 @@
+/*
+ * L2C-310 early resume code. This can be used by platforms to restore
+ * the settings of their L2 cache controller before restoring the
+ * processor state.
+ *
+ * This code can only be used to if you are running in the secure world.
+ */
+#include <linux/linkage.h>
+#include <asm/hardware/cache-l2x0.h>
+
+ .text
+
+ENTRY(l2c310_early_resume)
+ adr r0, 1f
+ ldr r2, [r0]
+ add r0, r2, r0
+
+ ldmia r0, {r1, r2, r3, r4, r5, r6, r7, r8}
+ @ r1 = phys address of L2C-310 controller
+ @ r2 = aux_ctrl
+ @ r3 = tag_latency
+ @ r4 = data_latency
+ @ r5 = filter_start
+ @ r6 = filter_end
+ @ r7 = prefetch_ctrl
+ @ r8 = pwr_ctrl
+
+ @ Check that the address has been initialised
+ teq r1, #0
+ moveq pc, lr
+
+ @ The prefetch and power control registers are revision dependent
+ @ and can be written whether or not the L2 cache is enabled
+ ldr r0, [r1, #L2X0_CACHE_ID]
+ and r0, r0, #L2X0_CACHE_ID_RTL_MASK
+ cmp r0, #L310_CACHE_ID_RTL_R2P0
+ strcs r7, [r1, #L310_PREFETCH_CTRL]
+ cmp r0, #L310_CACHE_ID_RTL_R3P0
+ strcs r8, [r1, #L310_POWER_CTRL]
+
+ @ Don't setup the L2 cache if it is already enabled
+ ldr r0, [r1, #L2X0_CTRL]
+ tst r0, #L2X0_CTRL_EN
+ movne pc, lr
+
+ str r3, [r1, #L310_TAG_LATENCY_CTRL]
+ str r4, [r1, #L310_DATA_LATENCY_CTRL]
+ str r6, [r1, #L310_ADDR_FILTER_END]
+ str r5, [r1, #L310_ADDR_FILTER_START]
+
+ str r2, [r1, #L2X0_AUX_CTRL]
+ mov r9, #L2X0_CTRL_EN
+ str r9, [r1, #L2X0_CTRL]
+ mov pc, lr
+ENDPROC(l2c310_early_resume)
+
+ .align
+1: .long l2x0_saved_regs - .
diff --git a/arch/arm/mm/mm.h b/arch/arm/mm/mm.h
index 7ea641b..ce727d4 100644
--- a/arch/arm/mm/mm.h
+++ b/arch/arm/mm/mm.h
@@ -2,6 +2,8 @@
#include <linux/list.h>
#include <linux/vmalloc.h>
+#include <asm/pgtable.h>
+
/* the upper-most page table pointer */
extern pmd_t *top_pmd;
@@ -93,3 +95,5 @@
void __init bootmem_init(void);
void arm_mm_memblock_reserve(void);
void dma_contiguous_remap(void);
+
+unsigned long __clear_cr(unsigned long mask);
diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
index b68c6b2..ab14b79 100644
--- a/arch/arm/mm/mmu.c
+++ b/arch/arm/mm/mmu.c
@@ -35,6 +35,7 @@
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
#include <asm/mach/pci.h>
+#include <asm/fixmap.h>
#include "mm.h"
#include "tcm.h"
@@ -117,28 +118,54 @@
};
#ifdef CONFIG_CPU_CP15
+static unsigned long initial_pmd_value __initdata = 0;
+
/*
- * These are useful for identifying cache coherency
- * problems by allowing the cache or the cache and
- * writebuffer to be turned off. (Note: the write
- * buffer should not be on and the cache off).
+ * Initialise the cache_policy variable with the initial state specified
+ * via the "pmd" value. This is used to ensure that on ARMv6 and later,
+ * the C code sets the page tables up with the same policy as the head
+ * assembly code, which avoids an illegal state where the TLBs can get
+ * confused. See comments in early_cachepolicy() for more information.
+ */
+void __init init_default_cache_policy(unsigned long pmd)
+{
+ int i;
+
+ initial_pmd_value = pmd;
+
+ pmd &= PMD_SECT_TEX(1) | PMD_SECT_BUFFERABLE | PMD_SECT_CACHEABLE;
+
+ for (i = 0; i < ARRAY_SIZE(cache_policies); i++)
+ if (cache_policies[i].pmd == pmd) {
+ cachepolicy = i;
+ break;
+ }
+
+ if (i == ARRAY_SIZE(cache_policies))
+ pr_err("ERROR: could not find cache policy\n");
+}
+
+/*
+ * These are useful for identifying cache coherency problems by allowing
+ * the cache or the cache and writebuffer to be turned off. (Note: the
+ * write buffer should not be on and the cache off).
*/
static int __init early_cachepolicy(char *p)
{
- int i;
+ int i, selected = -1;
for (i = 0; i < ARRAY_SIZE(cache_policies); i++) {
int len = strlen(cache_policies[i].policy);
if (memcmp(p, cache_policies[i].policy, len) == 0) {
- cachepolicy = i;
- cr_alignment &= ~cache_policies[i].cr_mask;
- cr_no_alignment &= ~cache_policies[i].cr_mask;
+ selected = i;
break;
}
}
- if (i == ARRAY_SIZE(cache_policies))
- printk(KERN_ERR "ERROR: unknown or unsupported cache policy\n");
+
+ if (selected == -1)
+ pr_err("ERROR: unknown or unsupported cache policy\n");
+
/*
* This restriction is partly to do with the way we boot; it is
* unpredictable to have memory mapped using two different sets of
@@ -146,12 +173,18 @@
* change these attributes once the initial assembly has setup the
* page tables.
*/
- if (cpu_architecture() >= CPU_ARCH_ARMv6) {
- printk(KERN_WARNING "Only cachepolicy=writeback supported on ARMv6 and later\n");
- cachepolicy = CPOLICY_WRITEBACK;
+ if (cpu_architecture() >= CPU_ARCH_ARMv6 && selected != cachepolicy) {
+ pr_warn("Only cachepolicy=%s supported on ARMv6 and later\n",
+ cache_policies[cachepolicy].policy);
+ return 0;
}
- flush_cache_all();
- set_cr(cr_alignment);
+
+ if (selected != cachepolicy) {
+ unsigned long cr = __clear_cr(cache_policies[selected].cr_mask);
+ cachepolicy = selected;
+ flush_cache_all();
+ set_cr(cr);
+ }
return 0;
}
early_param("cachepolicy", early_cachepolicy);
@@ -186,35 +219,6 @@
early_param("ecc", early_ecc);
#endif
-static int __init noalign_setup(char *__unused)
-{
- cr_alignment &= ~CR_A;
- cr_no_alignment &= ~CR_A;
- set_cr(cr_alignment);
- return 1;
-}
-__setup("noalign", noalign_setup);
-
-#ifndef CONFIG_SMP
-void adjust_cr(unsigned long mask, unsigned long set)
-{
- unsigned long flags;
-
- mask &= ~CR_A;
-
- set &= mask;
-
- local_irq_save(flags);
-
- cr_no_alignment = (cr_no_alignment & ~mask) | set;
- cr_alignment = (cr_alignment & ~mask) | set;
-
- set_cr((get_cr() & ~mask) | set);
-
- local_irq_restore(flags);
-}
-#endif
-
#else /* ifdef CONFIG_CPU_CP15 */
static int __init early_cachepolicy(char *p)
@@ -414,8 +418,17 @@
cachepolicy = CPOLICY_WRITEBACK;
ecc_mask = 0;
}
- if (is_smp())
- cachepolicy = CPOLICY_WRITEALLOC;
+
+ if (is_smp()) {
+ if (cachepolicy != CPOLICY_WRITEALLOC) {
+ pr_warn("Forcing write-allocate cache policy for SMP\n");
+ cachepolicy = CPOLICY_WRITEALLOC;
+ }
+ if (!(initial_pmd_value & PMD_SECT_S)) {
+ pr_warn("Forcing shared mappings for SMP\n");
+ initial_pmd_value |= PMD_SECT_S;
+ }
+ }
/*
* Strip out features not present on earlier architectures.
@@ -539,11 +552,12 @@
mem_types[MT_CACHECLEAN].prot_sect |= PMD_SECT_APX|PMD_SECT_AP_WRITE;
#endif
- if (is_smp()) {
- /*
- * Mark memory with the "shared" attribute
- * for SMP systems
- */
+ /*
+ * If the initial page tables were created with the S bit
+ * set, then we need to do the same here for the same
+ * reasons given in early_cachepolicy().
+ */
+ if (initial_pmd_value & PMD_SECT_S) {
user_pgprot |= L_PTE_SHARED;
kern_pgprot |= L_PTE_SHARED;
vecs_pgprot |= L_PTE_SHARED;
@@ -1061,74 +1075,47 @@
void __init sanity_check_meminfo(void)
{
phys_addr_t memblock_limit = 0;
- int i, j, highmem = 0;
+ int highmem = 0;
phys_addr_t vmalloc_limit = __pa(vmalloc_min - 1) + 1;
+ struct memblock_region *reg;
- for (i = 0, j = 0; i < meminfo.nr_banks; i++) {
- struct membank *bank = &meminfo.bank[j];
- phys_addr_t size_limit;
+ for_each_memblock(memory, reg) {
+ phys_addr_t block_start = reg->base;
+ phys_addr_t block_end = reg->base + reg->size;
+ phys_addr_t size_limit = reg->size;
- *bank = meminfo.bank[i];
- size_limit = bank->size;
-
- if (bank->start >= vmalloc_limit)
+ if (reg->base >= vmalloc_limit)
highmem = 1;
else
- size_limit = vmalloc_limit - bank->start;
+ size_limit = vmalloc_limit - reg->base;
- bank->highmem = highmem;
-#ifdef CONFIG_HIGHMEM
- /*
- * Split those memory banks which are partially overlapping
- * the vmalloc area greatly simplifying things later.
- */
- if (!highmem && bank->size > size_limit) {
- if (meminfo.nr_banks >= NR_BANKS) {
- printk(KERN_CRIT "NR_BANKS too low, "
- "ignoring high memory\n");
- } else {
- memmove(bank + 1, bank,
- (meminfo.nr_banks - i) * sizeof(*bank));
- meminfo.nr_banks++;
- i++;
- bank[1].size -= size_limit;
- bank[1].start = vmalloc_limit;
- bank[1].highmem = highmem = 1;
- j++;
+ if (!IS_ENABLED(CONFIG_HIGHMEM) || cache_is_vipt_aliasing()) {
+
+ if (highmem) {
+ pr_notice("Ignoring RAM at %pa-%pa (!CONFIG_HIGHMEM)\n",
+ &block_start, &block_end);
+ memblock_remove(reg->base, reg->size);
+ continue;
}
- bank->size = size_limit;
- }
-#else
- /*
- * Highmem banks not allowed with !CONFIG_HIGHMEM.
- */
- if (highmem) {
- printk(KERN_NOTICE "Ignoring RAM at %.8llx-%.8llx "
- "(!CONFIG_HIGHMEM).\n",
- (unsigned long long)bank->start,
- (unsigned long long)bank->start + bank->size - 1);
- continue;
+
+ if (reg->size > size_limit) {
+ phys_addr_t overlap_size = reg->size - size_limit;
+
+ pr_notice("Truncating RAM at %pa-%pa to -%pa",
+ &block_start, &block_end, &vmalloc_limit);
+ memblock_remove(vmalloc_limit, overlap_size);
+ block_end = vmalloc_limit;
+ }
}
- /*
- * Check whether this memory bank would partially overlap
- * the vmalloc area.
- */
- if (bank->size > size_limit) {
- printk(KERN_NOTICE "Truncating RAM at %.8llx-%.8llx "
- "to -%.8llx (vmalloc region overlap).\n",
- (unsigned long long)bank->start,
- (unsigned long long)bank->start + bank->size - 1,
- (unsigned long long)bank->start + size_limit - 1);
- bank->size = size_limit;
- }
-#endif
- if (!bank->highmem) {
- phys_addr_t bank_end = bank->start + bank->size;
-
- if (bank_end > arm_lowmem_limit)
- arm_lowmem_limit = bank_end;
+ if (!highmem) {
+ if (block_end > arm_lowmem_limit) {
+ if (reg->size > size_limit)
+ arm_lowmem_limit = vmalloc_limit;
+ else
+ arm_lowmem_limit = block_end;
+ }
/*
* Find the first non-section-aligned page, and point
@@ -1144,35 +1131,15 @@
* occurs before any free memory is mapped.
*/
if (!memblock_limit) {
- if (!IS_ALIGNED(bank->start, SECTION_SIZE))
- memblock_limit = bank->start;
- else if (!IS_ALIGNED(bank_end, SECTION_SIZE))
- memblock_limit = bank_end;
+ if (!IS_ALIGNED(block_start, SECTION_SIZE))
+ memblock_limit = block_start;
+ else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+ memblock_limit = arm_lowmem_limit;
}
- }
- j++;
- }
-#ifdef CONFIG_HIGHMEM
- if (highmem) {
- const char *reason = NULL;
- if (cache_is_vipt_aliasing()) {
- /*
- * Interactions between kmap and other mappings
- * make highmem support with aliasing VIPT caches
- * rather difficult.
- */
- reason = "with VIPT aliasing cache";
- }
- if (reason) {
- printk(KERN_CRIT "HIGHMEM is not supported %s, ignoring high memory\n",
- reason);
- while (j > 0 && meminfo.bank[j - 1].highmem)
- j--;
}
}
-#endif
- meminfo.nr_banks = j;
+
high_memory = __va(arm_lowmem_limit - 1) + 1;
/*
@@ -1359,6 +1326,9 @@
#ifdef CONFIG_HIGHMEM
pkmap_page_table = early_pte_alloc(pmd_off_k(PKMAP_BASE),
PKMAP_BASE, _PAGE_KERNEL_TABLE);
+
+ fixmap_page_table = early_pte_alloc(pmd_off_k(FIXADDR_START),
+ FIXADDR_START, _PAGE_KERNEL_TABLE);
#endif
}
@@ -1461,7 +1431,7 @@
* just complicate the code.
*/
flush_cache_louis();
- dsb();
+ dsb(ishst);
isb();
/* remap level 1 table */
diff --git a/arch/arm/mm/nommu.c b/arch/arm/mm/nommu.c
index 55764a7e..da1874f 100644
--- a/arch/arm/mm/nommu.c
+++ b/arch/arm/mm/nommu.c
@@ -88,30 +88,35 @@
void __init sanity_check_meminfo_mpu(void)
{
int i;
- struct membank *bank = meminfo.bank;
phys_addr_t phys_offset = PHYS_OFFSET;
phys_addr_t aligned_region_size, specified_mem_size, rounded_mem_size;
+ struct memblock_region *reg;
+ bool first = true;
+ phys_addr_t mem_start;
+ phys_addr_t mem_end;
- /* Initially only use memory continuous from PHYS_OFFSET */
- if (bank_phys_start(&bank[0]) != phys_offset)
- panic("First memory bank must be contiguous from PHYS_OFFSET");
+ for_each_memblock(memory, reg) {
+ if (first) {
+ /*
+ * Initially only use memory continuous from
+ * PHYS_OFFSET */
+ if (reg->base != phys_offset)
+ panic("First memory bank must be contiguous from PHYS_OFFSET");
- /* Banks have already been sorted by start address */
- for (i = 1; i < meminfo.nr_banks; i++) {
- if (bank[i].start <= bank_phys_end(&bank[0]) &&
- bank_phys_end(&bank[i]) > bank_phys_end(&bank[0])) {
- bank[0].size = bank_phys_end(&bank[i]) - bank[0].start;
+ mem_start = reg->base;
+ mem_end = reg->base + reg->size;
+ specified_mem_size = reg->size;
+ first = false;
} else {
- pr_notice("Ignoring RAM after 0x%.8lx. "
- "First non-contiguous (ignored) bank start: 0x%.8lx\n",
- (unsigned long)bank_phys_end(&bank[0]),
- (unsigned long)bank_phys_start(&bank[i]));
- break;
+ /*
+ * memblock auto merges contiguous blocks, remove
+ * all blocks afterwards
+ */
+ pr_notice("Ignoring RAM after %pa, memory at %pa ignored\n",
+ &mem_start, ®->base);
+ memblock_remove(reg->base, reg->size);
}
}
- /* All contiguous banks are now merged in to the first bank */
- meminfo.nr_banks = 1;
- specified_mem_size = bank[0].size;
/*
* MPU has curious alignment requirements: Size must be power of 2, and
@@ -128,23 +133,24 @@
*/
aligned_region_size = (phys_offset - 1) ^ (phys_offset);
/* Find the max power-of-two sized region that fits inside our bank */
- rounded_mem_size = (1 << __fls(bank[0].size)) - 1;
+ rounded_mem_size = (1 << __fls(specified_mem_size)) - 1;
/* The actual region size is the smaller of the two */
aligned_region_size = aligned_region_size < rounded_mem_size
? aligned_region_size + 1
: rounded_mem_size + 1;
- if (aligned_region_size != specified_mem_size)
- pr_warn("Truncating memory from 0x%.8lx to 0x%.8lx (MPU region constraints)",
- (unsigned long)specified_mem_size,
- (unsigned long)aligned_region_size);
+ if (aligned_region_size != specified_mem_size) {
+ pr_warn("Truncating memory from %pa to %pa (MPU region constraints)",
+ &specified_mem_size, &aligned_region_size);
+ memblock_remove(mem_start + aligned_region_size,
+ specified_mem_size - aligned_round_size);
- meminfo.bank[0].size = aligned_region_size;
- pr_debug("MPU Region from 0x%.8lx size 0x%.8lx (end 0x%.8lx))\n",
- (unsigned long)phys_offset,
- (unsigned long)aligned_region_size,
- (unsigned long)bank_phys_end(&bank[0]));
+ mem_end = mem_start + aligned_region_size;
+ }
+
+ pr_debug("MPU Region from %pa size %pa (end %pa))\n",
+ &phys_offset, &aligned_region_size, &mem_end);
}
@@ -292,7 +298,7 @@
{
phys_addr_t end;
sanity_check_meminfo_mpu();
- end = bank_phys_end(&meminfo.bank[meminfo.nr_banks - 1]);
+ end = memblock_end_of_DRAM();
high_memory = __va(end - 1) + 1;
}
diff --git a/arch/arm/mm/proc-v7-3level.S b/arch/arm/mm/proc-v7-3level.S
index 01a719e..22e3ad6 100644
--- a/arch/arm/mm/proc-v7-3level.S
+++ b/arch/arm/mm/proc-v7-3level.S
@@ -64,6 +64,14 @@
mov pc, lr
ENDPROC(cpu_v7_switch_mm)
+#ifdef __ARMEB__
+#define rl r3
+#define rh r2
+#else
+#define rl r2
+#define rh r3
+#endif
+
/*
* cpu_v7_set_pte_ext(ptep, pte)
*
@@ -73,13 +81,13 @@
*/
ENTRY(cpu_v7_set_pte_ext)
#ifdef CONFIG_MMU
- tst r2, #L_PTE_VALID
+ tst rl, #L_PTE_VALID
beq 1f
- tst r3, #1 << (57 - 32) @ L_PTE_NONE
- bicne r2, #L_PTE_VALID
+ tst rh, #1 << (57 - 32) @ L_PTE_NONE
+ bicne rl, #L_PTE_VALID
bne 1f
- tst r3, #1 << (55 - 32) @ L_PTE_DIRTY
- orreq r2, #L_PTE_RDONLY
+ tst rh, #1 << (55 - 32) @ L_PTE_DIRTY
+ orreq rl, #L_PTE_RDONLY
1: strd r2, r3, [r0]
ALT_SMP(W(nop))
ALT_UP (mcr p15, 0, r0, c7, c10, 1) @ flush_pte
diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S
index 195731d..3db2c2f 100644
--- a/arch/arm/mm/proc-v7.S
+++ b/arch/arm/mm/proc-v7.S
@@ -169,9 +169,31 @@
globl_equ cpu_pj4b_do_idle, cpu_v7_do_idle
#endif
globl_equ cpu_pj4b_dcache_clean_area, cpu_v7_dcache_clean_area
- globl_equ cpu_pj4b_do_suspend, cpu_v7_do_suspend
- globl_equ cpu_pj4b_do_resume, cpu_v7_do_resume
- globl_equ cpu_pj4b_suspend_size, cpu_v7_suspend_size
+#ifdef CONFIG_ARM_CPU_SUSPEND
+ENTRY(cpu_pj4b_do_suspend)
+ stmfd sp!, {r6 - r10}
+ mrc p15, 1, r6, c15, c1, 0 @ save CP15 - extra features
+ mrc p15, 1, r7, c15, c2, 0 @ save CP15 - Aux Func Modes Ctrl 0
+ mrc p15, 1, r8, c15, c1, 2 @ save CP15 - Aux Debug Modes Ctrl 2
+ mrc p15, 1, r9, c15, c1, 1 @ save CP15 - Aux Debug Modes Ctrl 1
+ mrc p15, 0, r10, c9, c14, 0 @ save CP15 - PMC
+ stmia r0!, {r6 - r10}
+ ldmfd sp!, {r6 - r10}
+ b cpu_v7_do_suspend
+ENDPROC(cpu_pj4b_do_suspend)
+
+ENTRY(cpu_pj4b_do_resume)
+ ldmia r0!, {r6 - r10}
+ mcr p15, 1, r6, c15, c1, 0 @ save CP15 - extra features
+ mcr p15, 1, r7, c15, c2, 0 @ save CP15 - Aux Func Modes Ctrl 0
+ mcr p15, 1, r8, c15, c1, 2 @ save CP15 - Aux Debug Modes Ctrl 2
+ mcr p15, 1, r9, c15, c1, 1 @ save CP15 - Aux Debug Modes Ctrl 1
+ mcr p15, 0, r10, c9, c14, 0 @ save CP15 - PMC
+ b cpu_v7_do_resume
+ENDPROC(cpu_pj4b_do_resume)
+#endif
+.globl cpu_pj4b_suspend_size
+.equ cpu_pj4b_suspend_size, 4 * 14
#endif
@@ -194,6 +216,7 @@
__v7_ca7mp_setup:
__v7_ca12mp_setup:
__v7_ca15mp_setup:
+__v7_ca17mp_setup:
mov r10, #0
1:
#ifdef CONFIG_SMP
@@ -505,6 +528,16 @@
.size __v7_ca15mp_proc_info, . - __v7_ca15mp_proc_info
/*
+ * ARM Ltd. Cortex A17 processor.
+ */
+ .type __v7_ca17mp_proc_info, #object
+__v7_ca17mp_proc_info:
+ .long 0x410fc0e0
+ .long 0xff0ffff0
+ __v7_proc __v7_ca17mp_setup
+ .size __v7_ca17mp_proc_info, . - __v7_ca17mp_proc_info
+
+ /*
* Qualcomm Inc. Krait processors.
*/
.type __krait_proc_info, #object
diff --git a/arch/arm/plat-samsung/s5p-sleep.S b/arch/arm/plat-samsung/s5p-sleep.S
index c500165..25c68ceb 100644
--- a/arch/arm/plat-samsung/s5p-sleep.S
+++ b/arch/arm/plat-samsung/s5p-sleep.S
@@ -22,7 +22,6 @@
*/
#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
.data
.align
diff --git a/arch/arm/vfp/entry.S b/arch/arm/vfp/entry.S
index f0759e7..fe6ca57 100644
--- a/arch/arm/vfp/entry.S
+++ b/arch/arm/vfp/entry.S
@@ -22,11 +22,10 @@
@ r9 = normal "successful" return address
@ r10 = this threads thread_info structure
@ lr = unrecognised instruction return address
-@ IRQs disabled.
+@ IRQs enabled.
@
ENTRY(do_vfp)
inc_preempt_count r10, r4
- enable_irq
ldr r4, .LCvfp
ldr r11, [r10, #TI_CPU] @ CPU number
add r10, r10, #TI_VFPSTATE @ r10 = workspace
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index f5f63b71..7295419 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -30,12 +30,17 @@
select HAVE_ARCH_JUMP_LABEL
select HAVE_ARCH_KGDB
select HAVE_ARCH_TRACEHOOK
+ select HAVE_C_RECORDMCOUNT
select HAVE_DEBUG_BUGVERBOSE
select HAVE_DEBUG_KMEMLEAK
select HAVE_DMA_API_DEBUG
select HAVE_DMA_ATTRS
select HAVE_DMA_CONTIGUOUS
+ select HAVE_DYNAMIC_FTRACE
select HAVE_EFFICIENT_UNALIGNED_ACCESS
+ select HAVE_FTRACE_MCOUNT_RECORD
+ select HAVE_FUNCTION_TRACER
+ select HAVE_FUNCTION_GRAPH_TRACER
select HAVE_GENERIC_DMA_COHERENT
select HAVE_HW_BREAKPOINT if PERF_EVENTS
select HAVE_MEMBLOCK
@@ -43,6 +48,7 @@
select HAVE_PERF_EVENTS
select HAVE_PERF_REGS
select HAVE_PERF_USER_STACK_DUMP
+ select HAVE_SYSCALL_TRACEPOINTS
select IRQ_DOMAIN
select MODULES_USE_ELF_RELA
select NO_BOOTMEM
@@ -245,6 +251,9 @@
config HAVE_ARCH_TRANSPARENT_HUGEPAGE
def_bool y
+config ARCH_HAS_CACHE_LINE_SIZE
+ def_bool y
+
source "mm/Kconfig"
config XEN_DOM0
@@ -283,6 +292,20 @@
This is useful if you cannot or don't want to change the
command-line options your boot loader passes to the kernel.
+config EFI
+ bool "UEFI runtime support"
+ depends on OF && !CPU_BIG_ENDIAN
+ select LIBFDT
+ select UCS2_STRING
+ select EFI_PARAMS_FROM_FDT
+ default y
+ help
+ This option provides support for runtime services provided
+ by UEFI firmware (such as non-volatile variables, realtime
+ clock, and platform reset). A UEFI stub is also provided to
+ allow the kernel to be booted as an EFI application. This
+ is only useful on systems that have UEFI firmware.
+
endmenu
menu "Userspace binary formats"
@@ -334,6 +357,8 @@
source "drivers/Kconfig"
+source "drivers/firmware/Kconfig"
+
source "fs/Kconfig"
source "arch/arm64/kvm/Kconfig"
@@ -343,5 +368,8 @@
source "security/Kconfig"
source "crypto/Kconfig"
+if CRYPTO
+source "arch/arm64/crypto/Kconfig"
+endif
source "lib/Kconfig"
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 2fceb71..8185a91 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -45,6 +45,7 @@
core-y += arch/arm64/kernel/ arch/arm64/mm/
core-$(CONFIG_KVM) += arch/arm64/kvm/
core-$(CONFIG_XEN) += arch/arm64/xen/
+core-$(CONFIG_CRYPTO) += arch/arm64/crypto/
libs-y := arch/arm64/lib/ $(libs-y)
libs-y += $(LIBGCC)
diff --git a/arch/arm64/boot/dts/apm-storm.dtsi b/arch/arm64/boot/dts/apm-storm.dtsi
index f8c40a6..c5f0a47 100644
--- a/arch/arm64/boot/dts/apm-storm.dtsi
+++ b/arch/arm64/boot/dts/apm-storm.dtsi
@@ -257,6 +257,19 @@
enable-offset = <0x0>;
enable-mask = <0x39>;
};
+
+ rtcclk: rtcclk@17000000 {
+ compatible = "apm,xgene-device-clock";
+ #clock-cells = <1>;
+ clocks = <&socplldiv2 0>;
+ reg = <0x0 0x17000000 0x0 0x2000>;
+ reg-names = "csr-reg";
+ csr-offset = <0xc>;
+ csr-mask = <0x2>;
+ enable-offset = <0x10>;
+ enable-mask = <0x2>;
+ clock-output-names = "rtcclk";
+ };
};
serial0: serial@1c020000 {
@@ -342,5 +355,13 @@
phys = <&phy3 0>;
phy-names = "sata-phy";
};
+
+ rtc: rtc@10510000 {
+ compatible = "apm,xgene-rtc";
+ reg = <0x0 0x10510000 0x0 0x400>;
+ interrupts = <0x0 0x46 0x4>;
+ #clock-cells = <1>;
+ clocks = <&rtcclk 0>;
+ };
};
};
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index 7959dd0..157e1d8 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -1,11 +1,11 @@
# CONFIG_LOCALVERSION_AUTO is not set
-# CONFIG_SWAP is not set
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
+CONFIG_AUDIT=y
+CONFIG_NO_HZ_IDLE=y
+CONFIG_HIGH_RES_TIMERS=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_BSD_PROCESS_ACCT_V3=y
-CONFIG_NO_HZ=y
-CONFIG_HIGH_RES_TIMERS=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=14
@@ -27,6 +27,7 @@
CONFIG_ARCH_XGENE=y
CONFIG_SMP=y
CONFIG_PREEMPT=y
+CONFIG_TRANSPARENT_HUGEPAGE=y
CONFIG_CMA=y
CONFIG_CMDLINE="console=ttyAMA0"
# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
@@ -44,7 +45,7 @@
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_DMA_CMA=y
-CONFIG_SCSI=y
+CONFIG_VIRTIO_BLK=y
# CONFIG_SCSI_PROC_FS is not set
CONFIG_BLK_DEV_SD=y
# CONFIG_SCSI_LOWLEVEL is not set
@@ -56,20 +57,18 @@
CONFIG_SMSC911X=y
# CONFIG_WLAN is not set
CONFIG_INPUT_EVDEV=y
-# CONFIG_SERIO_I8042 is not set
# CONFIG_SERIO_SERPORT is not set
CONFIG_LEGACY_PTY_COUNT=16
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
-CONFIG_SERIAL_OF_PLATFORM=y
CONFIG_SERIAL_AMBA_PL011=y
CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
+CONFIG_SERIAL_OF_PLATFORM=y
# CONFIG_HW_RANDOM is not set
# CONFIG_HWMON is not set
CONFIG_REGULATOR=y
CONFIG_REGULATOR_FIXED_VOLTAGE=y
CONFIG_FB=y
-# CONFIG_VGA_CONSOLE is not set
CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_LOGO=y
# CONFIG_LOGO_LINUX_MONO is not set
@@ -79,27 +78,38 @@
CONFIG_USB_STORAGE=y
CONFIG_MMC=y
CONFIG_MMC_ARMMMCI=y
+CONFIG_VIRTIO_MMIO=y
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_EXT2_FS=y
CONFIG_EXT3_FS=y
-CONFIG_EXT4_FS=y
# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
# CONFIG_EXT3_FS_XATTR is not set
+CONFIG_EXT4_FS=y
CONFIG_FUSE_FS=y
CONFIG_CUSE=y
CONFIG_VFAT_FS=y
CONFIG_TMPFS=y
+CONFIG_HUGETLBFS=y
# CONFIG_MISC_FILESYSTEMS is not set
CONFIG_NFS_FS=y
CONFIG_ROOT_NFS=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
-CONFIG_MAGIC_SYSRQ=y
-CONFIG_DEBUG_FS=y
-CONFIG_DEBUG_KERNEL=y
-# CONFIG_SCHED_DEBUG is not set
+CONFIG_VIRTUALIZATION=y
+CONFIG_KVM=y
CONFIG_DEBUG_INFO=y
+CONFIG_DEBUG_FS=y
+CONFIG_MAGIC_SYSRQ=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_LOCKUP_DETECTOR=y
+# CONFIG_SCHED_DEBUG is not set
# CONFIG_FTRACE is not set
-CONFIG_ATOMIC64_SELFTEST=y
-CONFIG_VIRTIO_MMIO=y
-CONFIG_VIRTIO_BLK=y
+CONFIG_CRYPTO_ANSI_CPRNG=y
+CONFIG_ARM64_CRYPTO=y
+CONFIG_CRYPTO_SHA1_ARM64_CE=y
+CONFIG_CRYPTO_SHA2_ARM64_CE=y
+CONFIG_CRYPTO_GHASH_ARM64_CE=y
+CONFIG_CRYPTO_AES_ARM64_CE=y
+CONFIG_CRYPTO_AES_ARM64_CE_CCM=y
+CONFIG_CRYPTO_AES_ARM64_CE_BLK=y
+CONFIG_CRYPTO_AES_ARM64_NEON_BLK=y
diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig
new file mode 100644
index 0000000..5562652
--- /dev/null
+++ b/arch/arm64/crypto/Kconfig
@@ -0,0 +1,53 @@
+
+menuconfig ARM64_CRYPTO
+ bool "ARM64 Accelerated Cryptographic Algorithms"
+ depends on ARM64
+ help
+ Say Y here to choose from a selection of cryptographic algorithms
+ implemented using ARM64 specific CPU features or instructions.
+
+if ARM64_CRYPTO
+
+config CRYPTO_SHA1_ARM64_CE
+ tristate "SHA-1 digest algorithm (ARMv8 Crypto Extensions)"
+ depends on ARM64 && KERNEL_MODE_NEON
+ select CRYPTO_HASH
+
+config CRYPTO_SHA2_ARM64_CE
+ tristate "SHA-224/SHA-256 digest algorithm (ARMv8 Crypto Extensions)"
+ depends on ARM64 && KERNEL_MODE_NEON
+ select CRYPTO_HASH
+
+config CRYPTO_GHASH_ARM64_CE
+ tristate "GHASH (for GCM chaining mode) using ARMv8 Crypto Extensions"
+ depends on ARM64 && KERNEL_MODE_NEON
+ select CRYPTO_HASH
+
+config CRYPTO_AES_ARM64_CE
+ tristate "AES core cipher using ARMv8 Crypto Extensions"
+ depends on ARM64 && KERNEL_MODE_NEON
+ select CRYPTO_ALGAPI
+ select CRYPTO_AES
+
+config CRYPTO_AES_ARM64_CE_CCM
+ tristate "AES in CCM mode using ARMv8 Crypto Extensions"
+ depends on ARM64 && KERNEL_MODE_NEON
+ select CRYPTO_ALGAPI
+ select CRYPTO_AES
+ select CRYPTO_AEAD
+
+config CRYPTO_AES_ARM64_CE_BLK
+ tristate "AES in ECB/CBC/CTR/XTS modes using ARMv8 Crypto Extensions"
+ depends on ARM64 && KERNEL_MODE_NEON
+ select CRYPTO_BLKCIPHER
+ select CRYPTO_AES
+ select CRYPTO_ABLK_HELPER
+
+config CRYPTO_AES_ARM64_NEON_BLK
+ tristate "AES in ECB/CBC/CTR/XTS modes using NEON instructions"
+ depends on ARM64 && KERNEL_MODE_NEON
+ select CRYPTO_BLKCIPHER
+ select CRYPTO_AES
+ select CRYPTO_ABLK_HELPER
+
+endif
diff --git a/arch/arm64/crypto/Makefile b/arch/arm64/crypto/Makefile
new file mode 100644
index 0000000..2070a56
--- /dev/null
+++ b/arch/arm64/crypto/Makefile
@@ -0,0 +1,38 @@
+#
+# linux/arch/arm64/crypto/Makefile
+#
+# Copyright (C) 2014 Linaro Ltd <ard.biesheuvel@linaro.org>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+
+obj-$(CONFIG_CRYPTO_SHA1_ARM64_CE) += sha1-ce.o
+sha1-ce-y := sha1-ce-glue.o sha1-ce-core.o
+
+obj-$(CONFIG_CRYPTO_SHA2_ARM64_CE) += sha2-ce.o
+sha2-ce-y := sha2-ce-glue.o sha2-ce-core.o
+
+obj-$(CONFIG_CRYPTO_GHASH_ARM64_CE) += ghash-ce.o
+ghash-ce-y := ghash-ce-glue.o ghash-ce-core.o
+
+obj-$(CONFIG_CRYPTO_AES_ARM64_CE) += aes-ce-cipher.o
+CFLAGS_aes-ce-cipher.o += -march=armv8-a+crypto
+
+obj-$(CONFIG_CRYPTO_AES_ARM64_CE_CCM) += aes-ce-ccm.o
+aes-ce-ccm-y := aes-ce-ccm-glue.o aes-ce-ccm-core.o
+
+obj-$(CONFIG_CRYPTO_AES_ARM64_CE_BLK) += aes-ce-blk.o
+aes-ce-blk-y := aes-glue-ce.o aes-ce.o
+
+obj-$(CONFIG_CRYPTO_AES_ARM64_NEON_BLK) += aes-neon-blk.o
+aes-neon-blk-y := aes-glue-neon.o aes-neon.o
+
+AFLAGS_aes-ce.o := -DINTERLEAVE=2 -DINTERLEAVE_INLINE
+AFLAGS_aes-neon.o := -DINTERLEAVE=4
+
+CFLAGS_aes-glue-ce.o := -DUSE_V8_CRYPTO_EXTENSIONS
+
+$(obj)/aes-glue-%.o: $(src)/aes-glue.c FORCE
+ $(call if_changed_dep,cc_o_c)
diff --git a/arch/arm64/crypto/aes-ce-ccm-core.S b/arch/arm64/crypto/aes-ce-ccm-core.S
new file mode 100644
index 0000000..432e484
--- /dev/null
+++ b/arch/arm64/crypto/aes-ce-ccm-core.S
@@ -0,0 +1,222 @@
+/*
+ * aesce-ccm-core.S - AES-CCM transform for ARMv8 with Crypto Extensions
+ *
+ * Copyright (C) 2013 - 2014 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+
+ .text
+ .arch armv8-a+crypto
+
+ /*
+ * void ce_aes_ccm_auth_data(u8 mac[], u8 const in[], u32 abytes,
+ * u32 *macp, u8 const rk[], u32 rounds);
+ */
+ENTRY(ce_aes_ccm_auth_data)
+ ldr w8, [x3] /* leftover from prev round? */
+ ld1 {v0.2d}, [x0] /* load mac */
+ cbz w8, 1f
+ sub w8, w8, #16
+ eor v1.16b, v1.16b, v1.16b
+0: ldrb w7, [x1], #1 /* get 1 byte of input */
+ subs w2, w2, #1
+ add w8, w8, #1
+ ins v1.b[0], w7
+ ext v1.16b, v1.16b, v1.16b, #1 /* rotate in the input bytes */
+ beq 8f /* out of input? */
+ cbnz w8, 0b
+ eor v0.16b, v0.16b, v1.16b
+1: ld1 {v3.2d}, [x4] /* load first round key */
+ prfm pldl1strm, [x1]
+ cmp w5, #12 /* which key size? */
+ add x6, x4, #16
+ sub w7, w5, #2 /* modified # of rounds */
+ bmi 2f
+ bne 5f
+ mov v5.16b, v3.16b
+ b 4f
+2: mov v4.16b, v3.16b
+ ld1 {v5.2d}, [x6], #16 /* load 2nd round key */
+3: aese v0.16b, v4.16b
+ aesmc v0.16b, v0.16b
+4: ld1 {v3.2d}, [x6], #16 /* load next round key */
+ aese v0.16b, v5.16b
+ aesmc v0.16b, v0.16b
+5: ld1 {v4.2d}, [x6], #16 /* load next round key */
+ subs w7, w7, #3
+ aese v0.16b, v3.16b
+ aesmc v0.16b, v0.16b
+ ld1 {v5.2d}, [x6], #16 /* load next round key */
+ bpl 3b
+ aese v0.16b, v4.16b
+ subs w2, w2, #16 /* last data? */
+ eor v0.16b, v0.16b, v5.16b /* final round */
+ bmi 6f
+ ld1 {v1.16b}, [x1], #16 /* load next input block */
+ eor v0.16b, v0.16b, v1.16b /* xor with mac */
+ bne 1b
+6: st1 {v0.2d}, [x0] /* store mac */
+ beq 10f
+ adds w2, w2, #16
+ beq 10f
+ mov w8, w2
+7: ldrb w7, [x1], #1
+ umov w6, v0.b[0]
+ eor w6, w6, w7
+ strb w6, [x0], #1
+ subs w2, w2, #1
+ beq 10f
+ ext v0.16b, v0.16b, v0.16b, #1 /* rotate out the mac bytes */
+ b 7b
+8: mov w7, w8
+ add w8, w8, #16
+9: ext v1.16b, v1.16b, v1.16b, #1
+ adds w7, w7, #1
+ bne 9b
+ eor v0.16b, v0.16b, v1.16b
+ st1 {v0.2d}, [x0]
+10: str w8, [x3]
+ ret
+ENDPROC(ce_aes_ccm_auth_data)
+
+ /*
+ * void ce_aes_ccm_final(u8 mac[], u8 const ctr[], u8 const rk[],
+ * u32 rounds);
+ */
+ENTRY(ce_aes_ccm_final)
+ ld1 {v3.2d}, [x2], #16 /* load first round key */
+ ld1 {v0.2d}, [x0] /* load mac */
+ cmp w3, #12 /* which key size? */
+ sub w3, w3, #2 /* modified # of rounds */
+ ld1 {v1.2d}, [x1] /* load 1st ctriv */
+ bmi 0f
+ bne 3f
+ mov v5.16b, v3.16b
+ b 2f
+0: mov v4.16b, v3.16b
+1: ld1 {v5.2d}, [x2], #16 /* load next round key */
+ aese v0.16b, v4.16b
+ aese v1.16b, v4.16b
+ aesmc v0.16b, v0.16b
+ aesmc v1.16b, v1.16b
+2: ld1 {v3.2d}, [x2], #16 /* load next round key */
+ aese v0.16b, v5.16b
+ aese v1.16b, v5.16b
+ aesmc v0.16b, v0.16b
+ aesmc v1.16b, v1.16b
+3: ld1 {v4.2d}, [x2], #16 /* load next round key */
+ subs w3, w3, #3
+ aese v0.16b, v3.16b
+ aese v1.16b, v3.16b
+ aesmc v0.16b, v0.16b
+ aesmc v1.16b, v1.16b
+ bpl 1b
+ aese v0.16b, v4.16b
+ aese v1.16b, v4.16b
+ /* final round key cancels out */
+ eor v0.16b, v0.16b, v1.16b /* en-/decrypt the mac */
+ st1 {v0.2d}, [x0] /* store result */
+ ret
+ENDPROC(ce_aes_ccm_final)
+
+ .macro aes_ccm_do_crypt,enc
+ ldr x8, [x6, #8] /* load lower ctr */
+ ld1 {v0.2d}, [x5] /* load mac */
+ rev x8, x8 /* keep swabbed ctr in reg */
+0: /* outer loop */
+ ld1 {v1.1d}, [x6] /* load upper ctr */
+ prfm pldl1strm, [x1]
+ add x8, x8, #1
+ rev x9, x8
+ cmp w4, #12 /* which key size? */
+ sub w7, w4, #2 /* get modified # of rounds */
+ ins v1.d[1], x9 /* no carry in lower ctr */
+ ld1 {v3.2d}, [x3] /* load first round key */
+ add x10, x3, #16
+ bmi 1f
+ bne 4f
+ mov v5.16b, v3.16b
+ b 3f
+1: mov v4.16b, v3.16b
+ ld1 {v5.2d}, [x10], #16 /* load 2nd round key */
+2: /* inner loop: 3 rounds, 2x interleaved */
+ aese v0.16b, v4.16b
+ aese v1.16b, v4.16b
+ aesmc v0.16b, v0.16b
+ aesmc v1.16b, v1.16b
+3: ld1 {v3.2d}, [x10], #16 /* load next round key */
+ aese v0.16b, v5.16b
+ aese v1.16b, v5.16b
+ aesmc v0.16b, v0.16b
+ aesmc v1.16b, v1.16b
+4: ld1 {v4.2d}, [x10], #16 /* load next round key */
+ subs w7, w7, #3
+ aese v0.16b, v3.16b
+ aese v1.16b, v3.16b
+ aesmc v0.16b, v0.16b
+ aesmc v1.16b, v1.16b
+ ld1 {v5.2d}, [x10], #16 /* load next round key */
+ bpl 2b
+ aese v0.16b, v4.16b
+ aese v1.16b, v4.16b
+ subs w2, w2, #16
+ bmi 6f /* partial block? */
+ ld1 {v2.16b}, [x1], #16 /* load next input block */
+ .if \enc == 1
+ eor v2.16b, v2.16b, v5.16b /* final round enc+mac */
+ eor v1.16b, v1.16b, v2.16b /* xor with crypted ctr */
+ .else
+ eor v2.16b, v2.16b, v1.16b /* xor with crypted ctr */
+ eor v1.16b, v2.16b, v5.16b /* final round enc */
+ .endif
+ eor v0.16b, v0.16b, v2.16b /* xor mac with pt ^ rk[last] */
+ st1 {v1.16b}, [x0], #16 /* write output block */
+ bne 0b
+ rev x8, x8
+ st1 {v0.2d}, [x5] /* store mac */
+ str x8, [x6, #8] /* store lsb end of ctr (BE) */
+5: ret
+
+6: eor v0.16b, v0.16b, v5.16b /* final round mac */
+ eor v1.16b, v1.16b, v5.16b /* final round enc */
+ st1 {v0.2d}, [x5] /* store mac */
+ add w2, w2, #16 /* process partial tail block */
+7: ldrb w9, [x1], #1 /* get 1 byte of input */
+ umov w6, v1.b[0] /* get top crypted ctr byte */
+ umov w7, v0.b[0] /* get top mac byte */
+ .if \enc == 1
+ eor w7, w7, w9
+ eor w9, w9, w6
+ .else
+ eor w9, w9, w6
+ eor w7, w7, w9
+ .endif
+ strb w9, [x0], #1 /* store out byte */
+ strb w7, [x5], #1 /* store mac byte */
+ subs w2, w2, #1
+ beq 5b
+ ext v0.16b, v0.16b, v0.16b, #1 /* shift out mac byte */
+ ext v1.16b, v1.16b, v1.16b, #1 /* shift out ctr byte */
+ b 7b
+ .endm
+
+ /*
+ * void ce_aes_ccm_encrypt(u8 out[], u8 const in[], u32 cbytes,
+ * u8 const rk[], u32 rounds, u8 mac[],
+ * u8 ctr[]);
+ * void ce_aes_ccm_decrypt(u8 out[], u8 const in[], u32 cbytes,
+ * u8 const rk[], u32 rounds, u8 mac[],
+ * u8 ctr[]);
+ */
+ENTRY(ce_aes_ccm_encrypt)
+ aes_ccm_do_crypt 1
+ENDPROC(ce_aes_ccm_encrypt)
+
+ENTRY(ce_aes_ccm_decrypt)
+ aes_ccm_do_crypt 0
+ENDPROC(ce_aes_ccm_decrypt)
diff --git a/arch/arm64/crypto/aes-ce-ccm-glue.c b/arch/arm64/crypto/aes-ce-ccm-glue.c
new file mode 100644
index 0000000..9e6cdde9
--- /dev/null
+++ b/arch/arm64/crypto/aes-ce-ccm-glue.c
@@ -0,0 +1,297 @@
+/*
+ * aes-ccm-glue.c - AES-CCM transform for ARMv8 with Crypto Extensions
+ *
+ * Copyright (C) 2013 - 2014 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <asm/neon.h>
+#include <asm/unaligned.h>
+#include <crypto/aes.h>
+#include <crypto/algapi.h>
+#include <crypto/scatterwalk.h>
+#include <linux/crypto.h>
+#include <linux/module.h>
+
+static int num_rounds(struct crypto_aes_ctx *ctx)
+{
+ /*
+ * # of rounds specified by AES:
+ * 128 bit key 10 rounds
+ * 192 bit key 12 rounds
+ * 256 bit key 14 rounds
+ * => n byte key => 6 + (n/4) rounds
+ */
+ return 6 + ctx->key_length / 4;
+}
+
+asmlinkage void ce_aes_ccm_auth_data(u8 mac[], u8 const in[], u32 abytes,
+ u32 *macp, u32 const rk[], u32 rounds);
+
+asmlinkage void ce_aes_ccm_encrypt(u8 out[], u8 const in[], u32 cbytes,
+ u32 const rk[], u32 rounds, u8 mac[],
+ u8 ctr[]);
+
+asmlinkage void ce_aes_ccm_decrypt(u8 out[], u8 const in[], u32 cbytes,
+ u32 const rk[], u32 rounds, u8 mac[],
+ u8 ctr[]);
+
+asmlinkage void ce_aes_ccm_final(u8 mac[], u8 const ctr[], u32 const rk[],
+ u32 rounds);
+
+static int ccm_setkey(struct crypto_aead *tfm, const u8 *in_key,
+ unsigned int key_len)
+{
+ struct crypto_aes_ctx *ctx = crypto_aead_ctx(tfm);
+ int ret;
+
+ ret = crypto_aes_expand_key(ctx, in_key, key_len);
+ if (!ret)
+ return 0;
+
+ tfm->base.crt_flags |= CRYPTO_TFM_RES_BAD_KEY_LEN;
+ return -EINVAL;
+}
+
+static int ccm_setauthsize(struct crypto_aead *tfm, unsigned int authsize)
+{
+ if ((authsize & 1) || authsize < 4)
+ return -EINVAL;
+ return 0;
+}
+
+static int ccm_init_mac(struct aead_request *req, u8 maciv[], u32 msglen)
+{
+ struct crypto_aead *aead = crypto_aead_reqtfm(req);
+ __be32 *n = (__be32 *)&maciv[AES_BLOCK_SIZE - 8];
+ u32 l = req->iv[0] + 1;
+
+ /* verify that CCM dimension 'L' is set correctly in the IV */
+ if (l < 2 || l > 8)
+ return -EINVAL;
+
+ /* verify that msglen can in fact be represented in L bytes */
+ if (l < 4 && msglen >> (8 * l))
+ return -EOVERFLOW;
+
+ /*
+ * Even if the CCM spec allows L values of up to 8, the Linux cryptoapi
+ * uses a u32 type to represent msglen so the top 4 bytes are always 0.
+ */
+ n[0] = 0;
+ n[1] = cpu_to_be32(msglen);
+
+ memcpy(maciv, req->iv, AES_BLOCK_SIZE - l);
+
+ /*
+ * Meaning of byte 0 according to CCM spec (RFC 3610/NIST 800-38C)
+ * - bits 0..2 : max # of bytes required to represent msglen, minus 1
+ * (already set by caller)
+ * - bits 3..5 : size of auth tag (1 => 4 bytes, 2 => 6 bytes, etc)
+ * - bit 6 : indicates presence of authenticate-only data
+ */
+ maciv[0] |= (crypto_aead_authsize(aead) - 2) << 2;
+ if (req->assoclen)
+ maciv[0] |= 0x40;
+
+ memset(&req->iv[AES_BLOCK_SIZE - l], 0, l);
+ return 0;
+}
+
+static void ccm_calculate_auth_mac(struct aead_request *req, u8 mac[])
+{
+ struct crypto_aead *aead = crypto_aead_reqtfm(req);
+ struct crypto_aes_ctx *ctx = crypto_aead_ctx(aead);
+ struct __packed { __be16 l; __be32 h; u16 len; } ltag;
+ struct scatter_walk walk;
+ u32 len = req->assoclen;
+ u32 macp = 0;
+
+ /* prepend the AAD with a length tag */
+ if (len < 0xff00) {
+ ltag.l = cpu_to_be16(len);
+ ltag.len = 2;
+ } else {
+ ltag.l = cpu_to_be16(0xfffe);
+ put_unaligned_be32(len, <ag.h);
+ ltag.len = 6;
+ }
+
+ ce_aes_ccm_auth_data(mac, (u8 *)<ag, ltag.len, &macp, ctx->key_enc,
+ num_rounds(ctx));
+ scatterwalk_start(&walk, req->assoc);
+
+ do {
+ u32 n = scatterwalk_clamp(&walk, len);
+ u8 *p;
+
+ if (!n) {
+ scatterwalk_start(&walk, sg_next(walk.sg));
+ n = scatterwalk_clamp(&walk, len);
+ }
+ p = scatterwalk_map(&walk);
+ ce_aes_ccm_auth_data(mac, p, n, &macp, ctx->key_enc,
+ num_rounds(ctx));
+ len -= n;
+
+ scatterwalk_unmap(p);
+ scatterwalk_advance(&walk, n);
+ scatterwalk_done(&walk, 0, len);
+ } while (len);
+}
+
+static int ccm_encrypt(struct aead_request *req)
+{
+ struct crypto_aead *aead = crypto_aead_reqtfm(req);
+ struct crypto_aes_ctx *ctx = crypto_aead_ctx(aead);
+ struct blkcipher_desc desc = { .info = req->iv };
+ struct blkcipher_walk walk;
+ u8 __aligned(8) mac[AES_BLOCK_SIZE];
+ u8 buf[AES_BLOCK_SIZE];
+ u32 len = req->cryptlen;
+ int err;
+
+ err = ccm_init_mac(req, mac, len);
+ if (err)
+ return err;
+
+ kernel_neon_begin_partial(6);
+
+ if (req->assoclen)
+ ccm_calculate_auth_mac(req, mac);
+
+ /* preserve the original iv for the final round */
+ memcpy(buf, req->iv, AES_BLOCK_SIZE);
+
+ blkcipher_walk_init(&walk, req->dst, req->src, len);
+ err = blkcipher_aead_walk_virt_block(&desc, &walk, aead,
+ AES_BLOCK_SIZE);
+
+ while (walk.nbytes) {
+ u32 tail = walk.nbytes % AES_BLOCK_SIZE;
+
+ if (walk.nbytes == len)
+ tail = 0;
+
+ ce_aes_ccm_encrypt(walk.dst.virt.addr, walk.src.virt.addr,
+ walk.nbytes - tail, ctx->key_enc,
+ num_rounds(ctx), mac, walk.iv);
+
+ len -= walk.nbytes - tail;
+ err = blkcipher_walk_done(&desc, &walk, tail);
+ }
+ if (!err)
+ ce_aes_ccm_final(mac, buf, ctx->key_enc, num_rounds(ctx));
+
+ kernel_neon_end();
+
+ if (err)
+ return err;
+
+ /* copy authtag to end of dst */
+ scatterwalk_map_and_copy(mac, req->dst, req->cryptlen,
+ crypto_aead_authsize(aead), 1);
+
+ return 0;
+}
+
+static int ccm_decrypt(struct aead_request *req)
+{
+ struct crypto_aead *aead = crypto_aead_reqtfm(req);
+ struct crypto_aes_ctx *ctx = crypto_aead_ctx(aead);
+ unsigned int authsize = crypto_aead_authsize(aead);
+ struct blkcipher_desc desc = { .info = req->iv };
+ struct blkcipher_walk walk;
+ u8 __aligned(8) mac[AES_BLOCK_SIZE];
+ u8 buf[AES_BLOCK_SIZE];
+ u32 len = req->cryptlen - authsize;
+ int err;
+
+ err = ccm_init_mac(req, mac, len);
+ if (err)
+ return err;
+
+ kernel_neon_begin_partial(6);
+
+ if (req->assoclen)
+ ccm_calculate_auth_mac(req, mac);
+
+ /* preserve the original iv for the final round */
+ memcpy(buf, req->iv, AES_BLOCK_SIZE);
+
+ blkcipher_walk_init(&walk, req->dst, req->src, len);
+ err = blkcipher_aead_walk_virt_block(&desc, &walk, aead,
+ AES_BLOCK_SIZE);
+
+ while (walk.nbytes) {
+ u32 tail = walk.nbytes % AES_BLOCK_SIZE;
+
+ if (walk.nbytes == len)
+ tail = 0;
+
+ ce_aes_ccm_decrypt(walk.dst.virt.addr, walk.src.virt.addr,
+ walk.nbytes - tail, ctx->key_enc,
+ num_rounds(ctx), mac, walk.iv);
+
+ len -= walk.nbytes - tail;
+ err = blkcipher_walk_done(&desc, &walk, tail);
+ }
+ if (!err)
+ ce_aes_ccm_final(mac, buf, ctx->key_enc, num_rounds(ctx));
+
+ kernel_neon_end();
+
+ if (err)
+ return err;
+
+ /* compare calculated auth tag with the stored one */
+ scatterwalk_map_and_copy(buf, req->src, req->cryptlen - authsize,
+ authsize, 0);
+
+ if (memcmp(mac, buf, authsize))
+ return -EBADMSG;
+ return 0;
+}
+
+static struct crypto_alg ccm_aes_alg = {
+ .cra_name = "ccm(aes)",
+ .cra_driver_name = "ccm-aes-ce",
+ .cra_priority = 300,
+ .cra_flags = CRYPTO_ALG_TYPE_AEAD,
+ .cra_blocksize = 1,
+ .cra_ctxsize = sizeof(struct crypto_aes_ctx),
+ .cra_alignmask = 7,
+ .cra_type = &crypto_aead_type,
+ .cra_module = THIS_MODULE,
+ .cra_aead = {
+ .ivsize = AES_BLOCK_SIZE,
+ .maxauthsize = AES_BLOCK_SIZE,
+ .setkey = ccm_setkey,
+ .setauthsize = ccm_setauthsize,
+ .encrypt = ccm_encrypt,
+ .decrypt = ccm_decrypt,
+ }
+};
+
+static int __init aes_mod_init(void)
+{
+ if (!(elf_hwcap & HWCAP_AES))
+ return -ENODEV;
+ return crypto_register_alg(&ccm_aes_alg);
+}
+
+static void __exit aes_mod_exit(void)
+{
+ crypto_unregister_alg(&ccm_aes_alg);
+}
+
+module_init(aes_mod_init);
+module_exit(aes_mod_exit);
+
+MODULE_DESCRIPTION("Synchronous AES in CCM mode using ARMv8 Crypto Extensions");
+MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("ccm(aes)");
diff --git a/arch/arm64/crypto/aes-ce-cipher.c b/arch/arm64/crypto/aes-ce-cipher.c
new file mode 100644
index 0000000..2075e1a
--- /dev/null
+++ b/arch/arm64/crypto/aes-ce-cipher.c
@@ -0,0 +1,155 @@
+/*
+ * aes-ce-cipher.c - core AES cipher using ARMv8 Crypto Extensions
+ *
+ * Copyright (C) 2013 - 2014 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <asm/neon.h>
+#include <crypto/aes.h>
+#include <linux/cpufeature.h>
+#include <linux/crypto.h>
+#include <linux/module.h>
+
+MODULE_DESCRIPTION("Synchronous AES cipher using ARMv8 Crypto Extensions");
+MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>");
+MODULE_LICENSE("GPL v2");
+
+struct aes_block {
+ u8 b[AES_BLOCK_SIZE];
+};
+
+static int num_rounds(struct crypto_aes_ctx *ctx)
+{
+ /*
+ * # of rounds specified by AES:
+ * 128 bit key 10 rounds
+ * 192 bit key 12 rounds
+ * 256 bit key 14 rounds
+ * => n byte key => 6 + (n/4) rounds
+ */
+ return 6 + ctx->key_length / 4;
+}
+
+static void aes_cipher_encrypt(struct crypto_tfm *tfm, u8 dst[], u8 const src[])
+{
+ struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm);
+ struct aes_block *out = (struct aes_block *)dst;
+ struct aes_block const *in = (struct aes_block *)src;
+ void *dummy0;
+ int dummy1;
+
+ kernel_neon_begin_partial(4);
+
+ __asm__(" ld1 {v0.16b}, %[in] ;"
+ " ld1 {v1.2d}, [%[key]], #16 ;"
+ " cmp %w[rounds], #10 ;"
+ " bmi 0f ;"
+ " bne 3f ;"
+ " mov v3.16b, v1.16b ;"
+ " b 2f ;"
+ "0: mov v2.16b, v1.16b ;"
+ " ld1 {v3.2d}, [%[key]], #16 ;"
+ "1: aese v0.16b, v2.16b ;"
+ " aesmc v0.16b, v0.16b ;"
+ "2: ld1 {v1.2d}, [%[key]], #16 ;"
+ " aese v0.16b, v3.16b ;"
+ " aesmc v0.16b, v0.16b ;"
+ "3: ld1 {v2.2d}, [%[key]], #16 ;"
+ " subs %w[rounds], %w[rounds], #3 ;"
+ " aese v0.16b, v1.16b ;"
+ " aesmc v0.16b, v0.16b ;"
+ " ld1 {v3.2d}, [%[key]], #16 ;"
+ " bpl 1b ;"
+ " aese v0.16b, v2.16b ;"
+ " eor v0.16b, v0.16b, v3.16b ;"
+ " st1 {v0.16b}, %[out] ;"
+
+ : [out] "=Q"(*out),
+ [key] "=r"(dummy0),
+ [rounds] "=r"(dummy1)
+ : [in] "Q"(*in),
+ "1"(ctx->key_enc),
+ "2"(num_rounds(ctx) - 2)
+ : "cc");
+
+ kernel_neon_end();
+}
+
+static void aes_cipher_decrypt(struct crypto_tfm *tfm, u8 dst[], u8 const src[])
+{
+ struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm);
+ struct aes_block *out = (struct aes_block *)dst;
+ struct aes_block const *in = (struct aes_block *)src;
+ void *dummy0;
+ int dummy1;
+
+ kernel_neon_begin_partial(4);
+
+ __asm__(" ld1 {v0.16b}, %[in] ;"
+ " ld1 {v1.2d}, [%[key]], #16 ;"
+ " cmp %w[rounds], #10 ;"
+ " bmi 0f ;"
+ " bne 3f ;"
+ " mov v3.16b, v1.16b ;"
+ " b 2f ;"
+ "0: mov v2.16b, v1.16b ;"
+ " ld1 {v3.2d}, [%[key]], #16 ;"
+ "1: aesd v0.16b, v2.16b ;"
+ " aesimc v0.16b, v0.16b ;"
+ "2: ld1 {v1.2d}, [%[key]], #16 ;"
+ " aesd v0.16b, v3.16b ;"
+ " aesimc v0.16b, v0.16b ;"
+ "3: ld1 {v2.2d}, [%[key]], #16 ;"
+ " subs %w[rounds], %w[rounds], #3 ;"
+ " aesd v0.16b, v1.16b ;"
+ " aesimc v0.16b, v0.16b ;"
+ " ld1 {v3.2d}, [%[key]], #16 ;"
+ " bpl 1b ;"
+ " aesd v0.16b, v2.16b ;"
+ " eor v0.16b, v0.16b, v3.16b ;"
+ " st1 {v0.16b}, %[out] ;"
+
+ : [out] "=Q"(*out),
+ [key] "=r"(dummy0),
+ [rounds] "=r"(dummy1)
+ : [in] "Q"(*in),
+ "1"(ctx->key_dec),
+ "2"(num_rounds(ctx) - 2)
+ : "cc");
+
+ kernel_neon_end();
+}
+
+static struct crypto_alg aes_alg = {
+ .cra_name = "aes",
+ .cra_driver_name = "aes-ce",
+ .cra_priority = 300,
+ .cra_flags = CRYPTO_ALG_TYPE_CIPHER,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct crypto_aes_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_cipher = {
+ .cia_min_keysize = AES_MIN_KEY_SIZE,
+ .cia_max_keysize = AES_MAX_KEY_SIZE,
+ .cia_setkey = crypto_aes_set_key,
+ .cia_encrypt = aes_cipher_encrypt,
+ .cia_decrypt = aes_cipher_decrypt
+ }
+};
+
+static int __init aes_mod_init(void)
+{
+ return crypto_register_alg(&aes_alg);
+}
+
+static void __exit aes_mod_exit(void)
+{
+ crypto_unregister_alg(&aes_alg);
+}
+
+module_cpu_feature_match(AES, aes_mod_init);
+module_exit(aes_mod_exit);
diff --git a/arch/arm64/crypto/aes-ce.S b/arch/arm64/crypto/aes-ce.S
new file mode 100644
index 0000000..685a18f
--- /dev/null
+++ b/arch/arm64/crypto/aes-ce.S
@@ -0,0 +1,133 @@
+/*
+ * linux/arch/arm64/crypto/aes-ce.S - AES cipher for ARMv8 with
+ * Crypto Extensions
+ *
+ * Copyright (C) 2013 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+
+#define AES_ENTRY(func) ENTRY(ce_ ## func)
+#define AES_ENDPROC(func) ENDPROC(ce_ ## func)
+
+ .arch armv8-a+crypto
+
+ /* preload all round keys */
+ .macro load_round_keys, rounds, rk
+ cmp \rounds, #12
+ blo 2222f /* 128 bits */
+ beq 1111f /* 192 bits */
+ ld1 {v17.16b-v18.16b}, [\rk], #32
+1111: ld1 {v19.16b-v20.16b}, [\rk], #32
+2222: ld1 {v21.16b-v24.16b}, [\rk], #64
+ ld1 {v25.16b-v28.16b}, [\rk], #64
+ ld1 {v29.16b-v31.16b}, [\rk]
+ .endm
+
+ /* prepare for encryption with key in rk[] */
+ .macro enc_prepare, rounds, rk, ignore
+ load_round_keys \rounds, \rk
+ .endm
+
+ /* prepare for encryption (again) but with new key in rk[] */
+ .macro enc_switch_key, rounds, rk, ignore
+ load_round_keys \rounds, \rk
+ .endm
+
+ /* prepare for decryption with key in rk[] */
+ .macro dec_prepare, rounds, rk, ignore
+ load_round_keys \rounds, \rk
+ .endm
+
+ .macro do_enc_Nx, de, mc, k, i0, i1, i2, i3
+ aes\de \i0\().16b, \k\().16b
+ .ifnb \i1
+ aes\de \i1\().16b, \k\().16b
+ .ifnb \i3
+ aes\de \i2\().16b, \k\().16b
+ aes\de \i3\().16b, \k\().16b
+ .endif
+ .endif
+ aes\mc \i0\().16b, \i0\().16b
+ .ifnb \i1
+ aes\mc \i1\().16b, \i1\().16b
+ .ifnb \i3
+ aes\mc \i2\().16b, \i2\().16b
+ aes\mc \i3\().16b, \i3\().16b
+ .endif
+ .endif
+ .endm
+
+ /* up to 4 interleaved encryption rounds with the same round key */
+ .macro round_Nx, enc, k, i0, i1, i2, i3
+ .ifc \enc, e
+ do_enc_Nx e, mc, \k, \i0, \i1, \i2, \i3
+ .else
+ do_enc_Nx d, imc, \k, \i0, \i1, \i2, \i3
+ .endif
+ .endm
+
+ /* up to 4 interleaved final rounds */
+ .macro fin_round_Nx, de, k, k2, i0, i1, i2, i3
+ aes\de \i0\().16b, \k\().16b
+ .ifnb \i1
+ aes\de \i1\().16b, \k\().16b
+ .ifnb \i3
+ aes\de \i2\().16b, \k\().16b
+ aes\de \i3\().16b, \k\().16b
+ .endif
+ .endif
+ eor \i0\().16b, \i0\().16b, \k2\().16b
+ .ifnb \i1
+ eor \i1\().16b, \i1\().16b, \k2\().16b
+ .ifnb \i3
+ eor \i2\().16b, \i2\().16b, \k2\().16b
+ eor \i3\().16b, \i3\().16b, \k2\().16b
+ .endif
+ .endif
+ .endm
+
+ /* up to 4 interleaved blocks */
+ .macro do_block_Nx, enc, rounds, i0, i1, i2, i3
+ cmp \rounds, #12
+ blo 2222f /* 128 bits */
+ beq 1111f /* 192 bits */
+ round_Nx \enc, v17, \i0, \i1, \i2, \i3
+ round_Nx \enc, v18, \i0, \i1, \i2, \i3
+1111: round_Nx \enc, v19, \i0, \i1, \i2, \i3
+ round_Nx \enc, v20, \i0, \i1, \i2, \i3
+2222: .irp key, v21, v22, v23, v24, v25, v26, v27, v28, v29
+ round_Nx \enc, \key, \i0, \i1, \i2, \i3
+ .endr
+ fin_round_Nx \enc, v30, v31, \i0, \i1, \i2, \i3
+ .endm
+
+ .macro encrypt_block, in, rounds, t0, t1, t2
+ do_block_Nx e, \rounds, \in
+ .endm
+
+ .macro encrypt_block2x, i0, i1, rounds, t0, t1, t2
+ do_block_Nx e, \rounds, \i0, \i1
+ .endm
+
+ .macro encrypt_block4x, i0, i1, i2, i3, rounds, t0, t1, t2
+ do_block_Nx e, \rounds, \i0, \i1, \i2, \i3
+ .endm
+
+ .macro decrypt_block, in, rounds, t0, t1, t2
+ do_block_Nx d, \rounds, \in
+ .endm
+
+ .macro decrypt_block2x, i0, i1, rounds, t0, t1, t2
+ do_block_Nx d, \rounds, \i0, \i1
+ .endm
+
+ .macro decrypt_block4x, i0, i1, i2, i3, rounds, t0, t1, t2
+ do_block_Nx d, \rounds, \i0, \i1, \i2, \i3
+ .endm
+
+#include "aes-modes.S"
diff --git a/arch/arm64/crypto/aes-glue.c b/arch/arm64/crypto/aes-glue.c
new file mode 100644
index 0000000..60f2f4c
--- /dev/null
+++ b/arch/arm64/crypto/aes-glue.c
@@ -0,0 +1,446 @@
+/*
+ * linux/arch/arm64/crypto/aes-glue.c - wrapper code for ARMv8 AES
+ *
+ * Copyright (C) 2013 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <asm/neon.h>
+#include <asm/hwcap.h>
+#include <crypto/aes.h>
+#include <crypto/ablk_helper.h>
+#include <crypto/algapi.h>
+#include <linux/module.h>
+#include <linux/cpufeature.h>
+
+#ifdef USE_V8_CRYPTO_EXTENSIONS
+#define MODE "ce"
+#define PRIO 300
+#define aes_ecb_encrypt ce_aes_ecb_encrypt
+#define aes_ecb_decrypt ce_aes_ecb_decrypt
+#define aes_cbc_encrypt ce_aes_cbc_encrypt
+#define aes_cbc_decrypt ce_aes_cbc_decrypt
+#define aes_ctr_encrypt ce_aes_ctr_encrypt
+#define aes_xts_encrypt ce_aes_xts_encrypt
+#define aes_xts_decrypt ce_aes_xts_decrypt
+MODULE_DESCRIPTION("AES-ECB/CBC/CTR/XTS using ARMv8 Crypto Extensions");
+#else
+#define MODE "neon"
+#define PRIO 200
+#define aes_ecb_encrypt neon_aes_ecb_encrypt
+#define aes_ecb_decrypt neon_aes_ecb_decrypt
+#define aes_cbc_encrypt neon_aes_cbc_encrypt
+#define aes_cbc_decrypt neon_aes_cbc_decrypt
+#define aes_ctr_encrypt neon_aes_ctr_encrypt
+#define aes_xts_encrypt neon_aes_xts_encrypt
+#define aes_xts_decrypt neon_aes_xts_decrypt
+MODULE_DESCRIPTION("AES-ECB/CBC/CTR/XTS using ARMv8 NEON");
+MODULE_ALIAS("ecb(aes)");
+MODULE_ALIAS("cbc(aes)");
+MODULE_ALIAS("ctr(aes)");
+MODULE_ALIAS("xts(aes)");
+#endif
+
+MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>");
+MODULE_LICENSE("GPL v2");
+
+/* defined in aes-modes.S */
+asmlinkage void aes_ecb_encrypt(u8 out[], u8 const in[], u8 const rk[],
+ int rounds, int blocks, int first);
+asmlinkage void aes_ecb_decrypt(u8 out[], u8 const in[], u8 const rk[],
+ int rounds, int blocks, int first);
+
+asmlinkage void aes_cbc_encrypt(u8 out[], u8 const in[], u8 const rk[],
+ int rounds, int blocks, u8 iv[], int first);
+asmlinkage void aes_cbc_decrypt(u8 out[], u8 const in[], u8 const rk[],
+ int rounds, int blocks, u8 iv[], int first);
+
+asmlinkage void aes_ctr_encrypt(u8 out[], u8 const in[], u8 const rk[],
+ int rounds, int blocks, u8 ctr[], int first);
+
+asmlinkage void aes_xts_encrypt(u8 out[], u8 const in[], u8 const rk1[],
+ int rounds, int blocks, u8 const rk2[], u8 iv[],
+ int first);
+asmlinkage void aes_xts_decrypt(u8 out[], u8 const in[], u8 const rk1[],
+ int rounds, int blocks, u8 const rk2[], u8 iv[],
+ int first);
+
+struct crypto_aes_xts_ctx {
+ struct crypto_aes_ctx key1;
+ struct crypto_aes_ctx __aligned(8) key2;
+};
+
+static int xts_set_key(struct crypto_tfm *tfm, const u8 *in_key,
+ unsigned int key_len)
+{
+ struct crypto_aes_xts_ctx *ctx = crypto_tfm_ctx(tfm);
+ int ret;
+
+ ret = crypto_aes_expand_key(&ctx->key1, in_key, key_len / 2);
+ if (!ret)
+ ret = crypto_aes_expand_key(&ctx->key2, &in_key[key_len / 2],
+ key_len / 2);
+ if (!ret)
+ return 0;
+
+ tfm->crt_flags |= CRYPTO_TFM_RES_BAD_KEY_LEN;
+ return -EINVAL;
+}
+
+static int ecb_encrypt(struct blkcipher_desc *desc, struct scatterlist *dst,
+ struct scatterlist *src, unsigned int nbytes)
+{
+ struct crypto_aes_ctx *ctx = crypto_blkcipher_ctx(desc->tfm);
+ int err, first, rounds = 6 + ctx->key_length / 4;
+ struct blkcipher_walk walk;
+ unsigned int blocks;
+
+ desc->flags &= ~CRYPTO_TFM_REQ_MAY_SLEEP;
+ blkcipher_walk_init(&walk, dst, src, nbytes);
+ err = blkcipher_walk_virt(desc, &walk);
+
+ kernel_neon_begin();
+ for (first = 1; (blocks = (walk.nbytes / AES_BLOCK_SIZE)); first = 0) {
+ aes_ecb_encrypt(walk.dst.virt.addr, walk.src.virt.addr,
+ (u8 *)ctx->key_enc, rounds, blocks, first);
+ err = blkcipher_walk_done(desc, &walk, 0);
+ }
+ kernel_neon_end();
+ return err;
+}
+
+static int ecb_decrypt(struct blkcipher_desc *desc, struct scatterlist *dst,
+ struct scatterlist *src, unsigned int nbytes)
+{
+ struct crypto_aes_ctx *ctx = crypto_blkcipher_ctx(desc->tfm);
+ int err, first, rounds = 6 + ctx->key_length / 4;
+ struct blkcipher_walk walk;
+ unsigned int blocks;
+
+ desc->flags &= ~CRYPTO_TFM_REQ_MAY_SLEEP;
+ blkcipher_walk_init(&walk, dst, src, nbytes);
+ err = blkcipher_walk_virt(desc, &walk);
+
+ kernel_neon_begin();
+ for (first = 1; (blocks = (walk.nbytes / AES_BLOCK_SIZE)); first = 0) {
+ aes_ecb_decrypt(walk.dst.virt.addr, walk.src.virt.addr,
+ (u8 *)ctx->key_dec, rounds, blocks, first);
+ err = blkcipher_walk_done(desc, &walk, 0);
+ }
+ kernel_neon_end();
+ return err;
+}
+
+static int cbc_encrypt(struct blkcipher_desc *desc, struct scatterlist *dst,
+ struct scatterlist *src, unsigned int nbytes)
+{
+ struct crypto_aes_ctx *ctx = crypto_blkcipher_ctx(desc->tfm);
+ int err, first, rounds = 6 + ctx->key_length / 4;
+ struct blkcipher_walk walk;
+ unsigned int blocks;
+
+ desc->flags &= ~CRYPTO_TFM_REQ_MAY_SLEEP;
+ blkcipher_walk_init(&walk, dst, src, nbytes);
+ err = blkcipher_walk_virt(desc, &walk);
+
+ kernel_neon_begin();
+ for (first = 1; (blocks = (walk.nbytes / AES_BLOCK_SIZE)); first = 0) {
+ aes_cbc_encrypt(walk.dst.virt.addr, walk.src.virt.addr,
+ (u8 *)ctx->key_enc, rounds, blocks, walk.iv,
+ first);
+ err = blkcipher_walk_done(desc, &walk, 0);
+ }
+ kernel_neon_end();
+ return err;
+}
+
+static int cbc_decrypt(struct blkcipher_desc *desc, struct scatterlist *dst,
+ struct scatterlist *src, unsigned int nbytes)
+{
+ struct crypto_aes_ctx *ctx = crypto_blkcipher_ctx(desc->tfm);
+ int err, first, rounds = 6 + ctx->key_length / 4;
+ struct blkcipher_walk walk;
+ unsigned int blocks;
+
+ desc->flags &= ~CRYPTO_TFM_REQ_MAY_SLEEP;
+ blkcipher_walk_init(&walk, dst, src, nbytes);
+ err = blkcipher_walk_virt(desc, &walk);
+
+ kernel_neon_begin();
+ for (first = 1; (blocks = (walk.nbytes / AES_BLOCK_SIZE)); first = 0) {
+ aes_cbc_decrypt(walk.dst.virt.addr, walk.src.virt.addr,
+ (u8 *)ctx->key_dec, rounds, blocks, walk.iv,
+ first);
+ err = blkcipher_walk_done(desc, &walk, 0);
+ }
+ kernel_neon_end();
+ return err;
+}
+
+static int ctr_encrypt(struct blkcipher_desc *desc, struct scatterlist *dst,
+ struct scatterlist *src, unsigned int nbytes)
+{
+ struct crypto_aes_ctx *ctx = crypto_blkcipher_ctx(desc->tfm);
+ int err, first, rounds = 6 + ctx->key_length / 4;
+ struct blkcipher_walk walk;
+ int blocks;
+
+ desc->flags &= ~CRYPTO_TFM_REQ_MAY_SLEEP;
+ blkcipher_walk_init(&walk, dst, src, nbytes);
+ err = blkcipher_walk_virt_block(desc, &walk, AES_BLOCK_SIZE);
+
+ first = 1;
+ kernel_neon_begin();
+ while ((blocks = (walk.nbytes / AES_BLOCK_SIZE))) {
+ aes_ctr_encrypt(walk.dst.virt.addr, walk.src.virt.addr,
+ (u8 *)ctx->key_enc, rounds, blocks, walk.iv,
+ first);
+ first = 0;
+ nbytes -= blocks * AES_BLOCK_SIZE;
+ if (nbytes && nbytes == walk.nbytes % AES_BLOCK_SIZE)
+ break;
+ err = blkcipher_walk_done(desc, &walk,
+ walk.nbytes % AES_BLOCK_SIZE);
+ }
+ if (nbytes) {
+ u8 *tdst = walk.dst.virt.addr + blocks * AES_BLOCK_SIZE;
+ u8 *tsrc = walk.src.virt.addr + blocks * AES_BLOCK_SIZE;
+ u8 __aligned(8) tail[AES_BLOCK_SIZE];
+
+ /*
+ * Minimum alignment is 8 bytes, so if nbytes is <= 8, we need
+ * to tell aes_ctr_encrypt() to only read half a block.
+ */
+ blocks = (nbytes <= 8) ? -1 : 1;
+
+ aes_ctr_encrypt(tail, tsrc, (u8 *)ctx->key_enc, rounds,
+ blocks, walk.iv, first);
+ memcpy(tdst, tail, nbytes);
+ err = blkcipher_walk_done(desc, &walk, 0);
+ }
+ kernel_neon_end();
+
+ return err;
+}
+
+static int xts_encrypt(struct blkcipher_desc *desc, struct scatterlist *dst,
+ struct scatterlist *src, unsigned int nbytes)
+{
+ struct crypto_aes_xts_ctx *ctx = crypto_blkcipher_ctx(desc->tfm);
+ int err, first, rounds = 6 + ctx->key1.key_length / 4;
+ struct blkcipher_walk walk;
+ unsigned int blocks;
+
+ desc->flags &= ~CRYPTO_TFM_REQ_MAY_SLEEP;
+ blkcipher_walk_init(&walk, dst, src, nbytes);
+ err = blkcipher_walk_virt(desc, &walk);
+
+ kernel_neon_begin();
+ for (first = 1; (blocks = (walk.nbytes / AES_BLOCK_SIZE)); first = 0) {
+ aes_xts_encrypt(walk.dst.virt.addr, walk.src.virt.addr,
+ (u8 *)ctx->key1.key_enc, rounds, blocks,
+ (u8 *)ctx->key2.key_enc, walk.iv, first);
+ err = blkcipher_walk_done(desc, &walk, 0);
+ }
+ kernel_neon_end();
+
+ return err;
+}
+
+static int xts_decrypt(struct blkcipher_desc *desc, struct scatterlist *dst,
+ struct scatterlist *src, unsigned int nbytes)
+{
+ struct crypto_aes_xts_ctx *ctx = crypto_blkcipher_ctx(desc->tfm);
+ int err, first, rounds = 6 + ctx->key1.key_length / 4;
+ struct blkcipher_walk walk;
+ unsigned int blocks;
+
+ desc->flags &= ~CRYPTO_TFM_REQ_MAY_SLEEP;
+ blkcipher_walk_init(&walk, dst, src, nbytes);
+ err = blkcipher_walk_virt(desc, &walk);
+
+ kernel_neon_begin();
+ for (first = 1; (blocks = (walk.nbytes / AES_BLOCK_SIZE)); first = 0) {
+ aes_xts_decrypt(walk.dst.virt.addr, walk.src.virt.addr,
+ (u8 *)ctx->key1.key_dec, rounds, blocks,
+ (u8 *)ctx->key2.key_enc, walk.iv, first);
+ err = blkcipher_walk_done(desc, &walk, 0);
+ }
+ kernel_neon_end();
+
+ return err;
+}
+
+static struct crypto_alg aes_algs[] = { {
+ .cra_name = "__ecb-aes-" MODE,
+ .cra_driver_name = "__driver-ecb-aes-" MODE,
+ .cra_priority = 0,
+ .cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct crypto_aes_ctx),
+ .cra_alignmask = 7,
+ .cra_type = &crypto_blkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_blkcipher = {
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = crypto_aes_set_key,
+ .encrypt = ecb_encrypt,
+ .decrypt = ecb_decrypt,
+ },
+}, {
+ .cra_name = "__cbc-aes-" MODE,
+ .cra_driver_name = "__driver-cbc-aes-" MODE,
+ .cra_priority = 0,
+ .cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct crypto_aes_ctx),
+ .cra_alignmask = 7,
+ .cra_type = &crypto_blkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_blkcipher = {
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = crypto_aes_set_key,
+ .encrypt = cbc_encrypt,
+ .decrypt = cbc_decrypt,
+ },
+}, {
+ .cra_name = "__ctr-aes-" MODE,
+ .cra_driver_name = "__driver-ctr-aes-" MODE,
+ .cra_priority = 0,
+ .cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER,
+ .cra_blocksize = 1,
+ .cra_ctxsize = sizeof(struct crypto_aes_ctx),
+ .cra_alignmask = 7,
+ .cra_type = &crypto_blkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_blkcipher = {
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = crypto_aes_set_key,
+ .encrypt = ctr_encrypt,
+ .decrypt = ctr_encrypt,
+ },
+}, {
+ .cra_name = "__xts-aes-" MODE,
+ .cra_driver_name = "__driver-xts-aes-" MODE,
+ .cra_priority = 0,
+ .cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct crypto_aes_xts_ctx),
+ .cra_alignmask = 7,
+ .cra_type = &crypto_blkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_blkcipher = {
+ .min_keysize = 2 * AES_MIN_KEY_SIZE,
+ .max_keysize = 2 * AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = xts_set_key,
+ .encrypt = xts_encrypt,
+ .decrypt = xts_decrypt,
+ },
+}, {
+ .cra_name = "ecb(aes)",
+ .cra_driver_name = "ecb-aes-" MODE,
+ .cra_priority = PRIO,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER|CRYPTO_ALG_ASYNC,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct async_helper_ctx),
+ .cra_alignmask = 7,
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_init = ablk_init,
+ .cra_exit = ablk_exit,
+ .cra_ablkcipher = {
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = ablk_set_key,
+ .encrypt = ablk_encrypt,
+ .decrypt = ablk_decrypt,
+ }
+}, {
+ .cra_name = "cbc(aes)",
+ .cra_driver_name = "cbc-aes-" MODE,
+ .cra_priority = PRIO,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER|CRYPTO_ALG_ASYNC,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct async_helper_ctx),
+ .cra_alignmask = 7,
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_init = ablk_init,
+ .cra_exit = ablk_exit,
+ .cra_ablkcipher = {
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = ablk_set_key,
+ .encrypt = ablk_encrypt,
+ .decrypt = ablk_decrypt,
+ }
+}, {
+ .cra_name = "ctr(aes)",
+ .cra_driver_name = "ctr-aes-" MODE,
+ .cra_priority = PRIO,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER|CRYPTO_ALG_ASYNC,
+ .cra_blocksize = 1,
+ .cra_ctxsize = sizeof(struct async_helper_ctx),
+ .cra_alignmask = 7,
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_init = ablk_init,
+ .cra_exit = ablk_exit,
+ .cra_ablkcipher = {
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = ablk_set_key,
+ .encrypt = ablk_encrypt,
+ .decrypt = ablk_decrypt,
+ }
+}, {
+ .cra_name = "xts(aes)",
+ .cra_driver_name = "xts-aes-" MODE,
+ .cra_priority = PRIO,
+ .cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER|CRYPTO_ALG_ASYNC,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct async_helper_ctx),
+ .cra_alignmask = 7,
+ .cra_type = &crypto_ablkcipher_type,
+ .cra_module = THIS_MODULE,
+ .cra_init = ablk_init,
+ .cra_exit = ablk_exit,
+ .cra_ablkcipher = {
+ .min_keysize = 2 * AES_MIN_KEY_SIZE,
+ .max_keysize = 2 * AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = ablk_set_key,
+ .encrypt = ablk_encrypt,
+ .decrypt = ablk_decrypt,
+ }
+} };
+
+static int __init aes_init(void)
+{
+ return crypto_register_algs(aes_algs, ARRAY_SIZE(aes_algs));
+}
+
+static void __exit aes_exit(void)
+{
+ crypto_unregister_algs(aes_algs, ARRAY_SIZE(aes_algs));
+}
+
+#ifdef USE_V8_CRYPTO_EXTENSIONS
+module_cpu_feature_match(AES, aes_init);
+#else
+module_init(aes_init);
+#endif
+module_exit(aes_exit);
diff --git a/arch/arm64/crypto/aes-modes.S b/arch/arm64/crypto/aes-modes.S
new file mode 100644
index 0000000..f6e372c
--- /dev/null
+++ b/arch/arm64/crypto/aes-modes.S
@@ -0,0 +1,532 @@
+/*
+ * linux/arch/arm64/crypto/aes-modes.S - chaining mode wrappers for AES
+ *
+ * Copyright (C) 2013 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+/* included by aes-ce.S and aes-neon.S */
+
+ .text
+ .align 4
+
+/*
+ * There are several ways to instantiate this code:
+ * - no interleave, all inline
+ * - 2-way interleave, 2x calls out of line (-DINTERLEAVE=2)
+ * - 2-way interleave, all inline (-DINTERLEAVE=2 -DINTERLEAVE_INLINE)
+ * - 4-way interleave, 4x calls out of line (-DINTERLEAVE=4)
+ * - 4-way interleave, all inline (-DINTERLEAVE=4 -DINTERLEAVE_INLINE)
+ *
+ * Macros imported by this code:
+ * - enc_prepare - setup NEON registers for encryption
+ * - dec_prepare - setup NEON registers for decryption
+ * - enc_switch_key - change to new key after having prepared for encryption
+ * - encrypt_block - encrypt a single block
+ * - decrypt block - decrypt a single block
+ * - encrypt_block2x - encrypt 2 blocks in parallel (if INTERLEAVE == 2)
+ * - decrypt_block2x - decrypt 2 blocks in parallel (if INTERLEAVE == 2)
+ * - encrypt_block4x - encrypt 4 blocks in parallel (if INTERLEAVE == 4)
+ * - decrypt_block4x - decrypt 4 blocks in parallel (if INTERLEAVE == 4)
+ */
+
+#if defined(INTERLEAVE) && !defined(INTERLEAVE_INLINE)
+#define FRAME_PUSH stp x29, x30, [sp,#-16]! ; mov x29, sp
+#define FRAME_POP ldp x29, x30, [sp],#16
+
+#if INTERLEAVE == 2
+
+aes_encrypt_block2x:
+ encrypt_block2x v0, v1, w3, x2, x6, w7
+ ret
+ENDPROC(aes_encrypt_block2x)
+
+aes_decrypt_block2x:
+ decrypt_block2x v0, v1, w3, x2, x6, w7
+ ret
+ENDPROC(aes_decrypt_block2x)
+
+#elif INTERLEAVE == 4
+
+aes_encrypt_block4x:
+ encrypt_block4x v0, v1, v2, v3, w3, x2, x6, w7
+ ret
+ENDPROC(aes_encrypt_block4x)
+
+aes_decrypt_block4x:
+ decrypt_block4x v0, v1, v2, v3, w3, x2, x6, w7
+ ret
+ENDPROC(aes_decrypt_block4x)
+
+#else
+#error INTERLEAVE should equal 2 or 4
+#endif
+
+ .macro do_encrypt_block2x
+ bl aes_encrypt_block2x
+ .endm
+
+ .macro do_decrypt_block2x
+ bl aes_decrypt_block2x
+ .endm
+
+ .macro do_encrypt_block4x
+ bl aes_encrypt_block4x
+ .endm
+
+ .macro do_decrypt_block4x
+ bl aes_decrypt_block4x
+ .endm
+
+#else
+#define FRAME_PUSH
+#define FRAME_POP
+
+ .macro do_encrypt_block2x
+ encrypt_block2x v0, v1, w3, x2, x6, w7
+ .endm
+
+ .macro do_decrypt_block2x
+ decrypt_block2x v0, v1, w3, x2, x6, w7
+ .endm
+
+ .macro do_encrypt_block4x
+ encrypt_block4x v0, v1, v2, v3, w3, x2, x6, w7
+ .endm
+
+ .macro do_decrypt_block4x
+ decrypt_block4x v0, v1, v2, v3, w3, x2, x6, w7
+ .endm
+
+#endif
+
+ /*
+ * aes_ecb_encrypt(u8 out[], u8 const in[], u8 const rk[], int rounds,
+ * int blocks, int first)
+ * aes_ecb_decrypt(u8 out[], u8 const in[], u8 const rk[], int rounds,
+ * int blocks, int first)
+ */
+
+AES_ENTRY(aes_ecb_encrypt)
+ FRAME_PUSH
+ cbz w5, .LecbencloopNx
+
+ enc_prepare w3, x2, x5
+
+.LecbencloopNx:
+#if INTERLEAVE >= 2
+ subs w4, w4, #INTERLEAVE
+ bmi .Lecbenc1x
+#if INTERLEAVE == 2
+ ld1 {v0.16b-v1.16b}, [x1], #32 /* get 2 pt blocks */
+ do_encrypt_block2x
+ st1 {v0.16b-v1.16b}, [x0], #32
+#else
+ ld1 {v0.16b-v3.16b}, [x1], #64 /* get 4 pt blocks */
+ do_encrypt_block4x
+ st1 {v0.16b-v3.16b}, [x0], #64
+#endif
+ b .LecbencloopNx
+.Lecbenc1x:
+ adds w4, w4, #INTERLEAVE
+ beq .Lecbencout
+#endif
+.Lecbencloop:
+ ld1 {v0.16b}, [x1], #16 /* get next pt block */
+ encrypt_block v0, w3, x2, x5, w6
+ st1 {v0.16b}, [x0], #16
+ subs w4, w4, #1
+ bne .Lecbencloop
+.Lecbencout:
+ FRAME_POP
+ ret
+AES_ENDPROC(aes_ecb_encrypt)
+
+
+AES_ENTRY(aes_ecb_decrypt)
+ FRAME_PUSH
+ cbz w5, .LecbdecloopNx
+
+ dec_prepare w3, x2, x5
+
+.LecbdecloopNx:
+#if INTERLEAVE >= 2
+ subs w4, w4, #INTERLEAVE
+ bmi .Lecbdec1x
+#if INTERLEAVE == 2
+ ld1 {v0.16b-v1.16b}, [x1], #32 /* get 2 ct blocks */
+ do_decrypt_block2x
+ st1 {v0.16b-v1.16b}, [x0], #32
+#else
+ ld1 {v0.16b-v3.16b}, [x1], #64 /* get 4 ct blocks */
+ do_decrypt_block4x
+ st1 {v0.16b-v3.16b}, [x0], #64
+#endif
+ b .LecbdecloopNx
+.Lecbdec1x:
+ adds w4, w4, #INTERLEAVE
+ beq .Lecbdecout
+#endif
+.Lecbdecloop:
+ ld1 {v0.16b}, [x1], #16 /* get next ct block */
+ decrypt_block v0, w3, x2, x5, w6
+ st1 {v0.16b}, [x0], #16
+ subs w4, w4, #1
+ bne .Lecbdecloop
+.Lecbdecout:
+ FRAME_POP
+ ret
+AES_ENDPROC(aes_ecb_decrypt)
+
+
+ /*
+ * aes_cbc_encrypt(u8 out[], u8 const in[], u8 const rk[], int rounds,
+ * int blocks, u8 iv[], int first)
+ * aes_cbc_decrypt(u8 out[], u8 const in[], u8 const rk[], int rounds,
+ * int blocks, u8 iv[], int first)
+ */
+
+AES_ENTRY(aes_cbc_encrypt)
+ cbz w6, .Lcbcencloop
+
+ ld1 {v0.16b}, [x5] /* get iv */
+ enc_prepare w3, x2, x5
+
+.Lcbcencloop:
+ ld1 {v1.16b}, [x1], #16 /* get next pt block */
+ eor v0.16b, v0.16b, v1.16b /* ..and xor with iv */
+ encrypt_block v0, w3, x2, x5, w6
+ st1 {v0.16b}, [x0], #16
+ subs w4, w4, #1
+ bne .Lcbcencloop
+ ret
+AES_ENDPROC(aes_cbc_encrypt)
+
+
+AES_ENTRY(aes_cbc_decrypt)
+ FRAME_PUSH
+ cbz w6, .LcbcdecloopNx
+
+ ld1 {v7.16b}, [x5] /* get iv */
+ dec_prepare w3, x2, x5
+
+.LcbcdecloopNx:
+#if INTERLEAVE >= 2
+ subs w4, w4, #INTERLEAVE
+ bmi .Lcbcdec1x
+#if INTERLEAVE == 2
+ ld1 {v0.16b-v1.16b}, [x1], #32 /* get 2 ct blocks */
+ mov v2.16b, v0.16b
+ mov v3.16b, v1.16b
+ do_decrypt_block2x
+ eor v0.16b, v0.16b, v7.16b
+ eor v1.16b, v1.16b, v2.16b
+ mov v7.16b, v3.16b
+ st1 {v0.16b-v1.16b}, [x0], #32
+#else
+ ld1 {v0.16b-v3.16b}, [x1], #64 /* get 4 ct blocks */
+ mov v4.16b, v0.16b
+ mov v5.16b, v1.16b
+ mov v6.16b, v2.16b
+ do_decrypt_block4x
+ sub x1, x1, #16
+ eor v0.16b, v0.16b, v7.16b
+ eor v1.16b, v1.16b, v4.16b
+ ld1 {v7.16b}, [x1], #16 /* reload 1 ct block */
+ eor v2.16b, v2.16b, v5.16b
+ eor v3.16b, v3.16b, v6.16b
+ st1 {v0.16b-v3.16b}, [x0], #64
+#endif
+ b .LcbcdecloopNx
+.Lcbcdec1x:
+ adds w4, w4, #INTERLEAVE
+ beq .Lcbcdecout
+#endif
+.Lcbcdecloop:
+ ld1 {v1.16b}, [x1], #16 /* get next ct block */
+ mov v0.16b, v1.16b /* ...and copy to v0 */
+ decrypt_block v0, w3, x2, x5, w6
+ eor v0.16b, v0.16b, v7.16b /* xor with iv => pt */
+ mov v7.16b, v1.16b /* ct is next iv */
+ st1 {v0.16b}, [x0], #16
+ subs w4, w4, #1
+ bne .Lcbcdecloop
+.Lcbcdecout:
+ FRAME_POP
+ ret
+AES_ENDPROC(aes_cbc_decrypt)
+
+
+ /*
+ * aes_ctr_encrypt(u8 out[], u8 const in[], u8 const rk[], int rounds,
+ * int blocks, u8 ctr[], int first)
+ */
+
+AES_ENTRY(aes_ctr_encrypt)
+ FRAME_PUSH
+ cbnz w6, .Lctrfirst /* 1st time around? */
+ umov x5, v4.d[1] /* keep swabbed ctr in reg */
+ rev x5, x5
+#if INTERLEAVE >= 2
+ cmn w5, w4 /* 32 bit overflow? */
+ bcs .Lctrinc
+ add x5, x5, #1 /* increment BE ctr */
+ b .LctrincNx
+#else
+ b .Lctrinc
+#endif
+.Lctrfirst:
+ enc_prepare w3, x2, x6
+ ld1 {v4.16b}, [x5]
+ umov x5, v4.d[1] /* keep swabbed ctr in reg */
+ rev x5, x5
+#if INTERLEAVE >= 2
+ cmn w5, w4 /* 32 bit overflow? */
+ bcs .Lctrloop
+.LctrloopNx:
+ subs w4, w4, #INTERLEAVE
+ bmi .Lctr1x
+#if INTERLEAVE == 2
+ mov v0.8b, v4.8b
+ mov v1.8b, v4.8b
+ rev x7, x5
+ add x5, x5, #1
+ ins v0.d[1], x7
+ rev x7, x5
+ add x5, x5, #1
+ ins v1.d[1], x7
+ ld1 {v2.16b-v3.16b}, [x1], #32 /* get 2 input blocks */
+ do_encrypt_block2x
+ eor v0.16b, v0.16b, v2.16b
+ eor v1.16b, v1.16b, v3.16b
+ st1 {v0.16b-v1.16b}, [x0], #32
+#else
+ ldr q8, =0x30000000200000001 /* addends 1,2,3[,0] */
+ dup v7.4s, w5
+ mov v0.16b, v4.16b
+ add v7.4s, v7.4s, v8.4s
+ mov v1.16b, v4.16b
+ rev32 v8.16b, v7.16b
+ mov v2.16b, v4.16b
+ mov v3.16b, v4.16b
+ mov v1.s[3], v8.s[0]
+ mov v2.s[3], v8.s[1]
+ mov v3.s[3], v8.s[2]
+ ld1 {v5.16b-v7.16b}, [x1], #48 /* get 3 input blocks */
+ do_encrypt_block4x
+ eor v0.16b, v5.16b, v0.16b
+ ld1 {v5.16b}, [x1], #16 /* get 1 input block */
+ eor v1.16b, v6.16b, v1.16b
+ eor v2.16b, v7.16b, v2.16b
+ eor v3.16b, v5.16b, v3.16b
+ st1 {v0.16b-v3.16b}, [x0], #64
+ add x5, x5, #INTERLEAVE
+#endif
+ cbz w4, .LctroutNx
+.LctrincNx:
+ rev x7, x5
+ ins v4.d[1], x7
+ b .LctrloopNx
+.LctroutNx:
+ sub x5, x5, #1
+ rev x7, x5
+ ins v4.d[1], x7
+ b .Lctrout
+.Lctr1x:
+ adds w4, w4, #INTERLEAVE
+ beq .Lctrout
+#endif
+.Lctrloop:
+ mov v0.16b, v4.16b
+ encrypt_block v0, w3, x2, x6, w7
+ subs w4, w4, #1
+ bmi .Lctrhalfblock /* blocks < 0 means 1/2 block */
+ ld1 {v3.16b}, [x1], #16
+ eor v3.16b, v0.16b, v3.16b
+ st1 {v3.16b}, [x0], #16
+ beq .Lctrout
+.Lctrinc:
+ adds x5, x5, #1 /* increment BE ctr */
+ rev x7, x5
+ ins v4.d[1], x7
+ bcc .Lctrloop /* no overflow? */
+ umov x7, v4.d[0] /* load upper word of ctr */
+ rev x7, x7 /* ... to handle the carry */
+ add x7, x7, #1
+ rev x7, x7
+ ins v4.d[0], x7
+ b .Lctrloop
+.Lctrhalfblock:
+ ld1 {v3.8b}, [x1]
+ eor v3.8b, v0.8b, v3.8b
+ st1 {v3.8b}, [x0]
+.Lctrout:
+ FRAME_POP
+ ret
+AES_ENDPROC(aes_ctr_encrypt)
+ .ltorg
+
+
+ /*
+ * aes_xts_decrypt(u8 out[], u8 const in[], u8 const rk1[], int rounds,
+ * int blocks, u8 const rk2[], u8 iv[], int first)
+ * aes_xts_decrypt(u8 out[], u8 const in[], u8 const rk1[], int rounds,
+ * int blocks, u8 const rk2[], u8 iv[], int first)
+ */
+
+ .macro next_tweak, out, in, const, tmp
+ sshr \tmp\().2d, \in\().2d, #63
+ and \tmp\().16b, \tmp\().16b, \const\().16b
+ add \out\().2d, \in\().2d, \in\().2d
+ ext \tmp\().16b, \tmp\().16b, \tmp\().16b, #8
+ eor \out\().16b, \out\().16b, \tmp\().16b
+ .endm
+
+.Lxts_mul_x:
+ .word 1, 0, 0x87, 0
+
+AES_ENTRY(aes_xts_encrypt)
+ FRAME_PUSH
+ cbz w7, .LxtsencloopNx
+
+ ld1 {v4.16b}, [x6]
+ enc_prepare w3, x5, x6
+ encrypt_block v4, w3, x5, x6, w7 /* first tweak */
+ enc_switch_key w3, x2, x6
+ ldr q7, .Lxts_mul_x
+ b .LxtsencNx
+
+.LxtsencloopNx:
+ ldr q7, .Lxts_mul_x
+ next_tweak v4, v4, v7, v8
+.LxtsencNx:
+#if INTERLEAVE >= 2
+ subs w4, w4, #INTERLEAVE
+ bmi .Lxtsenc1x
+#if INTERLEAVE == 2
+ ld1 {v0.16b-v1.16b}, [x1], #32 /* get 2 pt blocks */
+ next_tweak v5, v4, v7, v8
+ eor v0.16b, v0.16b, v4.16b
+ eor v1.16b, v1.16b, v5.16b
+ do_encrypt_block2x
+ eor v0.16b, v0.16b, v4.16b
+ eor v1.16b, v1.16b, v5.16b
+ st1 {v0.16b-v1.16b}, [x0], #32
+ cbz w4, .LxtsencoutNx
+ next_tweak v4, v5, v7, v8
+ b .LxtsencNx
+.LxtsencoutNx:
+ mov v4.16b, v5.16b
+ b .Lxtsencout
+#else
+ ld1 {v0.16b-v3.16b}, [x1], #64 /* get 4 pt blocks */
+ next_tweak v5, v4, v7, v8
+ eor v0.16b, v0.16b, v4.16b
+ next_tweak v6, v5, v7, v8
+ eor v1.16b, v1.16b, v5.16b
+ eor v2.16b, v2.16b, v6.16b
+ next_tweak v7, v6, v7, v8
+ eor v3.16b, v3.16b, v7.16b
+ do_encrypt_block4x
+ eor v3.16b, v3.16b, v7.16b
+ eor v0.16b, v0.16b, v4.16b
+ eor v1.16b, v1.16b, v5.16b
+ eor v2.16b, v2.16b, v6.16b
+ st1 {v0.16b-v3.16b}, [x0], #64
+ mov v4.16b, v7.16b
+ cbz w4, .Lxtsencout
+ b .LxtsencloopNx
+#endif
+.Lxtsenc1x:
+ adds w4, w4, #INTERLEAVE
+ beq .Lxtsencout
+#endif
+.Lxtsencloop:
+ ld1 {v1.16b}, [x1], #16
+ eor v0.16b, v1.16b, v4.16b
+ encrypt_block v0, w3, x2, x6, w7
+ eor v0.16b, v0.16b, v4.16b
+ st1 {v0.16b}, [x0], #16
+ subs w4, w4, #1
+ beq .Lxtsencout
+ next_tweak v4, v4, v7, v8
+ b .Lxtsencloop
+.Lxtsencout:
+ FRAME_POP
+ ret
+AES_ENDPROC(aes_xts_encrypt)
+
+
+AES_ENTRY(aes_xts_decrypt)
+ FRAME_PUSH
+ cbz w7, .LxtsdecloopNx
+
+ ld1 {v4.16b}, [x6]
+ enc_prepare w3, x5, x6
+ encrypt_block v4, w3, x5, x6, w7 /* first tweak */
+ dec_prepare w3, x2, x6
+ ldr q7, .Lxts_mul_x
+ b .LxtsdecNx
+
+.LxtsdecloopNx:
+ ldr q7, .Lxts_mul_x
+ next_tweak v4, v4, v7, v8
+.LxtsdecNx:
+#if INTERLEAVE >= 2
+ subs w4, w4, #INTERLEAVE
+ bmi .Lxtsdec1x
+#if INTERLEAVE == 2
+ ld1 {v0.16b-v1.16b}, [x1], #32 /* get 2 ct blocks */
+ next_tweak v5, v4, v7, v8
+ eor v0.16b, v0.16b, v4.16b
+ eor v1.16b, v1.16b, v5.16b
+ do_decrypt_block2x
+ eor v0.16b, v0.16b, v4.16b
+ eor v1.16b, v1.16b, v5.16b
+ st1 {v0.16b-v1.16b}, [x0], #32
+ cbz w4, .LxtsdecoutNx
+ next_tweak v4, v5, v7, v8
+ b .LxtsdecNx
+.LxtsdecoutNx:
+ mov v4.16b, v5.16b
+ b .Lxtsdecout
+#else
+ ld1 {v0.16b-v3.16b}, [x1], #64 /* get 4 ct blocks */
+ next_tweak v5, v4, v7, v8
+ eor v0.16b, v0.16b, v4.16b
+ next_tweak v6, v5, v7, v8
+ eor v1.16b, v1.16b, v5.16b
+ eor v2.16b, v2.16b, v6.16b
+ next_tweak v7, v6, v7, v8
+ eor v3.16b, v3.16b, v7.16b
+ do_decrypt_block4x
+ eor v3.16b, v3.16b, v7.16b
+ eor v0.16b, v0.16b, v4.16b
+ eor v1.16b, v1.16b, v5.16b
+ eor v2.16b, v2.16b, v6.16b
+ st1 {v0.16b-v3.16b}, [x0], #64
+ mov v4.16b, v7.16b
+ cbz w4, .Lxtsdecout
+ b .LxtsdecloopNx
+#endif
+.Lxtsdec1x:
+ adds w4, w4, #INTERLEAVE
+ beq .Lxtsdecout
+#endif
+.Lxtsdecloop:
+ ld1 {v1.16b}, [x1], #16
+ eor v0.16b, v1.16b, v4.16b
+ decrypt_block v0, w3, x2, x6, w7
+ eor v0.16b, v0.16b, v4.16b
+ st1 {v0.16b}, [x0], #16
+ subs w4, w4, #1
+ beq .Lxtsdecout
+ next_tweak v4, v4, v7, v8
+ b .Lxtsdecloop
+.Lxtsdecout:
+ FRAME_POP
+ ret
+AES_ENDPROC(aes_xts_decrypt)
diff --git a/arch/arm64/crypto/aes-neon.S b/arch/arm64/crypto/aes-neon.S
new file mode 100644
index 0000000..b93170e
--- /dev/null
+++ b/arch/arm64/crypto/aes-neon.S
@@ -0,0 +1,382 @@
+/*
+ * linux/arch/arm64/crypto/aes-neon.S - AES cipher for ARMv8 NEON
+ *
+ * Copyright (C) 2013 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+
+#define AES_ENTRY(func) ENTRY(neon_ ## func)
+#define AES_ENDPROC(func) ENDPROC(neon_ ## func)
+
+ /* multiply by polynomial 'x' in GF(2^8) */
+ .macro mul_by_x, out, in, temp, const
+ sshr \temp, \in, #7
+ add \out, \in, \in
+ and \temp, \temp, \const
+ eor \out, \out, \temp
+ .endm
+
+ /* preload the entire Sbox */
+ .macro prepare, sbox, shiftrows, temp
+ adr \temp, \sbox
+ movi v12.16b, #0x40
+ ldr q13, \shiftrows
+ movi v14.16b, #0x1b
+ ld1 {v16.16b-v19.16b}, [\temp], #64
+ ld1 {v20.16b-v23.16b}, [\temp], #64
+ ld1 {v24.16b-v27.16b}, [\temp], #64
+ ld1 {v28.16b-v31.16b}, [\temp]
+ .endm
+
+ /* do preload for encryption */
+ .macro enc_prepare, ignore0, ignore1, temp
+ prepare .LForward_Sbox, .LForward_ShiftRows, \temp
+ .endm
+
+ .macro enc_switch_key, ignore0, ignore1, temp
+ /* do nothing */
+ .endm
+
+ /* do preload for decryption */
+ .macro dec_prepare, ignore0, ignore1, temp
+ prepare .LReverse_Sbox, .LReverse_ShiftRows, \temp
+ .endm
+
+ /* apply SubBytes transformation using the the preloaded Sbox */
+ .macro sub_bytes, in
+ sub v9.16b, \in\().16b, v12.16b
+ tbl \in\().16b, {v16.16b-v19.16b}, \in\().16b
+ sub v10.16b, v9.16b, v12.16b
+ tbx \in\().16b, {v20.16b-v23.16b}, v9.16b
+ sub v11.16b, v10.16b, v12.16b
+ tbx \in\().16b, {v24.16b-v27.16b}, v10.16b
+ tbx \in\().16b, {v28.16b-v31.16b}, v11.16b
+ .endm
+
+ /* apply MixColumns transformation */
+ .macro mix_columns, in
+ mul_by_x v10.16b, \in\().16b, v9.16b, v14.16b
+ rev32 v8.8h, \in\().8h
+ eor \in\().16b, v10.16b, \in\().16b
+ shl v9.4s, v8.4s, #24
+ shl v11.4s, \in\().4s, #24
+ sri v9.4s, v8.4s, #8
+ sri v11.4s, \in\().4s, #8
+ eor v9.16b, v9.16b, v8.16b
+ eor v10.16b, v10.16b, v9.16b
+ eor \in\().16b, v10.16b, v11.16b
+ .endm
+
+ /* Inverse MixColumns: pre-multiply by { 5, 0, 4, 0 } */
+ .macro inv_mix_columns, in
+ mul_by_x v11.16b, \in\().16b, v10.16b, v14.16b
+ mul_by_x v11.16b, v11.16b, v10.16b, v14.16b
+ eor \in\().16b, \in\().16b, v11.16b
+ rev32 v11.8h, v11.8h
+ eor \in\().16b, \in\().16b, v11.16b
+ mix_columns \in
+ .endm
+
+ .macro do_block, enc, in, rounds, rk, rkp, i
+ ld1 {v15.16b}, [\rk]
+ add \rkp, \rk, #16
+ mov \i, \rounds
+1111: eor \in\().16b, \in\().16b, v15.16b /* ^round key */
+ tbl \in\().16b, {\in\().16b}, v13.16b /* ShiftRows */
+ sub_bytes \in
+ ld1 {v15.16b}, [\rkp], #16
+ subs \i, \i, #1
+ beq 2222f
+ .if \enc == 1
+ mix_columns \in
+ .else
+ inv_mix_columns \in
+ .endif
+ b 1111b
+2222: eor \in\().16b, \in\().16b, v15.16b /* ^round key */
+ .endm
+
+ .macro encrypt_block, in, rounds, rk, rkp, i
+ do_block 1, \in, \rounds, \rk, \rkp, \i
+ .endm
+
+ .macro decrypt_block, in, rounds, rk, rkp, i
+ do_block 0, \in, \rounds, \rk, \rkp, \i
+ .endm
+
+ /*
+ * Interleaved versions: functionally equivalent to the
+ * ones above, but applied to 2 or 4 AES states in parallel.
+ */
+
+ .macro sub_bytes_2x, in0, in1
+ sub v8.16b, \in0\().16b, v12.16b
+ sub v9.16b, \in1\().16b, v12.16b
+ tbl \in0\().16b, {v16.16b-v19.16b}, \in0\().16b
+ tbl \in1\().16b, {v16.16b-v19.16b}, \in1\().16b
+ sub v10.16b, v8.16b, v12.16b
+ sub v11.16b, v9.16b, v12.16b
+ tbx \in0\().16b, {v20.16b-v23.16b}, v8.16b
+ tbx \in1\().16b, {v20.16b-v23.16b}, v9.16b
+ sub v8.16b, v10.16b, v12.16b
+ sub v9.16b, v11.16b, v12.16b
+ tbx \in0\().16b, {v24.16b-v27.16b}, v10.16b
+ tbx \in1\().16b, {v24.16b-v27.16b}, v11.16b
+ tbx \in0\().16b, {v28.16b-v31.16b}, v8.16b
+ tbx \in1\().16b, {v28.16b-v31.16b}, v9.16b
+ .endm
+
+ .macro sub_bytes_4x, in0, in1, in2, in3
+ sub v8.16b, \in0\().16b, v12.16b
+ tbl \in0\().16b, {v16.16b-v19.16b}, \in0\().16b
+ sub v9.16b, \in1\().16b, v12.16b
+ tbl \in1\().16b, {v16.16b-v19.16b}, \in1\().16b
+ sub v10.16b, \in2\().16b, v12.16b
+ tbl \in2\().16b, {v16.16b-v19.16b}, \in2\().16b
+ sub v11.16b, \in3\().16b, v12.16b
+ tbl \in3\().16b, {v16.16b-v19.16b}, \in3\().16b
+ tbx \in0\().16b, {v20.16b-v23.16b}, v8.16b
+ tbx \in1\().16b, {v20.16b-v23.16b}, v9.16b
+ sub v8.16b, v8.16b, v12.16b
+ tbx \in2\().16b, {v20.16b-v23.16b}, v10.16b
+ sub v9.16b, v9.16b, v12.16b
+ tbx \in3\().16b, {v20.16b-v23.16b}, v11.16b
+ sub v10.16b, v10.16b, v12.16b
+ tbx \in0\().16b, {v24.16b-v27.16b}, v8.16b
+ sub v11.16b, v11.16b, v12.16b
+ tbx \in1\().16b, {v24.16b-v27.16b}, v9.16b
+ sub v8.16b, v8.16b, v12.16b
+ tbx \in2\().16b, {v24.16b-v27.16b}, v10.16b
+ sub v9.16b, v9.16b, v12.16b
+ tbx \in3\().16b, {v24.16b-v27.16b}, v11.16b
+ sub v10.16b, v10.16b, v12.16b
+ tbx \in0\().16b, {v28.16b-v31.16b}, v8.16b
+ sub v11.16b, v11.16b, v12.16b
+ tbx \in1\().16b, {v28.16b-v31.16b}, v9.16b
+ tbx \in2\().16b, {v28.16b-v31.16b}, v10.16b
+ tbx \in3\().16b, {v28.16b-v31.16b}, v11.16b
+ .endm
+
+ .macro mul_by_x_2x, out0, out1, in0, in1, tmp0, tmp1, const
+ sshr \tmp0\().16b, \in0\().16b, #7
+ add \out0\().16b, \in0\().16b, \in0\().16b
+ sshr \tmp1\().16b, \in1\().16b, #7
+ and \tmp0\().16b, \tmp0\().16b, \const\().16b
+ add \out1\().16b, \in1\().16b, \in1\().16b
+ and \tmp1\().16b, \tmp1\().16b, \const\().16b
+ eor \out0\().16b, \out0\().16b, \tmp0\().16b
+ eor \out1\().16b, \out1\().16b, \tmp1\().16b
+ .endm
+
+ .macro mix_columns_2x, in0, in1
+ mul_by_x_2x v8, v9, \in0, \in1, v10, v11, v14
+ rev32 v10.8h, \in0\().8h
+ rev32 v11.8h, \in1\().8h
+ eor \in0\().16b, v8.16b, \in0\().16b
+ eor \in1\().16b, v9.16b, \in1\().16b
+ shl v12.4s, v10.4s, #24
+ shl v13.4s, v11.4s, #24
+ eor v8.16b, v8.16b, v10.16b
+ sri v12.4s, v10.4s, #8
+ shl v10.4s, \in0\().4s, #24
+ eor v9.16b, v9.16b, v11.16b
+ sri v13.4s, v11.4s, #8
+ shl v11.4s, \in1\().4s, #24
+ sri v10.4s, \in0\().4s, #8
+ eor \in0\().16b, v8.16b, v12.16b
+ sri v11.4s, \in1\().4s, #8
+ eor \in1\().16b, v9.16b, v13.16b
+ eor \in0\().16b, v10.16b, \in0\().16b
+ eor \in1\().16b, v11.16b, \in1\().16b
+ .endm
+
+ .macro inv_mix_cols_2x, in0, in1
+ mul_by_x_2x v8, v9, \in0, \in1, v10, v11, v14
+ mul_by_x_2x v8, v9, v8, v9, v10, v11, v14
+ eor \in0\().16b, \in0\().16b, v8.16b
+ eor \in1\().16b, \in1\().16b, v9.16b
+ rev32 v8.8h, v8.8h
+ rev32 v9.8h, v9.8h
+ eor \in0\().16b, \in0\().16b, v8.16b
+ eor \in1\().16b, \in1\().16b, v9.16b
+ mix_columns_2x \in0, \in1
+ .endm
+
+ .macro inv_mix_cols_4x, in0, in1, in2, in3
+ mul_by_x_2x v8, v9, \in0, \in1, v10, v11, v14
+ mul_by_x_2x v10, v11, \in2, \in3, v12, v13, v14
+ mul_by_x_2x v8, v9, v8, v9, v12, v13, v14
+ mul_by_x_2x v10, v11, v10, v11, v12, v13, v14
+ eor \in0\().16b, \in0\().16b, v8.16b
+ eor \in1\().16b, \in1\().16b, v9.16b
+ eor \in2\().16b, \in2\().16b, v10.16b
+ eor \in3\().16b, \in3\().16b, v11.16b
+ rev32 v8.8h, v8.8h
+ rev32 v9.8h, v9.8h
+ rev32 v10.8h, v10.8h
+ rev32 v11.8h, v11.8h
+ eor \in0\().16b, \in0\().16b, v8.16b
+ eor \in1\().16b, \in1\().16b, v9.16b
+ eor \in2\().16b, \in2\().16b, v10.16b
+ eor \in3\().16b, \in3\().16b, v11.16b
+ mix_columns_2x \in0, \in1
+ mix_columns_2x \in2, \in3
+ .endm
+
+ .macro do_block_2x, enc, in0, in1 rounds, rk, rkp, i
+ ld1 {v15.16b}, [\rk]
+ add \rkp, \rk, #16
+ mov \i, \rounds
+1111: eor \in0\().16b, \in0\().16b, v15.16b /* ^round key */
+ eor \in1\().16b, \in1\().16b, v15.16b /* ^round key */
+ sub_bytes_2x \in0, \in1
+ tbl \in0\().16b, {\in0\().16b}, v13.16b /* ShiftRows */
+ tbl \in1\().16b, {\in1\().16b}, v13.16b /* ShiftRows */
+ ld1 {v15.16b}, [\rkp], #16
+ subs \i, \i, #1
+ beq 2222f
+ .if \enc == 1
+ mix_columns_2x \in0, \in1
+ ldr q13, .LForward_ShiftRows
+ .else
+ inv_mix_cols_2x \in0, \in1
+ ldr q13, .LReverse_ShiftRows
+ .endif
+ movi v12.16b, #0x40
+ b 1111b
+2222: eor \in0\().16b, \in0\().16b, v15.16b /* ^round key */
+ eor \in1\().16b, \in1\().16b, v15.16b /* ^round key */
+ .endm
+
+ .macro do_block_4x, enc, in0, in1, in2, in3, rounds, rk, rkp, i
+ ld1 {v15.16b}, [\rk]
+ add \rkp, \rk, #16
+ mov \i, \rounds
+1111: eor \in0\().16b, \in0\().16b, v15.16b /* ^round key */
+ eor \in1\().16b, \in1\().16b, v15.16b /* ^round key */
+ eor \in2\().16b, \in2\().16b, v15.16b /* ^round key */
+ eor \in3\().16b, \in3\().16b, v15.16b /* ^round key */
+ sub_bytes_4x \in0, \in1, \in2, \in3
+ tbl \in0\().16b, {\in0\().16b}, v13.16b /* ShiftRows */
+ tbl \in1\().16b, {\in1\().16b}, v13.16b /* ShiftRows */
+ tbl \in2\().16b, {\in2\().16b}, v13.16b /* ShiftRows */
+ tbl \in3\().16b, {\in3\().16b}, v13.16b /* ShiftRows */
+ ld1 {v15.16b}, [\rkp], #16
+ subs \i, \i, #1
+ beq 2222f
+ .if \enc == 1
+ mix_columns_2x \in0, \in1
+ mix_columns_2x \in2, \in3
+ ldr q13, .LForward_ShiftRows
+ .else
+ inv_mix_cols_4x \in0, \in1, \in2, \in3
+ ldr q13, .LReverse_ShiftRows
+ .endif
+ movi v12.16b, #0x40
+ b 1111b
+2222: eor \in0\().16b, \in0\().16b, v15.16b /* ^round key */
+ eor \in1\().16b, \in1\().16b, v15.16b /* ^round key */
+ eor \in2\().16b, \in2\().16b, v15.16b /* ^round key */
+ eor \in3\().16b, \in3\().16b, v15.16b /* ^round key */
+ .endm
+
+ .macro encrypt_block2x, in0, in1, rounds, rk, rkp, i
+ do_block_2x 1, \in0, \in1, \rounds, \rk, \rkp, \i
+ .endm
+
+ .macro decrypt_block2x, in0, in1, rounds, rk, rkp, i
+ do_block_2x 0, \in0, \in1, \rounds, \rk, \rkp, \i
+ .endm
+
+ .macro encrypt_block4x, in0, in1, in2, in3, rounds, rk, rkp, i
+ do_block_4x 1, \in0, \in1, \in2, \in3, \rounds, \rk, \rkp, \i
+ .endm
+
+ .macro decrypt_block4x, in0, in1, in2, in3, rounds, rk, rkp, i
+ do_block_4x 0, \in0, \in1, \in2, \in3, \rounds, \rk, \rkp, \i
+ .endm
+
+#include "aes-modes.S"
+
+ .text
+ .align 4
+.LForward_ShiftRows:
+ .byte 0x0, 0x5, 0xa, 0xf, 0x4, 0x9, 0xe, 0x3
+ .byte 0x8, 0xd, 0x2, 0x7, 0xc, 0x1, 0x6, 0xb
+
+.LReverse_ShiftRows:
+ .byte 0x0, 0xd, 0xa, 0x7, 0x4, 0x1, 0xe, 0xb
+ .byte 0x8, 0x5, 0x2, 0xf, 0xc, 0x9, 0x6, 0x3
+
+.LForward_Sbox:
+ .byte 0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5
+ .byte 0x30, 0x01, 0x67, 0x2b, 0xfe, 0xd7, 0xab, 0x76
+ .byte 0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0
+ .byte 0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0
+ .byte 0xb7, 0xfd, 0x93, 0x26, 0x36, 0x3f, 0xf7, 0xcc
+ .byte 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15
+ .byte 0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a
+ .byte 0x07, 0x12, 0x80, 0xe2, 0xeb, 0x27, 0xb2, 0x75
+ .byte 0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0
+ .byte 0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84
+ .byte 0x53, 0xd1, 0x00, 0xed, 0x20, 0xfc, 0xb1, 0x5b
+ .byte 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf
+ .byte 0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85
+ .byte 0x45, 0xf9, 0x02, 0x7f, 0x50, 0x3c, 0x9f, 0xa8
+ .byte 0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5
+ .byte 0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2
+ .byte 0xcd, 0x0c, 0x13, 0xec, 0x5f, 0x97, 0x44, 0x17
+ .byte 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73
+ .byte 0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88
+ .byte 0x46, 0xee, 0xb8, 0x14, 0xde, 0x5e, 0x0b, 0xdb
+ .byte 0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c
+ .byte 0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79
+ .byte 0xe7, 0xc8, 0x37, 0x6d, 0x8d, 0xd5, 0x4e, 0xa9
+ .byte 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08
+ .byte 0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6
+ .byte 0xe8, 0xdd, 0x74, 0x1f, 0x4b, 0xbd, 0x8b, 0x8a
+ .byte 0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e
+ .byte 0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e
+ .byte 0xe1, 0xf8, 0x98, 0x11, 0x69, 0xd9, 0x8e, 0x94
+ .byte 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf
+ .byte 0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68
+ .byte 0x41, 0x99, 0x2d, 0x0f, 0xb0, 0x54, 0xbb, 0x16
+
+.LReverse_Sbox:
+ .byte 0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38
+ .byte 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb
+ .byte 0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87
+ .byte 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb
+ .byte 0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d
+ .byte 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e
+ .byte 0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2
+ .byte 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25
+ .byte 0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16
+ .byte 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92
+ .byte 0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda
+ .byte 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84
+ .byte 0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a
+ .byte 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06
+ .byte 0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02
+ .byte 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b
+ .byte 0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea
+ .byte 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73
+ .byte 0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85
+ .byte 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e
+ .byte 0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89
+ .byte 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b
+ .byte 0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20
+ .byte 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4
+ .byte 0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31
+ .byte 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f
+ .byte 0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d
+ .byte 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef
+ .byte 0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0
+ .byte 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61
+ .byte 0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26
+ .byte 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d
diff --git a/arch/arm64/crypto/ghash-ce-core.S b/arch/arm64/crypto/ghash-ce-core.S
new file mode 100644
index 0000000..b9e6eaf41
--- /dev/null
+++ b/arch/arm64/crypto/ghash-ce-core.S
@@ -0,0 +1,95 @@
+/*
+ * Accelerated GHASH implementation with ARMv8 PMULL instructions.
+ *
+ * Copyright (C) 2014 Linaro Ltd. <ard.biesheuvel@linaro.org>
+ *
+ * Based on arch/x86/crypto/ghash-pmullni-intel_asm.S
+ *
+ * Copyright (c) 2009 Intel Corp.
+ * Author: Huang Ying <ying.huang@intel.com>
+ * Vinodh Gopal
+ * Erdinc Ozturk
+ * Deniz Karakoyunlu
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+ DATA .req v0
+ SHASH .req v1
+ IN1 .req v2
+ T1 .req v2
+ T2 .req v3
+ T3 .req v4
+ VZR .req v5
+
+ .text
+ .arch armv8-a+crypto
+
+ /*
+ * void pmull_ghash_update(int blocks, u64 dg[], const char *src,
+ * struct ghash_key const *k, const char *head)
+ */
+ENTRY(pmull_ghash_update)
+ ld1 {DATA.16b}, [x1]
+ ld1 {SHASH.16b}, [x3]
+ eor VZR.16b, VZR.16b, VZR.16b
+
+ /* do the head block first, if supplied */
+ cbz x4, 0f
+ ld1 {IN1.2d}, [x4]
+ b 1f
+
+0: ld1 {IN1.2d}, [x2], #16
+ sub w0, w0, #1
+1: ext IN1.16b, IN1.16b, IN1.16b, #8
+CPU_LE( rev64 IN1.16b, IN1.16b )
+ eor DATA.16b, DATA.16b, IN1.16b
+
+ /* multiply DATA by SHASH in GF(2^128) */
+ ext T2.16b, DATA.16b, DATA.16b, #8
+ ext T3.16b, SHASH.16b, SHASH.16b, #8
+ eor T2.16b, T2.16b, DATA.16b
+ eor T3.16b, T3.16b, SHASH.16b
+
+ pmull2 T1.1q, SHASH.2d, DATA.2d // a1 * b1
+ pmull DATA.1q, SHASH.1d, DATA.1d // a0 * b0
+ pmull T2.1q, T2.1d, T3.1d // (a1 + a0)(b1 + b0)
+ eor T2.16b, T2.16b, T1.16b // (a0 * b1) + (a1 * b0)
+ eor T2.16b, T2.16b, DATA.16b
+
+ ext T3.16b, VZR.16b, T2.16b, #8
+ ext T2.16b, T2.16b, VZR.16b, #8
+ eor DATA.16b, DATA.16b, T3.16b
+ eor T1.16b, T1.16b, T2.16b // <T1:DATA> is result of
+ // carry-less multiplication
+
+ /* first phase of the reduction */
+ shl T3.2d, DATA.2d, #1
+ eor T3.16b, T3.16b, DATA.16b
+ shl T3.2d, T3.2d, #5
+ eor T3.16b, T3.16b, DATA.16b
+ shl T3.2d, T3.2d, #57
+ ext T2.16b, VZR.16b, T3.16b, #8
+ ext T3.16b, T3.16b, VZR.16b, #8
+ eor DATA.16b, DATA.16b, T2.16b
+ eor T1.16b, T1.16b, T3.16b
+
+ /* second phase of the reduction */
+ ushr T2.2d, DATA.2d, #5
+ eor T2.16b, T2.16b, DATA.16b
+ ushr T2.2d, T2.2d, #1
+ eor T2.16b, T2.16b, DATA.16b
+ ushr T2.2d, T2.2d, #1
+ eor T1.16b, T1.16b, T2.16b
+ eor DATA.16b, DATA.16b, T1.16b
+
+ cbnz w0, 0b
+
+ st1 {DATA.16b}, [x1]
+ ret
+ENDPROC(pmull_ghash_update)
diff --git a/arch/arm64/crypto/ghash-ce-glue.c b/arch/arm64/crypto/ghash-ce-glue.c
new file mode 100644
index 0000000..b92baf3
--- /dev/null
+++ b/arch/arm64/crypto/ghash-ce-glue.c
@@ -0,0 +1,155 @@
+/*
+ * Accelerated GHASH implementation with ARMv8 PMULL instructions.
+ *
+ * Copyright (C) 2014 Linaro Ltd. <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ */
+
+#include <asm/neon.h>
+#include <asm/unaligned.h>
+#include <crypto/internal/hash.h>
+#include <linux/cpufeature.h>
+#include <linux/crypto.h>
+#include <linux/module.h>
+
+MODULE_DESCRIPTION("GHASH secure hash using ARMv8 Crypto Extensions");
+MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>");
+MODULE_LICENSE("GPL v2");
+
+#define GHASH_BLOCK_SIZE 16
+#define GHASH_DIGEST_SIZE 16
+
+struct ghash_key {
+ u64 a;
+ u64 b;
+};
+
+struct ghash_desc_ctx {
+ u64 digest[GHASH_DIGEST_SIZE/sizeof(u64)];
+ u8 buf[GHASH_BLOCK_SIZE];
+ u32 count;
+};
+
+asmlinkage void pmull_ghash_update(int blocks, u64 dg[], const char *src,
+ struct ghash_key const *k, const char *head);
+
+static int ghash_init(struct shash_desc *desc)
+{
+ struct ghash_desc_ctx *ctx = shash_desc_ctx(desc);
+
+ *ctx = (struct ghash_desc_ctx){};
+ return 0;
+}
+
+static int ghash_update(struct shash_desc *desc, const u8 *src,
+ unsigned int len)
+{
+ struct ghash_desc_ctx *ctx = shash_desc_ctx(desc);
+ unsigned int partial = ctx->count % GHASH_BLOCK_SIZE;
+
+ ctx->count += len;
+
+ if ((partial + len) >= GHASH_BLOCK_SIZE) {
+ struct ghash_key *key = crypto_shash_ctx(desc->tfm);
+ int blocks;
+
+ if (partial) {
+ int p = GHASH_BLOCK_SIZE - partial;
+
+ memcpy(ctx->buf + partial, src, p);
+ src += p;
+ len -= p;
+ }
+
+ blocks = len / GHASH_BLOCK_SIZE;
+ len %= GHASH_BLOCK_SIZE;
+
+ kernel_neon_begin_partial(6);
+ pmull_ghash_update(blocks, ctx->digest, src, key,
+ partial ? ctx->buf : NULL);
+ kernel_neon_end();
+ src += blocks * GHASH_BLOCK_SIZE;
+ }
+ if (len)
+ memcpy(ctx->buf + partial, src, len);
+ return 0;
+}
+
+static int ghash_final(struct shash_desc *desc, u8 *dst)
+{
+ struct ghash_desc_ctx *ctx = shash_desc_ctx(desc);
+ unsigned int partial = ctx->count % GHASH_BLOCK_SIZE;
+
+ if (partial) {
+ struct ghash_key *key = crypto_shash_ctx(desc->tfm);
+
+ memset(ctx->buf + partial, 0, GHASH_BLOCK_SIZE - partial);
+
+ kernel_neon_begin_partial(6);
+ pmull_ghash_update(1, ctx->digest, ctx->buf, key, NULL);
+ kernel_neon_end();
+ }
+ put_unaligned_be64(ctx->digest[1], dst);
+ put_unaligned_be64(ctx->digest[0], dst + 8);
+
+ *ctx = (struct ghash_desc_ctx){};
+ return 0;
+}
+
+static int ghash_setkey(struct crypto_shash *tfm,
+ const u8 *inkey, unsigned int keylen)
+{
+ struct ghash_key *key = crypto_shash_ctx(tfm);
+ u64 a, b;
+
+ if (keylen != GHASH_BLOCK_SIZE) {
+ crypto_shash_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
+ return -EINVAL;
+ }
+
+ /* perform multiplication by 'x' in GF(2^128) */
+ b = get_unaligned_be64(inkey);
+ a = get_unaligned_be64(inkey + 8);
+
+ key->a = (a << 1) | (b >> 63);
+ key->b = (b << 1) | (a >> 63);
+
+ if (b >> 63)
+ key->b ^= 0xc200000000000000UL;
+
+ return 0;
+}
+
+static struct shash_alg ghash_alg = {
+ .digestsize = GHASH_DIGEST_SIZE,
+ .init = ghash_init,
+ .update = ghash_update,
+ .final = ghash_final,
+ .setkey = ghash_setkey,
+ .descsize = sizeof(struct ghash_desc_ctx),
+ .base = {
+ .cra_name = "ghash",
+ .cra_driver_name = "ghash-ce",
+ .cra_priority = 200,
+ .cra_flags = CRYPTO_ALG_TYPE_SHASH,
+ .cra_blocksize = GHASH_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct ghash_key),
+ .cra_module = THIS_MODULE,
+ },
+};
+
+static int __init ghash_ce_mod_init(void)
+{
+ return crypto_register_shash(&ghash_alg);
+}
+
+static void __exit ghash_ce_mod_exit(void)
+{
+ crypto_unregister_shash(&ghash_alg);
+}
+
+module_cpu_feature_match(PMULL, ghash_ce_mod_init);
+module_exit(ghash_ce_mod_exit);
diff --git a/arch/arm64/crypto/sha1-ce-core.S b/arch/arm64/crypto/sha1-ce-core.S
new file mode 100644
index 0000000..09d57d9
--- /dev/null
+++ b/arch/arm64/crypto/sha1-ce-core.S
@@ -0,0 +1,153 @@
+/*
+ * sha1-ce-core.S - SHA-1 secure hash using ARMv8 Crypto Extensions
+ *
+ * Copyright (C) 2014 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+ .text
+ .arch armv8-a+crypto
+
+ k0 .req v0
+ k1 .req v1
+ k2 .req v2
+ k3 .req v3
+
+ t0 .req v4
+ t1 .req v5
+
+ dga .req q6
+ dgav .req v6
+ dgb .req s7
+ dgbv .req v7
+
+ dg0q .req q12
+ dg0s .req s12
+ dg0v .req v12
+ dg1s .req s13
+ dg1v .req v13
+ dg2s .req s14
+
+ .macro add_only, op, ev, rc, s0, dg1
+ .ifc \ev, ev
+ add t1.4s, v\s0\().4s, \rc\().4s
+ sha1h dg2s, dg0s
+ .ifnb \dg1
+ sha1\op dg0q, \dg1, t0.4s
+ .else
+ sha1\op dg0q, dg1s, t0.4s
+ .endif
+ .else
+ .ifnb \s0
+ add t0.4s, v\s0\().4s, \rc\().4s
+ .endif
+ sha1h dg1s, dg0s
+ sha1\op dg0q, dg2s, t1.4s
+ .endif
+ .endm
+
+ .macro add_update, op, ev, rc, s0, s1, s2, s3, dg1
+ sha1su0 v\s0\().4s, v\s1\().4s, v\s2\().4s
+ add_only \op, \ev, \rc, \s1, \dg1
+ sha1su1 v\s0\().4s, v\s3\().4s
+ .endm
+
+ /*
+ * The SHA1 round constants
+ */
+ .align 4
+.Lsha1_rcon:
+ .word 0x5a827999, 0x6ed9eba1, 0x8f1bbcdc, 0xca62c1d6
+
+ /*
+ * void sha1_ce_transform(int blocks, u8 const *src, u32 *state,
+ * u8 *head, long bytes)
+ */
+ENTRY(sha1_ce_transform)
+ /* load round constants */
+ adr x6, .Lsha1_rcon
+ ld1r {k0.4s}, [x6], #4
+ ld1r {k1.4s}, [x6], #4
+ ld1r {k2.4s}, [x6], #4
+ ld1r {k3.4s}, [x6]
+
+ /* load state */
+ ldr dga, [x2]
+ ldr dgb, [x2, #16]
+
+ /* load partial state (if supplied) */
+ cbz x3, 0f
+ ld1 {v8.4s-v11.4s}, [x3]
+ b 1f
+
+ /* load input */
+0: ld1 {v8.4s-v11.4s}, [x1], #64
+ sub w0, w0, #1
+
+1:
+CPU_LE( rev32 v8.16b, v8.16b )
+CPU_LE( rev32 v9.16b, v9.16b )
+CPU_LE( rev32 v10.16b, v10.16b )
+CPU_LE( rev32 v11.16b, v11.16b )
+
+2: add t0.4s, v8.4s, k0.4s
+ mov dg0v.16b, dgav.16b
+
+ add_update c, ev, k0, 8, 9, 10, 11, dgb
+ add_update c, od, k0, 9, 10, 11, 8
+ add_update c, ev, k0, 10, 11, 8, 9
+ add_update c, od, k0, 11, 8, 9, 10
+ add_update c, ev, k1, 8, 9, 10, 11
+
+ add_update p, od, k1, 9, 10, 11, 8
+ add_update p, ev, k1, 10, 11, 8, 9
+ add_update p, od, k1, 11, 8, 9, 10
+ add_update p, ev, k1, 8, 9, 10, 11
+ add_update p, od, k2, 9, 10, 11, 8
+
+ add_update m, ev, k2, 10, 11, 8, 9
+ add_update m, od, k2, 11, 8, 9, 10
+ add_update m, ev, k2, 8, 9, 10, 11
+ add_update m, od, k2, 9, 10, 11, 8
+ add_update m, ev, k3, 10, 11, 8, 9
+
+ add_update p, od, k3, 11, 8, 9, 10
+ add_only p, ev, k3, 9
+ add_only p, od, k3, 10
+ add_only p, ev, k3, 11
+ add_only p, od
+
+ /* update state */
+ add dgbv.2s, dgbv.2s, dg1v.2s
+ add dgav.4s, dgav.4s, dg0v.4s
+
+ cbnz w0, 0b
+
+ /*
+ * Final block: add padding and total bit count.
+ * Skip if we have no total byte count in x4. In that case, the input
+ * size was not a round multiple of the block size, and the padding is
+ * handled by the C code.
+ */
+ cbz x4, 3f
+ movi v9.2d, #0
+ mov x8, #0x80000000
+ movi v10.2d, #0
+ ror x7, x4, #29 // ror(lsl(x4, 3), 32)
+ fmov d8, x8
+ mov x4, #0
+ mov v11.d[0], xzr
+ mov v11.d[1], x7
+ b 2b
+
+ /* store new state */
+3: str dga, [x2]
+ str dgb, [x2, #16]
+ ret
+ENDPROC(sha1_ce_transform)
diff --git a/arch/arm64/crypto/sha1-ce-glue.c b/arch/arm64/crypto/sha1-ce-glue.c
new file mode 100644
index 0000000..6fe83f3
--- /dev/null
+++ b/arch/arm64/crypto/sha1-ce-glue.c
@@ -0,0 +1,174 @@
+/*
+ * sha1-ce-glue.c - SHA-1 secure hash using ARMv8 Crypto Extensions
+ *
+ * Copyright (C) 2014 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <asm/neon.h>
+#include <asm/unaligned.h>
+#include <crypto/internal/hash.h>
+#include <crypto/sha.h>
+#include <linux/cpufeature.h>
+#include <linux/crypto.h>
+#include <linux/module.h>
+
+MODULE_DESCRIPTION("SHA1 secure hash using ARMv8 Crypto Extensions");
+MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>");
+MODULE_LICENSE("GPL v2");
+
+asmlinkage void sha1_ce_transform(int blocks, u8 const *src, u32 *state,
+ u8 *head, long bytes);
+
+static int sha1_init(struct shash_desc *desc)
+{
+ struct sha1_state *sctx = shash_desc_ctx(desc);
+
+ *sctx = (struct sha1_state){
+ .state = { SHA1_H0, SHA1_H1, SHA1_H2, SHA1_H3, SHA1_H4 },
+ };
+ return 0;
+}
+
+static int sha1_update(struct shash_desc *desc, const u8 *data,
+ unsigned int len)
+{
+ struct sha1_state *sctx = shash_desc_ctx(desc);
+ unsigned int partial = sctx->count % SHA1_BLOCK_SIZE;
+
+ sctx->count += len;
+
+ if ((partial + len) >= SHA1_BLOCK_SIZE) {
+ int blocks;
+
+ if (partial) {
+ int p = SHA1_BLOCK_SIZE - partial;
+
+ memcpy(sctx->buffer + partial, data, p);
+ data += p;
+ len -= p;
+ }
+
+ blocks = len / SHA1_BLOCK_SIZE;
+ len %= SHA1_BLOCK_SIZE;
+
+ kernel_neon_begin_partial(16);
+ sha1_ce_transform(blocks, data, sctx->state,
+ partial ? sctx->buffer : NULL, 0);
+ kernel_neon_end();
+
+ data += blocks * SHA1_BLOCK_SIZE;
+ partial = 0;
+ }
+ if (len)
+ memcpy(sctx->buffer + partial, data, len);
+ return 0;
+}
+
+static int sha1_final(struct shash_desc *desc, u8 *out)
+{
+ static const u8 padding[SHA1_BLOCK_SIZE] = { 0x80, };
+
+ struct sha1_state *sctx = shash_desc_ctx(desc);
+ __be64 bits = cpu_to_be64(sctx->count << 3);
+ __be32 *dst = (__be32 *)out;
+ int i;
+
+ u32 padlen = SHA1_BLOCK_SIZE
+ - ((sctx->count + sizeof(bits)) % SHA1_BLOCK_SIZE);
+
+ sha1_update(desc, padding, padlen);
+ sha1_update(desc, (const u8 *)&bits, sizeof(bits));
+
+ for (i = 0; i < SHA1_DIGEST_SIZE / sizeof(__be32); i++)
+ put_unaligned_be32(sctx->state[i], dst++);
+
+ *sctx = (struct sha1_state){};
+ return 0;
+}
+
+static int sha1_finup(struct shash_desc *desc, const u8 *data,
+ unsigned int len, u8 *out)
+{
+ struct sha1_state *sctx = shash_desc_ctx(desc);
+ __be32 *dst = (__be32 *)out;
+ int blocks;
+ int i;
+
+ if (sctx->count || !len || (len % SHA1_BLOCK_SIZE)) {
+ sha1_update(desc, data, len);
+ return sha1_final(desc, out);
+ }
+
+ /*
+ * Use a fast path if the input is a multiple of 64 bytes. In
+ * this case, there is no need to copy data around, and we can
+ * perform the entire digest calculation in a single invocation
+ * of sha1_ce_transform()
+ */
+ blocks = len / SHA1_BLOCK_SIZE;
+
+ kernel_neon_begin_partial(16);
+ sha1_ce_transform(blocks, data, sctx->state, NULL, len);
+ kernel_neon_end();
+
+ for (i = 0; i < SHA1_DIGEST_SIZE / sizeof(__be32); i++)
+ put_unaligned_be32(sctx->state[i], dst++);
+
+ *sctx = (struct sha1_state){};
+ return 0;
+}
+
+static int sha1_export(struct shash_desc *desc, void *out)
+{
+ struct sha1_state *sctx = shash_desc_ctx(desc);
+ struct sha1_state *dst = out;
+
+ *dst = *sctx;
+ return 0;
+}
+
+static int sha1_import(struct shash_desc *desc, const void *in)
+{
+ struct sha1_state *sctx = shash_desc_ctx(desc);
+ struct sha1_state const *src = in;
+
+ *sctx = *src;
+ return 0;
+}
+
+static struct shash_alg alg = {
+ .init = sha1_init,
+ .update = sha1_update,
+ .final = sha1_final,
+ .finup = sha1_finup,
+ .export = sha1_export,
+ .import = sha1_import,
+ .descsize = sizeof(struct sha1_state),
+ .digestsize = SHA1_DIGEST_SIZE,
+ .statesize = sizeof(struct sha1_state),
+ .base = {
+ .cra_name = "sha1",
+ .cra_driver_name = "sha1-ce",
+ .cra_priority = 200,
+ .cra_flags = CRYPTO_ALG_TYPE_SHASH,
+ .cra_blocksize = SHA1_BLOCK_SIZE,
+ .cra_module = THIS_MODULE,
+ }
+};
+
+static int __init sha1_ce_mod_init(void)
+{
+ return crypto_register_shash(&alg);
+}
+
+static void __exit sha1_ce_mod_fini(void)
+{
+ crypto_unregister_shash(&alg);
+}
+
+module_cpu_feature_match(SHA1, sha1_ce_mod_init);
+module_exit(sha1_ce_mod_fini);
diff --git a/arch/arm64/crypto/sha2-ce-core.S b/arch/arm64/crypto/sha2-ce-core.S
new file mode 100644
index 0000000..7f29fc0
--- /dev/null
+++ b/arch/arm64/crypto/sha2-ce-core.S
@@ -0,0 +1,156 @@
+/*
+ * sha2-ce-core.S - core SHA-224/SHA-256 transform using v8 Crypto Extensions
+ *
+ * Copyright (C) 2014 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+ .text
+ .arch armv8-a+crypto
+
+ dga .req q20
+ dgav .req v20
+ dgb .req q21
+ dgbv .req v21
+
+ t0 .req v22
+ t1 .req v23
+
+ dg0q .req q24
+ dg0v .req v24
+ dg1q .req q25
+ dg1v .req v25
+ dg2q .req q26
+ dg2v .req v26
+
+ .macro add_only, ev, rc, s0
+ mov dg2v.16b, dg0v.16b
+ .ifeq \ev
+ add t1.4s, v\s0\().4s, \rc\().4s
+ sha256h dg0q, dg1q, t0.4s
+ sha256h2 dg1q, dg2q, t0.4s
+ .else
+ .ifnb \s0
+ add t0.4s, v\s0\().4s, \rc\().4s
+ .endif
+ sha256h dg0q, dg1q, t1.4s
+ sha256h2 dg1q, dg2q, t1.4s
+ .endif
+ .endm
+
+ .macro add_update, ev, rc, s0, s1, s2, s3
+ sha256su0 v\s0\().4s, v\s1\().4s
+ add_only \ev, \rc, \s1
+ sha256su1 v\s0\().4s, v\s2\().4s, v\s3\().4s
+ .endm
+
+ /*
+ * The SHA-256 round constants
+ */
+ .align 4
+.Lsha2_rcon:
+ .word 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5
+ .word 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5
+ .word 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3
+ .word 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174
+ .word 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc
+ .word 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da
+ .word 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7
+ .word 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967
+ .word 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13
+ .word 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85
+ .word 0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3
+ .word 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070
+ .word 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5
+ .word 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3
+ .word 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208
+ .word 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2
+
+ /*
+ * void sha2_ce_transform(int blocks, u8 const *src, u32 *state,
+ * u8 *head, long bytes)
+ */
+ENTRY(sha2_ce_transform)
+ /* load round constants */
+ adr x8, .Lsha2_rcon
+ ld1 { v0.4s- v3.4s}, [x8], #64
+ ld1 { v4.4s- v7.4s}, [x8], #64
+ ld1 { v8.4s-v11.4s}, [x8], #64
+ ld1 {v12.4s-v15.4s}, [x8]
+
+ /* load state */
+ ldp dga, dgb, [x2]
+
+ /* load partial input (if supplied) */
+ cbz x3, 0f
+ ld1 {v16.4s-v19.4s}, [x3]
+ b 1f
+
+ /* load input */
+0: ld1 {v16.4s-v19.4s}, [x1], #64
+ sub w0, w0, #1
+
+1:
+CPU_LE( rev32 v16.16b, v16.16b )
+CPU_LE( rev32 v17.16b, v17.16b )
+CPU_LE( rev32 v18.16b, v18.16b )
+CPU_LE( rev32 v19.16b, v19.16b )
+
+2: add t0.4s, v16.4s, v0.4s
+ mov dg0v.16b, dgav.16b
+ mov dg1v.16b, dgbv.16b
+
+ add_update 0, v1, 16, 17, 18, 19
+ add_update 1, v2, 17, 18, 19, 16
+ add_update 0, v3, 18, 19, 16, 17
+ add_update 1, v4, 19, 16, 17, 18
+
+ add_update 0, v5, 16, 17, 18, 19
+ add_update 1, v6, 17, 18, 19, 16
+ add_update 0, v7, 18, 19, 16, 17
+ add_update 1, v8, 19, 16, 17, 18
+
+ add_update 0, v9, 16, 17, 18, 19
+ add_update 1, v10, 17, 18, 19, 16
+ add_update 0, v11, 18, 19, 16, 17
+ add_update 1, v12, 19, 16, 17, 18
+
+ add_only 0, v13, 17
+ add_only 1, v14, 18
+ add_only 0, v15, 19
+ add_only 1
+
+ /* update state */
+ add dgav.4s, dgav.4s, dg0v.4s
+ add dgbv.4s, dgbv.4s, dg1v.4s
+
+ /* handled all input blocks? */
+ cbnz w0, 0b
+
+ /*
+ * Final block: add padding and total bit count.
+ * Skip if we have no total byte count in x4. In that case, the input
+ * size was not a round multiple of the block size, and the padding is
+ * handled by the C code.
+ */
+ cbz x4, 3f
+ movi v17.2d, #0
+ mov x8, #0x80000000
+ movi v18.2d, #0
+ ror x7, x4, #29 // ror(lsl(x4, 3), 32)
+ fmov d16, x8
+ mov x4, #0
+ mov v19.d[0], xzr
+ mov v19.d[1], x7
+ b 2b
+
+ /* store new state */
+3: stp dga, dgb, [x2]
+ ret
+ENDPROC(sha2_ce_transform)
diff --git a/arch/arm64/crypto/sha2-ce-glue.c b/arch/arm64/crypto/sha2-ce-glue.c
new file mode 100644
index 0000000..c294e67
--- /dev/null
+++ b/arch/arm64/crypto/sha2-ce-glue.c
@@ -0,0 +1,255 @@
+/*
+ * sha2-ce-glue.c - SHA-224/SHA-256 using ARMv8 Crypto Extensions
+ *
+ * Copyright (C) 2014 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <asm/neon.h>
+#include <asm/unaligned.h>
+#include <crypto/internal/hash.h>
+#include <crypto/sha.h>
+#include <linux/cpufeature.h>
+#include <linux/crypto.h>
+#include <linux/module.h>
+
+MODULE_DESCRIPTION("SHA-224/SHA-256 secure hash using ARMv8 Crypto Extensions");
+MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>");
+MODULE_LICENSE("GPL v2");
+
+asmlinkage int sha2_ce_transform(int blocks, u8 const *src, u32 *state,
+ u8 *head, long bytes);
+
+static int sha224_init(struct shash_desc *desc)
+{
+ struct sha256_state *sctx = shash_desc_ctx(desc);
+
+ *sctx = (struct sha256_state){
+ .state = {
+ SHA224_H0, SHA224_H1, SHA224_H2, SHA224_H3,
+ SHA224_H4, SHA224_H5, SHA224_H6, SHA224_H7,
+ }
+ };
+ return 0;
+}
+
+static int sha256_init(struct shash_desc *desc)
+{
+ struct sha256_state *sctx = shash_desc_ctx(desc);
+
+ *sctx = (struct sha256_state){
+ .state = {
+ SHA256_H0, SHA256_H1, SHA256_H2, SHA256_H3,
+ SHA256_H4, SHA256_H5, SHA256_H6, SHA256_H7,
+ }
+ };
+ return 0;
+}
+
+static int sha2_update(struct shash_desc *desc, const u8 *data,
+ unsigned int len)
+{
+ struct sha256_state *sctx = shash_desc_ctx(desc);
+ unsigned int partial = sctx->count % SHA256_BLOCK_SIZE;
+
+ sctx->count += len;
+
+ if ((partial + len) >= SHA256_BLOCK_SIZE) {
+ int blocks;
+
+ if (partial) {
+ int p = SHA256_BLOCK_SIZE - partial;
+
+ memcpy(sctx->buf + partial, data, p);
+ data += p;
+ len -= p;
+ }
+
+ blocks = len / SHA256_BLOCK_SIZE;
+ len %= SHA256_BLOCK_SIZE;
+
+ kernel_neon_begin_partial(28);
+ sha2_ce_transform(blocks, data, sctx->state,
+ partial ? sctx->buf : NULL, 0);
+ kernel_neon_end();
+
+ data += blocks * SHA256_BLOCK_SIZE;
+ partial = 0;
+ }
+ if (len)
+ memcpy(sctx->buf + partial, data, len);
+ return 0;
+}
+
+static void sha2_final(struct shash_desc *desc)
+{
+ static const u8 padding[SHA256_BLOCK_SIZE] = { 0x80, };
+
+ struct sha256_state *sctx = shash_desc_ctx(desc);
+ __be64 bits = cpu_to_be64(sctx->count << 3);
+ u32 padlen = SHA256_BLOCK_SIZE
+ - ((sctx->count + sizeof(bits)) % SHA256_BLOCK_SIZE);
+
+ sha2_update(desc, padding, padlen);
+ sha2_update(desc, (const u8 *)&bits, sizeof(bits));
+}
+
+static int sha224_final(struct shash_desc *desc, u8 *out)
+{
+ struct sha256_state *sctx = shash_desc_ctx(desc);
+ __be32 *dst = (__be32 *)out;
+ int i;
+
+ sha2_final(desc);
+
+ for (i = 0; i < SHA224_DIGEST_SIZE / sizeof(__be32); i++)
+ put_unaligned_be32(sctx->state[i], dst++);
+
+ *sctx = (struct sha256_state){};
+ return 0;
+}
+
+static int sha256_final(struct shash_desc *desc, u8 *out)
+{
+ struct sha256_state *sctx = shash_desc_ctx(desc);
+ __be32 *dst = (__be32 *)out;
+ int i;
+
+ sha2_final(desc);
+
+ for (i = 0; i < SHA256_DIGEST_SIZE / sizeof(__be32); i++)
+ put_unaligned_be32(sctx->state[i], dst++);
+
+ *sctx = (struct sha256_state){};
+ return 0;
+}
+
+static void sha2_finup(struct shash_desc *desc, const u8 *data,
+ unsigned int len)
+{
+ struct sha256_state *sctx = shash_desc_ctx(desc);
+ int blocks;
+
+ if (sctx->count || !len || (len % SHA256_BLOCK_SIZE)) {
+ sha2_update(desc, data, len);
+ sha2_final(desc);
+ return;
+ }
+
+ /*
+ * Use a fast path if the input is a multiple of 64 bytes. In
+ * this case, there is no need to copy data around, and we can
+ * perform the entire digest calculation in a single invocation
+ * of sha2_ce_transform()
+ */
+ blocks = len / SHA256_BLOCK_SIZE;
+
+ kernel_neon_begin_partial(28);
+ sha2_ce_transform(blocks, data, sctx->state, NULL, len);
+ kernel_neon_end();
+ data += blocks * SHA256_BLOCK_SIZE;
+}
+
+static int sha224_finup(struct shash_desc *desc, const u8 *data,
+ unsigned int len, u8 *out)
+{
+ struct sha256_state *sctx = shash_desc_ctx(desc);
+ __be32 *dst = (__be32 *)out;
+ int i;
+
+ sha2_finup(desc, data, len);
+
+ for (i = 0; i < SHA224_DIGEST_SIZE / sizeof(__be32); i++)
+ put_unaligned_be32(sctx->state[i], dst++);
+
+ *sctx = (struct sha256_state){};
+ return 0;
+}
+
+static int sha256_finup(struct shash_desc *desc, const u8 *data,
+ unsigned int len, u8 *out)
+{
+ struct sha256_state *sctx = shash_desc_ctx(desc);
+ __be32 *dst = (__be32 *)out;
+ int i;
+
+ sha2_finup(desc, data, len);
+
+ for (i = 0; i < SHA256_DIGEST_SIZE / sizeof(__be32); i++)
+ put_unaligned_be32(sctx->state[i], dst++);
+
+ *sctx = (struct sha256_state){};
+ return 0;
+}
+
+static int sha2_export(struct shash_desc *desc, void *out)
+{
+ struct sha256_state *sctx = shash_desc_ctx(desc);
+ struct sha256_state *dst = out;
+
+ *dst = *sctx;
+ return 0;
+}
+
+static int sha2_import(struct shash_desc *desc, const void *in)
+{
+ struct sha256_state *sctx = shash_desc_ctx(desc);
+ struct sha256_state const *src = in;
+
+ *sctx = *src;
+ return 0;
+}
+
+static struct shash_alg algs[] = { {
+ .init = sha224_init,
+ .update = sha2_update,
+ .final = sha224_final,
+ .finup = sha224_finup,
+ .export = sha2_export,
+ .import = sha2_import,
+ .descsize = sizeof(struct sha256_state),
+ .digestsize = SHA224_DIGEST_SIZE,
+ .statesize = sizeof(struct sha256_state),
+ .base = {
+ .cra_name = "sha224",
+ .cra_driver_name = "sha224-ce",
+ .cra_priority = 200,
+ .cra_flags = CRYPTO_ALG_TYPE_SHASH,
+ .cra_blocksize = SHA256_BLOCK_SIZE,
+ .cra_module = THIS_MODULE,
+ }
+}, {
+ .init = sha256_init,
+ .update = sha2_update,
+ .final = sha256_final,
+ .finup = sha256_finup,
+ .export = sha2_export,
+ .import = sha2_import,
+ .descsize = sizeof(struct sha256_state),
+ .digestsize = SHA256_DIGEST_SIZE,
+ .statesize = sizeof(struct sha256_state),
+ .base = {
+ .cra_name = "sha256",
+ .cra_driver_name = "sha256-ce",
+ .cra_priority = 200,
+ .cra_flags = CRYPTO_ALG_TYPE_SHASH,
+ .cra_blocksize = SHA256_BLOCK_SIZE,
+ .cra_module = THIS_MODULE,
+ }
+} };
+
+static int __init sha2_ce_mod_init(void)
+{
+ return crypto_register_shashes(algs, ARRAY_SIZE(algs));
+}
+
+static void __exit sha2_ce_mod_fini(void)
+{
+ crypto_unregister_shashes(algs, ARRAY_SIZE(algs));
+}
+
+module_cpu_feature_match(SHA2, sha2_ce_mod_init);
+module_exit(sha2_ce_mod_fini);
diff --git a/arch/arm64/include/asm/Kbuild b/arch/arm64/include/asm/Kbuild
index 83f71b3..42c7eec 100644
--- a/arch/arm64/include/asm/Kbuild
+++ b/arch/arm64/include/asm/Kbuild
@@ -40,6 +40,7 @@
generic-y += sembuf.h
generic-y += serial.h
generic-y += shmbuf.h
+generic-y += simd.h
generic-y += sizes.h
generic-y += socket.h
generic-y += sockios.h
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index fd3e392..5901480 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -21,6 +21,7 @@
#endif
#include <asm/ptrace.h>
+#include <asm/thread_info.h>
/*
* Stack pushing/popping (register pairs only). Equivalent to store decrement
@@ -68,23 +69,31 @@
msr daifclr, #8
.endm
- .macro disable_step, tmp
+ .macro disable_step_tsk, flgs, tmp
+ tbz \flgs, #TIF_SINGLESTEP, 9990f
mrs \tmp, mdscr_el1
bic \tmp, \tmp, #1
msr mdscr_el1, \tmp
+ isb // Synchronise with enable_dbg
+9990:
.endm
- .macro enable_step, tmp
+ .macro enable_step_tsk, flgs, tmp
+ tbz \flgs, #TIF_SINGLESTEP, 9990f
+ disable_dbg
mrs \tmp, mdscr_el1
orr \tmp, \tmp, #1
msr mdscr_el1, \tmp
+9990:
.endm
- .macro enable_dbg_if_not_stepping, tmp
- mrs \tmp, mdscr_el1
- tbnz \tmp, #0, 9990f
- enable_dbg
-9990:
+/*
+ * Enable both debug exceptions and interrupts. This is likely to be
+ * faster than two daifclr operations, since writes to this register
+ * are self-synchronising.
+ */
+ .macro enable_dbg_and_irq
+ msr daifclr, #(8 | 2)
.endm
/*
diff --git a/arch/arm64/include/asm/atomic.h b/arch/arm64/include/asm/atomic.h
index 57e8cb4..65f1569 100644
--- a/arch/arm64/include/asm/atomic.h
+++ b/arch/arm64/include/asm/atomic.h
@@ -157,7 +157,7 @@
*/
#define ATOMIC64_INIT(i) { (i) }
-#define atomic64_read(v) (*(volatile long long *)&(v)->counter)
+#define atomic64_read(v) (*(volatile long *)&(v)->counter)
#define atomic64_set(v,i) (((v)->counter) = (i))
static inline void atomic64_add(u64 i, atomic64_t *v)
diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h
index 48b9e70..6389d60 100644
--- a/arch/arm64/include/asm/barrier.h
+++ b/arch/arm64/include/asm/barrier.h
@@ -25,12 +25,12 @@
#define wfi() asm volatile("wfi" : : : "memory")
#define isb() asm volatile("isb" : : : "memory")
-#define dmb(opt) asm volatile("dmb sy" : : : "memory")
-#define dsb(opt) asm volatile("dsb sy" : : : "memory")
+#define dmb(opt) asm volatile("dmb " #opt : : : "memory")
+#define dsb(opt) asm volatile("dsb " #opt : : : "memory")
-#define mb() dsb()
-#define rmb() asm volatile("dsb ld" : : : "memory")
-#define wmb() asm volatile("dsb st" : : : "memory")
+#define mb() dsb(sy)
+#define rmb() dsb(ld)
+#define wmb() dsb(st)
#ifndef CONFIG_SMP
#define smp_mb() barrier()
@@ -40,7 +40,7 @@
#define smp_store_release(p, v) \
do { \
compiletime_assert_atomic_type(*p); \
- smp_mb(); \
+ barrier(); \
ACCESS_ONCE(*p) = (v); \
} while (0)
@@ -48,15 +48,15 @@
({ \
typeof(*p) ___p1 = ACCESS_ONCE(*p); \
compiletime_assert_atomic_type(*p); \
- smp_mb(); \
+ barrier(); \
___p1; \
})
#else
-#define smp_mb() asm volatile("dmb ish" : : : "memory")
-#define smp_rmb() asm volatile("dmb ishld" : : : "memory")
-#define smp_wmb() asm volatile("dmb ishst" : : : "memory")
+#define smp_mb() dmb(ish)
+#define smp_rmb() dmb(ishld)
+#define smp_wmb() dmb(ishst)
#define smp_store_release(p, v) \
do { \
diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h
index 390308a..88cc05b 100644
--- a/arch/arm64/include/asm/cache.h
+++ b/arch/arm64/include/asm/cache.h
@@ -16,6 +16,8 @@
#ifndef __ASM_CACHE_H
#define __ASM_CACHE_H
+#include <asm/cachetype.h>
+
#define L1_CACHE_SHIFT 6
#define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT)
@@ -27,6 +29,15 @@
* the CPU.
*/
#define ARCH_DMA_MINALIGN L1_CACHE_BYTES
-#define ARCH_SLAB_MINALIGN 8
+
+#ifndef __ASSEMBLY__
+
+static inline int cache_line_size(void)
+{
+ u32 cwg = cache_type_cwg();
+ return cwg ? 4 << cwg : L1_CACHE_BYTES;
+}
+
+#endif /* __ASSEMBLY__ */
#endif
diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h
index 4c60e64..a5176cf 100644
--- a/arch/arm64/include/asm/cacheflush.h
+++ b/arch/arm64/include/asm/cacheflush.h
@@ -123,7 +123,7 @@
static inline void __flush_icache_all(void)
{
asm("ic ialluis");
- dsb();
+ dsb(ish);
}
#define flush_dcache_mmap_lock(mapping) \
@@ -150,7 +150,7 @@
* set_pte_at() called from vmap_pte_range() does not
* have a DSB after cleaning the cache line.
*/
- dsb();
+ dsb(ish);
}
static inline void flush_cache_vunmap(unsigned long start, unsigned long end)
diff --git a/arch/arm64/include/asm/cachetype.h b/arch/arm64/include/asm/cachetype.h
index 85f5f51..4b23e75 100644
--- a/arch/arm64/include/asm/cachetype.h
+++ b/arch/arm64/include/asm/cachetype.h
@@ -20,12 +20,16 @@
#define CTR_L1IP_SHIFT 14
#define CTR_L1IP_MASK 3
+#define CTR_CWG_SHIFT 24
+#define CTR_CWG_MASK 15
#define ICACHE_POLICY_RESERVED 0
#define ICACHE_POLICY_AIVIVT 1
#define ICACHE_POLICY_VIPT 2
#define ICACHE_POLICY_PIPT 3
+#ifndef __ASSEMBLY__
+
static inline u32 icache_policy(void)
{
return (read_cpuid_cachetype() >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK;
@@ -45,4 +49,11 @@
return icache_policy() == ICACHE_POLICY_AIVIVT;
}
+static inline u32 cache_type_cwg(void)
+{
+ return (read_cpuid_cachetype() >> CTR_CWG_SHIFT) & CTR_CWG_MASK;
+}
+
+#endif /* __ASSEMBLY__ */
+
#endif /* __ASM_CACHETYPE_H */
diff --git a/arch/arm64/include/asm/cmpxchg.h b/arch/arm64/include/asm/cmpxchg.h
index 57c0fa7..ddb9d78 100644
--- a/arch/arm64/include/asm/cmpxchg.h
+++ b/arch/arm64/include/asm/cmpxchg.h
@@ -72,7 +72,12 @@
}
#define xchg(ptr,x) \
- ((__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr))))
+({ \
+ __typeof__(*(ptr)) __ret; \
+ __ret = (__typeof__(*(ptr))) \
+ __xchg((unsigned long)(x), (ptr), sizeof(*(ptr))); \
+ __ret; \
+})
static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
unsigned long new, int size)
diff --git a/arch/arm64/include/asm/compat.h b/arch/arm64/include/asm/compat.h
index e71f81f..253e33b 100644
--- a/arch/arm64/include/asm/compat.h
+++ b/arch/arm64/include/asm/compat.h
@@ -305,11 +305,6 @@
#else /* !CONFIG_COMPAT */
-static inline int is_compat_task(void)
-{
- return 0;
-}
-
static inline int is_compat_thread(struct thread_info *thread)
{
return 0;
diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
new file mode 100644
index 0000000..5a46c4e
--- /dev/null
+++ b/arch/arm64/include/asm/efi.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_EFI_H
+#define _ASM_EFI_H
+
+#include <asm/io.h>
+
+#ifdef CONFIG_EFI
+extern void efi_init(void);
+extern void efi_idmap_init(void);
+#else
+#define efi_init()
+#define efi_idmap_init()
+#endif
+
+#endif /* _ASM_EFI_H */
diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index c4a7f94..72674f4 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -18,9 +18,11 @@
#ifndef __ASM_ESR_H
#define __ASM_ESR_H
-#define ESR_EL1_EC_SHIFT (26)
-#define ESR_EL1_IL (1U << 25)
+#define ESR_EL1_WRITE (1 << 6)
+#define ESR_EL1_CM (1 << 8)
+#define ESR_EL1_IL (1 << 25)
+#define ESR_EL1_EC_SHIFT (26)
#define ESR_EL1_EC_UNKNOWN (0x00)
#define ESR_EL1_EC_WFI (0x01)
#define ESR_EL1_EC_CP15_32 (0x03)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index c43b4ac..50f559f 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -37,8 +37,21 @@
u32 fpcr;
};
};
+ /* the id of the last cpu to have restored this state */
+ unsigned int cpu;
};
+/*
+ * Struct for stacking the bottom 'n' FP/SIMD registers.
+ */
+struct fpsimd_partial_state {
+ u32 fpsr;
+ u32 fpcr;
+ u32 num_regs;
+ __uint128_t vregs[32];
+};
+
+
#if defined(__KERNEL__) && defined(CONFIG_COMPAT)
/* Masks for extracting the FPSR and FPCR from the FPSCR */
#define VFP_FPSCR_STAT_MASK 0xf800009f
@@ -58,6 +71,16 @@
extern void fpsimd_thread_switch(struct task_struct *next);
extern void fpsimd_flush_thread(void);
+extern void fpsimd_preserve_current_state(void);
+extern void fpsimd_restore_current_state(void);
+extern void fpsimd_update_current_state(struct fpsimd_state *state);
+
+extern void fpsimd_flush_task_state(struct task_struct *target);
+
+extern void fpsimd_save_partial_state(struct fpsimd_partial_state *state,
+ u32 num_regs);
+extern void fpsimd_load_partial_state(struct fpsimd_partial_state *state);
+
#endif
#endif
diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h
index bbec599c..768414d 100644
--- a/arch/arm64/include/asm/fpsimdmacros.h
+++ b/arch/arm64/include/asm/fpsimdmacros.h
@@ -62,3 +62,38 @@
ldr w\tmpnr, [\state, #16 * 2 + 4]
msr fpcr, x\tmpnr
.endm
+
+.altmacro
+.macro fpsimd_save_partial state, numnr, tmpnr1, tmpnr2
+ mrs x\tmpnr1, fpsr
+ str w\numnr, [\state, #8]
+ mrs x\tmpnr2, fpcr
+ stp w\tmpnr1, w\tmpnr2, [\state]
+ adr x\tmpnr1, 0f
+ add \state, \state, x\numnr, lsl #4
+ sub x\tmpnr1, x\tmpnr1, x\numnr, lsl #1
+ br x\tmpnr1
+ .irp qa, 30, 28, 26, 24, 22, 20, 18, 16, 14, 12, 10, 8, 6, 4, 2, 0
+ .irp qb, %(qa + 1)
+ stp q\qa, q\qb, [\state, # -16 * \qa - 16]
+ .endr
+ .endr
+0:
+.endm
+
+.macro fpsimd_restore_partial state, tmpnr1, tmpnr2
+ ldp w\tmpnr1, w\tmpnr2, [\state]
+ msr fpsr, x\tmpnr1
+ msr fpcr, x\tmpnr2
+ adr x\tmpnr1, 0f
+ ldr w\tmpnr2, [\state, #8]
+ add \state, \state, x\tmpnr2, lsl #4
+ sub x\tmpnr1, x\tmpnr1, x\tmpnr2, lsl #1
+ br x\tmpnr1
+ .irp qa, 30, 28, 26, 24, 22, 20, 18, 16, 14, 12, 10, 8, 6, 4, 2, 0
+ .irp qb, %(qa + 1)
+ ldp q\qa, q\qb, [\state, # -16 * \qa - 16]
+ .endr
+ .endr
+0:
+.endm
diff --git a/arch/arm64/include/asm/ftrace.h b/arch/arm64/include/asm/ftrace.h
new file mode 100644
index 0000000..c5534fa
--- /dev/null
+++ b/arch/arm64/include/asm/ftrace.h
@@ -0,0 +1,59 @@
+/*
+ * arch/arm64/include/asm/ftrace.h
+ *
+ * Copyright (C) 2013 Linaro Limited
+ * Author: AKASHI Takahiro <takahiro.akashi@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#ifndef __ASM_FTRACE_H
+#define __ASM_FTRACE_H
+
+#include <asm/insn.h>
+
+#define MCOUNT_ADDR ((unsigned long)_mcount)
+#define MCOUNT_INSN_SIZE AARCH64_INSN_SIZE
+
+#ifndef __ASSEMBLY__
+#include <linux/compat.h>
+
+extern void _mcount(unsigned long);
+extern void *return_address(unsigned int);
+
+struct dyn_arch_ftrace {
+ /* No extra data needed for arm64 */
+};
+
+extern unsigned long ftrace_graph_call;
+
+static inline unsigned long ftrace_call_adjust(unsigned long addr)
+{
+ /*
+ * addr is the address of the mcount call instruction.
+ * recordmcount does the necessary offset calculation.
+ */
+ return addr;
+}
+
+#define ftrace_return_address(n) return_address(n)
+
+/*
+ * Because AArch32 mode does not share the same syscall table with AArch64,
+ * tracing compat syscalls may result in reporting bogus syscalls or even
+ * hang-up, so just do not trace them.
+ * See kernel/trace/trace_syscalls.c
+ *
+ * x86 code says:
+ * If the user realy wants these, then they should use the
+ * raw syscall tracepoints with filtering.
+ */
+#define ARCH_TRACE_IGNORE_COMPAT_SYSCALLS
+static inline bool arch_trace_is_compat_syscall(struct pt_regs *regs)
+{
+ return is_compat_task();
+}
+#endif /* ifndef __ASSEMBLY__ */
+
+#endif /* __ASM_FTRACE_H */
diff --git a/arch/arm64/include/asm/hardirq.h b/arch/arm64/include/asm/hardirq.h
index ae4801d..0be6782 100644
--- a/arch/arm64/include/asm/hardirq.h
+++ b/arch/arm64/include/asm/hardirq.h
@@ -20,7 +20,7 @@
#include <linux/threads.h>
#include <asm/irq.h>
-#define NR_IPI 5
+#define NR_IPI 6
typedef struct {
unsigned int __softirq_pending;
diff --git a/arch/arm64/include/asm/insn.h b/arch/arm64/include/asm/insn.h
index c44ad39..dc1f73b 100644
--- a/arch/arm64/include/asm/insn.h
+++ b/arch/arm64/include/asm/insn.h
@@ -21,6 +21,7 @@
/* A64 instructions are always 32 bits. */
#define AARCH64_INSN_SIZE 4
+#ifndef __ASSEMBLY__
/*
* ARM Architecture Reference Manual for ARMv8 Profile-A, Issue A.a
* Section C3.1 "A64 instruction index by encoding":
@@ -104,5 +105,6 @@
int aarch64_insn_patch_text_nosync(void *addr, u32 insn);
int aarch64_insn_patch_text_sync(void *addrs[], u32 insns[], int cnt);
int aarch64_insn_patch_text(void *addrs[], u32 insns[], int cnt);
+#endif /* __ASSEMBLY__ */
#endif /* __ASM_INSN_H */
diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h
index a1bef78..e0ecdcf 100644
--- a/arch/arm64/include/asm/io.h
+++ b/arch/arm64/include/asm/io.h
@@ -230,19 +230,11 @@
extern void __iounmap(volatile void __iomem *addr);
extern void __iomem *ioremap_cache(phys_addr_t phys_addr, size_t size);
-#define PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_DIRTY)
-#define PROT_DEVICE_nGnRE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_ATTRINDX(MT_DEVICE_nGnRE))
-#define PROT_NORMAL_NC (PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL_NC))
-#define PROT_NORMAL (PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL))
-
#define ioremap(addr, size) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE))
#define ioremap_nocache(addr, size) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE))
#define ioremap_wc(addr, size) __ioremap((addr), (size), __pgprot(PROT_NORMAL_NC))
#define iounmap __iounmap
-#define PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF)
-#define PROT_SECT_DEVICE_nGnRE (PROT_SECT_DEFAULT | PTE_PXN | PTE_UXN | PMD_ATTRINDX(MT_DEVICE_nGnRE))
-
#define ARCH_HAS_IOREMAP_WC
#include <asm-generic/iomap.h>
diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h
index aff0292..c2f006c 100644
--- a/arch/arm64/include/asm/mmu.h
+++ b/arch/arm64/include/asm/mmu.h
@@ -31,5 +31,7 @@
extern void setup_mm_for_reboot(void);
extern void __iomem *early_io_map(phys_addr_t phys, unsigned long virt);
extern void init_mem_pgprot(void);
+/* create an identity mapping for memory (or io if map_io is true) */
+extern void create_id_mapping(phys_addr_t addr, phys_addr_t size, int map_io);
#endif
diff --git a/arch/arm64/include/asm/neon.h b/arch/arm64/include/asm/neon.h
index b0cc58a9..13ce4cc 100644
--- a/arch/arm64/include/asm/neon.h
+++ b/arch/arm64/include/asm/neon.h
@@ -8,7 +8,11 @@
* published by the Free Software Foundation.
*/
+#include <linux/types.h>
+
#define cpu_has_neon() (1)
-void kernel_neon_begin(void);
+#define kernel_neon_begin() kernel_neon_begin_partial(32)
+
+void kernel_neon_begin_partial(u32 num_regs);
void kernel_neon_end(void);
diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h
index 5fc8a66..955e8c5 100644
--- a/arch/arm64/include/asm/pgtable-hwdef.h
+++ b/arch/arm64/include/asm/pgtable-hwdef.h
@@ -29,6 +29,8 @@
*/
#define PUD_TABLE_BIT (_AT(pgdval_t, 1) << 1)
+#define PUD_TYPE_MASK (_AT(pgdval_t, 3) << 0)
+#define PUD_TYPE_SECT (_AT(pgdval_t, 1) << 0)
/*
* Level 2 descriptor (PMD).
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index e2f9674..598cc38 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -52,66 +52,59 @@
#endif
#define pgd_ERROR(pgd) __pgd_error(__FILE__, __LINE__, pgd_val(pgd))
-/*
- * The pgprot_* and protection_map entries will be fixed up at runtime to
- * include the cachable and bufferable bits based on memory policy, as well as
- * any architecture dependent bits like global/ASID and SMP shared mapping
- * bits.
- */
-#define _PAGE_DEFAULT PTE_TYPE_PAGE | PTE_AF
+#ifdef CONFIG_SMP
+#define PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED)
+#define PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S)
+#else
+#define PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF)
+#define PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF)
+#endif
-extern pgprot_t pgprot_default;
+#define PROT_DEVICE_nGnRE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_ATTRINDX(MT_DEVICE_nGnRE))
+#define PROT_NORMAL_NC (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_ATTRINDX(MT_NORMAL_NC))
+#define PROT_NORMAL (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_ATTRINDX(MT_NORMAL))
-#define __pgprot_modify(prot,mask,bits) \
- __pgprot((pgprot_val(prot) & ~(mask)) | (bits))
+#define PROT_SECT_DEVICE_nGnRE (PROT_SECT_DEFAULT | PMD_SECT_PXN | PMD_SECT_UXN | PMD_ATTRINDX(MT_DEVICE_nGnRE))
+#define PROT_SECT_NORMAL (PROT_SECT_DEFAULT | PMD_SECT_PXN | PMD_SECT_UXN | PMD_ATTRINDX(MT_NORMAL))
+#define PROT_SECT_NORMAL_EXEC (PROT_SECT_DEFAULT | PMD_SECT_UXN | PMD_ATTRINDX(MT_NORMAL))
-#define _MOD_PROT(p, b) __pgprot_modify(p, 0, b)
+#define _PAGE_DEFAULT (PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL))
-#define PAGE_NONE __pgprot_modify(pgprot_default, PTE_TYPE_MASK, PTE_PROT_NONE | PTE_PXN | PTE_UXN)
-#define PAGE_SHARED _MOD_PROT(pgprot_default, PTE_USER | PTE_NG | PTE_PXN | PTE_UXN | PTE_WRITE)
-#define PAGE_SHARED_EXEC _MOD_PROT(pgprot_default, PTE_USER | PTE_NG | PTE_PXN | PTE_WRITE)
-#define PAGE_COPY _MOD_PROT(pgprot_default, PTE_USER | PTE_NG | PTE_PXN | PTE_UXN)
-#define PAGE_COPY_EXEC _MOD_PROT(pgprot_default, PTE_USER | PTE_NG | PTE_PXN)
-#define PAGE_READONLY _MOD_PROT(pgprot_default, PTE_USER | PTE_NG | PTE_PXN | PTE_UXN)
-#define PAGE_READONLY_EXEC _MOD_PROT(pgprot_default, PTE_USER | PTE_NG | PTE_PXN)
-#define PAGE_KERNEL _MOD_PROT(pgprot_default, PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE)
-#define PAGE_KERNEL_EXEC _MOD_PROT(pgprot_default, PTE_UXN | PTE_DIRTY | PTE_WRITE)
+#define PAGE_KERNEL __pgprot(_PAGE_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE)
+#define PAGE_KERNEL_EXEC __pgprot(_PAGE_DEFAULT | PTE_UXN | PTE_DIRTY | PTE_WRITE)
-#define PAGE_HYP _MOD_PROT(pgprot_default, PTE_HYP)
+#define PAGE_HYP __pgprot(_PAGE_DEFAULT | PTE_HYP)
#define PAGE_HYP_DEVICE __pgprot(PROT_DEVICE_nGnRE | PTE_HYP)
-#define PAGE_S2 __pgprot_modify(pgprot_default, PTE_S2_MEMATTR_MASK, PTE_S2_MEMATTR(MT_S2_NORMAL) | PTE_S2_RDONLY)
+#define PAGE_S2 __pgprot(PROT_DEFAULT | PTE_S2_MEMATTR(MT_S2_NORMAL) | PTE_S2_RDONLY)
#define PAGE_S2_DEVICE __pgprot(PROT_DEFAULT | PTE_S2_MEMATTR(MT_S2_DEVICE_nGnRE) | PTE_S2_RDWR | PTE_UXN)
-#define __PAGE_NONE __pgprot(((_PAGE_DEFAULT) & ~PTE_TYPE_MASK) | PTE_PROT_NONE | PTE_PXN | PTE_UXN)
-#define __PAGE_SHARED __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_UXN | PTE_WRITE)
-#define __PAGE_SHARED_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_WRITE)
-#define __PAGE_COPY __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_UXN)
-#define __PAGE_COPY_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN)
-#define __PAGE_READONLY __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_UXN)
-#define __PAGE_READONLY_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN)
+#define PAGE_NONE __pgprot(((_PAGE_DEFAULT) & ~PTE_TYPE_MASK) | PTE_PROT_NONE | PTE_PXN | PTE_UXN)
+#define PAGE_SHARED __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_UXN | PTE_WRITE)
+#define PAGE_SHARED_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_WRITE)
+#define PAGE_COPY __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_UXN)
+#define PAGE_COPY_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN)
+#define PAGE_READONLY __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_UXN)
+#define PAGE_READONLY_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN)
-#endif /* __ASSEMBLY__ */
+#define __P000 PAGE_NONE
+#define __P001 PAGE_READONLY
+#define __P010 PAGE_COPY
+#define __P011 PAGE_COPY
+#define __P100 PAGE_READONLY_EXEC
+#define __P101 PAGE_READONLY_EXEC
+#define __P110 PAGE_COPY_EXEC
+#define __P111 PAGE_COPY_EXEC
-#define __P000 __PAGE_NONE
-#define __P001 __PAGE_READONLY
-#define __P010 __PAGE_COPY
-#define __P011 __PAGE_COPY
-#define __P100 __PAGE_READONLY_EXEC
-#define __P101 __PAGE_READONLY_EXEC
-#define __P110 __PAGE_COPY_EXEC
-#define __P111 __PAGE_COPY_EXEC
+#define __S000 PAGE_NONE
+#define __S001 PAGE_READONLY
+#define __S010 PAGE_SHARED
+#define __S011 PAGE_SHARED
+#define __S100 PAGE_READONLY_EXEC
+#define __S101 PAGE_READONLY_EXEC
+#define __S110 PAGE_SHARED_EXEC
+#define __S111 PAGE_SHARED_EXEC
-#define __S000 __PAGE_NONE
-#define __S001 __PAGE_READONLY
-#define __S010 __PAGE_SHARED
-#define __S011 __PAGE_SHARED
-#define __S100 __PAGE_READONLY_EXEC
-#define __S101 __PAGE_READONLY_EXEC
-#define __S110 __PAGE_SHARED_EXEC
-#define __S111 __PAGE_SHARED_EXEC
-
-#ifndef __ASSEMBLY__
/*
* ZERO_PAGE is a global shared page that is always zero: used
* for zero-mapped memory areas etc..
@@ -265,6 +258,7 @@
#define mk_pmd(page,prot) pfn_pmd(page_to_pfn(page),prot)
#define pmd_page(pmd) pfn_to_page(__phys_to_pfn(pmd_val(pmd) & PHYS_MASK))
+#define pud_pfn(pud) (((pud_val(pud) & PUD_MASK) & PHYS_MASK) >> PAGE_SHIFT)
#define set_pmd_at(mm, addr, pmdp, pmd) set_pte_at(mm, addr, (pte_t *)pmdp, pmd_pte(pmd))
@@ -273,6 +267,9 @@
return 1;
}
+#define __pgprot_modify(prot,mask,bits) \
+ __pgprot((pgprot_val(prot) & ~(mask)) | (bits))
+
/*
* Mark the prot value as uncacheable and unbufferable.
*/
@@ -295,11 +292,17 @@
#define pmd_sect(pmd) ((pmd_val(pmd) & PMD_TYPE_MASK) == \
PMD_TYPE_SECT)
+#ifdef ARM64_64K_PAGES
+#define pud_sect(pud) (0)
+#else
+#define pud_sect(pud) ((pud_val(pud) & PUD_TYPE_MASK) == \
+ PUD_TYPE_SECT)
+#endif
static inline void set_pmd(pmd_t *pmdp, pmd_t pmd)
{
*pmdp = pmd;
- dsb();
+ dsb(ishst);
}
static inline void pmd_clear(pmd_t *pmdp)
@@ -329,7 +332,7 @@
static inline void set_pud(pud_t *pudp, pud_t pud)
{
*pudp = pud;
- dsb();
+ dsb(ishst);
}
static inline void pud_clear(pud_t *pudp)
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 45b20cd..34de2a8 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -79,6 +79,7 @@
unsigned long tp_value;
struct fpsimd_state fpsimd_state;
unsigned long fault_address; /* fault info */
+ unsigned long fault_code; /* ESR_EL1 value */
struct debug_info debug; /* debugging */
};
diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h
index c7ba261..a429b59 100644
--- a/arch/arm64/include/asm/ptrace.h
+++ b/arch/arm64/include/asm/ptrace.h
@@ -135,6 +135,11 @@
#define user_stack_pointer(regs) \
(!compat_user_mode(regs)) ? ((regs)->sp) : ((regs)->compat_sp)
+static inline unsigned long regs_return_value(struct pt_regs *regs)
+{
+ return regs->regs[0];
+}
+
/*
* Are the current registers suitable for user mode? (used to maintain
* security in signal handlers)
diff --git a/arch/arm64/include/asm/sigcontext.h b/arch/arm64/include/asm/sigcontext.h
deleted file mode 100644
index dca1094..0000000
--- a/arch/arm64/include/asm/sigcontext.h
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-#ifndef __ASM_SIGCONTEXT_H
-#define __ASM_SIGCONTEXT_H
-
-#include <uapi/asm/sigcontext.h>
-
-/*
- * Auxiliary context saved in the sigcontext.__reserved array. Not exported to
- * user space as it will change with the addition of new context. User space
- * should check the magic/size information.
- */
-struct aux_context {
- struct fpsimd_context fpsimd;
- /* additional context to be added before "end" */
- struct _aarch64_ctx end;
-};
-#endif
diff --git a/arch/arm64/include/asm/string.h b/arch/arm64/include/asm/string.h
index 3ee8b30..64d2d48 100644
--- a/arch/arm64/include/asm/string.h
+++ b/arch/arm64/include/asm/string.h
@@ -22,6 +22,18 @@
#define __HAVE_ARCH_STRCHR
extern char *strchr(const char *, int c);
+#define __HAVE_ARCH_STRCMP
+extern int strcmp(const char *, const char *);
+
+#define __HAVE_ARCH_STRNCMP
+extern int strncmp(const char *, const char *, __kernel_size_t);
+
+#define __HAVE_ARCH_STRLEN
+extern __kernel_size_t strlen(const char *);
+
+#define __HAVE_ARCH_STRNLEN
+extern __kernel_size_t strnlen(const char *, __kernel_size_t);
+
#define __HAVE_ARCH_MEMCPY
extern void *memcpy(void *, const void *, __kernel_size_t);
@@ -34,4 +46,7 @@
#define __HAVE_ARCH_MEMSET
extern void *memset(void *, int, __kernel_size_t);
+#define __HAVE_ARCH_MEMCMP
+extern int memcmp(const void *, const void *, size_t);
+
#endif
diff --git a/arch/arm64/include/asm/syscall.h b/arch/arm64/include/asm/syscall.h
index 70ba9d4..383771e 100644
--- a/arch/arm64/include/asm/syscall.h
+++ b/arch/arm64/include/asm/syscall.h
@@ -18,6 +18,7 @@
#include <linux/err.h>
+extern const void *sys_call_table[];
static inline int syscall_get_nr(struct task_struct *task,
struct pt_regs *regs)
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index 7b8e3a2..e40b6d06 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -91,6 +91,9 @@
/*
* thread information flags:
* TIF_SYSCALL_TRACE - syscall trace active
+ * TIF_SYSCALL_TRACEPOINT - syscall tracepoint for ftrace
+ * TIF_SYSCALL_AUDIT - syscall auditing
+ * TIF_SECOMP - syscall secure computing
* TIF_SIGPENDING - signal pending
* TIF_NEED_RESCHED - rescheduling necessary
* TIF_NOTIFY_RESUME - callback before returning to user
@@ -99,7 +102,11 @@
#define TIF_SIGPENDING 0
#define TIF_NEED_RESCHED 1
#define TIF_NOTIFY_RESUME 2 /* callback before returning to user */
+#define TIF_FOREIGN_FPSTATE 3 /* CPU's FP state is not current's */
#define TIF_SYSCALL_TRACE 8
+#define TIF_SYSCALL_AUDIT 9
+#define TIF_SYSCALL_TRACEPOINT 10
+#define TIF_SECCOMP 11
#define TIF_MEMDIE 18 /* is terminating due to OOM killer */
#define TIF_FREEZE 19
#define TIF_RESTORE_SIGMASK 20
@@ -110,10 +117,18 @@
#define _TIF_SIGPENDING (1 << TIF_SIGPENDING)
#define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED)
#define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME)
+#define _TIF_FOREIGN_FPSTATE (1 << TIF_FOREIGN_FPSTATE)
+#define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE)
+#define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT)
+#define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT)
+#define _TIF_SECCOMP (1 << TIF_SECCOMP)
#define _TIF_32BIT (1 << TIF_32BIT)
#define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \
- _TIF_NOTIFY_RESUME)
+ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE)
+
+#define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \
+ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP)
#endif /* __KERNEL__ */
#endif /* __ASM_THREAD_INFO_H */
diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
index 8b48203..b9349c4 100644
--- a/arch/arm64/include/asm/tlbflush.h
+++ b/arch/arm64/include/asm/tlbflush.h
@@ -72,9 +72,9 @@
*/
static inline void flush_tlb_all(void)
{
- dsb();
+ dsb(ishst);
asm("tlbi vmalle1is");
- dsb();
+ dsb(ish);
isb();
}
@@ -82,9 +82,9 @@
{
unsigned long asid = (unsigned long)ASID(mm) << 48;
- dsb();
+ dsb(ishst);
asm("tlbi aside1is, %0" : : "r" (asid));
- dsb();
+ dsb(ish);
}
static inline void flush_tlb_page(struct vm_area_struct *vma,
@@ -93,16 +93,36 @@
unsigned long addr = uaddr >> 12 |
((unsigned long)ASID(vma->vm_mm) << 48);
- dsb();
+ dsb(ishst);
asm("tlbi vae1is, %0" : : "r" (addr));
- dsb();
+ dsb(ish);
}
-/*
- * Convert calls to our calling convention.
- */
-#define flush_tlb_range(vma,start,end) __cpu_flush_user_tlb_range(start,end,vma)
-#define flush_tlb_kernel_range(s,e) __cpu_flush_kern_tlb_range(s,e)
+static inline void flush_tlb_range(struct vm_area_struct *vma,
+ unsigned long start, unsigned long end)
+{
+ unsigned long asid = (unsigned long)ASID(vma->vm_mm) << 48;
+ unsigned long addr;
+ start = asid | (start >> 12);
+ end = asid | (end >> 12);
+
+ dsb(ishst);
+ for (addr = start; addr < end; addr += 1 << (PAGE_SHIFT - 12))
+ asm("tlbi vae1is, %0" : : "r"(addr));
+ dsb(ish);
+}
+
+static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end)
+{
+ unsigned long addr;
+ start >>= 12;
+ end >>= 12;
+
+ dsb(ishst);
+ for (addr = start; addr < end; addr += 1 << (PAGE_SHIFT - 12))
+ asm("tlbi vaae1is, %0" : : "r"(addr));
+ dsb(ish);
+}
/*
* On AArch64, the cache coherency is handled via the set_pte_at() function.
@@ -114,7 +134,7 @@
* set_pte() does not have a DSB, so make sure that the page table
* write is visible.
*/
- dsb();
+ dsb(ishst);
}
#define update_mmu_cache_pmd(vma, address, pmd) do { } while (0)
diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
index 0172e6d..7ebcd31 100644
--- a/arch/arm64/include/asm/topology.h
+++ b/arch/arm64/include/asm/topology.h
@@ -20,9 +20,6 @@
#define topology_core_cpumask(cpu) (&cpu_topology[cpu].core_sibling)
#define topology_thread_cpumask(cpu) (&cpu_topology[cpu].thread_sibling)
-#define mc_capable() (cpu_topology[0].cluster_id != -1)
-#define smt_capable() (cpu_topology[0].thread_id != -1)
-
void init_cpu_topology(void);
void store_cpu_topology(unsigned int cpuid);
const struct cpumask *cpu_coregroup_mask(int cpu);
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index a4654c6..e5f47df 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -29,3 +29,5 @@
#endif
#define __ARCH_WANT_SYS_CLONE
#include <uapi/asm/unistd.h>
+
+#define NR_syscalls (__NR_syscalls)
diff --git a/arch/arm64/include/uapi/asm/sigcontext.h b/arch/arm64/include/uapi/asm/sigcontext.h
index 690ad51..b72cf40 100644
--- a/arch/arm64/include/uapi/asm/sigcontext.h
+++ b/arch/arm64/include/uapi/asm/sigcontext.h
@@ -53,5 +53,12 @@
__uint128_t vregs[32];
};
+/* ESR_EL1 context */
+#define ESR_MAGIC 0x45535201
+
+struct esr_context {
+ struct _aarch64_ctx head;
+ u64 esr;
+};
#endif /* _UAPI__ASM_SIGCONTEXT_H */
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 7a6fce5..cdaedad 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -4,15 +4,22 @@
CPPFLAGS_vmlinux.lds := -DTEXT_OFFSET=$(TEXT_OFFSET)
AFLAGS_head.o := -DTEXT_OFFSET=$(TEXT_OFFSET)
+CFLAGS_efi-stub.o := -DTEXT_OFFSET=$(TEXT_OFFSET) \
+ -I$(src)/../../../scripts/dtc/libfdt
+
+CFLAGS_REMOVE_ftrace.o = -pg
+CFLAGS_REMOVE_insn.o = -pg
+CFLAGS_REMOVE_return_address.o = -pg
# Object file lists.
arm64-obj-y := cputable.o debug-monitors.o entry.o irq.o fpsimd.o \
entry-fpsimd.o process.o ptrace.o setup.o signal.o \
sys.o stacktrace.o time.o traps.o io.o vdso.o \
- hyp-stub.o psci.o cpu_ops.o insn.o
+ hyp-stub.o psci.o cpu_ops.o insn.o return_address.o
arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
sys_compat.o
+arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_SMP) += smp.o smp_spin_table.o topology.o
arm64-obj-$(CONFIG_PERF_EVENTS) += perf_regs.o
@@ -21,6 +28,7 @@
arm64-obj-$(CONFIG_ARM64_CPU_SUSPEND) += sleep.o suspend.o
arm64-obj-$(CONFIG_JUMP_LABEL) += jump_label.o
arm64-obj-$(CONFIG_KGDB) += kgdb.o
+arm64-obj-$(CONFIG_EFI) += efi.o efi-stub.o efi-entry.o
obj-y += $(arm64-obj-y) vdso/
obj-m += $(arm64-obj-m)
diff --git a/arch/arm64/kernel/arm64ksyms.c b/arch/arm64/kernel/arm64ksyms.c
index 338b568..a85843d 100644
--- a/arch/arm64/kernel/arm64ksyms.c
+++ b/arch/arm64/kernel/arm64ksyms.c
@@ -44,10 +44,15 @@
/* string / mem functions */
EXPORT_SYMBOL(strchr);
EXPORT_SYMBOL(strrchr);
+EXPORT_SYMBOL(strcmp);
+EXPORT_SYMBOL(strncmp);
+EXPORT_SYMBOL(strlen);
+EXPORT_SYMBOL(strnlen);
EXPORT_SYMBOL(memset);
EXPORT_SYMBOL(memcpy);
EXPORT_SYMBOL(memmove);
EXPORT_SYMBOL(memchr);
+EXPORT_SYMBOL(memcmp);
/* atomic bitops */
EXPORT_SYMBOL(set_bit);
@@ -56,3 +61,7 @@
EXPORT_SYMBOL(test_and_clear_bit);
EXPORT_SYMBOL(change_bit);
EXPORT_SYMBOL(test_and_change_bit);
+
+#ifdef CONFIG_FUNCTION_TRACER
+EXPORT_SYMBOL(_mcount);
+#endif
diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S
new file mode 100644
index 0000000..66716c9
--- /dev/null
+++ b/arch/arm64/kernel/efi-entry.S
@@ -0,0 +1,109 @@
+/*
+ * EFI entry point.
+ *
+ * Copyright (C) 2013, 2014 Red Hat, Inc.
+ * Author: Mark Salter <msalter@redhat.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+#include <linux/linkage.h>
+#include <linux/init.h>
+
+#include <asm/assembler.h>
+
+#define EFI_LOAD_ERROR 0x8000000000000001
+
+ __INIT
+
+ /*
+ * We arrive here from the EFI boot manager with:
+ *
+ * * CPU in little-endian mode
+ * * MMU on with identity-mapped RAM
+ * * Icache and Dcache on
+ *
+ * We will most likely be running from some place other than where
+ * we want to be. The kernel image wants to be placed at TEXT_OFFSET
+ * from start of RAM.
+ */
+ENTRY(efi_stub_entry)
+ /*
+ * Create a stack frame to save FP/LR with extra space
+ * for image_addr variable passed to efi_entry().
+ */
+ stp x29, x30, [sp, #-32]!
+
+ /*
+ * Call efi_entry to do the real work.
+ * x0 and x1 are already set up by firmware. Current runtime
+ * address of image is calculated and passed via *image_addr.
+ *
+ * unsigned long efi_entry(void *handle,
+ * efi_system_table_t *sys_table,
+ * unsigned long *image_addr) ;
+ */
+ adrp x8, _text
+ add x8, x8, #:lo12:_text
+ add x2, sp, 16
+ str x8, [x2]
+ bl efi_entry
+ cmn x0, #1
+ b.eq efi_load_fail
+
+ /*
+ * efi_entry() will have relocated the kernel image if necessary
+ * and we return here with device tree address in x0 and the kernel
+ * entry point stored at *image_addr. Save those values in registers
+ * which are callee preserved.
+ */
+ mov x20, x0 // DTB address
+ ldr x0, [sp, #16] // relocated _text address
+ mov x21, x0
+
+ /*
+ * Flush dcache covering current runtime addresses
+ * of kernel text/data. Then flush all of icache.
+ */
+ adrp x1, _text
+ add x1, x1, #:lo12:_text
+ adrp x2, _edata
+ add x2, x2, #:lo12:_edata
+ sub x1, x2, x1
+
+ bl __flush_dcache_area
+ ic ialluis
+
+ /* Turn off Dcache and MMU */
+ mrs x0, CurrentEL
+ cmp x0, #PSR_MODE_EL2t
+ ccmp x0, #PSR_MODE_EL2h, #0x4, ne
+ b.ne 1f
+ mrs x0, sctlr_el2
+ bic x0, x0, #1 << 0 // clear SCTLR.M
+ bic x0, x0, #1 << 2 // clear SCTLR.C
+ msr sctlr_el2, x0
+ isb
+ b 2f
+1:
+ mrs x0, sctlr_el1
+ bic x0, x0, #1 << 0 // clear SCTLR.M
+ bic x0, x0, #1 << 2 // clear SCTLR.C
+ msr sctlr_el1, x0
+ isb
+2:
+ /* Jump to kernel entry point */
+ mov x0, x20
+ mov x1, xzr
+ mov x2, xzr
+ mov x3, xzr
+ br x21
+
+efi_load_fail:
+ mov x0, #EFI_LOAD_ERROR
+ ldp x29, x30, [sp], #32
+ ret
+
+ENDPROC(efi_stub_entry)
diff --git a/arch/arm64/kernel/efi-stub.c b/arch/arm64/kernel/efi-stub.c
new file mode 100644
index 0000000..60e98a63
--- /dev/null
+++ b/arch/arm64/kernel/efi-stub.c
@@ -0,0 +1,81 @@
+/*
+ * Copyright (C) 2013, 2014 Linaro Ltd; <roy.franz@linaro.org>
+ *
+ * This file implements the EFI boot stub for the arm64 kernel.
+ * Adapted from ARM version by Mark Salter <msalter@redhat.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+#include <linux/efi.h>
+#include <linux/libfdt.h>
+#include <asm/sections.h>
+#include <generated/compile.h>
+#include <generated/utsrelease.h>
+
+/*
+ * AArch64 requires the DTB to be 8-byte aligned in the first 512MiB from
+ * start of kernel and may not cross a 2MiB boundary. We set alignment to
+ * 2MiB so we know it won't cross a 2MiB boundary.
+ */
+#define EFI_FDT_ALIGN SZ_2M /* used by allocate_new_fdt_and_exit_boot() */
+#define MAX_FDT_OFFSET SZ_512M
+
+#define efi_call_early(f, ...) sys_table_arg->boottime->f(__VA_ARGS__)
+
+static void efi_char16_printk(efi_system_table_t *sys_table_arg,
+ efi_char16_t *str);
+
+static efi_status_t efi_open_volume(efi_system_table_t *sys_table,
+ void *__image, void **__fh);
+static efi_status_t efi_file_close(void *handle);
+
+static efi_status_t
+efi_file_read(void *handle, unsigned long *size, void *addr);
+
+static efi_status_t
+efi_file_size(efi_system_table_t *sys_table, void *__fh,
+ efi_char16_t *filename_16, void **handle, u64 *file_sz);
+
+/* Include shared EFI stub code */
+#include "../../../drivers/firmware/efi/efi-stub-helper.c"
+#include "../../../drivers/firmware/efi/fdt.c"
+#include "../../../drivers/firmware/efi/arm-stub.c"
+
+
+static efi_status_t handle_kernel_image(efi_system_table_t *sys_table,
+ unsigned long *image_addr,
+ unsigned long *image_size,
+ unsigned long *reserve_addr,
+ unsigned long *reserve_size,
+ unsigned long dram_base,
+ efi_loaded_image_t *image)
+{
+ efi_status_t status;
+ unsigned long kernel_size, kernel_memsize = 0;
+
+ /* Relocate the image, if required. */
+ kernel_size = _edata - _text;
+ if (*image_addr != (dram_base + TEXT_OFFSET)) {
+ kernel_memsize = kernel_size + (_end - _edata);
+ status = efi_relocate_kernel(sys_table, image_addr,
+ kernel_size, kernel_memsize,
+ dram_base + TEXT_OFFSET,
+ PAGE_SIZE);
+ if (status != EFI_SUCCESS) {
+ pr_efi_err(sys_table, "Failed to relocate kernel\n");
+ return status;
+ }
+ if (*image_addr != (dram_base + TEXT_OFFSET)) {
+ pr_efi_err(sys_table, "Failed to alloc kernel memory\n");
+ efi_free(sys_table, kernel_memsize, *image_addr);
+ return EFI_ERROR;
+ }
+ *image_size = kernel_memsize;
+ }
+
+
+ return EFI_SUCCESS;
+}
diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
new file mode 100644
index 0000000..14db1f6
--- /dev/null
+++ b/arch/arm64/kernel/efi.c
@@ -0,0 +1,469 @@
+/*
+ * Extensible Firmware Interface
+ *
+ * Based on Extensible Firmware Interface Specification version 2.4
+ *
+ * Copyright (C) 2013, 2014 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/efi.h>
+#include <linux/export.h>
+#include <linux/memblock.h>
+#include <linux/bootmem.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+
+#include <asm/cacheflush.h>
+#include <asm/efi.h>
+#include <asm/tlbflush.h>
+#include <asm/mmu_context.h>
+
+struct efi_memory_map memmap;
+
+static efi_runtime_services_t *runtime;
+
+static u64 efi_system_table;
+
+static int uefi_debug __initdata;
+static int __init uefi_debug_setup(char *str)
+{
+ uefi_debug = 1;
+
+ return 0;
+}
+early_param("uefi_debug", uefi_debug_setup);
+
+static int __init is_normal_ram(efi_memory_desc_t *md)
+{
+ if (md->attribute & EFI_MEMORY_WB)
+ return 1;
+ return 0;
+}
+
+static void __init efi_setup_idmap(void)
+{
+ struct memblock_region *r;
+ efi_memory_desc_t *md;
+ u64 paddr, npages, size;
+
+ for_each_memblock(memory, r)
+ create_id_mapping(r->base, r->size, 0);
+
+ /* map runtime io spaces */
+ for_each_efi_memory_desc(&memmap, md) {
+ if (!(md->attribute & EFI_MEMORY_RUNTIME) || is_normal_ram(md))
+ continue;
+ paddr = md->phys_addr;
+ npages = md->num_pages;
+ memrange_efi_to_native(&paddr, &npages);
+ size = npages << PAGE_SHIFT;
+ create_id_mapping(paddr, size, 1);
+ }
+}
+
+static int __init uefi_init(void)
+{
+ efi_char16_t *c16;
+ char vendor[100] = "unknown";
+ int i, retval;
+
+ efi.systab = early_memremap(efi_system_table,
+ sizeof(efi_system_table_t));
+ if (efi.systab == NULL) {
+ pr_warn("Unable to map EFI system table.\n");
+ return -ENOMEM;
+ }
+
+ set_bit(EFI_BOOT, &efi.flags);
+ set_bit(EFI_64BIT, &efi.flags);
+
+ /*
+ * Verify the EFI Table
+ */
+ if (efi.systab->hdr.signature != EFI_SYSTEM_TABLE_SIGNATURE) {
+ pr_err("System table signature incorrect\n");
+ return -EINVAL;
+ }
+ if ((efi.systab->hdr.revision >> 16) < 2)
+ pr_warn("Warning: EFI system table version %d.%02d, expected 2.00 or greater\n",
+ efi.systab->hdr.revision >> 16,
+ efi.systab->hdr.revision & 0xffff);
+
+ /* Show what we know for posterity */
+ c16 = early_memremap(efi.systab->fw_vendor,
+ sizeof(vendor));
+ if (c16) {
+ for (i = 0; i < (int) sizeof(vendor) - 1 && *c16; ++i)
+ vendor[i] = c16[i];
+ vendor[i] = '\0';
+ }
+
+ pr_info("EFI v%u.%.02u by %s\n",
+ efi.systab->hdr.revision >> 16,
+ efi.systab->hdr.revision & 0xffff, vendor);
+
+ retval = efi_config_init(NULL);
+ if (retval == 0)
+ set_bit(EFI_CONFIG_TABLES, &efi.flags);
+
+ early_memunmap(c16, sizeof(vendor));
+ early_memunmap(efi.systab, sizeof(efi_system_table_t));
+
+ return retval;
+}
+
+static __initdata char memory_type_name[][32] = {
+ {"Reserved"},
+ {"Loader Code"},
+ {"Loader Data"},
+ {"Boot Code"},
+ {"Boot Data"},
+ {"Runtime Code"},
+ {"Runtime Data"},
+ {"Conventional Memory"},
+ {"Unusable Memory"},
+ {"ACPI Reclaim Memory"},
+ {"ACPI Memory NVS"},
+ {"Memory Mapped I/O"},
+ {"MMIO Port Space"},
+ {"PAL Code"},
+};
+
+/*
+ * Return true for RAM regions we want to permanently reserve.
+ */
+static __init int is_reserve_region(efi_memory_desc_t *md)
+{
+ if (!is_normal_ram(md))
+ return 0;
+
+ if (md->attribute & EFI_MEMORY_RUNTIME)
+ return 1;
+
+ if (md->type == EFI_ACPI_RECLAIM_MEMORY ||
+ md->type == EFI_RESERVED_TYPE)
+ return 1;
+
+ return 0;
+}
+
+static __init void reserve_regions(void)
+{
+ efi_memory_desc_t *md;
+ u64 paddr, npages, size;
+
+ if (uefi_debug)
+ pr_info("Processing EFI memory map:\n");
+
+ for_each_efi_memory_desc(&memmap, md) {
+ paddr = md->phys_addr;
+ npages = md->num_pages;
+
+ if (uefi_debug)
+ pr_info(" 0x%012llx-0x%012llx [%s]",
+ paddr, paddr + (npages << EFI_PAGE_SHIFT) - 1,
+ memory_type_name[md->type]);
+
+ memrange_efi_to_native(&paddr, &npages);
+ size = npages << PAGE_SHIFT;
+
+ if (is_normal_ram(md))
+ early_init_dt_add_memory_arch(paddr, size);
+
+ if (is_reserve_region(md) ||
+ md->type == EFI_BOOT_SERVICES_CODE ||
+ md->type == EFI_BOOT_SERVICES_DATA) {
+ memblock_reserve(paddr, size);
+ if (uefi_debug)
+ pr_cont("*");
+ }
+
+ if (uefi_debug)
+ pr_cont("\n");
+ }
+}
+
+
+static u64 __init free_one_region(u64 start, u64 end)
+{
+ u64 size = end - start;
+
+ if (uefi_debug)
+ pr_info(" EFI freeing: 0x%012llx-0x%012llx\n", start, end - 1);
+
+ free_bootmem_late(start, size);
+ return size;
+}
+
+static u64 __init free_region(u64 start, u64 end)
+{
+ u64 map_start, map_end, total = 0;
+
+ if (end <= start)
+ return total;
+
+ map_start = (u64)memmap.phys_map;
+ map_end = PAGE_ALIGN(map_start + (memmap.map_end - memmap.map));
+ map_start &= PAGE_MASK;
+
+ if (start < map_end && end > map_start) {
+ /* region overlaps UEFI memmap */
+ if (start < map_start)
+ total += free_one_region(start, map_start);
+
+ if (map_end < end)
+ total += free_one_region(map_end, end);
+ } else
+ total += free_one_region(start, end);
+
+ return total;
+}
+
+static void __init free_boot_services(void)
+{
+ u64 total_freed = 0;
+ u64 keep_end, free_start, free_end;
+ efi_memory_desc_t *md;
+
+ /*
+ * If kernel uses larger pages than UEFI, we have to be careful
+ * not to inadvertantly free memory we want to keep if there is
+ * overlap at the kernel page size alignment. We do not want to
+ * free is_reserve_region() memory nor the UEFI memmap itself.
+ *
+ * The memory map is sorted, so we keep track of the end of
+ * any previous region we want to keep, remember any region
+ * we want to free and defer freeing it until we encounter
+ * the next region we want to keep. This way, before freeing
+ * it, we can clip it as needed to avoid freeing memory we
+ * want to keep for UEFI.
+ */
+
+ keep_end = 0;
+ free_start = 0;
+
+ for_each_efi_memory_desc(&memmap, md) {
+ u64 paddr, npages, size;
+
+ if (is_reserve_region(md)) {
+ /*
+ * We don't want to free any memory from this region.
+ */
+ if (free_start) {
+ /* adjust free_end then free region */
+ if (free_end > md->phys_addr)
+ free_end -= PAGE_SIZE;
+ total_freed += free_region(free_start, free_end);
+ free_start = 0;
+ }
+ keep_end = md->phys_addr + (md->num_pages << EFI_PAGE_SHIFT);
+ continue;
+ }
+
+ if (md->type != EFI_BOOT_SERVICES_CODE &&
+ md->type != EFI_BOOT_SERVICES_DATA) {
+ /* no need to free this region */
+ continue;
+ }
+
+ /*
+ * We want to free memory from this region.
+ */
+ paddr = md->phys_addr;
+ npages = md->num_pages;
+ memrange_efi_to_native(&paddr, &npages);
+ size = npages << PAGE_SHIFT;
+
+ if (free_start) {
+ if (paddr <= free_end)
+ free_end = paddr + size;
+ else {
+ total_freed += free_region(free_start, free_end);
+ free_start = paddr;
+ free_end = paddr + size;
+ }
+ } else {
+ free_start = paddr;
+ free_end = paddr + size;
+ }
+ if (free_start < keep_end) {
+ free_start += PAGE_SIZE;
+ if (free_start >= free_end)
+ free_start = 0;
+ }
+ }
+ if (free_start)
+ total_freed += free_region(free_start, free_end);
+
+ if (total_freed)
+ pr_info("Freed 0x%llx bytes of EFI boot services memory",
+ total_freed);
+}
+
+void __init efi_init(void)
+{
+ struct efi_fdt_params params;
+
+ /* Grab UEFI information placed in FDT by stub */
+ if (!efi_get_fdt_params(¶ms, uefi_debug))
+ return;
+
+ efi_system_table = params.system_table;
+
+ memblock_reserve(params.mmap & PAGE_MASK,
+ PAGE_ALIGN(params.mmap_size + (params.mmap & ~PAGE_MASK)));
+ memmap.phys_map = (void *)params.mmap;
+ memmap.map = early_memremap(params.mmap, params.mmap_size);
+ memmap.map_end = memmap.map + params.mmap_size;
+ memmap.desc_size = params.desc_size;
+ memmap.desc_version = params.desc_ver;
+
+ if (uefi_init() < 0)
+ return;
+
+ reserve_regions();
+}
+
+void __init efi_idmap_init(void)
+{
+ if (!efi_enabled(EFI_BOOT))
+ return;
+
+ /* boot time idmap_pg_dir is incomplete, so fill in missing parts */
+ efi_setup_idmap();
+}
+
+static int __init remap_region(efi_memory_desc_t *md, void **new)
+{
+ u64 paddr, vaddr, npages, size;
+
+ paddr = md->phys_addr;
+ npages = md->num_pages;
+ memrange_efi_to_native(&paddr, &npages);
+ size = npages << PAGE_SHIFT;
+
+ if (is_normal_ram(md))
+ vaddr = (__force u64)ioremap_cache(paddr, size);
+ else
+ vaddr = (__force u64)ioremap(paddr, size);
+
+ if (!vaddr) {
+ pr_err("Unable to remap 0x%llx pages @ %p\n",
+ npages, (void *)paddr);
+ return 0;
+ }
+
+ /* adjust for any rounding when EFI and system pagesize differs */
+ md->virt_addr = vaddr + (md->phys_addr - paddr);
+
+ if (uefi_debug)
+ pr_info(" EFI remap 0x%012llx => %p\n",
+ md->phys_addr, (void *)md->virt_addr);
+
+ memcpy(*new, md, memmap.desc_size);
+ *new += memmap.desc_size;
+
+ return 1;
+}
+
+/*
+ * Switch UEFI from an identity map to a kernel virtual map
+ */
+static int __init arm64_enter_virtual_mode(void)
+{
+ efi_memory_desc_t *md;
+ phys_addr_t virtmap_phys;
+ void *virtmap, *virt_md;
+ efi_status_t status;
+ u64 mapsize;
+ int count = 0;
+ unsigned long flags;
+
+ if (!efi_enabled(EFI_BOOT)) {
+ pr_info("EFI services will not be available.\n");
+ return -1;
+ }
+
+ pr_info("Remapping and enabling EFI services.\n");
+
+ /* replace early memmap mapping with permanent mapping */
+ mapsize = memmap.map_end - memmap.map;
+ early_memunmap(memmap.map, mapsize);
+ memmap.map = (__force void *)ioremap_cache((phys_addr_t)memmap.phys_map,
+ mapsize);
+ memmap.map_end = memmap.map + mapsize;
+
+ efi.memmap = &memmap;
+
+ /* Map the runtime regions */
+ virtmap = kmalloc(mapsize, GFP_KERNEL);
+ if (!virtmap) {
+ pr_err("Failed to allocate EFI virtual memmap\n");
+ return -1;
+ }
+ virtmap_phys = virt_to_phys(virtmap);
+ virt_md = virtmap;
+
+ for_each_efi_memory_desc(&memmap, md) {
+ if (!(md->attribute & EFI_MEMORY_RUNTIME))
+ continue;
+ if (remap_region(md, &virt_md))
+ ++count;
+ }
+
+ efi.systab = (__force void *)efi_lookup_mapped_addr(efi_system_table);
+ if (efi.systab)
+ set_bit(EFI_SYSTEM_TABLES, &efi.flags);
+
+ local_irq_save(flags);
+ cpu_switch_mm(idmap_pg_dir, &init_mm);
+
+ /* Call SetVirtualAddressMap with the physical address of the map */
+ runtime = efi.systab->runtime;
+ efi.set_virtual_address_map = runtime->set_virtual_address_map;
+
+ status = efi.set_virtual_address_map(count * memmap.desc_size,
+ memmap.desc_size,
+ memmap.desc_version,
+ (efi_memory_desc_t *)virtmap_phys);
+ cpu_set_reserved_ttbr0();
+ flush_tlb_all();
+ local_irq_restore(flags);
+
+ kfree(virtmap);
+
+ free_boot_services();
+
+ if (status != EFI_SUCCESS) {
+ pr_err("Failed to set EFI virtual address map! [%lx]\n",
+ status);
+ return -1;
+ }
+
+ /* Set up runtime services function pointers */
+ runtime = efi.systab->runtime;
+ efi.get_time = runtime->get_time;
+ efi.set_time = runtime->set_time;
+ efi.get_wakeup_time = runtime->get_wakeup_time;
+ efi.set_wakeup_time = runtime->set_wakeup_time;
+ efi.get_variable = runtime->get_variable;
+ efi.get_next_variable = runtime->get_next_variable;
+ efi.set_variable = runtime->set_variable;
+ efi.query_variable_info = runtime->query_variable_info;
+ efi.update_capsule = runtime->update_capsule;
+ efi.query_capsule_caps = runtime->query_capsule_caps;
+ efi.get_next_high_mono_count = runtime->get_next_high_mono_count;
+ efi.reset_system = runtime->reset_system;
+
+ set_bit(EFI_RUNTIME_SERVICES, &efi.flags);
+
+ return 0;
+}
+early_initcall(arm64_enter_virtual_mode);
diff --git a/arch/arm64/kernel/entry-fpsimd.S b/arch/arm64/kernel/entry-fpsimd.S
index 6a27cd6..d358cca 100644
--- a/arch/arm64/kernel/entry-fpsimd.S
+++ b/arch/arm64/kernel/entry-fpsimd.S
@@ -41,3 +41,27 @@
fpsimd_restore x0, 8
ret
ENDPROC(fpsimd_load_state)
+
+#ifdef CONFIG_KERNEL_MODE_NEON
+
+/*
+ * Save the bottom n FP registers.
+ *
+ * x0 - pointer to struct fpsimd_partial_state
+ */
+ENTRY(fpsimd_save_partial_state)
+ fpsimd_save_partial x0, 1, 8, 9
+ ret
+ENDPROC(fpsimd_load_partial_state)
+
+/*
+ * Load the bottom n FP registers.
+ *
+ * x0 - pointer to struct fpsimd_partial_state
+ */
+ENTRY(fpsimd_load_partial_state)
+ fpsimd_restore_partial x0, 8, 9
+ ret
+ENDPROC(fpsimd_load_partial_state)
+
+#endif
diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S
new file mode 100644
index 0000000..b051871
--- /dev/null
+++ b/arch/arm64/kernel/entry-ftrace.S
@@ -0,0 +1,218 @@
+/*
+ * arch/arm64/kernel/entry-ftrace.S
+ *
+ * Copyright (C) 2013 Linaro Limited
+ * Author: AKASHI Takahiro <takahiro.akashi@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+#include <asm/ftrace.h>
+#include <asm/insn.h>
+
+/*
+ * Gcc with -pg will put the following code in the beginning of each function:
+ * mov x0, x30
+ * bl _mcount
+ * [function's body ...]
+ * "bl _mcount" may be replaced to "bl ftrace_caller" or NOP if dynamic
+ * ftrace is enabled.
+ *
+ * Please note that x0 as an argument will not be used here because we can
+ * get lr(x30) of instrumented function at any time by winding up call stack
+ * as long as the kernel is compiled without -fomit-frame-pointer.
+ * (or CONFIG_FRAME_POINTER, this is forced on arm64)
+ *
+ * stack layout after mcount_enter in _mcount():
+ *
+ * current sp/fp => 0:+-----+
+ * in _mcount() | x29 | -> instrumented function's fp
+ * +-----+
+ * | x30 | -> _mcount()'s lr (= instrumented function's pc)
+ * old sp => +16:+-----+
+ * when instrumented | |
+ * function calls | ... |
+ * _mcount() | |
+ * | |
+ * instrumented => +xx:+-----+
+ * function's fp | x29 | -> parent's fp
+ * +-----+
+ * | x30 | -> instrumented function's lr (= parent's pc)
+ * +-----+
+ * | ... |
+ */
+
+ .macro mcount_enter
+ stp x29, x30, [sp, #-16]!
+ mov x29, sp
+ .endm
+
+ .macro mcount_exit
+ ldp x29, x30, [sp], #16
+ ret
+ .endm
+
+ .macro mcount_adjust_addr rd, rn
+ sub \rd, \rn, #AARCH64_INSN_SIZE
+ .endm
+
+ /* for instrumented function's parent */
+ .macro mcount_get_parent_fp reg
+ ldr \reg, [x29]
+ ldr \reg, [\reg]
+ .endm
+
+ /* for instrumented function */
+ .macro mcount_get_pc0 reg
+ mcount_adjust_addr \reg, x30
+ .endm
+
+ .macro mcount_get_pc reg
+ ldr \reg, [x29, #8]
+ mcount_adjust_addr \reg, \reg
+ .endm
+
+ .macro mcount_get_lr reg
+ ldr \reg, [x29]
+ ldr \reg, [\reg, #8]
+ mcount_adjust_addr \reg, \reg
+ .endm
+
+ .macro mcount_get_lr_addr reg
+ ldr \reg, [x29]
+ add \reg, \reg, #8
+ .endm
+
+#ifndef CONFIG_DYNAMIC_FTRACE
+/*
+ * void _mcount(unsigned long return_address)
+ * @return_address: return address to instrumented function
+ *
+ * This function makes calls, if enabled, to:
+ * - tracer function to probe instrumented function's entry,
+ * - ftrace_graph_caller to set up an exit hook
+ */
+ENTRY(_mcount)
+#ifdef CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST
+ ldr x0, =ftrace_trace_stop
+ ldr x0, [x0] // if ftrace_trace_stop
+ ret // return;
+#endif
+ mcount_enter
+
+ ldr x0, =ftrace_trace_function
+ ldr x2, [x0]
+ adr x0, ftrace_stub
+ cmp x0, x2 // if (ftrace_trace_function
+ b.eq skip_ftrace_call // != ftrace_stub) {
+
+ mcount_get_pc x0 // function's pc
+ mcount_get_lr x1 // function's lr (= parent's pc)
+ blr x2 // (*ftrace_trace_function)(pc, lr);
+
+#ifndef CONFIG_FUNCTION_GRAPH_TRACER
+skip_ftrace_call: // return;
+ mcount_exit // }
+#else
+ mcount_exit // return;
+ // }
+skip_ftrace_call:
+ ldr x1, =ftrace_graph_return
+ ldr x2, [x1] // if ((ftrace_graph_return
+ cmp x0, x2 // != ftrace_stub)
+ b.ne ftrace_graph_caller
+
+ ldr x1, =ftrace_graph_entry // || (ftrace_graph_entry
+ ldr x2, [x1] // != ftrace_graph_entry_stub))
+ ldr x0, =ftrace_graph_entry_stub
+ cmp x0, x2
+ b.ne ftrace_graph_caller // ftrace_graph_caller();
+
+ mcount_exit
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
+ENDPROC(_mcount)
+
+#else /* CONFIG_DYNAMIC_FTRACE */
+/*
+ * _mcount() is used to build the kernel with -pg option, but all the branch
+ * instructions to _mcount() are replaced to NOP initially at kernel start up,
+ * and later on, NOP to branch to ftrace_caller() when enabled or branch to
+ * NOP when disabled per-function base.
+ */
+ENTRY(_mcount)
+ ret
+ENDPROC(_mcount)
+
+/*
+ * void ftrace_caller(unsigned long return_address)
+ * @return_address: return address to instrumented function
+ *
+ * This function is a counterpart of _mcount() in 'static' ftrace, and
+ * makes calls to:
+ * - tracer function to probe instrumented function's entry,
+ * - ftrace_graph_caller to set up an exit hook
+ */
+ENTRY(ftrace_caller)
+ mcount_enter
+
+ mcount_get_pc0 x0 // function's pc
+ mcount_get_lr x1 // function's lr
+
+ .global ftrace_call
+ftrace_call: // tracer(pc, lr);
+ nop // This will be replaced with "bl xxx"
+ // where xxx can be any kind of tracer.
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+ .global ftrace_graph_call
+ftrace_graph_call: // ftrace_graph_caller();
+ nop // If enabled, this will be replaced
+ // "b ftrace_graph_caller"
+#endif
+
+ mcount_exit
+ENDPROC(ftrace_caller)
+#endif /* CONFIG_DYNAMIC_FTRACE */
+
+ENTRY(ftrace_stub)
+ ret
+ENDPROC(ftrace_stub)
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+/*
+ * void ftrace_graph_caller(void)
+ *
+ * Called from _mcount() or ftrace_caller() when function_graph tracer is
+ * selected.
+ * This function w/ prepare_ftrace_return() fakes link register's value on
+ * the call stack in order to intercept instrumented function's return path
+ * and run return_to_handler() later on its exit.
+ */
+ENTRY(ftrace_graph_caller)
+ mcount_get_lr_addr x0 // pointer to function's saved lr
+ mcount_get_pc x1 // function's pc
+ mcount_get_parent_fp x2 // parent's fp
+ bl prepare_ftrace_return // prepare_ftrace_return(&lr, pc, fp)
+
+ mcount_exit
+ENDPROC(ftrace_graph_caller)
+
+/*
+ * void return_to_handler(void)
+ *
+ * Run ftrace_return_to_handler() before going back to parent.
+ * @fp is checked against the value passed by ftrace_graph_caller()
+ * only when CONFIG_FUNCTION_GRAPH_FP_TEST is enabled.
+ */
+ENTRY(return_to_handler)
+ str x0, [sp, #-16]!
+ mov x0, x29 // parent's fp
+ bl ftrace_return_to_handler// addr = ftrace_return_to_hander(fp);
+ mov x30, x0 // restore the original return address
+ ldr x0, [sp], #16
+ ret
+END(return_to_handler)
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 39ac630..bf017f4 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -60,6 +60,9 @@
push x0, x1
.if \el == 0
mrs x21, sp_el0
+ get_thread_info tsk // Ensure MDSCR_EL1.SS is clear,
+ ldr x19, [tsk, #TI_FLAGS] // since we can unmask debug
+ disable_step_tsk x19, x20 // exceptions when scheduling.
.else
add x21, sp, #S_FRAME_SIZE
.endif
@@ -259,7 +262,7 @@
* Data abort handling
*/
mrs x0, far_el1
- enable_dbg_if_not_stepping x2
+ enable_dbg
// re-enable interrupts if they were enabled in the aborted context
tbnz x23, #7, 1f // PSR_I_BIT
enable_irq
@@ -275,6 +278,7 @@
* Stack or PC alignment exception handling
*/
mrs x0, far_el1
+ enable_dbg
mov x1, x25
mov x2, sp
b do_sp_pc_abort
@@ -282,6 +286,7 @@
/*
* Undefined instruction
*/
+ enable_dbg
mov x0, sp
b do_undefinstr
el1_dbg:
@@ -294,10 +299,11 @@
mrs x0, far_el1
mov x2, sp // struct pt_regs
bl do_debug_exception
-
+ enable_dbg
kernel_exit 1
el1_inv:
// TODO: add support for undefined instructions in kernel mode
+ enable_dbg
mov x0, sp
mov x1, #BAD_SYNC
mrs x2, esr_el1
@@ -307,7 +313,7 @@
.align 6
el1_irq:
kernel_entry 1
- enable_dbg_if_not_stepping x0
+ enable_dbg
#ifdef CONFIG_TRACE_IRQFLAGS
bl trace_hardirqs_off
#endif
@@ -332,8 +338,7 @@
#ifdef CONFIG_PREEMPT
el1_preempt:
mov x24, lr
-1: enable_dbg
- bl preempt_schedule_irq // irq en/disable is done inside
+1: bl preempt_schedule_irq // irq en/disable is done inside
ldr x0, [tsk, #TI_FLAGS] // get new tasks TI_FLAGS
tbnz x0, #TIF_NEED_RESCHED, 1b // needs rescheduling?
ret x24
@@ -349,7 +354,7 @@
lsr x24, x25, #ESR_EL1_EC_SHIFT // exception class
cmp x24, #ESR_EL1_EC_SVC64 // SVC in 64-bit state
b.eq el0_svc
- adr lr, ret_from_exception
+ adr lr, ret_to_user
cmp x24, #ESR_EL1_EC_DABT_EL0 // data abort in EL0
b.eq el0_da
cmp x24, #ESR_EL1_EC_IABT_EL0 // instruction abort in EL0
@@ -378,7 +383,7 @@
lsr x24, x25, #ESR_EL1_EC_SHIFT // exception class
cmp x24, #ESR_EL1_EC_SVC32 // SVC in 32-bit state
b.eq el0_svc_compat
- adr lr, ret_from_exception
+ adr lr, ret_to_user
cmp x24, #ESR_EL1_EC_DABT_EL0 // data abort in EL0
b.eq el0_da
cmp x24, #ESR_EL1_EC_IABT_EL0 // instruction abort in EL0
@@ -423,11 +428,8 @@
*/
mrs x0, far_el1
bic x0, x0, #(0xff << 56)
- disable_step x1
- isb
- enable_dbg
// enable interrupts before calling the main handler
- enable_irq
+ enable_dbg_and_irq
mov x1, x25
mov x2, sp
b do_mem_abort
@@ -436,11 +438,8 @@
* Instruction abort handling
*/
mrs x0, far_el1
- disable_step x1
- isb
- enable_dbg
// enable interrupts before calling the main handler
- enable_irq
+ enable_dbg_and_irq
orr x1, x25, #1 << 24 // use reserved ISS bit for instruction aborts
mov x2, sp
b do_mem_abort
@@ -448,6 +447,7 @@
/*
* Floating Point or Advanced SIMD access
*/
+ enable_dbg
mov x0, x25
mov x1, sp
b do_fpsimd_acc
@@ -455,6 +455,7 @@
/*
* Floating Point or Advanced SIMD exception
*/
+ enable_dbg
mov x0, x25
mov x1, sp
b do_fpsimd_exc
@@ -463,11 +464,8 @@
* Stack or PC alignment exception handling
*/
mrs x0, far_el1
- disable_step x1
- isb
- enable_dbg
// enable interrupts before calling the main handler
- enable_irq
+ enable_dbg_and_irq
mov x1, x25
mov x2, sp
b do_sp_pc_abort
@@ -475,9 +473,9 @@
/*
* Undefined instruction
*/
- mov x0, sp
// enable interrupts before calling the main handler
- enable_irq
+ enable_dbg_and_irq
+ mov x0, sp
b do_undefinstr
el0_dbg:
/*
@@ -485,11 +483,13 @@
*/
tbnz x24, #0, el0_inv // EL0 only
mrs x0, far_el1
- disable_step x1
mov x1, x25
mov x2, sp
- b do_debug_exception
+ bl do_debug_exception
+ enable_dbg
+ b ret_to_user
el0_inv:
+ enable_dbg
mov x0, sp
mov x1, #BAD_SYNC
mrs x2, esr_el1
@@ -500,15 +500,12 @@
el0_irq:
kernel_entry 0
el0_irq_naked:
- disable_step x1
- isb
enable_dbg
#ifdef CONFIG_TRACE_IRQFLAGS
bl trace_hardirqs_off
#endif
irq_handler
- get_thread_info tsk
#ifdef CONFIG_TRACE_IRQFLAGS
bl trace_hardirqs_on
@@ -517,14 +514,6 @@
ENDPROC(el0_irq)
/*
- * This is the return code to user mode for abort handlers
- */
-ret_from_exception:
- get_thread_info tsk
- b ret_to_user
-ENDPROC(ret_from_exception)
-
-/*
* Register switch for AArch64. The callee-saved registers need to be saved
* and restored. On entry:
* x0 = previous task_struct (must be preserved across the switch)
@@ -563,10 +552,7 @@
ldr x1, [tsk, #TI_FLAGS]
and x2, x1, #_TIF_WORK_MASK
cbnz x2, fast_work_pending
- tbz x1, #TIF_SINGLESTEP, fast_exit
- disable_dbg
- enable_step x2
-fast_exit:
+ enable_step_tsk x1, x2
kernel_exit 0, ret = 1
/*
@@ -576,7 +562,7 @@
str x0, [sp, #S_X0] // returned x0
work_pending:
tbnz x1, #TIF_NEED_RESCHED, work_resched
- /* TIF_SIGPENDING or TIF_NOTIFY_RESUME case */
+ /* TIF_SIGPENDING, TIF_NOTIFY_RESUME or TIF_FOREIGN_FPSTATE case */
ldr x2, [sp, #S_PSTATE]
mov x0, sp // 'regs'
tst x2, #PSR_MODE_MASK // user mode regs?
@@ -585,7 +571,6 @@
bl do_notify_resume
b ret_to_user
work_resched:
- enable_dbg
bl schedule
/*
@@ -596,9 +581,7 @@
ldr x1, [tsk, #TI_FLAGS]
and x2, x1, #_TIF_WORK_MASK
cbnz x2, work_pending
- tbz x1, #TIF_SINGLESTEP, no_work_pending
- disable_dbg
- enable_step x2
+ enable_step_tsk x1, x2
no_work_pending:
kernel_exit 0, ret = 0
ENDPROC(ret_to_user)
@@ -625,14 +608,11 @@
mov sc_nr, #__NR_syscalls
el0_svc_naked: // compat entry point
stp x0, scno, [sp, #S_ORIG_X0] // save the original x0 and syscall number
- disable_step x16
- isb
- enable_dbg
- enable_irq
+ enable_dbg_and_irq
- get_thread_info tsk
- ldr x16, [tsk, #TI_FLAGS] // check for syscall tracing
- tbnz x16, #TIF_SYSCALL_TRACE, __sys_trace // are we tracing syscalls?
+ ldr x16, [tsk, #TI_FLAGS] // check for syscall hooks
+ tst x16, #_TIF_SYSCALL_WORK
+ b.ne __sys_trace
adr lr, ret_fast_syscall // return address
cmp scno, sc_nr // check upper syscall limit
b.hs ni_sys
@@ -648,9 +628,8 @@
* switches, and waiting for our parent to respond.
*/
__sys_trace:
- mov x1, sp
- mov w0, #0 // trace entry
- bl syscall_trace
+ mov x0, sp
+ bl syscall_trace_enter
adr lr, __sys_trace_return // return address
uxtw scno, w0 // syscall number (possibly new)
mov x1, sp // pointer to regs
@@ -665,9 +644,8 @@
__sys_trace_return:
str x0, [sp] // save returned x0
- mov x1, sp
- mov w0, #1 // trace exit
- bl syscall_trace
+ mov x0, sp
+ bl syscall_trace_exit
b ret_to_user
/*
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 4aef42a..ad8aebb 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -35,6 +35,60 @@
#define FPEXC_IDF (1 << 7)
/*
+ * In order to reduce the number of times the FPSIMD state is needlessly saved
+ * and restored, we need to keep track of two things:
+ * (a) for each task, we need to remember which CPU was the last one to have
+ * the task's FPSIMD state loaded into its FPSIMD registers;
+ * (b) for each CPU, we need to remember which task's userland FPSIMD state has
+ * been loaded into its FPSIMD registers most recently, or whether it has
+ * been used to perform kernel mode NEON in the meantime.
+ *
+ * For (a), we add a 'cpu' field to struct fpsimd_state, which gets updated to
+ * the id of the current CPU everytime the state is loaded onto a CPU. For (b),
+ * we add the per-cpu variable 'fpsimd_last_state' (below), which contains the
+ * address of the userland FPSIMD state of the task that was loaded onto the CPU
+ * the most recently, or NULL if kernel mode NEON has been performed after that.
+ *
+ * With this in place, we no longer have to restore the next FPSIMD state right
+ * when switching between tasks. Instead, we can defer this check to userland
+ * resume, at which time we verify whether the CPU's fpsimd_last_state and the
+ * task's fpsimd_state.cpu are still mutually in sync. If this is the case, we
+ * can omit the FPSIMD restore.
+ *
+ * As an optimization, we use the thread_info flag TIF_FOREIGN_FPSTATE to
+ * indicate whether or not the userland FPSIMD state of the current task is
+ * present in the registers. The flag is set unless the FPSIMD registers of this
+ * CPU currently contain the most recent userland FPSIMD state of the current
+ * task.
+ *
+ * For a certain task, the sequence may look something like this:
+ * - the task gets scheduled in; if both the task's fpsimd_state.cpu field
+ * contains the id of the current CPU, and the CPU's fpsimd_last_state per-cpu
+ * variable points to the task's fpsimd_state, the TIF_FOREIGN_FPSTATE flag is
+ * cleared, otherwise it is set;
+ *
+ * - the task returns to userland; if TIF_FOREIGN_FPSTATE is set, the task's
+ * userland FPSIMD state is copied from memory to the registers, the task's
+ * fpsimd_state.cpu field is set to the id of the current CPU, the current
+ * CPU's fpsimd_last_state pointer is set to this task's fpsimd_state and the
+ * TIF_FOREIGN_FPSTATE flag is cleared;
+ *
+ * - the task executes an ordinary syscall; upon return to userland, the
+ * TIF_FOREIGN_FPSTATE flag will still be cleared, so no FPSIMD state is
+ * restored;
+ *
+ * - the task executes a syscall which executes some NEON instructions; this is
+ * preceded by a call to kernel_neon_begin(), which copies the task's FPSIMD
+ * register contents to memory, clears the fpsimd_last_state per-cpu variable
+ * and sets the TIF_FOREIGN_FPSTATE flag;
+ *
+ * - the task gets preempted after kernel_neon_end() is called; as we have not
+ * returned from the 2nd syscall yet, TIF_FOREIGN_FPSTATE is still set so
+ * whatever is in the FPSIMD registers is not saved to memory, but discarded.
+ */
+static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
+
+/*
* Trapped FP/ASIMD access.
*/
void do_fpsimd_acc(unsigned int esr, struct pt_regs *regs)
@@ -72,43 +126,137 @@
void fpsimd_thread_switch(struct task_struct *next)
{
- /* check if not kernel threads */
- if (current->mm)
+ /*
+ * Save the current FPSIMD state to memory, but only if whatever is in
+ * the registers is in fact the most recent userland FPSIMD state of
+ * 'current'.
+ */
+ if (current->mm && !test_thread_flag(TIF_FOREIGN_FPSTATE))
fpsimd_save_state(¤t->thread.fpsimd_state);
- if (next->mm)
- fpsimd_load_state(&next->thread.fpsimd_state);
+
+ if (next->mm) {
+ /*
+ * If we are switching to a task whose most recent userland
+ * FPSIMD state is already in the registers of *this* cpu,
+ * we can skip loading the state from memory. Otherwise, set
+ * the TIF_FOREIGN_FPSTATE flag so the state will be loaded
+ * upon the next return to userland.
+ */
+ struct fpsimd_state *st = &next->thread.fpsimd_state;
+
+ if (__this_cpu_read(fpsimd_last_state) == st
+ && st->cpu == smp_processor_id())
+ clear_ti_thread_flag(task_thread_info(next),
+ TIF_FOREIGN_FPSTATE);
+ else
+ set_ti_thread_flag(task_thread_info(next),
+ TIF_FOREIGN_FPSTATE);
+ }
}
void fpsimd_flush_thread(void)
{
- preempt_disable();
memset(¤t->thread.fpsimd_state, 0, sizeof(struct fpsimd_state));
- fpsimd_load_state(¤t->thread.fpsimd_state);
+ set_thread_flag(TIF_FOREIGN_FPSTATE);
+}
+
+/*
+ * Save the userland FPSIMD state of 'current' to memory, but only if the state
+ * currently held in the registers does in fact belong to 'current'
+ */
+void fpsimd_preserve_current_state(void)
+{
+ preempt_disable();
+ if (!test_thread_flag(TIF_FOREIGN_FPSTATE))
+ fpsimd_save_state(¤t->thread.fpsimd_state);
preempt_enable();
}
+/*
+ * Load the userland FPSIMD state of 'current' from memory, but only if the
+ * FPSIMD state already held in the registers is /not/ the most recent FPSIMD
+ * state of 'current'
+ */
+void fpsimd_restore_current_state(void)
+{
+ preempt_disable();
+ if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
+ struct fpsimd_state *st = ¤t->thread.fpsimd_state;
+
+ fpsimd_load_state(st);
+ this_cpu_write(fpsimd_last_state, st);
+ st->cpu = smp_processor_id();
+ }
+ preempt_enable();
+}
+
+/*
+ * Load an updated userland FPSIMD state for 'current' from memory and set the
+ * flag that indicates that the FPSIMD register contents are the most recent
+ * FPSIMD state of 'current'
+ */
+void fpsimd_update_current_state(struct fpsimd_state *state)
+{
+ preempt_disable();
+ fpsimd_load_state(state);
+ if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
+ struct fpsimd_state *st = ¤t->thread.fpsimd_state;
+
+ this_cpu_write(fpsimd_last_state, st);
+ st->cpu = smp_processor_id();
+ }
+ preempt_enable();
+}
+
+/*
+ * Invalidate live CPU copies of task t's FPSIMD state
+ */
+void fpsimd_flush_task_state(struct task_struct *t)
+{
+ t->thread.fpsimd_state.cpu = NR_CPUS;
+}
+
#ifdef CONFIG_KERNEL_MODE_NEON
+static DEFINE_PER_CPU(struct fpsimd_partial_state, hardirq_fpsimdstate);
+static DEFINE_PER_CPU(struct fpsimd_partial_state, softirq_fpsimdstate);
+
/*
* Kernel-side NEON support functions
*/
-void kernel_neon_begin(void)
+void kernel_neon_begin_partial(u32 num_regs)
{
- /* Avoid using the NEON in interrupt context */
- BUG_ON(in_interrupt());
- preempt_disable();
+ if (in_interrupt()) {
+ struct fpsimd_partial_state *s = this_cpu_ptr(
+ in_irq() ? &hardirq_fpsimdstate : &softirq_fpsimdstate);
- if (current->mm)
- fpsimd_save_state(¤t->thread.fpsimd_state);
+ BUG_ON(num_regs > 32);
+ fpsimd_save_partial_state(s, roundup(num_regs, 2));
+ } else {
+ /*
+ * Save the userland FPSIMD state if we have one and if we
+ * haven't done so already. Clear fpsimd_last_state to indicate
+ * that there is no longer userland FPSIMD state in the
+ * registers.
+ */
+ preempt_disable();
+ if (current->mm &&
+ !test_and_set_thread_flag(TIF_FOREIGN_FPSTATE))
+ fpsimd_save_state(¤t->thread.fpsimd_state);
+ this_cpu_write(fpsimd_last_state, NULL);
+ }
}
-EXPORT_SYMBOL(kernel_neon_begin);
+EXPORT_SYMBOL(kernel_neon_begin_partial);
void kernel_neon_end(void)
{
- if (current->mm)
- fpsimd_load_state(¤t->thread.fpsimd_state);
-
- preempt_enable();
+ if (in_interrupt()) {
+ struct fpsimd_partial_state *s = this_cpu_ptr(
+ in_irq() ? &hardirq_fpsimdstate : &softirq_fpsimdstate);
+ fpsimd_load_partial_state(s);
+ } else {
+ preempt_enable();
+ }
}
EXPORT_SYMBOL(kernel_neon_end);
@@ -120,12 +268,12 @@
{
switch (cmd) {
case CPU_PM_ENTER:
- if (current->mm)
+ if (current->mm && !test_thread_flag(TIF_FOREIGN_FPSTATE))
fpsimd_save_state(¤t->thread.fpsimd_state);
break;
case CPU_PM_EXIT:
if (current->mm)
- fpsimd_load_state(¤t->thread.fpsimd_state);
+ set_thread_flag(TIF_FOREIGN_FPSTATE);
break;
case CPU_PM_ENTER_FAILED:
default:
diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c
new file mode 100644
index 0000000..7924d73
--- /dev/null
+++ b/arch/arm64/kernel/ftrace.c
@@ -0,0 +1,176 @@
+/*
+ * arch/arm64/kernel/ftrace.c
+ *
+ * Copyright (C) 2013 Linaro Limited
+ * Author: AKASHI Takahiro <takahiro.akashi@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/ftrace.h>
+#include <linux/swab.h>
+#include <linux/uaccess.h>
+
+#include <asm/cacheflush.h>
+#include <asm/ftrace.h>
+#include <asm/insn.h>
+
+#ifdef CONFIG_DYNAMIC_FTRACE
+/*
+ * Replace a single instruction, which may be a branch or NOP.
+ * If @validate == true, a replaced instruction is checked against 'old'.
+ */
+static int ftrace_modify_code(unsigned long pc, u32 old, u32 new,
+ bool validate)
+{
+ u32 replaced;
+
+ /*
+ * Note:
+ * Due to modules and __init, code can disappear and change,
+ * we need to protect against faulting as well as code changing.
+ * We do this by aarch64_insn_*() which use the probe_kernel_*().
+ *
+ * No lock is held here because all the modifications are run
+ * through stop_machine().
+ */
+ if (validate) {
+ if (aarch64_insn_read((void *)pc, &replaced))
+ return -EFAULT;
+
+ if (replaced != old)
+ return -EINVAL;
+ }
+ if (aarch64_insn_patch_text_nosync((void *)pc, new))
+ return -EPERM;
+
+ return 0;
+}
+
+/*
+ * Replace tracer function in ftrace_caller()
+ */
+int ftrace_update_ftrace_func(ftrace_func_t func)
+{
+ unsigned long pc;
+ u32 new;
+
+ pc = (unsigned long)&ftrace_call;
+ new = aarch64_insn_gen_branch_imm(pc, (unsigned long)func, true);
+
+ return ftrace_modify_code(pc, 0, new, false);
+}
+
+/*
+ * Turn on the call to ftrace_caller() in instrumented function
+ */
+int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+{
+ unsigned long pc = rec->ip;
+ u32 old, new;
+
+ old = aarch64_insn_gen_nop();
+ new = aarch64_insn_gen_branch_imm(pc, addr, true);
+
+ return ftrace_modify_code(pc, old, new, true);
+}
+
+/*
+ * Turn off the call to ftrace_caller() in instrumented function
+ */
+int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
+ unsigned long addr)
+{
+ unsigned long pc = rec->ip;
+ u32 old, new;
+
+ old = aarch64_insn_gen_branch_imm(pc, addr, true);
+ new = aarch64_insn_gen_nop();
+
+ return ftrace_modify_code(pc, old, new, true);
+}
+
+int __init ftrace_dyn_arch_init(void)
+{
+ return 0;
+}
+#endif /* CONFIG_DYNAMIC_FTRACE */
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+/*
+ * function_graph tracer expects ftrace_return_to_handler() to be called
+ * on the way back to parent. For this purpose, this function is called
+ * in _mcount() or ftrace_caller() to replace return address (*parent) on
+ * the call stack to return_to_handler.
+ *
+ * Note that @frame_pointer is used only for sanity check later.
+ */
+void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
+ unsigned long frame_pointer)
+{
+ unsigned long return_hooker = (unsigned long)&return_to_handler;
+ unsigned long old;
+ struct ftrace_graph_ent trace;
+ int err;
+
+ if (unlikely(atomic_read(¤t->tracing_graph_pause)))
+ return;
+
+ /*
+ * Note:
+ * No protection against faulting at *parent, which may be seen
+ * on other archs. It's unlikely on AArch64.
+ */
+ old = *parent;
+ *parent = return_hooker;
+
+ trace.func = self_addr;
+ trace.depth = current->curr_ret_stack + 1;
+
+ /* Only trace if the calling function expects to */
+ if (!ftrace_graph_entry(&trace)) {
+ *parent = old;
+ return;
+ }
+
+ err = ftrace_push_return_trace(old, self_addr, &trace.depth,
+ frame_pointer);
+ if (err == -EBUSY) {
+ *parent = old;
+ return;
+ }
+}
+
+#ifdef CONFIG_DYNAMIC_FTRACE
+/*
+ * Turn on/off the call to ftrace_graph_caller() in ftrace_caller()
+ * depending on @enable.
+ */
+static int ftrace_modify_graph_caller(bool enable)
+{
+ unsigned long pc = (unsigned long)&ftrace_graph_call;
+ u32 branch, nop;
+
+ branch = aarch64_insn_gen_branch_imm(pc,
+ (unsigned long)ftrace_graph_caller, false);
+ nop = aarch64_insn_gen_nop();
+
+ if (enable)
+ return ftrace_modify_code(pc, nop, branch, true);
+ else
+ return ftrace_modify_code(pc, branch, nop, true);
+}
+
+int ftrace_enable_ftrace_graph_caller(void)
+{
+ return ftrace_modify_graph_caller(true);
+}
+
+int ftrace_disable_ftrace_graph_caller(void)
+{
+ return ftrace_modify_graph_caller(false);
+}
+#endif /* CONFIG_DYNAMIC_FTRACE */
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 0fd5650..a96d3a6 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -108,8 +108,18 @@
/*
* DO NOT MODIFY. Image header expected by Linux boot-loaders.
*/
+#ifdef CONFIG_EFI
+efi_head:
+ /*
+ * This add instruction has no meaningful effect except that
+ * its opcode forms the magic "MZ" signature required by UEFI.
+ */
+ add x13, x18, #0x16
+ b stext
+#else
b stext // branch to kernel start, magic
.long 0 // reserved
+#endif
.quad TEXT_OFFSET // Image load offset from start of RAM
.quad 0 // reserved
.quad 0 // reserved
@@ -120,7 +130,109 @@
.byte 0x52
.byte 0x4d
.byte 0x64
+#ifdef CONFIG_EFI
+ .long pe_header - efi_head // Offset to the PE header.
+#else
.word 0 // reserved
+#endif
+
+#ifdef CONFIG_EFI
+ .align 3
+pe_header:
+ .ascii "PE"
+ .short 0
+coff_header:
+ .short 0xaa64 // AArch64
+ .short 2 // nr_sections
+ .long 0 // TimeDateStamp
+ .long 0 // PointerToSymbolTable
+ .long 1 // NumberOfSymbols
+ .short section_table - optional_header // SizeOfOptionalHeader
+ .short 0x206 // Characteristics.
+ // IMAGE_FILE_DEBUG_STRIPPED |
+ // IMAGE_FILE_EXECUTABLE_IMAGE |
+ // IMAGE_FILE_LINE_NUMS_STRIPPED
+optional_header:
+ .short 0x20b // PE32+ format
+ .byte 0x02 // MajorLinkerVersion
+ .byte 0x14 // MinorLinkerVersion
+ .long _edata - stext // SizeOfCode
+ .long 0 // SizeOfInitializedData
+ .long 0 // SizeOfUninitializedData
+ .long efi_stub_entry - efi_head // AddressOfEntryPoint
+ .long stext - efi_head // BaseOfCode
+
+extra_header_fields:
+ .quad 0 // ImageBase
+ .long 0x20 // SectionAlignment
+ .long 0x8 // FileAlignment
+ .short 0 // MajorOperatingSystemVersion
+ .short 0 // MinorOperatingSystemVersion
+ .short 0 // MajorImageVersion
+ .short 0 // MinorImageVersion
+ .short 0 // MajorSubsystemVersion
+ .short 0 // MinorSubsystemVersion
+ .long 0 // Win32VersionValue
+
+ .long _edata - efi_head // SizeOfImage
+
+ // Everything before the kernel image is considered part of the header
+ .long stext - efi_head // SizeOfHeaders
+ .long 0 // CheckSum
+ .short 0xa // Subsystem (EFI application)
+ .short 0 // DllCharacteristics
+ .quad 0 // SizeOfStackReserve
+ .quad 0 // SizeOfStackCommit
+ .quad 0 // SizeOfHeapReserve
+ .quad 0 // SizeOfHeapCommit
+ .long 0 // LoaderFlags
+ .long 0x6 // NumberOfRvaAndSizes
+
+ .quad 0 // ExportTable
+ .quad 0 // ImportTable
+ .quad 0 // ResourceTable
+ .quad 0 // ExceptionTable
+ .quad 0 // CertificationTable
+ .quad 0 // BaseRelocationTable
+
+ // Section table
+section_table:
+
+ /*
+ * The EFI application loader requires a relocation section
+ * because EFI applications must be relocatable. This is a
+ * dummy section as far as we are concerned.
+ */
+ .ascii ".reloc"
+ .byte 0
+ .byte 0 // end of 0 padding of section name
+ .long 0
+ .long 0
+ .long 0 // SizeOfRawData
+ .long 0 // PointerToRawData
+ .long 0 // PointerToRelocations
+ .long 0 // PointerToLineNumbers
+ .short 0 // NumberOfRelocations
+ .short 0 // NumberOfLineNumbers
+ .long 0x42100040 // Characteristics (section flags)
+
+
+ .ascii ".text"
+ .byte 0
+ .byte 0
+ .byte 0 // end of 0 padding of section name
+ .long _edata - stext // VirtualSize
+ .long stext - efi_head // VirtualAddress
+ .long _edata - stext // SizeOfRawData
+ .long stext - efi_head // PointerToRawData
+
+ .long 0 // PointerToRelocations (0 for executables)
+ .long 0 // PointerToLineNumbers (0 for executables)
+ .short 0 // NumberOfRelocations (0 for executables)
+ .short 0 // NumberOfLineNumbers (0 for executables)
+ .long 0xe0500020 // Characteristics (section flags)
+ .align 5
+#endif
ENTRY(stext)
mov x21, x0 // x21=FDT
@@ -230,11 +342,9 @@
cmp w20, #BOOT_CPU_MODE_EL2
b.ne 1f
add x1, x1, #4
-1: dc cvac, x1 // Clean potentially dirty cache line
- dsb sy
- str w20, [x1] // This CPU has booted in EL1
- dc civac, x1 // Clean&invalidate potentially stale cache line
- dsb sy
+1: str w20, [x1] // This CPU has booted in EL1
+ dmb sy
+ dc ivac, x1 // Invalidate potentially stale cache line
ret
ENDPROC(set_cpu_boot_mode_flag)
diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c
index bee7897..df1cf15 100644
--- a/arch/arm64/kernel/hw_breakpoint.c
+++ b/arch/arm64/kernel/hw_breakpoint.c
@@ -20,6 +20,7 @@
#define pr_fmt(fmt) "hw-breakpoint: " fmt
+#include <linux/compat.h>
#include <linux/cpu_pm.h>
#include <linux/errno.h>
#include <linux/hw_breakpoint.h>
@@ -27,7 +28,6 @@
#include <linux/ptrace.h>
#include <linux/smp.h>
-#include <asm/compat.h>
#include <asm/current.h>
#include <asm/debug-monitors.h>
#include <asm/hw_breakpoint.h>
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 6391485..43b7c34 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -20,6 +20,7 @@
#include <stdarg.h>
+#include <linux/compat.h>
#include <linux/export.h>
#include <linux/sched.h>
#include <linux/kernel.h>
@@ -113,32 +114,62 @@
}
#endif
+/*
+ * Called by kexec, immediately prior to machine_kexec().
+ *
+ * This must completely disable all secondary CPUs; simply causing those CPUs
+ * to execute e.g. a RAM-based pin loop is not sufficient. This allows the
+ * kexec'd kernel to use any and all RAM as it sees fit, without having to
+ * avoid any code or data used by any SW CPU pin loop. The CPU hotplug
+ * functionality embodied in disable_nonboot_cpus() to achieve this.
+ */
void machine_shutdown(void)
{
-#ifdef CONFIG_SMP
- smp_send_stop();
-#endif
+ disable_nonboot_cpus();
}
+/*
+ * Halting simply requires that the secondary CPUs stop performing any
+ * activity (executing tasks, handling interrupts). smp_send_stop()
+ * achieves this.
+ */
void machine_halt(void)
{
- machine_shutdown();
+ local_irq_disable();
+ smp_send_stop();
while (1);
}
+/*
+ * Power-off simply requires that the secondary CPUs stop performing any
+ * activity (executing tasks, handling interrupts). smp_send_stop()
+ * achieves this. When the system power is turned off, it will take all CPUs
+ * with it.
+ */
void machine_power_off(void)
{
- machine_shutdown();
+ local_irq_disable();
+ smp_send_stop();
if (pm_power_off)
pm_power_off();
}
+/*
+ * Restart requires that the secondary CPUs stop performing any activity
+ * while the primary CPU resets the system. Systems with a single CPU can
+ * use soft_restart() as their machine descriptor's .restart hook, since that
+ * will cause the only available CPU to reset. Systems with multiple CPUs must
+ * provide a HW restart implementation, to ensure that all CPUs reset at once.
+ * This is required so that any code running after reset on the primary CPU
+ * doesn't have to co-ordinate with other CPUs to ensure they aren't still
+ * executing pre-reset code, and using RAM that the primary CPU's code wishes
+ * to use. Implementing such co-ordination would be essentially impossible.
+ */
void machine_restart(char *cmd)
{
- machine_shutdown();
-
/* Disable interrupts first */
local_irq_disable();
+ smp_send_stop();
/* Now call the architecture specific reboot code. */
if (arm_pm_restart)
@@ -205,7 +236,7 @@
int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
{
- fpsimd_save_state(¤t->thread.fpsimd_state);
+ fpsimd_preserve_current_state();
*dst = *src;
return 0;
}
@@ -300,7 +331,7 @@
* Complete any pending TLB or cache maintenance on this CPU in case
* the thread migrates to a different CPU.
*/
- dsb();
+ dsb(ish);
/* the actual thread switch */
last = cpu_switch_to(prev, next);
diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c
index 6a8928b..3e926b9 100644
--- a/arch/arm64/kernel/ptrace.c
+++ b/arch/arm64/kernel/ptrace.c
@@ -19,6 +19,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <linux/compat.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/mm.h>
@@ -41,6 +42,9 @@
#include <asm/traps.h>
#include <asm/system_misc.h>
+#define CREATE_TRACE_POINTS
+#include <trace/events/syscalls.h>
+
/*
* TODO: does not yet catch signals sent when the child dies.
* in exit.c or in signal.c.
@@ -517,6 +521,7 @@
return ret;
target->thread.fpsimd_state.user_fpsimd = newstate;
+ fpsimd_flush_task_state(target);
return ret;
}
@@ -764,6 +769,7 @@
uregs->fpcr = fpscr & VFP_FPSCR_CTRL_MASK;
}
+ fpsimd_flush_task_state(target);
return ret;
}
@@ -1058,35 +1064,49 @@
return ptrace_request(child, request, addr, data);
}
-asmlinkage int syscall_trace(int dir, struct pt_regs *regs)
+enum ptrace_syscall_dir {
+ PTRACE_SYSCALL_ENTER = 0,
+ PTRACE_SYSCALL_EXIT,
+};
+
+static void tracehook_report_syscall(struct pt_regs *regs,
+ enum ptrace_syscall_dir dir)
{
+ int regno;
unsigned long saved_reg;
- if (!test_thread_flag(TIF_SYSCALL_TRACE))
- return regs->syscallno;
+ /*
+ * A scratch register (ip(r12) on AArch32, x7 on AArch64) is
+ * used to denote syscall entry/exit:
+ */
+ regno = (is_compat_task() ? 12 : 7);
+ saved_reg = regs->regs[regno];
+ regs->regs[regno] = dir;
- if (is_compat_task()) {
- /* AArch32 uses ip (r12) for scratch */
- saved_reg = regs->regs[12];
- regs->regs[12] = dir;
- } else {
- /*
- * Save X7. X7 is used to denote syscall entry/exit:
- * X7 = 0 -> entry, = 1 -> exit
- */
- saved_reg = regs->regs[7];
- regs->regs[7] = dir;
- }
-
- if (dir)
+ if (dir == PTRACE_SYSCALL_EXIT)
tracehook_report_syscall_exit(regs, 0);
else if (tracehook_report_syscall_entry(regs))
regs->syscallno = ~0UL;
- if (is_compat_task())
- regs->regs[12] = saved_reg;
- else
- regs->regs[7] = saved_reg;
+ regs->regs[regno] = saved_reg;
+}
+
+asmlinkage int syscall_trace_enter(struct pt_regs *regs)
+{
+ if (test_thread_flag(TIF_SYSCALL_TRACE))
+ tracehook_report_syscall(regs, PTRACE_SYSCALL_ENTER);
+
+ if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))
+ trace_sys_enter(regs, regs->syscallno);
return regs->syscallno;
}
+
+asmlinkage void syscall_trace_exit(struct pt_regs *regs)
+{
+ if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))
+ trace_sys_exit(regs, regs_return_value(regs));
+
+ if (test_thread_flag(TIF_SYSCALL_TRACE))
+ tracehook_report_syscall(regs, PTRACE_SYSCALL_EXIT);
+}
diff --git a/arch/arm64/kernel/return_address.c b/arch/arm64/kernel/return_address.c
new file mode 100644
index 0000000..89102a6
--- /dev/null
+++ b/arch/arm64/kernel/return_address.c
@@ -0,0 +1,55 @@
+/*
+ * arch/arm64/kernel/return_address.c
+ *
+ * Copyright (C) 2013 Linaro Limited
+ * Author: AKASHI Takahiro <takahiro.akashi@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/export.h>
+#include <linux/ftrace.h>
+
+#include <asm/stacktrace.h>
+
+struct return_address_data {
+ unsigned int level;
+ void *addr;
+};
+
+static int save_return_addr(struct stackframe *frame, void *d)
+{
+ struct return_address_data *data = d;
+
+ if (!data->level) {
+ data->addr = (void *)frame->pc;
+ return 1;
+ } else {
+ --data->level;
+ return 0;
+ }
+}
+
+void *return_address(unsigned int level)
+{
+ struct return_address_data data;
+ struct stackframe frame;
+ register unsigned long current_sp asm ("sp");
+
+ data.level = level + 2;
+ data.addr = NULL;
+
+ frame.fp = (unsigned long)__builtin_frame_address(0);
+ frame.sp = current_sp;
+ frame.pc = (unsigned long)return_address; /* dummy */
+
+ walk_stackframe(&frame, save_return_addr, &data);
+
+ if (!data.level)
+ return data.addr;
+ else
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(return_address);
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index 7ec7846..46d1125 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -25,6 +25,7 @@
#include <linux/utsname.h>
#include <linux/initrd.h>
#include <linux/console.h>
+#include <linux/cache.h>
#include <linux/bootmem.h>
#include <linux/seq_file.h>
#include <linux/screen_info.h>
@@ -41,6 +42,7 @@
#include <linux/memblock.h>
#include <linux/of_fdt.h>
#include <linux/of_platform.h>
+#include <linux/efi.h>
#include <asm/fixmap.h>
#include <asm/cputype.h>
@@ -55,6 +57,7 @@
#include <asm/traps.h>
#include <asm/memblock.h>
#include <asm/psci.h>
+#include <asm/efi.h>
unsigned int processor_id;
EXPORT_SYMBOL(processor_id);
@@ -198,6 +201,8 @@
{
struct cpu_info *cpu_info;
u64 features, block;
+ u32 cwg;
+ int cls;
cpu_info = lookup_processor_type(read_cpuid_id());
if (!cpu_info) {
@@ -215,6 +220,18 @@
elf_hwcap = 0;
/*
+ * Check for sane CTR_EL0.CWG value.
+ */
+ cwg = cache_type_cwg();
+ cls = cache_line_size();
+ if (!cwg)
+ pr_warn("No Cache Writeback Granule information, assuming cache line size %d\n",
+ cls);
+ if (L1_CACHE_BYTES < cls)
+ pr_warn("L1_CACHE_BYTES smaller than the Cache Writeback Granule (%d < %d)\n",
+ L1_CACHE_BYTES, cls);
+
+ /*
* ID_AA64ISAR0_EL1 contains 4-bit wide signed feature blocks.
* The blocks we test below represent incremental functionality
* for non-negative values. Negative values are reserved.
@@ -361,16 +378,18 @@
*cmdline_p = boot_command_line;
- init_mem_pgprot();
early_ioremap_init();
parse_early_param();
+ efi_init();
arm64_memblock_init();
paging_init();
request_standard_resources();
+ efi_idmap_init();
+
unflatten_device_tree();
psci_init();
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
index 890a591..6357b9c 100644
--- a/arch/arm64/kernel/signal.c
+++ b/arch/arm64/kernel/signal.c
@@ -17,6 +17,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <linux/compat.h>
#include <linux/errno.h>
#include <linux/signal.h>
#include <linux/personality.h>
@@ -25,7 +26,6 @@
#include <linux/tracehook.h>
#include <linux/ratelimit.h>
-#include <asm/compat.h>
#include <asm/debug-monitors.h>
#include <asm/elf.h>
#include <asm/cacheflush.h>
@@ -51,7 +51,7 @@
int err;
/* dump the hardware registers to the fpsimd_state structure */
- fpsimd_save_state(fpsimd);
+ fpsimd_preserve_current_state();
/* copy the FP and status/control registers */
err = __copy_to_user(ctx->vregs, fpsimd->vregs, sizeof(fpsimd->vregs));
@@ -86,11 +86,8 @@
__get_user_error(fpsimd.fpcr, &ctx->fpcr, err);
/* load the hardware registers from the fpsimd_state structure */
- if (!err) {
- preempt_disable();
- fpsimd_load_state(&fpsimd);
- preempt_enable();
- }
+ if (!err)
+ fpsimd_update_current_state(&fpsimd);
return err ? -EFAULT : 0;
}
@@ -100,8 +97,7 @@
{
sigset_t set;
int i, err;
- struct aux_context __user *aux =
- (struct aux_context __user *)sf->uc.uc_mcontext.__reserved;
+ void *aux = sf->uc.uc_mcontext.__reserved;
err = __copy_from_user(&set, &sf->uc.uc_sigmask, sizeof(set));
if (err == 0)
@@ -121,8 +117,11 @@
err |= !valid_user_regs(®s->user_regs);
- if (err == 0)
- err |= restore_fpsimd_context(&aux->fpsimd);
+ if (err == 0) {
+ struct fpsimd_context *fpsimd_ctx =
+ container_of(aux, struct fpsimd_context, head);
+ err |= restore_fpsimd_context(fpsimd_ctx);
+ }
return err;
}
@@ -167,8 +166,8 @@
struct pt_regs *regs, sigset_t *set)
{
int i, err = 0;
- struct aux_context __user *aux =
- (struct aux_context __user *)sf->uc.uc_mcontext.__reserved;
+ void *aux = sf->uc.uc_mcontext.__reserved;
+ struct _aarch64_ctx *end;
/* set up the stack frame for unwinding */
__put_user_error(regs->regs[29], &sf->fp, err);
@@ -185,12 +184,27 @@
err |= __copy_to_user(&sf->uc.uc_sigmask, set, sizeof(*set));
- if (err == 0)
- err |= preserve_fpsimd_context(&aux->fpsimd);
+ if (err == 0) {
+ struct fpsimd_context *fpsimd_ctx =
+ container_of(aux, struct fpsimd_context, head);
+ err |= preserve_fpsimd_context(fpsimd_ctx);
+ aux += sizeof(*fpsimd_ctx);
+ }
+
+ /* fault information, if valid */
+ if (current->thread.fault_code) {
+ struct esr_context *esr_ctx =
+ container_of(aux, struct esr_context, head);
+ __put_user_error(ESR_MAGIC, &esr_ctx->head.magic, err);
+ __put_user_error(sizeof(*esr_ctx), &esr_ctx->head.size, err);
+ __put_user_error(current->thread.fault_code, &esr_ctx->esr, err);
+ aux += sizeof(*esr_ctx);
+ }
/* set the "end" magic */
- __put_user_error(0, &aux->end.magic, err);
- __put_user_error(0, &aux->end.size, err);
+ end = aux;
+ __put_user_error(0, &end->magic, err);
+ __put_user_error(0, &end->size, err);
return err;
}
@@ -416,4 +430,8 @@
clear_thread_flag(TIF_NOTIFY_RESUME);
tracehook_notify_resume(regs);
}
+
+ if (thread_flags & _TIF_FOREIGN_FPSTATE)
+ fpsimd_restore_current_state();
+
}
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index b3fc9f5..3491c63 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -23,6 +23,7 @@
#include <linux/syscalls.h>
#include <linux/ratelimit.h>
+#include <asm/esr.h>
#include <asm/fpsimd.h>
#include <asm/signal32.h>
#include <asm/uaccess.h>
@@ -81,6 +82,8 @@
#define VFP_MAGIC 0x56465001
#define VFP_STORAGE_SIZE sizeof(struct compat_vfp_sigframe)
+#define FSR_WRITE_SHIFT (11)
+
struct compat_aux_sigframe {
struct compat_vfp_sigframe vfp;
@@ -219,7 +222,7 @@
* Note that this also saves V16-31, which aren't visible
* in AArch32.
*/
- fpsimd_save_state(fpsimd);
+ fpsimd_preserve_current_state();
/* Place structure header on the stack */
__put_user_error(magic, &frame->magic, err);
@@ -282,11 +285,8 @@
* We don't need to touch the exception register, so
* reload the hardware state.
*/
- if (!err) {
- preempt_disable();
- fpsimd_load_state(&fpsimd);
- preempt_enable();
- }
+ if (!err)
+ fpsimd_update_current_state(&fpsimd);
return err ? -EFAULT : 0;
}
@@ -500,7 +500,9 @@
__put_user_error(regs->pstate, &sf->uc.uc_mcontext.arm_cpsr, err);
__put_user_error((compat_ulong_t)0, &sf->uc.uc_mcontext.trap_no, err);
- __put_user_error((compat_ulong_t)0, &sf->uc.uc_mcontext.error_code, err);
+ /* set the compat FSR WnR */
+ __put_user_error(!!(current->thread.fault_code & ESR_EL1_WRITE) <<
+ FSR_WRITE_SHIFT, &sf->uc.uc_mcontext.error_code, err);
__put_user_error(current->thread.fault_address, &sf->uc.uc_mcontext.fault_address, err);
__put_user_error(set->sig[0], &sf->uc.uc_mcontext.oldmask, err);
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index c3cb160..40f38f4 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -35,6 +35,7 @@
#include <linux/clockchips.h>
#include <linux/completion.h>
#include <linux/of.h>
+#include <linux/irq_work.h>
#include <asm/atomic.h>
#include <asm/cacheflush.h>
@@ -62,6 +63,7 @@
IPI_CALL_FUNC_SINGLE,
IPI_CPU_STOP,
IPI_TIMER,
+ IPI_IRQ_WORK,
};
/*
@@ -477,6 +479,14 @@
smp_cross_call(cpumask_of(cpu), IPI_CALL_FUNC_SINGLE);
}
+#ifdef CONFIG_IRQ_WORK
+void arch_irq_work_raise(void)
+{
+ if (smp_cross_call)
+ smp_cross_call(cpumask_of(smp_processor_id()), IPI_IRQ_WORK);
+}
+#endif
+
static const char *ipi_types[NR_IPI] = {
#define S(x,s) [x - IPI_RESCHEDULE] = s
S(IPI_RESCHEDULE, "Rescheduling interrupts"),
@@ -484,6 +494,7 @@
S(IPI_CALL_FUNC_SINGLE, "Single function call interrupts"),
S(IPI_CPU_STOP, "CPU stop interrupts"),
S(IPI_TIMER, "Timer broadcast interrupts"),
+ S(IPI_IRQ_WORK, "IRQ work interrupts"),
};
void show_ipi_list(struct seq_file *p, int prec)
@@ -576,6 +587,14 @@
break;
#endif
+#ifdef CONFIG_IRQ_WORK
+ case IPI_IRQ_WORK:
+ irq_enter();
+ irq_work_run();
+ irq_exit();
+ break;
+#endif
+
default:
pr_crit("CPU%u: Unknown IPI message 0x%x\n", cpu, ipinr);
break;
diff --git a/arch/arm64/kernel/smp_spin_table.c b/arch/arm64/kernel/smp_spin_table.c
index 7a530d2..0347d38 100644
--- a/arch/arm64/kernel/smp_spin_table.c
+++ b/arch/arm64/kernel/smp_spin_table.c
@@ -30,7 +30,6 @@
volatile unsigned long secondary_holding_pen_release = INVALID_HWID;
static phys_addr_t cpu_release_addr[NR_CPUS];
-static DEFINE_RAW_SPINLOCK(boot_lock);
/*
* Write secondary_holding_pen_release in a way that is guaranteed to be
@@ -94,14 +93,6 @@
static int smp_spin_table_cpu_boot(unsigned int cpu)
{
- unsigned long timeout;
-
- /*
- * Set synchronisation state between this boot processor
- * and the secondary one
- */
- raw_spin_lock(&boot_lock);
-
/*
* Update the pen release flag.
*/
@@ -112,34 +103,7 @@
*/
sev();
- timeout = jiffies + (1 * HZ);
- while (time_before(jiffies, timeout)) {
- if (secondary_holding_pen_release == INVALID_HWID)
- break;
- udelay(10);
- }
-
- /*
- * Now the secondary core is starting up let it run its
- * calibrations, then wait for it to finish
- */
- raw_spin_unlock(&boot_lock);
-
- return secondary_holding_pen_release != INVALID_HWID ? -ENOSYS : 0;
-}
-
-static void smp_spin_table_cpu_postboot(void)
-{
- /*
- * Let the primary processor know we're out of the pen.
- */
- write_pen_release(INVALID_HWID);
-
- /*
- * Synchronise with the boot thread.
- */
- raw_spin_lock(&boot_lock);
- raw_spin_unlock(&boot_lock);
+ return 0;
}
const struct cpu_operations smp_spin_table_ops = {
@@ -147,5 +111,4 @@
.cpu_init = smp_spin_table_cpu_init,
.cpu_prepare = smp_spin_table_cpu_prepare,
.cpu_boot = smp_spin_table_cpu_boot,
- .cpu_postboot = smp_spin_table_cpu_postboot,
};
diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c
index 38f0558..55437ba 100644
--- a/arch/arm64/kernel/stacktrace.c
+++ b/arch/arm64/kernel/stacktrace.c
@@ -35,7 +35,7 @@
* ldp x29, x30, [sp]
* add sp, sp, #0x10
*/
-int unwind_frame(struct stackframe *frame)
+int notrace unwind_frame(struct stackframe *frame)
{
unsigned long high, low;
unsigned long fp = frame->fp;
diff --git a/arch/arm64/kernel/time.c b/arch/arm64/kernel/time.c
index 6815987..1a7125c 100644
--- a/arch/arm64/kernel/time.c
+++ b/arch/arm64/kernel/time.c
@@ -18,6 +18,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <linux/clockchips.h>
#include <linux/export.h>
#include <linux/kernel.h>
#include <linux/interrupt.h>
@@ -69,6 +70,8 @@
of_clk_init(NULL);
clocksource_of_init();
+ tick_setup_hrtimer_broadcast();
+
arch_timer_rate = arch_timer_get_rate();
if (!arch_timer_rate)
panic("Unable to initialise architected timer.\n");
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index 3e06b0b..43514f9 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -17,10 +17,192 @@
#include <linux/percpu.h>
#include <linux/node.h>
#include <linux/nodemask.h>
+#include <linux/of.h>
#include <linux/sched.h>
#include <asm/topology.h>
+static int __init get_cpu_for_node(struct device_node *node)
+{
+ struct device_node *cpu_node;
+ int cpu;
+
+ cpu_node = of_parse_phandle(node, "cpu", 0);
+ if (!cpu_node)
+ return -1;
+
+ for_each_possible_cpu(cpu) {
+ if (of_get_cpu_node(cpu, NULL) == cpu_node) {
+ of_node_put(cpu_node);
+ return cpu;
+ }
+ }
+
+ pr_crit("Unable to find CPU node for %s\n", cpu_node->full_name);
+
+ of_node_put(cpu_node);
+ return -1;
+}
+
+static int __init parse_core(struct device_node *core, int cluster_id,
+ int core_id)
+{
+ char name[10];
+ bool leaf = true;
+ int i = 0;
+ int cpu;
+ struct device_node *t;
+
+ do {
+ snprintf(name, sizeof(name), "thread%d", i);
+ t = of_get_child_by_name(core, name);
+ if (t) {
+ leaf = false;
+ cpu = get_cpu_for_node(t);
+ if (cpu >= 0) {
+ cpu_topology[cpu].cluster_id = cluster_id;
+ cpu_topology[cpu].core_id = core_id;
+ cpu_topology[cpu].thread_id = i;
+ } else {
+ pr_err("%s: Can't get CPU for thread\n",
+ t->full_name);
+ of_node_put(t);
+ return -EINVAL;
+ }
+ of_node_put(t);
+ }
+ i++;
+ } while (t);
+
+ cpu = get_cpu_for_node(core);
+ if (cpu >= 0) {
+ if (!leaf) {
+ pr_err("%s: Core has both threads and CPU\n",
+ core->full_name);
+ return -EINVAL;
+ }
+
+ cpu_topology[cpu].cluster_id = cluster_id;
+ cpu_topology[cpu].core_id = core_id;
+ } else if (leaf) {
+ pr_err("%s: Can't get CPU for leaf core\n", core->full_name);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int __init parse_cluster(struct device_node *cluster, int depth)
+{
+ char name[10];
+ bool leaf = true;
+ bool has_cores = false;
+ struct device_node *c;
+ static int cluster_id __initdata;
+ int core_id = 0;
+ int i, ret;
+
+ /*
+ * First check for child clusters; we currently ignore any
+ * information about the nesting of clusters and present the
+ * scheduler with a flat list of them.
+ */
+ i = 0;
+ do {
+ snprintf(name, sizeof(name), "cluster%d", i);
+ c = of_get_child_by_name(cluster, name);
+ if (c) {
+ leaf = false;
+ ret = parse_cluster(c, depth + 1);
+ of_node_put(c);
+ if (ret != 0)
+ return ret;
+ }
+ i++;
+ } while (c);
+
+ /* Now check for cores */
+ i = 0;
+ do {
+ snprintf(name, sizeof(name), "core%d", i);
+ c = of_get_child_by_name(cluster, name);
+ if (c) {
+ has_cores = true;
+
+ if (depth == 0) {
+ pr_err("%s: cpu-map children should be clusters\n",
+ c->full_name);
+ of_node_put(c);
+ return -EINVAL;
+ }
+
+ if (leaf) {
+ ret = parse_core(c, cluster_id, core_id++);
+ } else {
+ pr_err("%s: Non-leaf cluster with core %s\n",
+ cluster->full_name, name);
+ ret = -EINVAL;
+ }
+
+ of_node_put(c);
+ if (ret != 0)
+ return ret;
+ }
+ i++;
+ } while (c);
+
+ if (leaf && !has_cores)
+ pr_warn("%s: empty cluster\n", cluster->full_name);
+
+ if (leaf)
+ cluster_id++;
+
+ return 0;
+}
+
+static int __init parse_dt_topology(void)
+{
+ struct device_node *cn, *map;
+ int ret = 0;
+ int cpu;
+
+ cn = of_find_node_by_path("/cpus");
+ if (!cn) {
+ pr_err("No CPU information found in DT\n");
+ return 0;
+ }
+
+ /*
+ * When topology is provided cpu-map is essentially a root
+ * cluster with restricted subnodes.
+ */
+ map = of_get_child_by_name(cn, "cpu-map");
+ if (!map)
+ goto out;
+
+ ret = parse_cluster(map, 0);
+ if (ret != 0)
+ goto out_map;
+
+ /*
+ * Check that all cores are in the topology; the SMP code will
+ * only mark cores described in the DT as possible.
+ */
+ for_each_possible_cpu(cpu) {
+ if (cpu_topology[cpu].cluster_id == -1) {
+ pr_err("CPU%d: No topology information specified\n",
+ cpu);
+ ret = -EINVAL;
+ }
+ }
+
+out_map:
+ of_node_put(map);
+out:
+ of_node_put(cn);
+ return ret;
+}
+
/*
* cpu topology table
*/
@@ -39,13 +221,9 @@
if (cpuid_topo->cluster_id == -1) {
/*
- * DT does not contain topology information for this cpu
- * reset it to default behaviour
+ * DT does not contain topology information for this cpu.
*/
pr_debug("CPU%u: No topology information configured\n", cpuid);
- cpuid_topo->core_id = 0;
- cpumask_set_cpu(cpuid, &cpuid_topo->core_sibling);
- cpumask_set_cpu(cpuid, &cpuid_topo->thread_sibling);
return;
}
@@ -74,22 +252,32 @@
update_siblings_masks(cpuid);
}
-/*
- * init_cpu_topology is called at boot when only one cpu is running
- * which prevent simultaneous write access to cpu_topology array
- */
-void __init init_cpu_topology(void)
+static void __init reset_cpu_topology(void)
{
unsigned int cpu;
- /* init core mask and power*/
for_each_possible_cpu(cpu) {
struct cpu_topology *cpu_topo = &cpu_topology[cpu];
cpu_topo->thread_id = -1;
- cpu_topo->core_id = -1;
+ cpu_topo->core_id = 0;
cpu_topo->cluster_id = -1;
+
cpumask_clear(&cpu_topo->core_sibling);
+ cpumask_set_cpu(cpu, &cpu_topo->core_sibling);
cpumask_clear(&cpu_topo->thread_sibling);
+ cpumask_set_cpu(cpu, &cpu_topo->thread_sibling);
}
}
+
+void __init init_cpu_topology(void)
+{
+ reset_cpu_topology();
+
+ /*
+ * Discard anything that was parsed if we hit an error so we
+ * don't use partial information.
+ */
+ if (parse_dt_topology())
+ reset_cpu_topology();
+}
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 7ffaddd..c43cfa9 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -251,10 +251,13 @@
void arm64_notify_die(const char *str, struct pt_regs *regs,
struct siginfo *info, int err)
{
- if (user_mode(regs))
+ if (user_mode(regs)) {
+ current->thread.fault_address = 0;
+ current->thread.fault_code = err;
force_sig_info(info->si_signo, info, current);
- else
+ } else {
die(str, regs, err);
+ }
}
asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index 4ba7a55..f1e6d5c 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -13,7 +13,7 @@
#define ARM_EXIT_DISCARD(x) x
OUTPUT_ARCH(aarch64)
-ENTRY(stext)
+ENTRY(_text)
jiffies = jiffies_64;
diff --git a/arch/arm64/kvm/hyp.S b/arch/arm64/kvm/hyp.S
index 2c56012..b0d1512 100644
--- a/arch/arm64/kvm/hyp.S
+++ b/arch/arm64/kvm/hyp.S
@@ -630,9 +630,15 @@
* whole of Stage-1. Weep...
*/
tlbi ipas2e1is, x1
- dsb sy
+ /*
+ * We have to ensure completion of the invalidation at Stage-2,
+ * since a table walk on another CPU could refill a TLB with a
+ * complete (S1 + S2) walk based on the old Stage-2 mapping if
+ * the Stage-1 invalidation happened first.
+ */
+ dsb ish
tlbi vmalle1is
- dsb sy
+ dsb ish
isb
msr vttbr_el2, xzr
@@ -643,7 +649,7 @@
dsb ishst
tlbi alle1is
ic ialluis
- dsb sy
+ dsb ish
ret
ENDPROC(__kvm_flush_vm_context)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 0324458..c59a1bd 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -71,13 +71,13 @@
static void do_dc_cisw(u32 val)
{
asm volatile("dc cisw, %x0" : : "r" (val));
- dsb();
+ dsb(ish);
}
static void do_dc_csw(u32 val)
{
asm volatile("dc csw, %x0" : : "r" (val));
- dsb();
+ dsb(ish);
}
/* See note at ARM ARM B1.14.4 */
diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile
index 328ce1a9..d98d3e3 100644
--- a/arch/arm64/lib/Makefile
+++ b/arch/arm64/lib/Makefile
@@ -1,4 +1,5 @@
lib-y := bitops.o clear_user.o delay.o copy_from_user.o \
copy_to_user.o copy_in_user.o copy_page.o \
clear_page.o memchr.o memcpy.o memmove.o memset.o \
+ memcmp.o strcmp.o strncmp.o strlen.o strnlen.o \
strchr.o strrchr.o
diff --git a/arch/arm64/lib/memcmp.S b/arch/arm64/lib/memcmp.S
new file mode 100644
index 0000000..6ea0776
--- /dev/null
+++ b/arch/arm64/lib/memcmp.S
@@ -0,0 +1,258 @@
+/*
+ * Copyright (C) 2013 ARM Ltd.
+ * Copyright (C) 2013 Linaro.
+ *
+ * This code is based on glibc cortex strings work originally authored by Linaro
+ * and re-licensed under GPLv2 for the Linux kernel. The original code can
+ * be found @
+ *
+ * http://bazaar.launchpad.net/~linaro-toolchain-dev/cortex-strings/trunk/
+ * files/head:/src/aarch64/
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+/*
+* compare memory areas(when two memory areas' offset are different,
+* alignment handled by the hardware)
+*
+* Parameters:
+* x0 - const memory area 1 pointer
+* x1 - const memory area 2 pointer
+* x2 - the maximal compare byte length
+* Returns:
+* x0 - a compare result, maybe less than, equal to, or greater than ZERO
+*/
+
+/* Parameters and result. */
+src1 .req x0
+src2 .req x1
+limit .req x2
+result .req x0
+
+/* Internal variables. */
+data1 .req x3
+data1w .req w3
+data2 .req x4
+data2w .req w4
+has_nul .req x5
+diff .req x6
+endloop .req x7
+tmp1 .req x8
+tmp2 .req x9
+tmp3 .req x10
+pos .req x11
+limit_wd .req x12
+mask .req x13
+
+ENTRY(memcmp)
+ cbz limit, .Lret0
+ eor tmp1, src1, src2
+ tst tmp1, #7
+ b.ne .Lmisaligned8
+ ands tmp1, src1, #7
+ b.ne .Lmutual_align
+ sub limit_wd, limit, #1 /* limit != 0, so no underflow. */
+ lsr limit_wd, limit_wd, #3 /* Convert to Dwords. */
+ /*
+ * The input source addresses are at alignment boundary.
+ * Directly compare eight bytes each time.
+ */
+.Lloop_aligned:
+ ldr data1, [src1], #8
+ ldr data2, [src2], #8
+.Lstart_realigned:
+ subs limit_wd, limit_wd, #1
+ eor diff, data1, data2 /* Non-zero if differences found. */
+ csinv endloop, diff, xzr, cs /* Last Dword or differences. */
+ cbz endloop, .Lloop_aligned
+
+ /* Not reached the limit, must have found a diff. */
+ tbz limit_wd, #63, .Lnot_limit
+
+ /* Limit % 8 == 0 => the diff is in the last 8 bytes. */
+ ands limit, limit, #7
+ b.eq .Lnot_limit
+ /*
+ * The remained bytes less than 8. It is needed to extract valid data
+ * from last eight bytes of the intended memory range.
+ */
+ lsl limit, limit, #3 /* bytes-> bits. */
+ mov mask, #~0
+CPU_BE( lsr mask, mask, limit )
+CPU_LE( lsl mask, mask, limit )
+ bic data1, data1, mask
+ bic data2, data2, mask
+
+ orr diff, diff, mask
+ b .Lnot_limit
+
+.Lmutual_align:
+ /*
+ * Sources are mutually aligned, but are not currently at an
+ * alignment boundary. Round down the addresses and then mask off
+ * the bytes that precede the start point.
+ */
+ bic src1, src1, #7
+ bic src2, src2, #7
+ ldr data1, [src1], #8
+ ldr data2, [src2], #8
+ /*
+ * We can not add limit with alignment offset(tmp1) here. Since the
+ * addition probably make the limit overflown.
+ */
+ sub limit_wd, limit, #1/*limit != 0, so no underflow.*/
+ and tmp3, limit_wd, #7
+ lsr limit_wd, limit_wd, #3
+ add tmp3, tmp3, tmp1
+ add limit_wd, limit_wd, tmp3, lsr #3
+ add limit, limit, tmp1/* Adjust the limit for the extra. */
+
+ lsl tmp1, tmp1, #3/* Bytes beyond alignment -> bits.*/
+ neg tmp1, tmp1/* Bits to alignment -64. */
+ mov tmp2, #~0
+ /*mask off the non-intended bytes before the start address.*/
+CPU_BE( lsl tmp2, tmp2, tmp1 )/*Big-endian.Early bytes are at MSB*/
+ /* Little-endian. Early bytes are at LSB. */
+CPU_LE( lsr tmp2, tmp2, tmp1 )
+
+ orr data1, data1, tmp2
+ orr data2, data2, tmp2
+ b .Lstart_realigned
+
+ /*src1 and src2 have different alignment offset.*/
+.Lmisaligned8:
+ cmp limit, #8
+ b.lo .Ltiny8proc /*limit < 8: compare byte by byte*/
+
+ and tmp1, src1, #7
+ neg tmp1, tmp1
+ add tmp1, tmp1, #8/*valid length in the first 8 bytes of src1*/
+ and tmp2, src2, #7
+ neg tmp2, tmp2
+ add tmp2, tmp2, #8/*valid length in the first 8 bytes of src2*/
+ subs tmp3, tmp1, tmp2
+ csel pos, tmp1, tmp2, hi /*Choose the maximum.*/
+
+ sub limit, limit, pos
+ /*compare the proceeding bytes in the first 8 byte segment.*/
+.Ltinycmp:
+ ldrb data1w, [src1], #1
+ ldrb data2w, [src2], #1
+ subs pos, pos, #1
+ ccmp data1w, data2w, #0, ne /* NZCV = 0b0000. */
+ b.eq .Ltinycmp
+ cbnz pos, 1f /*diff occurred before the last byte.*/
+ cmp data1w, data2w
+ b.eq .Lstart_align
+1:
+ sub result, data1, data2
+ ret
+
+.Lstart_align:
+ lsr limit_wd, limit, #3
+ cbz limit_wd, .Lremain8
+
+ ands xzr, src1, #7
+ b.eq .Lrecal_offset
+ /*process more leading bytes to make src1 aligned...*/
+ add src1, src1, tmp3 /*backwards src1 to alignment boundary*/
+ add src2, src2, tmp3
+ sub limit, limit, tmp3
+ lsr limit_wd, limit, #3
+ cbz limit_wd, .Lremain8
+ /*load 8 bytes from aligned SRC1..*/
+ ldr data1, [src1], #8
+ ldr data2, [src2], #8
+
+ subs limit_wd, limit_wd, #1
+ eor diff, data1, data2 /*Non-zero if differences found.*/
+ csinv endloop, diff, xzr, ne
+ cbnz endloop, .Lunequal_proc
+ /*How far is the current SRC2 from the alignment boundary...*/
+ and tmp3, tmp3, #7
+
+.Lrecal_offset:/*src1 is aligned now..*/
+ neg pos, tmp3
+.Lloopcmp_proc:
+ /*
+ * Divide the eight bytes into two parts. First,backwards the src2
+ * to an alignment boundary,load eight bytes and compare from
+ * the SRC2 alignment boundary. If all 8 bytes are equal,then start
+ * the second part's comparison. Otherwise finish the comparison.
+ * This special handle can garantee all the accesses are in the
+ * thread/task space in avoid to overrange access.
+ */
+ ldr data1, [src1,pos]
+ ldr data2, [src2,pos]
+ eor diff, data1, data2 /* Non-zero if differences found. */
+ cbnz diff, .Lnot_limit
+
+ /*The second part process*/
+ ldr data1, [src1], #8
+ ldr data2, [src2], #8
+ eor diff, data1, data2 /* Non-zero if differences found. */
+ subs limit_wd, limit_wd, #1
+ csinv endloop, diff, xzr, ne/*if limit_wd is 0,will finish the cmp*/
+ cbz endloop, .Lloopcmp_proc
+.Lunequal_proc:
+ cbz diff, .Lremain8
+
+/*There is differnence occured in the latest comparison.*/
+.Lnot_limit:
+/*
+* For little endian,reverse the low significant equal bits into MSB,then
+* following CLZ can find how many equal bits exist.
+*/
+CPU_LE( rev diff, diff )
+CPU_LE( rev data1, data1 )
+CPU_LE( rev data2, data2 )
+
+ /*
+ * The MS-non-zero bit of DIFF marks either the first bit
+ * that is different, or the end of the significant data.
+ * Shifting left now will bring the critical information into the
+ * top bits.
+ */
+ clz pos, diff
+ lsl data1, data1, pos
+ lsl data2, data2, pos
+ /*
+ * We need to zero-extend (char is unsigned) the value and then
+ * perform a signed subtraction.
+ */
+ lsr data1, data1, #56
+ sub result, data1, data2, lsr #56
+ ret
+
+.Lremain8:
+ /* Limit % 8 == 0 =>. all data are equal.*/
+ ands limit, limit, #7
+ b.eq .Lret0
+
+.Ltiny8proc:
+ ldrb data1w, [src1], #1
+ ldrb data2w, [src2], #1
+ subs limit, limit, #1
+
+ ccmp data1w, data2w, #0, ne /* NZCV = 0b0000. */
+ b.eq .Ltiny8proc
+ sub result, data1, data2
+ ret
+.Lret0:
+ mov result, #0
+ ret
+ENDPROC(memcmp)
diff --git a/arch/arm64/lib/memcpy.S b/arch/arm64/lib/memcpy.S
index 27b5003..8a9a96d 100644
--- a/arch/arm64/lib/memcpy.S
+++ b/arch/arm64/lib/memcpy.S
@@ -1,5 +1,13 @@
/*
* Copyright (C) 2013 ARM Ltd.
+ * Copyright (C) 2013 Linaro.
+ *
+ * This code is based on glibc cortex strings work originally authored by Linaro
+ * and re-licensed under GPLv2 for the Linux kernel. The original code can
+ * be found @
+ *
+ * http://bazaar.launchpad.net/~linaro-toolchain-dev/cortex-strings/trunk/
+ * files/head:/src/aarch64/
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
@@ -16,6 +24,7 @@
#include <linux/linkage.h>
#include <asm/assembler.h>
+#include <asm/cache.h>
/*
* Copy a buffer from src to dest (alignment handled by the hardware)
@@ -27,27 +36,166 @@
* Returns:
* x0 - dest
*/
+dstin .req x0
+src .req x1
+count .req x2
+tmp1 .req x3
+tmp1w .req w3
+tmp2 .req x4
+tmp2w .req w4
+tmp3 .req x5
+tmp3w .req w5
+dst .req x6
+
+A_l .req x7
+A_h .req x8
+B_l .req x9
+B_h .req x10
+C_l .req x11
+C_h .req x12
+D_l .req x13
+D_h .req x14
+
ENTRY(memcpy)
- mov x4, x0
- subs x2, x2, #8
- b.mi 2f
-1: ldr x3, [x1], #8
- subs x2, x2, #8
- str x3, [x4], #8
- b.pl 1b
-2: adds x2, x2, #4
- b.mi 3f
- ldr w3, [x1], #4
- sub x2, x2, #4
- str w3, [x4], #4
-3: adds x2, x2, #2
- b.mi 4f
- ldrh w3, [x1], #2
- sub x2, x2, #2
- strh w3, [x4], #2
-4: adds x2, x2, #1
- b.mi 5f
- ldrb w3, [x1]
- strb w3, [x4]
-5: ret
+ mov dst, dstin
+ cmp count, #16
+ /*When memory length is less than 16, the accessed are not aligned.*/
+ b.lo .Ltiny15
+
+ neg tmp2, src
+ ands tmp2, tmp2, #15/* Bytes to reach alignment. */
+ b.eq .LSrcAligned
+ sub count, count, tmp2
+ /*
+ * Copy the leading memory data from src to dst in an increasing
+ * address order.By this way,the risk of overwritting the source
+ * memory data is eliminated when the distance between src and
+ * dst is less than 16. The memory accesses here are alignment.
+ */
+ tbz tmp2, #0, 1f
+ ldrb tmp1w, [src], #1
+ strb tmp1w, [dst], #1
+1:
+ tbz tmp2, #1, 2f
+ ldrh tmp1w, [src], #2
+ strh tmp1w, [dst], #2
+2:
+ tbz tmp2, #2, 3f
+ ldr tmp1w, [src], #4
+ str tmp1w, [dst], #4
+3:
+ tbz tmp2, #3, .LSrcAligned
+ ldr tmp1, [src],#8
+ str tmp1, [dst],#8
+
+.LSrcAligned:
+ cmp count, #64
+ b.ge .Lcpy_over64
+ /*
+ * Deal with small copies quickly by dropping straight into the
+ * exit block.
+ */
+.Ltail63:
+ /*
+ * Copy up to 48 bytes of data. At this point we only need the
+ * bottom 6 bits of count to be accurate.
+ */
+ ands tmp1, count, #0x30
+ b.eq .Ltiny15
+ cmp tmp1w, #0x20
+ b.eq 1f
+ b.lt 2f
+ ldp A_l, A_h, [src], #16
+ stp A_l, A_h, [dst], #16
+1:
+ ldp A_l, A_h, [src], #16
+ stp A_l, A_h, [dst], #16
+2:
+ ldp A_l, A_h, [src], #16
+ stp A_l, A_h, [dst], #16
+.Ltiny15:
+ /*
+ * Prefer to break one ldp/stp into several load/store to access
+ * memory in an increasing address order,rather than to load/store 16
+ * bytes from (src-16) to (dst-16) and to backward the src to aligned
+ * address,which way is used in original cortex memcpy. If keeping
+ * the original memcpy process here, memmove need to satisfy the
+ * precondition that src address is at least 16 bytes bigger than dst
+ * address,otherwise some source data will be overwritten when memove
+ * call memcpy directly. To make memmove simpler and decouple the
+ * memcpy's dependency on memmove, withdrew the original process.
+ */
+ tbz count, #3, 1f
+ ldr tmp1, [src], #8
+ str tmp1, [dst], #8
+1:
+ tbz count, #2, 2f
+ ldr tmp1w, [src], #4
+ str tmp1w, [dst], #4
+2:
+ tbz count, #1, 3f
+ ldrh tmp1w, [src], #2
+ strh tmp1w, [dst], #2
+3:
+ tbz count, #0, .Lexitfunc
+ ldrb tmp1w, [src]
+ strb tmp1w, [dst]
+
+.Lexitfunc:
+ ret
+
+.Lcpy_over64:
+ subs count, count, #128
+ b.ge .Lcpy_body_large
+ /*
+ * Less than 128 bytes to copy, so handle 64 here and then jump
+ * to the tail.
+ */
+ ldp A_l, A_h, [src],#16
+ stp A_l, A_h, [dst],#16
+ ldp B_l, B_h, [src],#16
+ ldp C_l, C_h, [src],#16
+ stp B_l, B_h, [dst],#16
+ stp C_l, C_h, [dst],#16
+ ldp D_l, D_h, [src],#16
+ stp D_l, D_h, [dst],#16
+
+ tst count, #0x3f
+ b.ne .Ltail63
+ ret
+
+ /*
+ * Critical loop. Start at a new cache line boundary. Assuming
+ * 64 bytes per line this ensures the entire loop is in one line.
+ */
+ .p2align L1_CACHE_SHIFT
+.Lcpy_body_large:
+ /* pre-get 64 bytes data. */
+ ldp A_l, A_h, [src],#16
+ ldp B_l, B_h, [src],#16
+ ldp C_l, C_h, [src],#16
+ ldp D_l, D_h, [src],#16
+1:
+ /*
+ * interlace the load of next 64 bytes data block with store of the last
+ * loaded 64 bytes data.
+ */
+ stp A_l, A_h, [dst],#16
+ ldp A_l, A_h, [src],#16
+ stp B_l, B_h, [dst],#16
+ ldp B_l, B_h, [src],#16
+ stp C_l, C_h, [dst],#16
+ ldp C_l, C_h, [src],#16
+ stp D_l, D_h, [dst],#16
+ ldp D_l, D_h, [src],#16
+ subs count, count, #64
+ b.ge 1b
+ stp A_l, A_h, [dst],#16
+ stp B_l, B_h, [dst],#16
+ stp C_l, C_h, [dst],#16
+ stp D_l, D_h, [dst],#16
+
+ tst count, #0x3f
+ b.ne .Ltail63
+ ret
ENDPROC(memcpy)
diff --git a/arch/arm64/lib/memmove.S b/arch/arm64/lib/memmove.S
index b79fdfa..57b19ea2 100644
--- a/arch/arm64/lib/memmove.S
+++ b/arch/arm64/lib/memmove.S
@@ -1,5 +1,13 @@
/*
* Copyright (C) 2013 ARM Ltd.
+ * Copyright (C) 2013 Linaro.
+ *
+ * This code is based on glibc cortex strings work originally authored by Linaro
+ * and re-licensed under GPLv2 for the Linux kernel. The original code can
+ * be found @
+ *
+ * http://bazaar.launchpad.net/~linaro-toolchain-dev/cortex-strings/trunk/
+ * files/head:/src/aarch64/
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
@@ -16,6 +24,7 @@
#include <linux/linkage.h>
#include <asm/assembler.h>
+#include <asm/cache.h>
/*
* Move a buffer from src to test (alignment handled by the hardware).
@@ -28,30 +37,161 @@
* Returns:
* x0 - dest
*/
+dstin .req x0
+src .req x1
+count .req x2
+tmp1 .req x3
+tmp1w .req w3
+tmp2 .req x4
+tmp2w .req w4
+tmp3 .req x5
+tmp3w .req w5
+dst .req x6
+
+A_l .req x7
+A_h .req x8
+B_l .req x9
+B_h .req x10
+C_l .req x11
+C_h .req x12
+D_l .req x13
+D_h .req x14
+
ENTRY(memmove)
- cmp x0, x1
- b.ls memcpy
- add x4, x0, x2
- add x1, x1, x2
- subs x2, x2, #8
- b.mi 2f
-1: ldr x3, [x1, #-8]!
- subs x2, x2, #8
- str x3, [x4, #-8]!
- b.pl 1b
-2: adds x2, x2, #4
- b.mi 3f
- ldr w3, [x1, #-4]!
- sub x2, x2, #4
- str w3, [x4, #-4]!
-3: adds x2, x2, #2
- b.mi 4f
- ldrh w3, [x1, #-2]!
- sub x2, x2, #2
- strh w3, [x4, #-2]!
-4: adds x2, x2, #1
- b.mi 5f
- ldrb w3, [x1, #-1]
- strb w3, [x4, #-1]
-5: ret
+ cmp dstin, src
+ b.lo memcpy
+ add tmp1, src, count
+ cmp dstin, tmp1
+ b.hs memcpy /* No overlap. */
+
+ add dst, dstin, count
+ add src, src, count
+ cmp count, #16
+ b.lo .Ltail15 /*probably non-alignment accesses.*/
+
+ ands tmp2, src, #15 /* Bytes to reach alignment. */
+ b.eq .LSrcAligned
+ sub count, count, tmp2
+ /*
+ * process the aligned offset length to make the src aligned firstly.
+ * those extra instructions' cost is acceptable. It also make the
+ * coming accesses are based on aligned address.
+ */
+ tbz tmp2, #0, 1f
+ ldrb tmp1w, [src, #-1]!
+ strb tmp1w, [dst, #-1]!
+1:
+ tbz tmp2, #1, 2f
+ ldrh tmp1w, [src, #-2]!
+ strh tmp1w, [dst, #-2]!
+2:
+ tbz tmp2, #2, 3f
+ ldr tmp1w, [src, #-4]!
+ str tmp1w, [dst, #-4]!
+3:
+ tbz tmp2, #3, .LSrcAligned
+ ldr tmp1, [src, #-8]!
+ str tmp1, [dst, #-8]!
+
+.LSrcAligned:
+ cmp count, #64
+ b.ge .Lcpy_over64
+
+ /*
+ * Deal with small copies quickly by dropping straight into the
+ * exit block.
+ */
+.Ltail63:
+ /*
+ * Copy up to 48 bytes of data. At this point we only need the
+ * bottom 6 bits of count to be accurate.
+ */
+ ands tmp1, count, #0x30
+ b.eq .Ltail15
+ cmp tmp1w, #0x20
+ b.eq 1f
+ b.lt 2f
+ ldp A_l, A_h, [src, #-16]!
+ stp A_l, A_h, [dst, #-16]!
+1:
+ ldp A_l, A_h, [src, #-16]!
+ stp A_l, A_h, [dst, #-16]!
+2:
+ ldp A_l, A_h, [src, #-16]!
+ stp A_l, A_h, [dst, #-16]!
+
+.Ltail15:
+ tbz count, #3, 1f
+ ldr tmp1, [src, #-8]!
+ str tmp1, [dst, #-8]!
+1:
+ tbz count, #2, 2f
+ ldr tmp1w, [src, #-4]!
+ str tmp1w, [dst, #-4]!
+2:
+ tbz count, #1, 3f
+ ldrh tmp1w, [src, #-2]!
+ strh tmp1w, [dst, #-2]!
+3:
+ tbz count, #0, .Lexitfunc
+ ldrb tmp1w, [src, #-1]
+ strb tmp1w, [dst, #-1]
+
+.Lexitfunc:
+ ret
+
+.Lcpy_over64:
+ subs count, count, #128
+ b.ge .Lcpy_body_large
+ /*
+ * Less than 128 bytes to copy, so handle 64 bytes here and then jump
+ * to the tail.
+ */
+ ldp A_l, A_h, [src, #-16]
+ stp A_l, A_h, [dst, #-16]
+ ldp B_l, B_h, [src, #-32]
+ ldp C_l, C_h, [src, #-48]
+ stp B_l, B_h, [dst, #-32]
+ stp C_l, C_h, [dst, #-48]
+ ldp D_l, D_h, [src, #-64]!
+ stp D_l, D_h, [dst, #-64]!
+
+ tst count, #0x3f
+ b.ne .Ltail63
+ ret
+
+ /*
+ * Critical loop. Start at a new cache line boundary. Assuming
+ * 64 bytes per line this ensures the entire loop is in one line.
+ */
+ .p2align L1_CACHE_SHIFT
+.Lcpy_body_large:
+ /* pre-load 64 bytes data. */
+ ldp A_l, A_h, [src, #-16]
+ ldp B_l, B_h, [src, #-32]
+ ldp C_l, C_h, [src, #-48]
+ ldp D_l, D_h, [src, #-64]!
+1:
+ /*
+ * interlace the load of next 64 bytes data block with store of the last
+ * loaded 64 bytes data.
+ */
+ stp A_l, A_h, [dst, #-16]
+ ldp A_l, A_h, [src, #-16]
+ stp B_l, B_h, [dst, #-32]
+ ldp B_l, B_h, [src, #-32]
+ stp C_l, C_h, [dst, #-48]
+ ldp C_l, C_h, [src, #-48]
+ stp D_l, D_h, [dst, #-64]!
+ ldp D_l, D_h, [src, #-64]!
+ subs count, count, #64
+ b.ge 1b
+ stp A_l, A_h, [dst, #-16]
+ stp B_l, B_h, [dst, #-32]
+ stp C_l, C_h, [dst, #-48]
+ stp D_l, D_h, [dst, #-64]!
+
+ tst count, #0x3f
+ b.ne .Ltail63
+ ret
ENDPROC(memmove)
diff --git a/arch/arm64/lib/memset.S b/arch/arm64/lib/memset.S
index 87e4a68..7c72dfd 100644
--- a/arch/arm64/lib/memset.S
+++ b/arch/arm64/lib/memset.S
@@ -1,5 +1,13 @@
/*
* Copyright (C) 2013 ARM Ltd.
+ * Copyright (C) 2013 Linaro.
+ *
+ * This code is based on glibc cortex strings work originally authored by Linaro
+ * and re-licensed under GPLv2 for the Linux kernel. The original code can
+ * be found @
+ *
+ * http://bazaar.launchpad.net/~linaro-toolchain-dev/cortex-strings/trunk/
+ * files/head:/src/aarch64/
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
@@ -16,6 +24,7 @@
#include <linux/linkage.h>
#include <asm/assembler.h>
+#include <asm/cache.h>
/*
* Fill in the buffer with character c (alignment handled by the hardware)
@@ -27,27 +36,181 @@
* Returns:
* x0 - buf
*/
+
+dstin .req x0
+val .req w1
+count .req x2
+tmp1 .req x3
+tmp1w .req w3
+tmp2 .req x4
+tmp2w .req w4
+zva_len_x .req x5
+zva_len .req w5
+zva_bits_x .req x6
+
+A_l .req x7
+A_lw .req w7
+dst .req x8
+tmp3w .req w9
+tmp3 .req x9
+
ENTRY(memset)
- mov x4, x0
- and w1, w1, #0xff
- orr w1, w1, w1, lsl #8
- orr w1, w1, w1, lsl #16
- orr x1, x1, x1, lsl #32
- subs x2, x2, #8
- b.mi 2f
-1: str x1, [x4], #8
- subs x2, x2, #8
- b.pl 1b
-2: adds x2, x2, #4
- b.mi 3f
- sub x2, x2, #4
- str w1, [x4], #4
-3: adds x2, x2, #2
- b.mi 4f
- sub x2, x2, #2
- strh w1, [x4], #2
-4: adds x2, x2, #1
- b.mi 5f
- strb w1, [x4]
-5: ret
+ mov dst, dstin /* Preserve return value. */
+ and A_lw, val, #255
+ orr A_lw, A_lw, A_lw, lsl #8
+ orr A_lw, A_lw, A_lw, lsl #16
+ orr A_l, A_l, A_l, lsl #32
+
+ cmp count, #15
+ b.hi .Lover16_proc
+ /*All store maybe are non-aligned..*/
+ tbz count, #3, 1f
+ str A_l, [dst], #8
+1:
+ tbz count, #2, 2f
+ str A_lw, [dst], #4
+2:
+ tbz count, #1, 3f
+ strh A_lw, [dst], #2
+3:
+ tbz count, #0, 4f
+ strb A_lw, [dst]
+4:
+ ret
+
+.Lover16_proc:
+ /*Whether the start address is aligned with 16.*/
+ neg tmp2, dst
+ ands tmp2, tmp2, #15
+ b.eq .Laligned
+/*
+* The count is not less than 16, we can use stp to store the start 16 bytes,
+* then adjust the dst aligned with 16.This process will make the current
+* memory address at alignment boundary.
+*/
+ stp A_l, A_l, [dst] /*non-aligned store..*/
+ /*make the dst aligned..*/
+ sub count, count, tmp2
+ add dst, dst, tmp2
+
+.Laligned:
+ cbz A_l, .Lzero_mem
+
+.Ltail_maybe_long:
+ cmp count, #64
+ b.ge .Lnot_short
+.Ltail63:
+ ands tmp1, count, #0x30
+ b.eq 3f
+ cmp tmp1w, #0x20
+ b.eq 1f
+ b.lt 2f
+ stp A_l, A_l, [dst], #16
+1:
+ stp A_l, A_l, [dst], #16
+2:
+ stp A_l, A_l, [dst], #16
+/*
+* The last store length is less than 16,use stp to write last 16 bytes.
+* It will lead some bytes written twice and the access is non-aligned.
+*/
+3:
+ ands count, count, #15
+ cbz count, 4f
+ add dst, dst, count
+ stp A_l, A_l, [dst, #-16] /* Repeat some/all of last store. */
+4:
+ ret
+
+ /*
+ * Critical loop. Start at a new cache line boundary. Assuming
+ * 64 bytes per line, this ensures the entire loop is in one line.
+ */
+ .p2align L1_CACHE_SHIFT
+.Lnot_short:
+ sub dst, dst, #16/* Pre-bias. */
+ sub count, count, #64
+1:
+ stp A_l, A_l, [dst, #16]
+ stp A_l, A_l, [dst, #32]
+ stp A_l, A_l, [dst, #48]
+ stp A_l, A_l, [dst, #64]!
+ subs count, count, #64
+ b.ge 1b
+ tst count, #0x3f
+ add dst, dst, #16
+ b.ne .Ltail63
+.Lexitfunc:
+ ret
+
+ /*
+ * For zeroing memory, check to see if we can use the ZVA feature to
+ * zero entire 'cache' lines.
+ */
+.Lzero_mem:
+ cmp count, #63
+ b.le .Ltail63
+ /*
+ * For zeroing small amounts of memory, it's not worth setting up
+ * the line-clear code.
+ */
+ cmp count, #128
+ b.lt .Lnot_short /*count is at least 128 bytes*/
+
+ mrs tmp1, dczid_el0
+ tbnz tmp1, #4, .Lnot_short
+ mov tmp3w, #4
+ and zva_len, tmp1w, #15 /* Safety: other bits reserved. */
+ lsl zva_len, tmp3w, zva_len
+
+ ands tmp3w, zva_len, #63
+ /*
+ * ensure the zva_len is not less than 64.
+ * It is not meaningful to use ZVA if the block size is less than 64.
+ */
+ b.ne .Lnot_short
+.Lzero_by_line:
+ /*
+ * Compute how far we need to go to become suitably aligned. We're
+ * already at quad-word alignment.
+ */
+ cmp count, zva_len_x
+ b.lt .Lnot_short /* Not enough to reach alignment. */
+ sub zva_bits_x, zva_len_x, #1
+ neg tmp2, dst
+ ands tmp2, tmp2, zva_bits_x
+ b.eq 2f /* Already aligned. */
+ /* Not aligned, check that there's enough to copy after alignment.*/
+ sub tmp1, count, tmp2
+ /*
+ * grantee the remain length to be ZVA is bigger than 64,
+ * avoid to make the 2f's process over mem range.*/
+ cmp tmp1, #64
+ ccmp tmp1, zva_len_x, #8, ge /* NZCV=0b1000 */
+ b.lt .Lnot_short
+ /*
+ * We know that there's at least 64 bytes to zero and that it's safe
+ * to overrun by 64 bytes.
+ */
+ mov count, tmp1
+1:
+ stp A_l, A_l, [dst]
+ stp A_l, A_l, [dst, #16]
+ stp A_l, A_l, [dst, #32]
+ subs tmp2, tmp2, #64
+ stp A_l, A_l, [dst, #48]
+ add dst, dst, #64
+ b.ge 1b
+ /* We've overrun a bit, so adjust dst downwards.*/
+ add dst, dst, tmp2
+2:
+ sub count, count, zva_len_x
+3:
+ dc zva, dst
+ add dst, dst, zva_len_x
+ subs count, count, zva_len_x
+ b.ge 3b
+ ands count, count, zva_bits_x
+ b.ne .Ltail_maybe_long
+ ret
ENDPROC(memset)
diff --git a/arch/arm64/lib/strcmp.S b/arch/arm64/lib/strcmp.S
new file mode 100644
index 0000000..42f828b
--- /dev/null
+++ b/arch/arm64/lib/strcmp.S
@@ -0,0 +1,234 @@
+/*
+ * Copyright (C) 2013 ARM Ltd.
+ * Copyright (C) 2013 Linaro.
+ *
+ * This code is based on glibc cortex strings work originally authored by Linaro
+ * and re-licensed under GPLv2 for the Linux kernel. The original code can
+ * be found @
+ *
+ * http://bazaar.launchpad.net/~linaro-toolchain-dev/cortex-strings/trunk/
+ * files/head:/src/aarch64/
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+/*
+ * compare two strings
+ *
+ * Parameters:
+ * x0 - const string 1 pointer
+ * x1 - const string 2 pointer
+ * Returns:
+ * x0 - an integer less than, equal to, or greater than zero
+ * if s1 is found, respectively, to be less than, to match,
+ * or be greater than s2.
+ */
+
+#define REP8_01 0x0101010101010101
+#define REP8_7f 0x7f7f7f7f7f7f7f7f
+#define REP8_80 0x8080808080808080
+
+/* Parameters and result. */
+src1 .req x0
+src2 .req x1
+result .req x0
+
+/* Internal variables. */
+data1 .req x2
+data1w .req w2
+data2 .req x3
+data2w .req w3
+has_nul .req x4
+diff .req x5
+syndrome .req x6
+tmp1 .req x7
+tmp2 .req x8
+tmp3 .req x9
+zeroones .req x10
+pos .req x11
+
+ENTRY(strcmp)
+ eor tmp1, src1, src2
+ mov zeroones, #REP8_01
+ tst tmp1, #7
+ b.ne .Lmisaligned8
+ ands tmp1, src1, #7
+ b.ne .Lmutual_align
+
+ /*
+ * NUL detection works on the principle that (X - 1) & (~X) & 0x80
+ * (=> (X - 1) & ~(X | 0x7f)) is non-zero iff a byte is zero, and
+ * can be done in parallel across the entire word.
+ */
+.Lloop_aligned:
+ ldr data1, [src1], #8
+ ldr data2, [src2], #8
+.Lstart_realigned:
+ sub tmp1, data1, zeroones
+ orr tmp2, data1, #REP8_7f
+ eor diff, data1, data2 /* Non-zero if differences found. */
+ bic has_nul, tmp1, tmp2 /* Non-zero if NUL terminator. */
+ orr syndrome, diff, has_nul
+ cbz syndrome, .Lloop_aligned
+ b .Lcal_cmpresult
+
+.Lmutual_align:
+ /*
+ * Sources are mutually aligned, but are not currently at an
+ * alignment boundary. Round down the addresses and then mask off
+ * the bytes that preceed the start point.
+ */
+ bic src1, src1, #7
+ bic src2, src2, #7
+ lsl tmp1, tmp1, #3 /* Bytes beyond alignment -> bits. */
+ ldr data1, [src1], #8
+ neg tmp1, tmp1 /* Bits to alignment -64. */
+ ldr data2, [src2], #8
+ mov tmp2, #~0
+ /* Big-endian. Early bytes are at MSB. */
+CPU_BE( lsl tmp2, tmp2, tmp1 ) /* Shift (tmp1 & 63). */
+ /* Little-endian. Early bytes are at LSB. */
+CPU_LE( lsr tmp2, tmp2, tmp1 ) /* Shift (tmp1 & 63). */
+
+ orr data1, data1, tmp2
+ orr data2, data2, tmp2
+ b .Lstart_realigned
+
+.Lmisaligned8:
+ /*
+ * Get the align offset length to compare per byte first.
+ * After this process, one string's address will be aligned.
+ */
+ and tmp1, src1, #7
+ neg tmp1, tmp1
+ add tmp1, tmp1, #8
+ and tmp2, src2, #7
+ neg tmp2, tmp2
+ add tmp2, tmp2, #8
+ subs tmp3, tmp1, tmp2
+ csel pos, tmp1, tmp2, hi /*Choose the maximum. */
+.Ltinycmp:
+ ldrb data1w, [src1], #1
+ ldrb data2w, [src2], #1
+ subs pos, pos, #1
+ ccmp data1w, #1, #0, ne /* NZCV = 0b0000. */
+ ccmp data1w, data2w, #0, cs /* NZCV = 0b0000. */
+ b.eq .Ltinycmp
+ cbnz pos, 1f /*find the null or unequal...*/
+ cmp data1w, #1
+ ccmp data1w, data2w, #0, cs
+ b.eq .Lstart_align /*the last bytes are equal....*/
+1:
+ sub result, data1, data2
+ ret
+
+.Lstart_align:
+ ands xzr, src1, #7
+ b.eq .Lrecal_offset
+ /*process more leading bytes to make str1 aligned...*/
+ add src1, src1, tmp3
+ add src2, src2, tmp3
+ /*load 8 bytes from aligned str1 and non-aligned str2..*/
+ ldr data1, [src1], #8
+ ldr data2, [src2], #8
+
+ sub tmp1, data1, zeroones
+ orr tmp2, data1, #REP8_7f
+ bic has_nul, tmp1, tmp2
+ eor diff, data1, data2 /* Non-zero if differences found. */
+ orr syndrome, diff, has_nul
+ cbnz syndrome, .Lcal_cmpresult
+ /*How far is the current str2 from the alignment boundary...*/
+ and tmp3, tmp3, #7
+.Lrecal_offset:
+ neg pos, tmp3
+.Lloopcmp_proc:
+ /*
+ * Divide the eight bytes into two parts. First,backwards the src2
+ * to an alignment boundary,load eight bytes from the SRC2 alignment
+ * boundary,then compare with the relative bytes from SRC1.
+ * If all 8 bytes are equal,then start the second part's comparison.
+ * Otherwise finish the comparison.
+ * This special handle can garantee all the accesses are in the
+ * thread/task space in avoid to overrange access.
+ */
+ ldr data1, [src1,pos]
+ ldr data2, [src2,pos]
+ sub tmp1, data1, zeroones
+ orr tmp2, data1, #REP8_7f
+ bic has_nul, tmp1, tmp2
+ eor diff, data1, data2 /* Non-zero if differences found. */
+ orr syndrome, diff, has_nul
+ cbnz syndrome, .Lcal_cmpresult
+
+ /*The second part process*/
+ ldr data1, [src1], #8
+ ldr data2, [src2], #8
+ sub tmp1, data1, zeroones
+ orr tmp2, data1, #REP8_7f
+ bic has_nul, tmp1, tmp2
+ eor diff, data1, data2 /* Non-zero if differences found. */
+ orr syndrome, diff, has_nul
+ cbz syndrome, .Lloopcmp_proc
+
+.Lcal_cmpresult:
+ /*
+ * reversed the byte-order as big-endian,then CLZ can find the most
+ * significant zero bits.
+ */
+CPU_LE( rev syndrome, syndrome )
+CPU_LE( rev data1, data1 )
+CPU_LE( rev data2, data2 )
+
+ /*
+ * For big-endian we cannot use the trick with the syndrome value
+ * as carry-propagation can corrupt the upper bits if the trailing
+ * bytes in the string contain 0x01.
+ * However, if there is no NUL byte in the dword, we can generate
+ * the result directly. We ca not just subtract the bytes as the
+ * MSB might be significant.
+ */
+CPU_BE( cbnz has_nul, 1f )
+CPU_BE( cmp data1, data2 )
+CPU_BE( cset result, ne )
+CPU_BE( cneg result, result, lo )
+CPU_BE( ret )
+CPU_BE( 1: )
+ /*Re-compute the NUL-byte detection, using a byte-reversed value. */
+CPU_BE( rev tmp3, data1 )
+CPU_BE( sub tmp1, tmp3, zeroones )
+CPU_BE( orr tmp2, tmp3, #REP8_7f )
+CPU_BE( bic has_nul, tmp1, tmp2 )
+CPU_BE( rev has_nul, has_nul )
+CPU_BE( orr syndrome, diff, has_nul )
+
+ clz pos, syndrome
+ /*
+ * The MS-non-zero bit of the syndrome marks either the first bit
+ * that is different, or the top bit of the first zero byte.
+ * Shifting left now will bring the critical information into the
+ * top bits.
+ */
+ lsl data1, data1, pos
+ lsl data2, data2, pos
+ /*
+ * But we need to zero-extend (char is unsigned) the value and then
+ * perform a signed 32-bit subtraction.
+ */
+ lsr data1, data1, #56
+ sub result, data1, data2, lsr #56
+ ret
+ENDPROC(strcmp)
diff --git a/arch/arm64/lib/strlen.S b/arch/arm64/lib/strlen.S
new file mode 100644
index 0000000..987b68b
--- /dev/null
+++ b/arch/arm64/lib/strlen.S
@@ -0,0 +1,126 @@
+/*
+ * Copyright (C) 2013 ARM Ltd.
+ * Copyright (C) 2013 Linaro.
+ *
+ * This code is based on glibc cortex strings work originally authored by Linaro
+ * and re-licensed under GPLv2 for the Linux kernel. The original code can
+ * be found @
+ *
+ * http://bazaar.launchpad.net/~linaro-toolchain-dev/cortex-strings/trunk/
+ * files/head:/src/aarch64/
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+/*
+ * calculate the length of a string
+ *
+ * Parameters:
+ * x0 - const string pointer
+ * Returns:
+ * x0 - the return length of specific string
+ */
+
+/* Arguments and results. */
+srcin .req x0
+len .req x0
+
+/* Locals and temporaries. */
+src .req x1
+data1 .req x2
+data2 .req x3
+data2a .req x4
+has_nul1 .req x5
+has_nul2 .req x6
+tmp1 .req x7
+tmp2 .req x8
+tmp3 .req x9
+tmp4 .req x10
+zeroones .req x11
+pos .req x12
+
+#define REP8_01 0x0101010101010101
+#define REP8_7f 0x7f7f7f7f7f7f7f7f
+#define REP8_80 0x8080808080808080
+
+ENTRY(strlen)
+ mov zeroones, #REP8_01
+ bic src, srcin, #15
+ ands tmp1, srcin, #15
+ b.ne .Lmisaligned
+ /*
+ * NUL detection works on the principle that (X - 1) & (~X) & 0x80
+ * (=> (X - 1) & ~(X | 0x7f)) is non-zero iff a byte is zero, and
+ * can be done in parallel across the entire word.
+ */
+ /*
+ * The inner loop deals with two Dwords at a time. This has a
+ * slightly higher start-up cost, but we should win quite quickly,
+ * especially on cores with a high number of issue slots per
+ * cycle, as we get much better parallelism out of the operations.
+ */
+.Lloop:
+ ldp data1, data2, [src], #16
+.Lrealigned:
+ sub tmp1, data1, zeroones
+ orr tmp2, data1, #REP8_7f
+ sub tmp3, data2, zeroones
+ orr tmp4, data2, #REP8_7f
+ bic has_nul1, tmp1, tmp2
+ bics has_nul2, tmp3, tmp4
+ ccmp has_nul1, #0, #0, eq /* NZCV = 0000 */
+ b.eq .Lloop
+
+ sub len, src, srcin
+ cbz has_nul1, .Lnul_in_data2
+CPU_BE( mov data2, data1 ) /*prepare data to re-calculate the syndrome*/
+ sub len, len, #8
+ mov has_nul2, has_nul1
+.Lnul_in_data2:
+ /*
+ * For big-endian, carry propagation (if the final byte in the
+ * string is 0x01) means we cannot use has_nul directly. The
+ * easiest way to get the correct byte is to byte-swap the data
+ * and calculate the syndrome a second time.
+ */
+CPU_BE( rev data2, data2 )
+CPU_BE( sub tmp1, data2, zeroones )
+CPU_BE( orr tmp2, data2, #REP8_7f )
+CPU_BE( bic has_nul2, tmp1, tmp2 )
+
+ sub len, len, #8
+ rev has_nul2, has_nul2
+ clz pos, has_nul2
+ add len, len, pos, lsr #3 /* Bits to bytes. */
+ ret
+
+.Lmisaligned:
+ cmp tmp1, #8
+ neg tmp1, tmp1
+ ldp data1, data2, [src], #16
+ lsl tmp1, tmp1, #3 /* Bytes beyond alignment -> bits. */
+ mov tmp2, #~0
+ /* Big-endian. Early bytes are at MSB. */
+CPU_BE( lsl tmp2, tmp2, tmp1 ) /* Shift (tmp1 & 63). */
+ /* Little-endian. Early bytes are at LSB. */
+CPU_LE( lsr tmp2, tmp2, tmp1 ) /* Shift (tmp1 & 63). */
+
+ orr data1, data1, tmp2
+ orr data2a, data2, tmp2
+ csinv data1, data1, xzr, le
+ csel data2, data2, data2a, le
+ b .Lrealigned
+ENDPROC(strlen)
diff --git a/arch/arm64/lib/strncmp.S b/arch/arm64/lib/strncmp.S
new file mode 100644
index 0000000..0224cf5
--- /dev/null
+++ b/arch/arm64/lib/strncmp.S
@@ -0,0 +1,310 @@
+/*
+ * Copyright (C) 2013 ARM Ltd.
+ * Copyright (C) 2013 Linaro.
+ *
+ * This code is based on glibc cortex strings work originally authored by Linaro
+ * and re-licensed under GPLv2 for the Linux kernel. The original code can
+ * be found @
+ *
+ * http://bazaar.launchpad.net/~linaro-toolchain-dev/cortex-strings/trunk/
+ * files/head:/src/aarch64/
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+/*
+ * compare two strings
+ *
+ * Parameters:
+ * x0 - const string 1 pointer
+ * x1 - const string 2 pointer
+ * x2 - the maximal length to be compared
+ * Returns:
+ * x0 - an integer less than, equal to, or greater than zero if s1 is found,
+ * respectively, to be less than, to match, or be greater than s2.
+ */
+
+#define REP8_01 0x0101010101010101
+#define REP8_7f 0x7f7f7f7f7f7f7f7f
+#define REP8_80 0x8080808080808080
+
+/* Parameters and result. */
+src1 .req x0
+src2 .req x1
+limit .req x2
+result .req x0
+
+/* Internal variables. */
+data1 .req x3
+data1w .req w3
+data2 .req x4
+data2w .req w4
+has_nul .req x5
+diff .req x6
+syndrome .req x7
+tmp1 .req x8
+tmp2 .req x9
+tmp3 .req x10
+zeroones .req x11
+pos .req x12
+limit_wd .req x13
+mask .req x14
+endloop .req x15
+
+ENTRY(strncmp)
+ cbz limit, .Lret0
+ eor tmp1, src1, src2
+ mov zeroones, #REP8_01
+ tst tmp1, #7
+ b.ne .Lmisaligned8
+ ands tmp1, src1, #7
+ b.ne .Lmutual_align
+ /* Calculate the number of full and partial words -1. */
+ /*
+ * when limit is mulitply of 8, if not sub 1,
+ * the judgement of last dword will wrong.
+ */
+ sub limit_wd, limit, #1 /* limit != 0, so no underflow. */
+ lsr limit_wd, limit_wd, #3 /* Convert to Dwords. */
+
+ /*
+ * NUL detection works on the principle that (X - 1) & (~X) & 0x80
+ * (=> (X - 1) & ~(X | 0x7f)) is non-zero iff a byte is zero, and
+ * can be done in parallel across the entire word.
+ */
+.Lloop_aligned:
+ ldr data1, [src1], #8
+ ldr data2, [src2], #8
+.Lstart_realigned:
+ subs limit_wd, limit_wd, #1
+ sub tmp1, data1, zeroones
+ orr tmp2, data1, #REP8_7f
+ eor diff, data1, data2 /* Non-zero if differences found. */
+ csinv endloop, diff, xzr, pl /* Last Dword or differences.*/
+ bics has_nul, tmp1, tmp2 /* Non-zero if NUL terminator. */
+ ccmp endloop, #0, #0, eq
+ b.eq .Lloop_aligned
+
+ /*Not reached the limit, must have found the end or a diff. */
+ tbz limit_wd, #63, .Lnot_limit
+
+ /* Limit % 8 == 0 => all bytes significant. */
+ ands limit, limit, #7
+ b.eq .Lnot_limit
+
+ lsl limit, limit, #3 /* Bits -> bytes. */
+ mov mask, #~0
+CPU_BE( lsr mask, mask, limit )
+CPU_LE( lsl mask, mask, limit )
+ bic data1, data1, mask
+ bic data2, data2, mask
+
+ /* Make sure that the NUL byte is marked in the syndrome. */
+ orr has_nul, has_nul, mask
+
+.Lnot_limit:
+ orr syndrome, diff, has_nul
+ b .Lcal_cmpresult
+
+.Lmutual_align:
+ /*
+ * Sources are mutually aligned, but are not currently at an
+ * alignment boundary. Round down the addresses and then mask off
+ * the bytes that precede the start point.
+ * We also need to adjust the limit calculations, but without
+ * overflowing if the limit is near ULONG_MAX.
+ */
+ bic src1, src1, #7
+ bic src2, src2, #7
+ ldr data1, [src1], #8
+ neg tmp3, tmp1, lsl #3 /* 64 - bits(bytes beyond align). */
+ ldr data2, [src2], #8
+ mov tmp2, #~0
+ sub limit_wd, limit, #1 /* limit != 0, so no underflow. */
+ /* Big-endian. Early bytes are at MSB. */
+CPU_BE( lsl tmp2, tmp2, tmp3 ) /* Shift (tmp1 & 63). */
+ /* Little-endian. Early bytes are at LSB. */
+CPU_LE( lsr tmp2, tmp2, tmp3 ) /* Shift (tmp1 & 63). */
+
+ and tmp3, limit_wd, #7
+ lsr limit_wd, limit_wd, #3
+ /* Adjust the limit. Only low 3 bits used, so overflow irrelevant.*/
+ add limit, limit, tmp1
+ add tmp3, tmp3, tmp1
+ orr data1, data1, tmp2
+ orr data2, data2, tmp2
+ add limit_wd, limit_wd, tmp3, lsr #3
+ b .Lstart_realigned
+
+/*when src1 offset is not equal to src2 offset...*/
+.Lmisaligned8:
+ cmp limit, #8
+ b.lo .Ltiny8proc /*limit < 8... */
+ /*
+ * Get the align offset length to compare per byte first.
+ * After this process, one string's address will be aligned.*/
+ and tmp1, src1, #7
+ neg tmp1, tmp1
+ add tmp1, tmp1, #8
+ and tmp2, src2, #7
+ neg tmp2, tmp2
+ add tmp2, tmp2, #8
+ subs tmp3, tmp1, tmp2
+ csel pos, tmp1, tmp2, hi /*Choose the maximum. */
+ /*
+ * Here, limit is not less than 8, so directly run .Ltinycmp
+ * without checking the limit.*/
+ sub limit, limit, pos
+.Ltinycmp:
+ ldrb data1w, [src1], #1
+ ldrb data2w, [src2], #1
+ subs pos, pos, #1
+ ccmp data1w, #1, #0, ne /* NZCV = 0b0000. */
+ ccmp data1w, data2w, #0, cs /* NZCV = 0b0000. */
+ b.eq .Ltinycmp
+ cbnz pos, 1f /*find the null or unequal...*/
+ cmp data1w, #1
+ ccmp data1w, data2w, #0, cs
+ b.eq .Lstart_align /*the last bytes are equal....*/
+1:
+ sub result, data1, data2
+ ret
+
+.Lstart_align:
+ lsr limit_wd, limit, #3
+ cbz limit_wd, .Lremain8
+ /*process more leading bytes to make str1 aligned...*/
+ ands xzr, src1, #7
+ b.eq .Lrecal_offset
+ add src1, src1, tmp3 /*tmp3 is positive in this branch.*/
+ add src2, src2, tmp3
+ ldr data1, [src1], #8
+ ldr data2, [src2], #8
+
+ sub limit, limit, tmp3
+ lsr limit_wd, limit, #3
+ subs limit_wd, limit_wd, #1
+
+ sub tmp1, data1, zeroones
+ orr tmp2, data1, #REP8_7f
+ eor diff, data1, data2 /* Non-zero if differences found. */
+ csinv endloop, diff, xzr, ne/*if limit_wd is 0,will finish the cmp*/
+ bics has_nul, tmp1, tmp2
+ ccmp endloop, #0, #0, eq /*has_null is ZERO: no null byte*/
+ b.ne .Lunequal_proc
+ /*How far is the current str2 from the alignment boundary...*/
+ and tmp3, tmp3, #7
+.Lrecal_offset:
+ neg pos, tmp3
+.Lloopcmp_proc:
+ /*
+ * Divide the eight bytes into two parts. First,backwards the src2
+ * to an alignment boundary,load eight bytes from the SRC2 alignment
+ * boundary,then compare with the relative bytes from SRC1.
+ * If all 8 bytes are equal,then start the second part's comparison.
+ * Otherwise finish the comparison.
+ * This special handle can garantee all the accesses are in the
+ * thread/task space in avoid to overrange access.
+ */
+ ldr data1, [src1,pos]
+ ldr data2, [src2,pos]
+ sub tmp1, data1, zeroones
+ orr tmp2, data1, #REP8_7f
+ bics has_nul, tmp1, tmp2 /* Non-zero if NUL terminator. */
+ eor diff, data1, data2 /* Non-zero if differences found. */
+ csinv endloop, diff, xzr, eq
+ cbnz endloop, .Lunequal_proc
+
+ /*The second part process*/
+ ldr data1, [src1], #8
+ ldr data2, [src2], #8
+ subs limit_wd, limit_wd, #1
+ sub tmp1, data1, zeroones
+ orr tmp2, data1, #REP8_7f
+ eor diff, data1, data2 /* Non-zero if differences found. */
+ csinv endloop, diff, xzr, ne/*if limit_wd is 0,will finish the cmp*/
+ bics has_nul, tmp1, tmp2
+ ccmp endloop, #0, #0, eq /*has_null is ZERO: no null byte*/
+ b.eq .Lloopcmp_proc
+
+.Lunequal_proc:
+ orr syndrome, diff, has_nul
+ cbz syndrome, .Lremain8
+.Lcal_cmpresult:
+ /*
+ * reversed the byte-order as big-endian,then CLZ can find the most
+ * significant zero bits.
+ */
+CPU_LE( rev syndrome, syndrome )
+CPU_LE( rev data1, data1 )
+CPU_LE( rev data2, data2 )
+ /*
+ * For big-endian we cannot use the trick with the syndrome value
+ * as carry-propagation can corrupt the upper bits if the trailing
+ * bytes in the string contain 0x01.
+ * However, if there is no NUL byte in the dword, we can generate
+ * the result directly. We can't just subtract the bytes as the
+ * MSB might be significant.
+ */
+CPU_BE( cbnz has_nul, 1f )
+CPU_BE( cmp data1, data2 )
+CPU_BE( cset result, ne )
+CPU_BE( cneg result, result, lo )
+CPU_BE( ret )
+CPU_BE( 1: )
+ /* Re-compute the NUL-byte detection, using a byte-reversed value.*/
+CPU_BE( rev tmp3, data1 )
+CPU_BE( sub tmp1, tmp3, zeroones )
+CPU_BE( orr tmp2, tmp3, #REP8_7f )
+CPU_BE( bic has_nul, tmp1, tmp2 )
+CPU_BE( rev has_nul, has_nul )
+CPU_BE( orr syndrome, diff, has_nul )
+ /*
+ * The MS-non-zero bit of the syndrome marks either the first bit
+ * that is different, or the top bit of the first zero byte.
+ * Shifting left now will bring the critical information into the
+ * top bits.
+ */
+ clz pos, syndrome
+ lsl data1, data1, pos
+ lsl data2, data2, pos
+ /*
+ * But we need to zero-extend (char is unsigned) the value and then
+ * perform a signed 32-bit subtraction.
+ */
+ lsr data1, data1, #56
+ sub result, data1, data2, lsr #56
+ ret
+
+.Lremain8:
+ /* Limit % 8 == 0 => all bytes significant. */
+ ands limit, limit, #7
+ b.eq .Lret0
+.Ltiny8proc:
+ ldrb data1w, [src1], #1
+ ldrb data2w, [src2], #1
+ subs limit, limit, #1
+
+ ccmp data1w, #1, #0, ne /* NZCV = 0b0000. */
+ ccmp data1w, data2w, #0, cs /* NZCV = 0b0000. */
+ b.eq .Ltiny8proc
+ sub result, data1, data2
+ ret
+
+.Lret0:
+ mov result, #0
+ ret
+ENDPROC(strncmp)
diff --git a/arch/arm64/lib/strnlen.S b/arch/arm64/lib/strnlen.S
new file mode 100644
index 0000000..2ca6657
--- /dev/null
+++ b/arch/arm64/lib/strnlen.S
@@ -0,0 +1,171 @@
+/*
+ * Copyright (C) 2013 ARM Ltd.
+ * Copyright (C) 2013 Linaro.
+ *
+ * This code is based on glibc cortex strings work originally authored by Linaro
+ * and re-licensed under GPLv2 for the Linux kernel. The original code can
+ * be found @
+ *
+ * http://bazaar.launchpad.net/~linaro-toolchain-dev/cortex-strings/trunk/
+ * files/head:/src/aarch64/
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+/*
+ * determine the length of a fixed-size string
+ *
+ * Parameters:
+ * x0 - const string pointer
+ * x1 - maximal string length
+ * Returns:
+ * x0 - the return length of specific string
+ */
+
+/* Arguments and results. */
+srcin .req x0
+len .req x0
+limit .req x1
+
+/* Locals and temporaries. */
+src .req x2
+data1 .req x3
+data2 .req x4
+data2a .req x5
+has_nul1 .req x6
+has_nul2 .req x7
+tmp1 .req x8
+tmp2 .req x9
+tmp3 .req x10
+tmp4 .req x11
+zeroones .req x12
+pos .req x13
+limit_wd .req x14
+
+#define REP8_01 0x0101010101010101
+#define REP8_7f 0x7f7f7f7f7f7f7f7f
+#define REP8_80 0x8080808080808080
+
+ENTRY(strnlen)
+ cbz limit, .Lhit_limit
+ mov zeroones, #REP8_01
+ bic src, srcin, #15
+ ands tmp1, srcin, #15
+ b.ne .Lmisaligned
+ /* Calculate the number of full and partial words -1. */
+ sub limit_wd, limit, #1 /* Limit != 0, so no underflow. */
+ lsr limit_wd, limit_wd, #4 /* Convert to Qwords. */
+
+ /*
+ * NUL detection works on the principle that (X - 1) & (~X) & 0x80
+ * (=> (X - 1) & ~(X | 0x7f)) is non-zero iff a byte is zero, and
+ * can be done in parallel across the entire word.
+ */
+ /*
+ * The inner loop deals with two Dwords at a time. This has a
+ * slightly higher start-up cost, but we should win quite quickly,
+ * especially on cores with a high number of issue slots per
+ * cycle, as we get much better parallelism out of the operations.
+ */
+.Lloop:
+ ldp data1, data2, [src], #16
+.Lrealigned:
+ sub tmp1, data1, zeroones
+ orr tmp2, data1, #REP8_7f
+ sub tmp3, data2, zeroones
+ orr tmp4, data2, #REP8_7f
+ bic has_nul1, tmp1, tmp2
+ bic has_nul2, tmp3, tmp4
+ subs limit_wd, limit_wd, #1
+ orr tmp1, has_nul1, has_nul2
+ ccmp tmp1, #0, #0, pl /* NZCV = 0000 */
+ b.eq .Lloop
+
+ cbz tmp1, .Lhit_limit /* No null in final Qword. */
+
+ /*
+ * We know there's a null in the final Qword. The easiest thing
+ * to do now is work out the length of the string and return
+ * MIN (len, limit).
+ */
+ sub len, src, srcin
+ cbz has_nul1, .Lnul_in_data2
+CPU_BE( mov data2, data1 ) /*perpare data to re-calculate the syndrome*/
+
+ sub len, len, #8
+ mov has_nul2, has_nul1
+.Lnul_in_data2:
+ /*
+ * For big-endian, carry propagation (if the final byte in the
+ * string is 0x01) means we cannot use has_nul directly. The
+ * easiest way to get the correct byte is to byte-swap the data
+ * and calculate the syndrome a second time.
+ */
+CPU_BE( rev data2, data2 )
+CPU_BE( sub tmp1, data2, zeroones )
+CPU_BE( orr tmp2, data2, #REP8_7f )
+CPU_BE( bic has_nul2, tmp1, tmp2 )
+
+ sub len, len, #8
+ rev has_nul2, has_nul2
+ clz pos, has_nul2
+ add len, len, pos, lsr #3 /* Bits to bytes. */
+ cmp len, limit
+ csel len, len, limit, ls /* Return the lower value. */
+ ret
+
+.Lmisaligned:
+ /*
+ * Deal with a partial first word.
+ * We're doing two things in parallel here;
+ * 1) Calculate the number of words (but avoiding overflow if
+ * limit is near ULONG_MAX) - to do this we need to work out
+ * limit + tmp1 - 1 as a 65-bit value before shifting it;
+ * 2) Load and mask the initial data words - we force the bytes
+ * before the ones we are interested in to 0xff - this ensures
+ * early bytes will not hit any zero detection.
+ */
+ ldp data1, data2, [src], #16
+
+ sub limit_wd, limit, #1
+ and tmp3, limit_wd, #15
+ lsr limit_wd, limit_wd, #4
+
+ add tmp3, tmp3, tmp1
+ add limit_wd, limit_wd, tmp3, lsr #4
+
+ neg tmp4, tmp1
+ lsl tmp4, tmp4, #3 /* Bytes beyond alignment -> bits. */
+
+ mov tmp2, #~0
+ /* Big-endian. Early bytes are at MSB. */
+CPU_BE( lsl tmp2, tmp2, tmp4 ) /* Shift (tmp1 & 63). */
+ /* Little-endian. Early bytes are at LSB. */
+CPU_LE( lsr tmp2, tmp2, tmp4 ) /* Shift (tmp1 & 63). */
+
+ cmp tmp1, #8
+
+ orr data1, data1, tmp2
+ orr data2a, data2, tmp2
+
+ csinv data1, data1, xzr, le
+ csel data2, data2, data2a, le
+ b .Lrealigned
+
+.Lhit_limit:
+ mov len, limit
+ ret
+ENDPROC(strnlen)
diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile
index b51d364..3ecb56c 100644
--- a/arch/arm64/mm/Makefile
+++ b/arch/arm64/mm/Makefile
@@ -1,5 +1,5 @@
obj-y := dma-mapping.o extable.o fault.o init.o \
cache.o copypage.o flush.o \
ioremap.o mmap.o pgd.o mmu.o \
- context.o tlb.o proc.o
+ context.o proc.o
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S
index fda7568..2366383 100644
--- a/arch/arm64/mm/cache.S
+++ b/arch/arm64/mm/cache.S
@@ -31,7 +31,7 @@
* Corrupted registers: x0-x7, x9-x11
*/
__flush_dcache_all:
- dsb sy // ensure ordering with previous memory accesses
+ dmb sy // ensure ordering with previous memory accesses
mrs x0, clidr_el1 // read clidr
and x3, x0, #0x7000000 // extract loc from clidr
lsr x3, x3, #23 // left align loc bit field
@@ -128,7 +128,7 @@
add x4, x4, x2
cmp x4, x1
b.lo 1b
- dsb sy
+ dsb ish
icache_line_size x2, x3
sub x3, x2, #1
@@ -139,7 +139,7 @@
cmp x4, x1
b.lo 1b
9: // ignore any faulting cache operation
- dsb sy
+ dsb ish
isb
ret
ENDPROC(flush_icache_range)
diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
index c851eb4..4164c5a 100644
--- a/arch/arm64/mm/dma-mapping.c
+++ b/arch/arm64/mm/dma-mapping.c
@@ -115,7 +115,7 @@
for (i = 0; i < (size >> PAGE_SHIFT); i++)
map[i] = page + i;
coherent_ptr = vmap(map, size >> PAGE_SHIFT, VM_MAP,
- __get_dma_pgprot(attrs, pgprot_default, false));
+ __get_dma_pgprot(attrs, __pgprot(PROT_NORMAL_NC), false));
kfree(map);
if (!coherent_ptr)
goto no_map;
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index c23751b..bcc965e2 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -32,6 +32,7 @@
#include <asm/exception.h>
#include <asm/debug-monitors.h>
+#include <asm/esr.h>
#include <asm/system_misc.h>
#include <asm/pgtable.h>
#include <asm/tlbflush.h>
@@ -123,6 +124,7 @@
}
tsk->thread.fault_address = addr;
+ tsk->thread.fault_code = esr;
si.si_signo = sig;
si.si_errno = 0;
si.si_code = code;
@@ -148,8 +150,6 @@
#define VM_FAULT_BADMAP 0x010000
#define VM_FAULT_BADACCESS 0x020000
-#define ESR_WRITE (1 << 6)
-#define ESR_CM (1 << 8)
#define ESR_LNX_EXEC (1 << 24)
static int __do_page_fault(struct mm_struct *mm, unsigned long addr,
@@ -218,7 +218,7 @@
if (esr & ESR_LNX_EXEC) {
vm_flags = VM_EXEC;
- } else if ((esr & ESR_WRITE) && !(esr & ESR_CM)) {
+ } else if ((esr & ESR_EL1_WRITE) && !(esr & ESR_EL1_CM)) {
vm_flags = VM_WRITE;
mm_flags |= FAULT_FLAG_WRITE;
}
@@ -525,7 +525,7 @@
info.si_errno = 0;
info.si_code = inf->code;
info.si_addr = (void __user *)addr;
- arm64_notify_die("", regs, &info, esr);
+ arm64_notify_die("", regs, &info, 0);
return 0;
}
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 0a472c4..c43f1dd 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -43,11 +43,6 @@
struct page *empty_zero_page;
EXPORT_SYMBOL(empty_zero_page);
-pgprot_t pgprot_default;
-EXPORT_SYMBOL(pgprot_default);
-
-static pmdval_t prot_sect_kernel;
-
struct cachepolicy {
const char policy[16];
u64 mair;
@@ -122,33 +117,6 @@
}
early_param("cachepolicy", early_cachepolicy);
-/*
- * Adjust the PMD section entries according to the CPU in use.
- */
-void __init init_mem_pgprot(void)
-{
- pteval_t default_pgprot;
- int i;
-
- default_pgprot = PTE_ATTRINDX(MT_NORMAL);
- prot_sect_kernel = PMD_TYPE_SECT | PMD_SECT_AF | PMD_ATTRINDX(MT_NORMAL);
-
-#ifdef CONFIG_SMP
- /*
- * Mark memory with the "shared" attribute for SMP systems
- */
- default_pgprot |= PTE_SHARED;
- prot_sect_kernel |= PMD_SECT_S;
-#endif
-
- for (i = 0; i < 16; i++) {
- unsigned long v = pgprot_val(protection_map[i]);
- protection_map[i] = __pgprot(v | default_pgprot);
- }
-
- pgprot_default = __pgprot(PTE_TYPE_PAGE | PTE_AF | default_pgprot);
-}
-
pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
unsigned long size, pgprot_t vma_prot)
{
@@ -168,7 +136,8 @@
}
static void __init alloc_init_pte(pmd_t *pmd, unsigned long addr,
- unsigned long end, unsigned long pfn)
+ unsigned long end, unsigned long pfn,
+ pgprot_t prot)
{
pte_t *pte;
@@ -180,16 +149,27 @@
pte = pte_offset_kernel(pmd, addr);
do {
- set_pte(pte, pfn_pte(pfn, PAGE_KERNEL_EXEC));
+ set_pte(pte, pfn_pte(pfn, prot));
pfn++;
} while (pte++, addr += PAGE_SIZE, addr != end);
}
static void __init alloc_init_pmd(pud_t *pud, unsigned long addr,
- unsigned long end, phys_addr_t phys)
+ unsigned long end, phys_addr_t phys,
+ int map_io)
{
pmd_t *pmd;
unsigned long next;
+ pmdval_t prot_sect;
+ pgprot_t prot_pte;
+
+ if (map_io) {
+ prot_sect = PROT_SECT_DEVICE_nGnRE;
+ prot_pte = __pgprot(PROT_DEVICE_nGnRE);
+ } else {
+ prot_sect = PROT_SECT_NORMAL_EXEC;
+ prot_pte = PAGE_KERNEL_EXEC;
+ }
/*
* Check for initial section mappings in the pgd/pud and remove them.
@@ -205,7 +185,7 @@
/* try section mapping first */
if (((addr | next | phys) & ~SECTION_MASK) == 0) {
pmd_t old_pmd =*pmd;
- set_pmd(pmd, __pmd(phys | prot_sect_kernel));
+ set_pmd(pmd, __pmd(phys | prot_sect));
/*
* Check for previous table entries created during
* boot (__create_page_tables) and flush them.
@@ -213,21 +193,46 @@
if (!pmd_none(old_pmd))
flush_tlb_all();
} else {
- alloc_init_pte(pmd, addr, next, __phys_to_pfn(phys));
+ alloc_init_pte(pmd, addr, next, __phys_to_pfn(phys),
+ prot_pte);
}
phys += next - addr;
} while (pmd++, addr = next, addr != end);
}
static void __init alloc_init_pud(pgd_t *pgd, unsigned long addr,
- unsigned long end, unsigned long phys)
+ unsigned long end, unsigned long phys,
+ int map_io)
{
pud_t *pud = pud_offset(pgd, addr);
unsigned long next;
do {
next = pud_addr_end(addr, end);
- alloc_init_pmd(pud, addr, next, phys);
+
+ /*
+ * For 4K granule only, attempt to put down a 1GB block
+ */
+ if (!map_io && (PAGE_SHIFT == 12) &&
+ ((addr | next | phys) & ~PUD_MASK) == 0) {
+ pud_t old_pud = *pud;
+ set_pud(pud, __pud(phys | PROT_SECT_NORMAL_EXEC));
+
+ /*
+ * If we have an old value for a pud, it will
+ * be pointing to a pmd table that we no longer
+ * need (from swapper_pg_dir).
+ *
+ * Look up the old pmd table and free it.
+ */
+ if (!pud_none(old_pud)) {
+ phys_addr_t table = __pa(pmd_offset(&old_pud, 0));
+ memblock_free(table, PAGE_SIZE);
+ flush_tlb_all();
+ }
+ } else {
+ alloc_init_pmd(pud, addr, next, phys, map_io);
+ }
phys += next - addr;
} while (pud++, addr = next, addr != end);
}
@@ -236,30 +241,44 @@
* Create the page directory entries and any necessary page tables for the
* mapping specified by 'md'.
*/
-static void __init create_mapping(phys_addr_t phys, unsigned long virt,
- phys_addr_t size)
+static void __init __create_mapping(pgd_t *pgd, phys_addr_t phys,
+ unsigned long virt, phys_addr_t size,
+ int map_io)
{
unsigned long addr, length, end, next;
- pgd_t *pgd;
-
- if (virt < VMALLOC_START) {
- pr_warning("BUG: not creating mapping for 0x%016llx at 0x%016lx - outside kernel range\n",
- phys, virt);
- return;
- }
addr = virt & PAGE_MASK;
length = PAGE_ALIGN(size + (virt & ~PAGE_MASK));
- pgd = pgd_offset_k(addr);
end = addr + length;
do {
next = pgd_addr_end(addr, end);
- alloc_init_pud(pgd, addr, next, phys);
+ alloc_init_pud(pgd, addr, next, phys, map_io);
phys += next - addr;
} while (pgd++, addr = next, addr != end);
}
+static void __init create_mapping(phys_addr_t phys, unsigned long virt,
+ phys_addr_t size)
+{
+ if (virt < VMALLOC_START) {
+ pr_warn("BUG: not creating mapping for %pa at 0x%016lx - outside kernel range\n",
+ &phys, virt);
+ return;
+ }
+ __create_mapping(pgd_offset_k(virt & PAGE_MASK), phys, virt, size, 0);
+}
+
+void __init create_id_mapping(phys_addr_t addr, phys_addr_t size, int map_io)
+{
+ if ((addr >> PGDIR_SHIFT) >= ARRAY_SIZE(idmap_pg_dir)) {
+ pr_warn("BUG: not creating id mapping for %pa\n", &addr);
+ return;
+ }
+ __create_mapping(&idmap_pg_dir[pgd_index(addr)],
+ addr, addr, size, map_io);
+}
+
static void __init map_mem(void)
{
struct memblock_region *reg;
@@ -370,6 +389,9 @@
if (pud_none(*pud))
return 0;
+ if (pud_sect(*pud))
+ return pfn_valid(pud_pfn(*pud));
+
pmd = pmd_offset(pud, addr);
if (pmd_none(*pmd))
return 0;
@@ -417,7 +439,7 @@
if (!p)
return -ENOMEM;
- set_pmd(pmd, __pmd(__pa(p) | prot_sect_kernel));
+ set_pmd(pmd, __pmd(__pa(p) | PROT_SECT_NORMAL));
} else
vmemmap_verify((pte_t *)pmd, node, addr, next);
} while (addr = next, addr != end);
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 9042aff..7736779 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -182,7 +182,7 @@
ENTRY(__cpu_setup)
ic iallu // I+BTB cache invalidate
tlbi vmalle1is // invalidate I + D TLBs
- dsb sy
+ dsb ish
mov x0, #3 << 20
msr cpacr_el1, x0 // Enable FP/ASIMD
diff --git a/arch/arm64/mm/tlb.S b/arch/arm64/mm/tlb.S
deleted file mode 100644
index 19da91e..0000000
--- a/arch/arm64/mm/tlb.S
+++ /dev/null
@@ -1,71 +0,0 @@
-/*
- * Based on arch/arm/mm/tlb.S
- *
- * Copyright (C) 1997-2002 Russell King
- * Copyright (C) 2012 ARM Ltd.
- * Written by Catalin Marinas <catalin.marinas@arm.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-#include <linux/linkage.h>
-#include <asm/assembler.h>
-#include <asm/asm-offsets.h>
-#include <asm/page.h>
-#include <asm/tlbflush.h>
-#include "proc-macros.S"
-
-/*
- * __cpu_flush_user_tlb_range(start, end, vma)
- *
- * Invalidate a range of TLB entries in the specified address space.
- *
- * - start - start address (may not be aligned)
- * - end - end address (exclusive, may not be aligned)
- * - vma - vma_struct describing address range
- */
-ENTRY(__cpu_flush_user_tlb_range)
- vma_vm_mm x3, x2 // get vma->vm_mm
- mmid w3, x3 // get vm_mm->context.id
- dsb sy
- lsr x0, x0, #12 // align address
- lsr x1, x1, #12
- bfi x0, x3, #48, #16 // start VA and ASID
- bfi x1, x3, #48, #16 // end VA and ASID
-1: tlbi vae1is, x0 // TLB invalidate by address and ASID
- add x0, x0, #1
- cmp x0, x1
- b.lo 1b
- dsb sy
- ret
-ENDPROC(__cpu_flush_user_tlb_range)
-
-/*
- * __cpu_flush_kern_tlb_range(start,end)
- *
- * Invalidate a range of kernel TLB entries.
- *
- * - start - start address (may not be aligned)
- * - end - end address (exclusive, may not be aligned)
- */
-ENTRY(__cpu_flush_kern_tlb_range)
- dsb sy
- lsr x0, x0, #12 // align address
- lsr x1, x1, #12
-1: tlbi vaae1is, x0 // TLB invalidate by address
- add x0, x0, #1
- cmp x0, x1
- b.lo 1b
- dsb sy
- isb
- ret
-ENDPROC(__cpu_flush_kern_tlb_range)
diff --git a/arch/blackfin/include/asm/ftrace.h b/arch/blackfin/include/asm/ftrace.h
index 8a02950..2f1c3c2 100644
--- a/arch/blackfin/include/asm/ftrace.h
+++ b/arch/blackfin/include/asm/ftrace.h
@@ -66,16 +66,7 @@
#endif /* CONFIG_FRAME_POINTER */
-#define HAVE_ARCH_CALLER_ADDR
-
-/* inline function or macro may lead to unexpected result */
-#define CALLER_ADDR0 ((unsigned long)__builtin_return_address(0))
-#define CALLER_ADDR1 ((unsigned long)return_address(1))
-#define CALLER_ADDR2 ((unsigned long)return_address(2))
-#define CALLER_ADDR3 ((unsigned long)return_address(3))
-#define CALLER_ADDR4 ((unsigned long)return_address(4))
-#define CALLER_ADDR5 ((unsigned long)return_address(5))
-#define CALLER_ADDR6 ((unsigned long)return_address(6))
+#define ftrace_return_address(n) return_address(n)
#endif /* __ASSEMBLY__ */
diff --git a/arch/blackfin/kernel/ptrace.c b/arch/blackfin/kernel/ptrace.c
index e1f88e0..8b8fe67 100644
--- a/arch/blackfin/kernel/ptrace.c
+++ b/arch/blackfin/kernel/ptrace.c
@@ -117,6 +117,7 @@
int
is_user_addr_valid(struct task_struct *child, unsigned long start, unsigned long len)
{
+ bool valid;
struct vm_area_struct *vma;
struct sram_list_struct *sraml;
@@ -124,9 +125,12 @@
if (start + len < start)
return -EIO;
+ down_read(&child->mm->mmap_sem);
vma = find_vma(child->mm, start);
- if (vma && start >= vma->vm_start && start + len <= vma->vm_end)
- return 0;
+ valid = vma && start >= vma->vm_start && start + len <= vma->vm_end;
+ up_read(&child->mm->mmap_sem);
+ if (valid)
+ return 0;
for (sraml = child->mm->context.sram_list; sraml; sraml = sraml->next)
if (start >= (unsigned long)sraml->addr
diff --git a/arch/cris/arch-v10/drivers/gpio.c b/arch/cris/arch-v10/drivers/gpio.c
index f4374ba..64285e0 100644
--- a/arch/cris/arch-v10/drivers/gpio.c
+++ b/arch/cris/arch-v10/drivers/gpio.c
@@ -833,8 +833,8 @@
printk(KERN_INFO "ETRAX 100LX GPIO driver v2.5, (c) 2001-2008 "
"Axis Communications AB\n");
/* We call etrax_gpio_wake_up_check() from timer interrupt and
- * from cpu_idle() in kernel/process.c
- * The check in cpu_idle() reduces latency from ~15 ms to ~6 ms
+ * from default_idle() in kernel/process.c
+ * The check in default_idle() reduces latency from ~15 ms to ~6 ms
* in some tests.
*/
res = request_irq(TIMER0_IRQ_NBR, gpio_poll_timer_interrupt,
diff --git a/arch/cris/arch-v32/drivers/mach-fs/gpio.c b/arch/cris/arch-v32/drivers/mach-fs/gpio.c
index 9e54273..009f4ee 100644
--- a/arch/cris/arch-v32/drivers/mach-fs/gpio.c
+++ b/arch/cris/arch-v32/drivers/mach-fs/gpio.c
@@ -958,11 +958,7 @@
printk(KERN_INFO "ETRAX FS GPIO driver v2.5, (c) 2003-2007 "
"Axis Communications AB\n");
- /* We call etrax_gpio_wake_up_check() from timer interrupt and
- * from cpu_idle() in kernel/process.c
- * The check in cpu_idle() reduces latency from ~15 ms to ~6 ms
- * in some tests.
- */
+ /* We call etrax_gpio_wake_up_check() from timer interrupt */
if (request_irq(TIMER0_INTR_VECT, gpio_poll_timer_interrupt,
IRQF_SHARED, "gpio poll", &alarmlist))
printk(KERN_ERR "timer0 irq for gpio\n");
diff --git a/arch/ia64/kernel/crash.c b/arch/ia64/kernel/crash.c
index b942f403..2955f35 100644
--- a/arch/ia64/kernel/crash.c
+++ b/arch/ia64/kernel/crash.c
@@ -237,7 +237,7 @@
}
#ifdef CONFIG_SYSCTL
-static ctl_table kdump_ctl_table[] = {
+static struct ctl_table kdump_ctl_table[] = {
{
.procname = "kdump_on_init",
.data = &kdump_on_init,
@@ -255,7 +255,7 @@
{ }
};
-static ctl_table sys_table[] = {
+static struct ctl_table sys_table[] = {
{
.procname = "kernel",
.mode = 0555,
diff --git a/arch/ia64/kernel/perfmon.c b/arch/ia64/kernel/perfmon.c
index d841c4b..5845ffe 100644
--- a/arch/ia64/kernel/perfmon.c
+++ b/arch/ia64/kernel/perfmon.c
@@ -521,7 +521,7 @@
pfm_sysctl_t pfm_sysctl;
EXPORT_SYMBOL(pfm_sysctl);
-static ctl_table pfm_ctl_table[]={
+static struct ctl_table pfm_ctl_table[] = {
{
.procname = "debug",
.data = &pfm_sysctl.debug,
@@ -552,7 +552,7 @@
},
{}
};
-static ctl_table pfm_sysctl_dir[] = {
+static struct ctl_table pfm_sysctl_dir[] = {
{
.procname = "perfmon",
.mode = 0555,
@@ -560,7 +560,7 @@
},
{}
};
-static ctl_table pfm_sysctl_root[] = {
+static struct ctl_table pfm_sysctl_root[] = {
{
.procname = "kernel",
.mode = 0555,
diff --git a/arch/m68k/include/asm/signal.h b/arch/m68k/include/asm/signal.h
index 214320b5..8c8ce5e 100644
--- a/arch/m68k/include/asm/signal.h
+++ b/arch/m68k/include/asm/signal.h
@@ -60,15 +60,6 @@
__const_sigismember(set,sig) : \
__gen_sigismember(set,sig))
-static inline int sigfindinword(unsigned long word)
-{
- asm ("bfffo %1{#0,#0},%0"
- : "=d" (word)
- : "d" (word & -word)
- : "cc");
- return word ^ 31;
-}
-
#endif /* !CONFIG_CPU_HAS_NO_BITFIELDS */
#ifndef __uClinux__
diff --git a/arch/microblaze/configs/mmu_defconfig b/arch/microblaze/configs/mmu_defconfig
index deaf45a..e2f6543 100644
--- a/arch/microblaze/configs/mmu_defconfig
+++ b/arch/microblaze/configs/mmu_defconfig
@@ -49,6 +49,7 @@
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_UARTLITE=y
CONFIG_SERIAL_UARTLITE_CONSOLE=y
+CONFIG_SERIAL_OF_PLATFORM=y
# CONFIG_HW_RANDOM is not set
CONFIG_XILINX_HWICAP=y
CONFIG_I2C=y
diff --git a/arch/microblaze/configs/nommu_defconfig b/arch/microblaze/configs/nommu_defconfig
index 10b5172..a29ebd4 100644
--- a/arch/microblaze/configs/nommu_defconfig
+++ b/arch/microblaze/configs/nommu_defconfig
@@ -58,6 +58,7 @@
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_UARTLITE=y
CONFIG_SERIAL_UARTLITE_CONSOLE=y
+CONFIG_SERIAL_OF_PLATFORM=y
# CONFIG_HW_RANDOM is not set
CONFIG_XILINX_HWICAP=y
CONFIG_I2C=y
diff --git a/arch/microblaze/include/asm/Kbuild b/arch/microblaze/include/asm/Kbuild
index c98ed95..35b3ecaf 100644
--- a/arch/microblaze/include/asm/Kbuild
+++ b/arch/microblaze/include/asm/Kbuild
@@ -2,6 +2,7 @@
generic-y += barrier.h
generic-y += clkdev.h
generic-y += cputime.h
+generic-y += device.h
generic-y += exec.h
generic-y += hash.h
generic-y += mcs_spinlock.h
diff --git a/arch/microblaze/include/asm/device.h b/arch/microblaze/include/asm/device.h
deleted file mode 100644
index 123b2fe..0000000
--- a/arch/microblaze/include/asm/device.h
+++ /dev/null
@@ -1,26 +0,0 @@
-/*
- * Arch specific extensions to struct device
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License v2. See the file "COPYING" in the main directory of this archive
- * for more details.
- */
-
-#ifndef _ASM_MICROBLAZE_DEVICE_H
-#define _ASM_MICROBLAZE_DEVICE_H
-
-struct device_node;
-
-struct dev_archdata {
- /* DMA operations on that device */
- struct dma_map_ops *dma_ops;
- void *dma_data;
-};
-
-struct pdev_archdata {
- u64 dma_mask;
-};
-
-#endif /* _ASM_MICROBLAZE_DEVICE_H */
-
-
diff --git a/arch/microblaze/include/asm/dma-mapping.h b/arch/microblaze/include/asm/dma-mapping.h
index 46460f1..ab35372 100644
--- a/arch/microblaze/include/asm/dma-mapping.h
+++ b/arch/microblaze/include/asm/dma-mapping.h
@@ -35,16 +35,6 @@
#define __dma_alloc_coherent(dev, gfp, size, handle) NULL
#define __dma_free_coherent(size, addr) ((void)0)
-static inline unsigned long device_to_mask(struct device *dev)
-{
- if (dev->dma_mask && *dev->dma_mask)
- return *dev->dma_mask;
- /* Assume devices without mask can take 32 bit addresses */
- return 0xfffffffful;
-}
-
-extern struct dma_map_ops *dma_ops;
-
/*
* Available generic sets of operations
*/
@@ -52,20 +42,7 @@
static inline struct dma_map_ops *get_dma_ops(struct device *dev)
{
- /* We don't handle the NULL dev case for ISA for now. We could
- * do it via an out of line call but it is not needed for now. The
- * only ISA DMA device we support is the floppy and we have a hack
- * in the floppy driver directly to get a device for us.
- */
- if (unlikely(!dev) || !dev->archdata.dma_ops)
- return NULL;
-
- return dev->archdata.dma_ops;
-}
-
-static inline void set_dma_ops(struct device *dev, struct dma_map_ops *ops)
-{
- dev->archdata.dma_ops = ops;
+ return &dma_direct_ops;
}
static inline int dma_supported(struct device *dev, u64 mask)
diff --git a/arch/microblaze/include/asm/io.h b/arch/microblaze/include/asm/io.h
index 1e4c332..433751b 100644
--- a/arch/microblaze/include/asm/io.h
+++ b/arch/microblaze/include/asm/io.h
@@ -19,17 +19,14 @@
#ifndef CONFIG_PCI
#define _IO_BASE 0
#define _ISA_MEM_BASE 0
-#define PCI_DRAM_OFFSET 0
#else
#define _IO_BASE isa_io_base
#define _ISA_MEM_BASE isa_mem_base
-#define PCI_DRAM_OFFSET pci_dram_offset
struct pci_dev;
extern void pci_iounmap(struct pci_dev *dev, void __iomem *);
#define pci_iounmap pci_iounmap
extern unsigned long isa_io_base;
-extern unsigned long pci_dram_offset;
extern resource_size_t isa_mem_base;
#endif
diff --git a/arch/microblaze/include/asm/pci.h b/arch/microblaze/include/asm/pci.h
index 3355240..468aca8 100644
--- a/arch/microblaze/include/asm/pci.h
+++ b/arch/microblaze/include/asm/pci.h
@@ -45,14 +45,6 @@
#define pcibios_assign_all_busses() 0
#ifdef CONFIG_PCI
-extern void set_pci_dma_ops(struct dma_map_ops *dma_ops);
-extern struct dma_map_ops *get_pci_dma_ops(void);
-#else /* CONFIG_PCI */
-#define set_pci_dma_ops(d)
-#define get_pci_dma_ops() NULL
-#endif
-
-#ifdef CONFIG_PCI
static inline void pci_dma_burst_advice(struct pci_dev *pdev,
enum pci_dma_burst_strategy *strat,
unsigned long *strategy_parameter)
diff --git a/arch/microblaze/kernel/dma.c b/arch/microblaze/kernel/dma.c
index da68d00..4633c36 100644
--- a/arch/microblaze/kernel/dma.c
+++ b/arch/microblaze/kernel/dma.c
@@ -13,23 +13,6 @@
#include <linux/export.h>
#include <linux/bug.h>
-/*
- * Generic direct DMA implementation
- *
- * This implementation supports a per-device offset that can be applied if
- * the address at which memory is visible to devices is not 0. Platform code
- * can set archdata.dma_data to an unsigned long holding the offset. By
- * default the offset is PCI_DRAM_OFFSET.
- */
-
-static unsigned long get_dma_direct_offset(struct device *dev)
-{
- if (likely(dev))
- return (unsigned long)dev->archdata.dma_data;
-
- return PCI_DRAM_OFFSET; /* FIXME Not sure if is correct */
-}
-
#define NOT_COHERENT_CACHE
static void *dma_direct_alloc_coherent(struct device *dev, size_t size,
@@ -51,7 +34,7 @@
return NULL;
ret = page_address(page);
memset(ret, 0, size);
- *dma_handle = virt_to_phys(ret) + get_dma_direct_offset(dev);
+ *dma_handle = virt_to_phys(ret);
return ret;
#endif
@@ -77,7 +60,7 @@
/* FIXME this part of code is untested */
for_each_sg(sgl, sg, nents, i) {
- sg->dma_address = sg_phys(sg) + get_dma_direct_offset(dev);
+ sg->dma_address = sg_phys(sg);
__dma_sync(page_to_phys(sg_page(sg)) + sg->offset,
sg->length, direction);
}
@@ -85,12 +68,6 @@
return nents;
}
-static void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sg,
- int nents, enum dma_data_direction direction,
- struct dma_attrs *attrs)
-{
-}
-
static int dma_direct_dma_supported(struct device *dev, u64 mask)
{
return 1;
@@ -104,7 +81,7 @@
struct dma_attrs *attrs)
{
__dma_sync(page_to_phys(page) + offset, size, direction);
- return page_to_phys(page) + offset + get_dma_direct_offset(dev);
+ return page_to_phys(page) + offset;
}
static inline void dma_direct_unmap_page(struct device *dev,
@@ -181,7 +158,6 @@
.alloc = dma_direct_alloc_coherent,
.free = dma_direct_free_coherent,
.map_sg = dma_direct_map_sg,
- .unmap_sg = dma_direct_unmap_sg,
.dma_supported = dma_direct_dma_supported,
.map_page = dma_direct_map_page,
.unmap_page = dma_direct_unmap_page,
diff --git a/arch/microblaze/kernel/head.S b/arch/microblaze/kernel/head.S
index 17645b2..4655ff3 100644
--- a/arch/microblaze/kernel/head.S
+++ b/arch/microblaze/kernel/head.S
@@ -205,7 +205,7 @@
GT16: /* TLB0 is 16MB */
addik r9, r0, 0x1000000 /* means TLB0 is 16MB */
TLB1:
- /* must be used r2 because of substract if failed */
+ /* must be used r2 because of subtract if failed */
addik r2, r11, -0x0400000
bgei r2, GT20 /* size is greater than 16MB */
/* size is >16MB and <20MB */
diff --git a/arch/microblaze/kernel/setup.c b/arch/microblaze/kernel/setup.c
index 67cc4b2..ab5b488 100644
--- a/arch/microblaze/kernel/setup.c
+++ b/arch/microblaze/kernel/setup.c
@@ -71,13 +71,9 @@
xilinx_pci_init();
-#ifdef CONFIG_VT
-#if defined(CONFIG_XILINX_CONSOLE)
- conswitchp = &xil_con;
-#elif defined(CONFIG_DUMMY_CONSOLE)
+#if defined(CONFIG_DUMMY_CONSOLE)
conswitchp = &dummy_con;
#endif
-#endif
}
#ifdef CONFIG_MTD_UCLINUX
@@ -229,31 +225,3 @@
device_initcall(debugfs_tlb);
# endif
#endif
-
-static int dflt_bus_notify(struct notifier_block *nb,
- unsigned long action, void *data)
-{
- struct device *dev = data;
-
- /* We are only intereted in device addition */
- if (action != BUS_NOTIFY_ADD_DEVICE)
- return 0;
-
- set_dma_ops(dev, &dma_direct_ops);
-
- return NOTIFY_DONE;
-}
-
-static struct notifier_block dflt_plat_bus_notifier = {
- .notifier_call = dflt_bus_notify,
- .priority = INT_MAX,
-};
-
-static int __init setup_bus_notifier(void)
-{
- bus_register_notifier(&platform_bus_type, &dflt_plat_bus_notifier);
-
- return 0;
-}
-
-arch_initcall(setup_bus_notifier);
diff --git a/arch/microblaze/pci/pci-common.c b/arch/microblaze/pci/pci-common.c
index a59de1b..9037914 100644
--- a/arch/microblaze/pci/pci-common.c
+++ b/arch/microblaze/pci/pci-common.c
@@ -47,24 +47,9 @@
/* ISA Memory physical address */
resource_size_t isa_mem_base;
-static struct dma_map_ops *pci_dma_ops = &dma_direct_ops;
-
unsigned long isa_io_base;
-unsigned long pci_dram_offset;
static int pci_bus_count;
-
-void set_pci_dma_ops(struct dma_map_ops *dma_ops)
-{
- pci_dma_ops = dma_ops;
-}
-
-struct dma_map_ops *get_pci_dma_ops(void)
-{
- return pci_dma_ops;
-}
-EXPORT_SYMBOL(get_pci_dma_ops);
-
struct pci_controller *pcibios_alloc_controller(struct device_node *dev)
{
struct pci_controller *phb;
@@ -866,10 +851,6 @@
*/
set_dev_node(&dev->dev, pcibus_to_node(dev->bus));
- /* Hook up default DMA ops */
- set_dma_ops(&dev->dev, pci_dma_ops);
- dev->dev.archdata.dma_data = (void *)PCI_DRAM_OFFSET;
-
/* Read default IRQs and fixup if necessary */
dev->irq = of_irq_parse_and_map_pci(dev, 0, 0);
}
diff --git a/arch/mips/Kbuild b/arch/mips/Kbuild
index d2cfe45..dd29533 100644
--- a/arch/mips/Kbuild
+++ b/arch/mips/Kbuild
@@ -16,7 +16,7 @@
obj-y += kernel/
obj-y += mm/
-obj-y += math-emu/
+obj-y += net/
ifdef CONFIG_KVM
obj-y += kvm/
diff --git a/arch/mips/Kbuild.platforms b/arch/mips/Kbuild.platforms
index 6e23912..f5e18bf3 100644
--- a/arch/mips/Kbuild.platforms
+++ b/arch/mips/Kbuild.platforms
@@ -18,6 +18,7 @@
platforms += mti-malta
platforms += mti-sead3
platforms += netlogic
+platforms += paravirt
platforms += pmcs-msp71xx
platforms += pnx833x
platforms += ralink
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 5e0014e..7a469ac 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -12,6 +12,7 @@
select HAVE_ARCH_KGDB
select HAVE_ARCH_SECCOMP_FILTER
select HAVE_ARCH_TRACEHOOK
+ select HAVE_BPF_JIT if !CPU_MICROMIPS
select ARCH_HAVE_CUSTOM_GPIO_H
select HAVE_FUNCTION_TRACER
select HAVE_FUNCTION_TRACE_MCOUNT_TEST
@@ -50,6 +51,8 @@
select CLONE_BACKWARDS
select HAVE_DEBUG_STACKOVERFLOW
select HAVE_CC_STACKPROTECTOR
+ select CPU_PM if CPU_IDLE
+ select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
menu "Machine selection"
@@ -83,6 +86,7 @@
select SYS_HAS_EARLY_PRINTK
select SYS_SUPPORTS_32BIT_KERNEL
select SYS_SUPPORTS_LITTLE_ENDIAN
+ select SYS_SUPPORTS_MIPS16
select SYS_SUPPORTS_ZBOOT_UART16550
select ARCH_REQUIRE_GPIOLIB
select VLYNQ
@@ -106,6 +110,7 @@
select SYS_HAS_EARLY_PRINTK
select SYS_SUPPORTS_32BIT_KERNEL
select SYS_SUPPORTS_BIG_ENDIAN
+ select SYS_SUPPORTS_MIPS16
help
Support for the Atheros AR71XX/AR724X/AR913X SoCs.
@@ -122,6 +127,7 @@
select NO_EXCEPT_FILL
select SYS_SUPPORTS_32BIT_KERNEL
select SYS_SUPPORTS_LITTLE_ENDIAN
+ select SYS_SUPPORTS_MIPS16
select SYS_HAS_EARLY_PRINTK
select USE_GENERIC_EARLY_PRINTK_8250
help
@@ -168,9 +174,9 @@
bool "DECstations"
select BOOT_ELF32
select CEVT_DS1287
- select CEVT_R4K
+ select CEVT_R4K if CPU_R4X00
select CSRC_IOASIC
- select CSRC_R4K
+ select CSRC_R4K if CPU_R4X00
select CPU_DADDI_WORKAROUNDS if 64BIT
select CPU_R4000_WORKAROUNDS if 64BIT
select CPU_R4400_WORKAROUNDS if 64BIT
@@ -248,6 +254,7 @@
select SYS_HAS_CPU_MIPS32_R2
select SYS_SUPPORTS_BIG_ENDIAN
select SYS_SUPPORTS_32BIT_KERNEL
+ select SYS_SUPPORTS_MIPS16
select SYS_SUPPORTS_MULTITHREADING
select SYS_HAS_EARLY_PRINTK
select ARCH_REQUIRE_GPIOLIB
@@ -330,6 +337,7 @@
select SYS_SUPPORTS_LITTLE_ENDIAN
select SYS_SUPPORTS_MIPS_CMP
select SYS_SUPPORTS_MIPS_CPS
+ select SYS_SUPPORTS_MIPS16
select SYS_SUPPORTS_MULTITHREADING
select SYS_SUPPORTS_SMARTMIPS
select SYS_SUPPORTS_ZBOOT
@@ -361,6 +369,7 @@
select SYS_SUPPORTS_LITTLE_ENDIAN
select SYS_SUPPORTS_SMARTMIPS
select SYS_SUPPORTS_MICROMIPS
+ select SYS_SUPPORTS_MIPS16
select USB_EHCI_BIG_ENDIAN_DESC
select USB_EHCI_BIG_ENDIAN_MMIO
select USE_OF
@@ -380,6 +389,7 @@
select CEVT_R4K
select CSRC_R4K
select SYS_HAS_CPU_VR41XX
+ select SYS_SUPPORTS_MIPS16
select ARCH_REQUIRE_GPIOLIB
config NXP_STB220
@@ -407,6 +417,7 @@
select SYS_HAS_CPU_MIPS32_R2
select SYS_SUPPORTS_32BIT_KERNEL
select SYS_SUPPORTS_BIG_ENDIAN
+ select SYS_SUPPORTS_MIPS16
select IRQ_CPU
select SERIAL_8250
select SERIAL_8250_CONSOLE
@@ -430,6 +441,7 @@
select SYS_HAS_CPU_MIPS32_R2
select SYS_SUPPORTS_32BIT_KERNEL
select SYS_SUPPORTS_LITTLE_ENDIAN
+ select SYS_SUPPORTS_MIPS16
select SYS_HAS_EARLY_PRINTK
select HAVE_MACH_CLKDEV
select CLKDEV_LOOKUP
@@ -674,7 +686,6 @@
select SYS_SUPPORTS_BIG_ENDIAN
select SYS_SUPPORTS_HIGHMEM
select SYS_SUPPORTS_LITTLE_ENDIAN
- select USE_GENERIC_EARLY_PRINTK_8250
help
The SNI RM200/300/400 are MIPS-based machines manufactured by
Siemens Nixdorf Informationssysteme (SNI), parent company of Pyramid
@@ -721,6 +732,11 @@
select ZONE_DMA32
select HOLES_IN_ZONE
select ARCH_REQUIRE_GPIOLIB
+ select LIBFDT
+ select USE_OF
+ select ARCH_SPARSEMEM_ENABLE
+ select SYS_SUPPORTS_SMP
+ select NR_CPUS_DEFAULT_16
help
This option supports all of the Octeon reference boards from Cavium
Networks. It builds a kernel that dynamically determines the Octeon
@@ -789,6 +805,25 @@
This board is based on Netlogic XLP Processor.
Say Y here if you have a XLP based board.
+config MIPS_PARAVIRT
+ bool "Para-Virtualized guest system"
+ select CEVT_R4K
+ select CSRC_R4K
+ select DMA_COHERENT
+ select SYS_SUPPORTS_64BIT_KERNEL
+ select SYS_SUPPORTS_32BIT_KERNEL
+ select SYS_SUPPORTS_BIG_ENDIAN
+ select SYS_SUPPORTS_SMP
+ select NR_CPUS_DEFAULT_4
+ select SYS_HAS_EARLY_PRINTK
+ select SYS_HAS_CPU_MIPS32_R2
+ select SYS_HAS_CPU_MIPS64_R2
+ select SYS_HAS_CPU_CAVIUM_OCTEON
+ select HW_HAS_PCI
+ select SWAP_IO_SPACE
+ help
+ This option supports guest running under ????
+
endchoice
source "arch/mips/alchemy/Kconfig"
@@ -809,6 +844,7 @@
source "arch/mips/loongson/Kconfig"
source "arch/mips/loongson1/Kconfig"
source "arch/mips/netlogic/Kconfig"
+source "arch/mips/paravirt/Kconfig"
endmenu
@@ -1059,6 +1095,7 @@
select SYS_SUPPORTS_32BIT_KERNEL
select SYS_SUPPORTS_LITTLE_ENDIAN
select SYS_SUPPORTS_BIG_ENDIAN
+ select SYS_SUPPORTS_MIPS16
select CPU_MIPSR2_IRQ_VI
config SOC_PNX8335
@@ -1398,16 +1435,11 @@
config CPU_CAVIUM_OCTEON
bool "Cavium Octeon processor"
depends on SYS_HAS_CPU_CAVIUM_OCTEON
- select ARCH_SPARSEMEM_ENABLE
select CPU_HAS_PREFETCH
select CPU_SUPPORTS_64BIT_KERNEL
- select SYS_SUPPORTS_SMP
- select NR_CPUS_DEFAULT_16
select WEAK_ORDERING
select CPU_SUPPORTS_HIGHMEM
select CPU_SUPPORTS_HUGEPAGES
- select LIBFDT
- select USE_OF
select USB_EHCI_BIG_ENDIAN_MMIO
select MIPS_L1_CACHE_SHIFT_7
help
@@ -1659,6 +1691,12 @@
config SYS_HAS_CPU_XLP
bool
+config MIPS_MALTA_PM
+ depends on MIPS_MALTA
+ depends on PCI
+ bool
+ default y
+
#
# CPU may reorder R->R, R->W, W->R, W->W
# Reordering beyond LL and SC is handled in WEAK_REORDERING_BEYOND_LLSC
@@ -1842,7 +1880,7 @@
config CEVT_GIC
bool "Use GIC global counter for clock events"
- depends on IRQ_GIC && !(MIPS_SEAD3 || MIPS_MT_SMTC)
+ depends on IRQ_GIC && !MIPS_SEAD3
help
Use the GIC global counter for the clock events. The R4K clock
event driver is always present, so if the platform ends up not
@@ -1895,19 +1933,8 @@
bool
default y if !(CPU_R3000 || CPU_R8000 || CPU_SB1 || CPU_TX39XX || CPU_CAVIUM_OCTEON)
-choice
- prompt "MIPS MT options"
-
-config MIPS_MT_DISABLED
- bool "Disable multithreading support"
- help
- Use this option if your platform does not support the MT ASE
- which is hardware multithreading support. On systems without
- an MT-enabled processor, this will be the only option that is
- available in this menu.
-
config MIPS_MT_SMP
- bool "Use 1 TC on each available VPE for SMP"
+ bool "MIPS MT SMP support (1 TC on each available VPE)"
depends on SYS_SUPPORTS_MULTITHREADING
select CPU_MIPSR2_IRQ_VI
select CPU_MIPSR2_IRQ_EI
@@ -1926,26 +1953,6 @@
Intel Hyperthreading feature. For further information go to
<http://www.imgtec.com/mips/mips-multithreading.asp>.
-config MIPS_MT_SMTC
- bool "Use all TCs on all VPEs for SMP (DEPRECATED)"
- depends on CPU_MIPS32_R2
- depends on SYS_SUPPORTS_MULTITHREADING
- depends on !MIPS_CPS
- select CPU_MIPSR2_IRQ_VI
- select CPU_MIPSR2_IRQ_EI
- select MIPS_MT
- select SMP
- select SMP_UP
- select SYS_SUPPORTS_SMP
- select NR_CPUS_DEFAULT_8
- help
- This is a kernel model which is known as SMTC. This is
- supported on cores with the MT ASE and presents all TCs
- available on all VPEs to support SMP. For further
- information see <http://www.linux-mips.org/wiki/34K#SMTC>.
-
-endchoice
-
config MIPS_MT
bool
@@ -1967,7 +1974,7 @@
config MIPS_MT_FPAFF
bool "Dynamic FPU affinity for FP-intensive threads"
default y
- depends on MIPS_MT_SMP || MIPS_MT_SMTC
+ depends on MIPS_MT_SMP
config MIPS_VPE_LOADER
bool "VPE loader support."
@@ -1989,29 +1996,6 @@
default "y"
depends on MIPS_VPE_LOADER && !MIPS_CMP
-config MIPS_MT_SMTC_IM_BACKSTOP
- bool "Use per-TC register bits as backstop for inhibited IM bits"
- depends on MIPS_MT_SMTC
- default n
- help
- To support multiple TC microthreads acting as "CPUs" within
- a VPE, VPE-wide interrupt mask bits must be specially manipulated
- during interrupt handling. To support legacy drivers and interrupt
- controller management code, SMTC has a "backstop" to track and
- if necessary restore the interrupt mask. This has some performance
- impact on interrupt service overhead.
-
-config MIPS_MT_SMTC_IRQAFF
- bool "Support IRQ affinity API"
- depends on MIPS_MT_SMTC
- default n
- help
- Enables SMP IRQ affinity API (/proc/irq/*/smp_affinity, etc.)
- for SMTC Linux kernel. Requires platform support, of which
- an example can be found in the MIPS kernel i8259 and Malta
- platform code. Adds some overhead to interrupt dispatch, and
- should be used only if you know what you are doing.
-
config MIPS_VPE_LOADER_TOM
bool "Load VPE program into memory hidden from linux"
depends on MIPS_VPE_LOADER
@@ -2039,7 +2023,7 @@
config MIPS_CMP
bool "MIPS CMP framework support (DEPRECATED)"
- depends on SYS_SUPPORTS_MIPS_CMP && !MIPS_MT_SMTC
+ depends on SYS_SUPPORTS_MIPS_CMP
select MIPS_GIC_IPI
select SYNC_R4K
select WEAK_ORDERING
@@ -2057,9 +2041,11 @@
depends on SYS_SUPPORTS_MIPS_CPS
select MIPS_CM
select MIPS_CPC
+ select MIPS_CPS_PM if HOTPLUG_CPU
select MIPS_GIC_IPI
select SMP
select SYNC_R4K if (CEVT_R4K || CSRC_R4K)
+ select SYS_SUPPORTS_HOTPLUG_CPU
select SYS_SUPPORTS_SMP
select WEAK_ORDERING
help
@@ -2069,6 +2055,9 @@
no external assistance. It is safe to enable this when hardware
support is unavailable.
+config MIPS_CPS_PM
+ bool
+
config MIPS_GIC_IPI
bool
@@ -2199,6 +2188,13 @@
config SYS_SUPPORTS_MICROMIPS
bool
+config SYS_SUPPORTS_MIPS16
+ bool
+ help
+ This option must be set if a kernel might be executed on a MIPS16-
+ enabled CPU even if MIPS16 is not actually being used. In other
+ words, it makes the kernel MIPS16-tolerant.
+
config CPU_SUPPORTS_MSA
bool
@@ -2239,7 +2235,7 @@
config HW_PERF_EVENTS
bool "Enable hardware performance counter support for perf events"
- depends on PERF_EVENTS && !MIPS_MT_SMTC && OPROFILE=n && (CPU_MIPS32 || CPU_MIPS64 || CPU_R10000 || CPU_SB1 || CPU_CAVIUM_OCTEON || CPU_XLP)
+ depends on PERF_EVENTS && OPROFILE=n && (CPU_MIPS32 || CPU_MIPS64 || CPU_R10000 || CPU_SB1 || CPU_CAVIUM_OCTEON || CPU_XLP)
default y
help
Enable hardware performance counter support for perf events. If
@@ -2297,8 +2293,8 @@
bool
config NR_CPUS
- int "Maximum number of CPUs (2-64)"
- range 2 64
+ int "Maximum number of CPUs (2-256)"
+ range 2 256
depends on SMP
default "4" if NR_CPUS_DEFAULT_4
default "8" if NR_CPUS_DEFAULT_8
@@ -2671,12 +2667,16 @@
config MIPS_EXTERNAL_TIMER
bool
-if CPU_SUPPORTS_CPUFREQ && MIPS_EXTERNAL_TIMER
menu "CPU Power Management"
+
+if CPU_SUPPORTS_CPUFREQ && MIPS_EXTERNAL_TIMER
source "drivers/cpufreq/Kconfig"
-endmenu
endif
+source "drivers/cpuidle/Kconfig"
+
+endmenu
+
source "net/Kconfig"
source "drivers/Kconfig"
diff --git a/arch/mips/Kconfig.debug b/arch/mips/Kconfig.debug
index 25de292..3a2b775 100644
--- a/arch/mips/Kconfig.debug
+++ b/arch/mips/Kconfig.debug
@@ -79,15 +79,6 @@
Normally, you will choose 'N' here.
-config SMTC_IDLE_HOOK_DEBUG
- bool "Enable additional debug checks before going into CPU idle loop"
- depends on DEBUG_KERNEL && MIPS_MT_SMTC
- help
- This option enables Enable additional debug checks before going into
- CPU idle loop. For details on these checks, see
- arch/mips/kernel/smtc.c. This debugging option result in significant
- overhead so should be disabled in production kernels.
-
config SB1XXX_CORELIS
bool "Corelis Debugger"
depends on SIBYTE_SB1xxx_SOC
diff --git a/arch/mips/Makefile b/arch/mips/Makefile
index 60a359c..a8521de 100644
--- a/arch/mips/Makefile
+++ b/arch/mips/Makefile
@@ -120,7 +120,7 @@
-fno-omit-frame-pointer
ifeq ($(CONFIG_CPU_HAS_MSA),y)
-toolchain-msa := $(call cc-option-yn,-mhard-float -mfp64 -mmsa)
+toolchain-msa := $(call cc-option-yn,-mhard-float -mfp64 -Wa$(comma)-mmsa)
cflags-$(toolchain-msa) += -DTOOLCHAIN_SUPPORTS_MSA
endif
@@ -251,6 +251,7 @@
head-y := arch/mips/kernel/head.o
libs-y += arch/mips/lib/
+libs-y += arch/mips/math-emu/
# See arch/mips/Kbuild for content of core part of the kernel
core-y += arch/mips/
diff --git a/arch/mips/alchemy/board-xxs1500.c b/arch/mips/alchemy/board-xxs1500.c
index bd55136..3fb814b 100644
--- a/arch/mips/alchemy/board-xxs1500.c
+++ b/arch/mips/alchemy/board-xxs1500.c
@@ -49,7 +49,7 @@
prom_init_cmdline();
memsize_str = prom_getenv("memsize");
- if (!memsize_str || strict_strtoul(memsize_str, 0, &memsize))
+ if (!memsize_str || kstrtoul(memsize_str, 0, &memsize))
memsize = 0x04000000;
add_memory_region(0, memsize, BOOT_MEM_RAM);
diff --git a/arch/mips/alchemy/common/setup.c b/arch/mips/alchemy/common/setup.c
index 566a1743..8267e3c 100644
--- a/arch/mips/alchemy/common/setup.c
+++ b/arch/mips/alchemy/common/setup.c
@@ -67,6 +67,12 @@
case ALCHEMY_CPU_AU1500:
case ALCHEMY_CPU_AU1100:
coherentio = 0;
+ break;
+ case ALCHEMY_CPU_AU1200:
+ /* Au1200 AB USB does not support coherent memory */
+ if (0 == (read_c0_prid() & PRID_REV_MASK))
+ coherentio = 0;
+ break;
}
board_setup(); /* board specific setup */
diff --git a/arch/mips/alchemy/common/usb.c b/arch/mips/alchemy/common/usb.c
index 2adc7ed..d193dbe 100644
--- a/arch/mips/alchemy/common/usb.c
+++ b/arch/mips/alchemy/common/usb.c
@@ -355,47 +355,25 @@
}
}
-static inline int au1200_coherency_bug(void)
-{
-#if defined(CONFIG_DMA_COHERENT)
- /* Au1200 AB USB does not support coherent memory */
- if (!(read_c0_prid() & PRID_REV_MASK)) {
- printk(KERN_INFO "Au1200 USB: this is chip revision AB !!\n");
- printk(KERN_INFO "Au1200 USB: update your board or re-configure"
- " the kernel\n");
- return -ENODEV;
- }
-#endif
- return 0;
-}
-
static inline int au1200_usb_control(int block, int enable)
{
void __iomem *base =
(void __iomem *)KSEG1ADDR(AU1200_USB_CTL_PHYS_ADDR);
- int ret = 0;
switch (block) {
case ALCHEMY_USB_OHCI0:
- ret = au1200_coherency_bug();
- if (ret && enable)
- goto out;
__au1200_ohci_control(base, enable);
break;
case ALCHEMY_USB_UDC0:
__au1200_udc_control(base, enable);
break;
case ALCHEMY_USB_EHCI0:
- ret = au1200_coherency_bug();
- if (ret && enable)
- goto out;
__au1200_ehci_control(base, enable);
break;
default:
- ret = -ENODEV;
+ return -ENODEV;
}
-out:
- return ret;
+ return 0;
}
diff --git a/arch/mips/alchemy/devboards/pm.c b/arch/mips/alchemy/devboards/pm.c
index b86bff3..61e90fe 100644
--- a/arch/mips/alchemy/devboards/pm.c
+++ b/arch/mips/alchemy/devboards/pm.c
@@ -158,7 +158,7 @@
int tmp;
if (ATTRCMP(timer_timeout)) {
- tmp = strict_strtoul(instr, 0, &l);
+ tmp = kstrtoul(instr, 0, &l);
if (tmp)
return tmp;
@@ -181,7 +181,7 @@
}
} else if (ATTRCMP(wakemsk)) {
- tmp = strict_strtoul(instr, 0, &l);
+ tmp = kstrtoul(instr, 0, &l);
if (tmp)
return tmp;
diff --git a/arch/mips/bcm47xx/prom.c b/arch/mips/bcm47xx/prom.c
index 0af808d..1a03a2f 100644
--- a/arch/mips/bcm47xx/prom.c
+++ b/arch/mips/bcm47xx/prom.c
@@ -69,15 +69,18 @@
* BCM47XX uses 128MB for addressing the ram, if the system contains
* less that that amount of ram it remaps the ram more often into the
* available space.
- * Accessing memory after 128MB will cause an exception.
- * max contains the biggest possible address supported by the platform.
- * If the method wants to try something above we assume 128MB ram.
*/
- off = (unsigned long)prom_init;
- max = off | ((128 << 20) - 1);
- for (mem = (1 << 20); mem < (128 << 20); mem += (1 << 20)) {
- if ((off + mem) > max) {
- mem = (128 << 20);
+
+ /* Physical address, without mapping to any kernel segment */
+ off = CPHYSADDR((unsigned long)prom_init);
+
+ /* Accessing memory after 128 MiB will cause an exception */
+ max = 128 << 20;
+
+ for (mem = 1 << 20; mem < max; mem += 1 << 20) {
+ /* Loop condition may be not enough, off may be over 1 MiB */
+ if (off + mem >= max) {
+ mem = max;
printk(KERN_DEBUG "assume 128MB RAM\n");
break;
}
diff --git a/arch/mips/cavium-octeon/Kconfig b/arch/mips/cavium-octeon/Kconfig
index 227705d..6028666 100644
--- a/arch/mips/cavium-octeon/Kconfig
+++ b/arch/mips/cavium-octeon/Kconfig
@@ -10,6 +10,17 @@
non-CN63XXP1 hardware, so it is recommended to select "n"
unless it is known the workarounds are needed.
+config CAVIUM_OCTEON_CVMSEG_SIZE
+ int "Number of L1 cache lines reserved for CVMSEG memory"
+ range 0 54
+ default 1
+ help
+ CVMSEG LM is a segment that accesses portions of the dcache as a
+ local memory; the larger CVMSEG is, the smaller the cache is.
+ This selects the size of CVMSEG LM, which is in cache blocks. The
+ legally range is from zero to 54 cache blocks (i.e. CVMSEG LM is
+ between zero and 6192 bytes).
+
endif # CPU_CAVIUM_OCTEON
if CAVIUM_OCTEON_SOC
@@ -23,17 +34,6 @@
with this option to be run at the same time as one built without this
option.
-config CAVIUM_OCTEON_CVMSEG_SIZE
- int "Number of L1 cache lines reserved for CVMSEG memory"
- range 0 54
- default 1
- help
- CVMSEG LM is a segment that accesses portions of the dcache as a
- local memory; the larger CVMSEG is, the smaller the cache is.
- This selects the size of CVMSEG LM, which is in cache blocks. The
- legally range is from zero to 54 cache blocks (i.e. CVMSEG LM is
- between zero and 6192 bytes).
-
config CAVIUM_OCTEON_LOCK_L2
bool "Lock often used kernel code in the L2"
default "y"
@@ -86,7 +86,6 @@
select IOMMU_HELPER
select NEED_SG_DMA_LENGTH
-
config OCTEON_ILM
tristate "Module to measure interrupt latency using Octeon CIU Timer"
help
diff --git a/arch/mips/cavium-octeon/executive/cvmx-helper.c b/arch/mips/cavium-octeon/executive/cvmx-helper.c
index 8553ad5..7e5cf7a 100644
--- a/arch/mips/cavium-octeon/executive/cvmx-helper.c
+++ b/arch/mips/cavium-octeon/executive/cvmx-helper.c
@@ -106,6 +106,158 @@
EXPORT_SYMBOL_GPL(cvmx_helper_ports_on_interface);
/**
+ * @INTERNAL
+ * Return interface mode for CN68xx.
+ */
+static cvmx_helper_interface_mode_t __cvmx_get_mode_cn68xx(int interface)
+{
+ union cvmx_mio_qlmx_cfg qlm_cfg;
+ switch (interface) {
+ case 0:
+ qlm_cfg.u64 = cvmx_read_csr(CVMX_MIO_QLMX_CFG(0));
+ /* QLM is disabled when QLM SPD is 15. */
+ if (qlm_cfg.s.qlm_spd == 15)
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+
+ if (qlm_cfg.s.qlm_cfg == 2)
+ return CVMX_HELPER_INTERFACE_MODE_SGMII;
+ else if (qlm_cfg.s.qlm_cfg == 3)
+ return CVMX_HELPER_INTERFACE_MODE_XAUI;
+ else
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ case 2:
+ case 3:
+ case 4:
+ qlm_cfg.u64 = cvmx_read_csr(CVMX_MIO_QLMX_CFG(interface));
+ /* QLM is disabled when QLM SPD is 15. */
+ if (qlm_cfg.s.qlm_spd == 15)
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+
+ if (qlm_cfg.s.qlm_cfg == 2)
+ return CVMX_HELPER_INTERFACE_MODE_SGMII;
+ else if (qlm_cfg.s.qlm_cfg == 3)
+ return CVMX_HELPER_INTERFACE_MODE_XAUI;
+ else
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ case 7:
+ qlm_cfg.u64 = cvmx_read_csr(CVMX_MIO_QLMX_CFG(3));
+ /* QLM is disabled when QLM SPD is 15. */
+ if (qlm_cfg.s.qlm_spd == 15) {
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ } else if (qlm_cfg.s.qlm_cfg != 0) {
+ qlm_cfg.u64 = cvmx_read_csr(CVMX_MIO_QLMX_CFG(1));
+ if (qlm_cfg.s.qlm_cfg != 0)
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ }
+ return CVMX_HELPER_INTERFACE_MODE_NPI;
+ case 8:
+ return CVMX_HELPER_INTERFACE_MODE_LOOP;
+ default:
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ }
+}
+
+/**
+ * @INTERNAL
+ * Return interface mode for an Octeon II
+ */
+static cvmx_helper_interface_mode_t __cvmx_get_mode_octeon2(int interface)
+{
+ union cvmx_gmxx_inf_mode mode;
+
+ if (OCTEON_IS_MODEL(OCTEON_CN68XX))
+ return __cvmx_get_mode_cn68xx(interface);
+
+ if (interface == 2)
+ return CVMX_HELPER_INTERFACE_MODE_NPI;
+
+ if (interface == 3)
+ return CVMX_HELPER_INTERFACE_MODE_LOOP;
+
+ /* Only present in CN63XX & CN66XX Octeon model */
+ if ((OCTEON_IS_MODEL(OCTEON_CN63XX) &&
+ (interface == 4 || interface == 5)) ||
+ (OCTEON_IS_MODEL(OCTEON_CN66XX) &&
+ interface >= 4 && interface <= 7)) {
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ }
+
+ if (OCTEON_IS_MODEL(OCTEON_CN66XX)) {
+ union cvmx_mio_qlmx_cfg mio_qlm_cfg;
+
+ /* QLM2 is SGMII0 and QLM1 is SGMII1 */
+ if (interface == 0)
+ mio_qlm_cfg.u64 = cvmx_read_csr(CVMX_MIO_QLMX_CFG(2));
+ else if (interface == 1)
+ mio_qlm_cfg.u64 = cvmx_read_csr(CVMX_MIO_QLMX_CFG(1));
+ else
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+
+ if (mio_qlm_cfg.s.qlm_spd == 15)
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+
+ if (mio_qlm_cfg.s.qlm_cfg == 9)
+ return CVMX_HELPER_INTERFACE_MODE_SGMII;
+ else if (mio_qlm_cfg.s.qlm_cfg == 11)
+ return CVMX_HELPER_INTERFACE_MODE_XAUI;
+ else
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ } else if (OCTEON_IS_MODEL(OCTEON_CN61XX)) {
+ union cvmx_mio_qlmx_cfg qlm_cfg;
+
+ if (interface == 0) {
+ qlm_cfg.u64 = cvmx_read_csr(CVMX_MIO_QLMX_CFG(2));
+ if (qlm_cfg.s.qlm_cfg == 2)
+ return CVMX_HELPER_INTERFACE_MODE_SGMII;
+ else if (qlm_cfg.s.qlm_cfg == 3)
+ return CVMX_HELPER_INTERFACE_MODE_XAUI;
+ else
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ } else if (interface == 1) {
+ qlm_cfg.u64 = cvmx_read_csr(CVMX_MIO_QLMX_CFG(0));
+ if (qlm_cfg.s.qlm_cfg == 2)
+ return CVMX_HELPER_INTERFACE_MODE_SGMII;
+ else if (qlm_cfg.s.qlm_cfg == 3)
+ return CVMX_HELPER_INTERFACE_MODE_XAUI;
+ else
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ }
+ } else if (OCTEON_IS_MODEL(OCTEON_CNF71XX)) {
+ if (interface == 0) {
+ union cvmx_mio_qlmx_cfg qlm_cfg;
+ qlm_cfg.u64 = cvmx_read_csr(CVMX_MIO_QLMX_CFG(0));
+ if (qlm_cfg.s.qlm_cfg == 2)
+ return CVMX_HELPER_INTERFACE_MODE_SGMII;
+ }
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ }
+
+ if (interface == 1 && OCTEON_IS_MODEL(OCTEON_CN63XX))
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+
+ mode.u64 = cvmx_read_csr(CVMX_GMXX_INF_MODE(interface));
+
+ if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+ switch (mode.cn63xx.mode) {
+ case 0:
+ return CVMX_HELPER_INTERFACE_MODE_SGMII;
+ case 1:
+ return CVMX_HELPER_INTERFACE_MODE_XAUI;
+ default:
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ }
+ } else {
+ if (!mode.s.en)
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+
+ if (mode.s.type)
+ return CVMX_HELPER_INTERFACE_MODE_GMII;
+ else
+ return CVMX_HELPER_INTERFACE_MODE_RGMII;
+ }
+}
+
+/**
* Get the operating mode of an interface. Depending on the Octeon
* chip and configuration, this function returns an enumeration
* of the type of packet I/O supported by an interface.
@@ -118,6 +270,20 @@
cvmx_helper_interface_mode_t cvmx_helper_interface_get_mode(int interface)
{
union cvmx_gmxx_inf_mode mode;
+
+ if (interface < 0 ||
+ interface >= cvmx_helper_get_number_of_interfaces())
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+
+ /*
+ * Octeon II models
+ */
+ if (OCTEON_IS_MODEL(OCTEON_CN6XXX) || OCTEON_IS_MODEL(OCTEON_CNF71XX))
+ return __cvmx_get_mode_octeon2(interface);
+
+ /*
+ * Octeon and Octeon Plus models
+ */
if (interface == 2)
return CVMX_HELPER_INTERFACE_MODE_NPI;
diff --git a/arch/mips/cavium-octeon/octeon-irq.c b/arch/mips/cavium-octeon/octeon-irq.c
index 3aa5b46..1b82ac6 100644
--- a/arch/mips/cavium-octeon/octeon-irq.c
+++ b/arch/mips/cavium-octeon/octeon-irq.c
@@ -1260,11 +1260,13 @@
for (i = 0; i < 4; i++)
octeon_irq_force_ciu_mapping(ciu_domain, i + OCTEON_IRQ_PCI_MSI0, 0, i + 40);
+ octeon_irq_force_ciu_mapping(ciu_domain, OCTEON_IRQ_TWSI, 0, 45);
octeon_irq_force_ciu_mapping(ciu_domain, OCTEON_IRQ_RML, 0, 46);
for (i = 0; i < 4; i++)
octeon_irq_force_ciu_mapping(ciu_domain, i + OCTEON_IRQ_TIMER0, 0, i + 52);
octeon_irq_force_ciu_mapping(ciu_domain, OCTEON_IRQ_USB0, 0, 56);
+ octeon_irq_force_ciu_mapping(ciu_domain, OCTEON_IRQ_TWSI2, 0, 59);
/* CIU_1 */
for (i = 0; i < 16; i++)
diff --git a/arch/mips/cavium-octeon/setup.c b/arch/mips/cavium-octeon/setup.c
index f1bec00..008e9c8 100644
--- a/arch/mips/cavium-octeon/setup.c
+++ b/arch/mips/cavium-octeon/setup.c
@@ -729,17 +729,6 @@
octeon_write_lcd("Linux");
#endif
-#ifdef CONFIG_CAVIUM_GDB
- /*
- * When debugging the linux kernel, force the cores to enter
- * the debug exception handler to break in.
- */
- if (octeon_get_boot_debug_flag()) {
- cvmx_write_csr(CVMX_CIU_DINT, 1 << cvmx_get_core_num());
- cvmx_read_csr(CVMX_CIU_DINT);
- }
-#endif
-
octeon_setup_delays();
/*
@@ -779,12 +768,6 @@
MAX_MEMORY = 32ull << 30;
if (*p == '@')
RESERVE_LOW_MEM = memparse(p + 1, &p);
- } else if (strcmp(arg, "ecc_verbose") == 0) {
-#ifdef CONFIG_CAVIUM_REPORT_SINGLE_BIT_ECC
- __cvmx_interrupt_ecc_report_single_bit_errors = 1;
- pr_notice("Reporting of single bit ECC errors is "
- "turned on\n");
-#endif
#ifdef CONFIG_KEXEC
} else if (strncmp(arg, "crashkernel=", 12) == 0) {
crashk_size = memparse(arg+12, &p);
diff --git a/arch/mips/cavium-octeon/smp.c b/arch/mips/cavium-octeon/smp.c
index 67a078f..a7b3ae1 100644
--- a/arch/mips/cavium-octeon/smp.c
+++ b/arch/mips/cavium-octeon/smp.c
@@ -218,15 +218,6 @@
*/
static void octeon_smp_finish(void)
{
-#ifdef CONFIG_CAVIUM_GDB
- unsigned long tmp;
- /* Pulse MCD0 signal on Ctrl-C to stop all the cores. Also set the MCD0
- to be not masked by this core so we know the signal is received by
- someone */
- asm volatile ("dmfc0 %0, $22\n"
- "ori %0, %0, 0x9100\n" "dmtc0 %0, $22\n" : "=r" (tmp));
-#endif
-
octeon_user_io_init();
/* to generate the first CPU timer interrupt */
@@ -234,21 +225,6 @@
local_irq_enable();
}
-/**
- * Hook for after all CPUs are online
- */
-static void octeon_cpus_done(void)
-{
-#ifdef CONFIG_CAVIUM_GDB
- unsigned long tmp;
- /* Pulse MCD0 signal on Ctrl-C to stop all the cores. Also set the MCD0
- to be not masked by this core so we know the signal is received by
- someone */
- asm volatile ("dmfc0 %0, $22\n"
- "ori %0, %0, 0x9100\n" "dmtc0 %0, $22\n" : "=r" (tmp));
-#endif
-}
-
#ifdef CONFIG_HOTPLUG_CPU
/* State of each CPU. */
@@ -405,7 +381,6 @@
.send_ipi_mask = octeon_send_ipi_mask,
.init_secondary = octeon_init_secondary,
.smp_finish = octeon_smp_finish,
- .cpus_done = octeon_cpus_done,
.boot_secondary = octeon_boot_secondary,
.smp_setup = octeon_smp_setup,
.prepare_cpus = octeon_prepare_cpus,
diff --git a/arch/mips/configs/ath79_defconfig b/arch/mips/configs/ath79_defconfig
index e3a3836..134879c 100644
--- a/arch/mips/configs/ath79_defconfig
+++ b/arch/mips/configs/ath79_defconfig
@@ -46,7 +46,6 @@
CONFIG_MTD_REDBOOT_PARTS=y
CONFIG_MTD_REDBOOT_DIRECTORY_BLOCK=-2
CONFIG_MTD_CMDLINE_PARTS=y
-CONFIG_MTD_CHAR=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
CONFIG_MTD_JEDECPROBE=y
@@ -54,7 +53,7 @@
CONFIG_MTD_COMPLEX_MAPPINGS=y
CONFIG_MTD_PHYSMAP=y
CONFIG_MTD_M25P80=y
-# CONFIG_M25PXX_USE_FAST_READ is not set
+CONFIG_MTD_SPI_NOR=y
CONFIG_NETDEVICES=y
# CONFIG_NET_PACKET_ENGINE is not set
CONFIG_ATH_COMMON=m
diff --git a/arch/mips/configs/db1xxx_defconfig b/arch/mips/configs/db1xxx_defconfig
index c99b6ee..a64b30b 100644
--- a/arch/mips/configs/db1xxx_defconfig
+++ b/arch/mips/configs/db1xxx_defconfig
@@ -113,6 +113,7 @@
CONFIG_MTD_NAND_ECC_BCH=y
CONFIG_MTD_NAND_AU1550=y
CONFIG_MTD_NAND_PLATFORM=y
+CONFIG_MTD_SPI_NOR=y
CONFIG_EEPROM_AT24=y
CONFIG_EEPROM_AT25=y
CONFIG_SCSI_TGT=y
diff --git a/arch/mips/configs/maltasmtc_defconfig b/arch/mips/configs/maltasmtc_defconfig
deleted file mode 100644
index eb316447..0000000
--- a/arch/mips/configs/maltasmtc_defconfig
+++ /dev/null
@@ -1,196 +0,0 @@
-CONFIG_MIPS_MALTA=y
-CONFIG_CPU_LITTLE_ENDIAN=y
-CONFIG_CPU_MIPS32_R2=y
-CONFIG_PAGE_SIZE_16KB=y
-CONFIG_MIPS_MT_SMTC=y
-# CONFIG_MIPS_MT_FPAFF is not set
-CONFIG_NR_CPUS=9
-CONFIG_HZ_48=y
-CONFIG_LOCALVERSION="smtc"
-CONFIG_SYSVIPC=y
-CONFIG_POSIX_MQUEUE=y
-CONFIG_AUDIT=y
-CONFIG_IKCONFIG=y
-CONFIG_IKCONFIG_PROC=y
-CONFIG_LOG_BUF_SHIFT=15
-CONFIG_SYSCTL_SYSCALL=y
-CONFIG_EMBEDDED=y
-CONFIG_SLAB=y
-CONFIG_MODULES=y
-CONFIG_MODULE_UNLOAD=y
-CONFIG_MODVERSIONS=y
-CONFIG_MODULE_SRCVERSION_ALL=y
-# CONFIG_BLK_DEV_BSG is not set
-CONFIG_PCI=y
-# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
-CONFIG_NET=y
-CONFIG_PACKET=y
-CONFIG_UNIX=y
-CONFIG_XFRM_USER=m
-CONFIG_NET_KEY=y
-CONFIG_INET=y
-CONFIG_IP_MULTICAST=y
-CONFIG_IP_ADVANCED_ROUTER=y
-CONFIG_IP_MULTIPLE_TABLES=y
-CONFIG_IP_ROUTE_MULTIPATH=y
-CONFIG_IP_ROUTE_VERBOSE=y
-CONFIG_IP_PNP=y
-CONFIG_IP_PNP_DHCP=y
-CONFIG_IP_PNP_BOOTP=y
-CONFIG_NET_IPIP=m
-CONFIG_IP_MROUTE=y
-CONFIG_IP_PIMSM_V1=y
-CONFIG_IP_PIMSM_V2=y
-CONFIG_SYN_COOKIES=y
-CONFIG_INET_AH=m
-CONFIG_INET_ESP=m
-CONFIG_INET_IPCOMP=m
-# CONFIG_INET_LRO is not set
-CONFIG_INET6_AH=m
-CONFIG_INET6_ESP=m
-CONFIG_INET6_IPCOMP=m
-CONFIG_IPV6_TUNNEL=m
-CONFIG_BRIDGE=m
-CONFIG_VLAN_8021Q=m
-CONFIG_ATALK=m
-CONFIG_DEV_APPLETALK=m
-CONFIG_IPDDP=m
-CONFIG_IPDDP_ENCAP=y
-CONFIG_NET_SCHED=y
-CONFIG_NET_SCH_CBQ=m
-CONFIG_NET_SCH_HTB=m
-CONFIG_NET_SCH_HFSC=m
-CONFIG_NET_SCH_PRIO=m
-CONFIG_NET_SCH_RED=m
-CONFIG_NET_SCH_SFQ=m
-CONFIG_NET_SCH_TEQL=m
-CONFIG_NET_SCH_TBF=m
-CONFIG_NET_SCH_GRED=m
-CONFIG_NET_SCH_DSMARK=m
-CONFIG_NET_SCH_NETEM=m
-CONFIG_NET_SCH_INGRESS=m
-CONFIG_NET_CLS_BASIC=m
-CONFIG_NET_CLS_TCINDEX=m
-CONFIG_NET_CLS_ROUTE4=m
-CONFIG_NET_CLS_FW=m
-CONFIG_NET_CLS_U32=m
-CONFIG_NET_CLS_RSVP=m
-CONFIG_NET_CLS_RSVP6=m
-CONFIG_NET_CLS_ACT=y
-CONFIG_NET_ACT_POLICE=y
-CONFIG_NET_CLS_IND=y
-# CONFIG_WIRELESS is not set
-CONFIG_DEVTMPFS=y
-CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
-CONFIG_IDE=y
-# CONFIG_IDE_PROC_FS is not set
-# CONFIG_IDEPCI_PCIBUS_ORDER is not set
-CONFIG_BLK_DEV_GENERIC=y
-CONFIG_BLK_DEV_PIIX=y
-CONFIG_SCSI=y
-CONFIG_BLK_DEV_SD=y
-CONFIG_CHR_DEV_SG=y
-# CONFIG_SCSI_LOWLEVEL is not set
-CONFIG_NETDEVICES=y
-# CONFIG_NET_VENDOR_3COM is not set
-# CONFIG_NET_VENDOR_ADAPTEC is not set
-# CONFIG_NET_VENDOR_ALTEON is not set
-CONFIG_PCNET32=y
-# CONFIG_NET_VENDOR_ATHEROS is not set
-# CONFIG_NET_VENDOR_BROADCOM is not set
-# CONFIG_NET_VENDOR_BROCADE is not set
-# CONFIG_NET_VENDOR_CHELSIO is not set
-# CONFIG_NET_VENDOR_CISCO is not set
-# CONFIG_NET_VENDOR_DEC is not set
-# CONFIG_NET_VENDOR_DLINK is not set
-# CONFIG_NET_VENDOR_EMULEX is not set
-# CONFIG_NET_VENDOR_EXAR is not set
-# CONFIG_NET_VENDOR_HP is not set
-# CONFIG_NET_VENDOR_INTEL is not set
-# CONFIG_NET_VENDOR_MARVELL is not set
-# CONFIG_NET_VENDOR_MELLANOX is not set
-# CONFIG_NET_VENDOR_MICREL is not set
-# CONFIG_NET_VENDOR_MYRI is not set
-# CONFIG_NET_VENDOR_NATSEMI is not set
-# CONFIG_NET_VENDOR_NVIDIA is not set
-# CONFIG_NET_VENDOR_OKI is not set
-# CONFIG_NET_PACKET_ENGINE is not set
-# CONFIG_NET_VENDOR_QLOGIC is not set
-# CONFIG_NET_VENDOR_REALTEK is not set
-# CONFIG_NET_VENDOR_RDC is not set
-# CONFIG_NET_VENDOR_SEEQ is not set
-# CONFIG_NET_VENDOR_SILAN is not set
-# CONFIG_NET_VENDOR_SIS is not set
-# CONFIG_NET_VENDOR_SMSC is not set
-# CONFIG_NET_VENDOR_STMICRO is not set
-# CONFIG_NET_VENDOR_SUN is not set
-# CONFIG_NET_VENDOR_TEHUTI is not set
-# CONFIG_NET_VENDOR_TI is not set
-# CONFIG_NET_VENDOR_TOSHIBA is not set
-# CONFIG_NET_VENDOR_VIA is not set
-# CONFIG_WLAN is not set
-# CONFIG_VT is not set
-CONFIG_LEGACY_PTY_COUNT=16
-CONFIG_SERIAL_8250=y
-CONFIG_SERIAL_8250_CONSOLE=y
-CONFIG_HW_RANDOM=y
-# CONFIG_HWMON is not set
-CONFIG_VIDEO_OUTPUT_CONTROL=m
-CONFIG_FB=y
-CONFIG_FIRMWARE_EDID=y
-CONFIG_FB_MATROX=y
-CONFIG_FB_MATROX_G=y
-CONFIG_USB=y
-CONFIG_USB_EHCI_HCD=y
-# CONFIG_USB_EHCI_TT_NEWSCHED is not set
-CONFIG_USB_UHCI_HCD=y
-CONFIG_USB_STORAGE=y
-CONFIG_NEW_LEDS=y
-CONFIG_LEDS_CLASS=y
-CONFIG_LEDS_TRIGGERS=y
-CONFIG_LEDS_TRIGGER_TIMER=y
-CONFIG_LEDS_TRIGGER_IDE_DISK=y
-CONFIG_LEDS_TRIGGER_HEARTBEAT=y
-CONFIG_LEDS_TRIGGER_BACKLIGHT=y
-CONFIG_LEDS_TRIGGER_DEFAULT_ON=y
-CONFIG_RTC_CLASS=y
-CONFIG_RTC_DRV_CMOS=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-CONFIG_XFS_FS=y
-CONFIG_XFS_QUOTA=y
-CONFIG_XFS_POSIX_ACL=y
-CONFIG_QUOTA=y
-CONFIG_QFMT_V2=y
-CONFIG_MSDOS_FS=m
-CONFIG_VFAT_FS=m
-CONFIG_PROC_KCORE=y
-CONFIG_TMPFS=y
-CONFIG_NFS_FS=y
-CONFIG_ROOT_NFS=y
-CONFIG_CIFS=m
-CONFIG_CIFS_WEAK_PW_HASH=y
-CONFIG_CIFS_XATTR=y
-CONFIG_CIFS_POSIX=y
-CONFIG_NLS_CODEPAGE_437=m
-CONFIG_NLS_ISO8859_1=m
-# CONFIG_FTRACE is not set
-CONFIG_CRYPTO_NULL=m
-CONFIG_CRYPTO_PCBC=m
-CONFIG_CRYPTO_HMAC=y
-CONFIG_CRYPTO_MICHAEL_MIC=m
-CONFIG_CRYPTO_SHA512=m
-CONFIG_CRYPTO_TGR192=m
-CONFIG_CRYPTO_WP512=m
-CONFIG_CRYPTO_ANUBIS=m
-CONFIG_CRYPTO_BLOWFISH=m
-CONFIG_CRYPTO_CAST5=m
-CONFIG_CRYPTO_CAST6=m
-CONFIG_CRYPTO_KHAZAD=m
-CONFIG_CRYPTO_SERPENT=m
-CONFIG_CRYPTO_TEA=m
-CONFIG_CRYPTO_TWOFISH=m
-# CONFIG_CRYPTO_ANSI_CPRNG is not set
-# CONFIG_CRYPTO_HW is not set
diff --git a/arch/mips/configs/maltasmvp_defconfig b/arch/mips/configs/maltasmvp_defconfig
index 10ef3be..f8a3231 100644
--- a/arch/mips/configs/maltasmvp_defconfig
+++ b/arch/mips/configs/maltasmvp_defconfig
@@ -4,10 +4,9 @@
CONFIG_PAGE_SIZE_16KB=y
CONFIG_MIPS_MT_SMP=y
CONFIG_SCHED_SMT=y
-CONFIG_MIPS_CMP=y
+CONFIG_MIPS_CPS=y
CONFIG_NR_CPUS=8
CONFIG_HZ_100=y
-CONFIG_LOCALVERSION="cmp"
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
CONFIG_AUDIT=y
diff --git a/arch/mips/configs/maltasmvp_eva_defconfig b/arch/mips/configs/maltasmvp_eva_defconfig
index 2d3002c..c83338a 100644
--- a/arch/mips/configs/maltasmvp_eva_defconfig
+++ b/arch/mips/configs/maltasmvp_eva_defconfig
@@ -5,10 +5,9 @@
CONFIG_PAGE_SIZE_16KB=y
CONFIG_MIPS_MT_SMP=y
CONFIG_SCHED_SMT=y
-CONFIG_MIPS_CMP=y
+CONFIG_MIPS_CPS=y
CONFIG_NR_CPUS=8
CONFIG_HZ_100=y
-CONFIG_LOCALVERSION="cmp"
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
CONFIG_AUDIT=y
diff --git a/arch/mips/configs/mips_paravirt_defconfig b/arch/mips/configs/mips_paravirt_defconfig
new file mode 100644
index 0000000..84cfcb4
--- /dev/null
+++ b/arch/mips/configs/mips_paravirt_defconfig
@@ -0,0 +1,103 @@
+CONFIG_MIPS_PARAVIRT=y
+CONFIG_CPU_MIPS64_R2=y
+CONFIG_64BIT=y
+CONFIG_TRANSPARENT_HUGEPAGE=y
+CONFIG_SMP=y
+CONFIG_HZ_1000=y
+CONFIG_PREEMPT=y
+CONFIG_SYSVIPC=y
+CONFIG_BSD_PROCESS_ACCT=y
+CONFIG_BSD_PROCESS_ACCT_V3=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_LOG_BUF_SHIFT=14
+CONFIG_RELAY=y
+CONFIG_BLK_DEV_INITRD=y
+CONFIG_EXPERT=y
+CONFIG_SLAB=y
+CONFIG_MODULES=y
+CONFIG_MODULE_UNLOAD=y
+# CONFIG_BLK_DEV_BSG is not set
+CONFIG_PCI=y
+CONFIG_MIPS32_COMPAT=y
+CONFIG_MIPS32_O32=y
+CONFIG_MIPS32_N32=y
+CONFIG_NET=y
+CONFIG_PACKET=y
+CONFIG_UNIX=y
+CONFIG_INET=y
+CONFIG_IP_MULTICAST=y
+CONFIG_IP_ADVANCED_ROUTER=y
+CONFIG_IP_MULTIPLE_TABLES=y
+CONFIG_IP_ROUTE_MULTIPATH=y
+CONFIG_IP_ROUTE_VERBOSE=y
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+CONFIG_IP_PNP_BOOTP=y
+CONFIG_IP_PNP_RARP=y
+CONFIG_IP_MROUTE=y
+CONFIG_IP_PIMSM_V1=y
+CONFIG_IP_PIMSM_V2=y
+CONFIG_SYN_COOKIES=y
+# CONFIG_INET_LRO is not set
+CONFIG_IPV6=y
+# CONFIG_WIRELESS is not set
+CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+# CONFIG_FW_LOADER is not set
+CONFIG_BLK_DEV_LOOP=y
+CONFIG_VIRTIO_BLK=y
+CONFIG_SCSI=y
+CONFIG_BLK_DEV_SD=y
+CONFIG_NETDEVICES=y
+CONFIG_VIRTIO_NET=y
+# CONFIG_NET_VENDOR_BROADCOM is not set
+# CONFIG_NET_VENDOR_INTEL is not set
+# CONFIG_NET_VENDOR_MARVELL is not set
+# CONFIG_NET_VENDOR_MICREL is not set
+# CONFIG_NET_VENDOR_NATSEMI is not set
+# CONFIG_NET_VENDOR_SMSC is not set
+# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_WIZNET is not set
+CONFIG_PHYLIB=y
+CONFIG_MARVELL_PHY=y
+CONFIG_BROADCOM_PHY=y
+CONFIG_BCM87XX_PHY=y
+# CONFIG_WLAN is not set
+# CONFIG_INPUT is not set
+# CONFIG_SERIO is not set
+# CONFIG_VT is not set
+CONFIG_VIRTIO_CONSOLE=y
+# CONFIG_HW_RANDOM is not set
+# CONFIG_HWMON is not set
+# CONFIG_USB_SUPPORT is not set
+CONFIG_VIRTIO_PCI=y
+CONFIG_VIRTIO_BALLOON=y
+CONFIG_VIRTIO_MMIO=y
+# CONFIG_IOMMU_SUPPORT is not set
+CONFIG_EXT4_FS=y
+CONFIG_EXT4_FS_POSIX_ACL=y
+CONFIG_EXT4_FS_SECURITY=y
+CONFIG_MSDOS_FS=y
+CONFIG_VFAT_FS=y
+CONFIG_PROC_KCORE=y
+CONFIG_TMPFS=y
+CONFIG_HUGETLBFS=y
+# CONFIG_MISC_FILESYSTEMS is not set
+CONFIG_NFS_FS=y
+CONFIG_NFS_V4=y
+CONFIG_NFS_V4_1=y
+CONFIG_ROOT_NFS=y
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_ASCII=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_NLS_UTF8=y
+CONFIG_DEBUG_INFO=y
+CONFIG_DEBUG_FS=y
+CONFIG_MAGIC_SYSRQ=y
+# CONFIG_SCHED_DEBUG is not set
+# CONFIG_FTRACE is not set
+CONFIG_CRYPTO_CBC=y
+CONFIG_CRYPTO_HMAC=y
+CONFIG_CRYPTO_MD5=y
+CONFIG_CRYPTO_DES=y
+# CONFIG_CRYPTO_ANSI_CPRNG is not set
diff --git a/arch/mips/configs/rt305x_defconfig b/arch/mips/configs/rt305x_defconfig
index d1741bc..d14ae2f 100644
--- a/arch/mips/configs/rt305x_defconfig
+++ b/arch/mips/configs/rt305x_defconfig
@@ -81,7 +81,6 @@
# CONFIG_FIRMWARE_IN_KERNEL is not set
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
-CONFIG_MTD_CHAR=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
CONFIG_MTD_CFI_AMDSTD=y
@@ -89,6 +88,7 @@
CONFIG_MTD_PHYSMAP=y
CONFIG_MTD_PHYSMAP_OF=y
CONFIG_MTD_M25P80=y
+CONFIG_MTD_SPI_NOR=y
CONFIG_EEPROM_93CX6=m
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
diff --git a/arch/mips/dec/Makefile b/arch/mips/dec/Makefile
index 3d5d2c5..bd74e05 100644
--- a/arch/mips/dec/Makefile
+++ b/arch/mips/dec/Makefile
@@ -3,7 +3,7 @@
#
obj-y := ecc-berr.o int-handler.o ioasic-irq.o kn01-berr.o \
- kn02-irq.o kn02xa-berr.o reset.o setup.o time.o
+ kn02-irq.o kn02xa-berr.o platform.o reset.o setup.o time.o
obj-$(CONFIG_TC) += tc.o
obj-$(CONFIG_CPU_HAS_WB) += wbflush.o
diff --git a/arch/mips/dec/platform.c b/arch/mips/dec/platform.c
new file mode 100644
index 0000000..c7ac86a
--- /dev/null
+++ b/arch/mips/dec/platform.c
@@ -0,0 +1,44 @@
+/*
+ * DEC platform devices.
+ *
+ * Copyright (c) 2014 Maciej W. Rozycki
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/ioport.h>
+#include <linux/kernel.h>
+#include <linux/mc146818rtc.h>
+#include <linux/platform_device.h>
+
+static struct resource dec_rtc_resources[] = {
+ {
+ .name = "rtc",
+ .flags = IORESOURCE_MEM,
+ },
+};
+
+static struct cmos_rtc_board_info dec_rtc_info = {
+ .flags = CMOS_RTC_FLAGS_NOFREQ,
+ .address_space = 64,
+};
+
+static struct platform_device dec_rtc_device = {
+ .name = "rtc_cmos",
+ .id = PLATFORM_DEVID_NONE,
+ .dev.platform_data = &dec_rtc_info,
+ .resource = dec_rtc_resources,
+ .num_resources = ARRAY_SIZE(dec_rtc_resources),
+};
+
+static int __init dec_add_devices(void)
+{
+ dec_rtc_resources[0].start = RTC_PORT(0);
+ dec_rtc_resources[0].end = RTC_PORT(0) + dec_kn_slot_size - 1;
+ return platform_device_register(&dec_rtc_device);
+}
+
+device_initcall(dec_add_devices);
diff --git a/arch/mips/dec/setup.c b/arch/mips/dec/setup.c
index 56e6e2c2..41bbffd 100644
--- a/arch/mips/dec/setup.c
+++ b/arch/mips/dec/setup.c
@@ -23,6 +23,7 @@
#include <asm/bootinfo.h>
#include <asm/cpu.h>
#include <asm/cpu-features.h>
+#include <asm/cpu-type.h>
#include <asm/irq.h>
#include <asm/irq_cpu.h>
#include <asm/mipsregs.h>
@@ -748,6 +749,10 @@
cpu_fpu_mask = 0;
dec_interrupt[DEC_IRQ_FPU] = -1;
}
+ /* Free the halt interrupt unused on R4k systems. */
+ if (current_cpu_type() == CPU_R4000SC ||
+ current_cpu_type() == CPU_R4400SC)
+ dec_interrupt[DEC_IRQ_HALT] = -1;
/* Register board interrupts: FPU and cascade. */
if (dec_interrupt[DEC_IRQ_FPU] >= 0)
diff --git a/arch/mips/include/asm/asmmacro.h b/arch/mips/include/asm/asmmacro.h
index b464b8b..935543f 100644
--- a/arch/mips/include/asm/asmmacro.h
+++ b/arch/mips/include/asm/asmmacro.h
@@ -17,26 +17,8 @@
#ifdef CONFIG_64BIT
#include <asm/asmmacro-64.h>
#endif
-#ifdef CONFIG_MIPS_MT_SMTC
-#include <asm/mipsmtregs.h>
-#endif
-#ifdef CONFIG_MIPS_MT_SMTC
- .macro local_irq_enable reg=t0
- mfc0 \reg, CP0_TCSTATUS
- ori \reg, \reg, TCSTATUS_IXMT
- xori \reg, \reg, TCSTATUS_IXMT
- mtc0 \reg, CP0_TCSTATUS
- _ehb
- .endm
-
- .macro local_irq_disable reg=t0
- mfc0 \reg, CP0_TCSTATUS
- ori \reg, \reg, TCSTATUS_IXMT
- mtc0 \reg, CP0_TCSTATUS
- _ehb
- .endm
-#elif defined(CONFIG_CPU_MIPSR2)
+#ifdef CONFIG_CPU_MIPSR2
.macro local_irq_enable reg=t0
ei
irq_enable_hazard
@@ -71,7 +53,7 @@
sw \reg, TI_PRE_COUNT($28)
#endif
.endm
-#endif /* CONFIG_MIPS_MT_SMTC */
+#endif /* CONFIG_CPU_MIPSR2 */
.macro fpu_save_16even thread tmp=t0
cfc1 \tmp, fcr31
@@ -267,13 +249,35 @@
.set pop
.endm
#else
+
+#ifdef CONFIG_CPU_MICROMIPS
+#define CFC_MSA_INSN 0x587e0056
+#define CTC_MSA_INSN 0x583e0816
+#define LDD_MSA_INSN 0x58000837
+#define STD_MSA_INSN 0x5800083f
+#define COPY_UW_MSA_INSN 0x58f00056
+#define COPY_UD_MSA_INSN 0x58f80056
+#define INSERT_W_MSA_INSN 0x59300816
+#define INSERT_D_MSA_INSN 0x59380816
+#else
+#define CFC_MSA_INSN 0x787e0059
+#define CTC_MSA_INSN 0x783e0819
+#define LDD_MSA_INSN 0x78000823
+#define STD_MSA_INSN 0x78000827
+#define COPY_UW_MSA_INSN 0x78f00059
+#define COPY_UD_MSA_INSN 0x78f80059
+#define INSERT_W_MSA_INSN 0x79300819
+#define INSERT_D_MSA_INSN 0x79380819
+#endif
+
/*
* Temporary until all toolchains in use include MSA support.
*/
.macro cfcmsa rd, cs
.set push
.set noat
- .word 0x787e0059 | (\cs << 11)
+ .insn
+ .word CFC_MSA_INSN | (\cs << 11)
move \rd, $1
.set pop
.endm
@@ -282,7 +286,7 @@
.set push
.set noat
move $1, \rs
- .word 0x783e0819 | (\cd << 6)
+ .word CTC_MSA_INSN | (\cd << 6)
.set pop
.endm
@@ -290,7 +294,7 @@
.set push
.set noat
add $1, \base, \off
- .word 0x78000823 | (\wd << 6)
+ .word LDD_MSA_INSN | (\wd << 6)
.set pop
.endm
@@ -298,14 +302,15 @@
.set push
.set noat
add $1, \base, \off
- .word 0x78000827 | (\wd << 6)
+ .word STD_MSA_INSN | (\wd << 6)
.set pop
.endm
.macro copy_u_w rd, ws, n
.set push
.set noat
- .word 0x78f00059 | (\n << 16) | (\ws << 11)
+ .insn
+ .word COPY_UW_MSA_INSN | (\n << 16) | (\ws << 11)
/* move triggers an assembler bug... */
or \rd, $1, zero
.set pop
@@ -314,7 +319,8 @@
.macro copy_u_d rd, ws, n
.set push
.set noat
- .word 0x78f80059 | (\n << 16) | (\ws << 11)
+ .insn
+ .word COPY_UD_MSA_INSN | (\n << 16) | (\ws << 11)
/* move triggers an assembler bug... */
or \rd, $1, zero
.set pop
@@ -325,7 +331,7 @@
.set noat
/* move triggers an assembler bug... */
or $1, \rs, zero
- .word 0x79300819 | (\n << 16) | (\wd << 6)
+ .word INSERT_W_MSA_INSN | (\n << 16) | (\wd << 6)
.set pop
.endm
@@ -334,7 +340,7 @@
.set noat
/* move triggers an assembler bug... */
or $1, \rs, zero
- .word 0x79380819 | (\n << 16) | (\wd << 6)
+ .word INSERT_D_MSA_INSN | (\n << 16) | (\wd << 6)
.set pop
.endm
#endif
diff --git a/arch/mips/include/asm/branch.h b/arch/mips/include/asm/branch.h
index e28a3e0..de781cf 100644
--- a/arch/mips/include/asm/branch.h
+++ b/arch/mips/include/asm/branch.h
@@ -8,6 +8,8 @@
#ifndef _ASM_BRANCH_H
#define _ASM_BRANCH_H
+#include <asm/cpu-features.h>
+#include <asm/mipsregs.h>
#include <asm/ptrace.h>
#include <asm/inst.h>
@@ -18,12 +20,40 @@
extern int __microMIPS_compute_return_epc(struct pt_regs *regs);
extern int __MIPS16e_compute_return_epc(struct pt_regs *regs);
+/*
+ * microMIPS bitfields
+ */
+#define MM_POOL32A_MINOR_MASK 0x3f
+#define MM_POOL32A_MINOR_SHIFT 0x6
+#define MM_MIPS32_COND_FC 0x30
+
+extern int __mm_isBranchInstr(struct pt_regs *regs,
+ struct mm_decoded_insn dec_insn, unsigned long *contpc);
+
+static inline int mm_isBranchInstr(struct pt_regs *regs,
+ struct mm_decoded_insn dec_insn, unsigned long *contpc)
+{
+ if (!cpu_has_mmips)
+ return 0;
+
+ return __mm_isBranchInstr(regs, dec_insn, contpc);
+}
static inline int delay_slot(struct pt_regs *regs)
{
return regs->cp0_cause & CAUSEF_BD;
}
+static inline void clear_delay_slot(struct pt_regs *regs)
+{
+ regs->cp0_cause &= ~CAUSEF_BD;
+}
+
+static inline void set_delay_slot(struct pt_regs *regs)
+{
+ regs->cp0_cause |= CAUSEF_BD;
+}
+
static inline unsigned long exception_epc(struct pt_regs *regs)
{
if (likely(!delay_slot(regs)))
diff --git a/arch/mips/include/asm/cacheflush.h b/arch/mips/include/asm/cacheflush.h
index 69468de..e08381a 100644
--- a/arch/mips/include/asm/cacheflush.h
+++ b/arch/mips/include/asm/cacheflush.h
@@ -113,6 +113,12 @@
extern void *kmap_coherent(struct page *page, unsigned long addr);
extern void kunmap_coherent(void);
+extern void *kmap_noncoherent(struct page *page, unsigned long addr);
+
+static inline void kunmap_noncoherent(void)
+{
+ kunmap_coherent();
+}
#define ARCH_HAS_FLUSH_KERNEL_DCACHE_PAGE
static inline void flush_kernel_dcache_page(struct page *page)
diff --git a/arch/mips/include/asm/cmp.h b/arch/mips/include/asm/cmp.h
index 89a73fb..033d973 100644
--- a/arch/mips/include/asm/cmp.h
+++ b/arch/mips/include/asm/cmp.h
@@ -10,7 +10,6 @@
extern void cmp_smp_finish(void);
extern void cmp_boot_secondary(int cpu, struct task_struct *t);
extern void cmp_init_secondary(void);
-extern void cmp_cpus_done(void);
extern void cmp_prepare_cpus(unsigned int max_cpus);
/* This is platform specific */
diff --git a/arch/mips/include/asm/cpu-features.h b/arch/mips/include/asm/cpu-features.h
index f56cc97..c7d8c99 100644
--- a/arch/mips/include/asm/cpu-features.h
+++ b/arch/mips/include/asm/cpu-features.h
@@ -110,9 +110,15 @@
#ifndef cpu_has_smartmips
#define cpu_has_smartmips (cpu_data[0].ases & MIPS_ASE_SMARTMIPS)
#endif
+
#ifndef cpu_has_rixi
-#define cpu_has_rixi (cpu_data[0].options & MIPS_CPU_RIXI)
+# ifdef CONFIG_64BIT
+# define cpu_has_rixi (cpu_data[0].options & MIPS_CPU_RIXI)
+# else /* CONFIG_32BIT */
+# define cpu_has_rixi ((cpu_data[0].options & MIPS_CPU_RIXI) && !cpu_has_64bits)
+# endif
#endif
+
#ifndef cpu_has_mmips
# ifdef CONFIG_SYS_SUPPORTS_MICROMIPS
# define cpu_has_mmips (cpu_data[0].options & MIPS_CPU_MICROMIPS)
@@ -120,6 +126,7 @@
# define cpu_has_mmips 0
# endif
#endif
+
#ifndef cpu_has_vtag_icache
#define cpu_has_vtag_icache (cpu_data[0].icache.flags & MIPS_CACHE_VTAG)
#endif
@@ -183,6 +190,17 @@
/*
* Shortcuts ...
*/
+#define cpu_has_mips_2_3_4_5 (cpu_has_mips_2 | cpu_has_mips_3_4_5)
+#define cpu_has_mips_3_4_5 (cpu_has_mips_3 | cpu_has_mips_4_5)
+#define cpu_has_mips_4_5 (cpu_has_mips_4 | cpu_has_mips_5)
+
+#define cpu_has_mips_2_3_4_5_r (cpu_has_mips_2 | cpu_has_mips_3_4_5_r)
+#define cpu_has_mips_3_4_5_r (cpu_has_mips_3 | cpu_has_mips_4_5_r)
+#define cpu_has_mips_4_5_r (cpu_has_mips_4 | cpu_has_mips_5_r)
+#define cpu_has_mips_5_r (cpu_has_mips_5 | cpu_has_mips_r)
+
+#define cpu_has_mips_4_5_r2 (cpu_has_mips_4_5 | cpu_has_mips_r2)
+
#define cpu_has_mips32 (cpu_has_mips32r1 | cpu_has_mips32r2)
#define cpu_has_mips64 (cpu_has_mips64r1 | cpu_has_mips64r2)
#define cpu_has_mips_r1 (cpu_has_mips32r1 | cpu_has_mips64r1)
diff --git a/arch/mips/include/asm/cpu-info.h b/arch/mips/include/asm/cpu-info.h
index ff2707ab..47d5967 100644
--- a/arch/mips/include/asm/cpu-info.h
+++ b/arch/mips/include/asm/cpu-info.h
@@ -65,18 +65,13 @@
#ifdef CONFIG_64BIT
int vmbits; /* Virtual memory size in bits */
#endif
-#if defined(CONFIG_MIPS_MT_SMP) || defined(CONFIG_MIPS_MT_SMTC)
+#ifdef CONFIG_MIPS_MT_SMP
/*
- * In the MIPS MT "SMTC" model, each TC is considered
- * to be a "CPU" for the purposes of scheduling, but
- * exception resources, ASID spaces, etc, are common
- * to all TCs within the same VPE.
+ * There is not necessarily a 1:1 mapping of VPE num to CPU number
+ * in particular on multi-core systems.
*/
int vpe_id; /* Virtual Processor number */
#endif
-#ifdef CONFIG_MIPS_MT_SMTC
- int tc_id; /* Thread Context number */
-#endif
void *data; /* Additional data */
unsigned int watch_reg_count; /* Number that exist */
unsigned int watch_reg_use_cnt; /* Usable by ptrace */
@@ -117,7 +112,7 @@
unsigned long n;
};
-#if defined(CONFIG_MIPS_MT_SMP) || defined(CONFIG_MIPS_MT_SMTC)
+#ifdef CONFIG_MIPS_MT_SMP
# define cpu_vpe_id(cpuinfo) ((cpuinfo)->vpe_id)
#else
# define cpu_vpe_id(cpuinfo) 0
diff --git a/arch/mips/include/asm/cpu-type.h b/arch/mips/include/asm/cpu-type.h
index 72190613..b4e2bd8 100644
--- a/arch/mips/include/asm/cpu-type.h
+++ b/arch/mips/include/asm/cpu-type.h
@@ -155,9 +155,6 @@
case CPU_RM7000:
case CPU_SR71000:
#endif
-#ifdef CONFIG_SYS_HAS_CPU_RM9000
- case CPU_RM9000:
-#endif
#ifdef CONFIG_SYS_HAS_CPU_SB1
case CPU_SB1:
case CPU_SB1A:
@@ -166,6 +163,7 @@
case CPU_CAVIUM_OCTEON:
case CPU_CAVIUM_OCTEON_PLUS:
case CPU_CAVIUM_OCTEON2:
+ case CPU_CAVIUM_OCTEON3:
#endif
#if defined(CONFIG_SYS_HAS_CPU_BMIPS32_3300) || \
diff --git a/arch/mips/include/asm/cpu.h b/arch/mips/include/asm/cpu.h
index 530eb8b..129d087 100644
--- a/arch/mips/include/asm/cpu.h
+++ b/arch/mips/include/asm/cpu.h
@@ -201,6 +201,7 @@
#define PRID_IMP_NETLOGIC_XLP3XX 0x1100
#define PRID_IMP_NETLOGIC_XLP2XX 0x1200
#define PRID_IMP_NETLOGIC_XLP9XX 0x1500
+#define PRID_IMP_NETLOGIC_XLP5XX 0x1300
/*
* Particular Revision values for bits 7:0 of the PRId register.
@@ -281,7 +282,7 @@
CPU_R4700, CPU_R5000, CPU_R5500, CPU_NEVADA, CPU_R5432, CPU_R10000,
CPU_R12000, CPU_R14000, CPU_VR41XX, CPU_VR4111, CPU_VR4121, CPU_VR4122,
CPU_VR4131, CPU_VR4133, CPU_VR4181, CPU_VR4181A, CPU_RM7000,
- CPU_SR71000, CPU_RM9000, CPU_TX49XX,
+ CPU_SR71000, CPU_TX49XX,
/*
* R8000 class processors
diff --git a/arch/mips/include/asm/dec/kn05.h b/arch/mips/include/asm/dec/kn05.h
index 56d22dc..8e14f67 100644
--- a/arch/mips/include/asm/dec/kn05.h
+++ b/arch/mips/include/asm/dec/kn05.h
@@ -49,12 +49,20 @@
#define KN4K_RES_15 (15*IOASIC_SLOT_SIZE) /* unused? */
/*
+ * MB ASIC interrupt bits.
+ */
+#define KN4K_MB_INR_MB 4 /* ??? */
+#define KN4K_MB_INR_MT 3 /* memory, I/O bus read/write errors */
+#define KN4K_MB_INR_RES_2 2 /* unused */
+#define KN4K_MB_INR_RTC 1 /* RTC */
+#define KN4K_MB_INR_TC 0 /* I/O ASIC cascade */
+
+/*
* Bits for the MB interrupt register.
* The register appears read-only.
*/
-#define KN4K_MB_INT_TC (1<<0) /* TURBOchannel? */
-#define KN4K_MB_INT_RTC (1<<1) /* RTC? */
-#define KN4K_MB_INT_MT (1<<3) /* I/O ASIC cascade */
+#define KN4K_MB_INT_IRQ (0x1f<<0) /* CPU Int[4:0] status. */
+#define KN4K_MB_INT_IRQ_N(n) (1<<(n)) /* Individual status bits. */
/*
* Bits for the MB control & status register.
@@ -70,6 +78,7 @@
#define KN4K_MB_CSR_NC (1<<14) /* ??? */
#define KN4K_MB_CSR_EE (1<<15) /* (bus) Exception Enable? */
#define KN4K_MB_CSR_MSK (0x1f<<16) /* CPU Int[4:0] mask */
+#define KN4K_MB_CSR_MSK_N(n) (1<<((n)+16)) /* Individual mask bits. */
#define KN4K_MB_CSR_FW (1<<21) /* ??? */
#define KN4K_MB_CSR_W (1<<31) /* ??? */
diff --git a/arch/mips/include/asm/fixmap.h b/arch/mips/include/asm/fixmap.h
index 8c012af..6842ffa 100644
--- a/arch/mips/include/asm/fixmap.h
+++ b/arch/mips/include/asm/fixmap.h
@@ -48,11 +48,7 @@
enum fixed_addresses {
#define FIX_N_COLOURS 8
FIX_CMAP_BEGIN,
-#ifdef CONFIG_MIPS_MT_SMTC
- FIX_CMAP_END = FIX_CMAP_BEGIN + (FIX_N_COLOURS * NR_CPUS * 2),
-#else
FIX_CMAP_END = FIX_CMAP_BEGIN + (FIX_N_COLOURS * 2),
-#endif
#ifdef CONFIG_HIGHMEM
/* reserved pte's for temporary kernel mappings */
FIX_KMAP_BEGIN = FIX_CMAP_END + 1,
diff --git a/arch/mips/include/asm/fpu.h b/arch/mips/include/asm/fpu.h
index 4d86b72..a939574 100644
--- a/arch/mips/include/asm/fpu.h
+++ b/arch/mips/include/asm/fpu.h
@@ -17,6 +17,7 @@
#include <asm/mipsregs.h>
#include <asm/cpu.h>
#include <asm/cpu-features.h>
+#include <asm/fpu_emulator.h>
#include <asm/hazards.h>
#include <asm/processor.h>
#include <asm/current.h>
@@ -28,7 +29,6 @@
struct sigcontext;
struct sigcontext32;
-extern void fpu_emulator_init_fpu(void);
extern void _init_fpu(void);
extern void _save_fp(struct task_struct *);
extern void _restore_fp(struct task_struct *);
@@ -156,15 +156,16 @@
int ret = 0;
preempt_disable();
+
if (cpu_has_fpu) {
ret = __own_fpu();
if (!ret)
_init_fpu();
- } else {
+ } else
fpu_emulator_init_fpu();
- }
preempt_enable();
+
return ret;
}
diff --git a/arch/mips/include/asm/fpu_emulator.h b/arch/mips/include/asm/fpu_emulator.h
index 2abb587..0195745 100644
--- a/arch/mips/include/asm/fpu_emulator.h
+++ b/arch/mips/include/asm/fpu_emulator.h
@@ -23,9 +23,12 @@
#ifndef _ASM_FPU_EMULATOR_H
#define _ASM_FPU_EMULATOR_H
+#include <linux/sched.h>
#include <asm/break.h>
+#include <asm/thread_info.h>
#include <asm/inst.h>
#include <asm/local.h>
+#include <asm/processor.h>
#ifdef CONFIG_DEBUG_FS
@@ -36,6 +39,11 @@
local_t cp1ops;
local_t cp1xops;
local_t errors;
+ local_t ieee754_inexact;
+ local_t ieee754_underflow;
+ local_t ieee754_overflow;
+ local_t ieee754_zerodiv;
+ local_t ieee754_invalidop;
};
DECLARE_PER_CPU(struct mips_fpu_emulator_stats, fpuemustats);
@@ -71,4 +79,17 @@
*/
#define BREAK_MATH (0x0000000d | (BRK_MEMU << 16))
+#define SIGNALLING_NAN 0x7ff800007ff80000LL
+
+static inline void fpu_emulator_init_fpu(void)
+{
+ struct task_struct *t = current;
+ int i;
+
+ t->thread.fpu.fcr31 = 0;
+
+ for (i = 0; i < 32; i++)
+ set_fpr64(&t->thread.fpu.fpr[i], 0, SIGNALLING_NAN);
+}
+
#endif /* _ASM_FPU_EMULATOR_H */
diff --git a/arch/mips/include/asm/gic.h b/arch/mips/include/asm/gic.h
index 0827166..10f6a99 100644
--- a/arch/mips/include/asm/gic.h
+++ b/arch/mips/include/asm/gic.h
@@ -380,6 +380,7 @@
extern cycle_t gic_read_count(void);
extern cycle_t gic_read_compare(void);
extern void gic_write_compare(cycle_t cnt);
+extern void gic_write_cpu_compare(cycle_t cnt, int cpu);
extern void gic_send_ipi(unsigned int intr);
extern unsigned int plat_ipi_call_int_xlate(unsigned int);
extern unsigned int plat_ipi_resched_int_xlate(unsigned int);
diff --git a/arch/mips/include/asm/gio_device.h b/arch/mips/include/asm/gio_device.h
index 0878701..4be1a57 100644
--- a/arch/mips/include/asm/gio_device.h
+++ b/arch/mips/include/asm/gio_device.h
@@ -50,7 +50,7 @@
extern int gio_register_driver(struct gio_driver *);
extern void gio_unregister_driver(struct gio_driver *);
-#define gio_get_drvdata(_dev) drv_get_drvdata(&(_dev)->dev)
-#define gio_set_drvdata(_dev, data) drv_set_drvdata(&(_dev)->dev, (data))
+#define gio_get_drvdata(_dev) dev_get_drvdata(&(_dev)->dev)
+#define gio_set_drvdata(_dev, data) dev_set_drvdata(&(_dev)->dev, (data))
extern void gio_set_master(struct gio_device *);
diff --git a/arch/mips/include/asm/idle.h b/arch/mips/include/asm/idle.h
index d192158..d9f932d 100644
--- a/arch/mips/include/asm/idle.h
+++ b/arch/mips/include/asm/idle.h
@@ -1,6 +1,7 @@
#ifndef __ASM_IDLE_H
#define __ASM_IDLE_H
+#include <linux/cpuidle.h>
#include <linux/linkage.h>
extern void (*cpu_wait)(void);
@@ -20,4 +21,17 @@
addr < (unsigned long)__pastwait;
}
+extern int mips_cpuidle_wait_enter(struct cpuidle_device *dev,
+ struct cpuidle_driver *drv, int index);
+
+#define MIPS_CPUIDLE_WAIT_STATE {\
+ .enter = mips_cpuidle_wait_enter,\
+ .exit_latency = 1,\
+ .target_residency = 1,\
+ .power_usage = UINT_MAX,\
+ .flags = CPUIDLE_FLAG_TIME_VALID,\
+ .name = "wait",\
+ .desc = "MIPS wait",\
+}
+
#endif /* __ASM_IDLE_H */
diff --git a/arch/mips/include/asm/irq.h b/arch/mips/include/asm/irq.h
index 7bc2cdb..ae1f7b2 100644
--- a/arch/mips/include/asm/irq.h
+++ b/arch/mips/include/asm/irq.h
@@ -26,104 +26,8 @@
#define irq_canonicalize(irq) (irq) /* Sane hardware, sane code ... */
#endif
-#ifdef CONFIG_MIPS_MT_SMTC
-
-struct irqaction;
-
-extern unsigned long irq_hwmask[];
-extern int setup_irq_smtc(unsigned int irq, struct irqaction * new,
- unsigned long hwmask);
-
-static inline void smtc_im_ack_irq(unsigned int irq)
-{
- if (irq_hwmask[irq] & ST0_IM)
- set_c0_status(irq_hwmask[irq] & ST0_IM);
-}
-
-#else
-
-static inline void smtc_im_ack_irq(unsigned int irq)
-{
-}
-
-#endif /* CONFIG_MIPS_MT_SMTC */
-
-#ifdef CONFIG_MIPS_MT_SMTC_IRQAFF
-#include <linux/cpumask.h>
-
-extern int plat_set_irq_affinity(struct irq_data *d,
- const struct cpumask *affinity, bool force);
-extern void smtc_forward_irq(struct irq_data *d);
-
-/*
- * IRQ affinity hook invoked at the beginning of interrupt dispatch
- * if option is enabled.
- *
- * Up through Linux 2.6.22 (at least) cpumask operations are very
- * inefficient on MIPS. Initial prototypes of SMTC IRQ affinity
- * used a "fast path" per-IRQ-descriptor cache of affinity information
- * to reduce latency. As there is a project afoot to optimize the
- * cpumask implementations, this version is optimistically assuming
- * that cpumask.h macro overhead is reasonable during interrupt dispatch.
- */
-static inline int handle_on_other_cpu(unsigned int irq)
-{
- struct irq_data *d = irq_get_irq_data(irq);
-
- if (cpumask_test_cpu(smp_processor_id(), d->affinity))
- return 0;
- smtc_forward_irq(d);
- return 1;
-}
-
-#else /* Not doing SMTC affinity */
-
-static inline int handle_on_other_cpu(unsigned int irq) { return 0; }
-
-#endif /* CONFIG_MIPS_MT_SMTC_IRQAFF */
-
-#ifdef CONFIG_MIPS_MT_SMTC_IM_BACKSTOP
-
-static inline void smtc_im_backstop(unsigned int irq)
-{
- if (irq_hwmask[irq] & 0x0000ff00)
- write_c0_tccontext(read_c0_tccontext() &
- ~(irq_hwmask[irq] & 0x0000ff00));
-}
-
-/*
- * Clear interrupt mask handling "backstop" if irq_hwmask
- * entry so indicates. This implies that the ack() or end()
- * functions will take over re-enabling the low-level mask.
- * Otherwise it will be done on return from exception.
- */
-static inline int smtc_handle_on_other_cpu(unsigned int irq)
-{
- int ret = handle_on_other_cpu(irq);
-
- if (!ret)
- smtc_im_backstop(irq);
- return ret;
-}
-
-#else
-
-static inline void smtc_im_backstop(unsigned int irq) { }
-static inline int smtc_handle_on_other_cpu(unsigned int irq)
-{
- return handle_on_other_cpu(irq);
-}
-
-#endif
-
extern void do_IRQ(unsigned int irq);
-#ifdef CONFIG_MIPS_MT_SMTC_IRQAFF
-
-extern void do_IRQ_no_affinity(unsigned int irq);
-
-#endif /* CONFIG_MIPS_MT_SMTC_IRQAFF */
-
extern void arch_init_irq(void);
extern void spurious_interrupt(void);
diff --git a/arch/mips/include/asm/irqflags.h b/arch/mips/include/asm/irqflags.h
index 45c0095..0fa5fdc 100644
--- a/arch/mips/include/asm/irqflags.h
+++ b/arch/mips/include/asm/irqflags.h
@@ -17,7 +17,7 @@
#include <linux/stringify.h>
#include <asm/hazards.h>
-#if defined(CONFIG_CPU_MIPSR2) && !defined(CONFIG_MIPS_MT_SMTC)
+#ifdef CONFIG_CPU_MIPSR2
static inline void arch_local_irq_disable(void)
{
@@ -118,30 +118,15 @@
unsigned long arch_local_irq_save(void);
void arch_local_irq_restore(unsigned long flags);
void __arch_local_irq_restore(unsigned long flags);
-#endif /* if defined(CONFIG_CPU_MIPSR2) && !defined(CONFIG_MIPS_MT_SMTC) */
-
-
-extern void smtc_ipi_replay(void);
+#endif /* CONFIG_CPU_MIPSR2 */
static inline void arch_local_irq_enable(void)
{
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * SMTC kernel needs to do a software replay of queued
- * IPIs, at the cost of call overhead on each local_irq_enable()
- */
- smtc_ipi_replay();
-#endif
__asm__ __volatile__(
" .set push \n"
" .set reorder \n"
" .set noat \n"
-#ifdef CONFIG_MIPS_MT_SMTC
- " mfc0 $1, $2, 1 # SMTC - clear TCStatus.IXMT \n"
- " ori $1, 0x400 \n"
- " xori $1, 0x400 \n"
- " mtc0 $1, $2, 1 \n"
-#elif defined(CONFIG_CPU_MIPSR2)
+#if defined(CONFIG_CPU_MIPSR2)
" ei \n"
#else
" mfc0 $1,$12 \n"
@@ -163,11 +148,7 @@
asm __volatile__(
" .set push \n"
" .set reorder \n"
-#ifdef CONFIG_MIPS_MT_SMTC
- " mfc0 %[flags], $2, 1 \n"
-#else
" mfc0 %[flags], $12 \n"
-#endif
" .set pop \n"
: [flags] "=r" (flags));
@@ -177,14 +158,7 @@
static inline int arch_irqs_disabled_flags(unsigned long flags)
{
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * SMTC model uses TCStatus.IXMT to disable interrupts for a thread/CPU
- */
- return flags & 0x400;
-#else
return !(flags & 1);
-#endif
}
#endif /* #ifndef __ASSEMBLY__ */
diff --git a/arch/mips/include/asm/kvm_para.h b/arch/mips/include/asm/kvm_para.h
new file mode 100644
index 0000000..5a9aa91
--- /dev/null
+++ b/arch/mips/include/asm/kvm_para.h
@@ -0,0 +1,109 @@
+#ifndef _ASM_MIPS_KVM_PARA_H
+#define _ASM_MIPS_KVM_PARA_H
+
+#include <uapi/asm/kvm_para.h>
+
+#define KVM_HYPERCALL ".word 0x42000028"
+
+/*
+ * Hypercalls for KVM.
+ *
+ * Hypercall number is passed in v0.
+ * Return value will be placed in v0.
+ * Up to 3 arguments are passed in a0, a1, and a2.
+ */
+static inline unsigned long kvm_hypercall0(unsigned long num)
+{
+ register unsigned long n asm("v0");
+ register unsigned long r asm("v0");
+
+ n = num;
+ __asm__ __volatile__(
+ KVM_HYPERCALL
+ : "=r" (r) : "r" (n) : "memory"
+ );
+
+ return r;
+}
+
+static inline unsigned long kvm_hypercall1(unsigned long num,
+ unsigned long arg0)
+{
+ register unsigned long n asm("v0");
+ register unsigned long r asm("v0");
+ register unsigned long a0 asm("a0");
+
+ n = num;
+ a0 = arg0;
+ __asm__ __volatile__(
+ KVM_HYPERCALL
+ : "=r" (r) : "r" (n), "r" (a0) : "memory"
+ );
+
+ return r;
+}
+
+static inline unsigned long kvm_hypercall2(unsigned long num,
+ unsigned long arg0, unsigned long arg1)
+{
+ register unsigned long n asm("v0");
+ register unsigned long r asm("v0");
+ register unsigned long a0 asm("a0");
+ register unsigned long a1 asm("a1");
+
+ n = num;
+ a0 = arg0;
+ a1 = arg1;
+ __asm__ __volatile__(
+ KVM_HYPERCALL
+ : "=r" (r) : "r" (n), "r" (a0), "r" (a1) : "memory"
+ );
+
+ return r;
+}
+
+static inline unsigned long kvm_hypercall3(unsigned long num,
+ unsigned long arg0, unsigned long arg1, unsigned long arg2)
+{
+ register unsigned long n asm("v0");
+ register unsigned long r asm("v0");
+ register unsigned long a0 asm("a0");
+ register unsigned long a1 asm("a1");
+ register unsigned long a2 asm("a2");
+
+ n = num;
+ a0 = arg0;
+ a1 = arg1;
+ a2 = arg2;
+ __asm__ __volatile__(
+ KVM_HYPERCALL
+ : "=r" (r) : "r" (n), "r" (a0), "r" (a1), "r" (a2) : "memory"
+ );
+
+ return r;
+}
+
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
+static inline unsigned int kvm_arch_para_features(void)
+{
+ return 0;
+}
+
+#ifdef CONFIG_MIPS_PARAVIRT
+static inline bool kvm_para_available(void)
+{
+ return true;
+}
+#else
+static inline bool kvm_para_available(void)
+{
+ return false;
+}
+#endif
+
+
+#endif /* _ASM_MIPS_KVM_PARA_H */
diff --git a/arch/mips/include/asm/mach-cavium-octeon/cpu-feature-overrides.h b/arch/mips/include/asm/mach-cavium-octeon/cpu-feature-overrides.h
index 94ed063..cf80228 100644
--- a/arch/mips/include/asm/mach-cavium-octeon/cpu-feature-overrides.h
+++ b/arch/mips/include/asm/mach-cavium-octeon/cpu-feature-overrides.h
@@ -22,7 +22,6 @@
#define cpu_has_3k_cache 0
#define cpu_has_4k_cache 0
#define cpu_has_tx39_cache 0
-#define cpu_has_fpu 0
#define cpu_has_counter 1
#define cpu_has_watch 1
#define cpu_has_divec 1
diff --git a/arch/mips/include/asm/mach-cavium-octeon/irq.h b/arch/mips/include/asm/mach-cavium-octeon/irq.h
index 60fc4c3..cceae32 100644
--- a/arch/mips/include/asm/mach-cavium-octeon/irq.h
+++ b/arch/mips/include/asm/mach-cavium-octeon/irq.h
@@ -35,6 +35,8 @@
OCTEON_IRQ_PCI_MSI2,
OCTEON_IRQ_PCI_MSI3,
+ OCTEON_IRQ_TWSI,
+ OCTEON_IRQ_TWSI2,
OCTEON_IRQ_RML,
OCTEON_IRQ_TIMER0,
OCTEON_IRQ_TIMER1,
diff --git a/arch/mips/include/asm/mach-ip22/cpu-feature-overrides.h b/arch/mips/include/asm/mach-ip22/cpu-feature-overrides.h
index 1bcb642..1dfe474 100644
--- a/arch/mips/include/asm/mach-ip22/cpu-feature-overrides.h
+++ b/arch/mips/include/asm/mach-ip22/cpu-feature-overrides.h
@@ -39,6 +39,10 @@
#define cpu_has_nofpuex 0
#define cpu_has_64bits 1
+#define cpu_has_mips_2 1
+#define cpu_has_mips_3 1
+#define cpu_has_mips_5 0
+
#define cpu_has_mips32r1 0
#define cpu_has_mips32r2 0
#define cpu_has_mips64r1 0
diff --git a/arch/mips/include/asm/mach-malta/kernel-entry-init.h b/arch/mips/include/asm/mach-malta/kernel-entry-init.h
index 7c5e17a..77eeda7 100644
--- a/arch/mips/include/asm/mach-malta/kernel-entry-init.h
+++ b/arch/mips/include/asm/mach-malta/kernel-entry-init.h
@@ -80,36 +80,6 @@
.endm
.macro kernel_entry_setup
-#ifdef CONFIG_MIPS_MT_SMTC
- mfc0 t0, CP0_CONFIG
- bgez t0, 9f
- mfc0 t0, CP0_CONFIG, 1
- bgez t0, 9f
- mfc0 t0, CP0_CONFIG, 2
- bgez t0, 9f
- mfc0 t0, CP0_CONFIG, 3
- and t0, 1<<2
- bnez t0, 0f
-9:
- /* Assume we came from YAMON... */
- PTR_LA v0, 0x9fc00534 /* YAMON print */
- lw v0, (v0)
- move a0, zero
- PTR_LA a1, nonmt_processor
- jal v0
-
- PTR_LA v0, 0x9fc00520 /* YAMON exit */
- lw v0, (v0)
- li a0, 1
- jal v0
-
-1: b 1b
-
- __INITDATA
-nonmt_processor:
- .asciz "SMTC kernel requires the MT ASE to run\n"
- __FINIT
-#endif
#ifdef CONFIG_EVA
sync
diff --git a/arch/mips/include/asm/mach-malta/malta-pm.h b/arch/mips/include/asm/mach-malta/malta-pm.h
new file mode 100644
index 0000000..c2c2e20
--- /dev/null
+++ b/arch/mips/include/asm/mach-malta/malta-pm.h
@@ -0,0 +1,37 @@
+/*
+ * Copyright (C) 2014 Imagination Technologies
+ * Author: Paul Burton <paul.burton@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#ifndef __ASM_MIPS_MACH_MALTA_PM_H__
+#define __ASM_MIPS_MACH_MALTA_PM_H__
+
+#include <asm/mips-boards/piix4.h>
+
+#ifdef CONFIG_MIPS_MALTA_PM
+
+/**
+ * mips_pm_suspend - enter a suspend state
+ * @state: the state to enter, one of PIIX4_FUNC3IO_PMCNTRL_SUS_TYP_*
+ *
+ * Enters a suspend state via the Malta's PIIX4. If the state to be entered
+ * is one which loses context (eg. SOFF) then this function will never
+ * return.
+ */
+extern int mips_pm_suspend(unsigned state);
+
+#else /* !CONFIG_MIPS_MALTA_PM */
+
+static inline int mips_pm_suspend(unsigned state)
+{
+ return -EINVAL;
+}
+
+#endif /* !CONFIG_MIPS_MALTA_PM */
+
+#endif /* __ASM_MIPS_MACH_MALTA_PM_H__ */
diff --git a/arch/mips/include/asm/mach-netlogic/topology.h b/arch/mips/include/asm/mach-netlogic/topology.h
index 0da99fa..ceeb1f5 100644
--- a/arch/mips/include/asm/mach-netlogic/topology.h
+++ b/arch/mips/include/asm/mach-netlogic/topology.h
@@ -10,10 +10,12 @@
#include <asm/mach-netlogic/multi-node.h>
+#ifdef CONFIG_SMP
#define topology_physical_package_id(cpu) cpu_to_node(cpu)
#define topology_core_id(cpu) (cpu_logical_map(cpu) / NLM_THREADS_PER_CORE)
#define topology_thread_cpumask(cpu) (&cpu_sibling_map[cpu])
#define topology_core_cpumask(cpu) cpumask_of_node(cpu_to_node(cpu))
+#endif
#include <asm-generic/topology.h>
diff --git a/arch/mips/include/asm/mach-paravirt/cpu-feature-overrides.h b/arch/mips/include/asm/mach-paravirt/cpu-feature-overrides.h
new file mode 100644
index 0000000..725e1ed
--- /dev/null
+++ b/arch/mips/include/asm/mach-paravirt/cpu-feature-overrides.h
@@ -0,0 +1,36 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2013 Cavium, Inc.
+ */
+#ifndef __ASM_MACH_PARAVIRT_CPU_FEATURE_OVERRIDES_H
+#define __ASM_MACH_PARAVIRT_CPU_FEATURE_OVERRIDES_H
+
+#define cpu_has_4kex 1
+#define cpu_has_3k_cache 0
+#define cpu_has_tx39_cache 0
+#define cpu_has_counter 1
+#define cpu_has_llsc 1
+/*
+ * We Disable LL/SC on non SMP systems as it is faster to disable
+ * interrupts for atomic access than a LL/SC.
+ */
+#ifdef CONFIG_SMP
+# define kernel_uses_llsc 1
+#else
+# define kernel_uses_llsc 0
+#endif
+
+#ifdef CONFIG_CPU_CAVIUM_OCTEON
+#define cpu_dcache_line_size() 128
+#define cpu_icache_line_size() 128
+#define cpu_has_octeon_cache 1
+#define cpu_has_4k_cache 0
+#else
+#define cpu_has_octeon_cache 0
+#define cpu_has_4k_cache 1
+#endif
+
+#endif /* __ASM_MACH_PARAVIRT_CPU_FEATURE_OVERRIDES_H */
diff --git a/arch/mips/include/asm/mach-paravirt/irq.h b/arch/mips/include/asm/mach-paravirt/irq.h
new file mode 100644
index 0000000..9b4d35e
--- /dev/null
+++ b/arch/mips/include/asm/mach-paravirt/irq.h
@@ -0,0 +1,19 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2013 Cavium, Inc.
+ */
+#ifndef __ASM_MACH_PARAVIRT_IRQ_H__
+#define __ASM_MACH_PARAVIRT_IRQ_H__
+
+#define NR_IRQS 64
+#define MIPS_CPU_IRQ_BASE 1
+
+#define MIPS_IRQ_PCIA (MIPS_CPU_IRQ_BASE + 8)
+
+#define MIPS_IRQ_MBOX0 (MIPS_CPU_IRQ_BASE + 32)
+#define MIPS_IRQ_MBOX1 (MIPS_CPU_IRQ_BASE + 33)
+
+#endif /* __ASM_MACH_PARAVIRT_IRQ_H__ */
diff --git a/arch/mips/include/asm/mach-paravirt/kernel-entry-init.h b/arch/mips/include/asm/mach-paravirt/kernel-entry-init.h
new file mode 100644
index 0000000..2f82bfa
--- /dev/null
+++ b/arch/mips/include/asm/mach-paravirt/kernel-entry-init.h
@@ -0,0 +1,50 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2013 Cavium, Inc
+ */
+#ifndef __ASM_MACH_PARAVIRT_KERNEL_ENTRY_H
+#define __ASM_MACH_PARAVIRT_KERNEL_ENTRY_H
+
+#define CP0_EBASE $15, 1
+
+ .macro kernel_entry_setup
+ mfc0 t0, CP0_EBASE
+ andi t0, t0, 0x3ff # CPUNum
+ beqz t0, 1f
+ # CPUs other than zero goto smp_bootstrap
+ j smp_bootstrap
+
+1:
+ .endm
+
+/*
+ * Do SMP slave processor setup necessary before we can safely execute
+ * C code.
+ */
+ .macro smp_slave_setup
+ mfc0 t0, CP0_EBASE
+ andi t0, t0, 0x3ff # CPUNum
+ slti t1, t0, NR_CPUS
+ bnez t1, 1f
+2:
+ di
+ wait
+ b 2b # Unknown CPU, loop forever.
+1:
+ PTR_LA t1, paravirt_smp_sp
+ PTR_SLL t0, PTR_SCALESHIFT
+ PTR_ADDU t1, t1, t0
+3:
+ PTR_L sp, 0(t1)
+ beqz sp, 3b # Spin until told to proceed.
+
+ PTR_LA t1, paravirt_smp_gp
+ PTR_ADDU t1, t1, t0
+ sync
+ PTR_L gp, 0(t1)
+ .endm
+
+#endif /* __ASM_MACH_PARAVIRT_KERNEL_ENTRY_H */
diff --git a/arch/mips/include/asm/mach-paravirt/war.h b/arch/mips/include/asm/mach-paravirt/war.h
new file mode 100644
index 0000000..36d3afb
--- /dev/null
+++ b/arch/mips/include/asm/mach-paravirt/war.h
@@ -0,0 +1,25 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2002, 2004, 2007 by Ralf Baechle <ralf@linux-mips.org>
+ * Copyright (C) 2013 Cavium Networks <support@caviumnetworks.com>
+ */
+#ifndef __ASM_MIPS_MACH_PARAVIRT_WAR_H
+#define __ASM_MIPS_MACH_PARAVIRT_WAR_H
+
+#define R4600_V1_INDEX_ICACHEOP_WAR 0
+#define R4600_V1_HIT_CACHEOP_WAR 0
+#define R4600_V2_HIT_CACHEOP_WAR 0
+#define R5432_CP0_INTERRUPT_WAR 0
+#define BCM1250_M3_WAR 0
+#define SIBYTE_1956_WAR 0
+#define MIPS4K_ICACHE_REFILL_WAR 0
+#define MIPS_CACHE_SYNC_WAR 0
+#define TX49XX_ICACHE_INDEX_INV_WAR 0
+#define ICACHE_REFILLS_WORKAROUND_WAR 0
+#define R10000_LLSC_WAR 0
+#define MIPS34K_MISSED_ITLB_WAR 0
+
+#endif /* __ASM_MIPS_MACH_PARAVIRT_WAR_H */
diff --git a/arch/mips/include/asm/mach-pmcs-msp71xx/msp_usb.h b/arch/mips/include/asm/mach-pmcs-msp71xx/msp_usb.h
index aa45e6a..fe1566f 100644
--- a/arch/mips/include/asm/mach-pmcs-msp71xx/msp_usb.h
+++ b/arch/mips/include/asm/mach-pmcs-msp71xx/msp_usb.h
@@ -25,11 +25,7 @@
#ifndef MSP_USB_H_
#define MSP_USB_H_
-#ifdef CONFIG_MSP_HAS_DUAL_USB
-#define NUM_USB_DEVS 2
-#else
#define NUM_USB_DEVS 1
-#endif
/* Register spaces for USB host 0 */
#define MSP_USB0_MAB_START (MSP_USB0_BASE + 0x0)
diff --git a/arch/mips/include/asm/mach-ralink/war.h b/arch/mips/include/asm/mach-ralink/war.h
index a7b712c..c074b5d 100644
--- a/arch/mips/include/asm/mach-ralink/war.h
+++ b/arch/mips/include/asm/mach-ralink/war.h
@@ -17,7 +17,6 @@
#define MIPS4K_ICACHE_REFILL_WAR 0
#define MIPS_CACHE_SYNC_WAR 0
#define TX49XX_ICACHE_INDEX_INV_WAR 0
-#define RM9000_CDEX_SMP_WAR 0
#define ICACHE_REFILLS_WORKAROUND_WAR 0
#define R10000_LLSC_WAR 0
#define MIPS34K_MISSED_ITLB_WAR 0
diff --git a/arch/mips/include/asm/mach-sead3/kernel-entry-init.h b/arch/mips/include/asm/mach-sead3/kernel-entry-init.h
index 3dfbd8e..6cccd4d 100644
--- a/arch/mips/include/asm/mach-sead3/kernel-entry-init.h
+++ b/arch/mips/include/asm/mach-sead3/kernel-entry-init.h
@@ -10,37 +10,6 @@
#define __ASM_MACH_MIPS_KERNEL_ENTRY_INIT_H
.macro kernel_entry_setup
-#ifdef CONFIG_MIPS_MT_SMTC
- mfc0 t0, CP0_CONFIG
- bgez t0, 9f
- mfc0 t0, CP0_CONFIG, 1
- bgez t0, 9f
- mfc0 t0, CP0_CONFIG, 2
- bgez t0, 9f
- mfc0 t0, CP0_CONFIG, 3
- and t0, 1<<2
- bnez t0, 0f
-9 :
- /* Assume we came from YAMON... */
- PTR_LA v0, 0x9fc00534 /* YAMON print */
- lw v0, (v0)
- move a0, zero
- PTR_LA a1, nonmt_processor
- jal v0
-
- PTR_LA v0, 0x9fc00520 /* YAMON exit */
- lw v0, (v0)
- li a0, 1
- jal v0
-
-1 : b 1b
-
- __INITDATA
-nonmt_processor :
- .asciz "SMTC kernel requires the MT ASE to run\n"
- __FINIT
-0 :
-#endif
.endm
/*
diff --git a/arch/mips/include/asm/mips-boards/piix4.h b/arch/mips/include/asm/mips-boards/piix4.h
index 9cf5404..9e340be 100644
--- a/arch/mips/include/asm/mips-boards/piix4.h
+++ b/arch/mips/include/asm/mips-boards/piix4.h
@@ -55,4 +55,16 @@
#define PIIX4_FUNC3_PMREGMISC 0x80
#define PIIX4_FUNC3_PMREGMISC_EN (1 << 0)
+/* Power Management IO Space */
+#define PIIX4_FUNC3IO_PMSTS 0x00
+#define PIIX4_FUNC3IO_PMSTS_PWRBTN_STS (1 << 8)
+#define PIIX4_FUNC3IO_PMCNTRL 0x04
+#define PIIX4_FUNC3IO_PMCNTRL_SUS_EN (1 << 13)
+#define PIIX4_FUNC3IO_PMCNTRL_SUS_TYP (0x7 << 10)
+#define PIIX4_FUNC3IO_PMCNTRL_SUS_TYP_SOFF (0x0 << 10)
+#define PIIX4_FUNC3IO_PMCNTRL_SUS_TYP_STR (0x1 << 10)
+
+/* Data for magic special PCI cycle */
+#define PIIX4_SUSPEND_MAGIC 0x00120002
+
#endif /* __ASM_MIPS_BOARDS_PIIX4_H */
diff --git a/arch/mips/include/asm/mips-cpc.h b/arch/mips/include/asm/mips-cpc.h
index 988507e..e139a53 100644
--- a/arch/mips/include/asm/mips-cpc.h
+++ b/arch/mips/include/asm/mips-cpc.h
@@ -72,7 +72,12 @@
#define MIPS_CPC_COCB_OFS 0x4000
/* Macros to ease the creation of register access functions */
-#define BUILD_CPC_R_(name, off) \
+#define BUILD_CPC_R_(name, off) \
+static inline u32 *addr_cpc_##name(void) \
+{ \
+ return (u32 *)(mips_cpc_base + (off)); \
+} \
+ \
static inline u32 read_cpc_##name(void) \
{ \
return __raw_readl(mips_cpc_base + (off)); \
@@ -147,4 +152,31 @@
#define CPC_Cx_OTHER_CORENUM_SHF 16
#define CPC_Cx_OTHER_CORENUM_MSK (_ULCAST_(0xff) << 16)
+#ifdef CONFIG_MIPS_CPC
+
+/**
+ * mips_cpc_lock_other - lock access to another core
+ * core: the other core to be accessed
+ *
+ * Call before operating upon a core via the 'other' register region in
+ * order to prevent the region being moved during access. Must be followed
+ * by a call to mips_cpc_unlock_other.
+ */
+extern void mips_cpc_lock_other(unsigned int core);
+
+/**
+ * mips_cpc_unlock_other - unlock access to another core
+ *
+ * Call after operating upon another core via the 'other' register region.
+ * Must be called after mips_cpc_lock_other.
+ */
+extern void mips_cpc_unlock_other(void);
+
+#else /* !CONFIG_MIPS_CPC */
+
+static inline void mips_cpc_lock_other(unsigned int core) { }
+static inline void mips_cpc_unlock_other(void) { }
+
+#endif /* !CONFIG_MIPS_CPC */
+
#endif /* __MIPS_ASM_MIPS_CPC_H__ */
diff --git a/arch/mips/include/asm/mips_mt.h b/arch/mips/include/asm/mips_mt.h
index a3df0c3..f6ba004 100644
--- a/arch/mips/include/asm/mips_mt.h
+++ b/arch/mips/include/asm/mips_mt.h
@@ -1,7 +1,6 @@
/*
- * Definitions and decalrations for MIPS MT support
- * that are common between SMTC, VSMP, and/or AP/SP
- * kernel models.
+ * Definitions and decalrations for MIPS MT support that are common between
+ * the VSMP, and AP/SP kernel models.
*/
#ifndef __ASM_MIPS_MT_H
#define __ASM_MIPS_MT_H
diff --git a/arch/mips/include/asm/mipsmtregs.h b/arch/mips/include/asm/mipsmtregs.h
index 6efa79a..5f8052c 100644
--- a/arch/mips/include/asm/mipsmtregs.h
+++ b/arch/mips/include/asm/mipsmtregs.h
@@ -36,6 +36,8 @@
#define read_c0_tcbind() __read_32bit_c0_register($2, 2)
+#define write_c0_tchalt(val) __write_32bit_c0_register($2, 4, val)
+
#define read_c0_tccontext() __read_32bit_c0_register($2, 5)
#define write_c0_tccontext(val) __write_32bit_c0_register($2, 5, val)
diff --git a/arch/mips/include/asm/mipsregs.h b/arch/mips/include/asm/mipsregs.h
index 3e025b5..98e9754 100644
--- a/arch/mips/include/asm/mipsregs.h
+++ b/arch/mips/include/asm/mipsregs.h
@@ -709,11 +709,18 @@
#ifndef __ASSEMBLY__
/*
- * Macros for handling the ISA mode bit for microMIPS.
+ * Macros for handling the ISA mode bit for MIPS16 and microMIPS.
*/
+#if defined(CONFIG_SYS_SUPPORTS_MIPS16) || \
+ defined(CONFIG_SYS_SUPPORTS_MICROMIPS)
#define get_isa16_mode(x) ((x) & 0x1)
#define msk_isa16_mode(x) ((x) & ~0x1)
#define set_isa16_mode(x) do { (x) |= 0x1; } while(0)
+#else
+#define get_isa16_mode(x) 0
+#define msk_isa16_mode(x) (x)
+#define set_isa16_mode(x) do { } while(0)
+#endif
/*
* microMIPS instructions can be 16-bit or 32-bit in length. This
@@ -1007,19 +1014,8 @@
#define write_c0_compare3(val) __write_32bit_c0_register($11, 7, val)
#define read_c0_status() __read_32bit_c0_register($12, 0)
-#ifdef CONFIG_MIPS_MT_SMTC
-#define write_c0_status(val) \
-do { \
- __write_32bit_c0_register($12, 0, val); \
- __ehb(); \
-} while (0)
-#else
-/*
- * Legacy non-SMTC code, which may be hazardous
- * but which might not support EHB
- */
+
#define write_c0_status(val) __write_32bit_c0_register($12, 0, val)
-#endif /* CONFIG_MIPS_MT_SMTC */
#define read_c0_cause() __read_32bit_c0_register($13, 0)
#define write_c0_cause(val) __write_32bit_c0_register($13, 0, val)
@@ -1743,11 +1739,6 @@
/*
* Manipulate bits in a c0 register.
*/
-#ifndef CONFIG_MIPS_MT_SMTC
-/*
- * SMTC Linux requires shutting-down microthread scheduling
- * during CP0 register read-modify-write sequences.
- */
#define __BUILD_SET_C0(name) \
static inline unsigned int \
set_c0_##name(unsigned int set) \
@@ -1786,121 +1777,6 @@
return res; \
}
-#else /* SMTC versions that manage MT scheduling */
-
-#include <linux/irqflags.h>
-
-/*
- * This is a duplicate of dmt() in mipsmtregs.h to avoid problems with
- * header file recursion.
- */
-static inline unsigned int __dmt(void)
-{
- int res;
-
- __asm__ __volatile__(
- " .set push \n"
- " .set mips32r2 \n"
- " .set noat \n"
- " .word 0x41610BC1 # dmt $1 \n"
- " ehb \n"
- " move %0, $1 \n"
- " .set pop \n"
- : "=r" (res));
-
- instruction_hazard();
-
- return res;
-}
-
-#define __VPECONTROL_TE_SHIFT 15
-#define __VPECONTROL_TE (1UL << __VPECONTROL_TE_SHIFT)
-
-#define __EMT_ENABLE __VPECONTROL_TE
-
-static inline void __emt(unsigned int previous)
-{
- if ((previous & __EMT_ENABLE))
- __asm__ __volatile__(
- " .set mips32r2 \n"
- " .word 0x41600be1 # emt \n"
- " ehb \n"
- " .set mips0 \n");
-}
-
-static inline void __ehb(void)
-{
- __asm__ __volatile__(
- " .set mips32r2 \n"
- " ehb \n" " .set mips0 \n");
-}
-
-/*
- * Note that local_irq_save/restore affect TC-specific IXMT state,
- * not Status.IE as in non-SMTC kernel.
- */
-
-#define __BUILD_SET_C0(name) \
-static inline unsigned int \
-set_c0_##name(unsigned int set) \
-{ \
- unsigned int res; \
- unsigned int new; \
- unsigned int omt; \
- unsigned long flags; \
- \
- local_irq_save(flags); \
- omt = __dmt(); \
- res = read_c0_##name(); \
- new = res | set; \
- write_c0_##name(new); \
- __emt(omt); \
- local_irq_restore(flags); \
- \
- return res; \
-} \
- \
-static inline unsigned int \
-clear_c0_##name(unsigned int clear) \
-{ \
- unsigned int res; \
- unsigned int new; \
- unsigned int omt; \
- unsigned long flags; \
- \
- local_irq_save(flags); \
- omt = __dmt(); \
- res = read_c0_##name(); \
- new = res & ~clear; \
- write_c0_##name(new); \
- __emt(omt); \
- local_irq_restore(flags); \
- \
- return res; \
-} \
- \
-static inline unsigned int \
-change_c0_##name(unsigned int change, unsigned int newbits) \
-{ \
- unsigned int res; \
- unsigned int new; \
- unsigned int omt; \
- unsigned long flags; \
- \
- local_irq_save(flags); \
- \
- omt = __dmt(); \
- res = read_c0_##name(); \
- new = res & ~change; \
- new |= (newbits & change); \
- write_c0_##name(new); \
- __emt(omt); \
- local_irq_restore(flags); \
- \
- return res; \
-}
-#endif
-
__BUILD_SET_C0(status)
__BUILD_SET_C0(cause)
__BUILD_SET_C0(config)
@@ -1916,6 +1792,15 @@
__BUILD_SET_C0(brcm_config)
__BUILD_SET_C0(brcm_mode)
+/*
+ * Return low 10 bits of ebase.
+ * Note that under KVM (MIPSVZ) this returns vcpu id.
+ */
+static inline unsigned int get_ebase_cpunum(void)
+{
+ return read_c0_ebase() & 0x3ff;
+}
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASM_MIPSREGS_H */
diff --git a/arch/mips/include/asm/mmu_context.h b/arch/mips/include/asm/mmu_context.h
index e277bba..2e373da 100644
--- a/arch/mips/include/asm/mmu_context.h
+++ b/arch/mips/include/asm/mmu_context.h
@@ -18,10 +18,6 @@
#include <asm/cacheflush.h>
#include <asm/hazards.h>
#include <asm/tlbflush.h>
-#ifdef CONFIG_MIPS_MT_SMTC
-#include <asm/mipsmtregs.h>
-#include <asm/smtc.h>
-#endif /* SMTC */
#include <asm-generic/mm_hooks.h>
#define TLBMISS_HANDLER_SETUP_PGD(pgd) \
@@ -31,11 +27,15 @@
} while (0)
#ifdef CONFIG_MIPS_PGD_C0_CONTEXT
+
+#define TLBMISS_HANDLER_RESTORE() \
+ write_c0_xcontext((unsigned long) smp_processor_id() << \
+ SMP_CPUID_REGSHIFT)
+
#define TLBMISS_HANDLER_SETUP() \
do { \
TLBMISS_HANDLER_SETUP_PGD(swapper_pg_dir); \
- write_c0_xcontext((unsigned long) smp_processor_id() << \
- SMP_CPUID_REGSHIFT); \
+ TLBMISS_HANDLER_RESTORE(); \
} while (0)
#else /* !CONFIG_MIPS_PGD_C0_CONTEXT: using pgd_current*/
@@ -47,9 +47,12 @@
*/
extern unsigned long pgd_current[];
-#define TLBMISS_HANDLER_SETUP() \
+#define TLBMISS_HANDLER_RESTORE() \
write_c0_context((unsigned long) smp_processor_id() << \
- SMP_CPUID_REGSHIFT); \
+ SMP_CPUID_REGSHIFT)
+
+#define TLBMISS_HANDLER_SETUP() \
+ TLBMISS_HANDLER_RESTORE(); \
back_to_back_c0_hazard(); \
TLBMISS_HANDLER_SETUP_PGD(swapper_pg_dir)
#endif /* CONFIG_MIPS_PGD_C0_CONTEXT*/
@@ -63,13 +66,6 @@
#define ASID_INC 0x10
#define ASID_MASK 0xff0
-#elif defined(CONFIG_MIPS_MT_SMTC)
-
-#define ASID_INC 0x1
-extern unsigned long smtc_asid_mask;
-#define ASID_MASK (smtc_asid_mask)
-#define HW_ASID_MASK 0xff
-/* End SMTC/34K debug hack */
#else /* FIXME: not correct for R6000 */
#define ASID_INC 0x1
@@ -92,7 +88,6 @@
#define ASID_VERSION_MASK ((unsigned long)~(ASID_MASK|(ASID_MASK-1)))
#define ASID_FIRST_VERSION ((unsigned long)(~ASID_VERSION_MASK) + 1)
-#ifndef CONFIG_MIPS_MT_SMTC
/* Normal, classic MIPS get_new_mmu_context */
static inline void
get_new_mmu_context(struct mm_struct *mm, unsigned long cpu)
@@ -115,12 +110,6 @@
cpu_context(cpu, mm) = asid_cache(cpu) = asid;
}
-#else /* CONFIG_MIPS_MT_SMTC */
-
-#define get_new_mmu_context(mm, cpu) smtc_get_new_mmu_context((mm), (cpu))
-
-#endif /* CONFIG_MIPS_MT_SMTC */
-
/*
* Initialize the context related info for a new mm_struct
* instance.
@@ -141,46 +130,12 @@
{
unsigned int cpu = smp_processor_id();
unsigned long flags;
-#ifdef CONFIG_MIPS_MT_SMTC
- unsigned long oldasid;
- unsigned long mtflags;
- int mytlb = (smtc_status & SMTC_TLB_SHARED) ? 0 : cpu_data[cpu].vpe_id;
local_irq_save(flags);
- mtflags = dvpe();
-#else /* Not SMTC */
- local_irq_save(flags);
-#endif /* CONFIG_MIPS_MT_SMTC */
/* Check if our ASID is of an older version and thus invalid */
if ((cpu_context(cpu, next) ^ asid_cache(cpu)) & ASID_VERSION_MASK)
get_new_mmu_context(next, cpu);
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * If the EntryHi ASID being replaced happens to be
- * the value flagged at ASID recycling time as having
- * an extended life, clear the bit showing it being
- * in use by this "CPU", and if that's the last bit,
- * free up the ASID value for use and flush any old
- * instances of it from the TLB.
- */
- oldasid = (read_c0_entryhi() & ASID_MASK);
- if(smtc_live_asid[mytlb][oldasid]) {
- smtc_live_asid[mytlb][oldasid] &= ~(0x1 << cpu);
- if(smtc_live_asid[mytlb][oldasid] == 0)
- smtc_flush_tlb_asid(oldasid);
- }
- /*
- * Tread softly on EntryHi, and so long as we support
- * having ASID_MASK smaller than the hardware maximum,
- * make sure no "soft" bits become "hard"...
- */
- write_c0_entryhi((read_c0_entryhi() & ~HW_ASID_MASK) |
- cpu_asid(cpu, next));
- ehb(); /* Make sure it propagates to TCStatus */
- evpe(mtflags);
-#else
write_c0_entryhi(cpu_asid(cpu, next));
-#endif /* CONFIG_MIPS_MT_SMTC */
TLBMISS_HANDLER_SETUP_PGD(next->pgd);
/*
@@ -213,34 +168,12 @@
unsigned long flags;
unsigned int cpu = smp_processor_id();
-#ifdef CONFIG_MIPS_MT_SMTC
- unsigned long oldasid;
- unsigned long mtflags;
- int mytlb = (smtc_status & SMTC_TLB_SHARED) ? 0 : cpu_data[cpu].vpe_id;
-#endif /* CONFIG_MIPS_MT_SMTC */
-
local_irq_save(flags);
/* Unconditionally get a new ASID. */
get_new_mmu_context(next, cpu);
-#ifdef CONFIG_MIPS_MT_SMTC
- /* See comments for similar code above */
- mtflags = dvpe();
- oldasid = read_c0_entryhi() & ASID_MASK;
- if(smtc_live_asid[mytlb][oldasid]) {
- smtc_live_asid[mytlb][oldasid] &= ~(0x1 << cpu);
- if(smtc_live_asid[mytlb][oldasid] == 0)
- smtc_flush_tlb_asid(oldasid);
- }
- /* See comments for similar code above */
- write_c0_entryhi((read_c0_entryhi() & ~HW_ASID_MASK) |
- cpu_asid(cpu, next));
- ehb(); /* Make sure it propagates to TCStatus */
- evpe(mtflags);
-#else
write_c0_entryhi(cpu_asid(cpu, next));
-#endif /* CONFIG_MIPS_MT_SMTC */
TLBMISS_HANDLER_SETUP_PGD(next->pgd);
/* mark mmu ownership change */
@@ -258,48 +191,15 @@
drop_mmu_context(struct mm_struct *mm, unsigned cpu)
{
unsigned long flags;
-#ifdef CONFIG_MIPS_MT_SMTC
- unsigned long oldasid;
- /* Can't use spinlock because called from TLB flush within DVPE */
- unsigned int prevvpe;
- int mytlb = (smtc_status & SMTC_TLB_SHARED) ? 0 : cpu_data[cpu].vpe_id;
-#endif /* CONFIG_MIPS_MT_SMTC */
local_irq_save(flags);
if (cpumask_test_cpu(cpu, mm_cpumask(mm))) {
get_new_mmu_context(mm, cpu);
-#ifdef CONFIG_MIPS_MT_SMTC
- /* See comments for similar code above */
- prevvpe = dvpe();
- oldasid = (read_c0_entryhi() & ASID_MASK);
- if (smtc_live_asid[mytlb][oldasid]) {
- smtc_live_asid[mytlb][oldasid] &= ~(0x1 << cpu);
- if(smtc_live_asid[mytlb][oldasid] == 0)
- smtc_flush_tlb_asid(oldasid);
- }
- /* See comments for similar code above */
- write_c0_entryhi((read_c0_entryhi() & ~HW_ASID_MASK)
- | cpu_asid(cpu, mm));
- ehb(); /* Make sure it propagates to TCStatus */
- evpe(prevvpe);
-#else /* not CONFIG_MIPS_MT_SMTC */
write_c0_entryhi(cpu_asid(cpu, mm));
-#endif /* CONFIG_MIPS_MT_SMTC */
} else {
/* will get a new context next time */
-#ifndef CONFIG_MIPS_MT_SMTC
cpu_context(cpu, mm) = 0;
-#else /* SMTC */
- int i;
-
- /* SMTC shares the TLB (and ASIDs) across VPEs */
- for_each_online_cpu(i) {
- if((smtc_status & SMTC_TLB_SHARED)
- || (cpu_data[i].vpe_id == cpu_data[cpu].vpe_id))
- cpu_context(i, mm) = 0;
- }
-#endif /* CONFIG_MIPS_MT_SMTC */
}
local_irq_restore(flags);
}
diff --git a/arch/mips/include/asm/module.h b/arch/mips/include/asm/module.h
index c2edae3..800fe57 100644
--- a/arch/mips/include/asm/module.h
+++ b/arch/mips/include/asm/module.h
@@ -144,13 +144,7 @@
#define MODULE_KERNEL_TYPE "64BIT "
#endif
-#ifdef CONFIG_MIPS_MT_SMTC
-#define MODULE_KERNEL_SMTC "MT_SMTC "
-#else
-#define MODULE_KERNEL_SMTC ""
-#endif
-
#define MODULE_ARCH_VERMAGIC \
- MODULE_PROC_FAMILY MODULE_KERNEL_TYPE MODULE_KERNEL_SMTC
+ MODULE_PROC_FAMILY MODULE_KERNEL_TYPE
#endif /* _ASM_MODULE_H */
diff --git a/arch/mips/include/asm/msa.h b/arch/mips/include/asm/msa.h
index a2aba6c..538f6d4 100644
--- a/arch/mips/include/asm/msa.h
+++ b/arch/mips/include/asm/msa.h
@@ -84,7 +84,7 @@
__asm__ __volatile__( \
" .set push\n" \
" .set msa\n" \
- " cfcmsa $" #cs ", %0\n" \
+ " ctcmsa $" #cs ", %0\n" \
" .set pop\n" \
: : "r"(val)); \
}
@@ -96,6 +96,13 @@
* allow compilation with toolchains that do not support MSA. Once all
* toolchains in use support MSA these can be removed.
*/
+#ifdef CONFIG_CPU_MICROMIPS
+#define CFC_MSA_INSN 0x587e0056
+#define CTC_MSA_INSN 0x583e0816
+#else
+#define CFC_MSA_INSN 0x787e0059
+#define CTC_MSA_INSN 0x783e0819
+#endif
#define __BUILD_MSA_CTL_REG(name, cs) \
static inline unsigned int read_msa_##name(void) \
@@ -104,7 +111,8 @@
__asm__ __volatile__( \
" .set push\n" \
" .set noat\n" \
- " .word 0x787e0059 | (" #cs " << 11)\n" \
+ " .insn\n" \
+ " .word #CFC_MSA_INSN | (" #cs " << 11)\n" \
" move %0, $1\n" \
" .set pop\n" \
: "=r"(reg)); \
@@ -117,7 +125,8 @@
" .set push\n" \
" .set noat\n" \
" move $1, %0\n" \
- " .word 0x783e0819 | (" #cs " << 6)\n" \
+ " .insn\n" \
+ " .word #CTC_MSA_INSN | (" #cs " << 6)\n" \
" .set pop\n" \
: : "r"(val)); \
}
diff --git a/arch/mips/include/asm/netlogic/mips-extns.h b/arch/mips/include/asm/netlogic/mips-extns.h
index de9aada..06f1f75 100644
--- a/arch/mips/include/asm/netlogic/mips-extns.h
+++ b/arch/mips/include/asm/netlogic/mips-extns.h
@@ -146,9 +146,10 @@
static inline int nlm_nodeid(void)
{
- uint32_t prid = read_c0_prid();
+ uint32_t prid = read_c0_prid() & PRID_IMP_MASK;
- if ((prid & 0xff00) == PRID_IMP_NETLOGIC_XLP9XX)
+ if ((prid == PRID_IMP_NETLOGIC_XLP9XX) ||
+ (prid == PRID_IMP_NETLOGIC_XLP5XX))
return (__read_32bit_c0_register($15, 1) >> 7) & 0x7;
else
return (__read_32bit_c0_register($15, 1) >> 5) & 0x3;
diff --git a/arch/mips/include/asm/netlogic/xlp-hal/iomap.h b/arch/mips/include/asm/netlogic/xlp-hal/iomap.h
index 1f23dfa..805bfd2 100644
--- a/arch/mips/include/asm/netlogic/xlp-hal/iomap.h
+++ b/arch/mips/include/asm/netlogic/xlp-hal/iomap.h
@@ -74,6 +74,8 @@
#define XLP_IO_USB_OHCI2_OFFSET(node) XLP_HDR_OFFSET(node, 0, 2, 4)
#define XLP_IO_USB_OHCI3_OFFSET(node) XLP_HDR_OFFSET(node, 0, 2, 5)
+#define XLP_IO_SATA_OFFSET(node) XLP_HDR_OFFSET(node, 0, 3, 2)
+
/* XLP2xx has an updated USB block */
#define XLP2XX_IO_USB_OFFSET(node, i) XLP_HDR_OFFSET(node, 0, 4, i)
#define XLP2XX_IO_USB_XHCI0_OFFSET(node) XLP_HDR_OFFSET(node, 0, 4, 1)
@@ -103,13 +105,11 @@
#define XLP_IO_SYS_OFFSET(node) XLP_HDR_OFFSET(node, 0, 6, 5)
#define XLP_IO_JTAG_OFFSET(node) XLP_HDR_OFFSET(node, 0, 6, 6)
+/* Flash */
#define XLP_IO_NOR_OFFSET(node) XLP_HDR_OFFSET(node, 0, 7, 0)
#define XLP_IO_NAND_OFFSET(node) XLP_HDR_OFFSET(node, 0, 7, 1)
#define XLP_IO_SPI_OFFSET(node) XLP_HDR_OFFSET(node, 0, 7, 2)
-/* SD flash */
-#define XLP_IO_SD_OFFSET(node) XLP_HDR_OFFSET(node, 0, 7, 3)
-#define XLP_IO_MMC_OFFSET(node, slot) \
- ((XLP_IO_SD_OFFSET(node))+(slot*0x100)+XLP_IO_PCI_HDRSZ)
+#define XLP_IO_MMC_OFFSET(node) XLP_HDR_OFFSET(node, 0, 7, 3)
/* Things have changed drastically in XLP 9XX */
#define XLP9XX_HDR_OFFSET(n, d, f) \
@@ -120,6 +120,8 @@
#define XLP9XX_IO_UART_OFFSET(node) XLP9XX_HDR_OFFSET(node, 2, 2)
#define XLP9XX_IO_SYS_OFFSET(node) XLP9XX_HDR_OFFSET(node, 6, 0)
#define XLP9XX_IO_FUSE_OFFSET(node) XLP9XX_HDR_OFFSET(node, 6, 1)
+#define XLP9XX_IO_CLOCK_OFFSET(node) XLP9XX_HDR_OFFSET(node, 6, 2)
+#define XLP9XX_IO_POWER_OFFSET(node) XLP9XX_HDR_OFFSET(node, 6, 3)
#define XLP9XX_IO_JTAG_OFFSET(node) XLP9XX_HDR_OFFSET(node, 6, 4)
#define XLP9XX_IO_PCIE_OFFSET(node, i) XLP9XX_HDR_OFFSET(node, 1, i)
@@ -135,11 +137,11 @@
/* XLP9XX on-chip SATA controller */
#define XLP9XX_IO_SATA_OFFSET(node) XLP9XX_HDR_OFFSET(node, 3, 2)
+/* Flash */
#define XLP9XX_IO_NOR_OFFSET(node) XLP9XX_HDR_OFFSET(node, 7, 0)
#define XLP9XX_IO_NAND_OFFSET(node) XLP9XX_HDR_OFFSET(node, 7, 1)
#define XLP9XX_IO_SPI_OFFSET(node) XLP9XX_HDR_OFFSET(node, 7, 2)
-/* SD flash */
-#define XLP9XX_IO_MMCSD_OFFSET(node) XLP9XX_HDR_OFFSET(node, 7, 3)
+#define XLP9XX_IO_MMC_OFFSET(node) XLP9XX_HDR_OFFSET(node, 7, 3)
/* PCI config header register id's */
#define XLP_PCI_CFGREG0 0x00
@@ -186,8 +188,10 @@
#define PCI_DEVICE_ID_NLM_NOR 0x1015
#define PCI_DEVICE_ID_NLM_NAND 0x1016
#define PCI_DEVICE_ID_NLM_MMC 0x1018
-#define PCI_DEVICE_ID_NLM_XHCI 0x101d
+#define PCI_DEVICE_ID_NLM_SATA 0x101A
+#define PCI_DEVICE_ID_NLM_XHCI 0x101D
+#define PCI_DEVICE_ID_XLP9XX_MMC 0x9018
#define PCI_DEVICE_ID_XLP9XX_SATA 0x901A
#define PCI_DEVICE_ID_XLP9XX_XHCI 0x901D
diff --git a/arch/mips/include/asm/netlogic/xlp-hal/pcibus.h b/arch/mips/include/asm/netlogic/xlp-hal/pcibus.h
index d4deb87..91540f4 100644
--- a/arch/mips/include/asm/netlogic/xlp-hal/pcibus.h
+++ b/arch/mips/include/asm/netlogic/xlp-hal/pcibus.h
@@ -69,6 +69,20 @@
#define PCIE_9XX_BYTE_SWAP_IO_BASE 0x25e
#define PCIE_9XX_BYTE_SWAP_IO_LIM 0x25f
+#define PCIE_9XX_BRIDGE_MSIX_ADDR_BASE 0x264
+#define PCIE_9XX_BRIDGE_MSIX_ADDR_LIMIT 0x265
+#define PCIE_9XX_MSI_STATUS 0x283
+#define PCIE_9XX_MSI_EN 0x284
+/* 128 MSIX vectors available in 9xx */
+#define PCIE_9XX_MSIX_STATUS0 0x286
+#define PCIE_9XX_MSIX_STATUSX(n) (n + 0x286)
+#define PCIE_9XX_MSIX_VEC 0x296
+#define PCIE_9XX_MSIX_VECX(n) (n + 0x296)
+#define PCIE_9XX_INT_STATUS0 0x397
+#define PCIE_9XX_INT_STATUS1 0x398
+#define PCIE_9XX_INT_EN0 0x399
+#define PCIE_9XX_INT_EN1 0x39a
+
/* other */
#define PCIE_NLINKS 4
diff --git a/arch/mips/include/asm/netlogic/xlp-hal/pic.h b/arch/mips/include/asm/netlogic/xlp-hal/pic.h
index f10bf3b..41cefe9 100644
--- a/arch/mips/include/asm/netlogic/xlp-hal/pic.h
+++ b/arch/mips/include/asm/netlogic/xlp-hal/pic.h
@@ -199,6 +199,10 @@
#define PIC_IRT_PCIE_LINK_3_INDEX 81
#define PIC_IRT_PCIE_LINK_INDEX(num) ((num) + PIC_IRT_PCIE_LINK_0_INDEX)
+#define PIC_9XX_IRT_PCIE_LINK_0_INDEX 191
+#define PIC_9XX_IRT_PCIE_LINK_INDEX(num) \
+ ((num) + PIC_9XX_IRT_PCIE_LINK_0_INDEX)
+
#define PIC_CLOCK_TIMER 7
#if !defined(LOCORE) && !defined(__ASSEMBLY__)
diff --git a/arch/mips/include/asm/netlogic/xlp-hal/sys.h b/arch/mips/include/asm/netlogic/xlp-hal/sys.h
index d9b107f..bc7bddf 100644
--- a/arch/mips/include/asm/netlogic/xlp-hal/sys.h
+++ b/arch/mips/include/asm/netlogic/xlp-hal/sys.h
@@ -118,6 +118,10 @@
#define SYS_SCRTCH3 0x4c
/* PLL registers XLP2XX */
+#define SYS_CPU_PLL_CTRL0(core) (0x1c0 + (core * 4))
+#define SYS_CPU_PLL_CTRL1(core) (0x1c1 + (core * 4))
+#define SYS_CPU_PLL_CTRL2(core) (0x1c2 + (core * 4))
+#define SYS_CPU_PLL_CTRL3(core) (0x1c3 + (core * 4))
#define SYS_PLL_CTRL0 0x240
#define SYS_PLL_CTRL1 0x241
#define SYS_PLL_CTRL2 0x242
@@ -147,6 +151,32 @@
#define SYS_SYS_PLL_MEM_REQ 0x2a3
#define SYS_PLL_MEM_STAT 0x2a4
+/* PLL registers XLP9XX */
+#define SYS_9XX_CPU_PLL_CTRL0(core) (0xc0 + (core * 4))
+#define SYS_9XX_CPU_PLL_CTRL1(core) (0xc1 + (core * 4))
+#define SYS_9XX_CPU_PLL_CTRL2(core) (0xc2 + (core * 4))
+#define SYS_9XX_CPU_PLL_CTRL3(core) (0xc3 + (core * 4))
+#define SYS_9XX_DMC_PLL_CTRL0 0x140
+#define SYS_9XX_DMC_PLL_CTRL1 0x141
+#define SYS_9XX_DMC_PLL_CTRL2 0x142
+#define SYS_9XX_DMC_PLL_CTRL3 0x143
+#define SYS_9XX_PLL_CTRL0 0x144
+#define SYS_9XX_PLL_CTRL1 0x145
+#define SYS_9XX_PLL_CTRL2 0x146
+#define SYS_9XX_PLL_CTRL3 0x147
+
+#define SYS_9XX_PLL_CTRL0_DEVX(x) (0x148 + (x) * 4)
+#define SYS_9XX_PLL_CTRL1_DEVX(x) (0x149 + (x) * 4)
+#define SYS_9XX_PLL_CTRL2_DEVX(x) (0x14a + (x) * 4)
+#define SYS_9XX_PLL_CTRL3_DEVX(x) (0x14b + (x) * 4)
+
+#define SYS_9XX_CPU_PLL_CHG_CTRL 0x188
+#define SYS_9XX_PLL_CHG_CTRL 0x189
+#define SYS_9XX_CLK_DEV_DIS 0x18a
+#define SYS_9XX_CLK_DEV_SEL 0x18b
+#define SYS_9XX_CLK_DEV_DIV 0x18d
+#define SYS_9XX_CLK_DEV_CHG 0x18f
+
/* Registers changed on 9XX */
#define SYS_9XX_POWER_ON_RESET_CFG 0x00
#define SYS_9XX_CHIP_RESET 0x01
@@ -170,6 +200,11 @@
#define nlm_get_fuse_regbase(node) \
(nlm_get_fuse_pcibase(node) + XLP_IO_PCI_HDRSZ)
+#define nlm_get_clock_pcibase(node) \
+ nlm_pcicfg_base(XLP9XX_IO_CLOCK_OFFSET(node))
+#define nlm_get_clock_regbase(node) \
+ (nlm_get_clock_pcibase(node) + XLP_IO_PCI_HDRSZ)
+
unsigned int nlm_get_pic_frequency(int node);
#endif
#endif
diff --git a/arch/mips/include/asm/netlogic/xlp-hal/xlp.h b/arch/mips/include/asm/netlogic/xlp-hal/xlp.h
index 2b0c959..a862b93 100644
--- a/arch/mips/include/asm/netlogic/xlp-hal/xlp.h
+++ b/arch/mips/include/asm/netlogic/xlp-hal/xlp.h
@@ -58,6 +58,10 @@
#define PIC_I2C_1_IRQ 31
#define PIC_I2C_2_IRQ 32
#define PIC_I2C_3_IRQ 33
+#define PIC_SPI_IRQ 34
+#define PIC_NAND_IRQ 37
+#define PIC_SATA_IRQ 38
+#define PIC_GPIO_IRQ 39
#define PIC_PCIE_LINK_MSI_IRQ_BASE 44 /* 44 - 47 MSI IRQ */
#define PIC_PCIE_LINK_MSI_IRQ(i) (44 + (i))
@@ -66,8 +70,9 @@
#define PIC_PCIE_MSIX_IRQ_BASE 48 /* 48 - 51 MSI-X IRQ */
#define PIC_PCIE_MSIX_IRQ(i) (48 + (i))
-#define NLM_MSIX_VEC_BASE 96 /* 96 - 127 - MSIX mapped */
-#define NLM_MSI_VEC_BASE 128 /* 128 -255 - MSI mapped */
+/* XLP9xx and XLP8xx has 128 and 32 MSIX vectors respectively */
+#define NLM_MSIX_VEC_BASE 96 /* 96 - 223 - MSIX mapped */
+#define NLM_MSI_VEC_BASE 224 /* 224 -351 - MSI mapped */
#define NLM_PIC_INDIRECT_VEC_BASE 512
#define NLM_GPIO_VEC_BASE 768
@@ -95,17 +100,19 @@
static inline int cpu_is_xlpii(void)
{
- int chip = read_c0_prid() & 0xff00;
+ int chip = read_c0_prid() & PRID_IMP_MASK;
return chip == PRID_IMP_NETLOGIC_XLP2XX ||
- chip == PRID_IMP_NETLOGIC_XLP9XX;
+ chip == PRID_IMP_NETLOGIC_XLP9XX ||
+ chip == PRID_IMP_NETLOGIC_XLP5XX;
}
static inline int cpu_is_xlp9xx(void)
{
- int chip = read_c0_prid() & 0xff00;
+ int chip = read_c0_prid() & PRID_IMP_MASK;
- return chip == PRID_IMP_NETLOGIC_XLP9XX;
+ return chip == PRID_IMP_NETLOGIC_XLP9XX ||
+ chip == PRID_IMP_NETLOGIC_XLP5XX;
}
#endif /* !__ASSEMBLY__ */
#endif /* _ASM_NLM_XLP_H */
diff --git a/arch/mips/include/asm/nile4.h b/arch/mips/include/asm/nile4.h
index 2e2436d..99e97f8 100644
--- a/arch/mips/include/asm/nile4.h
+++ b/arch/mips/include/asm/nile4.h
@@ -1,7 +1,7 @@
/*
* asm-mips/nile4.h -- NEC Vrc-5074 Nile 4 definitions
*
- * Copyright (C) 2000 Geert Uytterhoeven <geert@sonycom.com>
+ * Copyright (C) 2000 Geert Uytterhoeven <geert@linux-m68k.org>
* Sony Software Development Center Europe (SDCE), Brussels
*
* This file is based on the following documentation:
diff --git a/arch/mips/include/asm/octeon/octeon.h b/arch/mips/include/asm/octeon/octeon.h
index f5d77b9..d781f9e 100644
--- a/arch/mips/include/asm/octeon/octeon.h
+++ b/arch/mips/include/asm/octeon/octeon.h
@@ -211,7 +211,6 @@
extern void octeon_write_lcd(const char *s);
extern void octeon_check_cpu_bist(void);
-extern int octeon_get_boot_debug_flag(void);
extern int octeon_get_boot_uart(void);
struct uart_port;
diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h
index 008324d..539ddd1 100644
--- a/arch/mips/include/asm/pgtable.h
+++ b/arch/mips/include/asm/pgtable.h
@@ -32,6 +32,8 @@
_page_cachable_default)
#define PAGE_KERNEL __pgprot(_PAGE_PRESENT | __READABLE | __WRITEABLE | \
_PAGE_GLOBAL | _page_cachable_default)
+#define PAGE_KERNEL_NC __pgprot(_PAGE_PRESENT | __READABLE | __WRITEABLE | \
+ _PAGE_GLOBAL | _CACHE_CACHABLE_NONCOHERENT)
#define PAGE_USERIO __pgprot(_PAGE_PRESENT | (cpu_has_rixi ? 0 : _PAGE_READ) | _PAGE_WRITE | \
_page_cachable_default)
#define PAGE_KERNEL_UNCACHED __pgprot(_PAGE_PRESENT | __READABLE | \
diff --git a/arch/mips/include/asm/pm-cps.h b/arch/mips/include/asm/pm-cps.h
new file mode 100644
index 0000000..625eda5
--- /dev/null
+++ b/arch/mips/include/asm/pm-cps.h
@@ -0,0 +1,51 @@
+/*
+ * Copyright (C) 2014 Imagination Technologies
+ * Author: Paul Burton <paul.burton@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#ifndef __MIPS_ASM_PM_CPS_H__
+#define __MIPS_ASM_PM_CPS_H__
+
+/*
+ * The CM & CPC can only handle coherence & power control on a per-core basis,
+ * thus in an MT system the VPEs within each core are coupled and can only
+ * enter or exit states requiring CM or CPC assistance in unison.
+ */
+#ifdef CONFIG_MIPS_MT
+# define coupled_coherence cpu_has_mipsmt
+#else
+# define coupled_coherence 0
+#endif
+
+/* Enumeration of possible PM states */
+enum cps_pm_state {
+ CPS_PM_NC_WAIT, /* MIPS wait instruction, non-coherent */
+ CPS_PM_CLOCK_GATED, /* Core clock gated */
+ CPS_PM_POWER_GATED, /* Core power gated */
+ CPS_PM_STATE_COUNT,
+};
+
+/**
+ * cps_pm_support_state - determine whether the system supports a PM state
+ * @state: the state to test for support
+ *
+ * Returns true if the system supports the given state, otherwise false.
+ */
+extern bool cps_pm_support_state(enum cps_pm_state state);
+
+/**
+ * cps_pm_enter_state - enter a PM state
+ * @state: the state to enter
+ *
+ * Enter the given PM state. If coupled_coherence is non-zero then it is
+ * expected that this function be called at approximately the same time on
+ * each coupled CPU. Returns 0 on successful entry & exit, otherwise -errno.
+ */
+extern int cps_pm_enter_state(enum cps_pm_state state);
+
+#endif /* __MIPS_ASM_PM_CPS_H__ */
diff --git a/arch/mips/include/asm/pm.h b/arch/mips/include/asm/pm.h
new file mode 100644
index 0000000..7c03469
--- /dev/null
+++ b/arch/mips/include/asm/pm.h
@@ -0,0 +1,159 @@
+/*
+ * Copyright (C) 2014 Imagination Technologies Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ * PM helper macros for CPU power off (e.g. Suspend-to-RAM).
+ */
+
+#ifndef __ASM_PM_H
+#define __ASM_PM_H
+
+#ifdef __ASSEMBLY__
+
+#include <asm/asm-offsets.h>
+#include <asm/asm.h>
+#include <asm/mipsregs.h>
+#include <asm/regdef.h>
+
+/* Save CPU state to stack for suspend to RAM */
+.macro SUSPEND_SAVE_REGS
+ subu sp, PT_SIZE
+ /* Call preserved GPRs */
+ LONG_S $16, PT_R16(sp)
+ LONG_S $17, PT_R17(sp)
+ LONG_S $18, PT_R18(sp)
+ LONG_S $19, PT_R19(sp)
+ LONG_S $20, PT_R20(sp)
+ LONG_S $21, PT_R21(sp)
+ LONG_S $22, PT_R22(sp)
+ LONG_S $23, PT_R23(sp)
+ LONG_S $28, PT_R28(sp)
+ LONG_S $30, PT_R30(sp)
+ LONG_S $31, PT_R31(sp)
+ /* A couple of CP0 registers with space in pt_regs */
+ mfc0 k0, CP0_STATUS
+ LONG_S k0, PT_STATUS(sp)
+.endm
+
+/* Restore CPU state from stack after resume from RAM */
+.macro RESUME_RESTORE_REGS_RETURN
+ .set push
+ .set noreorder
+ /* A couple of CP0 registers with space in pt_regs */
+ LONG_L k0, PT_STATUS(sp)
+ mtc0 k0, CP0_STATUS
+ /* Call preserved GPRs */
+ LONG_L $16, PT_R16(sp)
+ LONG_L $17, PT_R17(sp)
+ LONG_L $18, PT_R18(sp)
+ LONG_L $19, PT_R19(sp)
+ LONG_L $20, PT_R20(sp)
+ LONG_L $21, PT_R21(sp)
+ LONG_L $22, PT_R22(sp)
+ LONG_L $23, PT_R23(sp)
+ LONG_L $28, PT_R28(sp)
+ LONG_L $30, PT_R30(sp)
+ LONG_L $31, PT_R31(sp)
+ /* Pop and return */
+ jr ra
+ addiu sp, PT_SIZE
+ .set pop
+.endm
+
+/* Get address of static suspend state into t1 */
+.macro LA_STATIC_SUSPEND
+ la t1, mips_static_suspend_state
+.endm
+
+/* Save important CPU state for early restoration to global data */
+.macro SUSPEND_SAVE_STATIC
+#ifdef CONFIG_EVA
+ /*
+ * Segment configuration is saved in global data where it can be easily
+ * reloaded without depending on the segment configuration.
+ */
+ mfc0 k0, CP0_PAGEMASK, 2 /* SegCtl0 */
+ LONG_S k0, SSS_SEGCTL0(t1)
+ mfc0 k0, CP0_PAGEMASK, 3 /* SegCtl1 */
+ LONG_S k0, SSS_SEGCTL1(t1)
+ mfc0 k0, CP0_PAGEMASK, 4 /* SegCtl2 */
+ LONG_S k0, SSS_SEGCTL2(t1)
+#endif
+ /* save stack pointer (pointing to GPRs) */
+ LONG_S sp, SSS_SP(t1)
+.endm
+
+/* Restore important CPU state early from global data */
+.macro RESUME_RESTORE_STATIC
+#ifdef CONFIG_EVA
+ /*
+ * Segment configuration must be restored prior to any access to
+ * allocated memory, as it may reside outside of the legacy kernel
+ * segments.
+ */
+ LONG_L k0, SSS_SEGCTL0(t1)
+ mtc0 k0, CP0_PAGEMASK, 2 /* SegCtl0 */
+ LONG_L k0, SSS_SEGCTL1(t1)
+ mtc0 k0, CP0_PAGEMASK, 3 /* SegCtl1 */
+ LONG_L k0, SSS_SEGCTL2(t1)
+ mtc0 k0, CP0_PAGEMASK, 4 /* SegCtl2 */
+ tlbw_use_hazard
+#endif
+ /* restore stack pointer (pointing to GPRs) */
+ LONG_L sp, SSS_SP(t1)
+.endm
+
+/* flush caches to make sure context has reached memory */
+.macro SUSPEND_CACHE_FLUSH
+ .extern __wback_cache_all
+ .set push
+ .set noreorder
+ la t1, __wback_cache_all
+ LONG_L t0, 0(t1)
+ jalr t0
+ nop
+ .set pop
+ .endm
+
+/* Save suspend state and flush data caches to RAM */
+.macro SUSPEND_SAVE
+ SUSPEND_SAVE_REGS
+ LA_STATIC_SUSPEND
+ SUSPEND_SAVE_STATIC
+ SUSPEND_CACHE_FLUSH
+.endm
+
+/* Restore saved state after resume from RAM and return */
+.macro RESUME_RESTORE_RETURN
+ LA_STATIC_SUSPEND
+ RESUME_RESTORE_STATIC
+ RESUME_RESTORE_REGS_RETURN
+.endm
+
+#else /* __ASSEMBLY__ */
+
+/**
+ * struct mips_static_suspend_state - Core saved CPU state across S2R.
+ * @segctl: CP0 Segment control registers.
+ * @sp: Stack frame where GP register context is saved.
+ *
+ * This structure contains minimal CPU state that must be saved in static kernel
+ * data in order to be able to restore the rest of the state. This includes
+ * segmentation configuration in the case of EVA being enabled, as they must be
+ * restored prior to any kmalloc'd memory being referenced (even the stack
+ * pointer).
+ */
+struct mips_static_suspend_state {
+#ifdef CONFIG_EVA
+ unsigned long segctl[3];
+#endif
+ unsigned long sp;
+};
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_PM_HELPERS_H */
diff --git a/arch/mips/include/asm/ptrace.h b/arch/mips/include/asm/ptrace.h
index bf1ac8d3..7e6e682a 100644
--- a/arch/mips/include/asm/ptrace.h
+++ b/arch/mips/include/asm/ptrace.h
@@ -39,9 +39,6 @@
unsigned long cp0_badvaddr;
unsigned long cp0_cause;
unsigned long cp0_epc;
-#ifdef CONFIG_MIPS_MT_SMTC
- unsigned long cp0_tcstatus;
-#endif /* CONFIG_MIPS_MT_SMTC */
#ifdef CONFIG_CPU_CAVIUM_OCTEON
unsigned long long mpl[3]; /* MTM{0,1,2} */
unsigned long long mtp[3]; /* MTP{0,1,2} */
diff --git a/arch/mips/include/asm/r4kcache.h b/arch/mips/include/asm/r4kcache.h
index ca64cbe..0b8bd28 100644
--- a/arch/mips/include/asm/r4kcache.h
+++ b/arch/mips/include/asm/r4kcache.h
@@ -43,11 +43,10 @@
: "i" (op), "R" (*(unsigned char *)(addr)))
#ifdef CONFIG_MIPS_MT
-/*
- * Temporary hacks for SMTC debug. Optionally force single-threaded
- * execution during I-cache flushes.
- */
+/*
+ * Optionally force single-threaded execution during I-cache flushes.
+ */
#define PROTECT_CACHE_FLUSHES 1
#ifdef PROTECT_CACHE_FLUSHES
@@ -524,6 +523,8 @@
__BUILD_BLAST_CACHE(d, dcache, Index_Writeback_Inv_D, Hit_Writeback_Inv_D, 64, )
__BUILD_BLAST_CACHE(i, icache, Index_Invalidate_I, Hit_Invalidate_I, 64, )
__BUILD_BLAST_CACHE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 64, )
+__BUILD_BLAST_CACHE(d, dcache, Index_Writeback_Inv_D, Hit_Writeback_Inv_D, 128, )
+__BUILD_BLAST_CACHE(i, icache, Index_Invalidate_I, Hit_Invalidate_I, 128, )
__BUILD_BLAST_CACHE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 128, )
__BUILD_BLAST_CACHE(inv_d, dcache, Index_Writeback_Inv_D, Hit_Invalidate_D, 16, )
diff --git a/arch/mips/include/asm/sgi/ip22.h b/arch/mips/include/asm/sgi/ip22.h
index 8db1a35..87ec9eaa 100644
--- a/arch/mips/include/asm/sgi/ip22.h
+++ b/arch/mips/include/asm/sgi/ip22.h
@@ -69,6 +69,8 @@
#define SGI_EISA_IRQ SGINT_LOCAL2 + 3 /* EISA interrupts */
#define SGI_KEYBD_IRQ SGINT_LOCAL2 + 4 /* keyboard */
#define SGI_SERIAL_IRQ SGINT_LOCAL2 + 5 /* onboard serial */
+#define SGI_GIOEXP0_IRQ (SGINT_LOCAL2 + 6) /* Indy GIO EXP0 */
+#define SGI_GIOEXP1_IRQ (SGINT_LOCAL2 + 7) /* Indy GIO EXP1 */
#define ip22_is_fullhouse() (sgioc->sysid & SGIOC_SYSID_FULLHOUSE)
diff --git a/arch/mips/include/asm/smp-cps.h b/arch/mips/include/asm/smp-cps.h
index d60d1a2..a06a08a 100644
--- a/arch/mips/include/asm/smp-cps.h
+++ b/arch/mips/include/asm/smp-cps.h
@@ -13,17 +13,28 @@
#ifndef __ASSEMBLY__
-struct boot_config {
- unsigned int core;
- unsigned int vpe;
+struct vpe_boot_config {
unsigned long pc;
unsigned long sp;
unsigned long gp;
};
-extern struct boot_config mips_cps_bootcfg;
+struct core_boot_config {
+ atomic_t vpe_mask;
+ struct vpe_boot_config *vpe_config;
+};
+
+extern struct core_boot_config *mips_cps_core_bootcfg;
extern void mips_cps_core_entry(void);
+extern void mips_cps_core_init(void);
+
+extern struct vpe_boot_config *mips_cps_boot_vpes(void);
+
+extern bool mips_cps_smp_in_use(void);
+
+extern void mips_cps_pm_save(void);
+extern void mips_cps_pm_restore(void);
#else /* __ASSEMBLY__ */
diff --git a/arch/mips/include/asm/smp-ops.h b/arch/mips/include/asm/smp-ops.h
index 73d35b1..6ba1fb8 100644
--- a/arch/mips/include/asm/smp-ops.h
+++ b/arch/mips/include/asm/smp-ops.h
@@ -26,7 +26,6 @@
void (*send_ipi_mask)(const struct cpumask *mask, unsigned int action);
void (*init_secondary)(void);
void (*smp_finish)(void);
- void (*cpus_done)(void);
void (*boot_secondary)(int cpu, struct task_struct *idle);
void (*smp_setup)(void);
void (*prepare_cpus)(unsigned int max_cpus);
diff --git a/arch/mips/include/asm/smp.h b/arch/mips/include/asm/smp.h
index efa02ac..b037334 100644
--- a/arch/mips/include/asm/smp.h
+++ b/arch/mips/include/asm/smp.h
@@ -46,6 +46,9 @@
extern volatile cpumask_t cpu_callin_map;
+/* Mask of CPUs which are currently definitely operating coherently */
+extern cpumask_t cpu_coherent_mask;
+
extern void asmlinkage smp_bootstrap(void);
/*
diff --git a/arch/mips/include/asm/smtc.h b/arch/mips/include/asm/smtc.h
deleted file mode 100644
index e56b439..0000000
--- a/arch/mips/include/asm/smtc.h
+++ /dev/null
@@ -1,78 +0,0 @@
-#ifndef _ASM_SMTC_MT_H
-#define _ASM_SMTC_MT_H
-
-/*
- * Definitions for SMTC multitasking on MIPS MT cores
- */
-
-#include <asm/mips_mt.h>
-#include <asm/smtc_ipi.h>
-
-/*
- * System-wide SMTC status information
- */
-
-extern unsigned int smtc_status;
-
-#define SMTC_TLB_SHARED 0x00000001
-#define SMTC_MTC_ACTIVE 0x00000002
-
-/*
- * TLB/ASID Management information
- */
-
-#define MAX_SMTC_TLBS 2
-#define MAX_SMTC_ASIDS 256
-#if NR_CPUS <= 8
-typedef char asiduse;
-#else
-#if NR_CPUS <= 16
-typedef short asiduse;
-#else
-typedef long asiduse;
-#endif
-#endif
-
-/*
- * VPE Management information
- */
-
-#define MAX_SMTC_VPES MAX_SMTC_TLBS /* FIXME: May not always be true. */
-
-extern asiduse smtc_live_asid[MAX_SMTC_TLBS][MAX_SMTC_ASIDS];
-
-struct mm_struct;
-struct task_struct;
-
-void smtc_get_new_mmu_context(struct mm_struct *mm, unsigned long cpu);
-void self_ipi(struct smtc_ipi *);
-void smtc_flush_tlb_asid(unsigned long asid);
-extern int smtc_build_cpu_map(int startslot);
-extern void smtc_prepare_cpus(int cpus);
-extern void smtc_smp_finish(void);
-extern void smtc_boot_secondary(int cpu, struct task_struct *t);
-extern void smtc_cpus_done(void);
-extern void smtc_init_secondary(void);
-
-
-/*
- * Sharing the TLB between multiple VPEs means that the
- * "random" index selection function is not allowed to
- * select the current value of the Index register. To
- * avoid additional TLB pressure, the Index registers
- * are "parked" with an non-Valid value.
- */
-
-#define PARKED_INDEX ((unsigned int)0x80000000)
-
-/*
- * Define low-level interrupt mask for IPIs, if necessary.
- * By default, use SW interrupt 1, which requires no external
- * hardware support, but which works only for single-core
- * MIPS MT systems.
- */
-#ifndef MIPS_CPU_IPI_IRQ
-#define MIPS_CPU_IPI_IRQ 1
-#endif
-
-#endif /* _ASM_SMTC_MT_H */
diff --git a/arch/mips/include/asm/smtc_ipi.h b/arch/mips/include/asm/smtc_ipi.h
deleted file mode 100644
index 15278db..0000000
--- a/arch/mips/include/asm/smtc_ipi.h
+++ /dev/null
@@ -1,129 +0,0 @@
-/*
- * Definitions used in MIPS MT SMTC "Interprocessor Interrupt" code.
- */
-#ifndef __ASM_SMTC_IPI_H
-#define __ASM_SMTC_IPI_H
-
-#include <linux/spinlock.h>
-
-//#define SMTC_IPI_DEBUG
-
-#ifdef SMTC_IPI_DEBUG
-#include <asm/mipsregs.h>
-#include <asm/mipsmtregs.h>
-#endif /* SMTC_IPI_DEBUG */
-
-/*
- * An IPI "message"
- */
-
-struct smtc_ipi {
- struct smtc_ipi *flink;
- int type;
- void *arg;
- int dest;
-#ifdef SMTC_IPI_DEBUG
- int sender;
- long stamp;
-#endif /* SMTC_IPI_DEBUG */
-};
-
-/*
- * Defined IPI Types
- */
-
-#define LINUX_SMP_IPI 1
-#define SMTC_CLOCK_TICK 2
-#define IRQ_AFFINITY_IPI 3
-
-/*
- * A queue of IPI messages
- */
-
-struct smtc_ipi_q {
- struct smtc_ipi *head;
- spinlock_t lock;
- struct smtc_ipi *tail;
- int depth;
- int resched_flag; /* reschedule already queued */
-};
-
-static inline void smtc_ipi_nq(struct smtc_ipi_q *q, struct smtc_ipi *p)
-{
- unsigned long flags;
-
- spin_lock_irqsave(&q->lock, flags);
- if (q->head == NULL)
- q->head = q->tail = p;
- else
- q->tail->flink = p;
- p->flink = NULL;
- q->tail = p;
- q->depth++;
-#ifdef SMTC_IPI_DEBUG
- p->sender = read_c0_tcbind();
- p->stamp = read_c0_count();
-#endif /* SMTC_IPI_DEBUG */
- spin_unlock_irqrestore(&q->lock, flags);
-}
-
-static inline struct smtc_ipi *__smtc_ipi_dq(struct smtc_ipi_q *q)
-{
- struct smtc_ipi *p;
-
- if (q->head == NULL)
- p = NULL;
- else {
- p = q->head;
- q->head = q->head->flink;
- q->depth--;
- /* Arguably unnecessary, but leaves queue cleaner */
- if (q->head == NULL)
- q->tail = NULL;
- }
-
- return p;
-}
-
-static inline struct smtc_ipi *smtc_ipi_dq(struct smtc_ipi_q *q)
-{
- unsigned long flags;
- struct smtc_ipi *p;
-
- spin_lock_irqsave(&q->lock, flags);
- p = __smtc_ipi_dq(q);
- spin_unlock_irqrestore(&q->lock, flags);
-
- return p;
-}
-
-static inline void smtc_ipi_req(struct smtc_ipi_q *q, struct smtc_ipi *p)
-{
- unsigned long flags;
-
- spin_lock_irqsave(&q->lock, flags);
- if (q->head == NULL) {
- q->head = q->tail = p;
- p->flink = NULL;
- } else {
- p->flink = q->head;
- q->head = p;
- }
- q->depth++;
- spin_unlock_irqrestore(&q->lock, flags);
-}
-
-static inline int smtc_ipi_qdepth(struct smtc_ipi_q *q)
-{
- unsigned long flags;
- int retval;
-
- spin_lock_irqsave(&q->lock, flags);
- retval = q->depth;
- spin_unlock_irqrestore(&q->lock, flags);
- return retval;
-}
-
-extern void smtc_send_ipi(int cpu, int type, unsigned int action);
-
-#endif /* __ASM_SMTC_IPI_H */
diff --git a/arch/mips/include/asm/smtc_proc.h b/arch/mips/include/asm/smtc_proc.h
deleted file mode 100644
index 25da651..0000000
--- a/arch/mips/include/asm/smtc_proc.h
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Definitions for SMTC /proc entries
- * Copyright(C) 2005 MIPS Technologies Inc.
- */
-#ifndef __ASM_SMTC_PROC_H
-#define __ASM_SMTC_PROC_H
-
-/*
- * per-"CPU" statistics
- */
-
-struct smtc_cpu_proc {
- unsigned long timerints;
- unsigned long selfipis;
-};
-
-extern struct smtc_cpu_proc smtc_cpu_stats[NR_CPUS];
-
-/* Count of number of recoveries of "stolen" FPU access rights on 34K */
-
-extern atomic_t smtc_fpu_recoveries;
-
-#endif /* __ASM_SMTC_PROC_H */
diff --git a/arch/mips/include/asm/stackframe.h b/arch/mips/include/asm/stackframe.h
index d301e10..b188c797 100644
--- a/arch/mips/include/asm/stackframe.h
+++ b/arch/mips/include/asm/stackframe.h
@@ -19,22 +19,12 @@
#include <asm/asm-offsets.h>
#include <asm/thread_info.h>
-/*
- * For SMTC kernel, global IE should be left set, and interrupts
- * controlled exclusively via IXMT.
- */
-#ifdef CONFIG_MIPS_MT_SMTC
-#define STATMASK 0x1e
-#elif defined(CONFIG_CPU_R3000) || defined(CONFIG_CPU_TX39XX)
+#if defined(CONFIG_CPU_R3000) || defined(CONFIG_CPU_TX39XX)
#define STATMASK 0x3f
#else
#define STATMASK 0x1f
#endif
-#ifdef CONFIG_MIPS_MT_SMTC
-#include <asm/mipsmtregs.h>
-#endif /* CONFIG_MIPS_MT_SMTC */
-
.macro SAVE_AT
.set push
.set noat
@@ -186,16 +176,6 @@
mfc0 v1, CP0_STATUS
LONG_S $2, PT_R2(sp)
LONG_S v1, PT_STATUS(sp)
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * Ideally, these instructions would be shuffled in
- * to cover the pipeline delay.
- */
- .set mips32
- mfc0 k0, CP0_TCSTATUS
- .set mips0
- LONG_S k0, PT_TCSTATUS(sp)
-#endif /* CONFIG_MIPS_MT_SMTC */
LONG_S $4, PT_R4(sp)
mfc0 v1, CP0_CAUSE
LONG_S $5, PT_R5(sp)
@@ -321,36 +301,6 @@
.set push
.set reorder
.set noat
-#ifdef CONFIG_MIPS_MT_SMTC
- .set mips32r2
- /*
- * We need to make sure the read-modify-write
- * of Status below isn't perturbed by an interrupt
- * or cross-TC access, so we need to do at least a DMT,
- * protected by an interrupt-inhibit. But setting IXMT
- * also creates a few-cycle window where an IPI could
- * be queued and not be detected before potentially
- * returning to a WAIT or user-mode loop. It must be
- * replayed.
- *
- * We're in the middle of a context switch, and
- * we can't dispatch it directly without trashing
- * some registers, so we'll try to detect this unlikely
- * case and program a software interrupt in the VPE,
- * as would be done for a cross-VPE IPI. To accommodate
- * the handling of that case, we're doing a DVPE instead
- * of just a DMT here to protect against other threads.
- * This is a lot of cruft to cover a tiny window.
- * If you can find a better design, implement it!
- *
- */
- mfc0 v0, CP0_TCSTATUS
- ori v0, TCSTATUS_IXMT
- mtc0 v0, CP0_TCSTATUS
- _ehb
- DVPE 5 # dvpe a1
- jal mips_ihb
-#endif /* CONFIG_MIPS_MT_SMTC */
mfc0 a0, CP0_STATUS
ori a0, STATMASK
xori a0, STATMASK
@@ -362,59 +312,6 @@
and v0, v1
or v0, a0
mtc0 v0, CP0_STATUS
-#ifdef CONFIG_MIPS_MT_SMTC
-/*
- * Only after EXL/ERL have been restored to status can we
- * restore TCStatus.IXMT.
- */
- LONG_L v1, PT_TCSTATUS(sp)
- _ehb
- mfc0 a0, CP0_TCSTATUS
- andi v1, TCSTATUS_IXMT
- bnez v1, 0f
-
-/*
- * We'd like to detect any IPIs queued in the tiny window
- * above and request an software interrupt to service them
- * when we ERET.
- *
- * Computing the offset into the IPIQ array of the executing
- * TC's IPI queue in-line would be tedious. We use part of
- * the TCContext register to hold 16 bits of offset that we
- * can add in-line to find the queue head.
- */
- mfc0 v0, CP0_TCCONTEXT
- la a2, IPIQ
- srl v0, v0, 16
- addu a2, a2, v0
- LONG_L v0, 0(a2)
- beqz v0, 0f
-/*
- * If we have a queue, provoke dispatch within the VPE by setting C_SW1
- */
- mfc0 v0, CP0_CAUSE
- ori v0, v0, C_SW1
- mtc0 v0, CP0_CAUSE
-0:
- /*
- * This test should really never branch but
- * let's be prudent here. Having atomized
- * the shared register modifications, we can
- * now EVPE, and must do so before interrupts
- * are potentially re-enabled.
- */
- andi a1, a1, MVPCONTROL_EVP
- beqz a1, 1f
- evpe
-1:
- /* We know that TCStatua.IXMT should be set from above */
- xori a0, a0, TCSTATUS_IXMT
- or a0, a0, v1
- mtc0 a0, CP0_TCSTATUS
- _ehb
-
- .set mips0
-#endif /* CONFIG_MIPS_MT_SMTC */
LONG_L v1, PT_EPC(sp)
MTC0 v1, CP0_EPC
LONG_L $31, PT_R31(sp)
@@ -467,33 +364,11 @@
* Set cp0 enable bit as sign that we're running on the kernel stack
*/
.macro CLI
-#if !defined(CONFIG_MIPS_MT_SMTC)
mfc0 t0, CP0_STATUS
li t1, ST0_CU0 | STATMASK
or t0, t1
xori t0, STATMASK
mtc0 t0, CP0_STATUS
-#else /* CONFIG_MIPS_MT_SMTC */
- /*
- * For SMTC, we need to set privilege
- * and disable interrupts only for the
- * current TC, using the TCStatus register.
- */
- mfc0 t0, CP0_TCSTATUS
- /* Fortunately CU 0 is in the same place in both registers */
- /* Set TCU0, TMX, TKSU (for later inversion) and IXMT */
- li t1, ST0_CU0 | 0x08001c00
- or t0, t1
- /* Clear TKSU, leave IXMT */
- xori t0, 0x00001800
- mtc0 t0, CP0_TCSTATUS
- _ehb
- /* We need to leave the global IE bit set, but clear EXL...*/
- mfc0 t0, CP0_STATUS
- ori t0, ST0_EXL | ST0_ERL
- xori t0, ST0_EXL | ST0_ERL
- mtc0 t0, CP0_STATUS
-#endif /* CONFIG_MIPS_MT_SMTC */
irq_disable_hazard
.endm
@@ -502,35 +377,11 @@
* Set cp0 enable bit as sign that we're running on the kernel stack
*/
.macro STI
-#if !defined(CONFIG_MIPS_MT_SMTC)
mfc0 t0, CP0_STATUS
li t1, ST0_CU0 | STATMASK
or t0, t1
xori t0, STATMASK & ~1
mtc0 t0, CP0_STATUS
-#else /* CONFIG_MIPS_MT_SMTC */
- /*
- * For SMTC, we need to set privilege
- * and enable interrupts only for the
- * current TC, using the TCStatus register.
- */
- _ehb
- mfc0 t0, CP0_TCSTATUS
- /* Fortunately CU 0 is in the same place in both registers */
- /* Set TCU0, TKSU (for later inversion) and IXMT */
- li t1, ST0_CU0 | 0x08001c00
- or t0, t1
- /* Clear TKSU *and* IXMT */
- xori t0, 0x00001c00
- mtc0 t0, CP0_TCSTATUS
- _ehb
- /* We need to leave the global IE bit set, but clear EXL...*/
- mfc0 t0, CP0_STATUS
- ori t0, ST0_EXL
- xori t0, ST0_EXL
- mtc0 t0, CP0_STATUS
- /* irq_enable_hazard below should expand to EHB for 24K/34K cpus */
-#endif /* CONFIG_MIPS_MT_SMTC */
irq_enable_hazard
.endm
@@ -540,32 +391,6 @@
* Set cp0 enable bit as sign that we're running on the kernel stack
*/
.macro KMODE
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * This gets baroque in SMTC. We want to
- * protect the non-atomic clearing of EXL
- * with DMT/EMT, but we don't want to take
- * an interrupt while DMT is still in effect.
- */
-
- /* KMODE gets invoked from both reorder and noreorder code */
- .set push
- .set mips32r2
- .set noreorder
- mfc0 v0, CP0_TCSTATUS
- andi v1, v0, TCSTATUS_IXMT
- ori v0, TCSTATUS_IXMT
- mtc0 v0, CP0_TCSTATUS
- _ehb
- DMT 2 # dmt v0
- /*
- * We don't know a priori if ra is "live"
- */
- move t0, ra
- jal mips_ihb
- nop /* delay slot */
- move ra, t0
-#endif /* CONFIG_MIPS_MT_SMTC */
mfc0 t0, CP0_STATUS
li t1, ST0_CU0 | (STATMASK & ~1)
#if defined(CONFIG_CPU_R3000) || defined(CONFIG_CPU_TX39XX)
@@ -576,25 +401,6 @@
or t0, t1
xori t0, STATMASK & ~1
mtc0 t0, CP0_STATUS
-#ifdef CONFIG_MIPS_MT_SMTC
- _ehb
- andi v0, v0, VPECONTROL_TE
- beqz v0, 2f
- nop /* delay slot */
- emt
-2:
- mfc0 v0, CP0_TCSTATUS
- /* Clear IXMT, then OR in previous value */
- ori v0, TCSTATUS_IXMT
- xori v0, TCSTATUS_IXMT
- or v0, v1, v0
- mtc0 v0, CP0_TCSTATUS
- /*
- * irq_disable_hazard below should expand to EHB
- * on 24K/34K CPUS
- */
- .set pop
-#endif /* CONFIG_MIPS_MT_SMTC */
irq_disable_hazard
.endm
diff --git a/arch/mips/include/asm/thread_info.h b/arch/mips/include/asm/thread_info.h
index d2d961d..7de8658 100644
--- a/arch/mips/include/asm/thread_info.h
+++ b/arch/mips/include/asm/thread_info.h
@@ -159,11 +159,7 @@
* We stash processor id into a COP0 register to retrieve it fast
* at kernel exception entry.
*/
-#if defined(CONFIG_MIPS_MT_SMTC)
-#define SMP_CPUID_REG 2, 2 /* TCBIND */
-#define ASM_SMP_CPUID_REG $2, 2
-#define SMP_CPUID_PTRSHIFT 19
-#elif defined(CONFIG_MIPS_PGD_C0_CONTEXT)
+#if defined(CONFIG_MIPS_PGD_C0_CONTEXT)
#define SMP_CPUID_REG 20, 0 /* XCONTEXT */
#define ASM_SMP_CPUID_REG $20
#define SMP_CPUID_PTRSHIFT 48
@@ -179,13 +175,8 @@
#define SMP_CPUID_REGSHIFT (SMP_CPUID_PTRSHIFT + 2)
#endif
-#ifdef CONFIG_MIPS_MT_SMTC
-#define ASM_CPUID_MFC0 mfc0
-#define UASM_i_CPUID_MFC0 uasm_i_mfc0
-#else
#define ASM_CPUID_MFC0 MFC0
#define UASM_i_CPUID_MFC0 UASM_i_MFC0
-#endif
#endif /* __KERNEL__ */
#endif /* _ASM_THREAD_INFO_H */
diff --git a/arch/mips/include/asm/time.h b/arch/mips/include/asm/time.h
index 24f534a..8f3047d 100644
--- a/arch/mips/include/asm/time.h
+++ b/arch/mips/include/asm/time.h
@@ -52,14 +52,11 @@
*/
extern unsigned int __weak get_c0_compare_int(void);
extern int r4k_clockevent_init(void);
-extern int smtc_clockevent_init(void);
extern int gic_clockevent_init(void);
static inline int mips_clockevent_init(void)
{
-#ifdef CONFIG_MIPS_MT_SMTC
- return smtc_clockevent_init();
-#elif defined(CONFIG_CEVT_GIC)
+#if defined(CONFIG_CEVT_GIC)
return (gic_clockevent_init() | r4k_clockevent_init());
#elif defined(CONFIG_CEVT_R4K)
return r4k_clockevent_init();
diff --git a/arch/mips/include/asm/timex.h b/arch/mips/include/asm/timex.h
index c542475..b05bb70 100644
--- a/arch/mips/include/asm/timex.h
+++ b/arch/mips/include/asm/timex.h
@@ -4,12 +4,16 @@
* for more details.
*
* Copyright (C) 1998, 1999, 2003 by Ralf Baechle
+ * Copyright (C) 2014 by Maciej W. Rozycki
*/
#ifndef _ASM_TIMEX_H
#define _ASM_TIMEX_H
#ifdef __KERNEL__
+#include <linux/compiler.h>
+
+#include <asm/cpu.h>
#include <asm/cpu-features.h>
#include <asm/mipsregs.h>
#include <asm/cpu-type.h>
@@ -45,30 +49,55 @@
* However for now the implementaton of this function doesn't get these
* fine details right.
*/
+static inline int can_use_mips_counter(unsigned int prid)
+{
+ int comp = (prid & PRID_COMP_MASK) != PRID_COMP_LEGACY;
+
+ if (__builtin_constant_p(cpu_has_counter) && !cpu_has_counter)
+ return 0;
+ else if (__builtin_constant_p(cpu_has_mips_r) && cpu_has_mips_r)
+ return 1;
+ else if (likely(!__builtin_constant_p(cpu_has_mips_r) && comp))
+ return 1;
+ /* Make sure we don't peek at cpu_data[0].options in the fast path! */
+ if (!__builtin_constant_p(cpu_has_counter))
+ asm volatile("" : "=m" (cpu_data[0].options));
+ if (likely(cpu_has_counter &&
+ prid >= (PRID_IMP_R4000 | PRID_REV_ENCODE_44(5, 0))))
+ return 1;
+ else
+ return 0;
+}
+
static inline cycles_t get_cycles(void)
{
- switch (boot_cpu_type()) {
- case CPU_R4400PC:
- case CPU_R4400SC:
- case CPU_R4400MC:
- if ((read_c0_prid() & 0xff) >= 0x0050)
- return read_c0_count();
- break;
-
- case CPU_R4000PC:
- case CPU_R4000SC:
- case CPU_R4000MC:
- break;
-
- default:
- if (cpu_has_counter)
- return read_c0_count();
- break;
- }
-
- return 0; /* no usable counter */
+ if (can_use_mips_counter(read_c0_prid()))
+ return read_c0_count();
+ else
+ return 0; /* no usable counter */
}
+/*
+ * Like get_cycles - but where c0_count is not available we desperately
+ * use c0_random in an attempt to get at least a little bit of entropy.
+ *
+ * R6000 and R6000A neither have a count register nor a random register.
+ * That leaves no entropy source in the CPU itself.
+ */
+static inline unsigned long random_get_entropy(void)
+{
+ unsigned int prid = read_c0_prid();
+ unsigned int imp = prid & PRID_IMP_MASK;
+
+ if (can_use_mips_counter(prid))
+ return read_c0_count();
+ else if (likely(imp != PRID_IMP_R6000 && imp != PRID_IMP_R6000A))
+ return read_c0_random();
+ else
+ return 0; /* no usable register */
+}
+#define random_get_entropy random_get_entropy
+
#endif /* __KERNEL__ */
#endif /* _ASM_TIMEX_H */
diff --git a/arch/mips/include/asm/uasm.h b/arch/mips/include/asm/uasm.h
index c33a956..f8d63b3 100644
--- a/arch/mips/include/asm/uasm.h
+++ b/arch/mips/include/asm/uasm.h
@@ -55,6 +55,9 @@
#define Ip_u2u1u3(op) \
void ISAOPC(op)(u32 **buf, unsigned int a, unsigned int b, unsigned int c)
+#define Ip_u3u2u1(op) \
+void ISAOPC(op)(u32 **buf, unsigned int a, unsigned int b, unsigned int c)
+
#define Ip_u3u1u2(op) \
void ISAOPC(op)(u32 **buf, unsigned int a, unsigned int b, unsigned int c)
@@ -74,6 +77,9 @@
#define Ip_u1u2(op) \
void ISAOPC(op)(u32 **buf, unsigned int a, unsigned int b)
+#define Ip_u2u1(op) \
+void ISAOPC(op)(u32 **buf, unsigned int a, unsigned int b)
+
#define Ip_u1s2(op) \
void ISAOPC(op)(u32 **buf, unsigned int a, signed int b)
@@ -99,6 +105,7 @@
Ip_u3u1u2(_daddu);
Ip_u2u1msbu3(_dins);
Ip_u2u1msbu3(_dinsm);
+Ip_u1u2(_divu);
Ip_u1u2u3(_dmfc0);
Ip_u1u2u3(_dmtc0);
Ip_u2u1u3(_drotr);
@@ -114,16 +121,22 @@
Ip_u2u1msbu3(_ins);
Ip_u1(_j);
Ip_u1(_jal);
+Ip_u2u1(_jalr);
Ip_u1(_jr);
+Ip_u2s3u1(_lb);
Ip_u2s3u1(_ld);
Ip_u3u1u2(_ldx);
+Ip_u2s3u1(_lh);
Ip_u2s3u1(_ll);
Ip_u2s3u1(_lld);
Ip_u1s2(_lui);
Ip_u2s3u1(_lw);
Ip_u3u1u2(_lwx);
Ip_u1u2u3(_mfc0);
+Ip_u1(_mfhi);
+Ip_u1(_mflo);
Ip_u1u2u3(_mtc0);
+Ip_u3u1u2(_mul);
Ip_u3u1u2(_or);
Ip_u2u1u3(_ori);
Ip_u2s3u1(_pref);
@@ -133,17 +146,25 @@
Ip_u2s3u1(_scd);
Ip_u2s3u1(_sd);
Ip_u2u1u3(_sll);
+Ip_u3u2u1(_sllv);
+Ip_u2u1s3(_sltiu);
+Ip_u3u1u2(_sltu);
Ip_u2u1u3(_sra);
Ip_u2u1u3(_srl);
+Ip_u3u2u1(_srlv);
Ip_u3u1u2(_subu);
Ip_u2s3u1(_sw);
+Ip_u1(_sync);
Ip_u1(_syscall);
Ip_0(_tlbp);
Ip_0(_tlbr);
Ip_0(_tlbwi);
Ip_0(_tlbwr);
+Ip_u1(_wait);
+Ip_u2u1(_wsbh);
Ip_u3u1u2(_xor);
Ip_u2u1u3(_xori);
+Ip_u2u1(_yield);
/* Handle labels. */
@@ -264,6 +285,8 @@
unsigned int bit, int lid);
void uasm_il_bbit1(u32 **p, struct uasm_reloc **r, unsigned int reg,
unsigned int bit, int lid);
+void uasm_il_beq(u32 **p, struct uasm_reloc **r, unsigned int r1,
+ unsigned int r2, int lid);
void uasm_il_beqz(u32 **p, struct uasm_reloc **r, unsigned int reg, int lid);
void uasm_il_beqzl(u32 **p, struct uasm_reloc **r, unsigned int reg, int lid);
void uasm_il_bgezl(u32 **p, struct uasm_reloc **r, unsigned int reg, int lid);
diff --git a/arch/mips/include/uapi/asm/Kbuild b/arch/mips/include/uapi/asm/Kbuild
index be7196e..96fe739 100644
--- a/arch/mips/include/uapi/asm/Kbuild
+++ b/arch/mips/include/uapi/asm/Kbuild
@@ -4,6 +4,7 @@
generic-y += auxvec.h
generic-y += ipcbuf.h
+header-y += bitfield.h
header-y += bitsperlong.h
header-y += break.h
header-y += byteorder.h
diff --git a/arch/mips/include/uapi/asm/bitfield.h b/arch/mips/include/uapi/asm/bitfield.h
new file mode 100644
index 0000000..ad98613
--- /dev/null
+++ b/arch/mips/include/uapi/asm/bitfield.h
@@ -0,0 +1,29 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2014 by Ralf Baechle <ralf@linux-mips.org>
+ */
+#ifndef __UAPI_ASM_BITFIELD_H
+#define __UAPI_ASM_BITFIELD_H
+
+/*
+ * * Damn ... bitfields depend from byteorder :-(
+ * */
+#ifdef __MIPSEB__
+#define __BITFIELD_FIELD(field, more) \
+ field; \
+ more
+
+#elif defined(__MIPSEL__)
+
+#define __BITFIELD_FIELD(field, more) \
+ more \
+ field;
+
+#else /* !defined (__MIPSEB__) && !defined (__MIPSEL__) */
+#error "MIPS but neither __MIPSEL__ nor __MIPSEB__?"
+#endif
+
+#endif /* __UAPI_ASM_BITFIELD_H */
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index 3125797..4b71602 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -13,6 +13,8 @@
#ifndef _UAPI_ASM_INST_H
#define _UAPI_ASM_INST_H
+#include <asm/bitfield.h>
+
/*
* Major opcodes; before MIPS IV cop1x was called cop3.
*/
@@ -74,16 +76,17 @@
enum spec3_op {
ext_op, dextm_op, dextu_op, dext_op,
ins_op, dinsm_op, dinsu_op, dins_op,
- lx_op = 0x0a, lwle_op = 0x19,
- lwre_op = 0x1a, cachee_op = 0x1b,
- sbe_op = 0x1c, she_op = 0x1d,
- sce_op = 0x1e, swe_op = 0x1f,
- bshfl_op = 0x20, swle_op = 0x21,
- swre_op = 0x22, prefe_op = 0x23,
- dbshfl_op = 0x24, lbue_op = 0x28,
- lhue_op = 0x29, lbe_op = 0x2c,
- lhe_op = 0x2d, lle_op = 0x2e,
- lwe_op = 0x2f, rdhwr_op = 0x3b
+ yield_op = 0x09, lx_op = 0x0a,
+ lwle_op = 0x19, lwre_op = 0x1a,
+ cachee_op = 0x1b, sbe_op = 0x1c,
+ she_op = 0x1d, sce_op = 0x1e,
+ swe_op = 0x1f, bshfl_op = 0x20,
+ swle_op = 0x21, swre_op = 0x22,
+ prefe_op = 0x23, dbshfl_op = 0x24,
+ lbue_op = 0x28, lhue_op = 0x29,
+ lbe_op = 0x2c, lhe_op = 0x2d,
+ lle_op = 0x2e, lwe_op = 0x2f,
+ rdhwr_op = 0x3b
};
/*
@@ -125,7 +128,8 @@
enum cop0_coi_func {
tlbr_op = 0x01, tlbwi_op = 0x02,
tlbwr_op = 0x06, tlbp_op = 0x08,
- rfe_op = 0x10, eret_op = 0x18
+ rfe_op = 0x10, eret_op = 0x18,
+ wait_op = 0x20,
};
/*
@@ -202,6 +206,16 @@
};
/*
+ * BSHFL opcodes
+ */
+enum bshfl_func {
+ wsbh_op = 0x2,
+ dshd_op = 0x5,
+ seb_op = 0x10,
+ seh_op = 0x18,
+};
+
+/*
* (microMIPS) Major opcodes.
*/
enum mm_major_op {
@@ -244,17 +258,22 @@
enum mm_32a_minor_op {
mm_sll32_op = 0x000,
mm_ins_op = 0x00c,
+ mm_sllv32_op = 0x010,
mm_ext_op = 0x02c,
mm_pool32axf_op = 0x03c,
mm_srl32_op = 0x040,
mm_sra_op = 0x080,
+ mm_srlv32_op = 0x090,
mm_rotr_op = 0x0c0,
mm_lwxs_op = 0x118,
mm_addu32_op = 0x150,
mm_subu32_op = 0x1d0,
+ mm_wsbh_op = 0x1ec,
+ mm_mul_op = 0x210,
mm_and_op = 0x250,
mm_or32_op = 0x290,
mm_xor32_op = 0x310,
+ mm_sltu_op = 0x390,
};
/*
@@ -294,15 +313,20 @@
mm_mfc0_op = 0x003,
mm_mtc0_op = 0x00b,
mm_tlbp_op = 0x00d,
+ mm_mfhi32_op = 0x035,
mm_jalr_op = 0x03c,
mm_tlbr_op = 0x04d,
+ mm_mflo32_op = 0x075,
mm_jalrhb_op = 0x07c,
mm_tlbwi_op = 0x08d,
mm_tlbwr_op = 0x0cd,
mm_jalrs_op = 0x13c,
mm_jalrshb_op = 0x17c,
+ mm_sync_op = 0x1ad,
mm_syscall_op = 0x22d,
+ mm_wait_op = 0x24d,
mm_eret_op = 0x3cd,
+ mm_divu_op = 0x5dc,
};
/*
@@ -480,24 +504,6 @@
*/
#define MM_NOP16 0x0c00
-/*
- * Damn ... bitfields depend from byteorder :-(
- */
-#ifdef __MIPSEB__
-#define __BITFIELD_FIELD(field, more) \
- field; \
- more
-
-#elif defined(__MIPSEL__)
-
-#define __BITFIELD_FIELD(field, more) \
- more \
- field;
-
-#else /* !defined (__MIPSEB__) && !defined (__MIPSEL__) */
-#error "MIPS but neither __MIPSEL__ nor __MIPSEB__?"
-#endif
-
struct j_format {
__BITFIELD_FIELD(unsigned int opcode : 6, /* Jump format */
__BITFIELD_FIELD(unsigned int target : 26,
diff --git a/arch/mips/include/uapi/asm/kvm_para.h b/arch/mips/include/uapi/asm/kvm_para.h
index 14fab8f..7e16d7c 100644
--- a/arch/mips/include/uapi/asm/kvm_para.h
+++ b/arch/mips/include/uapi/asm/kvm_para.h
@@ -1 +1,5 @@
-#include <asm-generic/kvm_para.h>
+#ifndef _UAPI_ASM_MIPS_KVM_PARA_H
+#define _UAPI_ASM_MIPS_KVM_PARA_H
+
+
+#endif /* _UAPI_ASM_MIPS_KVM_PARA_H */
diff --git a/arch/mips/include/uapi/asm/types.h b/arch/mips/include/uapi/asm/types.h
index 7ac9d0b..f3dd9ff 100644
--- a/arch/mips/include/uapi/asm/types.h
+++ b/arch/mips/include/uapi/asm/types.h
@@ -14,9 +14,12 @@
/*
* We don't use int-l64.h for the kernel anymore but still use it for
* userspace to avoid code changes.
+ *
+ * However, some user programs (e.g. perf) may not want this. They can
+ * flag __SANE_USERSPACE_TYPES__ to get int-ll64.h here.
*/
#ifndef __KERNEL__
-# if _MIPS_SZLONG == 64
+# if _MIPS_SZLONG == 64 && !defined(__SANE_USERSPACE_TYPES__)
# include <asm-generic/int-l64.h>
# else
# include <asm-generic/int-ll64.h>
diff --git a/arch/mips/kernel/Makefile b/arch/mips/kernel/Makefile
index 277dab3..008a2fe 100644
--- a/arch/mips/kernel/Makefile
+++ b/arch/mips/kernel/Makefile
@@ -17,7 +17,6 @@
obj-$(CONFIG_CEVT_BCM1480) += cevt-bcm1480.o
obj-$(CONFIG_CEVT_R4K) += cevt-r4k.o
-obj-$(CONFIG_MIPS_MT_SMTC) += cevt-smtc.o
obj-$(CONFIG_CEVT_DS1287) += cevt-ds1287.o
obj-$(CONFIG_CEVT_GIC) += cevt-gic.o
obj-$(CONFIG_CEVT_GT641XX) += cevt-gt641xx.o
@@ -42,7 +41,7 @@
obj-$(CONFIG_CPU_R3000) += r2300_fpu.o r2300_switch.o
obj-$(CONFIG_CPU_R6000) += r6000_fpu.o r4k_switch.o
obj-$(CONFIG_CPU_TX39XX) += r2300_fpu.o r2300_switch.o
-obj-$(CONFIG_CPU_CAVIUM_OCTEON) += octeon_switch.o
+obj-$(CONFIG_CPU_CAVIUM_OCTEON) += r4k_fpu.o octeon_switch.o
obj-$(CONFIG_SMP) += smp.o
obj-$(CONFIG_SMP_UP) += smp-up.o
@@ -50,7 +49,6 @@
obj-$(CONFIG_MIPS_MT) += mips-mt.o
obj-$(CONFIG_MIPS_MT_FPAFF) += mips-mt-fpaff.o
-obj-$(CONFIG_MIPS_MT_SMTC) += smtc.o smtc-asm.o smtc-proc.o
obj-$(CONFIG_MIPS_MT_SMP) += smp-mt.o
obj-$(CONFIG_MIPS_CMP) += smp-cmp.o
obj-$(CONFIG_MIPS_CPS) += smp-cps.o cps-vec.o
@@ -107,6 +105,9 @@
obj-$(CONFIG_MIPS_CM) += mips-cm.o
obj-$(CONFIG_MIPS_CPC) += mips-cpc.o
+obj-$(CONFIG_CPU_PM) += pm.o
+obj-$(CONFIG_MIPS_CPS_PM) += pm-cps.o
+
#
# DSP ASE supported for MIPS32 or MIPS64 Release 2 cores only. It is not
# safe to unconditionnaly use the assembler -mdsp / -mdspr2 switches
diff --git a/arch/mips/kernel/asm-offsets.c b/arch/mips/kernel/asm-offsets.c
index 0ea75c2..02f075d 100644
--- a/arch/mips/kernel/asm-offsets.c
+++ b/arch/mips/kernel/asm-offsets.c
@@ -14,6 +14,7 @@
#include <linux/mm.h>
#include <linux/kbuild.h>
#include <linux/suspend.h>
+#include <asm/pm.h>
#include <asm/ptrace.h>
#include <asm/processor.h>
#include <asm/smp-cps.h>
@@ -64,9 +65,6 @@
OFFSET(PT_BVADDR, pt_regs, cp0_badvaddr);
OFFSET(PT_STATUS, pt_regs, cp0_status);
OFFSET(PT_CAUSE, pt_regs, cp0_cause);
-#ifdef CONFIG_MIPS_MT_SMTC
- OFFSET(PT_TCSTATUS, pt_regs, cp0_tcstatus);
-#endif /* CONFIG_MIPS_MT_SMTC */
#ifdef CONFIG_CPU_CAVIUM_OCTEON
OFFSET(PT_MPL, pt_regs, mpl);
OFFSET(PT_MTP, pt_regs, mtp);
@@ -404,6 +402,20 @@
}
#endif
+#ifdef CONFIG_CPU_PM
+void output_pm_defines(void)
+{
+ COMMENT(" PM offsets. ");
+#ifdef CONFIG_EVA
+ OFFSET(SSS_SEGCTL0, mips_static_suspend_state, segctl[0]);
+ OFFSET(SSS_SEGCTL1, mips_static_suspend_state, segctl[1]);
+ OFFSET(SSS_SEGCTL2, mips_static_suspend_state, segctl[2]);
+#endif
+ OFFSET(SSS_SP, mips_static_suspend_state, sp);
+ BLANK();
+}
+#endif
+
void output_kvm_defines(void)
{
COMMENT(" KVM/MIPS Specfic offsets. ");
@@ -472,10 +484,14 @@
void output_cps_defines(void)
{
COMMENT(" MIPS CPS offsets. ");
- OFFSET(BOOTCFG_CORE, boot_config, core);
- OFFSET(BOOTCFG_VPE, boot_config, vpe);
- OFFSET(BOOTCFG_PC, boot_config, pc);
- OFFSET(BOOTCFG_SP, boot_config, sp);
- OFFSET(BOOTCFG_GP, boot_config, gp);
+
+ OFFSET(COREBOOTCFG_VPEMASK, core_boot_config, vpe_mask);
+ OFFSET(COREBOOTCFG_VPECONFIG, core_boot_config, vpe_config);
+ DEFINE(COREBOOTCFG_SIZE, sizeof(struct core_boot_config));
+
+ OFFSET(VPEBOOTCFG_PC, vpe_boot_config, pc);
+ OFFSET(VPEBOOTCFG_SP, vpe_boot_config, sp);
+ OFFSET(VPEBOOTCFG_GP, vpe_boot_config, gp);
+ DEFINE(VPEBOOTCFG_SIZE, sizeof(struct vpe_boot_config));
}
#endif
diff --git a/arch/mips/kernel/branch.c b/arch/mips/kernel/branch.c
index 76122ff..7b2df22 100644
--- a/arch/mips/kernel/branch.c
+++ b/arch/mips/kernel/branch.c
@@ -48,6 +48,202 @@
return epc;
}
+/* (microMIPS) Convert 16-bit register encoding to 32-bit register encoding. */
+static const unsigned int reg16to32map[8] = {16, 17, 2, 3, 4, 5, 6, 7};
+
+int __mm_isBranchInstr(struct pt_regs *regs, struct mm_decoded_insn dec_insn,
+ unsigned long *contpc)
+{
+ union mips_instruction insn = (union mips_instruction)dec_insn.insn;
+ int bc_false = 0;
+ unsigned int fcr31;
+ unsigned int bit;
+
+ if (!cpu_has_mmips)
+ return 0;
+
+ switch (insn.mm_i_format.opcode) {
+ case mm_pool32a_op:
+ if ((insn.mm_i_format.simmediate & MM_POOL32A_MINOR_MASK) ==
+ mm_pool32axf_op) {
+ switch (insn.mm_i_format.simmediate >>
+ MM_POOL32A_MINOR_SHIFT) {
+ case mm_jalr_op:
+ case mm_jalrhb_op:
+ case mm_jalrs_op:
+ case mm_jalrshb_op:
+ if (insn.mm_i_format.rt != 0) /* Not mm_jr */
+ regs->regs[insn.mm_i_format.rt] =
+ regs->cp0_epc +
+ dec_insn.pc_inc +
+ dec_insn.next_pc_inc;
+ *contpc = regs->regs[insn.mm_i_format.rs];
+ return 1;
+ }
+ }
+ break;
+ case mm_pool32i_op:
+ switch (insn.mm_i_format.rt) {
+ case mm_bltzals_op:
+ case mm_bltzal_op:
+ regs->regs[31] = regs->cp0_epc +
+ dec_insn.pc_inc +
+ dec_insn.next_pc_inc;
+ /* Fall through */
+ case mm_bltz_op:
+ if ((long)regs->regs[insn.mm_i_format.rs] < 0)
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ (insn.mm_i_format.simmediate << 1);
+ else
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ dec_insn.next_pc_inc;
+ return 1;
+ case mm_bgezals_op:
+ case mm_bgezal_op:
+ regs->regs[31] = regs->cp0_epc +
+ dec_insn.pc_inc +
+ dec_insn.next_pc_inc;
+ /* Fall through */
+ case mm_bgez_op:
+ if ((long)regs->regs[insn.mm_i_format.rs] >= 0)
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ (insn.mm_i_format.simmediate << 1);
+ else
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ dec_insn.next_pc_inc;
+ return 1;
+ case mm_blez_op:
+ if ((long)regs->regs[insn.mm_i_format.rs] <= 0)
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ (insn.mm_i_format.simmediate << 1);
+ else
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ dec_insn.next_pc_inc;
+ return 1;
+ case mm_bgtz_op:
+ if ((long)regs->regs[insn.mm_i_format.rs] <= 0)
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ (insn.mm_i_format.simmediate << 1);
+ else
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ dec_insn.next_pc_inc;
+ return 1;
+ case mm_bc2f_op:
+ case mm_bc1f_op:
+ bc_false = 1;
+ /* Fall through */
+ case mm_bc2t_op:
+ case mm_bc1t_op:
+ preempt_disable();
+ if (is_fpu_owner())
+ asm volatile("cfc1\t%0,$31" : "=r" (fcr31));
+ else
+ fcr31 = current->thread.fpu.fcr31;
+ preempt_enable();
+
+ if (bc_false)
+ fcr31 = ~fcr31;
+
+ bit = (insn.mm_i_format.rs >> 2);
+ bit += (bit != 0);
+ bit += 23;
+ if (fcr31 & (1 << bit))
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ (insn.mm_i_format.simmediate << 1);
+ else
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc + dec_insn.next_pc_inc;
+ return 1;
+ }
+ break;
+ case mm_pool16c_op:
+ switch (insn.mm_i_format.rt) {
+ case mm_jalr16_op:
+ case mm_jalrs16_op:
+ regs->regs[31] = regs->cp0_epc +
+ dec_insn.pc_inc + dec_insn.next_pc_inc;
+ /* Fall through */
+ case mm_jr16_op:
+ *contpc = regs->regs[insn.mm_i_format.rs];
+ return 1;
+ }
+ break;
+ case mm_beqz16_op:
+ if ((long)regs->regs[reg16to32map[insn.mm_b1_format.rs]] == 0)
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ (insn.mm_b1_format.simmediate << 1);
+ else
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc + dec_insn.next_pc_inc;
+ return 1;
+ case mm_bnez16_op:
+ if ((long)regs->regs[reg16to32map[insn.mm_b1_format.rs]] != 0)
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ (insn.mm_b1_format.simmediate << 1);
+ else
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc + dec_insn.next_pc_inc;
+ return 1;
+ case mm_b16_op:
+ *contpc = regs->cp0_epc + dec_insn.pc_inc +
+ (insn.mm_b0_format.simmediate << 1);
+ return 1;
+ case mm_beq32_op:
+ if (regs->regs[insn.mm_i_format.rs] ==
+ regs->regs[insn.mm_i_format.rt])
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ (insn.mm_i_format.simmediate << 1);
+ else
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ dec_insn.next_pc_inc;
+ return 1;
+ case mm_bne32_op:
+ if (regs->regs[insn.mm_i_format.rs] !=
+ regs->regs[insn.mm_i_format.rt])
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc +
+ (insn.mm_i_format.simmediate << 1);
+ else
+ *contpc = regs->cp0_epc +
+ dec_insn.pc_inc + dec_insn.next_pc_inc;
+ return 1;
+ case mm_jalx32_op:
+ regs->regs[31] = regs->cp0_epc +
+ dec_insn.pc_inc + dec_insn.next_pc_inc;
+ *contpc = regs->cp0_epc + dec_insn.pc_inc;
+ *contpc >>= 28;
+ *contpc <<= 28;
+ *contpc |= (insn.j_format.target << 2);
+ return 1;
+ case mm_jals32_op:
+ case mm_jal32_op:
+ regs->regs[31] = regs->cp0_epc +
+ dec_insn.pc_inc + dec_insn.next_pc_inc;
+ /* Fall through */
+ case mm_j32_op:
+ *contpc = regs->cp0_epc + dec_insn.pc_inc;
+ *contpc >>= 27;
+ *contpc <<= 27;
+ *contpc |= (insn.j_format.target << 1);
+ set_isa16_mode(*contpc);
+ return 1;
+ }
+ return 0;
+}
+
/*
* Compute return address and emulate branch in microMIPS mode after an
* exception only. It does not handle compact branches/jumps and cannot
@@ -366,7 +562,11 @@
case cop1_op:
preempt_disable();
if (is_fpu_owner())
- asm volatile("cfc1\t%0,$31" : "=r" (fcr31));
+ asm volatile(
+ ".set push\n"
+ "\t.set mips1\n"
+ "\tcfc1\t%0,$31\n"
+ "\t.set pop" : "=r" (fcr31));
else
fcr31 = current->thread.fpu.fcr31;
preempt_enable();
diff --git a/arch/mips/kernel/cevt-gic.c b/arch/mips/kernel/cevt-gic.c
index 594cbbf..6093716 100644
--- a/arch/mips/kernel/cevt-gic.c
+++ b/arch/mips/kernel/cevt-gic.c
@@ -26,7 +26,7 @@
cnt = gic_read_count();
cnt += (u64)delta;
- gic_write_compare(cnt);
+ gic_write_cpu_compare(cnt, cpumask_first(evt->cpumask));
res = ((int)(gic_read_count() - cnt) >= 0) ? -ETIME : 0;
return res;
}
@@ -73,7 +73,8 @@
cd = &per_cpu(gic_clockevent_device, cpu);
cd->name = "MIPS GIC";
- cd->features = CLOCK_EVT_FEAT_ONESHOT;
+ cd->features = CLOCK_EVT_FEAT_ONESHOT |
+ CLOCK_EVT_FEAT_C3STOP;
clockevent_set_clock(cd, gic_frequency);
diff --git a/arch/mips/kernel/cevt-r4k.c b/arch/mips/kernel/cevt-r4k.c
index 50d3f5a..bc127e2 100644
--- a/arch/mips/kernel/cevt-r4k.c
+++ b/arch/mips/kernel/cevt-r4k.c
@@ -12,17 +12,10 @@
#include <linux/smp.h>
#include <linux/irq.h>
-#include <asm/smtc_ipi.h>
#include <asm/time.h>
#include <asm/cevt-r4k.h>
#include <asm/gic.h>
-/*
- * The SMTC Kernel for the 34K, 1004K, et. al. replaces several
- * of these routines with SMTC-specific variants.
- */
-
-#ifndef CONFIG_MIPS_MT_SMTC
static int mips_next_event(unsigned long delta,
struct clock_event_device *evt)
{
@@ -36,8 +29,6 @@
return res;
}
-#endif /* CONFIG_MIPS_MT_SMTC */
-
void mips_set_clock_mode(enum clock_event_mode mode,
struct clock_event_device *evt)
{
@@ -47,7 +38,6 @@
DEFINE_PER_CPU(struct clock_event_device, mips_clockevent_device);
int cp0_timer_irq_installed;
-#ifndef CONFIG_MIPS_MT_SMTC
irqreturn_t c0_compare_interrupt(int irq, void *dev_id)
{
const int r2 = cpu_has_mips_r2;
@@ -72,9 +62,6 @@
/* Clear Count/Compare Interrupt */
write_c0_compare(read_c0_compare());
cd = &per_cpu(mips_clockevent_device, cpu);
-#ifdef CONFIG_CEVT_GIC
- if (!gic_present)
-#endif
cd->event_handler(cd);
}
@@ -82,8 +69,6 @@
return IRQ_HANDLED;
}
-#endif /* Not CONFIG_MIPS_MT_SMTC */
-
struct irqaction c0_compare_irqaction = {
.handler = c0_compare_interrupt,
.flags = IRQF_PERCPU | IRQF_TIMER,
@@ -170,7 +155,6 @@
return 1;
}
-#ifndef CONFIG_MIPS_MT_SMTC
int r4k_clockevent_init(void)
{
unsigned int cpu = smp_processor_id();
@@ -195,7 +179,9 @@
cd = &per_cpu(mips_clockevent_device, cpu);
cd->name = "MIPS";
- cd->features = CLOCK_EVT_FEAT_ONESHOT;
+ cd->features = CLOCK_EVT_FEAT_ONESHOT |
+ CLOCK_EVT_FEAT_C3STOP |
+ CLOCK_EVT_FEAT_PERCPU;
clockevent_set_clock(cd, mips_hpt_frequency);
@@ -210,9 +196,6 @@
cd->set_mode = mips_set_clock_mode;
cd->event_handler = mips_event_handler;
-#ifdef CONFIG_CEVT_GIC
- if (!gic_present)
-#endif
clockevents_register_device(cd);
if (cp0_timer_irq_installed)
@@ -225,4 +208,3 @@
return 0;
}
-#endif /* Not CONFIG_MIPS_MT_SMTC */
diff --git a/arch/mips/kernel/cevt-smtc.c b/arch/mips/kernel/cevt-smtc.c
deleted file mode 100644
index b6cf0a6..0000000
--- a/arch/mips/kernel/cevt-smtc.c
+++ /dev/null
@@ -1,324 +0,0 @@
-/*
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
- *
- * Copyright (C) 2007 MIPS Technologies, Inc.
- * Copyright (C) 2007 Ralf Baechle <ralf@linux-mips.org>
- * Copyright (C) 2008 Kevin D. Kissell, Paralogos sarl
- */
-#include <linux/clockchips.h>
-#include <linux/interrupt.h>
-#include <linux/percpu.h>
-#include <linux/smp.h>
-#include <linux/irq.h>
-
-#include <asm/smtc_ipi.h>
-#include <asm/time.h>
-#include <asm/cevt-r4k.h>
-
-/*
- * Variant clock event timer support for SMTC on MIPS 34K, 1004K
- * or other MIPS MT cores.
- *
- * Notes on SMTC Support:
- *
- * SMTC has multiple microthread TCs pretending to be Linux CPUs.
- * But there's only one Count/Compare pair per VPE, and Compare
- * interrupts are taken opportunisitically by available TCs
- * bound to the VPE with the Count register. The new timer
- * framework provides for global broadcasts, but we really
- * want VPE-level multicasts for best behavior. So instead
- * of invoking the high-level clock-event broadcast code,
- * this version of SMTC support uses the historical SMTC
- * multicast mechanisms "under the hood", appearing to the
- * generic clock layer as if the interrupts are per-CPU.
- *
- * The approach taken here is to maintain a set of NR_CPUS
- * virtual timers, and track which "CPU" needs to be alerted
- * at each event.
- *
- * It's unlikely that we'll see a MIPS MT core with more than
- * 2 VPEs, but we *know* that we won't need to handle more
- * VPEs than we have "CPUs". So NCPUs arrays of NCPUs elements
- * is always going to be overkill, but always going to be enough.
- */
-
-unsigned long smtc_nexttime[NR_CPUS][NR_CPUS];
-static int smtc_nextinvpe[NR_CPUS];
-
-/*
- * Timestamps stored are absolute values to be programmed
- * into Count register. Valid timestamps will never be zero.
- * If a Zero Count value is actually calculated, it is converted
- * to be a 1, which will introduce 1 or two CPU cycles of error
- * roughly once every four billion events, which at 1000 HZ means
- * about once every 50 days. If that's actually a problem, one
- * could alternate squashing 0 to 1 and to -1.
- */
-
-#define MAKEVALID(x) (((x) == 0L) ? 1L : (x))
-#define ISVALID(x) ((x) != 0L)
-
-/*
- * Time comparison is subtle, as it's really truncated
- * modular arithmetic.
- */
-
-#define IS_SOONER(a, b, reference) \
- (((a) - (unsigned long)(reference)) < ((b) - (unsigned long)(reference)))
-
-/*
- * CATCHUP_INCREMENT, used when the function falls behind the counter.
- * Could be an increasing function instead of a constant;
- */
-
-#define CATCHUP_INCREMENT 64
-
-static int mips_next_event(unsigned long delta,
- struct clock_event_device *evt)
-{
- unsigned long flags;
- unsigned int mtflags;
- unsigned long timestamp, reference, previous;
- unsigned long nextcomp = 0L;
- int vpe = current_cpu_data.vpe_id;
- int cpu = smp_processor_id();
- local_irq_save(flags);
- mtflags = dmt();
-
- /*
- * Maintain the per-TC virtual timer
- * and program the per-VPE shared Count register
- * as appropriate here...
- */
- reference = (unsigned long)read_c0_count();
- timestamp = MAKEVALID(reference + delta);
- /*
- * To really model the clock, we have to catch the case
- * where the current next-in-VPE timestamp is the old
- * timestamp for the calling CPE, but the new value is
- * in fact later. In that case, we have to do a full
- * scan and discover the new next-in-VPE CPU id and
- * timestamp.
- */
- previous = smtc_nexttime[vpe][cpu];
- if (cpu == smtc_nextinvpe[vpe] && ISVALID(previous)
- && IS_SOONER(previous, timestamp, reference)) {
- int i;
- int soonest = cpu;
-
- /*
- * Update timestamp array here, so that new
- * value gets considered along with those of
- * other virtual CPUs on the VPE.
- */
- smtc_nexttime[vpe][cpu] = timestamp;
- for_each_online_cpu(i) {
- if (ISVALID(smtc_nexttime[vpe][i])
- && IS_SOONER(smtc_nexttime[vpe][i],
- smtc_nexttime[vpe][soonest], reference)) {
- soonest = i;
- }
- }
- smtc_nextinvpe[vpe] = soonest;
- nextcomp = smtc_nexttime[vpe][soonest];
- /*
- * Otherwise, we don't have to process the whole array rank,
- * we just have to see if the event horizon has gotten closer.
- */
- } else {
- if (!ISVALID(smtc_nexttime[vpe][smtc_nextinvpe[vpe]]) ||
- IS_SOONER(timestamp,
- smtc_nexttime[vpe][smtc_nextinvpe[vpe]], reference)) {
- smtc_nextinvpe[vpe] = cpu;
- nextcomp = timestamp;
- }
- /*
- * Since next-in-VPE may me the same as the executing
- * virtual CPU, we update the array *after* checking
- * its value.
- */
- smtc_nexttime[vpe][cpu] = timestamp;
- }
-
- /*
- * It may be that, in fact, we don't need to update Compare,
- * but if we do, we want to make sure we didn't fall into
- * a crack just behind Count.
- */
- if (ISVALID(nextcomp)) {
- write_c0_compare(nextcomp);
- ehb();
- /*
- * We never return an error, we just make sure
- * that we trigger the handlers as quickly as
- * we can if we fell behind.
- */
- while ((nextcomp - (unsigned long)read_c0_count())
- > (unsigned long)LONG_MAX) {
- nextcomp += CATCHUP_INCREMENT;
- write_c0_compare(nextcomp);
- ehb();
- }
- }
- emt(mtflags);
- local_irq_restore(flags);
- return 0;
-}
-
-
-void smtc_distribute_timer(int vpe)
-{
- unsigned long flags;
- unsigned int mtflags;
- int cpu;
- struct clock_event_device *cd;
- unsigned long nextstamp;
- unsigned long reference;
-
-
-repeat:
- nextstamp = 0L;
- for_each_online_cpu(cpu) {
- /*
- * Find virtual CPUs within the current VPE who have
- * unserviced timer requests whose time is now past.
- */
- local_irq_save(flags);
- mtflags = dmt();
- if (cpu_data[cpu].vpe_id == vpe &&
- ISVALID(smtc_nexttime[vpe][cpu])) {
- reference = (unsigned long)read_c0_count();
- if ((smtc_nexttime[vpe][cpu] - reference)
- > (unsigned long)LONG_MAX) {
- smtc_nexttime[vpe][cpu] = 0L;
- emt(mtflags);
- local_irq_restore(flags);
- /*
- * We don't send IPIs to ourself.
- */
- if (cpu != smp_processor_id()) {
- smtc_send_ipi(cpu, SMTC_CLOCK_TICK, 0);
- } else {
- cd = &per_cpu(mips_clockevent_device, cpu);
- cd->event_handler(cd);
- }
- } else {
- /* Local to VPE but Valid Time not yet reached. */
- if (!ISVALID(nextstamp) ||
- IS_SOONER(smtc_nexttime[vpe][cpu], nextstamp,
- reference)) {
- smtc_nextinvpe[vpe] = cpu;
- nextstamp = smtc_nexttime[vpe][cpu];
- }
- emt(mtflags);
- local_irq_restore(flags);
- }
- } else {
- emt(mtflags);
- local_irq_restore(flags);
-
- }
- }
- /* Reprogram for interrupt at next soonest timestamp for VPE */
- if (ISVALID(nextstamp)) {
- write_c0_compare(nextstamp);
- ehb();
- if ((nextstamp - (unsigned long)read_c0_count())
- > (unsigned long)LONG_MAX)
- goto repeat;
- }
-}
-
-
-irqreturn_t c0_compare_interrupt(int irq, void *dev_id)
-{
- int cpu = smp_processor_id();
-
- /* If we're running SMTC, we've got MIPS MT and therefore MIPS32R2 */
- handle_perf_irq(1);
-
- if (read_c0_cause() & (1 << 30)) {
- /* Clear Count/Compare Interrupt */
- write_c0_compare(read_c0_compare());
- smtc_distribute_timer(cpu_data[cpu].vpe_id);
- }
- return IRQ_HANDLED;
-}
-
-
-int smtc_clockevent_init(void)
-{
- uint64_t mips_freq = mips_hpt_frequency;
- unsigned int cpu = smp_processor_id();
- struct clock_event_device *cd;
- unsigned int irq;
- int i;
- int j;
-
- if (!cpu_has_counter || !mips_hpt_frequency)
- return -ENXIO;
- if (cpu == 0) {
- for (i = 0; i < num_possible_cpus(); i++) {
- smtc_nextinvpe[i] = 0;
- for (j = 0; j < num_possible_cpus(); j++)
- smtc_nexttime[i][j] = 0L;
- }
- /*
- * SMTC also can't have the usablility test
- * run by secondary TCs once Compare is in use.
- */
- if (!c0_compare_int_usable())
- return -ENXIO;
- }
-
- /*
- * With vectored interrupts things are getting platform specific.
- * get_c0_compare_int is a hook to allow a platform to return the
- * interrupt number of it's liking.
- */
- irq = MIPS_CPU_IRQ_BASE + cp0_compare_irq;
- if (get_c0_compare_int)
- irq = get_c0_compare_int();
-
- cd = &per_cpu(mips_clockevent_device, cpu);
-
- cd->name = "MIPS";
- cd->features = CLOCK_EVT_FEAT_ONESHOT;
-
- /* Calculate the min / max delta */
- cd->mult = div_sc((unsigned long) mips_freq, NSEC_PER_SEC, 32);
- cd->shift = 32;
- cd->max_delta_ns = clockevent_delta2ns(0x7fffffff, cd);
- cd->min_delta_ns = clockevent_delta2ns(0x300, cd);
-
- cd->rating = 300;
- cd->irq = irq;
- cd->cpumask = cpumask_of(cpu);
- cd->set_next_event = mips_next_event;
- cd->set_mode = mips_set_clock_mode;
- cd->event_handler = mips_event_handler;
-
- clockevents_register_device(cd);
-
- /*
- * On SMTC we only want to do the data structure
- * initialization and IRQ setup once.
- */
- if (cpu)
- return 0;
- /*
- * And we need the hwmask associated with the c0_compare
- * vector to be initialized.
- */
- irq_hwmask[irq] = (0x100 << cp0_compare_irq);
- if (cp0_timer_irq_installed)
- return 0;
-
- cp0_timer_irq_installed = 1;
-
- setup_irq(irq, &c0_compare_irqaction);
-
- return 0;
-}
diff --git a/arch/mips/kernel/cps-vec.S b/arch/mips/kernel/cps-vec.S
index f7a46db..6f4f739 100644
--- a/arch/mips/kernel/cps-vec.S
+++ b/arch/mips/kernel/cps-vec.S
@@ -14,19 +14,43 @@
#include <asm/asmmacro.h>
#include <asm/cacheops.h>
#include <asm/mipsregs.h>
+#include <asm/mipsmtregs.h>
+#include <asm/pm.h>
-#define GCR_CL_COHERENCE_OFS 0x2008
+#define GCR_CL_COHERENCE_OFS 0x2008
+#define GCR_CL_ID_OFS 0x2028
+
+.extern mips_cm_base
+
+.set noreorder
+
+ /*
+ * Set dest to non-zero if the core supports the MT ASE, else zero. If
+ * MT is not supported then branch to nomt.
+ */
+ .macro has_mt dest, nomt
+ mfc0 \dest, CP0_CONFIG
+ bgez \dest, \nomt
+ mfc0 \dest, CP0_CONFIG, 1
+ bgez \dest, \nomt
+ mfc0 \dest, CP0_CONFIG, 2
+ bgez \dest, \nomt
+ mfc0 \dest, CP0_CONFIG, 3
+ andi \dest, \dest, MIPS_CONF3_MT
+ beqz \dest, \nomt
+ .endm
.section .text.cps-vec
.balign 0x1000
-.set noreorder
LEAF(mips_cps_core_entry)
/*
- * These first 8 bytes will be patched by cps_smp_setup to load the
- * base address of the CM GCRs into register v1.
+ * These first 12 bytes will be patched by cps_smp_setup to load the
+ * base address of the CM GCRs into register v1 and the CCA to use into
+ * register s0.
*/
.quad 0
+ .word 0
/* Check whether we're here due to an NMI */
mfc0 k0, CP0_STATUS
@@ -117,10 +141,11 @@
add a0, a0, t0
dcache_done:
- /* Set Kseg0 cacheable, coherent, write-back, write-allocate */
+ /* Set Kseg0 CCA to that in s0 */
mfc0 t0, CP0_CONFIG
ori t0, 0x7
- xori t0, 0x2
+ xori t0, 0x7
+ or t0, t0, s0
mtc0 t0, CP0_CONFIG
ehb
@@ -134,21 +159,24 @@
jr t0
nop
-1: /* We're up, cached & coherent */
+ /*
+ * We're up, cached & coherent. Perform any further required core-level
+ * initialisation.
+ */
+1: jal mips_cps_core_init
+ nop
/*
- * TODO: We should check the VPE number we intended to boot here, and
- * if non-zero we should start that VPE and stop this one. For
- * the moment this doesn't matter since CPUs are brought up
- * sequentially and in order, but once hotplug is implemented
- * this will need revisiting.
+ * Boot any other VPEs within this core that should be online, and
+ * deactivate this VPE if it should be offline.
*/
+ jal mips_cps_boot_vpes
+ nop
/* Off we go! */
- la t0, mips_cps_bootcfg
- lw t1, BOOTCFG_PC(t0)
- lw gp, BOOTCFG_GP(t0)
- lw sp, BOOTCFG_SP(t0)
+ lw t1, VPEBOOTCFG_PC(v0)
+ lw gp, VPEBOOTCFG_GP(v0)
+ lw sp, VPEBOOTCFG_SP(v0)
jr t1
nop
END(mips_cps_core_entry)
@@ -189,3 +217,271 @@
jr k0
nop
END(excep_ejtag)
+
+LEAF(mips_cps_core_init)
+#ifdef CONFIG_MIPS_MT
+ /* Check that the core implements the MT ASE */
+ has_mt t0, 3f
+ nop
+
+ .set push
+ .set mt
+
+ /* Only allow 1 TC per VPE to execute... */
+ dmt
+
+ /* ...and for the moment only 1 VPE */
+ dvpe
+ la t1, 1f
+ jr.hb t1
+ nop
+
+ /* Enter VPE configuration state */
+1: mfc0 t0, CP0_MVPCONTROL
+ ori t0, t0, MVPCONTROL_VPC
+ mtc0 t0, CP0_MVPCONTROL
+
+ /* Retrieve the number of VPEs within the core */
+ mfc0 t0, CP0_MVPCONF0
+ srl t0, t0, MVPCONF0_PVPE_SHIFT
+ andi t0, t0, (MVPCONF0_PVPE >> MVPCONF0_PVPE_SHIFT)
+ addi t7, t0, 1
+
+ /* If there's only 1, we're done */
+ beqz t0, 2f
+ nop
+
+ /* Loop through each VPE within this core */
+ li t5, 1
+
+1: /* Operate on the appropriate TC */
+ mtc0 t5, CP0_VPECONTROL
+ ehb
+
+ /* Bind TC to VPE (1:1 TC:VPE mapping) */
+ mttc0 t5, CP0_TCBIND
+
+ /* Set exclusive TC, non-active, master */
+ li t0, VPECONF0_MVP
+ sll t1, t5, VPECONF0_XTC_SHIFT
+ or t0, t0, t1
+ mttc0 t0, CP0_VPECONF0
+
+ /* Set TC non-active, non-allocatable */
+ mttc0 zero, CP0_TCSTATUS
+
+ /* Set TC halted */
+ li t0, TCHALT_H
+ mttc0 t0, CP0_TCHALT
+
+ /* Next VPE */
+ addi t5, t5, 1
+ slt t0, t5, t7
+ bnez t0, 1b
+ nop
+
+ /* Leave VPE configuration state */
+2: mfc0 t0, CP0_MVPCONTROL
+ xori t0, t0, MVPCONTROL_VPC
+ mtc0 t0, CP0_MVPCONTROL
+
+3: .set pop
+#endif
+ jr ra
+ nop
+ END(mips_cps_core_init)
+
+LEAF(mips_cps_boot_vpes)
+ /* Retrieve CM base address */
+ la t0, mips_cm_base
+ lw t0, 0(t0)
+
+ /* Calculate a pointer to this cores struct core_boot_config */
+ lw t0, GCR_CL_ID_OFS(t0)
+ li t1, COREBOOTCFG_SIZE
+ mul t0, t0, t1
+ la t1, mips_cps_core_bootcfg
+ lw t1, 0(t1)
+ addu t0, t0, t1
+
+ /* Calculate this VPEs ID. If the core doesn't support MT use 0 */
+ has_mt t6, 1f
+ li t9, 0
+
+ /* Find the number of VPEs present in the core */
+ mfc0 t1, CP0_MVPCONF0
+ srl t1, t1, MVPCONF0_PVPE_SHIFT
+ andi t1, t1, MVPCONF0_PVPE >> MVPCONF0_PVPE_SHIFT
+ addi t1, t1, 1
+
+ /* Calculate a mask for the VPE ID from EBase.CPUNum */
+ clz t1, t1
+ li t2, 31
+ subu t1, t2, t1
+ li t2, 1
+ sll t1, t2, t1
+ addiu t1, t1, -1
+
+ /* Retrieve the VPE ID from EBase.CPUNum */
+ mfc0 t9, $15, 1
+ and t9, t9, t1
+
+1: /* Calculate a pointer to this VPEs struct vpe_boot_config */
+ li t1, VPEBOOTCFG_SIZE
+ mul v0, t9, t1
+ lw t7, COREBOOTCFG_VPECONFIG(t0)
+ addu v0, v0, t7
+
+#ifdef CONFIG_MIPS_MT
+
+ /* If the core doesn't support MT then return */
+ bnez t6, 1f
+ nop
+ jr ra
+ nop
+
+ .set push
+ .set mt
+
+1: /* Enter VPE configuration state */
+ dvpe
+ la t1, 1f
+ jr.hb t1
+ nop
+1: mfc0 t1, CP0_MVPCONTROL
+ ori t1, t1, MVPCONTROL_VPC
+ mtc0 t1, CP0_MVPCONTROL
+ ehb
+
+ /* Loop through each VPE */
+ lw t6, COREBOOTCFG_VPEMASK(t0)
+ move t8, t6
+ li t5, 0
+
+ /* Check whether the VPE should be running. If not, skip it */
+1: andi t0, t6, 1
+ beqz t0, 2f
+ nop
+
+ /* Operate on the appropriate TC */
+ mfc0 t0, CP0_VPECONTROL
+ ori t0, t0, VPECONTROL_TARGTC
+ xori t0, t0, VPECONTROL_TARGTC
+ or t0, t0, t5
+ mtc0 t0, CP0_VPECONTROL
+ ehb
+
+ /* Skip the VPE if its TC is not halted */
+ mftc0 t0, CP0_TCHALT
+ beqz t0, 2f
+ nop
+
+ /* Calculate a pointer to the VPEs struct vpe_boot_config */
+ li t0, VPEBOOTCFG_SIZE
+ mul t0, t0, t5
+ addu t0, t0, t7
+
+ /* Set the TC restart PC */
+ lw t1, VPEBOOTCFG_PC(t0)
+ mttc0 t1, CP0_TCRESTART
+
+ /* Set the TC stack pointer */
+ lw t1, VPEBOOTCFG_SP(t0)
+ mttgpr t1, sp
+
+ /* Set the TC global pointer */
+ lw t1, VPEBOOTCFG_GP(t0)
+ mttgpr t1, gp
+
+ /* Copy config from this VPE */
+ mfc0 t0, CP0_CONFIG
+ mttc0 t0, CP0_CONFIG
+
+ /* Ensure no software interrupts are pending */
+ mttc0 zero, CP0_CAUSE
+ mttc0 zero, CP0_STATUS
+
+ /* Set TC active, not interrupt exempt */
+ mftc0 t0, CP0_TCSTATUS
+ li t1, ~TCSTATUS_IXMT
+ and t0, t0, t1
+ ori t0, t0, TCSTATUS_A
+ mttc0 t0, CP0_TCSTATUS
+
+ /* Clear the TC halt bit */
+ mttc0 zero, CP0_TCHALT
+
+ /* Set VPE active */
+ mftc0 t0, CP0_VPECONF0
+ ori t0, t0, VPECONF0_VPA
+ mttc0 t0, CP0_VPECONF0
+
+ /* Next VPE */
+2: srl t6, t6, 1
+ addi t5, t5, 1
+ bnez t6, 1b
+ nop
+
+ /* Leave VPE configuration state */
+ mfc0 t1, CP0_MVPCONTROL
+ xori t1, t1, MVPCONTROL_VPC
+ mtc0 t1, CP0_MVPCONTROL
+ ehb
+ evpe
+
+ /* Check whether this VPE is meant to be running */
+ li t0, 1
+ sll t0, t0, t9
+ and t0, t0, t8
+ bnez t0, 2f
+ nop
+
+ /* This VPE should be offline, halt the TC */
+ li t0, TCHALT_H
+ mtc0 t0, CP0_TCHALT
+ la t0, 1f
+1: jr.hb t0
+ nop
+
+2: .set pop
+
+#endif /* CONFIG_MIPS_MT */
+
+ /* Return */
+ jr ra
+ nop
+ END(mips_cps_boot_vpes)
+
+#if defined(CONFIG_MIPS_CPS_PM) && defined(CONFIG_CPU_PM)
+
+ /* Calculate a pointer to this CPUs struct mips_static_suspend_state */
+ .macro psstate dest
+ .set push
+ .set noat
+ lw $1, TI_CPU(gp)
+ sll $1, $1, LONGLOG
+ la \dest, __per_cpu_offset
+ addu $1, $1, \dest
+ lw $1, 0($1)
+ la \dest, cps_cpu_state
+ addu \dest, \dest, $1
+ .set pop
+ .endm
+
+LEAF(mips_cps_pm_save)
+ /* Save CPU state */
+ SUSPEND_SAVE_REGS
+ psstate t1
+ SUSPEND_SAVE_STATIC
+ jr v0
+ nop
+ END(mips_cps_pm_save)
+
+LEAF(mips_cps_pm_restore)
+ /* Restore CPU state */
+ psstate t1
+ RESUME_RESTORE_STATIC
+ RESUME_RESTORE_REGS_RETURN
+ END(mips_cps_pm_restore)
+
+#endif /* CONFIG_MIPS_CPS_PM && CONFIG_CPU_PM */
diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c
index 6e8fb85..d74f957 100644
--- a/arch/mips/kernel/cpu-probe.c
+++ b/arch/mips/kernel/cpu-probe.c
@@ -62,7 +62,7 @@
case CPU_34K:
/*
* Erratum "RPS May Cause Incorrect Instruction Execution"
- * This code only handles VPE0, any SMP/SMTC/RTOS code
+ * This code only handles VPE0, any SMP/RTOS code
* making use of VPE1 will be responsable for that VPE.
*/
if ((c->processor_id & PRID_REV_MASK) <= PRID_REV_34K_V1_0_2)
@@ -423,7 +423,7 @@
#ifndef CONFIG_MIPS_CPS
if (cpu_has_mips_r2) {
- c->core = read_c0_ebase() & 0x3ff;
+ c->core = get_ebase_cpunum();
if (cpu_has_mipsmt)
c->core >>= fls(core_nvpes()) - 1;
}
@@ -684,21 +684,6 @@
*/
c->tlbsize = (read_c0_info() & (1 << 29)) ? 64 : 48;
break;
- case PRID_IMP_RM9000:
- c->cputype = CPU_RM9000;
- __cpu_name[cpu] = "RM9000";
- set_isa(c, MIPS_CPU_ISA_IV);
- c->options = R4K_OPTS | MIPS_CPU_FPU | MIPS_CPU_32FPR |
- MIPS_CPU_LLSC;
- /*
- * Bit 29 in the info register of the RM9000
- * indicates if the TLB has 48 or 64 entries.
- *
- * 29 1 => 64 entry JTLB
- * 0 => 48 entry JTLB
- */
- c->tlbsize = (read_c0_info() & (1 << 29)) ? 64 : 48;
- break;
case PRID_IMP_R8000:
c->cputype = CPU_R8000;
__cpu_name[cpu] = "RM8000";
@@ -1041,6 +1026,7 @@
decode_configs(c);
/* JZRISC does not implement the CP0 counter. */
c->options &= ~MIPS_CPU_COUNTER;
+ BUG_ON(!__builtin_constant_p(cpu_has_counter) || cpu_has_counter);
switch (c->processor_id & PRID_IMP_MASK) {
case PRID_IMP_JZRISC:
c->cputype = CPU_JZRISC;
@@ -1074,6 +1060,7 @@
switch (c->processor_id & PRID_IMP_MASK) {
case PRID_IMP_NETLOGIC_XLP2XX:
case PRID_IMP_NETLOGIC_XLP9XX:
+ case PRID_IMP_NETLOGIC_XLP5XX:
c->cputype = CPU_XLP;
__cpu_name[cpu] = "Broadcom XLPII";
break;
diff --git a/arch/mips/kernel/entry.S b/arch/mips/kernel/entry.S
index e578685..4353d32 100644
--- a/arch/mips/kernel/entry.S
+++ b/arch/mips/kernel/entry.S
@@ -16,9 +16,6 @@
#include <asm/isadep.h>
#include <asm/thread_info.h>
#include <asm/war.h>
-#ifdef CONFIG_MIPS_MT_SMTC
-#include <asm/mipsmtregs.h>
-#endif
#ifndef CONFIG_PREEMPT
#define resume_kernel restore_all
@@ -89,41 +86,6 @@
bnez t0, syscall_exit_work
restore_all: # restore full frame
-#ifdef CONFIG_MIPS_MT_SMTC
-#ifdef CONFIG_MIPS_MT_SMTC_IM_BACKSTOP
-/* Re-arm any temporarily masked interrupts not explicitly "acked" */
- mfc0 v0, CP0_TCSTATUS
- ori v1, v0, TCSTATUS_IXMT
- mtc0 v1, CP0_TCSTATUS
- andi v0, TCSTATUS_IXMT
- _ehb
- mfc0 t0, CP0_TCCONTEXT
- DMT 9 # dmt t1
- jal mips_ihb
- mfc0 t2, CP0_STATUS
- andi t3, t0, 0xff00
- or t2, t2, t3
- mtc0 t2, CP0_STATUS
- _ehb
- andi t1, t1, VPECONTROL_TE
- beqz t1, 1f
- EMT
-1:
- mfc0 v1, CP0_TCSTATUS
- /* We set IXMT above, XOR should clear it here */
- xori v1, v1, TCSTATUS_IXMT
- or v1, v0, v1
- mtc0 v1, CP0_TCSTATUS
- _ehb
- xor t0, t0, t3
- mtc0 t0, CP0_TCCONTEXT
-#endif /* CONFIG_MIPS_MT_SMTC_IM_BACKSTOP */
-/* Detect and execute deferred IPI "interrupts" */
- LONG_L s0, TI_REGS($28)
- LONG_S sp, TI_REGS($28)
- jal deferred_smtc_ipi
- LONG_S s0, TI_REGS($28)
-#endif /* CONFIG_MIPS_MT_SMTC */
.set noat
RESTORE_TEMP
RESTORE_AT
diff --git a/arch/mips/kernel/genex.S b/arch/mips/kernel/genex.S
index a9ce340..ac35e12c 100644
--- a/arch/mips/kernel/genex.S
+++ b/arch/mips/kernel/genex.S
@@ -21,20 +21,6 @@
#include <asm/war.h>
#include <asm/thread_info.h>
-#ifdef CONFIG_MIPS_MT_SMTC
-#define PANIC_PIC(msg) \
- .set push; \
- .set nomicromips; \
- .set reorder; \
- PTR_LA a0,8f; \
- .set noat; \
- PTR_LA AT, panic; \
- jr AT; \
-9: b 9b; \
- .set pop; \
- TEXT(msg)
-#endif
-
__INIT
/*
@@ -251,15 +237,6 @@
SAVE_AT
.set push
.set noreorder
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * To keep from blindly blocking *all* interrupts
- * during service by SMTC kernel, we also want to
- * pass the IM value to be cleared.
- */
-FEXPORT(except_vec_vi_mori)
- ori a0, $0, 0
-#endif /* CONFIG_MIPS_MT_SMTC */
PTR_LA v1, except_vec_vi_handler
FEXPORT(except_vec_vi_lui)
lui v0, 0 /* Patched */
@@ -277,37 +254,10 @@
NESTED(except_vec_vi_handler, 0, sp)
SAVE_TEMP
SAVE_STATIC
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * SMTC has an interesting problem that interrupts are level-triggered,
- * and the CLI macro will clear EXL, potentially causing a duplicate
- * interrupt service invocation. So we need to clear the associated
- * IM bit of Status prior to doing CLI, and restore it after the
- * service routine has been invoked - we must assume that the
- * service routine will have cleared the state, and any active
- * level represents a new or otherwised unserviced event...
- */
- mfc0 t1, CP0_STATUS
- and t0, a0, t1
-#ifdef CONFIG_MIPS_MT_SMTC_IM_BACKSTOP
- mfc0 t2, CP0_TCCONTEXT
- or t2, t0, t2
- mtc0 t2, CP0_TCCONTEXT
-#endif /* CONFIG_MIPS_MT_SMTC_IM_BACKSTOP */
- xor t1, t1, t0
- mtc0 t1, CP0_STATUS
- _ehb
-#endif /* CONFIG_MIPS_MT_SMTC */
CLI
#ifdef CONFIG_TRACE_IRQFLAGS
move s0, v0
-#ifdef CONFIG_MIPS_MT_SMTC
- move s1, a0
-#endif
TRACE_IRQS_OFF
-#ifdef CONFIG_MIPS_MT_SMTC
- move a0, s1
-#endif
move v0, s0
#endif
@@ -496,9 +446,6 @@
.align 5
LEAF(handle_ri_rdhwr_vivt)
-#ifdef CONFIG_MIPS_MT_SMTC
- PANIC_PIC("handle_ri_rdhwr_vivt called")
-#else
.set push
.set noat
.set noreorder
@@ -517,7 +464,6 @@
.set pop
bltz k1, handle_ri /* slow path */
/* fall thru */
-#endif
END(handle_ri_rdhwr_vivt)
LEAF(handle_ri_rdhwr)
diff --git a/arch/mips/kernel/head.S b/arch/mips/kernel/head.S
index e712dcf..95afd66 100644
--- a/arch/mips/kernel/head.S
+++ b/arch/mips/kernel/head.S
@@ -35,33 +35,12 @@
*/
.macro setup_c0_status set clr
.set push
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * For SMTC, we need to set privilege and disable interrupts only for
- * the current TC, using the TCStatus register.
- */
- mfc0 t0, CP0_TCSTATUS
- /* Fortunately CU 0 is in the same place in both registers */
- /* Set TCU0, TMX, TKSU (for later inversion) and IXMT */
- li t1, ST0_CU0 | 0x08001c00
- or t0, t1
- /* Clear TKSU, leave IXMT */
- xori t0, 0x00001800
- mtc0 t0, CP0_TCSTATUS
- _ehb
- /* We need to leave the global IE bit set, but clear EXL...*/
- mfc0 t0, CP0_STATUS
- or t0, ST0_CU0 | ST0_EXL | ST0_ERL | \set | \clr
- xor t0, ST0_EXL | ST0_ERL | \clr
- mtc0 t0, CP0_STATUS
-#else
mfc0 t0, CP0_STATUS
or t0, ST0_CU0|\set|0x1f|\clr
xor t0, 0x1f|\clr
mtc0 t0, CP0_STATUS
.set noreorder
sll zero,3 # ehb
-#endif
.set pop
.endm
@@ -115,24 +94,6 @@
jr t0
0:
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * In SMTC kernel, "CLI" is thread-specific, in TCStatus.
- * We still need to enable interrupts globally in Status,
- * and clear EXL/ERL.
- *
- * TCContext is used to track interrupt levels under
- * service in SMTC kernel. Clear for boot TC before
- * allowing any interrupts.
- */
- mtc0 zero, CP0_TCCONTEXT
-
- mfc0 t0, CP0_STATUS
- ori t0, t0, 0xff1f
- xori t0, t0, 0x001e
- mtc0 t0, CP0_STATUS
-#endif /* CONFIG_MIPS_MT_SMTC */
-
PTR_LA t0, __bss_start # clear .bss
LONG_S zero, (t0)
PTR_LA t1, __bss_stop - LONGSIZE
@@ -164,25 +125,8 @@
* function after setting up the stack and gp registers.
*/
NESTED(smp_bootstrap, 16, sp)
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * Read-modify-writes of Status must be atomic, and this
- * is one case where CLI is invoked without EXL being
- * necessarily set. The CLI and setup_c0_status will
- * in fact be redundant for all but the first TC of
- * each VPE being booted.
- */
- DMT 10 # dmt t2 /* t0, t1 are used by CLI and setup_c0_status() */
- jal mips_ihb
-#endif /* CONFIG_MIPS_MT_SMTC */
smp_slave_setup
setup_c0_status_sec
-#ifdef CONFIG_MIPS_MT_SMTC
- andi t2, t2, VPECONTROL_TE
- beqz t2, 2f
- EMT # emt
-2:
-#endif /* CONFIG_MIPS_MT_SMTC */
j start_secondary
END(smp_bootstrap)
#endif /* CONFIG_SMP */
diff --git a/arch/mips/kernel/i8259.c b/arch/mips/kernel/i8259.c
index 2b91fe8..50b3648 100644
--- a/arch/mips/kernel/i8259.c
+++ b/arch/mips/kernel/i8259.c
@@ -42,9 +42,6 @@
.irq_disable = disable_8259A_irq,
.irq_unmask = enable_8259A_irq,
.irq_mask_ack = mask_and_ack_8259A,
-#ifdef CONFIG_MIPS_MT_SMTC_IRQAFF
- .irq_set_affinity = plat_set_irq_affinity,
-#endif /* CONFIG_MIPS_MT_SMTC_IRQAFF */
};
/*
@@ -180,7 +177,6 @@
outb(cached_master_mask, PIC_MASTER_IMR);
outb(0x60+irq, PIC_MASTER_CMD); /* 'Specific EOI to master */
}
- smtc_im_ack_irq(irq);
raw_spin_unlock_irqrestore(&i8259A_lock, flags);
return;
diff --git a/arch/mips/kernel/idle.c b/arch/mips/kernel/idle.c
index 837ff27..09ce459 100644
--- a/arch/mips/kernel/idle.c
+++ b/arch/mips/kernel/idle.c
@@ -224,29 +224,26 @@
cpu_wait = r4k_wait;
*/
break;
- case CPU_RM9000:
- if ((c->processor_id & 0x00ff) >= 0x40)
- cpu_wait = r4k_wait;
- break;
default:
break;
}
}
-static void smtc_idle_hook(void)
-{
-#ifdef CONFIG_MIPS_MT_SMTC
- void smtc_idle_loop_hook(void);
-
- smtc_idle_loop_hook();
-#endif
-}
-
void arch_cpu_idle(void)
{
- smtc_idle_hook();
if (cpu_wait)
cpu_wait();
else
local_irq_enable();
}
+
+#ifdef CONFIG_CPU_IDLE
+
+int mips_cpuidle_wait_enter(struct cpuidle_device *dev,
+ struct cpuidle_driver *drv, int index)
+{
+ arch_cpu_idle();
+ return index;
+}
+
+#endif
diff --git a/arch/mips/kernel/irq-gic.c b/arch/mips/kernel/irq-gic.c
index 8520dad..88e4c32 100644
--- a/arch/mips/kernel/irq-gic.c
+++ b/arch/mips/kernel/irq-gic.c
@@ -54,6 +54,21 @@
(int)(cnt & 0xffffffff));
}
+void gic_write_cpu_compare(cycle_t cnt, int cpu)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ GICWRITE(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), cpu);
+ GICWRITE(GIC_REG(VPE_OTHER, GIC_VPE_COMPARE_HI),
+ (int)(cnt >> 32));
+ GICWRITE(GIC_REG(VPE_OTHER, GIC_VPE_COMPARE_LO),
+ (int)(cnt & 0xffffffff));
+
+ local_irq_restore(flags);
+}
+
cycle_t gic_read_compare(void)
{
unsigned int hi, lo;
diff --git a/arch/mips/kernel/irq-msc01.c b/arch/mips/kernel/irq-msc01.c
index fab40f7..4858642 100644
--- a/arch/mips/kernel/irq-msc01.c
+++ b/arch/mips/kernel/irq-msc01.c
@@ -53,13 +53,9 @@
*/
static void level_mask_and_ack_msc_irq(struct irq_data *d)
{
- unsigned int irq = d->irq;
-
mask_msc_irq(d);
if (!cpu_has_veic)
MSCIC_WRITE(MSC01_IC_EOI, 0);
- /* This actually needs to be a call into platform code */
- smtc_im_ack_irq(irq);
}
/*
@@ -78,7 +74,6 @@
MSCIC_WRITE(MSC01_IC_SUP+irq*8, r | ~MSC01_IC_SUP_EDGE_BIT);
MSCIC_WRITE(MSC01_IC_SUP+irq*8, r);
}
- smtc_im_ack_irq(irq);
}
/*
diff --git a/arch/mips/kernel/irq.c b/arch/mips/kernel/irq.c
index 1818da4..d2bfbc2 100644
--- a/arch/mips/kernel/irq.c
+++ b/arch/mips/kernel/irq.c
@@ -73,7 +73,6 @@
*/
void ack_bad_irq(unsigned int irq)
{
- smtc_im_ack_irq(irq);
printk("unexpected IRQ # %d\n", irq);
}
@@ -142,23 +141,7 @@
{
irq_enter();
check_stack_overflow();
- if (!smtc_handle_on_other_cpu(irq))
- generic_handle_irq(irq);
- irq_exit();
-}
-
-#ifdef CONFIG_MIPS_MT_SMTC_IRQAFF
-/*
- * To avoid inefficient and in some cases pathological re-checking of
- * IRQ affinity, we have this variant that skips the affinity check.
- */
-
-void __irq_entry do_IRQ_no_affinity(unsigned int irq)
-{
- irq_enter();
- smtc_im_backstop(irq);
generic_handle_irq(irq);
irq_exit();
}
-#endif /* CONFIG_MIPS_MT_SMTC_IRQAFF */
diff --git a/arch/mips/kernel/mips-cpc.c b/arch/mips/kernel/mips-cpc.c
index c9dc674..ba47360 100644
--- a/arch/mips/kernel/mips-cpc.c
+++ b/arch/mips/kernel/mips-cpc.c
@@ -9,12 +9,18 @@
*/
#include <linux/errno.h>
+#include <linux/percpu.h>
+#include <linux/spinlock.h>
#include <asm/mips-cm.h>
#include <asm/mips-cpc.h>
void __iomem *mips_cpc_base;
+static DEFINE_PER_CPU_ALIGNED(spinlock_t, cpc_core_lock);
+
+static DEFINE_PER_CPU_ALIGNED(unsigned long, cpc_core_lock_flags);
+
phys_t __weak mips_cpc_phys_base(void)
{
u32 cpc_base;
@@ -39,6 +45,10 @@
int mips_cpc_probe(void)
{
phys_t addr;
+ unsigned cpu;
+
+ for_each_possible_cpu(cpu)
+ spin_lock_init(&per_cpu(cpc_core_lock, cpu));
addr = mips_cpc_phys_base();
if (!addr)
@@ -50,3 +60,21 @@
return 0;
}
+
+void mips_cpc_lock_other(unsigned int core)
+{
+ unsigned curr_core;
+ preempt_disable();
+ curr_core = current_cpu_data.core;
+ spin_lock_irqsave(&per_cpu(cpc_core_lock, curr_core),
+ per_cpu(cpc_core_lock_flags, curr_core));
+ write_cpc_cl_other(core << CPC_Cx_OTHER_CORENUM_SHF);
+}
+
+void mips_cpc_unlock_other(void)
+{
+ unsigned curr_core = current_cpu_data.core;
+ spin_unlock_irqrestore(&per_cpu(cpc_core_lock, curr_core),
+ per_cpu(cpc_core_lock_flags, curr_core));
+ preempt_enable();
+}
diff --git a/arch/mips/kernel/mips-mt-fpaff.c b/arch/mips/kernel/mips-mt-fpaff.c
index cb098628..362bb37 100644
--- a/arch/mips/kernel/mips-mt-fpaff.c
+++ b/arch/mips/kernel/mips-mt-fpaff.c
@@ -1,5 +1,5 @@
/*
- * General MIPS MT support routines, usable in AP/SP, SMVP, or SMTC kernels
+ * General MIPS MT support routines, usable in AP/SP and SMVP.
* Copyright (C) 2005 Mips Technologies, Inc
*/
#include <linux/cpu.h>
diff --git a/arch/mips/kernel/mips-mt.c b/arch/mips/kernel/mips-mt.c
index 6ded9bd..88b1ef5 100644
--- a/arch/mips/kernel/mips-mt.c
+++ b/arch/mips/kernel/mips-mt.c
@@ -1,5 +1,5 @@
/*
- * General MIPS MT support routines, usable in AP/SP, SMVP, or SMTC kernels
+ * General MIPS MT support routines, usable in AP/SP and SMVP.
* Copyright (C) 2005 Mips Technologies, Inc
*/
@@ -57,9 +57,6 @@
int tc;
unsigned long haltval;
unsigned long tcstatval;
-#ifdef CONFIG_MIPS_MT_SMTC
- void smtc_soft_dump(void);
-#endif /* CONFIG_MIPT_MT_SMTC */
local_irq_save(flags);
vpflags = dvpe();
@@ -116,9 +113,6 @@
if (!haltval)
write_tc_c0_tchalt(0);
}
-#ifdef CONFIG_MIPS_MT_SMTC
- smtc_soft_dump();
-#endif /* CONFIG_MIPT_MT_SMTC */
printk("===========================\n");
evpe(vpflags);
local_irq_restore(flags);
@@ -295,21 +289,11 @@
void mt_cflush_lockdown(void)
{
-#ifdef CONFIG_MIPS_MT_SMTC
- void smtc_cflush_lockdown(void);
-
- smtc_cflush_lockdown();
-#endif /* CONFIG_MIPS_MT_SMTC */
/* FILL IN VSMP and AP/SP VERSIONS HERE */
}
void mt_cflush_release(void)
{
-#ifdef CONFIG_MIPS_MT_SMTC
- void smtc_cflush_release(void);
-
- smtc_cflush_release();
-#endif /* CONFIG_MIPS_MT_SMTC */
/* FILL IN VSMP and AP/SP VERSIONS HERE */
}
diff --git a/arch/mips/kernel/octeon_switch.S b/arch/mips/kernel/octeon_switch.S
index 029e002..f654768 100644
--- a/arch/mips/kernel/octeon_switch.S
+++ b/arch/mips/kernel/octeon_switch.S
@@ -10,24 +10,12 @@
* Copyright (C) 2000 MIPS Technologies, Inc.
* written by Carsten Langgaard, carstenl@mips.com
*/
-#include <asm/asm.h>
-#include <asm/cachectl.h>
-#include <asm/fpregdef.h>
-#include <asm/mipsregs.h>
-#include <asm/asm-offsets.h>
-#include <asm/pgtable-bits.h>
-#include <asm/regdef.h>
-#include <asm/stackframe.h>
-#include <asm/thread_info.h>
-#include <asm/asmmacro.h>
-
-/*
- * Offset to the current process status flags, the first 32 bytes of the
- * stack are not used.
- */
-#define ST_OFF (_THREAD_SIZE - 32 - PT_SIZE + PT_STATUS)
-
+#define USE_ALTERNATE_RESUME_IMPL 1
+ .set push
+ .set arch=mips64r2
+#include "r4k_switch.S"
+ .set pop
/*
* task_struct *resume(task_struct *prev, task_struct *next,
* struct thread_info *next_ti, int usedfpu)
@@ -40,6 +28,61 @@
cpu_save_nonscratch a0
LONG_S ra, THREAD_REG31(a0)
+ /*
+ * check if we need to save FPU registers
+ */
+ PTR_L t3, TASK_THREAD_INFO(a0)
+ LONG_L t0, TI_FLAGS(t3)
+ li t1, _TIF_USEDFPU
+ and t2, t0, t1
+ beqz t2, 1f
+ nor t1, zero, t1
+
+ and t0, t0, t1
+ LONG_S t0, TI_FLAGS(t3)
+
+ /*
+ * clear saved user stack CU1 bit
+ */
+ LONG_L t0, ST_OFF(t3)
+ li t1, ~ST0_CU1
+ and t0, t0, t1
+ LONG_S t0, ST_OFF(t3)
+
+ .set push
+ .set arch=mips64r2
+ fpu_save_double a0 t0 t1 # c0_status passed in t0
+ # clobbers t1
+ .set pop
+1:
+
+ /* check if we need to save COP2 registers */
+ PTR_L t2, TASK_THREAD_INFO(a0)
+ LONG_L t0, ST_OFF(t2)
+ bbit0 t0, 30, 1f
+
+ /* Disable COP2 in the stored process state */
+ li t1, ST0_CU2
+ xor t0, t1
+ LONG_S t0, ST_OFF(t2)
+
+ /* Enable COP2 so we can save it */
+ mfc0 t0, CP0_STATUS
+ or t0, t1
+ mtc0 t0, CP0_STATUS
+
+ /* Save COP2 */
+ daddu a0, THREAD_CP2
+ jal octeon_cop2_save
+ dsubu a0, THREAD_CP2
+
+ /* Disable COP2 now that we are done */
+ mfc0 t0, CP0_STATUS
+ li t1, ST0_CU2
+ xor t0, t1
+ mtc0 t0, CP0_STATUS
+
+1:
#if CONFIG_CAVIUM_OCTEON_CVMSEG_SIZE > 0
/* Check if we need to store CVMSEG state */
mfc0 t0, $11,7 /* CvmMemCtl */
@@ -85,12 +128,7 @@
move $28, a2
cpu_restore_nonscratch a1
-#if (_THREAD_SIZE - 32) < 0x8000
- PTR_ADDIU t0, $28, _THREAD_SIZE - 32
-#else
- PTR_LI t0, _THREAD_SIZE - 32
- PTR_ADDU t0, $28
-#endif
+ PTR_ADDU t0, $28, _THREAD_SIZE - 32
set_saved_sp t0, t1, t2
mfc0 t1, CP0_STATUS /* Do we really need this? */
diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c
new file mode 100644
index 0000000..5aa4c6f
--- /dev/null
+++ b/arch/mips/kernel/pm-cps.c
@@ -0,0 +1,716 @@
+/*
+ * Copyright (C) 2014 Imagination Technologies
+ * Author: Paul Burton <paul.burton@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/init.h>
+#include <linux/percpu.h>
+#include <linux/slab.h>
+
+#include <asm/asm-offsets.h>
+#include <asm/cacheflush.h>
+#include <asm/cacheops.h>
+#include <asm/idle.h>
+#include <asm/mips-cm.h>
+#include <asm/mips-cpc.h>
+#include <asm/mipsmtregs.h>
+#include <asm/pm.h>
+#include <asm/pm-cps.h>
+#include <asm/smp-cps.h>
+#include <asm/uasm.h>
+
+/*
+ * cps_nc_entry_fn - type of a generated non-coherent state entry function
+ * @online: the count of online coupled VPEs
+ * @nc_ready_count: pointer to a non-coherent mapping of the core ready_count
+ *
+ * The code entering & exiting non-coherent states is generated at runtime
+ * using uasm, in order to ensure that the compiler cannot insert a stray
+ * memory access at an unfortunate time and to allow the generation of optimal
+ * core-specific code particularly for cache routines. If coupled_coherence
+ * is non-zero and this is the entry function for the CPS_PM_NC_WAIT state,
+ * returns the number of VPEs that were in the wait state at the point this
+ * VPE left it. Returns garbage if coupled_coherence is zero or this is not
+ * the entry function for CPS_PM_NC_WAIT.
+ */
+typedef unsigned (*cps_nc_entry_fn)(unsigned online, u32 *nc_ready_count);
+
+/*
+ * The entry point of the generated non-coherent idle state entry/exit
+ * functions. Actually per-core rather than per-CPU.
+ */
+static DEFINE_PER_CPU_READ_MOSTLY(cps_nc_entry_fn[CPS_PM_STATE_COUNT],
+ nc_asm_enter);
+
+/* Bitmap indicating which states are supported by the system */
+DECLARE_BITMAP(state_support, CPS_PM_STATE_COUNT);
+
+/*
+ * Indicates the number of coupled VPEs ready to operate in a non-coherent
+ * state. Actually per-core rather than per-CPU.
+ */
+static DEFINE_PER_CPU_ALIGNED(u32*, ready_count);
+static DEFINE_PER_CPU_ALIGNED(void*, ready_count_alloc);
+
+/* Indicates online CPUs coupled with the current CPU */
+static DEFINE_PER_CPU_ALIGNED(cpumask_t, online_coupled);
+
+/*
+ * Used to synchronize entry to deep idle states. Actually per-core rather
+ * than per-CPU.
+ */
+static DEFINE_PER_CPU_ALIGNED(atomic_t, pm_barrier);
+
+/* Saved CPU state across the CPS_PM_POWER_GATED state */
+DEFINE_PER_CPU_ALIGNED(struct mips_static_suspend_state, cps_cpu_state);
+
+/* A somewhat arbitrary number of labels & relocs for uasm */
+static struct uasm_label labels[32] __initdata;
+static struct uasm_reloc relocs[32] __initdata;
+
+/* CPU dependant sync types */
+static unsigned stype_intervention;
+static unsigned stype_memory;
+static unsigned stype_ordering;
+
+enum mips_reg {
+ zero, at, v0, v1, a0, a1, a2, a3,
+ t0, t1, t2, t3, t4, t5, t6, t7,
+ s0, s1, s2, s3, s4, s5, s6, s7,
+ t8, t9, k0, k1, gp, sp, fp, ra,
+};
+
+bool cps_pm_support_state(enum cps_pm_state state)
+{
+ return test_bit(state, state_support);
+}
+
+static void coupled_barrier(atomic_t *a, unsigned online)
+{
+ /*
+ * This function is effectively the same as
+ * cpuidle_coupled_parallel_barrier, which can't be used here since
+ * there's no cpuidle device.
+ */
+
+ if (!coupled_coherence)
+ return;
+
+ smp_mb__before_atomic_inc();
+ atomic_inc(a);
+
+ while (atomic_read(a) < online)
+ cpu_relax();
+
+ if (atomic_inc_return(a) == online * 2) {
+ atomic_set(a, 0);
+ return;
+ }
+
+ while (atomic_read(a) > online)
+ cpu_relax();
+}
+
+int cps_pm_enter_state(enum cps_pm_state state)
+{
+ unsigned cpu = smp_processor_id();
+ unsigned core = current_cpu_data.core;
+ unsigned online, left;
+ cpumask_t *coupled_mask = this_cpu_ptr(&online_coupled);
+ u32 *core_ready_count, *nc_core_ready_count;
+ void *nc_addr;
+ cps_nc_entry_fn entry;
+ struct core_boot_config *core_cfg;
+ struct vpe_boot_config *vpe_cfg;
+
+ /* Check that there is an entry function for this state */
+ entry = per_cpu(nc_asm_enter, core)[state];
+ if (!entry)
+ return -EINVAL;
+
+ /* Calculate which coupled CPUs (VPEs) are online */
+#ifdef CONFIG_MIPS_MT
+ if (cpu_online(cpu)) {
+ cpumask_and(coupled_mask, cpu_online_mask,
+ &cpu_sibling_map[cpu]);
+ online = cpumask_weight(coupled_mask);
+ cpumask_clear_cpu(cpu, coupled_mask);
+ } else
+#endif
+ {
+ cpumask_clear(coupled_mask);
+ online = 1;
+ }
+
+ /* Setup the VPE to run mips_cps_pm_restore when started again */
+ if (config_enabled(CONFIG_CPU_PM) && state == CPS_PM_POWER_GATED) {
+ core_cfg = &mips_cps_core_bootcfg[core];
+ vpe_cfg = &core_cfg->vpe_config[current_cpu_data.vpe_id];
+ vpe_cfg->pc = (unsigned long)mips_cps_pm_restore;
+ vpe_cfg->gp = (unsigned long)current_thread_info();
+ vpe_cfg->sp = 0;
+ }
+
+ /* Indicate that this CPU might not be coherent */
+ cpumask_clear_cpu(cpu, &cpu_coherent_mask);
+ smp_mb__after_clear_bit();
+
+ /* Create a non-coherent mapping of the core ready_count */
+ core_ready_count = per_cpu(ready_count, core);
+ nc_addr = kmap_noncoherent(virt_to_page(core_ready_count),
+ (unsigned long)core_ready_count);
+ nc_addr += ((unsigned long)core_ready_count & ~PAGE_MASK);
+ nc_core_ready_count = nc_addr;
+
+ /* Ensure ready_count is zero-initialised before the assembly runs */
+ ACCESS_ONCE(*nc_core_ready_count) = 0;
+ coupled_barrier(&per_cpu(pm_barrier, core), online);
+
+ /* Run the generated entry code */
+ left = entry(online, nc_core_ready_count);
+
+ /* Remove the non-coherent mapping of ready_count */
+ kunmap_noncoherent();
+
+ /* Indicate that this CPU is definitely coherent */
+ cpumask_set_cpu(cpu, &cpu_coherent_mask);
+
+ /*
+ * If this VPE is the first to leave the non-coherent wait state then
+ * it needs to wake up any coupled VPEs still running their wait
+ * instruction so that they return to cpuidle, which can then complete
+ * coordination between the coupled VPEs & provide the governor with
+ * a chance to reflect on the length of time the VPEs were in the
+ * idle state.
+ */
+ if (coupled_coherence && (state == CPS_PM_NC_WAIT) && (left == online))
+ arch_send_call_function_ipi_mask(coupled_mask);
+
+ return 0;
+}
+
+static void __init cps_gen_cache_routine(u32 **pp, struct uasm_label **pl,
+ struct uasm_reloc **pr,
+ const struct cache_desc *cache,
+ unsigned op, int lbl)
+{
+ unsigned cache_size = cache->ways << cache->waybit;
+ unsigned i;
+ const unsigned unroll_lines = 32;
+
+ /* If the cache isn't present this function has it easy */
+ if (cache->flags & MIPS_CACHE_NOT_PRESENT)
+ return;
+
+ /* Load base address */
+ UASM_i_LA(pp, t0, (long)CKSEG0);
+
+ /* Calculate end address */
+ if (cache_size < 0x8000)
+ uasm_i_addiu(pp, t1, t0, cache_size);
+ else
+ UASM_i_LA(pp, t1, (long)(CKSEG0 + cache_size));
+
+ /* Start of cache op loop */
+ uasm_build_label(pl, *pp, lbl);
+
+ /* Generate the cache ops */
+ for (i = 0; i < unroll_lines; i++)
+ uasm_i_cache(pp, op, i * cache->linesz, t0);
+
+ /* Update the base address */
+ uasm_i_addiu(pp, t0, t0, unroll_lines * cache->linesz);
+
+ /* Loop if we haven't reached the end address yet */
+ uasm_il_bne(pp, pr, t0, t1, lbl);
+ uasm_i_nop(pp);
+}
+
+static int __init cps_gen_flush_fsb(u32 **pp, struct uasm_label **pl,
+ struct uasm_reloc **pr,
+ const struct cpuinfo_mips *cpu_info,
+ int lbl)
+{
+ unsigned i, fsb_size = 8;
+ unsigned num_loads = (fsb_size * 3) / 2;
+ unsigned line_stride = 2;
+ unsigned line_size = cpu_info->dcache.linesz;
+ unsigned perf_counter, perf_event;
+ unsigned revision = cpu_info->processor_id & PRID_REV_MASK;
+
+ /*
+ * Determine whether this CPU requires an FSB flush, and if so which
+ * performance counter/event reflect stalls due to a full FSB.
+ */
+ switch (__get_cpu_type(cpu_info->cputype)) {
+ case CPU_INTERAPTIV:
+ perf_counter = 1;
+ perf_event = 51;
+ break;
+
+ case CPU_PROAPTIV:
+ /* Newer proAptiv cores don't require this workaround */
+ if (revision >= PRID_REV_ENCODE_332(1, 1, 0))
+ return 0;
+
+ /* On older ones it's unavailable */
+ return -1;
+
+ /* CPUs which do not require the workaround */
+ case CPU_P5600:
+ return 0;
+
+ default:
+ WARN_ONCE(1, "pm-cps: FSB flush unsupported for this CPU\n");
+ return -1;
+ }
+
+ /*
+ * Ensure that the fill/store buffer (FSB) is not holding the results
+ * of a prefetch, since if it is then the CPC sequencer may become
+ * stuck in the D3 (ClrBus) state whilst entering a low power state.
+ */
+
+ /* Preserve perf counter setup */
+ uasm_i_mfc0(pp, t2, 25, (perf_counter * 2) + 0); /* PerfCtlN */
+ uasm_i_mfc0(pp, t3, 25, (perf_counter * 2) + 1); /* PerfCntN */
+
+ /* Setup perf counter to count FSB full pipeline stalls */
+ uasm_i_addiu(pp, t0, zero, (perf_event << 5) | 0xf);
+ uasm_i_mtc0(pp, t0, 25, (perf_counter * 2) + 0); /* PerfCtlN */
+ uasm_i_ehb(pp);
+ uasm_i_mtc0(pp, zero, 25, (perf_counter * 2) + 1); /* PerfCntN */
+ uasm_i_ehb(pp);
+
+ /* Base address for loads */
+ UASM_i_LA(pp, t0, (long)CKSEG0);
+
+ /* Start of clear loop */
+ uasm_build_label(pl, *pp, lbl);
+
+ /* Perform some loads to fill the FSB */
+ for (i = 0; i < num_loads; i++)
+ uasm_i_lw(pp, zero, i * line_size * line_stride, t0);
+
+ /*
+ * Invalidate the new D-cache entries so that the cache will need
+ * refilling (via the FSB) if the loop is executed again.
+ */
+ for (i = 0; i < num_loads; i++) {
+ uasm_i_cache(pp, Hit_Invalidate_D,
+ i * line_size * line_stride, t0);
+ uasm_i_cache(pp, Hit_Writeback_Inv_SD,
+ i * line_size * line_stride, t0);
+ }
+
+ /* Completion barrier */
+ uasm_i_sync(pp, stype_memory);
+ uasm_i_ehb(pp);
+
+ /* Check whether the pipeline stalled due to the FSB being full */
+ uasm_i_mfc0(pp, t1, 25, (perf_counter * 2) + 1); /* PerfCntN */
+
+ /* Loop if it didn't */
+ uasm_il_beqz(pp, pr, t1, lbl);
+ uasm_i_nop(pp);
+
+ /* Restore perf counter 1. The count may well now be wrong... */
+ uasm_i_mtc0(pp, t2, 25, (perf_counter * 2) + 0); /* PerfCtlN */
+ uasm_i_ehb(pp);
+ uasm_i_mtc0(pp, t3, 25, (perf_counter * 2) + 1); /* PerfCntN */
+ uasm_i_ehb(pp);
+
+ return 0;
+}
+
+static void __init cps_gen_set_top_bit(u32 **pp, struct uasm_label **pl,
+ struct uasm_reloc **pr,
+ unsigned r_addr, int lbl)
+{
+ uasm_i_lui(pp, t0, uasm_rel_hi(0x80000000));
+ uasm_build_label(pl, *pp, lbl);
+ uasm_i_ll(pp, t1, 0, r_addr);
+ uasm_i_or(pp, t1, t1, t0);
+ uasm_i_sc(pp, t1, 0, r_addr);
+ uasm_il_beqz(pp, pr, t1, lbl);
+ uasm_i_nop(pp);
+}
+
+static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
+{
+ struct uasm_label *l = labels;
+ struct uasm_reloc *r = relocs;
+ u32 *buf, *p;
+ const unsigned r_online = a0;
+ const unsigned r_nc_count = a1;
+ const unsigned r_pcohctl = t7;
+ const unsigned max_instrs = 256;
+ unsigned cpc_cmd;
+ int err;
+ enum {
+ lbl_incready = 1,
+ lbl_poll_cont,
+ lbl_secondary_hang,
+ lbl_disable_coherence,
+ lbl_flush_fsb,
+ lbl_invicache,
+ lbl_flushdcache,
+ lbl_hang,
+ lbl_set_cont,
+ lbl_secondary_cont,
+ lbl_decready,
+ };
+
+ /* Allocate a buffer to hold the generated code */
+ p = buf = kcalloc(max_instrs, sizeof(u32), GFP_KERNEL);
+ if (!buf)
+ return NULL;
+
+ /* Clear labels & relocs ready for (re)use */
+ memset(labels, 0, sizeof(labels));
+ memset(relocs, 0, sizeof(relocs));
+
+ if (config_enabled(CONFIG_CPU_PM) && state == CPS_PM_POWER_GATED) {
+ /*
+ * Save CPU state. Note the non-standard calling convention
+ * with the return address placed in v0 to avoid clobbering
+ * the ra register before it is saved.
+ */
+ UASM_i_LA(&p, t0, (long)mips_cps_pm_save);
+ uasm_i_jalr(&p, v0, t0);
+ uasm_i_nop(&p);
+ }
+
+ /*
+ * Load addresses of required CM & CPC registers. This is done early
+ * because they're needed in both the enable & disable coherence steps
+ * but in the coupled case the enable step will only run on one VPE.
+ */
+ UASM_i_LA(&p, r_pcohctl, (long)addr_gcr_cl_coherence());
+
+ if (coupled_coherence) {
+ /* Increment ready_count */
+ uasm_i_sync(&p, stype_ordering);
+ uasm_build_label(&l, p, lbl_incready);
+ uasm_i_ll(&p, t1, 0, r_nc_count);
+ uasm_i_addiu(&p, t2, t1, 1);
+ uasm_i_sc(&p, t2, 0, r_nc_count);
+ uasm_il_beqz(&p, &r, t2, lbl_incready);
+ uasm_i_addiu(&p, t1, t1, 1);
+
+ /* Ordering barrier */
+ uasm_i_sync(&p, stype_ordering);
+
+ /*
+ * If this is the last VPE to become ready for non-coherence
+ * then it should branch below.
+ */
+ uasm_il_beq(&p, &r, t1, r_online, lbl_disable_coherence);
+ uasm_i_nop(&p);
+
+ if (state < CPS_PM_POWER_GATED) {
+ /*
+ * Otherwise this is not the last VPE to become ready
+ * for non-coherence. It needs to wait until coherence
+ * has been disabled before proceeding, which it will do
+ * by polling for the top bit of ready_count being set.
+ */
+ uasm_i_addiu(&p, t1, zero, -1);
+ uasm_build_label(&l, p, lbl_poll_cont);
+ uasm_i_lw(&p, t0, 0, r_nc_count);
+ uasm_il_bltz(&p, &r, t0, lbl_secondary_cont);
+ uasm_i_ehb(&p);
+ uasm_i_yield(&p, zero, t1);
+ uasm_il_b(&p, &r, lbl_poll_cont);
+ uasm_i_nop(&p);
+ } else {
+ /*
+ * The core will lose power & this VPE will not continue
+ * so it can simply halt here.
+ */
+ uasm_i_addiu(&p, t0, zero, TCHALT_H);
+ uasm_i_mtc0(&p, t0, 2, 4);
+ uasm_build_label(&l, p, lbl_secondary_hang);
+ uasm_il_b(&p, &r, lbl_secondary_hang);
+ uasm_i_nop(&p);
+ }
+ }
+
+ /*
+ * This is the point of no return - this VPE will now proceed to
+ * disable coherence. At this point we *must* be sure that no other
+ * VPE within the core will interfere with the L1 dcache.
+ */
+ uasm_build_label(&l, p, lbl_disable_coherence);
+
+ /* Invalidate the L1 icache */
+ cps_gen_cache_routine(&p, &l, &r, &cpu_data[cpu].icache,
+ Index_Invalidate_I, lbl_invicache);
+
+ /* Writeback & invalidate the L1 dcache */
+ cps_gen_cache_routine(&p, &l, &r, &cpu_data[cpu].dcache,
+ Index_Writeback_Inv_D, lbl_flushdcache);
+
+ /* Completion barrier */
+ uasm_i_sync(&p, stype_memory);
+ uasm_i_ehb(&p);
+
+ /*
+ * Disable all but self interventions. The load from COHCTL is defined
+ * by the interAptiv & proAptiv SUMs as ensuring that the operation
+ * resulting from the preceeding store is complete.
+ */
+ uasm_i_addiu(&p, t0, zero, 1 << cpu_data[cpu].core);
+ uasm_i_sw(&p, t0, 0, r_pcohctl);
+ uasm_i_lw(&p, t0, 0, r_pcohctl);
+
+ /* Sync to ensure previous interventions are complete */
+ uasm_i_sync(&p, stype_intervention);
+ uasm_i_ehb(&p);
+
+ /* Disable coherence */
+ uasm_i_sw(&p, zero, 0, r_pcohctl);
+ uasm_i_lw(&p, t0, 0, r_pcohctl);
+
+ if (state >= CPS_PM_CLOCK_GATED) {
+ err = cps_gen_flush_fsb(&p, &l, &r, &cpu_data[cpu],
+ lbl_flush_fsb);
+ if (err)
+ goto out_err;
+
+ /* Determine the CPC command to issue */
+ switch (state) {
+ case CPS_PM_CLOCK_GATED:
+ cpc_cmd = CPC_Cx_CMD_CLOCKOFF;
+ break;
+ case CPS_PM_POWER_GATED:
+ cpc_cmd = CPC_Cx_CMD_PWRDOWN;
+ break;
+ default:
+ BUG();
+ goto out_err;
+ }
+
+ /* Issue the CPC command */
+ UASM_i_LA(&p, t0, (long)addr_cpc_cl_cmd());
+ uasm_i_addiu(&p, t1, zero, cpc_cmd);
+ uasm_i_sw(&p, t1, 0, t0);
+
+ if (state == CPS_PM_POWER_GATED) {
+ /* If anything goes wrong just hang */
+ uasm_build_label(&l, p, lbl_hang);
+ uasm_il_b(&p, &r, lbl_hang);
+ uasm_i_nop(&p);
+
+ /*
+ * There's no point generating more code, the core is
+ * powered down & if powered back up will run from the
+ * reset vector not from here.
+ */
+ goto gen_done;
+ }
+
+ /* Completion barrier */
+ uasm_i_sync(&p, stype_memory);
+ uasm_i_ehb(&p);
+ }
+
+ if (state == CPS_PM_NC_WAIT) {
+ /*
+ * At this point it is safe for all VPEs to proceed with
+ * execution. This VPE will set the top bit of ready_count
+ * to indicate to the other VPEs that they may continue.
+ */
+ if (coupled_coherence)
+ cps_gen_set_top_bit(&p, &l, &r, r_nc_count,
+ lbl_set_cont);
+
+ /*
+ * VPEs which did not disable coherence will continue
+ * executing, after coherence has been disabled, from this
+ * point.
+ */
+ uasm_build_label(&l, p, lbl_secondary_cont);
+
+ /* Now perform our wait */
+ uasm_i_wait(&p, 0);
+ }
+
+ /*
+ * Re-enable coherence. Note that for CPS_PM_NC_WAIT all coupled VPEs
+ * will run this. The first will actually re-enable coherence & the
+ * rest will just be performing a rather unusual nop.
+ */
+ uasm_i_addiu(&p, t0, zero, CM_GCR_Cx_COHERENCE_COHDOMAINEN_MSK);
+ uasm_i_sw(&p, t0, 0, r_pcohctl);
+ uasm_i_lw(&p, t0, 0, r_pcohctl);
+
+ /* Completion barrier */
+ uasm_i_sync(&p, stype_memory);
+ uasm_i_ehb(&p);
+
+ if (coupled_coherence && (state == CPS_PM_NC_WAIT)) {
+ /* Decrement ready_count */
+ uasm_build_label(&l, p, lbl_decready);
+ uasm_i_sync(&p, stype_ordering);
+ uasm_i_ll(&p, t1, 0, r_nc_count);
+ uasm_i_addiu(&p, t2, t1, -1);
+ uasm_i_sc(&p, t2, 0, r_nc_count);
+ uasm_il_beqz(&p, &r, t2, lbl_decready);
+ uasm_i_andi(&p, v0, t1, (1 << fls(smp_num_siblings)) - 1);
+
+ /* Ordering barrier */
+ uasm_i_sync(&p, stype_ordering);
+ }
+
+ if (coupled_coherence && (state == CPS_PM_CLOCK_GATED)) {
+ /*
+ * At this point it is safe for all VPEs to proceed with
+ * execution. This VPE will set the top bit of ready_count
+ * to indicate to the other VPEs that they may continue.
+ */
+ cps_gen_set_top_bit(&p, &l, &r, r_nc_count, lbl_set_cont);
+
+ /*
+ * This core will be reliant upon another core sending a
+ * power-up command to the CPC in order to resume operation.
+ * Thus an arbitrary VPE can't trigger the core leaving the
+ * idle state and the one that disables coherence might as well
+ * be the one to re-enable it. The rest will continue from here
+ * after that has been done.
+ */
+ uasm_build_label(&l, p, lbl_secondary_cont);
+
+ /* Ordering barrier */
+ uasm_i_sync(&p, stype_ordering);
+ }
+
+ /* The core is coherent, time to return to C code */
+ uasm_i_jr(&p, ra);
+ uasm_i_nop(&p);
+
+gen_done:
+ /* Ensure the code didn't exceed the resources allocated for it */
+ BUG_ON((p - buf) > max_instrs);
+ BUG_ON((l - labels) > ARRAY_SIZE(labels));
+ BUG_ON((r - relocs) > ARRAY_SIZE(relocs));
+
+ /* Patch branch offsets */
+ uasm_resolve_relocs(relocs, labels);
+
+ /* Flush the icache */
+ local_flush_icache_range((unsigned long)buf, (unsigned long)p);
+
+ return buf;
+out_err:
+ kfree(buf);
+ return NULL;
+}
+
+static int __init cps_gen_core_entries(unsigned cpu)
+{
+ enum cps_pm_state state;
+ unsigned core = cpu_data[cpu].core;
+ unsigned dlinesz = cpu_data[cpu].dcache.linesz;
+ void *entry_fn, *core_rc;
+
+ for (state = CPS_PM_NC_WAIT; state < CPS_PM_STATE_COUNT; state++) {
+ if (per_cpu(nc_asm_enter, core)[state])
+ continue;
+ if (!test_bit(state, state_support))
+ continue;
+
+ entry_fn = cps_gen_entry_code(cpu, state);
+ if (!entry_fn) {
+ pr_err("Failed to generate core %u state %u entry\n",
+ core, state);
+ clear_bit(state, state_support);
+ }
+
+ per_cpu(nc_asm_enter, core)[state] = entry_fn;
+ }
+
+ if (!per_cpu(ready_count, core)) {
+ core_rc = kmalloc(dlinesz * 2, GFP_KERNEL);
+ if (!core_rc) {
+ pr_err("Failed allocate core %u ready_count\n", core);
+ return -ENOMEM;
+ }
+ per_cpu(ready_count_alloc, core) = core_rc;
+
+ /* Ensure ready_count is aligned to a cacheline boundary */
+ core_rc += dlinesz - 1;
+ core_rc = (void *)((unsigned long)core_rc & ~(dlinesz - 1));
+ per_cpu(ready_count, core) = core_rc;
+ }
+
+ return 0;
+}
+
+static int __init cps_pm_init(void)
+{
+ unsigned cpu;
+ int err;
+
+ /* Detect appropriate sync types for the system */
+ switch (current_cpu_data.cputype) {
+ case CPU_INTERAPTIV:
+ case CPU_PROAPTIV:
+ case CPU_M5150:
+ case CPU_P5600:
+ stype_intervention = 0x2;
+ stype_memory = 0x3;
+ stype_ordering = 0x10;
+ break;
+
+ default:
+ pr_warn("Power management is using heavyweight sync 0\n");
+ }
+
+ /* A CM is required for all non-coherent states */
+ if (!mips_cm_present()) {
+ pr_warn("pm-cps: no CM, non-coherent states unavailable\n");
+ goto out;
+ }
+
+ /*
+ * If interrupts were enabled whilst running a wait instruction on a
+ * non-coherent core then the VPE may end up processing interrupts
+ * whilst non-coherent. That would be bad.
+ */
+ if (cpu_wait == r4k_wait_irqoff)
+ set_bit(CPS_PM_NC_WAIT, state_support);
+ else
+ pr_warn("pm-cps: non-coherent wait unavailable\n");
+
+ /* Detect whether a CPC is present */
+ if (mips_cpc_present()) {
+ /* Detect whether clock gating is implemented */
+ if (read_cpc_cl_stat_conf() & CPC_Cx_STAT_CONF_CLKGAT_IMPL_MSK)
+ set_bit(CPS_PM_CLOCK_GATED, state_support);
+ else
+ pr_warn("pm-cps: CPC does not support clock gating\n");
+
+ /* Power gating is available with CPS SMP & any CPC */
+ if (mips_cps_smp_in_use())
+ set_bit(CPS_PM_POWER_GATED, state_support);
+ else
+ pr_warn("pm-cps: CPS SMP not in use, power gating unavailable\n");
+ } else {
+ pr_warn("pm-cps: no CPC, clock & power gating unavailable\n");
+ }
+
+ for_each_present_cpu(cpu) {
+ err = cps_gen_core_entries(cpu);
+ if (err)
+ return err;
+ }
+out:
+ return 0;
+}
+arch_initcall(cps_pm_init);
diff --git a/arch/mips/kernel/pm.c b/arch/mips/kernel/pm.c
new file mode 100644
index 0000000..fefdf39
--- /dev/null
+++ b/arch/mips/kernel/pm.c
@@ -0,0 +1,99 @@
+/*
+ * Copyright (C) 2014 Imagination Technologies Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ * CPU PM notifiers for saving/restoring general CPU state.
+ */
+
+#include <linux/cpu_pm.h>
+#include <linux/init.h>
+
+#include <asm/dsp.h>
+#include <asm/fpu.h>
+#include <asm/mmu_context.h>
+#include <asm/pm.h>
+#include <asm/watch.h>
+
+/* Used by PM helper macros in asm/pm.h */
+struct mips_static_suspend_state mips_static_suspend_state;
+
+/**
+ * mips_cpu_save() - Save general CPU state.
+ * Ensures that general CPU context is saved, notably FPU and DSP.
+ */
+static int mips_cpu_save(void)
+{
+ /* Save FPU state */
+ lose_fpu(1);
+
+ /* Save DSP state */
+ save_dsp(current);
+
+ return 0;
+}
+
+/**
+ * mips_cpu_restore() - Restore general CPU state.
+ * Restores important CPU context.
+ */
+static void mips_cpu_restore(void)
+{
+ unsigned int cpu = smp_processor_id();
+
+ /* Restore ASID */
+ if (current->mm)
+ write_c0_entryhi(cpu_asid(cpu, current->mm));
+
+ /* Restore DSP state */
+ restore_dsp(current);
+
+ /* Restore UserLocal */
+ if (cpu_has_userlocal)
+ write_c0_userlocal(current_thread_info()->tp_value);
+
+ /* Restore watch registers */
+ __restore_watch();
+}
+
+/**
+ * mips_pm_notifier() - Notifier for preserving general CPU context.
+ * @self: Notifier block.
+ * @cmd: CPU PM event.
+ * @v: Private data (unused).
+ *
+ * This is called when a CPU power management event occurs, and is used to
+ * ensure that important CPU context is preserved across a CPU power down.
+ */
+static int mips_pm_notifier(struct notifier_block *self, unsigned long cmd,
+ void *v)
+{
+ int ret;
+
+ switch (cmd) {
+ case CPU_PM_ENTER:
+ ret = mips_cpu_save();
+ if (ret)
+ return NOTIFY_STOP;
+ break;
+ case CPU_PM_ENTER_FAILED:
+ case CPU_PM_EXIT:
+ mips_cpu_restore();
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block mips_pm_notifier_block = {
+ .notifier_call = mips_pm_notifier,
+};
+
+static int __init mips_pm_init(void)
+{
+ return cpu_pm_register_notifier(&mips_pm_notifier_block);
+}
+arch_initcall(mips_pm_init);
diff --git a/arch/mips/kernel/process.c b/arch/mips/kernel/process.c
index 60e39dc..0a1ec0f 100644
--- a/arch/mips/kernel/process.c
+++ b/arch/mips/kernel/process.c
@@ -140,13 +140,6 @@
*/
childregs->cp0_status &= ~(ST0_CU2|ST0_CU1);
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * SMTC restores TCStatus after Status, and the CU bits
- * are aliased there.
- */
- childregs->cp0_tcstatus &= ~(ST0_CU2|ST0_CU1);
-#endif
clear_tsk_thread_flag(p, TIF_USEDFPU);
#ifdef CONFIG_MIPS_MT_FPAFF
diff --git a/arch/mips/kernel/r4k_switch.S b/arch/mips/kernel/r4k_switch.S
index abacac7..81ca3f7 100644
--- a/arch/mips/kernel/r4k_switch.S
+++ b/arch/mips/kernel/r4k_switch.S
@@ -28,6 +28,7 @@
*/
#define ST_OFF (_THREAD_SIZE - 32 - PT_SIZE + PT_STATUS)
+#ifndef USE_ALTERNATE_RESUME_IMPL
/*
* task_struct *resume(task_struct *prev, task_struct *next,
* struct thread_info *next_ti, s32 fp_save)
@@ -87,18 +88,6 @@
PTR_ADDU t0, $28, _THREAD_SIZE - 32
set_saved_sp t0, t1, t2
-#ifdef CONFIG_MIPS_MT_SMTC
- /* Read-modify-writes of Status must be atomic on a VPE */
- mfc0 t2, CP0_TCSTATUS
- ori t1, t2, TCSTATUS_IXMT
- mtc0 t1, CP0_TCSTATUS
- andi t2, t2, TCSTATUS_IXMT
- _ehb
- DMT 8 # dmt t0
- move t1,ra
- jal mips_ihb
- move ra,t1
-#endif /* CONFIG_MIPS_MT_SMTC */
mfc0 t1, CP0_STATUS /* Do we really need this? */
li a3, 0xff01
and t1, a3
@@ -107,22 +96,12 @@
and a2, a3
or a2, t1
mtc0 a2, CP0_STATUS
-#ifdef CONFIG_MIPS_MT_SMTC
- _ehb
- andi t0, t0, VPECONTROL_TE
- beqz t0, 1f
- emt
-1:
- mfc0 t1, CP0_TCSTATUS
- xori t1, t1, TCSTATUS_IXMT
- or t1, t1, t2
- mtc0 t1, CP0_TCSTATUS
- _ehb
-#endif /* CONFIG_MIPS_MT_SMTC */
move v0, a0
jr ra
END(resume)
+#endif /* USE_ALTERNATE_RESUME_IMPL */
+
/*
* Save a thread's fp context.
*/
@@ -176,19 +155,10 @@
#define FPU_DEFAULT 0x00000000
LEAF(_init_fpu)
-#ifdef CONFIG_MIPS_MT_SMTC
- /* Rather than manipulate per-VPE Status, set per-TC bit in TCStatus */
- mfc0 t0, CP0_TCSTATUS
- /* Bit position is the same for Status, TCStatus */
- li t1, ST0_CU1
- or t0, t1
- mtc0 t0, CP0_TCSTATUS
-#else /* Normal MIPS CU1 enable */
mfc0 t0, CP0_STATUS
li t1, ST0_CU1
or t0, t1
mtc0 t0, CP0_STATUS
-#endif /* CONFIG_MIPS_MT_SMTC */
enable_fpu_hazard
li t1, FPU_DEFAULT
diff --git a/arch/mips/kernel/rtlx-mt.c b/arch/mips/kernel/rtlx-mt.c
index 9c1aca0..5a66b97 100644
--- a/arch/mips/kernel/rtlx-mt.c
+++ b/arch/mips/kernel/rtlx-mt.c
@@ -36,7 +36,6 @@
unsigned long flags;
int i;
- /* Ought not to be strictly necessary for SMTC builds */
local_irq_save(flags);
vpeflags = dvpe();
set_c0_status(0x100 << MIPS_CPU_RTLX_IRQ);
diff --git a/arch/mips/kernel/smp-bmips.c b/arch/mips/kernel/smp-bmips.c
index ea4c2dc..df9e2bd9b 100644
--- a/arch/mips/kernel/smp-bmips.c
+++ b/arch/mips/kernel/smp-bmips.c
@@ -281,13 +281,6 @@
}
/*
- * Runs on CPU0 after all CPUs have been booted
- */
-static void bmips_cpus_done(void)
-{
-}
-
-/*
* BMIPS5000 raceless IPIs
*
* Each CPU has two inbound SW IRQs which are independent of all other CPUs.
@@ -434,7 +427,6 @@
.boot_secondary = bmips_boot_secondary,
.smp_finish = bmips_smp_finish,
.init_secondary = bmips_init_secondary,
- .cpus_done = bmips_cpus_done,
.send_ipi_single = bmips43xx_send_ipi_single,
.send_ipi_mask = bmips43xx_send_ipi_mask,
#ifdef CONFIG_HOTPLUG_CPU
@@ -449,7 +441,6 @@
.boot_secondary = bmips_boot_secondary,
.smp_finish = bmips_smp_finish,
.init_secondary = bmips_init_secondary,
- .cpus_done = bmips_cpus_done,
.send_ipi_single = bmips5000_send_ipi_single,
.send_ipi_mask = bmips5000_send_ipi_mask,
#ifdef CONFIG_HOTPLUG_CPU
diff --git a/arch/mips/kernel/smp-cmp.c b/arch/mips/kernel/smp-cmp.c
index 3ef55fb7..fc8a515 100644
--- a/arch/mips/kernel/smp-cmp.c
+++ b/arch/mips/kernel/smp-cmp.c
@@ -49,14 +49,11 @@
/* Enable per-cpu interrupts: platform specific */
-#if defined(CONFIG_MIPS_MT_SMP) || defined(CONFIG_MIPS_MT_SMTC)
+#ifdef CONFIG_MIPS_MT_SMP
if (cpu_has_mipsmt)
c->vpe_id = (read_c0_tcbind() >> TCBIND_CURVPE_SHIFT) &
TCBIND_CURVPE;
#endif
-#ifdef CONFIG_MIPS_MT_SMTC
- c->tc_id = (read_c0_tcbind() & TCBIND_CURTC) >> TCBIND_CURTC_SHIFT;
-#endif
}
static void cmp_smp_finish(void)
@@ -75,11 +72,6 @@
local_irq_enable();
}
-static void cmp_cpus_done(void)
-{
- pr_debug("SMPCMP: CPU%d: %s\n", smp_processor_id(), __func__);
-}
-
/*
* Setup the PC, SP, and GP of a secondary processor and start it running
* smp_bootstrap is the place to resume from
@@ -135,10 +127,6 @@
unsigned int mvpconf0 = read_c0_mvpconf0();
nvpe = ((mvpconf0 & MVPCONF0_PVPE) >> MVPCONF0_PVPE_SHIFT) + 1;
-#elif defined(CONFIG_MIPS_MT_SMTC)
- unsigned int mvpconf0 = read_c0_mvpconf0();
-
- nvpe = ((mvpconf0 & MVPCONF0_PTC) >> MVPCONF0_PTC_SHIFT) + 1;
#endif
smp_num_siblings = nvpe;
}
@@ -165,7 +153,6 @@
.send_ipi_mask = gic_send_ipi_mask,
.init_secondary = cmp_init_secondary,
.smp_finish = cmp_smp_finish,
- .cpus_done = cmp_cpus_done,
.boot_secondary = cmp_boot_secondary,
.smp_setup = cmp_smp_setup,
.prepare_cpus = cmp_prepare_cpus,
diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c
index 536eec0..df0598d 100644
--- a/arch/mips/kernel/smp-cps.c
+++ b/arch/mips/kernel/smp-cps.c
@@ -20,104 +20,43 @@
#include <asm/mips-cpc.h>
#include <asm/mips_mt.h>
#include <asm/mipsregs.h>
+#include <asm/pm-cps.h>
#include <asm/smp-cps.h>
#include <asm/time.h>
#include <asm/uasm.h>
static DECLARE_BITMAP(core_power, NR_CPUS);
-struct boot_config mips_cps_bootcfg;
+struct core_boot_config *mips_cps_core_bootcfg;
-static void init_core(void)
+static unsigned core_vpe_count(unsigned core)
{
- unsigned int nvpes, t;
- u32 mvpconf0, vpeconf0, vpecontrol, tcstatus, tcbind, status;
+ unsigned cfg;
- if (!cpu_has_mipsmt)
- return;
+ if (!config_enabled(CONFIG_MIPS_MT_SMP) || !cpu_has_mipsmt)
+ return 1;
- /* Enter VPE configuration state */
- dvpe();
- set_c0_mvpcontrol(MVPCONTROL_VPC);
-
- /* Retrieve the count of VPEs in this core */
- mvpconf0 = read_c0_mvpconf0();
- nvpes = ((mvpconf0 & MVPCONF0_PVPE) >> MVPCONF0_PVPE_SHIFT) + 1;
- smp_num_siblings = nvpes;
-
- for (t = 1; t < nvpes; t++) {
- /* Use a 1:1 mapping of TC index to VPE index */
- settc(t);
-
- /* Bind 1 TC to this VPE */
- tcbind = read_tc_c0_tcbind();
- tcbind &= ~TCBIND_CURVPE;
- tcbind |= t << TCBIND_CURVPE_SHIFT;
- write_tc_c0_tcbind(tcbind);
-
- /* Set exclusive TC, non-active, master */
- vpeconf0 = read_vpe_c0_vpeconf0();
- vpeconf0 &= ~(VPECONF0_XTC | VPECONF0_VPA);
- vpeconf0 |= t << VPECONF0_XTC_SHIFT;
- vpeconf0 |= VPECONF0_MVP;
- write_vpe_c0_vpeconf0(vpeconf0);
-
- /* Declare TC non-active, non-allocatable & interrupt exempt */
- tcstatus = read_tc_c0_tcstatus();
- tcstatus &= ~(TCSTATUS_A | TCSTATUS_DA);
- tcstatus |= TCSTATUS_IXMT;
- write_tc_c0_tcstatus(tcstatus);
-
- /* Halt the TC */
- write_tc_c0_tchalt(TCHALT_H);
-
- /* Allow only 1 TC to execute */
- vpecontrol = read_vpe_c0_vpecontrol();
- vpecontrol &= ~VPECONTROL_TE;
- write_vpe_c0_vpecontrol(vpecontrol);
-
- /* Copy (most of) Status from VPE 0 */
- status = read_c0_status();
- status &= ~(ST0_IM | ST0_IE | ST0_KSU);
- status |= ST0_CU0;
- write_vpe_c0_status(status);
-
- /* Copy Config from VPE 0 */
- write_vpe_c0_config(read_c0_config());
- write_vpe_c0_config7(read_c0_config7());
-
- /* Ensure no software interrupts are pending */
- write_vpe_c0_cause(0);
-
- /* Sync Count */
- write_vpe_c0_count(read_c0_count());
- }
-
- /* Leave VPE configuration state */
- clear_c0_mvpcontrol(MVPCONTROL_VPC);
+ write_gcr_cl_other(core << CM_GCR_Cx_OTHER_CORENUM_SHF);
+ cfg = read_gcr_co_config() & CM_GCR_Cx_CONFIG_PVPE_MSK;
+ return (cfg >> CM_GCR_Cx_CONFIG_PVPE_SHF) + 1;
}
static void __init cps_smp_setup(void)
{
unsigned int ncores, nvpes, core_vpes;
int c, v;
- u32 core_cfg, *entry_code;
/* Detect & record VPE topology */
ncores = mips_cm_numcores();
pr_info("VPE topology ");
for (c = nvpes = 0; c < ncores; c++) {
- if (cpu_has_mipsmt && config_enabled(CONFIG_MIPS_MT_SMP)) {
- write_gcr_cl_other(c << CM_GCR_Cx_OTHER_CORENUM_SHF);
- core_cfg = read_gcr_co_config();
- core_vpes = ((core_cfg & CM_GCR_Cx_CONFIG_PVPE_MSK) >>
- CM_GCR_Cx_CONFIG_PVPE_SHF) + 1;
- } else {
- core_vpes = 1;
- }
-
+ core_vpes = core_vpe_count(c);
pr_cont("%c%u", c ? ',' : '{', core_vpes);
+ /* Use the number of VPEs in core 0 for smp_num_siblings */
+ if (!c)
+ smp_num_siblings = core_vpes;
+
for (v = 0; v < min_t(int, core_vpes, NR_CPUS - nvpes); v++) {
cpu_data[nvpes + v].core = c;
#ifdef CONFIG_MIPS_MT_SMP
@@ -137,19 +76,14 @@
__cpu_logical_map[v] = v;
}
+ /* Set a coherent default CCA (CWB) */
+ change_c0_config(CONF_CM_CMASK, 0x5);
+
/* Core 0 is powered up (we're running on it) */
bitmap_set(core_power, 0, 1);
- /* Disable MT - we only want to run 1 TC per VPE */
- if (cpu_has_mipsmt)
- dmt();
-
/* Initialise core 0 */
- init_core();
-
- /* Patch the start of mips_cps_core_entry to provide the CM base */
- entry_code = (u32 *)&mips_cps_core_entry;
- UASM_i_LA(&entry_code, 3, (long)mips_cm_base);
+ mips_cps_core_init();
/* Make core 0 coherent with everything */
write_gcr_cl_coherence(0xff);
@@ -157,15 +91,99 @@
static void __init cps_prepare_cpus(unsigned int max_cpus)
{
+ unsigned ncores, core_vpes, c, cca;
+ bool cca_unsuitable;
+ u32 *entry_code;
+
mips_mt_set_cpuoptions();
+
+ /* Detect whether the CCA is unsuited to multi-core SMP */
+ cca = read_c0_config() & CONF_CM_CMASK;
+ switch (cca) {
+ case 0x4: /* CWBE */
+ case 0x5: /* CWB */
+ /* The CCA is coherent, multi-core is fine */
+ cca_unsuitable = false;
+ break;
+
+ default:
+ /* CCA is not coherent, multi-core is not usable */
+ cca_unsuitable = true;
+ }
+
+ /* Warn the user if the CCA prevents multi-core */
+ ncores = mips_cm_numcores();
+ if (cca_unsuitable && ncores > 1) {
+ pr_warn("Using only one core due to unsuitable CCA 0x%x\n",
+ cca);
+
+ for_each_present_cpu(c) {
+ if (cpu_data[c].core)
+ set_cpu_present(c, false);
+ }
+ }
+
+ /*
+ * Patch the start of mips_cps_core_entry to provide:
+ *
+ * v0 = CM base address
+ * s0 = kseg0 CCA
+ */
+ entry_code = (u32 *)&mips_cps_core_entry;
+ UASM_i_LA(&entry_code, 3, (long)mips_cm_base);
+ uasm_i_addiu(&entry_code, 16, 0, cca);
+ dma_cache_wback_inv((unsigned long)&mips_cps_core_entry,
+ (void *)entry_code - (void *)&mips_cps_core_entry);
+
+ /* Allocate core boot configuration structs */
+ mips_cps_core_bootcfg = kcalloc(ncores, sizeof(*mips_cps_core_bootcfg),
+ GFP_KERNEL);
+ if (!mips_cps_core_bootcfg) {
+ pr_err("Failed to allocate boot config for %u cores\n", ncores);
+ goto err_out;
+ }
+
+ /* Allocate VPE boot configuration structs */
+ for (c = 0; c < ncores; c++) {
+ core_vpes = core_vpe_count(c);
+ mips_cps_core_bootcfg[c].vpe_config = kcalloc(core_vpes,
+ sizeof(*mips_cps_core_bootcfg[c].vpe_config),
+ GFP_KERNEL);
+ if (!mips_cps_core_bootcfg[c].vpe_config) {
+ pr_err("Failed to allocate %u VPE boot configs\n",
+ core_vpes);
+ goto err_out;
+ }
+ }
+
+ /* Mark this CPU as booted */
+ atomic_set(&mips_cps_core_bootcfg[current_cpu_data.core].vpe_mask,
+ 1 << cpu_vpe_id(¤t_cpu_data));
+
+ return;
+err_out:
+ /* Clean up allocations */
+ if (mips_cps_core_bootcfg) {
+ for (c = 0; c < ncores; c++)
+ kfree(mips_cps_core_bootcfg[c].vpe_config);
+ kfree(mips_cps_core_bootcfg);
+ mips_cps_core_bootcfg = NULL;
+ }
+
+ /* Effectively disable SMP by declaring CPUs not present */
+ for_each_possible_cpu(c) {
+ if (c == 0)
+ continue;
+ set_cpu_present(c, false);
+ }
}
-static void boot_core(struct boot_config *cfg)
+static void boot_core(unsigned core)
{
u32 access;
/* Select the appropriate core */
- write_gcr_cl_other(cfg->core << CM_GCR_Cx_OTHER_CORENUM_SHF);
+ write_gcr_cl_other(core << CM_GCR_Cx_OTHER_CORENUM_SHF);
/* Set its reset vector */
write_gcr_co_reset_base(CKSEG1ADDR((unsigned long)mips_cps_core_entry));
@@ -175,104 +193,74 @@
/* Ensure the core can access the GCRs */
access = read_gcr_access();
- access |= 1 << (CM_GCR_ACCESS_ACCESSEN_SHF + cfg->core);
+ access |= 1 << (CM_GCR_ACCESS_ACCESSEN_SHF + core);
write_gcr_access(access);
- /* Copy cfg */
- mips_cps_bootcfg = *cfg;
-
if (mips_cpc_present()) {
- /* Select the appropriate core */
- write_cpc_cl_other(cfg->core << CPC_Cx_OTHER_CORENUM_SHF);
-
/* Reset the core */
+ mips_cpc_lock_other(core);
write_cpc_co_cmd(CPC_Cx_CMD_RESET);
+ mips_cpc_unlock_other();
} else {
/* Take the core out of reset */
write_gcr_co_reset_release(0);
}
/* The core is now powered up */
- bitmap_set(core_power, cfg->core, 1);
+ bitmap_set(core_power, core, 1);
}
-static void boot_vpe(void *info)
+static void remote_vpe_boot(void *dummy)
{
- struct boot_config *cfg = info;
- u32 tcstatus, vpeconf0;
-
- /* Enter VPE configuration state */
- dvpe();
- set_c0_mvpcontrol(MVPCONTROL_VPC);
-
- settc(cfg->vpe);
-
- /* Set the TC restart PC */
- write_tc_c0_tcrestart((unsigned long)&smp_bootstrap);
-
- /* Activate the TC, allow interrupts */
- tcstatus = read_tc_c0_tcstatus();
- tcstatus &= ~TCSTATUS_IXMT;
- tcstatus |= TCSTATUS_A;
- write_tc_c0_tcstatus(tcstatus);
-
- /* Clear the TC halt bit */
- write_tc_c0_tchalt(0);
-
- /* Activate the VPE */
- vpeconf0 = read_vpe_c0_vpeconf0();
- vpeconf0 |= VPECONF0_VPA;
- write_vpe_c0_vpeconf0(vpeconf0);
-
- /* Set the stack & global pointer registers */
- write_tc_gpr_sp(cfg->sp);
- write_tc_gpr_gp(cfg->gp);
-
- /* Leave VPE configuration state */
- clear_c0_mvpcontrol(MVPCONTROL_VPC);
-
- /* Enable other VPEs to execute */
- evpe(EVPE_ENABLE);
+ mips_cps_boot_vpes();
}
static void cps_boot_secondary(int cpu, struct task_struct *idle)
{
- struct boot_config cfg;
+ unsigned core = cpu_data[cpu].core;
+ unsigned vpe_id = cpu_vpe_id(&cpu_data[cpu]);
+ struct core_boot_config *core_cfg = &mips_cps_core_bootcfg[core];
+ struct vpe_boot_config *vpe_cfg = &core_cfg->vpe_config[vpe_id];
unsigned int remote;
int err;
- cfg.core = cpu_data[cpu].core;
- cfg.vpe = cpu_vpe_id(&cpu_data[cpu]);
- cfg.pc = (unsigned long)&smp_bootstrap;
- cfg.sp = __KSTK_TOS(idle);
- cfg.gp = (unsigned long)task_thread_info(idle);
+ vpe_cfg->pc = (unsigned long)&smp_bootstrap;
+ vpe_cfg->sp = __KSTK_TOS(idle);
+ vpe_cfg->gp = (unsigned long)task_thread_info(idle);
- if (!test_bit(cfg.core, core_power)) {
+ atomic_or(1 << cpu_vpe_id(&cpu_data[cpu]), &core_cfg->vpe_mask);
+
+ preempt_disable();
+
+ if (!test_bit(core, core_power)) {
/* Boot a VPE on a powered down core */
- boot_core(&cfg);
- return;
+ boot_core(core);
+ goto out;
}
- if (cfg.core != current_cpu_data.core) {
+ if (core != current_cpu_data.core) {
/* Boot a VPE on another powered up core */
for (remote = 0; remote < NR_CPUS; remote++) {
- if (cpu_data[remote].core != cfg.core)
+ if (cpu_data[remote].core != core)
continue;
if (cpu_online(remote))
break;
}
BUG_ON(remote >= NR_CPUS);
- err = smp_call_function_single(remote, boot_vpe, &cfg, 1);
+ err = smp_call_function_single(remote, remote_vpe_boot,
+ NULL, 1);
if (err)
panic("Failed to call remote CPU\n");
- return;
+ goto out;
}
BUG_ON(!cpu_has_mipsmt);
/* Boot a VPE on this core */
- boot_vpe(&cfg);
+ mips_cps_boot_vpes();
+out:
+ preempt_enable();
}
static void cps_init_secondary(void)
@@ -281,10 +269,6 @@
if (cpu_has_mipsmt)
dmt();
- /* TODO: revisit this assumption once hotplug is implemented */
- if (cpu_vpe_id(¤t_cpu_data) == 0)
- init_core();
-
change_c0_status(ST0_IM, STATUSF_IP3 | STATUSF_IP4 |
STATUSF_IP6 | STATUSF_IP7);
}
@@ -302,10 +286,148 @@
local_irq_enable();
}
-static void cps_cpus_done(void)
+#ifdef CONFIG_HOTPLUG_CPU
+
+static int cps_cpu_disable(void)
{
+ unsigned cpu = smp_processor_id();
+ struct core_boot_config *core_cfg;
+
+ if (!cpu)
+ return -EBUSY;
+
+ if (!cps_pm_support_state(CPS_PM_POWER_GATED))
+ return -EINVAL;
+
+ core_cfg = &mips_cps_core_bootcfg[current_cpu_data.core];
+ atomic_sub(1 << cpu_vpe_id(¤t_cpu_data), &core_cfg->vpe_mask);
+ smp_mb__after_atomic_dec();
+ set_cpu_online(cpu, false);
+ cpu_clear(cpu, cpu_callin_map);
+
+ return 0;
}
+static DECLARE_COMPLETION(cpu_death_chosen);
+static unsigned cpu_death_sibling;
+static enum {
+ CPU_DEATH_HALT,
+ CPU_DEATH_POWER,
+} cpu_death;
+
+void play_dead(void)
+{
+ unsigned cpu, core;
+
+ local_irq_disable();
+ idle_task_exit();
+ cpu = smp_processor_id();
+ cpu_death = CPU_DEATH_POWER;
+
+ if (cpu_has_mipsmt) {
+ core = cpu_data[cpu].core;
+
+ /* Look for another online VPE within the core */
+ for_each_online_cpu(cpu_death_sibling) {
+ if (cpu_data[cpu_death_sibling].core != core)
+ continue;
+
+ /*
+ * There is an online VPE within the core. Just halt
+ * this TC and leave the core alone.
+ */
+ cpu_death = CPU_DEATH_HALT;
+ break;
+ }
+ }
+
+ /* This CPU has chosen its way out */
+ complete(&cpu_death_chosen);
+
+ if (cpu_death == CPU_DEATH_HALT) {
+ /* Halt this TC */
+ write_c0_tchalt(TCHALT_H);
+ instruction_hazard();
+ } else {
+ /* Power down the core */
+ cps_pm_enter_state(CPS_PM_POWER_GATED);
+ }
+
+ /* This should never be reached */
+ panic("Failed to offline CPU %u", cpu);
+}
+
+static void wait_for_sibling_halt(void *ptr_cpu)
+{
+ unsigned cpu = (unsigned)ptr_cpu;
+ unsigned vpe_id = cpu_data[cpu].vpe_id;
+ unsigned halted;
+ unsigned long flags;
+
+ do {
+ local_irq_save(flags);
+ settc(vpe_id);
+ halted = read_tc_c0_tchalt();
+ local_irq_restore(flags);
+ } while (!(halted & TCHALT_H));
+}
+
+static void cps_cpu_die(unsigned int cpu)
+{
+ unsigned core = cpu_data[cpu].core;
+ unsigned stat;
+ int err;
+
+ /* Wait for the cpu to choose its way out */
+ if (!wait_for_completion_timeout(&cpu_death_chosen,
+ msecs_to_jiffies(5000))) {
+ pr_err("CPU%u: didn't offline\n", cpu);
+ return;
+ }
+
+ /*
+ * Now wait for the CPU to actually offline. Without doing this that
+ * offlining may race with one or more of:
+ *
+ * - Onlining the CPU again.
+ * - Powering down the core if another VPE within it is offlined.
+ * - A sibling VPE entering a non-coherent state.
+ *
+ * In the non-MT halt case (ie. infinite loop) the CPU is doing nothing
+ * with which we could race, so do nothing.
+ */
+ if (cpu_death == CPU_DEATH_POWER) {
+ /*
+ * Wait for the core to enter a powered down or clock gated
+ * state, the latter happening when a JTAG probe is connected
+ * in which case the CPC will refuse to power down the core.
+ */
+ do {
+ mips_cpc_lock_other(core);
+ stat = read_cpc_co_stat_conf();
+ stat &= CPC_Cx_STAT_CONF_SEQSTATE_MSK;
+ mips_cpc_unlock_other();
+ } while (stat != CPC_Cx_STAT_CONF_SEQSTATE_D0 &&
+ stat != CPC_Cx_STAT_CONF_SEQSTATE_D2 &&
+ stat != CPC_Cx_STAT_CONF_SEQSTATE_U2);
+
+ /* Indicate the core is powered off */
+ bitmap_clear(core_power, core, 1);
+ } else if (cpu_has_mipsmt) {
+ /*
+ * Have a CPU with access to the offlined CPUs registers wait
+ * for its TC to halt.
+ */
+ err = smp_call_function_single(cpu_death_sibling,
+ wait_for_sibling_halt,
+ (void *)cpu, 1);
+ if (err)
+ panic("Failed to call remote sibling CPU\n");
+ }
+}
+
+#endif /* CONFIG_HOTPLUG_CPU */
+
static struct plat_smp_ops cps_smp_ops = {
.smp_setup = cps_smp_setup,
.prepare_cpus = cps_prepare_cpus,
@@ -314,9 +436,18 @@
.smp_finish = cps_smp_finish,
.send_ipi_single = gic_send_ipi_single,
.send_ipi_mask = gic_send_ipi_mask,
- .cpus_done = cps_cpus_done,
+#ifdef CONFIG_HOTPLUG_CPU
+ .cpu_disable = cps_cpu_disable,
+ .cpu_die = cps_cpu_die,
+#endif
};
+bool mips_cps_smp_in_use(void)
+{
+ extern struct plat_smp_ops *mp_ops;
+ return mp_ops == &cps_smp_ops;
+}
+
int register_cps_smp_ops(void)
{
if (!mips_cm_present()) {
diff --git a/arch/mips/kernel/smp-gic.c b/arch/mips/kernel/smp-gic.c
index 3bb1f92..3b21a96 100644
--- a/arch/mips/kernel/smp-gic.c
+++ b/arch/mips/kernel/smp-gic.c
@@ -15,12 +15,14 @@
#include <linux/printk.h>
#include <asm/gic.h>
+#include <asm/mips-cpc.h>
#include <asm/smp-ops.h>
void gic_send_ipi_single(int cpu, unsigned int action)
{
unsigned long flags;
unsigned int intr;
+ unsigned int core = cpu_data[cpu].core;
pr_debug("CPU%d: %s cpu %d action %u status %08x\n",
smp_processor_id(), __func__, cpu, action, read_c0_status());
@@ -41,6 +43,15 @@
}
gic_send_ipi(intr);
+
+ if (mips_cpc_present() && (core != current_cpu_data.core)) {
+ while (!cpumask_test_cpu(cpu, &cpu_coherent_mask)) {
+ mips_cpc_lock_other(core);
+ write_cpc_co_cmd(CPC_Cx_CMD_PWRUP);
+ mips_cpc_unlock_other();
+ }
+ }
+
local_irq_restore(flags);
}
diff --git a/arch/mips/kernel/smp-mt.c b/arch/mips/kernel/smp-mt.c
index f8e1314..3babf6e 100644
--- a/arch/mips/kernel/smp-mt.c
+++ b/arch/mips/kernel/smp-mt.c
@@ -183,10 +183,6 @@
local_irq_enable();
}
-static void vsmp_cpus_done(void)
-{
-}
-
/*
* Setup the PC, SP, and GP of a secondary processor and start it
* running!
@@ -287,7 +283,6 @@
.send_ipi_mask = vsmp_send_ipi_mask,
.init_secondary = vsmp_init_secondary,
.smp_finish = vsmp_smp_finish,
- .cpus_done = vsmp_cpus_done,
.boot_secondary = vsmp_boot_secondary,
.smp_setup = vsmp_smp_setup,
.prepare_cpus = vsmp_prepare_cpus,
diff --git a/arch/mips/kernel/smp-up.c b/arch/mips/kernel/smp-up.c
index 7fde3e4..17878d7 100644
--- a/arch/mips/kernel/smp-up.c
+++ b/arch/mips/kernel/smp-up.c
@@ -36,11 +36,6 @@
{
}
-/* Hook for after all CPUs are online */
-static void up_cpus_done(void)
-{
-}
-
/*
* Firmware CPU startup hook
*/
@@ -73,7 +68,6 @@
.send_ipi_mask = up_send_ipi_mask,
.init_secondary = up_init_secondary,
.smp_finish = up_smp_finish,
- .cpus_done = up_cpus_done,
.boot_secondary = up_boot_secondary,
.smp_setup = up_smp_setup,
.prepare_cpus = up_prepare_cpus,
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 0a022ee..9bad52e 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -43,10 +43,6 @@
#include <asm/time.h>
#include <asm/setup.h>
-#ifdef CONFIG_MIPS_MT_SMTC
-#include <asm/mipsmtregs.h>
-#endif /* CONFIG_MIPS_MT_SMTC */
-
volatile cpumask_t cpu_callin_map; /* Bitmask of started secondaries */
int __cpu_number_map[NR_CPUS]; /* Map physical to logical */
@@ -66,6 +62,8 @@
/* representing cpus for which sibling maps can be computed */
static cpumask_t cpu_sibling_setup_map;
+cpumask_t cpu_coherent_mask;
+
static inline void set_cpu_sibling_map(int cpu)
{
int i;
@@ -102,12 +100,6 @@
{
unsigned int cpu;
-#ifdef CONFIG_MIPS_MT_SMTC
- /* Only do cpu_probe for first TC of CPU */
- if ((read_c0_tcbind() & TCBIND_CURTC) != 0)
- __cpu_name[smp_processor_id()] = __cpu_name[0];
- else
-#endif /* CONFIG_MIPS_MT_SMTC */
cpu_probe();
cpu_report();
per_cpu_trap_init(false);
@@ -124,6 +116,7 @@
cpu = smp_processor_id();
cpu_data[cpu].udelay_val = loops_per_jiffy;
+ cpu_set(cpu, cpu_coherent_mask);
notify_cpu_starting(cpu);
set_cpu_online(cpu, true);
@@ -173,7 +166,6 @@
void __init smp_cpus_done(unsigned int max_cpus)
{
- mp_ops->cpus_done();
}
/* called from main before smp_init() */
@@ -186,6 +178,7 @@
#ifndef CONFIG_HOTPLUG_CPU
init_cpu_present(cpu_possible_mask);
#endif
+ cpumask_copy(&cpu_coherent_mask, cpu_possible_mask);
}
/* preload SMP state for boot cpu */
@@ -238,13 +231,10 @@
* o collapses to normal function call on UP kernels
* o collapses to normal function call on systems with a single shared
* primary cache.
- * o CONFIG_MIPS_MT_SMTC currently implies there is only one physical core.
*/
static inline void smp_on_other_tlbs(void (*func) (void *info), void *info)
{
-#ifndef CONFIG_MIPS_MT_SMTC
smp_call_function(func, info, 1);
-#endif
}
static inline void smp_on_each_tlb(void (*func) (void *info), void *info)
@@ -404,3 +394,46 @@
}
EXPORT_SYMBOL(dump_send_ipi);
#endif
+
+#ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
+
+static DEFINE_PER_CPU(atomic_t, tick_broadcast_count);
+static DEFINE_PER_CPU(struct call_single_data, tick_broadcast_csd);
+
+void tick_broadcast(const struct cpumask *mask)
+{
+ atomic_t *count;
+ struct call_single_data *csd;
+ int cpu;
+
+ for_each_cpu(cpu, mask) {
+ count = &per_cpu(tick_broadcast_count, cpu);
+ csd = &per_cpu(tick_broadcast_csd, cpu);
+
+ if (atomic_inc_return(count) == 1)
+ smp_call_function_single_async(cpu, csd);
+ }
+}
+
+static void tick_broadcast_callee(void *info)
+{
+ int cpu = smp_processor_id();
+ tick_receive_broadcast();
+ atomic_set(&per_cpu(tick_broadcast_count, cpu), 0);
+}
+
+static int __init tick_broadcast_init(void)
+{
+ struct call_single_data *csd;
+ int cpu;
+
+ for (cpu = 0; cpu < NR_CPUS; cpu++) {
+ csd = &per_cpu(tick_broadcast_csd, cpu);
+ csd->func = tick_broadcast_callee;
+ }
+
+ return 0;
+}
+early_initcall(tick_broadcast_init);
+
+#endif /* CONFIG_GENERIC_CLOCKEVENTS_BROADCAST */
diff --git a/arch/mips/kernel/smtc-asm.S b/arch/mips/kernel/smtc-asm.S
deleted file mode 100644
index 2866863..0000000
--- a/arch/mips/kernel/smtc-asm.S
+++ /dev/null
@@ -1,133 +0,0 @@
-/*
- * Assembly Language Functions for MIPS MT SMTC support
- */
-
-/*
- * This file should be built into the kernel only if CONFIG_MIPS_MT_SMTC is set. */
-
-#include <asm/regdef.h>
-#include <asm/asmmacro.h>
-#include <asm/stackframe.h>
-#include <asm/irqflags.h>
-
-/*
- * "Software Interrupt" linkage.
- *
- * This is invoked when an "Interrupt" is sent from one TC to another,
- * where the TC to be interrupted is halted, has it's Restart address
- * and Status values saved by the "remote control" thread, then modified
- * to cause execution to begin here, in kenel mode. This code then
- * disguises the TC state as that of an exception and transfers
- * control to the general exception or vectored interrupt handler.
- */
- .set noreorder
-
-/*
-The __smtc_ipi_vector would use k0 and k1 as temporaries and
-1) Set EXL (this is per-VPE, so this can't be done by proxy!)
-2) Restore the K/CU and IXMT bits to the pre "exception" state
- (EXL means no interrupts and access to the kernel map).
-3) Set EPC to be the saved value of TCRestart.
-4) Jump to the exception handler entry point passed by the sender.
-
-CAN WE PROVE THAT WE WON'T DO THIS IF INTS DISABLED??
-*/
-
-/*
- * Reviled and slandered vision: Set EXL and restore K/CU/IXMT
- * state of pre-halt thread, then save everything and call
- * thought some function pointer to imaginary_exception, which
- * will parse a register value or memory message queue to
- * deliver things like interprocessor interrupts. On return
- * from that function, jump to the global ret_from_irq code
- * to invoke the scheduler and return as appropriate.
- */
-
-#define PT_PADSLOT4 (PT_R0-8)
-#define PT_PADSLOT5 (PT_R0-4)
-
- .text
- .align 5
-FEXPORT(__smtc_ipi_vector)
-#ifdef CONFIG_CPU_MICROMIPS
- nop
-#endif
- .set noat
- /* Disable thread scheduling to make Status update atomic */
- DMT 27 # dmt k1
- _ehb
- /* Set EXL */
- mfc0 k0,CP0_STATUS
- ori k0,k0,ST0_EXL
- mtc0 k0,CP0_STATUS
- _ehb
- /* Thread scheduling now inhibited by EXL. Restore TE state. */
- andi k1,k1,VPECONTROL_TE
- beqz k1,1f
- emt
-1:
- /*
- * The IPI sender has put some information on the anticipated
- * kernel stack frame. If we were in user mode, this will be
- * built above the saved kernel SP. If we were already in the
- * kernel, it will be built above the current CPU SP.
- *
- * Were we in kernel mode, as indicated by CU0?
- */
- sll k1,k0,3
- .set noreorder
- bltz k1,2f
- move k1,sp
- .set reorder
- /*
- * If previously in user mode, set CU0 and use kernel stack.
- */
- li k1,ST0_CU0
- or k1,k1,k0
- mtc0 k1,CP0_STATUS
- _ehb
- get_saved_sp
- /* Interrupting TC will have pre-set values in slots in the new frame */
-2: subu k1,k1,PT_SIZE
- /* Load TCStatus Value */
- lw k0,PT_TCSTATUS(k1)
- /* Write it to TCStatus to restore CU/KSU/IXMT state */
- mtc0 k0,$2,1
- _ehb
- lw k0,PT_EPC(k1)
- mtc0 k0,CP0_EPC
- /* Save all will redundantly recompute the SP, but use it for now */
- SAVE_ALL
- CLI
- TRACE_IRQS_OFF
- /* Function to be invoked passed stack pad slot 5 */
- lw t0,PT_PADSLOT5(sp)
- /* Argument from sender passed in stack pad slot 4 */
- lw a0,PT_PADSLOT4(sp)
- LONG_L s0, TI_REGS($28)
- LONG_S sp, TI_REGS($28)
- PTR_LA ra, ret_from_irq
- jr t0
-
-/*
- * Called from idle loop to provoke processing of queued IPIs
- * First IPI message in queue passed as argument.
- */
-
-LEAF(self_ipi)
- /* Before anything else, block interrupts */
- mfc0 t0,CP0_TCSTATUS
- ori t1,t0,TCSTATUS_IXMT
- mtc0 t1,CP0_TCSTATUS
- _ehb
- /* We know we're in kernel mode, so prepare stack frame */
- subu t1,sp,PT_SIZE
- sw ra,PT_EPC(t1)
- sw a0,PT_PADSLOT4(t1)
- la t2,ipi_decode
- sw t2,PT_PADSLOT5(t1)
- /* Save pre-disable value of TCStatus */
- sw t0,PT_TCSTATUS(t1)
- j __smtc_ipi_vector
- nop
-END(self_ipi)
diff --git a/arch/mips/kernel/smtc-proc.c b/arch/mips/kernel/smtc-proc.c
deleted file mode 100644
index 38635a9..0000000
--- a/arch/mips/kernel/smtc-proc.c
+++ /dev/null
@@ -1,102 +0,0 @@
-/*
- * /proc hooks for SMTC kernel
- * Copyright (C) 2005 Mips Technologies, Inc
- */
-
-#include <linux/kernel.h>
-#include <linux/sched.h>
-#include <linux/cpumask.h>
-#include <linux/interrupt.h>
-
-#include <asm/cpu.h>
-#include <asm/processor.h>
-#include <linux/atomic.h>
-#include <asm/hardirq.h>
-#include <asm/mmu_context.h>
-#include <asm/mipsregs.h>
-#include <asm/cacheflush.h>
-#include <linux/proc_fs.h>
-#include <linux/seq_file.h>
-
-#include <asm/smtc_proc.h>
-
-/*
- * /proc diagnostic and statistics hooks
- */
-
-/*
- * Statistics gathered
- */
-unsigned long selfipis[NR_CPUS];
-
-struct smtc_cpu_proc smtc_cpu_stats[NR_CPUS];
-
-atomic_t smtc_fpu_recoveries;
-
-static int smtc_proc_show(struct seq_file *m, void *v)
-{
- int i;
- extern unsigned long ebase;
-
- seq_printf(m, "SMTC Status Word: 0x%08x\n", smtc_status);
- seq_printf(m, "Config7: 0x%08x\n", read_c0_config7());
- seq_printf(m, "EBASE: 0x%08lx\n", ebase);
- seq_printf(m, "Counter Interrupts taken per CPU (TC)\n");
- for (i=0; i < NR_CPUS; i++)
- seq_printf(m, "%d: %ld\n", i, smtc_cpu_stats[i].timerints);
- seq_printf(m, "Self-IPIs by CPU:\n");
- for(i = 0; i < NR_CPUS; i++)
- seq_printf(m, "%d: %ld\n", i, smtc_cpu_stats[i].selfipis);
- seq_printf(m, "%d Recoveries of \"stolen\" FPU\n",
- atomic_read(&smtc_fpu_recoveries));
- return 0;
-}
-
-static int smtc_proc_open(struct inode *inode, struct file *file)
-{
- return single_open(file, smtc_proc_show, NULL);
-}
-
-static const struct file_operations smtc_proc_fops = {
- .open = smtc_proc_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
-};
-
-void init_smtc_stats(void)
-{
- int i;
-
- for (i=0; i<NR_CPUS; i++) {
- smtc_cpu_stats[i].timerints = 0;
- smtc_cpu_stats[i].selfipis = 0;
- }
-
- atomic_set(&smtc_fpu_recoveries, 0);
-
- proc_create("smtc", 0444, NULL, &smtc_proc_fops);
-}
-
-static int proc_cpuinfo_chain_call(struct notifier_block *nfb,
- unsigned long action_unused, void *data)
-{
- struct proc_cpuinfo_notifier_args *pcn = data;
- struct seq_file *m = pcn->m;
- unsigned long n = pcn->n;
-
- if (!cpu_has_mipsmt)
- return NOTIFY_OK;
-
- seq_printf(m, "VPE\t\t\t: %d\n", cpu_data[n].vpe_id);
- seq_printf(m, "TC\t\t\t: %d\n", cpu_data[n].tc_id);
-
- return NOTIFY_OK;
-}
-
-static int __init proc_cpuinfo_notifier_init(void)
-{
- return proc_cpuinfo_notifier(proc_cpuinfo_chain_call, 0);
-}
-
-subsys_initcall(proc_cpuinfo_notifier_init);
diff --git a/arch/mips/kernel/smtc.c b/arch/mips/kernel/smtc.c
deleted file mode 100644
index c1681d6..0000000
--- a/arch/mips/kernel/smtc.c
+++ /dev/null
@@ -1,1528 +0,0 @@
-/*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version 2
- * of the License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
- *
- * Copyright (C) 2004 Mips Technologies, Inc
- * Copyright (C) 2008 Kevin D. Kissell
- */
-
-#include <linux/clockchips.h>
-#include <linux/kernel.h>
-#include <linux/sched.h>
-#include <linux/smp.h>
-#include <linux/cpumask.h>
-#include <linux/interrupt.h>
-#include <linux/kernel_stat.h>
-#include <linux/module.h>
-#include <linux/ftrace.h>
-#include <linux/slab.h>
-
-#include <asm/cpu.h>
-#include <asm/processor.h>
-#include <linux/atomic.h>
-#include <asm/hardirq.h>
-#include <asm/hazards.h>
-#include <asm/irq.h>
-#include <asm/idle.h>
-#include <asm/mmu_context.h>
-#include <asm/mipsregs.h>
-#include <asm/cacheflush.h>
-#include <asm/time.h>
-#include <asm/addrspace.h>
-#include <asm/smtc.h>
-#include <asm/smtc_proc.h>
-#include <asm/setup.h>
-
-/*
- * SMTC Kernel needs to manipulate low-level CPU interrupt mask
- * in do_IRQ. These are passed in setup_irq_smtc() and stored
- * in this table.
- */
-unsigned long irq_hwmask[NR_IRQS];
-
-#define LOCK_MT_PRA() \
- local_irq_save(flags); \
- mtflags = dmt()
-
-#define UNLOCK_MT_PRA() \
- emt(mtflags); \
- local_irq_restore(flags)
-
-#define LOCK_CORE_PRA() \
- local_irq_save(flags); \
- mtflags = dvpe()
-
-#define UNLOCK_CORE_PRA() \
- evpe(mtflags); \
- local_irq_restore(flags)
-
-/*
- * Data structures purely associated with SMTC parallelism
- */
-
-
-/*
- * Table for tracking ASIDs whose lifetime is prolonged.
- */
-
-asiduse smtc_live_asid[MAX_SMTC_TLBS][MAX_SMTC_ASIDS];
-
-/*
- * Number of InterProcessor Interrupt (IPI) message buffers to allocate
- */
-
-#define IPIBUF_PER_CPU 4
-
-struct smtc_ipi_q IPIQ[NR_CPUS];
-static struct smtc_ipi_q freeIPIq;
-
-
-/*
- * Number of FPU contexts for each VPE
- */
-
-static int smtc_nconf1[MAX_SMTC_VPES];
-
-
-/* Forward declarations */
-
-void ipi_decode(struct smtc_ipi *);
-static void post_direct_ipi(int cpu, struct smtc_ipi *pipi);
-static void setup_cross_vpe_interrupts(unsigned int nvpe);
-void init_smtc_stats(void);
-
-/* Global SMTC Status */
-
-unsigned int smtc_status;
-
-/* Boot command line configuration overrides */
-
-static int vpe0limit;
-static int ipibuffers;
-static int nostlb;
-static int asidmask;
-unsigned long smtc_asid_mask = 0xff;
-
-static int __init vpe0tcs(char *str)
-{
- get_option(&str, &vpe0limit);
-
- return 1;
-}
-
-static int __init ipibufs(char *str)
-{
- get_option(&str, &ipibuffers);
- return 1;
-}
-
-static int __init stlb_disable(char *s)
-{
- nostlb = 1;
- return 1;
-}
-
-static int __init asidmask_set(char *str)
-{
- get_option(&str, &asidmask);
- switch (asidmask) {
- case 0x1:
- case 0x3:
- case 0x7:
- case 0xf:
- case 0x1f:
- case 0x3f:
- case 0x7f:
- case 0xff:
- smtc_asid_mask = (unsigned long)asidmask;
- break;
- default:
- printk("ILLEGAL ASID mask 0x%x from command line\n", asidmask);
- }
- return 1;
-}
-
-__setup("vpe0tcs=", vpe0tcs);
-__setup("ipibufs=", ipibufs);
-__setup("nostlb", stlb_disable);
-__setup("asidmask=", asidmask_set);
-
-#ifdef CONFIG_SMTC_IDLE_HOOK_DEBUG
-
-static int hang_trig;
-
-static int __init hangtrig_enable(char *s)
-{
- hang_trig = 1;
- return 1;
-}
-
-
-__setup("hangtrig", hangtrig_enable);
-
-#define DEFAULT_BLOCKED_IPI_LIMIT 32
-
-static int timerq_limit = DEFAULT_BLOCKED_IPI_LIMIT;
-
-static int __init tintq(char *str)
-{
- get_option(&str, &timerq_limit);
- return 1;
-}
-
-__setup("tintq=", tintq);
-
-static int imstuckcount[MAX_SMTC_VPES][8];
-/* vpemask represents IM/IE bits of per-VPE Status registers, low-to-high */
-static int vpemask[MAX_SMTC_VPES][8] = {
- {0, 0, 1, 0, 0, 0, 0, 1},
- {0, 0, 0, 0, 0, 0, 0, 1}
-};
-int tcnoprog[NR_CPUS];
-static atomic_t idle_hook_initialized = ATOMIC_INIT(0);
-static int clock_hang_reported[NR_CPUS];
-
-#endif /* CONFIG_SMTC_IDLE_HOOK_DEBUG */
-
-/*
- * Configure shared TLB - VPC configuration bit must be set by caller
- */
-
-static void smtc_configure_tlb(void)
-{
- int i, tlbsiz, vpes;
- unsigned long mvpconf0;
- unsigned long config1val;
-
- /* Set up ASID preservation table */
- for (vpes=0; vpes<MAX_SMTC_TLBS; vpes++) {
- for(i = 0; i < MAX_SMTC_ASIDS; i++) {
- smtc_live_asid[vpes][i] = 0;
- }
- }
- mvpconf0 = read_c0_mvpconf0();
-
- if ((vpes = ((mvpconf0 & MVPCONF0_PVPE)
- >> MVPCONF0_PVPE_SHIFT) + 1) > 1) {
- /* If we have multiple VPEs, try to share the TLB */
- if ((mvpconf0 & MVPCONF0_TLBS) && !nostlb) {
- /*
- * If TLB sizing is programmable, shared TLB
- * size is the total available complement.
- * Otherwise, we have to take the sum of all
- * static VPE TLB entries.
- */
- if ((tlbsiz = ((mvpconf0 & MVPCONF0_PTLBE)
- >> MVPCONF0_PTLBE_SHIFT)) == 0) {
- /*
- * If there's more than one VPE, there had better
- * be more than one TC, because we need one to bind
- * to each VPE in turn to be able to read
- * its configuration state!
- */
- settc(1);
- /* Stop the TC from doing anything foolish */
- write_tc_c0_tchalt(TCHALT_H);
- mips_ihb();
- /* No need to un-Halt - that happens later anyway */
- for (i=0; i < vpes; i++) {
- write_tc_c0_tcbind(i);
- /*
- * To be 100% sure we're really getting the right
- * information, we exit the configuration state
- * and do an IHB after each rebinding.
- */
- write_c0_mvpcontrol(
- read_c0_mvpcontrol() & ~ MVPCONTROL_VPC );
- mips_ihb();
- /*
- * Only count if the MMU Type indicated is TLB
- */
- if (((read_vpe_c0_config() & MIPS_CONF_MT) >> 7) == 1) {
- config1val = read_vpe_c0_config1();
- tlbsiz += ((config1val >> 25) & 0x3f) + 1;
- }
-
- /* Put core back in configuration state */
- write_c0_mvpcontrol(
- read_c0_mvpcontrol() | MVPCONTROL_VPC );
- mips_ihb();
- }
- }
- write_c0_mvpcontrol(read_c0_mvpcontrol() | MVPCONTROL_STLB);
- ehb();
-
- /*
- * Setup kernel data structures to use software total,
- * rather than read the per-VPE Config1 value. The values
- * for "CPU 0" gets copied to all the other CPUs as part
- * of their initialization in smtc_cpu_setup().
- */
-
- /* MIPS32 limits TLB indices to 64 */
- if (tlbsiz > 64)
- tlbsiz = 64;
- cpu_data[0].tlbsize = current_cpu_data.tlbsize = tlbsiz;
- smtc_status |= SMTC_TLB_SHARED;
- local_flush_tlb_all();
-
- printk("TLB of %d entry pairs shared by %d VPEs\n",
- tlbsiz, vpes);
- } else {
- printk("WARNING: TLB Not Sharable on SMTC Boot!\n");
- }
- }
-}
-
-
-/*
- * Incrementally build the CPU map out of constituent MIPS MT cores,
- * using the specified available VPEs and TCs. Plaform code needs
- * to ensure that each MIPS MT core invokes this routine on reset,
- * one at a time(!).
- *
- * This version of the build_cpu_map and prepare_cpus routines assumes
- * that *all* TCs of a MIPS MT core will be used for Linux, and that
- * they will be spread across *all* available VPEs (to minimise the
- * loss of efficiency due to exception service serialization).
- * An improved version would pick up configuration information and
- * possibly leave some TCs/VPEs as "slave" processors.
- *
- * Use c0_MVPConf0 to find out how many TCs are available, setting up
- * cpu_possible_mask and the logical/physical mappings.
- */
-
-int __init smtc_build_cpu_map(int start_cpu_slot)
-{
- int i, ntcs;
-
- /*
- * The CPU map isn't actually used for anything at this point,
- * so it's not clear what else we should do apart from set
- * everything up so that "logical" = "physical".
- */
- ntcs = ((read_c0_mvpconf0() & MVPCONF0_PTC) >> MVPCONF0_PTC_SHIFT) + 1;
- for (i=start_cpu_slot; i<NR_CPUS && i<ntcs; i++) {
- set_cpu_possible(i, true);
- __cpu_number_map[i] = i;
- __cpu_logical_map[i] = i;
- }
-#ifdef CONFIG_MIPS_MT_FPAFF
- /* Initialize map of CPUs with FPUs */
- cpus_clear(mt_fpu_cpumask);
-#endif
-
- /* One of those TC's is the one booting, and not a secondary... */
- printk("%i available secondary CPU TC(s)\n", i - 1);
-
- return i;
-}
-
-/*
- * Common setup before any secondaries are started
- * Make sure all CPUs are in a sensible state before we boot any of the
- * secondaries.
- *
- * For MIPS MT "SMTC" operation, we set up all TCs, spread as evenly
- * as possible across the available VPEs.
- */
-
-static void smtc_tc_setup(int vpe, int tc, int cpu)
-{
- static int cp1contexts[MAX_SMTC_VPES];
-
- /*
- * Make a local copy of the available FPU contexts in order
- * to keep track of TCs that can have one.
- */
- if (tc == 1)
- {
- /*
- * FIXME: Multi-core SMTC hasn't been tested and the
- * maximum number of VPEs may change.
- */
- cp1contexts[0] = smtc_nconf1[0] - 1;
- cp1contexts[1] = smtc_nconf1[1];
- }
-
- settc(tc);
- write_tc_c0_tchalt(TCHALT_H);
- mips_ihb();
- write_tc_c0_tcstatus((read_tc_c0_tcstatus()
- & ~(TCSTATUS_TKSU | TCSTATUS_DA | TCSTATUS_IXMT))
- | TCSTATUS_A);
- /*
- * TCContext gets an offset from the base of the IPIQ array
- * to be used in low-level code to detect the presence of
- * an active IPI queue.
- */
- write_tc_c0_tccontext((sizeof(struct smtc_ipi_q) * cpu) << 16);
-
- /* Bind TC to VPE. */
- write_tc_c0_tcbind(vpe);
-
- /* In general, all TCs should have the same cpu_data indications. */
- memcpy(&cpu_data[cpu], &cpu_data[0], sizeof(struct cpuinfo_mips));
-
- /* Check to see if there is a FPU context available for this TC. */
- if (!cp1contexts[vpe])
- cpu_data[cpu].options &= ~MIPS_CPU_FPU;
- else
- cp1contexts[vpe]--;
-
- /* Store the TC and VPE into the cpu_data structure. */
- cpu_data[cpu].vpe_id = vpe;
- cpu_data[cpu].tc_id = tc;
-
- /* FIXME: Multi-core SMTC hasn't been tested, but be prepared. */
- cpu_data[cpu].core = (read_vpe_c0_ebase() >> 1) & 0xff;
-}
-
-/*
- * Tweak to get Count registers synced as closely as possible. The
- * value seems good for 34K-class cores.
- */
-
-#define CP0_SKEW 8
-
-void smtc_prepare_cpus(int cpus)
-{
- int i, vpe, tc, ntc, nvpe, tcpervpe[NR_CPUS], slop, cpu;
- unsigned long flags;
- unsigned long val;
- int nipi;
- struct smtc_ipi *pipi;
-
- /* disable interrupts so we can disable MT */
- local_irq_save(flags);
- /* disable MT so we can configure */
- dvpe();
- dmt();
-
- spin_lock_init(&freeIPIq.lock);
-
- /*
- * We probably don't have as many VPEs as we do SMP "CPUs",
- * but it's possible - and in any case we'll never use more!
- */
- for (i=0; i<NR_CPUS; i++) {
- IPIQ[i].head = IPIQ[i].tail = NULL;
- spin_lock_init(&IPIQ[i].lock);
- IPIQ[i].depth = 0;
- IPIQ[i].resched_flag = 0; /* No reschedules queued initially */
- }
-
- /* cpu_data index starts at zero */
- cpu = 0;
- cpu_data[cpu].vpe_id = 0;
- cpu_data[cpu].tc_id = 0;
- cpu_data[cpu].core = (read_c0_ebase() >> 1) & 0xff;
- cpu++;
-
- /* Report on boot-time options */
- mips_mt_set_cpuoptions();
- if (vpelimit > 0)
- printk("Limit of %d VPEs set\n", vpelimit);
- if (tclimit > 0)
- printk("Limit of %d TCs set\n", tclimit);
- if (nostlb) {
- printk("Shared TLB Use Inhibited - UNSAFE for Multi-VPE Operation\n");
- }
- if (asidmask)
- printk("ASID mask value override to 0x%x\n", asidmask);
-
- /* Temporary */
-#ifdef CONFIG_SMTC_IDLE_HOOK_DEBUG
- if (hang_trig)
- printk("Logic Analyser Trigger on suspected TC hang\n");
-#endif /* CONFIG_SMTC_IDLE_HOOK_DEBUG */
-
- /* Put MVPE's into 'configuration state' */
- write_c0_mvpcontrol( read_c0_mvpcontrol() | MVPCONTROL_VPC );
-
- val = read_c0_mvpconf0();
- nvpe = ((val & MVPCONF0_PVPE) >> MVPCONF0_PVPE_SHIFT) + 1;
- if (vpelimit > 0 && nvpe > vpelimit)
- nvpe = vpelimit;
- ntc = ((val & MVPCONF0_PTC) >> MVPCONF0_PTC_SHIFT) + 1;
- if (ntc > NR_CPUS)
- ntc = NR_CPUS;
- if (tclimit > 0 && ntc > tclimit)
- ntc = tclimit;
- slop = ntc % nvpe;
- for (i = 0; i < nvpe; i++) {
- tcpervpe[i] = ntc / nvpe;
- if (slop) {
- if((slop - i) > 0) tcpervpe[i]++;
- }
- }
- /* Handle command line override for VPE0 */
- if (vpe0limit > ntc) vpe0limit = ntc;
- if (vpe0limit > 0) {
- int slopslop;
- if (vpe0limit < tcpervpe[0]) {
- /* Reducing TC count - distribute to others */
- slop = tcpervpe[0] - vpe0limit;
- slopslop = slop % (nvpe - 1);
- tcpervpe[0] = vpe0limit;
- for (i = 1; i < nvpe; i++) {
- tcpervpe[i] += slop / (nvpe - 1);
- if(slopslop && ((slopslop - (i - 1) > 0)))
- tcpervpe[i]++;
- }
- } else if (vpe0limit > tcpervpe[0]) {
- /* Increasing TC count - steal from others */
- slop = vpe0limit - tcpervpe[0];
- slopslop = slop % (nvpe - 1);
- tcpervpe[0] = vpe0limit;
- for (i = 1; i < nvpe; i++) {
- tcpervpe[i] -= slop / (nvpe - 1);
- if(slopslop && ((slopslop - (i - 1) > 0)))
- tcpervpe[i]--;
- }
- }
- }
-
- /* Set up shared TLB */
- smtc_configure_tlb();
-
- for (tc = 0, vpe = 0 ; (vpe < nvpe) && (tc < ntc) ; vpe++) {
- /* Get number of CP1 contexts for each VPE. */
- if (tc == 0)
- {
- /*
- * Do not call settc() for TC0 or the FPU context
- * value will be incorrect. Besides, we know that
- * we are TC0 anyway.
- */
- smtc_nconf1[0] = ((read_vpe_c0_vpeconf1() &
- VPECONF1_NCP1) >> VPECONF1_NCP1_SHIFT);
- if (nvpe == 2)
- {
- settc(1);
- smtc_nconf1[1] = ((read_vpe_c0_vpeconf1() &
- VPECONF1_NCP1) >> VPECONF1_NCP1_SHIFT);
- settc(0);
- }
- }
- if (tcpervpe[vpe] == 0)
- continue;
- if (vpe != 0)
- printk(", ");
- printk("VPE %d: TC", vpe);
- for (i = 0; i < tcpervpe[vpe]; i++) {
- /*
- * TC 0 is bound to VPE 0 at reset,
- * and is presumably executing this
- * code. Leave it alone!
- */
- if (tc != 0) {
- smtc_tc_setup(vpe, tc, cpu);
- if (vpe != 0) {
- /*
- * Set MVP bit (possibly again). Do it
- * here to catch CPUs that have no TCs
- * bound to the VPE at reset. In that
- * case, a TC must be bound to the VPE
- * before we can set VPEControl[MVP]
- */
- write_vpe_c0_vpeconf0(
- read_vpe_c0_vpeconf0() |
- VPECONF0_MVP);
- }
- cpu++;
- }
- printk(" %d", tc);
- tc++;
- }
- if (vpe != 0) {
- /*
- * Allow this VPE to control others.
- */
- write_vpe_c0_vpeconf0(read_vpe_c0_vpeconf0() |
- VPECONF0_MVP);
-
- /*
- * Clear any stale software interrupts from VPE's Cause
- */
- write_vpe_c0_cause(0);
-
- /*
- * Clear ERL/EXL of VPEs other than 0
- * and set restricted interrupt enable/mask.
- */
- write_vpe_c0_status((read_vpe_c0_status()
- & ~(ST0_BEV | ST0_ERL | ST0_EXL | ST0_IM))
- | (STATUSF_IP0 | STATUSF_IP1 | STATUSF_IP7
- | ST0_IE));
- /*
- * set config to be the same as vpe0,
- * particularly kseg0 coherency alg
- */
- write_vpe_c0_config(read_c0_config());
- /* Clear any pending timer interrupt */
- write_vpe_c0_compare(0);
- /* Propagate Config7 */
- write_vpe_c0_config7(read_c0_config7());
- write_vpe_c0_count(read_c0_count() + CP0_SKEW);
- ehb();
- }
- /* enable multi-threading within VPE */
- write_vpe_c0_vpecontrol(read_vpe_c0_vpecontrol() | VPECONTROL_TE);
- /* enable the VPE */
- write_vpe_c0_vpeconf0(read_vpe_c0_vpeconf0() | VPECONF0_VPA);
- }
-
- /*
- * Pull any physically present but unused TCs out of circulation.
- */
- while (tc < (((val & MVPCONF0_PTC) >> MVPCONF0_PTC_SHIFT) + 1)) {
- set_cpu_possible(tc, false);
- set_cpu_present(tc, false);
- tc++;
- }
-
- /* release config state */
- write_c0_mvpcontrol( read_c0_mvpcontrol() & ~ MVPCONTROL_VPC );
-
- printk("\n");
-
- /* Set up coprocessor affinity CPU mask(s) */
-
-#ifdef CONFIG_MIPS_MT_FPAFF
- for (tc = 0; tc < ntc; tc++) {
- if (cpu_data[tc].options & MIPS_CPU_FPU)
- cpu_set(tc, mt_fpu_cpumask);
- }
-#endif
-
- /* set up ipi interrupts... */
-
- /* If we have multiple VPEs running, set up the cross-VPE interrupt */
-
- setup_cross_vpe_interrupts(nvpe);
-
- /* Set up queue of free IPI "messages". */
- nipi = NR_CPUS * IPIBUF_PER_CPU;
- if (ipibuffers > 0)
- nipi = ipibuffers;
-
- pipi = kmalloc(nipi *sizeof(struct smtc_ipi), GFP_KERNEL);
- if (pipi == NULL)
- panic("kmalloc of IPI message buffers failed");
- else
- printk("IPI buffer pool of %d buffers\n", nipi);
- for (i = 0; i < nipi; i++) {
- smtc_ipi_nq(&freeIPIq, pipi);
- pipi++;
- }
-
- /* Arm multithreading and enable other VPEs - but all TCs are Halted */
- emt(EMT_ENABLE);
- evpe(EVPE_ENABLE);
- local_irq_restore(flags);
- /* Initialize SMTC /proc statistics/diagnostics */
- init_smtc_stats();
-}
-
-
-/*
- * Setup the PC, SP, and GP of a secondary processor and start it
- * running!
- * smp_bootstrap is the place to resume from
- * __KSTK_TOS(idle) is apparently the stack pointer
- * (unsigned long)idle->thread_info the gp
- *
- */
-void smtc_boot_secondary(int cpu, struct task_struct *idle)
-{
- extern u32 kernelsp[NR_CPUS];
- unsigned long flags;
- int mtflags;
-
- LOCK_MT_PRA();
- if (cpu_data[cpu].vpe_id != cpu_data[smp_processor_id()].vpe_id) {
- dvpe();
- }
- settc(cpu_data[cpu].tc_id);
-
- /* pc */
- write_tc_c0_tcrestart((unsigned long)&smp_bootstrap);
-
- /* stack pointer */
- kernelsp[cpu] = __KSTK_TOS(idle);
- write_tc_gpr_sp(__KSTK_TOS(idle));
-
- /* global pointer */
- write_tc_gpr_gp((unsigned long)task_thread_info(idle));
-
- smtc_status |= SMTC_MTC_ACTIVE;
- write_tc_c0_tchalt(0);
- if (cpu_data[cpu].vpe_id != cpu_data[smp_processor_id()].vpe_id) {
- evpe(EVPE_ENABLE);
- }
- UNLOCK_MT_PRA();
-}
-
-void smtc_init_secondary(void)
-{
-}
-
-void smtc_smp_finish(void)
-{
- int cpu = smp_processor_id();
-
- /*
- * Lowest-numbered CPU per VPE starts a clock tick.
- * Like per_cpu_trap_init() hack, this assumes that
- * SMTC init code assigns TCs consdecutively and
- * in ascending order across available VPEs.
- */
- if (cpu > 0 && (cpu_data[cpu].vpe_id != cpu_data[cpu - 1].vpe_id))
- write_c0_compare(read_c0_count() + mips_hpt_frequency/HZ);
-
- local_irq_enable();
-
- printk("TC %d going on-line as CPU %d\n",
- cpu_data[smp_processor_id()].tc_id, smp_processor_id());
-}
-
-void smtc_cpus_done(void)
-{
-}
-
-/*
- * Support for SMTC-optimized driver IRQ registration
- */
-
-/*
- * SMTC Kernel needs to manipulate low-level CPU interrupt mask
- * in do_IRQ. These are passed in setup_irq_smtc() and stored
- * in this table.
- */
-
-int setup_irq_smtc(unsigned int irq, struct irqaction * new,
- unsigned long hwmask)
-{
-#ifdef CONFIG_SMTC_IDLE_HOOK_DEBUG
- unsigned int vpe = current_cpu_data.vpe_id;
-
- vpemask[vpe][irq - MIPS_CPU_IRQ_BASE] = 1;
-#endif
- irq_hwmask[irq] = hwmask;
-
- return setup_irq(irq, new);
-}
-
-#ifdef CONFIG_MIPS_MT_SMTC_IRQAFF
-/*
- * Support for IRQ affinity to TCs
- */
-
-void smtc_set_irq_affinity(unsigned int irq, cpumask_t affinity)
-{
- /*
- * If a "fast path" cache of quickly decodable affinity state
- * is maintained, this is where it gets done, on a call up
- * from the platform affinity code.
- */
-}
-
-void smtc_forward_irq(struct irq_data *d)
-{
- unsigned int irq = d->irq;
- int target;
-
- /*
- * OK wise guy, now figure out how to get the IRQ
- * to be serviced on an authorized "CPU".
- *
- * Ideally, to handle the situation where an IRQ has multiple
- * eligible CPUS, we would maintain state per IRQ that would
- * allow a fair distribution of service requests. Since the
- * expected use model is any-or-only-one, for simplicity
- * and efficiency, we just pick the easiest one to find.
- */
-
- target = cpumask_first(d->affinity);
-
- /*
- * We depend on the platform code to have correctly processed
- * IRQ affinity change requests to ensure that the IRQ affinity
- * mask has been purged of bits corresponding to nonexistent and
- * offline "CPUs", and to TCs bound to VPEs other than the VPE
- * connected to the physical interrupt input for the interrupt
- * in question. Otherwise we have a nasty problem with interrupt
- * mask management. This is best handled in non-performance-critical
- * platform IRQ affinity setting code, to minimize interrupt-time
- * checks.
- */
-
- /* If no one is eligible, service locally */
- if (target >= NR_CPUS)
- do_IRQ_no_affinity(irq);
- else
- smtc_send_ipi(target, IRQ_AFFINITY_IPI, irq);
-}
-
-#endif /* CONFIG_MIPS_MT_SMTC_IRQAFF */
-
-/*
- * IPI model for SMTC is tricky, because interrupts aren't TC-specific.
- * Within a VPE one TC can interrupt another by different approaches.
- * The easiest to get right would probably be to make all TCs except
- * the target IXMT and set a software interrupt, but an IXMT-based
- * scheme requires that a handler must run before a new IPI could
- * be sent, which would break the "broadcast" loops in MIPS MT.
- * A more gonzo approach within a VPE is to halt the TC, extract
- * its Restart, Status, and a couple of GPRs, and program the Restart
- * address to emulate an interrupt.
- *
- * Within a VPE, one can be confident that the target TC isn't in
- * a critical EXL state when halted, since the write to the Halt
- * register could not have issued on the writing thread if the
- * halting thread had EXL set. So k0 and k1 of the target TC
- * can be used by the injection code. Across VPEs, one can't
- * be certain that the target TC isn't in a critical exception
- * state. So we try a two-step process of sending a software
- * interrupt to the target VPE, which either handles the event
- * itself (if it was the target) or injects the event within
- * the VPE.
- */
-
-static void smtc_ipi_qdump(void)
-{
- int i;
- struct smtc_ipi *temp;
-
- for (i = 0; i < NR_CPUS ;i++) {
- pr_info("IPIQ[%d]: head = 0x%x, tail = 0x%x, depth = %d\n",
- i, (unsigned)IPIQ[i].head, (unsigned)IPIQ[i].tail,
- IPIQ[i].depth);
- temp = IPIQ[i].head;
-
- while (temp != IPIQ[i].tail) {
- pr_debug("%d %d %d: ", temp->type, temp->dest,
- (int)temp->arg);
-#ifdef SMTC_IPI_DEBUG
- pr_debug("%u %lu\n", temp->sender, temp->stamp);
-#else
- pr_debug("\n");
-#endif
- temp = temp->flink;
- }
- }
-}
-
-/*
- * The standard atomic.h primitives don't quite do what we want
- * here: We need an atomic add-and-return-previous-value (which
- * could be done with atomic_add_return and a decrement) and an
- * atomic set/zero-and-return-previous-value (which can't really
- * be done with the atomic.h primitives). And since this is
- * MIPS MT, we can assume that we have LL/SC.
- */
-static inline int atomic_postincrement(atomic_t *v)
-{
- unsigned long result;
-
- unsigned long temp;
-
- __asm__ __volatile__(
- "1: ll %0, %2 \n"
- " addu %1, %0, 1 \n"
- " sc %1, %2 \n"
- " beqz %1, 1b \n"
- __WEAK_LLSC_MB
- : "=&r" (result), "=&r" (temp), "=m" (v->counter)
- : "m" (v->counter)
- : "memory");
-
- return result;
-}
-
-void smtc_send_ipi(int cpu, int type, unsigned int action)
-{
- int tcstatus;
- struct smtc_ipi *pipi;
- unsigned long flags;
- int mtflags;
- unsigned long tcrestart;
- int set_resched_flag = (type == LINUX_SMP_IPI &&
- action == SMP_RESCHEDULE_YOURSELF);
-
- if (cpu == smp_processor_id()) {
- printk("Cannot Send IPI to self!\n");
- return;
- }
- if (set_resched_flag && IPIQ[cpu].resched_flag != 0)
- return; /* There is a reschedule queued already */
-
- /* Set up a descriptor, to be delivered either promptly or queued */
- pipi = smtc_ipi_dq(&freeIPIq);
- if (pipi == NULL) {
- bust_spinlocks(1);
- mips_mt_regdump(dvpe());
- panic("IPI Msg. Buffers Depleted");
- }
- pipi->type = type;
- pipi->arg = (void *)action;
- pipi->dest = cpu;
- if (cpu_data[cpu].vpe_id != cpu_data[smp_processor_id()].vpe_id) {
- /* If not on same VPE, enqueue and send cross-VPE interrupt */
- IPIQ[cpu].resched_flag |= set_resched_flag;
- smtc_ipi_nq(&IPIQ[cpu], pipi);
- LOCK_CORE_PRA();
- settc(cpu_data[cpu].tc_id);
- write_vpe_c0_cause(read_vpe_c0_cause() | C_SW1);
- UNLOCK_CORE_PRA();
- } else {
- /*
- * Not sufficient to do a LOCK_MT_PRA (dmt) here,
- * since ASID shootdown on the other VPE may
- * collide with this operation.
- */
- LOCK_CORE_PRA();
- settc(cpu_data[cpu].tc_id);
- /* Halt the targeted TC */
- write_tc_c0_tchalt(TCHALT_H);
- mips_ihb();
-
- /*
- * Inspect TCStatus - if IXMT is set, we have to queue
- * a message. Otherwise, we set up the "interrupt"
- * of the other TC
- */
- tcstatus = read_tc_c0_tcstatus();
-
- if ((tcstatus & TCSTATUS_IXMT) != 0) {
- /*
- * If we're in the the irq-off version of the wait
- * loop, we need to force exit from the wait and
- * do a direct post of the IPI.
- */
- if (cpu_wait == r4k_wait_irqoff) {
- tcrestart = read_tc_c0_tcrestart();
- if (address_is_in_r4k_wait_irqoff(tcrestart)) {
- write_tc_c0_tcrestart(__pastwait);
- tcstatus &= ~TCSTATUS_IXMT;
- write_tc_c0_tcstatus(tcstatus);
- goto postdirect;
- }
- }
- /*
- * Otherwise we queue the message for the target TC
- * to pick up when he does a local_irq_restore()
- */
- write_tc_c0_tchalt(0);
- UNLOCK_CORE_PRA();
- IPIQ[cpu].resched_flag |= set_resched_flag;
- smtc_ipi_nq(&IPIQ[cpu], pipi);
- } else {
-postdirect:
- post_direct_ipi(cpu, pipi);
- write_tc_c0_tchalt(0);
- UNLOCK_CORE_PRA();
- }
- }
-}
-
-/*
- * Send IPI message to Halted TC, TargTC/TargVPE already having been set
- */
-static void post_direct_ipi(int cpu, struct smtc_ipi *pipi)
-{
- struct pt_regs *kstack;
- unsigned long tcstatus;
- unsigned long tcrestart;
- extern u32 kernelsp[NR_CPUS];
- extern void __smtc_ipi_vector(void);
-//printk("%s: on %d for %d\n", __func__, smp_processor_id(), cpu);
-
- /* Extract Status, EPC from halted TC */
- tcstatus = read_tc_c0_tcstatus();
- tcrestart = read_tc_c0_tcrestart();
- /* If TCRestart indicates a WAIT instruction, advance the PC */
- if ((tcrestart & 0x80000000)
- && ((*(unsigned int *)tcrestart & 0xfe00003f) == 0x42000020)) {
- tcrestart += 4;
- }
- /*
- * Save on TC's future kernel stack
- *
- * CU bit of Status is indicator that TC was
- * already running on a kernel stack...
- */
- if (tcstatus & ST0_CU0) {
- /* Note that this "- 1" is pointer arithmetic */
- kstack = ((struct pt_regs *)read_tc_gpr_sp()) - 1;
- } else {
- kstack = ((struct pt_regs *)kernelsp[cpu]) - 1;
- }
-
- kstack->cp0_epc = (long)tcrestart;
- /* Save TCStatus */
- kstack->cp0_tcstatus = tcstatus;
- /* Pass token of operation to be performed kernel stack pad area */
- kstack->pad0[4] = (unsigned long)pipi;
- /* Pass address of function to be called likewise */
- kstack->pad0[5] = (unsigned long)&ipi_decode;
- /* Set interrupt exempt and kernel mode */
- tcstatus |= TCSTATUS_IXMT;
- tcstatus &= ~TCSTATUS_TKSU;
- write_tc_c0_tcstatus(tcstatus);
- ehb();
- /* Set TC Restart address to be SMTC IPI vector */
- write_tc_c0_tcrestart(__smtc_ipi_vector);
-}
-
-static void ipi_resched_interrupt(void)
-{
- scheduler_ipi();
-}
-
-static void ipi_call_interrupt(void)
-{
- /* Invoke generic function invocation code in smp.c */
- smp_call_function_interrupt();
-}
-
-DECLARE_PER_CPU(struct clock_event_device, mips_clockevent_device);
-
-static void __irq_entry smtc_clock_tick_interrupt(void)
-{
- unsigned int cpu = smp_processor_id();
- struct clock_event_device *cd;
- int irq = MIPS_CPU_IRQ_BASE + 1;
-
- irq_enter();
- kstat_incr_irq_this_cpu(irq);
- cd = &per_cpu(mips_clockevent_device, cpu);
- cd->event_handler(cd);
- irq_exit();
-}
-
-void ipi_decode(struct smtc_ipi *pipi)
-{
- void *arg_copy = pipi->arg;
- int type_copy = pipi->type;
-
- smtc_ipi_nq(&freeIPIq, pipi);
-
- switch (type_copy) {
- case SMTC_CLOCK_TICK:
- smtc_clock_tick_interrupt();
- break;
-
- case LINUX_SMP_IPI:
- switch ((int)arg_copy) {
- case SMP_RESCHEDULE_YOURSELF:
- ipi_resched_interrupt();
- break;
- case SMP_CALL_FUNCTION:
- ipi_call_interrupt();
- break;
- default:
- printk("Impossible SMTC IPI Argument %p\n", arg_copy);
- break;
- }
- break;
-#ifdef CONFIG_MIPS_MT_SMTC_IRQAFF
- case IRQ_AFFINITY_IPI:
- /*
- * Accept a "forwarded" interrupt that was initially
- * taken by a TC who doesn't have affinity for the IRQ.
- */
- do_IRQ_no_affinity((int)arg_copy);
- break;
-#endif /* CONFIG_MIPS_MT_SMTC_IRQAFF */
- default:
- printk("Impossible SMTC IPI Type 0x%x\n", type_copy);
- break;
- }
-}
-
-/*
- * Similar to smtc_ipi_replay(), but invoked from context restore,
- * so it reuses the current exception frame rather than set up a
- * new one with self_ipi.
- */
-
-void deferred_smtc_ipi(void)
-{
- int cpu = smp_processor_id();
-
- /*
- * Test is not atomic, but much faster than a dequeue,
- * and the vast majority of invocations will have a null queue.
- * If irq_disabled when this was called, then any IPIs queued
- * after we test last will be taken on the next irq_enable/restore.
- * If interrupts were enabled, then any IPIs added after the
- * last test will be taken directly.
- */
-
- while (IPIQ[cpu].head != NULL) {
- struct smtc_ipi_q *q = &IPIQ[cpu];
- struct smtc_ipi *pipi;
- unsigned long flags;
-
- /*
- * It may be possible we'll come in with interrupts
- * already enabled.
- */
- local_irq_save(flags);
- spin_lock(&q->lock);
- pipi = __smtc_ipi_dq(q);
- spin_unlock(&q->lock);
- if (pipi != NULL) {
- if (pipi->type == LINUX_SMP_IPI &&
- (int)pipi->arg == SMP_RESCHEDULE_YOURSELF)
- IPIQ[cpu].resched_flag = 0;
- ipi_decode(pipi);
- }
- /*
- * The use of the __raw_local restore isn't
- * as obviously necessary here as in smtc_ipi_replay(),
- * but it's more efficient, given that we're already
- * running down the IPI queue.
- */
- __arch_local_irq_restore(flags);
- }
-}
-
-/*
- * Cross-VPE interrupts in the SMTC prototype use "software interrupts"
- * set via cross-VPE MTTR manipulation of the Cause register. It would be
- * in some regards preferable to have external logic for "doorbell" hardware
- * interrupts.
- */
-
-static int cpu_ipi_irq = MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_IRQ;
-
-static irqreturn_t ipi_interrupt(int irq, void *dev_idm)
-{
- int my_vpe = cpu_data[smp_processor_id()].vpe_id;
- int my_tc = cpu_data[smp_processor_id()].tc_id;
- int cpu;
- struct smtc_ipi *pipi;
- unsigned long tcstatus;
- int sent;
- unsigned long flags;
- unsigned int mtflags;
- unsigned int vpflags;
-
- /*
- * So long as cross-VPE interrupts are done via
- * MFTR/MTTR read-modify-writes of Cause, we need
- * to stop other VPEs whenever the local VPE does
- * anything similar.
- */
- local_irq_save(flags);
- vpflags = dvpe();
- clear_c0_cause(0x100 << MIPS_CPU_IPI_IRQ);
- set_c0_status(0x100 << MIPS_CPU_IPI_IRQ);
- irq_enable_hazard();
- evpe(vpflags);
- local_irq_restore(flags);
-
- /*
- * Cross-VPE Interrupt handler: Try to directly deliver IPIs
- * queued for TCs on this VPE other than the current one.
- * Return-from-interrupt should cause us to drain the queue
- * for the current TC, so we ought not to have to do it explicitly here.
- */
-
- for_each_online_cpu(cpu) {
- if (cpu_data[cpu].vpe_id != my_vpe)
- continue;
-
- pipi = smtc_ipi_dq(&IPIQ[cpu]);
- if (pipi != NULL) {
- if (cpu_data[cpu].tc_id != my_tc) {
- sent = 0;
- LOCK_MT_PRA();
- settc(cpu_data[cpu].tc_id);
- write_tc_c0_tchalt(TCHALT_H);
- mips_ihb();
- tcstatus = read_tc_c0_tcstatus();
- if ((tcstatus & TCSTATUS_IXMT) == 0) {
- post_direct_ipi(cpu, pipi);
- sent = 1;
- }
- write_tc_c0_tchalt(0);
- UNLOCK_MT_PRA();
- if (!sent) {
- smtc_ipi_req(&IPIQ[cpu], pipi);
- }
- } else {
- /*
- * ipi_decode() should be called
- * with interrupts off
- */
- local_irq_save(flags);
- if (pipi->type == LINUX_SMP_IPI &&
- (int)pipi->arg == SMP_RESCHEDULE_YOURSELF)
- IPIQ[cpu].resched_flag = 0;
- ipi_decode(pipi);
- local_irq_restore(flags);
- }
- }
- }
-
- return IRQ_HANDLED;
-}
-
-static void ipi_irq_dispatch(void)
-{
- do_IRQ(cpu_ipi_irq);
-}
-
-static struct irqaction irq_ipi = {
- .handler = ipi_interrupt,
- .flags = IRQF_PERCPU,
- .name = "SMTC_IPI"
-};
-
-static void setup_cross_vpe_interrupts(unsigned int nvpe)
-{
- if (nvpe < 1)
- return;
-
- if (!cpu_has_vint)
- panic("SMTC Kernel requires Vectored Interrupt support");
-
- set_vi_handler(MIPS_CPU_IPI_IRQ, ipi_irq_dispatch);
-
- setup_irq_smtc(cpu_ipi_irq, &irq_ipi, (0x100 << MIPS_CPU_IPI_IRQ));
-
- irq_set_handler(cpu_ipi_irq, handle_percpu_irq);
-}
-
-/*
- * SMTC-specific hacks invoked from elsewhere in the kernel.
- */
-
- /*
- * smtc_ipi_replay is called from raw_local_irq_restore
- */
-
-void smtc_ipi_replay(void)
-{
- unsigned int cpu = smp_processor_id();
-
- /*
- * To the extent that we've ever turned interrupts off,
- * we may have accumulated deferred IPIs. This is subtle.
- * we should be OK: If we pick up something and dispatch
- * it here, that's great. If we see nothing, but concurrent
- * with this operation, another TC sends us an IPI, IXMT
- * is clear, and we'll handle it as a real pseudo-interrupt
- * and not a pseudo-pseudo interrupt. The important thing
- * is to do the last check for queued message *after* the
- * re-enabling of interrupts.
- */
- while (IPIQ[cpu].head != NULL) {
- struct smtc_ipi_q *q = &IPIQ[cpu];
- struct smtc_ipi *pipi;
- unsigned long flags;
-
- /*
- * It's just possible we'll come in with interrupts
- * already enabled.
- */
- local_irq_save(flags);
-
- spin_lock(&q->lock);
- pipi = __smtc_ipi_dq(q);
- spin_unlock(&q->lock);
- /*
- ** But use a raw restore here to avoid recursion.
- */
- __arch_local_irq_restore(flags);
-
- if (pipi) {
- self_ipi(pipi);
- smtc_cpu_stats[cpu].selfipis++;
- }
- }
-}
-
-EXPORT_SYMBOL(smtc_ipi_replay);
-
-void smtc_idle_loop_hook(void)
-{
-#ifdef CONFIG_SMTC_IDLE_HOOK_DEBUG
- int im;
- int flags;
- int mtflags;
- int bit;
- int vpe;
- int tc;
- int hook_ntcs;
- /*
- * printk within DMT-protected regions can deadlock,
- * so buffer diagnostic messages for later output.
- */
- char *pdb_msg;
- char id_ho_db_msg[768]; /* worst-case use should be less than 700 */
-
- if (atomic_read(&idle_hook_initialized) == 0) { /* fast test */
- if (atomic_add_return(1, &idle_hook_initialized) == 1) {
- int mvpconf0;
- /* Tedious stuff to just do once */
- mvpconf0 = read_c0_mvpconf0();
- hook_ntcs = ((mvpconf0 & MVPCONF0_PTC) >> MVPCONF0_PTC_SHIFT) + 1;
- if (hook_ntcs > NR_CPUS)
- hook_ntcs = NR_CPUS;
- for (tc = 0; tc < hook_ntcs; tc++) {
- tcnoprog[tc] = 0;
- clock_hang_reported[tc] = 0;
- }
- for (vpe = 0; vpe < 2; vpe++)
- for (im = 0; im < 8; im++)
- imstuckcount[vpe][im] = 0;
- printk("Idle loop test hook initialized for %d TCs\n", hook_ntcs);
- atomic_set(&idle_hook_initialized, 1000);
- } else {
- /* Someone else is initializing in parallel - let 'em finish */
- while (atomic_read(&idle_hook_initialized) < 1000)
- ;
- }
- }
-
- /* Have we stupidly left IXMT set somewhere? */
- if (read_c0_tcstatus() & 0x400) {
- write_c0_tcstatus(read_c0_tcstatus() & ~0x400);
- ehb();
- printk("Dangling IXMT in cpu_idle()\n");
- }
-
- /* Have we stupidly left an IM bit turned off? */
-#define IM_LIMIT 2000
- local_irq_save(flags);
- mtflags = dmt();
- pdb_msg = &id_ho_db_msg[0];
- im = read_c0_status();
- vpe = current_cpu_data.vpe_id;
- for (bit = 0; bit < 8; bit++) {
- /*
- * In current prototype, I/O interrupts
- * are masked for VPE > 0
- */
- if (vpemask[vpe][bit]) {
- if (!(im & (0x100 << bit)))
- imstuckcount[vpe][bit]++;
- else
- imstuckcount[vpe][bit] = 0;
- if (imstuckcount[vpe][bit] > IM_LIMIT) {
- set_c0_status(0x100 << bit);
- ehb();
- imstuckcount[vpe][bit] = 0;
- pdb_msg += sprintf(pdb_msg,
- "Dangling IM %d fixed for VPE %d\n", bit,
- vpe);
- }
- }
- }
-
- emt(mtflags);
- local_irq_restore(flags);
- if (pdb_msg != &id_ho_db_msg[0])
- printk("CPU%d: %s", smp_processor_id(), id_ho_db_msg);
-#endif /* CONFIG_SMTC_IDLE_HOOK_DEBUG */
-
- smtc_ipi_replay();
-}
-
-void smtc_soft_dump(void)
-{
- int i;
-
- printk("Counter Interrupts taken per CPU (TC)\n");
- for (i=0; i < NR_CPUS; i++) {
- printk("%d: %ld\n", i, smtc_cpu_stats[i].timerints);
- }
- printk("Self-IPI invocations:\n");
- for (i=0; i < NR_CPUS; i++) {
- printk("%d: %ld\n", i, smtc_cpu_stats[i].selfipis);
- }
- smtc_ipi_qdump();
- printk("%d Recoveries of \"stolen\" FPU\n",
- atomic_read(&smtc_fpu_recoveries));
-}
-
-
-/*
- * TLB management routines special to SMTC
- */
-
-void smtc_get_new_mmu_context(struct mm_struct *mm, unsigned long cpu)
-{
- unsigned long flags, mtflags, tcstat, prevhalt, asid;
- int tlb, i;
-
- /*
- * It would be nice to be able to use a spinlock here,
- * but this is invoked from within TLB flush routines
- * that protect themselves with DVPE, so if a lock is
- * held by another TC, it'll never be freed.
- *
- * DVPE/DMT must not be done with interrupts enabled,
- * so even so most callers will already have disabled
- * them, let's be really careful...
- */
-
- local_irq_save(flags);
- if (smtc_status & SMTC_TLB_SHARED) {
- mtflags = dvpe();
- tlb = 0;
- } else {
- mtflags = dmt();
- tlb = cpu_data[cpu].vpe_id;
- }
- asid = asid_cache(cpu);
-
- do {
- if (!((asid += ASID_INC) & ASID_MASK) ) {
- if (cpu_has_vtag_icache)
- flush_icache_all();
- /* Traverse all online CPUs (hack requires contiguous range) */
- for_each_online_cpu(i) {
- /*
- * We don't need to worry about our own CPU, nor those of
- * CPUs who don't share our TLB.
- */
- if ((i != smp_processor_id()) &&
- ((smtc_status & SMTC_TLB_SHARED) ||
- (cpu_data[i].vpe_id == cpu_data[cpu].vpe_id))) {
- settc(cpu_data[i].tc_id);
- prevhalt = read_tc_c0_tchalt() & TCHALT_H;
- if (!prevhalt) {
- write_tc_c0_tchalt(TCHALT_H);
- mips_ihb();
- }
- tcstat = read_tc_c0_tcstatus();
- smtc_live_asid[tlb][(tcstat & ASID_MASK)] |= (asiduse)(0x1 << i);
- if (!prevhalt)
- write_tc_c0_tchalt(0);
- }
- }
- if (!asid) /* fix version if needed */
- asid = ASID_FIRST_VERSION;
- local_flush_tlb_all(); /* start new asid cycle */
- }
- } while (smtc_live_asid[tlb][(asid & ASID_MASK)]);
-
- /*
- * SMTC shares the TLB within VPEs and possibly across all VPEs.
- */
- for_each_online_cpu(i) {
- if ((smtc_status & SMTC_TLB_SHARED) ||
- (cpu_data[i].vpe_id == cpu_data[cpu].vpe_id))
- cpu_context(i, mm) = asid_cache(i) = asid;
- }
-
- if (smtc_status & SMTC_TLB_SHARED)
- evpe(mtflags);
- else
- emt(mtflags);
- local_irq_restore(flags);
-}
-
-/*
- * Invoked from macros defined in mmu_context.h
- * which must already have disabled interrupts
- * and done a DVPE or DMT as appropriate.
- */
-
-void smtc_flush_tlb_asid(unsigned long asid)
-{
- int entry;
- unsigned long ehi;
-
- entry = read_c0_wired();
-
- /* Traverse all non-wired entries */
- while (entry < current_cpu_data.tlbsize) {
- write_c0_index(entry);
- ehb();
- tlb_read();
- ehb();
- ehi = read_c0_entryhi();
- if ((ehi & ASID_MASK) == asid) {
- /*
- * Invalidate only entries with specified ASID,
- * makiing sure all entries differ.
- */
- write_c0_entryhi(CKSEG0 + (entry << (PAGE_SHIFT + 1)));
- write_c0_entrylo0(0);
- write_c0_entrylo1(0);
- mtc0_tlbw_hazard();
- tlb_write_indexed();
- }
- entry++;
- }
- write_c0_index(PARKED_INDEX);
- tlbw_use_hazard();
-}
-
-/*
- * Support for single-threading cache flush operations.
- */
-
-static int halt_state_save[NR_CPUS];
-
-/*
- * To really, really be sure that nothing is being done
- * by other TCs, halt them all. This code assumes that
- * a DVPE has already been done, so while their Halted
- * state is theoretically architecturally unstable, in
- * practice, it's not going to change while we're looking
- * at it.
- */
-
-void smtc_cflush_lockdown(void)
-{
- int cpu;
-
- for_each_online_cpu(cpu) {
- if (cpu != smp_processor_id()) {
- settc(cpu_data[cpu].tc_id);
- halt_state_save[cpu] = read_tc_c0_tchalt();
- write_tc_c0_tchalt(TCHALT_H);
- }
- }
- mips_ihb();
-}
-
-/* It would be cheating to change the cpu_online states during a flush! */
-
-void smtc_cflush_release(void)
-{
- int cpu;
-
- /*
- * Start with a hazard barrier to ensure
- * that all CACHE ops have played through.
- */
- mips_ihb();
-
- for_each_online_cpu(cpu) {
- if (cpu != smp_processor_id()) {
- settc(cpu_data[cpu].tc_id);
- write_tc_c0_tchalt(halt_state_save[cpu]);
- }
- }
- mips_ihb();
-}
diff --git a/arch/mips/kernel/sync-r4k.c b/arch/mips/kernel/sync-r4k.c
index c24ad5f..2242bdd 100644
--- a/arch/mips/kernel/sync-r4k.c
+++ b/arch/mips/kernel/sync-r4k.c
@@ -6,8 +6,6 @@
* not have done anything significant (but they may have had interrupts
* enabled briefly - prom_smp_finish() should not be responsible for enabling
* interrupts...)
- *
- * FIXME: broken for SMTC
*/
#include <linux/kernel.h>
@@ -33,14 +31,6 @@
unsigned long flags;
unsigned int initcount;
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * SMTC needs to synchronise per VPE, not per CPU
- * ignore for now
- */
- return;
-#endif
-
printk(KERN_INFO "Synchronize counters for CPU %u: ", cpu);
local_irq_save(flags);
@@ -110,14 +100,6 @@
int i;
unsigned int initcount;
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * SMTC needs to synchronise per VPE, not per CPU
- * ignore for now
- */
- return;
-#endif
-
/*
* Not every cpu is online at the time this gets called,
* so we first wait for the master to say everyone is ready
diff --git a/arch/mips/kernel/time.c b/arch/mips/kernel/time.c
index dcb8e5d..8d01709 100644
--- a/arch/mips/kernel/time.c
+++ b/arch/mips/kernel/time.c
@@ -26,7 +26,6 @@
#include <asm/cpu-features.h>
#include <asm/cpu-type.h>
#include <asm/div64.h>
-#include <asm/smtc_ipi.h>
#include <asm/time.h>
/*
diff --git a/arch/mips/kernel/traps.c b/arch/mips/kernel/traps.c
index 8119ac2..51706d6 100644
--- a/arch/mips/kernel/traps.c
+++ b/arch/mips/kernel/traps.c
@@ -15,6 +15,7 @@
#include <linux/bug.h>
#include <linux/compiler.h>
#include <linux/context_tracking.h>
+#include <linux/cpu_pm.h>
#include <linux/kexec.h>
#include <linux/init.h>
#include <linux/kernel.h>
@@ -370,9 +371,6 @@
{
static int die_counter;
int sig = SIGSEGV;
-#ifdef CONFIG_MIPS_MT_SMTC
- unsigned long dvpret;
-#endif /* CONFIG_MIPS_MT_SMTC */
oops_enter();
@@ -382,13 +380,7 @@
console_verbose();
raw_spin_lock_irq(&die_lock);
-#ifdef CONFIG_MIPS_MT_SMTC
- dvpret = dvpe();
-#endif /* CONFIG_MIPS_MT_SMTC */
bust_spinlocks(1);
-#ifdef CONFIG_MIPS_MT_SMTC
- mips_mt_regdump(dvpret);
-#endif /* CONFIG_MIPS_MT_SMTC */
printk("%s[#%d]:\n", str, ++die_counter);
show_registers(regs);
@@ -712,10 +704,12 @@
si.si_addr = fault_addr;
si.si_signo = sig;
if (sig == SIGSEGV) {
+ down_read(¤t->mm->mmap_sem);
if (find_vma(current->mm, (unsigned long)fault_addr))
si.si_code = SEGV_ACCERR;
else
si.si_code = SEGV_MAPERR;
+ up_read(¤t->mm->mmap_sem);
} else {
si.si_code = BUS_ADRERR;
}
@@ -1759,19 +1753,6 @@
extern char rollback_except_vec_vi;
char *vec_start = using_rollback_handler() ?
&rollback_except_vec_vi : &except_vec_vi;
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * We need to provide the SMTC vectored interrupt handler
- * not only with the address of the handler, but with the
- * Status.IM bit to be masked before going there.
- */
- extern char except_vec_vi_mori;
-#if defined(CONFIG_CPU_MICROMIPS) || defined(CONFIG_CPU_BIG_ENDIAN)
- const int mori_offset = &except_vec_vi_mori - vec_start + 2;
-#else
- const int mori_offset = &except_vec_vi_mori - vec_start;
-#endif
-#endif /* CONFIG_MIPS_MT_SMTC */
#if defined(CONFIG_CPU_MICROMIPS) || defined(CONFIG_CPU_BIG_ENDIAN)
const int lui_offset = &except_vec_vi_lui - vec_start + 2;
const int ori_offset = &except_vec_vi_ori - vec_start + 2;
@@ -1795,12 +1776,6 @@
#else
handler_len);
#endif
-#ifdef CONFIG_MIPS_MT_SMTC
- BUG_ON(n > 7); /* Vector index %d exceeds SMTC maximum. */
-
- h = (u16 *)(b + mori_offset);
- *h = (0x100 << n);
-#endif /* CONFIG_MIPS_MT_SMTC */
h = (u16 *)(b + lui_offset);
*h = (handler >> 16) & 0xffff;
h = (u16 *)(b + ori_offset);
@@ -1865,32 +1840,16 @@
}
__setup("noulri", ulri_disable);
-void per_cpu_trap_init(bool is_boot_cpu)
+/* configure STATUS register */
+static void configure_status(void)
{
- unsigned int cpu = smp_processor_id();
- unsigned int status_set = ST0_CU0;
- unsigned int hwrena = cpu_hwrena_impl_bits;
-#ifdef CONFIG_MIPS_MT_SMTC
- int secondaryTC = 0;
- int bootTC = (cpu == 0);
-
- /*
- * Only do per_cpu_trap_init() for first TC of Each VPE.
- * Note that this hack assumes that the SMTC init code
- * assigns TCs consecutively and in ascending order.
- */
-
- if (((read_c0_tcbind() & TCBIND_CURTC) != 0) &&
- ((read_c0_tcbind() & TCBIND_CURVPE) == cpu_data[cpu - 1].vpe_id))
- secondaryTC = 1;
-#endif /* CONFIG_MIPS_MT_SMTC */
-
/*
* Disable coprocessors and select 32-bit or 64-bit addressing
* and the 16/32 or 32/32 FPR register model. Reset the BEV
* flag that some firmware may have left set and the TS bit (for
* IP27). Set XX for ISA IV code to work.
*/
+ unsigned int status_set = ST0_CU0;
#ifdef CONFIG_64BIT
status_set |= ST0_FR|ST0_KX|ST0_SX|ST0_UX;
#endif
@@ -1901,6 +1860,12 @@
change_c0_status(ST0_CU|ST0_MX|ST0_RE|ST0_FR|ST0_BEV|ST0_TS|ST0_KX|ST0_SX|ST0_UX,
status_set);
+}
+
+/* configure HWRENA register */
+static void configure_hwrena(void)
+{
+ unsigned int hwrena = cpu_hwrena_impl_bits;
if (cpu_has_mips_r2)
hwrena |= 0x0000000f;
@@ -1910,11 +1875,10 @@
if (hwrena)
write_c0_hwrena(hwrena);
+}
-#ifdef CONFIG_MIPS_MT_SMTC
- if (!secondaryTC) {
-#endif /* CONFIG_MIPS_MT_SMTC */
-
+static void configure_exception_vector(void)
+{
if (cpu_has_veic || cpu_has_vint) {
unsigned long sr = set_c0_status(ST0_BEV);
write_c0_ebase(ebase);
@@ -1930,6 +1894,16 @@
} else
set_c0_cause(CAUSEF_IV);
}
+}
+
+void per_cpu_trap_init(bool is_boot_cpu)
+{
+ unsigned int cpu = smp_processor_id();
+
+ configure_status();
+ configure_hwrena();
+
+ configure_exception_vector();
/*
* Before R2 both interrupt numbers were fixed to 7, so on R2 only:
@@ -1949,10 +1923,6 @@
cp0_perfcount_irq = -1;
}
-#ifdef CONFIG_MIPS_MT_SMTC
- }
-#endif /* CONFIG_MIPS_MT_SMTC */
-
if (!cpu_data[cpu].asid_cache)
cpu_data[cpu].asid_cache = ASID_FIRST_VERSION;
@@ -1961,23 +1931,10 @@
BUG_ON(current->mm);
enter_lazy_tlb(&init_mm, current);
-#ifdef CONFIG_MIPS_MT_SMTC
- if (bootTC) {
-#endif /* CONFIG_MIPS_MT_SMTC */
/* Boot CPU's cache setup in setup_arch(). */
if (!is_boot_cpu)
cpu_cache_init();
tlb_init();
-#ifdef CONFIG_MIPS_MT_SMTC
- } else if (!secondaryTC) {
- /*
- * First TC in non-boot VPE must do subset of tlb_init()
- * for MMU countrol registers.
- */
- write_c0_pagemask(PM_DEFAULT_MASK);
- write_c0_wired(0);
- }
-#endif /* CONFIG_MIPS_MT_SMTC */
TLBMISS_HANDLER_SETUP();
}
@@ -2185,3 +2142,32 @@
cu2_notifier(default_cu2_call, 0x80000000); /* Run last */
}
+
+static int trap_pm_notifier(struct notifier_block *self, unsigned long cmd,
+ void *v)
+{
+ switch (cmd) {
+ case CPU_PM_ENTER_FAILED:
+ case CPU_PM_EXIT:
+ configure_status();
+ configure_hwrena();
+ configure_exception_vector();
+
+ /* Restore register with CPU number for TLB handlers */
+ TLBMISS_HANDLER_RESTORE();
+
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block trap_pm_notifier_block = {
+ .notifier_call = trap_pm_notifier,
+};
+
+static int __init trap_pm_init(void)
+{
+ return cpu_pm_register_notifier(&trap_pm_notifier_block);
+}
+arch_initcall(trap_pm_init);
diff --git a/arch/mips/kernel/vpe-mt.c b/arch/mips/kernel/vpe-mt.c
index 949ae0e..2e003b1 100644
--- a/arch/mips/kernel/vpe-mt.c
+++ b/arch/mips/kernel/vpe-mt.c
@@ -127,9 +127,8 @@
clear_c0_mvpcontrol(MVPCONTROL_VPC);
/*
- * SMTC/SMVP kernels manage VPE enable independently,
- * but uniprocessor kernels need to turn it on, even
- * if that wasn't the pre-dvpe() state.
+ * SMVP kernels manage VPE enable independently, but uniprocessor
+ * kernels need to turn it on, even if that wasn't the pre-dvpe() state.
*/
#ifdef CONFIG_SMP
evpe(vpeflags);
@@ -454,12 +453,11 @@
settc(tc);
- /* Any TC that is bound to VPE0 gets left as is - in
- * case we are running SMTC on VPE0. A TC that is bound
- * to any other VPE gets bound to VPE0, ideally I'd like
- * to make it homeless but it doesn't appear to let me
- * bind a TC to a non-existent VPE. Which is perfectly
- * reasonable.
+ /*
+ * A TC that is bound to any other VPE gets bound to
+ * VPE0, ideally I'd like to make it homeless but it
+ * doesn't appear to let me bind a TC to a non-existent
+ * VPE. Which is perfectly reasonable.
*
* The (un)bound state is visible to an EJTAG probe so
* may notify GDB...
diff --git a/arch/mips/lantiq/irq.c b/arch/mips/lantiq/irq.c
index 85685e1..030568a 100644
--- a/arch/mips/lantiq/irq.c
+++ b/arch/mips/lantiq/irq.c
@@ -61,7 +61,7 @@
/* we have a cascade of 8 irqs */
#define MIPS_CPU_IRQ_CASCADE 8
-#if defined(CONFIG_MIPS_MT_SMP) || defined(CONFIG_MIPS_MT_SMTC)
+#ifdef CONFIG_MIPS_MT_SMP
int gic_present;
#endif
@@ -440,7 +440,7 @@
arch_init_ipiirq(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_CALL_IRQ, &irq_call);
#endif
-#if !defined(CONFIG_MIPS_MT_SMP) && !defined(CONFIG_MIPS_MT_SMTC)
+#ifndef CONFIG_MIPS_MT_SMP
set_c0_status(IE_IRQ0 | IE_IRQ1 | IE_IRQ2 |
IE_IRQ3 | IE_IRQ4 | IE_IRQ5);
#else
diff --git a/arch/mips/lib/delay.c b/arch/mips/lib/delay.c
index 705cfb7..21d27c6 100644
--- a/arch/mips/lib/delay.c
+++ b/arch/mips/lib/delay.c
@@ -11,7 +11,9 @@
#include <linux/module.h>
#include <linux/param.h>
#include <linux/smp.h>
+#include <linux/stringify.h>
+#include <asm/asm.h>
#include <asm/compiler.h>
#include <asm/war.h>
@@ -27,11 +29,7 @@
" .set noreorder \n"
" .align 3 \n"
"1: bnez %0, 1b \n"
-#if BITS_PER_LONG == 32
- " subu %0, %1 \n"
-#else
- " dsubu %0, %1 \n"
-#endif
+ " " __stringify(LONG_SUBU) " %0, %1 \n"
" .set reorder \n"
: "=r" (loops)
: GCC_DADDI_IMM_ASM() (1), "0" (loops));
diff --git a/arch/mips/lib/mips-atomic.c b/arch/mips/lib/mips-atomic.c
index 6807f71..57bcdaf1 100644
--- a/arch/mips/lib/mips-atomic.c
+++ b/arch/mips/lib/mips-atomic.c
@@ -15,7 +15,7 @@
#include <linux/export.h>
#include <linux/stringify.h>
-#if !defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_MIPS_MT_SMTC)
+#ifndef CONFIG_CPU_MIPSR2
/*
* For cli() we have to insert nops to make sure that the new value
@@ -42,12 +42,7 @@
__asm__ __volatile__(
" .set push \n"
" .set noat \n"
-#ifdef CONFIG_MIPS_MT_SMTC
- " mfc0 $1, $2, 1 \n"
- " ori $1, 0x400 \n"
- " .set noreorder \n"
- " mtc0 $1, $2, 1 \n"
-#elif defined(CONFIG_CPU_MIPSR2)
+#if defined(CONFIG_CPU_MIPSR2)
/* see irqflags.h for inline function */
#else
" mfc0 $1,$12 \n"
@@ -77,13 +72,7 @@
" .set push \n"
" .set reorder \n"
" .set noat \n"
-#ifdef CONFIG_MIPS_MT_SMTC
- " mfc0 %[flags], $2, 1 \n"
- " ori $1, %[flags], 0x400 \n"
- " .set noreorder \n"
- " mtc0 $1, $2, 1 \n"
- " andi %[flags], %[flags], 0x400 \n"
-#elif defined(CONFIG_CPU_MIPSR2)
+#if defined(CONFIG_CPU_MIPSR2)
/* see irqflags.h for inline function */
#else
" mfc0 %[flags], $12 \n"
@@ -108,29 +97,13 @@
{
unsigned long __tmp1;
-#ifdef CONFIG_MIPS_MT_SMTC
- /*
- * SMTC kernel needs to do a software replay of queued
- * IPIs, at the cost of branch and call overhead on each
- * local_irq_restore()
- */
- if (unlikely(!(flags & 0x0400)))
- smtc_ipi_replay();
-#endif
preempt_disable();
__asm__ __volatile__(
" .set push \n"
" .set noreorder \n"
" .set noat \n"
-#ifdef CONFIG_MIPS_MT_SMTC
- " mfc0 $1, $2, 1 \n"
- " andi %[flags], 0x400 \n"
- " ori $1, 0x400 \n"
- " xori $1, 0x400 \n"
- " or %[flags], $1 \n"
- " mtc0 %[flags], $2, 1 \n"
-#elif defined(CONFIG_CPU_MIPSR2) && defined(CONFIG_IRQ_CPU)
+#if defined(CONFIG_CPU_MIPSR2) && defined(CONFIG_IRQ_CPU)
/* see irqflags.h for inline function */
#elif defined(CONFIG_CPU_MIPSR2)
/* see irqflags.h for inline function */
@@ -163,14 +136,7 @@
" .set push \n"
" .set noreorder \n"
" .set noat \n"
-#ifdef CONFIG_MIPS_MT_SMTC
- " mfc0 $1, $2, 1 \n"
- " andi %[flags], 0x400 \n"
- " ori $1, 0x400 \n"
- " xori $1, 0x400 \n"
- " or %[flags], $1 \n"
- " mtc0 %[flags], $2, 1 \n"
-#elif defined(CONFIG_CPU_MIPSR2) && defined(CONFIG_IRQ_CPU)
+#if defined(CONFIG_CPU_MIPSR2) && defined(CONFIG_IRQ_CPU)
/* see irqflags.h for inline function */
#elif defined(CONFIG_CPU_MIPSR2)
/* see irqflags.h for inline function */
@@ -192,4 +158,4 @@
}
EXPORT_SYMBOL(__arch_local_irq_restore);
-#endif /* !defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_MIPS_MT_SMTC) */
+#endif /* !CONFIG_CPU_MIPSR2 */
diff --git a/arch/mips/loongson/Kconfig b/arch/mips/loongson/Kconfig
index 603d79a..e6a86ccc 100644
--- a/arch/mips/loongson/Kconfig
+++ b/arch/mips/loongson/Kconfig
@@ -95,10 +95,11 @@
config CS5536_MFGPT
bool "CS5536 MFGPT Timer"
- depends on CS5536
+ depends on CS5536 && !HIGH_RES_TIMERS
select MIPS_EXTERNAL_TIMER
help
- This option enables the mfgpt0 timer of AMD CS5536.
+ This option enables the mfgpt0 timer of AMD CS5536. With this timer
+ switched on you can not use high resolution timers.
If you want to enable the Loongson2 CPUFreq Driver, Please enable
this option at first, otherwise, You will get wrong system time.
diff --git a/arch/mips/loongson/loongson-3/smp.c b/arch/mips/loongson/loongson-3/smp.c
index c665fe1..1e88940 100644
--- a/arch/mips/loongson/loongson-3/smp.c
+++ b/arch/mips/loongson/loongson-3/smp.c
@@ -279,13 +279,6 @@
loongson3_ipi_write64(startargs[0], (void *)(ipi_mailbox_buf[cpu]+0x0));
}
-/*
- * Final cleanup after all secondaries booted
- */
-static void __init loongson3_cpus_done(void)
-{
-}
-
#ifdef CONFIG_HOTPLUG_CPU
static int loongson3_cpu_disable(void)
@@ -432,7 +425,6 @@
.send_ipi_mask = loongson3_send_ipi_mask,
.init_secondary = loongson3_init_secondary,
.smp_finish = loongson3_smp_finish,
- .cpus_done = loongson3_cpus_done,
.boot_secondary = loongson3_boot_secondary,
.smp_setup = loongson3_smp_setup,
.prepare_cpus = loongson3_prepare_cpus,
diff --git a/arch/mips/loongson1/Kconfig b/arch/mips/loongson1/Kconfig
index fbf75f6..e23c25d 100644
--- a/arch/mips/loongson1/Kconfig
+++ b/arch/mips/loongson1/Kconfig
@@ -14,6 +14,7 @@
select SYS_SUPPORTS_32BIT_KERNEL
select SYS_SUPPORTS_LITTLE_ENDIAN
select SYS_SUPPORTS_HIGHMEM
+ select SYS_SUPPORTS_MIPS16
select SYS_HAS_EARLY_PRINTK
select COMMON_CLK
diff --git a/arch/mips/math-emu/Makefile b/arch/mips/math-emu/Makefile
index 121a848..619cfc1a 100644
--- a/arch/mips/math-emu/Makefile
+++ b/arch/mips/math-emu/Makefile
@@ -2,10 +2,12 @@
# Makefile for the Linux/MIPS kernel FPU emulation.
#
-obj-y := cp1emu.o ieee754m.o ieee754d.o ieee754dp.o ieee754sp.o ieee754.o \
- ieee754xcpt.o dp_frexp.o dp_modf.o dp_div.o dp_mul.o dp_sub.o \
- dp_add.o dp_fsp.o dp_cmp.o dp_logb.o dp_scalb.o dp_simple.o \
- dp_tint.o dp_fint.o dp_tlong.o dp_flong.o sp_frexp.o sp_modf.o \
- sp_div.o sp_mul.o sp_sub.o sp_add.o sp_fdp.o sp_cmp.o sp_logb.o \
- sp_scalb.o sp_simple.o sp_tint.o sp_fint.o sp_tlong.o sp_flong.o \
- dp_sqrt.o sp_sqrt.o kernel_linkage.o dsemul.o
+obj-y += cp1emu.o ieee754dp.o ieee754sp.o ieee754.o dp_div.o dp_mul.o \
+ dp_sub.o dp_add.o dp_fsp.o dp_cmp.o dp_simple.o dp_tint.o \
+ dp_fint.o dp_tlong.o dp_flong.o sp_div.o sp_mul.o sp_sub.o \
+ sp_add.o sp_fdp.o sp_cmp.o sp_simple.o sp_tint.o sp_fint.o \
+ sp_tlong.o sp_flong.o dsemul.o
+
+lib-y += ieee754d.o dp_sqrt.o sp_sqrt.o
+
+obj-$(CONFIG_DEBUG_FS) += me-debugfs.o
diff --git a/arch/mips/math-emu/cp1emu.c b/arch/mips/math-emu/cp1emu.c
index 7b3c9ac..736c17a 100644
--- a/arch/mips/math-emu/cp1emu.c
+++ b/arch/mips/math-emu/cp1emu.c
@@ -1,5 +1,5 @@
/*
- * cp1emu.c: a MIPS coprocessor 1 (fpu) instruction emulator
+ * cp1emu.c: a MIPS coprocessor 1 (FPU) instruction emulator
*
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
@@ -18,61 +18,46 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* A complete emulator for MIPS coprocessor 1 instructions. This is
* required for #float(switch) or #float(trap), where it catches all
* COP1 instructions via the "CoProcessor Unusable" exception.
*
* More surprisingly it is also required for #float(ieee), to help out
- * the hardware fpu at the boundaries of the IEEE-754 representation
+ * the hardware FPU at the boundaries of the IEEE-754 representation
* (denormalised values, infinities, underflow, etc). It is made
* quite nasty because emulation of some non-COP1 instructions is
* required, e.g. in branch delay slots.
*
- * Note if you know that you won't have an fpu, then you'll get much
+ * Note if you know that you won't have an FPU, then you'll get much
* better performance by compiling with -msoft-float!
*/
#include <linux/sched.h>
-#include <linux/module.h>
#include <linux/debugfs.h>
+#include <linux/kconfig.h>
+#include <linux/percpu-defs.h>
#include <linux/perf_event.h>
+#include <asm/branch.h>
#include <asm/inst.h>
-#include <asm/bootinfo.h>
-#include <asm/processor.h>
#include <asm/ptrace.h>
#include <asm/signal.h>
-#include <asm/mipsregs.h>
+#include <asm/uaccess.h>
+
+#include <asm/processor.h>
#include <asm/fpu_emulator.h>
#include <asm/fpu.h>
-#include <asm/uaccess.h>
-#include <asm/branch.h>
#include "ieee754.h"
-/* Strap kernel emulator for full MIPS IV emulation */
-
-#ifdef __mips
-#undef __mips
-#endif
-#define __mips 4
-
/* Function which emulates a floating point instruction. */
static int fpu_emu(struct pt_regs *, struct mips_fpu_struct *,
mips_instruction);
-#if __mips >= 4 && __mips != 32
static int fpux_emu(struct pt_regs *,
struct mips_fpu_struct *, mips_instruction, void *__user *);
-#endif
-
-/* Further private data for which no space exists in mips_fpu_struct */
-
-#ifdef CONFIG_DEBUG_FS
-DEFINE_PER_CPU(struct mips_fpu_emulator_stats, fpuemustats);
-#endif
/* Control registers */
@@ -82,27 +67,6 @@
/* Determine rounding mode from the RM bits of the FCSR */
#define modeindex(v) ((v) & FPU_CSR_RM)
-/* microMIPS bitfields */
-#define MM_POOL32A_MINOR_MASK 0x3f
-#define MM_POOL32A_MINOR_SHIFT 0x6
-#define MM_MIPS32_COND_FC 0x30
-
-/* Convert Mips rounding mode (0..3) to IEEE library modes. */
-static const unsigned char ieee_rm[4] = {
- [FPU_CSR_RN] = IEEE754_RN,
- [FPU_CSR_RZ] = IEEE754_RZ,
- [FPU_CSR_RU] = IEEE754_RU,
- [FPU_CSR_RD] = IEEE754_RD,
-};
-/* Convert IEEE library modes to Mips rounding mode (0..3). */
-static const unsigned char mips_rm[4] = {
- [IEEE754_RN] = FPU_CSR_RN,
- [IEEE754_RZ] = FPU_CSR_RZ,
- [IEEE754_RD] = FPU_CSR_RD,
- [IEEE754_RU] = FPU_CSR_RU,
-};
-
-#if __mips >= 4
/* convert condition code register number to csr bit */
static const unsigned int fpucondbit[8] = {
FPU_CSR_COND0,
@@ -114,10 +78,6 @@
FPU_CSR_COND6,
FPU_CSR_COND7
};
-#endif
-
-/* (microMIPS) Convert 16-bit register encoding to 32-bit register encoding. */
-static const unsigned int reg16to32map[8] = {16, 17, 2, 3, 4, 5, 6, 7};
/* (microMIPS) Convert certain microMIPS instructions to MIPS32 format. */
static const int sd_format[] = {16, 17, 0, 0, 0, 0, 0, 0};
@@ -466,199 +426,6 @@
return 0;
}
-int mm_isBranchInstr(struct pt_regs *regs, struct mm_decoded_insn dec_insn,
- unsigned long *contpc)
-{
- union mips_instruction insn = (union mips_instruction)dec_insn.insn;
- int bc_false = 0;
- unsigned int fcr31;
- unsigned int bit;
-
- if (!cpu_has_mmips)
- return 0;
-
- switch (insn.mm_i_format.opcode) {
- case mm_pool32a_op:
- if ((insn.mm_i_format.simmediate & MM_POOL32A_MINOR_MASK) ==
- mm_pool32axf_op) {
- switch (insn.mm_i_format.simmediate >>
- MM_POOL32A_MINOR_SHIFT) {
- case mm_jalr_op:
- case mm_jalrhb_op:
- case mm_jalrs_op:
- case mm_jalrshb_op:
- if (insn.mm_i_format.rt != 0) /* Not mm_jr */
- regs->regs[insn.mm_i_format.rt] =
- regs->cp0_epc +
- dec_insn.pc_inc +
- dec_insn.next_pc_inc;
- *contpc = regs->regs[insn.mm_i_format.rs];
- return 1;
- }
- }
- break;
- case mm_pool32i_op:
- switch (insn.mm_i_format.rt) {
- case mm_bltzals_op:
- case mm_bltzal_op:
- regs->regs[31] = regs->cp0_epc +
- dec_insn.pc_inc +
- dec_insn.next_pc_inc;
- /* Fall through */
- case mm_bltz_op:
- if ((long)regs->regs[insn.mm_i_format.rs] < 0)
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- (insn.mm_i_format.simmediate << 1);
- else
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- dec_insn.next_pc_inc;
- return 1;
- case mm_bgezals_op:
- case mm_bgezal_op:
- regs->regs[31] = regs->cp0_epc +
- dec_insn.pc_inc +
- dec_insn.next_pc_inc;
- /* Fall through */
- case mm_bgez_op:
- if ((long)regs->regs[insn.mm_i_format.rs] >= 0)
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- (insn.mm_i_format.simmediate << 1);
- else
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- dec_insn.next_pc_inc;
- return 1;
- case mm_blez_op:
- if ((long)regs->regs[insn.mm_i_format.rs] <= 0)
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- (insn.mm_i_format.simmediate << 1);
- else
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- dec_insn.next_pc_inc;
- return 1;
- case mm_bgtz_op:
- if ((long)regs->regs[insn.mm_i_format.rs] <= 0)
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- (insn.mm_i_format.simmediate << 1);
- else
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- dec_insn.next_pc_inc;
- return 1;
- case mm_bc2f_op:
- case mm_bc1f_op:
- bc_false = 1;
- /* Fall through */
- case mm_bc2t_op:
- case mm_bc1t_op:
- preempt_disable();
- if (is_fpu_owner())
- asm volatile("cfc1\t%0,$31" : "=r" (fcr31));
- else
- fcr31 = current->thread.fpu.fcr31;
- preempt_enable();
-
- if (bc_false)
- fcr31 = ~fcr31;
-
- bit = (insn.mm_i_format.rs >> 2);
- bit += (bit != 0);
- bit += 23;
- if (fcr31 & (1 << bit))
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- (insn.mm_i_format.simmediate << 1);
- else
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc + dec_insn.next_pc_inc;
- return 1;
- }
- break;
- case mm_pool16c_op:
- switch (insn.mm_i_format.rt) {
- case mm_jalr16_op:
- case mm_jalrs16_op:
- regs->regs[31] = regs->cp0_epc +
- dec_insn.pc_inc + dec_insn.next_pc_inc;
- /* Fall through */
- case mm_jr16_op:
- *contpc = regs->regs[insn.mm_i_format.rs];
- return 1;
- }
- break;
- case mm_beqz16_op:
- if ((long)regs->regs[reg16to32map[insn.mm_b1_format.rs]] == 0)
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- (insn.mm_b1_format.simmediate << 1);
- else
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc + dec_insn.next_pc_inc;
- return 1;
- case mm_bnez16_op:
- if ((long)regs->regs[reg16to32map[insn.mm_b1_format.rs]] != 0)
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- (insn.mm_b1_format.simmediate << 1);
- else
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc + dec_insn.next_pc_inc;
- return 1;
- case mm_b16_op:
- *contpc = regs->cp0_epc + dec_insn.pc_inc +
- (insn.mm_b0_format.simmediate << 1);
- return 1;
- case mm_beq32_op:
- if (regs->regs[insn.mm_i_format.rs] ==
- regs->regs[insn.mm_i_format.rt])
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- (insn.mm_i_format.simmediate << 1);
- else
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- dec_insn.next_pc_inc;
- return 1;
- case mm_bne32_op:
- if (regs->regs[insn.mm_i_format.rs] !=
- regs->regs[insn.mm_i_format.rt])
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc +
- (insn.mm_i_format.simmediate << 1);
- else
- *contpc = regs->cp0_epc +
- dec_insn.pc_inc + dec_insn.next_pc_inc;
- return 1;
- case mm_jalx32_op:
- regs->regs[31] = regs->cp0_epc +
- dec_insn.pc_inc + dec_insn.next_pc_inc;
- *contpc = regs->cp0_epc + dec_insn.pc_inc;
- *contpc >>= 28;
- *contpc <<= 28;
- *contpc |= (insn.j_format.target << 2);
- return 1;
- case mm_jals32_op:
- case mm_jal32_op:
- regs->regs[31] = regs->cp0_epc +
- dec_insn.pc_inc + dec_insn.next_pc_inc;
- /* Fall through */
- case mm_j32_op:
- *contpc = regs->cp0_epc + dec_insn.pc_inc;
- *contpc >>= 27;
- *contpc <<= 27;
- *contpc |= (insn.j_format.target << 1);
- set_isa16_mode(*contpc);
- return 1;
- }
- return 0;
-}
-
/*
* Redundant with logic already in kernel/branch.c,
* embedded in compute_return_epc. At some point,
@@ -817,7 +584,11 @@
if (insn.i_format.rs == bc_op) {
preempt_disable();
if (is_fpu_owner())
- asm volatile("cfc1\t%0,$31" : "=r" (fcr31));
+ asm volatile(
+ ".set push\n"
+ "\t.set mips1\n"
+ "\tcfc1\t%0,$31\n"
+ "\t.set pop" : "=r" (fcr31));
else
fcr31 = current->thread.fpu.fcr31;
preempt_enable();
@@ -867,23 +638,25 @@
*/
static inline int cop1_64bit(struct pt_regs *xcp)
{
-#if defined(CONFIG_64BIT) && !defined(CONFIG_MIPS32_O32)
- return 1;
-#elif defined(CONFIG_32BIT) && !defined(CONFIG_MIPS_O32_FP64_SUPPORT)
- return 0;
-#else
+ if (config_enabled(CONFIG_64BIT) && !config_enabled(CONFIG_MIPS32_O32))
+ return 1;
+ else if (config_enabled(CONFIG_32BIT) &&
+ !config_enabled(CONFIG_MIPS_O32_FP64_SUPPORT))
+ return 0;
+
return !test_thread_flag(TIF_32BIT_FPREGS);
-#endif
}
-#define SIFROMREG(si, x) do { \
+#define SIFROMREG(si, x) \
+do { \
if (cop1_64bit(xcp)) \
(si) = get_fpr32(&ctx->fpr[x], 0); \
else \
(si) = get_fpr32(&ctx->fpr[(x) & ~1], (x) & 1); \
} while (0)
-#define SITOREG(si, x) do { \
+#define SITOREG(si, x) \
+do { \
if (cop1_64bit(xcp)) { \
unsigned i; \
set_fpr32(&ctx->fpr[x], 0, si); \
@@ -896,17 +669,19 @@
#define SIFROMHREG(si, x) ((si) = get_fpr32(&ctx->fpr[x], 1))
-#define SITOHREG(si, x) do { \
+#define SITOHREG(si, x) \
+do { \
unsigned i; \
set_fpr32(&ctx->fpr[x], 1, si); \
for (i = 2; i < ARRAY_SIZE(ctx->fpr[x].val32); i++) \
set_fpr32(&ctx->fpr[x], i, 0); \
} while (0)
-#define DIFROMREG(di, x) \
+#define DIFROMREG(di, x) \
((di) = get_fpr64(&ctx->fpr[(x) & ~(cop1_64bit(xcp) == 0)], 0))
-#define DITOREG(di, x) do { \
+#define DITOREG(di, x) \
+do { \
unsigned fpr, i; \
fpr = (x) & ~(cop1_64bit(xcp) == 0); \
set_fpr64(&ctx->fpr[fpr], 0, di); \
@@ -927,23 +702,36 @@
static int cop1Emulate(struct pt_regs *xcp, struct mips_fpu_struct *ctx,
struct mm_decoded_insn dec_insn, void *__user *fault_addr)
{
- mips_instruction ir;
unsigned long contpc = xcp->cp0_epc + dec_insn.pc_inc;
- unsigned int cond;
- int pc_inc;
+ unsigned int cond, cbit;
+ mips_instruction ir;
+ int likely, pc_inc;
+ u32 __user *wva;
+ u64 __user *dva;
+ u32 value;
+ u32 wval;
+ u64 dval;
+ int sig;
+
+ /*
+ * These are giving gcc a gentle hint about what to expect in
+ * dec_inst in order to do better optimization.
+ */
+ if (!cpu_has_mmips && dec_insn.micro_mips_mode)
+ unreachable();
/* XXX NEC Vr54xx bug workaround */
- if (xcp->cp0_cause & CAUSEF_BD) {
+ if (delay_slot(xcp)) {
if (dec_insn.micro_mips_mode) {
if (!mm_isBranchInstr(xcp, dec_insn, &contpc))
- xcp->cp0_cause &= ~CAUSEF_BD;
+ clear_delay_slot(xcp);
} else {
if (!isBranchInstr(xcp, dec_insn, &contpc))
- xcp->cp0_cause &= ~CAUSEF_BD;
+ clear_delay_slot(xcp);
}
}
- if (xcp->cp0_cause & CAUSEF_BD) {
+ if (delay_slot(xcp)) {
/*
* The instruction to be emulated is in a branch delay slot
* which means that we have to emulate the branch instruction
@@ -985,96 +773,85 @@
return SIGILL;
}
- emul:
+emul:
perf_sw_event(PERF_COUNT_SW_EMULATION_FAULTS, 1, xcp, 0);
MIPS_FPU_EMU_INC_STATS(emulated);
switch (MIPSInst_OPCODE(ir)) {
- case ldc1_op:{
- u64 __user *va = (u64 __user *) (xcp->regs[MIPSInst_RS(ir)] +
- MIPSInst_SIMM(ir));
- u64 val;
-
+ case ldc1_op:
+ dva = (u64 __user *) (xcp->regs[MIPSInst_RS(ir)] +
+ MIPSInst_SIMM(ir));
MIPS_FPU_EMU_INC_STATS(loads);
- if (!access_ok(VERIFY_READ, va, sizeof(u64))) {
+ if (!access_ok(VERIFY_READ, dva, sizeof(u64))) {
MIPS_FPU_EMU_INC_STATS(errors);
- *fault_addr = va;
+ *fault_addr = dva;
return SIGBUS;
}
- if (__get_user(val, va)) {
+ if (__get_user(dval, dva)) {
MIPS_FPU_EMU_INC_STATS(errors);
- *fault_addr = va;
+ *fault_addr = dva;
return SIGSEGV;
}
- DITOREG(val, MIPSInst_RT(ir));
+ DITOREG(dval, MIPSInst_RT(ir));
break;
- }
- case sdc1_op:{
- u64 __user *va = (u64 __user *) (xcp->regs[MIPSInst_RS(ir)] +
- MIPSInst_SIMM(ir));
- u64 val;
-
+ case sdc1_op:
+ dva = (u64 __user *) (xcp->regs[MIPSInst_RS(ir)] +
+ MIPSInst_SIMM(ir));
MIPS_FPU_EMU_INC_STATS(stores);
- DIFROMREG(val, MIPSInst_RT(ir));
- if (!access_ok(VERIFY_WRITE, va, sizeof(u64))) {
+ DIFROMREG(dval, MIPSInst_RT(ir));
+ if (!access_ok(VERIFY_WRITE, dva, sizeof(u64))) {
MIPS_FPU_EMU_INC_STATS(errors);
- *fault_addr = va;
+ *fault_addr = dva;
return SIGBUS;
}
- if (__put_user(val, va)) {
+ if (__put_user(dval, dva)) {
MIPS_FPU_EMU_INC_STATS(errors);
- *fault_addr = va;
+ *fault_addr = dva;
return SIGSEGV;
}
break;
- }
- case lwc1_op:{
- u32 __user *va = (u32 __user *) (xcp->regs[MIPSInst_RS(ir)] +
- MIPSInst_SIMM(ir));
- u32 val;
-
+ case lwc1_op:
+ wva = (u32 __user *) (xcp->regs[MIPSInst_RS(ir)] +
+ MIPSInst_SIMM(ir));
MIPS_FPU_EMU_INC_STATS(loads);
- if (!access_ok(VERIFY_READ, va, sizeof(u32))) {
+ if (!access_ok(VERIFY_READ, wva, sizeof(u32))) {
MIPS_FPU_EMU_INC_STATS(errors);
- *fault_addr = va;
+ *fault_addr = wva;
return SIGBUS;
}
- if (__get_user(val, va)) {
+ if (__get_user(wval, wva)) {
MIPS_FPU_EMU_INC_STATS(errors);
- *fault_addr = va;
+ *fault_addr = wva;
return SIGSEGV;
}
- SITOREG(val, MIPSInst_RT(ir));
+ SITOREG(wval, MIPSInst_RT(ir));
break;
- }
- case swc1_op:{
- u32 __user *va = (u32 __user *) (xcp->regs[MIPSInst_RS(ir)] +
- MIPSInst_SIMM(ir));
- u32 val;
-
+ case swc1_op:
+ wva = (u32 __user *) (xcp->regs[MIPSInst_RS(ir)] +
+ MIPSInst_SIMM(ir));
MIPS_FPU_EMU_INC_STATS(stores);
- SIFROMREG(val, MIPSInst_RT(ir));
- if (!access_ok(VERIFY_WRITE, va, sizeof(u32))) {
+ SIFROMREG(wval, MIPSInst_RT(ir));
+ if (!access_ok(VERIFY_WRITE, wva, sizeof(u32))) {
MIPS_FPU_EMU_INC_STATS(errors);
- *fault_addr = va;
+ *fault_addr = wva;
return SIGBUS;
}
- if (__put_user(val, va)) {
+ if (__put_user(wval, wva)) {
MIPS_FPU_EMU_INC_STATS(errors);
- *fault_addr = va;
+ *fault_addr = wva;
return SIGSEGV;
}
break;
- }
case cop1_op:
switch (MIPSInst_RS(ir)) {
-
-#if defined(__mips64)
case dmfc_op:
+ if (!cpu_has_mips_3_4_5 && !cpu_has_mips64)
+ return SIGILL;
+
/* copregister fs -> gpr[rt] */
if (MIPSInst_RT(ir) != 0) {
DIFROMREG(xcp->regs[MIPSInst_RT(ir)],
@@ -1083,10 +860,12 @@
break;
case dmtc_op:
+ if (!cpu_has_mips_3_4_5 && !cpu_has_mips64)
+ return SIGILL;
+
/* copregister fs <- rt */
DITOREG(xcp->regs[MIPSInst_RT(ir)], MIPSInst_RD(ir));
break;
-#endif
case mfhc_op:
if (!cpu_has_mips_r2)
@@ -1120,19 +899,14 @@
SITOREG(xcp->regs[MIPSInst_RT(ir)], MIPSInst_RD(ir));
break;
- case cfc_op:{
+ case cfc_op:
/* cop control register rd -> gpr[rt] */
- u32 value;
-
if (MIPSInst_RD(ir) == FPCREG_CSR) {
value = ctx->fcr31;
- value = (value & ~FPU_CSR_RM) |
- mips_rm[modeindex(value)];
-#ifdef CSRTRACE
- printk("%p gpr[%d]<-csr=%08x\n",
- (void *) (xcp->cp0_epc),
- MIPSInst_RT(ir), value);
-#endif
+ value = (value & ~FPU_CSR_RM) | modeindex(value);
+ pr_debug("%p gpr[%d]<-csr=%08x\n",
+ (void *) (xcp->cp0_epc),
+ MIPSInst_RT(ir), value);
}
else if (MIPSInst_RD(ir) == FPCREG_RID)
value = 0;
@@ -1141,12 +915,9 @@
if (MIPSInst_RT(ir))
xcp->regs[MIPSInst_RT(ir)] = value;
break;
- }
- case ctc_op:{
+ case ctc_op:
/* copregister rd <- rt */
- u32 value;
-
if (MIPSInst_RT(ir) == 0)
value = 0;
else
@@ -1155,37 +926,33 @@
/* we only have one writable control reg
*/
if (MIPSInst_RD(ir) == FPCREG_CSR) {
-#ifdef CSRTRACE
- printk("%p gpr[%d]->csr=%08x\n",
- (void *) (xcp->cp0_epc),
- MIPSInst_RT(ir), value);
-#endif
+ pr_debug("%p gpr[%d]->csr=%08x\n",
+ (void *) (xcp->cp0_epc),
+ MIPSInst_RT(ir), value);
/*
* Don't write reserved bits,
* and convert to ieee library modes
*/
- ctx->fcr31 = (value &
- ~(FPU_CSR_RSVD | FPU_CSR_RM)) |
- ieee_rm[modeindex(value)];
+ ctx->fcr31 = (value & ~(FPU_CSR_RSVD | FPU_CSR_RM)) |
+ modeindex(value);
}
if ((ctx->fcr31 >> 5) & ctx->fcr31 & FPU_CSR_ALL_E) {
return SIGFPE;
}
break;
- }
- case bc_op:{
- int likely = 0;
-
- if (xcp->cp0_cause & CAUSEF_BD)
+ case bc_op:
+ if (delay_slot(xcp))
return SIGILL;
-#if __mips >= 4
- cond = ctx->fcr31 & fpucondbit[MIPSInst_RT(ir) >> 2];
-#else
- cond = ctx->fcr31 & FPU_CSR_COND;
-#endif
+ if (cpu_has_mips_4_5_r)
+ cbit = fpucondbit[MIPSInst_RT(ir) >> 2];
+ else
+ cbit = FPU_CSR_COND;
+ cond = ctx->fcr31 & cbit;
+
+ likely = 0;
switch (MIPSInst_RT(ir) & 3) {
case bcfl_op:
likely = 1;
@@ -1201,10 +968,10 @@
return SIGILL;
}
- xcp->cp0_cause |= CAUSEF_BD;
+ set_delay_slot(xcp);
if (cond) {
- /* branch taken: emulate dslot
- * instruction
+ /*
+ * Branch taken: emulate dslot instruction
*/
xcp->cp0_epc += dec_insn.pc_inc;
@@ -1238,23 +1005,37 @@
switch (MIPSInst_OPCODE(ir)) {
case lwc1_op:
+ goto emul;
+
case swc1_op:
-#if (__mips >= 2 || defined(__mips64))
+ goto emul;
+
case ldc1_op:
case sdc1_op:
-#endif
- case cop1_op:
-#if __mips >= 4 && __mips != 32
- case cop1x_op:
-#endif
- /* its one of ours */
+ if (cpu_has_mips_2_3_4_5 ||
+ cpu_has_mips64)
+ goto emul;
+
+ return SIGILL;
goto emul;
-#if __mips >= 4
+
+ case cop1_op:
+ goto emul;
+
+ case cop1x_op:
+ if (cpu_has_mips_4_5 || cpu_has_mips64)
+ /* its one of ours */
+ goto emul;
+
+ return SIGILL;
+
case spec_op:
+ if (!cpu_has_mips_4_5_r)
+ return SIGILL;
+
if (MIPSInst_FUNC(ir) == movc_op)
goto emul;
break;
-#endif
}
/*
@@ -1262,10 +1043,7 @@
* instruction in the dslot
*/
return mips_dsemul(xcp, ir, contpc);
- }
- else {
- /* branch not taken */
- if (likely) {
+ } else if (likely) { /* branch not taken */
/*
* branch likely nullifies
* dslot if not taken
@@ -1277,34 +1055,31 @@
* dslot as normal insn
*/
}
- }
break;
- }
default:
if (!(MIPSInst_RS(ir) & 0x10))
return SIGILL;
- {
- int sig;
- /* a real fpu computation instruction */
- if ((sig = fpu_emu(xcp, ctx, ir)))
- return sig;
- }
+ /* a real fpu computation instruction */
+ if ((sig = fpu_emu(xcp, ctx, ir)))
+ return sig;
}
break;
-#if __mips >= 4 && __mips != 32
- case cop1x_op:{
- int sig = fpux_emu(xcp, ctx, ir, fault_addr);
+ case cop1x_op:
+ if (!cpu_has_mips_4_5 && !cpu_has_mips64)
+ return SIGILL;
+
+ sig = fpux_emu(xcp, ctx, ir, fault_addr);
if (sig)
return sig;
break;
- }
-#endif
-#if __mips >= 4
case spec_op:
+ if (!cpu_has_mips_4_5_r)
+ return SIGILL;
+
if (MIPSInst_FUNC(ir) != movc_op)
return SIGILL;
cond = fpucondbit[MIPSInst_RT(ir) >> 2];
@@ -1312,8 +1087,6 @@
xcp->regs[MIPSInst_RD(ir)] =
xcp->regs[MIPSInst_RS(ir)];
break;
-#endif
-
default:
sigill:
return SIGILL;
@@ -1321,7 +1094,7 @@
/* we did it !! */
xcp->cp0_epc = contpc;
- xcp->cp0_cause &= ~CAUSEF_BD;
+ clear_delay_slot(xcp);
return 0;
}
@@ -1342,44 +1115,42 @@
};
-#if __mips >= 4 && __mips != 32
-
/*
* Additional MIPS4 instructions
*/
-#define DEF3OP(name, p, f1, f2, f3) \
-static ieee754##p fpemu_##p##_##name(ieee754##p r, ieee754##p s, \
- ieee754##p t) \
-{ \
- struct _ieee754_csr ieee754_csr_save; \
- s = f1(s, t); \
- ieee754_csr_save = ieee754_csr; \
- s = f2(s, r); \
- ieee754_csr_save.cx |= ieee754_csr.cx; \
- ieee754_csr_save.sx |= ieee754_csr.sx; \
- s = f3(s); \
- ieee754_csr.cx |= ieee754_csr_save.cx; \
- ieee754_csr.sx |= ieee754_csr_save.sx; \
- return s; \
+#define DEF3OP(name, p, f1, f2, f3) \
+static union ieee754##p fpemu_##p##_##name(union ieee754##p r, \
+ union ieee754##p s, union ieee754##p t) \
+{ \
+ struct _ieee754_csr ieee754_csr_save; \
+ s = f1(s, t); \
+ ieee754_csr_save = ieee754_csr; \
+ s = f2(s, r); \
+ ieee754_csr_save.cx |= ieee754_csr.cx; \
+ ieee754_csr_save.sx |= ieee754_csr.sx; \
+ s = f3(s); \
+ ieee754_csr.cx |= ieee754_csr_save.cx; \
+ ieee754_csr.sx |= ieee754_csr_save.sx; \
+ return s; \
}
-static ieee754dp fpemu_dp_recip(ieee754dp d)
+static union ieee754dp fpemu_dp_recip(union ieee754dp d)
{
return ieee754dp_div(ieee754dp_one(0), d);
}
-static ieee754dp fpemu_dp_rsqrt(ieee754dp d)
+static union ieee754dp fpemu_dp_rsqrt(union ieee754dp d)
{
return ieee754dp_div(ieee754dp_one(0), ieee754dp_sqrt(d));
}
-static ieee754sp fpemu_sp_recip(ieee754sp s)
+static union ieee754sp fpemu_sp_recip(union ieee754sp s)
{
return ieee754sp_div(ieee754sp_one(0), s);
}
-static ieee754sp fpemu_sp_rsqrt(ieee754sp s)
+static union ieee754sp fpemu_sp_rsqrt(union ieee754sp s)
{
return ieee754sp_div(ieee754sp_one(0), ieee754sp_sqrt(s));
}
@@ -1403,8 +1174,8 @@
switch (MIPSInst_FMA_FFMT(ir)) {
case s_fmt:{ /* 0 */
- ieee754sp(*handler) (ieee754sp, ieee754sp, ieee754sp);
- ieee754sp fd, fr, fs, ft;
+ union ieee754sp(*handler) (union ieee754sp, union ieee754sp, union ieee754sp);
+ union ieee754sp fd, fr, fs, ft;
u32 __user *va;
u32 val;
@@ -1467,18 +1238,26 @@
SPTOREG(fd, MIPSInst_FD(ir));
copcsr:
- if (ieee754_cxtest(IEEE754_INEXACT))
+ if (ieee754_cxtest(IEEE754_INEXACT)) {
+ MIPS_FPU_EMU_INC_STATS(ieee754_inexact);
rcsr |= FPU_CSR_INE_X | FPU_CSR_INE_S;
- if (ieee754_cxtest(IEEE754_UNDERFLOW))
+ }
+ if (ieee754_cxtest(IEEE754_UNDERFLOW)) {
+ MIPS_FPU_EMU_INC_STATS(ieee754_underflow);
rcsr |= FPU_CSR_UDF_X | FPU_CSR_UDF_S;
- if (ieee754_cxtest(IEEE754_OVERFLOW))
+ }
+ if (ieee754_cxtest(IEEE754_OVERFLOW)) {
+ MIPS_FPU_EMU_INC_STATS(ieee754_overflow);
rcsr |= FPU_CSR_OVF_X | FPU_CSR_OVF_S;
- if (ieee754_cxtest(IEEE754_INVALID_OPERATION))
+ }
+ if (ieee754_cxtest(IEEE754_INVALID_OPERATION)) {
+ MIPS_FPU_EMU_INC_STATS(ieee754_invalidop);
rcsr |= FPU_CSR_INV_X | FPU_CSR_INV_S;
+ }
ctx->fcr31 = (ctx->fcr31 & ~FPU_CSR_ALL_X) | rcsr;
if ((ctx->fcr31 >> 5) & ctx->fcr31 & FPU_CSR_ALL_E) {
- /*printk ("SIGFPE: fpu csr = %08x\n",
+ /*printk ("SIGFPE: FPU csr = %08x\n",
ctx->fcr31); */
return SIGFPE;
}
@@ -1492,8 +1271,8 @@
}
case d_fmt:{ /* 1 */
- ieee754dp(*handler) (ieee754dp, ieee754dp, ieee754dp);
- ieee754dp fd, fr, fs, ft;
+ union ieee754dp(*handler) (union ieee754dp, union ieee754dp, union ieee754dp);
+ union ieee754dp fd, fr, fs, ft;
u64 __user *va;
u64 val;
@@ -1574,7 +1353,6 @@
return 0;
}
-#endif
@@ -1586,23 +1364,25 @@
{
int rfmt; /* resulting format */
unsigned rcsr = 0; /* resulting csr */
+ unsigned int oldrm;
+ unsigned int cbit;
unsigned cond;
union {
- ieee754dp d;
- ieee754sp s;
+ union ieee754dp d;
+ union ieee754sp s;
int w;
-#ifdef __mips64
s64 l;
-#endif
} rv; /* resulting value */
+ u64 bits;
MIPS_FPU_EMU_INC_STATS(cp1ops);
switch (rfmt = (MIPSInst_FFMT(ir) & 0xf)) {
- case s_fmt:{ /* 0 */
+ case s_fmt: { /* 0 */
union {
- ieee754sp(*b) (ieee754sp, ieee754sp);
- ieee754sp(*u) (ieee754sp);
+ union ieee754sp(*b) (union ieee754sp, union ieee754sp);
+ union ieee754sp(*u) (union ieee754sp);
} handler;
+ union ieee754sp fs, ft;
switch (MIPSInst_FUNC(ir)) {
/* binary ops */
@@ -1620,148 +1400,167 @@
goto scopbop;
/* unary ops */
-#if __mips >= 2 || defined(__mips64)
case fsqrt_op:
+ if (!cpu_has_mips_4_5_r)
+ return SIGILL;
+
handler.u = ieee754sp_sqrt;
goto scopuop;
-#endif
-#if __mips >= 4 && __mips != 32
+
+ /*
+ * Note that on some MIPS IV implementations such as the
+ * R5000 and R8000 the FSQRT and FRECIP instructions do not
+ * achieve full IEEE-754 accuracy - however this emulator does.
+ */
case frsqrt_op:
+ if (!cpu_has_mips_4_5_r2)
+ return SIGILL;
+
handler.u = fpemu_sp_rsqrt;
goto scopuop;
+
case frecip_op:
+ if (!cpu_has_mips_4_5_r2)
+ return SIGILL;
+
handler.u = fpemu_sp_recip;
goto scopuop;
-#endif
-#if __mips >= 4
+
case fmovc_op:
+ if (!cpu_has_mips_4_5_r)
+ return SIGILL;
+
cond = fpucondbit[MIPSInst_FT(ir) >> 2];
if (((ctx->fcr31 & cond) != 0) !=
((MIPSInst_FT(ir) & 1) != 0))
return 0;
SPFROMREG(rv.s, MIPSInst_FS(ir));
break;
+
case fmovz_op:
+ if (!cpu_has_mips_4_5_r)
+ return SIGILL;
+
if (xcp->regs[MIPSInst_FT(ir)] != 0)
return 0;
SPFROMREG(rv.s, MIPSInst_FS(ir));
break;
+
case fmovn_op:
+ if (!cpu_has_mips_4_5_r)
+ return SIGILL;
+
if (xcp->regs[MIPSInst_FT(ir)] == 0)
return 0;
SPFROMREG(rv.s, MIPSInst_FS(ir));
break;
-#endif
+
case fabs_op:
handler.u = ieee754sp_abs;
goto scopuop;
+
case fneg_op:
handler.u = ieee754sp_neg;
goto scopuop;
+
case fmov_op:
/* an easy one */
SPFROMREG(rv.s, MIPSInst_FS(ir));
goto copcsr;
/* binary op on handler */
- scopbop:
- {
- ieee754sp fs, ft;
+scopbop:
+ SPFROMREG(fs, MIPSInst_FS(ir));
+ SPFROMREG(ft, MIPSInst_FT(ir));
- SPFROMREG(fs, MIPSInst_FS(ir));
- SPFROMREG(ft, MIPSInst_FT(ir));
-
- rv.s = (*handler.b) (fs, ft);
- goto copcsr;
- }
- scopuop:
- {
- ieee754sp fs;
-
- SPFROMREG(fs, MIPSInst_FS(ir));
- rv.s = (*handler.u) (fs);
- goto copcsr;
- }
- copcsr:
- if (ieee754_cxtest(IEEE754_INEXACT))
+ rv.s = (*handler.b) (fs, ft);
+ goto copcsr;
+scopuop:
+ SPFROMREG(fs, MIPSInst_FS(ir));
+ rv.s = (*handler.u) (fs);
+ goto copcsr;
+copcsr:
+ if (ieee754_cxtest(IEEE754_INEXACT)) {
+ MIPS_FPU_EMU_INC_STATS(ieee754_inexact);
rcsr |= FPU_CSR_INE_X | FPU_CSR_INE_S;
- if (ieee754_cxtest(IEEE754_UNDERFLOW))
+ }
+ if (ieee754_cxtest(IEEE754_UNDERFLOW)) {
+ MIPS_FPU_EMU_INC_STATS(ieee754_underflow);
rcsr |= FPU_CSR_UDF_X | FPU_CSR_UDF_S;
- if (ieee754_cxtest(IEEE754_OVERFLOW))
+ }
+ if (ieee754_cxtest(IEEE754_OVERFLOW)) {
+ MIPS_FPU_EMU_INC_STATS(ieee754_overflow);
rcsr |= FPU_CSR_OVF_X | FPU_CSR_OVF_S;
- if (ieee754_cxtest(IEEE754_ZERO_DIVIDE))
+ }
+ if (ieee754_cxtest(IEEE754_ZERO_DIVIDE)) {
+ MIPS_FPU_EMU_INC_STATS(ieee754_zerodiv);
rcsr |= FPU_CSR_DIV_X | FPU_CSR_DIV_S;
- if (ieee754_cxtest(IEEE754_INVALID_OPERATION))
+ }
+ if (ieee754_cxtest(IEEE754_INVALID_OPERATION)) {
+ MIPS_FPU_EMU_INC_STATS(ieee754_invalidop);
rcsr |= FPU_CSR_INV_X | FPU_CSR_INV_S;
+ }
break;
/* unary conv ops */
case fcvts_op:
return SIGILL; /* not defined */
- case fcvtd_op:{
- ieee754sp fs;
+ case fcvtd_op:
SPFROMREG(fs, MIPSInst_FS(ir));
rv.d = ieee754dp_fsp(fs);
rfmt = d_fmt;
goto copcsr;
- }
- case fcvtw_op:{
- ieee754sp fs;
+ case fcvtw_op:
SPFROMREG(fs, MIPSInst_FS(ir));
rv.w = ieee754sp_tint(fs);
rfmt = w_fmt;
goto copcsr;
- }
-#if __mips >= 2 || defined(__mips64)
case fround_op:
case ftrunc_op:
case fceil_op:
- case ffloor_op:{
- unsigned int oldrm = ieee754_csr.rm;
- ieee754sp fs;
+ case ffloor_op:
+ if (!cpu_has_mips_2_3_4_5 && !cpu_has_mips64)
+ return SIGILL;
+ oldrm = ieee754_csr.rm;
SPFROMREG(fs, MIPSInst_FS(ir));
- ieee754_csr.rm = ieee_rm[modeindex(MIPSInst_FUNC(ir))];
+ ieee754_csr.rm = modeindex(MIPSInst_FUNC(ir));
rv.w = ieee754sp_tint(fs);
ieee754_csr.rm = oldrm;
rfmt = w_fmt;
goto copcsr;
- }
-#endif /* __mips >= 2 */
-#if defined(__mips64)
- case fcvtl_op:{
- ieee754sp fs;
+ case fcvtl_op:
+ if (!cpu_has_mips_3_4_5 && !cpu_has_mips64)
+ return SIGILL;
SPFROMREG(fs, MIPSInst_FS(ir));
rv.l = ieee754sp_tlong(fs);
rfmt = l_fmt;
goto copcsr;
- }
case froundl_op:
case ftruncl_op:
case fceill_op:
- case ffloorl_op:{
- unsigned int oldrm = ieee754_csr.rm;
- ieee754sp fs;
+ case ffloorl_op:
+ if (!cpu_has_mips_3_4_5 && !cpu_has_mips64)
+ return SIGILL;
+ oldrm = ieee754_csr.rm;
SPFROMREG(fs, MIPSInst_FS(ir));
- ieee754_csr.rm = ieee_rm[modeindex(MIPSInst_FUNC(ir))];
+ ieee754_csr.rm = modeindex(MIPSInst_FUNC(ir));
rv.l = ieee754sp_tlong(fs);
ieee754_csr.rm = oldrm;
rfmt = l_fmt;
goto copcsr;
- }
-#endif /* defined(__mips64) */
default:
if (MIPSInst_FUNC(ir) >= fcmp_op) {
unsigned cmpop = MIPSInst_FUNC(ir) - fcmp_op;
- ieee754sp fs, ft;
+ union ieee754sp fs, ft;
SPFROMREG(fs, MIPSInst_FS(ir));
SPFROMREG(ft, MIPSInst_FT(ir));
@@ -1774,19 +1573,18 @@
else
goto copcsr;
- }
- else {
+ } else
return SIGILL;
- }
break;
}
break;
}
- case d_fmt:{
+ case d_fmt: {
+ union ieee754dp fs, ft;
union {
- ieee754dp(*b) (ieee754dp, ieee754dp);
- ieee754dp(*u) (ieee754dp);
+ union ieee754dp(*b) (union ieee754dp, union ieee754dp);
+ union ieee754dp(*u) (union ieee754dp);
} handler;
switch (MIPSInst_FUNC(ir)) {
@@ -1805,21 +1603,33 @@
goto dcopbop;
/* unary ops */
-#if __mips >= 2 || defined(__mips64)
case fsqrt_op:
+ if (!cpu_has_mips_2_3_4_5_r)
+ return SIGILL;
+
handler.u = ieee754dp_sqrt;
goto dcopuop;
-#endif
-#if __mips >= 4 && __mips != 32
+ /*
+ * Note that on some MIPS IV implementations such as the
+ * R5000 and R8000 the FSQRT and FRECIP instructions do not
+ * achieve full IEEE-754 accuracy - however this emulator does.
+ */
case frsqrt_op:
+ if (!cpu_has_mips_4_5_r2)
+ return SIGILL;
+
handler.u = fpemu_dp_rsqrt;
goto dcopuop;
case frecip_op:
+ if (!cpu_has_mips_4_5_r2)
+ return SIGILL;
+
handler.u = fpemu_dp_recip;
goto dcopuop;
-#endif
-#if __mips >= 4
case fmovc_op:
+ if (!cpu_has_mips_4_5_r)
+ return SIGILL;
+
cond = fpucondbit[MIPSInst_FT(ir) >> 2];
if (((ctx->fcr31 & cond) != 0) !=
((MIPSInst_FT(ir) & 1) != 0))
@@ -1827,16 +1637,21 @@
DPFROMREG(rv.d, MIPSInst_FS(ir));
break;
case fmovz_op:
+ if (!cpu_has_mips_4_5_r)
+ return SIGILL;
+
if (xcp->regs[MIPSInst_FT(ir)] != 0)
return 0;
DPFROMREG(rv.d, MIPSInst_FS(ir));
break;
case fmovn_op:
+ if (!cpu_has_mips_4_5_r)
+ return SIGILL;
+
if (xcp->regs[MIPSInst_FT(ir)] == 0)
return 0;
DPFROMREG(rv.d, MIPSInst_FS(ir));
break;
-#endif
case fabs_op:
handler.u = ieee754dp_abs;
goto dcopuop;
@@ -1851,91 +1666,78 @@
goto copcsr;
/* binary op on handler */
- dcopbop:{
- ieee754dp fs, ft;
+dcopbop:
+ DPFROMREG(fs, MIPSInst_FS(ir));
+ DPFROMREG(ft, MIPSInst_FT(ir));
- DPFROMREG(fs, MIPSInst_FS(ir));
- DPFROMREG(ft, MIPSInst_FT(ir));
+ rv.d = (*handler.b) (fs, ft);
+ goto copcsr;
+dcopuop:
+ DPFROMREG(fs, MIPSInst_FS(ir));
+ rv.d = (*handler.u) (fs);
+ goto copcsr;
- rv.d = (*handler.b) (fs, ft);
- goto copcsr;
- }
- dcopuop:{
- ieee754dp fs;
-
- DPFROMREG(fs, MIPSInst_FS(ir));
- rv.d = (*handler.u) (fs);
- goto copcsr;
- }
-
- /* unary conv ops */
- case fcvts_op:{
- ieee754dp fs;
-
+ /*
+ * unary conv ops
+ */
+ case fcvts_op:
DPFROMREG(fs, MIPSInst_FS(ir));
rv.s = ieee754sp_fdp(fs);
rfmt = s_fmt;
goto copcsr;
- }
+
case fcvtd_op:
return SIGILL; /* not defined */
- case fcvtw_op:{
- ieee754dp fs;
-
+ case fcvtw_op:
DPFROMREG(fs, MIPSInst_FS(ir));
rv.w = ieee754dp_tint(fs); /* wrong */
rfmt = w_fmt;
goto copcsr;
- }
-#if __mips >= 2 || defined(__mips64)
case fround_op:
case ftrunc_op:
case fceil_op:
- case ffloor_op:{
- unsigned int oldrm = ieee754_csr.rm;
- ieee754dp fs;
+ case ffloor_op:
+ if (!cpu_has_mips_2_3_4_5_r)
+ return SIGILL;
+ oldrm = ieee754_csr.rm;
DPFROMREG(fs, MIPSInst_FS(ir));
- ieee754_csr.rm = ieee_rm[modeindex(MIPSInst_FUNC(ir))];
+ ieee754_csr.rm = modeindex(MIPSInst_FUNC(ir));
rv.w = ieee754dp_tint(fs);
ieee754_csr.rm = oldrm;
rfmt = w_fmt;
goto copcsr;
- }
-#endif
-#if defined(__mips64)
- case fcvtl_op:{
- ieee754dp fs;
+ case fcvtl_op:
+ if (!cpu_has_mips_3_4_5 && !cpu_has_mips64)
+ return SIGILL;
DPFROMREG(fs, MIPSInst_FS(ir));
rv.l = ieee754dp_tlong(fs);
rfmt = l_fmt;
goto copcsr;
- }
case froundl_op:
case ftruncl_op:
case fceill_op:
- case ffloorl_op:{
- unsigned int oldrm = ieee754_csr.rm;
- ieee754dp fs;
+ case ffloorl_op:
+ if (!cpu_has_mips_3_4_5 && !cpu_has_mips64)
+ return SIGILL;
+ oldrm = ieee754_csr.rm;
DPFROMREG(fs, MIPSInst_FS(ir));
- ieee754_csr.rm = ieee_rm[modeindex(MIPSInst_FUNC(ir))];
+ ieee754_csr.rm = modeindex(MIPSInst_FUNC(ir));
rv.l = ieee754dp_tlong(fs);
ieee754_csr.rm = oldrm;
rfmt = l_fmt;
goto copcsr;
- }
-#endif /* __mips >= 3 */
default:
if (MIPSInst_FUNC(ir) >= fcmp_op) {
unsigned cmpop = MIPSInst_FUNC(ir) - fcmp_op;
- ieee754dp fs, ft;
+ union ieee754dp fs, ft;
DPFROMREG(fs, MIPSInst_FS(ir));
DPFROMREG(ft, MIPSInst_FT(ir));
@@ -1957,11 +1759,8 @@
break;
}
break;
- }
- case w_fmt:{
- ieee754sp fs;
-
+ case w_fmt:
switch (MIPSInst_FUNC(ir)) {
case fcvts_op:
/* convert word to single precision real */
@@ -1981,9 +1780,11 @@
break;
}
-#if defined(__mips64)
- case l_fmt:{
- u64 bits;
+ case l_fmt:
+
+ if (!cpu_has_mips_3_4_5 && !cpu_has_mips64)
+ return SIGILL;
+
DIFROMREG(bits, MIPSInst_FS(ir));
switch (MIPSInst_FUNC(ir)) {
@@ -2001,8 +1802,6 @@
return SIGILL;
}
break;
- }
-#endif
default:
return SIGILL;
@@ -2017,7 +1816,7 @@
*/
ctx->fcr31 = (ctx->fcr31 & ~FPU_CSR_ALL_X) | rcsr;
if ((ctx->fcr31 >> 5) & ctx->fcr31 & FPU_CSR_ALL_E) {
- /*printk ("SIGFPE: fpu csr = %08x\n",ctx->fcr31); */
+ /*printk ("SIGFPE: FPU csr = %08x\n",ctx->fcr31); */
return SIGFPE;
}
@@ -2025,18 +1824,18 @@
* Now we can safely write the result back to the register file.
*/
switch (rfmt) {
- case -1:{
-#if __mips >= 4
- cond = fpucondbit[MIPSInst_FD(ir) >> 2];
-#else
- cond = FPU_CSR_COND;
-#endif
- if (rv.w)
- ctx->fcr31 |= cond;
+ case -1:
+
+ if (cpu_has_mips_4_5_r)
+ cbit = fpucondbit[MIPSInst_RT(ir) >> 2];
else
- ctx->fcr31 &= ~cond;
+ cbit = FPU_CSR_COND;
+ if (rv.w)
+ ctx->fcr31 |= cbit;
+ else
+ ctx->fcr31 &= ~cbit;
break;
- }
+
case d_fmt:
DPTOREG(rv.d, MIPSInst_FD(ir));
break;
@@ -2046,11 +1845,12 @@
case w_fmt:
SITOREG(rv.w, MIPSInst_FD(ir));
break;
-#if defined(__mips64)
case l_fmt:
+ if (!cpu_has_mips_3_4_5 && !cpu_has_mips64)
+ return SIGILL;
+
DITOREG(rv.l, MIPSInst_FD(ir));
break;
-#endif
default:
return SIGILL;
}
@@ -2138,11 +1938,7 @@
* ieee754_csr. But ieee754_csr.rm is ieee
* library modes. (not mips rounding mode)
*/
- /* convert to ieee library modes */
- ieee754_csr.rm = ieee_rm[ieee754_csr.rm];
sig = cop1Emulate(xcp, ctx, dec_insn, fault_addr);
- /* revert to mips rounding mode */
- ieee754_csr.rm = mips_rm[ieee754_csr.rm];
}
if (has_fpu)
@@ -2155,58 +1951,8 @@
/* SIGILL indicates a non-fpu instruction */
if (sig == SIGILL && xcp->cp0_epc != oldepc)
- /* but if epc has advanced, then ignore it */
+ /* but if EPC has advanced, then ignore it */
sig = 0;
return sig;
}
-
-#ifdef CONFIG_DEBUG_FS
-
-static int fpuemu_stat_get(void *data, u64 *val)
-{
- int cpu;
- unsigned long sum = 0;
- for_each_online_cpu(cpu) {
- struct mips_fpu_emulator_stats *ps;
- local_t *pv;
- ps = &per_cpu(fpuemustats, cpu);
- pv = (void *)ps + (unsigned long)data;
- sum += local_read(pv);
- }
- *val = sum;
- return 0;
-}
-DEFINE_SIMPLE_ATTRIBUTE(fops_fpuemu_stat, fpuemu_stat_get, NULL, "%llu\n");
-
-extern struct dentry *mips_debugfs_dir;
-static int __init debugfs_fpuemu(void)
-{
- struct dentry *d, *dir;
-
- if (!mips_debugfs_dir)
- return -ENODEV;
- dir = debugfs_create_dir("fpuemustats", mips_debugfs_dir);
- if (!dir)
- return -ENOMEM;
-
-#define FPU_STAT_CREATE(M) \
- do { \
- d = debugfs_create_file(#M , S_IRUGO, dir, \
- (void *)offsetof(struct mips_fpu_emulator_stats, M), \
- &fops_fpuemu_stat); \
- if (!d) \
- return -ENOMEM; \
- } while (0)
-
- FPU_STAT_CREATE(emulated);
- FPU_STAT_CREATE(loads);
- FPU_STAT_CREATE(stores);
- FPU_STAT_CREATE(cp1ops);
- FPU_STAT_CREATE(cp1xops);
- FPU_STAT_CREATE(errors);
-
- return 0;
-}
-__initcall(debugfs_fpuemu);
-#endif
diff --git a/arch/mips/math-emu/dp_add.c b/arch/mips/math-emu/dp_add.c
index c57c8ad..7f64577 100644
--- a/arch/mips/math-emu/dp_add.c
+++ b/arch/mips/math-emu/dp_add.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,24 +16,22 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
- *
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754dp.h"
-ieee754dp ieee754dp_add(ieee754dp x, ieee754dp y)
+union ieee754dp ieee754dp_add(union ieee754dp x, union ieee754dp y)
{
+ int s;
+
COMPXDP;
COMPYDP;
EXPLODEXDP;
EXPLODEYDP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXDP;
FLUSHYDP;
@@ -52,8 +48,8 @@
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_nanxcpt(ieee754dp_indef(), "add", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_nanxcpt(ieee754dp_indef());
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN):
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN):
@@ -69,14 +65,14 @@
return x;
- /* Infinity handling
- */
-
+ /*
+ * Infinity handling
+ */
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF):
if (xs == ys)
return x;
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_xcpt(ieee754dp_indef(), "add", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_indef();
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF):
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF):
@@ -88,15 +84,14 @@
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM):
return x;
- /* Zero handling
- */
-
+ /*
+ * Zero handling
+ */
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO):
if (xs == ys)
return x;
else
- return ieee754dp_zero(ieee754_csr.rm ==
- IEEE754_RD);
+ return ieee754dp_zero(ieee754_csr.rm == FPU_CSR_RD);
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO):
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO):
@@ -125,20 +120,24 @@
assert(xm & DP_HIDDEN_BIT);
assert(ym & DP_HIDDEN_BIT);
- /* provide guard,round and stick bit space */
+ /*
+ * Provide guard,round and stick bit space.
+ */
xm <<= 3;
ym <<= 3;
if (xe > ye) {
- /* have to shift y fraction right to align
+ /*
+ * Have to shift y fraction right to align.
*/
- int s = xe - ye;
+ s = xe - ye;
ym = XDPSRS(ym, s);
ye += s;
} else if (ye > xe) {
- /* have to shift x fraction right to align
+ /*
+ * Have to shift x fraction right to align.
*/
- int s = ye - xe;
+ s = ye - xe;
xm = XDPSRS(xm, s);
xe += s;
}
@@ -146,14 +145,15 @@
assert(xe <= DP_EMAX);
if (xs == ys) {
- /* generate 28 bit result of adding two 27 bit numbers
- * leaving result in xm,xs,xe
+ /*
+ * Generate 28 bit result of adding two 27 bit numbers
+ * leaving result in xm, xs and xe.
*/
xm = xm + ym;
xe = xe;
xs = xs;
- if (xm >> (DP_MBITS + 1 + 3)) { /* carry out */
+ if (xm >> (DP_FBITS + 1 + 3)) { /* carry out */
xm = XDPSRS1(xm);
xe++;
}
@@ -168,15 +168,16 @@
xs = ys;
}
if (xm == 0)
- return ieee754dp_zero(ieee754_csr.rm ==
- IEEE754_RD);
+ return ieee754dp_zero(ieee754_csr.rm == FPU_CSR_RD);
- /* normalize to rounding precision */
- while ((xm >> (DP_MBITS + 3)) == 0) {
+ /*
+ * Normalize to rounding precision.
+ */
+ while ((xm >> (DP_FBITS + 3)) == 0) {
xm <<= 1;
xe--;
}
-
}
- DPNORMRET2(xs, xe, xm, "add", x, y);
+
+ return ieee754dp_format(xs, xe, xm);
}
diff --git a/arch/mips/math-emu/dp_cmp.c b/arch/mips/math-emu/dp_cmp.c
index 0f32486..30f95f6 100644
--- a/arch/mips/math-emu/dp_cmp.c
+++ b/arch/mips/math-emu/dp_cmp.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,16 +16,16 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754dp.h"
-int ieee754dp_cmp(ieee754dp x, ieee754dp y, int cmp, int sig)
+int ieee754dp_cmp(union ieee754dp x, union ieee754dp y, int cmp, int sig)
{
+ s64 vx;
+ s64 vy;
+
COMPXDP;
COMPYDP;
@@ -35,21 +33,21 @@
EXPLODEYDP;
FLUSHXDP;
FLUSHYDP;
- CLEARCX; /* Even clear inexact flag here */
+ ieee754_clearcx(); /* Even clear inexact flag here */
if (ieee754dp_isnan(x) || ieee754dp_isnan(y)) {
if (sig || xc == IEEE754_CLASS_SNAN || yc == IEEE754_CLASS_SNAN)
- SETCX(IEEE754_INVALID_OPERATION);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
if (cmp & IEEE754_CUN)
return 1;
if (cmp & (IEEE754_CLT | IEEE754_CGT)) {
- if (sig && SETANDTESTCX(IEEE754_INVALID_OPERATION))
- return ieee754si_xcpt(0, "fcmpf", x);
+ if (sig && ieee754_setandtestcx(IEEE754_INVALID_OPERATION))
+ return 0;
}
return 0;
} else {
- s64 vx = x.bits;
- s64 vy = y.bits;
+ vx = x.bits;
+ vy = y.bits;
if (vx < 0)
vx = -vx ^ DP_SIGN_BIT;
diff --git a/arch/mips/math-emu/dp_div.c b/arch/mips/math-emu/dp_div.c
index a1bce1b..bef0e55 100644
--- a/arch/mips/math-emu/dp_div.c
+++ b/arch/mips/math-emu/dp_div.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,23 +16,24 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754dp.h"
-ieee754dp ieee754dp_div(ieee754dp x, ieee754dp y)
+union ieee754dp ieee754dp_div(union ieee754dp x, union ieee754dp y)
{
+ u64 rm;
+ int re;
+ u64 bm;
+
COMPXDP;
COMPYDP;
EXPLODEXDP;
EXPLODEYDP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXDP;
FLUSHYDP;
@@ -51,8 +50,8 @@
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_nanxcpt(ieee754dp_indef(), "div", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_nanxcpt(ieee754dp_indef());
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN):
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN):
@@ -68,12 +67,12 @@
return x;
- /* Infinity handling
- */
-
+ /*
+ * Infinity handling
+ */
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_xcpt(ieee754dp_indef(), "div", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_indef();
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF):
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF):
@@ -85,17 +84,17 @@
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM):
return ieee754dp_inf(xs ^ ys);
- /* Zero handling
- */
-
+ /*
+ * Zero handling
+ */
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_xcpt(ieee754dp_indef(), "div", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_indef();
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO):
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO):
- SETCX(IEEE754_ZERO_DIVIDE);
- return ieee754dp_xcpt(ieee754dp_inf(xs ^ ys), "div", x, y);
+ ieee754_setcx(IEEE754_ZERO_DIVIDE);
+ return ieee754dp_inf(xs ^ ys);
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM):
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM):
@@ -122,35 +121,34 @@
xm <<= 3;
ym <<= 3;
- {
- /* now the dirty work */
+ /* now the dirty work */
- u64 rm = 0;
- int re = xe - ye;
- u64 bm;
+ rm = 0;
+ re = xe - ye;
- for (bm = DP_MBIT(DP_MBITS + 2); bm; bm >>= 1) {
- if (xm >= ym) {
- xm -= ym;
- rm |= bm;
- if (xm == 0)
- break;
- }
- xm <<= 1;
+ for (bm = DP_MBIT(DP_FBITS + 2); bm; bm >>= 1) {
+ if (xm >= ym) {
+ xm -= ym;
+ rm |= bm;
+ if (xm == 0)
+ break;
}
- rm <<= 1;
- if (xm)
- rm |= 1; /* have remainder, set sticky */
-
- assert(rm);
-
- /* normalise rm to rounding precision ?
- */
- while ((rm >> (DP_MBITS + 3)) == 0) {
- rm <<= 1;
- re--;
- }
-
- DPNORMRET2(xs == ys ? 0 : 1, re, rm, "div", x, y);
+ xm <<= 1;
}
+
+ rm <<= 1;
+ if (xm)
+ rm |= 1; /* have remainder, set sticky */
+
+ assert(rm);
+
+ /*
+ * Normalise rm to rounding precision ?
+ */
+ while ((rm >> (DP_FBITS + 3)) == 0) {
+ rm <<= 1;
+ re--;
+ }
+
+ return ieee754dp_format(xs == ys ? 0 : 1, re, rm);
}
diff --git a/arch/mips/math-emu/dp_fint.c b/arch/mips/math-emu/dp_fint.c
index 8857128..10258f0 100644
--- a/arch/mips/math-emu/dp_fint.c
+++ b/arch/mips/math-emu/dp_fint.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,21 +16,18 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754dp.h"
-ieee754dp ieee754dp_fint(int x)
+union ieee754dp ieee754dp_fint(int x)
{
u64 xm;
int xe;
int xs;
- CLEARCX;
+ ieee754_clearcx();
if (x == 0)
return ieee754dp_zero(0);
@@ -51,29 +46,11 @@
xm = x;
}
-#if 1
/* normalize - result can never be inexact or overflow */
- xe = DP_MBITS;
- while ((xm >> DP_MBITS) == 0) {
+ xe = DP_FBITS;
+ while ((xm >> DP_FBITS) == 0) {
xm <<= 1;
xe--;
}
return builddp(xs, xe + DP_EBIAS, xm & ~DP_HIDDEN_BIT);
-#else
- /* normalize */
- xe = DP_MBITS + 3;
- while ((xm >> (DP_MBITS + 3)) == 0) {
- xm <<= 1;
- xe--;
- }
- DPNORMRET1(xs, xe, xm, "fint", x);
-#endif
-}
-
-ieee754dp ieee754dp_funs(unsigned int u)
-{
- if ((int) u < 0)
- return ieee754dp_add(ieee754dp_1e31(),
- ieee754dp_fint(u & ~(1 << 31)));
- return ieee754dp_fint(u);
}
diff --git a/arch/mips/math-emu/dp_flong.c b/arch/mips/math-emu/dp_flong.c
index 14fc01e..a267c2e 100644
--- a/arch/mips/math-emu/dp_flong.c
+++ b/arch/mips/math-emu/dp_flong.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,21 +16,18 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754dp.h"
-ieee754dp ieee754dp_flong(s64 x)
+union ieee754dp ieee754dp_flong(s64 x)
{
u64 xm;
int xe;
int xs;
- CLEARCX;
+ ieee754_clearcx();
if (x == 0)
return ieee754dp_zero(0);
@@ -52,26 +47,19 @@
}
/* normalize */
- xe = DP_MBITS + 3;
- if (xm >> (DP_MBITS + 1 + 3)) {
+ xe = DP_FBITS + 3;
+ if (xm >> (DP_FBITS + 1 + 3)) {
/* shunt out overflow bits */
- while (xm >> (DP_MBITS + 1 + 3)) {
+ while (xm >> (DP_FBITS + 1 + 3)) {
XDPSRSX1();
}
} else {
/* normalize in grs extended double precision */
- while ((xm >> (DP_MBITS + 3)) == 0) {
+ while ((xm >> (DP_FBITS + 3)) == 0) {
xm <<= 1;
xe--;
}
}
- DPNORMRET1(xs, xe, xm, "dp_flong", x);
-}
-ieee754dp ieee754dp_fulong(u64 u)
-{
- if ((s64) u < 0)
- return ieee754dp_add(ieee754dp_1e63(),
- ieee754dp_flong(u & ~(1ULL << 63)));
- return ieee754dp_flong(u);
+ return ieee754dp_format(xs, xe, xm);
}
diff --git a/arch/mips/math-emu/dp_frexp.c b/arch/mips/math-emu/dp_frexp.c
deleted file mode 100644
index cb15a5e..0000000
--- a/arch/mips/math-emu/dp_frexp.c
+++ /dev/null
@@ -1,52 +0,0 @@
-/* IEEE754 floating point arithmetic
- * double precision: common utilities
- */
-/*
- * MIPS floating point support
- * Copyright (C) 1994-2000 Algorithmics Ltd.
- *
- * ########################################################################
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License (Version 2) as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
- */
-
-
-#include "ieee754dp.h"
-
-/* close to ieeep754dp_logb
-*/
-ieee754dp ieee754dp_frexp(ieee754dp x, int *eptr)
-{
- COMPXDP;
- CLEARCX;
- EXPLODEXDP;
-
- switch (xc) {
- case IEEE754_CLASS_SNAN:
- case IEEE754_CLASS_QNAN:
- case IEEE754_CLASS_INF:
- case IEEE754_CLASS_ZERO:
- *eptr = 0;
- return x;
- case IEEE754_CLASS_DNORM:
- DPDNORMX;
- break;
- case IEEE754_CLASS_NORM:
- break;
- }
- *eptr = xe + 1;
- return builddp(xs, -1 + DP_EBIAS, xm & ~DP_HIDDEN_BIT);
-}
diff --git a/arch/mips/math-emu/dp_fsp.c b/arch/mips/math-emu/dp_fsp.c
index daed683..ffb69c5 100644
--- a/arch/mips/math-emu/dp_fsp.c
+++ b/arch/mips/math-emu/dp_fsp.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,56 +16,58 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
+#include "ieee754sp.h"
#include "ieee754dp.h"
-ieee754dp ieee754dp_fsp(ieee754sp x)
+union ieee754dp ieee754dp_fsp(union ieee754sp x)
{
COMPXSP;
EXPLODEXSP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXSP;
switch (xc) {
case IEEE754_CLASS_SNAN:
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_nanxcpt(ieee754dp_indef(), "fsp");
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_nanxcpt(ieee754dp_indef());
+
case IEEE754_CLASS_QNAN:
return ieee754dp_nanxcpt(builddp(xs,
DP_EMAX + 1 + DP_EBIAS,
((u64) xm
- << (DP_MBITS -
- SP_MBITS))), "fsp",
- x);
+ << (DP_FBITS -
+ SP_FBITS))));
case IEEE754_CLASS_INF:
return ieee754dp_inf(xs);
+
case IEEE754_CLASS_ZERO:
return ieee754dp_zero(xs);
+
case IEEE754_CLASS_DNORM:
/* normalize */
- while ((xm >> SP_MBITS) == 0) {
+ while ((xm >> SP_FBITS) == 0) {
xm <<= 1;
xe--;
}
break;
+
case IEEE754_CLASS_NORM:
break;
}
- /* CAN'T possibly overflow,underflow, or need rounding
+ /*
+ * Can't possibly overflow,underflow, or need rounding
*/
/* drop the hidden bit */
xm &= ~SP_HIDDEN_BIT;
return builddp(xs, xe + DP_EBIAS,
- (u64) xm << (DP_MBITS - SP_MBITS));
+ (u64) xm << (DP_FBITS - SP_FBITS));
}
diff --git a/arch/mips/math-emu/dp_logb.c b/arch/mips/math-emu/dp_logb.c
deleted file mode 100644
index 151127e..0000000
--- a/arch/mips/math-emu/dp_logb.c
+++ /dev/null
@@ -1,53 +0,0 @@
-/* IEEE754 floating point arithmetic
- * double precision: common utilities
- */
-/*
- * MIPS floating point support
- * Copyright (C) 1994-2000 Algorithmics Ltd.
- *
- * ########################################################################
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License (Version 2) as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
- */
-
-
-#include "ieee754dp.h"
-
-ieee754dp ieee754dp_logb(ieee754dp x)
-{
- COMPXDP;
-
- CLEARCX;
-
- EXPLODEXDP;
-
- switch (xc) {
- case IEEE754_CLASS_SNAN:
- return ieee754dp_nanxcpt(x, "logb", x);
- case IEEE754_CLASS_QNAN:
- return x;
- case IEEE754_CLASS_INF:
- return ieee754dp_inf(0);
- case IEEE754_CLASS_ZERO:
- return ieee754dp_inf(1);
- case IEEE754_CLASS_DNORM:
- DPDNORMX;
- break;
- case IEEE754_CLASS_NORM:
- break;
- }
- return ieee754dp_fint(xe);
-}
diff --git a/arch/mips/math-emu/dp_modf.c b/arch/mips/math-emu/dp_modf.c
deleted file mode 100644
index b01f9cf..0000000
--- a/arch/mips/math-emu/dp_modf.c
+++ /dev/null
@@ -1,79 +0,0 @@
-/* IEEE754 floating point arithmetic
- * double precision: common utilities
- */
-/*
- * MIPS floating point support
- * Copyright (C) 1994-2000 Algorithmics Ltd.
- *
- * ########################################################################
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License (Version 2) as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
- */
-
-
-#include "ieee754dp.h"
-
-/* modf function is always exact for a finite number
-*/
-ieee754dp ieee754dp_modf(ieee754dp x, ieee754dp *ip)
-{
- COMPXDP;
-
- CLEARCX;
-
- EXPLODEXDP;
-
- switch (xc) {
- case IEEE754_CLASS_SNAN:
- case IEEE754_CLASS_QNAN:
- case IEEE754_CLASS_INF:
- case IEEE754_CLASS_ZERO:
- *ip = x;
- return x;
- case IEEE754_CLASS_DNORM:
- /* far to small */
- *ip = ieee754dp_zero(xs);
- return x;
- case IEEE754_CLASS_NORM:
- break;
- }
- if (xe < 0) {
- *ip = ieee754dp_zero(xs);
- return x;
- }
- if (xe >= DP_MBITS) {
- *ip = x;
- return ieee754dp_zero(xs);
- }
- /* generate ipart mantissa by clearing bottom bits
- */
- *ip = builddp(xs, xe + DP_EBIAS,
- ((xm >> (DP_MBITS - xe)) << (DP_MBITS - xe)) &
- ~DP_HIDDEN_BIT);
-
- /* generate fpart mantissa by clearing top bits
- * and normalizing (must be able to normalize)
- */
- xm = (xm << (64 - (DP_MBITS - xe))) >> (64 - (DP_MBITS - xe));
- if (xm == 0)
- return ieee754dp_zero(xs);
-
- while ((xm >> DP_MBITS) == 0) {
- xm <<= 1;
- xe--;
- }
- return builddp(xs, xe + DP_EBIAS, xm & ~DP_HIDDEN_BIT);
-}
diff --git a/arch/mips/math-emu/dp_mul.c b/arch/mips/math-emu/dp_mul.c
index 09175f4..d3acded 100644
--- a/arch/mips/math-emu/dp_mul.c
+++ b/arch/mips/math-emu/dp_mul.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,23 +16,32 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754dp.h"
-ieee754dp ieee754dp_mul(ieee754dp x, ieee754dp y)
+union ieee754dp ieee754dp_mul(union ieee754dp x, union ieee754dp y)
{
+ int re;
+ int rs;
+ u64 rm;
+ unsigned lxm;
+ unsigned hxm;
+ unsigned lym;
+ unsigned hym;
+ u64 lrm;
+ u64 hrm;
+ u64 t;
+ u64 at;
+
COMPXDP;
COMPYDP;
EXPLODEXDP;
EXPLODEYDP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXDP;
FLUSHYDP;
@@ -51,8 +58,8 @@
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_nanxcpt(ieee754dp_indef(), "mul", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_nanxcpt(ieee754dp_indef());
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN):
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN):
@@ -68,12 +75,13 @@
return x;
- /* Infinity handling */
-
+ /*
+ * Infinity handling
+ */
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO):
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_xcpt(ieee754dp_indef(), "mul", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_indef();
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF):
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF):
@@ -107,70 +115,59 @@
/* rm = xm * ym, re = xe+ye basically */
assert(xm & DP_HIDDEN_BIT);
assert(ym & DP_HIDDEN_BIT);
- {
- int re = xe + ye;
- int rs = xs ^ ys;
- u64 rm;
- /* shunt to top of word */
- xm <<= 64 - (DP_MBITS + 1);
- ym <<= 64 - (DP_MBITS + 1);
+ re = xe + ye;
+ rs = xs ^ ys;
- /* multiply 32bits xm,ym to give high 32bits rm with stickness
- */
+ /* shunt to top of word */
+ xm <<= 64 - (DP_FBITS + 1);
+ ym <<= 64 - (DP_FBITS + 1);
- /* 32 * 32 => 64 */
+ /*
+ * Multiply 32 bits xm, ym to give high 32 bits rm with stickness.
+ */
+
+ /* 32 * 32 => 64 */
#define DPXMULT(x, y) ((u64)(x) * (u64)y)
- {
- unsigned lxm = xm;
- unsigned hxm = xm >> 32;
- unsigned lym = ym;
- unsigned hym = ym >> 32;
- u64 lrm;
- u64 hrm;
+ lxm = xm;
+ hxm = xm >> 32;
+ lym = ym;
+ hym = ym >> 32;
- lrm = DPXMULT(lxm, lym);
- hrm = DPXMULT(hxm, hym);
+ lrm = DPXMULT(lxm, lym);
+ hrm = DPXMULT(hxm, hym);
- {
- u64 t = DPXMULT(lxm, hym);
- {
- u64 at =
- lrm + (t << 32);
- hrm += at < lrm;
- lrm = at;
- }
- hrm = hrm + (t >> 32);
- }
+ t = DPXMULT(lxm, hym);
- {
- u64 t = DPXMULT(hxm, lym);
- {
- u64 at =
- lrm + (t << 32);
- hrm += at < lrm;
- lrm = at;
- }
- hrm = hrm + (t >> 32);
- }
- rm = hrm | (lrm != 0);
- }
+ at = lrm + (t << 32);
+ hrm += at < lrm;
+ lrm = at;
- /*
- * sticky shift down to normal rounding precision
- */
- if ((s64) rm < 0) {
- rm =
- (rm >> (64 - (DP_MBITS + 1 + 3))) |
- ((rm << (DP_MBITS + 1 + 3)) != 0);
+ hrm = hrm + (t >> 32);
+
+ t = DPXMULT(hxm, lym);
+
+ at = lrm + (t << 32);
+ hrm += at < lrm;
+ lrm = at;
+
+ hrm = hrm + (t >> 32);
+
+ rm = hrm | (lrm != 0);
+
+ /*
+ * Sticky shift down to normal rounding precision.
+ */
+ if ((s64) rm < 0) {
+ rm = (rm >> (64 - (DP_FBITS + 1 + 3))) |
+ ((rm << (DP_FBITS + 1 + 3)) != 0);
re++;
- } else {
- rm =
- (rm >> (64 - (DP_MBITS + 1 + 3 + 1))) |
- ((rm << (DP_MBITS + 1 + 3 + 1)) != 0);
- }
- assert(rm & (DP_HIDDEN_BIT << 3));
- DPNORMRET2(rs, re, rm, "mul", x, y);
+ } else {
+ rm = (rm >> (64 - (DP_FBITS + 1 + 3 + 1))) |
+ ((rm << (DP_FBITS + 1 + 3 + 1)) != 0);
}
+ assert(rm & (DP_HIDDEN_BIT << 3));
+
+ return ieee754dp_format(rs, re, rm);
}
diff --git a/arch/mips/math-emu/dp_scalb.c b/arch/mips/math-emu/dp_scalb.c
deleted file mode 100644
index 6f5df43..0000000
--- a/arch/mips/math-emu/dp_scalb.c
+++ /dev/null
@@ -1,57 +0,0 @@
-/* IEEE754 floating point arithmetic
- * double precision: common utilities
- */
-/*
- * MIPS floating point support
- * Copyright (C) 1994-2000 Algorithmics Ltd.
- *
- * ########################################################################
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License (Version 2) as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
- */
-
-
-#include "ieee754dp.h"
-
-ieee754dp ieee754dp_scalb(ieee754dp x, int n)
-{
- COMPXDP;
-
- CLEARCX;
-
- EXPLODEXDP;
-
- switch (xc) {
- case IEEE754_CLASS_SNAN:
- return ieee754dp_nanxcpt(x, "scalb", x, n);
- case IEEE754_CLASS_QNAN:
- case IEEE754_CLASS_INF:
- case IEEE754_CLASS_ZERO:
- return x;
- case IEEE754_CLASS_DNORM:
- DPDNORMX;
- break;
- case IEEE754_CLASS_NORM:
- break;
- }
- DPNORMRET2(xs, xe + n, xm << 3, "scalb", x, n);
-}
-
-
-ieee754dp ieee754dp_ldexp(ieee754dp x, int n)
-{
- return ieee754dp_scalb(x, n);
-}
diff --git a/arch/mips/math-emu/dp_simple.c b/arch/mips/math-emu/dp_simple.c
index 79ce267..bccbe90 100644
--- a/arch/mips/math-emu/dp_simple.c
+++ b/arch/mips/math-emu/dp_simple.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,33 +16,17 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754dp.h"
-int ieee754dp_finite(ieee754dp x)
-{
- return DPBEXP(x) != DP_EMAX + 1 + DP_EBIAS;
-}
-
-ieee754dp ieee754dp_copysign(ieee754dp x, ieee754dp y)
-{
- CLEARCX;
- DPSIGN(x) = DPSIGN(y);
- return x;
-}
-
-
-ieee754dp ieee754dp_neg(ieee754dp x)
+union ieee754dp ieee754dp_neg(union ieee754dp x)
{
COMPXDP;
EXPLODEXDP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXDP;
/*
@@ -55,30 +37,29 @@
DPSIGN(x) ^= 1;
if (xc == IEEE754_CLASS_SNAN) {
- ieee754dp y = ieee754dp_indef();
- SETCX(IEEE754_INVALID_OPERATION);
+ union ieee754dp y = ieee754dp_indef();
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
DPSIGN(y) = DPSIGN(x);
- return ieee754dp_nanxcpt(y, "neg");
+ return ieee754dp_nanxcpt(y);
}
return x;
}
-
-ieee754dp ieee754dp_abs(ieee754dp x)
+union ieee754dp ieee754dp_abs(union ieee754dp x)
{
COMPXDP;
EXPLODEXDP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXDP;
/* Clear sign ALWAYS, irrespective of NaN */
DPSIGN(x) = 0;
if (xc == IEEE754_CLASS_SNAN) {
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_nanxcpt(ieee754dp_indef(), "abs");
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_nanxcpt(ieee754dp_indef());
}
return x;
diff --git a/arch/mips/math-emu/dp_sqrt.c b/arch/mips/math-emu/dp_sqrt.c
index b874d60..041bbb61 100644
--- a/arch/mips/math-emu/dp_sqrt.c
+++ b/arch/mips/math-emu/dp_sqrt.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,12 +16,9 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754dp.h"
static const unsigned table[] = {
@@ -34,44 +29,49 @@
1742, 661, 130
};
-ieee754dp ieee754dp_sqrt(ieee754dp x)
+union ieee754dp ieee754dp_sqrt(union ieee754dp x)
{
struct _ieee754_csr oldcsr;
- ieee754dp y, z, t;
+ union ieee754dp y, z, t;
unsigned scalx, yh;
COMPXDP;
EXPLODEXDP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXDP;
/* x == INF or NAN? */
switch (xc) {
case IEEE754_CLASS_QNAN:
/* sqrt(Nan) = Nan */
- return ieee754dp_nanxcpt(x, "sqrt");
+ return ieee754dp_nanxcpt(x);
+
case IEEE754_CLASS_SNAN:
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_nanxcpt(ieee754dp_indef(), "sqrt");
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_nanxcpt(ieee754dp_indef());
+
case IEEE754_CLASS_ZERO:
/* sqrt(0) = 0 */
return x;
+
case IEEE754_CLASS_INF:
if (xs) {
/* sqrt(-Inf) = Nan */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_nanxcpt(ieee754dp_indef(), "sqrt");
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_nanxcpt(ieee754dp_indef());
}
/* sqrt(+Inf) = Inf */
return x;
+
case IEEE754_CLASS_DNORM:
DPDNORMX;
/* fall through */
+
case IEEE754_CLASS_NORM:
if (xs) {
/* sqrt(-x) = Nan */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_nanxcpt(ieee754dp_indef(), "sqrt");
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_nanxcpt(ieee754dp_indef());
}
break;
}
@@ -80,7 +80,7 @@
oldcsr = ieee754_csr;
ieee754_csr.mx &= ~IEEE754_INEXACT;
ieee754_csr.sx &= ~IEEE754_INEXACT;
- ieee754_csr.rm = IEEE754_RN;
+ ieee754_csr.rm = FPU_CSR_RN;
/* adjust exponent to prevent overflow */
scalx = 0;
@@ -110,19 +110,19 @@
/* triple to almost 56 sig. bits: y ~= sqrt(x) to within 1 ulp */
/* t=y*y; z=t; pt[n0]+=0x00100000; t+=z; z=(x-z)*y; */
z = t = ieee754dp_mul(y, y);
- t.parts.bexp += 0x001;
+ t.bexp += 0x001;
t = ieee754dp_add(t, z);
z = ieee754dp_mul(ieee754dp_sub(x, z), y);
/* t=z/(t+x) ; pt[n0]+=0x00100000; y+=t; */
t = ieee754dp_div(z, ieee754dp_add(t, x));
- t.parts.bexp += 0x001;
+ t.bexp += 0x001;
y = ieee754dp_add(y, t);
/* twiddle last bit to force y correctly rounded */
/* set RZ, clear INEX flag */
- ieee754_csr.rm = IEEE754_RZ;
+ ieee754_csr.rm = FPU_CSR_RZ;
ieee754_csr.sx &= ~IEEE754_INEXACT;
/* t=x/y; ...chopped quotient, possibly inexact */
@@ -139,10 +139,10 @@
oldcsr.sx |= IEEE754_INEXACT;
switch (oldcsr.rm) {
- case IEEE754_RP:
+ case FPU_CSR_RU:
y.bits += 1;
/* drop through */
- case IEEE754_RN:
+ case FPU_CSR_RN:
t.bits += 1;
break;
}
@@ -155,7 +155,7 @@
}
/* py[n0]=py[n0]+scalx; ...scale back y */
- y.parts.bexp += scalx;
+ y.bexp += scalx;
/* restore rounding mode, possibly set inexact */
ieee754_csr = oldcsr;
diff --git a/arch/mips/math-emu/dp_sub.c b/arch/mips/math-emu/dp_sub.c
index 91e0a4b..7a17402 100644
--- a/arch/mips/math-emu/dp_sub.c
+++ b/arch/mips/math-emu/dp_sub.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,23 +16,22 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754dp.h"
-ieee754dp ieee754dp_sub(ieee754dp x, ieee754dp y)
+union ieee754dp ieee754dp_sub(union ieee754dp x, union ieee754dp y)
{
+ int s;
+
COMPXDP;
COMPYDP;
EXPLODEXDP;
EXPLODEYDP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXDP;
FLUSHYDP;
@@ -51,8 +48,8 @@
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_nanxcpt(ieee754dp_indef(), "sub", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_nanxcpt(ieee754dp_indef());
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN):
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN):
@@ -68,14 +65,14 @@
return x;
- /* Infinity handling
- */
-
+ /*
+ * Infinity handling
+ */
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF):
if (xs != ys)
return x;
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754dp_xcpt(ieee754dp_indef(), "sub", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754dp_indef();
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF):
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF):
@@ -87,15 +84,14 @@
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM):
return x;
- /* Zero handling
- */
-
+ /*
+ * Zero handling
+ */
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO):
if (xs != ys)
return x;
else
- return ieee754dp_zero(ieee754_csr.rm ==
- IEEE754_RD);
+ return ieee754dp_zero(ieee754_csr.rm == FPU_CSR_RD);
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO):
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO):
@@ -136,15 +132,17 @@
ym <<= 3;
if (xe > ye) {
- /* have to shift y fraction right to align
+ /*
+ * Have to shift y fraction right to align
*/
- int s = xe - ye;
+ s = xe - ye;
ym = XDPSRS(ym, s);
ye += s;
} else if (ye > xe) {
- /* have to shift x fraction right to align
+ /*
+ * Have to shift x fraction right to align
*/
- int s = ye - xe;
+ s = ye - xe;
xm = XDPSRS(xm, s);
xe += s;
}
@@ -158,7 +156,7 @@
xe = xe;
xs = xs;
- if (xm >> (DP_MBITS + 1 + 3)) { /* carry out */
+ if (xm >> (DP_FBITS + 1 + 3)) { /* carry out */
xm = XDPSRS1(xm); /* shift preserving sticky */
xe++;
}
@@ -173,7 +171,7 @@
xs = ys;
}
if (xm == 0) {
- if (ieee754_csr.rm == IEEE754_RD)
+ if (ieee754_csr.rm == FPU_CSR_RD)
return ieee754dp_zero(1); /* round negative inf. => sign = -1 */
else
return ieee754dp_zero(0); /* other round modes => sign = 1 */
@@ -181,10 +179,11 @@
/* normalize to rounding precision
*/
- while ((xm >> (DP_MBITS + 3)) == 0) {
+ while ((xm >> (DP_FBITS + 3)) == 0) {
xm <<= 1;
xe--;
}
}
- DPNORMRET2(xs, xe, xm, "sub", x, y);
+
+ return ieee754dp_format(xs, xe, xm);
}
diff --git a/arch/mips/math-emu/dp_tint.c b/arch/mips/math-emu/dp_tint.c
index 0ebe859..6ffc336 100644
--- a/arch/mips/math-emu/dp_tint.c
+++ b/arch/mips/math-emu/dp_tint.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,20 +16,21 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
-#include <linux/kernel.h>
#include "ieee754dp.h"
-int ieee754dp_tint(ieee754dp x)
+int ieee754dp_tint(union ieee754dp x)
{
+ u64 residue;
+ int round;
+ int sticky;
+ int odd;
+
COMPXDP;
- CLEARCX;
+ ieee754_clearcx();
EXPLODEXDP;
FLUSHXDP;
@@ -40,10 +39,12 @@
case IEEE754_CLASS_SNAN:
case IEEE754_CLASS_QNAN:
case IEEE754_CLASS_INF:
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754si_xcpt(ieee754si_indef(), "dp_tint", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754si_indef();
+
case IEEE754_CLASS_ZERO:
return 0;
+
case IEEE754_CLASS_DNORM:
case IEEE754_CLASS_NORM:
break;
@@ -51,44 +52,39 @@
if (xe > 31) {
/* Set invalid. We will only use overflow for floating
point overflow */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754si_xcpt(ieee754si_indef(), "dp_tint", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754si_indef();
}
/* oh gawd */
- if (xe > DP_MBITS) {
- xm <<= xe - DP_MBITS;
- } else if (xe < DP_MBITS) {
- u64 residue;
- int round;
- int sticky;
- int odd;
-
+ if (xe > DP_FBITS) {
+ xm <<= xe - DP_FBITS;
+ } else if (xe < DP_FBITS) {
if (xe < -1) {
residue = xm;
round = 0;
sticky = residue != 0;
xm = 0;
} else {
- residue = xm << (64 - DP_MBITS + xe);
+ residue = xm << (64 - DP_FBITS + xe);
round = (residue >> 63) != 0;
sticky = (residue << 1) != 0;
- xm >>= DP_MBITS - xe;
+ xm >>= DP_FBITS - xe;
}
/* Note: At this point upper 32 bits of xm are guaranteed
to be zero */
odd = (xm & 0x1) != 0x0;
switch (ieee754_csr.rm) {
- case IEEE754_RN:
+ case FPU_CSR_RN:
if (round && (sticky || odd))
xm++;
break;
- case IEEE754_RZ:
+ case FPU_CSR_RZ:
break;
- case IEEE754_RU: /* toward +Infinity */
+ case FPU_CSR_RU: /* toward +Infinity */
if ((round || sticky) && !xs)
xm++;
break;
- case IEEE754_RD: /* toward -Infinity */
+ case FPU_CSR_RD: /* toward -Infinity */
if ((round || sticky) && xs)
xm++;
break;
@@ -96,27 +92,14 @@
/* look for valid corner case 0x80000000 */
if ((xm >> 31) != 0 && (xs == 0 || xm != 0x80000000)) {
/* This can happen after rounding */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754si_xcpt(ieee754si_indef(), "dp_tint", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754si_indef();
}
if (round || sticky)
- SETCX(IEEE754_INEXACT);
+ ieee754_setcx(IEEE754_INEXACT);
}
if (xs)
return -xm;
else
return xm;
}
-
-
-unsigned int ieee754dp_tuns(ieee754dp x)
-{
- ieee754dp hb = ieee754dp_1e31();
-
- /* what if x < 0 ?? */
- if (ieee754dp_lt(x, hb))
- return (unsigned) ieee754dp_tint(x);
-
- return (unsigned) ieee754dp_tint(ieee754dp_sub(x, hb)) |
- ((unsigned) 1 << 31);
-}
diff --git a/arch/mips/math-emu/dp_tlong.c b/arch/mips/math-emu/dp_tlong.c
index 133ce2b..9cdc145 100644
--- a/arch/mips/math-emu/dp_tlong.c
+++ b/arch/mips/math-emu/dp_tlong.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,19 +16,21 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754dp.h"
-s64 ieee754dp_tlong(ieee754dp x)
+s64 ieee754dp_tlong(union ieee754dp x)
{
+ u64 residue;
+ int round;
+ int sticky;
+ int odd;
+
COMPXDP;
- CLEARCX;
+ ieee754_clearcx();
EXPLODEXDP;
FLUSHXDP;
@@ -39,10 +39,12 @@
case IEEE754_CLASS_SNAN:
case IEEE754_CLASS_QNAN:
case IEEE754_CLASS_INF:
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754di_xcpt(ieee754di_indef(), "dp_tlong", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754di_indef();
+
case IEEE754_CLASS_ZERO:
return 0;
+
case IEEE754_CLASS_DNORM:
case IEEE754_CLASS_NORM:
break;
@@ -53,18 +55,13 @@
return -0x8000000000000000LL;
/* Set invalid. We will only use overflow for floating
point overflow */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754di_xcpt(ieee754di_indef(), "dp_tlong", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754di_indef();
}
/* oh gawd */
- if (xe > DP_MBITS) {
- xm <<= xe - DP_MBITS;
- } else if (xe < DP_MBITS) {
- u64 residue;
- int round;
- int sticky;
- int odd;
-
+ if (xe > DP_FBITS) {
+ xm <<= xe - DP_FBITS;
+ } else if (xe < DP_FBITS) {
if (xe < -1) {
residue = xm;
round = 0;
@@ -75,51 +72,38 @@
* so we do it in two steps. Be aware that xe
* may be -1 */
residue = xm << (xe + 1);
- residue <<= 63 - DP_MBITS;
+ residue <<= 63 - DP_FBITS;
round = (residue >> 63) != 0;
sticky = (residue << 1) != 0;
- xm >>= DP_MBITS - xe;
+ xm >>= DP_FBITS - xe;
}
odd = (xm & 0x1) != 0x0;
switch (ieee754_csr.rm) {
- case IEEE754_RN:
+ case FPU_CSR_RN:
if (round && (sticky || odd))
xm++;
break;
- case IEEE754_RZ:
+ case FPU_CSR_RZ:
break;
- case IEEE754_RU: /* toward +Infinity */
+ case FPU_CSR_RU: /* toward +Infinity */
if ((round || sticky) && !xs)
xm++;
break;
- case IEEE754_RD: /* toward -Infinity */
+ case FPU_CSR_RD: /* toward -Infinity */
if ((round || sticky) && xs)
xm++;
break;
}
if ((xm >> 63) != 0) {
/* This can happen after rounding */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754di_xcpt(ieee754di_indef(), "dp_tlong", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754di_indef();
}
if (round || sticky)
- SETCX(IEEE754_INEXACT);
+ ieee754_setcx(IEEE754_INEXACT);
}
if (xs)
return -xm;
else
return xm;
}
-
-
-u64 ieee754dp_tulong(ieee754dp x)
-{
- ieee754dp hb = ieee754dp_1e63();
-
- /* what if x < 0 ?? */
- if (ieee754dp_lt(x, hb))
- return (u64) ieee754dp_tlong(x);
-
- return (u64) ieee754dp_tlong(ieee754dp_sub(x, hb)) |
- (1ULL << 63);
-}
diff --git a/arch/mips/math-emu/dsemul.c b/arch/mips/math-emu/dsemul.c
index 7ea622a..4f514f3 100644
--- a/arch/mips/math-emu/dsemul.c
+++ b/arch/mips/math-emu/dsemul.c
@@ -1,30 +1,12 @@
-#include <linux/compiler.h>
-#include <linux/mm.h>
-#include <linux/signal.h>
-#include <linux/smp.h>
-
-#include <asm/asm.h>
-#include <asm/bootinfo.h>
-#include <asm/byteorder.h>
-#include <asm/cpu.h>
-#include <asm/inst.h>
-#include <asm/processor.h>
-#include <asm/uaccess.h>
#include <asm/branch.h>
-#include <asm/mipsregs.h>
#include <asm/cacheflush.h>
-
#include <asm/fpu_emulator.h>
+#include <asm/inst.h>
+#include <asm/mipsregs.h>
+#include <asm/uaccess.h>
#include "ieee754.h"
-/* Strap kernel emulator for full MIPS IV emulation */
-
-#ifdef __mips
-#undef __mips
-#endif
-#define __mips 4
-
/*
* Emulate the arbritrary instruction ir at xcp->cp0_epc. Required when
* we have to emulate the instruction in a COP1 branch delay slot. Do
@@ -59,13 +41,11 @@
(ir == 0)) {
/* NOP is easy */
regs->cp0_epc = cpc;
- regs->cp0_cause &= ~CAUSEF_BD;
+ clear_delay_slot(regs);
return 0;
}
-#ifdef DSEMUL_TRACE
- printk("dsemul %lx %lx\n", regs->cp0_epc, cpc);
-#endif
+ pr_debug("dsemul %lx %lx\n", regs->cp0_epc, cpc);
/*
* The strategy is to push the instruction onto the user stack
@@ -167,9 +147,8 @@
* emulating the branch delay instruction.
*/
-#ifdef DSEMUL_TRACE
- printk("dsemulret\n");
-#endif
+ pr_debug("dsemulret\n");
+
if (__get_user(epc, &fr->epc)) { /* Saved EPC */
/* This is not a good situation to be in */
force_sig(SIGBUS, current);
diff --git a/arch/mips/math-emu/ieee754.c b/arch/mips/math-emu/ieee754.c
index 0015cf1..53f1d22 100644
--- a/arch/mips/math-emu/ieee754.c
+++ b/arch/mips/math-emu/ieee754.c
@@ -10,8 +10,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -23,105 +21,69 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
+#include <linux/compiler.h>
-#include "ieee754int.h"
+#include "ieee754.h"
#include "ieee754sp.h"
#include "ieee754dp.h"
-#define DP_EBIAS 1023
-#define DP_EMIN (-1022)
-#define DP_EMAX 1023
+/*
+ * Special constants
+ */
-#define SP_EBIAS 127
-#define SP_EMIN (-126)
-#define SP_EMAX 127
-
-/* special constants
-*/
-
-
-#if (defined(BYTE_ORDER) && BYTE_ORDER == LITTLE_ENDIAN) || defined(__MIPSEL__)
-#define SPSTR(s, b, m) {m, b, s}
-#define DPSTR(s, b, mh, ml) {ml, mh, b, s}
-#endif
-
-#ifdef __MIPSEB__
-#define SPSTR(s, b, m) {s, b, m}
-#define DPSTR(s, b, mh, ml) {s, b, mh, ml}
-#endif
-
-const struct ieee754dp_konst __ieee754dp_spcvals[] = {
- DPSTR(0, DP_EMIN - 1 + DP_EBIAS, 0, 0), /* + zero */
- DPSTR(1, DP_EMIN - 1 + DP_EBIAS, 0, 0), /* - zero */
- DPSTR(0, DP_EBIAS, 0, 0), /* + 1.0 */
- DPSTR(1, DP_EBIAS, 0, 0), /* - 1.0 */
- DPSTR(0, 3 + DP_EBIAS, 0x40000, 0), /* + 10.0 */
- DPSTR(1, 3 + DP_EBIAS, 0x40000, 0), /* - 10.0 */
- DPSTR(0, DP_EMAX + 1 + DP_EBIAS, 0, 0), /* + infinity */
- DPSTR(1, DP_EMAX + 1 + DP_EBIAS, 0, 0), /* - infinity */
- DPSTR(0, DP_EMAX+1+DP_EBIAS, 0x7FFFF, 0xFFFFFFFF), /* + indef quiet Nan */
- DPSTR(0, DP_EMAX + DP_EBIAS, 0xFFFFF, 0xFFFFFFFF), /* + max */
- DPSTR(1, DP_EMAX + DP_EBIAS, 0xFFFFF, 0xFFFFFFFF), /* - max */
- DPSTR(0, DP_EMIN + DP_EBIAS, 0, 0), /* + min normal */
- DPSTR(1, DP_EMIN + DP_EBIAS, 0, 0), /* - min normal */
- DPSTR(0, DP_EMIN - 1 + DP_EBIAS, 0, 1), /* + min denormal */
- DPSTR(1, DP_EMIN - 1 + DP_EBIAS, 0, 1), /* - min denormal */
- DPSTR(0, 31 + DP_EBIAS, 0, 0), /* + 1.0e31 */
- DPSTR(0, 63 + DP_EBIAS, 0, 0), /* + 1.0e63 */
-};
-
-const struct ieee754sp_konst __ieee754sp_spcvals[] = {
- SPSTR(0, SP_EMIN - 1 + SP_EBIAS, 0), /* + zero */
- SPSTR(1, SP_EMIN - 1 + SP_EBIAS, 0), /* - zero */
- SPSTR(0, SP_EBIAS, 0), /* + 1.0 */
- SPSTR(1, SP_EBIAS, 0), /* - 1.0 */
- SPSTR(0, 3 + SP_EBIAS, 0x200000), /* + 10.0 */
- SPSTR(1, 3 + SP_EBIAS, 0x200000), /* - 10.0 */
- SPSTR(0, SP_EMAX + 1 + SP_EBIAS, 0), /* + infinity */
- SPSTR(1, SP_EMAX + 1 + SP_EBIAS, 0), /* - infinity */
- SPSTR(0, SP_EMAX+1+SP_EBIAS, 0x3FFFFF), /* + indef quiet Nan */
- SPSTR(0, SP_EMAX + SP_EBIAS, 0x7FFFFF), /* + max normal */
- SPSTR(1, SP_EMAX + SP_EBIAS, 0x7FFFFF), /* - max normal */
- SPSTR(0, SP_EMIN + SP_EBIAS, 0), /* + min normal */
- SPSTR(1, SP_EMIN + SP_EBIAS, 0), /* - min normal */
- SPSTR(0, SP_EMIN - 1 + SP_EBIAS, 1), /* + min denormal */
- SPSTR(1, SP_EMIN - 1 + SP_EBIAS, 1), /* - min denormal */
- SPSTR(0, 31 + SP_EBIAS, 0), /* + 1.0e31 */
- SPSTR(0, 63 + SP_EBIAS, 0), /* + 1.0e63 */
-};
-
-
-int ieee754si_xcpt(int r, const char *op, ...)
-{
- struct ieee754xctx ax;
-
- if (!TSTX())
- return r;
- ax.op = op;
- ax.rt = IEEE754_RT_SI;
- ax.rv.si = r;
- va_start(ax.ap, op);
- ieee754_xcpt(&ax);
- va_end(ax.ap);
- return ax.rv.si;
+#define DPCNST(s, b, m) \
+{ \
+ .sign = (s), \
+ .bexp = (b) + DP_EBIAS, \
+ .mant = (m) \
}
-s64 ieee754di_xcpt(s64 r, const char *op, ...)
-{
- struct ieee754xctx ax;
+const union ieee754dp __ieee754dp_spcvals[] = {
+ DPCNST(0, DP_EMIN - 1, 0x0000000000000ULL), /* + zero */
+ DPCNST(1, DP_EMIN - 1, 0x0000000000000ULL), /* - zero */
+ DPCNST(0, 0, 0x0000000000000ULL), /* + 1.0 */
+ DPCNST(1, 0, 0x0000000000000ULL), /* - 1.0 */
+ DPCNST(0, 3, 0x4000000000000ULL), /* + 10.0 */
+ DPCNST(1, 3, 0x4000000000000ULL), /* - 10.0 */
+ DPCNST(0, DP_EMAX + 1, 0x0000000000000ULL), /* + infinity */
+ DPCNST(1, DP_EMAX + 1, 0x0000000000000ULL), /* - infinity */
+ DPCNST(0, DP_EMAX + 1, 0x7FFFFFFFFFFFFULL), /* + indef quiet Nan */
+ DPCNST(0, DP_EMAX, 0xFFFFFFFFFFFFFULL), /* + max */
+ DPCNST(1, DP_EMAX, 0xFFFFFFFFFFFFFULL), /* - max */
+ DPCNST(0, DP_EMIN, 0x0000000000000ULL), /* + min normal */
+ DPCNST(1, DP_EMIN, 0x0000000000000ULL), /* - min normal */
+ DPCNST(0, DP_EMIN - 1, 0x0000000000001ULL), /* + min denormal */
+ DPCNST(1, DP_EMIN - 1, 0x0000000000001ULL), /* - min denormal */
+ DPCNST(0, 31, 0x0000000000000ULL), /* + 1.0e31 */
+ DPCNST(0, 63, 0x0000000000000ULL), /* + 1.0e63 */
+};
- if (!TSTX())
- return r;
- ax.op = op;
- ax.rt = IEEE754_RT_DI;
- ax.rv.di = r;
- va_start(ax.ap, op);
- ieee754_xcpt(&ax);
- va_end(ax.ap);
- return ax.rv.di;
+#define SPCNST(s, b, m) \
+{ \
+ .sign = (s), \
+ .bexp = (b) + SP_EBIAS, \
+ .mant = (m) \
}
+
+const union ieee754sp __ieee754sp_spcvals[] = {
+ SPCNST(0, SP_EMIN - 1, 0x000000), /* + zero */
+ SPCNST(1, SP_EMIN - 1, 0x000000), /* - zero */
+ SPCNST(0, 0, 0x000000), /* + 1.0 */
+ SPCNST(1, 0, 0x000000), /* - 1.0 */
+ SPCNST(0, 3, 0x200000), /* + 10.0 */
+ SPCNST(1, 3, 0x200000), /* - 10.0 */
+ SPCNST(0, SP_EMAX + 1, 0x000000), /* + infinity */
+ SPCNST(1, SP_EMAX + 1, 0x000000), /* - infinity */
+ SPCNST(0, SP_EMAX + 1, 0x3FFFFF), /* + indef quiet Nan */
+ SPCNST(0, SP_EMAX, 0x7FFFFF), /* + max normal */
+ SPCNST(1, SP_EMAX, 0x7FFFFF), /* - max normal */
+ SPCNST(0, SP_EMIN, 0x000000), /* + min normal */
+ SPCNST(1, SP_EMIN, 0x000000), /* - min normal */
+ SPCNST(0, SP_EMIN - 1, 0x000001), /* + min denormal */
+ SPCNST(1, SP_EMIN - 1, 0x000001), /* - min denormal */
+ SPCNST(0, 31, 0x000000), /* + 1.0e31 */
+ SPCNST(0, 63, 0x000000), /* + 1.0e63 */
+};
diff --git a/arch/mips/math-emu/ieee754.h b/arch/mips/math-emu/ieee754.h
index 22796e0..43c4fb5 100644
--- a/arch/mips/math-emu/ieee754.h
+++ b/arch/mips/math-emu/ieee754.h
@@ -13,7 +13,7 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Nov 7, 2000
* Modification to allow integration with Linux kernel
@@ -24,186 +24,93 @@
#ifndef __ARCH_MIPS_MATH_EMU_IEEE754_H
#define __ARCH_MIPS_MATH_EMU_IEEE754_H
+#include <linux/compiler.h>
#include <asm/byteorder.h>
+#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/sched.h>
+#include <asm/bitfield.h>
-/*
- * Not very pretty, but the Linux kernel's normal va_list definition
- * does not allow it to be used as a structure element, as it is here.
- */
-#ifndef _STDARG_H
-#include <stdarg.h>
-#endif
-
-#ifdef __LITTLE_ENDIAN
-struct ieee754dp_konst {
- unsigned mantlo:32;
- unsigned manthi:20;
- unsigned bexp:11;
- unsigned sign:1;
-};
-struct ieee754sp_konst {
- unsigned mant:23;
- unsigned bexp:8;
- unsigned sign:1;
-};
-
-typedef union _ieee754dp {
- struct ieee754dp_konst oparts;
+union ieee754dp {
struct {
- u64 mant:52;
- unsigned int bexp:11;
- unsigned int sign:1;
- } parts;
+ __BITFIELD_FIELD(unsigned int sign:1,
+ __BITFIELD_FIELD(unsigned int bexp:11,
+ __BITFIELD_FIELD(u64 mant:52,
+ ;)))
+ };
u64 bits;
- double d;
-} ieee754dp;
-
-typedef union _ieee754sp {
- struct ieee754sp_konst parts;
- float f;
- u32 bits;
-} ieee754sp;
-#endif
-
-#ifdef __BIG_ENDIAN
-struct ieee754dp_konst {
- unsigned sign:1;
- unsigned bexp:11;
- unsigned manthi:20;
- unsigned mantlo:32;
};
-typedef union _ieee754dp {
- struct ieee754dp_konst oparts;
+union ieee754sp {
struct {
- unsigned int sign:1;
- unsigned int bexp:11;
- u64 mant:52;
- } parts;
- double d;
- u64 bits;
-} ieee754dp;
-
-struct ieee754sp_konst {
- unsigned sign:1;
- unsigned bexp:8;
- unsigned mant:23;
-};
-
-typedef union _ieee754sp {
- struct ieee754sp_konst parts;
- float f;
+ __BITFIELD_FIELD(unsigned sign:1,
+ __BITFIELD_FIELD(unsigned bexp:8,
+ __BITFIELD_FIELD(unsigned mant:23,
+ ;)))
+ };
u32 bits;
-} ieee754sp;
-#endif
+};
/*
* single precision (often aka float)
*/
-int ieee754sp_finite(ieee754sp x);
-int ieee754sp_class(ieee754sp x);
+int ieee754sp_class(union ieee754sp x);
-ieee754sp ieee754sp_abs(ieee754sp x);
-ieee754sp ieee754sp_neg(ieee754sp x);
-ieee754sp ieee754sp_scalb(ieee754sp x, int);
-ieee754sp ieee754sp_logb(ieee754sp x);
+union ieee754sp ieee754sp_abs(union ieee754sp x);
+union ieee754sp ieee754sp_neg(union ieee754sp x);
-/* x with sign of y */
-ieee754sp ieee754sp_copysign(ieee754sp x, ieee754sp y);
+union ieee754sp ieee754sp_add(union ieee754sp x, union ieee754sp y);
+union ieee754sp ieee754sp_sub(union ieee754sp x, union ieee754sp y);
+union ieee754sp ieee754sp_mul(union ieee754sp x, union ieee754sp y);
+union ieee754sp ieee754sp_div(union ieee754sp x, union ieee754sp y);
-ieee754sp ieee754sp_add(ieee754sp x, ieee754sp y);
-ieee754sp ieee754sp_sub(ieee754sp x, ieee754sp y);
-ieee754sp ieee754sp_mul(ieee754sp x, ieee754sp y);
-ieee754sp ieee754sp_div(ieee754sp x, ieee754sp y);
+union ieee754sp ieee754sp_fint(int x);
+union ieee754sp ieee754sp_flong(s64 x);
+union ieee754sp ieee754sp_fdp(union ieee754dp x);
-ieee754sp ieee754sp_fint(int x);
-ieee754sp ieee754sp_funs(unsigned x);
-ieee754sp ieee754sp_flong(s64 x);
-ieee754sp ieee754sp_fulong(u64 x);
-ieee754sp ieee754sp_fdp(ieee754dp x);
+int ieee754sp_tint(union ieee754sp x);
+s64 ieee754sp_tlong(union ieee754sp x);
-int ieee754sp_tint(ieee754sp x);
-unsigned int ieee754sp_tuns(ieee754sp x);
-s64 ieee754sp_tlong(ieee754sp x);
-u64 ieee754sp_tulong(ieee754sp x);
+int ieee754sp_cmp(union ieee754sp x, union ieee754sp y, int cop, int sig);
-int ieee754sp_cmp(ieee754sp x, ieee754sp y, int cop, int sig);
-/*
- * basic sp math
- */
-ieee754sp ieee754sp_modf(ieee754sp x, ieee754sp * ip);
-ieee754sp ieee754sp_frexp(ieee754sp x, int *exp);
-ieee754sp ieee754sp_ldexp(ieee754sp x, int exp);
-
-ieee754sp ieee754sp_ceil(ieee754sp x);
-ieee754sp ieee754sp_floor(ieee754sp x);
-ieee754sp ieee754sp_trunc(ieee754sp x);
-
-ieee754sp ieee754sp_sqrt(ieee754sp x);
+union ieee754sp ieee754sp_sqrt(union ieee754sp x);
/*
* double precision (often aka double)
*/
-int ieee754dp_finite(ieee754dp x);
-int ieee754dp_class(ieee754dp x);
+int ieee754dp_class(union ieee754dp x);
-/* x with sign of y */
-ieee754dp ieee754dp_copysign(ieee754dp x, ieee754dp y);
+union ieee754dp ieee754dp_add(union ieee754dp x, union ieee754dp y);
+union ieee754dp ieee754dp_sub(union ieee754dp x, union ieee754dp y);
+union ieee754dp ieee754dp_mul(union ieee754dp x, union ieee754dp y);
+union ieee754dp ieee754dp_div(union ieee754dp x, union ieee754dp y);
-ieee754dp ieee754dp_add(ieee754dp x, ieee754dp y);
-ieee754dp ieee754dp_sub(ieee754dp x, ieee754dp y);
-ieee754dp ieee754dp_mul(ieee754dp x, ieee754dp y);
-ieee754dp ieee754dp_div(ieee754dp x, ieee754dp y);
+union ieee754dp ieee754dp_abs(union ieee754dp x);
+union ieee754dp ieee754dp_neg(union ieee754dp x);
-ieee754dp ieee754dp_abs(ieee754dp x);
-ieee754dp ieee754dp_neg(ieee754dp x);
-ieee754dp ieee754dp_scalb(ieee754dp x, int);
+union ieee754dp ieee754dp_fint(int x);
+union ieee754dp ieee754dp_flong(s64 x);
+union ieee754dp ieee754dp_fsp(union ieee754sp x);
-/* return exponent as integer in floating point format
- */
-ieee754dp ieee754dp_logb(ieee754dp x);
+int ieee754dp_tint(union ieee754dp x);
+s64 ieee754dp_tlong(union ieee754dp x);
-ieee754dp ieee754dp_fint(int x);
-ieee754dp ieee754dp_funs(unsigned x);
-ieee754dp ieee754dp_flong(s64 x);
-ieee754dp ieee754dp_fulong(u64 x);
-ieee754dp ieee754dp_fsp(ieee754sp x);
+int ieee754dp_cmp(union ieee754dp x, union ieee754dp y, int cop, int sig);
-ieee754dp ieee754dp_ceil(ieee754dp x);
-ieee754dp ieee754dp_floor(ieee754dp x);
-ieee754dp ieee754dp_trunc(ieee754dp x);
-
-int ieee754dp_tint(ieee754dp x);
-unsigned int ieee754dp_tuns(ieee754dp x);
-s64 ieee754dp_tlong(ieee754dp x);
-u64 ieee754dp_tulong(ieee754dp x);
-
-int ieee754dp_cmp(ieee754dp x, ieee754dp y, int cop, int sig);
-/*
- * basic sp math
- */
-ieee754dp ieee754dp_modf(ieee754dp x, ieee754dp * ip);
-ieee754dp ieee754dp_frexp(ieee754dp x, int *exp);
-ieee754dp ieee754dp_ldexp(ieee754dp x, int exp);
-
-ieee754dp ieee754dp_ceil(ieee754dp x);
-ieee754dp ieee754dp_floor(ieee754dp x);
-ieee754dp ieee754dp_trunc(ieee754dp x);
-
-ieee754dp ieee754dp_sqrt(ieee754dp x);
+union ieee754dp ieee754dp_sqrt(union ieee754dp x);
/* 5 types of floating point number
*/
-#define IEEE754_CLASS_NORM 0x00
-#define IEEE754_CLASS_ZERO 0x01
-#define IEEE754_CLASS_DNORM 0x02
-#define IEEE754_CLASS_INF 0x03
-#define IEEE754_CLASS_SNAN 0x04
-#define IEEE754_CLASS_QNAN 0x05
+enum {
+ IEEE754_CLASS_NORM = 0x00,
+ IEEE754_CLASS_ZERO = 0x01,
+ IEEE754_CLASS_DNORM = 0x02,
+ IEEE754_CLASS_INF = 0x03,
+ IEEE754_CLASS_SNAN = 0x04,
+ IEEE754_CLASS_QNAN = 0x05,
+};
/* exception numbers */
#define IEEE754_INEXACT 0x01
@@ -219,114 +126,84 @@
#define IEEE754_CGT 0x04
#define IEEE754_CUN 0x08
-/* rounding mode
-*/
-#define IEEE754_RN 0 /* round to nearest */
-#define IEEE754_RZ 1 /* round toward zero */
-#define IEEE754_RD 2 /* round toward -Infinity */
-#define IEEE754_RU 3 /* round toward +Infinity */
-
-/* other naming */
-#define IEEE754_RM IEEE754_RD
-#define IEEE754_RP IEEE754_RU
-
/* "normal" comparisons
*/
-static inline int ieee754sp_eq(ieee754sp x, ieee754sp y)
+static inline int ieee754sp_eq(union ieee754sp x, union ieee754sp y)
{
return ieee754sp_cmp(x, y, IEEE754_CEQ, 0);
}
-static inline int ieee754sp_ne(ieee754sp x, ieee754sp y)
+static inline int ieee754sp_ne(union ieee754sp x, union ieee754sp y)
{
return ieee754sp_cmp(x, y,
IEEE754_CLT | IEEE754_CGT | IEEE754_CUN, 0);
}
-static inline int ieee754sp_lt(ieee754sp x, ieee754sp y)
+static inline int ieee754sp_lt(union ieee754sp x, union ieee754sp y)
{
return ieee754sp_cmp(x, y, IEEE754_CLT, 0);
}
-static inline int ieee754sp_le(ieee754sp x, ieee754sp y)
+static inline int ieee754sp_le(union ieee754sp x, union ieee754sp y)
{
return ieee754sp_cmp(x, y, IEEE754_CLT | IEEE754_CEQ, 0);
}
-static inline int ieee754sp_gt(ieee754sp x, ieee754sp y)
+static inline int ieee754sp_gt(union ieee754sp x, union ieee754sp y)
{
return ieee754sp_cmp(x, y, IEEE754_CGT, 0);
}
-static inline int ieee754sp_ge(ieee754sp x, ieee754sp y)
+static inline int ieee754sp_ge(union ieee754sp x, union ieee754sp y)
{
return ieee754sp_cmp(x, y, IEEE754_CGT | IEEE754_CEQ, 0);
}
-static inline int ieee754dp_eq(ieee754dp x, ieee754dp y)
+static inline int ieee754dp_eq(union ieee754dp x, union ieee754dp y)
{
return ieee754dp_cmp(x, y, IEEE754_CEQ, 0);
}
-static inline int ieee754dp_ne(ieee754dp x, ieee754dp y)
+static inline int ieee754dp_ne(union ieee754dp x, union ieee754dp y)
{
return ieee754dp_cmp(x, y,
IEEE754_CLT | IEEE754_CGT | IEEE754_CUN, 0);
}
-static inline int ieee754dp_lt(ieee754dp x, ieee754dp y)
+static inline int ieee754dp_lt(union ieee754dp x, union ieee754dp y)
{
return ieee754dp_cmp(x, y, IEEE754_CLT, 0);
}
-static inline int ieee754dp_le(ieee754dp x, ieee754dp y)
+static inline int ieee754dp_le(union ieee754dp x, union ieee754dp y)
{
return ieee754dp_cmp(x, y, IEEE754_CLT | IEEE754_CEQ, 0);
}
-static inline int ieee754dp_gt(ieee754dp x, ieee754dp y)
+static inline int ieee754dp_gt(union ieee754dp x, union ieee754dp y)
{
return ieee754dp_cmp(x, y, IEEE754_CGT, 0);
}
-static inline int ieee754dp_ge(ieee754dp x, ieee754dp y)
+static inline int ieee754dp_ge(union ieee754dp x, union ieee754dp y)
{
return ieee754dp_cmp(x, y, IEEE754_CGT | IEEE754_CEQ, 0);
}
-
-/*
- * Like strtod
- */
-ieee754dp ieee754dp_fstr(const char *s, char **endp);
-char *ieee754dp_tstr(ieee754dp x, int prec, int fmt, int af);
-
-
/*
* The control status register
*/
struct _ieee754_csr {
-#ifdef __BIG_ENDIAN
- unsigned pad0:7;
- unsigned nod:1; /* set 1 for no denormalised numbers */
- unsigned c:1; /* condition */
- unsigned pad1:5;
- unsigned cx:6; /* exceptions this operation */
- unsigned mx:5; /* exception enable mask */
- unsigned sx:5; /* exceptions total */
- unsigned rm:2; /* current rounding mode */
-#endif
-#ifdef __LITTLE_ENDIAN
- unsigned rm:2; /* current rounding mode */
- unsigned sx:5; /* exceptions total */
- unsigned mx:5; /* exception enable mask */
- unsigned cx:6; /* exceptions this operation */
- unsigned pad1:5;
- unsigned c:1; /* condition */
- unsigned nod:1; /* set 1 for no denormalised numbers */
- unsigned pad0:7;
-#endif
+ __BITFIELD_FIELD(unsigned pad0:7,
+ __BITFIELD_FIELD(unsigned nod:1, /* set 1 for no denormalised numbers */
+ __BITFIELD_FIELD(unsigned c:1, /* condition */
+ __BITFIELD_FIELD(unsigned pad1:5,
+ __BITFIELD_FIELD(unsigned cx:6, /* exceptions this operation */
+ __BITFIELD_FIELD(unsigned mx:5, /* exception enable mask */
+ __BITFIELD_FIELD(unsigned sx:5, /* exceptions total */
+ __BITFIELD_FIELD(unsigned rm:2, /* current rounding mode */
+ ;))))))))
};
#define ieee754_csr (*(struct _ieee754_csr *)(¤t->thread.fpu.fcr31))
@@ -377,8 +254,8 @@
}
/* debugging */
-ieee754sp ieee754sp_dump(char *s, ieee754sp x);
-ieee754dp ieee754dp_dump(char *s, ieee754dp x);
+union ieee754sp ieee754sp_dump(char *s, union ieee754sp x);
+union ieee754dp ieee754dp_dump(char *s, union ieee754dp x);
#define IEEE754_SPCVAL_PZERO 0
#define IEEE754_SPCVAL_NZERO 1
@@ -398,10 +275,10 @@
#define IEEE754_SPCVAL_P1E31 15 /* + 1.0e31 */
#define IEEE754_SPCVAL_P1E63 16 /* + 1.0e63 */
-extern const struct ieee754dp_konst __ieee754dp_spcvals[];
-extern const struct ieee754sp_konst __ieee754sp_spcvals[];
-#define ieee754dp_spcvals ((const ieee754dp *)__ieee754dp_spcvals)
-#define ieee754sp_spcvals ((const ieee754sp *)__ieee754sp_spcvals)
+extern const union ieee754dp __ieee754dp_spcvals[];
+extern const union ieee754sp __ieee754sp_spcvals[];
+#define ieee754dp_spcvals ((const union ieee754dp *)__ieee754dp_spcvals)
+#define ieee754sp_spcvals ((const union ieee754sp *)__ieee754sp_spcvals)
/*
* Return infinity with given sign
@@ -431,28 +308,15 @@
/*
* Indefinite integer value
*/
-#define ieee754si_indef() INT_MAX
-#ifdef LONG_LONG_MAX
-#define ieee754di_indef() LONG_LONG_MAX
-#else
-#define ieee754di_indef() ((s64)(~0ULL>>1))
-#endif
+static inline int ieee754si_indef(void)
+{
+ return INT_MAX;
+}
-/* IEEE exception context, passed to handler */
-struct ieee754xctx {
- const char *op; /* operation name */
- int rt; /* result type */
- union {
- ieee754sp sp; /* single precision */
- ieee754dp dp; /* double precision */
-#ifdef IEEE854_XP
- ieee754xp xp; /* extended precision */
-#endif
- int si; /* standard signed integer (32bits) */
- s64 di; /* extended signed integer (64bits) */
- } rv; /* default result format implied by op */
- va_list ap;
-};
+static inline s64 ieee754di_indef(void)
+{
+ return S64_MAX;
+}
/* result types for xctx.rt */
#define IEEE754_RT_SP 0
@@ -461,8 +325,6 @@
#define IEEE754_RT_SI 3
#define IEEE754_RT_DI 4
-extern void ieee754_xcpt(struct ieee754xctx *xcp);
-
/* compat */
#define ieee754dp_fix(x) ieee754dp_tint(x)
#define ieee754sp_fix(x) ieee754sp_tint(x)
diff --git a/arch/mips/math-emu/ieee754d.c b/arch/mips/math-emu/ieee754d.c
index 9599bdd..a04e8a7 100644
--- a/arch/mips/math-emu/ieee754d.c
+++ b/arch/mips/math-emu/ieee754d.c
@@ -16,7 +16,7 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Nov 7, 2000
* Modified to build and operate in Linux kernel environment.
@@ -25,38 +25,13 @@
* Copyright (C) 2000 MIPS Technologies, Inc. All rights reserved.
*/
-#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/printk.h>
#include "ieee754.h"
+#include "ieee754sp.h"
+#include "ieee754dp.h"
-#define DP_EBIAS 1023
-#define DP_EMIN (-1022)
-#define DP_EMAX 1023
-#define DP_FBITS 52
-
-#define SP_EBIAS 127
-#define SP_EMIN (-126)
-#define SP_EMAX 127
-#define SP_FBITS 23
-
-#define DP_MBIT(x) ((u64)1 << (x))
-#define DP_HIDDEN_BIT DP_MBIT(DP_FBITS)
-#define DP_SIGN_BIT DP_MBIT(63)
-
-
-#define SP_MBIT(x) ((u32)1 << (x))
-#define SP_HIDDEN_BIT SP_MBIT(SP_FBITS)
-#define SP_SIGN_BIT SP_MBIT(31)
-
-
-#define SPSIGN(sp) (sp.parts.sign)
-#define SPBEXP(sp) (sp.parts.bexp)
-#define SPMANT(sp) (sp.parts.mant)
-
-#define DPSIGN(dp) (dp.parts.sign)
-#define DPBEXP(dp) (dp.parts.bexp)
-#define DPMANT(dp) (dp.parts.mant)
-
-ieee754dp ieee754dp_dump(char *m, ieee754dp x)
+union ieee754dp ieee754dp_dump(char *m, union ieee754dp x)
{
int i;
@@ -96,7 +71,7 @@
return x;
}
-ieee754sp ieee754sp_dump(char *m, ieee754sp x)
+union ieee754sp ieee754sp_dump(char *m, union ieee754sp x)
{
int i;
diff --git a/arch/mips/math-emu/ieee754dp.c b/arch/mips/math-emu/ieee754dp.c
index 068e56b..fd13467 100644
--- a/arch/mips/math-emu/ieee754dp.c
+++ b/arch/mips/math-emu/ieee754dp.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,104 +16,68 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
+#include <linux/compiler.h>
#include "ieee754dp.h"
-int ieee754dp_class(ieee754dp x)
+int ieee754dp_class(union ieee754dp x)
{
COMPXDP;
EXPLODEXDP;
return xc;
}
-int ieee754dp_isnan(ieee754dp x)
+int ieee754dp_isnan(union ieee754dp x)
{
return ieee754dp_class(x) >= IEEE754_CLASS_SNAN;
}
-int ieee754dp_issnan(ieee754dp x)
+static inline int ieee754dp_issnan(union ieee754dp x)
{
assert(ieee754dp_isnan(x));
- return ((DPMANT(x) & DP_MBIT(DP_MBITS-1)) == DP_MBIT(DP_MBITS-1));
+ return ((DPMANT(x) & DP_MBIT(DP_FBITS-1)) == DP_MBIT(DP_FBITS-1));
}
-ieee754dp ieee754dp_xcpt(ieee754dp r, const char *op, ...)
+union ieee754dp __cold ieee754dp_nanxcpt(union ieee754dp r)
{
- struct ieee754xctx ax;
- if (!TSTX())
- return r;
-
- ax.op = op;
- ax.rt = IEEE754_RT_DP;
- ax.rv.dp = r;
- va_start(ax.ap, op);
- ieee754_xcpt(&ax);
- va_end(ax.ap);
- return ax.rv.dp;
-}
-
-ieee754dp ieee754dp_nanxcpt(ieee754dp r, const char *op, ...)
-{
- struct ieee754xctx ax;
-
assert(ieee754dp_isnan(r));
if (!ieee754dp_issnan(r)) /* QNAN does not cause invalid op !! */
return r;
- if (!SETANDTESTCX(IEEE754_INVALID_OPERATION)) {
+ if (!ieee754_setandtestcx(IEEE754_INVALID_OPERATION)) {
/* not enabled convert to a quiet NaN */
- DPMANT(r) &= (~DP_MBIT(DP_MBITS-1));
+ DPMANT(r) &= (~DP_MBIT(DP_FBITS-1));
if (ieee754dp_isnan(r))
return r;
else
return ieee754dp_indef();
}
- ax.op = op;
- ax.rt = 0;
- ax.rv.dp = r;
- va_start(ax.ap, op);
- ieee754_xcpt(&ax);
- va_end(ax.ap);
- return ax.rv.dp;
+ return r;
}
-ieee754dp ieee754dp_bestnan(ieee754dp x, ieee754dp y)
-{
- assert(ieee754dp_isnan(x));
- assert(ieee754dp_isnan(y));
-
- if (DPMANT(x) > DPMANT(y))
- return x;
- else
- return y;
-}
-
-
-static u64 get_rounding(int sn, u64 xm)
+static u64 ieee754dp_get_rounding(int sn, u64 xm)
{
/* inexact must round of 3 bits
*/
if (xm & (DP_MBIT(3) - 1)) {
switch (ieee754_csr.rm) {
- case IEEE754_RZ:
+ case FPU_CSR_RZ:
break;
- case IEEE754_RN:
+ case FPU_CSR_RN:
xm += 0x3 + ((xm >> 3) & 1);
/* xm += (xm&0x8)?0x4:0x3 */
break;
- case IEEE754_RU: /* toward +Infinity */
+ case FPU_CSR_RU: /* toward +Infinity */
if (!sn) /* ?? */
xm += 0x8;
break;
- case IEEE754_RD: /* toward -Infinity */
+ case FPU_CSR_RD: /* toward -Infinity */
if (sn) /* ?? */
xm += 0x8;
break;
@@ -130,11 +92,11 @@
* xe is an unbiased exponent
* xm is 3bit extended precision value.
*/
-ieee754dp ieee754dp_format(int sn, int xe, u64 xm)
+union ieee754dp ieee754dp_format(int sn, int xe, u64 xm)
{
assert(xm); /* we don't gen exact zeros (probably should) */
- assert((xm >> (DP_MBITS + 1 + 3)) == 0); /* no execess */
+ assert((xm >> (DP_FBITS + 1 + 3)) == 0); /* no execess */
assert(xm & (DP_HIDDEN_BIT << 3));
if (xe < DP_EMIN) {
@@ -142,32 +104,32 @@
int es = DP_EMIN - xe;
if (ieee754_csr.nod) {
- SETCX(IEEE754_UNDERFLOW);
- SETCX(IEEE754_INEXACT);
+ ieee754_setcx(IEEE754_UNDERFLOW);
+ ieee754_setcx(IEEE754_INEXACT);
switch(ieee754_csr.rm) {
- case IEEE754_RN:
- case IEEE754_RZ:
+ case FPU_CSR_RN:
+ case FPU_CSR_RZ:
return ieee754dp_zero(sn);
- case IEEE754_RU: /* toward +Infinity */
- if(sn == 0)
+ case FPU_CSR_RU: /* toward +Infinity */
+ if (sn == 0)
return ieee754dp_min(0);
else
return ieee754dp_zero(1);
- case IEEE754_RD: /* toward -Infinity */
- if(sn == 0)
+ case FPU_CSR_RD: /* toward -Infinity */
+ if (sn == 0)
return ieee754dp_zero(0);
else
return ieee754dp_min(1);
}
}
- if (xe == DP_EMIN - 1
- && get_rounding(sn, xm) >> (DP_MBITS + 1 + 3))
+ if (xe == DP_EMIN - 1 &&
+ ieee754dp_get_rounding(sn, xm) >> (DP_FBITS + 1 + 3))
{
/* Not tiny after rounding */
- SETCX(IEEE754_INEXACT);
- xm = get_rounding(sn, xm);
+ ieee754_setcx(IEEE754_INEXACT);
+ xm = ieee754dp_get_rounding(sn, xm);
xm >>= 1;
/* Clear grs bits */
xm &= ~(DP_MBIT(3) - 1);
@@ -183,17 +145,17 @@
}
}
if (xm & (DP_MBIT(3) - 1)) {
- SETCX(IEEE754_INEXACT);
+ ieee754_setcx(IEEE754_INEXACT);
if ((xm & (DP_HIDDEN_BIT << 3)) == 0) {
- SETCX(IEEE754_UNDERFLOW);
+ ieee754_setcx(IEEE754_UNDERFLOW);
}
/* inexact must round of 3 bits
*/
- xm = get_rounding(sn, xm);
+ xm = ieee754dp_get_rounding(sn, xm);
/* adjust exponent for rounding add overflowing
*/
- if (xm >> (DP_MBITS + 3 + 1)) {
+ if (xm >> (DP_FBITS + 3 + 1)) {
/* add causes mantissa overflow */
xm >>= 1;
xe++;
@@ -202,24 +164,24 @@
/* strip grs bits */
xm >>= 3;
- assert((xm >> (DP_MBITS + 1)) == 0); /* no execess */
+ assert((xm >> (DP_FBITS + 1)) == 0); /* no execess */
assert(xe >= DP_EMIN);
if (xe > DP_EMAX) {
- SETCX(IEEE754_OVERFLOW);
- SETCX(IEEE754_INEXACT);
+ ieee754_setcx(IEEE754_OVERFLOW);
+ ieee754_setcx(IEEE754_INEXACT);
/* -O can be table indexed by (rm,sn) */
switch (ieee754_csr.rm) {
- case IEEE754_RN:
+ case FPU_CSR_RN:
return ieee754dp_inf(sn);
- case IEEE754_RZ:
+ case FPU_CSR_RZ:
return ieee754dp_max(sn);
- case IEEE754_RU: /* toward +Infinity */
+ case FPU_CSR_RU: /* toward +Infinity */
if (sn == 0)
return ieee754dp_inf(0);
else
return ieee754dp_max(1);
- case IEEE754_RD: /* toward -Infinity */
+ case FPU_CSR_RD: /* toward -Infinity */
if (sn == 0)
return ieee754dp_max(0);
else
@@ -232,10 +194,10 @@
/* we underflow (tiny/zero) */
assert(xe == DP_EMIN);
if (ieee754_csr.mx & IEEE754_UNDERFLOW)
- SETCX(IEEE754_UNDERFLOW);
+ ieee754_setcx(IEEE754_UNDERFLOW);
return builddp(sn, DP_EMIN - 1 + DP_EBIAS, xm);
} else {
- assert((xm >> (DP_MBITS + 1)) == 0); /* no execess */
+ assert((xm >> (DP_FBITS + 1)) == 0); /* no execess */
assert(xm & DP_HIDDEN_BIT);
return builddp(sn, xe + DP_EBIAS, xm & ~DP_HIDDEN_BIT);
diff --git a/arch/mips/math-emu/ieee754dp.h b/arch/mips/math-emu/ieee754dp.h
index f139c72..61fd6fd 100644
--- a/arch/mips/math-emu/ieee754dp.h
+++ b/arch/mips/math-emu/ieee754dp.h
@@ -6,8 +6,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -19,64 +17,66 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
+#include <linux/compiler.h>
#include "ieee754int.h"
#define assert(expr) ((void)0)
+#define DP_EBIAS 1023
+#define DP_EMIN (-1022)
+#define DP_EMAX 1023
+#define DP_FBITS 52
+#define DP_MBITS 52
+
+#define DP_MBIT(x) ((u64)1 << (x))
+#define DP_HIDDEN_BIT DP_MBIT(DP_FBITS)
+#define DP_SIGN_BIT DP_MBIT(63)
+
+#define DPSIGN(dp) (dp.sign)
+#define DPBEXP(dp) (dp.bexp)
+#define DPMANT(dp) (dp.mant)
+
+static inline int ieee754dp_finite(union ieee754dp x)
+{
+ return DPBEXP(x) != DP_EMAX + 1 + DP_EBIAS;
+}
+
/* 3bit extended double precision sticky right shift */
#define XDPSRS(v,rs) \
- ((rs > (DP_MBITS+3))?1:((v) >> (rs)) | ((v) << (64-(rs)) != 0))
+ ((rs > (DP_FBITS+3))?1:((v) >> (rs)) | ((v) << (64-(rs)) != 0))
#define XDPSRSX1() \
- (xe++, (xm = (xm >> 1) | (xm & 1)))
+ (xe++, (xm = (xm >> 1) | (xm & 1)))
#define XDPSRS1(v) \
- (((v) >> 1) | ((v) & 1))
+ (((v) >> 1) | ((v) & 1))
/* convert denormal to normalized with extended exponent */
#define DPDNORMx(m,e) \
- while( (m >> DP_MBITS) == 0) { m <<= 1; e--; }
+ while ((m >> DP_FBITS) == 0) { m <<= 1; e--; }
#define DPDNORMX DPDNORMx(xm, xe)
#define DPDNORMY DPDNORMx(ym, ye)
-static inline ieee754dp builddp(int s, int bx, u64 m)
+static inline union ieee754dp builddp(int s, int bx, u64 m)
{
- ieee754dp r;
+ union ieee754dp r;
assert((s) == 0 || (s) == 1);
assert((bx) >= DP_EMIN - 1 + DP_EBIAS
&& (bx) <= DP_EMAX + 1 + DP_EBIAS);
- assert(((m) >> DP_MBITS) == 0);
+ assert(((m) >> DP_FBITS) == 0);
- r.parts.sign = s;
- r.parts.bexp = bx;
- r.parts.mant = m;
+ r.sign = s;
+ r.bexp = bx;
+ r.mant = m;
+
return r;
}
-extern int ieee754dp_isnan(ieee754dp);
-extern int ieee754dp_issnan(ieee754dp);
-extern int ieee754si_xcpt(int, const char *, ...);
-extern s64 ieee754di_xcpt(s64, const char *, ...);
-extern ieee754dp ieee754dp_xcpt(ieee754dp, const char *, ...);
-extern ieee754dp ieee754dp_nanxcpt(ieee754dp, const char *, ...);
-extern ieee754dp ieee754dp_bestnan(ieee754dp, ieee754dp);
-extern ieee754dp ieee754dp_format(int, int, u64);
-
-
-#define DPNORMRET2(s, e, m, name, a0, a1) \
-{ \
- ieee754dp V = ieee754dp_format(s, e, m); \
- if(TSTX()) \
- return ieee754dp_xcpt(V, name, a0, a1); \
- else \
- return V; \
-}
-
-#define DPNORMRET1(s, e, m, name, a0) DPNORMRET2(s, e, m, name, a0, a0)
+extern int ieee754dp_isnan(union ieee754dp);
+extern union ieee754dp __cold ieee754dp_nanxcpt(union ieee754dp);
+extern union ieee754dp ieee754dp_format(int, int, u64);
diff --git a/arch/mips/math-emu/ieee754int.h b/arch/mips/math-emu/ieee754int.h
index 4b6c6fb3..f0365bb 100644
--- a/arch/mips/math-emu/ieee754int.h
+++ b/arch/mips/math-emu/ieee754int.h
@@ -6,8 +6,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -19,146 +17,125 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
+#ifndef __IEEE754INT_H
+#define __IEEE754INT_H
#include "ieee754.h"
-#define DP_EBIAS 1023
-#define DP_EMIN (-1022)
-#define DP_EMAX 1023
-#define DP_MBITS 52
-
-#define SP_EBIAS 127
-#define SP_EMIN (-126)
-#define SP_EMAX 127
-#define SP_MBITS 23
-
-#define DP_MBIT(x) ((u64)1 << (x))
-#define DP_HIDDEN_BIT DP_MBIT(DP_MBITS)
-#define DP_SIGN_BIT DP_MBIT(63)
-
-#define SP_MBIT(x) ((u32)1 << (x))
-#define SP_HIDDEN_BIT SP_MBIT(SP_MBITS)
-#define SP_SIGN_BIT SP_MBIT(31)
-
-
-#define SPSIGN(sp) (sp.parts.sign)
-#define SPBEXP(sp) (sp.parts.bexp)
-#define SPMANT(sp) (sp.parts.mant)
-
-#define DPSIGN(dp) (dp.parts.sign)
-#define DPBEXP(dp) (dp.parts.bexp)
-#define DPMANT(dp) (dp.parts.mant)
-
#define CLPAIR(x, y) ((x)*6+(y))
-#define CLEARCX \
- (ieee754_csr.cx = 0)
+static inline void ieee754_clearcx(void)
+{
+ ieee754_csr.cx = 0;
+}
-#define SETCX(x) \
- (ieee754_csr.cx |= (x), ieee754_csr.sx |= (x))
+static inline void ieee754_setcx(const unsigned int flags)
+{
+ ieee754_csr.cx |= flags;
+ ieee754_csr.sx |= flags;
+}
-#define SETANDTESTCX(x) \
- (SETCX(x), ieee754_csr.mx & (x))
+static inline int ieee754_setandtestcx(const unsigned int x)
+{
+ ieee754_setcx(x);
-#define TSTX() \
- (ieee754_csr.cx & ieee754_csr.mx)
-
+ return ieee754_csr.mx & x;
+}
#define COMPXSP \
- unsigned xm; int xe; int xs __maybe_unused; int xc
+ unsigned xm; int xe; int xs __maybe_unused; int xc
#define COMPYSP \
- unsigned ym; int ye; int ys; int yc
+ unsigned ym; int ye; int ys; int yc
-#define EXPLODESP(v, vc, vs, ve, vm) \
-{\
- vs = SPSIGN(v);\
- ve = SPBEXP(v);\
- vm = SPMANT(v);\
- if(ve == SP_EMAX+1+SP_EBIAS){\
- if(vm == 0)\
- vc = IEEE754_CLASS_INF;\
- else if(vm & SP_MBIT(SP_MBITS-1)) \
- vc = IEEE754_CLASS_SNAN;\
- else \
- vc = IEEE754_CLASS_QNAN;\
- } else if(ve == SP_EMIN-1+SP_EBIAS) {\
- if(vm) {\
- ve = SP_EMIN;\
- vc = IEEE754_CLASS_DNORM;\
- } else\
- vc = IEEE754_CLASS_ZERO;\
- } else {\
- ve -= SP_EBIAS;\
- vm |= SP_HIDDEN_BIT;\
- vc = IEEE754_CLASS_NORM;\
- }\
+#define EXPLODESP(v, vc, vs, ve, vm) \
+{ \
+ vs = SPSIGN(v); \
+ ve = SPBEXP(v); \
+ vm = SPMANT(v); \
+ if (ve == SP_EMAX+1+SP_EBIAS) { \
+ if (vm == 0) \
+ vc = IEEE754_CLASS_INF; \
+ else if (vm & SP_MBIT(SP_FBITS-1)) \
+ vc = IEEE754_CLASS_SNAN; \
+ else \
+ vc = IEEE754_CLASS_QNAN; \
+ } else if (ve == SP_EMIN-1+SP_EBIAS) { \
+ if (vm) { \
+ ve = SP_EMIN; \
+ vc = IEEE754_CLASS_DNORM; \
+ } else \
+ vc = IEEE754_CLASS_ZERO; \
+ } else { \
+ ve -= SP_EBIAS; \
+ vm |= SP_HIDDEN_BIT; \
+ vc = IEEE754_CLASS_NORM; \
+ } \
}
#define EXPLODEXSP EXPLODESP(x, xc, xs, xe, xm)
#define EXPLODEYSP EXPLODESP(y, yc, ys, ye, ym)
#define COMPXDP \
-u64 xm; int xe; int xs __maybe_unused; int xc
+ u64 xm; int xe; int xs __maybe_unused; int xc
#define COMPYDP \
-u64 ym; int ye; int ys; int yc
+ u64 ym; int ye; int ys; int yc
-#define EXPLODEDP(v, vc, vs, ve, vm) \
-{\
- vm = DPMANT(v);\
- vs = DPSIGN(v);\
- ve = DPBEXP(v);\
- if(ve == DP_EMAX+1+DP_EBIAS){\
- if(vm == 0)\
- vc = IEEE754_CLASS_INF;\
- else if(vm & DP_MBIT(DP_MBITS-1)) \
- vc = IEEE754_CLASS_SNAN;\
- else \
- vc = IEEE754_CLASS_QNAN;\
- } else if(ve == DP_EMIN-1+DP_EBIAS) {\
- if(vm) {\
- ve = DP_EMIN;\
- vc = IEEE754_CLASS_DNORM;\
- } else\
- vc = IEEE754_CLASS_ZERO;\
- } else {\
- ve -= DP_EBIAS;\
- vm |= DP_HIDDEN_BIT;\
- vc = IEEE754_CLASS_NORM;\
- }\
+#define EXPLODEDP(v, vc, vs, ve, vm) \
+{ \
+ vm = DPMANT(v); \
+ vs = DPSIGN(v); \
+ ve = DPBEXP(v); \
+ if (ve == DP_EMAX+1+DP_EBIAS) { \
+ if (vm == 0) \
+ vc = IEEE754_CLASS_INF; \
+ else if (vm & DP_MBIT(DP_FBITS-1)) \
+ vc = IEEE754_CLASS_SNAN; \
+ else \
+ vc = IEEE754_CLASS_QNAN; \
+ } else if (ve == DP_EMIN-1+DP_EBIAS) { \
+ if (vm) { \
+ ve = DP_EMIN; \
+ vc = IEEE754_CLASS_DNORM; \
+ } else \
+ vc = IEEE754_CLASS_ZERO; \
+ } else { \
+ ve -= DP_EBIAS; \
+ vm |= DP_HIDDEN_BIT; \
+ vc = IEEE754_CLASS_NORM; \
+ } \
}
#define EXPLODEXDP EXPLODEDP(x, xc, xs, xe, xm)
#define EXPLODEYDP EXPLODEDP(y, yc, ys, ye, ym)
-#define FLUSHDP(v, vc, vs, ve, vm) \
- if(vc==IEEE754_CLASS_DNORM) {\
- if(ieee754_csr.nod) {\
- SETCX(IEEE754_INEXACT);\
- vc = IEEE754_CLASS_ZERO;\
- ve = DP_EMIN-1+DP_EBIAS;\
- vm = 0;\
- v = ieee754dp_zero(vs);\
- }\
+#define FLUSHDP(v, vc, vs, ve, vm) \
+ if (vc==IEEE754_CLASS_DNORM) { \
+ if (ieee754_csr.nod) { \
+ ieee754_setcx(IEEE754_INEXACT); \
+ vc = IEEE754_CLASS_ZERO; \
+ ve = DP_EMIN-1+DP_EBIAS; \
+ vm = 0; \
+ v = ieee754dp_zero(vs); \
+ } \
}
-#define FLUSHSP(v, vc, vs, ve, vm) \
- if(vc==IEEE754_CLASS_DNORM) {\
- if(ieee754_csr.nod) {\
- SETCX(IEEE754_INEXACT);\
- vc = IEEE754_CLASS_ZERO;\
- ve = SP_EMIN-1+SP_EBIAS;\
- vm = 0;\
- v = ieee754sp_zero(vs);\
- }\
+#define FLUSHSP(v, vc, vs, ve, vm) \
+ if (vc==IEEE754_CLASS_DNORM) { \
+ if (ieee754_csr.nod) { \
+ ieee754_setcx(IEEE754_INEXACT); \
+ vc = IEEE754_CLASS_ZERO; \
+ ve = SP_EMIN-1+SP_EBIAS; \
+ vm = 0; \
+ v = ieee754sp_zero(vs); \
+ } \
}
#define FLUSHXDP FLUSHDP(x, xc, xs, xe, xm)
#define FLUSHYDP FLUSHDP(y, yc, ys, ye, ym)
#define FLUSHXSP FLUSHSP(x, xc, xs, xe, xm)
#define FLUSHYSP FLUSHSP(y, yc, ys, ye, ym)
+
+#endif /* __IEEE754INT_H */
diff --git a/arch/mips/math-emu/ieee754m.c b/arch/mips/math-emu/ieee754m.c
deleted file mode 100644
index 24190f3..0000000
--- a/arch/mips/math-emu/ieee754m.c
+++ /dev/null
@@ -1,55 +0,0 @@
-/*
- * floor, trunc, ceil
- */
-/*
- * MIPS floating point support
- * Copyright (C) 1994-2000 Algorithmics Ltd.
- *
- * ########################################################################
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License (Version 2) as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
- */
-
-
-#include "ieee754.h"
-
-ieee754dp ieee754dp_floor(ieee754dp x)
-{
- ieee754dp i;
-
- if (ieee754dp_lt(ieee754dp_modf(x, &i), ieee754dp_zero(0)))
- return ieee754dp_sub(i, ieee754dp_one(0));
- else
- return i;
-}
-
-ieee754dp ieee754dp_ceil(ieee754dp x)
-{
- ieee754dp i;
-
- if (ieee754dp_gt(ieee754dp_modf(x, &i), ieee754dp_zero(0)))
- return ieee754dp_add(i, ieee754dp_one(0));
- else
- return i;
-}
-
-ieee754dp ieee754dp_trunc(ieee754dp x)
-{
- ieee754dp i;
-
- (void) ieee754dp_modf(x, &i);
- return i;
-}
diff --git a/arch/mips/math-emu/ieee754sp.c b/arch/mips/math-emu/ieee754sp.c
index 15d1e36..d348efe 100644
--- a/arch/mips/math-emu/ieee754sp.c
+++ b/arch/mips/math-emu/ieee754sp.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,105 +16,68 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
+#include <linux/compiler.h>
#include "ieee754sp.h"
-int ieee754sp_class(ieee754sp x)
+int ieee754sp_class(union ieee754sp x)
{
COMPXSP;
EXPLODEXSP;
return xc;
}
-int ieee754sp_isnan(ieee754sp x)
+int ieee754sp_isnan(union ieee754sp x)
{
return ieee754sp_class(x) >= IEEE754_CLASS_SNAN;
}
-int ieee754sp_issnan(ieee754sp x)
+static inline int ieee754sp_issnan(union ieee754sp x)
{
assert(ieee754sp_isnan(x));
- return (SPMANT(x) & SP_MBIT(SP_MBITS-1));
+ return (SPMANT(x) & SP_MBIT(SP_FBITS-1));
}
-ieee754sp ieee754sp_xcpt(ieee754sp r, const char *op, ...)
+union ieee754sp __cold ieee754sp_nanxcpt(union ieee754sp r)
{
- struct ieee754xctx ax;
-
- if (!TSTX())
- return r;
-
- ax.op = op;
- ax.rt = IEEE754_RT_SP;
- ax.rv.sp = r;
- va_start(ax.ap, op);
- ieee754_xcpt(&ax);
- va_end(ax.ap);
- return ax.rv.sp;
-}
-
-ieee754sp ieee754sp_nanxcpt(ieee754sp r, const char *op, ...)
-{
- struct ieee754xctx ax;
-
assert(ieee754sp_isnan(r));
if (!ieee754sp_issnan(r)) /* QNAN does not cause invalid op !! */
return r;
- if (!SETANDTESTCX(IEEE754_INVALID_OPERATION)) {
+ if (!ieee754_setandtestcx(IEEE754_INVALID_OPERATION)) {
/* not enabled convert to a quiet NaN */
- SPMANT(r) &= (~SP_MBIT(SP_MBITS-1));
+ SPMANT(r) &= (~SP_MBIT(SP_FBITS-1));
if (ieee754sp_isnan(r))
return r;
else
return ieee754sp_indef();
}
- ax.op = op;
- ax.rt = 0;
- ax.rv.sp = r;
- va_start(ax.ap, op);
- ieee754_xcpt(&ax);
- va_end(ax.ap);
- return ax.rv.sp;
+ return r;
}
-ieee754sp ieee754sp_bestnan(ieee754sp x, ieee754sp y)
-{
- assert(ieee754sp_isnan(x));
- assert(ieee754sp_isnan(y));
-
- if (SPMANT(x) > SPMANT(y))
- return x;
- else
- return y;
-}
-
-
-static unsigned get_rounding(int sn, unsigned xm)
+static unsigned ieee754sp_get_rounding(int sn, unsigned xm)
{
/* inexact must round of 3 bits
*/
if (xm & (SP_MBIT(3) - 1)) {
switch (ieee754_csr.rm) {
- case IEEE754_RZ:
+ case FPU_CSR_RZ:
break;
- case IEEE754_RN:
+ case FPU_CSR_RN:
xm += 0x3 + ((xm >> 3) & 1);
/* xm += (xm&0x8)?0x4:0x3 */
break;
- case IEEE754_RU: /* toward +Infinity */
+ case FPU_CSR_RU: /* toward +Infinity */
if (!sn) /* ?? */
xm += 0x8;
break;
- case IEEE754_RD: /* toward -Infinity */
+ case FPU_CSR_RD: /* toward -Infinity */
if (sn) /* ?? */
xm += 0x8;
break;
@@ -131,11 +92,11 @@
* xe is an unbiased exponent
* xm is 3bit extended precision value.
*/
-ieee754sp ieee754sp_format(int sn, int xe, unsigned xm)
+union ieee754sp ieee754sp_format(int sn, int xe, unsigned xm)
{
assert(xm); /* we don't gen exact zeros (probably should) */
- assert((xm >> (SP_MBITS + 1 + 3)) == 0); /* no execess */
+ assert((xm >> (SP_FBITS + 1 + 3)) == 0); /* no execess */
assert(xm & (SP_HIDDEN_BIT << 3));
if (xe < SP_EMIN) {
@@ -143,38 +104,37 @@
int es = SP_EMIN - xe;
if (ieee754_csr.nod) {
- SETCX(IEEE754_UNDERFLOW);
- SETCX(IEEE754_INEXACT);
+ ieee754_setcx(IEEE754_UNDERFLOW);
+ ieee754_setcx(IEEE754_INEXACT);
switch(ieee754_csr.rm) {
- case IEEE754_RN:
- case IEEE754_RZ:
+ case FPU_CSR_RN:
+ case FPU_CSR_RZ:
return ieee754sp_zero(sn);
- case IEEE754_RU: /* toward +Infinity */
- if(sn == 0)
+ case FPU_CSR_RU: /* toward +Infinity */
+ if (sn == 0)
return ieee754sp_min(0);
else
return ieee754sp_zero(1);
- case IEEE754_RD: /* toward -Infinity */
- if(sn == 0)
+ case FPU_CSR_RD: /* toward -Infinity */
+ if (sn == 0)
return ieee754sp_zero(0);
else
return ieee754sp_min(1);
}
}
- if (xe == SP_EMIN - 1
- && get_rounding(sn, xm) >> (SP_MBITS + 1 + 3))
+ if (xe == SP_EMIN - 1 &&
+ ieee754sp_get_rounding(sn, xm) >> (SP_FBITS + 1 + 3))
{
/* Not tiny after rounding */
- SETCX(IEEE754_INEXACT);
- xm = get_rounding(sn, xm);
+ ieee754_setcx(IEEE754_INEXACT);
+ xm = ieee754sp_get_rounding(sn, xm);
xm >>= 1;
/* Clear grs bits */
xm &= ~(SP_MBIT(3) - 1);
xe++;
- }
- else {
+ } else {
/* sticky right shift es bits
*/
SPXSRSXn(es);
@@ -183,17 +143,17 @@
}
}
if (xm & (SP_MBIT(3) - 1)) {
- SETCX(IEEE754_INEXACT);
+ ieee754_setcx(IEEE754_INEXACT);
if ((xm & (SP_HIDDEN_BIT << 3)) == 0) {
- SETCX(IEEE754_UNDERFLOW);
+ ieee754_setcx(IEEE754_UNDERFLOW);
}
/* inexact must round of 3 bits
*/
- xm = get_rounding(sn, xm);
+ xm = ieee754sp_get_rounding(sn, xm);
/* adjust exponent for rounding add overflowing
*/
- if (xm >> (SP_MBITS + 1 + 3)) {
+ if (xm >> (SP_FBITS + 1 + 3)) {
/* add causes mantissa overflow */
xm >>= 1;
xe++;
@@ -202,24 +162,24 @@
/* strip grs bits */
xm >>= 3;
- assert((xm >> (SP_MBITS + 1)) == 0); /* no execess */
+ assert((xm >> (SP_FBITS + 1)) == 0); /* no execess */
assert(xe >= SP_EMIN);
if (xe > SP_EMAX) {
- SETCX(IEEE754_OVERFLOW);
- SETCX(IEEE754_INEXACT);
+ ieee754_setcx(IEEE754_OVERFLOW);
+ ieee754_setcx(IEEE754_INEXACT);
/* -O can be table indexed by (rm,sn) */
switch (ieee754_csr.rm) {
- case IEEE754_RN:
+ case FPU_CSR_RN:
return ieee754sp_inf(sn);
- case IEEE754_RZ:
+ case FPU_CSR_RZ:
return ieee754sp_max(sn);
- case IEEE754_RU: /* toward +Infinity */
+ case FPU_CSR_RU: /* toward +Infinity */
if (sn == 0)
return ieee754sp_inf(0);
else
return ieee754sp_max(1);
- case IEEE754_RD: /* toward -Infinity */
+ case FPU_CSR_RD: /* toward -Infinity */
if (sn == 0)
return ieee754sp_max(0);
else
@@ -232,10 +192,10 @@
/* we underflow (tiny/zero) */
assert(xe == SP_EMIN);
if (ieee754_csr.mx & IEEE754_UNDERFLOW)
- SETCX(IEEE754_UNDERFLOW);
+ ieee754_setcx(IEEE754_UNDERFLOW);
return buildsp(sn, SP_EMIN - 1 + SP_EBIAS, xm);
} else {
- assert((xm >> (SP_MBITS + 1)) == 0); /* no execess */
+ assert((xm >> (SP_FBITS + 1)) == 0); /* no execess */
assert(xm & SP_HIDDEN_BIT);
return buildsp(sn, xe + SP_EBIAS, xm & ~SP_HIDDEN_BIT);
diff --git a/arch/mips/math-emu/ieee754sp.h b/arch/mips/math-emu/ieee754sp.h
index 754fd546..ad268e3 100644
--- a/arch/mips/math-emu/ieee754sp.h
+++ b/arch/mips/math-emu/ieee754sp.h
@@ -6,8 +6,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -19,70 +17,71 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
+#include <linux/compiler.h>
#include "ieee754int.h"
#define assert(expr) ((void)0)
+#define SP_EBIAS 127
+#define SP_EMIN (-126)
+#define SP_EMAX 127
+#define SP_FBITS 23
+#define SP_MBITS 23
+
+#define SP_MBIT(x) ((u32)1 << (x))
+#define SP_HIDDEN_BIT SP_MBIT(SP_FBITS)
+#define SP_SIGN_BIT SP_MBIT(31)
+
+#define SPSIGN(sp) (sp.sign)
+#define SPBEXP(sp) (sp.bexp)
+#define SPMANT(sp) (sp.mant)
+
+static inline int ieee754sp_finite(union ieee754sp x)
+{
+ return SPBEXP(x) != SP_EMAX + 1 + SP_EBIAS;
+}
+
/* 3bit extended single precision sticky right shift */
-#define SPXSRSXn(rs) \
- (xe += rs, \
- xm = (rs > (SP_MBITS+3))?1:((xm) >> (rs)) | ((xm) << (32-(rs)) != 0))
+#define SPXSRSXn(rs) \
+ (xe += rs, \
+ xm = (rs > (SP_FBITS+3))?1:((xm) >> (rs)) | ((xm) << (32-(rs)) != 0))
#define SPXSRSX1() \
- (xe++, (xm = (xm >> 1) | (xm & 1)))
+ (xe++, (xm = (xm >> 1) | (xm & 1)))
-#define SPXSRSYn(rs) \
- (ye+=rs, \
- ym = (rs > (SP_MBITS+3))?1:((ym) >> (rs)) | ((ym) << (32-(rs)) != 0))
+#define SPXSRSYn(rs) \
+ (ye+=rs, \
+ ym = (rs > (SP_FBITS+3))?1:((ym) >> (rs)) | ((ym) << (32-(rs)) != 0))
#define SPXSRSY1() \
- (ye++, (ym = (ym >> 1) | (ym & 1)))
+ (ye++, (ym = (ym >> 1) | (ym & 1)))
/* convert denormal to normalized with extended exponent */
#define SPDNORMx(m,e) \
- while( (m >> SP_MBITS) == 0) { m <<= 1; e--; }
+ while ((m >> SP_FBITS) == 0) { m <<= 1; e--; }
#define SPDNORMX SPDNORMx(xm, xe)
#define SPDNORMY SPDNORMx(ym, ye)
-static inline ieee754sp buildsp(int s, int bx, unsigned m)
+static inline union ieee754sp buildsp(int s, int bx, unsigned m)
{
- ieee754sp r;
+ union ieee754sp r;
assert((s) == 0 || (s) == 1);
assert((bx) >= SP_EMIN - 1 + SP_EBIAS
&& (bx) <= SP_EMAX + 1 + SP_EBIAS);
- assert(((m) >> SP_MBITS) == 0);
+ assert(((m) >> SP_FBITS) == 0);
- r.parts.sign = s;
- r.parts.bexp = bx;
- r.parts.mant = m;
+ r.sign = s;
+ r.bexp = bx;
+ r.mant = m;
return r;
}
-extern int ieee754sp_isnan(ieee754sp);
-extern int ieee754sp_issnan(ieee754sp);
-extern int ieee754si_xcpt(int, const char *, ...);
-extern s64 ieee754di_xcpt(s64, const char *, ...);
-extern ieee754sp ieee754sp_xcpt(ieee754sp, const char *, ...);
-extern ieee754sp ieee754sp_nanxcpt(ieee754sp, const char *, ...);
-extern ieee754sp ieee754sp_bestnan(ieee754sp, ieee754sp);
-extern ieee754sp ieee754sp_format(int, int, unsigned);
-
-
-#define SPNORMRET2(s, e, m, name, a0, a1) \
-{ \
- ieee754sp V = ieee754sp_format(s, e, m); \
- if(TSTX()) \
- return ieee754sp_xcpt(V, name, a0, a1); \
- else \
- return V; \
-}
-
-#define SPNORMRET1(s, e, m, name, a0) SPNORMRET2(s, e, m, name, a0, a0)
+extern int ieee754sp_isnan(union ieee754sp);
+extern union ieee754sp __cold ieee754sp_nanxcpt(union ieee754sp);
+extern union ieee754sp ieee754sp_format(int, int, unsigned);
diff --git a/arch/mips/math-emu/ieee754xcpt.c b/arch/mips/math-emu/ieee754xcpt.c
deleted file mode 100644
index 96716711..0000000
--- a/arch/mips/math-emu/ieee754xcpt.c
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * MIPS floating point support
- * Copyright (C) 1994-2000 Algorithmics Ltd.
- *
- * ########################################################################
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License (Version 2) as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
- */
-
-/**************************************************************************
- * Nov 7, 2000
- * Added preprocessor hacks to map to Linux kernel diagnostics.
- *
- * Kevin D. Kissell, kevink@mips.com and Carsten Langgaard, carstenl@mips.com
- * Copyright (C) 2000 MIPS Technologies, Inc. All rights reserved.
- *************************************************************************/
-
-#include <linux/kernel.h>
-#include "ieee754.h"
-
-/*
- * Very naff exception handler (you can plug in your own and
- * override this).
- */
-
-static const char *const rtnames[] = {
- "sp", "dp", "xp", "si", "di"
-};
-
-void ieee754_xcpt(struct ieee754xctx *xcp)
-{
- printk(KERN_DEBUG "floating point exception in \"%s\", type=%s\n",
- xcp->op, rtnames[xcp->rt]);
-}
diff --git a/arch/mips/math-emu/kernel_linkage.c b/arch/mips/math-emu/kernel_linkage.c
deleted file mode 100644
index eb58a85..0000000
--- a/arch/mips/math-emu/kernel_linkage.c
+++ /dev/null
@@ -1,45 +0,0 @@
-/*
- * Kevin D. Kissell, kevink@mips and Carsten Langgaard, carstenl@mips.com
- * Copyright (C) 2000 MIPS Technologies, Inc. All rights reserved.
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License (Version 2) as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * Routines corresponding to Linux kernel FP context
- * manipulation primitives for the Algorithmics MIPS
- * FPU Emulator
- */
-#include <linux/sched.h>
-#include <asm/processor.h>
-#include <asm/signal.h>
-#include <asm/uaccess.h>
-
-#include <asm/fpu.h>
-#include <asm/fpu_emulator.h>
-
-#define SIGNALLING_NAN 0x7ff800007ff80000LL
-
-void fpu_emulator_init_fpu(void)
-{
- static int first = 1;
- int i;
-
- if (first) {
- first = 0;
- printk("Algorithmics/MIPS FPU Emulator v1.5\n");
- }
-
- current->thread.fpu.fcr31 = 0;
- for (i = 0; i < 32; i++)
- set_fpr64(¤t->thread.fpu.fpr[i], 0, SIGNALLING_NAN);
-}
diff --git a/arch/mips/math-emu/me-debugfs.c b/arch/mips/math-emu/me-debugfs.c
new file mode 100644
index 0000000..becdd63
--- /dev/null
+++ b/arch/mips/math-emu/me-debugfs.c
@@ -0,0 +1,67 @@
+#include <linux/cpumask.h>
+#include <linux/debugfs.h>
+#include <linux/fs.h>
+#include <linux/init.h>
+#include <linux/percpu.h>
+#include <linux/types.h>
+#include <asm/fpu_emulator.h>
+#include <asm/local.h>
+
+DEFINE_PER_CPU(struct mips_fpu_emulator_stats, fpuemustats);
+
+static int fpuemu_stat_get(void *data, u64 *val)
+{
+ int cpu;
+ unsigned long sum = 0;
+
+ for_each_online_cpu(cpu) {
+ struct mips_fpu_emulator_stats *ps;
+ local_t *pv;
+
+ ps = &per_cpu(fpuemustats, cpu);
+ pv = (void *)ps + (unsigned long)data;
+ sum += local_read(pv);
+ }
+ *val = sum;
+ return 0;
+}
+DEFINE_SIMPLE_ATTRIBUTE(fops_fpuemu_stat, fpuemu_stat_get, NULL, "%llu\n");
+
+extern struct dentry *mips_debugfs_dir;
+static int __init debugfs_fpuemu(void)
+{
+ struct dentry *d, *dir;
+
+ if (!mips_debugfs_dir)
+ return -ENODEV;
+ dir = debugfs_create_dir("fpuemustats", mips_debugfs_dir);
+ if (!dir)
+ return -ENOMEM;
+
+#define FPU_EMU_STAT_OFFSET(m) \
+ offsetof(struct mips_fpu_emulator_stats, m)
+
+#define FPU_STAT_CREATE(m) \
+do { \
+ d = debugfs_create_file(#m , S_IRUGO, dir, \
+ (void *)FPU_EMU_STAT_OFFSET(m), \
+ &fops_fpuemu_stat); \
+ if (!d) \
+ return -ENOMEM; \
+} while (0)
+
+ FPU_STAT_CREATE(emulated);
+ FPU_STAT_CREATE(loads);
+ FPU_STAT_CREATE(stores);
+ FPU_STAT_CREATE(cp1ops);
+ FPU_STAT_CREATE(cp1xops);
+ FPU_STAT_CREATE(errors);
+ FPU_STAT_CREATE(ieee754_inexact);
+ FPU_STAT_CREATE(ieee754_underflow);
+ FPU_STAT_CREATE(ieee754_overflow);
+ FPU_STAT_CREATE(ieee754_zerodiv);
+ FPU_STAT_CREATE(ieee754_invalidop);
+
+ return 0;
+}
+__initcall(debugfs_fpuemu);
diff --git a/arch/mips/math-emu/sp_add.c b/arch/mips/math-emu/sp_add.c
index c446e64..2d84d46 100644
--- a/arch/mips/math-emu/sp_add.c
+++ b/arch/mips/math-emu/sp_add.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,23 +16,22 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754sp.h"
-ieee754sp ieee754sp_add(ieee754sp x, ieee754sp y)
+union ieee754sp ieee754sp_add(union ieee754sp x, union ieee754sp y)
{
+ int s;
+
COMPXSP;
COMPYSP;
EXPLODEXSP;
EXPLODEYSP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXSP;
FLUSHYSP;
@@ -51,8 +48,8 @@
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_nanxcpt(ieee754sp_indef(), "add", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_nanxcpt(ieee754sp_indef());
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN):
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN):
@@ -68,14 +65,14 @@
return x;
- /* Infinity handling
- */
-
+ /*
+ * Infinity handling
+ */
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF):
if (xs == ys)
return x;
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_xcpt(ieee754sp_indef(), "add", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_indef();
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF):
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF):
@@ -87,15 +84,14 @@
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM):
return x;
- /* Zero handling
- */
-
+ /*
+ * Zero handling
+ */
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO):
if (xs == ys)
return x;
else
- return ieee754sp_zero(ieee754_csr.rm ==
- IEEE754_RD);
+ return ieee754sp_zero(ieee754_csr.rm == FPU_CSR_RD);
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO):
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO):
@@ -108,6 +104,8 @@
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM):
SPDNORMX;
+ /* FALL THROUGH */
+
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM):
SPDNORMY;
break;
@@ -122,33 +120,38 @@
assert(xm & SP_HIDDEN_BIT);
assert(ym & SP_HIDDEN_BIT);
- /* provide guard,round and stick bit space */
+ /*
+ * Provide guard, round and stick bit space.
+ */
xm <<= 3;
ym <<= 3;
if (xe > ye) {
- /* have to shift y fraction right to align
+ /*
+ * Have to shift y fraction right to align.
*/
- int s = xe - ye;
+ s = xe - ye;
SPXSRSYn(s);
} else if (ye > xe) {
- /* have to shift x fraction right to align
+ /*
+ * Have to shift x fraction right to align.
*/
- int s = ye - xe;
+ s = ye - xe;
SPXSRSXn(s);
}
assert(xe == ye);
assert(xe <= SP_EMAX);
if (xs == ys) {
- /* generate 28 bit result of adding two 27 bit numbers
- * leaving result in xm,xs,xe
+ /*
+ * Generate 28 bit result of adding two 27 bit numbers
+ * leaving result in xm, xs and xe.
*/
xm = xm + ym;
xe = xe;
xs = xs;
- if (xm >> (SP_MBITS + 1 + 3)) { /* carry out */
+ if (xm >> (SP_FBITS + 1 + 3)) { /* carry out */
SPXSRSX1();
}
} else {
@@ -162,15 +165,16 @@
xs = ys;
}
if (xm == 0)
- return ieee754sp_zero(ieee754_csr.rm ==
- IEEE754_RD);
+ return ieee754sp_zero(ieee754_csr.rm == FPU_CSR_RD);
- /* normalize in extended single precision */
- while ((xm >> (SP_MBITS + 3)) == 0) {
+ /*
+ * Normalize in extended single precision
+ */
+ while ((xm >> (SP_FBITS + 3)) == 0) {
xm <<= 1;
xe--;
}
-
}
- SPNORMRET2(xs, xe, xm, "add", x, y);
+
+ return ieee754sp_format(xs, xe, xm);
}
diff --git a/arch/mips/math-emu/sp_cmp.c b/arch/mips/math-emu/sp_cmp.c
index 716cf37..addbccb 100644
--- a/arch/mips/math-emu/sp_cmp.c
+++ b/arch/mips/math-emu/sp_cmp.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,16 +16,16 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754sp.h"
-int ieee754sp_cmp(ieee754sp x, ieee754sp y, int cmp, int sig)
+int ieee754sp_cmp(union ieee754sp x, union ieee754sp y, int cmp, int sig)
{
+ int vx;
+ int vy;
+
COMPXSP;
COMPYSP;
@@ -35,21 +33,21 @@
EXPLODEYSP;
FLUSHXSP;
FLUSHYSP;
- CLEARCX; /* Even clear inexact flag here */
+ ieee754_clearcx(); /* Even clear inexact flag here */
if (ieee754sp_isnan(x) || ieee754sp_isnan(y)) {
if (sig || xc == IEEE754_CLASS_SNAN || yc == IEEE754_CLASS_SNAN)
- SETCX(IEEE754_INVALID_OPERATION);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
if (cmp & IEEE754_CUN)
return 1;
if (cmp & (IEEE754_CLT | IEEE754_CGT)) {
- if (sig && SETANDTESTCX(IEEE754_INVALID_OPERATION))
- return ieee754si_xcpt(0, "fcmpf", x);
+ if (sig && ieee754_setandtestcx(IEEE754_INVALID_OPERATION))
+ return 0;
}
return 0;
} else {
- int vx = x.bits;
- int vy = y.bits;
+ vx = x.bits;
+ vy = y.bits;
if (vx < 0)
vx = -vx ^ SP_SIGN_BIT;
diff --git a/arch/mips/math-emu/sp_div.c b/arch/mips/math-emu/sp_div.c
index d7747928c..721f317 100644
--- a/arch/mips/math-emu/sp_div.c
+++ b/arch/mips/math-emu/sp_div.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,23 +16,24 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754sp.h"
-ieee754sp ieee754sp_div(ieee754sp x, ieee754sp y)
+union ieee754sp ieee754sp_div(union ieee754sp x, union ieee754sp y)
{
+ unsigned rm;
+ int re;
+ unsigned bm;
+
COMPXSP;
COMPYSP;
EXPLODEXSP;
EXPLODEYSP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXSP;
FLUSHYSP;
@@ -51,8 +50,8 @@
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_nanxcpt(ieee754sp_indef(), "div", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_nanxcpt(ieee754sp_indef());
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN):
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN):
@@ -68,12 +67,12 @@
return x;
- /* Infinity handling
- */
-
+ /*
+ * Infinity handling
+ */
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_xcpt(ieee754sp_indef(), "div", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_indef();
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF):
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF):
@@ -85,17 +84,17 @@
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM):
return ieee754sp_inf(xs ^ ys);
- /* Zero handling
- */
-
+ /*
+ * Zero handling
+ */
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_xcpt(ieee754sp_indef(), "div", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_indef();
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO):
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO):
- SETCX(IEEE754_ZERO_DIVIDE);
- return ieee754sp_xcpt(ieee754sp_inf(xs ^ ys), "div", x, y);
+ ieee754_setcx(IEEE754_ZERO_DIVIDE);
+ return ieee754sp_inf(xs ^ ys);
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM):
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM):
@@ -122,35 +121,33 @@
xm <<= 3;
ym <<= 3;
- {
- /* now the dirty work */
+ /* now the dirty work */
- unsigned rm = 0;
- int re = xe - ye;
- unsigned bm;
+ rm = 0;
+ re = xe - ye;
- for (bm = SP_MBIT(SP_MBITS + 2); bm; bm >>= 1) {
- if (xm >= ym) {
- xm -= ym;
- rm |= bm;
- if (xm == 0)
- break;
- }
- xm <<= 1;
+ for (bm = SP_MBIT(SP_FBITS + 2); bm; bm >>= 1) {
+ if (xm >= ym) {
+ xm -= ym;
+ rm |= bm;
+ if (xm == 0)
+ break;
}
- rm <<= 1;
- if (xm)
- rm |= 1; /* have remainder, set sticky */
-
- assert(rm);
-
- /* normalise rm to rounding precision ?
- */
- while ((rm >> (SP_MBITS + 3)) == 0) {
- rm <<= 1;
- re--;
- }
-
- SPNORMRET2(xs == ys ? 0 : 1, re, rm, "div", x, y);
+ xm <<= 1;
}
+
+ rm <<= 1;
+ if (xm)
+ rm |= 1; /* have remainder, set sticky */
+
+ assert(rm);
+
+ /* normalise rm to rounding precision ?
+ */
+ while ((rm >> (SP_FBITS + 3)) == 0) {
+ rm <<= 1;
+ re--;
+ }
+
+ return ieee754sp_format(xs == ys ? 0 : 1, re, rm);
}
diff --git a/arch/mips/math-emu/sp_fdp.c b/arch/mips/math-emu/sp_fdp.c
index e1515aa..1b266fb 100644
--- a/arch/mips/math-emu/sp_fdp.c
+++ b/arch/mips/math-emu/sp_fdp.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,59 +16,61 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754sp.h"
+#include "ieee754dp.h"
-ieee754sp ieee754sp_fdp(ieee754dp x)
+union ieee754sp ieee754sp_fdp(union ieee754dp x)
{
+ u32 rm;
+
COMPXDP;
- ieee754sp nan;
+ union ieee754sp nan;
EXPLODEXDP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXDP;
switch (xc) {
case IEEE754_CLASS_SNAN:
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_nanxcpt(ieee754sp_indef(), "fdp");
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_nanxcpt(ieee754sp_indef());
+
case IEEE754_CLASS_QNAN:
nan = buildsp(xs, SP_EMAX + 1 + SP_EBIAS, (u32)
- (xm >> (DP_MBITS - SP_MBITS)));
+ (xm >> (DP_FBITS - SP_FBITS)));
if (!ieee754sp_isnan(nan))
nan = ieee754sp_indef();
- return ieee754sp_nanxcpt(nan, "fdp", x);
+ return ieee754sp_nanxcpt(nan);
+
case IEEE754_CLASS_INF:
return ieee754sp_inf(xs);
+
case IEEE754_CLASS_ZERO:
return ieee754sp_zero(xs);
+
case IEEE754_CLASS_DNORM:
/* can't possibly be sp representable */
- SETCX(IEEE754_UNDERFLOW);
- SETCX(IEEE754_INEXACT);
- if ((ieee754_csr.rm == IEEE754_RU && !xs) ||
- (ieee754_csr.rm == IEEE754_RD && xs))
- return ieee754sp_xcpt(ieee754sp_mind(xs), "fdp", x);
- return ieee754sp_xcpt(ieee754sp_zero(xs), "fdp", x);
+ ieee754_setcx(IEEE754_UNDERFLOW);
+ ieee754_setcx(IEEE754_INEXACT);
+ if ((ieee754_csr.rm == FPU_CSR_RU && !xs) ||
+ (ieee754_csr.rm == FPU_CSR_RD && xs))
+ return ieee754sp_mind(xs);
+ return ieee754sp_zero(xs);
+
case IEEE754_CLASS_NORM:
break;
}
- {
- u32 rm;
+ /*
+ * Convert from DP_FBITS to SP_FBITS+3 with sticky right shift.
+ */
+ rm = (xm >> (DP_FBITS - (SP_FBITS + 3))) |
+ ((xm << (64 - (DP_FBITS - (SP_FBITS + 3)))) != 0);
- /* convert from DP_MBITS to SP_MBITS+3 with sticky right shift
- */
- rm = (xm >> (DP_MBITS - (SP_MBITS + 3))) |
- ((xm << (64 - (DP_MBITS - (SP_MBITS + 3)))) != 0);
-
- SPNORMRET1(xs, xe, rm, "fdp", x);
- }
+ return ieee754sp_format(xs, xe, rm);
}
diff --git a/arch/mips/math-emu/sp_fint.c b/arch/mips/math-emu/sp_fint.c
index 9694d6c0..d5d8495 100644
--- a/arch/mips/math-emu/sp_fint.c
+++ b/arch/mips/math-emu/sp_fint.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,21 +16,18 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754sp.h"
-ieee754sp ieee754sp_fint(int x)
+union ieee754sp ieee754sp_fint(int x)
{
unsigned xm;
int xe;
int xs;
- CLEARCX;
+ ieee754_clearcx();
if (x == 0)
return ieee754sp_zero(0);
@@ -50,30 +45,21 @@
} else {
xm = x;
}
- xe = SP_MBITS + 3;
+ xe = SP_FBITS + 3;
- if (xm >> (SP_MBITS + 1 + 3)) {
+ if (xm >> (SP_FBITS + 1 + 3)) {
/* shunt out overflow bits
*/
- while (xm >> (SP_MBITS + 1 + 3)) {
+ while (xm >> (SP_FBITS + 1 + 3)) {
SPXSRSX1();
}
} else {
/* normalize in grs extended single precision
*/
- while ((xm >> (SP_MBITS + 3)) == 0) {
+ while ((xm >> (SP_FBITS + 3)) == 0) {
xm <<= 1;
xe--;
}
}
- SPNORMRET1(xs, xe, xm, "fint", x);
-}
-
-
-ieee754sp ieee754sp_funs(unsigned int u)
-{
- if ((int) u < 0)
- return ieee754sp_add(ieee754sp_1e31(),
- ieee754sp_fint(u & ~(1 << 31)));
- return ieee754sp_fint(u);
+ return ieee754sp_format(xs, xe, xm);
}
diff --git a/arch/mips/math-emu/sp_flong.c b/arch/mips/math-emu/sp_flong.c
index 16a651f..012e30c 100644
--- a/arch/mips/math-emu/sp_flong.c
+++ b/arch/mips/math-emu/sp_flong.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,21 +16,18 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754sp.h"
-ieee754sp ieee754sp_flong(s64 x)
+union ieee754sp ieee754sp_flong(s64 x)
{
u64 xm; /* <--- need 64-bit mantissa temp */
int xe;
int xs;
- CLEARCX;
+ ieee754_clearcx();
if (x == 0)
return ieee754sp_zero(0);
@@ -50,29 +45,20 @@
} else {
xm = x;
}
- xe = SP_MBITS + 3;
+ xe = SP_FBITS + 3;
- if (xm >> (SP_MBITS + 1 + 3)) {
+ if (xm >> (SP_FBITS + 1 + 3)) {
/* shunt out overflow bits
*/
- while (xm >> (SP_MBITS + 1 + 3)) {
+ while (xm >> (SP_FBITS + 1 + 3)) {
SPXSRSX1();
}
} else {
/* normalize in grs extended single precision */
- while ((xm >> (SP_MBITS + 3)) == 0) {
+ while ((xm >> (SP_FBITS + 3)) == 0) {
xm <<= 1;
xe--;
}
}
- SPNORMRET1(xs, xe, xm, "sp_flong", x);
-}
-
-
-ieee754sp ieee754sp_fulong(u64 u)
-{
- if ((s64) u < 0)
- return ieee754sp_add(ieee754sp_1e63(),
- ieee754sp_flong(u & ~(1ULL << 63)));
- return ieee754sp_flong(u);
+ return ieee754sp_format(xs, xe, xm);
}
diff --git a/arch/mips/math-emu/sp_frexp.c b/arch/mips/math-emu/sp_frexp.c
deleted file mode 100644
index 5bc993c..0000000
--- a/arch/mips/math-emu/sp_frexp.c
+++ /dev/null
@@ -1,52 +0,0 @@
-/* IEEE754 floating point arithmetic
- * single precision
- */
-/*
- * MIPS floating point support
- * Copyright (C) 1994-2000 Algorithmics Ltd.
- *
- * ########################################################################
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License (Version 2) as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
- */
-
-
-#include "ieee754sp.h"
-
-/* close to ieeep754sp_logb
-*/
-ieee754sp ieee754sp_frexp(ieee754sp x, int *eptr)
-{
- COMPXSP;
- CLEARCX;
- EXPLODEXSP;
-
- switch (xc) {
- case IEEE754_CLASS_SNAN:
- case IEEE754_CLASS_QNAN:
- case IEEE754_CLASS_INF:
- case IEEE754_CLASS_ZERO:
- *eptr = 0;
- return x;
- case IEEE754_CLASS_DNORM:
- SPDNORMX;
- break;
- case IEEE754_CLASS_NORM:
- break;
- }
- *eptr = xe + 1;
- return buildsp(xs, -1 + SP_EBIAS, xm & ~SP_HIDDEN_BIT);
-}
diff --git a/arch/mips/math-emu/sp_logb.c b/arch/mips/math-emu/sp_logb.c
deleted file mode 100644
index 9c14e0c..0000000
--- a/arch/mips/math-emu/sp_logb.c
+++ /dev/null
@@ -1,53 +0,0 @@
-/* IEEE754 floating point arithmetic
- * single precision
- */
-/*
- * MIPS floating point support
- * Copyright (C) 1994-2000 Algorithmics Ltd.
- *
- * ########################################################################
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License (Version 2) as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
- */
-
-
-#include "ieee754sp.h"
-
-ieee754sp ieee754sp_logb(ieee754sp x)
-{
- COMPXSP;
-
- CLEARCX;
-
- EXPLODEXSP;
-
- switch (xc) {
- case IEEE754_CLASS_SNAN:
- return ieee754sp_nanxcpt(x, "logb", x);
- case IEEE754_CLASS_QNAN:
- return x;
- case IEEE754_CLASS_INF:
- return ieee754sp_inf(0);
- case IEEE754_CLASS_ZERO:
- return ieee754sp_inf(1);
- case IEEE754_CLASS_DNORM:
- SPDNORMX;
- break;
- case IEEE754_CLASS_NORM:
- break;
- }
- return ieee754sp_fint(xe);
-}
diff --git a/arch/mips/math-emu/sp_modf.c b/arch/mips/math-emu/sp_modf.c
deleted file mode 100644
index 25a0fba..0000000
--- a/arch/mips/math-emu/sp_modf.c
+++ /dev/null
@@ -1,79 +0,0 @@
-/* IEEE754 floating point arithmetic
- * single precision
- */
-/*
- * MIPS floating point support
- * Copyright (C) 1994-2000 Algorithmics Ltd.
- *
- * ########################################################################
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License (Version 2) as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
- */
-
-
-#include "ieee754sp.h"
-
-/* modf function is always exact for a finite number
-*/
-ieee754sp ieee754sp_modf(ieee754sp x, ieee754sp *ip)
-{
- COMPXSP;
-
- CLEARCX;
-
- EXPLODEXSP;
-
- switch (xc) {
- case IEEE754_CLASS_SNAN:
- case IEEE754_CLASS_QNAN:
- case IEEE754_CLASS_INF:
- case IEEE754_CLASS_ZERO:
- *ip = x;
- return x;
- case IEEE754_CLASS_DNORM:
- /* far to small */
- *ip = ieee754sp_zero(xs);
- return x;
- case IEEE754_CLASS_NORM:
- break;
- }
- if (xe < 0) {
- *ip = ieee754sp_zero(xs);
- return x;
- }
- if (xe >= SP_MBITS) {
- *ip = x;
- return ieee754sp_zero(xs);
- }
- /* generate ipart mantissa by clearing bottom bits
- */
- *ip = buildsp(xs, xe + SP_EBIAS,
- ((xm >> (SP_MBITS - xe)) << (SP_MBITS - xe)) &
- ~SP_HIDDEN_BIT);
-
- /* generate fpart mantissa by clearing top bits
- * and normalizing (must be able to normalize)
- */
- xm = (xm << (32 - (SP_MBITS - xe))) >> (32 - (SP_MBITS - xe));
- if (xm == 0)
- return ieee754sp_zero(xs);
-
- while ((xm >> SP_MBITS) == 0) {
- xm <<= 1;
- xe--;
- }
- return buildsp(xs, xe + SP_EBIAS, xm & ~SP_HIDDEN_BIT);
-}
diff --git a/arch/mips/math-emu/sp_mul.c b/arch/mips/math-emu/sp_mul.c
index fa4675c..890c13a 100644
--- a/arch/mips/math-emu/sp_mul.c
+++ b/arch/mips/math-emu/sp_mul.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,23 +16,32 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754sp.h"
-ieee754sp ieee754sp_mul(ieee754sp x, ieee754sp y)
+union ieee754sp ieee754sp_mul(union ieee754sp x, union ieee754sp y)
{
+ int re;
+ int rs;
+ unsigned rm;
+ unsigned short lxm;
+ unsigned short hxm;
+ unsigned short lym;
+ unsigned short hym;
+ unsigned lrm;
+ unsigned hrm;
+ unsigned t;
+ unsigned at;
+
COMPXSP;
COMPYSP;
EXPLODEXSP;
EXPLODEYSP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXSP;
FLUSHYSP;
@@ -51,8 +58,8 @@
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_nanxcpt(ieee754sp_indef(), "mul", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_nanxcpt(ieee754sp_indef());
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN):
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN):
@@ -68,12 +75,13 @@
return x;
- /* Infinity handling */
-
+ /*
+ * Infinity handling
+ */
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO):
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_xcpt(ieee754sp_indef(), "mul", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_indef();
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF):
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF):
@@ -108,63 +116,50 @@
assert(xm & SP_HIDDEN_BIT);
assert(ym & SP_HIDDEN_BIT);
- {
- int re = xe + ye;
- int rs = xs ^ ys;
- unsigned rm;
+ re = xe + ye;
+ rs = xs ^ ys;
- /* shunt to top of word */
- xm <<= 32 - (SP_MBITS + 1);
- ym <<= 32 - (SP_MBITS + 1);
+ /* shunt to top of word */
+ xm <<= 32 - (SP_FBITS + 1);
+ ym <<= 32 - (SP_FBITS + 1);
- /* multiply 32bits xm,ym to give high 32bits rm with stickness
- */
- {
- unsigned short lxm = xm & 0xffff;
- unsigned short hxm = xm >> 16;
- unsigned short lym = ym & 0xffff;
- unsigned short hym = ym >> 16;
- unsigned lrm;
- unsigned hrm;
+ /*
+ * Multiply 32 bits xm, ym to give high 32 bits rm with stickness.
+ */
+ lxm = xm & 0xffff;
+ hxm = xm >> 16;
+ lym = ym & 0xffff;
+ hym = ym >> 16;
- lrm = lxm * lym; /* 16 * 16 => 32 */
- hrm = hxm * hym; /* 16 * 16 => 32 */
+ lrm = lxm * lym; /* 16 * 16 => 32 */
+ hrm = hxm * hym; /* 16 * 16 => 32 */
- {
- unsigned t = lxm * hym; /* 16 * 16 => 32 */
- {
- unsigned at = lrm + (t << 16);
- hrm += at < lrm;
- lrm = at;
- }
- hrm = hrm + (t >> 16);
- }
+ t = lxm * hym; /* 16 * 16 => 32 */
+ at = lrm + (t << 16);
+ hrm += at < lrm;
+ lrm = at;
+ hrm = hrm + (t >> 16);
- {
- unsigned t = hxm * lym; /* 16 * 16 => 32 */
- {
- unsigned at = lrm + (t << 16);
- hrm += at < lrm;
- lrm = at;
- }
- hrm = hrm + (t >> 16);
- }
- rm = hrm | (lrm != 0);
- }
+ t = hxm * lym; /* 16 * 16 => 32 */
+ at = lrm + (t << 16);
+ hrm += at < lrm;
+ lrm = at;
+ hrm = hrm + (t >> 16);
- /*
- * sticky shift down to normal rounding precision
- */
- if ((int) rm < 0) {
- rm = (rm >> (32 - (SP_MBITS + 1 + 3))) |
- ((rm << (SP_MBITS + 1 + 3)) != 0);
- re++;
- } else {
- rm = (rm >> (32 - (SP_MBITS + 1 + 3 + 1))) |
- ((rm << (SP_MBITS + 1 + 3 + 1)) != 0);
- }
- assert(rm & (SP_HIDDEN_BIT << 3));
+ rm = hrm | (lrm != 0);
- SPNORMRET2(rs, re, rm, "mul", x, y);
+ /*
+ * Sticky shift down to normal rounding precision.
+ */
+ if ((int) rm < 0) {
+ rm = (rm >> (32 - (SP_FBITS + 1 + 3))) |
+ ((rm << (SP_FBITS + 1 + 3)) != 0);
+ re++;
+ } else {
+ rm = (rm >> (32 - (SP_FBITS + 1 + 3 + 1))) |
+ ((rm << (SP_FBITS + 1 + 3 + 1)) != 0);
}
+ assert(rm & (SP_HIDDEN_BIT << 3));
+
+ return ieee754sp_format(rs, re, rm);
}
diff --git a/arch/mips/math-emu/sp_scalb.c b/arch/mips/math-emu/sp_scalb.c
deleted file mode 100644
index dd76196..0000000
--- a/arch/mips/math-emu/sp_scalb.c
+++ /dev/null
@@ -1,57 +0,0 @@
-/* IEEE754 floating point arithmetic
- * single precision
- */
-/*
- * MIPS floating point support
- * Copyright (C) 1994-2000 Algorithmics Ltd.
- *
- * ########################################################################
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License (Version 2) as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
- */
-
-
-#include "ieee754sp.h"
-
-ieee754sp ieee754sp_scalb(ieee754sp x, int n)
-{
- COMPXSP;
-
- CLEARCX;
-
- EXPLODEXSP;
-
- switch (xc) {
- case IEEE754_CLASS_SNAN:
- return ieee754sp_nanxcpt(x, "scalb", x, n);
- case IEEE754_CLASS_QNAN:
- case IEEE754_CLASS_INF:
- case IEEE754_CLASS_ZERO:
- return x;
- case IEEE754_CLASS_DNORM:
- SPDNORMX;
- break;
- case IEEE754_CLASS_NORM:
- break;
- }
- SPNORMRET2(xs, xe + n, xm << 3, "scalb", x, n);
-}
-
-
-ieee754sp ieee754sp_ldexp(ieee754sp x, int n)
-{
- return ieee754sp_scalb(x, n);
-}
diff --git a/arch/mips/math-emu/sp_simple.c b/arch/mips/math-emu/sp_simple.c
index ae4fcfa..f1ffaa9a 100644
--- a/arch/mips/math-emu/sp_simple.c
+++ b/arch/mips/math-emu/sp_simple.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,33 +16,17 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754sp.h"
-int ieee754sp_finite(ieee754sp x)
-{
- return SPBEXP(x) != SP_EMAX + 1 + SP_EBIAS;
-}
-
-ieee754sp ieee754sp_copysign(ieee754sp x, ieee754sp y)
-{
- CLEARCX;
- SPSIGN(x) = SPSIGN(y);
- return x;
-}
-
-
-ieee754sp ieee754sp_neg(ieee754sp x)
+union ieee754sp ieee754sp_neg(union ieee754sp x)
{
COMPXSP;
EXPLODEXSP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXSP;
/*
@@ -55,30 +37,29 @@
SPSIGN(x) ^= 1;
if (xc == IEEE754_CLASS_SNAN) {
- ieee754sp y = ieee754sp_indef();
- SETCX(IEEE754_INVALID_OPERATION);
+ union ieee754sp y = ieee754sp_indef();
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
SPSIGN(y) = SPSIGN(x);
- return ieee754sp_nanxcpt(y, "neg");
+ return ieee754sp_nanxcpt(y);
}
return x;
}
-
-ieee754sp ieee754sp_abs(ieee754sp x)
+union ieee754sp ieee754sp_abs(union ieee754sp x)
{
COMPXSP;
EXPLODEXSP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXSP;
/* Clear sign ALWAYS, irrespective of NaN */
SPSIGN(x) = 0;
if (xc == IEEE754_CLASS_SNAN) {
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_nanxcpt(ieee754sp_indef(), "abs");
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_nanxcpt(ieee754sp_indef());
}
return x;
diff --git a/arch/mips/math-emu/sp_sqrt.c b/arch/mips/math-emu/sp_sqrt.c
index fed2017..b7c098a 100644
--- a/arch/mips/math-emu/sp_sqrt.c
+++ b/arch/mips/math-emu/sp_sqrt.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,15 +16,12 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754sp.h"
-ieee754sp ieee754sp_sqrt(ieee754sp x)
+union ieee754sp ieee754sp_sqrt(union ieee754sp x)
{
int ix, s, q, m, t, i;
unsigned int r;
@@ -35,34 +30,38 @@
/* take care of Inf and NaN */
EXPLODEXSP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXSP;
/* x == INF or NAN? */
switch (xc) {
case IEEE754_CLASS_QNAN:
/* sqrt(Nan) = Nan */
- return ieee754sp_nanxcpt(x, "sqrt");
+ return ieee754sp_nanxcpt(x);
+
case IEEE754_CLASS_SNAN:
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_nanxcpt(ieee754sp_indef(), "sqrt");
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_nanxcpt(ieee754sp_indef());
+
case IEEE754_CLASS_ZERO:
/* sqrt(0) = 0 */
return x;
+
case IEEE754_CLASS_INF:
if (xs) {
/* sqrt(-Inf) = Nan */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_nanxcpt(ieee754sp_indef(), "sqrt");
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_nanxcpt(ieee754sp_indef());
}
/* sqrt(+Inf) = Inf */
return x;
+
case IEEE754_CLASS_DNORM:
case IEEE754_CLASS_NORM:
if (xs) {
/* sqrt(-x) = Nan */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_nanxcpt(ieee754sp_indef(), "sqrt");
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_nanxcpt(ieee754sp_indef());
}
break;
}
@@ -99,12 +98,12 @@
}
if (ix != 0) {
- SETCX(IEEE754_INEXACT);
+ ieee754_setcx(IEEE754_INEXACT);
switch (ieee754_csr.rm) {
- case IEEE754_RP:
+ case FPU_CSR_RU:
q += 2;
break;
- case IEEE754_RN:
+ case FPU_CSR_RN:
q += (q & 1);
break;
}
diff --git a/arch/mips/math-emu/sp_sub.c b/arch/mips/math-emu/sp_sub.c
index e595c6f..8592e49 100644
--- a/arch/mips/math-emu/sp_sub.c
+++ b/arch/mips/math-emu/sp_sub.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,23 +16,22 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754sp.h"
-ieee754sp ieee754sp_sub(ieee754sp x, ieee754sp y)
+union ieee754sp ieee754sp_sub(union ieee754sp x, union ieee754sp y)
{
+ int s;
+
COMPXSP;
COMPYSP;
EXPLODEXSP;
EXPLODEYSP;
- CLEARCX;
+ ieee754_clearcx();
FLUSHXSP;
FLUSHYSP;
@@ -51,8 +48,8 @@
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM):
case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF):
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_nanxcpt(ieee754sp_indef(), "sub", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_nanxcpt(ieee754sp_indef());
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN):
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN):
@@ -68,14 +65,14 @@
return x;
- /* Infinity handling
- */
-
+ /*
+ * Infinity handling
+ */
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF):
if (xs != ys)
return x;
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754sp_xcpt(ieee754sp_indef(), "sub", x, y);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754sp_indef();
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF):
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF):
@@ -87,15 +84,14 @@
case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM):
return x;
- /* Zero handling
- */
-
+ /*
+ * Zero handling
+ */
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO):
if (xs != ys)
return x;
else
- return ieee754sp_zero(ieee754_csr.rm ==
- IEEE754_RD);
+ return ieee754sp_zero(ieee754_csr.rm == FPU_CSR_RD);
case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO):
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO):
@@ -104,7 +100,7 @@
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM):
case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM):
/* quick fix up */
- DPSIGN(y) ^= 1;
+ SPSIGN(y) ^= 1;
return y;
case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM):
@@ -133,14 +129,16 @@
ym <<= 3;
if (xe > ye) {
- /* have to shift y fraction right to align
+ /*
+ * have to shift y fraction right to align
*/
- int s = xe - ye;
+ s = xe - ye;
SPXSRSYn(s);
} else if (ye > xe) {
- /* have to shift x fraction right to align
+ /*
+ * have to shift x fraction right to align
*/
- int s = ye - xe;
+ s = ye - xe;
SPXSRSXn(s);
}
assert(xe == ye);
@@ -153,7 +151,7 @@
xe = xe;
xs = xs;
- if (xm >> (SP_MBITS + 1 + 3)) { /* carry out */
+ if (xm >> (SP_FBITS + 1 + 3)) { /* carry out */
SPXSRSX1(); /* shift preserving sticky */
}
} else {
@@ -167,17 +165,18 @@
xs = ys;
}
if (xm == 0) {
- if (ieee754_csr.rm == IEEE754_RD)
+ if (ieee754_csr.rm == FPU_CSR_RD)
return ieee754sp_zero(1); /* round negative inf. => sign = -1 */
else
return ieee754sp_zero(0); /* other round modes => sign = 1 */
}
/* normalize to rounding precision
*/
- while ((xm >> (SP_MBITS + 3)) == 0) {
+ while ((xm >> (SP_FBITS + 3)) == 0) {
xm <<= 1;
xe--;
}
}
- SPNORMRET2(xs, xe, xm, "sub", x, y);
+
+ return ieee754sp_format(xs, xe, xm);
}
diff --git a/arch/mips/math-emu/sp_tint.c b/arch/mips/math-emu/sp_tint.c
index 0fe9acc..091299a 100644
--- a/arch/mips/math-emu/sp_tint.c
+++ b/arch/mips/math-emu/sp_tint.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,20 +16,21 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
-#include <linux/kernel.h>
#include "ieee754sp.h"
-int ieee754sp_tint(ieee754sp x)
+int ieee754sp_tint(union ieee754sp x)
{
+ u32 residue;
+ int round;
+ int sticky;
+ int odd;
+
COMPXSP;
- CLEARCX;
+ ieee754_clearcx();
EXPLODEXSP;
FLUSHXSP;
@@ -40,10 +39,12 @@
case IEEE754_CLASS_SNAN:
case IEEE754_CLASS_QNAN:
case IEEE754_CLASS_INF:
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754si_xcpt(ieee754si_indef(), "sp_tint", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754si_indef();
+
case IEEE754_CLASS_ZERO:
return 0;
+
case IEEE754_CLASS_DNORM:
case IEEE754_CLASS_NORM:
break;
@@ -54,18 +55,13 @@
return -0x80000000;
/* Set invalid. We will only use overflow for floating
point overflow */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754si_xcpt(ieee754si_indef(), "sp_tint", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754si_indef();
}
/* oh gawd */
- if (xe > SP_MBITS) {
- xm <<= xe - SP_MBITS;
+ if (xe > SP_FBITS) {
+ xm <<= xe - SP_FBITS;
} else {
- u32 residue;
- int round;
- int sticky;
- int odd;
-
if (xe < -1) {
residue = xm;
round = 0;
@@ -76,51 +72,38 @@
* so we do it in two steps. Be aware that xe
* may be -1 */
residue = xm << (xe + 1);
- residue <<= 31 - SP_MBITS;
+ residue <<= 31 - SP_FBITS;
round = (residue >> 31) != 0;
sticky = (residue << 1) != 0;
- xm >>= SP_MBITS - xe;
+ xm >>= SP_FBITS - xe;
}
odd = (xm & 0x1) != 0x0;
switch (ieee754_csr.rm) {
- case IEEE754_RN:
+ case FPU_CSR_RN:
if (round && (sticky || odd))
xm++;
break;
- case IEEE754_RZ:
+ case FPU_CSR_RZ:
break;
- case IEEE754_RU: /* toward +Infinity */
+ case FPU_CSR_RU: /* toward +Infinity */
if ((round || sticky) && !xs)
xm++;
break;
- case IEEE754_RD: /* toward -Infinity */
+ case FPU_CSR_RD: /* toward -Infinity */
if ((round || sticky) && xs)
xm++;
break;
}
if ((xm >> 31) != 0) {
/* This can happen after rounding */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754si_xcpt(ieee754si_indef(), "sp_tint", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754si_indef();
}
if (round || sticky)
- SETCX(IEEE754_INEXACT);
+ ieee754_setcx(IEEE754_INEXACT);
}
if (xs)
return -xm;
else
return xm;
}
-
-
-unsigned int ieee754sp_tuns(ieee754sp x)
-{
- ieee754sp hb = ieee754sp_1e31();
-
- /* what if x < 0 ?? */
- if (ieee754sp_lt(x, hb))
- return (unsigned) ieee754sp_tint(x);
-
- return (unsigned) ieee754sp_tint(ieee754sp_sub(x, hb)) |
- ((unsigned) 1 << 31);
-}
diff --git a/arch/mips/math-emu/sp_tlong.c b/arch/mips/math-emu/sp_tlong.c
index d0ca6e2..9f3c742 100644
--- a/arch/mips/math-emu/sp_tlong.c
+++ b/arch/mips/math-emu/sp_tlong.c
@@ -5,8 +5,6 @@
* MIPS floating point support
* Copyright (C) 1994-2000 Algorithmics Ltd.
*
- * ########################################################################
- *
* This program is free software; you can distribute it and/or modify it
* under the terms of the GNU General Public License (Version 2) as
* published by the Free Software Foundation.
@@ -18,19 +16,22 @@
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
- *
- * ########################################################################
+ * 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*/
-
#include "ieee754sp.h"
+#include "ieee754dp.h"
-s64 ieee754sp_tlong(ieee754sp x)
+s64 ieee754sp_tlong(union ieee754sp x)
{
+ u32 residue;
+ int round;
+ int sticky;
+ int odd;
+
COMPXDP; /* <-- need 64-bit mantissa tmp */
- CLEARCX;
+ ieee754_clearcx();
EXPLODEXSP;
FLUSHXSP;
@@ -39,10 +40,12 @@
case IEEE754_CLASS_SNAN:
case IEEE754_CLASS_QNAN:
case IEEE754_CLASS_INF:
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754di_xcpt(ieee754di_indef(), "sp_tlong", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754di_indef();
+
case IEEE754_CLASS_ZERO:
return 0;
+
case IEEE754_CLASS_DNORM:
case IEEE754_CLASS_NORM:
break;
@@ -53,69 +56,51 @@
return -0x8000000000000000LL;
/* Set invalid. We will only use overflow for floating
point overflow */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754di_xcpt(ieee754di_indef(), "sp_tlong", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754di_indef();
}
/* oh gawd */
- if (xe > SP_MBITS) {
- xm <<= xe - SP_MBITS;
- } else if (xe < SP_MBITS) {
- u32 residue;
- int round;
- int sticky;
- int odd;
-
+ if (xe > SP_FBITS) {
+ xm <<= xe - SP_FBITS;
+ } else if (xe < SP_FBITS) {
if (xe < -1) {
residue = xm;
round = 0;
sticky = residue != 0;
xm = 0;
} else {
- residue = xm << (32 - SP_MBITS + xe);
+ residue = xm << (32 - SP_FBITS + xe);
round = (residue >> 31) != 0;
sticky = (residue << 1) != 0;
- xm >>= SP_MBITS - xe;
+ xm >>= SP_FBITS - xe;
}
odd = (xm & 0x1) != 0x0;
switch (ieee754_csr.rm) {
- case IEEE754_RN:
+ case FPU_CSR_RN:
if (round && (sticky || odd))
xm++;
break;
- case IEEE754_RZ:
+ case FPU_CSR_RZ:
break;
- case IEEE754_RU: /* toward +Infinity */
+ case FPU_CSR_RU: /* toward +Infinity */
if ((round || sticky) && !xs)
xm++;
break;
- case IEEE754_RD: /* toward -Infinity */
+ case FPU_CSR_RD: /* toward -Infinity */
if ((round || sticky) && xs)
xm++;
break;
}
if ((xm >> 63) != 0) {
/* This can happen after rounding */
- SETCX(IEEE754_INVALID_OPERATION);
- return ieee754di_xcpt(ieee754di_indef(), "sp_tlong", x);
+ ieee754_setcx(IEEE754_INVALID_OPERATION);
+ return ieee754di_indef();
}
if (round || sticky)
- SETCX(IEEE754_INEXACT);
+ ieee754_setcx(IEEE754_INEXACT);
}
if (xs)
return -xm;
else
return xm;
}
-
-
-u64 ieee754sp_tulong(ieee754sp x)
-{
- ieee754sp hb = ieee754sp_1e63();
-
- /* what if x < 0 ?? */
- if (ieee754sp_lt(x, hb))
- return (u64) ieee754sp_tlong(x);
-
- return (u64) ieee754sp_tlong(ieee754sp_sub(x, hb)) |
- (1ULL << 63);
-}
diff --git a/arch/mips/mm/c-octeon.c b/arch/mips/mm/c-octeon.c
index f41a5c5..05b1d7c 100644
--- a/arch/mips/mm/c-octeon.c
+++ b/arch/mips/mm/c-octeon.c
@@ -137,8 +137,10 @@
{
struct vm_area_struct *vma;
+ down_read(¤t->mm->mmap_sem);
vma = find_vma(current->mm, addr);
octeon_flush_icache_all_cores(vma);
+ up_read(¤t->mm->mmap_sem);
}
diff --git a/arch/mips/mm/c-r4k.c b/arch/mips/mm/c-r4k.c
index 1c74a6a..f2e8302 100644
--- a/arch/mips/mm/c-r4k.c
+++ b/arch/mips/mm/c-r4k.c
@@ -7,6 +7,7 @@
* Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002 Ralf Baechle (ralf@gnu.org)
* Copyright (C) 1999, 2000 Silicon Graphics, Inc.
*/
+#include <linux/cpu_pm.h>
#include <linux/hardirq.h>
#include <linux/init.h>
#include <linux/highmem.h>
@@ -50,7 +51,7 @@
{
preempt_disable();
-#if !defined(CONFIG_MIPS_MT_SMP) && !defined(CONFIG_MIPS_MT_SMTC)
+#ifndef CONFIG_MIPS_MT_SMP
smp_call_function(func, info, 1);
#endif
func(info);
@@ -105,22 +106,37 @@
static inline void r4k_blast_dcache_page_dc64(unsigned long addr)
{
- R4600_HIT_CACHEOP_WAR_IMPL;
blast_dcache64_page(addr);
}
+static inline void r4k_blast_dcache_page_dc128(unsigned long addr)
+{
+ blast_dcache128_page(addr);
+}
+
static void r4k_blast_dcache_page_setup(void)
{
unsigned long dc_lsize = cpu_dcache_line_size();
- if (dc_lsize == 0)
+ switch (dc_lsize) {
+ case 0:
r4k_blast_dcache_page = (void *)cache_noop;
- else if (dc_lsize == 16)
+ break;
+ case 16:
r4k_blast_dcache_page = blast_dcache16_page;
- else if (dc_lsize == 32)
+ break;
+ case 32:
r4k_blast_dcache_page = r4k_blast_dcache_page_dc32;
- else if (dc_lsize == 64)
+ break;
+ case 64:
r4k_blast_dcache_page = r4k_blast_dcache_page_dc64;
+ break;
+ case 128:
+ r4k_blast_dcache_page = r4k_blast_dcache_page_dc128;
+ break;
+ default:
+ break;
+ }
}
#ifndef CONFIG_EVA
@@ -159,6 +175,8 @@
r4k_blast_dcache_page_indexed = blast_dcache32_page_indexed;
else if (dc_lsize == 64)
r4k_blast_dcache_page_indexed = blast_dcache64_page_indexed;
+ else if (dc_lsize == 128)
+ r4k_blast_dcache_page_indexed = blast_dcache128_page_indexed;
}
void (* r4k_blast_dcache)(void);
@@ -176,6 +194,8 @@
r4k_blast_dcache = blast_dcache32;
else if (dc_lsize == 64)
r4k_blast_dcache = blast_dcache64;
+ else if (dc_lsize == 128)
+ r4k_blast_dcache = blast_dcache128;
}
/* force code alignment (used for TX49XX_ICACHE_INDEX_INV_WAR) */
@@ -265,6 +285,8 @@
r4k_blast_icache_page = blast_icache32_page;
else if (ic_lsize == 64)
r4k_blast_icache_page = blast_icache64_page;
+ else if (ic_lsize == 128)
+ r4k_blast_icache_page = blast_icache128_page;
}
#ifndef CONFIG_EVA
@@ -338,6 +360,8 @@
r4k_blast_icache = blast_icache32;
} else if (ic_lsize == 64)
r4k_blast_icache = blast_icache64;
+ else if (ic_lsize == 128)
+ r4k_blast_icache = blast_icache128;
}
static void (* r4k_blast_scache_page)(unsigned long addr);
@@ -428,7 +452,7 @@
static inline int has_valid_asid(const struct mm_struct *mm)
{
-#if defined(CONFIG_MIPS_MT_SMP) || defined(CONFIG_MIPS_MT_SMTC)
+#ifdef CONFIG_MIPS_MT_SMP
int i;
for_each_online_cpu(i)
@@ -1094,6 +1118,21 @@
c->dcache.waybit = 0;
break;
+ case CPU_CAVIUM_OCTEON3:
+ /* For now lie about the number of ways. */
+ c->icache.linesz = 128;
+ c->icache.sets = 16;
+ c->icache.ways = 8;
+ c->icache.flags |= MIPS_CACHE_VTAG;
+ icache_size = c->icache.sets * c->icache.ways * c->icache.linesz;
+
+ c->dcache.linesz = 128;
+ c->dcache.ways = 8;
+ c->dcache.sets = 8;
+ dcache_size = c->dcache.sets * c->dcache.ways * c->dcache.linesz;
+ c->options |= MIPS_CPU_PREFETCH;
+ break;
+
default:
if (!(config & MIPS_CONF_M))
panic("Don't know how to probe P-caches on this cpu.");
@@ -1414,6 +1453,7 @@
loongson3_sc_init();
return;
+ case CPU_CAVIUM_OCTEON3:
case CPU_XLP:
/* don't need to worry about L2, fully coherent */
return;
@@ -1644,3 +1684,26 @@
coherency_setup();
board_cache_error_setup = r4k_cache_error_setup;
}
+
+static int r4k_cache_pm_notifier(struct notifier_block *self, unsigned long cmd,
+ void *v)
+{
+ switch (cmd) {
+ case CPU_PM_ENTER_FAILED:
+ case CPU_PM_EXIT:
+ coherency_setup();
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block r4k_cache_pm_notifier_block = {
+ .notifier_call = r4k_cache_pm_notifier,
+};
+
+int __init r4k_cache_init_pm(void)
+{
+ return cpu_pm_register_notifier(&r4k_cache_pm_notifier_block);
+}
+arch_initcall(r4k_cache_init_pm);
diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c
index 4fc74c7..6e44133 100644
--- a/arch/mips/mm/init.c
+++ b/arch/mips/mm/init.c
@@ -44,27 +44,6 @@
#include <asm/tlb.h>
#include <asm/fixmap.h>
-/* Atomicity and interruptability */
-#ifdef CONFIG_MIPS_MT_SMTC
-
-#include <asm/mipsmtregs.h>
-
-#define ENTER_CRITICAL(flags) \
- { \
- unsigned int mvpflags; \
- local_irq_save(flags);\
- mvpflags = dvpe()
-#define EXIT_CRITICAL(flags) \
- evpe(mvpflags); \
- local_irq_restore(flags); \
- }
-#else
-
-#define ENTER_CRITICAL(flags) local_irq_save(flags)
-#define EXIT_CRITICAL(flags) local_irq_restore(flags)
-
-#endif /* CONFIG_MIPS_MT_SMTC */
-
/*
* We have up to 8 empty zeroed pages so we can map one of the right colour
* when needed. This is necessary only on R4000 / R4400 SC and MC versions
@@ -100,21 +79,7 @@
zero_page_mask = ((PAGE_SIZE << order) - 1) & PAGE_MASK;
}
-#ifdef CONFIG_MIPS_MT_SMTC
-static pte_t *kmap_coherent_pte;
-static void __init kmap_coherent_init(void)
-{
- unsigned long vaddr;
-
- /* cache the first coherent kmap pte */
- vaddr = __fix_to_virt(FIX_CMAP_BEGIN);
- kmap_coherent_pte = kmap_get_fixmap_pte(vaddr);
-}
-#else
-static inline void kmap_coherent_init(void) {}
-#endif
-
-void *kmap_coherent(struct page *page, unsigned long addr)
+static void *__kmap_pgprot(struct page *page, unsigned long addr, pgprot_t prot)
{
enum fixed_addresses idx;
unsigned long vaddr, flags, entrylo;
@@ -126,58 +91,48 @@
pagefault_disable();
idx = (addr >> PAGE_SHIFT) & (FIX_N_COLOURS - 1);
-#ifdef CONFIG_MIPS_MT_SMTC
- idx += FIX_N_COLOURS * smp_processor_id() +
- (in_interrupt() ? (FIX_N_COLOURS * NR_CPUS) : 0);
-#else
idx += in_interrupt() ? FIX_N_COLOURS : 0;
-#endif
vaddr = __fix_to_virt(FIX_CMAP_END - idx);
- pte = mk_pte(page, PAGE_KERNEL);
+ pte = mk_pte(page, prot);
#if defined(CONFIG_64BIT_PHYS_ADDR) && defined(CONFIG_CPU_MIPS32)
entrylo = pte.pte_high;
#else
entrylo = pte_to_entrylo(pte_val(pte));
#endif
- ENTER_CRITICAL(flags);
+ local_irq_save(flags);
old_ctx = read_c0_entryhi();
write_c0_entryhi(vaddr & (PAGE_MASK << 1));
write_c0_entrylo0(entrylo);
write_c0_entrylo1(entrylo);
-#ifdef CONFIG_MIPS_MT_SMTC
- set_pte(kmap_coherent_pte - (FIX_CMAP_END - idx), pte);
- /* preload TLB instead of local_flush_tlb_one() */
- mtc0_tlbw_hazard();
- tlb_probe();
- tlb_probe_hazard();
- tlbidx = read_c0_index();
- mtc0_tlbw_hazard();
- if (tlbidx < 0)
- tlb_write_random();
- else
- tlb_write_indexed();
-#else
tlbidx = read_c0_wired();
write_c0_wired(tlbidx + 1);
write_c0_index(tlbidx);
mtc0_tlbw_hazard();
tlb_write_indexed();
-#endif
tlbw_use_hazard();
write_c0_entryhi(old_ctx);
- EXIT_CRITICAL(flags);
+ local_irq_restore(flags);
return (void*) vaddr;
}
+void *kmap_coherent(struct page *page, unsigned long addr)
+{
+ return __kmap_pgprot(page, addr, PAGE_KERNEL);
+}
+
+void *kmap_noncoherent(struct page *page, unsigned long addr)
+{
+ return __kmap_pgprot(page, addr, PAGE_KERNEL_NC);
+}
+
void kunmap_coherent(void)
{
-#ifndef CONFIG_MIPS_MT_SMTC
unsigned int wired;
unsigned long flags, old_ctx;
- ENTER_CRITICAL(flags);
+ local_irq_save(flags);
old_ctx = read_c0_entryhi();
wired = read_c0_wired() - 1;
write_c0_wired(wired);
@@ -189,8 +144,7 @@
tlb_write_indexed();
tlbw_use_hazard();
write_c0_entryhi(old_ctx);
- EXIT_CRITICAL(flags);
-#endif
+ local_irq_restore(flags);
pagefault_enable();
}
@@ -256,7 +210,7 @@
void __init fixrange_init(unsigned long start, unsigned long end,
pgd_t *pgd_base)
{
-#if defined(CONFIG_HIGHMEM) || defined(CONFIG_MIPS_MT_SMTC)
+#ifdef CONFIG_HIGHMEM
pgd_t *pgd;
pud_t *pud;
pmd_t *pmd;
@@ -327,8 +281,6 @@
#ifdef CONFIG_HIGHMEM
kmap_init();
#endif
- kmap_coherent_init();
-
#ifdef CONFIG_ZONE_DMA
max_zone_pfns[ZONE_DMA] = MAX_DMA_PFN;
#endif
diff --git a/arch/mips/mm/tlb-r4k.c b/arch/mips/mm/tlb-r4k.c
index eeaf50f..3914e27 100644
--- a/arch/mips/mm/tlb-r4k.c
+++ b/arch/mips/mm/tlb-r4k.c
@@ -8,6 +8,7 @@
* Carsten Langgaard, carstenl@mips.com
* Copyright (C) 2002 MIPS Technologies, Inc. All rights reserved.
*/
+#include <linux/cpu_pm.h>
#include <linux/init.h>
#include <linux/sched.h>
#include <linux/smp.h>
@@ -25,28 +26,6 @@
extern void build_tlb_refill_handler(void);
-/* Atomicity and interruptability */
-#ifdef CONFIG_MIPS_MT_SMTC
-
-#include <asm/smtc.h>
-#include <asm/mipsmtregs.h>
-
-#define ENTER_CRITICAL(flags) \
- { \
- unsigned int mvpflags; \
- local_irq_save(flags);\
- mvpflags = dvpe()
-#define EXIT_CRITICAL(flags) \
- evpe(mvpflags); \
- local_irq_restore(flags); \
- }
-#else
-
-#define ENTER_CRITICAL(flags) local_irq_save(flags)
-#define EXIT_CRITICAL(flags) local_irq_restore(flags)
-
-#endif /* CONFIG_MIPS_MT_SMTC */
-
/*
* LOONGSON2/3 has a 4 entry itlb which is a subset of dtlb,
* unfortunately, itlb is not totally transparent to software.
@@ -75,7 +54,7 @@
unsigned long old_ctx;
int entry, ftlbhighset;
- ENTER_CRITICAL(flags);
+ local_irq_save(flags);
/* Save old context and create impossible VPN2 value */
old_ctx = read_c0_entryhi();
write_c0_entrylo0(0);
@@ -112,7 +91,7 @@
tlbw_use_hazard();
write_c0_entryhi(old_ctx);
flush_itlb();
- EXIT_CRITICAL(flags);
+ local_irq_restore(flags);
}
EXPORT_SYMBOL(local_flush_tlb_all);
@@ -142,7 +121,7 @@
if (cpu_context(cpu, mm) != 0) {
unsigned long size, flags;
- ENTER_CRITICAL(flags);
+ local_irq_save(flags);
start = round_down(start, PAGE_SIZE << 1);
end = round_up(end, PAGE_SIZE << 1);
size = (end - start) >> (PAGE_SHIFT + 1);
@@ -176,7 +155,7 @@
drop_mmu_context(mm, cpu);
}
flush_itlb();
- EXIT_CRITICAL(flags);
+ local_irq_restore(flags);
}
}
@@ -184,7 +163,7 @@
{
unsigned long size, flags;
- ENTER_CRITICAL(flags);
+ local_irq_save(flags);
size = (end - start + (PAGE_SIZE - 1)) >> PAGE_SHIFT;
size = (size + 1) >> 1;
if (size <= (current_cpu_data.tlbsizeftlbsets ?
@@ -220,7 +199,7 @@
local_flush_tlb_all();
}
flush_itlb();
- EXIT_CRITICAL(flags);
+ local_irq_restore(flags);
}
void local_flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
@@ -233,7 +212,7 @@
newpid = cpu_asid(cpu, vma->vm_mm);
page &= (PAGE_MASK << 1);
- ENTER_CRITICAL(flags);
+ local_irq_save(flags);
oldpid = read_c0_entryhi();
write_c0_entryhi(page | newpid);
mtc0_tlbw_hazard();
@@ -253,7 +232,7 @@
finish:
write_c0_entryhi(oldpid);
flush_itlb_vm(vma);
- EXIT_CRITICAL(flags);
+ local_irq_restore(flags);
}
}
@@ -266,7 +245,7 @@
unsigned long flags;
int oldpid, idx;
- ENTER_CRITICAL(flags);
+ local_irq_save(flags);
oldpid = read_c0_entryhi();
page &= (PAGE_MASK << 1);
write_c0_entryhi(page);
@@ -285,7 +264,7 @@
}
write_c0_entryhi(oldpid);
flush_itlb();
- EXIT_CRITICAL(flags);
+ local_irq_restore(flags);
}
/*
@@ -308,7 +287,7 @@
if (current->active_mm != vma->vm_mm)
return;
- ENTER_CRITICAL(flags);
+ local_irq_save(flags);
pid = read_c0_entryhi() & ASID_MASK;
address &= (PAGE_MASK << 1);
@@ -358,7 +337,7 @@
}
tlbw_use_hazard();
flush_itlb_vm(vma);
- EXIT_CRITICAL(flags);
+ local_irq_restore(flags);
}
void add_wired_entry(unsigned long entrylo0, unsigned long entrylo1,
@@ -369,7 +348,7 @@
unsigned long old_pagemask;
unsigned long old_ctx;
- ENTER_CRITICAL(flags);
+ local_irq_save(flags);
/* Save old context and create impossible VPN2 value */
old_ctx = read_c0_entryhi();
old_pagemask = read_c0_pagemask();
@@ -389,7 +368,7 @@
tlbw_use_hazard(); /* What is the hazard here? */
write_c0_pagemask(old_pagemask);
local_flush_tlb_all();
- EXIT_CRITICAL(flags);
+ local_irq_restore(flags);
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
@@ -399,13 +378,13 @@
unsigned int mask;
unsigned long flags;
- ENTER_CRITICAL(flags);
+ local_irq_save(flags);
write_c0_pagemask(PM_HUGE_MASK);
back_to_back_c0_hazard();
mask = read_c0_pagemask();
write_c0_pagemask(PM_DEFAULT_MASK);
- EXIT_CRITICAL(flags);
+ local_irq_restore(flags);
return mask == PM_HUGE_MASK;
}
@@ -421,7 +400,10 @@
__setup("ntlb=", set_ntlb);
-void tlb_init(void)
+/*
+ * Configure TLB (for init or after a CPU has been powered off).
+ */
+static void r4k_tlb_configure(void)
{
/*
* You should never change this register:
@@ -453,6 +435,11 @@
local_flush_tlb_all();
/* Did I tell you that ARC SUCKS? */
+}
+
+void tlb_init(void)
+{
+ r4k_tlb_configure();
if (ntlb) {
if (ntlb > 1 && ntlb <= current_cpu_data.tlbsize) {
@@ -466,3 +453,26 @@
build_tlb_refill_handler();
}
+
+static int r4k_tlb_pm_notifier(struct notifier_block *self, unsigned long cmd,
+ void *v)
+{
+ switch (cmd) {
+ case CPU_PM_ENTER_FAILED:
+ case CPU_PM_EXIT:
+ r4k_tlb_configure();
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block r4k_tlb_pm_notifier_block = {
+ .notifier_call = r4k_tlb_pm_notifier,
+};
+
+static int __init r4k_tlb_init_pm(void)
+{
+ return cpu_pm_register_notifier(&r4k_tlb_pm_notifier_block);
+}
+arch_initcall(r4k_tlb_init_pm);
diff --git a/arch/mips/mm/tlbex.c b/arch/mips/mm/tlbex.c
index f99ec587..e80e10b 100644
--- a/arch/mips/mm/tlbex.c
+++ b/arch/mips/mm/tlbex.c
@@ -1256,7 +1256,7 @@
memset(relocs, 0, sizeof(relocs));
memset(final_handler, 0, sizeof(final_handler));
- if ((scratch_reg >= 0 || scratchpad_available()) && use_bbit_insns()) {
+ if (IS_ENABLED(CONFIG_64BIT) && (scratch_reg >= 0 || scratchpad_available()) && use_bbit_insns()) {
htlb_info = build_fast_tlb_refill_handler(&p, &l, &r, K0, K1,
scratch_reg);
vmalloc_mode = refill_scratch;
diff --git a/arch/mips/mm/uasm-micromips.c b/arch/mips/mm/uasm-micromips.c
index b8d580c..775c280 100644
--- a/arch/mips/mm/uasm-micromips.c
+++ b/arch/mips/mm/uasm-micromips.c
@@ -63,6 +63,7 @@
{ insn_cache, M(mm_pool32b_op, 0, 0, mm_cache_func, 0, 0), RT | RS | SIMM },
{ insn_daddu, 0, 0 },
{ insn_daddiu, 0, 0 },
+ { insn_divu, M(mm_pool32a_op, 0, 0, 0, mm_divu_op, mm_pool32axf_op), RT | RS },
{ insn_dmfc0, 0, 0 },
{ insn_dmtc0, 0, 0 },
{ insn_dsll, 0, 0 },
@@ -78,14 +79,20 @@
{ insn_ext, M(mm_pool32a_op, 0, 0, 0, 0, mm_ext_op), RT | RS | RD | RE },
{ insn_j, M(mm_j32_op, 0, 0, 0, 0, 0), JIMM },
{ insn_jal, M(mm_jal32_op, 0, 0, 0, 0, 0), JIMM },
+ { insn_jalr, M(mm_pool32a_op, 0, 0, 0, mm_jalr_op, mm_pool32axf_op), RT | RS },
{ insn_jr, M(mm_pool32a_op, 0, 0, 0, mm_jalr_op, mm_pool32axf_op), RS },
+ { insn_lb, M(mm_lb32_op, 0, 0, 0, 0, 0), RT | RS | SIMM },
{ insn_ld, 0, 0 },
+ { insn_lh, M(mm_lh32_op, 0, 0, 0, 0, 0), RS | RS | SIMM },
{ insn_ll, M(mm_pool32c_op, 0, 0, (mm_ll_func << 1), 0, 0), RS | RT | SIMM },
{ insn_lld, 0, 0 },
{ insn_lui, M(mm_pool32i_op, mm_lui_op, 0, 0, 0, 0), RS | SIMM },
{ insn_lw, M(mm_lw32_op, 0, 0, 0, 0, 0), RT | RS | SIMM },
{ insn_mfc0, M(mm_pool32a_op, 0, 0, 0, mm_mfc0_op, mm_pool32axf_op), RT | RS | RD },
+ { insn_mfhi, M(mm_pool32a_op, 0, 0, 0, mm_mfhi32_op, mm_pool32axf_op), RS },
+ { insn_mflo, M(mm_pool32a_op, 0, 0, 0, mm_mflo32_op, mm_pool32axf_op), RS },
{ insn_mtc0, M(mm_pool32a_op, 0, 0, 0, mm_mtc0_op, mm_pool32axf_op), RT | RS | RD },
+ { insn_mul, M(mm_pool32a_op, 0, 0, 0, 0, mm_mul_op), RT | RS | RD },
{ insn_or, M(mm_pool32a_op, 0, 0, 0, 0, mm_or32_op), RT | RS | RD },
{ insn_ori, M(mm_ori32_op, 0, 0, 0, 0, 0), RT | RS | UIMM },
{ insn_pref, M(mm_pool32c_op, 0, 0, (mm_pref_func << 1), 0, 0), RT | RS | SIMM },
@@ -94,15 +101,22 @@
{ insn_scd, 0, 0 },
{ insn_sd, 0, 0 },
{ insn_sll, M(mm_pool32a_op, 0, 0, 0, 0, mm_sll32_op), RT | RS | RD },
+ { insn_sllv, M(mm_pool32a_op, 0, 0, 0, 0, mm_sllv32_op), RT | RS | RD },
+ { insn_sltiu, M(mm_sltiu32_op, 0, 0, 0, 0, 0), RT | RS | SIMM },
+ { insn_sltu, M(mm_pool32a_op, 0, 0, 0, 0, mm_sltu_op), RT | RS | RD },
{ insn_sra, M(mm_pool32a_op, 0, 0, 0, 0, mm_sra_op), RT | RS | RD },
{ insn_srl, M(mm_pool32a_op, 0, 0, 0, 0, mm_srl32_op), RT | RS | RD },
+ { insn_srlv, M(mm_pool32a_op, 0, 0, 0, 0, mm_srlv32_op), RT | RS | RD },
{ insn_rotr, M(mm_pool32a_op, 0, 0, 0, 0, mm_rotr_op), RT | RS | RD },
{ insn_subu, M(mm_pool32a_op, 0, 0, 0, 0, mm_subu32_op), RT | RS | RD },
{ insn_sw, M(mm_sw32_op, 0, 0, 0, 0, 0), RT | RS | SIMM },
+ { insn_sync, M(mm_pool32a_op, 0, 0, 0, mm_sync_op, mm_pool32axf_op), RS },
{ insn_tlbp, M(mm_pool32a_op, 0, 0, 0, mm_tlbp_op, mm_pool32axf_op), 0 },
{ insn_tlbr, M(mm_pool32a_op, 0, 0, 0, mm_tlbr_op, mm_pool32axf_op), 0 },
{ insn_tlbwi, M(mm_pool32a_op, 0, 0, 0, mm_tlbwi_op, mm_pool32axf_op), 0 },
{ insn_tlbwr, M(mm_pool32a_op, 0, 0, 0, mm_tlbwr_op, mm_pool32axf_op), 0 },
+ { insn_wait, M(mm_pool32a_op, 0, 0, 0, mm_wait_op, mm_pool32axf_op), SCIMM },
+ { insn_wsbh, M(mm_pool32a_op, 0, 0, 0, mm_wsbh_op, mm_pool32axf_op), RT | RS },
{ insn_xor, M(mm_pool32a_op, 0, 0, 0, 0, mm_xor32_op), RT | RS | RD },
{ insn_xori, M(mm_xori32_op, 0, 0, 0, 0, 0), RT | RS | UIMM },
{ insn_dins, 0, 0 },
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 3abd609..38792c2 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -67,6 +67,7 @@
{ insn_daddu, M(spec_op, 0, 0, 0, 0, daddu_op), RS | RT | RD },
{ insn_dinsm, M(spec3_op, 0, 0, 0, 0, dinsm_op), RS | RT | RD | RE },
{ insn_dins, M(spec3_op, 0, 0, 0, 0, dins_op), RS | RT | RD | RE },
+ { insn_divu, M(spec_op, 0, 0, 0, 0, divu_op), RS | RT },
{ insn_dmfc0, M(cop0_op, dmfc_op, 0, 0, 0, 0), RT | RD | SET},
{ insn_dmtc0, M(cop0_op, dmtc_op, 0, 0, 0, 0), RT | RD | SET},
{ insn_drotr32, M(spec_op, 1, 0, 0, 0, dsrl32_op), RT | RD | RE },
@@ -82,17 +83,23 @@
{ insn_ins, M(spec3_op, 0, 0, 0, 0, ins_op), RS | RT | RD | RE },
{ insn_j, M(j_op, 0, 0, 0, 0, 0), JIMM },
{ insn_jal, M(jal_op, 0, 0, 0, 0, 0), JIMM },
+ { insn_jalr, M(spec_op, 0, 0, 0, 0, jalr_op), RS | RD },
{ insn_j, M(j_op, 0, 0, 0, 0, 0), JIMM },
{ insn_jr, M(spec_op, 0, 0, 0, 0, jr_op), RS },
+ { insn_lb, M(lb_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_ld, M(ld_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_ldx, M(spec3_op, 0, 0, 0, ldx_op, lx_op), RS | RT | RD },
+ { insn_lh, M(lw_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_lld, M(lld_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_ll, M(ll_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_lui, M(lui_op, 0, 0, 0, 0, 0), RT | SIMM },
{ insn_lw, M(lw_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_lwx, M(spec3_op, 0, 0, 0, lwx_op, lx_op), RS | RT | RD },
{ insn_mfc0, M(cop0_op, mfc_op, 0, 0, 0, 0), RT | RD | SET},
+ { insn_mfhi, M(spec_op, 0, 0, 0, 0, mfhi_op), RD },
+ { insn_mflo, M(spec_op, 0, 0, 0, 0, mflo_op), RD },
{ insn_mtc0, M(cop0_op, mtc_op, 0, 0, 0, 0), RT | RD | SET},
+ { insn_mul, M(spec2_op, 0, 0, 0, 0, mul_op), RS | RT | RD},
{ insn_ori, M(ori_op, 0, 0, 0, 0, 0), RS | RT | UIMM },
{ insn_or, M(spec_op, 0, 0, 0, 0, or_op), RS | RT | RD },
{ insn_pref, M(pref_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
@@ -102,17 +109,25 @@
{ insn_sc, M(sc_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_sd, M(sd_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_sll, M(spec_op, 0, 0, 0, 0, sll_op), RT | RD | RE },
+ { insn_sllv, M(spec_op, 0, 0, 0, 0, sllv_op), RS | RT | RD },
+ { insn_sltiu, M(sltiu_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
+ { insn_sltu, M(spec_op, 0, 0, 0, 0, sltu_op), RS | RT | RD },
{ insn_sra, M(spec_op, 0, 0, 0, 0, sra_op), RT | RD | RE },
{ insn_srl, M(spec_op, 0, 0, 0, 0, srl_op), RT | RD | RE },
+ { insn_srlv, M(spec_op, 0, 0, 0, 0, srlv_op), RS | RT | RD },
{ insn_subu, M(spec_op, 0, 0, 0, 0, subu_op), RS | RT | RD },
{ insn_sw, M(sw_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
+ { insn_sync, M(spec_op, 0, 0, 0, 0, sync_op), RE },
{ insn_syscall, M(spec_op, 0, 0, 0, 0, syscall_op), SCIMM},
{ insn_tlbp, M(cop0_op, cop_op, 0, 0, 0, tlbp_op), 0 },
{ insn_tlbr, M(cop0_op, cop_op, 0, 0, 0, tlbr_op), 0 },
{ insn_tlbwi, M(cop0_op, cop_op, 0, 0, 0, tlbwi_op), 0 },
{ insn_tlbwr, M(cop0_op, cop_op, 0, 0, 0, tlbwr_op), 0 },
+ { insn_wait, M(cop0_op, cop_op, 0, 0, 0, wait_op), SCIMM },
+ { insn_wsbh, M(spec3_op, 0, 0, 0, wsbh_op, bshfl_op), RT | RD },
{ insn_xori, M(xori_op, 0, 0, 0, 0, 0), RS | RT | UIMM },
{ insn_xor, M(spec_op, 0, 0, 0, 0, xor_op), RS | RT | RD },
+ { insn_yield, M(spec3_op, 0, 0, 0, 0, yield_op), RS | RD },
{ insn_invalid, 0, 0 }
};
diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
index b9d14b6..0051580 100644
--- a/arch/mips/mm/uasm.c
+++ b/arch/mips/mm/uasm.c
@@ -47,14 +47,16 @@
insn_addiu, insn_addu, insn_and, insn_andi, insn_bbit0, insn_bbit1,
insn_beq, insn_beql, insn_bgez, insn_bgezl, insn_bltz, insn_bltzl,
insn_bne, insn_cache, insn_daddiu, insn_daddu, insn_dins, insn_dinsm,
- insn_dmfc0, insn_dmtc0, insn_drotr, insn_drotr32, insn_dsll,
+ insn_divu, insn_dmfc0, insn_dmtc0, insn_drotr, insn_drotr32, insn_dsll,
insn_dsll32, insn_dsra, insn_dsrl, insn_dsrl32, insn_dsubu, insn_eret,
- insn_ext, insn_ins, insn_j, insn_jal, insn_jr, insn_ld, insn_ldx,
- insn_ll, insn_lld, insn_lui, insn_lw, insn_lwx, insn_mfc0, insn_mtc0,
+ insn_ext, insn_ins, insn_j, insn_jal, insn_jalr, insn_jr, insn_lb,
+ insn_ld, insn_ldx, insn_lh, insn_ll, insn_lld, insn_lui, insn_lw,
+ insn_lwx, insn_mfc0, insn_mfhi, insn_mflo, insn_mtc0, insn_mul,
insn_or, insn_ori, insn_pref, insn_rfe, insn_rotr, insn_sc, insn_scd,
- insn_sd, insn_sll, insn_sra, insn_srl, insn_subu, insn_sw,
- insn_syscall, insn_tlbp, insn_tlbr, insn_tlbwi, insn_tlbwr, insn_xor,
- insn_xori,
+ insn_sd, insn_sll, insn_sllv, insn_sltiu, insn_sltu, insn_sra,
+ insn_srl, insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall,
+ insn_tlbp, insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh,
+ insn_xor, insn_xori, insn_yield,
};
struct insn {
@@ -144,6 +146,13 @@
} \
UASM_EXPORT_SYMBOL(uasm_i##op);
+#define I_u3u2u1(op) \
+Ip_u3u2u1(op) \
+{ \
+ build_insn(buf, insn##op, c, b, a); \
+} \
+UASM_EXPORT_SYMBOL(uasm_i##op);
+
#define I_u3u1u2(op) \
Ip_u3u1u2(op) \
{ \
@@ -200,6 +209,13 @@
} \
UASM_EXPORT_SYMBOL(uasm_i##op);
+#define I_u2u1(op) \
+Ip_u1u2(op) \
+{ \
+ build_insn(buf, insn##op, b, a); \
+} \
+UASM_EXPORT_SYMBOL(uasm_i##op);
+
#define I_u1s2(op) \
Ip_u1s2(op) \
{ \
@@ -237,6 +253,7 @@
I_u1u2u3(_dmtc0)
I_u2u1s3(_daddiu)
I_u3u1u2(_daddu)
+I_u1u2(_divu)
I_u2u1u3(_dsll)
I_u2u1u3(_dsll32)
I_u2u1u3(_dsra)
@@ -250,14 +267,20 @@
I_u2u1msbu3(_ins)
I_u1(_j)
I_u1(_jal)
+I_u2u1(_jalr)
I_u1(_jr)
+I_u2s3u1(_lb)
I_u2s3u1(_ld)
+I_u2s3u1(_lh)
I_u2s3u1(_ll)
I_u2s3u1(_lld)
I_u1s2(_lui)
I_u2s3u1(_lw)
I_u1u2u3(_mfc0)
+I_u1(_mfhi)
+I_u1(_mflo)
I_u1u2u3(_mtc0)
+I_u3u1u2(_mul)
I_u2u1u3(_ori)
I_u3u1u2(_or)
I_0(_rfe)
@@ -265,17 +288,25 @@
I_u2s3u1(_scd)
I_u2s3u1(_sd)
I_u2u1u3(_sll)
+I_u3u2u1(_sllv)
+I_u2u1s3(_sltiu)
+I_u3u1u2(_sltu)
I_u2u1u3(_sra)
I_u2u1u3(_srl)
+I_u3u2u1(_srlv)
I_u2u1u3(_rotr)
I_u3u1u2(_subu)
I_u2s3u1(_sw)
+I_u1(_sync)
I_0(_tlbp)
I_0(_tlbr)
I_0(_tlbwi)
I_0(_tlbwr)
+I_u1(_wait);
+I_u2u1(_wsbh)
I_u3u1u2(_xor)
I_u2u1u3(_xori)
+I_u2u1(_yield)
I_u2u1msbu3(_dins);
I_u2u1msb32u3(_dinsm);
I_u1(_syscall);
@@ -469,6 +500,14 @@
}
UASM_EXPORT_SYMBOL(ISAFUNC(uasm_il_b));
+void ISAFUNC(uasm_il_beq)(u32 **p, struct uasm_reloc **r, unsigned int r1,
+ unsigned int r2, int lid)
+{
+ uasm_r_mips_pc16(r, *p, lid);
+ ISAFUNC(uasm_i_beq)(p, r1, r2, 0);
+}
+UASM_EXPORT_SYMBOL(ISAFUNC(uasm_il_beq));
+
void ISAFUNC(uasm_il_beqz)(u32 **p, struct uasm_reloc **r, unsigned int reg,
int lid)
{
diff --git a/arch/mips/mti-malta/Makefile b/arch/mips/mti-malta/Makefile
index eae0ba3..b9510ea 100644
--- a/arch/mips/mti-malta/Makefile
+++ b/arch/mips/mti-malta/Makefile
@@ -9,5 +9,4 @@
malta-int.o malta-memory.o malta-platform.o \
malta-reset.o malta-setup.o malta-time.o
-# FIXME FIXME FIXME
-obj-$(CONFIG_MIPS_MT_SMTC) += malta-smtc.o
+obj-$(CONFIG_MIPS_MALTA_PM) += malta-pm.o
diff --git a/arch/mips/mti-malta/malta-init.c b/arch/mips/mti-malta/malta-init.c
index 4f9e44d..0f60256 100644
--- a/arch/mips/mti-malta/malta-init.c
+++ b/arch/mips/mti-malta/malta-init.c
@@ -116,8 +116,6 @@
return CPC_BASE_ADDR;
}
-extern struct plat_smp_ops msmtc_smp_ops;
-
void __init prom_init(void)
{
mips_display_message("LINUX");
@@ -304,8 +302,4 @@
return;
if (!register_vsmp_smp_ops())
return;
-
-#ifdef CONFIG_MIPS_MT_SMTC
- register_smp_ops(&msmtc_smp_ops);
-#endif
}
diff --git a/arch/mips/mti-malta/malta-int.c b/arch/mips/mti-malta/malta-int.c
index b71ee80..ecc2785 100644
--- a/arch/mips/mti-malta/malta-int.c
+++ b/arch/mips/mti-malta/malta-int.c
@@ -504,28 +504,9 @@
} else if (cpu_has_vint) {
set_vi_handler(MIPSCPU_INT_I8259A, malta_hw0_irqdispatch);
set_vi_handler(MIPSCPU_INT_COREHI, corehi_irqdispatch);
-#ifdef CONFIG_MIPS_MT_SMTC
- setup_irq_smtc(MIPS_CPU_IRQ_BASE+MIPSCPU_INT_I8259A, &i8259irq,
- (0x100 << MIPSCPU_INT_I8259A));
- setup_irq_smtc(MIPS_CPU_IRQ_BASE+MIPSCPU_INT_COREHI,
- &corehi_irqaction, (0x100 << MIPSCPU_INT_COREHI));
- /*
- * Temporary hack to ensure that the subsidiary device
- * interrupts coing in via the i8259A, but associated
- * with low IRQ numbers, will restore the Status.IM
- * value associated with the i8259A.
- */
- {
- int i;
-
- for (i = 0; i < 16; i++)
- irq_hwmask[i] = (0x100 << MIPSCPU_INT_I8259A);
- }
-#else /* Not SMTC */
setup_irq(MIPS_CPU_IRQ_BASE+MIPSCPU_INT_I8259A, &i8259irq);
setup_irq(MIPS_CPU_IRQ_BASE+MIPSCPU_INT_COREHI,
&corehi_irqaction);
-#endif /* CONFIG_MIPS_MT_SMTC */
} else {
setup_irq(MIPS_CPU_IRQ_BASE+MIPSCPU_INT_I8259A, &i8259irq);
setup_irq(MIPS_CPU_IRQ_BASE+MIPSCPU_INT_COREHI,
diff --git a/arch/mips/mti-malta/malta-memory.c b/arch/mips/mti-malta/malta-memory.c
index f2364e4..6d97730 100644
--- a/arch/mips/mti-malta/malta-memory.c
+++ b/arch/mips/mti-malta/malta-memory.c
@@ -26,8 +26,8 @@
fw_memblock_t * __init fw_getmdesc(int eva)
{
- char *memsize_str, *ememsize_str __maybe_unused = NULL, *ptr;
- unsigned long memsize = 0, ememsize __maybe_unused = 0;
+ char *memsize_str, *ememsize_str = NULL, *ptr;
+ unsigned long memsize = 0, ememsize = 0;
static char cmdline[COMMAND_LINE_SIZE] __initdata;
int tmp;
diff --git a/arch/mips/mti-malta/malta-pm.c b/arch/mips/mti-malta/malta-pm.c
new file mode 100644
index 0000000..c1e456c
--- /dev/null
+++ b/arch/mips/mti-malta/malta-pm.c
@@ -0,0 +1,96 @@
+/*
+ * Copyright (C) 2014 Imagination Technologies
+ * Author: Paul Burton <paul.burton@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/delay.h>
+#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/pci.h>
+
+#include <asm/mach-malta/malta-pm.h>
+
+static struct pci_bus *pm_pci_bus;
+static resource_size_t pm_io_offset;
+
+int mips_pm_suspend(unsigned state)
+{
+ int spec_devid;
+ u16 sts;
+
+ if (!pm_pci_bus || !pm_io_offset)
+ return -ENODEV;
+
+ /* Ensure the power button status is clear */
+ while (1) {
+ sts = inw(pm_io_offset + PIIX4_FUNC3IO_PMSTS);
+ if (!(sts & PIIX4_FUNC3IO_PMSTS_PWRBTN_STS))
+ break;
+ outw(sts, pm_io_offset + PIIX4_FUNC3IO_PMSTS);
+ }
+
+ /* Enable entry to suspend */
+ outw(state | PIIX4_FUNC3IO_PMCNTRL_SUS_EN,
+ pm_io_offset + PIIX4_FUNC3IO_PMCNTRL);
+
+ /* If the special cycle occurs too soon this doesn't work... */
+ mdelay(10);
+
+ /*
+ * The PIIX4 will enter the suspend state only after seeing a special
+ * cycle with the correct magic data on the PCI bus. Generate that
+ * cycle now.
+ */
+ spec_devid = PCI_DEVID(0, PCI_DEVFN(0x1f, 0x7));
+ pci_bus_write_config_dword(pm_pci_bus, spec_devid, 0,
+ PIIX4_SUSPEND_MAGIC);
+
+ /* Give the system some time to power down */
+ mdelay(1000);
+
+ return 0;
+}
+
+static int __init malta_pm_setup(void)
+{
+ struct pci_dev *dev;
+ int res, io_region = PCI_BRIDGE_RESOURCES;
+
+ /* Find a reference to the PCI bus */
+ pm_pci_bus = pci_find_next_bus(NULL);
+ if (!pm_pci_bus) {
+ pr_warn("malta-pm: failed to find reference to PCI bus\n");
+ return -ENODEV;
+ }
+
+ /* Find the PIIX4 PM device */
+ dev = pci_get_subsys(PCI_VENDOR_ID_INTEL,
+ PCI_DEVICE_ID_INTEL_82371AB_3, PCI_ANY_ID,
+ PCI_ANY_ID, NULL);
+ if (!dev) {
+ pr_warn("malta-pm: failed to find PIIX4 PM\n");
+ return -ENODEV;
+ }
+
+ /* Request access to the PIIX4 PM IO registers */
+ res = pci_request_region(dev, io_region, "PIIX4 PM IO registers");
+ if (res) {
+ pr_warn("malta-pm: failed to request PM IO registers (%d)\n",
+ res);
+ pci_dev_put(dev);
+ return -ENODEV;
+ }
+
+ /* Find the offset to the PIIX4 PM IO registers */
+ pm_io_offset = pci_resource_start(dev, io_region);
+
+ pci_dev_put(dev);
+ return 0;
+}
+
+late_initcall(malta_pm_setup);
diff --git a/arch/mips/mti-malta/malta-reset.c b/arch/mips/mti-malta/malta-reset.c
index d627d4b..2fd2cc2 100644
--- a/arch/mips/mti-malta/malta-reset.c
+++ b/arch/mips/mti-malta/malta-reset.c
@@ -10,6 +10,7 @@
#include <linux/pm.h>
#include <asm/reboot.h>
+#include <asm/mach-malta/malta-pm.h>
#define SOFTRES_REG 0x1f000500
#define GORESET 0x42
@@ -24,17 +25,22 @@
static void mips_machine_halt(void)
{
- unsigned int __iomem *softres_reg =
- ioremap(SOFTRES_REG, sizeof(unsigned int));
+ while (true);
+}
- __raw_writel(GORESET, softres_reg);
+static void mips_machine_power_off(void)
+{
+ mips_pm_suspend(PIIX4_FUNC3IO_PMCNTRL_SUS_TYP_SOFF);
+
+ pr_info("Failed to power down, resetting\n");
+ mips_machine_restart(NULL);
}
static int __init mips_reboot_setup(void)
{
_machine_restart = mips_machine_restart;
_machine_halt = mips_machine_halt;
- pm_power_off = mips_machine_halt;
+ pm_power_off = mips_machine_power_off;
return 0;
}
diff --git a/arch/mips/mti-malta/malta-setup.c b/arch/mips/mti-malta/malta-setup.c
index bf62151..db7c9e5 100644
--- a/arch/mips/mti-malta/malta-setup.c
+++ b/arch/mips/mti-malta/malta-setup.c
@@ -77,11 +77,7 @@
return "MIPS Malta";
}
-#if defined(CONFIG_MIPS_MT_SMTC)
-const char display_string[] = " SMTC LINUX ON MALTA ";
-#else
const char display_string[] = " LINUX ON MALTA ";
-#endif /* CONFIG_MIPS_MT_SMTC */
#ifdef CONFIG_BLK_DEV_FD
static void __init fd_activate(void)
diff --git a/arch/mips/mti-malta/malta-smtc.c b/arch/mips/mti-malta/malta-smtc.c
deleted file mode 100644
index c484990..0000000
--- a/arch/mips/mti-malta/malta-smtc.c
+++ /dev/null
@@ -1,162 +0,0 @@
-/*
- * Malta Platform-specific hooks for SMP operation
- */
-#include <linux/irq.h>
-#include <linux/init.h>
-
-#include <asm/mipsregs.h>
-#include <asm/mipsmtregs.h>
-#include <asm/smtc.h>
-#include <asm/smtc_ipi.h>
-
-/* VPE/SMP Prototype implements platform interfaces directly */
-
-/*
- * Cause the specified action to be performed on a targeted "CPU"
- */
-
-static void msmtc_send_ipi_single(int cpu, unsigned int action)
-{
- /* "CPU" may be TC of same VPE, VPE of same CPU, or different CPU */
- smtc_send_ipi(cpu, LINUX_SMP_IPI, action);
-}
-
-static void msmtc_send_ipi_mask(const struct cpumask *mask, unsigned int action)
-{
- unsigned int i;
-
- for_each_cpu(i, mask)
- msmtc_send_ipi_single(i, action);
-}
-
-/*
- * Post-config but pre-boot cleanup entry point
- */
-static void msmtc_init_secondary(void)
-{
- int myvpe;
-
- /* Don't enable Malta I/O interrupts (IP2) for secondary VPEs */
- myvpe = read_c0_tcbind() & TCBIND_CURVPE;
- if (myvpe != 0) {
- /* Ideally, this should be done only once per VPE, but... */
- clear_c0_status(ST0_IM);
- set_c0_status((0x100 << cp0_compare_irq)
- | (0x100 << MIPS_CPU_IPI_IRQ));
- if (cp0_perfcount_irq >= 0)
- set_c0_status(0x100 << cp0_perfcount_irq);
- }
-
- smtc_init_secondary();
-}
-
-/*
- * Platform "CPU" startup hook
- */
-static void msmtc_boot_secondary(int cpu, struct task_struct *idle)
-{
- smtc_boot_secondary(cpu, idle);
-}
-
-/*
- * SMP initialization finalization entry point
- */
-static void msmtc_smp_finish(void)
-{
- smtc_smp_finish();
-}
-
-/*
- * Hook for after all CPUs are online
- */
-
-static void msmtc_cpus_done(void)
-{
-}
-
-/*
- * Platform SMP pre-initialization
- *
- * As noted above, we can assume a single CPU for now
- * but it may be multithreaded.
- */
-
-static void __init msmtc_smp_setup(void)
-{
- /*
- * we won't get the definitive value until
- * we've run smtc_prepare_cpus later, but
- * we would appear to need an upper bound now.
- */
- smp_num_siblings = smtc_build_cpu_map(0);
-}
-
-static void __init msmtc_prepare_cpus(unsigned int max_cpus)
-{
- smtc_prepare_cpus(max_cpus);
-}
-
-struct plat_smp_ops msmtc_smp_ops = {
- .send_ipi_single = msmtc_send_ipi_single,
- .send_ipi_mask = msmtc_send_ipi_mask,
- .init_secondary = msmtc_init_secondary,
- .smp_finish = msmtc_smp_finish,
- .cpus_done = msmtc_cpus_done,
- .boot_secondary = msmtc_boot_secondary,
- .smp_setup = msmtc_smp_setup,
- .prepare_cpus = msmtc_prepare_cpus,
-};
-
-#ifdef CONFIG_MIPS_MT_SMTC_IRQAFF
-/*
- * IRQ affinity hook
- */
-
-
-int plat_set_irq_affinity(struct irq_data *d, const struct cpumask *affinity,
- bool force)
-{
- cpumask_t tmask;
- int cpu = 0;
- void smtc_set_irq_affinity(unsigned int irq, cpumask_t aff);
-
- /*
- * On the legacy Malta development board, all I/O interrupts
- * are routed through the 8259 and combined in a single signal
- * to the CPU daughterboard, and on the CoreFPGA2/3 34K models,
- * that signal is brought to IP2 of both VPEs. To avoid racing
- * concurrent interrupt service events, IP2 is enabled only on
- * one VPE, by convention VPE0. So long as no bits are ever
- * cleared in the affinity mask, there will never be any
- * interrupt forwarding. But as soon as a program or operator
- * sets affinity for one of the related IRQs, we need to make
- * sure that we don't ever try to forward across the VPE boundary,
- * at least not until we engineer a system where the interrupt
- * _ack() or _end() function can somehow know that it corresponds
- * to an interrupt taken on another VPE, and perform the appropriate
- * restoration of Status.IM state using MFTR/MTTR instead of the
- * normal local behavior. We also ensure that no attempt will
- * be made to forward to an offline "CPU".
- */
-
- cpumask_copy(&tmask, affinity);
- for_each_cpu(cpu, affinity) {
- if ((cpu_data[cpu].vpe_id != 0) || !cpu_online(cpu))
- cpu_clear(cpu, tmask);
- }
- cpumask_copy(d->affinity, &tmask);
-
- if (cpus_empty(tmask))
- /*
- * We could restore a default mask here, but the
- * runtime code can anyway deal with the null set
- */
- printk(KERN_WARNING
- "IRQ affinity leaves no legal CPU for IRQ %d\n", d->irq);
-
- /* Do any generic SMTC IRQ affinity setup */
- smtc_set_irq_affinity(d->irq, tmask);
-
- return IRQ_SET_MASK_OK_NOCOPY;
-}
-#endif /* CONFIG_MIPS_MT_SMTC_IRQAFF */
diff --git a/arch/mips/mti-sead3/sead3-pic32-i2c-drv.c b/arch/mips/mti-sead3/sead3-pic32-i2c-drv.c
index b921e5ec..80fe194 100644
--- a/arch/mips/mti-sead3/sead3-pic32-i2c-drv.c
+++ b/arch/mips/mti-sead3/sead3-pic32-i2c-drv.c
@@ -312,16 +312,13 @@
pr_debug("i2c_platform_probe\n");
r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (!r) {
- ret = -ENODEV;
- goto out;
- }
+ if (!r)
+ return -ENODEV;
- priv = kzalloc(sizeof(struct i2c_platform_data), GFP_KERNEL);
- if (!priv) {
- ret = -ENOMEM;
- goto out;
- }
+ priv = devm_kzalloc(&pdev->dev, sizeof(struct i2c_platform_data),
+ GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
/* FIXME: need to allocate resource in PIC32 space */
#if 0
@@ -330,10 +327,8 @@
#else
priv->base = r->start;
#endif
- if (!priv->base) {
- ret = -EBUSY;
- goto out_mem;
- }
+ if (!priv->base)
+ return -EBUSY;
priv->xfer_timeout = 200;
priv->ack_timeout = 200;
@@ -348,17 +343,13 @@
i2c_platform_setup(priv);
ret = i2c_add_numbered_adapter(&priv->adap);
- if (ret == 0) {
- platform_set_drvdata(pdev, priv);
- return 0;
+ if (ret) {
+ i2c_platform_disable(priv);
+ return ret;
}
- i2c_platform_disable(priv);
-
-out_mem:
- kfree(priv);
-out:
- return ret;
+ platform_set_drvdata(pdev, priv);
+ return 0;
}
static int i2c_platform_remove(struct platform_device *pdev)
@@ -369,7 +360,6 @@
platform_set_drvdata(pdev, NULL);
i2c_del_adapter(&priv->adap);
i2c_platform_disable(priv);
- kfree(priv);
return 0;
}
diff --git a/arch/mips/net/Makefile b/arch/mips/net/Makefile
new file mode 100644
index 0000000..ae74b3a
--- /dev/null
+++ b/arch/mips/net/Makefile
@@ -0,0 +1,3 @@
+# MIPS networking code
+
+obj-$(CONFIG_BPF_JIT) += bpf_jit.o
diff --git a/arch/mips/net/bpf_jit.c b/arch/mips/net/bpf_jit.c
new file mode 100644
index 0000000..a67b975
--- /dev/null
+++ b/arch/mips/net/bpf_jit.c
@@ -0,0 +1,1399 @@
+/*
+ * Just-In-Time compiler for BPF filters on MIPS
+ *
+ * Copyright (c) 2014 Imagination Technologies Ltd.
+ * Author: Markos Chandras <markos.chandras@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; version 2 of the License.
+ */
+
+#include <linux/bitops.h>
+#include <linux/compiler.h>
+#include <linux/errno.h>
+#include <linux/filter.h>
+#include <linux/if_vlan.h>
+#include <linux/kconfig.h>
+#include <linux/moduleloader.h>
+#include <linux/netdevice.h>
+#include <linux/string.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <asm/bitops.h>
+#include <asm/cacheflush.h>
+#include <asm/cpu-features.h>
+#include <asm/uasm.h>
+
+#include "bpf_jit.h"
+
+/* ABI
+ *
+ * s0 1st scratch register
+ * s1 2nd scratch register
+ * s2 offset register
+ * s3 BPF register A
+ * s4 BPF register X
+ * s5 *skb
+ * s6 *scratch memory
+ *
+ * On entry (*bpf_func)(*skb, *filter)
+ * a0 = MIPS_R_A0 = skb;
+ * a1 = MIPS_R_A1 = filter;
+ *
+ * Stack
+ * ...
+ * M[15]
+ * M[14]
+ * M[13]
+ * ...
+ * M[0] <-- r_M
+ * saved reg k-1
+ * saved reg k-2
+ * ...
+ * saved reg 0 <-- r_sp
+ * <no argument area>
+ *
+ * Packet layout
+ *
+ * <--------------------- len ------------------------>
+ * <--skb-len(r_skb_hl)-->< ----- skb->data_len ------>
+ * ----------------------------------------------------
+ * | skb->data |
+ * ----------------------------------------------------
+ */
+
+#define RSIZE (sizeof(unsigned long))
+#define ptr typeof(unsigned long)
+
+/* ABI specific return values */
+#ifdef CONFIG_32BIT /* O32 */
+#ifdef CONFIG_CPU_LITTLE_ENDIAN
+#define r_err MIPS_R_V1
+#define r_val MIPS_R_V0
+#else /* CONFIG_CPU_LITTLE_ENDIAN */
+#define r_err MIPS_R_V0
+#define r_val MIPS_R_V1
+#endif
+#else /* N64 */
+#define r_err MIPS_R_V0
+#define r_val MIPS_R_V0
+#endif
+
+#define r_ret MIPS_R_V0
+
+/*
+ * Use 2 scratch registers to avoid pipeline interlocks.
+ * There is no overhead during epilogue and prologue since
+ * any of the $s0-$s6 registers will only be preserved if
+ * they are going to actually be used.
+ */
+#define r_s0 MIPS_R_S0 /* scratch reg 1 */
+#define r_s1 MIPS_R_S1 /* scratch reg 2 */
+#define r_off MIPS_R_S2
+#define r_A MIPS_R_S3
+#define r_X MIPS_R_S4
+#define r_skb MIPS_R_S5
+#define r_M MIPS_R_S6
+#define r_tmp_imm MIPS_R_T6 /* No need to preserve this */
+#define r_tmp MIPS_R_T7 /* No need to preserve this */
+#define r_zero MIPS_R_ZERO
+#define r_sp MIPS_R_SP
+#define r_ra MIPS_R_RA
+
+#define SCRATCH_OFF(k) (4 * (k))
+
+/* JIT flags */
+#define SEEN_CALL (1 << BPF_MEMWORDS)
+#define SEEN_SREG_SFT (BPF_MEMWORDS + 1)
+#define SEEN_SREG_BASE (1 << SEEN_SREG_SFT)
+#define SEEN_SREG(x) (SEEN_SREG_BASE << (x))
+#define SEEN_S0 SEEN_SREG(0)
+#define SEEN_S1 SEEN_SREG(1)
+#define SEEN_OFF SEEN_SREG(2)
+#define SEEN_A SEEN_SREG(3)
+#define SEEN_X SEEN_SREG(4)
+#define SEEN_SKB SEEN_SREG(5)
+#define SEEN_MEM SEEN_SREG(6)
+
+/* Arguments used by JIT */
+#define ARGS_USED_BY_JIT 2 /* only applicable to 64-bit */
+
+#define FLAG_NEED_X_RESET (1 << 0)
+
+#define SBIT(x) (1 << (x)) /* Signed version of BIT() */
+
+/**
+ * struct jit_ctx - JIT context
+ * @skf: The sk_filter
+ * @prologue_bytes: Number of bytes for prologue
+ * @idx: Instruction index
+ * @flags: JIT flags
+ * @offsets: Instruction offsets
+ * @target: Memory location for the compiled filter
+ */
+struct jit_ctx {
+ const struct sk_filter *skf;
+ unsigned int prologue_bytes;
+ u32 idx;
+ u32 flags;
+ u32 *offsets;
+ u32 *target;
+};
+
+
+static inline int optimize_div(u32 *k)
+{
+ /* power of 2 divides can be implemented with right shift */
+ if (!(*k & (*k-1))) {
+ *k = ilog2(*k);
+ return 1;
+ }
+
+ return 0;
+}
+
+/* Simply emit the instruction if the JIT memory space has been allocated */
+#define emit_instr(ctx, func, ...) \
+do { \
+ if ((ctx)->target != NULL) { \
+ u32 *p = &(ctx)->target[ctx->idx]; \
+ uasm_i_##func(&p, ##__VA_ARGS__); \
+ } \
+ (ctx)->idx++; \
+} while (0)
+
+/* Determine if immediate is within the 16-bit signed range */
+static inline bool is_range16(s32 imm)
+{
+ if (imm >= SBIT(15) || imm < -SBIT(15))
+ return true;
+ return false;
+}
+
+static inline void emit_addu(unsigned int dst, unsigned int src1,
+ unsigned int src2, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, addu, dst, src1, src2);
+}
+
+static inline void emit_nop(struct jit_ctx *ctx)
+{
+ emit_instr(ctx, nop);
+}
+
+/* Load a u32 immediate to a register */
+static inline void emit_load_imm(unsigned int dst, u32 imm, struct jit_ctx *ctx)
+{
+ if (ctx->target != NULL) {
+ /* addiu can only handle s16 */
+ if (is_range16(imm)) {
+ u32 *p = &ctx->target[ctx->idx];
+ uasm_i_lui(&p, r_tmp_imm, (s32)imm >> 16);
+ p = &ctx->target[ctx->idx + 1];
+ uasm_i_ori(&p, dst, r_tmp_imm, imm & 0xffff);
+ } else {
+ u32 *p = &ctx->target[ctx->idx];
+ uasm_i_addiu(&p, dst, r_zero, imm);
+ }
+ }
+ ctx->idx++;
+
+ if (is_range16(imm))
+ ctx->idx++;
+}
+
+static inline void emit_or(unsigned int dst, unsigned int src1,
+ unsigned int src2, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, or, dst, src1, src2);
+}
+
+static inline void emit_ori(unsigned int dst, unsigned src, u32 imm,
+ struct jit_ctx *ctx)
+{
+ if (imm >= BIT(16)) {
+ emit_load_imm(r_tmp, imm, ctx);
+ emit_or(dst, src, r_tmp, ctx);
+ } else {
+ emit_instr(ctx, ori, dst, src, imm);
+ }
+}
+
+
+static inline void emit_daddu(unsigned int dst, unsigned int src1,
+ unsigned int src2, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, daddu, dst, src1, src2);
+}
+
+static inline void emit_daddiu(unsigned int dst, unsigned int src,
+ int imm, struct jit_ctx *ctx)
+{
+ /*
+ * Only used for stack, so the imm is relatively small
+ * and it fits in 15-bits
+ */
+ emit_instr(ctx, daddiu, dst, src, imm);
+}
+
+static inline void emit_addiu(unsigned int dst, unsigned int src,
+ u32 imm, struct jit_ctx *ctx)
+{
+ if (is_range16(imm)) {
+ emit_load_imm(r_tmp, imm, ctx);
+ emit_addu(dst, r_tmp, src, ctx);
+ } else {
+ emit_instr(ctx, addiu, dst, src, imm);
+ }
+}
+
+static inline void emit_and(unsigned int dst, unsigned int src1,
+ unsigned int src2, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, and, dst, src1, src2);
+}
+
+static inline void emit_andi(unsigned int dst, unsigned int src,
+ u32 imm, struct jit_ctx *ctx)
+{
+ /* If imm does not fit in u16 then load it to register */
+ if (imm >= BIT(16)) {
+ emit_load_imm(r_tmp, imm, ctx);
+ emit_and(dst, src, r_tmp, ctx);
+ } else {
+ emit_instr(ctx, andi, dst, src, imm);
+ }
+}
+
+static inline void emit_xor(unsigned int dst, unsigned int src1,
+ unsigned int src2, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, xor, dst, src1, src2);
+}
+
+static inline void emit_xori(ptr dst, ptr src, u32 imm, struct jit_ctx *ctx)
+{
+ /* If imm does not fit in u16 then load it to register */
+ if (imm >= BIT(16)) {
+ emit_load_imm(r_tmp, imm, ctx);
+ emit_xor(dst, src, r_tmp, ctx);
+ } else {
+ emit_instr(ctx, xori, dst, src, imm);
+ }
+}
+
+static inline void emit_stack_offset(int offset, struct jit_ctx *ctx)
+{
+ if (config_enabled(CONFIG_64BIT))
+ emit_instr(ctx, daddiu, r_sp, r_sp, offset);
+ else
+ emit_instr(ctx, addiu, r_sp, r_sp, offset);
+
+}
+
+static inline void emit_subu(unsigned int dst, unsigned int src1,
+ unsigned int src2, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, subu, dst, src1, src2);
+}
+
+static inline void emit_neg(unsigned int reg, struct jit_ctx *ctx)
+{
+ emit_subu(reg, r_zero, reg, ctx);
+}
+
+static inline void emit_sllv(unsigned int dst, unsigned int src,
+ unsigned int sa, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, sllv, dst, src, sa);
+}
+
+static inline void emit_sll(unsigned int dst, unsigned int src,
+ unsigned int sa, struct jit_ctx *ctx)
+{
+ /* sa is 5-bits long */
+ BUG_ON(sa >= BIT(5));
+ emit_instr(ctx, sll, dst, src, sa);
+}
+
+static inline void emit_srlv(unsigned int dst, unsigned int src,
+ unsigned int sa, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, srlv, dst, src, sa);
+}
+
+static inline void emit_srl(unsigned int dst, unsigned int src,
+ unsigned int sa, struct jit_ctx *ctx)
+{
+ /* sa is 5-bits long */
+ BUG_ON(sa >= BIT(5));
+ emit_instr(ctx, srl, dst, src, sa);
+}
+
+static inline void emit_sltu(unsigned int dst, unsigned int src1,
+ unsigned int src2, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, sltu, dst, src1, src2);
+}
+
+static inline void emit_sltiu(unsigned dst, unsigned int src,
+ unsigned int imm, struct jit_ctx *ctx)
+{
+ /* 16 bit immediate */
+ if (is_range16((s32)imm)) {
+ emit_load_imm(r_tmp, imm, ctx);
+ emit_sltu(dst, src, r_tmp, ctx);
+ } else {
+ emit_instr(ctx, sltiu, dst, src, imm);
+ }
+
+}
+
+/* Store register on the stack */
+static inline void emit_store_stack_reg(ptr reg, ptr base,
+ unsigned int offset,
+ struct jit_ctx *ctx)
+{
+ if (config_enabled(CONFIG_64BIT))
+ emit_instr(ctx, sd, reg, offset, base);
+ else
+ emit_instr(ctx, sw, reg, offset, base);
+}
+
+static inline void emit_store(ptr reg, ptr base, unsigned int offset,
+ struct jit_ctx *ctx)
+{
+ emit_instr(ctx, sw, reg, offset, base);
+}
+
+static inline void emit_load_stack_reg(ptr reg, ptr base,
+ unsigned int offset,
+ struct jit_ctx *ctx)
+{
+ if (config_enabled(CONFIG_64BIT))
+ emit_instr(ctx, ld, reg, offset, base);
+ else
+ emit_instr(ctx, lw, reg, offset, base);
+}
+
+static inline void emit_load(unsigned int reg, unsigned int base,
+ unsigned int offset, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, lw, reg, offset, base);
+}
+
+static inline void emit_load_byte(unsigned int reg, unsigned int base,
+ unsigned int offset, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, lb, reg, offset, base);
+}
+
+static inline void emit_half_load(unsigned int reg, unsigned int base,
+ unsigned int offset, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, lh, reg, offset, base);
+}
+
+static inline void emit_mul(unsigned int dst, unsigned int src1,
+ unsigned int src2, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, mul, dst, src1, src2);
+}
+
+static inline void emit_div(unsigned int dst, unsigned int src,
+ struct jit_ctx *ctx)
+{
+ if (ctx->target != NULL) {
+ u32 *p = &ctx->target[ctx->idx];
+ uasm_i_divu(&p, dst, src);
+ p = &ctx->target[ctx->idx + 1];
+ uasm_i_mfhi(&p, dst);
+ }
+ ctx->idx += 2; /* 2 insts */
+}
+
+static inline void emit_mod(unsigned int dst, unsigned int src,
+ struct jit_ctx *ctx)
+{
+ if (ctx->target != NULL) {
+ u32 *p = &ctx->target[ctx->idx];
+ uasm_i_divu(&p, dst, src);
+ p = &ctx->target[ctx->idx + 1];
+ uasm_i_mflo(&p, dst);
+ }
+ ctx->idx += 2; /* 2 insts */
+}
+
+static inline void emit_dsll(unsigned int dst, unsigned int src,
+ unsigned int sa, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, dsll, dst, src, sa);
+}
+
+static inline void emit_dsrl32(unsigned int dst, unsigned int src,
+ unsigned int sa, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, dsrl32, dst, src, sa);
+}
+
+static inline void emit_wsbh(unsigned int dst, unsigned int src,
+ struct jit_ctx *ctx)
+{
+ emit_instr(ctx, wsbh, dst, src);
+}
+
+/* load a function pointer to register */
+static inline void emit_load_func(unsigned int reg, ptr imm,
+ struct jit_ctx *ctx)
+{
+ if (config_enabled(CONFIG_64BIT)) {
+ /* At this point imm is always 64-bit */
+ emit_load_imm(r_tmp, (u64)imm >> 32, ctx);
+ emit_dsll(r_tmp_imm, r_tmp, 16, ctx); /* left shift by 16 */
+ emit_ori(r_tmp, r_tmp_imm, (imm >> 16) & 0xffff, ctx);
+ emit_dsll(r_tmp_imm, r_tmp, 16, ctx); /* left shift by 16 */
+ emit_ori(reg, r_tmp_imm, imm & 0xffff, ctx);
+ } else {
+ emit_load_imm(reg, imm, ctx);
+ }
+}
+
+/* Move to real MIPS register */
+static inline void emit_reg_move(ptr dst, ptr src, struct jit_ctx *ctx)
+{
+ if (config_enabled(CONFIG_64BIT))
+ emit_daddu(dst, src, r_zero, ctx);
+ else
+ emit_addu(dst, src, r_zero, ctx);
+}
+
+/* Move to JIT (32-bit) register */
+static inline void emit_jit_reg_move(ptr dst, ptr src, struct jit_ctx *ctx)
+{
+ emit_addu(dst, src, r_zero, ctx);
+}
+
+/* Compute the immediate value for PC-relative branches. */
+static inline u32 b_imm(unsigned int tgt, struct jit_ctx *ctx)
+{
+ if (ctx->target == NULL)
+ return 0;
+
+ /*
+ * We want a pc-relative branch. We only do forward branches
+ * so tgt is always after pc. tgt is the instruction offset
+ * we want to jump to.
+
+ * Branch on MIPS:
+ * I: target_offset <- sign_extend(offset)
+ * I+1: PC += target_offset (delay slot)
+ *
+ * ctx->idx currently points to the branch instruction
+ * but the offset is added to the delay slot so we need
+ * to subtract 4.
+ */
+ return ctx->offsets[tgt] -
+ (ctx->idx * 4 - ctx->prologue_bytes) - 4;
+}
+
+static inline void emit_bcond(int cond, unsigned int reg1, unsigned int reg2,
+ unsigned int imm, struct jit_ctx *ctx)
+{
+ if (ctx->target != NULL) {
+ u32 *p = &ctx->target[ctx->idx];
+
+ switch (cond) {
+ case MIPS_COND_EQ:
+ uasm_i_beq(&p, reg1, reg2, imm);
+ break;
+ case MIPS_COND_NE:
+ uasm_i_bne(&p, reg1, reg2, imm);
+ break;
+ case MIPS_COND_ALL:
+ uasm_i_b(&p, imm);
+ break;
+ default:
+ pr_warn("%s: Unhandled branch conditional: %d\n",
+ __func__, cond);
+ }
+ }
+ ctx->idx++;
+}
+
+static inline void emit_b(unsigned int imm, struct jit_ctx *ctx)
+{
+ emit_bcond(MIPS_COND_ALL, r_zero, r_zero, imm, ctx);
+}
+
+static inline void emit_jalr(unsigned int link, unsigned int reg,
+ struct jit_ctx *ctx)
+{
+ emit_instr(ctx, jalr, link, reg);
+}
+
+static inline void emit_jr(unsigned int reg, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, jr, reg);
+}
+
+static inline u16 align_sp(unsigned int num)
+{
+ /* Double word alignment for 32-bit, quadword for 64-bit */
+ unsigned int align = config_enabled(CONFIG_64BIT) ? 16 : 8;
+ num = (num + (align - 1)) & -align;
+ return num;
+}
+
+static inline void update_on_xread(struct jit_ctx *ctx)
+{
+ if (!(ctx->flags & SEEN_X))
+ ctx->flags |= FLAG_NEED_X_RESET;
+
+ ctx->flags |= SEEN_X;
+}
+
+static bool is_load_to_a(u16 inst)
+{
+ switch (inst) {
+ case BPF_S_LD_W_LEN:
+ case BPF_S_LD_W_ABS:
+ case BPF_S_LD_H_ABS:
+ case BPF_S_LD_B_ABS:
+ case BPF_S_ANC_CPU:
+ case BPF_S_ANC_IFINDEX:
+ case BPF_S_ANC_MARK:
+ case BPF_S_ANC_PROTOCOL:
+ case BPF_S_ANC_RXHASH:
+ case BPF_S_ANC_VLAN_TAG:
+ case BPF_S_ANC_VLAN_TAG_PRESENT:
+ case BPF_S_ANC_QUEUE:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static void save_bpf_jit_regs(struct jit_ctx *ctx, unsigned offset)
+{
+ int i = 0, real_off = 0;
+ u32 sflags, tmp_flags;
+
+ /* Adjust the stack pointer */
+ emit_stack_offset(-align_sp(offset), ctx);
+
+ if (ctx->flags & SEEN_CALL) {
+ /* Argument save area */
+ if (config_enabled(CONFIG_64BIT))
+ /* Bottom of current frame */
+ real_off = align_sp(offset) - RSIZE;
+ else
+ /* Top of previous frame */
+ real_off = align_sp(offset) + RSIZE;
+ emit_store_stack_reg(MIPS_R_A0, r_sp, real_off, ctx);
+ emit_store_stack_reg(MIPS_R_A1, r_sp, real_off + RSIZE, ctx);
+
+ real_off = 0;
+ }
+
+ tmp_flags = sflags = ctx->flags >> SEEN_SREG_SFT;
+ /* sflags is essentially a bitmap */
+ while (tmp_flags) {
+ if ((sflags >> i) & 0x1) {
+ emit_store_stack_reg(MIPS_R_S0 + i, r_sp, real_off,
+ ctx);
+ real_off += RSIZE;
+ }
+ i++;
+ tmp_flags >>= 1;
+ }
+
+ /* save return address */
+ if (ctx->flags & SEEN_CALL) {
+ emit_store_stack_reg(r_ra, r_sp, real_off, ctx);
+ real_off += RSIZE;
+ }
+
+ /* Setup r_M leaving the alignment gap if necessary */
+ if (ctx->flags & SEEN_MEM) {
+ if (real_off % (RSIZE * 2))
+ real_off += RSIZE;
+ emit_addiu(r_M, r_sp, real_off, ctx);
+ }
+}
+
+static void restore_bpf_jit_regs(struct jit_ctx *ctx,
+ unsigned int offset)
+{
+ int i, real_off = 0;
+ u32 sflags, tmp_flags;
+
+ if (ctx->flags & SEEN_CALL) {
+ if (config_enabled(CONFIG_64BIT))
+ /* Bottom of current frame */
+ real_off = align_sp(offset) - RSIZE;
+ else
+ /* Top of previous frame */
+ real_off = align_sp(offset) + RSIZE;
+ emit_load_stack_reg(MIPS_R_A0, r_sp, real_off, ctx);
+ emit_load_stack_reg(MIPS_R_A1, r_sp, real_off + RSIZE, ctx);
+
+ real_off = 0;
+ }
+
+ tmp_flags = sflags = ctx->flags >> SEEN_SREG_SFT;
+ /* sflags is a bitmap */
+ i = 0;
+ while (tmp_flags) {
+ if ((sflags >> i) & 0x1) {
+ emit_load_stack_reg(MIPS_R_S0 + i, r_sp, real_off,
+ ctx);
+ real_off += RSIZE;
+ }
+ i++;
+ tmp_flags >>= 1;
+ }
+
+ /* restore return address */
+ if (ctx->flags & SEEN_CALL)
+ emit_load_stack_reg(r_ra, r_sp, real_off, ctx);
+
+ /* Restore the sp and discard the scrach memory */
+ emit_stack_offset(align_sp(offset), ctx);
+}
+
+static unsigned int get_stack_depth(struct jit_ctx *ctx)
+{
+ int sp_off = 0;
+
+
+ /* How may s* regs do we need to preserved? */
+ sp_off += hweight32(ctx->flags >> SEEN_SREG_SFT) * RSIZE;
+
+ if (ctx->flags & SEEN_MEM)
+ sp_off += 4 * BPF_MEMWORDS; /* BPF_MEMWORDS are 32-bit */
+
+ if (ctx->flags & SEEN_CALL)
+ /*
+ * The JIT code make calls to external functions using 2
+ * arguments. Therefore, for o32 we don't need to allocate
+ * space because we don't care if the argumetns are lost
+ * across calls. We do need however to preserve incoming
+ * arguments but the space is already allocated for us by
+ * the caller. On the other hand, for n64, we need to allocate
+ * this space ourselves. We need to preserve $ra as well.
+ */
+ sp_off += config_enabled(CONFIG_64BIT) ?
+ (ARGS_USED_BY_JIT + 1) * RSIZE : RSIZE;
+
+ /*
+ * Subtract the bytes for the last registers since we only care about
+ * the location on the stack pointer.
+ */
+ return sp_off - RSIZE;
+}
+
+static void build_prologue(struct jit_ctx *ctx)
+{
+ u16 first_inst = ctx->skf->insns[0].code;
+ int sp_off;
+
+ /* Calculate the total offset for the stack pointer */
+ sp_off = get_stack_depth(ctx);
+ save_bpf_jit_regs(ctx, sp_off);
+
+ if (ctx->flags & SEEN_SKB)
+ emit_reg_move(r_skb, MIPS_R_A0, ctx);
+
+ if (ctx->flags & FLAG_NEED_X_RESET)
+ emit_jit_reg_move(r_X, r_zero, ctx);
+
+ /* Do not leak kernel data to userspace */
+ if ((first_inst != BPF_S_RET_K) && !(is_load_to_a(first_inst)))
+ emit_jit_reg_move(r_A, r_zero, ctx);
+}
+
+static void build_epilogue(struct jit_ctx *ctx)
+{
+ unsigned int sp_off;
+
+ /* Calculate the total offset for the stack pointer */
+
+ sp_off = get_stack_depth(ctx);
+ restore_bpf_jit_regs(ctx, sp_off);
+
+ /* Return */
+ emit_jr(r_ra, ctx);
+ emit_nop(ctx);
+}
+
+static u64 jit_get_skb_b(struct sk_buff *skb, unsigned offset)
+{
+ u8 ret;
+ int err;
+
+ err = skb_copy_bits(skb, offset, &ret, 1);
+
+ return (u64)err << 32 | ret;
+}
+
+static u64 jit_get_skb_h(struct sk_buff *skb, unsigned offset)
+{
+ u16 ret;
+ int err;
+
+ err = skb_copy_bits(skb, offset, &ret, 2);
+
+ return (u64)err << 32 | ntohs(ret);
+}
+
+static u64 jit_get_skb_w(struct sk_buff *skb, unsigned offset)
+{
+ u32 ret;
+ int err;
+
+ err = skb_copy_bits(skb, offset, &ret, 4);
+
+ return (u64)err << 32 | ntohl(ret);
+}
+
+#define PKT_TYPE_MAX 7
+static int pkt_type_offset(void)
+{
+ struct sk_buff skb_probe = {
+ .pkt_type = ~0,
+ };
+ char *ct = (char *)&skb_probe;
+ unsigned int off;
+
+ for (off = 0; off < sizeof(struct sk_buff); off++) {
+ if (ct[off] == PKT_TYPE_MAX)
+ return off;
+ }
+ pr_err_once("Please fix pkt_type_offset(), as pkt_type couldn't be found\n");
+ return -1;
+}
+
+static int build_body(struct jit_ctx *ctx)
+{
+ void *load_func[] = {jit_get_skb_b, jit_get_skb_h, jit_get_skb_w};
+ const struct sk_filter *prog = ctx->skf;
+ const struct sock_filter *inst;
+ unsigned int i, off, load_order, condt;
+ u32 k, b_off __maybe_unused;
+
+ for (i = 0; i < prog->len; i++) {
+ inst = &(prog->insns[i]);
+ pr_debug("%s: code->0x%02x, jt->0x%x, jf->0x%x, k->0x%x\n",
+ __func__, inst->code, inst->jt, inst->jf, inst->k);
+ k = inst->k;
+
+ if (ctx->target == NULL)
+ ctx->offsets[i] = ctx->idx * 4;
+
+ switch (inst->code) {
+ case BPF_S_LD_IMM:
+ /* A <- k ==> li r_A, k */
+ ctx->flags |= SEEN_A;
+ emit_load_imm(r_A, k, ctx);
+ break;
+ case BPF_S_LD_W_LEN:
+ BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, len) != 4);
+ /* A <- len ==> lw r_A, offset(skb) */
+ ctx->flags |= SEEN_SKB | SEEN_A;
+ off = offsetof(struct sk_buff, len);
+ emit_load(r_A, r_skb, off, ctx);
+ break;
+ case BPF_S_LD_MEM:
+ /* A <- M[k] ==> lw r_A, offset(M) */
+ ctx->flags |= SEEN_MEM | SEEN_A;
+ emit_load(r_A, r_M, SCRATCH_OFF(k), ctx);
+ break;
+ case BPF_S_LD_W_ABS:
+ /* A <- P[k:4] */
+ load_order = 2;
+ goto load;
+ case BPF_S_LD_H_ABS:
+ /* A <- P[k:2] */
+ load_order = 1;
+ goto load;
+ case BPF_S_LD_B_ABS:
+ /* A <- P[k:1] */
+ load_order = 0;
+load:
+ emit_load_imm(r_off, k, ctx);
+load_common:
+ ctx->flags |= SEEN_CALL | SEEN_OFF | SEEN_S0 |
+ SEEN_SKB | SEEN_A;
+
+ emit_load_func(r_s0, (ptr)load_func[load_order],
+ ctx);
+ emit_reg_move(MIPS_R_A0, r_skb, ctx);
+ emit_jalr(MIPS_R_RA, r_s0, ctx);
+ /* Load second argument to delay slot */
+ emit_reg_move(MIPS_R_A1, r_off, ctx);
+ /* Check the error value */
+ if (config_enabled(CONFIG_64BIT)) {
+ /* Get error code from the top 32-bits */
+ emit_dsrl32(r_s0, r_val, 0, ctx);
+ /* Branch to 3 instructions ahead */
+ emit_bcond(MIPS_COND_NE, r_s0, r_zero, 3 << 2,
+ ctx);
+ } else {
+ /* Branch to 3 instructions ahead */
+ emit_bcond(MIPS_COND_NE, r_err, r_zero, 3 << 2,
+ ctx);
+ }
+ emit_nop(ctx);
+ /* We are good */
+ emit_b(b_imm(i + 1, ctx), ctx);
+ emit_jit_reg_move(r_A, r_val, ctx);
+ /* Return with error */
+ emit_b(b_imm(prog->len, ctx), ctx);
+ emit_reg_move(r_ret, r_zero, ctx);
+ break;
+ case BPF_S_LD_W_IND:
+ /* A <- P[X + k:4] */
+ load_order = 2;
+ goto load_ind;
+ case BPF_S_LD_H_IND:
+ /* A <- P[X + k:2] */
+ load_order = 1;
+ goto load_ind;
+ case BPF_S_LD_B_IND:
+ /* A <- P[X + k:1] */
+ load_order = 0;
+load_ind:
+ update_on_xread(ctx);
+ ctx->flags |= SEEN_OFF | SEEN_X;
+ emit_addiu(r_off, r_X, k, ctx);
+ goto load_common;
+ case BPF_S_LDX_IMM:
+ /* X <- k */
+ ctx->flags |= SEEN_X;
+ emit_load_imm(r_X, k, ctx);
+ break;
+ case BPF_S_LDX_MEM:
+ /* X <- M[k] */
+ ctx->flags |= SEEN_X | SEEN_MEM;
+ emit_load(r_X, r_M, SCRATCH_OFF(k), ctx);
+ break;
+ case BPF_S_LDX_W_LEN:
+ /* X <- len */
+ ctx->flags |= SEEN_X | SEEN_SKB;
+ off = offsetof(struct sk_buff, len);
+ emit_load(r_X, r_skb, off, ctx);
+ break;
+ case BPF_S_LDX_B_MSH:
+ /* X <- 4 * (P[k:1] & 0xf) */
+ ctx->flags |= SEEN_X | SEEN_CALL | SEEN_S0 | SEEN_SKB;
+ /* Load offset to a1 */
+ emit_load_func(r_s0, (ptr)jit_get_skb_b, ctx);
+ /*
+ * This may emit two instructions so it may not fit
+ * in the delay slot. So use a0 in the delay slot.
+ */
+ emit_load_imm(MIPS_R_A1, k, ctx);
+ emit_jalr(MIPS_R_RA, r_s0, ctx);
+ emit_reg_move(MIPS_R_A0, r_skb, ctx); /* delay slot */
+ /* Check the error value */
+ if (config_enabled(CONFIG_64BIT)) {
+ /* Top 32-bits of $v0 on 64-bit */
+ emit_dsrl32(r_s0, r_val, 0, ctx);
+ emit_bcond(MIPS_COND_NE, r_s0, r_zero,
+ 3 << 2, ctx);
+ } else {
+ emit_bcond(MIPS_COND_NE, r_err, r_zero,
+ 3 << 2, ctx);
+ }
+ /* No need for delay slot */
+ /* We are good */
+ /* X <- P[1:K] & 0xf */
+ emit_andi(r_X, r_val, 0xf, ctx);
+ /* X << 2 */
+ emit_b(b_imm(i + 1, ctx), ctx);
+ emit_sll(r_X, r_X, 2, ctx); /* delay slot */
+ /* Return with error */
+ emit_b(b_imm(prog->len, ctx), ctx);
+ emit_load_imm(r_ret, 0, ctx); /* delay slot */
+ break;
+ case BPF_S_ST:
+ /* M[k] <- A */
+ ctx->flags |= SEEN_MEM | SEEN_A;
+ emit_store(r_A, r_M, SCRATCH_OFF(k), ctx);
+ break;
+ case BPF_S_STX:
+ /* M[k] <- X */
+ ctx->flags |= SEEN_MEM | SEEN_X;
+ emit_store(r_X, r_M, SCRATCH_OFF(k), ctx);
+ break;
+ case BPF_S_ALU_ADD_K:
+ /* A += K */
+ ctx->flags |= SEEN_A;
+ emit_addiu(r_A, r_A, k, ctx);
+ break;
+ case BPF_S_ALU_ADD_X:
+ /* A += X */
+ ctx->flags |= SEEN_A | SEEN_X;
+ emit_addu(r_A, r_A, r_X, ctx);
+ break;
+ case BPF_S_ALU_SUB_K:
+ /* A -= K */
+ ctx->flags |= SEEN_A;
+ emit_addiu(r_A, r_A, -k, ctx);
+ break;
+ case BPF_S_ALU_SUB_X:
+ /* A -= X */
+ ctx->flags |= SEEN_A | SEEN_X;
+ emit_subu(r_A, r_A, r_X, ctx);
+ break;
+ case BPF_S_ALU_MUL_K:
+ /* A *= K */
+ /* Load K to scratch register before MUL */
+ ctx->flags |= SEEN_A | SEEN_S0;
+ emit_load_imm(r_s0, k, ctx);
+ emit_mul(r_A, r_A, r_s0, ctx);
+ break;
+ case BPF_S_ALU_MUL_X:
+ /* A *= X */
+ update_on_xread(ctx);
+ ctx->flags |= SEEN_A | SEEN_X;
+ emit_mul(r_A, r_A, r_X, ctx);
+ break;
+ case BPF_S_ALU_DIV_K:
+ /* A /= k */
+ if (k == 1)
+ break;
+ if (optimize_div(&k)) {
+ ctx->flags |= SEEN_A;
+ emit_srl(r_A, r_A, k, ctx);
+ break;
+ }
+ ctx->flags |= SEEN_A | SEEN_S0;
+ emit_load_imm(r_s0, k, ctx);
+ emit_div(r_A, r_s0, ctx);
+ break;
+ case BPF_S_ALU_MOD_K:
+ /* A %= k */
+ if (k == 1 || optimize_div(&k)) {
+ ctx->flags |= SEEN_A;
+ emit_jit_reg_move(r_A, r_zero, ctx);
+ } else {
+ ctx->flags |= SEEN_A | SEEN_S0;
+ emit_load_imm(r_s0, k, ctx);
+ emit_mod(r_A, r_s0, ctx);
+ }
+ break;
+ case BPF_S_ALU_DIV_X:
+ /* A /= X */
+ update_on_xread(ctx);
+ ctx->flags |= SEEN_X | SEEN_A;
+ /* Check if r_X is zero */
+ emit_bcond(MIPS_COND_EQ, r_X, r_zero,
+ b_imm(prog->len, ctx), ctx);
+ emit_load_imm(r_val, 0, ctx); /* delay slot */
+ emit_div(r_A, r_X, ctx);
+ break;
+ case BPF_S_ALU_MOD_X:
+ /* A %= X */
+ update_on_xread(ctx);
+ ctx->flags |= SEEN_X | SEEN_A;
+ /* Check if r_X is zero */
+ emit_bcond(MIPS_COND_EQ, r_X, r_zero,
+ b_imm(prog->len, ctx), ctx);
+ emit_load_imm(r_val, 0, ctx); /* delay slot */
+ emit_mod(r_A, r_X, ctx);
+ break;
+ case BPF_S_ALU_OR_K:
+ /* A |= K */
+ ctx->flags |= SEEN_A;
+ emit_ori(r_A, r_A, k, ctx);
+ break;
+ case BPF_S_ALU_OR_X:
+ /* A |= X */
+ update_on_xread(ctx);
+ ctx->flags |= SEEN_A;
+ emit_ori(r_A, r_A, r_X, ctx);
+ break;
+ case BPF_S_ALU_XOR_K:
+ /* A ^= k */
+ ctx->flags |= SEEN_A;
+ emit_xori(r_A, r_A, k, ctx);
+ break;
+ case BPF_S_ANC_ALU_XOR_X:
+ case BPF_S_ALU_XOR_X:
+ /* A ^= X */
+ update_on_xread(ctx);
+ ctx->flags |= SEEN_A;
+ emit_xor(r_A, r_A, r_X, ctx);
+ break;
+ case BPF_S_ALU_AND_K:
+ /* A &= K */
+ ctx->flags |= SEEN_A;
+ emit_andi(r_A, r_A, k, ctx);
+ break;
+ case BPF_S_ALU_AND_X:
+ /* A &= X */
+ update_on_xread(ctx);
+ ctx->flags |= SEEN_A | SEEN_X;
+ emit_and(r_A, r_A, r_X, ctx);
+ break;
+ case BPF_S_ALU_LSH_K:
+ /* A <<= K */
+ ctx->flags |= SEEN_A;
+ emit_sll(r_A, r_A, k, ctx);
+ break;
+ case BPF_S_ALU_LSH_X:
+ /* A <<= X */
+ ctx->flags |= SEEN_A | SEEN_X;
+ update_on_xread(ctx);
+ emit_sllv(r_A, r_A, r_X, ctx);
+ break;
+ case BPF_S_ALU_RSH_K:
+ /* A >>= K */
+ ctx->flags |= SEEN_A;
+ emit_srl(r_A, r_A, k, ctx);
+ break;
+ case BPF_S_ALU_RSH_X:
+ ctx->flags |= SEEN_A | SEEN_X;
+ update_on_xread(ctx);
+ emit_srlv(r_A, r_A, r_X, ctx);
+ break;
+ case BPF_S_ALU_NEG:
+ /* A = -A */
+ ctx->flags |= SEEN_A;
+ emit_neg(r_A, ctx);
+ break;
+ case BPF_S_JMP_JA:
+ /* pc += K */
+ emit_b(b_imm(i + k + 1, ctx), ctx);
+ emit_nop(ctx);
+ break;
+ case BPF_S_JMP_JEQ_K:
+ /* pc += ( A == K ) ? pc->jt : pc->jf */
+ condt = MIPS_COND_EQ | MIPS_COND_K;
+ goto jmp_cmp;
+ case BPF_S_JMP_JEQ_X:
+ ctx->flags |= SEEN_X;
+ /* pc += ( A == X ) ? pc->jt : pc->jf */
+ condt = MIPS_COND_EQ | MIPS_COND_X;
+ goto jmp_cmp;
+ case BPF_S_JMP_JGE_K:
+ /* pc += ( A >= K ) ? pc->jt : pc->jf */
+ condt = MIPS_COND_GE | MIPS_COND_K;
+ goto jmp_cmp;
+ case BPF_S_JMP_JGE_X:
+ ctx->flags |= SEEN_X;
+ /* pc += ( A >= X ) ? pc->jt : pc->jf */
+ condt = MIPS_COND_GE | MIPS_COND_X;
+ goto jmp_cmp;
+ case BPF_S_JMP_JGT_K:
+ /* pc += ( A > K ) ? pc->jt : pc->jf */
+ condt = MIPS_COND_GT | MIPS_COND_K;
+ goto jmp_cmp;
+ case BPF_S_JMP_JGT_X:
+ ctx->flags |= SEEN_X;
+ /* pc += ( A > X ) ? pc->jt : pc->jf */
+ condt = MIPS_COND_GT | MIPS_COND_X;
+jmp_cmp:
+ /* Greater or Equal */
+ if ((condt & MIPS_COND_GE) ||
+ (condt & MIPS_COND_GT)) {
+ if (condt & MIPS_COND_K) { /* K */
+ ctx->flags |= SEEN_S0 | SEEN_A;
+ emit_sltiu(r_s0, r_A, k, ctx);
+ } else { /* X */
+ ctx->flags |= SEEN_S0 | SEEN_A |
+ SEEN_X;
+ emit_sltu(r_s0, r_A, r_X, ctx);
+ }
+ /* A < (K|X) ? r_scrach = 1 */
+ b_off = b_imm(i + inst->jf + 1, ctx);
+ emit_bcond(MIPS_COND_GT, r_s0, r_zero, b_off,
+ ctx);
+ emit_nop(ctx);
+ /* A > (K|X) ? scratch = 0 */
+ if (condt & MIPS_COND_GT) {
+ /* Checking for equality */
+ ctx->flags |= SEEN_S0 | SEEN_A | SEEN_X;
+ if (condt & MIPS_COND_K)
+ emit_load_imm(r_s0, k, ctx);
+ else
+ emit_jit_reg_move(r_s0, r_X,
+ ctx);
+ b_off = b_imm(i + inst->jf + 1, ctx);
+ emit_bcond(MIPS_COND_EQ, r_A, r_s0,
+ b_off, ctx);
+ emit_nop(ctx);
+ /* Finally, A > K|X */
+ b_off = b_imm(i + inst->jt + 1, ctx);
+ emit_b(b_off, ctx);
+ emit_nop(ctx);
+ } else {
+ /* A >= (K|X) so jump */
+ b_off = b_imm(i + inst->jt + 1, ctx);
+ emit_b(b_off, ctx);
+ emit_nop(ctx);
+ }
+ } else {
+ /* A == K|X */
+ if (condt & MIPS_COND_K) { /* K */
+ ctx->flags |= SEEN_S0 | SEEN_A;
+ emit_load_imm(r_s0, k, ctx);
+ /* jump true */
+ b_off = b_imm(i + inst->jt + 1, ctx);
+ emit_bcond(MIPS_COND_EQ, r_A, r_s0,
+ b_off, ctx);
+ emit_nop(ctx);
+ /* jump false */
+ b_off = b_imm(i + inst->jf + 1,
+ ctx);
+ emit_bcond(MIPS_COND_NE, r_A, r_s0,
+ b_off, ctx);
+ emit_nop(ctx);
+ } else { /* X */
+ /* jump true */
+ ctx->flags |= SEEN_A | SEEN_X;
+ b_off = b_imm(i + inst->jt + 1,
+ ctx);
+ emit_bcond(MIPS_COND_EQ, r_A, r_X,
+ b_off, ctx);
+ emit_nop(ctx);
+ /* jump false */
+ b_off = b_imm(i + inst->jf + 1, ctx);
+ emit_bcond(MIPS_COND_NE, r_A, r_X,
+ b_off, ctx);
+ emit_nop(ctx);
+ }
+ }
+ break;
+ case BPF_S_JMP_JSET_K:
+ ctx->flags |= SEEN_S0 | SEEN_S1 | SEEN_A;
+ /* pc += (A & K) ? pc -> jt : pc -> jf */
+ emit_load_imm(r_s1, k, ctx);
+ emit_and(r_s0, r_A, r_s1, ctx);
+ /* jump true */
+ b_off = b_imm(i + inst->jt + 1, ctx);
+ emit_bcond(MIPS_COND_NE, r_s0, r_zero, b_off, ctx);
+ emit_nop(ctx);
+ /* jump false */
+ b_off = b_imm(i + inst->jf + 1, ctx);
+ emit_b(b_off, ctx);
+ emit_nop(ctx);
+ break;
+ case BPF_S_JMP_JSET_X:
+ ctx->flags |= SEEN_S0 | SEEN_X | SEEN_A;
+ /* pc += (A & X) ? pc -> jt : pc -> jf */
+ emit_and(r_s0, r_A, r_X, ctx);
+ /* jump true */
+ b_off = b_imm(i + inst->jt + 1, ctx);
+ emit_bcond(MIPS_COND_NE, r_s0, r_zero, b_off, ctx);
+ emit_nop(ctx);
+ /* jump false */
+ b_off = b_imm(i + inst->jf + 1, ctx);
+ emit_b(b_off, ctx);
+ emit_nop(ctx);
+ break;
+ case BPF_S_RET_A:
+ ctx->flags |= SEEN_A;
+ if (i != prog->len - 1)
+ /*
+ * If this is not the last instruction
+ * then jump to the epilogue
+ */
+ emit_b(b_imm(prog->len, ctx), ctx);
+ emit_reg_move(r_ret, r_A, ctx); /* delay slot */
+ break;
+ case BPF_S_RET_K:
+ /*
+ * It can emit two instructions so it does not fit on
+ * the delay slot.
+ */
+ emit_load_imm(r_ret, k, ctx);
+ if (i != prog->len - 1) {
+ /*
+ * If this is not the last instruction
+ * then jump to the epilogue
+ */
+ emit_b(b_imm(prog->len, ctx), ctx);
+ emit_nop(ctx);
+ }
+ break;
+ case BPF_S_MISC_TAX:
+ /* X = A */
+ ctx->flags |= SEEN_X | SEEN_A;
+ emit_jit_reg_move(r_X, r_A, ctx);
+ break;
+ case BPF_S_MISC_TXA:
+ /* A = X */
+ ctx->flags |= SEEN_A | SEEN_X;
+ update_on_xread(ctx);
+ emit_jit_reg_move(r_A, r_X, ctx);
+ break;
+ /* AUX */
+ case BPF_S_ANC_PROTOCOL:
+ /* A = ntohs(skb->protocol */
+ ctx->flags |= SEEN_SKB | SEEN_OFF | SEEN_A;
+ BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff,
+ protocol) != 2);
+ off = offsetof(struct sk_buff, protocol);
+ emit_half_load(r_A, r_skb, off, ctx);
+#ifdef CONFIG_CPU_LITTLE_ENDIAN
+ /* This needs little endian fixup */
+ if (cpu_has_mips_r2) {
+ /* R2 and later have the wsbh instruction */
+ emit_wsbh(r_A, r_A, ctx);
+ } else {
+ /* Get first byte */
+ emit_andi(r_tmp_imm, r_A, 0xff, ctx);
+ /* Shift it */
+ emit_sll(r_tmp, r_tmp_imm, 8, ctx);
+ /* Get second byte */
+ emit_srl(r_tmp_imm, r_A, 8, ctx);
+ emit_andi(r_tmp_imm, r_tmp_imm, 0xff, ctx);
+ /* Put everyting together in r_A */
+ emit_or(r_A, r_tmp, r_tmp_imm, ctx);
+ }
+#endif
+ break;
+ case BPF_S_ANC_CPU:
+ ctx->flags |= SEEN_A | SEEN_OFF;
+ /* A = current_thread_info()->cpu */
+ BUILD_BUG_ON(FIELD_SIZEOF(struct thread_info,
+ cpu) != 4);
+ off = offsetof(struct thread_info, cpu);
+ /* $28/gp points to the thread_info struct */
+ emit_load(r_A, 28, off, ctx);
+ break;
+ case BPF_S_ANC_IFINDEX:
+ /* A = skb->dev->ifindex */
+ ctx->flags |= SEEN_SKB | SEEN_A | SEEN_S0;
+ off = offsetof(struct sk_buff, dev);
+ emit_load(r_s0, r_skb, off, ctx);
+ /* error (0) in the delay slot */
+ emit_bcond(MIPS_COND_EQ, r_s0, r_zero,
+ b_imm(prog->len, ctx), ctx);
+ emit_reg_move(r_ret, r_zero, ctx);
+ BUILD_BUG_ON(FIELD_SIZEOF(struct net_device,
+ ifindex) != 4);
+ off = offsetof(struct net_device, ifindex);
+ emit_load(r_A, r_s0, off, ctx);
+ break;
+ case BPF_S_ANC_MARK:
+ ctx->flags |= SEEN_SKB | SEEN_A;
+ BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, mark) != 4);
+ off = offsetof(struct sk_buff, mark);
+ emit_load(r_A, r_skb, off, ctx);
+ break;
+ case BPF_S_ANC_RXHASH:
+ ctx->flags |= SEEN_SKB | SEEN_A;
+ BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, hash) != 4);
+ off = offsetof(struct sk_buff, hash);
+ emit_load(r_A, r_skb, off, ctx);
+ break;
+ case BPF_S_ANC_VLAN_TAG:
+ case BPF_S_ANC_VLAN_TAG_PRESENT:
+ ctx->flags |= SEEN_SKB | SEEN_S0 | SEEN_A;
+ BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff,
+ vlan_tci) != 2);
+ off = offsetof(struct sk_buff, vlan_tci);
+ emit_half_load(r_s0, r_skb, off, ctx);
+ if (inst->code == BPF_S_ANC_VLAN_TAG)
+ emit_and(r_A, r_s0, VLAN_VID_MASK, ctx);
+ else
+ emit_and(r_A, r_s0, VLAN_TAG_PRESENT, ctx);
+ break;
+ case BPF_S_ANC_PKTTYPE:
+ off = pkt_type_offset();
+
+ if (off < 0)
+ return -1;
+ emit_load_byte(r_tmp, r_skb, off, ctx);
+ /* Keep only the last 3 bits */
+ emit_andi(r_A, r_tmp, PKT_TYPE_MAX, ctx);
+ break;
+ case BPF_S_ANC_QUEUE:
+ ctx->flags |= SEEN_SKB | SEEN_A;
+ BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff,
+ queue_mapping) != 2);
+ BUILD_BUG_ON(offsetof(struct sk_buff,
+ queue_mapping) > 0xff);
+ off = offsetof(struct sk_buff, queue_mapping);
+ emit_half_load(r_A, r_skb, off, ctx);
+ break;
+ default:
+ pr_warn("%s: Unhandled opcode: 0x%02x\n", __FILE__,
+ inst->code);
+ return -1;
+ }
+ }
+
+ /* compute offsets only during the first pass */
+ if (ctx->target == NULL)
+ ctx->offsets[i] = ctx->idx * 4;
+
+ return 0;
+}
+
+int bpf_jit_enable __read_mostly;
+
+void bpf_jit_compile(struct sk_filter *fp)
+{
+ struct jit_ctx ctx;
+ unsigned int alloc_size, tmp_idx;
+
+ if (!bpf_jit_enable)
+ return;
+
+ memset(&ctx, 0, sizeof(ctx));
+
+ ctx.offsets = kcalloc(fp->len, sizeof(*ctx.offsets), GFP_KERNEL);
+ if (ctx.offsets == NULL)
+ return;
+
+ ctx.skf = fp;
+
+ if (build_body(&ctx))
+ goto out;
+
+ tmp_idx = ctx.idx;
+ build_prologue(&ctx);
+ ctx.prologue_bytes = (ctx.idx - tmp_idx) * 4;
+ /* just to complete the ctx.idx count */
+ build_epilogue(&ctx);
+
+ alloc_size = 4 * ctx.idx;
+ ctx.target = module_alloc(alloc_size);
+ if (ctx.target == NULL)
+ goto out;
+
+ /* Clean it */
+ memset(ctx.target, 0, alloc_size);
+
+ ctx.idx = 0;
+
+ /* Generate the actual JIT code */
+ build_prologue(&ctx);
+ build_body(&ctx);
+ build_epilogue(&ctx);
+
+ /* Update the icache */
+ flush_icache_range((ptr)ctx.target, (ptr)(ctx.target + ctx.idx));
+
+ if (bpf_jit_enable > 1)
+ /* Dump JIT code */
+ bpf_jit_dump(fp->len, alloc_size, 2, ctx.target);
+
+ fp->bpf_func = (void *)ctx.target;
+ fp->jited = 1;
+
+out:
+ kfree(ctx.offsets);
+}
+
+void bpf_jit_free(struct sk_filter *fp)
+{
+ if (fp->jited)
+ module_free(NULL, fp->bpf_func);
+ kfree(fp);
+}
diff --git a/arch/mips/net/bpf_jit.h b/arch/mips/net/bpf_jit.h
new file mode 100644
index 0000000..3a5751b
--- /dev/null
+++ b/arch/mips/net/bpf_jit.h
@@ -0,0 +1,44 @@
+/*
+ * Just-In-Time compiler for BPF filters on MIPS
+ *
+ * Copyright (c) 2014 Imagination Technologies Ltd.
+ * Author: Markos Chandras <markos.chandras@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; version 2 of the License.
+ */
+
+#ifndef BPF_JIT_MIPS_OP_H
+#define BPF_JIT_MIPS_OP_H
+
+/* Registers used by JIT */
+#define MIPS_R_ZERO 0
+#define MIPS_R_V0 2
+#define MIPS_R_V1 3
+#define MIPS_R_A0 4
+#define MIPS_R_A1 5
+#define MIPS_R_T6 14
+#define MIPS_R_T7 15
+#define MIPS_R_S0 16
+#define MIPS_R_S1 17
+#define MIPS_R_S2 18
+#define MIPS_R_S3 19
+#define MIPS_R_S4 20
+#define MIPS_R_S5 21
+#define MIPS_R_S6 22
+#define MIPS_R_S7 23
+#define MIPS_R_SP 29
+#define MIPS_R_RA 31
+
+/* Conditional codes */
+#define MIPS_COND_EQ 0x1
+#define MIPS_COND_GE (0x1 << 1)
+#define MIPS_COND_GT (0x1 << 2)
+#define MIPS_COND_NE (0x1 << 3)
+#define MIPS_COND_ALL (0x1 << 4)
+/* Conditionals on X register or K immediate */
+#define MIPS_COND_X (0x1 << 5)
+#define MIPS_COND_K (0x1 << 6)
+
+#endif /* BPF_JIT_MIPS_OP_H */
diff --git a/arch/mips/netlogic/common/irq.c b/arch/mips/netlogic/common/irq.c
index 5afc4b7..c100b9a 100644
--- a/arch/mips/netlogic/common/irq.c
+++ b/arch/mips/netlogic/common/irq.c
@@ -203,6 +203,8 @@
xirq = nlm_irq_to_xirq(node, irq);
pic_data = irq_get_handler_data(xirq);
+ if (WARN_ON(!pic_data))
+ return;
pic_data->extra_ack = xack;
}
diff --git a/arch/mips/netlogic/common/reset.S b/arch/mips/netlogic/common/reset.S
index b231fe1..701c4bc 100644
--- a/arch/mips/netlogic/common/reset.S
+++ b/arch/mips/netlogic/common/reset.S
@@ -35,6 +35,7 @@
#include <asm/asm.h>
#include <asm/asm-offsets.h>
+#include <asm/cpu.h>
#include <asm/cacheops.h>
#include <asm/regdef.h>
#include <asm/mipsregs.h>
@@ -74,13 +75,25 @@
.endm
/*
+ * Allow access to physical mem >64G by enabling ELPA in PAGEGRAIN
+ * register. This is needed before going to C code since the SP can
+ * in this region. Called from all HW threads.
+ */
+.macro xlp_early_mmu_init
+ mfc0 t0, CP0_PAGEMASK, 1
+ li t1, (1 << 29) /* ELPA bit */
+ or t0, t1
+ mtc0 t0, CP0_PAGEMASK, 1
+.endm
+
+/*
* L1D cache has to be flushed before enabling threads in XLP.
* On XLP8xx/XLP3xx, we do a low level flush using processor control
* registers. On XLPII CPUs, usual cache instructions work.
*/
.macro xlp_flush_l1_dcache
mfc0 t0, CP0_EBASE, 0
- andi t0, t0, 0xff00
+ andi t0, t0, PRID_IMP_MASK
slt t1, t0, 0x1200
beqz t1, 15f
nop
@@ -159,11 +172,15 @@
1: /* Entry point on core wakeup */
mfc0 t0, CP0_EBASE, 0 /* processor ID */
- andi t0, 0xff00
+ andi t0, PRID_IMP_MASK
li t1, 0x1500 /* XLP 9xx */
beq t0, t1, 2f /* does not need to set coherent */
nop
+ li t1, 0x1300 /* XLP 5xx */
+ beq t0, t1, 2f /* does not need to set coherent */
+ nop
+
/* set bit in SYS coherent register for the core */
mfc0 t0, CP0_EBASE, 1
mfc0 t1, CP0_EBASE, 1
@@ -197,6 +214,9 @@
EXPORT(nlm_boot_siblings)
/* core L1D flush before enable threads */
xlp_flush_l1_dcache
+ /* save ra and sp, will be used later (only for boot cpu) */
+ dmtc0 ra, $22, 6
+ dmtc0 sp, $22, 7
/* Enable hw threads by writing to MAP_THREADMODE of the core */
li t0, CKSEG1ADDR(RESET_DATA_PHYS)
lw t1, BOOT_THREAD_MODE(t0) /* t1 <- thread mode */
@@ -225,6 +245,8 @@
#endif
mtc0 t1, CP0_STATUS
+ xlp_early_mmu_init
+
/* mark CPU ready */
li t3, CKSEG1ADDR(RESET_DATA_PHYS)
ADDIU t1, t3, BOOT_CPU_READY
@@ -238,14 +260,12 @@
nop
/*
- * For the boot CPU, we have to restore registers and
- * return
+ * For the boot CPU, we have to restore ra and sp and return, rest
+ * of the registers will be restored by the caller
*/
-4: dmfc0 t0, $4, 2 /* restore SP from UserLocal */
- li t1, 0xfadebeef
- dmtc0 t1, $4, 2 /* restore SP from UserLocal */
- PTR_SUBU sp, t0, PT_SIZE
- RESTORE_ALL
+4:
+ dmfc0 ra, $22, 6
+ dmfc0 sp, $22, 7
jr ra
nop
EXPORT(nlm_reset_entry_end)
@@ -253,6 +273,7 @@
LEAF(nlm_init_boot_cpu)
#ifdef CONFIG_CPU_XLP
xlp_config_lsu
+ xlp_early_mmu_init
#endif
jr ra
nop
diff --git a/arch/mips/netlogic/common/smp.c b/arch/mips/netlogic/common/smp.c
index 6baae15..4fde7ac 100644
--- a/arch/mips/netlogic/common/smp.c
+++ b/arch/mips/netlogic/common/smp.c
@@ -135,10 +135,6 @@
local_irq_enable();
}
-void nlm_cpus_done(void)
-{
-}
-
/*
* Boot all other cpus in the system, initialize them, and bring them into
* the boot function
@@ -198,7 +194,7 @@
cpumask_scnprintf(buf, ARRAY_SIZE(buf), cpu_possible_mask);
pr_info("Possible CPU mask: %s\n", buf);
- /* check with the cores we have worken up */
+ /* check with the cores we have woken up */
for (ncore = 0, i = 0; i < NLM_NR_NODES; i++)
ncore += hweight32(nlm_get_node(i)->coremask);
@@ -213,6 +209,7 @@
{
uint32_t core0_thr_mask, core_thr_mask;
int threadmode, i, j;
+ char buf[64];
core0_thr_mask = 0;
for (i = 0; i < NLM_THREADS_PER_CORE; i++)
@@ -247,8 +244,8 @@
return threadmode;
unsupp:
- panic("Unsupported CPU mask %lx",
- (unsigned long)cpumask_bits(wakeup_mask)[0]);
+ cpumask_scnprintf(buf, ARRAY_SIZE(buf), wakeup_mask);
+ panic("Unsupported CPU mask %s", buf);
return 0;
}
@@ -277,7 +274,6 @@
.send_ipi_mask = nlm_send_ipi_mask,
.init_secondary = nlm_init_secondary,
.smp_finish = nlm_smp_finish,
- .cpus_done = nlm_cpus_done,
.boot_secondary = nlm_boot_secondary,
.smp_setup = nlm_smp_setup,
.prepare_cpus = nlm_prepare_cpus,
diff --git a/arch/mips/netlogic/common/smpboot.S b/arch/mips/netlogic/common/smpboot.S
index 8597657..805355b 100644
--- a/arch/mips/netlogic/common/smpboot.S
+++ b/arch/mips/netlogic/common/smpboot.S
@@ -54,8 +54,9 @@
.set noat
.set arch=xlr /* for mfcr/mtcr, XLR is sufficient */
-FEXPORT(xlp_boot_core0_siblings) /* "Master" cpu starts from here */
- dmtc0 sp, $4, 2 /* SP saved in UserLocal */
+/* Called by the boot cpu to wake up its sibling threads */
+NESTED(xlp_boot_core0_siblings, PT_SIZE, sp)
+ /* CPU register contents lost when enabling threads, save them first */
SAVE_ALL
sync
/* find the location to which nlm_boot_siblings was relocated */
@@ -65,9 +66,12 @@
dsubu t2, t1
daddu t2, t0
/* call it */
- jr t2
+ jalr t2
nop
- /* not reached */
+ RESTORE_ALL
+ jr ra
+ nop
+END(xlp_boot_core0_siblings)
NESTED(nlm_boot_secondary_cpus, 16, sp)
/* Initialize CP0 Status */
diff --git a/arch/mips/netlogic/common/time.c b/arch/mips/netlogic/common/time.c
index 13391b8..0c0a1a6 100644
--- a/arch/mips/netlogic/common/time.c
+++ b/arch/mips/netlogic/common/time.c
@@ -82,6 +82,7 @@
static void nlm_init_pic_timer(void)
{
uint64_t picbase = nlm_get_node(0)->picbase;
+ u32 picfreq;
nlm_pic_set_timer(picbase, PIC_CLOCK_TIMER, ~0ULL, 0, 0);
if (current_cpu_data.cputype == CPU_XLR) {
@@ -92,7 +93,9 @@
csrc_pic.read = nlm_get_pic_timer;
}
csrc_pic.rating = 1000;
- clocksource_register_hz(&csrc_pic, pic_timer_freq());
+ picfreq = pic_timer_freq();
+ clocksource_register_hz(&csrc_pic, picfreq);
+ pr_info("PIC clock source added, frequency %d\n", picfreq);
}
void __init plat_time_init(void)
diff --git a/arch/mips/netlogic/dts/xlp_gvp.dts b/arch/mips/netlogic/dts/xlp_gvp.dts
index 047d27f..bb4ecd1 100644
--- a/arch/mips/netlogic/dts/xlp_gvp.dts
+++ b/arch/mips/netlogic/dts/xlp_gvp.dts
@@ -26,11 +26,12 @@
interrupt-parent = <&pic>;
interrupts = <17>;
};
- pic: pic@4000 {
- interrupt-controller;
+ pic: pic@110000 {
+ compatible = "netlogic,xlp-pic";
#address-cells = <0>;
#interrupt-cells = <1>;
reg = <0 0x110000 0x200>;
+ interrupt-controller;
};
nor_flash@1,0 {
diff --git a/arch/mips/netlogic/xlp/Makefile b/arch/mips/netlogic/xlp/Makefile
index ed9a93c..be358a8 100644
--- a/arch/mips/netlogic/xlp/Makefile
+++ b/arch/mips/netlogic/xlp/Makefile
@@ -2,3 +2,5 @@
obj-$(CONFIG_SMP) += wakeup.o
obj-$(CONFIG_USB) += usb-init.o
obj-$(CONFIG_USB) += usb-init-xlp2.o
+obj-$(CONFIG_SATA_AHCI) += ahci-init.o
+obj-$(CONFIG_SATA_AHCI) += ahci-init-xlp2.o
diff --git a/arch/mips/netlogic/xlp/ahci-init-xlp2.c b/arch/mips/netlogic/xlp/ahci-init-xlp2.c
new file mode 100644
index 0000000..c83dbf3
--- /dev/null
+++ b/arch/mips/netlogic/xlp/ahci-init-xlp2.c
@@ -0,0 +1,377 @@
+/*
+ * Copyright (c) 2003-2014 Broadcom Corporation
+ * All Rights Reserved
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the Broadcom
+ * license below:
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY BROADCOM ``AS IS'' AND ANY EXPRESS OR
+ * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL BROADCOM OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+ * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
+ * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
+ * IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/kernel.h>
+#include <linux/delay.h>
+#include <linux/init.h>
+#include <linux/pci.h>
+#include <linux/irq.h>
+#include <linux/bitops.h>
+#include <linux/pci_ids.h>
+#include <linux/nodemask.h>
+
+#include <asm/cpu.h>
+#include <asm/mipsregs.h>
+
+#include <asm/netlogic/common.h>
+#include <asm/netlogic/haldefs.h>
+#include <asm/netlogic/mips-extns.h>
+#include <asm/netlogic/xlp-hal/xlp.h>
+#include <asm/netlogic/xlp-hal/iomap.h>
+
+#define SATA_CTL 0x0
+#define SATA_STATUS 0x1 /* Status Reg */
+#define SATA_INT 0x2 /* Interrupt Reg */
+#define SATA_INT_MASK 0x3 /* Interrupt Mask Reg */
+#define SATA_BIU_TIMEOUT 0x4
+#define AXIWRSPERRLOG 0x5
+#define AXIRDSPERRLOG 0x6
+#define BiuTimeoutLow 0x7
+#define BiuTimeoutHi 0x8
+#define BiuSlvErLow 0x9
+#define BiuSlvErHi 0xa
+#define IO_CONFIG_SWAP_DIS 0xb
+#define CR_REG_TIMER 0xc
+#define CORE_ID 0xd
+#define AXI_SLAVE_OPT1 0xe
+#define PHY_MEM_ACCESS 0xf
+#define PHY0_CNTRL 0x10
+#define PHY0_STAT 0x11
+#define PHY0_RX_ALIGN 0x12
+#define PHY0_RX_EQ_LO 0x13
+#define PHY0_RX_EQ_HI 0x14
+#define PHY0_BIST_LOOP 0x15
+#define PHY1_CNTRL 0x16
+#define PHY1_STAT 0x17
+#define PHY1_RX_ALIGN 0x18
+#define PHY1_RX_EQ_LO 0x19
+#define PHY1_RX_EQ_HI 0x1a
+#define PHY1_BIST_LOOP 0x1b
+#define RdExBase 0x1c
+#define RdExLimit 0x1d
+#define CacheAllocBase 0x1e
+#define CacheAllocLimit 0x1f
+#define BiuSlaveCmdGstNum 0x20
+
+/*SATA_CTL Bits */
+#define SATA_RST_N BIT(0) /* Active low reset sata_core phy */
+#define SataCtlReserve0 BIT(1)
+#define M_CSYSREQ BIT(2) /* AXI master low power, not used */
+#define S_CSYSREQ BIT(3) /* AXI slave low power, not used */
+#define P0_CP_DET BIT(8) /* Reserved, bring in from pad */
+#define P0_MP_SW BIT(9) /* Mech Switch */
+#define P0_DISABLE BIT(10) /* disable p0 */
+#define P0_ACT_LED_EN BIT(11) /* Active LED enable */
+#define P0_IRST_HARD_SYNTH BIT(12) /* PHY hard synth reset */
+#define P0_IRST_HARD_TXRX BIT(13) /* PHY lane hard reset */
+#define P0_IRST_POR BIT(14) /* PHY power on reset*/
+#define P0_IPDTXL BIT(15) /* PHY Tx lane dis/power down */
+#define P0_IPDRXL BIT(16) /* PHY Rx lane dis/power down */
+#define P0_IPDIPDMSYNTH BIT(17) /* PHY synthesizer dis/porwer down */
+#define P0_CP_POD_EN BIT(18) /* CP_POD enable */
+#define P0_AT_BYPASS BIT(19) /* P0 address translation by pass */
+#define P1_CP_DET BIT(20) /* Reserved,Cold Detect */
+#define P1_MP_SW BIT(21) /* Mech Switch */
+#define P1_DISABLE BIT(22) /* disable p1 */
+#define P1_ACT_LED_EN BIT(23) /* Active LED enable */
+#define P1_IRST_HARD_SYNTH BIT(24) /* PHY hard synth reset */
+#define P1_IRST_HARD_TXRX BIT(25) /* PHY lane hard reset */
+#define P1_IRST_POR BIT(26) /* PHY power on reset*/
+#define P1_IPDTXL BIT(27) /* PHY Tx lane dis/porwer down */
+#define P1_IPDRXL BIT(28) /* PHY Rx lane dis/porwer down */
+#define P1_IPDIPDMSYNTH BIT(29) /* PHY synthesizer dis/porwer down */
+#define P1_CP_POD_EN BIT(30)
+#define P1_AT_BYPASS BIT(31) /* P1 address translation by pass */
+
+/* Status register */
+#define M_CACTIVE BIT(0) /* m_cactive, not used */
+#define S_CACTIVE BIT(1) /* s_cactive, not used */
+#define P0_PHY_READY BIT(8) /* phy is ready */
+#define P0_CP_POD BIT(9) /* Cold PowerOn */
+#define P0_SLUMBER BIT(10) /* power mode slumber */
+#define P0_PATIAL BIT(11) /* power mode patial */
+#define P0_PHY_SIG_DET BIT(12) /* phy dignal detect */
+#define P0_PHY_CALI BIT(13) /* phy calibration done */
+#define P1_PHY_READY BIT(16) /* phy is ready */
+#define P1_CP_POD BIT(17) /* Cold PowerOn */
+#define P1_SLUMBER BIT(18) /* power mode slumber */
+#define P1_PATIAL BIT(19) /* power mode patial */
+#define P1_PHY_SIG_DET BIT(20) /* phy dignal detect */
+#define P1_PHY_CALI BIT(21) /* phy calibration done */
+
+/* SATA CR_REG_TIMER bits */
+#define CR_TIME_SCALE (0x1000 << 0)
+
+/* SATA PHY specific registers start and end address */
+#define RXCDRCALFOSC0 0x0065
+#define CALDUTY 0x006e
+#define RXDPIF 0x8065
+#define PPMDRIFTMAX_HI 0x80A4
+
+#define nlm_read_sata_reg(b, r) nlm_read_reg(b, r)
+#define nlm_write_sata_reg(b, r, v) nlm_write_reg(b, r, v)
+#define nlm_get_sata_pcibase(node) \
+ nlm_pcicfg_base(XLP9XX_IO_SATA_OFFSET(node))
+#define nlm_get_sata_regbase(node) \
+ (nlm_get_sata_pcibase(node) + 0x100)
+
+/* SATA PHY config for register block 1 0x0065 .. 0x006e */
+static const u8 sata_phy_config1[] = {
+ 0xC9, 0xC9, 0x07, 0x07, 0x18, 0x18, 0x01, 0x01, 0x22, 0x00
+};
+
+/* SATA PHY config for register block 2 0x0x8065 .. 0x0x80A4 */
+static const u8 sata_phy_config2[] = {
+ 0xAA, 0x00, 0x4C, 0xC9, 0xC9, 0x07, 0x07, 0x18,
+ 0x18, 0x05, 0x0C, 0x10, 0x00, 0x10, 0x00, 0xFF,
+ 0xCF, 0xF7, 0xE1, 0xF5, 0xFD, 0xFD, 0xFF, 0xFF,
+ 0xFF, 0xFF, 0xE3, 0xE7, 0xDB, 0xF5, 0xFD, 0xFD,
+ 0xF5, 0xF5, 0xFF, 0xFF, 0xE3, 0xE7, 0xDB, 0xF5,
+ 0xFD, 0xFD, 0xF5, 0xF5, 0xFF, 0xFF, 0xFF, 0xF5,
+ 0x3F, 0x00, 0x32, 0x00, 0x03, 0x01, 0x05, 0x05,
+ 0x04, 0x00, 0x00, 0x08, 0x04, 0x00, 0x00, 0x04,
+};
+
+const int sata_phy_debug = 0; /* set to verify PHY writes */
+
+static void sata_clear_glue_reg(u64 regbase, u32 off, u32 bit)
+{
+ u32 reg_val;
+
+ reg_val = nlm_read_sata_reg(regbase, off);
+ nlm_write_sata_reg(regbase, off, (reg_val & ~bit));
+}
+
+static void sata_set_glue_reg(u64 regbase, u32 off, u32 bit)
+{
+ u32 reg_val;
+
+ reg_val = nlm_read_sata_reg(regbase, off);
+ nlm_write_sata_reg(regbase, off, (reg_val | bit));
+}
+
+static void write_phy_reg(u64 regbase, u32 addr, u32 physel, u8 data)
+{
+ nlm_write_sata_reg(regbase, PHY_MEM_ACCESS,
+ (1u << 31) | (physel << 24) | (data << 16) | addr);
+ udelay(850);
+}
+
+static u8 read_phy_reg(u64 regbase, u32 addr, u32 physel)
+{
+ u32 val;
+
+ nlm_write_sata_reg(regbase, PHY_MEM_ACCESS,
+ (0 << 31) | (physel << 24) | (0 << 16) | addr);
+ udelay(850);
+ val = nlm_read_sata_reg(regbase, PHY_MEM_ACCESS);
+ return (val >> 16) & 0xff;
+}
+
+static void config_sata_phy(u64 regbase)
+{
+ u32 port, i, reg;
+
+ for (port = 0; port < 2; port++) {
+ for (i = 0, reg = RXCDRCALFOSC0; reg <= CALDUTY; reg++, i++)
+ write_phy_reg(regbase, reg, port, sata_phy_config1[i]);
+
+ for (i = 0, reg = RXDPIF; reg <= PPMDRIFTMAX_HI; reg++, i++)
+ write_phy_reg(regbase, reg, port, sata_phy_config2[i]);
+ }
+}
+
+static void check_phy_register(u64 regbase, u32 addr, u32 physel, u8 xdata)
+{
+ u8 data;
+
+ data = read_phy_reg(regbase, addr, physel);
+ pr_info("PHY read addr = 0x%x physel = %d data = 0x%x %s\n",
+ addr, physel, data, data == xdata ? "TRUE" : "FALSE");
+}
+
+static void verify_sata_phy_config(u64 regbase)
+{
+ u32 port, i, reg;
+
+ for (port = 0; port < 2; port++) {
+ for (i = 0, reg = RXCDRCALFOSC0; reg <= CALDUTY; reg++, i++)
+ check_phy_register(regbase, reg, port,
+ sata_phy_config1[i]);
+
+ for (i = 0, reg = RXDPIF; reg <= PPMDRIFTMAX_HI; reg++, i++)
+ check_phy_register(regbase, reg, port,
+ sata_phy_config2[i]);
+ }
+}
+
+static void nlm_sata_firmware_init(int node)
+{
+ u32 reg_val;
+ u64 regbase;
+ int n;
+
+ pr_info("Initializing XLP9XX On-chip AHCI...\n");
+ regbase = nlm_get_sata_regbase(node);
+
+ /* Reset port0 */
+ sata_clear_glue_reg(regbase, SATA_CTL, P0_IRST_POR);
+ sata_clear_glue_reg(regbase, SATA_CTL, P0_IRST_HARD_TXRX);
+ sata_clear_glue_reg(regbase, SATA_CTL, P0_IRST_HARD_SYNTH);
+ sata_clear_glue_reg(regbase, SATA_CTL, P0_IPDTXL);
+ sata_clear_glue_reg(regbase, SATA_CTL, P0_IPDRXL);
+ sata_clear_glue_reg(regbase, SATA_CTL, P0_IPDIPDMSYNTH);
+
+ /* port1 */
+ sata_clear_glue_reg(regbase, SATA_CTL, P1_IRST_POR);
+ sata_clear_glue_reg(regbase, SATA_CTL, P1_IRST_HARD_TXRX);
+ sata_clear_glue_reg(regbase, SATA_CTL, P1_IRST_HARD_SYNTH);
+ sata_clear_glue_reg(regbase, SATA_CTL, P1_IPDTXL);
+ sata_clear_glue_reg(regbase, SATA_CTL, P1_IPDRXL);
+ sata_clear_glue_reg(regbase, SATA_CTL, P1_IPDIPDMSYNTH);
+ udelay(300);
+
+ /* Set PHY */
+ sata_set_glue_reg(regbase, SATA_CTL, P0_IPDTXL);
+ sata_set_glue_reg(regbase, SATA_CTL, P0_IPDRXL);
+ sata_set_glue_reg(regbase, SATA_CTL, P0_IPDIPDMSYNTH);
+ sata_set_glue_reg(regbase, SATA_CTL, P1_IPDTXL);
+ sata_set_glue_reg(regbase, SATA_CTL, P1_IPDRXL);
+ sata_set_glue_reg(regbase, SATA_CTL, P1_IPDIPDMSYNTH);
+
+ udelay(1000);
+ sata_set_glue_reg(regbase, SATA_CTL, P0_IRST_POR);
+ udelay(1000);
+ sata_set_glue_reg(regbase, SATA_CTL, P1_IRST_POR);
+ udelay(1000);
+
+ /* setup PHY */
+ config_sata_phy(regbase);
+ if (sata_phy_debug)
+ verify_sata_phy_config(regbase);
+
+ udelay(1000);
+ sata_set_glue_reg(regbase, SATA_CTL, P0_IRST_HARD_TXRX);
+ sata_set_glue_reg(regbase, SATA_CTL, P0_IRST_HARD_SYNTH);
+ sata_set_glue_reg(regbase, SATA_CTL, P1_IRST_HARD_TXRX);
+ sata_set_glue_reg(regbase, SATA_CTL, P1_IRST_HARD_SYNTH);
+ udelay(300);
+
+ /* Override reset in serial PHY mode */
+ sata_set_glue_reg(regbase, CR_REG_TIMER, CR_TIME_SCALE);
+ /* Set reset SATA */
+ sata_set_glue_reg(regbase, SATA_CTL, SATA_RST_N);
+ sata_set_glue_reg(regbase, SATA_CTL, M_CSYSREQ);
+ sata_set_glue_reg(regbase, SATA_CTL, S_CSYSREQ);
+
+ pr_debug("Waiting for PHYs to come up.\n");
+ n = 10000;
+ do {
+ reg_val = nlm_read_sata_reg(regbase, SATA_STATUS);
+ if ((reg_val & P1_PHY_READY) && (reg_val & P0_PHY_READY))
+ break;
+ udelay(10);
+ } while (--n > 0);
+
+ if (reg_val & P0_PHY_READY)
+ pr_info("PHY0 is up.\n");
+ else
+ pr_info("PHY0 is down.\n");
+ if (reg_val & P1_PHY_READY)
+ pr_info("PHY1 is up.\n");
+ else
+ pr_info("PHY1 is down.\n");
+
+ pr_info("XLP AHCI Init Done.\n");
+}
+
+static int __init nlm_ahci_init(void)
+{
+ int node;
+
+ if (!cpu_is_xlp9xx())
+ return 0;
+ for (node = 0; node < NLM_NR_NODES; node++)
+ if (nlm_node_present(node))
+ nlm_sata_firmware_init(node);
+ return 0;
+}
+
+static void nlm_sata_intr_ack(struct irq_data *data)
+{
+ u64 regbase;
+ u32 val;
+ int node;
+
+ node = data->irq / NLM_IRQS_PER_NODE;
+ regbase = nlm_get_sata_regbase(node);
+ val = nlm_read_sata_reg(regbase, SATA_INT);
+ sata_set_glue_reg(regbase, SATA_INT, val);
+}
+
+static void nlm_sata_fixup_bar(struct pci_dev *dev)
+{
+ dev->resource[5] = dev->resource[0];
+ memset(&dev->resource[0], 0, sizeof(dev->resource[0]));
+}
+
+static void nlm_sata_fixup_final(struct pci_dev *dev)
+{
+ u32 val;
+ u64 regbase;
+ int node;
+
+ /* Find end bridge function to find node */
+ node = xlp_socdev_to_node(dev);
+ regbase = nlm_get_sata_regbase(node);
+
+ /* clear pending interrupts and then enable them */
+ val = nlm_read_sata_reg(regbase, SATA_INT);
+ sata_set_glue_reg(regbase, SATA_INT, val);
+
+ /* Enable only the core interrupt */
+ sata_set_glue_reg(regbase, SATA_INT_MASK, 0x1);
+
+ dev->irq = nlm_irq_to_xirq(node, PIC_SATA_IRQ);
+ nlm_set_pic_extra_ack(node, PIC_SATA_IRQ, nlm_sata_intr_ack);
+}
+
+arch_initcall(nlm_ahci_init);
+
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, PCI_DEVICE_ID_XLP9XX_SATA,
+ nlm_sata_fixup_bar);
+
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM, PCI_DEVICE_ID_XLP9XX_SATA,
+ nlm_sata_fixup_final);
diff --git a/arch/mips/netlogic/xlp/ahci-init.c b/arch/mips/netlogic/xlp/ahci-init.c
new file mode 100644
index 0000000..a9d0fae
--- /dev/null
+++ b/arch/mips/netlogic/xlp/ahci-init.c
@@ -0,0 +1,209 @@
+/*
+ * Copyright (c) 2003-2014 Broadcom Corporation
+ * All Rights Reserved
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the Broadcom
+ * license below:
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY BROADCOM ``AS IS'' AND ANY EXPRESS OR
+ * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL BROADCOM OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+ * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
+ * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
+ * IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/kernel.h>
+#include <linux/delay.h>
+#include <linux/init.h>
+#include <linux/pci.h>
+#include <linux/irq.h>
+#include <linux/bitops.h>
+
+#include <asm/cpu.h>
+#include <asm/mipsregs.h>
+
+#include <asm/netlogic/haldefs.h>
+#include <asm/netlogic/xlp-hal/xlp.h>
+#include <asm/netlogic/common.h>
+#include <asm/netlogic/xlp-hal/iomap.h>
+#include <asm/netlogic/mips-extns.h>
+
+#define SATA_CTL 0x0
+#define SATA_STATUS 0x1 /* Status Reg */
+#define SATA_INT 0x2 /* Interrupt Reg */
+#define SATA_INT_MASK 0x3 /* Interrupt Mask Reg */
+#define SATA_CR_REG_TIMER 0x4 /* PHY Conrol Timer Reg */
+#define SATA_CORE_ID 0x5 /* Core ID Reg */
+#define SATA_AXI_SLAVE_OPT1 0x6 /* AXI Slave Options Reg */
+#define SATA_PHY_LOS_LEV 0x7 /* PHY LOS Level Reg */
+#define SATA_PHY_MULTI 0x8 /* PHY Multiplier Reg */
+#define SATA_PHY_CLK_SEL 0x9 /* Clock Select Reg */
+#define SATA_PHY_AMP1_GEN1 0xa /* PHY Transmit Amplitude Reg 1 */
+#define SATA_PHY_AMP1_GEN2 0xb /* PHY Transmit Amplitude Reg 2 */
+#define SATA_PHY_AMP1_GEN3 0xc /* PHY Transmit Amplitude Reg 3 */
+#define SATA_PHY_PRE1 0xd /* PHY Transmit Preemphasis Reg 1 */
+#define SATA_PHY_PRE2 0xe /* PHY Transmit Preemphasis Reg 2 */
+#define SATA_PHY_PRE3 0xf /* PHY Transmit Preemphasis Reg 3 */
+#define SATA_SPDMODE 0x10 /* Speed Mode Reg */
+#define SATA_REFCLK 0x11 /* Reference Clock Control Reg */
+#define SATA_BYTE_SWAP_DIS 0x12 /* byte swap disable */
+
+/*SATA_CTL Bits */
+#define SATA_RST_N BIT(0)
+#define PHY0_RESET_N BIT(16)
+#define PHY1_RESET_N BIT(17)
+#define PHY2_RESET_N BIT(18)
+#define PHY3_RESET_N BIT(19)
+#define M_CSYSREQ BIT(2)
+#define S_CSYSREQ BIT(3)
+
+/*SATA_STATUS Bits */
+#define P0_PHY_READY BIT(4)
+#define P1_PHY_READY BIT(5)
+#define P2_PHY_READY BIT(6)
+#define P3_PHY_READY BIT(7)
+
+#define nlm_read_sata_reg(b, r) nlm_read_reg(b, r)
+#define nlm_write_sata_reg(b, r, v) nlm_write_reg(b, r, v)
+#define nlm_get_sata_pcibase(node) \
+ nlm_pcicfg_base(XLP_IO_SATA_OFFSET(node))
+/* SATA device specific configuration registers are starts at 0x900 offset */
+#define nlm_get_sata_regbase(node) \
+ (nlm_get_sata_pcibase(node) + 0x900)
+
+static void sata_clear_glue_reg(uint64_t regbase, uint32_t off, uint32_t bit)
+{
+ uint32_t reg_val;
+
+ reg_val = nlm_read_sata_reg(regbase, off);
+ nlm_write_sata_reg(regbase, off, (reg_val & ~bit));
+}
+
+static void sata_set_glue_reg(uint64_t regbase, uint32_t off, uint32_t bit)
+{
+ uint32_t reg_val;
+
+ reg_val = nlm_read_sata_reg(regbase, off);
+ nlm_write_sata_reg(regbase, off, (reg_val | bit));
+}
+
+static void nlm_sata_firmware_init(int node)
+{
+ uint32_t reg_val;
+ uint64_t regbase;
+ int i;
+
+ pr_info("XLP AHCI Initialization started.\n");
+ regbase = nlm_get_sata_regbase(node);
+
+ /* Reset SATA */
+ sata_clear_glue_reg(regbase, SATA_CTL, SATA_RST_N);
+ /* Reset PHY */
+ sata_clear_glue_reg(regbase, SATA_CTL,
+ (PHY3_RESET_N | PHY2_RESET_N
+ | PHY1_RESET_N | PHY0_RESET_N));
+
+ /* Set SATA */
+ sata_set_glue_reg(regbase, SATA_CTL, SATA_RST_N);
+ /* Set PHY */
+ sata_set_glue_reg(regbase, SATA_CTL,
+ (PHY3_RESET_N | PHY2_RESET_N
+ | PHY1_RESET_N | PHY0_RESET_N));
+
+ pr_debug("Waiting for PHYs to come up.\n");
+ i = 0;
+ do {
+ reg_val = nlm_read_sata_reg(regbase, SATA_STATUS);
+ i++;
+ } while (((reg_val & 0xF0) != 0xF0) && (i < 10000));
+
+ for (i = 0; i < 4; i++) {
+ if (reg_val & (P0_PHY_READY << i))
+ pr_info("PHY%d is up.\n", i);
+ else
+ pr_info("PHY%d is down.\n", i);
+ }
+
+ pr_info("XLP AHCI init done.\n");
+}
+
+static int __init nlm_ahci_init(void)
+{
+ int node = 0;
+ int chip = read_c0_prid() & PRID_REV_MASK;
+
+ if (chip == PRID_IMP_NETLOGIC_XLP3XX)
+ nlm_sata_firmware_init(node);
+ return 0;
+}
+
+static void nlm_sata_intr_ack(struct irq_data *data)
+{
+ uint32_t val = 0;
+ uint64_t regbase;
+
+ regbase = nlm_get_sata_regbase(nlm_nodeid());
+ val = nlm_read_sata_reg(regbase, SATA_INT);
+ sata_set_glue_reg(regbase, SATA_INT, val);
+}
+
+static void nlm_sata_fixup_bar(struct pci_dev *dev)
+{
+ /*
+ * The AHCI resource is in BAR 0, move it to
+ * BAR 5, where it is expected
+ */
+ dev->resource[5] = dev->resource[0];
+ memset(&dev->resource[0], 0, sizeof(dev->resource[0]));
+}
+
+static void nlm_sata_fixup_final(struct pci_dev *dev)
+{
+ uint32_t val;
+ uint64_t regbase;
+ int node = 0; /* XLP3XX does not support multi-node */
+
+ regbase = nlm_get_sata_regbase(node);
+
+ /* clear pending interrupts and then enable them */
+ val = nlm_read_sata_reg(regbase, SATA_INT);
+ sata_set_glue_reg(regbase, SATA_INT, val);
+
+ /* Mask the core interrupt. If all the interrupts
+ * are enabled there are spurious interrupt flow
+ * happening, to avoid only enable core interrupt
+ * mask.
+ */
+ sata_set_glue_reg(regbase, SATA_INT_MASK, 0x1);
+
+ dev->irq = PIC_SATA_IRQ;
+ nlm_set_pic_extra_ack(node, PIC_SATA_IRQ, nlm_sata_intr_ack);
+}
+
+arch_initcall(nlm_ahci_init);
+
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_NETLOGIC, PCI_DEVICE_ID_NLM_SATA,
+ nlm_sata_fixup_bar);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_NETLOGIC, PCI_DEVICE_ID_NLM_SATA,
+ nlm_sata_fixup_final);
diff --git a/arch/mips/netlogic/xlp/dt.c b/arch/mips/netlogic/xlp/dt.c
index bdde331..7cc4603 100644
--- a/arch/mips/netlogic/xlp/dt.c
+++ b/arch/mips/netlogic/xlp/dt.c
@@ -48,9 +48,10 @@
void __init *xlp_dt_init(void *fdtp)
{
if (!fdtp) {
- switch (current_cpu_data.processor_id & 0xff00) {
+ switch (current_cpu_data.processor_id & PRID_IMP_MASK) {
#ifdef CONFIG_DT_XLP_GVP
case PRID_IMP_NETLOGIC_XLP9XX:
+ case PRID_IMP_NETLOGIC_XLP5XX:
fdtp = __dtb_xlp_gvp_begin;
break;
#endif
diff --git a/arch/mips/netlogic/xlp/nlm_hal.c b/arch/mips/netlogic/xlp/nlm_hal.c
index 997cd9ee1..bc24beb 100644
--- a/arch/mips/netlogic/xlp/nlm_hal.c
+++ b/arch/mips/netlogic/xlp/nlm_hal.c
@@ -54,6 +54,8 @@
struct nlm_soc_info *nodep;
nodep = nlm_get_node(node);
+ if (node == 0)
+ nodep->coremask = 1; /* node 0, boot cpu */
nodep->sysbase = nlm_get_sys_regbase(node);
nodep->picbase = nlm_get_pic_regbase(node);
nodep->ebase = read_c0_ebase() & (~((1 << 12) - 1));
@@ -64,31 +66,39 @@
spin_lock_init(&nodep->piclock);
}
-int nlm_irq_to_irt(int irq)
+static int xlp9xx_irq_to_irt(int irq)
+{
+ switch (irq) {
+ case PIC_GPIO_IRQ:
+ return 12;
+ case PIC_9XX_XHCI_0_IRQ:
+ return 114;
+ case PIC_9XX_XHCI_1_IRQ:
+ return 115;
+ case PIC_UART_0_IRQ:
+ return 133;
+ case PIC_UART_1_IRQ:
+ return 134;
+ case PIC_SATA_IRQ:
+ return 143;
+ case PIC_SPI_IRQ:
+ return 152;
+ case PIC_MMC_IRQ:
+ return 153;
+ case PIC_PCIE_LINK_LEGACY_IRQ(0):
+ case PIC_PCIE_LINK_LEGACY_IRQ(1):
+ case PIC_PCIE_LINK_LEGACY_IRQ(2):
+ case PIC_PCIE_LINK_LEGACY_IRQ(3):
+ return 191 + irq - PIC_PCIE_LINK_LEGACY_IRQ_BASE;
+ }
+ return -1;
+}
+
+static int xlp_irq_to_irt(int irq)
{
uint64_t pcibase;
int devoff, irt;
- /* bypass for 9xx */
- if (cpu_is_xlp9xx()) {
- switch (irq) {
- case PIC_9XX_XHCI_0_IRQ:
- return 114;
- case PIC_9XX_XHCI_1_IRQ:
- return 115;
- case PIC_UART_0_IRQ:
- return 133;
- case PIC_UART_1_IRQ:
- return 134;
- case PIC_PCIE_LINK_LEGACY_IRQ(0):
- case PIC_PCIE_LINK_LEGACY_IRQ(1):
- case PIC_PCIE_LINK_LEGACY_IRQ(2):
- case PIC_PCIE_LINK_LEGACY_IRQ(3):
- return 191 + irq - PIC_PCIE_LINK_LEGACY_IRQ_BASE;
- }
- return -1;
- }
-
devoff = 0;
switch (irq) {
case PIC_UART_0_IRQ:
@@ -98,7 +108,7 @@
devoff = XLP_IO_UART1_OFFSET(0);
break;
case PIC_MMC_IRQ:
- devoff = XLP_IO_SD_OFFSET(0);
+ devoff = XLP_IO_MMC_OFFSET(0);
break;
case PIC_I2C_0_IRQ: /* I2C will be fixed up */
case PIC_I2C_1_IRQ:
@@ -109,6 +119,18 @@
else
devoff = XLP_IO_I2C0_OFFSET(0);
break;
+ case PIC_SATA_IRQ:
+ devoff = XLP_IO_SATA_OFFSET(0);
+ break;
+ case PIC_GPIO_IRQ:
+ devoff = XLP_IO_GPIO_OFFSET(0);
+ break;
+ case PIC_NAND_IRQ:
+ devoff = XLP_IO_NAND_OFFSET(0);
+ break;
+ case PIC_SPI_IRQ:
+ devoff = XLP_IO_SPI_OFFSET(0);
+ break;
default:
if (cpu_is_xlpii()) {
switch (irq) {
@@ -164,61 +186,123 @@
/* HW bug, PCI IRT entries are bad on early silicon, fix */
irt = PIC_IRT_PCIE_LINK_INDEX(irq -
PIC_PCIE_LINK_LEGACY_IRQ_BASE);
- } else if (irq >= PIC_PCIE_LINK_MSI_IRQ(0) &&
- irq <= PIC_PCIE_LINK_MSI_IRQ(3)) {
- irt = -2;
- } else if (irq >= PIC_PCIE_MSIX_IRQ(0) &&
- irq <= PIC_PCIE_MSIX_IRQ(3)) {
- irt = -2;
} else {
irt = -1;
}
return irt;
}
-unsigned int nlm_get_core_frequency(int node, int core)
+int nlm_irq_to_irt(int irq)
+{
+ /* return -2 for irqs without 1-1 mapping */
+ if (irq >= PIC_PCIE_LINK_MSI_IRQ(0) && irq <= PIC_PCIE_LINK_MSI_IRQ(3))
+ return -2;
+ if (irq >= PIC_PCIE_MSIX_IRQ(0) && irq <= PIC_PCIE_MSIX_IRQ(3))
+ return -2;
+
+ if (cpu_is_xlp9xx())
+ return xlp9xx_irq_to_irt(irq);
+ else
+ return xlp_irq_to_irt(irq);
+}
+
+static unsigned int nlm_xlp2_get_core_frequency(int node, int core)
+{
+ unsigned int pll_post_div, ctrl_val0, ctrl_val1, denom;
+ uint64_t num, sysbase, clockbase;
+
+ if (cpu_is_xlp9xx()) {
+ clockbase = nlm_get_clock_regbase(node);
+ ctrl_val0 = nlm_read_sys_reg(clockbase,
+ SYS_9XX_CPU_PLL_CTRL0(core));
+ ctrl_val1 = nlm_read_sys_reg(clockbase,
+ SYS_9XX_CPU_PLL_CTRL1(core));
+ } else {
+ sysbase = nlm_get_node(node)->sysbase;
+ ctrl_val0 = nlm_read_sys_reg(sysbase,
+ SYS_CPU_PLL_CTRL0(core));
+ ctrl_val1 = nlm_read_sys_reg(sysbase,
+ SYS_CPU_PLL_CTRL1(core));
+ }
+
+ /* Find PLL post divider value */
+ switch ((ctrl_val0 >> 24) & 0x7) {
+ case 1:
+ pll_post_div = 2;
+ break;
+ case 3:
+ pll_post_div = 4;
+ break;
+ case 7:
+ pll_post_div = 8;
+ break;
+ case 6:
+ pll_post_div = 16;
+ break;
+ case 0:
+ default:
+ pll_post_div = 1;
+ break;
+ }
+
+ num = 1000000ULL * (400 * 3 + 100 * (ctrl_val1 & 0x3f));
+ denom = 3 * pll_post_div;
+ do_div(num, denom);
+
+ return (unsigned int)num;
+}
+
+static unsigned int nlm_xlp_get_core_frequency(int node, int core)
{
unsigned int pll_divf, pll_divr, dfs_div, ext_div;
unsigned int rstval, dfsval, denom;
uint64_t num, sysbase;
sysbase = nlm_get_node(node)->sysbase;
- if (cpu_is_xlp9xx())
- rstval = nlm_read_sys_reg(sysbase, SYS_9XX_POWER_ON_RESET_CFG);
- else
- rstval = nlm_read_sys_reg(sysbase, SYS_POWER_ON_RESET_CFG);
- if (cpu_is_xlpii()) {
- num = 1000000ULL * (400 * 3 + 100 * (rstval >> 26));
- denom = 3;
- } else {
- dfsval = nlm_read_sys_reg(sysbase, SYS_CORE_DFS_DIV_VALUE);
- pll_divf = ((rstval >> 10) & 0x7f) + 1;
- pll_divr = ((rstval >> 8) & 0x3) + 1;
- ext_div = ((rstval >> 30) & 0x3) + 1;
- dfs_div = ((dfsval >> (core * 4)) & 0xf) + 1;
+ rstval = nlm_read_sys_reg(sysbase, SYS_POWER_ON_RESET_CFG);
+ dfsval = nlm_read_sys_reg(sysbase, SYS_CORE_DFS_DIV_VALUE);
+ pll_divf = ((rstval >> 10) & 0x7f) + 1;
+ pll_divr = ((rstval >> 8) & 0x3) + 1;
+ ext_div = ((rstval >> 30) & 0x3) + 1;
+ dfs_div = ((dfsval >> (core * 4)) & 0xf) + 1;
- num = 800000000ULL * pll_divf;
- denom = 3 * pll_divr * ext_div * dfs_div;
- }
+ num = 800000000ULL * pll_divf;
+ denom = 3 * pll_divr * ext_div * dfs_div;
do_div(num, denom);
+
return (unsigned int)num;
}
-/* Calculate Frequency to the PIC from PLL.
- * freq_out = ( ref_freq/2 * (6 + ctrl2[7:0]) + ctrl2[20:8]/2^13 ) /
- * ((2^ctrl0[7:5]) * Table(ctrl0[26:24]))
- */
-static unsigned int nlm_2xx_get_pic_frequency(int node)
+unsigned int nlm_get_core_frequency(int node, int core)
{
- u32 ctrl_val0, ctrl_val2, vco_post_div, pll_post_div;
+ if (cpu_is_xlpii())
+ return nlm_xlp2_get_core_frequency(node, core);
+ else
+ return nlm_xlp_get_core_frequency(node, core);
+}
+
+/*
+ * Calculate PIC frequency from PLL registers.
+ * freq_out = (ref_freq/2 * (6 + ctrl2[7:0]) + ctrl2[20:8]/2^13) /
+ * ((2^ctrl0[7:5]) * Table(ctrl0[26:24]))
+ */
+static unsigned int nlm_xlp2_get_pic_frequency(int node)
+{
+ u32 ctrl_val0, ctrl_val2, vco_post_div, pll_post_div, cpu_xlp9xx;
u32 mdiv, fdiv, pll_out_freq_den, reg_select, ref_div, pic_div;
- u64 ref_clk, sysbase, pll_out_freq_num, ref_clk_select;
+ u64 sysbase, pll_out_freq_num, ref_clk_select, clockbase, ref_clk;
sysbase = nlm_get_node(node)->sysbase;
+ clockbase = nlm_get_clock_regbase(node);
+ cpu_xlp9xx = cpu_is_xlp9xx();
/* Find ref_clk_base */
- ref_clk_select =
- (nlm_read_sys_reg(sysbase, SYS_POWER_ON_RESET_CFG) >> 18) & 0x3;
+ if (cpu_xlp9xx)
+ ref_clk_select = (nlm_read_sys_reg(sysbase,
+ SYS_9XX_POWER_ON_RESET_CFG) >> 18) & 0x3;
+ else
+ ref_clk_select = (nlm_read_sys_reg(sysbase,
+ SYS_POWER_ON_RESET_CFG) >> 18) & 0x3;
switch (ref_clk_select) {
case 0:
ref_clk = 200000000ULL;
@@ -239,30 +323,70 @@
}
/* Find the clock source PLL device for PIC */
- reg_select = (nlm_read_sys_reg(sysbase, SYS_CLK_DEV_SEL) >> 22) & 0x3;
- switch (reg_select) {
- case 0:
- ctrl_val0 = nlm_read_sys_reg(sysbase, SYS_PLL_CTRL0);
- ctrl_val2 = nlm_read_sys_reg(sysbase, SYS_PLL_CTRL2);
- break;
- case 1:
- ctrl_val0 = nlm_read_sys_reg(sysbase, SYS_PLL_CTRL0_DEVX(0));
- ctrl_val2 = nlm_read_sys_reg(sysbase, SYS_PLL_CTRL2_DEVX(0));
- break;
- case 2:
- ctrl_val0 = nlm_read_sys_reg(sysbase, SYS_PLL_CTRL0_DEVX(1));
- ctrl_val2 = nlm_read_sys_reg(sysbase, SYS_PLL_CTRL2_DEVX(1));
- break;
- case 3:
- ctrl_val0 = nlm_read_sys_reg(sysbase, SYS_PLL_CTRL0_DEVX(2));
- ctrl_val2 = nlm_read_sys_reg(sysbase, SYS_PLL_CTRL2_DEVX(2));
- break;
+ if (cpu_xlp9xx) {
+ reg_select = nlm_read_sys_reg(clockbase,
+ SYS_9XX_CLK_DEV_SEL) & 0x3;
+ switch (reg_select) {
+ case 0:
+ ctrl_val0 = nlm_read_sys_reg(clockbase,
+ SYS_9XX_PLL_CTRL0);
+ ctrl_val2 = nlm_read_sys_reg(clockbase,
+ SYS_9XX_PLL_CTRL2);
+ break;
+ case 1:
+ ctrl_val0 = nlm_read_sys_reg(clockbase,
+ SYS_9XX_PLL_CTRL0_DEVX(0));
+ ctrl_val2 = nlm_read_sys_reg(clockbase,
+ SYS_9XX_PLL_CTRL2_DEVX(0));
+ break;
+ case 2:
+ ctrl_val0 = nlm_read_sys_reg(clockbase,
+ SYS_9XX_PLL_CTRL0_DEVX(1));
+ ctrl_val2 = nlm_read_sys_reg(clockbase,
+ SYS_9XX_PLL_CTRL2_DEVX(1));
+ break;
+ case 3:
+ ctrl_val0 = nlm_read_sys_reg(clockbase,
+ SYS_9XX_PLL_CTRL0_DEVX(2));
+ ctrl_val2 = nlm_read_sys_reg(clockbase,
+ SYS_9XX_PLL_CTRL2_DEVX(2));
+ break;
+ }
+ } else {
+ reg_select = (nlm_read_sys_reg(sysbase,
+ SYS_CLK_DEV_SEL) >> 22) & 0x3;
+ switch (reg_select) {
+ case 0:
+ ctrl_val0 = nlm_read_sys_reg(sysbase,
+ SYS_PLL_CTRL0);
+ ctrl_val2 = nlm_read_sys_reg(sysbase,
+ SYS_PLL_CTRL2);
+ break;
+ case 1:
+ ctrl_val0 = nlm_read_sys_reg(sysbase,
+ SYS_PLL_CTRL0_DEVX(0));
+ ctrl_val2 = nlm_read_sys_reg(sysbase,
+ SYS_PLL_CTRL2_DEVX(0));
+ break;
+ case 2:
+ ctrl_val0 = nlm_read_sys_reg(sysbase,
+ SYS_PLL_CTRL0_DEVX(1));
+ ctrl_val2 = nlm_read_sys_reg(sysbase,
+ SYS_PLL_CTRL2_DEVX(1));
+ break;
+ case 3:
+ ctrl_val0 = nlm_read_sys_reg(sysbase,
+ SYS_PLL_CTRL0_DEVX(2));
+ ctrl_val2 = nlm_read_sys_reg(sysbase,
+ SYS_PLL_CTRL2_DEVX(2));
+ break;
+ }
}
vco_post_div = (ctrl_val0 >> 5) & 0x7;
pll_post_div = (ctrl_val0 >> 24) & 0x7;
mdiv = ctrl_val2 & 0xff;
- fdiv = (ctrl_val2 >> 8) & 0xfff;
+ fdiv = (ctrl_val2 >> 8) & 0x1fff;
/* Find PLL post divider value */
switch (pll_post_div) {
@@ -292,7 +416,12 @@
do_div(pll_out_freq_num, pll_out_freq_den);
/* PIC post divider, which happens after PLL */
- pic_div = (nlm_read_sys_reg(sysbase, SYS_CLK_DEV_DIV) >> 22) & 0x3;
+ if (cpu_xlp9xx)
+ pic_div = nlm_read_sys_reg(clockbase,
+ SYS_9XX_CLK_DEV_DIV) & 0x3;
+ else
+ pic_div = (nlm_read_sys_reg(sysbase,
+ SYS_CLK_DEV_DIV) >> 22) & 0x3;
do_div(pll_out_freq_num, 1 << pic_div);
return pll_out_freq_num;
@@ -300,12 +429,8 @@
unsigned int nlm_get_pic_frequency(int node)
{
- /* TODO Has to calculate freq as like 2xx */
- if (cpu_is_xlp9xx())
- return 250000000;
-
if (cpu_is_xlpii())
- return nlm_2xx_get_pic_frequency(node);
+ return nlm_xlp2_get_pic_frequency(node);
else
return 133333333;
}
diff --git a/arch/mips/netlogic/xlp/setup.c b/arch/mips/netlogic/xlp/setup.c
index 8c60a2d..4fdd9fd 100644
--- a/arch/mips/netlogic/xlp/setup.c
+++ b/arch/mips/netlogic/xlp/setup.c
@@ -121,8 +121,9 @@
const char *get_system_type(void)
{
- switch (read_c0_prid() & 0xff00) {
+ switch (read_c0_prid() & PRID_IMP_MASK) {
case PRID_IMP_NETLOGIC_XLP9XX:
+ case PRID_IMP_NETLOGIC_XLP5XX:
case PRID_IMP_NETLOGIC_XLP2XX:
return "Broadcom XLPII Series";
default:
diff --git a/arch/mips/netlogic/xlp/wakeup.c b/arch/mips/netlogic/xlp/wakeup.c
index 9a92617..e5f44d2 100644
--- a/arch/mips/netlogic/xlp/wakeup.c
+++ b/arch/mips/netlogic/xlp/wakeup.c
@@ -135,11 +135,19 @@
if (cpu_is_xlp9xx()) {
fusebase = nlm_get_fuse_regbase(n);
fusemask = nlm_read_reg(fusebase, FUSE_9XX_DEVCFG6);
- mask = 0xfffff;
+ switch (read_c0_prid() & PRID_IMP_MASK) {
+ case PRID_IMP_NETLOGIC_XLP5XX:
+ mask = 0xff;
+ break;
+ case PRID_IMP_NETLOGIC_XLP9XX:
+ default:
+ mask = 0xfffff;
+ break;
+ }
} else {
fusemask = nlm_read_sys_reg(nodep->sysbase,
SYS_EFUSE_DEVICE_CFG_STATUS0);
- switch (read_c0_prid() & 0xff00) {
+ switch (read_c0_prid() & PRID_IMP_MASK) {
case PRID_IMP_NETLOGIC_XLP3XX:
mask = 0xf;
break;
@@ -159,10 +167,6 @@
*/
syscoremask = (1 << hweight32(~fusemask & mask)) - 1;
- /* The boot cpu */
- if (n == 0)
- nodep->coremask = 1;
-
pr_info("Node %d - SYS/FUSE coremask %x\n", n, syscoremask);
for (core = 0; core < nlm_cores_per_node(); core++) {
/* we will be on node 0 core 0 */
diff --git a/arch/mips/paravirt/Kconfig b/arch/mips/paravirt/Kconfig
new file mode 100644
index 0000000..ecae586
--- /dev/null
+++ b/arch/mips/paravirt/Kconfig
@@ -0,0 +1,6 @@
+if MIPS_PARAVIRT
+
+config MIPS_PCI_VIRTIO
+ def_bool y
+
+endif # MIPS_PARAVIRT
diff --git a/arch/mips/paravirt/Makefile b/arch/mips/paravirt/Makefile
new file mode 100644
index 0000000..5023af7
--- /dev/null
+++ b/arch/mips/paravirt/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for MIPS para-virtualized specific kernel interface routines
+# under Linux.
+#
+# This file is subject to the terms and conditions of the GNU General Public
+# License. See the file "COPYING" in the main directory of this archive
+# for more details.
+#
+# Copyright (C) 2013 Cavium, Inc.
+#
+
+obj-y := setup.o serial.o paravirt-irq.o
+
+obj-$(CONFIG_SMP) += paravirt-smp.o
diff --git a/arch/mips/paravirt/Platform b/arch/mips/paravirt/Platform
new file mode 100644
index 0000000..7e76ef2
--- /dev/null
+++ b/arch/mips/paravirt/Platform
@@ -0,0 +1,8 @@
+#
+# Generic para-virtualized guest.
+#
+platform-$(CONFIG_MIPS_PARAVIRT) += paravirt/
+cflags-$(CONFIG_MIPS_PARAVIRT) += \
+ -I$(srctree)/arch/mips/include/asm/mach-paravirt
+
+load-$(CONFIG_MIPS_PARAVIRT) = 0xffffffff80010000
diff --git a/arch/mips/paravirt/paravirt-irq.c b/arch/mips/paravirt/paravirt-irq.c
new file mode 100644
index 0000000..8987b06
--- /dev/null
+++ b/arch/mips/paravirt/paravirt-irq.c
@@ -0,0 +1,368 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2013 Cavium, Inc.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/cpumask.h>
+#include <linux/kernel.h>
+#include <linux/mutex.h>
+
+#include <asm/io.h>
+
+#define MBOX_BITS_PER_CPU 2
+
+static int cpunum_for_cpu(int cpu)
+{
+#ifdef CONFIG_SMP
+ return cpu_logical_map(cpu);
+#else
+ return get_ebase_cpunum();
+#endif
+}
+
+struct core_chip_data {
+ struct mutex core_irq_mutex;
+ bool current_en;
+ bool desired_en;
+ u8 bit;
+};
+
+static struct core_chip_data irq_core_chip_data[8];
+
+static void irq_core_ack(struct irq_data *data)
+{
+ struct core_chip_data *cd = irq_data_get_irq_chip_data(data);
+ unsigned int bit = cd->bit;
+
+ /*
+ * We don't need to disable IRQs to make these atomic since
+ * they are already disabled earlier in the low level
+ * interrupt code.
+ */
+ clear_c0_status(0x100 << bit);
+ /* The two user interrupts must be cleared manually. */
+ if (bit < 2)
+ clear_c0_cause(0x100 << bit);
+}
+
+static void irq_core_eoi(struct irq_data *data)
+{
+ struct core_chip_data *cd = irq_data_get_irq_chip_data(data);
+
+ /*
+ * We don't need to disable IRQs to make these atomic since
+ * they are already disabled earlier in the low level
+ * interrupt code.
+ */
+ set_c0_status(0x100 << cd->bit);
+}
+
+static void irq_core_set_enable_local(void *arg)
+{
+ struct irq_data *data = arg;
+ struct core_chip_data *cd = irq_data_get_irq_chip_data(data);
+ unsigned int mask = 0x100 << cd->bit;
+
+ /*
+ * Interrupts are already disabled, so these are atomic.
+ */
+ if (cd->desired_en)
+ set_c0_status(mask);
+ else
+ clear_c0_status(mask);
+
+}
+
+static void irq_core_disable(struct irq_data *data)
+{
+ struct core_chip_data *cd = irq_data_get_irq_chip_data(data);
+ cd->desired_en = false;
+}
+
+static void irq_core_enable(struct irq_data *data)
+{
+ struct core_chip_data *cd = irq_data_get_irq_chip_data(data);
+ cd->desired_en = true;
+}
+
+static void irq_core_bus_lock(struct irq_data *data)
+{
+ struct core_chip_data *cd = irq_data_get_irq_chip_data(data);
+
+ mutex_lock(&cd->core_irq_mutex);
+}
+
+static void irq_core_bus_sync_unlock(struct irq_data *data)
+{
+ struct core_chip_data *cd = irq_data_get_irq_chip_data(data);
+
+ if (cd->desired_en != cd->current_en) {
+ on_each_cpu(irq_core_set_enable_local, data, 1);
+ cd->current_en = cd->desired_en;
+ }
+
+ mutex_unlock(&cd->core_irq_mutex);
+}
+
+static struct irq_chip irq_chip_core = {
+ .name = "Core",
+ .irq_enable = irq_core_enable,
+ .irq_disable = irq_core_disable,
+ .irq_ack = irq_core_ack,
+ .irq_eoi = irq_core_eoi,
+ .irq_bus_lock = irq_core_bus_lock,
+ .irq_bus_sync_unlock = irq_core_bus_sync_unlock,
+
+ .irq_cpu_online = irq_core_eoi,
+ .irq_cpu_offline = irq_core_ack,
+ .flags = IRQCHIP_ONOFFLINE_ENABLED,
+};
+
+static void __init irq_init_core(void)
+{
+ int i;
+ int irq;
+ struct core_chip_data *cd;
+
+ /* Start with a clean slate */
+ clear_c0_status(ST0_IM);
+ clear_c0_cause(CAUSEF_IP0 | CAUSEF_IP1);
+
+ for (i = 0; i < ARRAY_SIZE(irq_core_chip_data); i++) {
+ cd = irq_core_chip_data + i;
+ cd->current_en = false;
+ cd->desired_en = false;
+ cd->bit = i;
+ mutex_init(&cd->core_irq_mutex);
+
+ irq = MIPS_CPU_IRQ_BASE + i;
+
+ switch (i) {
+ case 0: /* SW0 */
+ case 1: /* SW1 */
+ case 5: /* IP5 */
+ case 6: /* IP6 */
+ case 7: /* IP7 */
+ irq_set_chip_data(irq, cd);
+ irq_set_chip_and_handler(irq, &irq_chip_core,
+ handle_percpu_irq);
+ break;
+ default:
+ break;
+ }
+ }
+}
+
+static void __iomem *mips_irq_chip;
+#define MIPS_IRQ_CHIP_NUM_BITS 0
+#define MIPS_IRQ_CHIP_REGS 8
+
+static int mips_irq_cpu_stride;
+static int mips_irq_chip_reg_raw;
+static int mips_irq_chip_reg_src;
+static int mips_irq_chip_reg_en;
+static int mips_irq_chip_reg_raw_w1s;
+static int mips_irq_chip_reg_raw_w1c;
+static int mips_irq_chip_reg_en_w1s;
+static int mips_irq_chip_reg_en_w1c;
+
+static void irq_pci_enable(struct irq_data *data)
+{
+ u32 mask = 1u << data->irq;
+
+ __raw_writel(mask, mips_irq_chip + mips_irq_chip_reg_en_w1s);
+}
+
+static void irq_pci_disable(struct irq_data *data)
+{
+ u32 mask = 1u << data->irq;
+
+ __raw_writel(mask, mips_irq_chip + mips_irq_chip_reg_en_w1c);
+}
+
+static void irq_pci_ack(struct irq_data *data)
+{
+}
+
+static void irq_pci_mask(struct irq_data *data)
+{
+ u32 mask = 1u << data->irq;
+
+ __raw_writel(mask, mips_irq_chip + mips_irq_chip_reg_en_w1c);
+}
+
+static void irq_pci_unmask(struct irq_data *data)
+{
+ u32 mask = 1u << data->irq;
+
+ __raw_writel(mask, mips_irq_chip + mips_irq_chip_reg_en_w1s);
+}
+
+static struct irq_chip irq_chip_pci = {
+ .name = "PCI",
+ .irq_enable = irq_pci_enable,
+ .irq_disable = irq_pci_disable,
+ .irq_ack = irq_pci_ack,
+ .irq_mask = irq_pci_mask,
+ .irq_unmask = irq_pci_unmask,
+};
+
+static void irq_mbox_all(struct irq_data *data, void __iomem *base)
+{
+ int cpu;
+ unsigned int mbox = data->irq - MIPS_IRQ_MBOX0;
+ u32 mask;
+
+ WARN_ON(mbox >= MBOX_BITS_PER_CPU);
+
+ for_each_online_cpu(cpu) {
+ unsigned int cpuid = cpunum_for_cpu(cpu);
+ mask = 1 << (cpuid * MBOX_BITS_PER_CPU + mbox);
+ __raw_writel(mask, base + (cpuid * mips_irq_cpu_stride));
+ }
+}
+
+static void irq_mbox_enable(struct irq_data *data)
+{
+ irq_mbox_all(data, mips_irq_chip + mips_irq_chip_reg_en_w1s + sizeof(u32));
+}
+
+static void irq_mbox_disable(struct irq_data *data)
+{
+ irq_mbox_all(data, mips_irq_chip + mips_irq_chip_reg_en_w1c + sizeof(u32));
+}
+
+static void irq_mbox_ack(struct irq_data *data)
+{
+ u32 mask;
+ unsigned int mbox = data->irq - MIPS_IRQ_MBOX0;
+
+ WARN_ON(mbox >= MBOX_BITS_PER_CPU);
+
+ mask = 1 << (get_ebase_cpunum() * MBOX_BITS_PER_CPU + mbox);
+ __raw_writel(mask, mips_irq_chip + mips_irq_chip_reg_raw_w1c + sizeof(u32));
+}
+
+void irq_mbox_ipi(int cpu, unsigned int actions)
+{
+ unsigned int cpuid = cpunum_for_cpu(cpu);
+ u32 mask;
+
+ WARN_ON(actions >= (1 << MBOX_BITS_PER_CPU));
+
+ mask = actions << (cpuid * MBOX_BITS_PER_CPU);
+ __raw_writel(mask, mips_irq_chip + mips_irq_chip_reg_raw_w1s + sizeof(u32));
+}
+
+static void irq_mbox_cpu_onoffline(struct irq_data *data, void __iomem *base)
+{
+ unsigned int mbox = data->irq - MIPS_IRQ_MBOX0;
+ unsigned int cpuid = get_ebase_cpunum();
+ u32 mask;
+
+ WARN_ON(mbox >= MBOX_BITS_PER_CPU);
+
+ mask = 1 << (cpuid * MBOX_BITS_PER_CPU + mbox);
+ __raw_writel(mask, base + (cpuid * mips_irq_cpu_stride));
+
+}
+
+static void irq_mbox_cpu_online(struct irq_data *data)
+{
+ irq_mbox_cpu_onoffline(data, mips_irq_chip + mips_irq_chip_reg_en_w1s + sizeof(u32));
+}
+
+static void irq_mbox_cpu_offline(struct irq_data *data)
+{
+ irq_mbox_cpu_onoffline(data, mips_irq_chip + mips_irq_chip_reg_en_w1c + sizeof(u32));
+}
+
+static struct irq_chip irq_chip_mbox = {
+ .name = "MBOX",
+ .irq_enable = irq_mbox_enable,
+ .irq_disable = irq_mbox_disable,
+ .irq_ack = irq_mbox_ack,
+ .irq_cpu_online = irq_mbox_cpu_online,
+ .irq_cpu_offline = irq_mbox_cpu_offline,
+ .flags = IRQCHIP_ONOFFLINE_ENABLED,
+};
+
+static void __init irq_pci_init(void)
+{
+ int i, stride;
+ u32 num_bits;
+
+ mips_irq_chip = ioremap(0x1e010000, 4096);
+
+ num_bits = __raw_readl(mips_irq_chip + MIPS_IRQ_CHIP_NUM_BITS);
+ stride = 8 * (1 + ((num_bits - 1) / 64));
+
+
+ pr_notice("mips_irq_chip: %u bits, reg stride: %d\n", num_bits, stride);
+ mips_irq_chip_reg_raw = MIPS_IRQ_CHIP_REGS + 0 * stride;
+ mips_irq_chip_reg_raw_w1s = MIPS_IRQ_CHIP_REGS + 1 * stride;
+ mips_irq_chip_reg_raw_w1c = MIPS_IRQ_CHIP_REGS + 2 * stride;
+ mips_irq_chip_reg_src = MIPS_IRQ_CHIP_REGS + 3 * stride;
+ mips_irq_chip_reg_en = MIPS_IRQ_CHIP_REGS + 4 * stride;
+ mips_irq_chip_reg_en_w1s = MIPS_IRQ_CHIP_REGS + 5 * stride;
+ mips_irq_chip_reg_en_w1c = MIPS_IRQ_CHIP_REGS + 6 * stride;
+ mips_irq_cpu_stride = stride * 4;
+
+ for (i = 0; i < 4; i++)
+ irq_set_chip_and_handler(i + MIPS_IRQ_PCIA, &irq_chip_pci, handle_level_irq);
+
+ for (i = 0; i < 2; i++)
+ irq_set_chip_and_handler(i + MIPS_IRQ_MBOX0, &irq_chip_mbox, handle_percpu_irq);
+
+
+ set_c0_status(STATUSF_IP2);
+}
+
+static void irq_pci_dispatch(void)
+{
+ unsigned int cpuid = get_ebase_cpunum();
+ u32 en;
+
+ en = __raw_readl(mips_irq_chip + mips_irq_chip_reg_src +
+ (cpuid * mips_irq_cpu_stride));
+
+ if (!en) {
+ en = __raw_readl(mips_irq_chip + mips_irq_chip_reg_src + (cpuid * mips_irq_cpu_stride) + sizeof(u32));
+ en = (en >> (2 * cpuid)) & 3;
+
+ if (!en)
+ spurious_interrupt();
+ else
+ do_IRQ(__ffs(en) + MIPS_IRQ_MBOX0); /* MBOX type */
+ } else {
+ do_IRQ(__ffs(en));
+ }
+}
+
+
+void __init arch_init_irq(void)
+{
+ irq_init_core();
+ irq_pci_init();
+}
+
+asmlinkage void plat_irq_dispatch(void)
+{
+ unsigned int pending = read_c0_cause() & read_c0_status() & ST0_IM;
+ int ip;
+
+ if (unlikely(!pending)) {
+ spurious_interrupt();
+ return;
+ }
+
+ ip = ffs(pending) - 1 - STATUSB_IP0;
+ if (ip == 2)
+ irq_pci_dispatch();
+ else
+ do_IRQ(MIPS_CPU_IRQ_BASE + ip);
+}
diff --git a/arch/mips/paravirt/paravirt-smp.c b/arch/mips/paravirt/paravirt-smp.c
new file mode 100644
index 0000000..0164b0c
--- /dev/null
+++ b/arch/mips/paravirt/paravirt-smp.c
@@ -0,0 +1,143 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2013 Cavium, Inc.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/cpumask.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+
+#include <asm/mipsregs.h>
+#include <asm/setup.h>
+#include <asm/time.h>
+#include <asm/smp.h>
+
+/*
+ * Writing the sp releases the CPU, so writes must be ordered, gp
+ * first, then sp.
+ */
+unsigned long paravirt_smp_sp[NR_CPUS];
+unsigned long paravirt_smp_gp[NR_CPUS];
+
+static int numcpus = 1;
+
+static int __init set_numcpus(char *str)
+{
+ int newval;
+
+ if (get_option(&str, &newval)) {
+ if (newval < 1 || newval >= NR_CPUS)
+ goto bad;
+ numcpus = newval;
+ return 0;
+ }
+bad:
+ return -EINVAL;
+}
+early_param("numcpus", set_numcpus);
+
+
+static void paravirt_smp_setup(void)
+{
+ int id;
+ unsigned int cpunum = get_ebase_cpunum();
+
+ if (WARN_ON(cpunum >= NR_CPUS))
+ return;
+
+ /* The present CPUs are initially just the boot cpu (CPU 0). */
+ for (id = 0; id < NR_CPUS; id++) {
+ set_cpu_possible(id, id == 0);
+ set_cpu_present(id, id == 0);
+ }
+ __cpu_number_map[cpunum] = 0;
+ __cpu_logical_map[0] = cpunum;
+
+ for (id = 0; id < numcpus; id++) {
+ set_cpu_possible(id, true);
+ set_cpu_present(id, true);
+ __cpu_number_map[id] = id;
+ __cpu_logical_map[id] = id;
+ }
+}
+
+void irq_mbox_ipi(int cpu, unsigned int actions);
+static void paravirt_send_ipi_single(int cpu, unsigned int action)
+{
+ irq_mbox_ipi(cpu, action);
+}
+
+static void paravirt_send_ipi_mask(const struct cpumask *mask, unsigned int action)
+{
+ unsigned int cpu;
+
+ for_each_cpu_mask(cpu, *mask)
+ paravirt_send_ipi_single(cpu, action);
+}
+
+static void paravirt_init_secondary(void)
+{
+ unsigned int sr;
+
+ sr = set_c0_status(ST0_BEV);
+ write_c0_ebase((u32)ebase);
+
+ sr |= STATUSF_IP2; /* Interrupt controller on IP2 */
+ write_c0_status(sr);
+
+ irq_cpu_online();
+}
+
+static void paravirt_smp_finish(void)
+{
+ /* to generate the first CPU timer interrupt */
+ write_c0_compare(read_c0_count() + mips_hpt_frequency / HZ);
+ local_irq_enable();
+}
+
+static void paravirt_boot_secondary(int cpu, struct task_struct *idle)
+{
+ paravirt_smp_gp[cpu] = (unsigned long)task_thread_info(idle);
+ smp_wmb();
+ paravirt_smp_sp[cpu] = __KSTK_TOS(idle);
+}
+
+static irqreturn_t paravirt_reched_interrupt(int irq, void *dev_id)
+{
+ scheduler_ipi();
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t paravirt_function_interrupt(int irq, void *dev_id)
+{
+ smp_call_function_interrupt();
+ return IRQ_HANDLED;
+}
+
+static void paravirt_prepare_cpus(unsigned int max_cpus)
+{
+ if (request_irq(MIPS_IRQ_MBOX0, paravirt_reched_interrupt,
+ IRQF_PERCPU | IRQF_NO_THREAD, "Scheduler",
+ paravirt_reched_interrupt)) {
+ panic("Cannot request_irq for SchedulerIPI");
+ }
+ if (request_irq(MIPS_IRQ_MBOX1, paravirt_function_interrupt,
+ IRQF_PERCPU | IRQF_NO_THREAD, "SMP-Call",
+ paravirt_function_interrupt)) {
+ panic("Cannot request_irq for SMP-Call");
+ }
+}
+
+struct plat_smp_ops paravirt_smp_ops = {
+ .send_ipi_single = paravirt_send_ipi_single,
+ .send_ipi_mask = paravirt_send_ipi_mask,
+ .init_secondary = paravirt_init_secondary,
+ .smp_finish = paravirt_smp_finish,
+ .boot_secondary = paravirt_boot_secondary,
+ .smp_setup = paravirt_smp_setup,
+ .prepare_cpus = paravirt_prepare_cpus,
+};
diff --git a/arch/mips/paravirt/serial.c b/arch/mips/paravirt/serial.c
new file mode 100644
index 0000000..02b665c
--- /dev/null
+++ b/arch/mips/paravirt/serial.c
@@ -0,0 +1,40 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2013 Cavium, Inc.
+ */
+
+#include <linux/kernel.h>
+#include <linux/virtio_console.h>
+#include <linux/kvm_para.h>
+
+/*
+ * Emit one character to the boot console.
+ */
+int prom_putchar(char c)
+{
+ kvm_hypercall3(KVM_HC_MIPS_CONSOLE_OUTPUT, 0 /* port 0 */,
+ (unsigned long)&c, 1 /* len == 1 */);
+
+ return 1;
+}
+
+#ifdef CONFIG_VIRTIO_CONSOLE
+static int paravirt_put_chars(u32 vtermno, const char *buf, int count)
+{
+ kvm_hypercall3(KVM_HC_MIPS_CONSOLE_OUTPUT, vtermno,
+ (unsigned long)buf, count);
+
+ return count;
+}
+
+static int __init paravirt_cons_init(void)
+{
+ virtio_cons_early_init(paravirt_put_chars);
+ return 0;
+}
+core_initcall(paravirt_cons_init);
+
+#endif
diff --git a/arch/mips/paravirt/setup.c b/arch/mips/paravirt/setup.c
new file mode 100644
index 0000000..cb8448b
--- /dev/null
+++ b/arch/mips/paravirt/setup.c
@@ -0,0 +1,67 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2013 Cavium, Inc.
+ */
+
+#include <linux/kernel.h>
+#include <linux/kvm_para.h>
+
+#include <asm/reboot.h>
+#include <asm/bootinfo.h>
+#include <asm/smp-ops.h>
+#include <asm/time.h>
+
+extern struct plat_smp_ops paravirt_smp_ops;
+
+const char *get_system_type(void)
+{
+ return "MIPS Para-Virtualized Guest";
+}
+
+void __init plat_time_init(void)
+{
+ mips_hpt_frequency = kvm_hypercall0(KVM_HC_MIPS_GET_CLOCK_FREQ);
+
+ preset_lpj = mips_hpt_frequency / (2 * HZ);
+}
+
+static void pv_machine_halt(void)
+{
+ kvm_hypercall0(KVM_HC_MIPS_EXIT_VM);
+}
+
+/*
+ * Early entry point for arch setup
+ */
+void __init prom_init(void)
+{
+ int i;
+ int argc = fw_arg0;
+ char **argv = (char **)fw_arg1;
+
+#ifdef CONFIG_32BIT
+ set_io_port_base(KSEG1ADDR(0x1e000000));
+#else /* CONFIG_64BIT */
+ set_io_port_base(PHYS_TO_XKSEG_UNCACHED(0x1e000000));
+#endif
+
+ for (i = 0; i < argc; i++) {
+ strlcat(arcs_cmdline, argv[i], COMMAND_LINE_SIZE);
+ if (i < argc - 1)
+ strlcat(arcs_cmdline, " ", COMMAND_LINE_SIZE);
+ }
+ _machine_halt = pv_machine_halt;
+ register_smp_ops(¶virt_smp_ops);
+}
+
+void __init plat_mem_setup(void)
+{
+ /* Do nothing, the "mem=???" parser handles our memory. */
+}
+
+void __init prom_free_prom_memory(void)
+{
+}
diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile
index d61138a..ff8a553 100644
--- a/arch/mips/pci/Makefile
+++ b/arch/mips/pci/Makefile
@@ -21,7 +21,7 @@
obj-$(CONFIG_MIPS_ALCHEMY) += pci-alchemy.o
obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o
obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o
-
+obj-$(CONFIG_MIPS_PCI_VIRTIO) += pci-virtio-guest.o
#
# These are still pretty much in the old state, watch, go blind.
#
diff --git a/arch/mips/pci/fixup-malta.c b/arch/mips/pci/fixup-malta.c
index 2f9e52a..40e920c 100644
--- a/arch/mips/pci/fixup-malta.c
+++ b/arch/mips/pci/fixup-malta.c
@@ -68,6 +68,7 @@
{
unsigned char reg_val;
u32 reg_val32;
+ u16 reg_val16;
/* PIIX PIRQC[A:D] irq mappings */
static int piixirqmap[PIIX4_FUNC0_PIRQRC_IRQ_ROUTING_MAX] = {
0, 0, 0, 3,
@@ -107,6 +108,11 @@
pci_read_config_byte(pdev, PIIX4_FUNC0_SERIRQC, ®_val);
reg_val |= PIIX4_FUNC0_SERIRQC_EN | PIIX4_FUNC0_SERIRQC_CONT;
pci_write_config_byte(pdev, PIIX4_FUNC0_SERIRQC, reg_val);
+
+ /* Enable response to special cycles */
+ pci_read_config_word(pdev, PCI_COMMAND, ®_val16);
+ pci_write_config_word(pdev, PCI_COMMAND,
+ reg_val16 | PCI_COMMAND_SPECIAL);
}
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82371AB_0,
diff --git a/arch/mips/pci/msi-octeon.c b/arch/mips/pci/msi-octeon.c
index 2b91b0e..ab0c5d1 100644
--- a/arch/mips/pci/msi-octeon.c
+++ b/arch/mips/pci/msi-octeon.c
@@ -15,6 +15,7 @@
#include <asm/octeon/cvmx-npi-defs.h>
#include <asm/octeon/cvmx-pci-defs.h>
#include <asm/octeon/cvmx-npei-defs.h>
+#include <asm/octeon/cvmx-sli-defs.h>
#include <asm/octeon/cvmx-pexp-defs.h>
#include <asm/octeon/pci-octeon.h>
@@ -162,6 +163,11 @@
msg.address_lo = (0 + CVMX_NPEI_PCIE_MSI_RCV) & 0xffffffff;
msg.address_hi = (0 + CVMX_NPEI_PCIE_MSI_RCV) >> 32;
break;
+ case OCTEON_DMA_BAR_TYPE_PCIE2:
+ /* When using PCIe2, Bar 0 is based at 0 */
+ msg.address_lo = (0 + CVMX_SLI_PCIE_MSI_RCV) & 0xffffffff;
+ msg.address_hi = (0 + CVMX_SLI_PCIE_MSI_RCV) >> 32;
+ break;
default:
panic("arch_setup_msi_irq: Invalid octeon_dma_bar_type");
}
diff --git a/arch/mips/pci/msi-xlp.c b/arch/mips/pci/msi-xlp.c
index 3249685..fa374fe 100644
--- a/arch/mips/pci/msi-xlp.c
+++ b/arch/mips/pci/msi-xlp.c
@@ -56,8 +56,8 @@
#include <asm/netlogic/xlp-hal/bridge.h>
#define XLP_MSIVEC_PER_LINK 32
-#define XLP_MSIXVEC_TOTAL 32
-#define XLP_MSIXVEC_PER_LINK 8
+#define XLP_MSIXVEC_TOTAL (cpu_is_xlp9xx() ? 128 : 32)
+#define XLP_MSIXVEC_PER_LINK (cpu_is_xlp9xx() ? 32 : 8)
/* 128 MSI irqs per node, mapped starting at NLM_MSI_VEC_BASE */
static inline int nlm_link_msiirq(int link, int msivec)
@@ -65,35 +65,44 @@
return NLM_MSI_VEC_BASE + link * XLP_MSIVEC_PER_LINK + msivec;
}
+/* get the link MSI vector from irq number */
static inline int nlm_irq_msivec(int irq)
{
- return irq % XLP_MSIVEC_PER_LINK;
+ return (irq - NLM_MSI_VEC_BASE) % XLP_MSIVEC_PER_LINK;
}
+/* get the link from the irq number */
static inline int nlm_irq_msilink(int irq)
{
- return (irq % (XLP_MSIVEC_PER_LINK * PCIE_NLINKS)) /
- XLP_MSIVEC_PER_LINK;
+ int total_msivec = XLP_MSIVEC_PER_LINK * PCIE_NLINKS;
+
+ return ((irq - NLM_MSI_VEC_BASE) % total_msivec) /
+ XLP_MSIVEC_PER_LINK;
}
/*
- * Only 32 MSI-X vectors are possible because there are only 32 PIC
- * interrupts for MSI. We split them statically and use 8 MSI-X vectors
- * per link - this keeps the allocation and lookup simple.
+ * For XLP 8xx/4xx/3xx/2xx, only 32 MSI-X vectors are possible because
+ * there are only 32 PIC interrupts for MSI. We split them statically
+ * and use 8 MSI-X vectors per link - this keeps the allocation and
+ * lookup simple.
+ * On XLP 9xx, there are 32 vectors per link, and the interrupts are
+ * not routed thru PIC, so we can use all 128 MSI-X vectors.
*/
static inline int nlm_link_msixirq(int link, int bit)
{
return NLM_MSIX_VEC_BASE + link * XLP_MSIXVEC_PER_LINK + bit;
}
+/* get the link MSI vector from irq number */
static inline int nlm_irq_msixvec(int irq)
{
- return irq % XLP_MSIXVEC_TOTAL; /* works when given xirq */
+ return (irq - NLM_MSIX_VEC_BASE) % XLP_MSIXVEC_TOTAL;
}
-static inline int nlm_irq_msixlink(int irq)
+/* get the link from MSIX vec */
+static inline int nlm_irq_msixlink(int msixvec)
{
- return nlm_irq_msixvec(irq) / XLP_MSIXVEC_PER_LINK;
+ return msixvec / XLP_MSIXVEC_PER_LINK;
}
/*
@@ -129,7 +138,11 @@
vec = nlm_irq_msivec(d->irq);
spin_lock_irqsave(&md->msi_lock, flags);
md->msi_enabled_mask |= 1u << vec;
- nlm_write_reg(md->lnkbase, PCIE_MSI_EN, md->msi_enabled_mask);
+ if (cpu_is_xlp9xx())
+ nlm_write_reg(md->lnkbase, PCIE_9XX_MSI_EN,
+ md->msi_enabled_mask);
+ else
+ nlm_write_reg(md->lnkbase, PCIE_MSI_EN, md->msi_enabled_mask);
spin_unlock_irqrestore(&md->msi_lock, flags);
}
@@ -142,7 +155,11 @@
vec = nlm_irq_msivec(d->irq);
spin_lock_irqsave(&md->msi_lock, flags);
md->msi_enabled_mask &= ~(1u << vec);
- nlm_write_reg(md->lnkbase, PCIE_MSI_EN, md->msi_enabled_mask);
+ if (cpu_is_xlp9xx())
+ nlm_write_reg(md->lnkbase, PCIE_9XX_MSI_EN,
+ md->msi_enabled_mask);
+ else
+ nlm_write_reg(md->lnkbase, PCIE_MSI_EN, md->msi_enabled_mask);
spin_unlock_irqrestore(&md->msi_lock, flags);
}
@@ -156,11 +173,18 @@
xlp_msi_disable(d);
/* Ack MSI on bridge */
- nlm_write_reg(md->lnkbase, PCIE_MSI_STATUS, 1u << vec);
+ if (cpu_is_xlp9xx())
+ nlm_write_reg(md->lnkbase, PCIE_9XX_MSI_STATUS, 1u << vec);
+ else
+ nlm_write_reg(md->lnkbase, PCIE_MSI_STATUS, 1u << vec);
/* Ack at eirr and PIC */
ack_c0_eirr(PIC_PCIE_LINK_MSI_IRQ(link));
- nlm_pic_ack(md->node->picbase, PIC_IRT_PCIE_LINK_INDEX(link));
+ if (cpu_is_xlp9xx())
+ nlm_pic_ack(md->node->picbase,
+ PIC_9XX_IRT_PCIE_LINK_INDEX(link));
+ else
+ nlm_pic_ack(md->node->picbase, PIC_IRT_PCIE_LINK_INDEX(link));
}
static struct irq_chip xlp_msi_chip = {
@@ -172,30 +196,45 @@
};
/*
- * The MSI-X interrupt handling is different from MSI, there are 32
- * MSI-X interrupts generated by the PIC and each of these correspond
- * to a MSI-X vector (0-31) that can be assigned.
+ * XLP8XX/4XX/3XX/2XX:
+ * The MSI-X interrupt handling is different from MSI, there are 32 MSI-X
+ * interrupts generated by the PIC and each of these correspond to a MSI-X
+ * vector (0-31) that can be assigned.
*
- * We divide the MSI-X vectors to 8 per link and do a per-link
- * allocation
+ * We divide the MSI-X vectors to 8 per link and do a per-link allocation
+ *
+ * XLP9XX:
+ * 32 MSI-X vectors are available per link, and the interrupts are not routed
+ * thru the PIC. PIC ack not needed.
*
* Enable and disable done using standard MSI functions.
*/
static void xlp_msix_mask_ack(struct irq_data *d)
{
- struct xlp_msi_data *md = irq_data_get_irq_handler_data(d);
+ struct xlp_msi_data *md;
int link, msixvec;
+ uint32_t status_reg, bit;
msixvec = nlm_irq_msixvec(d->irq);
- link = nlm_irq_msixlink(d->irq);
+ link = nlm_irq_msixlink(msixvec);
mask_msi_irq(d);
+ md = irq_data_get_irq_handler_data(d);
/* Ack MSI on bridge */
- nlm_write_reg(md->lnkbase, PCIE_MSIX_STATUS, 1u << msixvec);
+ if (cpu_is_xlp9xx()) {
+ status_reg = PCIE_9XX_MSIX_STATUSX(link);
+ bit = msixvec % XLP_MSIXVEC_PER_LINK;
+ } else {
+ status_reg = PCIE_MSIX_STATUS;
+ bit = msixvec;
+ }
+ nlm_write_reg(md->lnkbase, status_reg, 1u << bit);
/* Ack at eirr and PIC */
ack_c0_eirr(PIC_PCIE_MSIX_IRQ(link));
- nlm_pic_ack(md->node->picbase, PIC_IRT_PCIE_MSIX_INDEX(msixvec));
+ if (!cpu_is_xlp9xx())
+ nlm_pic_ack(md->node->picbase,
+ PIC_IRT_PCIE_MSIX_INDEX(msixvec));
}
static struct irq_chip xlp_msix_chip = {
@@ -219,10 +258,18 @@
{
u32 val;
- val = nlm_read_reg(lnkbase, PCIE_INT_EN0);
- if ((val & 0x200) == 0) {
- val |= 0x200; /* MSI Interrupt enable */
- nlm_write_reg(lnkbase, PCIE_INT_EN0, val);
+ if (cpu_is_xlp9xx()) {
+ val = nlm_read_reg(lnkbase, PCIE_9XX_INT_EN0);
+ if ((val & 0x200) == 0) {
+ val |= 0x200; /* MSI Interrupt enable */
+ nlm_write_reg(lnkbase, PCIE_9XX_INT_EN0, val);
+ }
+ } else {
+ val = nlm_read_reg(lnkbase, PCIE_INT_EN0);
+ if ((val & 0x200) == 0) {
+ val |= 0x200;
+ nlm_write_reg(lnkbase, PCIE_INT_EN0, val);
+ }
}
val = nlm_read_reg(lnkbase, 0x1); /* CMD */
@@ -269,9 +316,12 @@
spin_lock_irqsave(&md->msi_lock, flags);
if (md->msi_alloc_mask == 0) {
- /* switch the link IRQ to MSI range */
xlp_config_link_msi(lnkbase, lirq, msiaddr);
- irt = PIC_IRT_PCIE_LINK_INDEX(link);
+ /* switch the link IRQ to MSI range */
+ if (cpu_is_xlp9xx())
+ irt = PIC_9XX_IRT_PCIE_LINK_INDEX(link);
+ else
+ irt = PIC_IRT_PCIE_LINK_INDEX(link);
nlm_setup_pic_irq(node, lirq, lirq, irt);
nlm_pic_init_irt(nlm_get_node(node)->picbase, irt, lirq,
node * nlm_threads_per_node(), 1 /*en */);
@@ -311,10 +361,19 @@
val |= 0x80000000U;
nlm_write_reg(lnkbase, 0x2C, val);
}
- val = nlm_read_reg(lnkbase, PCIE_INT_EN0);
- if ((val & 0x200) == 0) {
- val |= 0x200; /* MSI Interrupt enable */
- nlm_write_reg(lnkbase, PCIE_INT_EN0, val);
+
+ if (cpu_is_xlp9xx()) {
+ val = nlm_read_reg(lnkbase, PCIE_9XX_INT_EN0);
+ if ((val & 0x200) == 0) {
+ val |= 0x200; /* MSI Interrupt enable */
+ nlm_write_reg(lnkbase, PCIE_9XX_INT_EN0, val);
+ }
+ } else {
+ val = nlm_read_reg(lnkbase, PCIE_INT_EN0);
+ if ((val & 0x200) == 0) {
+ val |= 0x200; /* MSI Interrupt enable */
+ nlm_write_reg(lnkbase, PCIE_INT_EN0, val);
+ }
}
val = nlm_read_reg(lnkbase, 0x1); /* CMD */
@@ -329,10 +388,19 @@
val |= (1 << 8) | lirq;
nlm_write_pci_reg(lnkbase, 0xf, val);
- /* MSI-X addresses */
- nlm_write_reg(lnkbase, PCIE_BRIDGE_MSIX_ADDR_BASE, msixaddr >> 8);
- nlm_write_reg(lnkbase, PCIE_BRIDGE_MSIX_ADDR_LIMIT,
- (msixaddr + MSI_ADDR_SZ) >> 8);
+ if (cpu_is_xlp9xx()) {
+ /* MSI-X addresses */
+ nlm_write_reg(lnkbase, PCIE_9XX_BRIDGE_MSIX_ADDR_BASE,
+ msixaddr >> 8);
+ nlm_write_reg(lnkbase, PCIE_9XX_BRIDGE_MSIX_ADDR_LIMIT,
+ (msixaddr + MSI_ADDR_SZ) >> 8);
+ } else {
+ /* MSI-X addresses */
+ nlm_write_reg(lnkbase, PCIE_BRIDGE_MSIX_ADDR_BASE,
+ msixaddr >> 8);
+ nlm_write_reg(lnkbase, PCIE_BRIDGE_MSIX_ADDR_LIMIT,
+ (msixaddr + MSI_ADDR_SZ) >> 8);
+ }
}
/*
@@ -369,6 +437,7 @@
xirq += t;
msixvec = nlm_irq_msixvec(xirq);
+
msg.address_hi = msixaddr >> 32;
msg.address_lo = msixaddr & 0xffffffff;
msg.data = 0xc00 | msixvec;
@@ -409,7 +478,7 @@
{
struct nlm_soc_info *nodep;
struct xlp_msi_data *md;
- int irq, i, irt, msixvec;
+ int irq, i, irt, msixvec, val;
pr_info("[%d %d] Init node PCI IRT\n", node, link);
nodep = nlm_get_node(node);
@@ -430,19 +499,28 @@
irq_set_handler_data(i, md);
}
- for (i = 0; i < XLP_MSIXVEC_PER_LINK; i++) {
- /* Initialize MSI-X irts to generate one interrupt per link */
- msixvec = link * XLP_MSIXVEC_PER_LINK + i;
- irt = PIC_IRT_PCIE_MSIX_INDEX(msixvec);
- nlm_pic_init_irt(nodep->picbase, irt, PIC_PCIE_MSIX_IRQ(link),
- node * nlm_threads_per_node(), 1 /* enable */);
+ for (i = 0; i < XLP_MSIXVEC_PER_LINK ; i++) {
+ if (cpu_is_xlp9xx()) {
+ val = ((node * nlm_threads_per_node()) << 7 |
+ PIC_PCIE_MSIX_IRQ(link) << 1 | 0 << 0);
+ nlm_write_pcie_reg(md->lnkbase, PCIE_9XX_MSIX_VECX(i +
+ (link * XLP_MSIXVEC_PER_LINK)), val);
+ } else {
+ /* Initialize MSI-X irts to generate one interrupt
+ * per link
+ */
+ msixvec = link * XLP_MSIXVEC_PER_LINK + i;
+ irt = PIC_IRT_PCIE_MSIX_INDEX(msixvec);
+ nlm_pic_init_irt(nodep->picbase, irt,
+ PIC_PCIE_MSIX_IRQ(link),
+ node * nlm_threads_per_node(), 1);
+ }
/* Initialize MSI-X extended irq space for the link */
irq = nlm_irq_to_xirq(node, nlm_link_msixirq(link, i));
irq_set_chip_and_handler(irq, &xlp_msix_chip, handle_level_irq);
irq_set_handler_data(irq, md);
}
-
}
void nlm_dispatch_msi(int node, int lirq)
@@ -454,7 +532,11 @@
link = lirq - PIC_PCIE_LINK_MSI_IRQ_BASE;
irqbase = nlm_irq_to_xirq(node, nlm_link_msiirq(link, 0));
md = irq_get_handler_data(irqbase);
- status = nlm_read_reg(md->lnkbase, PCIE_MSI_STATUS) &
+ if (cpu_is_xlp9xx())
+ status = nlm_read_reg(md->lnkbase, PCIE_9XX_MSI_STATUS) &
+ md->msi_enabled_mask;
+ else
+ status = nlm_read_reg(md->lnkbase, PCIE_MSI_STATUS) &
md->msi_enabled_mask;
while (status) {
i = __ffs(status);
@@ -472,10 +554,14 @@
link = lirq - PIC_PCIE_MSIX_IRQ_BASE;
irqbase = nlm_irq_to_xirq(node, nlm_link_msixirq(link, 0));
md = irq_get_handler_data(irqbase);
- status = nlm_read_reg(md->lnkbase, PCIE_MSIX_STATUS);
+ if (cpu_is_xlp9xx())
+ status = nlm_read_reg(md->lnkbase, PCIE_9XX_MSIX_STATUSX(link));
+ else
+ status = nlm_read_reg(md->lnkbase, PCIE_MSIX_STATUS);
/* narrow it down to the MSI-x vectors for our link */
- status = (status >> (link * XLP_MSIXVEC_PER_LINK)) &
+ if (!cpu_is_xlp9xx())
+ status = (status >> (link * XLP_MSIXVEC_PER_LINK)) &
((1 << XLP_MSIXVEC_PER_LINK) - 1);
while (status) {
diff --git a/arch/mips/pci/ops-pmcmsp.c b/arch/mips/pci/ops-pmcmsp.c
index 3d27800..50034f9 100644
--- a/arch/mips/pci/ops-pmcmsp.c
+++ b/arch/mips/pci/ops-pmcmsp.c
@@ -7,7 +7,7 @@
* Author: Jun Sun, jsun@mvista.com or jsun@junsun.net
*
* Much of the code is derived from the original DDB5074 port by
- * Geert Uytterhoeven <geert@sonycom.com>
+ * Geert Uytterhoeven <geert@linux-m68k.org>
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/arch/mips/pci/ops-tx3927.c b/arch/mips/pci/ops-tx3927.c
index 02d64f77..d35dc9c 100644
--- a/arch/mips/pci/ops-tx3927.c
+++ b/arch/mips/pci/ops-tx3927.c
@@ -11,7 +11,7 @@
* Define the pci_ops for TX3927.
*
* Much of the code is derived from the original DDB5074 port by
- * Geert Uytterhoeven <geert@sonycom.com>
+ * Geert Uytterhoeven <geert@linux-m68k.org>
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/arch/mips/pci/ops-tx4927.c b/arch/mips/pci/ops-tx4927.c
index 3d5df51..0e046d8 100644
--- a/arch/mips/pci/ops-tx4927.c
+++ b/arch/mips/pci/ops-tx4927.c
@@ -202,17 +202,20 @@
unsigned long val;
if (!strncmp(str, "trdyto=", 7)) {
- if (strict_strtoul(str + 7, 0, &val) == 0)
+ u8 val = 0;
+ if (kstrtou8(str + 7, 0, &val) == 0)
tx4927_pci_opts.trdyto = val;
return NULL;
}
if (!strncmp(str, "retryto=", 8)) {
- if (strict_strtoul(str + 8, 0, &val) == 0)
+ u8 val = 0;
+ if (kstrtou8(str + 8, 0, &val) == 0)
tx4927_pci_opts.retryto = val;
return NULL;
}
if (!strncmp(str, "gbwc=", 5)) {
- if (strict_strtoul(str + 5, 0, &val) == 0)
+ u16 val;
+ if (kstrtou16(str + 5, 0, &val) == 0)
tx4927_pci_opts.gbwc = val;
return NULL;
}
diff --git a/arch/mips/pci/pci-virtio-guest.c b/arch/mips/pci/pci-virtio-guest.c
new file mode 100644
index 0000000..40a078b
--- /dev/null
+++ b/arch/mips/pci/pci-virtio-guest.c
@@ -0,0 +1,131 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2013 Cavium, Inc.
+ */
+
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/pci.h>
+
+#include <uapi/asm/bitfield.h>
+#include <asm/byteorder.h>
+#include <asm/io.h>
+
+#define PCI_CONFIG_ADDRESS 0xcf8
+#define PCI_CONFIG_DATA 0xcfc
+
+union pci_config_address {
+ struct {
+ __BITFIELD_FIELD(unsigned enable_bit : 1, /* 31 */
+ __BITFIELD_FIELD(unsigned reserved : 7, /* 30 .. 24 */
+ __BITFIELD_FIELD(unsigned bus_number : 8, /* 23 .. 16 */
+ __BITFIELD_FIELD(unsigned devfn_number : 8, /* 15 .. 8 */
+ __BITFIELD_FIELD(unsigned register_number : 8, /* 7 .. 0 */
+ )))));
+ };
+ u32 w;
+};
+
+int pcibios_plat_dev_init(struct pci_dev *dev)
+{
+ return 0;
+}
+
+int pcibios_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)
+{
+ return ((pin + slot) % 4)+ MIPS_IRQ_PCIA;
+}
+
+static void pci_virtio_guest_write_config_addr(struct pci_bus *bus,
+ unsigned int devfn, int reg)
+{
+ union pci_config_address pca = { .w = 0 };
+
+ pca.register_number = reg;
+ pca.devfn_number = devfn;
+ pca.bus_number = bus->number;
+ pca.enable_bit = 1;
+
+ outl(pca.w, PCI_CONFIG_ADDRESS);
+}
+
+static int pci_virtio_guest_write_config(struct pci_bus *bus,
+ unsigned int devfn, int reg, int size, u32 val)
+{
+ pci_virtio_guest_write_config_addr(bus, devfn, reg);
+
+ switch (size) {
+ case 1:
+ outb(val, PCI_CONFIG_DATA + (reg & 3));
+ break;
+ case 2:
+ outw(val, PCI_CONFIG_DATA + (reg & 2));
+ break;
+ case 4:
+ outl(val, PCI_CONFIG_DATA);
+ break;
+ }
+
+ return PCIBIOS_SUCCESSFUL;
+}
+
+static int pci_virtio_guest_read_config(struct pci_bus *bus, unsigned int devfn,
+ int reg, int size, u32 *val)
+{
+ pci_virtio_guest_write_config_addr(bus, devfn, reg);
+
+ switch (size) {
+ case 1:
+ *val = inb(PCI_CONFIG_DATA + (reg & 3));
+ break;
+ case 2:
+ *val = inw(PCI_CONFIG_DATA + (reg & 2));
+ break;
+ case 4:
+ *val = inl(PCI_CONFIG_DATA);
+ break;
+ }
+ return PCIBIOS_SUCCESSFUL;
+}
+
+static struct pci_ops pci_virtio_guest_ops = {
+ .read = pci_virtio_guest_read_config,
+ .write = pci_virtio_guest_write_config,
+};
+
+static struct resource pci_virtio_guest_mem_resource = {
+ .name = "Virtio MEM",
+ .flags = IORESOURCE_MEM,
+ .start = 0x10000000,
+ .end = 0x1dffffff
+};
+
+static struct resource pci_virtio_guest_io_resource = {
+ .name = "Virtio IO",
+ .flags = IORESOURCE_IO,
+ .start = 0,
+ .end = 0xffff
+};
+
+static struct pci_controller pci_virtio_guest_controller = {
+ .pci_ops = &pci_virtio_guest_ops,
+ .mem_resource = &pci_virtio_guest_mem_resource,
+ .io_resource = &pci_virtio_guest_io_resource,
+};
+
+static int __init pci_virtio_guest_setup(void)
+{
+ pr_err("pci_virtio_guest_setup\n");
+
+ /* Virtio comes pre-assigned */
+ pci_set_flags(PCI_PROBE_ONLY);
+
+ pci_virtio_guest_controller.io_map_base = mips_io_port_base;
+ register_pci_controller(&pci_virtio_guest_controller);
+ return 0;
+}
+arch_initcall(pci_virtio_guest_setup);
diff --git a/arch/mips/pmcs-msp71xx/Makefile b/arch/mips/pmcs-msp71xx/Makefile
index 9201c8b..d4f7220 100644
--- a/arch/mips/pmcs-msp71xx/Makefile
+++ b/arch/mips/pmcs-msp71xx/Makefile
@@ -10,4 +10,3 @@
obj-$(CONFIG_MSP_HAS_MAC) += msp_eth.o
obj-$(CONFIG_MSP_HAS_USB) += msp_usb.o
obj-$(CONFIG_MIPS_MT_SMP) += msp_smp.o
-obj-$(CONFIG_MIPS_MT_SMTC) += msp_smtc.o
diff --git a/arch/mips/pmcs-msp71xx/msp_eth.c b/arch/mips/pmcs-msp71xx/msp_eth.c
index c584df3..15679b4 100644
--- a/arch/mips/pmcs-msp71xx/msp_eth.c
+++ b/arch/mips/pmcs-msp71xx/msp_eth.c
@@ -38,73 +38,6 @@
#define MSP_ETHERNET_GPIO1 15
#define MSP_ETHERNET_GPIO2 16
-#ifdef CONFIG_MSP_HAS_TSMAC
-#define MSP_TSMAC_SIZE 0x10020
-#define MSP_TSMAC_ID "pmc_tsmac"
-
-static struct resource msp_tsmac0_resources[] = {
- [0] = {
- .start = MSP_MAC0_BASE,
- .end = MSP_MAC0_BASE + MSP_TSMAC_SIZE - 1,
- .flags = IORESOURCE_MEM,
- },
- [1] = {
- .start = MSP_INT_MAC0,
- .end = MSP_INT_MAC0,
- .flags = IORESOURCE_IRQ,
- },
-};
-
-static struct resource msp_tsmac1_resources[] = {
- [0] = {
- .start = MSP_MAC1_BASE,
- .end = MSP_MAC1_BASE + MSP_TSMAC_SIZE - 1,
- .flags = IORESOURCE_MEM,
- },
- [1] = {
- .start = MSP_INT_MAC1,
- .end = MSP_INT_MAC1,
- .flags = IORESOURCE_IRQ,
- },
-};
-static struct resource msp_tsmac2_resources[] = {
- [0] = {
- .start = MSP_MAC2_BASE,
- .end = MSP_MAC2_BASE + MSP_TSMAC_SIZE - 1,
- .flags = IORESOURCE_MEM,
- },
- [1] = {
- .start = MSP_INT_SAR,
- .end = MSP_INT_SAR,
- .flags = IORESOURCE_IRQ,
- },
-};
-
-
-static struct platform_device tsmac_device[] = {
- [0] = {
- .name = MSP_TSMAC_ID,
- .id = 0,
- .num_resources = ARRAY_SIZE(msp_tsmac0_resources),
- .resource = msp_tsmac0_resources,
- },
- [1] = {
- .name = MSP_TSMAC_ID,
- .id = 1,
- .num_resources = ARRAY_SIZE(msp_tsmac1_resources),
- .resource = msp_tsmac1_resources,
- },
- [2] = {
- .name = MSP_TSMAC_ID,
- .id = 2,
- .num_resources = ARRAY_SIZE(msp_tsmac2_resources),
- .resource = msp_tsmac2_resources,
- },
-};
-#define msp_eth_devs tsmac_device
-
-#else
-/* If it is not TSMAC assume MSP_ETH (100Mbps) */
#define MSP_ETH_ID "pmc_mspeth"
#define MSP_ETH_SIZE 0xE0
static struct resource msp_eth0_resources[] = {
@@ -152,7 +85,6 @@
};
#define msp_eth_devs mspeth_device
-#endif
int __init msp_eth_setup(void)
{
int i, ret = 0;
@@ -161,14 +93,6 @@
msp_gpio_pin_mode(MSP_GPIO_OUTPUT, MSP_ETHERNET_GPIO0);
msp_gpio_pin_hi(MSP_ETHERNET_GPIO0);
-#ifdef CONFIG_MSP_HAS_TSMAC
- /* 3 phys on boards with TSMAC */
- msp_gpio_pin_mode(MSP_GPIO_OUTPUT, MSP_ETHERNET_GPIO1);
- msp_gpio_pin_hi(MSP_ETHERNET_GPIO1);
-
- msp_gpio_pin_mode(MSP_GPIO_OUTPUT, MSP_ETHERNET_GPIO2);
- msp_gpio_pin_hi(MSP_ETHERNET_GPIO2);
-#endif
for (i = 0; i < ARRAY_SIZE(msp_eth_devs); i++) {
ret = platform_device_register(&msp_eth_devs[i]);
printk(KERN_INFO "device: %d, return value = %d\n", i, ret);
diff --git a/arch/mips/pmcs-msp71xx/msp_irq.c b/arch/mips/pmcs-msp71xx/msp_irq.c
index 9da5619..941744a 100644
--- a/arch/mips/pmcs-msp71xx/msp_irq.c
+++ b/arch/mips/pmcs-msp71xx/msp_irq.c
@@ -32,7 +32,7 @@
/* vectored interrupt implementation */
-/* SW0/1 interrupts are used for SMP/SMTC */
+/* SW0/1 interrupts are used for SMP */
static inline void mac0_int_dispatch(void) { do_IRQ(MSP_INT_MAC0); }
static inline void mac1_int_dispatch(void) { do_IRQ(MSP_INT_MAC1); }
static inline void mac2_int_dispatch(void) { do_IRQ(MSP_INT_SAR); }
@@ -138,14 +138,6 @@
set_vi_handler(MSP_INT_SEC, sec_int_dispatch);
#ifdef CONFIG_MIPS_MT_SMP
msp_vsmp_int_init();
-#elif defined CONFIG_MIPS_MT_SMTC
- /*Set hwmask for all platform devices */
- irq_hwmask[MSP_INT_MAC0] = C_IRQ0;
- irq_hwmask[MSP_INT_MAC1] = C_IRQ1;
- irq_hwmask[MSP_INT_USB] = C_IRQ2;
- irq_hwmask[MSP_INT_SAR] = C_IRQ3;
- irq_hwmask[MSP_INT_SEC] = C_IRQ5;
-
#endif /* CONFIG_MIPS_MT_SMP */
#endif /* CONFIG_MIPS_MT */
/* setup the cascaded interrupts */
@@ -153,8 +145,10 @@
setup_irq(MSP_INT_PER, &per_cascade_msp);
#else
- /* setup the 2nd-level SLP register based interrupt controller */
- /* VSMP /SMTC support support is not enabled for SLP */
+ /*
+ * Setup the 2nd-level SLP register based interrupt controller.
+ * VSMP support support is not enabled for SLP.
+ */
msp_slp_irq_init();
/* setup the cascaded SLP/PER interrupts */
diff --git a/arch/mips/pmcs-msp71xx/msp_irq_cic.c b/arch/mips/pmcs-msp71xx/msp_irq_cic.c
index e49b499..b8df2f7 100644
--- a/arch/mips/pmcs-msp71xx/msp_irq_cic.c
+++ b/arch/mips/pmcs-msp71xx/msp_irq_cic.c
@@ -120,10 +120,9 @@
* hurt for the others
*/
*CIC_STS_REG = (1 << (d->irq - MSP_CIC_INTBASE));
- smtc_im_ack_irq(d->irq);
}
-/*Note: Limiting to VSMP . Not tested in SMTC */
+/* Note: Limiting to VSMP. */
#ifdef CONFIG_MIPS_MT_SMP
static int msp_cic_irq_set_affinity(struct irq_data *d,
@@ -183,10 +182,6 @@
for (i = MSP_CIC_INTBASE ; i < MSP_CIC_INTBASE + 32 ; i++) {
irq_set_chip_and_handler(i, &msp_cic_irq_controller,
handle_level_irq);
-#ifdef CONFIG_MIPS_MT_SMTC
- /* Mask of CIC interrupt */
- irq_hwmask[i] = C_IRQ4;
-#endif
}
/* Initialize the PER interrupt sub-system */
diff --git a/arch/mips/pmcs-msp71xx/msp_irq_per.c b/arch/mips/pmcs-msp71xx/msp_irq_per.c
index d1fd530..a111836 100644
--- a/arch/mips/pmcs-msp71xx/msp_irq_per.c
+++ b/arch/mips/pmcs-msp71xx/msp_irq_per.c
@@ -113,9 +113,6 @@
/* initialize all the IRQ descriptors */
for (i = MSP_PER_INTBASE; i < MSP_PER_INTBASE + 32; i++) {
irq_set_chip(i, &msp_per_irq_controller);
-#ifdef CONFIG_MIPS_MT_SMTC
- irq_hwmask[i] = C_IRQ4;
-#endif
}
}
diff --git a/arch/mips/pmcs-msp71xx/msp_setup.c b/arch/mips/pmcs-msp71xx/msp_setup.c
index 7e98076..4f925e0 100644
--- a/arch/mips/pmcs-msp71xx/msp_setup.c
+++ b/arch/mips/pmcs-msp71xx/msp_setup.c
@@ -27,7 +27,6 @@
#endif
extern void msp_serial_setup(void);
-extern void pmctwiled_setup(void);
#if defined(CONFIG_PMC_MSP7120_EVAL) || \
defined(CONFIG_PMC_MSP7120_GW) || \
@@ -148,8 +147,6 @@
pm_power_off = msp_power_off;
}
-extern struct plat_smp_ops msp_smtc_smp_ops;
-
void __init prom_init(void)
{
unsigned long family;
@@ -230,17 +227,5 @@
*/
msp_serial_setup();
- if (register_vsmp_smp_ops()) {
-#ifdef CONFIG_MIPS_MT_SMTC
- register_smp_ops(&msp_smtc_smp_ops);
-#endif
- }
-
-#ifdef CONFIG_PMCTWILED
- /*
- * Setup LED states before the subsys_initcall loads other
- * dependent drivers/modules.
- */
- pmctwiled_setup();
-#endif
+ register_vsmp_smp_ops();
}
diff --git a/arch/mips/pmcs-msp71xx/msp_smtc.c b/arch/mips/pmcs-msp71xx/msp_smtc.c
deleted file mode 100644
index 6b5607f..0000000
--- a/arch/mips/pmcs-msp71xx/msp_smtc.c
+++ /dev/null
@@ -1,104 +0,0 @@
-/*
- * MSP71xx Platform-specific hooks for SMP operation
- */
-#include <linux/irq.h>
-#include <linux/init.h>
-
-#include <asm/mipsmtregs.h>
-#include <asm/mipsregs.h>
-#include <asm/smtc.h>
-#include <asm/smtc_ipi.h>
-
-/* VPE/SMP Prototype implements platform interfaces directly */
-
-/*
- * Cause the specified action to be performed on a targeted "CPU"
- */
-
-static void msp_smtc_send_ipi_single(int cpu, unsigned int action)
-{
- /* "CPU" may be TC of same VPE, VPE of same CPU, or different CPU */
- smtc_send_ipi(cpu, LINUX_SMP_IPI, action);
-}
-
-static void msp_smtc_send_ipi_mask(const struct cpumask *mask,
- unsigned int action)
-{
- unsigned int i;
-
- for_each_cpu(i, mask)
- msp_smtc_send_ipi_single(i, action);
-}
-
-/*
- * Post-config but pre-boot cleanup entry point
- */
-static void msp_smtc_init_secondary(void)
-{
- int myvpe;
-
- /* Don't enable Malta I/O interrupts (IP2) for secondary VPEs */
- myvpe = read_c0_tcbind() & TCBIND_CURVPE;
- if (myvpe > 0)
- change_c0_status(ST0_IM, STATUSF_IP0 | STATUSF_IP1 |
- STATUSF_IP6 | STATUSF_IP7);
- smtc_init_secondary();
-}
-
-/*
- * Platform "CPU" startup hook
- */
-static void msp_smtc_boot_secondary(int cpu, struct task_struct *idle)
-{
- smtc_boot_secondary(cpu, idle);
-}
-
-/*
- * SMP initialization finalization entry point
- */
-static void msp_smtc_smp_finish(void)
-{
- smtc_smp_finish();
-}
-
-/*
- * Hook for after all CPUs are online
- */
-
-static void msp_smtc_cpus_done(void)
-{
-}
-
-/*
- * Platform SMP pre-initialization
- *
- * As noted above, we can assume a single CPU for now
- * but it may be multithreaded.
- */
-
-static void __init msp_smtc_smp_setup(void)
-{
- /*
- * we won't get the definitive value until
- * we've run smtc_prepare_cpus later, but
- */
-
- if (read_c0_config3() & (1 << 2))
- smp_num_siblings = smtc_build_cpu_map(0);
-}
-
-static void __init msp_smtc_prepare_cpus(unsigned int max_cpus)
-{
- smtc_prepare_cpus(max_cpus);
-}
-
-struct plat_smp_ops msp_smtc_smp_ops = {
- .send_ipi_single = msp_smtc_send_ipi_single,
- .send_ipi_mask = msp_smtc_send_ipi_mask,
- .init_secondary = msp_smtc_init_secondary,
- .smp_finish = msp_smtc_smp_finish,
- .cpus_done = msp_smtc_cpus_done,
- .boot_secondary = msp_smtc_boot_secondary,
- .smp_setup = msp_smtc_smp_setup,
- .prepare_cpus = msp_smtc_prepare_cpus,
-};
diff --git a/arch/mips/pmcs-msp71xx/msp_usb.c b/arch/mips/pmcs-msp71xx/msp_usb.c
index 4dab915..c87c5f8 100644
--- a/arch/mips/pmcs-msp71xx/msp_usb.c
+++ b/arch/mips/pmcs-msp71xx/msp_usb.c
@@ -75,47 +75,6 @@
.resource = msp_usbhost0_resources,
},
};
-
-/* MSP7140/MSP82XX has two USB2 hosts. */
-#ifdef CONFIG_MSP_HAS_DUAL_USB
-static u64 msp_usbhost1_dma_mask = 0xffffffffUL;
-
-static struct resource msp_usbhost1_resources[] = {
- [0] = { /* EHCI-HS operational and capabilities registers */
- .start = MSP_USB1_HS_START,
- .end = MSP_USB1_HS_END,
- .flags = IORESOURCE_MEM,
- },
- [1] = {
- .start = MSP_INT_USB,
- .end = MSP_INT_USB,
- .flags = IORESOURCE_IRQ,
- },
- [2] = { /* MSBus-to-AMBA bridge register space */
- .start = MSP_USB1_MAB_START,
- .end = MSP_USB1_MAB_END,
- .flags = IORESOURCE_MEM,
- },
- [3] = { /* Identification and general hardware parameters */
- .start = MSP_USB1_ID_START,
- .end = MSP_USB1_ID_END,
- .flags = IORESOURCE_MEM,
- },
-};
-
-static struct mspusb_device msp_usbhost1_device = {
- .dev = {
- .name = "pmcmsp-ehci",
- .id = 1,
- .dev = {
- .dma_mask = &msp_usbhost1_dma_mask,
- .coherent_dma_mask = 0xffffffffUL,
- },
- .num_resources = ARRAY_SIZE(msp_usbhost1_resources),
- .resource = msp_usbhost1_resources,
- },
-};
-#endif /* CONFIG_MSP_HAS_DUAL_USB */
#endif /* CONFIG_USB_EHCI_HCD */
#if defined(CONFIG_USB_GADGET)
@@ -157,46 +116,6 @@
.resource = msp_usbdev0_resources,
},
};
-
-#ifdef CONFIG_MSP_HAS_DUAL_USB
-static struct resource msp_usbdev1_resources[] = {
- [0] = { /* EHCI-HS operational and capabilities registers */
- .start = MSP_USB1_HS_START,
- .end = MSP_USB1_HS_END,
- .flags = IORESOURCE_MEM,
- },
- [1] = {
- .start = MSP_INT_USB,
- .end = MSP_INT_USB,
- .flags = IORESOURCE_IRQ,
- },
- [2] = { /* MSBus-to-AMBA bridge register space */
- .start = MSP_USB1_MAB_START,
- .end = MSP_USB1_MAB_END,
- .flags = IORESOURCE_MEM,
- },
- [3] = { /* Identification and general hardware parameters */
- .start = MSP_USB1_ID_START,
- .end = MSP_USB1_ID_END,
- .flags = IORESOURCE_MEM,
- },
-};
-
-/* This may need to be converted to a mspusb_device, too. */
-static struct mspusb_device msp_usbdev1_device = {
- .dev = {
- .name = "msp71xx_udc",
- .id = 0,
- .dev = {
- .dma_mask = &msp_usbdev_dma_mask,
- .coherent_dma_mask = 0xffffffffUL,
- },
- .num_resources = ARRAY_SIZE(msp_usbdev1_resources),
- .resource = msp_usbdev1_resources,
- },
-};
-
-#endif /* CONFIG_MSP_HAS_DUAL_USB */
#endif /* CONFIG_USB_GADGET */
static int __init msp_usb_setup(void)
@@ -231,10 +150,6 @@
#if defined(CONFIG_USB_EHCI_HCD)
msp_devs[0] = &msp_usbhost0_device.dev;
ppfinit("platform add USB HOST done %s.\n", msp_devs[0]->name);
-#ifdef CONFIG_MSP_HAS_DUAL_USB
- msp_devs[1] = &msp_usbhost1_device.dev;
- ppfinit("platform add USB HOST done %s.\n", msp_devs[1]->name);
-#endif
#else
ppfinit("%s: echi_hcd not supported\n", __FILE__);
#endif /* CONFIG_USB_EHCI_HCD */
@@ -244,11 +159,6 @@
msp_devs[0] = &msp_usbdev0_device.dev;
ppfinit("platform add USB DEVICE done %s.\n"
, msp_devs[0]->name);
-#ifdef CONFIG_MSP_HAS_DUAL_USB
- msp_devs[1] = &msp_usbdev1_device.dev;
- ppfinit("platform add USB DEVICE done %s.\n"
- , msp_devs[1]->name);
-#endif
#else
ppfinit("%s: usb_gadget not supported\n", __FILE__);
#endif /* CONFIG_USB_GADGET */
diff --git a/arch/mips/pnx833x/common/platform.c b/arch/mips/pnx833x/common/platform.c
index 2b7e837..b4b774b 100644
--- a/arch/mips/pnx833x/common/platform.c
+++ b/arch/mips/pnx833x/common/platform.c
@@ -33,11 +33,6 @@
#include <linux/mtd/nand.h>
#include <linux/mtd/partitions.h>
-#ifdef CONFIG_I2C_PNX0105
-/* Until i2c driver available in kernel.*/
-#include <linux/i2c-pnx0105.h>
-#endif
-
#include <irq.h>
#include <irq-mapping.h>
#include <pnx833x.h>
@@ -134,70 +129,6 @@
.resource = pnx833x_usb_ehci_resources,
};
-#ifdef CONFIG_I2C_PNX0105
-static struct resource pnx833x_i2c0_resources[] = {
- {
- .start = PNX833X_I2C0_PORTS_START,
- .end = PNX833X_I2C0_PORTS_END,
- .flags = IORESOURCE_MEM,
- },
- {
- .start = PNX833X_PIC_I2C0_INT,
- .end = PNX833X_PIC_I2C0_INT,
- .flags = IORESOURCE_IRQ,
- },
-};
-
-static struct resource pnx833x_i2c1_resources[] = {
- {
- .start = PNX833X_I2C1_PORTS_START,
- .end = PNX833X_I2C1_PORTS_END,
- .flags = IORESOURCE_MEM,
- },
- {
- .start = PNX833X_PIC_I2C1_INT,
- .end = PNX833X_PIC_I2C1_INT,
- .flags = IORESOURCE_IRQ,
- },
-};
-
-static struct i2c_pnx0105_dev pnx833x_i2c_dev[] = {
- {
- .base = PNX833X_I2C0_PORTS_START,
- .irq = -1, /* should be PNX833X_PIC_I2C0_INT but polling is faster */
- .clock = 6, /* 0 == 400 kHz, 4 == 100 kHz(Maximum HDMI), 6 = 50kHz(Preferred HDCP) */
- .bus_addr = 0, /* no slave support */
- },
- {
- .base = PNX833X_I2C1_PORTS_START,
- .irq = -1, /* on high freq, polling is faster */
- /*.irq = PNX833X_PIC_I2C1_INT,*/
- .clock = 4, /* 0 == 400 kHz, 4 == 100 kHz. 100 kHz seems a safe default for now */
- .bus_addr = 0, /* no slave support */
- },
-};
-
-static struct platform_device pnx833x_i2c0_device = {
- .name = "i2c-pnx0105",
- .id = 0,
- .dev = {
- .platform_data = &pnx833x_i2c_dev[0],
- },
- .num_resources = ARRAY_SIZE(pnx833x_i2c0_resources),
- .resource = pnx833x_i2c0_resources,
-};
-
-static struct platform_device pnx833x_i2c1_device = {
- .name = "i2c-pnx0105",
- .id = 1,
- .dev = {
- .platform_data = &pnx833x_i2c_dev[1],
- },
- .num_resources = ARRAY_SIZE(pnx833x_i2c1_resources),
- .resource = pnx833x_i2c1_resources,
-};
-#endif
-
static u64 ethernet_dmamask = DMA_BIT_MASK(32);
static struct resource pnx833x_ethernet_resources[] = {
@@ -294,10 +225,6 @@
static struct platform_device *pnx833x_platform_devices[] __initdata = {
&pnx833x_uart_device,
&pnx833x_usb_ehci_device,
-#ifdef CONFIG_I2C_PNX0105
- &pnx833x_i2c0_device,
- &pnx833x_i2c1_device,
-#endif
&pnx833x_ethernet_device,
&pnx833x_sata_device,
&pnx833x_flash_nand,
diff --git a/arch/mips/sgi-ip22/ip22-gio.c b/arch/mips/sgi-ip22/ip22-gio.c
index ab0e379..8e52446 100644
--- a/arch/mips/sgi-ip22/ip22-gio.c
+++ b/arch/mips/sgi-ip22/ip22-gio.c
@@ -19,6 +19,9 @@
} gio_name_table[] = {
{ .name = "SGI Impact", .id = 0x10 },
{ .name = "Phobos G160", .id = 0x35 },
+ { .name = "Phobos G130", .id = 0x36 },
+ { .name = "Phobos G100", .id = 0x37 },
+ { .name = "Set Engineering GFE", .id = 0x38 },
/* fake IDs */
{ .name = "SGI Newport", .id = 0x7e },
{ .name = "SGI GR2/GR3", .id = 0x7f },
@@ -293,7 +296,16 @@
* data matches
*/
ptr8 = (void *)CKSEG1ADDR(addr + 3);
- get_dbe(tmp8, ptr8);
+ if (get_dbe(tmp8, ptr8)) {
+ /*
+ * 32bit access worked, but 8bit doesn't
+ * so we don't see phantom reads on
+ * a pipelined bus, but a real card which
+ * doesn't support 8 bit reads
+ */
+ *res = tmp32;
+ return 1;
+ }
ptr16 = (void *)CKSEG1ADDR(addr + 2);
get_dbe(tmp16, ptr16);
if (tmp8 == (tmp16 & 0xff) &&
@@ -324,7 +336,7 @@
}
-static void ip22_check_gio(int slotno, unsigned long addr)
+static void ip22_check_gio(int slotno, unsigned long addr, int irq)
{
const char *name = "Unknown";
struct gio_device *gio_dev;
@@ -338,9 +350,9 @@
else {
if (!ip22_gio_id(addr, &tmp)) {
/*
- * no GIO signature at start address of slot, but
- * Newport doesn't have one, so let's check usea
- * status register
+ * no GIO signature at start address of slot
+ * since Newport doesn't have one, we check if
+ * user status register is readable
*/
if (ip22_gio_id(addr + NEWPORT_USTATUS_OFFS, &tmp))
tmp = 0x7e;
@@ -369,6 +381,7 @@
gio_dev->resource.start = addr;
gio_dev->resource.end = addr + 0x3fffff;
gio_dev->resource.flags = IORESOURCE_MEM;
+ gio_dev->irq = irq;
dev_set_name(&gio_dev->dev, "%d", slotno);
gio_device_register(gio_dev);
} else
@@ -408,16 +421,17 @@
request_resource(&iomem_resource, &gio_bus_resource);
printk(KERN_INFO "GIO: Probing bus...\n");
- if (ip22_is_fullhouse() ||
- !get_dbe(pbdma, (unsigned int *)&hpc3c1->pbdma[1])) {
- /* Indigo2 and ChallengeS */
- ip22_check_gio(0, GIO_SLOT_GFX_BASE);
- ip22_check_gio(1, GIO_SLOT_EXP0_BASE);
+ if (ip22_is_fullhouse()) {
+ /* Indigo2 */
+ ip22_check_gio(0, GIO_SLOT_GFX_BASE, SGI_GIO_1_IRQ);
+ ip22_check_gio(1, GIO_SLOT_EXP0_BASE, SGI_GIO_1_IRQ);
} else {
- /* Indy */
- ip22_check_gio(0, GIO_SLOT_GFX_BASE);
- ip22_check_gio(1, GIO_SLOT_EXP0_BASE);
- ip22_check_gio(2, GIO_SLOT_EXP1_BASE);
+ /* Indy/Challenge S */
+ if (get_dbe(pbdma, (unsigned int *)&hpc3c1->pbdma[1]))
+ ip22_check_gio(0, GIO_SLOT_GFX_BASE,
+ SGI_GIO_0_IRQ);
+ ip22_check_gio(1, GIO_SLOT_EXP0_BASE, SGI_GIOEXP0_IRQ);
+ ip22_check_gio(2, GIO_SLOT_EXP1_BASE, SGI_GIOEXP1_IRQ);
}
} else
device_unregister(&gio_bus);
diff --git a/arch/mips/sgi-ip22/ip22-int.c b/arch/mips/sgi-ip22/ip22-int.c
index 58b40ae..c66889f 100644
--- a/arch/mips/sgi-ip22/ip22-int.c
+++ b/arch/mips/sgi-ip22/ip22-int.c
@@ -119,9 +119,14 @@
} else
irq = lc0msk_to_irqnr[mask];
- /* if irq == 0, then the interrupt has already been cleared */
+ /*
+ * workaround for INT2 bug; if irq == 0, INT2 has seen a fifo full
+ * irq, but failed to latch it into status register
+ */
if (irq)
do_IRQ(irq);
+ else
+ do_IRQ(SGINT_LOCAL0 + 0);
}
static void indy_local1_irqdispatch(void)
diff --git a/arch/mips/sgi-ip27/ip27-smp.c b/arch/mips/sgi-ip27/ip27-smp.c
index f4ea8aa..f9ae6a8 100644
--- a/arch/mips/sgi-ip27/ip27-smp.c
+++ b/arch/mips/sgi-ip27/ip27-smp.c
@@ -186,10 +186,6 @@
local_irq_enable();
}
-static void __init ip27_cpus_done(void)
-{
-}
-
/*
* Launch a slave into smp_bootstrap(). It doesn't take an argument, and we
* set sp to the kernel stack of the newly created idle process, gp to the proc
@@ -236,7 +232,6 @@
.send_ipi_mask = ip27_send_ipi_mask,
.init_secondary = ip27_init_secondary,
.smp_finish = ip27_smp_finish,
- .cpus_done = ip27_cpus_done,
.boot_secondary = ip27_boot_secondary,
.smp_setup = ip27_smp_setup,
.prepare_cpus = ip27_prepare_cpus,
diff --git a/arch/mips/sibyte/bcm1480/irq.c b/arch/mips/sibyte/bcm1480/irq.c
index 59cfe26..373fbbc8 100644
--- a/arch/mips/sibyte/bcm1480/irq.c
+++ b/arch/mips/sibyte/bcm1480/irq.c
@@ -347,19 +347,8 @@
unsigned int cpu = smp_processor_id();
unsigned int pending;
-#ifdef CONFIG_SIBYTE_BCM1480_PROF
- /* Set compare to count to silence count/compare timer interrupts */
- write_c0_compare(read_c0_count());
-#endif
-
pending = read_c0_cause() & read_c0_status();
-#ifdef CONFIG_SIBYTE_BCM1480_PROF
- if (pending & CAUSEF_IP7) /* Cpu performance counter interrupt */
- sbprof_cpu_intr();
- else
-#endif
-
if (pending & CAUSEF_IP4)
do_IRQ(K_BCM1480_INT_TIMER_0 + cpu);
#ifdef CONFIG_SMP
diff --git a/arch/mips/sibyte/bcm1480/smp.c b/arch/mips/sibyte/bcm1480/smp.c
index 70d9182..af7d44e 100644
--- a/arch/mips/sibyte/bcm1480/smp.c
+++ b/arch/mips/sibyte/bcm1480/smp.c
@@ -115,13 +115,6 @@
}
/*
- * Final cleanup after all secondaries booted
- */
-static void bcm1480_cpus_done(void)
-{
-}
-
-/*
* Setup the PC, SP, and GP of a secondary processor and start it
* running!
*/
@@ -170,7 +163,6 @@
.send_ipi_mask = bcm1480_send_ipi_mask,
.init_secondary = bcm1480_init_secondary,
.smp_finish = bcm1480_smp_finish,
- .cpus_done = bcm1480_cpus_done,
.boot_secondary = bcm1480_boot_secondary,
.smp_setup = bcm1480_smp_setup,
.prepare_cpus = bcm1480_prepare_cpus,
diff --git a/arch/mips/sibyte/sb1250/smp.c b/arch/mips/sibyte/sb1250/smp.c
index db97611..c0c4b3f 100644
--- a/arch/mips/sibyte/sb1250/smp.c
+++ b/arch/mips/sibyte/sb1250/smp.c
@@ -103,13 +103,6 @@
}
/*
- * Final cleanup after all secondaries booted
- */
-static void sb1250_cpus_done(void)
-{
-}
-
-/*
* Setup the PC, SP, and GP of a secondary processor and start it
* running!
*/
@@ -158,7 +151,6 @@
.send_ipi_mask = sb1250_send_ipi_mask,
.init_secondary = sb1250_init_secondary,
.smp_finish = sb1250_smp_finish,
- .cpus_done = sb1250_cpus_done,
.boot_secondary = sb1250_boot_secondary,
.smp_setup = sb1250_smp_setup,
.prepare_cpus = sb1250_prepare_cpus,
diff --git a/arch/mips/txx9/generic/setup.c b/arch/mips/txx9/generic/setup.c
index 2b0b83c..dd2cf25 100644
--- a/arch/mips/txx9/generic/setup.c
+++ b/arch/mips/txx9/generic/setup.c
@@ -309,8 +309,8 @@
txx9_board_vec = find_board_byname(str + 6);
continue;
} else if (strncmp(str, "masterclk=", 10) == 0) {
- unsigned long val;
- if (strict_strtoul(str + 10, 10, &val) == 0)
+ unsigned int val;
+ if (kstrtouint(str + 10, 10, &val) == 0)
txx9_master_clock = val;
continue;
} else if (strcmp(str, "icdisable") == 0) {
diff --git a/arch/parisc/include/asm/ftrace.h b/arch/parisc/include/asm/ftrace.h
index 72c0faf..544ed8e 100644
--- a/arch/parisc/include/asm/ftrace.h
+++ b/arch/parisc/include/asm/ftrace.h
@@ -24,15 +24,7 @@
extern unsigned long return_address(unsigned int);
-#define HAVE_ARCH_CALLER_ADDR
-
-#define CALLER_ADDR0 ((unsigned long)__builtin_return_address(0))
-#define CALLER_ADDR1 return_address(1)
-#define CALLER_ADDR2 return_address(2)
-#define CALLER_ADDR3 return_address(3)
-#define CALLER_ADDR4 return_address(4)
-#define CALLER_ADDR5 return_address(5)
-#define CALLER_ADDR6 return_address(6)
+#define ftrace_return_address(n) return_address(n)
#endif /* __ASSEMBLY__ */
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index e099899..bd6dd6e 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -453,6 +453,14 @@
default "4"
depends on NEED_MULTIPLE_NODES
+config USE_PERCPU_NUMA_NODE_ID
+ def_bool y
+ depends on NUMA
+
+config HAVE_MEMORYLESS_NODES
+ def_bool y
+ depends on NUMA
+
config ARCH_SELECT_MEMORY_MODEL
def_bool y
depends on PPC64
diff --git a/arch/powerpc/Makefile b/arch/powerpc/Makefile
index ce4c68a..5687e29 100644
--- a/arch/powerpc/Makefile
+++ b/arch/powerpc/Makefile
@@ -113,8 +113,13 @@
endif
endif
-CFLAGS-$(CONFIG_PPC64) := -mtraceback=no -mcall-aixdesc
-CFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mabi=elfv1)
+CFLAGS-$(CONFIG_PPC64) := -mtraceback=no
+ifeq ($(CONFIG_CPU_LITTLE_ENDIAN),y)
+CFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mabi=elfv2,-mcall-aixdesc)
+AFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mabi=elfv2)
+else
+CFLAGS-$(CONFIG_PPC64) += -mcall-aixdesc
+endif
CFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mcmodel=medium,-mminimal-toc)
CFLAGS-$(CONFIG_PPC64) += $(call cc-option,-mno-pointers-to-nested-functions)
CFLAGS-$(CONFIG_PPC32) := -ffixed-r2 $(MULTIPLEWORD)
@@ -153,7 +158,7 @@
asinstr := $(call as-instr,lis 9$(comma)foo@high,-DHAVE_AS_ATHIGH=1)
KBUILD_CPPFLAGS += -Iarch/$(ARCH) $(asinstr)
-KBUILD_AFLAGS += -Iarch/$(ARCH)
+KBUILD_AFLAGS += -Iarch/$(ARCH) $(AFLAGS-y)
KBUILD_CFLAGS += -msoft-float -pipe -Iarch/$(ARCH) $(CFLAGS-y)
CPP = $(CC) -E $(KBUILD_CFLAGS)
@@ -161,6 +166,11 @@
KBUILD_LDFLAGS_MODULE += arch/powerpc/lib/crtsavres.o
+ifeq ($(CONFIG_476FPE_ERR46),y)
+ KBUILD_LDFLAGS_MODULE += --ppc476-workaround \
+ -T $(srctree)/arch/powerpc/platforms/44x/ppc476_modules.lds
+endif
+
# No AltiVec or VSX instructions when building kernel
KBUILD_CFLAGS += $(call cc-option,-mno-altivec)
KBUILD_CFLAGS += $(call cc-option,-mno-vsx)
diff --git a/arch/powerpc/boot/Makefile b/arch/powerpc/boot/Makefile
index a1f8c7f1..426dce7 100644
--- a/arch/powerpc/boot/Makefile
+++ b/arch/powerpc/boot/Makefile
@@ -22,8 +22,14 @@
BOOTCFLAGS := -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
-fno-strict-aliasing -Os -msoft-float -pipe \
-fomit-frame-pointer -fno-builtin -fPIC -nostdinc \
- -isystem $(shell $(CROSS32CC) -print-file-name=include) \
- -mbig-endian
+ -isystem $(shell $(CROSS32CC) -print-file-name=include)
+ifdef CONFIG_PPC64_BOOT_WRAPPER
+BOOTCFLAGS += -m64
+endif
+ifdef CONFIG_CPU_BIG_ENDIAN
+BOOTCFLAGS += -mbig-endian
+endif
+
BOOTAFLAGS := -D__ASSEMBLY__ $(BOOTCFLAGS) -traditional -nostdinc
ifdef CONFIG_DEBUG_INFO
@@ -47,6 +53,7 @@
$(obj)/treeboot-walnut.o: BOOTCFLAGS += -mcpu=405
$(obj)/treeboot-iss4xx.o: BOOTCFLAGS += -mcpu=405
$(obj)/treeboot-currituck.o: BOOTCFLAGS += -mcpu=405
+$(obj)/treeboot-akebono.o: BOOTCFLAGS += -mcpu=405
$(obj)/virtex405-head.o: BOOTAFLAGS += -mcpu=405
@@ -86,6 +93,7 @@
cuboot-taishan.c cuboot-katmai.c \
cuboot-warp.c cuboot-yosemite.c \
treeboot-iss4xx.c treeboot-currituck.c \
+ treeboot-akebono.c \
simpleboot.c fixed-head.S virtex.c
src-plat-$(CONFIG_8xx) += cuboot-8xx.c fixed-head.S ep88xc.c redboot-8xx.c
src-plat-$(CONFIG_PPC_MPC52xx) += cuboot-52xx.c
@@ -99,6 +107,11 @@
src-plat-$(CONFIG_AMIGAONE) += cuboot-amigaone.c
src-plat-$(CONFIG_PPC_PS3) += ps3-head.S ps3-hvcall.S ps3.c
src-plat-$(CONFIG_EPAPR_BOOT) += epapr.c epapr-wrapper.c
+src-plat-$(CONFIG_PPC_PSERIES) += pseries-head.S
+src-plat-$(CONFIG_PPC_POWERNV) += pseries-head.S
+src-plat-$(CONFIG_PPC_IBM_CELL_BLADE) += pseries-head.S
+src-plat-$(CONFIG_PPC_CELLEB) += pseries-head.S
+src-plat-$(CONFIG_PPC_CELL_QPACE) += pseries-head.S
src-wlib := $(sort $(src-wlib-y))
src-plat := $(sort $(src-plat-y))
@@ -137,7 +150,11 @@
$(obj)/empty.c:
@touch $@
-$(obj)/zImage.lds $(obj)/zImage.coff.lds $(obj)/zImage.ps3.lds: $(obj)/%: $(srctree)/$(src)/%.S
+$(obj)/zImage.lds: $(obj)/%: $(srctree)/$(src)/%.S
+ $(CROSS32CC) $(cpp_flags) -E -Wp,-MD,$(depfile) -P -Upowerpc \
+ -D__ASSEMBLY__ -DLINKER_SCRIPT -o $@ $<
+
+$(obj)/zImage.coff.lds $(obj)/zImage.ps3.lds : $(obj)/%: $(srctree)/$(src)/%.S
@cp $< $@
clean-files := $(zlib) $(zlibheader) $(zliblinuxheader) \
@@ -235,6 +252,7 @@
image-$(CONFIG_ISS4xx) += treeImage.iss4xx \
treeImage.iss4xx-mpic
image-$(CONFIG_CURRITUCK) += treeImage.currituck
+image-$(CONFIG_AKEBONO) += treeImage.akebono
# Board ports in arch/powerpc/platform/8xx/Kconfig
image-$(CONFIG_MPC86XADS) += cuImage.mpc866ads
diff --git a/arch/powerpc/boot/addnote.c b/arch/powerpc/boot/addnote.c
index 349b553..9d9f6f3 100644
--- a/arch/powerpc/boot/addnote.c
+++ b/arch/powerpc/boot/addnote.c
@@ -6,6 +6,8 @@
*
* Copyright 2000 Paul Mackerras.
*
+ * Adapted for 64 bit little endian images by Andrew Tauferner.
+ *
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version
@@ -55,36 +57,61 @@
#define ROUNDUP(len) (((len) + 3) & ~3)
-unsigned char buf[512];
+unsigned char buf[1024];
+#define ELFDATA2LSB 1
+#define ELFDATA2MSB 2
+static int e_data = ELFDATA2MSB;
+#define ELFCLASS32 1
+#define ELFCLASS64 2
+static int e_class = ELFCLASS32;
#define GET_16BE(off) ((buf[off] << 8) + (buf[(off)+1]))
-#define GET_32BE(off) ((GET_16BE(off) << 16) + GET_16BE((off)+2))
+#define GET_32BE(off) ((GET_16BE(off) << 16U) + GET_16BE((off)+2U))
+#define GET_64BE(off) ((((unsigned long long)GET_32BE(off)) << 32ULL) + \
+ ((unsigned long long)GET_32BE((off)+4ULL)))
+#define PUT_16BE(off, v)(buf[off] = ((v) >> 8) & 0xff, \
+ buf[(off) + 1] = (v) & 0xff)
+#define PUT_32BE(off, v)(PUT_16BE((off), (v) >> 16L), PUT_16BE((off) + 2, (v)))
+#define PUT_64BE(off, v)((PUT_32BE((off), (v) >> 32L), \
+ PUT_32BE((off) + 4, (v))))
-#define PUT_16BE(off, v) (buf[off] = ((v) >> 8) & 0xff, \
- buf[(off) + 1] = (v) & 0xff)
-#define PUT_32BE(off, v) (PUT_16BE((off), (v) >> 16), \
- PUT_16BE((off) + 2, (v)))
+#define GET_16LE(off) ((buf[off]) + (buf[(off)+1] << 8))
+#define GET_32LE(off) (GET_16LE(off) + (GET_16LE((off)+2U) << 16U))
+#define GET_64LE(off) ((unsigned long long)GET_32LE(off) + \
+ (((unsigned long long)GET_32LE((off)+4ULL)) << 32ULL))
+#define PUT_16LE(off, v) (buf[off] = (v) & 0xff, \
+ buf[(off) + 1] = ((v) >> 8) & 0xff)
+#define PUT_32LE(off, v) (PUT_16LE((off), (v)), PUT_16LE((off) + 2, (v) >> 16L))
+#define PUT_64LE(off, v) (PUT_32LE((off), (v)), PUT_32LE((off) + 4, (v) >> 32L))
+
+#define GET_16(off) (e_data == ELFDATA2MSB ? GET_16BE(off) : GET_16LE(off))
+#define GET_32(off) (e_data == ELFDATA2MSB ? GET_32BE(off) : GET_32LE(off))
+#define GET_64(off) (e_data == ELFDATA2MSB ? GET_64BE(off) : GET_64LE(off))
+#define PUT_16(off, v) (e_data == ELFDATA2MSB ? PUT_16BE(off, v) : \
+ PUT_16LE(off, v))
+#define PUT_32(off, v) (e_data == ELFDATA2MSB ? PUT_32BE(off, v) : \
+ PUT_32LE(off, v))
+#define PUT_64(off, v) (e_data == ELFDATA2MSB ? PUT_64BE(off, v) : \
+ PUT_64LE(off, v))
/* Structure of an ELF file */
#define E_IDENT 0 /* ELF header */
-#define E_PHOFF 28
-#define E_PHENTSIZE 42
-#define E_PHNUM 44
-#define E_HSIZE 52 /* size of ELF header */
+#define E_PHOFF (e_class == ELFCLASS32 ? 28 : 32)
+#define E_PHENTSIZE (e_class == ELFCLASS32 ? 42 : 54)
+#define E_PHNUM (e_class == ELFCLASS32 ? 44 : 56)
+#define E_HSIZE (e_class == ELFCLASS32 ? 52 : 64)
#define EI_MAGIC 0 /* offsets in E_IDENT area */
#define EI_CLASS 4
#define EI_DATA 5
#define PH_TYPE 0 /* ELF program header */
-#define PH_OFFSET 4
-#define PH_FILESZ 16
-#define PH_HSIZE 32 /* size of program header */
+#define PH_OFFSET (e_class == ELFCLASS32 ? 4 : 8)
+#define PH_FILESZ (e_class == ELFCLASS32 ? 16 : 32)
+#define PH_HSIZE (e_class == ELFCLASS32 ? 32 : 56)
#define PT_NOTE 4 /* Program header type = note */
-#define ELFCLASS32 1
-#define ELFDATA2MSB 2
unsigned char elf_magic[4] = { 0x7f, 'E', 'L', 'F' };
@@ -92,8 +119,8 @@
main(int ac, char **av)
{
int fd, n, i;
- int ph, ps, np;
- int nnote, nnote2, ns;
+ unsigned long ph, ps, np;
+ long nnote, nnote2, ns;
if (ac != 2) {
fprintf(stderr, "Usage: %s elf-file\n", av[0]);
@@ -114,26 +141,27 @@
exit(1);
}
- if (n < E_HSIZE || memcmp(&buf[E_IDENT+EI_MAGIC], elf_magic, 4) != 0)
+ if (memcmp(&buf[E_IDENT+EI_MAGIC], elf_magic, 4) != 0)
+ goto notelf;
+ e_class = buf[E_IDENT+EI_CLASS];
+ if (e_class != ELFCLASS32 && e_class != ELFCLASS64)
+ goto notelf;
+ e_data = buf[E_IDENT+EI_DATA];
+ if (e_data != ELFDATA2MSB && e_data != ELFDATA2LSB)
+ goto notelf;
+ if (n < E_HSIZE)
goto notelf;
- if (buf[E_IDENT+EI_CLASS] != ELFCLASS32
- || buf[E_IDENT+EI_DATA] != ELFDATA2MSB) {
- fprintf(stderr, "%s is not a big-endian 32-bit ELF image\n",
- av[1]);
- exit(1);
- }
-
- ph = GET_32BE(E_PHOFF);
- ps = GET_16BE(E_PHENTSIZE);
- np = GET_16BE(E_PHNUM);
+ ph = (e_class == ELFCLASS32 ? GET_32(E_PHOFF) : GET_64(E_PHOFF));
+ ps = GET_16(E_PHENTSIZE);
+ np = GET_16(E_PHNUM);
if (ph < E_HSIZE || ps < PH_HSIZE || np < 1)
goto notelf;
if (ph + (np + 2) * ps + nnote + nnote2 > n)
goto nospace;
for (i = 0; i < np; ++i) {
- if (GET_32BE(ph + PH_TYPE) == PT_NOTE) {
+ if (GET_32(ph + PH_TYPE) == PT_NOTE) {
fprintf(stderr, "%s already has a note entry\n",
av[1]);
exit(0);
@@ -148,15 +176,22 @@
/* fill in the program header entry */
ns = ph + 2 * ps;
- PUT_32BE(ph + PH_TYPE, PT_NOTE);
- PUT_32BE(ph + PH_OFFSET, ns);
- PUT_32BE(ph + PH_FILESZ, nnote);
+ PUT_32(ph + PH_TYPE, PT_NOTE);
+ if (e_class == ELFCLASS32)
+ PUT_32(ph + PH_OFFSET, ns);
+ else
+ PUT_64(ph + PH_OFFSET, ns);
+
+ if (e_class == ELFCLASS32)
+ PUT_32(ph + PH_FILESZ, nnote);
+ else
+ PUT_64(ph + PH_FILESZ, nnote);
/* fill in the note area we point to */
/* XXX we should probably make this a proper section */
- PUT_32BE(ns, strlen(arch) + 1);
- PUT_32BE(ns + 4, N_DESCR * 4);
- PUT_32BE(ns + 8, 0x1275);
+ PUT_32(ns, strlen(arch) + 1);
+ PUT_32(ns + 4, N_DESCR * 4);
+ PUT_32(ns + 8, 0x1275);
strcpy((char *) &buf[ns + 12], arch);
ns += 12 + strlen(arch) + 1;
for (i = 0; i < N_DESCR; ++i, ns += 4)
@@ -164,21 +199,28 @@
/* fill in the second program header entry and the RPA note area */
ph += ps;
- PUT_32BE(ph + PH_TYPE, PT_NOTE);
- PUT_32BE(ph + PH_OFFSET, ns);
- PUT_32BE(ph + PH_FILESZ, nnote2);
+ PUT_32(ph + PH_TYPE, PT_NOTE);
+ if (e_class == ELFCLASS32)
+ PUT_32(ph + PH_OFFSET, ns);
+ else
+ PUT_64(ph + PH_OFFSET, ns);
+
+ if (e_class == ELFCLASS32)
+ PUT_32(ph + PH_FILESZ, nnote);
+ else
+ PUT_64(ph + PH_FILESZ, nnote2);
/* fill in the note area we point to */
- PUT_32BE(ns, strlen(rpaname) + 1);
- PUT_32BE(ns + 4, sizeof(rpanote));
- PUT_32BE(ns + 8, 0x12759999);
+ PUT_32(ns, strlen(rpaname) + 1);
+ PUT_32(ns + 4, sizeof(rpanote));
+ PUT_32(ns + 8, 0x12759999);
strcpy((char *) &buf[ns + 12], rpaname);
ns += 12 + ROUNDUP(strlen(rpaname) + 1);
for (i = 0; i < N_RPA_DESCR; ++i, ns += 4)
PUT_32BE(ns, rpanote[i]);
/* Update the number of program headers */
- PUT_16BE(E_PHNUM, np + 2);
+ PUT_16(E_PHNUM, np + 2);
/* write back */
lseek(fd, (long) 0, SEEK_SET);
diff --git a/arch/powerpc/boot/crt0.S b/arch/powerpc/boot/crt0.S
index 0f7428a..14de4f8 100644
--- a/arch/powerpc/boot/crt0.S
+++ b/arch/powerpc/boot/crt0.S
@@ -1,17 +1,20 @@
/*
* Copyright (C) Paul Mackerras 1997.
*
+ * Adapted for 64 bit LE PowerPC by Andrew Tauferner
+ *
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*
- * NOTE: this code runs in 32 bit mode, is position-independent,
- * and is packaged as ELF32.
*/
#include "ppc_asm.h"
+RELA = 7
+RELACOUNT = 0x6ffffff9
+
.text
/* A procedure descriptor used when booting this as a COFF file.
* When making COFF, this comes first in the link and we're
@@ -21,6 +24,20 @@
_zimage_start_opd:
.long 0x500000, 0, 0, 0
+#ifdef __powerpc64__
+.balign 8
+p_start: .llong _start
+p_etext: .llong _etext
+p_bss_start: .llong __bss_start
+p_end: .llong _end
+
+p_toc: .llong __toc_start + 0x8000 - p_base
+p_dyn: .llong __dynamic_start - p_base
+p_rela: .llong __rela_dyn_start - p_base
+p_prom: .llong 0
+ .weak _platform_stack_top
+p_pstack: .llong _platform_stack_top
+#else
p_start: .long _start
p_etext: .long _etext
p_bss_start: .long __bss_start
@@ -28,6 +45,7 @@
.weak _platform_stack_top
p_pstack: .long _platform_stack_top
+#endif
.weak _zimage_start
.globl _zimage_start
@@ -38,6 +56,7 @@
and the address where we're running. */
bl .+4
p_base: mflr r10 /* r10 now points to runtime addr of p_base */
+#ifndef __powerpc64__
/* grab the link address of the dynamic section in r11 */
addis r11,r10,(_GLOBAL_OFFSET_TABLE_-p_base)@ha
lwz r11,(_GLOBAL_OFFSET_TABLE_-p_base)@l(r11)
@@ -51,8 +70,6 @@
/* The dynamic section contains a series of tagged entries.
* We need the RELA and RELACOUNT entries. */
-RELA = 7
-RELACOUNT = 0x6ffffff9
li r9,0
li r0,0
9: lwz r8,0(r12) /* get tag */
@@ -120,9 +137,164 @@
li r0,0
stwu r0,-16(r1) /* establish a stack frame */
6:
+#else /* __powerpc64__ */
+ /* Save the prom pointer at p_prom. */
+ std r5,(p_prom-p_base)(r10)
+ /* Set r2 to the TOC. */
+ ld r2,(p_toc-p_base)(r10)
+ add r2,r2,r10
+
+ /* Grab the link address of the dynamic section in r11. */
+ ld r11,-32768(r2)
+ cmpwi r11,0
+ beq 3f /* if not linked -pie then no dynamic section */
+
+ ld r11,(p_dyn-p_base)(r10)
+ add r11,r11,r10
+ ld r9,(p_rela-p_base)(r10)
+ add r9,r9,r10
+
+ li r7,0
+ li r8,0
+9: ld r6,0(r11) /* get tag */
+ cmpdi r6,0
+ beq 12f /* end of list */
+ cmpdi r6,RELA
+ bne 10f
+ ld r7,8(r11) /* get RELA pointer in r7 */
+ b 11f
+10: addis r6,r6,(-RELACOUNT)@ha
+ cmpdi r6,RELACOUNT@l
+ bne 11f
+ ld r8,8(r11) /* get RELACOUNT value in r8 */
+11: addi r11,r11,16
+ b 9b
+12:
+ cmpdi r7,0 /* check we have both RELA and RELACOUNT */
+ cmpdi cr1,r8,0
+ beq 3f
+ beq cr1,3f
+
+ /* Calcuate the runtime offset. */
+ subf r7,r7,r9
+
+ /* Run through the list of relocations and process the
+ * R_PPC64_RELATIVE ones. */
+ mtctr r8
+13: ld r0,8(r9) /* ELF64_R_TYPE(reloc->r_info) */
+ cmpdi r0,22 /* R_PPC64_RELATIVE */
+ bne 3f
+ ld r6,0(r9) /* reloc->r_offset */
+ ld r0,16(r9) /* reloc->r_addend */
+ add r0,r0,r7
+ stdx r0,r7,r6
+ addi r9,r9,24
+ bdnz 13b
+
+ /* Do a cache flush for our text, in case the loader didn't */
+3: ld r9,p_start-p_base(r10) /* note: these are relocated now */
+ ld r8,p_etext-p_base(r10)
+4: dcbf r0,r9
+ icbi r0,r9
+ addi r9,r9,0x20
+ cmpld cr0,r9,r8
+ blt 4b
+ sync
+ isync
+
+ /* Clear the BSS */
+ ld r9,p_bss_start-p_base(r10)
+ ld r8,p_end-p_base(r10)
+ li r0,0
+5: std r0,0(r9)
+ addi r9,r9,8
+ cmpld cr0,r9,r8
+ blt 5b
+
+ /* Possibly set up a custom stack */
+ ld r8,p_pstack-p_base(r10)
+ cmpdi r8,0
+ beq 6f
+ ld r1,0(r8)
+ li r0,0
+ stdu r0,-16(r1) /* establish a stack frame */
+6:
+#endif /* __powerpc64__ */
/* Call platform_init() */
bl platform_init
/* Call start */
b start
+
+#ifdef __powerpc64__
+
+#define PROM_FRAME_SIZE 512
+#define SAVE_GPR(n, base) std n,8*(n)(base)
+#define REST_GPR(n, base) ld n,8*(n)(base)
+#define SAVE_2GPRS(n, base) SAVE_GPR(n, base); SAVE_GPR(n+1, base)
+#define SAVE_4GPRS(n, base) SAVE_2GPRS(n, base); SAVE_2GPRS(n+2, base)
+#define SAVE_8GPRS(n, base) SAVE_4GPRS(n, base); SAVE_4GPRS(n+4, base)
+#define SAVE_10GPRS(n, base) SAVE_8GPRS(n, base); SAVE_2GPRS(n+8, base)
+#define REST_2GPRS(n, base) REST_GPR(n, base); REST_GPR(n+1, base)
+#define REST_4GPRS(n, base) REST_2GPRS(n, base); REST_2GPRS(n+2, base)
+#define REST_8GPRS(n, base) REST_4GPRS(n, base); REST_4GPRS(n+4, base)
+#define REST_10GPRS(n, base) REST_8GPRS(n, base); REST_2GPRS(n+8, base)
+
+/* prom handles the jump into and return from firmware. The prom args pointer
+ is loaded in r3. */
+.globl prom
+prom:
+ mflr r0
+ std r0,16(r1)
+ stdu r1,-PROM_FRAME_SIZE(r1) /* Save SP and create stack space */
+
+ SAVE_GPR(2, r1)
+ SAVE_GPR(13, r1)
+ SAVE_8GPRS(14, r1)
+ SAVE_10GPRS(22, r1)
+ mfcr r10
+ std r10,8*32(r1)
+ mfmsr r10
+ std r10,8*33(r1)
+
+ /* remove MSR_LE from msr but keep MSR_SF */
+ mfmsr r10
+ rldicr r10,r10,0,62
+ mtsrr1 r10
+
+ /* Load FW address, set LR to label 1, and jump to FW */
+ bl 0f
+0: mflr r10
+ addi r11,r10,(1f-0b)
+ mtlr r11
+
+ ld r10,(p_prom-0b)(r10)
+ mtsrr0 r10
+
+ rfid
+
+1: /* Return from OF */
+ FIXUP_ENDIAN
+
+ /* Restore registers and return. */
+ rldicl r1,r1,0,32
+
+ /* Restore the MSR (back to 64 bits) */
+ ld r10,8*(33)(r1)
+ mtmsr r10
+ isync
+
+ /* Restore other registers */
+ REST_GPR(2, r1)
+ REST_GPR(13, r1)
+ REST_8GPRS(14, r1)
+ REST_10GPRS(22, r1)
+ ld r10,8*32(r1)
+ mtcr r10
+
+ addi r1,r1,PROM_FRAME_SIZE
+ ld r0,16(r1)
+ mtlr r0
+ blr
+#endif
diff --git a/arch/powerpc/boot/dcr.h b/arch/powerpc/boot/dcr.h
index cc73f7a..bf8f4ed 100644
--- a/arch/powerpc/boot/dcr.h
+++ b/arch/powerpc/boot/dcr.h
@@ -15,6 +15,10 @@
asm volatile("mfdcrx %0,%1" : "=r"(rval) : "r"(rn)); \
rval; \
})
+#define mtdcrx(rn, val) \
+ ({ \
+ asm volatile("mtdcrx %0,%1" : : "r"(rn), "r" (val)); \
+ })
/* 440GP/440GX SDRAM controller DCRs */
#define DCRN_SDRAM0_CFGADDR 0x010
diff --git a/arch/powerpc/boot/dts/akebono.dts b/arch/powerpc/boot/dts/akebono.dts
new file mode 100644
index 0000000..f92ecfe
--- /dev/null
+++ b/arch/powerpc/boot/dts/akebono.dts
@@ -0,0 +1,415 @@
+/*
+ * Device Tree Source for IBM Embedded PPC 476 Platform
+ *
+ * Copyright © 2013 Tony Breeds IBM Corporation
+ * Copyright © 2013 Alistair Popple IBM Corporation
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without
+ * any warranty of any kind, whether express or implied.
+ */
+
+/dts-v1/;
+
+/memreserve/ 0x01f00000 0x00100000; // spin table
+
+/ {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ model = "ibm,akebono";
+ compatible = "ibm,akebono", "ibm,476gtr";
+ dcr-parent = <&{/cpus/cpu@0}>;
+
+ aliases {
+ serial0 = &UART0;
+ };
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ cpu@0 {
+ device_type = "cpu";
+ model = "PowerPC,476";
+ reg = <0>;
+ clock-frequency = <1600000000>; // 1.6 GHz
+ timebase-frequency = <100000000>; // 100Mhz
+ i-cache-line-size = <32>;
+ d-cache-line-size = <32>;
+ i-cache-size = <32768>;
+ d-cache-size = <32768>;
+ dcr-controller;
+ dcr-access-method = "native";
+ status = "ok";
+ };
+ cpu@1 {
+ device_type = "cpu";
+ model = "PowerPC,476";
+ reg = <1>;
+ clock-frequency = <1600000000>; // 1.6 GHz
+ timebase-frequency = <100000000>; // 100Mhz
+ i-cache-line-size = <32>;
+ d-cache-line-size = <32>;
+ i-cache-size = <32768>;
+ d-cache-size = <32768>;
+ dcr-controller;
+ dcr-access-method = "native";
+ status = "disabled";
+ enable-method = "spin-table";
+ cpu-release-addr = <0x0 0x01f00000>;
+ };
+ };
+
+ memory {
+ device_type = "memory";
+ reg = <0x0 0x0 0x0 0x0>; // filled in by zImage
+ };
+
+ MPIC: interrupt-controller {
+ compatible = "chrp,open-pic";
+ interrupt-controller;
+ dcr-reg = <0xffc00000 0x00040000>;
+ #address-cells = <0>;
+ #size-cells = <0>;
+ #interrupt-cells = <2>;
+ single-cpu-affinity;
+ };
+
+ plb {
+ compatible = "ibm,plb6";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+ clock-frequency = <200000000>; // 200Mhz
+
+ HSTA0: hsta@310000e0000 {
+ compatible = "ibm,476gtr-hsta-msi", "ibm,hsta-msi";
+ reg = <0x310 0x000e0000 0x0 0xf0>;
+ interrupt-parent = <&MPIC>;
+ interrupts = <108 0
+ 109 0
+ 110 0
+ 111 0
+ 112 0
+ 113 0
+ 114 0
+ 115 0
+ 116 0
+ 117 0
+ 118 0
+ 119 0
+ 120 0
+ 121 0
+ 122 0
+ 123 0>;
+ };
+
+ MAL0: mcmal {
+ compatible = "ibm,mcmal-476gtr", "ibm,mcmal2";
+ dcr-reg = <0xc0000000 0x062>;
+ num-tx-chans = <1>;
+ num-rx-chans = <1>;
+ #address-cells = <0>;
+ #size-cells = <0>;
+ interrupt-parent = <&MPIC>;
+ interrupts = < /*TXEOB*/ 77 0x4
+ /*RXEOB*/ 78 0x4
+ /*SERR*/ 76 0x4
+ /*TXDE*/ 79 0x4
+ /*RXDE*/ 80 0x4>;
+ };
+
+ SATA0: sata@30000010000 {
+ compatible = "ibm,476gtr-ahci";
+ reg = <0x300 0x00010000 0x0 0x10000>;
+ interrupt-parent = <&MPIC>;
+ interrupts = <93 2>;
+ };
+
+ EHCI0: ehci@30010000000 {
+ compatible = "ibm,476gtr-ehci", "generic-ehci";
+ reg = <0x300 0x10000000 0x0 0x10000>;
+ interrupt-parent = <&MPIC>;
+ interrupts = <85 2>;
+ };
+
+ SD0: sd@30000000000 {
+ compatible = "ibm,476gtr-sdhci", "generic-sdhci";
+ reg = <0x300 0x00000000 0x0 0x10000>;
+ interrupts = <91 2>;
+ interrupt-parent = <&MPIC>;
+ };
+
+ OHCI0: ohci@30010010000 {
+ compatible = "ibm,476gtr-ohci", "generic-ohci";
+ reg = <0x300 0x10010000 0x0 0x10000>;
+ interrupt-parent = <&MPIC>;
+ interrupts = <89 1>;
+ };
+
+ OHCI1: ohci@30010020000 {
+ compatible = "ibm,476gtr-ohci", "generic-ohci";
+ reg = <0x300 0x10020000 0x0 0x10000>;
+ interrupt-parent = <&MPIC>;
+ interrupts = <88 1>;
+ };
+
+ POB0: opb {
+ compatible = "ibm,opb-4xx", "ibm,opb";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ /* Wish there was a nicer way of specifying a full
+ * 32-bit range
+ */
+ ranges = <0x00000000 0x0000033f 0x00000000 0x80000000
+ 0x80000000 0x0000033f 0x80000000 0x80000000>;
+ clock-frequency = <100000000>;
+
+ RGMII0: emac-rgmii-wol@50004 {
+ compatible = "ibm,rgmii-wol-476gtr", "ibm,rgmii-wol";
+ reg = <0x50004 0x00000008>;
+ has-mdio;
+ };
+
+ EMAC0: ethernet@30000 {
+ device_type = "network";
+ compatible = "ibm,emac-476gtr", "ibm,emac4sync";
+ interrupt-parent = <&EMAC0>;
+ interrupts = <0x0 0x1>;
+ #interrupt-cells = <1>;
+ #address-cells = <0>;
+ #size-cells = <0>;
+ interrupt-map = </*Status*/ 0x0 &MPIC 81 0x4
+ /*Wake*/ 0x1 &MPIC 82 0x4>;
+ reg = <0x30000 0x78>;
+
+ /* local-mac-address will normally be added by
+ * the wrapper. If your device doesn't support
+ * passing data to the wrapper (in the form
+ * local-mac-addr=<hwaddr>) then you will need
+ * to set it manually here. */
+ //local-mac-address = [000000000000];
+
+ mal-device = <&MAL0>;
+ mal-tx-channel = <0>;
+ mal-rx-channel = <0>;
+ cell-index = <0>;
+ max-frame-size = <9000>;
+ rx-fifo-size = <4096>;
+ tx-fifo-size = <2048>;
+ rx-fifo-size-gige = <16384>;
+ phy-mode = "rgmii";
+ phy-map = <0x00000000>;
+ rgmii-wol-device = <&RGMII0>;
+ has-inverted-stacr-oc;
+ has-new-stacr-staopc;
+ };
+
+ UART0: serial@10000 {
+ device_type = "serial";
+ compatible = "ns16750", "ns16550";
+ reg = <0x10000 0x00000008>;
+ virtual-reg = <0xe8010000>;
+ clock-frequency = <1851851>;
+ current-speed = <38400>;
+ interrupt-parent = <&MPIC>;
+ interrupts = <39 2>;
+ };
+
+ IIC0: i2c@00000000 {
+ compatible = "ibm,iic-476gtr", "ibm,iic";
+ reg = <0x0 0x00000020>;
+ interrupt-parent = <&MPIC>;
+ interrupts = <37 2>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ rtc@68 {
+ compatible = "stm,m41t80", "m41st85";
+ reg = <0x68>;
+ };
+ };
+
+ IIC1: i2c@00000100 {
+ compatible = "ibm,iic-476gtr", "ibm,iic";
+ reg = <0x100 0x00000020>;
+ interrupt-parent = <&MPIC>;
+ interrupts = <38 2>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ avr@58 {
+ compatible = "ibm,akebono-avr";
+ reg = <0x58>;
+ };
+ };
+
+ FPGA0: fpga@ebc00000 {
+ compatible = "ibm,akebono-fpga";
+ reg = <0xebc00000 0x8>;
+ };
+ };
+
+ PCIE0: pciex@10100000000 {
+ device_type = "pci";
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ compatible = "ibm,plb-pciex-476fpe", "ibm,plb-pciex";
+ primary;
+ port = <0x0>; /* port number */
+ reg = <0x00000101 0x00000000 0x0 0x10000000 /* Config space access */
+ 0x00000100 0x00000000 0x0 0x00001000>; /* UTL Registers space access */
+ dcr-reg = <0xc0 0x20>;
+
+// pci_space < pci_addr > < cpu_addr > < size >
+ ranges = <0x02000000 0x00000000 0x80000000 0x00000110 0x80000000 0x0 0x80000000
+ 0x01000000 0x0 0x0 0x00000140 0x0 0x0 0x00010000>;
+
+ /* Inbound starting at 0x0 to 0x40000000000. In order to use MSI
+ * PCI devices must be able to write to the HSTA module.
+ */
+ dma-ranges = <0x42000000 0x0 0x0 0x0 0x0 0x400 0x0>;
+
+ /* This drives busses 0 to 0xf */
+ bus-range = <0x0 0xf>;
+
+ /* Legacy interrupts (note the weird polarity, the bridge seems
+ * to invert PCIe legacy interrupts).
+ * We are de-swizzling here because the numbers are actually for
+ * port of the root complex virtual P2P bridge. But I want
+ * to avoid putting a node for it in the tree, so the numbers
+ * below are basically de-swizzled numbers.
+ * The real slot is on idsel 0, so the swizzling is 1:1
+ */
+ interrupt-map-mask = <0x0 0x0 0x0 0x7>;
+ interrupt-map = <
+ 0x0 0x0 0x0 0x1 &MPIC 45 0x2 /* int A */
+ 0x0 0x0 0x0 0x2 &MPIC 46 0x2 /* int B */
+ 0x0 0x0 0x0 0x3 &MPIC 47 0x2 /* int C */
+ 0x0 0x0 0x0 0x4 &MPIC 48 0x2 /* int D */>;
+ };
+
+ PCIE1: pciex@20100000000 {
+ device_type = "pci";
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ compatible = "ibm,plb-pciex-476fpe", "ibm,plb-pciex";
+ primary;
+ port = <0x1>; /* port number */
+ reg = <0x00000201 0x00000000 0x0 0x10000000 /* Config space access */
+ 0x00000200 0x00000000 0x0 0x00001000>; /* UTL Registers space access */
+ dcr-reg = <0x100 0x20>;
+
+// pci_space < pci_addr > < cpu_addr > < size >
+ ranges = <0x02000000 0x00000000 0x80000000 0x00000210 0x80000000 0x0 0x80000000
+ 0x01000000 0x0 0x0 0x00000240 0x0 0x0 0x00010000>;
+
+ /* Inbound starting at 0x0 to 0x40000000000. In order to use MSI
+ * PCI devices must be able to write to the HSTA module.
+ */
+ dma-ranges = <0x42000000 0x0 0x0 0x0 0x0 0x400 0x0>;
+
+ /* This drives busses 0 to 0xf */
+ bus-range = <0x0 0xf>;
+
+ /* Legacy interrupts (note the weird polarity, the bridge seems
+ * to invert PCIe legacy interrupts).
+ * We are de-swizzling here because the numbers are actually for
+ * port of the root complex virtual P2P bridge. But I want
+ * to avoid putting a node for it in the tree, so the numbers
+ * below are basically de-swizzled numbers.
+ * The real slot is on idsel 0, so the swizzling is 1:1
+ */
+ interrupt-map-mask = <0x0 0x0 0x0 0x7>;
+ interrupt-map = <
+ 0x0 0x0 0x0 0x1 &MPIC 53 0x2 /* int A */
+ 0x0 0x0 0x0 0x2 &MPIC 54 0x2 /* int B */
+ 0x0 0x0 0x0 0x3 &MPIC 55 0x2 /* int C */
+ 0x0 0x0 0x0 0x4 &MPIC 56 0x2 /* int D */>;
+ };
+
+ PCIE2: pciex@18100000000 {
+ device_type = "pci";
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ compatible = "ibm,plb-pciex-476fpe", "ibm,plb-pciex";
+ primary;
+ port = <0x2>; /* port number */
+ reg = <0x00000181 0x00000000 0x0 0x10000000 /* Config space access */
+ 0x00000180 0x00000000 0x0 0x00001000>; /* UTL Registers space access */
+ dcr-reg = <0xe0 0x20>;
+
+// pci_space < pci_addr > < cpu_addr > < size >
+ ranges = <0x02000000 0x00000000 0x80000000 0x00000190 0x80000000 0x0 0x80000000
+ 0x01000000 0x0 0x0 0x000001c0 0x0 0x0 0x00010000>;
+
+ /* Inbound starting at 0x0 to 0x40000000000. In order to use MSI
+ * PCI devices must be able to write to the HSTA module.
+ */
+ dma-ranges = <0x42000000 0x0 0x0 0x0 0x0 0x400 0x0>;
+
+ /* This drives busses 0 to 0xf */
+ bus-range = <0x0 0xf>;
+
+ /* Legacy interrupts (note the weird polarity, the bridge seems
+ * to invert PCIe legacy interrupts).
+ * We are de-swizzling here because the numbers are actually for
+ * port of the root complex virtual P2P bridge. But I want
+ * to avoid putting a node for it in the tree, so the numbers
+ * below are basically de-swizzled numbers.
+ * The real slot is on idsel 0, so the swizzling is 1:1
+ */
+ interrupt-map-mask = <0x0 0x0 0x0 0x7>;
+ interrupt-map = <
+ 0x0 0x0 0x0 0x1 &MPIC 61 0x2 /* int A */
+ 0x0 0x0 0x0 0x2 &MPIC 62 0x2 /* int B */
+ 0x0 0x0 0x0 0x3 &MPIC 63 0x2 /* int C */
+ 0x0 0x0 0x0 0x4 &MPIC 64 0x2 /* int D */>;
+ };
+
+ PCIE3: pciex@28100000000 {
+ device_type = "pci";
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ compatible = "ibm,plb-pciex-476fpe", "ibm,plb-pciex";
+ primary;
+ port = <0x3>; /* port number */
+ reg = <0x00000281 0x00000000 0x0 0x10000000 /* Config space access */
+ 0x00000280 0x00000000 0x0 0x00001000>; /* UTL Registers space access */
+ dcr-reg = <0x120 0x20>;
+
+// pci_space < pci_addr > < cpu_addr > < size >
+ ranges = <0x02000000 0x00000000 0x80000000 0x00000290 0x80000000 0x0 0x80000000
+ 0x01000000 0x0 0x0 0x000002c0 0x0 0x0 0x00010000>;
+
+ /* Inbound starting at 0x0 to 0x40000000000. In order to use MSI
+ * PCI devices must be able to write to the HSTA module.
+ */
+ dma-ranges = <0x42000000 0x0 0x0 0x0 0x0 0x400 0x0>;
+
+ /* This drives busses 0 to 0xf */
+ bus-range = <0x0 0xf>;
+
+ /* Legacy interrupts (note the weird polarity, the bridge seems
+ * to invert PCIe legacy interrupts).
+ * We are de-swizzling here because the numbers are actually for
+ * port of the root complex virtual P2P bridge. But I want
+ * to avoid putting a node for it in the tree, so the numbers
+ * below are basically de-swizzled numbers.
+ * The real slot is on idsel 0, so the swizzling is 1:1
+ */
+ interrupt-map-mask = <0x0 0x0 0x0 0x7>;
+ interrupt-map = <
+ 0x0 0x0 0x0 0x1 &MPIC 69 0x2 /* int A */
+ 0x0 0x0 0x0 0x2 &MPIC 70 0x2 /* int B */
+ 0x0 0x0 0x0 0x3 &MPIC 71 0x2 /* int C */
+ 0x0 0x0 0x0 0x4 &MPIC 72 0x2 /* int D */>;
+ };
+ };
+
+ chosen {
+ linux,stdout-path = &UART0;
+ };
+};
diff --git a/arch/powerpc/boot/dts/b4860emu.dts b/arch/powerpc/boot/dts/b4860emu.dts
index 7290021..85646b4 100644
--- a/arch/powerpc/boot/dts/b4860emu.dts
+++ b/arch/powerpc/boot/dts/b4860emu.dts
@@ -61,21 +61,25 @@
device_type = "cpu";
reg = <0 1>;
next-level-cache = <&L2>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu1: PowerPC,e6500@2 {
device_type = "cpu";
reg = <2 3>;
next-level-cache = <&L2>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu2: PowerPC,e6500@4 {
device_type = "cpu";
reg = <4 5>;
next-level-cache = <&L2>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu3: PowerPC,e6500@6 {
device_type = "cpu";
reg = <6 7>;
next-level-cache = <&L2>;
+ fsl,portid-mapping = <0x80000000>;
};
};
};
@@ -157,7 +161,7 @@
};
corenet-cf@18000 {
- compatible = "fsl,b4-corenet-cf";
+ compatible = "fsl,corenet2-cf", "fsl,corenet-cf";
reg = <0x18000 0x1000>;
interrupts = <16 2 1 0>;
fsl,ccf-num-csdids = <32>;
@@ -167,6 +171,7 @@
iommu@20000 {
compatible = "fsl,pamu-v1.0", "fsl,pamu";
reg = <0x20000 0x4000>;
+ fsl,portid-mapping = <0x8000>;
#address-cells = <1>;
#size-cells = <1>;
interrupts = <
diff --git a/arch/powerpc/boot/dts/bsc9132qds.dts b/arch/powerpc/boot/dts/bsc9132qds.dts
new file mode 100644
index 0000000..6cab106
--- /dev/null
+++ b/arch/powerpc/boot/dts/bsc9132qds.dts
@@ -0,0 +1,35 @@
+/*
+ * BSC9132 QDS Device Tree Source
+ *
+ * Copyright 2014 Freescale Semiconductor Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+/include/ "fsl/bsc9132si-pre.dtsi"
+
+/ {
+ model = "fsl,bsc9132qds";
+ compatible = "fsl,bsc9132qds";
+
+ memory {
+ device_type = "memory";
+ };
+
+ ifc: ifc@ff71e000 {
+ /* NOR, NAND Flash on board */
+ ranges = <0x0 0x0 0x0 0x88000000 0x08000000
+ 0x1 0x0 0x0 0xff800000 0x00010000>;
+ reg = <0x0 0xff71e000 0x0 0x2000>;
+ };
+
+ soc: soc@ff700000 {
+ ranges = <0x0 0x0 0xff700000 0x100000>;
+ };
+};
+
+/include/ "bsc9132qds.dtsi"
+/include/ "fsl/bsc9132si-post.dtsi"
diff --git a/arch/powerpc/boot/dts/bsc9132qds.dtsi b/arch/powerpc/boot/dts/bsc9132qds.dtsi
new file mode 100644
index 0000000..af8e888
--- /dev/null
+++ b/arch/powerpc/boot/dts/bsc9132qds.dtsi
@@ -0,0 +1,101 @@
+/*
+ * BSC9132 QDS Device Tree Source stub (no addresses or top-level ranges)
+ *
+ * Copyright 2014 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+&ifc {
+ nor@0,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "cfi-flash";
+ reg = <0x0 0x0 0x8000000>;
+ bank-width = <2>;
+ device-width = <1>;
+ };
+
+ nand@1,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "fsl,ifc-nand";
+ reg = <0x1 0x0 0x4000>;
+ };
+};
+
+&soc {
+ spi@7000 {
+ flash@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "spansion,s25sl12801";
+ reg = <0>;
+ spi-max-frequency = <30000000>;
+ };
+ };
+
+ i2c@3000 {
+ fpga: fpga@66 {
+ compatible = "fsl,bsc9132qds-fpga", "fsl,fpga-qixis-i2c";
+ reg = <0x66>;
+ };
+ };
+
+ usb@22000 {
+ phy_type = "ulpi";
+ };
+
+ mdio@24000 {
+ phy0: ethernet-phy@0 {
+ reg = <0x0>;
+ };
+
+ phy1: ethernet-phy@1 {
+ reg = <0x1>;
+ };
+
+ tbi0: tbi-phy@11 {
+ reg = <0x1f>;
+ device_type = "tbi-phy";
+ };
+ };
+
+ enet0: ethernet@b0000 {
+ phy-handle = <&phy0>;
+ tbi-handle = <&tbi0>;
+ phy-connection-type = "sgmii";
+ };
+
+ enet1: ethernet@b1000 {
+ phy-handle = <&phy1>;
+ tbi-handle = <&tbi0>;
+ phy-connection-type = "sgmii";
+ };
+};
diff --git a/arch/powerpc/boot/dts/fsl/b4420si-post.dtsi b/arch/powerpc/boot/dts/fsl/b4420si-post.dtsi
index 60566f99..d678944 100644
--- a/arch/powerpc/boot/dts/fsl/b4420si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/b4420si-post.dtsi
@@ -76,10 +76,6 @@
compatible = "fsl,b4420-l3-cache-controller", "cache";
};
- corenet-cf@18000 {
- compatible = "fsl,b4420-corenet-cf";
- };
-
guts: global-utilities@e0000 {
compatible = "fsl,b4420-device-config", "fsl,qoriq-device-config-2.0";
};
diff --git a/arch/powerpc/boot/dts/fsl/b4420si-pre.dtsi b/arch/powerpc/boot/dts/fsl/b4420si-pre.dtsi
index 2419731..338af7e 100644
--- a/arch/powerpc/boot/dts/fsl/b4420si-pre.dtsi
+++ b/arch/powerpc/boot/dts/fsl/b4420si-pre.dtsi
@@ -66,12 +66,14 @@
reg = <0 1>;
clocks = <&mux0>;
next-level-cache = <&L2>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu1: PowerPC,e6500@2 {
device_type = "cpu";
reg = <2 3>;
clocks = <&mux0>;
next-level-cache = <&L2>;
+ fsl,portid-mapping = <0x80000000>;
};
};
};
diff --git a/arch/powerpc/boot/dts/fsl/b4860si-post.dtsi b/arch/powerpc/boot/dts/fsl/b4860si-post.dtsi
index cbc354b..582381d 100644
--- a/arch/powerpc/boot/dts/fsl/b4860si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/b4860si-post.dtsi
@@ -120,10 +120,6 @@
compatible = "fsl,b4860-l3-cache-controller", "cache";
};
- corenet-cf@18000 {
- compatible = "fsl,b4860-corenet-cf";
- };
-
guts: global-utilities@e0000 {
compatible = "fsl,b4860-device-config", "fsl,qoriq-device-config-2.0";
};
diff --git a/arch/powerpc/boot/dts/fsl/b4860si-pre.dtsi b/arch/powerpc/boot/dts/fsl/b4860si-pre.dtsi
index 142ac86..1948f73 100644
--- a/arch/powerpc/boot/dts/fsl/b4860si-pre.dtsi
+++ b/arch/powerpc/boot/dts/fsl/b4860si-pre.dtsi
@@ -66,24 +66,28 @@
reg = <0 1>;
clocks = <&mux0>;
next-level-cache = <&L2>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu1: PowerPC,e6500@2 {
device_type = "cpu";
reg = <2 3>;
clocks = <&mux0>;
next-level-cache = <&L2>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu2: PowerPC,e6500@4 {
device_type = "cpu";
reg = <4 5>;
clocks = <&mux0>;
next-level-cache = <&L2>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu3: PowerPC,e6500@6 {
device_type = "cpu";
reg = <6 7>;
clocks = <&mux0>;
next-level-cache = <&L2>;
+ fsl,portid-mapping = <0x80000000>;
};
};
};
diff --git a/arch/powerpc/boot/dts/fsl/b4si-post.dtsi b/arch/powerpc/boot/dts/fsl/b4si-post.dtsi
index 4f6e482..1a54ba7 100644
--- a/arch/powerpc/boot/dts/fsl/b4si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/b4si-post.dtsi
@@ -158,7 +158,7 @@
};
corenet-cf@18000 {
- compatible = "fsl,b4-corenet-cf";
+ compatible = "fsl,corenet2-cf", "fsl,corenet-cf";
reg = <0x18000 0x1000>;
interrupts = <16 2 1 0>;
fsl,ccf-num-csdids = <32>;
@@ -168,6 +168,7 @@
iommu@20000 {
compatible = "fsl,pamu-v1.0", "fsl,pamu";
reg = <0x20000 0x4000>;
+ fsl,portid-mapping = <0x8000>;
#address-cells = <1>;
#size-cells = <1>;
interrupts = <
diff --git a/arch/powerpc/boot/dts/fsl/bsc9132si-post.dtsi b/arch/powerpc/boot/dts/fsl/bsc9132si-post.dtsi
new file mode 100644
index 0000000..c723071
--- /dev/null
+++ b/arch/powerpc/boot/dts/fsl/bsc9132si-post.dtsi
@@ -0,0 +1,185 @@
+/*
+ * BSC9132 Silicon/SoC Device Tree Source (post include)
+ *
+ * Copyright 2014 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+&ifc {
+ #address-cells = <2>;
+ #size-cells = <1>;
+ compatible = "fsl,ifc", "simple-bus";
+ /* FIXME: Test whether interrupts are split */
+ interrupts = <16 2 0 0 20 2 0 0>;
+};
+
+&soc {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ device_type = "soc";
+ compatible = "fsl,bsc9132-immr", "simple-bus";
+ bus-frequency = <0>; // Filled out by uboot.
+
+ ecm-law@0 {
+ compatible = "fsl,ecm-law";
+ reg = <0x0 0x1000>;
+ fsl,num-laws = <12>;
+ };
+
+ ecm@1000 {
+ compatible = "fsl,bsc9132-ecm", "fsl,ecm";
+ reg = <0x1000 0x1000>;
+ interrupts = <16 2 0 0>;
+ };
+
+ memory-controller@2000 {
+ compatible = "fsl,bsc9132-memory-controller";
+ reg = <0x2000 0x1000>;
+ interrupts = <16 2 1 8>;
+ };
+
+/include/ "pq3-i2c-0.dtsi"
+ i2c@3000 {
+ interrupts = <17 2 0 0>;
+ };
+
+/include/ "pq3-i2c-1.dtsi"
+ i2c@3100 {
+ interrupts = <17 2 0 0>;
+ };
+
+/include/ "pq3-duart-0.dtsi"
+ serial0: serial@4500 {
+ interrupts = <18 2 0 0>;
+ };
+
+ serial1: serial@4600 {
+ interrupts = <18 2 0 0 >;
+ };
+/include/ "pq3-espi-0.dtsi"
+ spi0: spi@7000 {
+ fsl,espi-num-chipselects = <1>;
+ interrupts = <22 0x2 0 0>;
+ };
+
+/include/ "pq3-gpio-0.dtsi"
+ gpio-controller@f000 {
+ interrupts = <19 0x2 0 0>;
+ };
+
+ L2: l2-cache-controller@20000 {
+ compatible = "fsl,bsc9132-l2-cache-controller";
+ reg = <0x20000 0x1000>;
+ cache-line-size = <32>; // 32 bytes
+ cache-size = <0x40000>; // L2,256K
+ interrupts = <16 2 1 0>;
+ };
+
+/include/ "pq3-dma-0.dtsi"
+
+dma@21300 {
+
+ dma-channel@0 {
+ interrupts = <62 2 0 0>;
+ };
+
+ dma-channel@80 {
+ interrupts = <63 2 0 0>;
+ };
+
+ dma-channel@100 {
+ interrupts = <64 2 0 0>;
+ };
+
+ dma-channel@180 {
+ interrupts = <65 2 0 0>;
+ };
+};
+
+/include/ "pq3-usb2-dr-0.dtsi"
+usb@22000 {
+ compatible = "fsl-usb2-dr","fsl-usb2-dr-v2.2";
+ interrupts = <40 0x2 0 0>;
+};
+
+/include/ "pq3-esdhc-0.dtsi"
+ sdhc@2e000 {
+ fsl,sdhci-auto-cmd12;
+ interrupts = <41 0x2 0 0>;
+ };
+
+/include/ "pq3-sec4.4-0.dtsi"
+crypto@30000 {
+ interrupts = <57 2 0 0>;
+
+ sec_jr0: jr@1000 {
+ interrupts = <58 2 0 0>;
+ };
+
+ sec_jr1: jr@2000 {
+ interrupts = <59 2 0 0>;
+ };
+
+ sec_jr2: jr@3000 {
+ interrupts = <60 2 0 0>;
+ };
+
+ sec_jr3: jr@4000 {
+ interrupts = <61 2 0 0>;
+ };
+};
+
+/include/ "pq3-mpic.dtsi"
+/include/ "pq3-mpic-timer-B.dtsi"
+
+/include/ "pq3-etsec2-0.dtsi"
+enet0: ethernet@b0000 {
+ queue-group@b0000 {
+ fsl,rx-bit-map = <0xff>;
+ fsl,tx-bit-map = <0xff>;
+ interrupts = <26 2 0 0 27 2 0 0 28 2 0 0>;
+ };
+};
+
+/include/ "pq3-etsec2-1.dtsi"
+enet1: ethernet@b1000 {
+ queue-group@b1000 {
+ fsl,rx-bit-map = <0xff>;
+ fsl,tx-bit-map = <0xff>;
+ interrupts = <33 2 0 0 34 2 0 0 35 2 0 0>;
+ };
+};
+
+global-utilities@e0000 {
+ compatible = "fsl,bsc9132-guts";
+ reg = <0xe0000 0x1000>;
+ fsl,has-rstcr;
+ };
+};
diff --git a/arch/powerpc/boot/dts/fsl/bsc9132si-pre.dtsi b/arch/powerpc/boot/dts/fsl/bsc9132si-pre.dtsi
new file mode 100644
index 0000000..301a9db
--- /dev/null
+++ b/arch/powerpc/boot/dts/fsl/bsc9132si-pre.dtsi
@@ -0,0 +1,66 @@
+/*
+ * BSC9132 Silicon/SoC Device Tree Source (pre include)
+ *
+ * Copyright 2014 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/dts-v1/;
+
+/include/ "e500v2_power_isa.dtsi"
+
+/ {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+
+ aliases {
+ serial0 = &serial0;
+ ethernet0 = &enet0;
+ ethernet1 = &enet1;
+ };
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ cpu0: PowerPC,e500v2@0 {
+ device_type = "cpu";
+ reg = <0x0>;
+ next-level-cache = <&L2>;
+ };
+
+ cpu1: PowerPC,e500v2@1 {
+ device_type = "cpu";
+ reg = <0x1>;
+ next-level-cache = <&L2>;
+ };
+ };
+};
diff --git a/arch/powerpc/boot/dts/fsl/p2041si-post.dtsi b/arch/powerpc/boot/dts/fsl/p2041si-post.dtsi
index e2987a3..5290df8 100644
--- a/arch/powerpc/boot/dts/fsl/p2041si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/p2041si-post.dtsi
@@ -246,7 +246,7 @@
};
corenet-cf@18000 {
- compatible = "fsl,corenet-cf";
+ compatible = "fsl,corenet1-cf", "fsl,corenet-cf";
reg = <0x18000 0x1000>;
interrupts = <16 2 1 31>;
fsl,ccf-num-csdids = <32>;
@@ -262,6 +262,7 @@
interrupts = <
24 2 0 0
16 2 1 30>;
+ fsl,portid-mapping = <0x0f000000>;
pamu0: pamu@0 {
reg = <0 0x1000>;
diff --git a/arch/powerpc/boot/dts/fsl/p2041si-pre.dtsi b/arch/powerpc/boot/dts/fsl/p2041si-pre.dtsi
index 22f3b14..b1ea147 100644
--- a/arch/powerpc/boot/dts/fsl/p2041si-pre.dtsi
+++ b/arch/powerpc/boot/dts/fsl/p2041si-pre.dtsi
@@ -83,6 +83,7 @@
reg = <0>;
clocks = <&mux0>;
next-level-cache = <&L2_0>;
+ fsl,portid-mapping = <0x80000000>;
L2_0: l2-cache {
next-level-cache = <&cpc>;
};
@@ -92,6 +93,7 @@
reg = <1>;
clocks = <&mux1>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x40000000>;
L2_1: l2-cache {
next-level-cache = <&cpc>;
};
@@ -101,6 +103,7 @@
reg = <2>;
clocks = <&mux2>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x20000000>;
L2_2: l2-cache {
next-level-cache = <&cpc>;
};
@@ -110,6 +113,7 @@
reg = <3>;
clocks = <&mux3>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x10000000>;
L2_3: l2-cache {
next-level-cache = <&cpc>;
};
diff --git a/arch/powerpc/boot/dts/fsl/p3041si-post.dtsi b/arch/powerpc/boot/dts/fsl/p3041si-post.dtsi
index 7af6d45f..cd63cb1 100644
--- a/arch/powerpc/boot/dts/fsl/p3041si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/p3041si-post.dtsi
@@ -273,7 +273,7 @@
};
corenet-cf@18000 {
- compatible = "fsl,corenet-cf";
+ compatible = "fsl,corenet1-cf", "fsl,corenet-cf";
reg = <0x18000 0x1000>;
interrupts = <16 2 1 31>;
fsl,ccf-num-csdids = <32>;
@@ -289,6 +289,7 @@
interrupts = <
24 2 0 0
16 2 1 30>;
+ fsl,portid-mapping = <0x0f000000>;
pamu0: pamu@0 {
reg = <0 0x1000>;
diff --git a/arch/powerpc/boot/dts/fsl/p3041si-pre.dtsi b/arch/powerpc/boot/dts/fsl/p3041si-pre.dtsi
index 468e8be..dc5f4b3 100644
--- a/arch/powerpc/boot/dts/fsl/p3041si-pre.dtsi
+++ b/arch/powerpc/boot/dts/fsl/p3041si-pre.dtsi
@@ -84,6 +84,7 @@
reg = <0>;
clocks = <&mux0>;
next-level-cache = <&L2_0>;
+ fsl,portid-mapping = <0x80000000>;
L2_0: l2-cache {
next-level-cache = <&cpc>;
};
@@ -93,6 +94,7 @@
reg = <1>;
clocks = <&mux1>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x40000000>;
L2_1: l2-cache {
next-level-cache = <&cpc>;
};
@@ -102,6 +104,7 @@
reg = <2>;
clocks = <&mux2>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x20000000>;
L2_2: l2-cache {
next-level-cache = <&cpc>;
};
@@ -111,6 +114,7 @@
reg = <3>;
clocks = <&mux3>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x10000000>;
L2_3: l2-cache {
next-level-cache = <&cpc>;
};
diff --git a/arch/powerpc/boot/dts/fsl/p4080si-post.dtsi b/arch/powerpc/boot/dts/fsl/p4080si-post.dtsi
index 2415e1f..12947cc 100644
--- a/arch/powerpc/boot/dts/fsl/p4080si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/p4080si-post.dtsi
@@ -281,7 +281,7 @@
};
corenet-cf@18000 {
- compatible = "fsl,corenet-cf";
+ compatible = "fsl,corenet1-cf", "fsl,corenet-cf";
reg = <0x18000 0x1000>;
interrupts = <16 2 1 31>;
fsl,ccf-num-csdids = <32>;
@@ -297,6 +297,7 @@
interrupts = <
24 2 0 0
16 2 1 30>;
+ fsl,portid-mapping = <0x00f80000>;
pamu0: pamu@0 {
reg = <0 0x1000>;
diff --git a/arch/powerpc/boot/dts/fsl/p4080si-pre.dtsi b/arch/powerpc/boot/dts/fsl/p4080si-pre.dtsi
index 0040b5a5..38bde09 100644
--- a/arch/powerpc/boot/dts/fsl/p4080si-pre.dtsi
+++ b/arch/powerpc/boot/dts/fsl/p4080si-pre.dtsi
@@ -83,6 +83,7 @@
reg = <0>;
clocks = <&mux0>;
next-level-cache = <&L2_0>;
+ fsl,portid-mapping = <0x80000000>;
L2_0: l2-cache {
next-level-cache = <&cpc>;
};
@@ -92,6 +93,7 @@
reg = <1>;
clocks = <&mux1>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x40000000>;
L2_1: l2-cache {
next-level-cache = <&cpc>;
};
@@ -101,6 +103,7 @@
reg = <2>;
clocks = <&mux2>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x20000000>;
L2_2: l2-cache {
next-level-cache = <&cpc>;
};
@@ -110,6 +113,7 @@
reg = <3>;
clocks = <&mux3>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x10000000>;
L2_3: l2-cache {
next-level-cache = <&cpc>;
};
@@ -119,6 +123,7 @@
reg = <4>;
clocks = <&mux4>;
next-level-cache = <&L2_4>;
+ fsl,portid-mapping = <0x08000000>;
L2_4: l2-cache {
next-level-cache = <&cpc>;
};
@@ -128,6 +133,7 @@
reg = <5>;
clocks = <&mux5>;
next-level-cache = <&L2_5>;
+ fsl,portid-mapping = <0x04000000>;
L2_5: l2-cache {
next-level-cache = <&cpc>;
};
@@ -137,6 +143,7 @@
reg = <6>;
clocks = <&mux6>;
next-level-cache = <&L2_6>;
+ fsl,portid-mapping = <0x02000000>;
L2_6: l2-cache {
next-level-cache = <&cpc>;
};
@@ -146,6 +153,7 @@
reg = <7>;
clocks = <&mux7>;
next-level-cache = <&L2_7>;
+ fsl,portid-mapping = <0x01000000>;
L2_7: l2-cache {
next-level-cache = <&cpc>;
};
diff --git a/arch/powerpc/boot/dts/fsl/p5020si-post.dtsi b/arch/powerpc/boot/dts/fsl/p5020si-post.dtsi
index 2985de4..4c4a2b0 100644
--- a/arch/powerpc/boot/dts/fsl/p5020si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/p5020si-post.dtsi
@@ -278,7 +278,7 @@
};
corenet-cf@18000 {
- compatible = "fsl,corenet-cf";
+ compatible = "fsl,corenet1-cf", "fsl,corenet-cf";
reg = <0x18000 0x1000>;
interrupts = <16 2 1 31>;
fsl,ccf-num-csdids = <32>;
@@ -294,6 +294,7 @@
interrupts = <
24 2 0 0
16 2 1 30>;
+ fsl,portid-mapping = <0x3c000000>;
pamu0: pamu@0 {
reg = <0 0x1000>;
diff --git a/arch/powerpc/boot/dts/fsl/p5020si-pre.dtsi b/arch/powerpc/boot/dts/fsl/p5020si-pre.dtsi
index fe1a2e6..1cc61e1 100644
--- a/arch/powerpc/boot/dts/fsl/p5020si-pre.dtsi
+++ b/arch/powerpc/boot/dts/fsl/p5020si-pre.dtsi
@@ -90,6 +90,7 @@
reg = <0>;
clocks = <&mux0>;
next-level-cache = <&L2_0>;
+ fsl,portid-mapping = <0x80000000>;
L2_0: l2-cache {
next-level-cache = <&cpc>;
};
@@ -99,6 +100,7 @@
reg = <1>;
clocks = <&mux1>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x40000000>;
L2_1: l2-cache {
next-level-cache = <&cpc>;
};
diff --git a/arch/powerpc/boot/dts/fsl/p5040si-post.dtsi b/arch/powerpc/boot/dts/fsl/p5040si-post.dtsi
index 546a899..67296fd 100644
--- a/arch/powerpc/boot/dts/fsl/p5040si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/p5040si-post.dtsi
@@ -233,7 +233,7 @@
};
corenet-cf@18000 {
- compatible = "fsl,corenet-cf";
+ compatible = "fsl,corenet1-cf", "fsl,corenet-cf";
reg = <0x18000 0x1000>;
interrupts = <16 2 1 31>;
fsl,ccf-num-csdids = <32>;
@@ -248,6 +248,7 @@
#size-cells = <1>;
interrupts = <24 2 0 0
16 2 1 30>;
+ fsl,portid-mapping = <0x0f800000>;
pamu0: pamu@0 {
reg = <0 0x1000>;
diff --git a/arch/powerpc/boot/dts/fsl/p5040si-pre.dtsi b/arch/powerpc/boot/dts/fsl/p5040si-pre.dtsi
index 3674686..b048a2b 100644
--- a/arch/powerpc/boot/dts/fsl/p5040si-pre.dtsi
+++ b/arch/powerpc/boot/dts/fsl/p5040si-pre.dtsi
@@ -83,6 +83,7 @@
reg = <0>;
clocks = <&mux0>;
next-level-cache = <&L2_0>;
+ fsl,portid-mapping = <0x80000000>;
L2_0: l2-cache {
next-level-cache = <&cpc>;
};
@@ -92,6 +93,7 @@
reg = <1>;
clocks = <&mux1>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x40000000>;
L2_1: l2-cache {
next-level-cache = <&cpc>;
};
@@ -101,6 +103,7 @@
reg = <2>;
clocks = <&mux2>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x20000000>;
L2_2: l2-cache {
next-level-cache = <&cpc>;
};
@@ -110,6 +113,7 @@
reg = <3>;
clocks = <&mux3>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x10000000>;
L2_3: l2-cache {
next-level-cache = <&cpc>;
};
diff --git a/arch/powerpc/boot/dts/fsl/t1040si-post.dtsi b/arch/powerpc/boot/dts/fsl/t1040si-post.dtsi
new file mode 100644
index 0000000..12e597e
--- /dev/null
+++ b/arch/powerpc/boot/dts/fsl/t1040si-post.dtsi
@@ -0,0 +1,430 @@
+/*
+ * T1040 Silicon/SoC Device Tree Source (post include)
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+&ifc {
+ #address-cells = <2>;
+ #size-cells = <1>;
+ compatible = "fsl,ifc", "simple-bus";
+ interrupts = <25 2 0 0>;
+};
+
+&pci0 {
+ compatible = "fsl,t1040-pcie", "fsl,qoriq-pcie-v2.4", "fsl,qoriq-pcie";
+ device_type = "pci";
+ #size-cells = <2>;
+ #address-cells = <3>;
+ bus-range = <0x0 0xff>;
+ interrupts = <20 2 0 0>;
+ fsl,iommu-parent = <&pamu0>;
+ pcie@0 {
+ reg = <0 0 0 0 0>;
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ device_type = "pci";
+ interrupts = <20 2 0 0>;
+ interrupt-map-mask = <0xf800 0 0 7>;
+ interrupt-map = <
+ /* IDSEL 0x0 */
+ 0000 0 0 1 &mpic 40 1 0 0
+ 0000 0 0 2 &mpic 1 1 0 0
+ 0000 0 0 3 &mpic 2 1 0 0
+ 0000 0 0 4 &mpic 3 1 0 0
+ >;
+ };
+};
+
+&pci1 {
+ compatible = "fsl,t1040-pcie", "fsl,qoriq-pcie-v2.4", "fsl,qoriq-pcie";
+ device_type = "pci";
+ #size-cells = <2>;
+ #address-cells = <3>;
+ bus-range = <0 0xff>;
+ interrupts = <21 2 0 0>;
+ fsl,iommu-parent = <&pamu0>;
+ pcie@0 {
+ reg = <0 0 0 0 0>;
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ device_type = "pci";
+ interrupts = <21 2 0 0>;
+ interrupt-map-mask = <0xf800 0 0 7>;
+ interrupt-map = <
+ /* IDSEL 0x0 */
+ 0000 0 0 1 &mpic 41 1 0 0
+ 0000 0 0 2 &mpic 5 1 0 0
+ 0000 0 0 3 &mpic 6 1 0 0
+ 0000 0 0 4 &mpic 7 1 0 0
+ >;
+ };
+};
+
+&pci2 {
+ compatible = "fsl,t1040-pcie", "fsl,qoriq-pcie-v2.4", "fsl,qoriq-pcie";
+ device_type = "pci";
+ #size-cells = <2>;
+ #address-cells = <3>;
+ bus-range = <0x0 0xff>;
+ interrupts = <22 2 0 0>;
+ fsl,iommu-parent = <&pamu0>;
+ pcie@0 {
+ reg = <0 0 0 0 0>;
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ device_type = "pci";
+ interrupts = <22 2 0 0>;
+ interrupt-map-mask = <0xf800 0 0 7>;
+ interrupt-map = <
+ /* IDSEL 0x0 */
+ 0000 0 0 1 &mpic 42 1 0 0
+ 0000 0 0 2 &mpic 9 1 0 0
+ 0000 0 0 3 &mpic 10 1 0 0
+ 0000 0 0 4 &mpic 11 1 0 0
+ >;
+ };
+};
+
+&pci3 {
+ compatible = "fsl,t1040-pcie", "fsl,qoriq-pcie-v2.4", "fsl,qoriq-pcie";
+ device_type = "pci";
+ #size-cells = <2>;
+ #address-cells = <3>;
+ bus-range = <0x0 0xff>;
+ interrupts = <23 2 0 0>;
+ fsl,iommu-parent = <&pamu0>;
+ pcie@0 {
+ reg = <0 0 0 0 0>;
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ device_type = "pci";
+ interrupts = <23 2 0 0>;
+ interrupt-map-mask = <0xf800 0 0 7>;
+ interrupt-map = <
+ /* IDSEL 0x0 */
+ 0000 0 0 1 &mpic 43 1 0 0
+ 0000 0 0 2 &mpic 0 1 0 0
+ 0000 0 0 3 &mpic 4 1 0 0
+ 0000 0 0 4 &mpic 8 1 0 0
+ >;
+ };
+};
+
+&dcsr {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "fsl,dcsr", "simple-bus";
+
+ dcsr-epu@0 {
+ compatible = "fsl,t1040-dcsr-epu", "fsl,dcsr-epu";
+ interrupts = <52 2 0 0
+ 84 2 0 0
+ 85 2 0 0>;
+ reg = <0x0 0x1000>;
+ };
+ dcsr-npc {
+ compatible = "fsl,t1040-dcsr-cnpc", "fsl,dcsr-cnpc";
+ reg = <0x1000 0x1000 0x1002000 0x10000>;
+ };
+ dcsr-nxc@2000 {
+ compatible = "fsl,dcsr-nxc";
+ reg = <0x2000 0x1000>;
+ };
+ dcsr-corenet {
+ compatible = "fsl,dcsr-corenet";
+ reg = <0x8000 0x1000 0x1A000 0x1000>;
+ };
+ dcsr-dpaa@9000 {
+ compatible = "fsl,t1040-dcsr-dpaa", "fsl,dcsr-dpaa";
+ reg = <0x9000 0x1000>;
+ };
+ dcsr-ocn@11000 {
+ compatible = "fsl,t1040-dcsr-ocn", "fsl,dcsr-ocn";
+ reg = <0x11000 0x1000>;
+ };
+ dcsr-ddr@12000 {
+ compatible = "fsl,dcsr-ddr";
+ dev-handle = <&ddr1>;
+ reg = <0x12000 0x1000>;
+ };
+ dcsr-nal@18000 {
+ compatible = "fsl,t1040-dcsr-nal", "fsl,dcsr-nal";
+ reg = <0x18000 0x1000>;
+ };
+ dcsr-rcpm@22000 {
+ compatible = "fsl,t1040-dcsr-rcpm", "fsl,dcsr-rcpm";
+ reg = <0x22000 0x1000>;
+ };
+ dcsr-snpc@30000 {
+ compatible = "fsl,t1040-dcsr-snpc", "fsl,dcsr-snpc";
+ reg = <0x30000 0x1000 0x1022000 0x10000>;
+ };
+ dcsr-snpc@31000 {
+ compatible = "fsl,t1040-dcsr-snpc", "fsl,dcsr-snpc";
+ reg = <0x31000 0x1000 0x1042000 0x10000>;
+ };
+ dcsr-cpu-sb-proxy@100000 {
+ compatible = "fsl,dcsr-e5500-sb-proxy", "fsl,dcsr-cpu-sb-proxy";
+ cpu-handle = <&cpu0>;
+ reg = <0x100000 0x1000 0x101000 0x1000>;
+ };
+ dcsr-cpu-sb-proxy@108000 {
+ compatible = "fsl,dcsr-e5500-sb-proxy", "fsl,dcsr-cpu-sb-proxy";
+ cpu-handle = <&cpu1>;
+ reg = <0x108000 0x1000 0x109000 0x1000>;
+ };
+ dcsr-cpu-sb-proxy@110000 {
+ compatible = "fsl,dcsr-e5500-sb-proxy", "fsl,dcsr-cpu-sb-proxy";
+ cpu-handle = <&cpu2>;
+ reg = <0x110000 0x1000 0x111000 0x1000>;
+ };
+ dcsr-cpu-sb-proxy@118000 {
+ compatible = "fsl,dcsr-e5500-sb-proxy", "fsl,dcsr-cpu-sb-proxy";
+ cpu-handle = <&cpu3>;
+ reg = <0x118000 0x1000 0x119000 0x1000>;
+ };
+};
+
+&soc {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ device_type = "soc";
+ compatible = "simple-bus";
+
+ soc-sram-error {
+ compatible = "fsl,soc-sram-error";
+ interrupts = <16 2 1 29>;
+ };
+
+ corenet-law@0 {
+ compatible = "fsl,corenet-law";
+ reg = <0x0 0x1000>;
+ fsl,num-laws = <16>;
+ };
+
+ ddr1: memory-controller@8000 {
+ compatible = "fsl,qoriq-memory-controller-v5.0",
+ "fsl,qoriq-memory-controller";
+ reg = <0x8000 0x1000>;
+ interrupts = <16 2 1 23>;
+ };
+
+ cpc: l3-cache-controller@10000 {
+ compatible = "fsl,t1040-l3-cache-controller", "cache";
+ reg = <0x10000 0x1000>;
+ interrupts = <16 2 1 27>;
+ };
+
+ corenet-cf@18000 {
+ compatible = "fsl,corenet2-cf", "fsl,corenet-cf";
+ reg = <0x18000 0x1000>;
+ interrupts = <16 2 1 31>;
+ fsl,ccf-num-csdids = <32>;
+ fsl,ccf-num-snoopids = <32>;
+ };
+
+ iommu@20000 {
+ compatible = "fsl,pamu-v1.0", "fsl,pamu";
+ reg = <0x20000 0x1000>;
+ ranges = <0 0x20000 0x1000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ interrupts = <
+ 24 2 0 0
+ 16 2 1 30>;
+ pamu0: pamu@0 {
+ reg = <0 0x1000>;
+ fsl,primary-cache-geometry = <128 1>;
+ fsl,secondary-cache-geometry = <16 2>;
+ };
+ };
+
+/include/ "qoriq-mpic.dtsi"
+
+ guts: global-utilities@e0000 {
+ compatible = "fsl,t1040-device-config", "fsl,qoriq-device-config-2.0";
+ reg = <0xe0000 0xe00>;
+ fsl,has-rstcr;
+ fsl,liodn-bits = <12>;
+ };
+
+ clockgen: global-utilities@e1000 {
+ compatible = "fsl,t1040-clockgen", "fsl,qoriq-clockgen-2.0";
+ ranges = <0x0 0xe1000 0x1000>;
+ reg = <0xe1000 0x1000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ sysclk: sysclk {
+ #clock-cells = <0>;
+ compatible = "fsl,qoriq-sysclk-2.0";
+ clock-output-names = "sysclk", "fixed-clock";
+ };
+
+
+ pll0: pll0@800 {
+ #clock-cells = <1>;
+ reg = <0x800 4>;
+ compatible = "fsl,qoriq-core-pll-2.0";
+ clocks = <&sysclk>;
+ clock-output-names = "pll0", "pll0-div2", "pll0-div4";
+ };
+
+ pll1: pll1@820 {
+ #clock-cells = <1>;
+ reg = <0x820 4>;
+ compatible = "fsl,qoriq-core-pll-2.0";
+ clocks = <&sysclk>;
+ clock-output-names = "pll1", "pll1-div2", "pll1-div4";
+ };
+
+ mux0: mux0@0 {
+ #clock-cells = <0>;
+ reg = <0x0 4>;
+ compatible = "fsl,qoriq-core-mux-2.0";
+ clocks = <&pll0 0>, <&pll0 1>, <&pll0 2>,
+ <&pll1 0>, <&pll1 1>, <&pll1 2>;
+ clock-names = "pll0", "pll0-div2", "pll1-div4",
+ "pll1", "pll1-div2", "pll1-div4";
+ clock-output-names = "cmux0";
+ };
+
+ mux1: mux1@20 {
+ #clock-cells = <0>;
+ reg = <0x20 4>;
+ compatible = "fsl,qoriq-core-mux-2.0";
+ clocks = <&pll0 0>, <&pll0 1>, <&pll0 2>,
+ <&pll1 0>, <&pll1 1>, <&pll1 2>;
+ clock-names = "pll0", "pll0-div2", "pll1-div4",
+ "pll1", "pll1-div2", "pll1-div4";
+ clock-output-names = "cmux1";
+ };
+
+ mux2: mux2@40 {
+ #clock-cells = <0>;
+ reg = <0x40 4>;
+ compatible = "fsl,qoriq-core-mux-2.0";
+ clocks = <&pll0 0>, <&pll0 1>, <&pll0 2>,
+ <&pll1 0>, <&pll1 1>, <&pll1 2>;
+ clock-names = "pll0", "pll0-div2", "pll1-div4",
+ "pll1", "pll1-div2", "pll1-div4";
+ clock-output-names = "cmux2";
+ };
+
+ mux3: mux3@60 {
+ #clock-cells = <0>;
+ reg = <0x60 4>;
+ compatible = "fsl,qoriq-core-mux-2.0";
+ clocks = <&pll0 0>, <&pll0 1>, <&pll0 2>,
+ <&pll1 0>, <&pll1 1>, <&pll1 2>;
+ clock-names = "pll0_0", "pll0_1", "pll0_2",
+ "pll1_0", "pll1_1", "pll1_2";
+ clock-output-names = "cmux3";
+ };
+ };
+
+ rcpm: global-utilities@e2000 {
+ compatible = "fsl,t1040-rcpm", "fsl,qoriq-rcpm-2.0";
+ reg = <0xe2000 0x1000>;
+ };
+
+ sfp: sfp@e8000 {
+ compatible = "fsl,t1040-sfp";
+ reg = <0xe8000 0x1000>;
+ };
+
+ serdes: serdes@ea000 {
+ compatible = "fsl,t1040-serdes";
+ reg = <0xea000 0x4000>;
+ };
+
+/include/ "elo3-dma-0.dtsi"
+/include/ "elo3-dma-1.dtsi"
+/include/ "qoriq-espi-0.dtsi"
+ spi@110000 {
+ fsl,espi-num-chipselects = <4>;
+ };
+
+/include/ "qoriq-esdhc-0.dtsi"
+ sdhc@114000 {
+ compatible = "fsl,t1040-esdhc", "fsl,esdhc";
+ fsl,iommu-parent = <&pamu0>;
+ fsl,liodn-reg = <&guts 0x530>; /* eSDHCLIODNR */
+ sdhci,auto-cmd12;
+ };
+/include/ "qoriq-i2c-0.dtsi"
+/include/ "qoriq-i2c-1.dtsi"
+/include/ "qoriq-duart-0.dtsi"
+/include/ "qoriq-duart-1.dtsi"
+/include/ "qoriq-gpio-0.dtsi"
+/include/ "qoriq-gpio-1.dtsi"
+/include/ "qoriq-gpio-2.dtsi"
+/include/ "qoriq-gpio-3.dtsi"
+/include/ "qoriq-usb2-mph-0.dtsi"
+ usb0: usb@210000 {
+ compatible = "fsl-usb2-mph-v2.4", "fsl-usb2-mph";
+ fsl,iommu-parent = <&pamu0>;
+ fsl,liodn-reg = <&guts 0x520>; /* USB1LIODNR */
+ phy_type = "utmi";
+ port0;
+ };
+/include/ "qoriq-usb2-dr-0.dtsi"
+ usb1: usb@211000 {
+ compatible = "fsl-usb2-dr-v2.4", "fsl-usb2-dr";
+ fsl,iommu-parent = <&pamu0>;
+ fsl,liodn-reg = <&guts 0x524>; /* USB2LIODNR */
+ dr_mode = "host";
+ phy_type = "utmi";
+ };
+
+ display@180000 {
+ compatible = "fsl,t1040-diu", "fsl,diu";
+ reg = <0x180000 1000>;
+ interrupts = <74 2 0 0>;
+ };
+
+/include/ "qoriq-sata2-0.dtsi"
+ sata@220000 {
+ fsl,iommu-parent = <&pamu0>;
+ fsl,liodn-reg = <&guts 0x550>; /* SATA1LIODNR */
+ };
+/include/ "qoriq-sata2-1.dtsi"
+ sata@221000 {
+ fsl,iommu-parent = <&pamu0>;
+ fsl,liodn-reg = <&guts 0x554>; /* SATA2LIODNR */
+ };
+/include/ "qoriq-sec5.0-0.dtsi"
+};
diff --git a/arch/powerpc/boot/dts/fsl/t1042si-post.dtsi b/arch/powerpc/boot/dts/fsl/t1042si-post.dtsi
new file mode 100644
index 0000000..319b74f
--- /dev/null
+++ b/arch/powerpc/boot/dts/fsl/t1042si-post.dtsi
@@ -0,0 +1,37 @@
+/*
+ * T1042 Silicon/SoC Device Tree Source (post include)
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/include/ "t1040si-post.dtsi"
+
+/* Place holder for ethernet related device tree nodes */
diff --git a/arch/powerpc/boot/dts/fsl/t104xsi-pre.dtsi b/arch/powerpc/boot/dts/fsl/t104xsi-pre.dtsi
new file mode 100644
index 0000000..bbb7025
--- /dev/null
+++ b/arch/powerpc/boot/dts/fsl/t104xsi-pre.dtsi
@@ -0,0 +1,104 @@
+/*
+ * T1040/T1042 Silicon/SoC Device Tree Source (pre include)
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/dts-v1/;
+
+/include/ "e5500_power_isa.dtsi"
+
+/ {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+
+ aliases {
+ ccsr = &soc;
+ dcsr = &dcsr;
+
+ serial0 = &serial0;
+ serial1 = &serial1;
+ serial2 = &serial2;
+ serial3 = &serial3;
+ pci0 = &pci0;
+ pci1 = &pci1;
+ pci2 = &pci2;
+ pci3 = &pci3;
+ usb0 = &usb0;
+ usb1 = &usb1;
+ sdhc = &sdhc;
+
+ crypto = &crypto;
+ };
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ cpu0: PowerPC,e5500@0 {
+ device_type = "cpu";
+ reg = <0>;
+ clocks = <&mux0>;
+ next-level-cache = <&L2_1>;
+ L2_1: l2-cache {
+ next-level-cache = <&cpc>;
+ };
+ };
+ cpu1: PowerPC,e5500@1 {
+ device_type = "cpu";
+ reg = <1>;
+ clocks = <&mux1>;
+ next-level-cache = <&L2_2>;
+ L2_2: l2-cache {
+ next-level-cache = <&cpc>;
+ };
+ };
+ cpu2: PowerPC,e5500@2 {
+ device_type = "cpu";
+ reg = <2>;
+ clocks = <&mux2>;
+ next-level-cache = <&L2_3>;
+ L2_3: l2-cache {
+ next-level-cache = <&cpc>;
+ };
+ };
+ cpu3: PowerPC,e5500@3 {
+ device_type = "cpu";
+ reg = <3>;
+ clocks = <&mux3>;
+ next-level-cache = <&L2_4>;
+ L2_4: l2-cache {
+ next-level-cache = <&cpc>;
+ };
+ };
+ };
+};
diff --git a/arch/powerpc/boot/dts/fsl/t4240si-post.dtsi b/arch/powerpc/boot/dts/fsl/t4240si-post.dtsi
index f99d74f..793669b 100644
--- a/arch/powerpc/boot/dts/fsl/t4240si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/t4240si-post.dtsi
@@ -343,7 +343,7 @@
};
corenet-cf@18000 {
- compatible = "fsl,corenet-cf";
+ compatible = "fsl,corenet2-cf", "fsl,corenet-cf";
reg = <0x18000 0x1000>;
interrupts = <16 2 1 31>;
fsl,ccf-num-csdids = <32>;
@@ -353,6 +353,7 @@
iommu@20000 {
compatible = "fsl,pamu-v1.0", "fsl,pamu";
reg = <0x20000 0x6000>;
+ fsl,portid-mapping = <0x8000>;
interrupts = <
24 2 0 0
16 2 1 30>;
diff --git a/arch/powerpc/boot/dts/fsl/t4240si-pre.dtsi b/arch/powerpc/boot/dts/fsl/t4240si-pre.dtsi
index 0b8ccc5b4..d2f157e 100644
--- a/arch/powerpc/boot/dts/fsl/t4240si-pre.dtsi
+++ b/arch/powerpc/boot/dts/fsl/t4240si-pre.dtsi
@@ -69,72 +69,84 @@
reg = <0 1>;
clocks = <&mux0>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu1: PowerPC,e6500@2 {
device_type = "cpu";
reg = <2 3>;
clocks = <&mux0>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu2: PowerPC,e6500@4 {
device_type = "cpu";
reg = <4 5>;
clocks = <&mux0>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu3: PowerPC,e6500@6 {
device_type = "cpu";
reg = <6 7>;
clocks = <&mux0>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu4: PowerPC,e6500@8 {
device_type = "cpu";
reg = <8 9>;
clocks = <&mux1>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x40000000>;
};
cpu5: PowerPC,e6500@10 {
device_type = "cpu";
reg = <10 11>;
clocks = <&mux1>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x40000000>;
};
cpu6: PowerPC,e6500@12 {
device_type = "cpu";
reg = <12 13>;
clocks = <&mux1>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x40000000>;
};
cpu7: PowerPC,e6500@14 {
device_type = "cpu";
reg = <14 15>;
clocks = <&mux1>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x40000000>;
};
cpu8: PowerPC,e6500@16 {
device_type = "cpu";
reg = <16 17>;
clocks = <&mux2>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x20000000>;
};
cpu9: PowerPC,e6500@18 {
device_type = "cpu";
reg = <18 19>;
clocks = <&mux2>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x20000000>;
};
cpu10: PowerPC,e6500@20 {
device_type = "cpu";
reg = <20 21>;
clocks = <&mux2>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x20000000>;
};
cpu11: PowerPC,e6500@22 {
device_type = "cpu";
reg = <22 23>;
clocks = <&mux2>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x20000000>;
};
};
};
diff --git a/arch/powerpc/boot/dts/kmcoge4.dts b/arch/powerpc/boot/dts/kmcoge4.dts
new file mode 100644
index 0000000..89b4119
--- /dev/null
+++ b/arch/powerpc/boot/dts/kmcoge4.dts
@@ -0,0 +1,152 @@
+/*
+ * Keymile kmcoge4 Device Tree Source, based on the P2041RDB DTS
+ *
+ * (C) Copyright 2014
+ * Valentin Longchamp, Keymile AG, valentin.longchamp@keymile.com
+ *
+ * Copyright 2011 Freescale Semiconductor Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+/include/ "fsl/p2041si-pre.dtsi"
+
+/ {
+ model = "keymile,kmcoge4";
+ compatible = "keymile,kmcoge4", "keymile,kmp204x";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+
+ memory {
+ device_type = "memory";
+ };
+
+ dcsr: dcsr@f00000000 {
+ ranges = <0x00000000 0xf 0x00000000 0x01008000>;
+ };
+
+ soc: soc@ffe000000 {
+ ranges = <0x00000000 0xf 0xfe000000 0x1000000>;
+ reg = <0xf 0xfe000000 0 0x00001000>;
+ spi@110000 {
+ flash@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "spansion,s25fl256s1";
+ reg = <0>;
+ spi-max-frequency = <20000000>; /* input clock */
+ };
+
+ network_clock@1 {
+ compatible = "zarlink,zl30343";
+ reg = <1>;
+ spi-max-frequency = <8000000>;
+ };
+
+ flash@2 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "micron,m25p32";
+ reg = <2>;
+ spi-max-frequency = <15000000>;
+ };
+ };
+
+ i2c@119000 {
+ status = "disabled";
+ };
+
+ i2c@119100 {
+ status = "disabled";
+ };
+
+ usb0: usb@210000 {
+ status = "disabled";
+ };
+
+ usb1: usb@211000 {
+ status = "disabled";
+ };
+
+ sata@220000 {
+ status = "disabled";
+ };
+
+ sata@221000 {
+ status = "disabled";
+ };
+ };
+
+ rio: rapidio@ffe0c0000 {
+ status = "disabled";
+ };
+
+ lbc: localbus@ffe124000 {
+ reg = <0xf 0xfe124000 0 0x1000>;
+ ranges = <0 0 0xf 0xffa00000 0x00040000 /* LB 0 */
+ 1 0 0xf 0xfb000000 0x00010000 /* LB 1 */
+ 2 0 0xf 0xd0000000 0x10000000 /* LB 2 */
+ 3 0 0xf 0xe0000000 0x10000000>; /* LB 3 */
+
+ nand@0,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "fsl,elbc-fcm-nand";
+ reg = <0 0 0x40000>;
+ };
+
+ board-control@1,0 {
+ compatible = "keymile,qriox";
+ reg = <1 0 0x80>;
+ };
+
+ chassis-mgmt@3,0 {
+ compatible = "keymile,bfticu";
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ reg = <3 0 0x100>;
+ interrupt-parent = <&mpic>;
+ interrupts = <6 1 0 0>;
+ };
+ };
+
+ pci0: pcie@ffe200000 {
+ reg = <0xf 0xfe200000 0 0x1000>;
+ ranges = <0x02000000 0 0xe0000000 0xc 0x00000000 0x0 0x20000000
+ 0x01000000 0 0x00000000 0xf 0xf8000000 0x0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x20000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+
+ pci1: pcie@ffe201000 {
+ status = "disabled";
+ };
+
+ pci2: pcie@ffe202000 {
+ reg = <0xf 0xfe202000 0 0x1000>;
+ ranges = <0x02000000 0 0xe0000000 0xc 0x20000000 0 0x20000000
+ 0x01000000 0 0x00000000 0xf 0xf8010000 0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x20000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+};
+
+/include/ "fsl/p2041si-post.dtsi"
diff --git a/arch/powerpc/boot/dts/mpc8308_p1m.dts b/arch/powerpc/boot/dts/mpc8308_p1m.dts
index 651e4f5..57f86cd 100644
--- a/arch/powerpc/boot/dts/mpc8308_p1m.dts
+++ b/arch/powerpc/boot/dts/mpc8308_p1m.dts
@@ -296,7 +296,7 @@
};
dma@2c000 {
- compatible = "fsl,mpc8308-dma", "fsl,mpc5121-dma";
+ compatible = "fsl,mpc8308-dma";
reg = <0x2c000 0x1800>;
interrupts = <3 0x8
94 0x8>;
diff --git a/arch/powerpc/boot/dts/mpc8308rdb.dts b/arch/powerpc/boot/dts/mpc8308rdb.dts
index 9ce45f2..d0211f0 100644
--- a/arch/powerpc/boot/dts/mpc8308rdb.dts
+++ b/arch/powerpc/boot/dts/mpc8308rdb.dts
@@ -265,7 +265,7 @@
};
dma@2c000 {
- compatible = "fsl,mpc8308-dma", "fsl,mpc5121-dma";
+ compatible = "fsl,mpc8308-dma";
reg = <0x2c000 0x1800>;
interrupts = <3 0x8
94 0x8>;
diff --git a/arch/powerpc/boot/dts/oca4080.dts b/arch/powerpc/boot/dts/oca4080.dts
new file mode 100644
index 0000000..3d4c751
--- /dev/null
+++ b/arch/powerpc/boot/dts/oca4080.dts
@@ -0,0 +1,118 @@
+/*
+ * OCA4080 Device Tree Source
+ *
+ * Copyright 2014 Prodrive Technologies B.V.
+ *
+ * Based on:
+ * P4080DS Device Tree Source
+ * Copyright 2009-2011 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/include/ "fsl/p4080si-pre.dtsi"
+
+/ {
+ model = "fsl,OCA4080";
+ compatible = "fsl,OCA4080";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+
+ memory {
+ device_type = "memory";
+ };
+
+ dcsr: dcsr@f00000000 {
+ ranges = <0x00000000 0xf 0x00000000 0x01008000>;
+ };
+
+ soc: soc@ffe000000 {
+ ranges = <0x00000000 0xf 0xfe000000 0x1000000>;
+ reg = <0xf 0xfe000000 0 0x00001000>;
+
+ i2c@118000 {
+ status = "disabled";
+ };
+
+ i2c@118100 {
+ status = "disabled";
+ };
+
+ i2c@119000 {
+ status = "disabled";
+ };
+
+ i2c@119100 {
+ status = "disabled";
+ };
+
+ usb0: usb@210000 {
+ status = "disabled";
+ };
+
+ usb1: usb@211000 {
+ status = "disabled";
+ };
+ };
+
+ rio: rapidio@ffe0c0000 {
+ reg = <0xf 0xfe0c0000 0 0x11000>;
+
+ port1 {
+ ranges = <0 0 0xc 0x20000000 0 0x10000000>;
+ };
+ };
+
+ lbc: localbus@ffe124000 {
+ reg = <0xf 0xfe124000 0 0x1000>;
+ ranges = <0 0 0xf 0xef800000 0x800000>;
+
+ flash@0,0 {
+ compatible = "cfi-flash";
+ reg = <0 0 0x00800000>;
+ bank-width = <2>;
+ device-width = <2>;
+ };
+ };
+
+ pci0: pcie@ffe200000 {
+ status = "disabled";
+ };
+
+ pci1: pcie@ffe201000 {
+ status = "disabled";
+ };
+
+ pci2: pcie@ffe202000 {
+ status = "disabled";
+ };
+};
+
+/include/ "fsl/p4080si-post.dtsi"
diff --git a/arch/powerpc/boot/dts/p1023rds.dts b/arch/powerpc/boot/dts/p1023rds.dts
deleted file mode 100644
index beb6cb1..0000000
--- a/arch/powerpc/boot/dts/p1023rds.dts
+++ /dev/null
@@ -1,219 +0,0 @@
-/*
- * P1023 RDS Device Tree Source
- *
- * Copyright 2010-2011 Freescale Semiconductor Inc.
- *
- * Author: Roy Zang <tie-fei.zang@freescale.com>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- * * Neither the name of Freescale Semiconductor nor the
- * names of its contributors may be used to endorse or promote products
- * derived from this software without specific prior written permission.
- *
- *
- * ALTERNATIVELY, this software may be distributed under the terms of the
- * GNU General Public License ("GPL") as published by the Free Software
- * Foundation, either version 2 of that License or (at your option) any
- * later version.
- *
- * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY
- * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
- * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
- * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
- * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
- * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
- * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
- * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
- * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-/include/ "fsl/p1023si-pre.dtsi"
-
-/ {
- model = "fsl,P1023";
- compatible = "fsl,P1023RDS";
- #address-cells = <2>;
- #size-cells = <2>;
- interrupt-parent = <&mpic>;
-
- memory {
- device_type = "memory";
- };
-
- soc: soc@ff600000 {
- ranges = <0x0 0x0 0xff600000 0x200000>;
-
- i2c@3000 {
- rtc@68 {
- compatible = "dallas,ds1374";
- reg = <0x68>;
- };
- };
-
- spi@7000 {
- fsl_dataflash@0 {
- #address-cells = <1>;
- #size-cells = <1>;
- compatible = "atmel,at45db081d";
- reg = <0>;
- spi-max-frequency = <40000000>; /* input clock */
- partition@u-boot {
- /* 512KB for u-boot Bootloader Image */
- label = "u-boot-spi";
- reg = <0x00000000 0x00080000>;
- read-only;
- };
- partition@dtb {
- /* 512KB for DTB Image */
- label = "dtb-spi";
- reg = <0x00080000 0x00080000>;
- read-only;
- };
- };
- };
-
- usb@22000 {
- dr_mode = "host";
- phy_type = "ulpi";
- };
- };
-
- lbc: localbus@ff605000 {
- reg = <0 0xff605000 0 0x1000>;
-
- /* NOR Flash, BCSR */
- ranges = <0x0 0x0 0x0 0xee000000 0x02000000
- 0x1 0x0 0x0 0xe0000000 0x00008000>;
-
- nor@0,0 {
- #address-cells = <1>;
- #size-cells = <1>;
- compatible = "cfi-flash";
- reg = <0x0 0x0 0x02000000>;
- bank-width = <2>;
- device-width = <1>;
- partition@0 {
- label = "ramdisk";
- reg = <0x00000000 0x01c00000>;
- };
- partition@1c00000 {
- label = "kernel";
- reg = <0x01c00000 0x002e0000>;
- };
- partiton@1ee0000 {
- label = "dtb";
- reg = <0x01ee0000 0x00020000>;
- };
- partition@1f00000 {
- label = "firmware";
- reg = <0x01f00000 0x00080000>;
- read-only;
- };
- partition@1f80000 {
- label = "u-boot";
- reg = <0x01f80000 0x00080000>;
- read-only;
- };
- };
-
- fpga@1,0 {
- #address-cells = <1>;
- #size-cells = <1>;
- compatible = "fsl,p1023rds-fpga";
- reg = <1 0 0x8000>;
- ranges = <0 1 0 0x8000>;
-
- bcsr@20 {
- compatible = "fsl,p1023rds-bcsr";
- reg = <0x20 0x20>;
- };
- };
- };
-
- pci0: pcie@ff60a000 {
- reg = <0 0xff60a000 0 0x1000>;
- ranges = <0x2000000 0x0 0xc0000000 0 0xc0000000 0x0 0x20000000
- 0x1000000 0x0 0x00000000 0 0xffc20000 0x0 0x10000>;
- pcie@0 {
- /* IRQ[0:3] are pulled up on board, set to active-low */
- interrupt-map-mask = <0xf800 0 0 7>;
- interrupt-map = <
- /* IDSEL 0x0 */
- 0000 0 0 1 &mpic 0 1 0 0
- 0000 0 0 2 &mpic 1 1 0 0
- 0000 0 0 3 &mpic 2 1 0 0
- 0000 0 0 4 &mpic 3 1 0 0
- >;
- ranges = <0x2000000 0x0 0xc0000000
- 0x2000000 0x0 0xc0000000
- 0x0 0x20000000
-
- 0x1000000 0x0 0x0
- 0x1000000 0x0 0x0
- 0x0 0x100000>;
- };
- };
-
- board_pci1: pci1: pcie@ff609000 {
- reg = <0 0xff609000 0 0x1000>;
- ranges = <0x2000000 0x0 0xa0000000 0 0xa0000000 0x0 0x20000000
- 0x1000000 0x0 0x00000000 0 0xffc10000 0x0 0x10000>;
- pcie@0 {
- /*
- * IRQ[4:6] only for PCIe, set to active-high,
- * IRQ[7] is pulled up on board, set to active-low
- */
- interrupt-map-mask = <0xf800 0 0 7>;
- interrupt-map = <
- /* IDSEL 0x0 */
- 0000 0 0 1 &mpic 4 2 0 0
- 0000 0 0 2 &mpic 5 2 0 0
- 0000 0 0 3 &mpic 6 2 0 0
- 0000 0 0 4 &mpic 7 1 0 0
- >;
- ranges = <0x2000000 0x0 0xa0000000
- 0x2000000 0x0 0xa0000000
- 0x0 0x20000000
-
- 0x1000000 0x0 0x0
- 0x1000000 0x0 0x0
- 0x0 0x100000>;
- };
- };
-
- pci2: pcie@ff60b000 {
- reg = <0 0xff60b000 0 0x1000>;
- ranges = <0x2000000 0x0 0x80000000 0 0x80000000 0x0 0x20000000
- 0x1000000 0x0 0x00000000 0 0xffc00000 0x0 0x10000>;
- pcie@0 {
- /*
- * IRQ[8:10] are pulled up on board, set to active-low
- * IRQ[11] only for PCIe, set to active-high,
- */
- interrupt-map-mask = <0xf800 0 0 7>;
- interrupt-map = <
- /* IDSEL 0x0 */
- 0000 0 0 1 &mpic 8 1 0 0
- 0000 0 0 2 &mpic 9 1 0 0
- 0000 0 0 3 &mpic 10 1 0 0
- 0000 0 0 4 &mpic 11 2 0 0
- >;
- ranges = <0x2000000 0x0 0x80000000
- 0x2000000 0x0 0x80000000
- 0x0 0x20000000
-
- 0x1000000 0x0 0x0
- 0x1000000 0x0 0x0
- 0x0 0x100000>;
- };
- };
-};
-
-/include/ "fsl/p1023si-post.dtsi"
diff --git a/arch/powerpc/boot/dts/t1040qds.dts b/arch/powerpc/boot/dts/t1040qds.dts
new file mode 100644
index 0000000..973c29c
--- /dev/null
+++ b/arch/powerpc/boot/dts/t1040qds.dts
@@ -0,0 +1,46 @@
+/*
+ * T1040QDS Device Tree Source
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/include/ "fsl/t104xsi-pre.dtsi"
+/include/ "t104xqds.dtsi"
+
+/ {
+ model = "fsl,T1040QDS";
+ compatible = "fsl,T1040QDS";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+};
+
+/include/ "fsl/t1040si-post.dtsi"
diff --git a/arch/powerpc/boot/dts/t1042qds.dts b/arch/powerpc/boot/dts/t1042qds.dts
new file mode 100644
index 0000000..45bd037
--- /dev/null
+++ b/arch/powerpc/boot/dts/t1042qds.dts
@@ -0,0 +1,46 @@
+/*
+ * T1042QDS Device Tree Source
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/include/ "fsl/t104xsi-pre.dtsi"
+/include/ "t104xqds.dtsi"
+
+/ {
+ model = "fsl,T1042QDS";
+ compatible = "fsl,T1042QDS";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+};
+
+/include/ "fsl/t1042si-post.dtsi"
diff --git a/arch/powerpc/boot/dts/t104xqds.dtsi b/arch/powerpc/boot/dts/t104xqds.dtsi
new file mode 100644
index 0000000..234f4b5
--- /dev/null
+++ b/arch/powerpc/boot/dts/t104xqds.dtsi
@@ -0,0 +1,166 @@
+/*
+ * T104xQDS Device Tree Source
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/ {
+ model = "fsl,T1040QDS";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+
+ ifc: localbus@ffe124000 {
+ reg = <0xf 0xfe124000 0 0x2000>;
+ ranges = <0 0 0xf 0xe8000000 0x08000000
+ 2 0 0xf 0xff800000 0x00010000
+ 3 0 0xf 0xffdf0000 0x00008000>;
+
+ nor@0,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "cfi-flash";
+ reg = <0x0 0x0 0x8000000>;
+
+ bank-width = <2>;
+ device-width = <1>;
+ };
+
+ nand@2,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "fsl,ifc-nand";
+ reg = <0x2 0x0 0x10000>;
+ };
+
+ board-control@3,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "fsl,fpga-qixis";
+ reg = <3 0 0x300>;
+ };
+ };
+
+ memory {
+ device_type = "memory";
+ };
+
+ dcsr: dcsr@f00000000 {
+ ranges = <0x00000000 0xf 0x00000000 0x01072000>;
+ };
+
+ soc: soc@ffe000000 {
+ ranges = <0x00000000 0xf 0xfe000000 0x1000000>;
+ reg = <0xf 0xfe000000 0 0x00001000>;
+
+ spi@110000 {
+ flash@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "micron,n25q128a11";
+ reg = <0>;
+ spi-max-frequency = <10000000>; /* input clock */
+ };
+ };
+
+ i2c@118000 {
+ pca9547@77 {
+ compatible = "philips,pca9547";
+ reg = <0x77>;
+ };
+ rtc@68 {
+ compatible = "dallas,ds3232";
+ reg = <0x68>;
+ interrupts = <0x1 0x1 0 0>;
+ };
+ };
+ };
+
+ pci0: pcie@ffe240000 {
+ reg = <0xf 0xfe240000 0 0x10000>;
+ ranges = <0x02000000 0 0xe0000000 0xc 0x00000000 0x0 0x10000000
+ 0x01000000 0 0x00000000 0xf 0xf8000000 0x0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x10000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+
+ pci1: pcie@ffe250000 {
+ reg = <0xf 0xfe250000 0 0x10000>;
+ ranges = <0x02000000 0x0 0xe0000000 0xc 0x10000000 0x0 0x10000000
+ 0x01000000 0x0 0x00000000 0xf 0xf8010000 0x0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x10000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+
+ pci2: pcie@ffe260000 {
+ reg = <0xf 0xfe260000 0 0x10000>;
+ ranges = <0x02000000 0 0xe0000000 0xc 0x20000000 0 0x10000000
+ 0x01000000 0 0x00000000 0xf 0xf8020000 0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x10000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+
+ pci3: pcie@ffe270000 {
+ reg = <0xf 0xfe270000 0 0x10000>;
+ ranges = <0x02000000 0 0xe0000000 0xc 0x30000000 0 0x10000000
+ 0x01000000 0 0x00000000 0xf 0xf8030000 0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x10000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+};
diff --git a/arch/powerpc/boot/dts/t4240emu.dts b/arch/powerpc/boot/dts/t4240emu.dts
index ee24ab3..bc12127a 100644
--- a/arch/powerpc/boot/dts/t4240emu.dts
+++ b/arch/powerpc/boot/dts/t4240emu.dts
@@ -60,63 +60,75 @@
device_type = "cpu";
reg = <0 1>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu1: PowerPC,e6500@2 {
device_type = "cpu";
reg = <2 3>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu2: PowerPC,e6500@4 {
device_type = "cpu";
reg = <4 5>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu3: PowerPC,e6500@6 {
device_type = "cpu";
reg = <6 7>;
next-level-cache = <&L2_1>;
+ fsl,portid-mapping = <0x80000000>;
};
cpu4: PowerPC,e6500@8 {
device_type = "cpu";
reg = <8 9>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x40000000>;
};
cpu5: PowerPC,e6500@10 {
device_type = "cpu";
reg = <10 11>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x40000000>;
};
cpu6: PowerPC,e6500@12 {
device_type = "cpu";
reg = <12 13>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x40000000>;
};
cpu7: PowerPC,e6500@14 {
device_type = "cpu";
reg = <14 15>;
next-level-cache = <&L2_2>;
+ fsl,portid-mapping = <0x40000000>;
};
cpu8: PowerPC,e6500@16 {
device_type = "cpu";
reg = <16 17>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x20000000>;
};
cpu9: PowerPC,e6500@18 {
device_type = "cpu";
reg = <18 19>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x20000000>;
};
cpu10: PowerPC,e6500@20 {
device_type = "cpu";
reg = <20 21>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x20000000>;
};
cpu11: PowerPC,e6500@22 {
device_type = "cpu";
reg = <22 23>;
next-level-cache = <&L2_3>;
+ fsl,portid-mapping = <0x20000000>;
};
};
};
@@ -213,7 +225,7 @@
};
corenet-cf@18000 {
- compatible = "fsl,corenet-cf";
+ compatible = "fsl,corenet2-cf", "fsl,corenet-cf";
reg = <0x18000 0x1000>;
interrupts = <16 2 1 31>;
fsl,ccf-num-csdids = <32>;
@@ -223,6 +235,7 @@
iommu@20000 {
compatible = "fsl,pamu-v1.0", "fsl,pamu";
reg = <0x20000 0x6000>;
+ fsl,portid-mapping = <0x8000>;
interrupts = <
24 2 0 0
16 2 1 30>;
diff --git a/arch/powerpc/boot/elf_util.c b/arch/powerpc/boot/elf_util.c
index 1567a0c..316552d 100644
--- a/arch/powerpc/boot/elf_util.c
+++ b/arch/powerpc/boot/elf_util.c
@@ -26,7 +26,11 @@
elf64->e_ident[EI_MAG2] == ELFMAG2 &&
elf64->e_ident[EI_MAG3] == ELFMAG3 &&
elf64->e_ident[EI_CLASS] == ELFCLASS64 &&
+#ifdef __LITTLE_ENDIAN__
+ elf64->e_ident[EI_DATA] == ELFDATA2LSB &&
+#else
elf64->e_ident[EI_DATA] == ELFDATA2MSB &&
+#endif
(elf64->e_type == ET_EXEC ||
elf64->e_type == ET_DYN) &&
elf64->e_machine == EM_PPC64))
diff --git a/arch/powerpc/boot/of.c b/arch/powerpc/boot/of.c
index 62e2f43..7ca910c 100644
--- a/arch/powerpc/boot/of.c
+++ b/arch/powerpc/boot/of.c
@@ -40,8 +40,8 @@
#ifdef DEBUG
printf(" trying: 0x%08lx\n\r", claim_base);
#endif
- addr = (unsigned long)of_claim(claim_base, size, 0);
- if ((void *)addr != (void *)-1)
+ addr = (unsigned long) of_claim(claim_base, size, 0);
+ if (addr != PROM_ERROR)
break;
}
if (addr == 0)
diff --git a/arch/powerpc/boot/of.h b/arch/powerpc/boot/of.h
index e4c68f7..c8c1750 100644
--- a/arch/powerpc/boot/of.h
+++ b/arch/powerpc/boot/of.h
@@ -1,12 +1,15 @@
#ifndef _PPC_BOOT_OF_H_
#define _PPC_BOOT_OF_H_
+#include "swab.h"
+
typedef void *phandle;
-typedef void *ihandle;
+typedef u32 ihandle;
void of_init(void *promptr);
int of_call_prom(const char *service, int nargs, int nret, ...);
-void *of_claim(unsigned long virt, unsigned long size, unsigned long align);
+unsigned int of_claim(unsigned long virt, unsigned long size,
+ unsigned long align);
void *of_vmlinux_alloc(unsigned long size);
void of_exit(void);
void *of_finddevice(const char *name);
@@ -18,4 +21,16 @@
/* Console functions */
void of_console_init(void);
+typedef u32 __be32;
+
+#ifdef __LITTLE_ENDIAN__
+#define cpu_to_be32(x) swab32(x)
+#define be32_to_cpu(x) swab32(x)
+#else
+#define cpu_to_be32(x) (x)
+#define be32_to_cpu(x) (x)
+#endif
+
+#define PROM_ERROR (-1u)
+
#endif /* _PPC_BOOT_OF_H_ */
diff --git a/arch/powerpc/boot/ofconsole.c b/arch/powerpc/boot/ofconsole.c
index ce0e024..8b75470 100644
--- a/arch/powerpc/boot/ofconsole.c
+++ b/arch/powerpc/boot/ofconsole.c
@@ -18,7 +18,7 @@
#include "of.h"
-static void *of_stdout_handle;
+static unsigned int of_stdout_handle;
static int of_console_open(void)
{
@@ -27,8 +27,10 @@
if (((devp = of_finddevice("/chosen")) != NULL)
&& (of_getprop(devp, "stdout", &of_stdout_handle,
sizeof(of_stdout_handle))
- == sizeof(of_stdout_handle)))
+ == sizeof(of_stdout_handle))) {
+ of_stdout_handle = be32_to_cpu(of_stdout_handle);
return 0;
+ }
return -1;
}
diff --git a/arch/powerpc/boot/oflib.c b/arch/powerpc/boot/oflib.c
index b0ec9cf..46c98a4 100644
--- a/arch/powerpc/boot/oflib.c
+++ b/arch/powerpc/boot/oflib.c
@@ -16,74 +16,83 @@
#include "of.h"
+typedef u32 prom_arg_t;
+
+/* The following structure is used to communicate with open firmware.
+ * All arguments in and out are in big endian format. */
+struct prom_args {
+ __be32 service; /* Address of service name string. */
+ __be32 nargs; /* Number of input arguments. */
+ __be32 nret; /* Number of output arguments. */
+ __be32 args[10]; /* Input/output arguments. */
+};
+
+#ifdef __powerpc64__
+extern int prom(void *);
+#else
static int (*prom) (void *);
+#endif
void of_init(void *promptr)
{
+#ifndef __powerpc64__
prom = (int (*)(void *))promptr;
+#endif
}
+#define ADDR(x) (u32)(unsigned long)(x)
+
int of_call_prom(const char *service, int nargs, int nret, ...)
{
int i;
- struct prom_args {
- const char *service;
- int nargs;
- int nret;
- unsigned int args[12];
- } args;
+ struct prom_args args;
va_list list;
- args.service = service;
- args.nargs = nargs;
- args.nret = nret;
+ args.service = cpu_to_be32(ADDR(service));
+ args.nargs = cpu_to_be32(nargs);
+ args.nret = cpu_to_be32(nret);
va_start(list, nret);
for (i = 0; i < nargs; i++)
- args.args[i] = va_arg(list, unsigned int);
+ args.args[i] = cpu_to_be32(va_arg(list, prom_arg_t));
va_end(list);
for (i = 0; i < nret; i++)
args.args[nargs+i] = 0;
if (prom(&args) < 0)
- return -1;
+ return PROM_ERROR;
- return (nret > 0)? args.args[nargs]: 0;
+ return (nret > 0) ? be32_to_cpu(args.args[nargs]) : 0;
}
static int of_call_prom_ret(const char *service, int nargs, int nret,
- unsigned int *rets, ...)
+ prom_arg_t *rets, ...)
{
int i;
- struct prom_args {
- const char *service;
- int nargs;
- int nret;
- unsigned int args[12];
- } args;
+ struct prom_args args;
va_list list;
- args.service = service;
- args.nargs = nargs;
- args.nret = nret;
+ args.service = cpu_to_be32(ADDR(service));
+ args.nargs = cpu_to_be32(nargs);
+ args.nret = cpu_to_be32(nret);
va_start(list, rets);
for (i = 0; i < nargs; i++)
- args.args[i] = va_arg(list, unsigned int);
+ args.args[i] = cpu_to_be32(va_arg(list, prom_arg_t));
va_end(list);
for (i = 0; i < nret; i++)
args.args[nargs+i] = 0;
if (prom(&args) < 0)
- return -1;
+ return PROM_ERROR;
- if (rets != (void *) 0)
+ if (rets != NULL)
for (i = 1; i < nret; ++i)
- rets[i-1] = args.args[nargs+i];
+ rets[i-1] = be32_to_cpu(args.args[nargs+i]);
- return (nret > 0)? args.args[nargs]: 0;
+ return (nret > 0) ? be32_to_cpu(args.args[nargs]) : 0;
}
/* returns true if s2 is a prefix of s1 */
@@ -103,7 +112,7 @@
*/
static int need_map = -1;
static ihandle chosen_mmu;
-static phandle memory;
+static ihandle memory;
static int check_of_version(void)
{
@@ -132,10 +141,10 @@
printf("no mmu\n");
return 0;
}
- memory = (ihandle) of_call_prom("open", 1, 1, "/memory");
- if (memory == (ihandle) -1) {
- memory = (ihandle) of_call_prom("open", 1, 1, "/memory@0");
- if (memory == (ihandle) -1) {
+ memory = of_call_prom("open", 1, 1, "/memory");
+ if (memory == PROM_ERROR) {
+ memory = of_call_prom("open", 1, 1, "/memory@0");
+ if (memory == PROM_ERROR) {
printf("no memory node\n");
return 0;
}
@@ -144,40 +153,41 @@
return 1;
}
-void *of_claim(unsigned long virt, unsigned long size, unsigned long align)
+unsigned int of_claim(unsigned long virt, unsigned long size,
+ unsigned long align)
{
int ret;
- unsigned int result;
+ prom_arg_t result;
if (need_map < 0)
need_map = check_of_version();
if (align || !need_map)
- return (void *) of_call_prom("claim", 3, 1, virt, size, align);
+ return of_call_prom("claim", 3, 1, virt, size, align);
ret = of_call_prom_ret("call-method", 5, 2, &result, "claim", memory,
align, size, virt);
if (ret != 0 || result == -1)
- return (void *) -1;
+ return -1;
ret = of_call_prom_ret("call-method", 5, 2, &result, "claim", chosen_mmu,
align, size, virt);
/* 0x12 == coherent + read/write */
ret = of_call_prom("call-method", 6, 1, "map", chosen_mmu,
0x12, size, virt, virt);
- return (void *) virt;
+ return virt;
}
void *of_vmlinux_alloc(unsigned long size)
{
unsigned long start = (unsigned long)_start, end = (unsigned long)_end;
- void *addr;
+ unsigned long addr;
void *p;
/* With some older POWER4 firmware we need to claim the area the kernel
* will reside in. Newer firmwares don't need this so we just ignore
* the return value.
*/
- addr = of_claim(start, end - start, 0);
- printf("Trying to claim from 0x%lx to 0x%lx (0x%lx) got %p\r\n",
+ addr = (unsigned long) of_claim(start, end - start, 0);
+ printf("Trying to claim from 0x%lx to 0x%lx (0x%lx) got %lx\r\n",
start, end, end - start, addr);
p = malloc(size);
@@ -197,7 +207,7 @@
*/
void *of_finddevice(const char *name)
{
- return (phandle) of_call_prom("finddevice", 1, 1, name);
+ return (void *) (unsigned long) of_call_prom("finddevice", 1, 1, name);
}
int of_getprop(const void *phandle, const char *name, void *buf,
diff --git a/arch/powerpc/boot/ppc_asm.h b/arch/powerpc/boot/ppc_asm.h
index eb0e98b..35ea60c 100644
--- a/arch/powerpc/boot/ppc_asm.h
+++ b/arch/powerpc/boot/ppc_asm.h
@@ -62,4 +62,16 @@
#define SPRN_TBRL 268
#define SPRN_TBRU 269
+#define FIXUP_ENDIAN \
+ tdi 0, 0, 0x48; /* Reverse endian of b . + 8 */ \
+ b $+36; /* Skip trampoline if endian is good */ \
+ .long 0x05009f42; /* bcl 20,31,$+4 */ \
+ .long 0xa602487d; /* mflr r10 */ \
+ .long 0x1c004a39; /* addi r10,r10,28 */ \
+ .long 0xa600607d; /* mfmsr r11 */ \
+ .long 0x01006b69; /* xori r11,r11,1 */ \
+ .long 0xa6035a7d; /* mtsrr0 r10 */ \
+ .long 0xa6037b7d; /* mtsrr1 r11 */ \
+ .long 0x2400004c /* rfid */
+
#endif /* _PPC64_PPC_ASM_H */
diff --git a/arch/powerpc/boot/pseries-head.S b/arch/powerpc/boot/pseries-head.S
new file mode 100644
index 0000000..6ef6e02
--- /dev/null
+++ b/arch/powerpc/boot/pseries-head.S
@@ -0,0 +1,8 @@
+#include "ppc_asm.h"
+
+ .text
+
+ .globl _zimage_start
+_zimage_start:
+ FIXUP_ENDIAN
+ b _zimage_start_lib
diff --git a/arch/powerpc/boot/stdio.c b/arch/powerpc/boot/stdio.c
index 5b57800..a701261b 100644
--- a/arch/powerpc/boot/stdio.c
+++ b/arch/powerpc/boot/stdio.c
@@ -21,6 +21,18 @@
return sc - s;
}
+#ifdef __powerpc64__
+
+# define do_div(n, base) ({ \
+ unsigned int __base = (base); \
+ unsigned int __rem; \
+ __rem = ((unsigned long long)(n)) % __base; \
+ (n) = ((unsigned long long)(n)) / __base; \
+ __rem; \
+})
+
+#else
+
extern unsigned int __div64_32(unsigned long long *dividend,
unsigned int divisor);
@@ -39,6 +51,8 @@
__rem; \
})
+#endif /* __powerpc64__ */
+
static int skip_atoi(const char **s)
{
int i, c;
diff --git a/arch/powerpc/boot/swab.h b/arch/powerpc/boot/swab.h
new file mode 100644
index 0000000..d0e1431
--- /dev/null
+++ b/arch/powerpc/boot/swab.h
@@ -0,0 +1,29 @@
+#ifndef _PPC_BOOT_SWAB_H_
+#define _PPC_BOOT_SWAB_H_
+
+static inline u16 swab16(u16 x)
+{
+ return ((x & (u16)0x00ffU) << 8) |
+ ((x & (u16)0xff00U) >> 8);
+}
+
+static inline u32 swab32(u32 x)
+{
+ return ((x & (u32)0x000000ffUL) << 24) |
+ ((x & (u32)0x0000ff00UL) << 8) |
+ ((x & (u32)0x00ff0000UL) >> 8) |
+ ((x & (u32)0xff000000UL) >> 24);
+}
+
+static inline u64 swab64(u64 x)
+{
+ return (u64)((x & (u64)0x00000000000000ffULL) << 56) |
+ (u64)((x & (u64)0x000000000000ff00ULL) << 40) |
+ (u64)((x & (u64)0x0000000000ff0000ULL) << 24) |
+ (u64)((x & (u64)0x00000000ff000000ULL) << 8) |
+ (u64)((x & (u64)0x000000ff00000000ULL) >> 8) |
+ (u64)((x & (u64)0x0000ff0000000000ULL) >> 24) |
+ (u64)((x & (u64)0x00ff000000000000ULL) >> 40) |
+ (u64)((x & (u64)0xff00000000000000ULL) >> 56);
+}
+#endif /* _PPC_BOOT_SWAB_H_ */
diff --git a/arch/powerpc/boot/treeboot-akebono.c b/arch/powerpc/boot/treeboot-akebono.c
new file mode 100644
index 0000000..b73174c
--- /dev/null
+++ b/arch/powerpc/boot/treeboot-akebono.c
@@ -0,0 +1,163 @@
+/*
+ * Copyright © 2013 Tony Breeds IBM Corporation
+ * Copyright © 2013 Alistair Popple IBM Corporation
+ *
+ * Based on earlier code:
+ * Copyright (C) Paul Mackerras 1997.
+ *
+ * Matt Porter <mporter@kernel.crashing.org>
+ * Copyright 2002-2005 MontaVista Software Inc.
+ *
+ * Eugene Surovegin <eugene.surovegin@zultys.com> or <ebs@ebshome.net>
+ * Copyright (c) 2003, 2004 Zultys Technologies
+ *
+ * Copyright 2007 David Gibson, IBM Corporation.
+ * Copyright 2010 Ben. Herrenschmidt, IBM Corporation.
+ * Copyright © 2011 David Kleikamp IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+#include <stdarg.h>
+#include <stddef.h>
+#include "types.h"
+#include "elf.h"
+#include "string.h"
+#include "stdlib.h"
+#include "stdio.h"
+#include "page.h"
+#include "ops.h"
+#include "reg.h"
+#include "io.h"
+#include "dcr.h"
+#include "4xx.h"
+#include "44x.h"
+#include "libfdt.h"
+
+BSS_STACK(4096);
+
+#define SPRN_PIR 0x11E /* Processor Indentification Register */
+#define USERDATA_LEN 256 /* Length of userdata passed in by PIBS */
+#define MAX_RANKS 0x4
+#define DDR3_MR0CF 0x80010011U
+#define CCTL0_MCO2 0x8000080FU
+#define CCTL0_MCO3 0x80000810U
+#define CCTL0_MCO4 0x80000811U
+#define CCTL0_MCO5 0x80000812U
+#define CCTL0_MCO6 0x80000813U
+
+static unsigned long long ibm_akebono_memsize;
+static long long unsigned mac_addr;
+
+static unsigned long long ibm_akebono_detect_memsize(void)
+{
+ u32 reg;
+ unsigned i;
+ unsigned long long memsize = 0;
+
+ for (i = 0; i < MAX_RANKS; i++) {
+ reg = mfdcrx(DDR3_MR0CF + i);
+
+ if (!(reg & 1))
+ continue;
+
+ reg &= 0x0000f000;
+ reg >>= 12;
+ memsize += (0x800000ULL << reg);
+ }
+
+ return memsize;
+}
+
+static void ibm_akebono_fixups(void)
+{
+ void *emac;
+ u32 reg;
+
+ dt_fixup_memory(0x0ULL, ibm_akebono_memsize);
+
+ /* Fixup the SD timeout frequency */
+ mtdcrx(CCTL0_MCO4, 0x1);
+
+ /* Disable SD high-speed mode (which seems to be broken) */
+ reg = mfdcrx(CCTL0_MCO2) & ~0x2;
+ mtdcrx(CCTL0_MCO2, reg);
+
+ /* Set the MAC address */
+ emac = finddevice("/plb/opb/ethernet");
+ if (emac > 0) {
+ if (mac_addr)
+ setprop(emac, "local-mac-address",
+ ((u8 *) &mac_addr) + 2 , 6);
+ }
+}
+
+void platform_init(char *userdata)
+{
+ unsigned long end_of_ram, avail_ram;
+ u32 pir_reg;
+ int node, size;
+ const u32 *timebase;
+ int len, i, userdata_len;
+ char *end;
+
+ userdata[USERDATA_LEN - 1] = '\0';
+ userdata_len = strlen(userdata);
+ for (i = 0; i < userdata_len - 15; i++) {
+ if (strncmp(&userdata[i], "local-mac-addr=", 15) == 0) {
+ if (i > 0 && userdata[i - 1] != ' ') {
+ /* We've only found a substring ending
+ * with local-mac-addr so this isn't
+ * our mac address. */
+ continue;
+ }
+
+ mac_addr = strtoull(&userdata[i + 15], &end, 16);
+
+ /* Remove the "local-mac-addr=<...>" from the kernel
+ * command line, including the tailing space if
+ * present. */
+ if (*end == ' ')
+ end++;
+
+ len = ((int) end) - ((int) &userdata[i]);
+ memmove(&userdata[i], end,
+ userdata_len - (len + i) + 1);
+ break;
+ }
+ }
+
+ loader_info.cmdline = userdata;
+ loader_info.cmdline_len = 256;
+
+ ibm_akebono_memsize = ibm_akebono_detect_memsize();
+ if (ibm_akebono_memsize >> 32)
+ end_of_ram = ~0UL;
+ else
+ end_of_ram = ibm_akebono_memsize;
+ avail_ram = end_of_ram - (unsigned long)_end;
+
+ simple_alloc_init(_end, avail_ram, 128, 64);
+ platform_ops.fixups = ibm_akebono_fixups;
+ platform_ops.exit = ibm44x_dbcr_reset;
+ pir_reg = mfspr(SPRN_PIR);
+
+ /* Make sure FDT blob is sane */
+ if (fdt_check_header(_dtb_start) != 0)
+ fatal("Invalid device tree blob\n");
+
+ node = fdt_node_offset_by_prop_value(_dtb_start, -1, "device_type",
+ "cpu", sizeof("cpu"));
+ if (!node)
+ fatal("Cannot find cpu node\n");
+ timebase = fdt_getprop(_dtb_start, node, "timebase-frequency", &size);
+ if (timebase && (size == 4))
+ timebase_period_ns = 1000000000 / *timebase;
+
+ fdt_set_boot_cpuid_phys(_dtb_start, pir_reg);
+ fdt_init(_dtb_start);
+
+ serial_console_init();
+}
diff --git a/arch/powerpc/boot/util.S b/arch/powerpc/boot/util.S
index 6636b1d7..243b849 100644
--- a/arch/powerpc/boot/util.S
+++ b/arch/powerpc/boot/util.S
@@ -45,7 +45,7 @@
mfspr r4,SPRN_PVR
srwi r4,r4,16
cmpwi 0,r4,1 /* 601 ? */
- bne .udelay_not_601
+ bne .Ludelay_not_601
00: li r0,86 /* Instructions / microsecond? */
mtctr r0
10: addi r0,r0,0 /* NOP */
@@ -54,7 +54,7 @@
bne 00b
blr
-.udelay_not_601:
+.Ludelay_not_601:
mulli r4,r3,1000 /* nanoseconds */
/* Change r4 to be the number of ticks using:
* (nanoseconds + (timebase_period_ns - 1 )) / timebase_period_ns
diff --git a/arch/powerpc/boot/wrapper b/arch/powerpc/boot/wrapper
index d27a255..ae0f88e 100755
--- a/arch/powerpc/boot/wrapper
+++ b/arch/powerpc/boot/wrapper
@@ -40,6 +40,7 @@
binary=
gzip=.gz
pie=
+format=
# cross-compilation prefix
CROSS=
@@ -136,6 +137,14 @@
kernel=vmlinux
fi
+elfformat="`${CROSS}objdump -p "$kernel" | grep 'file format' | awk '{print $4}'`"
+case "$elfformat" in
+ elf64-powerpcle) format=elf64lppc ;;
+ elf64-powerpc) format=elf32ppc ;;
+ elf32-powerpc) format=elf32ppc ;;
+esac
+
+
platformo=$object/"$platform".o
lds=$object/zImage.lds
ext=strip
@@ -152,8 +161,12 @@
make_space=n
;;
pseries)
- platformo="$object/of.o $object/epapr.o"
+ platformo="$object/pseries-head.o $object/of.o $object/epapr.o"
link_address='0x4000000'
+ if [ "$format" != "elf32ppc" ]; then
+ link_address=
+ pie=-pie
+ fi
make_space=n
;;
maple)
@@ -257,6 +270,9 @@
treeboot-currituck)
link_address='0x1000000'
;;
+treeboot-akebono)
+ link_address='0x1000000'
+ ;;
treeboot-iss4xx-mpic)
platformo="$object/treeboot-iss4xx.o"
;;
@@ -379,7 +395,7 @@
if [ -n "$link_address" ] ; then
text_start="-Ttext $link_address"
fi
- ${CROSS}ld -m elf32ppc -T $lds $text_start $pie -o "$ofile" \
+ ${CROSS}ld -m $format -T $lds $text_start $pie -o "$ofile" \
$platformo $tmp $object/wrapper.a
rm $tmp
fi
diff --git a/arch/powerpc/boot/zImage.lds.S b/arch/powerpc/boot/zImage.lds.S
index 2bd8731..861e721 100644
--- a/arch/powerpc/boot/zImage.lds.S
+++ b/arch/powerpc/boot/zImage.lds.S
@@ -1,4 +1,10 @@
+#include <asm-generic/vmlinux.lds.h>
+
+#ifdef CONFIG_PPC64_BOOT_WRAPPER
+OUTPUT_ARCH(powerpc:common64)
+#else
OUTPUT_ARCH(powerpc:common)
+#endif
ENTRY(_zimage_start)
EXTERN(_zimage_start)
SECTIONS
@@ -16,7 +22,9 @@
*(.rodata*)
*(.data*)
*(.sdata*)
+#ifndef CONFIG_PPC64_BOOT_WRAPPER
*(.got2)
+#endif
}
.dynsym : { *(.dynsym) }
.dynstr : { *(.dynstr) }
@@ -27,7 +35,13 @@
}
.hash : { *(.hash) }
.interp : { *(.interp) }
- .rela.dyn : { *(.rela*) }
+ .rela.dyn :
+ {
+#ifdef CONFIG_PPC64_BOOT_WRAPPER
+ __rela_dyn_start = .;
+#endif
+ *(.rela*)
+ }
. = ALIGN(8);
.kernel:dtb :
@@ -53,6 +67,15 @@
_initrd_end = .;
}
+#ifdef CONFIG_PPC64_BOOT_WRAPPER
+ .got :
+ {
+ __toc_start = .;
+ *(.got)
+ *(.toc)
+ }
+#endif
+
. = ALIGN(4096);
.bss :
{
diff --git a/arch/powerpc/configs/44x/akebono_defconfig b/arch/powerpc/configs/44x/akebono_defconfig
new file mode 100644
index 0000000..7e2530cd
--- /dev/null
+++ b/arch/powerpc/configs/44x/akebono_defconfig
@@ -0,0 +1,148 @@
+CONFIG_44x=y
+CONFIG_SMP=y
+CONFIG_SYSVIPC=y
+CONFIG_POSIX_MQUEUE=y
+CONFIG_LOG_BUF_SHIFT=14
+CONFIG_BLK_DEV_INITRD=y
+CONFIG_RD_BZIP2=y
+CONFIG_RD_LZMA=y
+CONFIG_RD_XZ=y
+CONFIG_EXPERT=y
+CONFIG_KALLSYMS_ALL=y
+# CONFIG_SLUB_CPU_PARTIAL is not set
+CONFIG_PROFILING=y
+CONFIG_OPROFILE=y
+CONFIG_MODULES=y
+CONFIG_MODULE_UNLOAD=y
+# CONFIG_BLK_DEV_BSG is not set
+# CONFIG_POWERNV_MSI is not set
+CONFIG_PPC_47x=y
+# CONFIG_EBONY is not set
+CONFIG_AKEBONO=y
+CONFIG_HIGHMEM=y
+CONFIG_HZ_100=y
+CONFIG_IRQ_ALL_CPUS=y
+# CONFIG_COMPACTION is not set
+CONFIG_CMDLINE_BOOL=y
+CONFIG_CMDLINE=""
+# CONFIG_SUSPEND is not set
+CONFIG_PCI_MSI=y
+CONFIG_NET=y
+CONFIG_PACKET=y
+CONFIG_UNIX=y
+CONFIG_INET=y
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+CONFIG_IP_PNP_BOOTP=y
+# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
+# CONFIG_INET_XFRM_MODE_TUNNEL is not set
+# CONFIG_INET_XFRM_MODE_BEET is not set
+# CONFIG_INET_LRO is not set
+# CONFIG_IPV6 is not set
+CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_DEVTMPFS=y
+CONFIG_DEVTMPFS_MOUNT=y
+CONFIG_CONNECTOR=y
+CONFIG_MTD=y
+CONFIG_MTD_BLOCK=y
+CONFIG_MTD_JEDECPROBE=y
+CONFIG_MTD_CFI_AMDSTD=y
+CONFIG_MTD_PHYSMAP_OF=y
+CONFIG_PROC_DEVICETREE=y
+CONFIG_BLK_DEV_RAM=y
+CONFIG_BLK_DEV_RAM_SIZE=35000
+# CONFIG_SCSI_PROC_FS is not set
+CONFIG_BLK_DEV_SD=y
+# CONFIG_SCSI_LOWLEVEL is not set
+# CONFIG_SATA_PMP is not set
+# CONFIG_ATA_SFF is not set
+# CONFIG_NET_VENDOR_3COM is not set
+# CONFIG_NET_VENDOR_ADAPTEC is not set
+# CONFIG_NET_VENDOR_ALTEON is not set
+# CONFIG_NET_VENDOR_AMD is not set
+# CONFIG_NET_VENDOR_ARC is not set
+# CONFIG_NET_VENDOR_ATHEROS is not set
+# CONFIG_NET_CADENCE is not set
+# CONFIG_NET_VENDOR_BROADCOM is not set
+# CONFIG_NET_VENDOR_BROCADE is not set
+# CONFIG_NET_VENDOR_CHELSIO is not set
+# CONFIG_NET_VENDOR_CISCO is not set
+# CONFIG_NET_VENDOR_DEC is not set
+# CONFIG_NET_VENDOR_DLINK is not set
+# CONFIG_NET_VENDOR_EMULEX is not set
+# CONFIG_NET_VENDOR_EXAR is not set
+# CONFIG_NET_VENDOR_HP is not set
+CONFIG_IBM_EMAC=y
+# CONFIG_NET_VENDOR_MARVELL is not set
+# CONFIG_NET_VENDOR_MELLANOX is not set
+# CONFIG_NET_VENDOR_MICREL is not set
+# CONFIG_NET_VENDOR_MYRI is not set
+# CONFIG_NET_VENDOR_NATSEMI is not set
+# CONFIG_NET_VENDOR_NVIDIA is not set
+# CONFIG_NET_VENDOR_OKI is not set
+# CONFIG_NET_VENDOR_QLOGIC is not set
+# CONFIG_NET_VENDOR_REALTEK is not set
+# CONFIG_NET_VENDOR_RDC is not set
+# CONFIG_NET_VENDOR_SEEQ is not set
+# CONFIG_NET_VENDOR_SILAN is not set
+# CONFIG_NET_VENDOR_SIS is not set
+# CONFIG_NET_VENDOR_SMSC is not set
+# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_SUN is not set
+# CONFIG_NET_VENDOR_TEHUTI is not set
+# CONFIG_NET_VENDOR_TI is not set
+# CONFIG_NET_VENDOR_VIA is not set
+# CONFIG_NET_VENDOR_WIZNET is not set
+# CONFIG_NET_VENDOR_XILINX is not set
+# CONFIG_KEYBOARD_ATKBD is not set
+# CONFIG_MOUSE_PS2 is not set
+# CONFIG_SERIO is not set
+# CONFIG_VT is not set
+CONFIG_SERIAL_8250=y
+# CONFIG_SERIAL_8250_DEPRECATED_OPTIONS is not set
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SERIAL_8250_EXTENDED=y
+CONFIG_SERIAL_8250_SHARE_IRQ=y
+CONFIG_SERIAL_OF_PLATFORM=y
+# CONFIG_HW_RANDOM is not set
+CONFIG_I2C_CHARDEV=y
+# CONFIG_HWMON is not set
+CONFIG_THERMAL=y
+# CONFIG_USB_DEFAULT_PERSIST is not set
+CONFIG_USB_EHCI_HCD=y
+CONFIG_USB_OHCI_HCD=y
+# CONFIG_USB_OHCI_HCD_PCI is not set
+CONFIG_USB_STORAGE=y
+CONFIG_MMC=y
+CONFIG_RTC_CLASS=y
+CONFIG_RTC_DRV_M41T80=y
+CONFIG_EXT2_FS=y
+CONFIG_EXT3_FS=y
+# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
+CONFIG_EXT3_FS_POSIX_ACL=y
+CONFIG_EXT3_FS_SECURITY=y
+# CONFIG_DNOTIFY is not set
+# CONFIG_INOTIFY_USER is not set
+CONFIG_VFAT_FS=y
+CONFIG_PROC_KCORE=y
+CONFIG_TMPFS=y
+CONFIG_CRAMFS=y
+# CONFIG_NETWORK_FILESYSTEMS is not set
+CONFIG_NLS_DEFAULT="n"
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_DEBUG_INFO=y
+CONFIG_DEBUG_FS=y
+CONFIG_MAGIC_SYSRQ=y
+CONFIG_DETECT_HUNG_TASK=y
+CONFIG_XMON=y
+CONFIG_XMON_DEFAULT=y
+CONFIG_PPC_EARLY_DEBUG=y
+CONFIG_PPC_EARLY_DEBUG_44x_PHYSLOW=0x00010000
+CONFIG_PPC_EARLY_DEBUG_44x_PHYSHIGH=0x33f
+CONFIG_CRYPTO_PCBC=y
+CONFIG_CRYPTO_MD5=y
+CONFIG_CRYPTO_SHA1_PPC=y
+CONFIG_CRYPTO_DES=y
+# CONFIG_CRYPTO_ANSI_CPRNG is not set
+# CONFIG_CRYPTO_HW is not set
diff --git a/arch/powerpc/configs/85xx/kmp204x_defconfig b/arch/powerpc/configs/85xx/kmp204x_defconfig
new file mode 100644
index 0000000..e9a81e5
--- /dev/null
+++ b/arch/powerpc/configs/85xx/kmp204x_defconfig
@@ -0,0 +1,225 @@
+CONFIG_PPC_85xx=y
+CONFIG_SMP=y
+CONFIG_NR_CPUS=8
+CONFIG_SYSVIPC=y
+CONFIG_POSIX_MQUEUE=y
+CONFIG_AUDIT=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_BSD_PROCESS_ACCT=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_LOG_BUF_SHIFT=14
+CONFIG_CGROUPS=y
+CONFIG_CGROUP_SCHED=y
+CONFIG_RELAY=y
+CONFIG_BLK_DEV_INITRD=y
+CONFIG_KALLSYMS_ALL=y
+CONFIG_EMBEDDED=y
+CONFIG_PERF_EVENTS=y
+CONFIG_SLAB=y
+CONFIG_MODULES=y
+CONFIG_MODULE_UNLOAD=y
+CONFIG_MODULE_FORCE_UNLOAD=y
+CONFIG_MODVERSIONS=y
+# CONFIG_BLK_DEV_BSG is not set
+CONFIG_PARTITION_ADVANCED=y
+CONFIG_MAC_PARTITION=y
+CONFIG_CORENET_GENERIC=y
+CONFIG_MPIC_MSGR=y
+CONFIG_HIGHMEM=y
+# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
+CONFIG_BINFMT_MISC=m
+CONFIG_KEXEC=y
+CONFIG_FORCE_MAX_ZONEORDER=13
+CONFIG_PCI=y
+CONFIG_PCIEPORTBUS=y
+# CONFIG_PCIEASPM is not set
+CONFIG_PCI_MSI=y
+CONFIG_ADVANCED_OPTIONS=y
+CONFIG_LOWMEM_SIZE_BOOL=y
+CONFIG_LOWMEM_SIZE=0x20000000
+CONFIG_NET=y
+CONFIG_PACKET=y
+CONFIG_UNIX=y
+CONFIG_XFRM_USER=y
+CONFIG_XFRM_SUB_POLICY=y
+CONFIG_XFRM_STATISTICS=y
+CONFIG_NET_KEY=y
+CONFIG_NET_KEY_MIGRATE=y
+CONFIG_INET=y
+CONFIG_IP_MULTICAST=y
+CONFIG_IP_ADVANCED_ROUTER=y
+CONFIG_IP_MULTIPLE_TABLES=y
+CONFIG_IP_ROUTE_MULTIPATH=y
+CONFIG_IP_ROUTE_VERBOSE=y
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+CONFIG_IP_PNP_BOOTP=y
+CONFIG_IP_PNP_RARP=y
+CONFIG_NET_IPIP=y
+CONFIG_IP_MROUTE=y
+CONFIG_IP_PIMSM_V1=y
+CONFIG_IP_PIMSM_V2=y
+CONFIG_INET_AH=y
+CONFIG_INET_ESP=y
+CONFIG_INET_IPCOMP=y
+# CONFIG_INET_LRO is not set
+CONFIG_IPV6=y
+CONFIG_IP_SCTP=m
+CONFIG_TIPC=y
+CONFIG_NET_SCHED=y
+CONFIG_NET_SCH_CBQ=y
+CONFIG_NET_SCH_HTB=y
+CONFIG_NET_SCH_HFSC=y
+CONFIG_NET_SCH_PRIO=y
+CONFIG_NET_SCH_MULTIQ=y
+CONFIG_NET_SCH_RED=y
+CONFIG_NET_SCH_SFQ=y
+CONFIG_NET_SCH_TEQL=y
+CONFIG_NET_SCH_TBF=y
+CONFIG_NET_SCH_GRED=y
+CONFIG_NET_CLS_BASIC=y
+CONFIG_NET_CLS_TCINDEX=y
+CONFIG_NET_CLS_U32=y
+CONFIG_CLS_U32_PERF=y
+CONFIG_CLS_U32_MARK=y
+CONFIG_NET_CLS_FLOW=y
+CONFIG_NET_CLS_CGROUP=y
+CONFIG_UEVENT_HELPER_PATH="/sbin/mdev"
+CONFIG_DEVTMPFS=y
+CONFIG_MTD=y
+CONFIG_MTD_CMDLINE_PARTS=y
+CONFIG_MTD_BLOCK=y
+CONFIG_MTD_CFI=y
+CONFIG_MTD_CFI_AMDSTD=y
+CONFIG_MTD_PHYSMAP_OF=y
+CONFIG_MTD_M25P80=y
+CONFIG_MTD_PHRAM=y
+CONFIG_MTD_NAND=y
+CONFIG_MTD_NAND_ECC_BCH=y
+CONFIG_MTD_NAND_FSL_ELBC=y
+CONFIG_MTD_UBI=y
+CONFIG_MTD_UBI_GLUEBI=y
+CONFIG_BLK_DEV_LOOP=y
+CONFIG_BLK_DEV_RAM=y
+CONFIG_BLK_DEV_RAM_COUNT=2
+CONFIG_BLK_DEV_RAM_SIZE=2048
+CONFIG_EEPROM_AT24=y
+CONFIG_SCSI=y
+CONFIG_BLK_DEV_SD=y
+CONFIG_CHR_DEV_ST=y
+CONFIG_BLK_DEV_SR=y
+CONFIG_CHR_DEV_SG=y
+CONFIG_SCSI_MULTI_LUN=y
+CONFIG_SCSI_LOGGING=y
+CONFIG_SCSI_SYM53C8XX_2=y
+CONFIG_NETDEVICES=y
+# CONFIG_NET_VENDOR_3COM is not set
+# CONFIG_NET_VENDOR_ADAPTEC is not set
+# CONFIG_NET_VENDOR_ALTEON is not set
+# CONFIG_NET_VENDOR_AMD is not set
+# CONFIG_NET_VENDOR_ATHEROS is not set
+# CONFIG_NET_CADENCE is not set
+# CONFIG_NET_VENDOR_BROADCOM is not set
+# CONFIG_NET_VENDOR_BROCADE is not set
+# CONFIG_NET_VENDOR_CHELSIO is not set
+# CONFIG_NET_VENDOR_CISCO is not set
+# CONFIG_NET_VENDOR_DEC is not set
+# CONFIG_NET_VENDOR_DLINK is not set
+# CONFIG_NET_VENDOR_EMULEX is not set
+# CONFIG_NET_VENDOR_EXAR is not set
+CONFIG_FSL_PQ_MDIO=y
+CONFIG_FSL_XGMAC_MDIO=y
+# CONFIG_NET_VENDOR_HP is not set
+# CONFIG_NET_VENDOR_INTEL is not set
+# CONFIG_NET_VENDOR_MARVELL is not set
+# CONFIG_NET_VENDOR_MELLANOX is not set
+# CONFIG_NET_VENDOR_MICREL is not set
+# CONFIG_NET_VENDOR_MICROCHIP is not set
+# CONFIG_NET_VENDOR_MYRI is not set
+# CONFIG_NET_VENDOR_NATSEMI is not set
+# CONFIG_NET_VENDOR_NVIDIA is not set
+# CONFIG_NET_VENDOR_OKI is not set
+# CONFIG_NET_PACKET_ENGINE is not set
+# CONFIG_NET_VENDOR_QLOGIC is not set
+# CONFIG_NET_VENDOR_REALTEK is not set
+# CONFIG_NET_VENDOR_RDC is not set
+# CONFIG_NET_VENDOR_SEEQ is not set
+# CONFIG_NET_VENDOR_SILAN is not set
+# CONFIG_NET_VENDOR_SIS is not set
+# CONFIG_NET_VENDOR_SMSC is not set
+# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_SUN is not set
+# CONFIG_NET_VENDOR_TEHUTI is not set
+# CONFIG_NET_VENDOR_TI is not set
+# CONFIG_NET_VENDOR_VIA is not set
+# CONFIG_NET_VENDOR_WIZNET is not set
+# CONFIG_NET_VENDOR_XILINX is not set
+CONFIG_MARVELL_PHY=y
+CONFIG_VITESSE_PHY=y
+CONFIG_FIXED_PHY=y
+# CONFIG_WLAN is not set
+# CONFIG_INPUT_MOUSEDEV is not set
+# CONFIG_INPUT_KEYBOARD is not set
+# CONFIG_INPUT_MOUSE is not set
+CONFIG_SERIO_LIBPS2=y
+# CONFIG_LEGACY_PTYS is not set
+CONFIG_PPC_EPAPR_HV_BYTECHAN=y
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SERIAL_8250_MANY_PORTS=y
+CONFIG_SERIAL_8250_DETECT_IRQ=y
+CONFIG_SERIAL_8250_RSA=y
+CONFIG_NVRAM=y
+CONFIG_I2C=y
+CONFIG_I2C_CHARDEV=y
+CONFIG_I2C_MUX=y
+CONFIG_I2C_MUX_PCA954x=y
+CONFIG_I2C_MPC=y
+CONFIG_SPI=y
+CONFIG_SPI_FSL_SPI=y
+CONFIG_SPI_FSL_ESPI=y
+CONFIG_SPI_SPIDEV=m
+CONFIG_PTP_1588_CLOCK=y
+# CONFIG_HWMON is not set
+# CONFIG_USB_SUPPORT is not set
+CONFIG_EDAC=y
+CONFIG_EDAC_MM_EDAC=y
+CONFIG_EDAC_MPC85XX=y
+CONFIG_RTC_CLASS=y
+CONFIG_RTC_DRV_DS3232=y
+CONFIG_RTC_DRV_CMOS=y
+CONFIG_UIO=y
+CONFIG_STAGING=y
+# CONFIG_NET_VENDOR_SILICOM is not set
+CONFIG_CLK_PPC_CORENET=y
+CONFIG_EXT2_FS=y
+CONFIG_NTFS_FS=y
+CONFIG_PROC_KCORE=y
+CONFIG_TMPFS=y
+CONFIG_JFFS2_FS=y
+CONFIG_UBIFS_FS=y
+CONFIG_CRAMFS=y
+CONFIG_SQUASHFS=y
+CONFIG_SQUASHFS_XZ=y
+CONFIG_NFS_FS=y
+CONFIG_NFS_V4=y
+CONFIG_ROOT_NFS=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_NLS_UTF8=m
+CONFIG_CRC_ITU_T=m
+CONFIG_DEBUG_INFO=y
+CONFIG_MAGIC_SYSRQ=y
+CONFIG_DEBUG_SHIRQ=y
+CONFIG_DETECT_HUNG_TASK=y
+CONFIG_SCHEDSTATS=y
+CONFIG_RCU_TRACE=y
+CONFIG_UPROBE_EVENT=y
+CONFIG_CRYPTO_NULL=y
+CONFIG_CRYPTO_PCBC=m
+CONFIG_CRYPTO_MD4=y
+CONFIG_CRYPTO_SHA256=y
+CONFIG_CRYPTO_SHA512=y
+# CONFIG_CRYPTO_ANSI_CPRNG is not set
+CONFIG_CRYPTO_DEV_FSL_CAAM=y
diff --git a/arch/powerpc/configs/corenet32_smp_defconfig b/arch/powerpc/configs/corenet32_smp_defconfig
index bbd794d..c19ff05 100644
--- a/arch/powerpc/configs/corenet32_smp_defconfig
+++ b/arch/powerpc/configs/corenet32_smp_defconfig
@@ -72,6 +72,7 @@
CONFIG_MTD_CHAR=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
+CONFIG_MTD_CFI_INTELEXT=y
CONFIG_MTD_CFI_AMDSTD=y
CONFIG_MTD_PHYSMAP_OF=y
CONFIG_MTD_M25P80=y
diff --git a/arch/powerpc/configs/mpc85xx_defconfig b/arch/powerpc/configs/mpc85xx_defconfig
index 19f0fbe..55765c8 100644
--- a/arch/powerpc/configs/mpc85xx_defconfig
+++ b/arch/powerpc/configs/mpc85xx_defconfig
@@ -32,7 +32,6 @@
CONFIG_P1022_DS=y
CONFIG_P1022_RDK=y
CONFIG_P1023_RDB=y
-CONFIG_P1023_RDS=y
CONFIG_SOCRATES=y
CONFIG_KSI8560=y
CONFIG_XES_MPC85xx=y
diff --git a/arch/powerpc/configs/mpc85xx_smp_defconfig b/arch/powerpc/configs/mpc85xx_smp_defconfig
index 062312e..5c6ecdc 100644
--- a/arch/powerpc/configs/mpc85xx_smp_defconfig
+++ b/arch/powerpc/configs/mpc85xx_smp_defconfig
@@ -35,7 +35,6 @@
CONFIG_P1022_DS=y
CONFIG_P1022_RDK=y
CONFIG_P1023_RDB=y
-CONFIG_P1023_RDS=y
CONFIG_SOCRATES=y
CONFIG_KSI8560=y
CONFIG_XES_MPC85xx=y
diff --git a/arch/powerpc/include/asm/code-patching.h b/arch/powerpc/include/asm/code-patching.h
index 97e02f9..37991e1 100644
--- a/arch/powerpc/include/asm/code-patching.h
+++ b/arch/powerpc/include/asm/code-patching.h
@@ -42,15 +42,47 @@
} while (0)
#endif
+#define OP_RT_RA_MASK 0xffff0000UL
+#define LIS_R2 0x3c020000UL
+#define ADDIS_R2_R12 0x3c4c0000UL
+#define ADDI_R2_R2 0x38420000UL
+
static inline unsigned long ppc_function_entry(void *func)
{
-#ifdef CONFIG_PPC64
+#if defined(CONFIG_PPC64)
+#if defined(_CALL_ELF) && _CALL_ELF == 2
+ u32 *insn = func;
+
/*
- * On PPC64 the function pointer actually points to the function's
- * descriptor. The first entry in the descriptor is the address
- * of the function text.
+ * A PPC64 ABIv2 function may have a local and a global entry
+ * point. We need to use the local entry point when patching
+ * functions, so identify and step over the global entry point
+ * sequence.
+ *
+ * The global entry point sequence is always of the form:
+ *
+ * addis r2,r12,XXXX
+ * addi r2,r2,XXXX
+ *
+ * A linker optimisation may convert the addis to lis:
+ *
+ * lis r2,XXXX
+ * addi r2,r2,XXXX
+ */
+ if ((((*insn & OP_RT_RA_MASK) == ADDIS_R2_R12) ||
+ ((*insn & OP_RT_RA_MASK) == LIS_R2)) &&
+ ((*(insn+1) & OP_RT_RA_MASK) == ADDI_R2_R2))
+ return (unsigned long)(insn + 2);
+ else
+ return (unsigned long)func;
+#else
+ /*
+ * On PPC64 ABIv1 the function pointer actually points to the
+ * function's descriptor. The first entry in the descriptor is the
+ * address of the function text.
*/
return ((func_descr_t *)func)->entry;
+#endif
#else
return (unsigned long)func;
#endif
diff --git a/arch/powerpc/include/asm/context_tracking.h b/arch/powerpc/include/asm/context_tracking.h
index b6f5a33..4001492 100644
--- a/arch/powerpc/include/asm/context_tracking.h
+++ b/arch/powerpc/include/asm/context_tracking.h
@@ -2,9 +2,9 @@
#define _ASM_POWERPC_CONTEXT_TRACKING_H
#ifdef CONFIG_CONTEXT_TRACKING
-#define SCHEDULE_USER bl .schedule_user
+#define SCHEDULE_USER bl schedule_user
#else
-#define SCHEDULE_USER bl .schedule
+#define SCHEDULE_USER bl schedule
#endif
#endif
diff --git a/arch/powerpc/include/asm/cputhreads.h b/arch/powerpc/include/asm/cputhreads.h
index ac3eedb..2bf8e93 100644
--- a/arch/powerpc/include/asm/cputhreads.h
+++ b/arch/powerpc/include/asm/cputhreads.h
@@ -18,10 +18,12 @@
#ifdef CONFIG_SMP
extern int threads_per_core;
+extern int threads_per_subcore;
extern int threads_shift;
extern cpumask_t threads_core_mask;
#else
#define threads_per_core 1
+#define threads_per_subcore 1
#define threads_shift 0
#define threads_core_mask (CPU_MASK_CPU0)
#endif
@@ -74,6 +76,11 @@
return cpu & (threads_per_core - 1);
}
+static inline int cpu_thread_in_subcore(int cpu)
+{
+ return cpu & (threads_per_subcore - 1);
+}
+
static inline int cpu_first_thread_sibling(int cpu)
{
return cpu & ~(threads_per_core - 1);
diff --git a/arch/powerpc/include/asm/debug.h b/arch/powerpc/include/asm/debug.h
index d251630..a954e49 100644
--- a/arch/powerpc/include/asm/debug.h
+++ b/arch/powerpc/include/asm/debug.h
@@ -46,7 +46,8 @@
static inline int debugger_fault_handler(struct pt_regs *regs) { return 0; }
#endif
-int set_breakpoint(struct arch_hw_breakpoint *brk);
+void set_breakpoint(struct arch_hw_breakpoint *brk);
+void __set_breakpoint(struct arch_hw_breakpoint *brk);
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
extern void do_send_trap(struct pt_regs *regs, unsigned long address,
unsigned long error_code, int signal_code, int brkpt);
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index d4dd41f..b76f58c 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -32,6 +32,22 @@
#ifdef CONFIG_EEH
+/* EEH subsystem flags */
+#define EEH_ENABLED 0x1 /* EEH enabled */
+#define EEH_FORCE_DISABLED 0x2 /* EEH disabled */
+#define EEH_PROBE_MODE_DEV 0x4 /* From PCI device */
+#define EEH_PROBE_MODE_DEVTREE 0x8 /* From device tree */
+
+/*
+ * Delay for PE reset, all in ms
+ *
+ * PCI specification has reset hold time of 100 milliseconds.
+ * We have 250 milliseconds here. The PCI bus settlement time
+ * is specified as 1.5 seconds and we have 1.8 seconds.
+ */
+#define EEH_PE_RST_HOLD_TIME 250
+#define EEH_PE_RST_SETTLE_TIME 1800
+
/*
* The struct is used to trace PE related EEH functionality.
* In theory, there will have one instance of the struct to
@@ -53,7 +69,7 @@
#define EEH_PE_ISOLATED (1 << 0) /* Isolated PE */
#define EEH_PE_RECOVERING (1 << 1) /* Recovering PE */
-#define EEH_PE_PHB_DEAD (1 << 2) /* Dead PHB */
+#define EEH_PE_RESET (1 << 2) /* PE reset in progress */
#define EEH_PE_KEEP (1 << 8) /* Keep PE on hotplug */
@@ -92,6 +108,7 @@
#define EEH_DEV_NO_HANDLER (1 << 8) /* No error handler */
#define EEH_DEV_SYSFS (1 << 9) /* Sysfs created */
+#define EEH_DEV_REMOVED (1 << 10) /* Removed permanently */
struct eeh_dev {
int mode; /* EEH mode */
@@ -99,7 +116,9 @@
int config_addr; /* Config address */
int pe_config_addr; /* PE config address */
u32 config_space[16]; /* Saved PCI config space */
- u8 pcie_cap; /* Saved PCIe capability */
+ int pcix_cap; /* Saved PCIx capability */
+ int pcie_cap; /* Saved PCIe capability */
+ int aer_cap; /* Saved AER capability */
struct eeh_pe *pe; /* Associated PE */
struct list_head list; /* Form link list in the PE */
struct pci_controller *phb; /* Associated PHB */
@@ -171,37 +190,40 @@
int (*restore_config)(struct device_node *dn);
};
+extern int eeh_subsystem_flags;
extern struct eeh_ops *eeh_ops;
-extern bool eeh_subsystem_enabled;
extern raw_spinlock_t confirm_error_lock;
-extern int eeh_probe_mode;
static inline bool eeh_enabled(void)
{
- return eeh_subsystem_enabled;
+ if ((eeh_subsystem_flags & EEH_FORCE_DISABLED) ||
+ !(eeh_subsystem_flags & EEH_ENABLED))
+ return false;
+
+ return true;
}
static inline void eeh_set_enable(bool mode)
{
- eeh_subsystem_enabled = mode;
+ if (mode)
+ eeh_subsystem_flags |= EEH_ENABLED;
+ else
+ eeh_subsystem_flags &= ~EEH_ENABLED;
}
-#define EEH_PROBE_MODE_DEV (1<<0) /* From PCI device */
-#define EEH_PROBE_MODE_DEVTREE (1<<1) /* From device tree */
-
static inline void eeh_probe_mode_set(int flag)
{
- eeh_probe_mode = flag;
+ eeh_subsystem_flags |= flag;
}
static inline int eeh_probe_mode_devtree(void)
{
- return (eeh_probe_mode == EEH_PROBE_MODE_DEVTREE);
+ return (eeh_subsystem_flags & EEH_PROBE_MODE_DEVTREE);
}
static inline int eeh_probe_mode_dev(void)
{
- return (eeh_probe_mode == EEH_PROBE_MODE_DEV);
+ return (eeh_subsystem_flags & EEH_PROBE_MODE_DEV);
}
static inline void eeh_serialize_lock(unsigned long *flags)
diff --git a/arch/powerpc/include/asm/elf.h b/arch/powerpc/include/asm/elf.h
index 935b5e7..888d8f3 100644
--- a/arch/powerpc/include/asm/elf.h
+++ b/arch/powerpc/include/asm/elf.h
@@ -90,6 +90,8 @@
do { \
if (((ex).e_flags & 0x3) == 2) \
set_thread_flag(TIF_ELF2ABI); \
+ else \
+ clear_thread_flag(TIF_ELF2ABI); \
if ((ex).e_ident[EI_CLASS] == ELFCLASS32) \
set_thread_flag(TIF_32BIT); \
else \
diff --git a/arch/powerpc/include/asm/exception-64e.h b/arch/powerpc/include/asm/exception-64e.h
index a563d9af..a8b52b6 100644
--- a/arch/powerpc/include/asm/exception-64e.h
+++ b/arch/powerpc/include/asm/exception-64e.h
@@ -174,10 +174,10 @@
mtlr r16;
#define TLB_MISS_STATS_D(name) \
addi r9,r13,MMSTAT_DSTATS+name; \
- bl .tlb_stat_inc;
+ bl tlb_stat_inc;
#define TLB_MISS_STATS_I(name) \
addi r9,r13,MMSTAT_ISTATS+name; \
- bl .tlb_stat_inc;
+ bl tlb_stat_inc;
#define TLB_MISS_STATS_X(name) \
ld r8,PACA_EXTLB+EX_TLB_ESR(r13); \
cmpdi cr2,r8,-1; \
@@ -185,7 +185,7 @@
addi r9,r13,MMSTAT_DSTATS+name; \
b 62f; \
61: addi r9,r13,MMSTAT_ISTATS+name; \
-62: bl .tlb_stat_inc;
+62: bl tlb_stat_inc;
#define TLB_MISS_STATS_SAVE_INFO \
std r14,EX_TLB_ESR(r12); /* save ESR */
#define TLB_MISS_STATS_SAVE_INFO_BOLTED \
diff --git a/arch/powerpc/include/asm/exception-64s.h b/arch/powerpc/include/asm/exception-64s.h
index aeaa56c..8f35cd7 100644
--- a/arch/powerpc/include/asm/exception-64s.h
+++ b/arch/powerpc/include/asm/exception-64s.h
@@ -517,7 +517,7 @@
#define DISABLE_INTS RECONCILE_IRQ_STATE(r10,r11)
#define ADD_NVGPRS \
- bl .save_nvgprs
+ bl save_nvgprs
#define RUNLATCH_ON \
BEGIN_FTR_SECTION \
diff --git a/arch/powerpc/include/asm/ftrace.h b/arch/powerpc/include/asm/ftrace.h
index 169d039e..e366187 100644
--- a/arch/powerpc/include/asm/ftrace.h
+++ b/arch/powerpc/include/asm/ftrace.h
@@ -61,6 +61,7 @@
#endif
#if defined(CONFIG_FTRACE_SYSCALLS) && defined(CONFIG_PPC64) && !defined(__ASSEMBLY__)
+#if !defined(_CALL_ELF) || _CALL_ELF != 2
#define ARCH_HAS_SYSCALL_MATCH_SYM_NAME
static inline bool arch_syscall_match_sym_name(const char *sym, const char *name)
{
@@ -72,6 +73,7 @@
*/
return !strcmp(sym + 4, name + 3);
}
+#endif
#endif /* CONFIG_FTRACE_SYSCALLS && CONFIG_PPC64 && !__ASSEMBLY__ */
#endif /* _ASM_POWERPC_FTRACE */
diff --git a/arch/powerpc/include/asm/hw_breakpoint.h b/arch/powerpc/include/asm/hw_breakpoint.h
index eb0f4ac..ac6432d 100644
--- a/arch/powerpc/include/asm/hw_breakpoint.h
+++ b/arch/powerpc/include/asm/hw_breakpoint.h
@@ -79,7 +79,7 @@
brk.address = 0;
brk.type = 0;
brk.len = 0;
- set_breakpoint(&brk);
+ __set_breakpoint(&brk);
}
extern void thread_change_pc(struct task_struct *tsk, struct pt_regs *regs);
diff --git a/arch/powerpc/include/asm/irqflags.h b/arch/powerpc/include/asm/irqflags.h
index f51a558..e20eb95 100644
--- a/arch/powerpc/include/asm/irqflags.h
+++ b/arch/powerpc/include/asm/irqflags.h
@@ -20,9 +20,9 @@
*/
#define TRACE_WITH_FRAME_BUFFER(func) \
mflr r0; \
- stdu r1, -32(r1); \
+ stdu r1, -STACK_FRAME_OVERHEAD(r1); \
std r0, 16(r1); \
- stdu r1, -32(r1); \
+ stdu r1, -STACK_FRAME_OVERHEAD(r1); \
bl func; \
ld r1, 0(r1); \
ld r1, 0(r1);
@@ -36,8 +36,8 @@
* have to call a C function so call a wrapper that saves all the
* C-clobbered registers.
*/
-#define TRACE_ENABLE_INTS TRACE_WITH_FRAME_BUFFER(.trace_hardirqs_on)
-#define TRACE_DISABLE_INTS TRACE_WITH_FRAME_BUFFER(.trace_hardirqs_off)
+#define TRACE_ENABLE_INTS TRACE_WITH_FRAME_BUFFER(trace_hardirqs_on)
+#define TRACE_DISABLE_INTS TRACE_WITH_FRAME_BUFFER(trace_hardirqs_off)
/*
* This is used by assembly code to soft-disable interrupts first and
diff --git a/arch/powerpc/include/asm/kprobes.h b/arch/powerpc/include/asm/kprobes.h
index 7b6feab..af15d4d 100644
--- a/arch/powerpc/include/asm/kprobes.h
+++ b/arch/powerpc/include/asm/kprobes.h
@@ -30,6 +30,7 @@
#include <linux/ptrace.h>
#include <linux/percpu.h>
#include <asm/probes.h>
+#include <asm/code-patching.h>
#define __ARCH_WANT_KPROBES_INSN_SLOT
@@ -56,9 +57,9 @@
if ((colon = strchr(name, ':')) != NULL) { \
colon++; \
if (*colon != '\0' && *colon != '.') \
- addr = *(kprobe_opcode_t **)addr; \
+ addr = (kprobe_opcode_t *)ppc_function_entry(addr); \
} else if (name[0] != '.') \
- addr = *(kprobe_opcode_t **)addr; \
+ addr = (kprobe_opcode_t *)ppc_function_entry(addr); \
} else { \
char dot_name[KSYM_NAME_LEN]; \
dot_name[0] = '.'; \
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 4a7cc45..9c89cdd 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -337,6 +337,10 @@
vcpu->kvm->arch.kvm_ops->fast_vcpu_kick(vcpu);
}
+extern void kvm_hv_vm_activated(void);
+extern void kvm_hv_vm_deactivated(void);
+extern bool kvm_hv_mode_active(void);
+
#else
static inline void __init kvm_cma_reserve(void)
{}
@@ -356,6 +360,9 @@
{
kvm_vcpu_kick(vcpu);
}
+
+static inline bool kvm_hv_mode_active(void) { return false; }
+
#endif
#ifdef CONFIG_KVM_XICS
diff --git a/arch/powerpc/include/asm/linkage.h b/arch/powerpc/include/asm/linkage.h
index b36f650..e3ad5c7 100644
--- a/arch/powerpc/include/asm/linkage.h
+++ b/arch/powerpc/include/asm/linkage.h
@@ -2,6 +2,7 @@
#define _ASM_POWERPC_LINKAGE_H
#ifdef CONFIG_PPC64
+#if !defined(_CALL_ELF) || _CALL_ELF != 2
#define cond_syscall(x) \
asm ("\t.weak " #x "\n\t.set " #x ", sys_ni_syscall\n" \
"\t.weak ." #x "\n\t.set ." #x ", .sys_ni_syscall\n")
@@ -9,5 +10,6 @@
asm ("\t.globl " #alias "\n\t.set " #alias ", " #name "\n" \
"\t.globl ." #alias "\n\t.set ." #alias ", ." #name)
#endif
+#endif
#endif /* _ASM_POWERPC_LINKAGE_H */
diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
index 5b6c03f..f92b0b5 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -98,6 +98,9 @@
void (*iommu_save)(void);
void (*iommu_restore)(void);
#endif
+#ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
+ unsigned long (*memory_block_size)(void);
+#endif
#endif /* CONFIG_PPC64 */
void (*pci_dma_dev_setup)(struct pci_dev *dev);
@@ -113,6 +116,8 @@
/* Optional, may be NULL. */
void (*show_cpuinfo)(struct seq_file *m);
void (*show_percpuinfo)(struct seq_file *m, int i);
+ /* Returns the current operating frequency of "cpu" in Hz */
+ unsigned long (*get_proc_freq)(unsigned int cpu);
void (*init_IRQ)(void);
@@ -241,6 +246,9 @@
/* Called during PCI resource reassignment */
resource_size_t (*pcibios_window_alignment)(struct pci_bus *, unsigned long type);
+ /* Reset the secondary bus of bridge */
+ void (*pcibios_reset_secondary_bus)(struct pci_dev *dev);
+
/* Called to shutdown machine specific hardware not already controlled
* by other drivers.
*/
diff --git a/arch/powerpc/include/asm/module.h b/arch/powerpc/include/asm/module.h
index 49fa55b..dcfcad1 100644
--- a/arch/powerpc/include/asm/module.h
+++ b/arch/powerpc/include/asm/module.h
@@ -35,6 +35,7 @@
#ifdef __powerpc64__
unsigned int stubs_section; /* Index of stubs section in module */
unsigned int toc_section; /* What section is the TOC? */
+ bool toc_fixed; /* Have we fixed up .TOC.? */
#ifdef CONFIG_DYNAMIC_FTRACE
unsigned long toc;
unsigned long tramp;
@@ -77,6 +78,9 @@
# endif /* MODULE */
#endif
+bool is_module_trampoline(u32 *insns);
+int module_trampoline_target(struct module *mod, u32 *trampoline,
+ unsigned long *target);
struct exception_table_entry;
void sort_ex_table(struct exception_table_entry *start,
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 66ad7a7..cb15cbb 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -154,6 +154,7 @@
#define OPAL_LPC_READ 67
#define OPAL_LPC_WRITE 68
#define OPAL_RETURN_CPU 69
+#define OPAL_REINIT_CPUS 70
#define OPAL_ELOG_READ 71
#define OPAL_ELOG_WRITE 72
#define OPAL_ELOG_ACK 73
@@ -509,7 +510,7 @@
struct OpalMemoryErrorData {
enum OpalMemErr_Version version:8; /* 0x00 */
enum OpalMemErrType type:8; /* 0x01 */
- uint16_t flags; /* 0x02 */
+ __be16 flags; /* 0x02 */
uint8_t reserved_1[4]; /* 0x04 */
union {
@@ -517,15 +518,15 @@
struct {
enum OpalMemErr_ResilErrType resil_err_type:8;
uint8_t reserved_1[7];
- uint64_t physical_address_start;
- uint64_t physical_address_end;
+ __be64 physical_address_start;
+ __be64 physical_address_end;
} resilience;
/* Dynamic memory deallocation error info */
struct {
enum OpalMemErr_DynErrType dyn_err_type:8;
uint8_t reserved_1[7];
- uint64_t physical_address_start;
- uint64_t physical_address_end;
+ __be64 physical_address_start;
+ __be64 physical_address_end;
} dyn_dealloc;
} u;
};
@@ -725,6 +726,11 @@
uint64_t pestB[OPAL_PHB3_NUM_PEST_REGS];
};
+enum {
+ OPAL_REINIT_CPUS_HILE_BE = (1 << 0),
+ OPAL_REINIT_CPUS_HILE_LE = (1 << 1),
+};
+
typedef struct oppanel_line {
const char * line;
uint64_t line_len;
@@ -849,6 +855,7 @@
uint16_t *pci_error_type, uint16_t *severity);
int64_t opal_pci_poll(uint64_t phb_id);
int64_t opal_return_cpu(void);
+int64_t opal_reinit_cpus(uint64_t flags);
int64_t opal_xscom_read(uint32_t gcid, uint64_t pcb_addr, __be64 *val);
int64_t opal_xscom_write(uint32_t gcid, uint64_t pcb_addr, uint64_t val);
@@ -916,6 +923,7 @@
extern unsigned long opal_get_boot_time(void);
extern void opal_nvram_init(void);
extern void opal_flash_init(void);
+extern void opal_flash_term_callback(void);
extern int opal_elog_init(void);
extern void opal_platform_dump_init(void);
extern void opal_sys_param_init(void);
diff --git a/arch/powerpc/include/asm/paca.h b/arch/powerpc/include/asm/paca.h
index 8e956a0..bb0bd25 100644
--- a/arch/powerpc/include/asm/paca.h
+++ b/arch/powerpc/include/asm/paca.h
@@ -92,7 +92,10 @@
struct slb_shadow *slb_shadow_ptr;
struct dtl_entry *dispatch_log;
struct dtl_entry *dispatch_log_end;
+#endif /* CONFIG_PPC_STD_MMU_64 */
+ u64 dscr_default; /* per-CPU default DSCR */
+#ifdef CONFIG_PPC_STD_MMU_64
/*
* Now, starting in cacheline 2, the exception save areas
*/
diff --git a/arch/powerpc/include/asm/ppc-pci.h b/arch/powerpc/include/asm/ppc-pci.h
index ed57fa7..db1e2b8 100644
--- a/arch/powerpc/include/asm/ppc-pci.h
+++ b/arch/powerpc/include/asm/ppc-pci.h
@@ -58,6 +58,7 @@
int rtas_read_config(struct pci_dn *, int where, int size, u32 *val);
void eeh_pe_state_mark(struct eeh_pe *pe, int state);
void eeh_pe_state_clear(struct eeh_pe *pe, int state);
+void eeh_pe_dev_mode_mark(struct eeh_pe *pe, int mode);
void eeh_sysfs_add_device(struct pci_dev *pdev);
void eeh_sysfs_remove_device(struct pci_dev *pdev);
diff --git a/arch/powerpc/include/asm/ppc_asm.h b/arch/powerpc/include/asm/ppc_asm.h
index cded7c1..9ea266e 100644
--- a/arch/powerpc/include/asm/ppc_asm.h
+++ b/arch/powerpc/include/asm/ppc_asm.h
@@ -57,7 +57,7 @@
LDX_BE r10,0,r10; /* get log write index */ \
cmpd cr1,r11,r10; \
beq+ cr1,33f; \
- bl .accumulate_stolen_time; \
+ bl accumulate_stolen_time; \
ld r12,_MSR(r1); \
andi. r10,r12,MSR_PR; /* Restore cr0 (coming from user) */ \
33: \
@@ -189,9 +189,45 @@
#define __STK_REG(i) (112 + ((i)-14)*8)
#define STK_REG(i) __STK_REG(__REG_##i)
+#if defined(_CALL_ELF) && _CALL_ELF == 2
+#define STK_GOT 24
+#define __STK_PARAM(i) (32 + ((i)-3)*8)
+#else
+#define STK_GOT 40
#define __STK_PARAM(i) (48 + ((i)-3)*8)
+#endif
#define STK_PARAM(i) __STK_PARAM(__REG_##i)
+#if defined(_CALL_ELF) && _CALL_ELF == 2
+
+#define _GLOBAL(name) \
+ .section ".text"; \
+ .align 2 ; \
+ .type name,@function; \
+ .globl name; \
+name:
+
+#define _GLOBAL_TOC(name) \
+ .section ".text"; \
+ .align 2 ; \
+ .type name,@function; \
+ .globl name; \
+name: \
+0: addis r2,r12,(.TOC.-0b)@ha; \
+ addi r2,r2,(.TOC.-0b)@l; \
+ .localentry name,.-name
+
+#define _KPROBE(name) \
+ .section ".kprobes.text","a"; \
+ .align 2 ; \
+ .type name,@function; \
+ .globl name; \
+name:
+
+#define DOTSYM(a) a
+
+#else
+
#define XGLUE(a,b) a##b
#define GLUE(a,b) XGLUE(a,b)
@@ -209,19 +245,7 @@
.type GLUE(.,name),@function; \
GLUE(.,name):
-#define _INIT_GLOBAL(name) \
- __REF; \
- .align 2 ; \
- .globl name; \
- .globl GLUE(.,name); \
- .section ".opd","aw"; \
-name: \
- .quad GLUE(.,name); \
- .quad .TOC.@tocbase; \
- .quad 0; \
- .previous; \
- .type GLUE(.,name),@function; \
-GLUE(.,name):
+#define _GLOBAL_TOC(name) _GLOBAL(name)
#define _KPROBE(name) \
.section ".kprobes.text","a"; \
@@ -237,29 +261,9 @@
.type GLUE(.,name),@function; \
GLUE(.,name):
-#define _STATIC(name) \
- .section ".text"; \
- .align 2 ; \
- .section ".opd","aw"; \
-name: \
- .quad GLUE(.,name); \
- .quad .TOC.@tocbase; \
- .quad 0; \
- .previous; \
- .type GLUE(.,name),@function; \
-GLUE(.,name):
+#define DOTSYM(a) GLUE(.,a)
-#define _INIT_STATIC(name) \
- __REF; \
- .align 2 ; \
- .section ".opd","aw"; \
-name: \
- .quad GLUE(.,name); \
- .quad .TOC.@tocbase; \
- .quad 0; \
- .previous; \
- .type GLUE(.,name),@function; \
-GLUE(.,name):
+#endif
#else /* 32-bit */
diff --git a/arch/powerpc/include/asm/processor.h b/arch/powerpc/include/asm/processor.h
index d660dc3..6d59072 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -449,7 +449,7 @@
enum idle_boot_override {IDLE_NO_OVERRIDE = 0, IDLE_POWERSAVE_OFF};
extern int powersave_nap; /* set if nap mode can be used in idle loop */
-extern void power7_nap(void);
+extern void power7_nap(int check_irq);
extern void power7_sleep(void);
extern void flush_instruction_cache(void);
extern void hard_reset_now(void);
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index 4852bcf..bffd89d 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -215,6 +215,7 @@
#define SPRN_TEXASR 0x82 /* Transaction EXception & Summary */
#define TEXASR_FS __MASK(63-36) /* Transaction Failure Summary */
#define SPRN_TEXASRU 0x83 /* '' '' '' Upper 32 */
+#define TEXASR_FS __MASK(63-36) /* TEXASR Failure Summary */
#define SPRN_TFHAR 0x80 /* Transaction Failure Handler Addr */
#define SPRN_CTRLF 0x088
#define SPRN_CTRLT 0x098
@@ -224,6 +225,7 @@
#define CTRL_TE 0x00c00000 /* thread enable */
#define CTRL_RUNLATCH 0x1
#define SPRN_DAWR 0xB4
+#define SPRN_RPR 0xBA /* Relative Priority Register */
#define SPRN_CIABR 0xBB
#define CIABR_PRIV 0x3
#define CIABR_PRIV_USER 1
@@ -272,8 +274,10 @@
#define SPRN_HSRR1 0x13B /* Hypervisor Save/Restore 1 */
#define SPRN_IC 0x350 /* Virtual Instruction Count */
#define SPRN_VTB 0x351 /* Virtual Time Base */
+#define SPRN_LDBAR 0x352 /* LD Base Address Register */
#define SPRN_PMICR 0x354 /* Power Management Idle Control Reg */
#define SPRN_PMSR 0x355 /* Power Management Status Reg */
+#define SPRN_PMMAR 0x356 /* Power Management Memory Activity Register */
#define SPRN_PMCR 0x374 /* Power Management Control Register */
/* HFSCR and FSCR bit numbers are the same */
@@ -433,6 +437,12 @@
#define HID0_BTCD (1<<1) /* Branch target cache disable */
#define HID0_NOPDST (1<<1) /* No-op dst, dstt, etc. instr. */
#define HID0_NOPTI (1<<0) /* No-op dcbt and dcbst instr. */
+/* POWER8 HID0 bits */
+#define HID0_POWER8_4LPARMODE __MASK(61)
+#define HID0_POWER8_2LPARMODE __MASK(57)
+#define HID0_POWER8_1TO2LPAR __MASK(52)
+#define HID0_POWER8_1TO4LPAR __MASK(51)
+#define HID0_POWER8_DYNLPARDIS __MASK(48)
#define SPRN_HID1 0x3F1 /* Hardware Implementation Register 1 */
#ifdef CONFIG_6xx
diff --git a/arch/powerpc/include/asm/sections.h b/arch/powerpc/include/asm/sections.h
index 5217903..a5e930a 100644
--- a/arch/powerpc/include/asm/sections.h
+++ b/arch/powerpc/include/asm/sections.h
@@ -50,6 +50,7 @@
#endif
}
+#if !defined(_CALL_ELF) || _CALL_ELF != 2
#undef dereference_function_descriptor
static inline void *dereference_function_descriptor(void *ptr)
{
@@ -60,6 +61,7 @@
ptr = p;
return ptr;
}
+#endif
#endif
diff --git a/arch/powerpc/include/asm/smp.h b/arch/powerpc/include/asm/smp.h
index ff51046..5a6614a 100644
--- a/arch/powerpc/include/asm/smp.h
+++ b/arch/powerpc/include/asm/smp.h
@@ -68,14 +68,6 @@
void generic_set_cpu_dead(unsigned int cpu);
void generic_set_cpu_up(unsigned int cpu);
int generic_check_cpu_restart(unsigned int cpu);
-
-extern void inhibit_secondary_onlining(void);
-extern void uninhibit_secondary_onlining(void);
-
-#else /* HOTPLUG_CPU */
-static inline void inhibit_secondary_onlining(void) {}
-static inline void uninhibit_secondary_onlining(void) {}
-
#endif
#ifdef CONFIG_PPC64
diff --git a/arch/powerpc/include/asm/string.h b/arch/powerpc/include/asm/string.h
index 0dffad6..e40010a 100644
--- a/arch/powerpc/include/asm/string.h
+++ b/arch/powerpc/include/asm/string.h
@@ -10,9 +10,7 @@
#define __HAVE_ARCH_STRNCMP
#define __HAVE_ARCH_STRCAT
#define __HAVE_ARCH_MEMSET
-#ifdef __BIG_ENDIAN__
#define __HAVE_ARCH_MEMCPY
-#endif
#define __HAVE_ARCH_MEMMOVE
#define __HAVE_ARCH_MEMCMP
#define __HAVE_ARCH_MEMCHR
@@ -24,9 +22,7 @@
extern int strncmp(const char *, const char *, __kernel_size_t);
extern char * strcat(char *, const char *);
extern void * memset(void *,int,__kernel_size_t);
-#ifdef __BIG_ENDIAN__
extern void * memcpy(void *,const void *,__kernel_size_t);
-#endif
extern void * memmove(void *,const void *,__kernel_size_t);
extern int memcmp(const void *,const void *,__kernel_size_t);
extern void * memchr(const void *,int,__kernel_size_t);
diff --git a/arch/powerpc/include/asm/systbl.h b/arch/powerpc/include/asm/systbl.h
index ea4dc3a..babbeca 100644
--- a/arch/powerpc/include/asm/systbl.h
+++ b/arch/powerpc/include/asm/systbl.h
@@ -62,7 +62,7 @@
SYSCALL(ni_syscall)
SYSCALL_SPU(setpgid)
SYSCALL(ni_syscall)
-SYSX(sys_ni_syscall,sys_olduname, sys_olduname)
+SYSX(sys_ni_syscall,sys_olduname,sys_olduname)
SYSCALL_SPU(umask)
SYSCALL_SPU(chroot)
COMPAT_SYS(ustat)
@@ -190,7 +190,7 @@
SYSCALL_SPU(capget)
SYSCALL_SPU(capset)
COMPAT_SYS(sigaltstack)
-COMPAT_SYS_SPU(sendfile)
+SYSX_SPU(sys_sendfile64,compat_sys_sendfile,sys_sendfile)
SYSCALL(ni_syscall)
SYSCALL(ni_syscall)
PPC_SYS(vfork)
@@ -258,7 +258,7 @@
COMPAT_SYS_SPU(utimes)
COMPAT_SYS_SPU(statfs64)
COMPAT_SYS_SPU(fstatfs64)
-SYSX(sys_ni_syscall, ppc_fadvise64_64, ppc_fadvise64_64)
+SYSX(sys_ni_syscall,ppc_fadvise64_64,ppc_fadvise64_64)
PPC_SYS_SPU(rtas)
OLDSYS(debug_setcontext)
SYSCALL(ni_syscall)
@@ -295,7 +295,7 @@
SYSCALL_SPU(mknodat)
SYSCALL_SPU(fchownat)
COMPAT_SYS_SPU(futimesat)
-SYSX_SPU(sys_newfstatat, sys_fstatat64, sys_fstatat64)
+SYSX_SPU(sys_newfstatat,sys_fstatat64,sys_fstatat64)
SYSCALL_SPU(unlinkat)
SYSCALL_SPU(renameat)
SYSCALL_SPU(linkat)
diff --git a/arch/powerpc/include/asm/topology.h b/arch/powerpc/include/asm/topology.h
index 6c8a8c5..5f1048e 100644
--- a/arch/powerpc/include/asm/topology.h
+++ b/arch/powerpc/include/asm/topology.h
@@ -16,19 +16,6 @@
#include <asm/mmzone.h>
-static inline int cpu_to_node(int cpu)
-{
- int nid;
-
- nid = numa_cpu_lookup_table[cpu];
-
- /*
- * During early boot, the numa-cpu lookup table might not have been
- * setup for all CPUs yet. In such cases, default to node 0.
- */
- return (nid < 0) ? 0 : nid;
-}
-
#define parent_node(node) (node)
#define cpumask_of_node(node) ((node) == -1 ? \
diff --git a/arch/powerpc/include/uapi/asm/Kbuild b/arch/powerpc/include/uapi/asm/Kbuild
index 48be855e..7a3f795 100644
--- a/arch/powerpc/include/uapi/asm/Kbuild
+++ b/arch/powerpc/include/uapi/asm/Kbuild
@@ -15,7 +15,6 @@
header-y += ipcbuf.h
header-y += kvm.h
header-y += kvm_para.h
-header-y += linkage.h
header-y += mman.h
header-y += msgbuf.h
header-y += nvram.h
diff --git a/arch/powerpc/include/uapi/asm/elf.h b/arch/powerpc/include/uapi/asm/elf.h
index 7e39c91..59dad11 100644
--- a/arch/powerpc/include/uapi/asm/elf.h
+++ b/arch/powerpc/include/uapi/asm/elf.h
@@ -291,9 +291,17 @@
#define R_PPC64_DTPREL16_HIGHERA 104 /* half16 (sym+add)@dtprel@highera */
#define R_PPC64_DTPREL16_HIGHEST 105 /* half16 (sym+add)@dtprel@highest */
#define R_PPC64_DTPREL16_HIGHESTA 106 /* half16 (sym+add)@dtprel@highesta */
+#define R_PPC64_TLSGD 107
+#define R_PPC64_TLSLD 108
+#define R_PPC64_TOCSAVE 109
+
+#define R_PPC64_REL16 249
+#define R_PPC64_REL16_LO 250
+#define R_PPC64_REL16_HI 251
+#define R_PPC64_REL16_HA 252
/* Keep this the last entry. */
-#define R_PPC64_NUM 107
+#define R_PPC64_NUM 253
/* There's actually a third entry here, but it's unused */
struct ppc64_opd_entry
diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c
index 93e1465..f5995a9 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -248,6 +248,7 @@
#endif
DEFINE(PACAHWCPUID, offsetof(struct paca_struct, hw_cpu_id));
DEFINE(PACAKEXECSTATE, offsetof(struct paca_struct, kexec_state));
+ DEFINE(PACA_DSCR, offsetof(struct paca_struct, dscr_default));
DEFINE(PACA_STARTTIME, offsetof(struct paca_struct, starttime));
DEFINE(PACA_STARTTIME_USER, offsetof(struct paca_struct, starttime_user));
DEFINE(PACA_USER_TIME, offsetof(struct paca_struct, user_time));
diff --git a/arch/powerpc/kernel/cpu_setup_fsl_booke.S b/arch/powerpc/kernel/cpu_setup_fsl_booke.S
index cc2d896..4f1393d 100644
--- a/arch/powerpc/kernel/cpu_setup_fsl_booke.S
+++ b/arch/powerpc/kernel/cpu_setup_fsl_booke.S
@@ -94,12 +94,12 @@
_GLOBAL(__setup_cpu_e6500)
mflr r6
#ifdef CONFIG_PPC64
- bl .setup_altivec_ivors
+ bl setup_altivec_ivors
/* Touch IVOR42 only if the CPU supports E.HV category */
mfspr r10,SPRN_MMUCFG
rlwinm. r10,r10,0,MMUCFG_LPIDSIZE
beq 1f
- bl .setup_lrat_ivor
+ bl setup_lrat_ivor
1:
#endif
bl setup_pw20_idle
@@ -164,15 +164,15 @@
#ifdef CONFIG_PPC_BOOK3E_64
_GLOBAL(__restore_cpu_e6500)
mflr r5
- bl .setup_altivec_ivors
+ bl setup_altivec_ivors
/* Touch IVOR42 only if the CPU supports E.HV category */
mfspr r10,SPRN_MMUCFG
rlwinm. r10,r10,0,MMUCFG_LPIDSIZE
beq 1f
- bl .setup_lrat_ivor
+ bl setup_lrat_ivor
1:
- bl .setup_pw20_idle
- bl .setup_altivec_idle
+ bl setup_pw20_idle
+ bl setup_altivec_idle
bl __restore_cpu_e5500
mtlr r5
blr
@@ -181,9 +181,9 @@
mflr r4
bl __e500_icache_setup
bl __e500_dcache_setup
- bl .__setup_base_ivors
- bl .setup_perfmon_ivor
- bl .setup_doorbell_ivors
+ bl __setup_base_ivors
+ bl setup_perfmon_ivor
+ bl setup_doorbell_ivors
/*
* We only want to touch IVOR38-41 if we're running on hardware
* that supports category E.HV. The architectural way to determine
@@ -192,7 +192,7 @@
mfspr r10,SPRN_MMUCFG
rlwinm. r10,r10,0,MMUCFG_LPIDSIZE
beq 1f
- bl .setup_ehv_ivors
+ bl setup_ehv_ivors
1:
mtlr r4
blr
@@ -201,9 +201,9 @@
mflr r5
bl __e500_icache_setup
bl __e500_dcache_setup
- bl .__setup_base_ivors
- bl .setup_perfmon_ivor
- bl .setup_doorbell_ivors
+ bl __setup_base_ivors
+ bl setup_perfmon_ivor
+ bl setup_doorbell_ivors
/*
* We only want to touch IVOR38-41 if we're running on hardware
* that supports category E.HV. The architectural way to determine
@@ -212,7 +212,7 @@
mfspr r10,SPRN_MMUCFG
rlwinm. r10,r10,0,MMUCFG_LPIDSIZE
beq 1f
- bl .setup_ehv_ivors
+ bl setup_ehv_ivors
b 2f
1:
ld r10,CPU_SPEC_FEATURES(r4)
diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
index e7b76a6..7051ea3 100644
--- a/arch/powerpc/kernel/eeh.c
+++ b/arch/powerpc/kernel/eeh.c
@@ -22,6 +22,7 @@
*/
#include <linux/delay.h>
+#include <linux/debugfs.h>
#include <linux/sched.h>
#include <linux/init.h>
#include <linux/list.h>
@@ -35,6 +36,7 @@
#include <linux/of.h>
#include <linux/atomic.h>
+#include <asm/debug.h>
#include <asm/eeh.h>
#include <asm/eeh_event.h>
#include <asm/io.h>
@@ -87,23 +89,22 @@
/* Time to wait for a PCI slot to report status, in milliseconds */
#define PCI_BUS_RESET_WAIT_MSEC (5*60*1000)
+/*
+ * EEH probe mode support, which is part of the flags,
+ * is to support multiple platforms for EEH. Some platforms
+ * like pSeries do PCI emunation based on device tree.
+ * However, other platforms like powernv probe PCI devices
+ * from hardware. The flag is used to distinguish that.
+ * In addition, struct eeh_ops::probe would be invoked for
+ * particular OF node or PCI device so that the corresponding
+ * PE would be created there.
+ */
+int eeh_subsystem_flags;
+EXPORT_SYMBOL(eeh_subsystem_flags);
+
/* Platform dependent EEH operations */
struct eeh_ops *eeh_ops = NULL;
-bool eeh_subsystem_enabled = false;
-EXPORT_SYMBOL(eeh_subsystem_enabled);
-
-/*
- * EEH probe mode support. The intention is to support multiple
- * platforms for EEH. Some platforms like pSeries do PCI emunation
- * based on device tree. However, other platforms like powernv probe
- * PCI devices from hardware. The flag is used to distinguish that.
- * In addition, struct eeh_ops::probe would be invoked for particular
- * OF node or PCI device so that the corresponding PE would be created
- * there.
- */
-int eeh_probe_mode;
-
/* Lock to avoid races due to multiple reports of an error */
DEFINE_RAW_SPINLOCK(confirm_error_lock);
@@ -133,6 +134,15 @@
#define IS_BRIDGE(class_code) (((class_code)<<16) == PCI_BASE_CLASS_BRIDGE)
+static int __init eeh_setup(char *str)
+{
+ if (!strcmp(str, "off"))
+ eeh_subsystem_flags |= EEH_FORCE_DISABLED;
+
+ return 1;
+}
+__setup("eeh=", eeh_setup);
+
/**
* eeh_gather_pci_data - Copy assorted PCI config space registers to buff
* @edev: device to report data for
@@ -145,73 +155,67 @@
static size_t eeh_gather_pci_data(struct eeh_dev *edev, char * buf, size_t len)
{
struct device_node *dn = eeh_dev_to_of_node(edev);
- struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
u32 cfg;
int cap, i;
int n = 0;
n += scnprintf(buf+n, len-n, "%s\n", dn->full_name);
- printk(KERN_WARNING "EEH: of node=%s\n", dn->full_name);
+ pr_warn("EEH: of node=%s\n", dn->full_name);
eeh_ops->read_config(dn, PCI_VENDOR_ID, 4, &cfg);
n += scnprintf(buf+n, len-n, "dev/vend:%08x\n", cfg);
- printk(KERN_WARNING "EEH: PCI device/vendor: %08x\n", cfg);
+ pr_warn("EEH: PCI device/vendor: %08x\n", cfg);
eeh_ops->read_config(dn, PCI_COMMAND, 4, &cfg);
n += scnprintf(buf+n, len-n, "cmd/stat:%x\n", cfg);
- printk(KERN_WARNING "EEH: PCI cmd/status register: %08x\n", cfg);
-
- if (!dev) {
- printk(KERN_WARNING "EEH: no PCI device for this of node\n");
- return n;
- }
+ pr_warn("EEH: PCI cmd/status register: %08x\n", cfg);
/* Gather bridge-specific registers */
- if (dev->class >> 16 == PCI_BASE_CLASS_BRIDGE) {
+ if (edev->mode & EEH_DEV_BRIDGE) {
eeh_ops->read_config(dn, PCI_SEC_STATUS, 2, &cfg);
n += scnprintf(buf+n, len-n, "sec stat:%x\n", cfg);
- printk(KERN_WARNING "EEH: Bridge secondary status: %04x\n", cfg);
+ pr_warn("EEH: Bridge secondary status: %04x\n", cfg);
eeh_ops->read_config(dn, PCI_BRIDGE_CONTROL, 2, &cfg);
n += scnprintf(buf+n, len-n, "brdg ctl:%x\n", cfg);
- printk(KERN_WARNING "EEH: Bridge control: %04x\n", cfg);
+ pr_warn("EEH: Bridge control: %04x\n", cfg);
}
/* Dump out the PCI-X command and status regs */
- cap = pci_find_capability(dev, PCI_CAP_ID_PCIX);
+ cap = edev->pcix_cap;
if (cap) {
eeh_ops->read_config(dn, cap, 4, &cfg);
n += scnprintf(buf+n, len-n, "pcix-cmd:%x\n", cfg);
- printk(KERN_WARNING "EEH: PCI-X cmd: %08x\n", cfg);
+ pr_warn("EEH: PCI-X cmd: %08x\n", cfg);
eeh_ops->read_config(dn, cap+4, 4, &cfg);
n += scnprintf(buf+n, len-n, "pcix-stat:%x\n", cfg);
- printk(KERN_WARNING "EEH: PCI-X status: %08x\n", cfg);
+ pr_warn("EEH: PCI-X status: %08x\n", cfg);
}
- /* If PCI-E capable, dump PCI-E cap 10, and the AER */
- if (pci_is_pcie(dev)) {
+ /* If PCI-E capable, dump PCI-E cap 10 */
+ cap = edev->pcie_cap;
+ if (cap) {
n += scnprintf(buf+n, len-n, "pci-e cap10:\n");
- printk(KERN_WARNING
- "EEH: PCI-E capabilities and status follow:\n");
+ pr_warn("EEH: PCI-E capabilities and status follow:\n");
for (i=0; i<=8; i++) {
- eeh_ops->read_config(dn, dev->pcie_cap+4*i, 4, &cfg);
+ eeh_ops->read_config(dn, cap+4*i, 4, &cfg);
n += scnprintf(buf+n, len-n, "%02x:%x\n", 4*i, cfg);
- printk(KERN_WARNING "EEH: PCI-E %02x: %08x\n", i, cfg);
+ pr_warn("EEH: PCI-E %02x: %08x\n", i, cfg);
}
+ }
- cap = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ERR);
- if (cap) {
- n += scnprintf(buf+n, len-n, "pci-e AER:\n");
- printk(KERN_WARNING
- "EEH: PCI-E AER capability register set follows:\n");
+ /* If AER capable, dump it */
+ cap = edev->aer_cap;
+ if (cap) {
+ n += scnprintf(buf+n, len-n, "pci-e AER:\n");
+ pr_warn("EEH: PCI-E AER capability register set follows:\n");
- for (i=0; i<14; i++) {
- eeh_ops->read_config(dn, cap+4*i, 4, &cfg);
- n += scnprintf(buf+n, len-n, "%02x:%x\n", 4*i, cfg);
- printk(KERN_WARNING "EEH: PCI-E AER %02x: %08x\n", i, cfg);
- }
+ for (i=0; i<14; i++) {
+ eeh_ops->read_config(dn, cap+4*i, 4, &cfg);
+ n += scnprintf(buf+n, len-n, "%02x:%x\n", 4*i, cfg);
+ pr_warn("EEH: PCI-E AER %02x: %08x\n", i, cfg);
}
}
@@ -232,21 +236,19 @@
{
size_t loglen = 0;
struct eeh_dev *edev, *tmp;
- bool valid_cfg_log = true;
/*
* When the PHB is fenced or dead, it's pointless to collect
* the data from PCI config space because it should return
* 0xFF's. For ER, we still retrieve the data from the PCI
* config space.
+ *
+ * For pHyp, we have to enable IO for log retrieval. Otherwise,
+ * 0xFF's is always returned from PCI config space.
*/
- if (eeh_probe_mode_dev() &&
- (pe->type & EEH_PE_PHB) &&
- (pe->state & (EEH_PE_ISOLATED | EEH_PE_PHB_DEAD)))
- valid_cfg_log = false;
-
- if (valid_cfg_log) {
- eeh_pci_enable(pe, EEH_OPT_THAW_MMIO);
+ if (!(pe->type & EEH_PE_PHB)) {
+ if (eeh_probe_mode_devtree())
+ eeh_pci_enable(pe, EEH_OPT_THAW_MMIO);
eeh_ops->configure_bridge(pe);
eeh_pe_restore_bars(pe);
@@ -309,7 +311,7 @@
/* If the PHB has been in problematic state */
eeh_serialize_lock(&flags);
- if (phb_pe->state & (EEH_PE_ISOLATED | EEH_PE_PHB_DEAD)) {
+ if (phb_pe->state & EEH_PE_ISOLATED) {
ret = 0;
goto out;
}
@@ -515,16 +517,42 @@
*/
int eeh_pci_enable(struct eeh_pe *pe, int function)
{
- int rc;
+ int rc, flags = (EEH_STATE_MMIO_ACTIVE | EEH_STATE_DMA_ACTIVE);
+
+ /*
+ * pHyp doesn't allow to enable IO or DMA on unfrozen PE.
+ * Also, it's pointless to enable them on unfrozen PE. So
+ * we have the check here.
+ */
+ if (function == EEH_OPT_THAW_MMIO ||
+ function == EEH_OPT_THAW_DMA) {
+ rc = eeh_ops->get_state(pe, NULL);
+ if (rc < 0)
+ return rc;
+
+ /* Needn't to enable or already enabled */
+ if ((rc == EEH_STATE_NOT_SUPPORT) ||
+ ((rc & flags) == flags))
+ return 0;
+ }
rc = eeh_ops->set_option(pe, function);
if (rc)
- pr_warning("%s: Unexpected state change %d on PHB#%d-PE#%x, err=%d\n",
- __func__, function, pe->phb->global_number, pe->addr, rc);
+ pr_warn("%s: Unexpected state change %d on "
+ "PHB#%d-PE#%x, err=%d\n",
+ __func__, function, pe->phb->global_number,
+ pe->addr, rc);
rc = eeh_ops->wait_state(pe, PCI_BUS_RESET_WAIT_MSEC);
- if (rc > 0 && (rc & EEH_STATE_MMIO_ENABLED) &&
- (function == EEH_OPT_THAW_MMIO))
+ if (rc <= 0)
+ return rc;
+
+ if ((function == EEH_OPT_THAW_MMIO) &&
+ (rc & EEH_STATE_MMIO_ENABLED))
+ return 0;
+
+ if ((function == EEH_OPT_THAW_DMA) &&
+ (rc & EEH_STATE_DMA_ENABLED))
return 0;
return rc;
@@ -612,26 +640,7 @@
else
eeh_ops->reset(pe, EEH_RESET_HOT);
- /* The PCI bus requires that the reset be held high for at least
- * a 100 milliseconds. We wait a bit longer 'just in case'.
- */
-#define PCI_BUS_RST_HOLD_TIME_MSEC 250
- msleep(PCI_BUS_RST_HOLD_TIME_MSEC);
-
- /* We might get hit with another EEH freeze as soon as the
- * pci slot reset line is dropped. Make sure we don't miss
- * these, and clear the flag now.
- */
- eeh_pe_state_clear(pe, EEH_PE_ISOLATED);
-
eeh_ops->reset(pe, EEH_RESET_DEACTIVATE);
-
- /* After a PCI slot has been reset, the PCI Express spec requires
- * a 1.5 second idle time for the bus to stabilize, before starting
- * up traffic.
- */
-#define PCI_BUS_SETTLE_TIME_MSEC 1800
- msleep(PCI_BUS_SETTLE_TIME_MSEC);
}
/**
@@ -651,6 +660,10 @@
for (i=0; i<3; i++) {
eeh_reset_pe_once(pe);
+ /*
+ * EEH_PE_ISOLATED is expected to be removed after
+ * BAR restore.
+ */
rc = eeh_ops->wait_state(pe, PCI_BUS_RESET_WAIT_MSEC);
if ((rc & flags) == flags)
return 0;
@@ -826,8 +839,8 @@
&hose_list, list_node)
pci_walk_bus(hose->bus, eeh_ops->dev_probe, NULL);
} else {
- pr_warning("%s: Invalid probe mode %d\n",
- __func__, eeh_probe_mode);
+ pr_warn("%s: Invalid probe mode %x",
+ __func__, eeh_subsystem_flags);
return -EINVAL;
}
@@ -1102,10 +1115,45 @@
.release = single_release,
};
+#ifdef CONFIG_DEBUG_FS
+static int eeh_enable_dbgfs_set(void *data, u64 val)
+{
+ if (val)
+ eeh_subsystem_flags &= ~EEH_FORCE_DISABLED;
+ else
+ eeh_subsystem_flags |= EEH_FORCE_DISABLED;
+
+ /* Notify the backend */
+ if (eeh_ops->post_init)
+ eeh_ops->post_init();
+
+ return 0;
+}
+
+static int eeh_enable_dbgfs_get(void *data, u64 *val)
+{
+ if (eeh_enabled())
+ *val = 0x1ul;
+ else
+ *val = 0x0ul;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(eeh_enable_dbgfs_ops, eeh_enable_dbgfs_get,
+ eeh_enable_dbgfs_set, "0x%llx\n");
+#endif
+
static int __init eeh_init_proc(void)
{
- if (machine_is(pseries) || machine_is(powernv))
+ if (machine_is(pseries) || machine_is(powernv)) {
proc_create("powerpc/eeh", 0, NULL, &proc_eeh_operations);
+#ifdef CONFIG_DEBUG_FS
+ debugfs_create_file("eeh_enable", 0600,
+ powerpc_debugfs_root, NULL,
+ &eeh_enable_dbgfs_ops);
+#endif
+ }
+
return 0;
}
__initcall(eeh_init_proc);
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index bb61ca5..7100a5b 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -171,6 +171,15 @@
}
}
+static bool eeh_dev_removed(struct eeh_dev *edev)
+{
+ /* EEH device removed ? */
+ if (!edev || (edev->mode & EEH_DEV_REMOVED))
+ return true;
+
+ return false;
+}
+
/**
* eeh_report_error - Report pci error to each device driver
* @data: eeh device
@@ -187,10 +196,8 @@
enum pci_ers_result rc, *res = userdata;
struct pci_driver *driver;
- /* We might not have the associated PCI device,
- * then we should continue for next one.
- */
- if (!dev) return NULL;
+ if (!dev || eeh_dev_removed(edev))
+ return NULL;
dev->error_state = pci_channel_io_frozen;
driver = eeh_pcid_get(dev);
@@ -230,6 +237,9 @@
enum pci_ers_result rc, *res = userdata;
struct pci_driver *driver;
+ if (!dev || eeh_dev_removed(edev))
+ return NULL;
+
driver = eeh_pcid_get(dev);
if (!driver) return NULL;
@@ -267,7 +277,8 @@
enum pci_ers_result rc, *res = userdata;
struct pci_driver *driver;
- if (!dev) return NULL;
+ if (!dev || eeh_dev_removed(edev))
+ return NULL;
dev->error_state = pci_channel_io_normal;
driver = eeh_pcid_get(dev);
@@ -307,7 +318,8 @@
struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
struct pci_driver *driver;
- if (!dev) return NULL;
+ if (!dev || eeh_dev_removed(edev))
+ return NULL;
dev->error_state = pci_channel_io_normal;
driver = eeh_pcid_get(dev);
@@ -343,7 +355,8 @@
struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
struct pci_driver *driver;
- if (!dev) return NULL;
+ if (!dev || eeh_dev_removed(edev))
+ return NULL;
dev->error_state = pci_channel_io_perm_failure;
driver = eeh_pcid_get(dev);
@@ -380,6 +393,16 @@
if (!dev || (dev->hdr_type & PCI_HEADER_TYPE_BRIDGE))
return NULL;
+ /*
+ * We rely on count-based pcibios_release_device() to
+ * detach permanently offlined PEs. Unfortunately, that's
+ * not reliable enough. We might have the permanently
+ * offlined PEs attached, but we needn't take care of
+ * them and their child devices.
+ */
+ if (eeh_dev_removed(edev))
+ return NULL;
+
driver = eeh_pcid_get(dev);
if (driver) {
eeh_pcid_put(dev);
@@ -417,6 +440,36 @@
return NULL;
}
+/*
+ * Explicitly clear PE's frozen state for PowerNV where
+ * we have frozen PE until BAR restore is completed. It's
+ * harmless to clear it for pSeries. To be consistent with
+ * PE reset (for 3 times), we try to clear the frozen state
+ * for 3 times as well.
+ */
+static int eeh_clear_pe_frozen_state(struct eeh_pe *pe)
+{
+ int i, rc;
+
+ for (i = 0; i < 3; i++) {
+ rc = eeh_pci_enable(pe, EEH_OPT_THAW_MMIO);
+ if (rc)
+ continue;
+ rc = eeh_pci_enable(pe, EEH_OPT_THAW_DMA);
+ if (!rc)
+ break;
+ }
+
+ /* The PE has been isolated, clear it */
+ if (rc)
+ pr_warn("%s: Can't clear frozen PHB#%x-PE#%x (%d)\n",
+ __func__, pe->phb->global_number, pe->addr, rc);
+ else
+ eeh_pe_state_clear(pe, EEH_PE_ISOLATED);
+
+ return rc;
+}
+
/**
* eeh_reset_device - Perform actual reset of a pci slot
* @pe: EEH PE
@@ -451,19 +504,33 @@
eeh_pe_dev_traverse(pe, eeh_rmv_device, &removed);
}
- /* Reset the pci controller. (Asserts RST#; resets config space).
+ /*
+ * Reset the pci controller. (Asserts RST#; resets config space).
* Reconfigure bridges and devices. Don't try to bring the system
* up if the reset failed for some reason.
+ *
+ * During the reset, it's very dangerous to have uncontrolled PCI
+ * config accesses. So we prefer to block them. However, controlled
+ * PCI config accesses initiated from EEH itself are allowed.
*/
+ eeh_pe_state_mark(pe, EEH_PE_RESET);
rc = eeh_reset_pe(pe);
- if (rc)
+ if (rc) {
+ eeh_pe_state_clear(pe, EEH_PE_RESET);
return rc;
+ }
pci_lock_rescan_remove();
/* Restore PE */
eeh_ops->configure_bridge(pe);
eeh_pe_restore_bars(pe);
+ eeh_pe_state_clear(pe, EEH_PE_RESET);
+
+ /* Clear frozen state */
+ rc = eeh_clear_pe_frozen_state(pe);
+ if (rc)
+ return rc;
/* Give the system 5 seconds to finish running the user-space
* hotplug shutdown scripts, e.g. ifdown for ethernet. Yes,
@@ -573,7 +640,6 @@
result = PCI_ERS_RESULT_NEED_RESET;
} else {
pr_info("EEH: Notify device drivers to resume I/O\n");
- result = PCI_ERS_RESULT_NONE;
eeh_pe_dev_traverse(pe, eeh_report_mmio_enabled, &result);
}
}
@@ -585,10 +651,17 @@
if (rc < 0)
goto hard_fail;
- if (rc)
+ if (rc) {
result = PCI_ERS_RESULT_NEED_RESET;
- else
+ } else {
+ /*
+ * We didn't do PE reset for the case. The PE
+ * is still in frozen state. Clear it before
+ * resuming the PE.
+ */
+ eeh_pe_state_clear(pe, EEH_PE_ISOLATED);
result = PCI_ERS_RESULT_RECOVERED;
+ }
}
/* If any device has a hard failure, then shut off everything. */
@@ -650,8 +723,17 @@
/* Notify all devices that they're about to go down. */
eeh_pe_dev_traverse(pe, eeh_report_failure, NULL);
- /* Shut down the device drivers for good. */
+ /* Mark the PE to be removed permanently */
+ pe->freeze_count = EEH_MAX_ALLOWED_FREEZES + 1;
+
+ /*
+ * Shut down the device drivers for good. We mark
+ * all removed devices correctly to avoid access
+ * the their PCI config any more.
+ */
if (frozen_bus) {
+ eeh_pe_dev_mode_mark(pe, EEH_DEV_REMOVED);
+
pci_lock_rescan_remove();
pcibios_remove_pci_devices(frozen_bus);
pci_unlock_rescan_remove();
@@ -682,8 +764,7 @@
phb_pe = eeh_phb_pe_get(hose);
if (!phb_pe) continue;
- eeh_pe_state_mark(phb_pe,
- EEH_PE_ISOLATED | EEH_PE_PHB_DEAD);
+ eeh_pe_state_mark(phb_pe, EEH_PE_ISOLATED);
}
eeh_serialize_unlock(flags);
@@ -699,8 +780,7 @@
eeh_remove_event(pe);
if (rc == EEH_NEXT_ERR_DEAD_PHB)
- eeh_pe_state_mark(pe,
- EEH_PE_ISOLATED | EEH_PE_PHB_DEAD);
+ eeh_pe_state_mark(pe, EEH_PE_ISOLATED);
else
eeh_pe_state_mark(pe,
EEH_PE_ISOLATED | EEH_PE_RECOVERING);
@@ -724,12 +804,14 @@
if (rc == EEH_NEXT_ERR_FROZEN_PE ||
rc == EEH_NEXT_ERR_FENCED_PHB) {
eeh_handle_normal_event(pe);
+ eeh_pe_state_clear(pe, EEH_PE_RECOVERING);
} else {
pci_lock_rescan_remove();
list_for_each_entry(hose, &hose_list, list_node) {
phb_pe = eeh_phb_pe_get(hose);
if (!phb_pe ||
- !(phb_pe->state & EEH_PE_PHB_DEAD))
+ !(phb_pe->state & EEH_PE_ISOLATED) ||
+ (phb_pe->state & EEH_PE_RECOVERING))
continue;
/* Notify all devices to be down */
diff --git a/arch/powerpc/kernel/eeh_pe.c b/arch/powerpc/kernel/eeh_pe.c
index f0c353f..995c2a2 100644
--- a/arch/powerpc/kernel/eeh_pe.c
+++ b/arch/powerpc/kernel/eeh_pe.c
@@ -503,13 +503,17 @@
struct eeh_dev *edev, *tmp;
struct pci_dev *pdev;
- /*
- * Mark the PE with the indicated state. Also,
- * the associated PCI device will be put into
- * I/O frozen state to avoid I/O accesses from
- * the PCI device driver.
- */
+ /* Keep the state of permanently removed PE intact */
+ if ((pe->freeze_count > EEH_MAX_ALLOWED_FREEZES) &&
+ (state & (EEH_PE_ISOLATED | EEH_PE_RECOVERING)))
+ return NULL;
+
pe->state |= state;
+
+ /* Offline PCI devices if applicable */
+ if (state != EEH_PE_ISOLATED)
+ return NULL;
+
eeh_pe_for_each_dev(pe, edev, tmp) {
pdev = eeh_dev_to_pci_dev(edev);
if (pdev)
@@ -532,6 +536,27 @@
eeh_pe_traverse(pe, __eeh_pe_state_mark, &state);
}
+static void *__eeh_pe_dev_mode_mark(void *data, void *flag)
+{
+ struct eeh_dev *edev = data;
+ int mode = *((int *)flag);
+
+ edev->mode |= mode;
+
+ return NULL;
+}
+
+/**
+ * eeh_pe_dev_state_mark - Mark state for all device under the PE
+ * @pe: EEH PE
+ *
+ * Mark specific state for all child devices of the PE.
+ */
+void eeh_pe_dev_mode_mark(struct eeh_pe *pe, int mode)
+{
+ eeh_pe_dev_traverse(pe, __eeh_pe_dev_mode_mark, &mode);
+}
+
/**
* __eeh_pe_state_clear - Clear state for the PE
* @data: EEH PE
@@ -546,8 +571,16 @@
struct eeh_pe *pe = (struct eeh_pe *)data;
int state = *((int *)flag);
+ /* Keep the state of permanently removed PE intact */
+ if ((pe->freeze_count > EEH_MAX_ALLOWED_FREEZES) &&
+ (state & EEH_PE_ISOLATED))
+ return NULL;
+
pe->state &= ~state;
- pe->check_count = 0;
+
+ /* Clear check count since last isolation */
+ if (state & EEH_PE_ISOLATED)
+ pe->check_count = 0;
return NULL;
}
diff --git a/arch/powerpc/kernel/eeh_sysfs.c b/arch/powerpc/kernel/eeh_sysfs.c
index 5d753d4..e2595ba 100644
--- a/arch/powerpc/kernel/eeh_sysfs.c
+++ b/arch/powerpc/kernel/eeh_sysfs.c
@@ -59,6 +59,9 @@
struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
int rc=0;
+ if (!eeh_enabled())
+ return;
+
if (edev && (edev->mode & EEH_DEV_SYSFS))
return;
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 662c6dd..911d453 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -39,8 +39,8 @@
* System calls.
*/
.section ".toc","aw"
-.SYS_CALL_TABLE:
- .tc .sys_call_table[TC],.sys_call_table
+SYS_CALL_TABLE:
+ .tc sys_call_table[TC],sys_call_table
/* This value is used to mark exception frames on the stack. */
exception_marker:
@@ -106,7 +106,7 @@
LDX_BE r10,0,r10 /* get log write index */
cmpd cr1,r11,r10
beq+ cr1,33f
- bl .accumulate_stolen_time
+ bl accumulate_stolen_time
REST_GPR(0,r1)
REST_4GPRS(3,r1)
REST_2GPRS(7,r1)
@@ -143,7 +143,7 @@
std r10,SOFTE(r1)
#ifdef SHOW_SYSCALLS
- bl .do_show_syscall
+ bl do_show_syscall
REST_GPR(0,r1)
REST_4GPRS(3,r1)
REST_2GPRS(7,r1)
@@ -162,7 +162,7 @@
* Need to vector to 32 Bit or default sys_call_table here,
* based on caller's run-mode / personality.
*/
- ld r11,.SYS_CALL_TABLE@toc(2)
+ ld r11,SYS_CALL_TABLE@toc(2)
andi. r10,r10,_TIF_32BIT
beq 15f
addi r11,r11,8 /* use 32-bit syscall entries */
@@ -174,14 +174,14 @@
clrldi r8,r8,32
15:
slwi r0,r0,4
- ldx r10,r11,r0 /* Fetch system call handler [ptr] */
- mtctr r10
+ ldx r12,r11,r0 /* Fetch system call handler [ptr] */
+ mtctr r12
bctrl /* Call handler */
syscall_exit:
std r3,RESULT(r1)
#ifdef SHOW_SYSCALLS
- bl .do_show_syscall_exit
+ bl do_show_syscall_exit
ld r3,RESULT(r1)
#endif
CURRENT_THREAD_INFO(r12, r1)
@@ -248,9 +248,9 @@
/* Traced system call support */
syscall_dotrace:
- bl .save_nvgprs
+ bl save_nvgprs
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .do_syscall_trace_enter
+ bl do_syscall_trace_enter
/*
* Restore argument registers possibly just changed.
* We use the return value of do_syscall_trace_enter
@@ -308,7 +308,7 @@
4: /* Anything else left to do? */
SET_DEFAULT_THREAD_PPR(r3, r10) /* Set thread.ppr = 3 */
andi. r0,r9,(_TIF_SYSCALL_T_OR_A|_TIF_SINGLESTEP)
- beq .ret_from_except_lite
+ beq ret_from_except_lite
/* Re-enable interrupts */
#ifdef CONFIG_PPC_BOOK3E
@@ -319,10 +319,10 @@
mtmsrd r10,1
#endif /* CONFIG_PPC_BOOK3E */
- bl .save_nvgprs
+ bl save_nvgprs
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .do_syscall_trace_leave
- b .ret_from_except
+ bl do_syscall_trace_leave
+ b ret_from_except
/* Save non-volatile GPRs, if not already saved. */
_GLOBAL(save_nvgprs)
@@ -345,52 +345,48 @@
*/
_GLOBAL(ppc_fork)
- bl .save_nvgprs
- bl .sys_fork
+ bl save_nvgprs
+ bl sys_fork
b syscall_exit
_GLOBAL(ppc_vfork)
- bl .save_nvgprs
- bl .sys_vfork
+ bl save_nvgprs
+ bl sys_vfork
b syscall_exit
_GLOBAL(ppc_clone)
- bl .save_nvgprs
- bl .sys_clone
+ bl save_nvgprs
+ bl sys_clone
b syscall_exit
_GLOBAL(ppc32_swapcontext)
- bl .save_nvgprs
- bl .compat_sys_swapcontext
+ bl save_nvgprs
+ bl compat_sys_swapcontext
b syscall_exit
_GLOBAL(ppc64_swapcontext)
- bl .save_nvgprs
- bl .sys_swapcontext
+ bl save_nvgprs
+ bl sys_swapcontext
b syscall_exit
_GLOBAL(ret_from_fork)
- bl .schedule_tail
+ bl schedule_tail
REST_NVGPRS(r1)
li r3,0
b syscall_exit
_GLOBAL(ret_from_kernel_thread)
- bl .schedule_tail
+ bl schedule_tail
REST_NVGPRS(r1)
- ld r14, 0(r14)
mtlr r14
mr r3,r15
+#if defined(_CALL_ELF) && _CALL_ELF == 2
+ mr r12,r14
+#endif
blrl
li r3,0
b syscall_exit
- .section ".toc","aw"
-DSCR_DEFAULT:
- .tc dscr_default[TC],dscr_default
-
- .section ".text"
-
/*
* This routine switches between two different tasks. The process
* state of one is saved on its kernel stack. Then the state
@@ -575,11 +571,10 @@
#ifdef CONFIG_PPC64
BEGIN_FTR_SECTION
lwz r6,THREAD_DSCR_INHERIT(r4)
- ld r7,DSCR_DEFAULT@toc(2)
ld r0,THREAD_DSCR(r4)
cmpwi r6,0
bne 1f
- ld r0,0(r7)
+ ld r0,PACA_DSCR(r13)
1:
BEGIN_FTR_SECTION_NESTED(70)
mfspr r8, SPRN_FSCR
@@ -611,7 +606,7 @@
_GLOBAL(ret_from_except)
ld r11,_TRAP(r1)
andi. r0,r11,1
- bne .ret_from_except_lite
+ bne ret_from_except_lite
REST_NVGPRS(r1)
_GLOBAL(ret_from_except_lite)
@@ -661,23 +656,23 @@
#endif
1: andi. r0,r4,_TIF_NEED_RESCHED
beq 2f
- bl .restore_interrupts
+ bl restore_interrupts
SCHEDULE_USER
- b .ret_from_except_lite
+ b ret_from_except_lite
2:
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
andi. r0,r4,_TIF_USER_WORK_MASK & ~_TIF_RESTORE_TM
bne 3f /* only restore TM if nothing else to do */
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .restore_tm_state
+ bl restore_tm_state
b restore
3:
#endif
- bl .save_nvgprs
- bl .restore_interrupts
+ bl save_nvgprs
+ bl restore_interrupts
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .do_notify_resume
- b .ret_from_except
+ bl do_notify_resume
+ b ret_from_except
resume_kernel:
/* check current_thread_info, _TIF_EMULATE_STACK_STORE */
@@ -730,7 +725,7 @@
* sure we are soft-disabled first and reconcile irq state.
*/
RECONCILE_IRQ_STATE(r3,r4)
-1: bl .preempt_schedule_irq
+1: bl preempt_schedule_irq
/* Re-test flags and eventually loop */
CURRENT_THREAD_INFO(r9, r1)
@@ -792,7 +787,7 @@
*/
do_restore:
#ifdef CONFIG_PPC_BOOK3E
- b .exception_return_book3e
+ b exception_return_book3e
#else
/*
* Clear the reservation. If we know the CPU tracks the address of
@@ -907,7 +902,7 @@
*
* Still, this might be useful for things like hash_page
*/
- bl .__check_irq_replay
+ bl __check_irq_replay
cmpwi cr0,r3,0
beq restore_no_replay
@@ -928,13 +923,13 @@
cmpwi cr0,r3,0x500
bne 1f
addi r3,r1,STACK_FRAME_OVERHEAD;
- bl .do_IRQ
- b .ret_from_except
+ bl do_IRQ
+ b ret_from_except
1: cmpwi cr0,r3,0x900
bne 1f
addi r3,r1,STACK_FRAME_OVERHEAD;
- bl .timer_interrupt
- b .ret_from_except
+ bl timer_interrupt
+ b ret_from_except
#ifdef CONFIG_PPC_DOORBELL
1:
#ifdef CONFIG_PPC_BOOK3E
@@ -948,14 +943,14 @@
#endif /* CONFIG_PPC_BOOK3E */
bne 1f
addi r3,r1,STACK_FRAME_OVERHEAD;
- bl .doorbell_exception
- b .ret_from_except
+ bl doorbell_exception
+ b ret_from_except
#endif /* CONFIG_PPC_DOORBELL */
-1: b .ret_from_except /* What else to do here ? */
+1: b ret_from_except /* What else to do here ? */
unrecov_restore:
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .unrecoverable_exception
+ bl unrecoverable_exception
b unrecov_restore
#ifdef CONFIG_PPC_RTAS
@@ -1021,7 +1016,7 @@
std r6,PACASAVEDMSR(r13)
/* Setup our real return addr */
- LOAD_REG_ADDR(r4,.rtas_return_loc)
+ LOAD_REG_ADDR(r4,rtas_return_loc)
clrldi r4,r4,2 /* convert to realmode address */
mtlr r4
@@ -1045,7 +1040,7 @@
rfid
b . /* prevent speculative execution */
-_STATIC(rtas_return_loc)
+rtas_return_loc:
FIXUP_ENDIAN
/* relocation is off at this point */
@@ -1054,7 +1049,7 @@
bcl 20,31,$+4
0: mflr r3
- ld r3,(1f-0b)(r3) /* get &.rtas_restore_regs */
+ ld r3,(1f-0b)(r3) /* get &rtas_restore_regs */
mfmsr r6
li r0,MSR_RI
@@ -1071,9 +1066,9 @@
b . /* prevent speculative execution */
.align 3
-1: .llong .rtas_restore_regs
+1: .llong rtas_restore_regs
-_STATIC(rtas_restore_regs)
+rtas_restore_regs:
/* relocation is on at this point */
REST_GPR(2, r1) /* Restore the TOC */
REST_GPR(13, r1) /* Restore paca */
@@ -1173,7 +1168,7 @@
_GLOBAL(_mcount)
blr
-_GLOBAL(ftrace_caller)
+_GLOBAL_TOC(ftrace_caller)
/* Taken from output of objdump from lib64/glibc */
mflr r3
ld r11, 0(r1)
@@ -1197,10 +1192,7 @@
_GLOBAL(ftrace_stub)
blr
#else
-_GLOBAL(mcount)
- blr
-
-_GLOBAL(_mcount)
+_GLOBAL_TOC(_mcount)
/* Taken from output of objdump from lib64/glibc */
mflr r3
ld r11, 0(r1)
@@ -1238,7 +1230,7 @@
ld r11, 112(r1)
addi r3, r11, 16
- bl .prepare_ftrace_return
+ bl prepare_ftrace_return
nop
ld r0, 128(r1)
@@ -1254,7 +1246,7 @@
mr r31, r1
stdu r1, -112(r1)
- bl .ftrace_return_to_handler
+ bl ftrace_return_to_handler
nop
/* return value has real return address */
@@ -1284,7 +1276,7 @@
*/
ld r2, PACATOC(r13)
- bl .ftrace_return_to_handler
+ bl ftrace_return_to_handler
nop
/* return value has real return address */
diff --git a/arch/powerpc/kernel/epapr_paravirt.c b/arch/powerpc/kernel/epapr_paravirt.c
index 60d1a22..59e4ba74 100644
--- a/arch/powerpc/kernel/epapr_paravirt.c
+++ b/arch/powerpc/kernel/epapr_paravirt.c
@@ -30,6 +30,7 @@
#endif
bool epapr_paravirt_enabled;
+static bool __maybe_unused epapr_has_idle;
static int __init early_init_dt_scan_epapr(unsigned long node,
const char *uname,
@@ -56,7 +57,7 @@
#if !defined(CONFIG_64BIT) || defined(CONFIG_PPC_BOOK3E_64)
if (of_get_flat_dt_prop(node, "has-idle", NULL))
- ppc_md.power_save = epapr_ev_idle;
+ epapr_has_idle = true;
#endif
epapr_paravirt_enabled = true;
@@ -71,3 +72,14 @@
return 0;
}
+static int __init epapr_idle_init(void)
+{
+#if !defined(CONFIG_64BIT) || defined(CONFIG_PPC_BOOK3E_64)
+ if (epapr_has_idle)
+ ppc_md.power_save = epapr_ev_idle;
+#endif
+
+ return 0;
+}
+
+postcore_initcall(epapr_idle_init);
diff --git a/arch/powerpc/kernel/exceptions-64e.S b/arch/powerpc/kernel/exceptions-64e.S
index c1bee3c..771b4e9 100644
--- a/arch/powerpc/kernel/exceptions-64e.S
+++ b/arch/powerpc/kernel/exceptions-64e.S
@@ -499,7 +499,7 @@
CHECK_NAPPING(); \
addi r3,r1,STACK_FRAME_OVERHEAD; \
bl hdlr; \
- b .ret_from_except_lite;
+ b ret_from_except_lite;
/* This value is used to mark exception frames on the stack. */
.section ".toc","aw"
@@ -550,11 +550,11 @@
CRIT_EXCEPTION_PROLOG(0x100, BOOKE_INTERRUPT_CRITICAL,
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON_CRIT(0x100)
- bl .save_nvgprs
+ bl save_nvgprs
bl special_reg_save
CHECK_NAPPING();
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .unknown_exception
+ bl unknown_exception
b ret_from_crit_except
/* Machine Check Interrupt */
@@ -562,11 +562,11 @@
MC_EXCEPTION_PROLOG(0x000, BOOKE_INTERRUPT_MACHINE_CHECK,
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON_MC(0x000)
- bl .save_nvgprs
+ bl save_nvgprs
bl special_reg_save
CHECK_NAPPING();
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .machine_check_exception
+ bl machine_check_exception
b ret_from_mc_except
/* Data Storage Interrupt */
@@ -591,7 +591,7 @@
/* External Input Interrupt */
MASKABLE_EXCEPTION(0x500, BOOKE_INTERRUPT_EXTERNAL,
- external_input, .do_IRQ, ACK_NONE)
+ external_input, do_IRQ, ACK_NONE)
/* Alignment */
START_EXCEPTION(alignment);
@@ -612,9 +612,9 @@
std r14,_DSISR(r1)
addi r3,r1,STACK_FRAME_OVERHEAD
ld r14,PACA_EXGEN+EX_R14(r13)
- bl .save_nvgprs
- bl .program_check_exception
- b .ret_from_except
+ bl save_nvgprs
+ bl program_check_exception
+ b ret_from_except
/* Floating Point Unavailable Interrupt */
START_EXCEPTION(fp_unavailable);
@@ -625,13 +625,13 @@
ld r12,_MSR(r1)
andi. r0,r12,MSR_PR;
beq- 1f
- bl .load_up_fpu
+ bl load_up_fpu
b fast_exception_return
1: INTS_DISABLE
- bl .save_nvgprs
+ bl save_nvgprs
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .kernel_fp_unavailable_exception
- b .ret_from_except
+ bl kernel_fp_unavailable_exception
+ b ret_from_except
/* Altivec Unavailable Interrupt */
START_EXCEPTION(altivec_unavailable);
@@ -644,16 +644,16 @@
ld r12,_MSR(r1)
andi. r0,r12,MSR_PR;
beq- 1f
- bl .load_up_altivec
+ bl load_up_altivec
b fast_exception_return
1:
END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
#endif
INTS_DISABLE
- bl .save_nvgprs
+ bl save_nvgprs
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .altivec_unavailable_exception
- b .ret_from_except
+ bl altivec_unavailable_exception
+ b ret_from_except
/* AltiVec Assist */
START_EXCEPTION(altivec_assist);
@@ -662,39 +662,39 @@
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x220)
INTS_DISABLE
- bl .save_nvgprs
+ bl save_nvgprs
addi r3,r1,STACK_FRAME_OVERHEAD
#ifdef CONFIG_ALTIVEC
BEGIN_FTR_SECTION
- bl .altivec_assist_exception
+ bl altivec_assist_exception
END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
#else
- bl .unknown_exception
+ bl unknown_exception
#endif
- b .ret_from_except
+ b ret_from_except
/* Decrementer Interrupt */
MASKABLE_EXCEPTION(0x900, BOOKE_INTERRUPT_DECREMENTER,
- decrementer, .timer_interrupt, ACK_DEC)
+ decrementer, timer_interrupt, ACK_DEC)
/* Fixed Interval Timer Interrupt */
MASKABLE_EXCEPTION(0x980, BOOKE_INTERRUPT_FIT,
- fixed_interval, .unknown_exception, ACK_FIT)
+ fixed_interval, unknown_exception, ACK_FIT)
/* Watchdog Timer Interrupt */
START_EXCEPTION(watchdog);
CRIT_EXCEPTION_PROLOG(0x9f0, BOOKE_INTERRUPT_WATCHDOG,
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON_CRIT(0x9f0)
- bl .save_nvgprs
+ bl save_nvgprs
bl special_reg_save
CHECK_NAPPING();
addi r3,r1,STACK_FRAME_OVERHEAD
#ifdef CONFIG_BOOKE_WDT
- bl .WatchdogException
+ bl WatchdogException
#else
- bl .unknown_exception
+ bl unknown_exception
#endif
b ret_from_crit_except
@@ -712,10 +712,10 @@
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0xf20)
INTS_DISABLE
- bl .save_nvgprs
+ bl save_nvgprs
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .unknown_exception
- b .ret_from_except
+ bl unknown_exception
+ b ret_from_except
/* Debug exception as a critical interrupt*/
START_EXCEPTION(debug_crit);
@@ -774,9 +774,9 @@
mr r4,r14
ld r14,PACA_EXCRIT+EX_R14(r13)
ld r15,PACA_EXCRIT+EX_R15(r13)
- bl .save_nvgprs
- bl .DebugException
- b .ret_from_except
+ bl save_nvgprs
+ bl DebugException
+ b ret_from_except
kernel_dbg_exc:
b . /* NYI */
@@ -839,9 +839,9 @@
mr r4,r14
ld r14,PACA_EXDBG+EX_R14(r13)
ld r15,PACA_EXDBG+EX_R15(r13)
- bl .save_nvgprs
- bl .DebugException
- b .ret_from_except
+ bl save_nvgprs
+ bl DebugException
+ b ret_from_except
START_EXCEPTION(perfmon);
NORMAL_EXCEPTION_PROLOG(0x260, BOOKE_INTERRUPT_PERFORMANCE_MONITOR,
@@ -850,23 +850,23 @@
INTS_DISABLE
CHECK_NAPPING()
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .performance_monitor_exception
- b .ret_from_except_lite
+ bl performance_monitor_exception
+ b ret_from_except_lite
/* Doorbell interrupt */
MASKABLE_EXCEPTION(0x280, BOOKE_INTERRUPT_DOORBELL,
- doorbell, .doorbell_exception, ACK_NONE)
+ doorbell, doorbell_exception, ACK_NONE)
/* Doorbell critical Interrupt */
START_EXCEPTION(doorbell_crit);
CRIT_EXCEPTION_PROLOG(0x2a0, BOOKE_INTERRUPT_DOORBELL_CRITICAL,
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON_CRIT(0x2a0)
- bl .save_nvgprs
+ bl save_nvgprs
bl special_reg_save
CHECK_NAPPING();
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .unknown_exception
+ bl unknown_exception
b ret_from_crit_except
/*
@@ -878,21 +878,21 @@
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x2c0)
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .save_nvgprs
+ bl save_nvgprs
INTS_RESTORE_HARD
- bl .unknown_exception
- b .ret_from_except
+ bl unknown_exception
+ b ret_from_except
/* Guest Doorbell critical Interrupt */
START_EXCEPTION(guest_doorbell_crit);
CRIT_EXCEPTION_PROLOG(0x2e0, BOOKE_INTERRUPT_GUEST_DBELL_CRIT,
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON_CRIT(0x2e0)
- bl .save_nvgprs
+ bl save_nvgprs
bl special_reg_save
CHECK_NAPPING();
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .unknown_exception
+ bl unknown_exception
b ret_from_crit_except
/* Hypervisor call */
@@ -901,10 +901,10 @@
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x310)
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .save_nvgprs
+ bl save_nvgprs
INTS_RESTORE_HARD
- bl .unknown_exception
- b .ret_from_except
+ bl unknown_exception
+ b ret_from_except
/* Embedded Hypervisor priviledged */
START_EXCEPTION(ehpriv);
@@ -912,10 +912,10 @@
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x320)
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .save_nvgprs
+ bl save_nvgprs
INTS_RESTORE_HARD
- bl .unknown_exception
- b .ret_from_except
+ bl unknown_exception
+ b ret_from_except
/* LRAT Error interrupt */
START_EXCEPTION(lrat_error);
@@ -1014,16 +1014,16 @@
mr r5,r15
ld r14,PACA_EXGEN+EX_R14(r13)
ld r15,PACA_EXGEN+EX_R15(r13)
- bl .do_page_fault
+ bl do_page_fault
cmpdi r3,0
bne- 1f
- b .ret_from_except_lite
-1: bl .save_nvgprs
+ b ret_from_except_lite
+1: bl save_nvgprs
mr r5,r3
addi r3,r1,STACK_FRAME_OVERHEAD
ld r4,_DAR(r1)
- bl .bad_page_fault
- b .ret_from_except
+ bl bad_page_fault
+ b ret_from_except
/*
* Alignment exception doesn't fit entirely in the 0x100 bytes so it
@@ -1035,10 +1035,10 @@
addi r3,r1,STACK_FRAME_OVERHEAD
ld r14,PACA_EXGEN+EX_R14(r13)
ld r15,PACA_EXGEN+EX_R15(r13)
- bl .save_nvgprs
+ bl save_nvgprs
INTS_RESTORE_HARD
- bl .alignment_exception
- b .ret_from_except
+ bl alignment_exception
+ b ret_from_except
/*
* We branch here from entry_64.S for the last stage of the exception
@@ -1172,7 +1172,7 @@
std r12,0(r11)
ld r2,PACATOC(r13)
1: addi r3,r1,STACK_FRAME_OVERHEAD
- bl .kernel_bad_stack
+ bl kernel_bad_stack
b 1b
/*
@@ -1521,13 +1521,13 @@
* and always use AS 0, so we just set it up to match our link
* address and never use 0 based addresses.
*/
- bl .initial_tlb_book3e
+ bl initial_tlb_book3e
/* Init global core bits */
- bl .init_core_book3e
+ bl init_core_book3e
/* Init per-thread bits */
- bl .init_thread_book3e
+ bl init_thread_book3e
/* Return to common init code */
tovirt(r28,r28)
@@ -1548,7 +1548,7 @@
*/
_GLOBAL(book3e_secondary_core_init_tlb_set)
li r4,1
- b .generic_secondary_smp_init
+ b generic_secondary_smp_init
_GLOBAL(book3e_secondary_core_init)
mflr r28
@@ -1558,18 +1558,18 @@
bne 2f
/* Setup TLB for this core */
- bl .initial_tlb_book3e
+ bl initial_tlb_book3e
/* We can return from the above running at a different
* address, so recalculate r2 (TOC)
*/
- bl .relative_toc
+ bl relative_toc
/* Init global core bits */
-2: bl .init_core_book3e
+2: bl init_core_book3e
/* Init per-thread bits */
-3: bl .init_thread_book3e
+3: bl init_thread_book3e
/* Return to common init code at proper virtual address.
*
@@ -1596,14 +1596,14 @@
mflr r28
b 3b
-_STATIC(init_core_book3e)
+init_core_book3e:
/* Establish the interrupt vector base */
LOAD_REG_IMMEDIATE(r3, interrupt_base_book3e)
mtspr SPRN_IVPR,r3
sync
blr
-_STATIC(init_thread_book3e)
+init_thread_book3e:
lis r3,(SPRN_EPCR_ICM | SPRN_EPCR_GICM)@h
mtspr SPRN_EPCR,r3
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 3afd391..20f11eb 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -132,12 +132,12 @@
#endif
beq cr1,2f
- b .power7_wakeup_noloss
-2: b .power7_wakeup_loss
+ b power7_wakeup_noloss
+2: b power7_wakeup_loss
/* Fast Sleep wakeup on PowerNV */
8: GET_PACA(r13)
- b .power7_wakeup_tb_loss
+ b power7_wakeup_tb_loss
9:
END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
@@ -211,16 +211,16 @@
#endif /* __DISABLED__ */
mfspr r12,SPRN_SRR1
#ifndef CONFIG_RELOCATABLE
- b .slb_miss_realmode
+ b slb_miss_realmode
#else
/*
- * We can't just use a direct branch to .slb_miss_realmode
+ * We can't just use a direct branch to slb_miss_realmode
* because the distance from here to there depends on where
* the kernel ends up being put.
*/
mfctr r11
ld r10,PACAKBASE(r13)
- LOAD_HANDLER(r10, .slb_miss_realmode)
+ LOAD_HANDLER(r10, slb_miss_realmode)
mtctr r10
bctr
#endif
@@ -243,11 +243,11 @@
#endif /* __DISABLED__ */
mfspr r12,SPRN_SRR1
#ifndef CONFIG_RELOCATABLE
- b .slb_miss_realmode
+ b slb_miss_realmode
#else
mfctr r11
ld r10,PACAKBASE(r13)
- LOAD_HANDLER(r10, .slb_miss_realmode)
+ LOAD_HANDLER(r10, slb_miss_realmode)
mtctr r10
bctr
#endif
@@ -524,7 +524,7 @@
std r12,PACA_EXSLB+EX_R12(r13)
GET_SCRATCH0(r10)
std r10,PACA_EXSLB+EX_R13(r13)
- EXCEPTION_PROLOG_PSERIES_1(.do_stab_bolted, EXC_STD)
+ EXCEPTION_PROLOG_PSERIES_1(do_stab_bolted, EXC_STD)
KVM_HANDLER_SKIP(PACA_EXGEN, EXC_STD, 0x300)
KVM_HANDLER_SKIP(PACA_EXSLB, EXC_STD, 0x380)
@@ -769,38 +769,38 @@
/*** Common interrupt handlers ***/
- STD_EXCEPTION_COMMON(0x100, system_reset, .system_reset_exception)
+ STD_EXCEPTION_COMMON(0x100, system_reset, system_reset_exception)
STD_EXCEPTION_COMMON_ASYNC(0x500, hardware_interrupt, do_IRQ)
- STD_EXCEPTION_COMMON_ASYNC(0x900, decrementer, .timer_interrupt)
- STD_EXCEPTION_COMMON(0x980, hdecrementer, .hdec_interrupt)
+ STD_EXCEPTION_COMMON_ASYNC(0x900, decrementer, timer_interrupt)
+ STD_EXCEPTION_COMMON(0x980, hdecrementer, hdec_interrupt)
#ifdef CONFIG_PPC_DOORBELL
- STD_EXCEPTION_COMMON_ASYNC(0xa00, doorbell_super, .doorbell_exception)
+ STD_EXCEPTION_COMMON_ASYNC(0xa00, doorbell_super, doorbell_exception)
#else
- STD_EXCEPTION_COMMON_ASYNC(0xa00, doorbell_super, .unknown_exception)
+ STD_EXCEPTION_COMMON_ASYNC(0xa00, doorbell_super, unknown_exception)
#endif
- STD_EXCEPTION_COMMON(0xb00, trap_0b, .unknown_exception)
- STD_EXCEPTION_COMMON(0xd00, single_step, .single_step_exception)
- STD_EXCEPTION_COMMON(0xe00, trap_0e, .unknown_exception)
- STD_EXCEPTION_COMMON(0xe40, emulation_assist, .emulation_assist_interrupt)
- STD_EXCEPTION_COMMON(0xe60, hmi_exception, .unknown_exception)
+ STD_EXCEPTION_COMMON(0xb00, trap_0b, unknown_exception)
+ STD_EXCEPTION_COMMON(0xd00, single_step, single_step_exception)
+ STD_EXCEPTION_COMMON(0xe00, trap_0e, unknown_exception)
+ STD_EXCEPTION_COMMON(0xe40, emulation_assist, emulation_assist_interrupt)
+ STD_EXCEPTION_COMMON(0xe60, hmi_exception, unknown_exception)
#ifdef CONFIG_PPC_DOORBELL
- STD_EXCEPTION_COMMON_ASYNC(0xe80, h_doorbell, .doorbell_exception)
+ STD_EXCEPTION_COMMON_ASYNC(0xe80, h_doorbell, doorbell_exception)
#else
- STD_EXCEPTION_COMMON_ASYNC(0xe80, h_doorbell, .unknown_exception)
+ STD_EXCEPTION_COMMON_ASYNC(0xe80, h_doorbell, unknown_exception)
#endif
- STD_EXCEPTION_COMMON_ASYNC(0xf00, performance_monitor, .performance_monitor_exception)
- STD_EXCEPTION_COMMON(0x1300, instruction_breakpoint, .instruction_breakpoint_exception)
- STD_EXCEPTION_COMMON(0x1502, denorm, .unknown_exception)
+ STD_EXCEPTION_COMMON_ASYNC(0xf00, performance_monitor, performance_monitor_exception)
+ STD_EXCEPTION_COMMON(0x1300, instruction_breakpoint, instruction_breakpoint_exception)
+ STD_EXCEPTION_COMMON(0x1502, denorm, unknown_exception)
#ifdef CONFIG_ALTIVEC
- STD_EXCEPTION_COMMON(0x1700, altivec_assist, .altivec_assist_exception)
+ STD_EXCEPTION_COMMON(0x1700, altivec_assist, altivec_assist_exception)
#else
- STD_EXCEPTION_COMMON(0x1700, altivec_assist, .unknown_exception)
+ STD_EXCEPTION_COMMON(0x1700, altivec_assist, unknown_exception)
#endif
#ifdef CONFIG_CBE_RAS
- STD_EXCEPTION_COMMON(0x1200, cbe_system_error, .cbe_system_error_exception)
- STD_EXCEPTION_COMMON(0x1600, cbe_maintenance, .cbe_maintenance_exception)
- STD_EXCEPTION_COMMON(0x1800, cbe_thermal, .cbe_thermal_exception)
+ STD_EXCEPTION_COMMON(0x1200, cbe_system_error, cbe_system_error_exception)
+ STD_EXCEPTION_COMMON(0x1600, cbe_maintenance, cbe_maintenance_exception)
+ STD_EXCEPTION_COMMON(0x1800, cbe_thermal, cbe_thermal_exception)
#endif /* CONFIG_CBE_RAS */
/*
@@ -829,16 +829,16 @@
mfspr r3,SPRN_DAR
mfspr r12,SPRN_SRR1
#ifndef CONFIG_RELOCATABLE
- b .slb_miss_realmode
+ b slb_miss_realmode
#else
/*
- * We can't just use a direct branch to .slb_miss_realmode
+ * We can't just use a direct branch to slb_miss_realmode
* because the distance from here to there depends on where
* the kernel ends up being put.
*/
mfctr r11
ld r10,PACAKBASE(r13)
- LOAD_HANDLER(r10, .slb_miss_realmode)
+ LOAD_HANDLER(r10, slb_miss_realmode)
mtctr r10
bctr
#endif
@@ -854,11 +854,11 @@
mfspr r3,SPRN_SRR0 /* SRR0 is faulting address */
mfspr r12,SPRN_SRR1
#ifndef CONFIG_RELOCATABLE
- b .slb_miss_realmode
+ b slb_miss_realmode
#else
mfctr r11
ld r10,PACAKBASE(r13)
- LOAD_HANDLER(r10, .slb_miss_realmode)
+ LOAD_HANDLER(r10, slb_miss_realmode)
mtctr r10
bctr
#endif
@@ -966,7 +966,7 @@
b system_call_common
ppc64_runlatch_on_trampoline:
- b .__ppc64_runlatch_on
+ b __ppc64_runlatch_on
/*
* Here we have detected that the kernel stack pointer is bad.
@@ -1025,7 +1025,7 @@
std r12,RESULT(r1)
std r11,STACK_FRAME_OVERHEAD-16(r1)
1: addi r3,r1,STACK_FRAME_OVERHEAD
- bl .kernel_bad_stack
+ bl kernel_bad_stack
b 1b
/*
@@ -1046,7 +1046,7 @@
ld r3,PACA_EXGEN+EX_DAR(r13)
lwz r4,PACA_EXGEN+EX_DSISR(r13)
li r5,0x300
- b .do_hash_page /* Try to handle as hpte fault */
+ b do_hash_page /* Try to handle as hpte fault */
.align 7
.globl h_data_storage_common
@@ -1056,11 +1056,11 @@
mfspr r10,SPRN_HDSISR
stw r10,PACA_EXGEN+EX_DSISR(r13)
EXCEPTION_PROLOG_COMMON(0xe00, PACA_EXGEN)
- bl .save_nvgprs
+ bl save_nvgprs
DISABLE_INTS
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .unknown_exception
- b .ret_from_except
+ bl unknown_exception
+ b ret_from_except
.align 7
.globl instruction_access_common
@@ -1071,9 +1071,9 @@
ld r3,_NIP(r1)
andis. r4,r12,0x5820
li r5,0x400
- b .do_hash_page /* Try to handle as hpte fault */
+ b do_hash_page /* Try to handle as hpte fault */
- STD_EXCEPTION_COMMON(0xe20, h_instr_storage, .unknown_exception)
+ STD_EXCEPTION_COMMON(0xe20, h_instr_storage, unknown_exception)
/*
* Here is the common SLB miss user that is used when going to virtual
@@ -1088,7 +1088,7 @@
stw r9,PACA_EXGEN+EX_CCR(r13)
std r10,PACA_EXGEN+EX_LR(r13)
std r11,PACA_EXGEN+EX_SRR0(r13)
- bl .slb_allocate_user
+ bl slb_allocate_user
ld r10,PACA_EXGEN+EX_LR(r13)
ld r3,PACA_EXGEN+EX_R3(r13)
@@ -1131,9 +1131,9 @@
unrecov_user_slb:
EXCEPTION_PROLOG_COMMON(0x4200, PACA_EXGEN)
DISABLE_INTS
- bl .save_nvgprs
+ bl save_nvgprs
1: addi r3,r1,STACK_FRAME_OVERHEAD
- bl .unrecoverable_exception
+ bl unrecoverable_exception
b 1b
#endif /* __DISABLED__ */
@@ -1158,10 +1158,10 @@
lwz r4,PACA_EXGEN+EX_DSISR(r13)
std r3,_DAR(r1)
std r4,_DSISR(r1)
- bl .save_nvgprs
+ bl save_nvgprs
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .machine_check_exception
- b .ret_from_except
+ bl machine_check_exception
+ b ret_from_except
.align 7
.globl alignment_common
@@ -1175,31 +1175,31 @@
lwz r4,PACA_EXGEN+EX_DSISR(r13)
std r3,_DAR(r1)
std r4,_DSISR(r1)
- bl .save_nvgprs
+ bl save_nvgprs
DISABLE_INTS
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .alignment_exception
- b .ret_from_except
+ bl alignment_exception
+ b ret_from_except
.align 7
.globl program_check_common
program_check_common:
EXCEPTION_PROLOG_COMMON(0x700, PACA_EXGEN)
- bl .save_nvgprs
+ bl save_nvgprs
DISABLE_INTS
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .program_check_exception
- b .ret_from_except
+ bl program_check_exception
+ b ret_from_except
.align 7
.globl fp_unavailable_common
fp_unavailable_common:
EXCEPTION_PROLOG_COMMON(0x800, PACA_EXGEN)
bne 1f /* if from user, just load it up */
- bl .save_nvgprs
+ bl save_nvgprs
DISABLE_INTS
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .kernel_fp_unavailable_exception
+ bl kernel_fp_unavailable_exception
BUG_OPCODE
1:
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
@@ -1211,15 +1211,15 @@
bne- 2f
END_FTR_SECTION_IFSET(CPU_FTR_TM)
#endif
- bl .load_up_fpu
+ bl load_up_fpu
b fast_exception_return
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
2: /* User process was in a transaction */
- bl .save_nvgprs
+ bl save_nvgprs
DISABLE_INTS
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .fp_unavailable_tm
- b .ret_from_except
+ bl fp_unavailable_tm
+ b ret_from_except
#endif
.align 7
.globl altivec_unavailable_common
@@ -1237,24 +1237,24 @@
bne- 2f
END_FTR_SECTION_NESTED(CPU_FTR_TM, CPU_FTR_TM, 69)
#endif
- bl .load_up_altivec
+ bl load_up_altivec
b fast_exception_return
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
2: /* User process was in a transaction */
- bl .save_nvgprs
+ bl save_nvgprs
DISABLE_INTS
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .altivec_unavailable_tm
- b .ret_from_except
+ bl altivec_unavailable_tm
+ b ret_from_except
#endif
1:
END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
#endif
- bl .save_nvgprs
+ bl save_nvgprs
DISABLE_INTS
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .altivec_unavailable_exception
- b .ret_from_except
+ bl altivec_unavailable_exception
+ b ret_from_except
.align 7
.globl vsx_unavailable_common
@@ -1272,26 +1272,26 @@
bne- 2f
END_FTR_SECTION_NESTED(CPU_FTR_TM, CPU_FTR_TM, 69)
#endif
- b .load_up_vsx
+ b load_up_vsx
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
2: /* User process was in a transaction */
- bl .save_nvgprs
+ bl save_nvgprs
DISABLE_INTS
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .vsx_unavailable_tm
- b .ret_from_except
+ bl vsx_unavailable_tm
+ b ret_from_except
#endif
1:
END_FTR_SECTION_IFSET(CPU_FTR_VSX)
#endif
- bl .save_nvgprs
+ bl save_nvgprs
DISABLE_INTS
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .vsx_unavailable_exception
- b .ret_from_except
+ bl vsx_unavailable_exception
+ b ret_from_except
- STD_EXCEPTION_COMMON(0xf60, facility_unavailable, .facility_unavailable_exception)
- STD_EXCEPTION_COMMON(0xf80, hv_facility_unavailable, .facility_unavailable_exception)
+ STD_EXCEPTION_COMMON(0xf60, facility_unavailable, facility_unavailable_exception)
+ STD_EXCEPTION_COMMON(0xf80, hv_facility_unavailable, facility_unavailable_exception)
.align 7
.globl __end_handlers
@@ -1386,9 +1386,9 @@
machine_check_handle_early:
std r0,GPR0(r1) /* Save r0 */
EXCEPTION_PROLOG_COMMON_3(0x200)
- bl .save_nvgprs
+ bl save_nvgprs
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .machine_check_early
+ bl machine_check_early
ld r12,_MSR(r1)
#ifdef CONFIG_PPC_P7_NAP
/*
@@ -1408,11 +1408,11 @@
/* Supervisor state loss */
li r0,1
stb r0,PACA_NAPSTATELOST(r13)
-3: bl .machine_check_queue_event
+3: bl machine_check_queue_event
MACHINE_CHECK_HANDLER_WINDUP
GET_PACA(r13)
ld r1,PACAR1(r13)
- b .power7_enter_nap_mode
+ b power7_enter_nap_mode
4:
#endif
/*
@@ -1444,7 +1444,7 @@
andi. r11,r12,MSR_RI
bne 2f
1: addi r3,r1,STACK_FRAME_OVERHEAD
- bl .unrecoverable_exception
+ bl unrecoverable_exception
b 1b
2:
/*
@@ -1452,7 +1452,7 @@
* Queue up the MCE event so that we can log it later, while
* returning from kernel or opal call.
*/
- bl .machine_check_queue_event
+ bl machine_check_queue_event
MACHINE_CHECK_HANDLER_WINDUP
rfid
9:
@@ -1468,7 +1468,7 @@
* r3 is saved in paca->slb_r3
* We assume we aren't going to take any exceptions during this procedure.
*/
-_GLOBAL(slb_miss_realmode)
+slb_miss_realmode:
mflr r10
#ifdef CONFIG_RELOCATABLE
mtctr r11
@@ -1477,7 +1477,7 @@
stw r9,PACA_EXSLB+EX_CCR(r13) /* save CR in exc. frame */
std r10,PACA_EXSLB+EX_LR(r13) /* save LR */
- bl .slb_allocate_realmode
+ bl slb_allocate_realmode
/* All done -- return from exception. */
@@ -1517,9 +1517,9 @@
unrecov_slb:
EXCEPTION_PROLOG_COMMON(0x4100, PACA_EXSLB)
DISABLE_INTS
- bl .save_nvgprs
+ bl save_nvgprs
1: addi r3,r1,STACK_FRAME_OVERHEAD
- bl .unrecoverable_exception
+ bl unrecoverable_exception
b 1b
@@ -1536,7 +1536,7 @@
* Hash table stuff
*/
.align 7
-_STATIC(do_hash_page)
+do_hash_page:
std r3,_DAR(r1)
std r4,_DSISR(r1)
@@ -1573,7 +1573,7 @@
*
* at return r3 = 0 for success, 1 for page fault, negative for error
*/
- bl .hash_page /* build HPTE if possible */
+ bl hash_page /* build HPTE if possible */
cmpdi r3,0 /* see if hash_page succeeded */
/* Success */
@@ -1587,35 +1587,35 @@
11: ld r4,_DAR(r1)
ld r5,_DSISR(r1)
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .do_page_fault
+ bl do_page_fault
cmpdi r3,0
beq+ 12f
- bl .save_nvgprs
+ bl save_nvgprs
mr r5,r3
addi r3,r1,STACK_FRAME_OVERHEAD
lwz r4,_DAR(r1)
- bl .bad_page_fault
- b .ret_from_except
+ bl bad_page_fault
+ b ret_from_except
/* We have a data breakpoint exception - handle it */
handle_dabr_fault:
- bl .save_nvgprs
+ bl save_nvgprs
ld r4,_DAR(r1)
ld r5,_DSISR(r1)
addi r3,r1,STACK_FRAME_OVERHEAD
- bl .do_break
-12: b .ret_from_except_lite
+ bl do_break
+12: b ret_from_except_lite
/* We have a page fault that hash_page could handle but HV refused
* the PTE insertion
*/
-13: bl .save_nvgprs
+13: bl save_nvgprs
mr r5,r3
addi r3,r1,STACK_FRAME_OVERHEAD
ld r4,_DAR(r1)
- bl .low_hash_fault
- b .ret_from_except
+ bl low_hash_fault
+ b ret_from_except
/*
* We come here as a result of a DSI at a point where we don't want
@@ -1624,16 +1624,16 @@
* were soft-disabled. We want to invoke the exception handler for
* the access, or panic if there isn't a handler.
*/
-77: bl .save_nvgprs
+77: bl save_nvgprs
mr r4,r3
addi r3,r1,STACK_FRAME_OVERHEAD
li r5,SIGSEGV
- bl .bad_page_fault
- b .ret_from_except
+ bl bad_page_fault
+ b ret_from_except
/* here we have a segment miss */
do_ste_alloc:
- bl .ste_allocate /* try to insert stab entry */
+ bl ste_allocate /* try to insert stab entry */
cmpdi r3,0
bne- handle_page_fault
b fast_exception_return
@@ -1646,7 +1646,7 @@
* We assume (DAR >> 60) == 0xc.
*/
.align 7
-_GLOBAL(do_stab_bolted)
+do_stab_bolted:
stw r9,PACA_EXSLB+EX_CCR(r13) /* save CR in exc. frame */
std r11,PACA_EXSLB+EX_SRR0(r13) /* save SRR0 in exc. frame */
mfspr r11,SPRN_DAR /* ea */
diff --git a/arch/powerpc/kernel/fadump.c b/arch/powerpc/kernel/fadump.c
index 7213d930..742694c1 100644
--- a/arch/powerpc/kernel/fadump.c
+++ b/arch/powerpc/kernel/fadump.c
@@ -69,7 +69,7 @@
*/
token = of_get_flat_dt_prop(node, "ibm,configure-kernel-dump", NULL);
if (!token)
- return 0;
+ return 1;
fw_dump.fadump_supported = 1;
fw_dump.ibm_configure_kernel_dump = *token;
@@ -92,7 +92,7 @@
&size);
if (!sections)
- return 0;
+ return 1;
num_sections = size / (3 * sizeof(u32));
@@ -110,6 +110,7 @@
break;
}
}
+
return 1;
}
@@ -645,7 +646,7 @@
}
/* Lower 4 bytes of reg_value contains logical cpu id */
cpu = reg_entry->reg_value & FADUMP_CPU_ID_MASK;
- if (!cpumask_test_cpu(cpu, &fdh->cpu_online_mask)) {
+ if (fdh && !cpumask_test_cpu(cpu, &fdh->cpu_online_mask)) {
SKIP_TO_NEXT_CPU(reg_entry);
continue;
}
@@ -662,9 +663,11 @@
}
fadump_final_note(note_buf);
- pr_debug("Updating elfcore header (%llx) with cpu notes\n",
+ if (fdh) {
+ pr_debug("Updating elfcore header (%llx) with cpu notes\n",
fdh->elfcorehdr_addr);
- fadump_update_elfcore_header((char *)__va(fdh->elfcorehdr_addr));
+ fadump_update_elfcore_header((char *)__va(fdh->elfcorehdr_addr));
+ }
return 0;
error_out:
diff --git a/arch/powerpc/kernel/ftrace.c b/arch/powerpc/kernel/ftrace.c
index 6a014c7..f202d07 100644
--- a/arch/powerpc/kernel/ftrace.c
+++ b/arch/powerpc/kernel/ftrace.c
@@ -105,11 +105,9 @@
struct dyn_ftrace *rec, unsigned long addr)
{
unsigned int op;
- unsigned int jmp[5];
unsigned long ptr;
unsigned long ip = rec->ip;
- unsigned long tramp;
- int offset;
+ void *tramp;
/* read where this goes */
if (probe_kernel_read(&op, (void *)ip, sizeof(int)))
@@ -122,96 +120,41 @@
}
/* lets find where the pointer goes */
- tramp = find_bl_target(ip, op);
+ tramp = (void *)find_bl_target(ip, op);
- /*
- * On PPC64 the trampoline looks like:
- * 0x3d, 0x82, 0x00, 0x00, addis r12,r2, <high>
- * 0x39, 0x8c, 0x00, 0x00, addi r12,r12, <low>
- * Where the bytes 2,3,6 and 7 make up the 32bit offset
- * to the TOC that holds the pointer.
- * to jump to.
- * 0xf8, 0x41, 0x00, 0x28, std r2,40(r1)
- * 0xe9, 0x6c, 0x00, 0x20, ld r11,32(r12)
- * The actually address is 32 bytes from the offset
- * into the TOC.
- * 0xe8, 0x4c, 0x00, 0x28, ld r2,40(r12)
- */
+ pr_devel("ip:%lx jumps to %p", ip, tramp);
- pr_devel("ip:%lx jumps to %lx r2: %lx", ip, tramp, mod->arch.toc);
-
- /* Find where the trampoline jumps to */
- if (probe_kernel_read(jmp, (void *)tramp, sizeof(jmp))) {
- printk(KERN_ERR "Failed to read %lx\n", tramp);
- return -EFAULT;
- }
-
- pr_devel(" %08x %08x", jmp[0], jmp[1]);
-
- /* verify that this is what we expect it to be */
- if (((jmp[0] & 0xffff0000) != 0x3d820000) ||
- ((jmp[1] & 0xffff0000) != 0x398c0000) ||
- (jmp[2] != 0xf8410028) ||
- (jmp[3] != 0xe96c0020) ||
- (jmp[4] != 0xe84c0028)) {
+ if (!is_module_trampoline(tramp)) {
printk(KERN_ERR "Not a trampoline\n");
return -EINVAL;
}
- /* The bottom half is signed extended */
- offset = ((unsigned)((unsigned short)jmp[0]) << 16) +
- (int)((short)jmp[1]);
-
- pr_devel(" %x ", offset);
-
- /* get the address this jumps too */
- tramp = mod->arch.toc + offset + 32;
- pr_devel("toc: %lx", tramp);
-
- if (probe_kernel_read(jmp, (void *)tramp, 8)) {
- printk(KERN_ERR "Failed to read %lx\n", tramp);
+ if (module_trampoline_target(mod, tramp, &ptr)) {
+ printk(KERN_ERR "Failed to get trampoline target\n");
return -EFAULT;
}
- pr_devel(" %08x %08x\n", jmp[0], jmp[1]);
-
-#ifdef __LITTLE_ENDIAN__
- ptr = ((unsigned long)jmp[1] << 32) + jmp[0];
-#else
- ptr = ((unsigned long)jmp[0] << 32) + jmp[1];
-#endif
+ pr_devel("trampoline target %lx", ptr);
/* This should match what was called */
if (ptr != ppc_function_entry((void *)addr)) {
- printk(KERN_ERR "addr does not match %lx\n", ptr);
+ printk(KERN_ERR "addr %lx does not match expected %lx\n",
+ ptr, ppc_function_entry((void *)addr));
return -EINVAL;
}
/*
- * We want to nop the line, but the next line is
- * 0xe8, 0x41, 0x00, 0x28 ld r2,40(r1)
- * This needs to be turned to a nop too.
- */
- if (probe_kernel_read(&op, (void *)(ip+4), MCOUNT_INSN_SIZE))
- return -EFAULT;
-
- if (op != 0xe8410028) {
- printk(KERN_ERR "Next line is not ld! (%08x)\n", op);
- return -EINVAL;
- }
-
- /*
- * Milton Miller pointed out that we can not blindly do nops.
- * If a task was preempted when calling a trace function,
- * the nops will remove the way to restore the TOC in r2
- * and the r2 TOC will get corrupted.
- */
-
- /*
- * Replace:
- * bl <tramp> <==== will be replaced with "b 1f"
- * ld r2,40(r1)
- * 1:
+ * Our original call site looks like:
+ *
+ * bl <tramp>
+ * ld r2,XX(r1)
+ *
+ * Milton Miller pointed out that we can not simply nop the branch.
+ * If a task was preempted when calling a trace function, the nops
+ * will remove the way to restore the TOC in r2 and the r2 TOC will
+ * get corrupted.
+ *
+ * Use a b +8 to jump over the load.
*/
op = 0x48000008; /* b +8 */
@@ -349,19 +292,24 @@
__ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
{
unsigned int op[2];
- unsigned long ip = rec->ip;
+ void *ip = (void *)rec->ip;
/* read where this goes */
- if (probe_kernel_read(op, (void *)ip, MCOUNT_INSN_SIZE * 2))
+ if (probe_kernel_read(op, ip, sizeof(op)))
return -EFAULT;
/*
- * It should be pointing to two nops or
- * b +8; ld r2,40(r1)
+ * We expect to see:
+ *
+ * b +8
+ * ld r2,XX(r1)
+ *
+ * The load offset is different depending on the ABI. For simplicity
+ * just mask it out when doing the compare.
*/
- if (((op[0] != 0x48000008) || (op[1] != 0xe8410028)) &&
- ((op[0] != PPC_INST_NOP) || (op[1] != PPC_INST_NOP))) {
- printk(KERN_ERR "Expected NOPs but have %x %x\n", op[0], op[1]);
+ if ((op[0] != 0x48000008) || ((op[1] & 0xffff00000) != 0xe8410000)) {
+ printk(KERN_ERR "Unexpected call sequence: %x %x\n",
+ op[0], op[1]);
return -EINVAL;
}
@@ -371,23 +319,16 @@
return -EINVAL;
}
- /* create the branch to the trampoline */
- op[0] = create_branch((unsigned int *)ip,
- rec->arch.mod->arch.tramp, BRANCH_SET_LINK);
- if (!op[0]) {
- printk(KERN_ERR "REL24 out of range!\n");
+ /* Ensure branch is within 24 bits */
+ if (create_branch(ip, rec->arch.mod->arch.tramp, BRANCH_SET_LINK)) {
+ printk(KERN_ERR "Branch out of range");
return -EINVAL;
}
- /* ld r2,40(r1) */
- op[1] = 0xe8410028;
-
- pr_devel("write to %lx\n", rec->ip);
-
- if (probe_kernel_write((void *)ip, op, MCOUNT_INSN_SIZE * 2))
- return -EPERM;
-
- flush_icache_range(ip, ip + 8);
+ if (patch_branch(ip, rec->arch.mod->arch.tramp, BRANCH_SET_LINK)) {
+ printk(KERN_ERR "REL24 out of range!\n");
+ return -EINVAL;
+ }
return 0;
}
diff --git a/arch/powerpc/kernel/head_64.S b/arch/powerpc/kernel/head_64.S
index b7363bd..a95145d 100644
--- a/arch/powerpc/kernel/head_64.S
+++ b/arch/powerpc/kernel/head_64.S
@@ -70,16 +70,15 @@
/* NOP this out unconditionally */
BEGIN_FTR_SECTION
FIXUP_ENDIAN
- b .__start_initialization_multiplatform
+ b __start_initialization_multiplatform
END_FTR_SECTION(0, 1)
/* Catch branch to 0 in real mode */
trap
- /* Secondary processors spin on this value until it becomes nonzero.
- * When it does it contains the real address of the descriptor
- * of the function that the cpu should jump to to continue
- * initialization.
+ /* Secondary processors spin on this value until it becomes non-zero.
+ * When non-zero, it contains the real address of the function the cpu
+ * should jump to.
*/
.balign 8
.globl __secondary_hold_spinloop
@@ -140,16 +139,15 @@
tovirt(r26,r26)
#endif
/* All secondary cpus wait here until told to start. */
-100: ld r4,__secondary_hold_spinloop-_stext(r26)
- cmpdi 0,r4,0
+100: ld r12,__secondary_hold_spinloop-_stext(r26)
+ cmpdi 0,r12,0
beq 100b
#if defined(CONFIG_SMP) || defined(CONFIG_KEXEC)
#ifdef CONFIG_PPC_BOOK3E
- tovirt(r4,r4)
+ tovirt(r12,r12)
#endif
- ld r4,0(r4) /* deref function descriptor */
- mtctr r4
+ mtctr r12
mr r3,r24
/*
* it may be the case that other platforms have r4 right to
@@ -186,16 +184,16 @@
mr r24,r3
/* turn on 64-bit mode */
- bl .enable_64b_mode
+ bl enable_64b_mode
/* get a valid TOC pointer, wherever we're mapped at */
- bl .relative_toc
+ bl relative_toc
tovirt(r2,r2)
#ifdef CONFIG_PPC_BOOK3E
/* Book3E initialization */
mr r3,r24
- bl .book3e_secondary_thread_init
+ bl book3e_secondary_thread_init
#endif
b generic_secondary_common_init
@@ -214,17 +212,17 @@
mr r25,r4
/* turn on 64-bit mode */
- bl .enable_64b_mode
+ bl enable_64b_mode
/* get a valid TOC pointer, wherever we're mapped at */
- bl .relative_toc
+ bl relative_toc
tovirt(r2,r2)
#ifdef CONFIG_PPC_BOOK3E
/* Book3E initialization */
mr r3,r24
mr r4,r25
- bl .book3e_secondary_core_init
+ bl book3e_secondary_core_init
#endif
generic_secondary_common_init:
@@ -236,7 +234,7 @@
ld r13,0(r13) /* Get base vaddr of paca array */
#ifndef CONFIG_SMP
addi r13,r13,PACA_SIZE /* know r13 if used accidentally */
- b .kexec_wait /* wait for next kernel if !SMP */
+ b kexec_wait /* wait for next kernel if !SMP */
#else
LOAD_REG_ADDR(r7, nr_cpu_ids) /* Load nr_cpu_ids address */
lwz r7,0(r7) /* also the max paca allocated */
@@ -250,7 +248,7 @@
blt 1b
mr r3,r24 /* not found, copy phys to r3 */
- b .kexec_wait /* next kernel might do better */
+ b kexec_wait /* next kernel might do better */
2: SET_PACA(r13)
#ifdef CONFIG_PPC_BOOK3E
@@ -264,11 +262,13 @@
/* See if we need to call a cpu state restore handler */
LOAD_REG_ADDR(r23, cur_cpu_spec)
ld r23,0(r23)
- ld r23,CPU_SPEC_RESTORE(r23)
- cmpdi 0,r23,0
+ ld r12,CPU_SPEC_RESTORE(r23)
+ cmpdi 0,r12,0
beq 3f
- ld r23,0(r23)
- mtctr r23
+#if !defined(_CALL_ELF) || _CALL_ELF != 2
+ ld r12,0(r12)
+#endif
+ mtctr r12
bctrl
3: LOAD_REG_ADDR(r3, spinning_secondaries) /* Decrement spinning_secondaries */
@@ -299,7 +299,7 @@
* Assumes we're mapped EA == RA if the MMU is on.
*/
#ifdef CONFIG_PPC_BOOK3S
-_STATIC(__mmu_off)
+__mmu_off:
mfmsr r3
andi. r0,r3,MSR_IR|MSR_DR
beqlr
@@ -324,12 +324,12 @@
* DT block, r4 is a physical pointer to the kernel itself
*
*/
-_GLOBAL(__start_initialization_multiplatform)
+__start_initialization_multiplatform:
/* Make sure we are running in 64 bits mode */
- bl .enable_64b_mode
+ bl enable_64b_mode
/* Get TOC pointer (current runtime address) */
- bl .relative_toc
+ bl relative_toc
/* find out where we are now */
bcl 20,31,$+4
@@ -342,7 +342,7 @@
*/
cmpldi cr0,r5,0
beq 1f
- b .__boot_from_prom /* yes -> prom */
+ b __boot_from_prom /* yes -> prom */
1:
/* Save parameters */
mr r31,r3
@@ -354,8 +354,8 @@
#endif
#ifdef CONFIG_PPC_BOOK3E
- bl .start_initialization_book3e
- b .__after_prom_start
+ bl start_initialization_book3e
+ b __after_prom_start
#else
/* Setup some critical 970 SPRs before switching MMU off */
mfspr r0,SPRN_PVR
@@ -368,15 +368,15 @@
beq 1f
cmpwi r0,0x45 /* 970GX */
bne 2f
-1: bl .__cpu_preinit_ppc970
+1: bl __cpu_preinit_ppc970
2:
/* Switch off MMU if not already off */
- bl .__mmu_off
- b .__after_prom_start
+ bl __mmu_off
+ b __after_prom_start
#endif /* CONFIG_PPC_BOOK3E */
-_INIT_STATIC(__boot_from_prom)
+__boot_from_prom:
#ifdef CONFIG_PPC_OF_BOOT_TRAMPOLINE
/* Save parameters */
mr r31,r3
@@ -395,7 +395,7 @@
#ifdef CONFIG_RELOCATABLE
/* Relocate code for where we are now */
mr r3,r26
- bl .relocate
+ bl relocate
#endif
/* Restore parameters */
@@ -407,14 +407,14 @@
/* Do all of the interaction with OF client interface */
mr r8,r26
- bl .prom_init
+ bl prom_init
#endif /* #CONFIG_PPC_OF_BOOT_TRAMPOLINE */
/* We never return. We also hit that trap if trying to boot
* from OF while CONFIG_PPC_OF_BOOT_TRAMPOLINE isn't selected */
trap
-_STATIC(__after_prom_start)
+__after_prom_start:
#ifdef CONFIG_RELOCATABLE
/* process relocations for the final address of the kernel */
lis r25,PAGE_OFFSET@highest /* compute virtual base of kernel */
@@ -424,7 +424,7 @@
bne 1f
add r25,r25,r26
1: mr r3,r25
- bl .relocate
+ bl relocate
#endif
/*
@@ -464,12 +464,12 @@
lis r5,(copy_to_here - _stext)@ha
addi r5,r5,(copy_to_here - _stext)@l /* # bytes of memory to copy */
- bl .copy_and_flush /* copy the first n bytes */
+ bl copy_and_flush /* copy the first n bytes */
/* this includes the code being */
/* executed here. */
addis r8,r3,(4f - _stext)@ha /* Jump to the copy of this code */
- addi r8,r8,(4f - _stext)@l /* that we just made */
- mtctr r8
+ addi r12,r8,(4f - _stext)@l /* that we just made */
+ mtctr r12
bctr
.balign 8
@@ -478,9 +478,9 @@
4: /* Now copy the rest of the kernel up to _end */
addis r5,r26,(p_end - _stext)@ha
ld r5,(p_end - _stext)@l(r5) /* get _end */
-5: bl .copy_and_flush /* copy the rest */
+5: bl copy_and_flush /* copy the rest */
-9: b .start_here_multiplatform
+9: b start_here_multiplatform
/*
* Copy routine used to copy the kernel to start at physical address 0
@@ -544,7 +544,7 @@
_GLOBAL(pmac_secondary_start)
/* turn on 64-bit mode */
- bl .enable_64b_mode
+ bl enable_64b_mode
li r0,0
mfspr r3,SPRN_HID4
@@ -556,11 +556,11 @@
slbia
/* get TOC pointer (real address) */
- bl .relative_toc
+ bl relative_toc
tovirt(r2,r2)
/* Copy some CPU settings from CPU 0 */
- bl .__restore_cpu_ppc970
+ bl __restore_cpu_ppc970
/* pSeries do that early though I don't think we really need it */
mfmsr r3
@@ -619,7 +619,7 @@
std r14,PACAKSAVE(r13)
/* Do early setup for that CPU (stab, slb, hash table pointer) */
- bl .early_setup_secondary
+ bl early_setup_secondary
/*
* setup the new stack pointer, but *don't* use this until
@@ -639,7 +639,7 @@
stb r0,PACAIRQHAPPENED(r13)
/* enable MMU and jump to start_secondary */
- LOAD_REG_ADDR(r3, .start_secondary_prolog)
+ LOAD_REG_ADDR(r3, start_secondary_prolog)
LOAD_REG_IMMEDIATE(r4, MSR_KERNEL)
mtspr SPRN_SRR0,r3
@@ -652,11 +652,11 @@
* zero the stack back-chain pointer and get the TOC virtual address
* before going into C code.
*/
-_GLOBAL(start_secondary_prolog)
+start_secondary_prolog:
ld r2,PACATOC(r13)
li r3,0
std r3,0(r1) /* Zero the stack frame pointer */
- bl .start_secondary
+ bl start_secondary
b .
/*
* Reset stack pointer and call start_secondary
@@ -667,14 +667,14 @@
ld r1,PACAKSAVE(r13) /* Reload kernel stack pointer */
li r3,0
std r3,0(r1) /* Zero the stack frame pointer */
- bl .start_secondary
+ bl start_secondary
b .
#endif
/*
* This subroutine clobbers r11 and r12
*/
-_GLOBAL(enable_64b_mode)
+enable_64b_mode:
mfmsr r11 /* grab the current MSR */
#ifdef CONFIG_PPC_BOOK3E
oris r11,r11,0x8000 /* CM bit set, we'll set ICM later */
@@ -715,9 +715,9 @@
/*
* This is where the main kernel code starts.
*/
-_INIT_STATIC(start_here_multiplatform)
+start_here_multiplatform:
/* set up the TOC */
- bl .relative_toc
+ bl relative_toc
tovirt(r2,r2)
/* Clear out the BSS. It may have been done in prom_init,
@@ -776,9 +776,9 @@
/* Restore parameters passed from prom_init/kexec */
mr r3,r31
- bl .early_setup /* also sets r13 and SPRG_PACA */
+ bl early_setup /* also sets r13 and SPRG_PACA */
- LOAD_REG_ADDR(r3, .start_here_common)
+ LOAD_REG_ADDR(r3, start_here_common)
ld r4,PACAKMSR(r13)
mtspr SPRN_SRR0,r3
mtspr SPRN_SRR1,r4
@@ -786,7 +786,8 @@
b . /* prevent speculative execution */
/* This is where all platforms converge execution */
-_INIT_GLOBAL(start_here_common)
+
+start_here_common:
/* relocation is on at this point */
std r1,PACAKSAVE(r13)
@@ -794,7 +795,7 @@
ld r2,PACATOC(r13)
/* Do more system initializations in virtual mode */
- bl .setup_system
+ bl setup_system
/* Mark interrupts soft and hard disabled (they might be enabled
* in the PACA when doing hotplug)
@@ -805,7 +806,7 @@
stb r0,PACAIRQHAPPENED(r13)
/* Generic kernel entry */
- bl .start_kernel
+ bl start_kernel
/* Not reached */
BUG_OPCODE
diff --git a/arch/powerpc/kernel/hw_breakpoint.c b/arch/powerpc/kernel/hw_breakpoint.c
index b0a1792..0bb5918 100644
--- a/arch/powerpc/kernel/hw_breakpoint.c
+++ b/arch/powerpc/kernel/hw_breakpoint.c
@@ -72,7 +72,7 @@
* If so, DABR will be populated in single_step_dabr_instruction().
*/
if (current->thread.last_hit_ubp != bp)
- set_breakpoint(info);
+ __set_breakpoint(info);
return 0;
}
@@ -198,7 +198,7 @@
info = counter_arch_bp(tsk->thread.last_hit_ubp);
regs->msr &= ~MSR_SE;
- set_breakpoint(info);
+ __set_breakpoint(info);
tsk->thread.last_hit_ubp = NULL;
}
@@ -284,7 +284,7 @@
if (!(info->type & HW_BRK_TYPE_EXTRANEOUS_IRQ))
perf_bp_event(bp, regs);
- set_breakpoint(info);
+ __set_breakpoint(info);
out:
rcu_read_unlock();
return rc;
@@ -316,7 +316,7 @@
if (!(info->type & HW_BRK_TYPE_EXTRANEOUS_IRQ))
perf_bp_event(bp, regs);
- set_breakpoint(info);
+ __set_breakpoint(info);
current->thread.last_hit_ubp = NULL;
/*
diff --git a/arch/powerpc/kernel/idle_book3e.S b/arch/powerpc/kernel/idle_book3e.S
index bfb73cc..48c21ac 100644
--- a/arch/powerpc/kernel/idle_book3e.S
+++ b/arch/powerpc/kernel/idle_book3e.S
@@ -43,7 +43,7 @@
*/
#ifdef CONFIG_TRACE_IRQFLAGS
stdu r1,-128(r1)
- bl .trace_hardirqs_on
+ bl trace_hardirqs_on
addi r1,r1,128
#endif
li r0,1
diff --git a/arch/powerpc/kernel/idle_power4.S b/arch/powerpc/kernel/idle_power4.S
index e3edaa18..f57a193 100644
--- a/arch/powerpc/kernel/idle_power4.S
+++ b/arch/powerpc/kernel/idle_power4.S
@@ -46,7 +46,7 @@
mflr r0
std r0,16(r1)
stdu r1,-128(r1)
- bl .trace_hardirqs_on
+ bl trace_hardirqs_on
addi r1,r1,128
ld r0,16(r1)
mtlr r0
diff --git a/arch/powerpc/kernel/idle_power7.S b/arch/powerpc/kernel/idle_power7.S
index c3ab869..2480256 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -39,6 +39,10 @@
* Pass requested state in r3:
* 0 - nap
* 1 - sleep
+ *
+ * To check IRQ_HAPPENED in r4
+ * 0 - don't check
+ * 1 - check
*/
_GLOBAL(power7_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
@@ -58,7 +62,7 @@
/* Make sure FPU, VSX etc... are flushed as we may lose
* state when going to nap mode
*/
- bl .discard_lazy_cpu_state
+ bl discard_lazy_cpu_state
#endif /* CONFIG_SMP */
/* Hard disable interrupts */
@@ -71,6 +75,8 @@
lbz r0,PACAIRQHAPPENED(r13)
cmpwi cr0,r0,0
beq 1f
+ cmpwi cr0,r4,0
+ beq 1f
addi r1,r1,INT_FRAME_SIZE
ld r0,16(r1)
mtlr r0
@@ -114,15 +120,18 @@
lwz r4,ADDROFF(powersave_nap)(r3)
cmpwi 0,r4,0
beqlr
+ li r3, 1
/* fall through */
_GLOBAL(power7_nap)
+ mr r4,r3
li r3,0
b power7_powersave_common
/* No return */
_GLOBAL(power7_sleep)
li r3,1
+ li r4,0
b power7_powersave_common
/* No return */
@@ -168,7 +177,7 @@
_GLOBAL(power7_wakeup_noloss)
lbz r0,PACA_NAPSTATELOST(r13)
cmpwi r0,0
- bne .power7_wakeup_loss
+ bne power7_wakeup_loss
ld r1,PACAR1(r13)
ld r4,_MSR(r1)
ld r5,_NIP(r1)
diff --git a/arch/powerpc/kernel/irq.c b/arch/powerpc/kernel/irq.c
index ca1cd74..248ee7e 100644
--- a/arch/powerpc/kernel/irq.c
+++ b/arch/powerpc/kernel/irq.c
@@ -304,7 +304,7 @@
* being re-enabled and generally sanitized the lazy irq state,
* and in the latter case it will leave with interrupts hard
* disabled and marked as such, so the local_irq_enable() call
- * in cpu_idle() will properly re-enable everything.
+ * in arch_cpu_idle() will properly re-enable everything.
*/
bool prep_irq_for_idle(void)
{
diff --git a/arch/powerpc/kernel/legacy_serial.c b/arch/powerpc/kernel/legacy_serial.c
index 40bd7bd..9362588 100644
--- a/arch/powerpc/kernel/legacy_serial.c
+++ b/arch/powerpc/kernel/legacy_serial.c
@@ -48,6 +48,9 @@
static unsigned int legacy_serial_count;
static int legacy_serial_console = -1;
+static const upf_t legacy_port_flags = UPF_BOOT_AUTOCONF | UPF_SKIP_TEST |
+ UPF_SHARE_IRQ | UPF_FIXED_PORT;
+
static unsigned int tsi_serial_in(struct uart_port *p, int offset)
{
unsigned int tmp;
@@ -71,8 +74,9 @@
phys_addr_t taddr, unsigned long irq,
upf_t flags, int irq_check_parent)
{
- const __be32 *clk, *spd;
+ const __be32 *clk, *spd, *rs;
u32 clock = BASE_BAUD * 16;
+ u32 shift = 0;
int index;
/* get clock freq. if present */
@@ -83,6 +87,11 @@
/* get default speed if present */
spd = of_get_property(np, "current-speed", NULL);
+ /* get register shift if present */
+ rs = of_get_property(np, "reg-shift", NULL);
+ if (rs && *rs)
+ shift = be32_to_cpup(rs);
+
/* If we have a location index, then try to use it */
if (want_index >= 0 && want_index < MAX_LEGACY_SERIAL_PORTS)
index = want_index;
@@ -126,6 +135,7 @@
legacy_serial_ports[index].uartclk = clock;
legacy_serial_ports[index].irq = irq;
legacy_serial_ports[index].flags = flags;
+ legacy_serial_ports[index].regshift = shift;
legacy_serial_infos[index].taddr = taddr;
legacy_serial_infos[index].np = of_node_get(np);
legacy_serial_infos[index].clock = clock;
@@ -153,8 +163,6 @@
{
u64 addr;
const __be32 *addrp;
- upf_t flags = UPF_BOOT_AUTOCONF | UPF_SKIP_TEST | UPF_SHARE_IRQ
- | UPF_FIXED_PORT;
struct device_node *tsi = of_get_parent(np);
/* We only support ports that have a clock frequency properly
@@ -163,9 +171,8 @@
if (of_get_property(np, "clock-frequency", NULL) == NULL)
return -1;
- /* if reg-shift or offset, don't try to use it */
- if ((of_get_property(np, "reg-shift", NULL) != NULL) ||
- (of_get_property(np, "reg-offset", NULL) != NULL))
+ /* if reg-offset don't try to use it */
+ if ((of_get_property(np, "reg-offset", NULL) != NULL))
return -1;
/* if rtas uses this device, don't try to use it as well */
@@ -185,9 +192,11 @@
* IO port value. It will be fixed up later along with the irq
*/
if (tsi && !strcmp(tsi->type, "tsi-bridge"))
- return add_legacy_port(np, -1, UPIO_TSI, addr, addr, NO_IRQ, flags, 0);
+ return add_legacy_port(np, -1, UPIO_TSI, addr, addr,
+ NO_IRQ, legacy_port_flags, 0);
else
- return add_legacy_port(np, -1, UPIO_MEM, addr, addr, NO_IRQ, flags, 0);
+ return add_legacy_port(np, -1, UPIO_MEM, addr, addr,
+ NO_IRQ, legacy_port_flags, 0);
}
static int __init add_legacy_isa_port(struct device_node *np,
@@ -233,7 +242,7 @@
/* Add port, irq will be dealt with later */
return add_legacy_port(np, index, UPIO_PORT, be32_to_cpu(reg[1]),
- taddr, NO_IRQ, UPF_BOOT_AUTOCONF, 0);
+ taddr, NO_IRQ, legacy_port_flags, 0);
}
@@ -306,7 +315,7 @@
* IO port value. It will be fixed up later along with the irq
*/
return add_legacy_port(np, index, iotype, base, addr, NO_IRQ,
- UPF_BOOT_AUTOCONF, np != pci_dev);
+ legacy_port_flags, np != pci_dev);
}
#endif
@@ -315,17 +324,20 @@
struct legacy_serial_info *info = &legacy_serial_infos[console];
struct plat_serial8250_port *port = &legacy_serial_ports[console];
void __iomem *addr;
+ unsigned int stride;
+
+ stride = 1 << port->regshift;
/* Check if a translated MMIO address has been found */
if (info->taddr) {
addr = ioremap(info->taddr, 0x1000);
if (addr == NULL)
return;
- udbg_uart_init_mmio(addr, 1);
+ udbg_uart_init_mmio(addr, stride);
} else {
/* Check if it's PIO and we support untranslated PIO */
if (port->iotype == UPIO_PORT && isa_io_special)
- udbg_uart_init_pio(port->iobase, 1);
+ udbg_uart_init_pio(port->iobase, stride);
else
return;
}
diff --git a/arch/powerpc/kernel/misc_64.S b/arch/powerpc/kernel/misc_64.S
index 3d02495..4e314b9 100644
--- a/arch/powerpc/kernel/misc_64.S
+++ b/arch/powerpc/kernel/misc_64.S
@@ -34,7 +34,7 @@
std r0,16(r1)
stdu r1,THREAD_SIZE-STACK_FRAME_OVERHEAD(r3)
mr r1,r3
- bl .__do_softirq
+ bl __do_softirq
ld r1,0(r1)
ld r0,16(r1)
mtlr r0
@@ -45,7 +45,7 @@
std r0,16(r1)
stdu r1,THREAD_SIZE-STACK_FRAME_OVERHEAD(r4)
mr r1,r4
- bl .__do_irq
+ bl __do_irq
ld r1,0(r1)
ld r0,16(r1)
mtlr r0
@@ -506,7 +506,7 @@
stb r4,PACAKEXECSTATE(r13)
SYNC
- b .kexec_wait
+ b kexec_wait
/*
* switch to real mode (turn mmu off)
@@ -576,7 +576,7 @@
/* copy dest pages, flush whole dest image */
mr r3,r29
- bl .kexec_copy_flush /* (image) */
+ bl kexec_copy_flush /* (image) */
/* turn off mmu */
bl real_mode
@@ -586,7 +586,7 @@
mr r4,r30 /* start, aka phys mem offset */
li r5,0x100
li r6,0
- bl .copy_and_flush /* (dest, src, copy limit, start offset) */
+ bl copy_and_flush /* (dest, src, copy limit, start offset) */
1: /* assume normal blr return */
/* release other cpus to the new kernel secondary start at 0x60 */
@@ -595,8 +595,12 @@
stw r6,kexec_flag-1b(5)
/* clear out hardware hash page table and tlb */
- ld r5,0(r27) /* deref function descriptor */
- mtctr r5
+#if !defined(_CALL_ELF) || _CALL_ELF != 2
+ ld r12,0(r27) /* deref function descriptor */
+#else
+ mr r12,r27
+#endif
+ mtctr r12
bctrl /* ppc_md.hpte_clear_all(void); */
/*
@@ -630,3 +634,31 @@
li r5,0
blr /* image->start(physid, image->start, 0); */
#endif /* CONFIG_KEXEC */
+
+#ifdef CONFIG_MODULES
+#if defined(_CALL_ELF) && _CALL_ELF == 2
+
+#ifdef CONFIG_MODVERSIONS
+.weak __crc_TOC.
+.section "___kcrctab+TOC.","a"
+.globl __kcrctab_TOC.
+__kcrctab_TOC.:
+ .llong __crc_TOC.
+#endif
+
+/*
+ * Export a fake .TOC. since both modpost and depmod will complain otherwise.
+ * Both modpost and depmod strip the leading . so we do the same here.
+ */
+.section "__ksymtab_strings","a"
+__kstrtab_TOC.:
+ .asciz "TOC."
+
+.section "___ksymtab+TOC.","a"
+/* This symbol name is important: it's used by modpost to find exported syms */
+.globl __ksymtab_TOC.
+__ksymtab_TOC.:
+ .llong 0 /* .value */
+ .llong __kstrtab_TOC.
+#endif /* ELFv2 */
+#endif /* MODULES */
diff --git a/arch/powerpc/kernel/module_64.c b/arch/powerpc/kernel/module_64.c
index 12664c1..077d2ce6 100644
--- a/arch/powerpc/kernel/module_64.c
+++ b/arch/powerpc/kernel/module_64.c
@@ -22,6 +22,7 @@
#include <linux/vmalloc.h>
#include <linux/ftrace.h>
#include <linux/bug.h>
+#include <linux/uaccess.h>
#include <asm/module.h>
#include <asm/firmware.h>
#include <asm/code-patching.h>
@@ -41,46 +42,170 @@
#define DEBUGP(fmt , ...)
#endif
+#if defined(_CALL_ELF) && _CALL_ELF == 2
+#define R2_STACK_OFFSET 24
+
+/* An address is simply the address of the function. */
+typedef unsigned long func_desc_t;
+
+static func_desc_t func_desc(unsigned long addr)
+{
+ return addr;
+}
+static unsigned long func_addr(unsigned long addr)
+{
+ return addr;
+}
+static unsigned long stub_func_addr(func_desc_t func)
+{
+ return func;
+}
+
+/* PowerPC64 specific values for the Elf64_Sym st_other field. */
+#define STO_PPC64_LOCAL_BIT 5
+#define STO_PPC64_LOCAL_MASK (7 << STO_PPC64_LOCAL_BIT)
+#define PPC64_LOCAL_ENTRY_OFFSET(other) \
+ (((1 << (((other) & STO_PPC64_LOCAL_MASK) >> STO_PPC64_LOCAL_BIT)) >> 2) << 2)
+
+static unsigned int local_entry_offset(const Elf64_Sym *sym)
+{
+ /* sym->st_other indicates offset to local entry point
+ * (otherwise it will assume r12 is the address of the start
+ * of function and try to derive r2 from it). */
+ return PPC64_LOCAL_ENTRY_OFFSET(sym->st_other);
+}
+#else
+#define R2_STACK_OFFSET 40
+
+/* An address is address of the OPD entry, which contains address of fn. */
+typedef struct ppc64_opd_entry func_desc_t;
+
+static func_desc_t func_desc(unsigned long addr)
+{
+ return *(struct ppc64_opd_entry *)addr;
+}
+static unsigned long func_addr(unsigned long addr)
+{
+ return func_desc(addr).funcaddr;
+}
+static unsigned long stub_func_addr(func_desc_t func)
+{
+ return func.funcaddr;
+}
+static unsigned int local_entry_offset(const Elf64_Sym *sym)
+{
+ return 0;
+}
+#endif
+
/* Like PPC32, we need little trampolines to do > 24-bit jumps (into
the kernel itself). But on PPC64, these need to be used for every
jump, actually, to reset r2 (TOC+0x8000). */
struct ppc64_stub_entry
{
- /* 28 byte jump instruction sequence (7 instructions) */
- unsigned char jump[28];
- unsigned char unused[4];
+ /* 28 byte jump instruction sequence (7 instructions). We only
+ * need 6 instructions on ABIv2 but we always allocate 7 so
+ * so we don't have to modify the trampoline load instruction. */
+ u32 jump[7];
+ u32 unused;
/* Data for the above code */
- struct ppc64_opd_entry opd;
+ func_desc_t funcdata;
};
-/* We use a stub to fix up r2 (TOC ptr) and to jump to the (external)
- function which may be more than 24-bits away. We could simply
- patch the new r2 value and function pointer into the stub, but it's
- significantly shorter to put these values at the end of the stub
- code, and patch the stub address (32-bits relative to the TOC ptr,
- r2) into the stub. */
-static struct ppc64_stub_entry ppc64_stub =
-{ .jump = {
-#ifdef __LITTLE_ENDIAN__
- 0x00, 0x00, 0x82, 0x3d, /* addis r12,r2, <high> */
- 0x00, 0x00, 0x8c, 0x39, /* addi r12,r12, <low> */
+/*
+ * PPC64 uses 24 bit jumps, but we need to jump into other modules or
+ * the kernel which may be further. So we jump to a stub.
+ *
+ * For ELFv1 we need to use this to set up the new r2 value (aka TOC
+ * pointer). For ELFv2 it's the callee's responsibility to set up the
+ * new r2, but for both we need to save the old r2.
+ *
+ * We could simply patch the new r2 value and function pointer into
+ * the stub, but it's significantly shorter to put these values at the
+ * end of the stub code, and patch the stub address (32-bits relative
+ * to the TOC ptr, r2) into the stub.
+ */
+
+static u32 ppc64_stub_insns[] = {
+ 0x3d620000, /* addis r11,r2, <high> */
+ 0x396b0000, /* addi r11,r11, <low> */
/* Save current r2 value in magic place on the stack. */
- 0x28, 0x00, 0x41, 0xf8, /* std r2,40(r1) */
- 0x20, 0x00, 0x6c, 0xe9, /* ld r11,32(r12) */
- 0x28, 0x00, 0x4c, 0xe8, /* ld r2,40(r12) */
- 0xa6, 0x03, 0x69, 0x7d, /* mtctr r11 */
- 0x20, 0x04, 0x80, 0x4e /* bctr */
-#else
- 0x3d, 0x82, 0x00, 0x00, /* addis r12,r2, <high> */
- 0x39, 0x8c, 0x00, 0x00, /* addi r12,r12, <low> */
- /* Save current r2 value in magic place on the stack. */
- 0xf8, 0x41, 0x00, 0x28, /* std r2,40(r1) */
- 0xe9, 0x6c, 0x00, 0x20, /* ld r11,32(r12) */
- 0xe8, 0x4c, 0x00, 0x28, /* ld r2,40(r12) */
- 0x7d, 0x69, 0x03, 0xa6, /* mtctr r11 */
- 0x4e, 0x80, 0x04, 0x20 /* bctr */
+ 0xf8410000|R2_STACK_OFFSET, /* std r2,R2_STACK_OFFSET(r1) */
+ 0xe98b0020, /* ld r12,32(r11) */
+#if !defined(_CALL_ELF) || _CALL_ELF != 2
+ /* Set up new r2 from function descriptor */
+ 0xe84b0028, /* ld r2,40(r11) */
#endif
-} };
+ 0x7d8903a6, /* mtctr r12 */
+ 0x4e800420 /* bctr */
+};
+
+#ifdef CONFIG_DYNAMIC_FTRACE
+
+static u32 ppc64_stub_mask[] = {
+ 0xffff0000,
+ 0xffff0000,
+ 0xffffffff,
+ 0xffffffff,
+#if !defined(_CALL_ELF) || _CALL_ELF != 2
+ 0xffffffff,
+#endif
+ 0xffffffff,
+ 0xffffffff
+};
+
+bool is_module_trampoline(u32 *p)
+{
+ unsigned int i;
+ u32 insns[ARRAY_SIZE(ppc64_stub_insns)];
+
+ BUILD_BUG_ON(sizeof(ppc64_stub_insns) != sizeof(ppc64_stub_mask));
+
+ if (probe_kernel_read(insns, p, sizeof(insns)))
+ return -EFAULT;
+
+ for (i = 0; i < ARRAY_SIZE(ppc64_stub_insns); i++) {
+ u32 insna = insns[i];
+ u32 insnb = ppc64_stub_insns[i];
+ u32 mask = ppc64_stub_mask[i];
+
+ if ((insna & mask) != (insnb & mask))
+ return false;
+ }
+
+ return true;
+}
+
+int module_trampoline_target(struct module *mod, u32 *trampoline,
+ unsigned long *target)
+{
+ u32 buf[2];
+ u16 upper, lower;
+ long offset;
+ void *toc_entry;
+
+ if (probe_kernel_read(buf, trampoline, sizeof(buf)))
+ return -EFAULT;
+
+ upper = buf[0] & 0xffff;
+ lower = buf[1] & 0xffff;
+
+ /* perform the addis/addi, both signed */
+ offset = ((short)upper << 16) + (short)lower;
+
+ /*
+ * Now get the address this trampoline jumps to. This
+ * is always 32 bytes into our trampoline stub.
+ */
+ toc_entry = (void *)mod->arch.toc + offset + 32;
+
+ if (probe_kernel_read(target, toc_entry, sizeof(*target)))
+ return -EFAULT;
+
+ return 0;
+}
+
+#endif
/* Count how many different 24-bit relocations (different symbol,
different addend) */
@@ -183,6 +308,7 @@
return relocs * sizeof(struct ppc64_stub_entry);
}
+/* Still needed for ELFv2, for .TOC. */
static void dedotify_versions(struct modversion_info *vers,
unsigned long size)
{
@@ -193,7 +319,7 @@
memmove(vers->name, vers->name+1, strlen(vers->name));
}
-/* Undefined symbols which refer to .funcname, hack to funcname */
+/* Undefined symbols which refer to .funcname, hack to funcname (or .TOC.) */
static void dedotify(Elf64_Sym *syms, unsigned int numsyms, char *strtab)
{
unsigned int i;
@@ -207,6 +333,24 @@
}
}
+static Elf64_Sym *find_dot_toc(Elf64_Shdr *sechdrs,
+ const char *strtab,
+ unsigned int symindex)
+{
+ unsigned int i, numsyms;
+ Elf64_Sym *syms;
+
+ syms = (Elf64_Sym *)sechdrs[symindex].sh_addr;
+ numsyms = sechdrs[symindex].sh_size / sizeof(Elf64_Sym);
+
+ for (i = 1; i < numsyms; i++) {
+ if (syms[i].st_shndx == SHN_UNDEF
+ && strcmp(strtab + syms[i].st_name, "TOC.") == 0)
+ return &syms[i];
+ }
+ return NULL;
+}
+
int module_frob_arch_sections(Elf64_Ehdr *hdr,
Elf64_Shdr *sechdrs,
char *secstrings,
@@ -271,21 +415,12 @@
/* Patch stub to reference function and correct r2 value. */
static inline int create_stub(Elf64_Shdr *sechdrs,
struct ppc64_stub_entry *entry,
- struct ppc64_opd_entry *opd,
+ unsigned long addr,
struct module *me)
{
- Elf64_Half *loc1, *loc2;
long reladdr;
- *entry = ppc64_stub;
-
-#ifdef __LITTLE_ENDIAN__
- loc1 = (Elf64_Half *)&entry->jump[0];
- loc2 = (Elf64_Half *)&entry->jump[4];
-#else
- loc1 = (Elf64_Half *)&entry->jump[2];
- loc2 = (Elf64_Half *)&entry->jump[6];
-#endif
+ memcpy(entry->jump, ppc64_stub_insns, sizeof(ppc64_stub_insns));
/* Stub uses address relative to r2. */
reladdr = (unsigned long)entry - my_r2(sechdrs, me);
@@ -296,35 +431,33 @@
}
DEBUGP("Stub %p get data from reladdr %li\n", entry, reladdr);
- *loc1 = PPC_HA(reladdr);
- *loc2 = PPC_LO(reladdr);
- entry->opd.funcaddr = opd->funcaddr;
- entry->opd.r2 = opd->r2;
+ entry->jump[0] |= PPC_HA(reladdr);
+ entry->jump[1] |= PPC_LO(reladdr);
+ entry->funcdata = func_desc(addr);
return 1;
}
-/* Create stub to jump to function described in this OPD: we need the
+/* Create stub to jump to function described in this OPD/ptr: we need the
stub to set up the TOC ptr (r2) for the function. */
static unsigned long stub_for_addr(Elf64_Shdr *sechdrs,
- unsigned long opdaddr,
+ unsigned long addr,
struct module *me)
{
struct ppc64_stub_entry *stubs;
- struct ppc64_opd_entry *opd = (void *)opdaddr;
unsigned int i, num_stubs;
num_stubs = sechdrs[me->arch.stubs_section].sh_size / sizeof(*stubs);
/* Find this stub, or if that fails, the next avail. entry */
stubs = (void *)sechdrs[me->arch.stubs_section].sh_addr;
- for (i = 0; stubs[i].opd.funcaddr; i++) {
+ for (i = 0; stub_func_addr(stubs[i].funcdata); i++) {
BUG_ON(i >= num_stubs);
- if (stubs[i].opd.funcaddr == opd->funcaddr)
+ if (stub_func_addr(stubs[i].funcdata) == func_addr(addr))
return (unsigned long)&stubs[i];
}
- if (!create_stub(sechdrs, &stubs[i], opd, me))
+ if (!create_stub(sechdrs, &stubs[i], addr, me))
return 0;
return (unsigned long)&stubs[i];
@@ -339,7 +472,8 @@
me->name, *instruction);
return 0;
}
- *instruction = 0xe8410028; /* ld r2,40(r1) */
+ /* ld r2,R2_STACK_OFFSET(r1) */
+ *instruction = 0xe8410000 | R2_STACK_OFFSET;
return 1;
}
@@ -357,6 +491,17 @@
DEBUGP("Applying ADD relocate section %u to %u\n", relsec,
sechdrs[relsec].sh_info);
+
+ /* First time we're called, we can fix up .TOC. */
+ if (!me->arch.toc_fixed) {
+ sym = find_dot_toc(sechdrs, strtab, symindex);
+ /* It's theoretically possible that a module doesn't want a
+ * .TOC. so don't fail it just for that. */
+ if (sym)
+ sym->st_value = my_r2(sechdrs, me);
+ me->arch.toc_fixed = true;
+ }
+
for (i = 0; i < sechdrs[relsec].sh_size / sizeof(*rela); i++) {
/* This is where to make the change */
location = (void *)sechdrs[sechdrs[relsec].sh_info].sh_addr
@@ -453,7 +598,8 @@
return -ENOENT;
if (!restore_r2((u32 *)location + 1, me))
return -ENOEXEC;
- }
+ } else
+ value += local_entry_offset(sym);
/* Convert value to relative */
value -= (unsigned long)location;
@@ -474,6 +620,31 @@
*location = value - (unsigned long)location;
break;
+ case R_PPC64_TOCSAVE:
+ /*
+ * Marker reloc indicates we don't have to save r2.
+ * That would only save us one instruction, so ignore
+ * it.
+ */
+ break;
+
+ case R_PPC64_REL16_HA:
+ /* Subtract location pointer */
+ value -= (unsigned long)location;
+ value = ((value + 0x8000) >> 16);
+ *((uint16_t *) location)
+ = (*((uint16_t *) location) & ~0xffff)
+ | (value & 0xffff);
+ break;
+
+ case R_PPC64_REL16_LO:
+ /* Subtract location pointer */
+ value -= (unsigned long)location;
+ *((uint16_t *) location)
+ = (*((uint16_t *) location) & ~0xffff)
+ | (value & 0xffff);
+ break;
+
default:
printk("%s: Unknown ADD relocation: %lu\n",
me->name,
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 24d342e..b49c72f 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -21,6 +21,7 @@
#include <linux/string.h>
#include <linux/init.h>
#include <linux/bootmem.h>
+#include <linux/delay.h>
#include <linux/export.h>
#include <linux/of_address.h>
#include <linux/of_pci.h>
@@ -120,6 +121,25 @@
return 1;
}
+void pcibios_reset_secondary_bus(struct pci_dev *dev)
+{
+ u16 ctrl;
+
+ if (ppc_md.pcibios_reset_secondary_bus) {
+ ppc_md.pcibios_reset_secondary_bus(dev);
+ return;
+ }
+
+ pci_read_config_word(dev, PCI_BRIDGE_CONTROL, &ctrl);
+ ctrl |= PCI_BRIDGE_CTL_BUS_RESET;
+ pci_write_config_word(dev, PCI_BRIDGE_CONTROL, ctrl);
+ msleep(2);
+
+ ctrl &= ~PCI_BRIDGE_CTL_BUS_RESET;
+ pci_write_config_word(dev, PCI_BRIDGE_CONTROL, ctrl);
+ ssleep(1);
+}
+
static resource_size_t pcibios_io_size(const struct pci_controller *hose)
{
#ifdef CONFIG_PPC64
@@ -646,60 +666,36 @@
void pci_process_bridge_OF_ranges(struct pci_controller *hose,
struct device_node *dev, int primary)
{
- const __be32 *ranges;
- int rlen;
- int pna = of_n_addr_cells(dev);
- int np = pna + 5;
int memno = 0;
- u32 pci_space;
- unsigned long long pci_addr, cpu_addr, pci_next, cpu_next, size;
struct resource *res;
+ struct of_pci_range range;
+ struct of_pci_range_parser parser;
printk(KERN_INFO "PCI host bridge %s %s ranges:\n",
dev->full_name, primary ? "(primary)" : "");
- /* Get ranges property */
- ranges = of_get_property(dev, "ranges", &rlen);
- if (ranges == NULL)
+ /* Check for ranges property */
+ if (of_pci_range_parser_init(&parser, dev))
return;
/* Parse it */
- while ((rlen -= np * 4) >= 0) {
- /* Read next ranges element */
- pci_space = of_read_number(ranges, 1);
- pci_addr = of_read_number(ranges + 1, 2);
- cpu_addr = of_translate_address(dev, ranges + 3);
- size = of_read_number(ranges + pna + 3, 2);
- ranges += np;
-
+ for_each_of_pci_range(&parser, &range) {
/* If we failed translation or got a zero-sized region
* (some FW try to feed us with non sensical zero sized regions
* such as power3 which look like some kind of attempt at exposing
* the VGA memory hole)
*/
- if (cpu_addr == OF_BAD_ADDR || size == 0)
+ if (range.cpu_addr == OF_BAD_ADDR || range.size == 0)
continue;
- /* Now consume following elements while they are contiguous */
- for (; rlen >= np * sizeof(u32);
- ranges += np, rlen -= np * 4) {
- if (of_read_number(ranges, 1) != pci_space)
- break;
- pci_next = of_read_number(ranges + 1, 2);
- cpu_next = of_translate_address(dev, ranges + 3);
- if (pci_next != pci_addr + size ||
- cpu_next != cpu_addr + size)
- break;
- size += of_read_number(ranges + pna + 3, 2);
- }
-
/* Act based on address space type */
res = NULL;
- switch ((pci_space >> 24) & 0x3) {
- case 1: /* PCI IO space */
+ switch (range.flags & IORESOURCE_TYPE_BITS) {
+ case IORESOURCE_IO:
printk(KERN_INFO
" IO 0x%016llx..0x%016llx -> 0x%016llx\n",
- cpu_addr, cpu_addr + size - 1, pci_addr);
+ range.cpu_addr, range.cpu_addr + range.size - 1,
+ range.pci_addr);
/* We support only one IO range */
if (hose->pci_io_size) {
@@ -709,11 +705,12 @@
}
#ifdef CONFIG_PPC32
/* On 32 bits, limit I/O space to 16MB */
- if (size > 0x01000000)
- size = 0x01000000;
+ if (range.size > 0x01000000)
+ range.size = 0x01000000;
/* 32 bits needs to map IOs here */
- hose->io_base_virt = ioremap(cpu_addr, size);
+ hose->io_base_virt = ioremap(range.cpu_addr,
+ range.size);
/* Expect trouble if pci_addr is not 0 */
if (primary)
@@ -723,20 +720,20 @@
/* pci_io_size and io_base_phys always represent IO
* space starting at 0 so we factor in pci_addr
*/
- hose->pci_io_size = pci_addr + size;
- hose->io_base_phys = cpu_addr - pci_addr;
+ hose->pci_io_size = range.pci_addr + range.size;
+ hose->io_base_phys = range.cpu_addr - range.pci_addr;
/* Build resource */
res = &hose->io_resource;
- res->flags = IORESOURCE_IO;
- res->start = pci_addr;
+ range.cpu_addr = range.pci_addr;
break;
- case 2: /* PCI Memory space */
- case 3: /* PCI 64 bits Memory space */
+ case IORESOURCE_MEM:
printk(KERN_INFO
" MEM 0x%016llx..0x%016llx -> 0x%016llx %s\n",
- cpu_addr, cpu_addr + size - 1, pci_addr,
- (pci_space & 0x40000000) ? "Prefetch" : "");
+ range.cpu_addr, range.cpu_addr + range.size - 1,
+ range.pci_addr,
+ (range.pci_space & 0x40000000) ?
+ "Prefetch" : "");
/* We support only 3 memory ranges */
if (memno >= 3) {
@@ -745,28 +742,21 @@
continue;
}
/* Handles ISA memory hole space here */
- if (pci_addr == 0) {
+ if (range.pci_addr == 0) {
if (primary || isa_mem_base == 0)
- isa_mem_base = cpu_addr;
- hose->isa_mem_phys = cpu_addr;
- hose->isa_mem_size = size;
+ isa_mem_base = range.cpu_addr;
+ hose->isa_mem_phys = range.cpu_addr;
+ hose->isa_mem_size = range.size;
}
/* Build resource */
- hose->mem_offset[memno] = cpu_addr - pci_addr;
+ hose->mem_offset[memno] = range.cpu_addr -
+ range.pci_addr;
res = &hose->mem_resources[memno++];
- res->flags = IORESOURCE_MEM;
- if (pci_space & 0x40000000)
- res->flags |= IORESOURCE_PREFETCH;
- res->start = cpu_addr;
break;
}
if (res != NULL) {
- res->name = dev->full_name;
- res->end = res->start + size - 1;
- res->parent = NULL;
- res->sibling = NULL;
- res->child = NULL;
+ of_pci_range_to_resource(&range, dev, res);
}
}
}
diff --git a/arch/powerpc/kernel/pci_of_scan.c b/arch/powerpc/kernel/pci_of_scan.c
index 059e244..44562aa 100644
--- a/arch/powerpc/kernel/pci_of_scan.c
+++ b/arch/powerpc/kernel/pci_of_scan.c
@@ -304,6 +304,9 @@
struct pci_dev *dev = NULL;
const __be32 *reg;
int reglen, devfn;
+#ifdef CONFIG_EEH
+ struct eeh_dev *edev = of_node_to_eeh_dev(dn);
+#endif
pr_debug(" * %s\n", dn->full_name);
if (!of_device_is_available(dn))
@@ -321,6 +324,12 @@
return dev;
}
+ /* Device removed permanently ? */
+#ifdef CONFIG_EEH
+ if (edev && (edev->mode & EEH_DEV_REMOVED))
+ return NULL;
+#endif
+
/* create a new pci_dev for this device */
dev = of_create_pci_dev(dn, bus, devfn);
if (!dev)
diff --git a/arch/powerpc/kernel/ppc_ksyms.c b/arch/powerpc/kernel/ppc_ksyms.c
index 450850a..48d17d6f 100644
--- a/arch/powerpc/kernel/ppc_ksyms.c
+++ b/arch/powerpc/kernel/ppc_ksyms.c
@@ -155,9 +155,7 @@
#endif
long long __bswapdi2(long long);
EXPORT_SYMBOL(__bswapdi2);
-#ifdef __BIG_ENDIAN__
EXPORT_SYMBOL(memcpy);
-#endif
EXPORT_SYMBOL(memset);
EXPORT_SYMBOL(memmove);
EXPORT_SYMBOL(memcmp);
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 31d0215..8a1edbe2 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -54,6 +54,7 @@
#ifdef CONFIG_PPC64
#include <asm/firmware.h>
#endif
+#include <asm/code-patching.h>
#include <linux/kprobes.h>
#include <linux/kdebug.h>
@@ -495,14 +496,21 @@
return 0;
}
-int set_breakpoint(struct arch_hw_breakpoint *brk)
+void __set_breakpoint(struct arch_hw_breakpoint *brk)
{
__get_cpu_var(current_brk) = *brk;
if (cpu_has_feature(CPU_FTR_DAWR))
- return set_dawr(brk);
+ set_dawr(brk);
+ else
+ set_dabr(brk);
+}
- return set_dabr(brk);
+void set_breakpoint(struct arch_hw_breakpoint *brk)
+{
+ preempt_disable();
+ __set_breakpoint(brk);
+ preempt_enable();
}
#ifdef CONFIG_PPC64
@@ -834,7 +842,7 @@
*/
#ifndef CONFIG_HAVE_HW_BREAKPOINT
if (unlikely(!hw_brk_match(&__get_cpu_var(current_brk), &new->thread.hw_brk)))
- set_breakpoint(&new->thread.hw_brk);
+ __set_breakpoint(&new->thread.hw_brk);
#endif /* CONFIG_HAVE_HW_BREAKPOINT */
#endif
@@ -1108,7 +1116,9 @@
struct thread_info *ti = (void *)task_stack_page(p);
memset(childregs, 0, sizeof(struct pt_regs));
childregs->gpr[1] = sp + sizeof(struct pt_regs);
- childregs->gpr[14] = usp; /* function */
+ /* function */
+ if (usp)
+ childregs->gpr[14] = ppc_function_entry((void *)usp);
#ifdef CONFIG_PPC64
clear_tsk_thread_flag(p, TIF_32BIT);
childregs->softe = 1;
@@ -1187,17 +1197,7 @@
if (cpu_has_feature(CPU_FTR_HAS_PPR))
p->thread.ppr = INIT_PPR;
#endif
- /*
- * The PPC64 ABI makes use of a TOC to contain function
- * pointers. The function (ret_from_except) is actually a pointer
- * to the TOC entry. The first entry is a pointer to the actual
- * function.
- */
-#ifdef CONFIG_PPC64
- kregs->nip = *((unsigned long *)f);
-#else
- kregs->nip = (unsigned long)f;
-#endif
+ kregs->nip = ppc_function_entry(f);
return 0;
}
diff --git a/arch/powerpc/kernel/prom_init_check.sh b/arch/powerpc/kernel/prom_init_check.sh
index b0c263d..77aa1e9 100644
--- a/arch/powerpc/kernel/prom_init_check.sh
+++ b/arch/powerpc/kernel/prom_init_check.sh
@@ -23,7 +23,7 @@
reloc_got2 kernstart_addr memstart_addr linux_banner _stext
opal_query_takeover opal_do_takeover opal_enter_rtas opal_secondary_entry
boot_command_line __prom_init_toc_start __prom_init_toc_end
-btext_setup_display"
+btext_setup_display TOC."
NM="$1"
OBJ="$2"
diff --git a/arch/powerpc/kernel/rtas_pci.c b/arch/powerpc/kernel/rtas_pci.c
index 7d4c717..c168337 100644
--- a/arch/powerpc/kernel/rtas_pci.c
+++ b/arch/powerpc/kernel/rtas_pci.c
@@ -80,10 +80,6 @@
if (ret)
return PCIBIOS_DEVICE_NOT_FOUND;
- if (returnval == EEH_IO_ERROR_VALUE(size) &&
- eeh_dev_check_failure(of_node_to_eeh_dev(pdn->node)))
- return PCIBIOS_DEVICE_NOT_FOUND;
-
return PCIBIOS_SUCCESSFUL;
}
@@ -92,18 +88,39 @@
int where, int size, u32 *val)
{
struct device_node *busdn, *dn;
-
- busdn = pci_bus_to_OF_node(bus);
+ struct pci_dn *pdn;
+ bool found = false;
+#ifdef CONFIG_EEH
+ struct eeh_dev *edev;
+#endif
+ int ret;
/* Search only direct children of the bus */
+ *val = 0xFFFFFFFF;
+ busdn = pci_bus_to_OF_node(bus);
for (dn = busdn->child; dn; dn = dn->sibling) {
- struct pci_dn *pdn = PCI_DN(dn);
+ pdn = PCI_DN(dn);
if (pdn && pdn->devfn == devfn
- && of_device_is_available(dn))
- return rtas_read_config(pdn, where, size, val);
+ && of_device_is_available(dn)) {
+ found = true;
+ break;
+ }
}
- return PCIBIOS_DEVICE_NOT_FOUND;
+ if (!found)
+ return PCIBIOS_DEVICE_NOT_FOUND;
+#ifdef CONFIG_EEH
+ edev = of_node_to_eeh_dev(dn);
+ if (edev && edev->pe && edev->pe->state & EEH_PE_RESET)
+ return PCIBIOS_DEVICE_NOT_FOUND;
+#endif
+
+ ret = rtas_read_config(pdn, where, size, val);
+ if (*val == EEH_IO_ERROR_VALUE(size) &&
+ eeh_dev_check_failure(of_node_to_eeh_dev(dn)))
+ return PCIBIOS_DEVICE_NOT_FOUND;
+
+ return ret;
}
int rtas_write_config(struct pci_dn *pdn, int where, int size, u32 val)
@@ -136,17 +153,34 @@
int where, int size, u32 val)
{
struct device_node *busdn, *dn;
-
- busdn = pci_bus_to_OF_node(bus);
+ struct pci_dn *pdn;
+ bool found = false;
+#ifdef CONFIG_EEH
+ struct eeh_dev *edev;
+#endif
+ int ret;
/* Search only direct children of the bus */
+ busdn = pci_bus_to_OF_node(bus);
for (dn = busdn->child; dn; dn = dn->sibling) {
- struct pci_dn *pdn = PCI_DN(dn);
+ pdn = PCI_DN(dn);
if (pdn && pdn->devfn == devfn
- && of_device_is_available(dn))
- return rtas_write_config(pdn, where, size, val);
+ && of_device_is_available(dn)) {
+ found = true;
+ break;
+ }
}
- return PCIBIOS_DEVICE_NOT_FOUND;
+
+ if (!found)
+ return PCIBIOS_DEVICE_NOT_FOUND;
+#ifdef CONFIG_EEH
+ edev = of_node_to_eeh_dev(dn);
+ if (edev && edev->pe && (edev->pe->state & EEH_PE_RESET))
+ return PCIBIOS_DEVICE_NOT_FOUND;
+#endif
+ ret = rtas_write_config(pdn, where, size, val);
+
+ return ret;
}
static struct pci_ops rtas_pci_ops = {
diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index 79b7612..d4d4183 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -212,6 +212,7 @@
{
unsigned long cpu_id = (unsigned long)v - 1;
unsigned int pvr;
+ unsigned long proc_freq;
unsigned short maj;
unsigned short min;
@@ -263,12 +264,19 @@
#endif /* CONFIG_TAU */
/*
- * Assume here that all clock rates are the same in a
- * smp system. -- Cort
+ * Platforms that have variable clock rates, should implement
+ * the method ppc_md.get_proc_freq() that reports the clock
+ * rate of a given cpu. The rest can use ppc_proc_freq to
+ * report the clock rate that is same across all cpus.
*/
- if (ppc_proc_freq)
+ if (ppc_md.get_proc_freq)
+ proc_freq = ppc_md.get_proc_freq(cpu_id);
+ else
+ proc_freq = ppc_proc_freq;
+
+ if (proc_freq)
seq_printf(m, "clock\t\t: %lu.%06luMHz\n",
- ppc_proc_freq / 1000000, ppc_proc_freq % 1000000);
+ proc_freq / 1000000, proc_freq % 1000000);
if (ppc_md.show_percpuinfo != NULL)
ppc_md.show_percpuinfo(m, cpu_id);
@@ -382,9 +390,10 @@
#ifdef CONFIG_SMP
-int threads_per_core, threads_shift;
+int threads_per_core, threads_per_subcore, threads_shift;
cpumask_t threads_core_mask;
EXPORT_SYMBOL_GPL(threads_per_core);
+EXPORT_SYMBOL_GPL(threads_per_subcore);
EXPORT_SYMBOL_GPL(threads_shift);
EXPORT_SYMBOL_GPL(threads_core_mask);
@@ -393,6 +402,7 @@
int i;
threads_per_core = tpc;
+ threads_per_subcore = tpc;
cpumask_clear(&threads_core_mask);
/* This implementation only supports power of 2 number of threads
@@ -718,33 +728,6 @@
arch_initcall(powerpc_debugfs_init);
#endif
-#ifdef CONFIG_BOOKE_WDT
-extern u32 booke_wdt_enabled;
-extern u32 booke_wdt_period;
-
-/* Checks wdt=x and wdt_period=xx command-line option */
-notrace int __init early_parse_wdt(char *p)
-{
- if (p && strncmp(p, "0", 1) != 0)
- booke_wdt_enabled = 1;
-
- return 0;
-}
-early_param("wdt", early_parse_wdt);
-
-int __init early_parse_wdt_period(char *p)
-{
- unsigned long ret;
- if (p) {
- if (!kstrtol(p, 0, &ret))
- booke_wdt_period = ret;
- }
-
- return 0;
-}
-early_param("wdt_period", early_parse_wdt_period);
-#endif /* CONFIG_BOOKE_WDT */
-
void ppc_printk_progress(char *s, unsigned short hex)
{
pr_info("%s\n", s);
diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
index fbe2437..ee082d7 100644
--- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c
@@ -36,6 +36,7 @@
#include <linux/lockdep.h>
#include <linux/memblock.h>
#include <linux/hugetlb.h>
+#include <linux/memory.h>
#include <asm/io.h>
#include <asm/kdump.h>
@@ -341,7 +342,7 @@
ptr = (unsigned long *)((unsigned long)&__secondary_hold_spinloop
- PHYSICAL_START);
- *ptr = __pa(generic_secondary_smp_init);
+ *ptr = ppc_function_entry(generic_secondary_smp_init);
/* And wait a bit for them to catch up */
for (i = 0; i < 100000; i++) {
@@ -780,6 +781,15 @@
}
#endif
+#ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
+unsigned long memory_block_size_bytes(void)
+{
+ if (ppc_md.memory_block_size)
+ return ppc_md.memory_block_size();
+
+ return MIN_MEMORY_BLOCK_SIZE;
+}
+#endif
#if defined(CONFIG_PPC_INDIRECT_PIO) || defined(CONFIG_PPC_INDIRECT_MMIO)
struct ppc_pci_io ppc_pci_io;
diff --git a/arch/powerpc/kernel/signal.c b/arch/powerpc/kernel/signal.c
index 8fc4177..1c794ce 100644
--- a/arch/powerpc/kernel/signal.c
+++ b/arch/powerpc/kernel/signal.c
@@ -134,7 +134,7 @@
*/
if (current->thread.hw_brk.address &&
current->thread.hw_brk.type)
- set_breakpoint(¤t->thread.hw_brk);
+ __set_breakpoint(¤t->thread.hw_brk);
#endif
/* Re-enable the breakpoints for the signal stack */
thread_change_pc(current, regs);
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 10ffffe..7753af2 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -36,6 +36,7 @@
#include <linux/atomic.h>
#include <asm/irq.h>
#include <asm/hw_irq.h>
+#include <asm/kvm_ppc.h>
#include <asm/page.h>
#include <asm/pgtable.h>
#include <asm/prom.h>
@@ -390,6 +391,7 @@
#ifdef CONFIG_PPC64
paca[boot_cpuid].__current = current;
#endif
+ set_numa_node(numa_cpu_lookup_table[boot_cpuid]);
current_set[boot_cpuid] = task_thread_info(current);
}
@@ -457,38 +459,9 @@
return per_cpu(cpu_state, cpu) == CPU_UP_PREPARE;
}
-static atomic_t secondary_inhibit_count;
-
-/*
- * Don't allow secondary CPU threads to come online
- */
-void inhibit_secondary_onlining(void)
+static bool secondaries_inhibited(void)
{
- /*
- * This makes secondary_inhibit_count stable during cpu
- * online/offline operations.
- */
- get_online_cpus();
-
- atomic_inc(&secondary_inhibit_count);
- put_online_cpus();
-}
-EXPORT_SYMBOL_GPL(inhibit_secondary_onlining);
-
-/*
- * Allow secondary CPU threads to come online again
- */
-void uninhibit_secondary_onlining(void)
-{
- get_online_cpus();
- atomic_dec(&secondary_inhibit_count);
- put_online_cpus();
-}
-EXPORT_SYMBOL_GPL(uninhibit_secondary_onlining);
-
-static int secondaries_inhibited(void)
-{
- return atomic_read(&secondary_inhibit_count);
+ return kvm_hv_mode_active();
}
#else /* HOTPLUG_CPU */
@@ -517,7 +490,7 @@
* Don't allow secondary threads to come online if inhibited
*/
if (threads_per_core > 1 && secondaries_inhibited() &&
- cpu % threads_per_core != 0)
+ cpu_thread_in_subcore(cpu))
return -EBUSY;
if (smp_ops == NULL ||
@@ -750,6 +723,12 @@
}
traverse_core_siblings(cpu, true);
+ /*
+ * numa_node_id() works after this.
+ */
+ set_numa_node(numa_cpu_lookup_table[cpu]);
+ set_numa_mem(local_memory_node(numa_cpu_lookup_table[cpu]));
+
smp_wmb();
notify_cpu_starting(cpu);
set_cpu_online(cpu, true);
diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c
index d90d4b7..67fd2fd 100644
--- a/arch/powerpc/kernel/sysfs.c
+++ b/arch/powerpc/kernel/sysfs.c
@@ -404,7 +404,7 @@
}
EXPORT_SYMBOL(ppc_enable_pmcs);
-#define __SYSFS_SPRSETUP(NAME, ADDRESS, EXTRA) \
+#define __SYSFS_SPRSETUP_READ_WRITE(NAME, ADDRESS, EXTRA) \
static void read_##NAME(void *val) \
{ \
*(unsigned long *)val = mfspr(ADDRESS); \
@@ -413,7 +413,9 @@
{ \
EXTRA; \
mtspr(ADDRESS, *(unsigned long *)val); \
-} \
+}
+
+#define __SYSFS_SPRSETUP_SHOW_STORE(NAME) \
static ssize_t show_##NAME(struct device *dev, \
struct device_attribute *attr, \
char *buf) \
@@ -436,10 +438,15 @@
return count; \
}
-#define SYSFS_PMCSETUP(NAME, ADDRESS) \
- __SYSFS_SPRSETUP(NAME, ADDRESS, ppc_enable_pmcs())
-#define SYSFS_SPRSETUP(NAME, ADDRESS) \
- __SYSFS_SPRSETUP(NAME, ADDRESS, )
+#define SYSFS_PMCSETUP(NAME, ADDRESS) \
+ __SYSFS_SPRSETUP_READ_WRITE(NAME, ADDRESS, ppc_enable_pmcs()) \
+ __SYSFS_SPRSETUP_SHOW_STORE(NAME)
+#define SYSFS_SPRSETUP(NAME, ADDRESS) \
+ __SYSFS_SPRSETUP_READ_WRITE(NAME, ADDRESS, ) \
+ __SYSFS_SPRSETUP_SHOW_STORE(NAME)
+
+#define SYSFS_SPRSETUP_SHOW_STORE(NAME) \
+ __SYSFS_SPRSETUP_SHOW_STORE(NAME)
/* Let's define all possible registers, we'll only hook up the ones
* that are implemented on the current processor
@@ -477,7 +484,6 @@
SYSFS_PMCSETUP(mmcra, SPRN_MMCRA);
SYSFS_SPRSETUP(purr, SPRN_PURR);
SYSFS_SPRSETUP(spurr, SPRN_SPURR);
-SYSFS_SPRSETUP(dscr, SPRN_DSCR);
SYSFS_SPRSETUP(pir, SPRN_PIR);
/*
@@ -487,12 +493,27 @@
*/
static DEVICE_ATTR(mmcra, 0600, show_mmcra, store_mmcra);
static DEVICE_ATTR(spurr, 0400, show_spurr, NULL);
-static DEVICE_ATTR(dscr, 0600, show_dscr, store_dscr);
static DEVICE_ATTR(purr, 0400, show_purr, store_purr);
static DEVICE_ATTR(pir, 0400, show_pir, NULL);
-unsigned long dscr_default = 0;
-EXPORT_SYMBOL(dscr_default);
+static unsigned long dscr_default;
+
+static void read_dscr(void *val)
+{
+ *(unsigned long *)val = get_paca()->dscr_default;
+}
+
+static void write_dscr(void *val)
+{
+ get_paca()->dscr_default = *(unsigned long *)val;
+ if (!current->thread.dscr_inherit) {
+ current->thread.dscr = *(unsigned long *)val;
+ mtspr(SPRN_DSCR, *(unsigned long *)val);
+ }
+}
+
+SYSFS_SPRSETUP_SHOW_STORE(dscr);
+static DEVICE_ATTR(dscr, 0600, show_dscr, store_dscr);
static void add_write_permission_dev_attr(struct device_attribute *attr)
{
@@ -505,14 +526,6 @@
return sprintf(buf, "%lx\n", dscr_default);
}
-static void update_dscr(void *dummy)
-{
- if (!current->thread.dscr_inherit) {
- current->thread.dscr = dscr_default;
- mtspr(SPRN_DSCR, dscr_default);
- }
-}
-
static ssize_t __used store_dscr_default(struct device *dev,
struct device_attribute *attr, const char *buf,
size_t count)
@@ -525,7 +538,7 @@
return -EINVAL;
dscr_default = val;
- on_each_cpu(update_dscr, NULL, 1);
+ on_each_cpu(write_dscr, &val, 1);
return count;
}
diff --git a/arch/powerpc/kernel/systbl.S b/arch/powerpc/kernel/systbl.S
index 93219c3..895c50c 100644
--- a/arch/powerpc/kernel/systbl.S
+++ b/arch/powerpc/kernel/systbl.S
@@ -17,12 +17,12 @@
#include <asm/ppc_asm.h>
#ifdef CONFIG_PPC64
-#define SYSCALL(func) .llong .sys_##func,.sys_##func
-#define COMPAT_SYS(func) .llong .sys_##func,.compat_sys_##func
-#define PPC_SYS(func) .llong .ppc_##func,.ppc_##func
-#define OLDSYS(func) .llong .sys_ni_syscall,.sys_ni_syscall
-#define SYS32ONLY(func) .llong .sys_ni_syscall,.compat_sys_##func
-#define SYSX(f, f3264, f32) .llong .f,.f3264
+#define SYSCALL(func) .llong DOTSYM(sys_##func),DOTSYM(sys_##func)
+#define COMPAT_SYS(func) .llong DOTSYM(sys_##func),DOTSYM(compat_sys_##func)
+#define PPC_SYS(func) .llong DOTSYM(ppc_##func),DOTSYM(ppc_##func)
+#define OLDSYS(func) .llong DOTSYM(sys_ni_syscall),DOTSYM(sys_ni_syscall)
+#define SYS32ONLY(func) .llong DOTSYM(sys_ni_syscall),DOTSYM(compat_sys_##func)
+#define SYSX(f, f3264, f32) .llong DOTSYM(f),DOTSYM(f3264)
#else
#define SYSCALL(func) .long sys_##func
#define COMPAT_SYS(func) .long sys_##func
@@ -36,6 +36,8 @@
#define PPC_SYS_SPU(func) PPC_SYS(func)
#define SYSX_SPU(f, f3264, f32) SYSX(f, f3264, f32)
+.section .rodata,"a"
+
#ifdef CONFIG_PPC64
#define sys_sigpending sys_ni_syscall
#define sys_old_getrlimit sys_ni_syscall
@@ -43,5 +45,7 @@
.p2align 3
#endif
-_GLOBAL(sys_call_table)
+.globl sys_call_table
+sys_call_table:
+
#include <asm/systbl.h>
diff --git a/arch/powerpc/kernel/tm.S b/arch/powerpc/kernel/tm.S
index 03567c0..2a324f4 100644
--- a/arch/powerpc/kernel/tm.S
+++ b/arch/powerpc/kernel/tm.S
@@ -10,6 +10,7 @@
#include <asm/ppc-opcode.h>
#include <asm/ptrace.h>
#include <asm/reg.h>
+#include <asm/bug.h>
#ifdef CONFIG_VSX
/* See fpu.S, this is borrowed from there */
@@ -41,7 +42,6 @@
/* Stack frame offsets for local variables. */
#define TM_FRAME_L0 TM_FRAME_SIZE-16
#define TM_FRAME_L1 TM_FRAME_SIZE-8
-#define STACK_PARAM(x) (48+((x)*8))
/* In order to access the TM SPRs, TM must be enabled. So, do so: */
@@ -78,12 +78,6 @@
TABORT(R3)
blr
- .section ".toc","aw"
-DSCR_DEFAULT:
- .tc dscr_default[TC],dscr_default
-
- .section ".text"
-
/* void tm_reclaim(struct thread_struct *thread,
* unsigned long orig_msr,
* uint8_t cause)
@@ -108,12 +102,12 @@
mflr r0
stw r6, 8(r1)
std r0, 16(r1)
- std r2, 40(r1)
+ std r2, STK_GOT(r1)
stdu r1, -TM_FRAME_SIZE(r1)
/* We've a struct pt_regs at [r1+STACK_FRAME_OVERHEAD]. */
- std r3, STACK_PARAM(0)(r1)
+ std r3, STK_PARAM(R3)(r1)
SAVE_NVGPRS(r1)
/* We need to setup MSR for VSX register save instructions. Here we
@@ -175,6 +169,13 @@
stfd fr0,FPSTATE_FPSCR(r7)
dont_backup_fp:
+ /* Do sanity check on MSR to make sure we are suspended */
+ li r7, (MSR_TS_S)@higher
+ srdi r6, r14, 32
+ and r6, r6, r7
+1: tdeqi r6, 0
+ EMIT_BUG_ENTRY 1b,__FILE__,__LINE__,0
+
/* The moment we treclaim, ALL of our GPRs will switch
* to user register state. (FPRs, CCR etc. also!)
* Use an sprg and a tm_scratch in the PACA to shuffle.
@@ -202,7 +203,7 @@
/* Now get some more GPRS free */
std r7, GPR7(r1) /* Temporary stash */
std r12, GPR12(r1) /* '' '' '' */
- ld r12, STACK_PARAM(0)(r1) /* Param 0, thread_struct * */
+ ld r12, STK_PARAM(R3)(r1) /* Param 0, thread_struct * */
std r11, THREAD_TM_PPR(r12) /* Store PPR and free r11 */
@@ -289,11 +290,10 @@
ld r0, 16(r1)
mtcr r4
mtlr r0
- ld r2, 40(r1)
+ ld r2, STK_GOT(r1)
- /* Load system default DSCR */
- ld r4, DSCR_DEFAULT@toc(r2)
- ld r0, 0(r4)
+ /* Load CPU's default DSCR */
+ ld r0, PACA_DSCR(r13)
mtspr SPRN_DSCR, r0
blr
@@ -312,7 +312,7 @@
mflr r0
stw r5, 8(r1)
std r0, 16(r1)
- std r2, 40(r1)
+ std r2, STK_GOT(r1)
stdu r1, -TM_FRAME_SIZE(r1)
/* We've a struct pt_regs at [r1+STACK_FRAME_OVERHEAD].
@@ -320,8 +320,6 @@
*/
SAVE_NVGPRS(r1)
- std r1, PACAR1(r13)
-
/* Load complete register state from ts_ckpt* registers */
addi r7, r3, PT_CKPT_REGS /* Thread's ckpt_regs */
@@ -385,12 +383,10 @@
/* ******************** CR,LR,CCR,MSR ********** */
ld r4, _CTR(r7)
ld r5, _LINK(r7)
- ld r6, _CCR(r7)
ld r8, _XER(r7)
mtctr r4
mtlr r5
- mtcr r6
mtxer r8
/* ******************** TAR ******************** */
@@ -406,7 +402,8 @@
li r4, 0
mtmsrd r4, 1
- REST_4GPRS(0, r7) /* GPR0-3 */
+ REST_GPR(0, r7) /* GPR0 */
+ REST_2GPRS(2, r7) /* GPR2-3 */
REST_GPR(4, r7) /* GPR4 */
REST_4GPRS(8, r7) /* GPR8-11 */
REST_2GPRS(12, r7) /* GPR12-13 */
@@ -418,6 +415,31 @@
mtspr SPRN_DSCR, r5
mtspr SPRN_PPR, r6
+ /* Do final sanity check on TEXASR to make sure FS is set. Do this
+ * here before we load up the userspace r1 so any bugs we hit will get
+ * a call chain */
+ mfspr r5, SPRN_TEXASR
+ srdi r5, r5, 16
+ li r6, (TEXASR_FS)@h
+ and r6, r6, r5
+1: tdeqi r6, 0
+ EMIT_BUG_ENTRY 1b,__FILE__,__LINE__,0
+
+ /* Do final sanity check on MSR to make sure we are not transactional
+ * or suspended
+ */
+ mfmsr r6
+ li r5, (MSR_TS_MASK)@higher
+ srdi r6, r6, 32
+ and r6, r6, r5
+1: tdnei r6, 0
+ EMIT_BUG_ENTRY 1b,__FILE__,__LINE__,0
+
+ /* Restore CR */
+ ld r6, _CCR(r7)
+ mtcr r6
+
+ REST_GPR(1, r7) /* GPR1 */
REST_GPR(5, r7) /* GPR5-7 */
REST_GPR(6, r7)
ld r7, GPR7(r7)
@@ -448,11 +470,10 @@
ld r0, 16(r1)
mtcr r4
mtlr r0
- ld r2, 40(r1)
+ ld r2, STK_GOT(r1)
- /* Load system default DSCR */
- ld r4, DSCR_DEFAULT@toc(r2)
- ld r0, 0(r4)
+ /* Load CPU's default DSCR */
+ ld r0, PACA_DSCR(r13)
mtspr SPRN_DSCR, r0
blr
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index aba05bb..7a12edb 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -1236,7 +1236,7 @@
int core;
struct kvmppc_vcore *vcore;
- core = id / threads_per_core;
+ core = id / threads_per_subcore;
if (core >= KVM_MAX_VCORES)
goto out;
@@ -1286,7 +1286,7 @@
init_waitqueue_head(&vcore->wq);
vcore->preempt_tb = TB_NIL;
vcore->lpcr = kvm->arch.lpcr;
- vcore->first_vcpuid = core * threads_per_core;
+ vcore->first_vcpuid = core * threads_per_subcore;
vcore->kvm = kvm;
}
kvm->arch.vcores[core] = vcore;
@@ -1476,16 +1476,19 @@
static int on_primary_thread(void)
{
int cpu = smp_processor_id();
- int thr = cpu_thread_in_core(cpu);
+ int thr;
- if (thr)
+ /* Are we on a primary subcore? */
+ if (cpu_thread_in_subcore(cpu))
return 0;
- while (++thr < threads_per_core)
+
+ thr = 0;
+ while (++thr < threads_per_subcore)
if (cpu_online(cpu + thr))
return 0;
/* Grab all hw threads so they can't go into the kernel */
- for (thr = 1; thr < threads_per_core; ++thr) {
+ for (thr = 1; thr < threads_per_subcore; ++thr) {
if (kvmppc_grab_hwthread(cpu + thr)) {
/* Couldn't grab one; let the others go */
do {
@@ -1544,15 +1547,18 @@
}
/*
- * Make sure we are running on thread 0, and that
- * secondary threads are offline.
+ * Make sure we are running on primary threads, and that secondary
+ * threads are offline. Also check if the number of threads in this
+ * guest are greater than the current system threads per guest.
*/
- if (threads_per_core > 1 && !on_primary_thread()) {
+ if ((threads_per_core > 1) &&
+ ((vc->num_threads > threads_per_subcore) || !on_primary_thread())) {
list_for_each_entry(vcpu, &vc->runnable_threads, arch.run_list)
vcpu->arch.ret = -EBUSY;
goto out;
}
+
vc->pcpu = smp_processor_id();
list_for_each_entry(vcpu, &vc->runnable_threads, arch.run_list) {
kvmppc_start_thread(vcpu);
@@ -1580,7 +1586,7 @@
/* wait for secondary threads to finish writing their state to memory */
if (vc->nap_count < vc->n_woken)
kvmppc_wait_for_nap(vc);
- for (i = 0; i < threads_per_core; ++i)
+ for (i = 0; i < threads_per_subcore; ++i)
kvmppc_release_hwthread(vc->pcpu + i);
/* prevent other vcpu threads from doing kvmppc_start_thread() now */
vc->vcore_state = VCORE_EXITING;
@@ -2305,10 +2311,10 @@
spin_lock_init(&kvm->arch.slot_phys_lock);
/*
- * Don't allow secondary CPU threads to come online
- * while any KVM VMs exist.
+ * Track that we now have a HV mode VM active. This blocks secondary
+ * CPU threads from coming online.
*/
- inhibit_secondary_onlining();
+ kvm_hv_vm_activated();
return 0;
}
@@ -2324,7 +2330,7 @@
static void kvmppc_core_destroy_vm_hv(struct kvm *kvm)
{
- uninhibit_secondary_onlining();
+ kvm_hv_vm_deactivated();
kvmppc_free_vcores(kvm);
if (kvm->arch.rma) {
diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c
index 8cd0dae..7cde8a6 100644
--- a/arch/powerpc/kvm/book3s_hv_builtin.c
+++ b/arch/powerpc/kvm/book3s_hv_builtin.c
@@ -6,6 +6,7 @@
* published by the Free Software Foundation.
*/
+#include <linux/cpu.h>
#include <linux/kvm_host.h>
#include <linux/preempt.h>
#include <linux/export.h>
@@ -181,3 +182,33 @@
kvm_cma_declare_contiguous(selected_size, align_size);
}
}
+
+/*
+ * When running HV mode KVM we need to block certain operations while KVM VMs
+ * exist in the system. We use a counter of VMs to track this.
+ *
+ * One of the operations we need to block is onlining of secondaries, so we
+ * protect hv_vm_count with get/put_online_cpus().
+ */
+static atomic_t hv_vm_count;
+
+void kvm_hv_vm_activated(void)
+{
+ get_online_cpus();
+ atomic_inc(&hv_vm_count);
+ put_online_cpus();
+}
+EXPORT_SYMBOL_GPL(kvm_hv_vm_activated);
+
+void kvm_hv_vm_deactivated(void)
+{
+ get_online_cpus();
+ atomic_dec(&hv_vm_count);
+ put_online_cpus();
+}
+EXPORT_SYMBOL_GPL(kvm_hv_vm_deactivated);
+
+bool kvm_hv_mode_active(void)
+{
+ return atomic_read(&hv_vm_count) != 0;
+}
diff --git a/arch/powerpc/kvm/book3s_hv_interrupts.S b/arch/powerpc/kvm/book3s_hv_interrupts.S
index e18e3cf..8c86422 100644
--- a/arch/powerpc/kvm/book3s_hv_interrupts.S
+++ b/arch/powerpc/kvm/book3s_hv_interrupts.S
@@ -171,7 +171,7 @@
#endif /* CONFIG_SMP */
/* Jump to partition switch code */
- bl .kvmppc_hv_entry_trampoline
+ bl kvmppc_hv_entry_trampoline
nop
/*
diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index 9747934..77356fd 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -292,8 +292,7 @@
beq kvm_no_guest
/* Set HSTATE_DSCR(r13) to something sensible */
- LOAD_REG_ADDR(r6, dscr_default)
- ld r6, 0(r6)
+ ld r6, PACA_DSCR(r13)
std r6, HSTATE_DSCR(r13)
bl kvmppc_hv_entry
@@ -1799,7 +1798,7 @@
/* Search the hash table. */
mr r3, r9 /* vcpu pointer */
li r7, 1 /* data fault */
- bl .kvmppc_hpte_hv_fault
+ bl kvmppc_hpte_hv_fault
ld r9, HSTATE_KVM_VCPU(r13)
ld r10, VCPU_PC(r9)
ld r11, VCPU_MSR(r9)
@@ -1873,7 +1872,7 @@
mr r4, r10
mr r6, r11
li r7, 0 /* instruction fault */
- bl .kvmppc_hpte_hv_fault
+ bl kvmppc_hpte_hv_fault
ld r9, HSTATE_KVM_VCPU(r13)
ld r10, VCPU_PC(r9)
ld r11, VCPU_MSR(r9)
@@ -1947,16 +1946,16 @@
.globl hcall_real_table
hcall_real_table:
.long 0 /* 0 - unused */
- .long .kvmppc_h_remove - hcall_real_table
- .long .kvmppc_h_enter - hcall_real_table
- .long .kvmppc_h_read - hcall_real_table
+ .long DOTSYM(kvmppc_h_remove) - hcall_real_table
+ .long DOTSYM(kvmppc_h_enter) - hcall_real_table
+ .long DOTSYM(kvmppc_h_read) - hcall_real_table
.long 0 /* 0x10 - H_CLEAR_MOD */
.long 0 /* 0x14 - H_CLEAR_REF */
- .long .kvmppc_h_protect - hcall_real_table
- .long .kvmppc_h_get_tce - hcall_real_table
- .long .kvmppc_h_put_tce - hcall_real_table
+ .long DOTSYM(kvmppc_h_protect) - hcall_real_table
+ .long DOTSYM(kvmppc_h_get_tce) - hcall_real_table
+ .long DOTSYM(kvmppc_h_put_tce) - hcall_real_table
.long 0 /* 0x24 - H_SET_SPRG0 */
- .long .kvmppc_h_set_dabr - hcall_real_table
+ .long DOTSYM(kvmppc_h_set_dabr) - hcall_real_table
.long 0 /* 0x2c */
.long 0 /* 0x30 */
.long 0 /* 0x34 */
@@ -1972,11 +1971,11 @@
.long 0 /* 0x5c */
.long 0 /* 0x60 */
#ifdef CONFIG_KVM_XICS
- .long .kvmppc_rm_h_eoi - hcall_real_table
- .long .kvmppc_rm_h_cppr - hcall_real_table
- .long .kvmppc_rm_h_ipi - hcall_real_table
+ .long DOTSYM(kvmppc_rm_h_eoi) - hcall_real_table
+ .long DOTSYM(kvmppc_rm_h_cppr) - hcall_real_table
+ .long DOTSYM(kvmppc_rm_h_ipi) - hcall_real_table
.long 0 /* 0x70 - H_IPOLL */
- .long .kvmppc_rm_h_xirr - hcall_real_table
+ .long DOTSYM(kvmppc_rm_h_xirr) - hcall_real_table
#else
.long 0 /* 0x64 - H_EOI */
.long 0 /* 0x68 - H_CPPR */
@@ -2010,7 +2009,7 @@
.long 0 /* 0xd4 */
.long 0 /* 0xd8 */
.long 0 /* 0xdc */
- .long .kvmppc_h_cede - hcall_real_table
+ .long DOTSYM(kvmppc_h_cede) - hcall_real_table
.long 0 /* 0xe4 */
.long 0 /* 0xe8 */
.long 0 /* 0xec */
@@ -2027,11 +2026,11 @@
.long 0 /* 0x118 */
.long 0 /* 0x11c */
.long 0 /* 0x120 */
- .long .kvmppc_h_bulk_remove - hcall_real_table
+ .long DOTSYM(kvmppc_h_bulk_remove) - hcall_real_table
.long 0 /* 0x128 */
.long 0 /* 0x12c */
.long 0 /* 0x130 */
- .long .kvmppc_h_set_xdabr - hcall_real_table
+ .long DOTSYM(kvmppc_h_set_xdabr) - hcall_real_table
hcall_real_table_end:
ignore_hdec:
@@ -2256,7 +2255,7 @@
/* Try to handle a machine check in real mode */
machine_check_realmode:
mr r3, r9 /* get vcpu pointer */
- bl .kvmppc_realmode_machine_check
+ bl kvmppc_realmode_machine_check
nop
cmpdi r3, 0 /* continue exiting from guest? */
ld r9, HSTATE_KVM_VCPU(r13)
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index bab20f4..61c738a 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -426,7 +426,7 @@
#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
case KVM_CAP_PPC_SMT:
if (hv_enabled)
- r = threads_per_core;
+ r = threads_per_subcore;
else
r = 0;
break;
diff --git a/arch/powerpc/lib/Makefile b/arch/powerpc/lib/Makefile
index 95a20e1..59fa2de 100644
--- a/arch/powerpc/lib/Makefile
+++ b/arch/powerpc/lib/Makefile
@@ -23,9 +23,7 @@
obj-$(CONFIG_PPC64) += checksum_wrappers_64.o
endif
-ifeq ($(CONFIG_CPU_LITTLE_ENDIAN),)
obj-$(CONFIG_PPC64) += memcpy_power7.o memcpy_64.o
-endif
obj-$(CONFIG_PPC_EMULATE_SSTEP) += sstep.o ldstfp.o
diff --git a/arch/powerpc/lib/copypage_64.S b/arch/powerpc/lib/copypage_64.S
index 9f9434a..a3c4dc4 100644
--- a/arch/powerpc/lib/copypage_64.S
+++ b/arch/powerpc/lib/copypage_64.S
@@ -16,11 +16,11 @@
.tc ppc64_caches[TC],ppc64_caches
.section ".text"
-_GLOBAL(copy_page)
+_GLOBAL_TOC(copy_page)
BEGIN_FTR_SECTION
lis r5,PAGE_SIZE@h
FTR_SECTION_ELSE
- b .copypage_power7
+ b copypage_power7
ALT_FTR_SECTION_END_IFCLR(CPU_FTR_VMX_COPY)
ori r5,r5,PAGE_SIZE@l
BEGIN_FTR_SECTION
diff --git a/arch/powerpc/lib/copypage_power7.S b/arch/powerpc/lib/copypage_power7.S
index 395c594..d7dafb3 100644
--- a/arch/powerpc/lib/copypage_power7.S
+++ b/arch/powerpc/lib/copypage_power7.S
@@ -56,15 +56,15 @@
#ifdef CONFIG_ALTIVEC
mflr r0
- std r3,48(r1)
- std r4,56(r1)
+ std r3,-STACKFRAMESIZE+STK_REG(R31)(r1)
+ std r4,-STACKFRAMESIZE+STK_REG(R30)(r1)
std r0,16(r1)
stdu r1,-STACKFRAMESIZE(r1)
- bl .enter_vmx_copy
+ bl enter_vmx_copy
cmpwi r3,0
ld r0,STACKFRAMESIZE+16(r1)
- ld r3,STACKFRAMESIZE+48(r1)
- ld r4,STACKFRAMESIZE+56(r1)
+ ld r3,STK_REG(R31)(r1)
+ ld r4,STK_REG(R30)(r1)
mtlr r0
li r0,(PAGE_SIZE/128)
@@ -103,7 +103,7 @@
addi r3,r3,128
bdnz 1b
- b .exit_vmx_copy /* tail call optimise */
+ b exit_vmx_copy /* tail call optimise */
#else
li r0,(PAGE_SIZE/128)
diff --git a/arch/powerpc/lib/copyuser_64.S b/arch/powerpc/lib/copyuser_64.S
index 596a285..0860ee4 100644
--- a/arch/powerpc/lib/copyuser_64.S
+++ b/arch/powerpc/lib/copyuser_64.S
@@ -18,7 +18,7 @@
#endif
.align 7
-_GLOBAL(__copy_tofrom_user)
+_GLOBAL_TOC(__copy_tofrom_user)
BEGIN_FTR_SECTION
nop
FTR_SECTION_ELSE
diff --git a/arch/powerpc/lib/copyuser_power7.S b/arch/powerpc/lib/copyuser_power7.S
index e8e9c36..c46c876 100644
--- a/arch/powerpc/lib/copyuser_power7.S
+++ b/arch/powerpc/lib/copyuser_power7.S
@@ -66,7 +66,7 @@
ld r15,STK_REG(R15)(r1)
ld r14,STK_REG(R14)(r1)
.Ldo_err3:
- bl .exit_vmx_usercopy
+ bl exit_vmx_usercopy
ld r0,STACKFRAMESIZE+16(r1)
mtlr r0
b .Lexit
@@ -85,9 +85,9 @@
.Lexit:
addi r1,r1,STACKFRAMESIZE
.Ldo_err1:
- ld r3,48(r1)
- ld r4,56(r1)
- ld r5,64(r1)
+ ld r3,-STACKFRAMESIZE+STK_REG(R31)(r1)
+ ld r4,-STACKFRAMESIZE+STK_REG(R30)(r1)
+ ld r5,-STACKFRAMESIZE+STK_REG(R29)(r1)
b __copy_tofrom_user_base
@@ -96,18 +96,18 @@
cmpldi r5,16
cmpldi cr1,r5,4096
- std r3,48(r1)
- std r4,56(r1)
- std r5,64(r1)
+ std r3,-STACKFRAMESIZE+STK_REG(R31)(r1)
+ std r4,-STACKFRAMESIZE+STK_REG(R30)(r1)
+ std r5,-STACKFRAMESIZE+STK_REG(R29)(r1)
blt .Lshort_copy
bgt cr1,.Lvmx_copy
#else
cmpldi r5,16
- std r3,48(r1)
- std r4,56(r1)
- std r5,64(r1)
+ std r3,-STACKFRAMESIZE+STK_REG(R31)(r1)
+ std r4,-STACKFRAMESIZE+STK_REG(R30)(r1)
+ std r5,-STACKFRAMESIZE+STK_REG(R29)(r1)
blt .Lshort_copy
#endif
@@ -295,12 +295,12 @@
mflr r0
std r0,16(r1)
stdu r1,-STACKFRAMESIZE(r1)
- bl .enter_vmx_usercopy
+ bl enter_vmx_usercopy
cmpwi cr1,r3,0
ld r0,STACKFRAMESIZE+16(r1)
- ld r3,STACKFRAMESIZE+48(r1)
- ld r4,STACKFRAMESIZE+56(r1)
- ld r5,STACKFRAMESIZE+64(r1)
+ ld r3,STK_REG(R31)(r1)
+ ld r4,STK_REG(R30)(r1)
+ ld r5,STK_REG(R29)(r1)
mtlr r0
/*
@@ -514,7 +514,7 @@
err3; stb r0,0(r3)
15: addi r1,r1,STACKFRAMESIZE
- b .exit_vmx_usercopy /* tail call optimise */
+ b exit_vmx_usercopy /* tail call optimise */
.Lvmx_unaligned_copy:
/* Get the destination 16B aligned */
@@ -717,5 +717,5 @@
err3; stb r0,0(r3)
15: addi r1,r1,STACKFRAMESIZE
- b .exit_vmx_usercopy /* tail call optimise */
+ b exit_vmx_usercopy /* tail call optimise */
#endif /* CONFiG_ALTIVEC */
diff --git a/arch/powerpc/lib/hweight_64.S b/arch/powerpc/lib/hweight_64.S
index 9b96ff2..19e6600 100644
--- a/arch/powerpc/lib/hweight_64.S
+++ b/arch/powerpc/lib/hweight_64.S
@@ -24,7 +24,7 @@
_GLOBAL(__arch_hweight8)
BEGIN_FTR_SECTION
- b .__sw_hweight8
+ b __sw_hweight8
nop
nop
FTR_SECTION_ELSE
@@ -35,7 +35,7 @@
_GLOBAL(__arch_hweight16)
BEGIN_FTR_SECTION
- b .__sw_hweight16
+ b __sw_hweight16
nop
nop
nop
@@ -57,7 +57,7 @@
_GLOBAL(__arch_hweight32)
BEGIN_FTR_SECTION
- b .__sw_hweight32
+ b __sw_hweight32
nop
nop
nop
@@ -82,7 +82,7 @@
_GLOBAL(__arch_hweight64)
BEGIN_FTR_SECTION
- b .__sw_hweight64
+ b __sw_hweight64
nop
nop
nop
diff --git a/arch/powerpc/lib/mem_64.S b/arch/powerpc/lib/mem_64.S
index f4fcb0b..0738f96 100644
--- a/arch/powerpc/lib/mem_64.S
+++ b/arch/powerpc/lib/mem_64.S
@@ -79,8 +79,8 @@
_GLOBAL(memmove)
cmplw 0,r3,r4
- bgt .backwards_memcpy
- b .memcpy
+ bgt backwards_memcpy
+ b memcpy
_GLOBAL(backwards_memcpy)
rlwinm. r7,r5,32-3,3,31 /* r0 = r5 >> 3 */
diff --git a/arch/powerpc/lib/memcpy_64.S b/arch/powerpc/lib/memcpy_64.S
index 72ad055..32a06ec 100644
--- a/arch/powerpc/lib/memcpy_64.S
+++ b/arch/powerpc/lib/memcpy_64.S
@@ -10,14 +10,29 @@
#include <asm/ppc_asm.h>
.align 7
-_GLOBAL(memcpy)
+_GLOBAL_TOC(memcpy)
BEGIN_FTR_SECTION
- std r3,48(r1) /* save destination pointer for return value */
+#ifdef __LITTLE_ENDIAN__
+ cmpdi cr7,r5,0
+#else
+ std r3,-STACKFRAMESIZE+STK_REG(R31)(r1) /* save destination pointer for return value */
+#endif
FTR_SECTION_ELSE
#ifndef SELFTEST
b memcpy_power7
#endif
ALT_FTR_SECTION_END_IFCLR(CPU_FTR_VMX_COPY)
+#ifdef __LITTLE_ENDIAN__
+ /* dumb little-endian memcpy that will get replaced at runtime */
+ addi r9,r3,-1
+ addi r4,r4,-1
+ beqlr cr7
+ mtctr r5
+1: lbzu r10,1(r4)
+ stbu r10,1(r9)
+ bdnz 1b
+ blr
+#else
PPC_MTOCRF(0x01,r5)
cmpldi cr1,r5,16
neg r6,r3 # LS 3 bits = # bytes to 8-byte dest bdry
@@ -73,7 +88,7 @@
2: bf cr7*4+3,3f
lbz r9,8(r4)
stb r9,0(r3)
-3: ld r3,48(r1) /* return dest pointer */
+3: ld r3,-STACKFRAMESIZE+STK_REG(R31)(r1) /* return dest pointer */
blr
.Lsrc_unaligned:
@@ -156,7 +171,7 @@
2: bf cr7*4+3,3f
rotldi r9,r9,8
stb r9,0(r3)
-3: ld r3,48(r1) /* return dest pointer */
+3: ld r3,-STACKFRAMESIZE+STK_REG(R31)(r1) /* return dest pointer */
blr
.Ldst_unaligned:
@@ -201,5 +216,6 @@
3: bf cr7*4+3,4f
lbz r0,0(r4)
stb r0,0(r3)
-4: ld r3,48(r1) /* return dest pointer */
+4: ld r3,-STACKFRAMESIZE+STK_REG(R31)(r1) /* return dest pointer */
blr
+#endif
diff --git a/arch/powerpc/lib/memcpy_power7.S b/arch/powerpc/lib/memcpy_power7.S
index e4177db..2ff5c14 100644
--- a/arch/powerpc/lib/memcpy_power7.S
+++ b/arch/powerpc/lib/memcpy_power7.S
@@ -33,14 +33,14 @@
cmpldi r5,16
cmpldi cr1,r5,4096
- std r3,48(r1)
+ std r3,-STACKFRAMESIZE+STK_REG(R31)(r1)
blt .Lshort_copy
bgt cr1,.Lvmx_copy
#else
cmpldi r5,16
- std r3,48(r1)
+ std r3,-STACKFRAMESIZE+STK_REG(R31)(r1)
blt .Lshort_copy
#endif
@@ -216,7 +216,7 @@
lbz r0,0(r4)
stb r0,0(r3)
-15: ld r3,48(r1)
+15: ld r3,-STACKFRAMESIZE+STK_REG(R31)(r1)
blr
.Lunwind_stack_nonvmx_copy:
@@ -226,16 +226,16 @@
#ifdef CONFIG_ALTIVEC
.Lvmx_copy:
mflr r0
- std r4,56(r1)
- std r5,64(r1)
+ std r4,-STACKFRAMESIZE+STK_REG(R30)(r1)
+ std r5,-STACKFRAMESIZE+STK_REG(R29)(r1)
std r0,16(r1)
stdu r1,-STACKFRAMESIZE(r1)
- bl .enter_vmx_copy
+ bl enter_vmx_copy
cmpwi cr1,r3,0
ld r0,STACKFRAMESIZE+16(r1)
- ld r3,STACKFRAMESIZE+48(r1)
- ld r4,STACKFRAMESIZE+56(r1)
- ld r5,STACKFRAMESIZE+64(r1)
+ ld r3,STK_REG(R31)(r1)
+ ld r4,STK_REG(R30)(r1)
+ ld r5,STK_REG(R29)(r1)
mtlr r0
/*
@@ -447,8 +447,8 @@
stb r0,0(r3)
15: addi r1,r1,STACKFRAMESIZE
- ld r3,48(r1)
- b .exit_vmx_copy /* tail call optimise */
+ ld r3,-STACKFRAMESIZE+STK_REG(R31)(r1)
+ b exit_vmx_copy /* tail call optimise */
.Lvmx_unaligned_copy:
/* Get the destination 16B aligned */
@@ -651,6 +651,6 @@
stb r0,0(r3)
15: addi r1,r1,STACKFRAMESIZE
- ld r3,48(r1)
- b .exit_vmx_copy /* tail call optimise */
+ ld r3,-STACKFRAMESIZE+STK_REG(R31)(r1)
+ b exit_vmx_copy /* tail call optimise */
#endif /* CONFiG_ALTIVEC */
diff --git a/arch/powerpc/lib/string_64.S b/arch/powerpc/lib/string_64.S
index 3b1e480..7bd9549 100644
--- a/arch/powerpc/lib/string_64.S
+++ b/arch/powerpc/lib/string_64.S
@@ -77,7 +77,7 @@
mr r3,r4
blr
-_GLOBAL(__clear_user)
+_GLOBAL_TOC(__clear_user)
cmpdi r4,32
neg r6,r3
li r0,0
diff --git a/arch/powerpc/mm/hash_low_64.S b/arch/powerpc/mm/hash_low_64.S
index 1136d26..057cbbb 100644
--- a/arch/powerpc/mm/hash_low_64.S
+++ b/arch/powerpc/mm/hash_low_64.S
@@ -159,7 +159,7 @@
BEGIN_FTR_SECTION
mr r4,r30
mr r5,r7
- bl .hash_page_do_lazy_icache
+ bl hash_page_do_lazy_icache
END_FTR_SECTION(CPU_FTR_NOEXECUTE|CPU_FTR_COHERENT_ICACHE, CPU_FTR_NOEXECUTE)
/* At this point, r3 contains new PP bits, save them in
@@ -201,7 +201,8 @@
li r8,MMU_PAGE_4K /* page size */
li r9,MMU_PAGE_4K /* actual page size */
ld r10,STK_PARAM(R9)(r1) /* segment size */
-_GLOBAL(htab_call_hpte_insert1)
+.globl htab_call_hpte_insert1
+htab_call_hpte_insert1:
bl . /* Patched by htab_finish_init() */
cmpdi 0,r3,0
bge htab_pte_insert_ok /* Insertion successful */
@@ -225,7 +226,8 @@
li r8,MMU_PAGE_4K /* page size */
li r9,MMU_PAGE_4K /* actual page size */
ld r10,STK_PARAM(R9)(r1) /* segment size */
-_GLOBAL(htab_call_hpte_insert2)
+.globl htab_call_hpte_insert2
+htab_call_hpte_insert2:
bl . /* Patched by htab_finish_init() */
cmpdi 0,r3,0
bge+ htab_pte_insert_ok /* Insertion successful */
@@ -242,7 +244,8 @@
2: and r0,r5,r27
rldicr r3,r0,3,63-3 /* r0 = (hash & mask) << 3 */
/* Call ppc_md.hpte_remove */
-_GLOBAL(htab_call_hpte_remove)
+.globl htab_call_hpte_remove
+htab_call_hpte_remove:
bl . /* Patched by htab_finish_init() */
/* Try all again */
@@ -296,7 +299,8 @@
li r7,MMU_PAGE_4K /* actual page size */
ld r8,STK_PARAM(R9)(r1) /* segment size */
ld r9,STK_PARAM(R8)(r1) /* get "local" param */
-_GLOBAL(htab_call_hpte_updatepp)
+.globl htab_call_hpte_updatepp
+htab_call_hpte_updatepp:
bl . /* Patched by htab_finish_init() */
/* if we failed because typically the HPTE wasn't really here
@@ -471,7 +475,7 @@
BEGIN_FTR_SECTION
mr r4,r30
mr r5,r7
- bl .hash_page_do_lazy_icache
+ bl hash_page_do_lazy_icache
END_FTR_SECTION(CPU_FTR_NOEXECUTE|CPU_FTR_COHERENT_ICACHE, CPU_FTR_NOEXECUTE)
/* At this point, r3 contains new PP bits, save them in
@@ -526,7 +530,8 @@
li r8,MMU_PAGE_4K /* page size */
li r9,MMU_PAGE_4K /* actual page size */
ld r10,STK_PARAM(R9)(r1) /* segment size */
-_GLOBAL(htab_call_hpte_insert1)
+.globl htab_call_hpte_insert1
+htab_call_hpte_insert1:
bl . /* patched by htab_finish_init() */
cmpdi 0,r3,0
bge htab_pte_insert_ok /* Insertion successful */
@@ -554,7 +559,8 @@
li r8,MMU_PAGE_4K /* page size */
li r9,MMU_PAGE_4K /* actual page size */
ld r10,STK_PARAM(R9)(r1) /* segment size */
-_GLOBAL(htab_call_hpte_insert2)
+.globl htab_call_hpte_insert2
+htab_call_hpte_insert2:
bl . /* patched by htab_finish_init() */
cmpdi 0,r3,0
bge+ htab_pte_insert_ok /* Insertion successful */
@@ -571,7 +577,8 @@
2: and r0,r5,r27
rldicr r3,r0,3,63-3 /* r0 = (hash & mask) << 3 */
/* Call ppc_md.hpte_remove */
-_GLOBAL(htab_call_hpte_remove)
+.globl htab_call_hpte_remove
+htab_call_hpte_remove:
bl . /* patched by htab_finish_init() */
/* Try all again */
@@ -588,7 +595,7 @@
li r6,MMU_PAGE_64K /* psize */
ld r7,STK_PARAM(R9)(r1) /* ssize */
ld r8,STK_PARAM(R8)(r1) /* local */
- bl .flush_hash_page
+ bl flush_hash_page
/* Clear out _PAGE_HPTE_SUB bits in the new linux PTE */
lis r0,_PAGE_HPTE_SUB@h
ori r0,r0,_PAGE_HPTE_SUB@l
@@ -660,7 +667,8 @@
li r7,MMU_PAGE_4K /* actual page size */
ld r8,STK_PARAM(R9)(r1) /* segment size */
ld r9,STK_PARAM(R8)(r1) /* get "local" param */
-_GLOBAL(htab_call_hpte_updatepp)
+.globl htab_call_hpte_updatepp
+htab_call_hpte_updatepp:
bl . /* patched by htab_finish_init() */
/* if we failed because typically the HPTE wasn't really here
@@ -812,7 +820,7 @@
BEGIN_FTR_SECTION
mr r4,r30
mr r5,r7
- bl .hash_page_do_lazy_icache
+ bl hash_page_do_lazy_icache
END_FTR_SECTION(CPU_FTR_NOEXECUTE|CPU_FTR_COHERENT_ICACHE, CPU_FTR_NOEXECUTE)
/* At this point, r3 contains new PP bits, save them in
@@ -857,7 +865,8 @@
li r8,MMU_PAGE_64K
li r9,MMU_PAGE_64K /* actual page size */
ld r10,STK_PARAM(R9)(r1) /* segment size */
-_GLOBAL(ht64_call_hpte_insert1)
+.globl ht64_call_hpte_insert1
+ht64_call_hpte_insert1:
bl . /* patched by htab_finish_init() */
cmpdi 0,r3,0
bge ht64_pte_insert_ok /* Insertion successful */
@@ -881,7 +890,8 @@
li r8,MMU_PAGE_64K
li r9,MMU_PAGE_64K /* actual page size */
ld r10,STK_PARAM(R9)(r1) /* segment size */
-_GLOBAL(ht64_call_hpte_insert2)
+.globl ht64_call_hpte_insert2
+ht64_call_hpte_insert2:
bl . /* patched by htab_finish_init() */
cmpdi 0,r3,0
bge+ ht64_pte_insert_ok /* Insertion successful */
@@ -898,7 +908,8 @@
2: and r0,r5,r27
rldicr r3,r0,3,63-3 /* r0 = (hash & mask) << 3 */
/* Call ppc_md.hpte_remove */
-_GLOBAL(ht64_call_hpte_remove)
+.globl ht64_call_hpte_remove
+ht64_call_hpte_remove:
bl . /* patched by htab_finish_init() */
/* Try all again */
@@ -952,7 +963,8 @@
li r7,MMU_PAGE_64K /* actual page size */
ld r8,STK_PARAM(R9)(r1) /* segment size */
ld r9,STK_PARAM(R8)(r1) /* get "local" param */
-_GLOBAL(ht64_call_hpte_updatepp)
+.globl ht64_call_hpte_updatepp
+ht64_call_hpte_updatepp:
bl . /* patched by htab_finish_init() */
/* if we failed because typically the HPTE wasn't really here
diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c
index 350aa58..88fdd9d 100644
--- a/arch/powerpc/mm/hash_utils_64.c
+++ b/arch/powerpc/mm/hash_utils_64.c
@@ -449,6 +449,24 @@
mmu_psize_defs[bpsize].penc[apsize] = -1;
}
+#ifdef CONFIG_PPC_64K_PAGES
+
+static bool might_have_hea(void)
+{
+ /*
+ * The HEA ethernet adapter requires awareness of the
+ * GX bus. Without that awareness we can easily assume
+ * we will never see an HEA ethernet device.
+ */
+#ifdef CONFIG_IBMEBUS
+ return !cpu_has_feature(CPU_FTR_ARCH_207S);
+#else
+ return false;
+#endif
+}
+
+#endif /* #ifdef CONFIG_PPC_64K_PAGES */
+
static void __init htab_init_page_sizes(void)
{
int rc;
@@ -503,10 +521,11 @@
mmu_linear_psize = MMU_PAGE_64K;
if (mmu_has_feature(MMU_FTR_CI_LARGE_PAGE)) {
/*
- * Don't use 64k pages for ioremap on pSeries, since
- * that would stop us accessing the HEA ethernet.
+ * When running on pSeries using 64k pages for ioremap
+ * would stop us accessing the HEA ethernet. So if we
+ * have the chance of ever seeing one, stay at 4k.
*/
- if (!machine_is(pseries))
+ if (!might_have_hea() || !machine_is(pseries))
mmu_io_psize = MMU_PAGE_64K;
} else
mmu_ci_restrictions = 1;
@@ -607,47 +626,43 @@
}
#endif /* CONFIG_MEMORY_HOTPLUG */
-#define FUNCTION_TEXT(A) ((*(unsigned long *)(A)))
+extern u32 htab_call_hpte_insert1[];
+extern u32 htab_call_hpte_insert2[];
+extern u32 htab_call_hpte_remove[];
+extern u32 htab_call_hpte_updatepp[];
+extern u32 ht64_call_hpte_insert1[];
+extern u32 ht64_call_hpte_insert2[];
+extern u32 ht64_call_hpte_remove[];
+extern u32 ht64_call_hpte_updatepp[];
static void __init htab_finish_init(void)
{
- extern unsigned int *htab_call_hpte_insert1;
- extern unsigned int *htab_call_hpte_insert2;
- extern unsigned int *htab_call_hpte_remove;
- extern unsigned int *htab_call_hpte_updatepp;
-
#ifdef CONFIG_PPC_HAS_HASH_64K
- extern unsigned int *ht64_call_hpte_insert1;
- extern unsigned int *ht64_call_hpte_insert2;
- extern unsigned int *ht64_call_hpte_remove;
- extern unsigned int *ht64_call_hpte_updatepp;
-
patch_branch(ht64_call_hpte_insert1,
- FUNCTION_TEXT(ppc_md.hpte_insert),
+ ppc_function_entry(ppc_md.hpte_insert),
BRANCH_SET_LINK);
patch_branch(ht64_call_hpte_insert2,
- FUNCTION_TEXT(ppc_md.hpte_insert),
+ ppc_function_entry(ppc_md.hpte_insert),
BRANCH_SET_LINK);
patch_branch(ht64_call_hpte_remove,
- FUNCTION_TEXT(ppc_md.hpte_remove),
+ ppc_function_entry(ppc_md.hpte_remove),
BRANCH_SET_LINK);
patch_branch(ht64_call_hpte_updatepp,
- FUNCTION_TEXT(ppc_md.hpte_updatepp),
+ ppc_function_entry(ppc_md.hpte_updatepp),
BRANCH_SET_LINK);
-
#endif /* CONFIG_PPC_HAS_HASH_64K */
patch_branch(htab_call_hpte_insert1,
- FUNCTION_TEXT(ppc_md.hpte_insert),
+ ppc_function_entry(ppc_md.hpte_insert),
BRANCH_SET_LINK);
patch_branch(htab_call_hpte_insert2,
- FUNCTION_TEXT(ppc_md.hpte_insert),
+ ppc_function_entry(ppc_md.hpte_insert),
BRANCH_SET_LINK);
patch_branch(htab_call_hpte_remove,
- FUNCTION_TEXT(ppc_md.hpte_remove),
+ ppc_function_entry(ppc_md.hpte_remove),
BRANCH_SET_LINK);
patch_branch(htab_call_hpte_updatepp,
- FUNCTION_TEXT(ppc_md.hpte_updatepp),
+ ppc_function_entry(ppc_md.hpte_updatepp),
BRANCH_SET_LINK);
}
@@ -964,6 +979,22 @@
trap, vsid, ssize, psize, lpsize, pte);
}
+static void check_paca_psize(unsigned long ea, struct mm_struct *mm,
+ int psize, bool user_region)
+{
+ if (user_region) {
+ if (psize != get_paca_psize(ea)) {
+ get_paca()->context = mm->context;
+ slb_flush_and_rebolt();
+ }
+ } else if (get_paca()->vmalloc_sllp !=
+ mmu_psize_defs[mmu_vmalloc_psize].sllp) {
+ get_paca()->vmalloc_sllp =
+ mmu_psize_defs[mmu_vmalloc_psize].sllp;
+ slb_vmalloc_update();
+ }
+}
+
/* Result code is:
* 0 - handled
* 1 - normal page fault
@@ -1085,6 +1116,8 @@
WARN_ON(1);
}
#endif
+ check_paca_psize(ea, mm, psize, user_region);
+
goto bail;
}
@@ -1125,17 +1158,8 @@
#endif
}
}
- if (user_region) {
- if (psize != get_paca_psize(ea)) {
- get_paca()->context = mm->context;
- slb_flush_and_rebolt();
- }
- } else if (get_paca()->vmalloc_sllp !=
- mmu_psize_defs[mmu_vmalloc_psize].sllp) {
- get_paca()->vmalloc_sllp =
- mmu_psize_defs[mmu_vmalloc_psize].sllp;
- slb_vmalloc_update();
- }
+
+ check_paca_psize(ea, mm, psize, user_region);
#endif /* CONFIG_PPC_64K_PAGES */
#ifdef CONFIG_PPC_HAS_HASH_64K
diff --git a/arch/powerpc/mm/slb.c b/arch/powerpc/mm/slb.c
index 964a5f6..0399a67 100644
--- a/arch/powerpc/mm/slb.c
+++ b/arch/powerpc/mm/slb.c
@@ -256,10 +256,14 @@
patch_instruction(insn_addr, insn);
}
+extern u32 slb_compare_rr_to_size[];
+extern u32 slb_miss_kernel_load_linear[];
+extern u32 slb_miss_kernel_load_io[];
+extern u32 slb_compare_rr_to_size[];
+extern u32 slb_miss_kernel_load_vmemmap[];
+
void slb_set_size(u16 size)
{
- extern unsigned int *slb_compare_rr_to_size;
-
if (mmu_slb_size == size)
return;
@@ -272,11 +276,7 @@
unsigned long linear_llp, vmalloc_llp, io_llp;
unsigned long lflags, vflags;
static int slb_encoding_inited;
- extern unsigned int *slb_miss_kernel_load_linear;
- extern unsigned int *slb_miss_kernel_load_io;
- extern unsigned int *slb_compare_rr_to_size;
#ifdef CONFIG_SPARSEMEM_VMEMMAP
- extern unsigned int *slb_miss_kernel_load_vmemmap;
unsigned long vmemmap_llp;
#endif
diff --git a/arch/powerpc/mm/slb_low.S b/arch/powerpc/mm/slb_low.S
index 17aa6df..736d18b 100644
--- a/arch/powerpc/mm/slb_low.S
+++ b/arch/powerpc/mm/slb_low.S
@@ -35,7 +35,7 @@
* check for bad kernel/user address
* (ea & ~REGION_MASK) >= PGTABLE_RANGE
*/
- rldicr. r9,r3,4,(63 - 46 - 4)
+ rldicr. r9,r3,4,(63 - PGTABLE_EADDR_SIZE - 4)
bne- 8f
srdi r9,r3,60 /* get region */
@@ -59,7 +59,8 @@
/* Linear mapping encoding bits, the "li" instruction below will
* be patched by the kernel at boot
*/
-_GLOBAL(slb_miss_kernel_load_linear)
+.globl slb_miss_kernel_load_linear
+slb_miss_kernel_load_linear:
li r11,0
/*
* context = (MAX_USER_CONTEXT) + ((ea >> 60) - 0xc) + 1
@@ -79,7 +80,8 @@
/* Check virtual memmap region. To be patches at kernel boot */
cmpldi cr0,r9,0xf
bne 1f
-_GLOBAL(slb_miss_kernel_load_vmemmap)
+.globl slb_miss_kernel_load_vmemmap
+slb_miss_kernel_load_vmemmap:
li r11,0
b 6f
1:
@@ -95,7 +97,8 @@
b 6f
5:
/* IO mapping */
- _GLOBAL(slb_miss_kernel_load_io)
+.globl slb_miss_kernel_load_io
+slb_miss_kernel_load_io:
li r11,0
6:
/*
@@ -250,7 +253,8 @@
7: ld r10,PACASTABRR(r13)
addi r10,r10,1
/* This gets soft patched on boot. */
-_GLOBAL(slb_compare_rr_to_size)
+.globl slb_compare_rr_to_size
+slb_compare_rr_to_size:
cmpldi r10,0
blt+ 4f
diff --git a/arch/powerpc/mm/tlb_nohash.c b/arch/powerpc/mm/tlb_nohash.c
index ae3d5b79..92cb18d 100644
--- a/arch/powerpc/mm/tlb_nohash.c
+++ b/arch/powerpc/mm/tlb_nohash.c
@@ -596,8 +596,13 @@
/* XXX This should be decided at runtime based on supported
* page sizes in the TLB, but for now let's assume 16M is
* always there and a good fit (which it probably is)
+ *
+ * Freescale booke only supports 4K pages in TLB0, so use that.
*/
- mmu_vmemmap_psize = MMU_PAGE_16M;
+ if (mmu_has_feature(MMU_FTR_TYPE_FSL_E))
+ mmu_vmemmap_psize = MMU_PAGE_4K;
+ else
+ mmu_vmemmap_psize = MMU_PAGE_16M;
/* XXX This code only checks for TLB 0 capabilities and doesn't
* check what page size combos are supported by the HW. It
diff --git a/arch/powerpc/platforms/44x/Kconfig b/arch/powerpc/platforms/44x/Kconfig
index dc1a264..4d88f6a 100644
--- a/arch/powerpc/platforms/44x/Kconfig
+++ b/arch/powerpc/platforms/44x/Kconfig
@@ -199,6 +199,34 @@
help
This option enables support for the IBM Currituck (476fpe) evaluation board
+config AKEBONO
+ bool "IBM Akebono (476gtr) Support"
+ depends on PPC_47x
+ default n
+ select SWIOTLB
+ select 476FPE
+ select PPC4xx_PCI_EXPRESS
+ select PCI_MSI
+ select PPC4xx_HSTA_MSI
+ select I2C
+ select I2C_IBM_IIC
+ select NETDEVICES
+ select ETHERNET
+ select NET_VENDOR_IBM
+ select IBM_EMAC_EMAC4
+ select IBM_EMAC_RGMII_WOL
+ select USB
+ select USB_OHCI_HCD_PLATFORM
+ select USB_EHCI_HCD_PLATFORM
+ select MMC_SDHCI
+ select MMC_SDHCI_PLTFM
+ select MMC_SDHCI_OF_476GTR
+ select ATA
+ select SATA_AHCI_PLATFORM
+ help
+ This option enables support for the IBM Akebono (476gtr) evaluation board
+
+
config ICON
bool "Icon"
depends on 44x
@@ -323,6 +351,20 @@
select IBM_EMAC_EMAC4
select IBM_EMAC_TAH
+config 476FPE_ERR46
+ depends on 476FPE
+ bool "Enable linker work around for PPC476FPE errata #46"
+ help
+ This option enables a work around for an icache bug on 476
+ that can cause execution of stale instructions when falling
+ through pages (IBM errata #46). It requires a recent version
+ of binutils which supports the --ppc476-workaround option.
+
+ The work around enables the appropriate linker options and
+ ensures that all module output sections are aligned to 4K
+ page boundaries. The work around is only required when
+ building modules.
+
# 44x errata/workaround config symbols, selected by the CPU models above
config IBM440EP_ERR42
bool
diff --git a/arch/powerpc/platforms/44x/Makefile b/arch/powerpc/platforms/44x/Makefile
index d03833a..26d35b5 100644
--- a/arch/powerpc/platforms/44x/Makefile
+++ b/arch/powerpc/platforms/44x/Makefile
@@ -10,4 +10,5 @@
obj-$(CONFIG_XILINX_ML510) += virtex_ml510.o
obj-$(CONFIG_ISS4xx) += iss4xx.o
obj-$(CONFIG_CANYONLANDS)+= canyonlands.o
-obj-$(CONFIG_CURRITUCK) += currituck.o
+obj-$(CONFIG_CURRITUCK) += ppc476.o
+obj-$(CONFIG_AKEBONO) += ppc476.o
diff --git a/arch/powerpc/platforms/44x/currituck.c b/arch/powerpc/platforms/44x/ppc476.c
similarity index 72%
rename from arch/powerpc/platforms/44x/currituck.c
rename to arch/powerpc/platforms/44x/ppc476.c
index 7f1b71a..33986c1 100644
--- a/arch/powerpc/platforms/44x/currituck.c
+++ b/arch/powerpc/platforms/44x/ppc476.c
@@ -1,7 +1,8 @@
/*
- * Currituck board specific routines
+ * PowerPC 476FPE board specific routines
*
- * Copyright © 2011 Tony Breeds IBM Corporation
+ * Copyright © 2013 Tony Breeds IBM Corporation
+ * Copyright © 2013 Alistair Popple IBM Corporation
*
* Based on earlier code:
* Matt Porter <mporter@kernel.crashing.org>
@@ -35,8 +36,9 @@
#include <asm/mmu.h>
#include <linux/pci.h>
+#include <linux/i2c.h>
-static __initdata struct of_device_id ppc47x_of_bus[] = {
+static struct of_device_id ppc47x_of_bus[] __initdata = {
{ .compatible = "ibm,plb4", },
{ .compatible = "ibm,plb6", },
{ .compatible = "ibm,opb", },
@@ -55,15 +57,69 @@
}
DECLARE_PCI_FIXUP_HEADER(0x1033, 0x0035, quirk_ppc_currituck_usb_fixup);
+/* Akebono has an AVR microcontroller attached to the I2C bus
+ * which is used to power off/reset the system. */
+
+/* AVR I2C Commands */
+#define AVR_PWRCTL_CMD (0x26)
+
+/* Flags for the power control I2C commands */
+#define AVR_PWRCTL_PWROFF (0x01)
+#define AVR_PWRCTL_RESET (0x02)
+
+static struct i2c_client *avr_i2c_client;
+static void avr_halt_system(int pwrctl_flags)
+{
+ /* Request the AVR to reset the system */
+ i2c_smbus_write_byte_data(avr_i2c_client,
+ AVR_PWRCTL_CMD, pwrctl_flags);
+
+ /* Wait for system to be reset */
+ while (1)
+ ;
+}
+
+static void avr_power_off_system(void)
+{
+ avr_halt_system(AVR_PWRCTL_PWROFF);
+}
+
+static void avr_reset_system(char *cmd)
+{
+ avr_halt_system(AVR_PWRCTL_RESET);
+}
+
+static int avr_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ avr_i2c_client = client;
+ ppc_md.restart = avr_reset_system;
+ ppc_md.power_off = avr_power_off_system;
+ return 0;
+}
+
+static const struct i2c_device_id avr_id[] = {
+ { "akebono-avr", 0 },
+ { }
+};
+
+static struct i2c_driver avr_driver = {
+ .driver = {
+ .name = "akebono-avr",
+ },
+ .probe = avr_probe,
+ .id_table = avr_id,
+};
+
static int __init ppc47x_device_probe(void)
{
+ i2c_add_driver(&avr_driver);
of_platform_bus_probe(NULL, ppc47x_of_bus, NULL);
return 0;
}
machine_device_initcall(ppc47x, ppc47x_device_probe);
-/* We can have either UICs or MPICs */
static void __init ppc47x_init_irq(void)
{
struct device_node *np;
@@ -157,43 +213,36 @@
{
/* No need to check the DMA config as we /know/ our windows are all of
- * RAM. Lets hope that doesn't change */
+ * RAM. Lets hope that doesn't change */
swiotlb_detect_4g();
ppc47x_smp_init();
}
-/*
- * Called very early, MMU is off, device-tree isn't unflattened
- */
-static int __init ppc47x_probe(void)
-{
- unsigned long root = of_get_flat_dt_root();
-
- if (!of_flat_dt_is_compatible(root, "ibm,currituck"))
- return 0;
-
- return 1;
-}
-
static int board_rev = -1;
static int __init ppc47x_get_board_rev(void)
{
- u8 fpga_reg0;
- void *fpga;
- struct device_node *np;
+ int reg;
+ u8 *fpga;
+ struct device_node *np = NULL;
- np = of_find_compatible_node(NULL, NULL, "ibm,currituck-fpga");
+ if (of_machine_is_compatible("ibm,currituck")) {
+ np = of_find_compatible_node(NULL, NULL, "ibm,currituck-fpga");
+ reg = 0;
+ } else if (of_machine_is_compatible("ibm,akebono")) {
+ np = of_find_compatible_node(NULL, NULL, "ibm,akebono-fpga");
+ reg = 2;
+ }
+
if (!np)
goto fail;
- fpga = of_iomap(np, 0);
+ fpga = (u8 *) of_iomap(np, 0);
of_node_put(np);
if (!fpga)
goto fail;
- fpga_reg0 = ioread8(fpga);
- board_rev = fpga_reg0 & 0x03;
+ board_rev = ioread8(fpga + reg) & 0x03;
pr_info("%s: Found board revision %d\n", __func__, board_rev);
iounmap(fpga);
return 0;
@@ -208,7 +257,7 @@
static void ppc47x_pci_irq_fixup(struct pci_dev *dev)
{
if (dev->vendor == 0x1033 && (dev->device == 0x0035 ||
- dev->device == 0x00e0)) {
+ dev->device == 0x00e0)) {
if (board_rev == 0) {
dev->irq = irq_create_mapping(NULL, 47);
pr_info("%s: Mapping irq %d\n", __func__, dev->irq);
@@ -221,13 +270,30 @@
}
}
+/*
+ * Called very early, MMU is off, device-tree isn't unflattened
+ */
+static int __init ppc47x_probe(void)
+{
+ unsigned long root = of_get_flat_dt_root();
+
+ if (of_flat_dt_is_compatible(root, "ibm,akebono"))
+ return 1;
+
+ if (of_flat_dt_is_compatible(root, "ibm,currituck")) {
+ ppc_md.pci_irq_fixup = ppc47x_pci_irq_fixup;
+ return 1;
+ }
+
+ return 0;
+}
+
define_machine(ppc47x) {
.name = "PowerPC 47x",
.probe = ppc47x_probe,
.progress = udbg_progress,
.init_IRQ = ppc47x_init_irq,
.setup_arch = ppc47x_setup_arch,
- .pci_irq_fixup = ppc47x_pci_irq_fixup,
.restart = ppc4xx_reset_system,
.calibrate_decr = generic_calibrate_decr,
};
diff --git a/arch/powerpc/platforms/44x/ppc476_modules.lds b/arch/powerpc/platforms/44x/ppc476_modules.lds
new file mode 100644
index 0000000..9fec5d3
--- /dev/null
+++ b/arch/powerpc/platforms/44x/ppc476_modules.lds
@@ -0,0 +1,15 @@
+SECTIONS
+{
+ .text : ALIGN(4096)
+ {
+ *(.text .text.* .fixup)
+ }
+ .init.text : ALIGN(4096)
+ {
+ *(.init.text .init.text.*)
+ }
+ .exit.text : ALIGN(4096)
+ {
+ *(.exit.text .exit.text.*)
+ }
+}
diff --git a/arch/powerpc/platforms/85xx/Kconfig b/arch/powerpc/platforms/85xx/Kconfig
index c17aae8..f442120 100644
--- a/arch/powerpc/platforms/85xx/Kconfig
+++ b/arch/powerpc/platforms/85xx/Kconfig
@@ -38,6 +38,15 @@
help
This option enables support for the C293PCIE board
+config BSC9132_QDS
+ bool "Freescale BSC9132QDS"
+ select DEFAULT_UIMAGE
+ help
+ This option enables support for the Freescale BSC9132 QDS board.
+ BSC9132 is a heterogeneous SoC containing dual e500v2 powerpc cores
+ and dual StarCore SC3850 DSP cores.
+ Manufacturer : Freescale Semiconductor, Inc
+
config MPC8540_ADS
bool "Freescale MPC8540 ADS"
select DEFAULT_UIMAGE
@@ -117,11 +126,11 @@
This option enables support for the Freescale / iVeia P1022RDK
reference board.
-config P1023_RDS
- bool "Freescale P1023 RDS/RDB"
+config P1023_RDB
+ bool "Freescale P1023 RDB"
select DEFAULT_UIMAGE
help
- This option enables support for the P1023 RDS and RDB boards
+ This option enables support for the P1023 RDB board.
config TWR_P102x
bool "Freescale TWR-P102x"
@@ -263,11 +272,11 @@
help
This option enables support for the FSL CoreNet based boards.
For 32bit kernel, the following boards are supported:
- P2041 RDB, P3041 DS and P4080 DS
+ P2041 RDB, P3041 DS, P4080 DS, kmcoge4, and OCA4080
For 64bit kernel, the following boards are supported:
T4240 QDS and B4 QDS
The following boards are supported for both 32bit and 64bit kernel:
- P5020 DS and P5040 DS
+ P5020 DS, P5040 DS and T104xQDS
endif # FSL_SOC_BOOKE
diff --git a/arch/powerpc/platforms/85xx/Makefile b/arch/powerpc/platforms/85xx/Makefile
index 25cebe7..7303260 100644
--- a/arch/powerpc/platforms/85xx/Makefile
+++ b/arch/powerpc/platforms/85xx/Makefile
@@ -6,6 +6,7 @@
obj-y += common.o
obj-$(CONFIG_BSC9131_RDB) += bsc913x_rdb.o
+obj-$(CONFIG_BSC9132_QDS) += bsc913x_qds.o
obj-$(CONFIG_C293_PCIE) += c293pcie.o
obj-$(CONFIG_MPC8540_ADS) += mpc85xx_ads.o
obj-$(CONFIG_MPC8560_ADS) += mpc85xx_ads.o
@@ -17,7 +18,7 @@
obj-$(CONFIG_P1010_RDB) += p1010rdb.o
obj-$(CONFIG_P1022_DS) += p1022_ds.o
obj-$(CONFIG_P1022_RDK) += p1022_rdk.o
-obj-$(CONFIG_P1023_RDS) += p1023_rds.o
+obj-$(CONFIG_P1023_RDB) += p1023_rdb.o
obj-$(CONFIG_TWR_P102x) += twr_p102x.o
obj-$(CONFIG_CORENET_GENERIC) += corenet_generic.o
obj-$(CONFIG_STX_GP3) += stx_gp3.o
diff --git a/arch/powerpc/platforms/85xx/bsc913x_qds.c b/arch/powerpc/platforms/85xx/bsc913x_qds.c
new file mode 100644
index 0000000..f0927e5
--- /dev/null
+++ b/arch/powerpc/platforms/85xx/bsc913x_qds.c
@@ -0,0 +1,74 @@
+/*
+ * BSC913xQDS Board Setup
+ *
+ * Author:
+ * Harninder Rai <harninder.rai@freescale.com>
+ * Priyanka Jain <Priyanka.Jain@freescale.com>
+ *
+ * Copyright 2014 Freescale Semiconductor Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/of_platform.h>
+#include <linux/pci.h>
+#include <asm/mpic.h>
+#include <sysdev/fsl_soc.h>
+#include <asm/udbg.h>
+
+#include "mpc85xx.h"
+#include "smp.h"
+
+void __init bsc913x_qds_pic_init(void)
+{
+ struct mpic *mpic = mpic_alloc(NULL, 0, MPIC_BIG_ENDIAN |
+ MPIC_SINGLE_DEST_CPU,
+ 0, 256, " OpenPIC ");
+
+ if (!mpic)
+ pr_err("bsc913x: Failed to allocate MPIC structure\n");
+ else
+ mpic_init(mpic);
+}
+
+/*
+ * Setup the architecture
+ */
+static void __init bsc913x_qds_setup_arch(void)
+{
+ if (ppc_md.progress)
+ ppc_md.progress("bsc913x_qds_setup_arch()", 0);
+
+#if defined(CONFIG_SMP)
+ mpc85xx_smp_init();
+#endif
+
+ pr_info("bsc913x board from Freescale Semiconductor\n");
+}
+
+machine_device_initcall(bsc9132_qds, mpc85xx_common_publish_devices);
+
+/*
+ * Called very early, device-tree isn't unflattened
+ */
+
+static int __init bsc9132_qds_probe(void)
+{
+ unsigned long root = of_get_flat_dt_root();
+
+ return of_flat_dt_is_compatible(root, "fsl,bsc9132qds");
+}
+
+define_machine(bsc9132_qds) {
+ .name = "BSC9132 QDS",
+ .probe = bsc9132_qds_probe,
+ .setup_arch = bsc913x_qds_setup_arch,
+ .init_IRQ = bsc913x_qds_pic_init,
+ .get_irq = mpic_get_irq,
+ .restart = fsl_rstcr_restart,
+ .calibrate_decr = generic_calibrate_decr,
+ .progress = udbg_progress,
+};
diff --git a/arch/powerpc/platforms/85xx/corenet_generic.c b/arch/powerpc/platforms/85xx/corenet_generic.c
index 8e4b1e1..5db1e11 100644
--- a/arch/powerpc/platforms/85xx/corenet_generic.c
+++ b/arch/powerpc/platforms/85xx/corenet_generic.c
@@ -67,7 +67,7 @@
swiotlb_detect_4g();
- pr_info("%s board from Freescale Semiconductor\n", ppc_md.name);
+ pr_info("%s board\n", ppc_md.name);
mpc85xx_qe_init();
}
@@ -115,6 +115,7 @@
static const char * const boards[] __initconst = {
"fsl,P2041RDB",
"fsl,P3041DS",
+ "fsl,OCA4080",
"fsl,P4080DS",
"fsl,P5020DS",
"fsl,P5040DS",
@@ -122,12 +123,16 @@
"fsl,B4860QDS",
"fsl,B4420QDS",
"fsl,B4220QDS",
+ "fsl,T1040QDS",
+ "fsl,T1042QDS",
+ "keymile,kmcoge4",
NULL
};
static const char * const hv_boards[] __initconst = {
"fsl,P2041RDB-hv",
"fsl,P3041DS-hv",
+ "fsl,OCA4080-hv",
"fsl,P4080DS-hv",
"fsl,P5020DS-hv",
"fsl,P5040DS-hv",
@@ -135,6 +140,8 @@
"fsl,B4860QDS-hv",
"fsl,B4420QDS-hv",
"fsl,B4220QDS-hv",
+ "fsl,T1040QDS-hv",
+ "fsl,T1042QDS-hv",
NULL
};
diff --git a/arch/powerpc/platforms/85xx/p1023_rds.c b/arch/powerpc/platforms/85xx/p1023_rdb.c
similarity index 75%
rename from arch/powerpc/platforms/85xx/p1023_rds.c
rename to arch/powerpc/platforms/85xx/p1023_rdb.c
index 0e61400..d5b7509 100644
--- a/arch/powerpc/platforms/85xx/p1023_rds.c
+++ b/arch/powerpc/platforms/85xx/p1023_rdb.c
@@ -4,7 +4,7 @@
* Author: Roy Zang <tie-fei.zang@freescale.com>
*
* Description:
- * P1023 RDS Board Setup
+ * P1023 RDB Board Setup
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
@@ -41,12 +41,12 @@
* Setup the architecture
*
*/
-static void __init mpc85xx_rds_setup_arch(void)
+static void __init mpc85xx_rdb_setup_arch(void)
{
struct device_node *np;
if (ppc_md.progress)
- ppc_md.progress("p1023_rds_setup_arch()", 0);
+ ppc_md.progress("p1023_rdb_setup_arch()", 0);
/* Map BCSR area */
np = of_find_node_by_name(NULL, "bcsr");
@@ -85,10 +85,9 @@
fsl_pci_assign_primary();
}
-machine_arch_initcall(p1023_rds, mpc85xx_common_publish_devices);
machine_arch_initcall(p1023_rdb, mpc85xx_common_publish_devices);
-static void __init mpc85xx_rds_pic_init(void)
+static void __init mpc85xx_rdb_pic_init(void)
{
struct mpic *mpic = mpic_alloc(NULL, 0, MPIC_BIG_ENDIAN |
MPIC_SINGLE_DEST_CPU,
@@ -99,14 +98,6 @@
mpic_init(mpic);
}
-static int __init p1023_rds_probe(void)
-{
- unsigned long root = of_get_flat_dt_root();
-
- return of_flat_dt_is_compatible(root, "fsl,P1023RDS");
-
-}
-
static int __init p1023_rdb_probe(void)
{
unsigned long root = of_get_flat_dt_root();
@@ -115,26 +106,11 @@
}
-define_machine(p1023_rds) {
- .name = "P1023 RDS",
- .probe = p1023_rds_probe,
- .setup_arch = mpc85xx_rds_setup_arch,
- .init_IRQ = mpc85xx_rds_pic_init,
- .get_irq = mpic_get_irq,
- .restart = fsl_rstcr_restart,
- .calibrate_decr = generic_calibrate_decr,
- .progress = udbg_progress,
-#ifdef CONFIG_PCI
- .pcibios_fixup_bus = fsl_pcibios_fixup_bus,
- .pcibios_fixup_phb = fsl_pcibios_fixup_phb,
-#endif
-};
-
define_machine(p1023_rdb) {
.name = "P1023 RDB",
.probe = p1023_rdb_probe,
- .setup_arch = mpc85xx_rds_setup_arch,
- .init_IRQ = mpc85xx_rds_pic_init,
+ .setup_arch = mpc85xx_rdb_setup_arch,
+ .init_IRQ = mpc85xx_rdb_pic_init,
.get_irq = mpic_get_irq,
.restart = fsl_rstcr_restart,
.calibrate_decr = generic_calibrate_decr,
diff --git a/arch/powerpc/platforms/85xx/smp.c b/arch/powerpc/platforms/85xx/smp.c
index 6382098..ba093f5 100644
--- a/arch/powerpc/platforms/85xx/smp.c
+++ b/arch/powerpc/platforms/85xx/smp.c
@@ -27,6 +27,7 @@
#include <asm/cacheflush.h>
#include <asm/dbell.h>
#include <asm/fsl_guts.h>
+#include <asm/code-patching.h>
#include <sysdev/fsl_soc.h>
#include <sysdev/mpic.h>
@@ -267,7 +268,7 @@
flush_spin_table(spin_table);
out_be32(&spin_table->pir, hw_cpu);
out_be64((u64 *)(&spin_table->addr_h),
- __pa((u64)*((unsigned long long *)generic_secondary_smp_init)));
+ __pa(ppc_function_entry(generic_secondary_smp_init)));
flush_spin_table(spin_table);
#endif
diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype
index d9e2b19b7..43b65ad 100644
--- a/arch/powerpc/platforms/Kconfig.cputype
+++ b/arch/powerpc/platforms/Kconfig.cputype
@@ -422,6 +422,7 @@
config CPU_LITTLE_ENDIAN
bool "Build little endian kernel"
+ select PPC64_BOOT_WRAPPER
help
Build a little endian kernel.
@@ -430,3 +431,7 @@
little endian powerpc.
endchoice
+
+config PPC64_BOOT_WRAPPER
+ def_bool n
+ depends on CPU_LITTLE_ENDIAN
diff --git a/arch/powerpc/platforms/cell/smp.c b/arch/powerpc/platforms/cell/smp.c
index 90745ea..c8017a7 100644
--- a/arch/powerpc/platforms/cell/smp.c
+++ b/arch/powerpc/platforms/cell/smp.c
@@ -40,6 +40,7 @@
#include <asm/firmware.h>
#include <asm/rtas.h>
#include <asm/cputhreads.h>
+#include <asm/code-patching.h>
#include "interrupt.h"
#include <asm/udbg.h>
@@ -70,8 +71,8 @@
static inline int smp_startup_cpu(unsigned int lcpu)
{
int status;
- unsigned long start_here = __pa((u32)*((unsigned long *)
- generic_secondary_smp_init));
+ unsigned long start_here =
+ __pa(ppc_function_entry(generic_secondary_smp_init));
unsigned int pcpu;
int start_cpu;
diff --git a/arch/powerpc/platforms/embedded6xx/Kconfig b/arch/powerpc/platforms/embedded6xx/Kconfig
index 2a7024d..a25f496 100644
--- a/arch/powerpc/platforms/embedded6xx/Kconfig
+++ b/arch/powerpc/platforms/embedded6xx/Kconfig
@@ -65,6 +65,7 @@
select PPC_INDIRECT_PCI
select PPC_I8259
select PPC_NATIVE
+ select PPC_UDBG_16550
help
This option enables support for the Motorola (now Emerson) MVME5100
board.
diff --git a/arch/powerpc/platforms/pasemi/powersave.S b/arch/powerpc/platforms/pasemi/powersave.S
index 56f45ad..81ab555 100644
--- a/arch/powerpc/platforms/pasemi/powersave.S
+++ b/arch/powerpc/platforms/pasemi/powersave.S
@@ -66,7 +66,7 @@
std r3, 48(r1)
/* Only do power savings when in astate 0 */
- bl .check_astate
+ bl check_astate
cmpwi r3,0
bne 1f
diff --git a/arch/powerpc/platforms/powernv/Makefile b/arch/powerpc/platforms/powernv/Makefile
index 63cebb9..4ad0d34 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -1,7 +1,7 @@
obj-y += setup.o opal-takeover.o opal-wrappers.o opal.o opal-async.o
obj-y += opal-rtc.o opal-nvram.o opal-lpc.o opal-flash.o
obj-y += rng.o opal-elog.o opal-dump.o opal-sysparam.o opal-sensor.o
-obj-y += opal-msglog.o
+obj-y += opal-msglog.o subcore.o subcore-asm.o
obj-$(CONFIG_SMP) += smp.o
obj-$(CONFIG_PCI) += pci.o pci-p5ioc2.o pci-ioda.o
diff --git a/arch/powerpc/platforms/powernv/eeh-ioda.c b/arch/powerpc/platforms/powernv/eeh-ioda.c
index 5b51079..753f08e 100644
--- a/arch/powerpc/platforms/powernv/eeh-ioda.c
+++ b/arch/powerpc/platforms/powernv/eeh-ioda.c
@@ -42,11 +42,19 @@
{
uint64_t changed_evts = (uint64_t)change;
- /* We simply send special EEH event */
- if ((changed_evts & OPAL_EVENT_PCI_ERROR) &&
- (events & OPAL_EVENT_PCI_ERROR) &&
- eeh_enabled())
+ /*
+ * We simply send special EEH event if EEH has
+ * been enabled, or clear pending events in
+ * case that we enable EEH soon
+ */
+ if (!(changed_evts & OPAL_EVENT_PCI_ERROR) ||
+ !(events & OPAL_EVENT_PCI_ERROR))
+ return 0;
+
+ if (eeh_enabled())
eeh_send_failure_event(NULL);
+ else
+ opal_notifier_update_evt(OPAL_EVENT_PCI_ERROR, 0x0ul);
return 0;
}
@@ -141,7 +149,9 @@
}
#ifdef CONFIG_DEBUG_FS
- if (phb->dbgfs) {
+ if (!phb->has_dbgfs && phb->dbgfs) {
+ phb->has_dbgfs = 1;
+
debugfs_create_file("err_injct_outbound", 0600,
phb->dbgfs, hose,
&ioda_eeh_outb_dbgfs_ops);
@@ -154,7 +164,14 @@
}
#endif
- phb->eeh_state |= PNV_EEH_STATE_ENABLED;
+ /* If EEH is enabled, we're going to rely on that.
+ * Otherwise, we restore to conventional mechanism
+ * to clear frozen PE during PCI config access.
+ */
+ if (eeh_enabled())
+ phb->flags |= PNV_PHB_FLAG_EEH;
+ else
+ phb->flags &= ~PNV_PHB_FLAG_EEH;
return 0;
}
@@ -268,6 +285,21 @@
return EEH_STATE_NOT_SUPPORT;
}
+ /*
+ * If we're in middle of PE reset, return normal
+ * state to keep EEH core going. For PHB reset, we
+ * still expect to have fenced PHB cleared with
+ * PHB reset.
+ */
+ if (!(pe->type & EEH_PE_PHB) &&
+ (pe->state & EEH_PE_RESET)) {
+ result = (EEH_STATE_MMIO_ACTIVE |
+ EEH_STATE_DMA_ACTIVE |
+ EEH_STATE_MMIO_ENABLED |
+ EEH_STATE_DMA_ENABLED);
+ return result;
+ }
+
/* Retrieve PE status through OPAL */
pe_no = pe->addr;
ret = opal_pci_eeh_freeze_status(phb->opal_id, pe_no,
@@ -347,52 +379,6 @@
return result;
}
-static int ioda_eeh_pe_clear(struct eeh_pe *pe)
-{
- struct pci_controller *hose;
- struct pnv_phb *phb;
- u32 pe_no;
- u8 fstate;
- u16 pcierr;
- s64 ret;
-
- pe_no = pe->addr;
- hose = pe->phb;
- phb = pe->phb->private_data;
-
- /* Clear the EEH error on the PE */
- ret = opal_pci_eeh_freeze_clear(phb->opal_id,
- pe_no, OPAL_EEH_ACTION_CLEAR_FREEZE_ALL);
- if (ret) {
- pr_err("%s: Failed to clear EEH error for "
- "PHB#%x-PE#%x, err=%lld\n",
- __func__, hose->global_number, pe_no, ret);
- return -EIO;
- }
-
- /*
- * Read the PE state back and verify that the frozen
- * state has been removed.
- */
- ret = opal_pci_eeh_freeze_status(phb->opal_id, pe_no,
- &fstate, &pcierr, NULL);
- if (ret) {
- pr_err("%s: Failed to get EEH status on "
- "PHB#%x-PE#%x\n, err=%lld\n",
- __func__, hose->global_number, pe_no, ret);
- return -EIO;
- }
-
- if (fstate != OPAL_EEH_STOPPED_NOT_FROZEN) {
- pr_err("%s: Frozen state not cleared on "
- "PHB#%x-PE#%x, sts=%x\n",
- __func__, hose->global_number, pe_no, fstate);
- return -EIO;
- }
-
- return 0;
-}
-
static s64 ioda_eeh_phb_poll(struct pnv_phb *phb)
{
s64 rc = OPAL_HARDWARE;
@@ -402,13 +388,16 @@
if (rc <= 0)
break;
- msleep(rc);
+ if (system_state < SYSTEM_RUNNING)
+ udelay(1000 * rc);
+ else
+ msleep(rc);
}
return rc;
}
-static int ioda_eeh_phb_reset(struct pci_controller *hose, int option)
+int ioda_eeh_phb_reset(struct pci_controller *hose, int option)
{
struct pnv_phb *phb = hose->private_data;
s64 rc = OPAL_HARDWARE;
@@ -431,9 +420,17 @@
/*
* Poll state of the PHB until the request is done
- * successfully.
+ * successfully. The PHB reset is usually PHB complete
+ * reset followed by hot reset on root bus. So we also
+ * need the PCI bus settlement delay.
*/
rc = ioda_eeh_phb_poll(phb);
+ if (option == EEH_RESET_DEACTIVATE) {
+ if (system_state < SYSTEM_RUNNING)
+ udelay(1000 * EEH_PE_RST_SETTLE_TIME);
+ else
+ msleep(EEH_PE_RST_SETTLE_TIME);
+ }
out:
if (rc != OPAL_SUCCESS)
return -EIO;
@@ -471,6 +468,8 @@
/* Poll state of the PHB until the request is done */
rc = ioda_eeh_phb_poll(phb);
+ if (option == EEH_RESET_DEACTIVATE)
+ msleep(EEH_PE_RST_SETTLE_TIME);
out:
if (rc != OPAL_SUCCESS)
return -EIO;
@@ -478,32 +477,71 @@
return 0;
}
-static int ioda_eeh_bridge_reset(struct pci_controller *hose,
- struct pci_dev *dev, int option)
-{
- u16 ctrl;
+static int ioda_eeh_bridge_reset(struct pci_dev *dev, int option)
- pr_debug("%s: Reset device %04x:%02x:%02x.%01x with option %d\n",
- __func__, hose->global_number, dev->bus->number,
- PCI_SLOT(dev->devfn), PCI_FUNC(dev->devfn), option);
+{
+ struct device_node *dn = pci_device_to_OF_node(dev);
+ struct eeh_dev *edev = of_node_to_eeh_dev(dn);
+ int aer = edev ? edev->aer_cap : 0;
+ u32 ctrl;
+
+ pr_debug("%s: Reset PCI bus %04x:%02x with option %d\n",
+ __func__, pci_domain_nr(dev->bus),
+ dev->bus->number, option);
switch (option) {
case EEH_RESET_FUNDAMENTAL:
case EEH_RESET_HOT:
- pci_read_config_word(dev, PCI_BRIDGE_CONTROL, &ctrl);
+ /* Don't report linkDown event */
+ if (aer) {
+ eeh_ops->read_config(dn, aer + PCI_ERR_UNCOR_MASK,
+ 4, &ctrl);
+ ctrl |= PCI_ERR_UNC_SURPDN;
+ eeh_ops->write_config(dn, aer + PCI_ERR_UNCOR_MASK,
+ 4, ctrl);
+ }
+
+ eeh_ops->read_config(dn, PCI_BRIDGE_CONTROL, 2, &ctrl);
ctrl |= PCI_BRIDGE_CTL_BUS_RESET;
- pci_write_config_word(dev, PCI_BRIDGE_CONTROL, ctrl);
+ eeh_ops->write_config(dn, PCI_BRIDGE_CONTROL, 2, ctrl);
+ msleep(EEH_PE_RST_HOLD_TIME);
+
break;
case EEH_RESET_DEACTIVATE:
- pci_read_config_word(dev, PCI_BRIDGE_CONTROL, &ctrl);
+ eeh_ops->read_config(dn, PCI_BRIDGE_CONTROL, 2, &ctrl);
ctrl &= ~PCI_BRIDGE_CTL_BUS_RESET;
- pci_write_config_word(dev, PCI_BRIDGE_CONTROL, ctrl);
+ eeh_ops->write_config(dn, PCI_BRIDGE_CONTROL, 2, ctrl);
+ msleep(EEH_PE_RST_SETTLE_TIME);
+
+ /* Continue reporting linkDown event */
+ if (aer) {
+ eeh_ops->read_config(dn, aer + PCI_ERR_UNCOR_MASK,
+ 4, &ctrl);
+ ctrl &= ~PCI_ERR_UNC_SURPDN;
+ eeh_ops->write_config(dn, aer + PCI_ERR_UNCOR_MASK,
+ 4, ctrl);
+ }
+
break;
}
return 0;
}
+void pnv_pci_reset_secondary_bus(struct pci_dev *dev)
+{
+ struct pci_controller *hose;
+
+ if (pci_is_root_bus(dev->bus)) {
+ hose = pci_bus_to_host(dev->bus);
+ ioda_eeh_root_reset(hose, EEH_RESET_HOT);
+ ioda_eeh_root_reset(hose, EEH_RESET_DEACTIVATE);
+ } else {
+ ioda_eeh_bridge_reset(dev, EEH_RESET_HOT);
+ ioda_eeh_bridge_reset(dev, EEH_RESET_DEACTIVATE);
+ }
+}
+
/**
* ioda_eeh_reset - Reset the indicated PE
* @pe: EEH PE
@@ -523,27 +561,18 @@
int ret;
/*
- * Anyway, we have to clear the problematic state for the
- * corresponding PE. However, we needn't do it if the PE
- * is PHB associated. That means the PHB is having fatal
- * errors and it needs reset. Further more, the AIB interface
- * isn't reliable any more.
- */
- if (!(pe->type & EEH_PE_PHB) &&
- (option == EEH_RESET_HOT ||
- option == EEH_RESET_FUNDAMENTAL)) {
- ret = ioda_eeh_pe_clear(pe);
- if (ret)
- return -EIO;
- }
-
- /*
- * The rules applied to reset, either fundamental or hot reset:
+ * For PHB reset, we always have complete reset. For those PEs whose
+ * primary bus derived from root complex (root bus) or root port
+ * (usually bus#1), we apply hot or fundamental reset on the root port.
+ * For other PEs, we always have hot reset on the PE primary bus.
*
- * We always reset the direct upstream bridge of the PE. If the
- * direct upstream bridge isn't root bridge, we always take hot
- * reset no matter what option (fundamental or hot) is. Otherwise,
- * we should do the reset according to the required option.
+ * Here, we have different design to pHyp, which always clear the
+ * frozen state during PE reset. However, the good idea here from
+ * benh is to keep frozen state before we get PE reset done completely
+ * (until BAR restore). With the frozen state, HW drops illegal IO
+ * or MMIO access, which can incur recrusive frozen PE during PE
+ * reset. The side effect is that EEH core has to clear the frozen
+ * state explicitly after BAR restore.
*/
if (pe->type & EEH_PE_PHB) {
ret = ioda_eeh_phb_reset(hose, option);
@@ -553,7 +582,7 @@
pci_is_root_bus(bus->parent))
ret = ioda_eeh_root_reset(hose, option);
else
- ret = ioda_eeh_bridge_reset(hose, bus->self, option);
+ ret = ioda_eeh_bridge_reset(bus->self, option);
}
return ret;
@@ -640,22 +669,6 @@
}
}
-static int ioda_eeh_get_phb_pe(struct pci_controller *hose,
- struct eeh_pe **pe)
-{
- struct eeh_pe *phb_pe;
-
- phb_pe = eeh_phb_pe_get(hose);
- if (!phb_pe) {
- pr_warning("%s Can't find PE for PHB#%d\n",
- __func__, hose->global_number);
- return -EEXIST;
- }
-
- *pe = phb_pe;
- return 0;
-}
-
static int ioda_eeh_get_pe(struct pci_controller *hose,
u16 pe_no, struct eeh_pe **pe)
{
@@ -663,7 +676,8 @@
struct eeh_dev dev;
/* Find the PHB PE */
- if (ioda_eeh_get_phb_pe(hose, &phb_pe))
+ phb_pe = eeh_phb_pe_get(hose);
+ if (!phb_pe)
return -EEXIST;
/* Find the PE according to PE# */
@@ -691,6 +705,7 @@
{
struct pci_controller *hose;
struct pnv_phb *phb;
+ struct eeh_pe *phb_pe;
u64 frozen_pe_no;
u16 err_type, severity;
long rc;
@@ -707,10 +722,12 @@
list_for_each_entry(hose, &hose_list, list_node) {
/*
* If the subordinate PCI buses of the PHB has been
- * removed, we needn't take care of it any more.
+ * removed or is exactly under error recovery, we
+ * needn't take care of it any more.
*/
phb = hose->private_data;
- if (phb->eeh_state & PNV_EEH_STATE_REMOVED)
+ phb_pe = eeh_phb_pe_get(hose);
+ if (!phb_pe || (phb_pe->state & EEH_PE_ISOLATED))
continue;
rc = opal_pci_next_error(phb->opal_id,
@@ -743,12 +760,6 @@
switch (err_type) {
case OPAL_EEH_IOC_ERROR:
if (severity == OPAL_EEH_SEV_IOC_DEAD) {
- list_for_each_entry(hose, &hose_list,
- list_node) {
- phb = hose->private_data;
- phb->eeh_state |= PNV_EEH_STATE_REMOVED;
- }
-
pr_err("EEH: dead IOC detected\n");
ret = EEH_NEXT_ERR_DEAD_IOC;
} else if (severity == OPAL_EEH_SEV_INF) {
@@ -761,17 +772,12 @@
break;
case OPAL_EEH_PHB_ERROR:
if (severity == OPAL_EEH_SEV_PHB_DEAD) {
- if (ioda_eeh_get_phb_pe(hose, pe))
- break;
-
+ *pe = phb_pe;
pr_err("EEH: dead PHB#%x detected\n",
hose->global_number);
- phb->eeh_state |= PNV_EEH_STATE_REMOVED;
ret = EEH_NEXT_ERR_DEAD_PHB;
} else if (severity == OPAL_EEH_SEV_PHB_FENCED) {
- if (ioda_eeh_get_phb_pe(hose, pe))
- break;
-
+ *pe = phb_pe;
pr_err("EEH: fenced PHB#%x detected\n",
hose->global_number);
ret = EEH_NEXT_ERR_FENCED_PHB;
@@ -789,17 +795,21 @@
* If we can't find the corresponding PE, the
* PEEV / PEST would be messy. So we force an
* fenced PHB so that it can be recovered.
+ *
+ * If the PE has been marked as isolated, that
+ * should have been removed permanently or in
+ * progress with recovery. We needn't report
+ * it again.
*/
if (ioda_eeh_get_pe(hose, frozen_pe_no, pe)) {
- if (!ioda_eeh_get_phb_pe(hose, pe)) {
- pr_err("EEH: Escalated fenced PHB#%x "
- "detected for PE#%llx\n",
- hose->global_number,
- frozen_pe_no);
- ret = EEH_NEXT_ERR_FENCED_PHB;
- } else {
- ret = EEH_NEXT_ERR_NONE;
- }
+ *pe = phb_pe;
+ pr_err("EEH: Escalated fenced PHB#%x "
+ "detected for PE#%llx\n",
+ hose->global_number,
+ frozen_pe_no);
+ ret = EEH_NEXT_ERR_FENCED_PHB;
+ } else if ((*pe)->state & EEH_PE_ISOLATED) {
+ ret = EEH_NEXT_ERR_NONE;
} else {
pr_err("EEH: Frozen PE#%x on PHB#%x detected\n",
(*pe)->addr, (*pe)->phb->global_number);
diff --git a/arch/powerpc/platforms/powernv/eeh-powernv.c b/arch/powerpc/platforms/powernv/eeh-powernv.c
index a59788e..56a206f 100644
--- a/arch/powerpc/platforms/powernv/eeh-powernv.c
+++ b/arch/powerpc/platforms/powernv/eeh-powernv.c
@@ -126,6 +126,7 @@
edev->mode &= 0xFFFFFF00;
if (dev->hdr_type == PCI_HEADER_TYPE_BRIDGE)
edev->mode |= EEH_DEV_BRIDGE;
+ edev->pcix_cap = pci_find_capability(dev, PCI_CAP_ID_PCIX);
if (pci_is_pcie(dev)) {
edev->pcie_cap = pci_pcie_cap(dev);
@@ -133,6 +134,9 @@
edev->mode |= EEH_DEV_ROOT_PORT;
else if (pci_pcie_type(dev) == PCI_EXP_TYPE_DOWNSTREAM)
edev->mode |= EEH_DEV_DS_PORT;
+
+ edev->aer_cap = pci_find_ext_capability(dev,
+ PCI_EXT_CAP_ID_ERR);
}
edev->config_addr = ((dev->bus->number << 8) | dev->devfn);
diff --git a/arch/powerpc/platforms/powernv/opal-flash.c b/arch/powerpc/platforms/powernv/opal-flash.c
index dc487ff..5c21d9c 100644
--- a/arch/powerpc/platforms/powernv/opal-flash.c
+++ b/arch/powerpc/platforms/powernv/opal-flash.c
@@ -20,6 +20,7 @@
#include <linux/mm.h>
#include <linux/vmalloc.h>
#include <linux/pagemap.h>
+#include <linux/delay.h>
#include <asm/opal.h>
@@ -130,7 +131,8 @@
{
long ret;
void *buf = validate_flash_data.buf;
- __be32 size, result;
+ __be32 size = cpu_to_be32(validate_flash_data.buf_size);
+ __be32 result;
ret = opal_validate_flash(__pa(buf), &size, &result);
@@ -290,11 +292,6 @@
/* First entry address */
addr = __pa(list);
- pr_alert("FLASH: Image is %u bytes\n", image_data.size);
- pr_alert("FLASH: Image update requested\n");
- pr_alert("FLASH: Image will be updated during system reboot\n");
- pr_alert("FLASH: This will take several minutes. Do not power off!\n");
-
flash:
rc = opal_update_flash(addr);
@@ -302,6 +299,47 @@
return rc;
}
+/* Return CPUs to OPAL before starting FW update */
+static void flash_return_cpu(void *info)
+{
+ int cpu = smp_processor_id();
+
+ if (!cpu_online(cpu))
+ return;
+
+ /* Disable IRQ */
+ hard_irq_disable();
+
+ /* Return the CPU to OPAL */
+ opal_return_cpu();
+}
+
+/* This gets called just before system reboots */
+void opal_flash_term_callback(void)
+{
+ struct cpumask mask;
+
+ if (update_flash_data.status != FLASH_IMG_READY)
+ return;
+
+ pr_alert("FLASH: Flashing new firmware\n");
+ pr_alert("FLASH: Image is %u bytes\n", image_data.size);
+ pr_alert("FLASH: Performing flash and reboot/shutdown\n");
+ pr_alert("FLASH: This will take several minutes. Do not power off!\n");
+
+ /* Small delay to help getting the above message out */
+ msleep(500);
+
+ /* Return secondary CPUs to firmware */
+ cpumask_copy(&mask, cpu_online_mask);
+ cpumask_clear_cpu(smp_processor_id(), &mask);
+ if (!cpumask_empty(&mask))
+ smp_call_function_many(&mask,
+ flash_return_cpu, NULL, false);
+ /* Hard disable interrupts */
+ hard_irq_disable();
+}
+
/*
* Show candidate image status
*/
diff --git a/arch/powerpc/platforms/powernv/opal-lpc.c b/arch/powerpc/platforms/powernv/opal-lpc.c
index 79d83ca..f04b4d8 100644
--- a/arch/powerpc/platforms/powernv/opal-lpc.c
+++ b/arch/powerpc/platforms/powernv/opal-lpc.c
@@ -12,12 +12,17 @@
#include <linux/kernel.h>
#include <linux/of.h>
#include <linux/bug.h>
+#include <linux/debugfs.h>
+#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/machdep.h>
#include <asm/firmware.h>
#include <asm/xics.h>
#include <asm/opal.h>
#include <asm/prom.h>
+#include <asm/uaccess.h>
+#include <asm/debug.h>
static int opal_lpc_chip_id = -1;
@@ -176,6 +181,152 @@
.outsl = opal_lpc_outsl,
};
+#ifdef CONFIG_DEBUG_FS
+struct lpc_debugfs_entry {
+ enum OpalLPCAddressType lpc_type;
+};
+
+static ssize_t lpc_debug_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *ppos)
+{
+ struct lpc_debugfs_entry *lpc = filp->private_data;
+ u32 data, pos, len, todo;
+ int rc;
+
+ if (!access_ok(VERIFY_WRITE, ubuf, count))
+ return -EFAULT;
+
+ todo = count;
+ while (todo) {
+ pos = *ppos;
+
+ /*
+ * Select access size based on count and alignment and
+ * access type. IO and MEM only support byte acceses,
+ * FW supports all 3.
+ */
+ len = 1;
+ if (lpc->lpc_type == OPAL_LPC_FW) {
+ if (todo > 3 && (pos & 3) == 0)
+ len = 4;
+ else if (todo > 1 && (pos & 1) == 0)
+ len = 2;
+ }
+ rc = opal_lpc_read(opal_lpc_chip_id, lpc->lpc_type, pos,
+ &data, len);
+ if (rc)
+ return -ENXIO;
+ switch(len) {
+ case 4:
+ rc = __put_user((u32)data, (u32 __user *)ubuf);
+ break;
+ case 2:
+ rc = __put_user((u16)data, (u16 __user *)ubuf);
+ break;
+ default:
+ rc = __put_user((u8)data, (u8 __user *)ubuf);
+ break;
+ }
+ if (rc)
+ return -EFAULT;
+ *ppos += len;
+ ubuf += len;
+ todo -= len;
+ }
+
+ return count;
+}
+
+static ssize_t lpc_debug_write(struct file *filp, const char __user *ubuf,
+ size_t count, loff_t *ppos)
+{
+ struct lpc_debugfs_entry *lpc = filp->private_data;
+ u32 data, pos, len, todo;
+ int rc;
+
+ if (!access_ok(VERIFY_READ, ubuf, count))
+ return -EFAULT;
+
+ todo = count;
+ while (todo) {
+ pos = *ppos;
+
+ /*
+ * Select access size based on count and alignment and
+ * access type. IO and MEM only support byte acceses,
+ * FW supports all 3.
+ */
+ len = 1;
+ if (lpc->lpc_type == OPAL_LPC_FW) {
+ if (todo > 3 && (pos & 3) == 0)
+ len = 4;
+ else if (todo > 1 && (pos & 1) == 0)
+ len = 2;
+ }
+ switch(len) {
+ case 4:
+ rc = __get_user(data, (u32 __user *)ubuf);
+ break;
+ case 2:
+ rc = __get_user(data, (u16 __user *)ubuf);
+ break;
+ default:
+ rc = __get_user(data, (u8 __user *)ubuf);
+ break;
+ }
+ if (rc)
+ return -EFAULT;
+
+ rc = opal_lpc_write(opal_lpc_chip_id, lpc->lpc_type, pos,
+ data, len);
+ if (rc)
+ return -ENXIO;
+ *ppos += len;
+ ubuf += len;
+ todo -= len;
+ }
+
+ return count;
+}
+
+static const struct file_operations lpc_fops = {
+ .read = lpc_debug_read,
+ .write = lpc_debug_write,
+ .open = simple_open,
+ .llseek = default_llseek,
+};
+
+static int opal_lpc_debugfs_create_type(struct dentry *folder,
+ const char *fname,
+ enum OpalLPCAddressType type)
+{
+ struct lpc_debugfs_entry *entry;
+ entry = kzalloc(sizeof(*entry), GFP_KERNEL);
+ if (!entry)
+ return -ENOMEM;
+ entry->lpc_type = type;
+ debugfs_create_file(fname, 0600, folder, entry, &lpc_fops);
+ return 0;
+}
+
+static int opal_lpc_init_debugfs(void)
+{
+ struct dentry *root;
+ int rc = 0;
+
+ if (opal_lpc_chip_id < 0)
+ return -ENODEV;
+
+ root = debugfs_create_dir("lpc", powerpc_debugfs_root);
+
+ rc |= opal_lpc_debugfs_create_type(root, "io", OPAL_LPC_IO);
+ rc |= opal_lpc_debugfs_create_type(root, "mem", OPAL_LPC_MEM);
+ rc |= opal_lpc_debugfs_create_type(root, "fw", OPAL_LPC_FW);
+ return rc;
+}
+device_initcall(opal_lpc_init_debugfs);
+#endif /* CONFIG_DEBUG_FS */
+
void opal_lpc_init(void)
{
struct device_node *np;
diff --git a/arch/powerpc/platforms/powernv/opal-memory-errors.c b/arch/powerpc/platforms/powernv/opal-memory-errors.c
index ec413223..b17a34b 100644
--- a/arch/powerpc/platforms/powernv/opal-memory-errors.c
+++ b/arch/powerpc/platforms/powernv/opal-memory-errors.c
@@ -47,12 +47,12 @@
__func__, merr_evt->type);
switch (merr_evt->type) {
case OPAL_MEM_ERR_TYPE_RESILIENCE:
- paddr_start = merr_evt->u.resilience.physical_address_start;
- paddr_end = merr_evt->u.resilience.physical_address_end;
+ paddr_start = be64_to_cpu(merr_evt->u.resilience.physical_address_start);
+ paddr_end = be64_to_cpu(merr_evt->u.resilience.physical_address_end);
break;
case OPAL_MEM_ERR_TYPE_DYN_DALLOC:
- paddr_start = merr_evt->u.dyn_dealloc.physical_address_start;
- paddr_end = merr_evt->u.dyn_dealloc.physical_address_end;
+ paddr_start = be64_to_cpu(merr_evt->u.dyn_dealloc.physical_address_start);
+ paddr_end = be64_to_cpu(merr_evt->u.dyn_dealloc.physical_address_end);
break;
default:
return;
diff --git a/arch/powerpc/platforms/powernv/opal-takeover.S b/arch/powerpc/platforms/powernv/opal-takeover.S
index 3cd2628..11a3169 100644
--- a/arch/powerpc/platforms/powernv/opal-takeover.S
+++ b/arch/powerpc/platforms/powernv/opal-takeover.S
@@ -21,11 +21,13 @@
_GLOBAL(opal_query_takeover)
mfcr r0
stw r0,8(r1)
+ stdu r1,-STACKFRAMESIZE(r1)
std r3,STK_PARAM(R3)(r1)
std r4,STK_PARAM(R4)(r1)
li r3,H_HAL_TAKEOVER
li r4,H_HAL_TAKEOVER_QUERY_MAGIC
HVSC
+ addi r1,r1,STACKFRAMESIZE
ld r10,STK_PARAM(R3)(r1)
std r4,0(r10)
ld r10,STK_PARAM(R4)(r1)
diff --git a/arch/powerpc/platforms/powernv/opal-wrappers.S b/arch/powerpc/platforms/powernv/opal-wrappers.S
index f531ffe..4abbff2 100644
--- a/arch/powerpc/platforms/powernv/opal-wrappers.S
+++ b/arch/powerpc/platforms/powernv/opal-wrappers.S
@@ -32,7 +32,7 @@
std r12,PACASAVEDMSR(r13); \
andc r12,r12,r0; \
mtmsrd r12,1; \
- LOAD_REG_ADDR(r0,.opal_return); \
+ LOAD_REG_ADDR(r0,opal_return); \
mtlr r0; \
li r0,MSR_DR|MSR_IR|MSR_LE;\
andc r12,r12,r0; \
@@ -44,7 +44,7 @@
mtspr SPRN_HSRR0,r12; \
hrfid
-_STATIC(opal_return)
+opal_return:
/*
* Fixup endian on OPAL return... we should be able to simplify
* this by instead converting the below trampoline to a set of
@@ -124,6 +124,7 @@
OPAL_CALL(opal_lpc_read, OPAL_LPC_READ);
OPAL_CALL(opal_lpc_write, OPAL_LPC_WRITE);
OPAL_CALL(opal_return_cpu, OPAL_RETURN_CPU);
+OPAL_CALL(opal_reinit_cpus, OPAL_REINIT_CPUS);
OPAL_CALL(opal_read_elog, OPAL_ELOG_READ);
OPAL_CALL(opal_send_ack_elog, OPAL_ELOG_ACK);
OPAL_CALL(opal_get_elog_size, OPAL_ELOG_SIZE);
diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
index f343183..1999756 100644
--- a/arch/powerpc/platforms/powernv/opal.c
+++ b/arch/powerpc/platforms/powernv/opal.c
@@ -57,6 +57,21 @@
static uint64_t last_notified_mask = 0x0ul;
static atomic_t opal_notifier_hold = ATOMIC_INIT(0);
+static void opal_reinit_cores(void)
+{
+ /* Do the actual re-init, This will clobber all FPRs, VRs, etc...
+ *
+ * It will preserve non volatile GPRs and HSPRG0/1. It will
+ * also restore HIDs and other SPRs to their original value
+ * but it might clobber a bunch.
+ */
+#ifdef __BIG_ENDIAN__
+ opal_reinit_cpus(OPAL_REINIT_CPUS_HILE_BE);
+#else
+ opal_reinit_cpus(OPAL_REINIT_CPUS_HILE_LE);
+#endif
+}
+
int __init early_init_dt_scan_opal(unsigned long node,
const char *uname, int depth, void *data)
{
@@ -96,6 +111,13 @@
printk("OPAL V1 detected !\n");
}
+ /* Reinit all cores with the right endian */
+ opal_reinit_cores();
+
+ /* Restore some bits */
+ if (cur_cpu_spec->cpu_restore)
+ cur_cpu_spec->cpu_restore();
+
return 1;
}
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 98824aa..de19ede 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -13,6 +13,7 @@
#include <linux/kernel.h>
#include <linux/pci.h>
+#include <linux/crash_dump.h>
#include <linux/debugfs.h>
#include <linux/delay.h>
#include <linux/string.h>
@@ -663,15 +664,15 @@
* errors, and on the first pass the data will be a relative
* bus number, print that out instead.
*/
- tbl->it_busno = 0;
pe->tce_inval_reg_phys = be64_to_cpup(swinvp);
tbl->it_index = (unsigned long)ioremap(pe->tce_inval_reg_phys,
8);
- tbl->it_type = TCE_PCI_SWINV_CREATE | TCE_PCI_SWINV_FREE |
- TCE_PCI_SWINV_PAIR;
+ tbl->it_type |= (TCE_PCI_SWINV_CREATE |
+ TCE_PCI_SWINV_FREE |
+ TCE_PCI_SWINV_PAIR);
}
iommu_init_table(tbl, phb->hose->node);
- iommu_register_group(tbl, pci_domain_nr(pe->pbus), pe->pe_number);
+ iommu_register_group(tbl, phb->hose->global_number, pe->pe_number);
if (pe->pdev)
set_iommu_table_base_and_group(&pe->pdev->dev, tbl);
@@ -793,14 +794,13 @@
* errors, and on the first pass the data will be a relative
* bus number, print that out instead.
*/
- tbl->it_busno = 0;
pe->tce_inval_reg_phys = be64_to_cpup(swinvp);
tbl->it_index = (unsigned long)ioremap(pe->tce_inval_reg_phys,
8);
- tbl->it_type = TCE_PCI_SWINV_CREATE | TCE_PCI_SWINV_FREE;
+ tbl->it_type |= (TCE_PCI_SWINV_CREATE | TCE_PCI_SWINV_FREE);
}
iommu_init_table(tbl, phb->hose->node);
- iommu_register_group(tbl, pci_domain_nr(pe->pbus), pe->pe_number);
+ iommu_register_group(tbl, phb->hose->global_number, pe->pe_number);
if (pe->pdev)
set_iommu_table_base_and_group(&pe->pdev->dev, tbl);
@@ -1386,12 +1386,24 @@
ppc_md.pcibios_fixup = pnv_pci_ioda_fixup;
ppc_md.pcibios_enable_device_hook = pnv_pci_enable_device_hook;
ppc_md.pcibios_window_alignment = pnv_pci_window_alignment;
+ ppc_md.pcibios_reset_secondary_bus = pnv_pci_reset_secondary_bus;
pci_add_flags(PCI_REASSIGN_ALL_RSRC);
/* Reset IODA tables to a clean state */
rc = opal_pci_reset(phb_id, OPAL_PCI_IODA_TABLE_RESET, OPAL_ASSERT_RESET);
if (rc)
pr_warning(" OPAL Error %ld performing IODA table reset !\n", rc);
+
+ /* If we're running in kdump kerenl, the previous kerenl never
+ * shutdown PCI devices correctly. We already got IODA table
+ * cleaned out. So we have to issue PHB reset to stop all PCI
+ * transactions from previous kerenl.
+ */
+ if (is_kdump_kernel()) {
+ pr_info(" Issue PHB reset ...\n");
+ ioda_eeh_phb_reset(hose, EEH_RESET_FUNDAMENTAL);
+ ioda_eeh_phb_reset(hose, OPAL_DEASSERT_RESET);
+ }
}
void __init pnv_pci_init_ioda2_phb(struct device_node *np)
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index 8518817..eefbfcc 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -131,65 +131,60 @@
int i;
data = (struct OpalIoP7IOCPhbErrorData *)common;
- pr_info("P7IOC PHB#%d Diag-data (Version: %d)\n\n",
+ pr_info("P7IOC PHB#%d Diag-data (Version: %d)\n",
hose->global_number, common->version);
if (data->brdgCtl)
- pr_info(" brdgCtl: %08x\n",
+ pr_info("brdgCtl: %08x\n",
data->brdgCtl);
if (data->portStatusReg || data->rootCmplxStatus ||
data->busAgentStatus)
- pr_info(" UtlSts: %08x %08x %08x\n",
+ pr_info("UtlSts: %08x %08x %08x\n",
data->portStatusReg, data->rootCmplxStatus,
data->busAgentStatus);
if (data->deviceStatus || data->slotStatus ||
data->linkStatus || data->devCmdStatus ||
data->devSecStatus)
- pr_info(" RootSts: %08x %08x %08x %08x %08x\n",
+ pr_info("RootSts: %08x %08x %08x %08x %08x\n",
data->deviceStatus, data->slotStatus,
data->linkStatus, data->devCmdStatus,
data->devSecStatus);
if (data->rootErrorStatus || data->uncorrErrorStatus ||
data->corrErrorStatus)
- pr_info(" RootErrSts: %08x %08x %08x\n",
+ pr_info("RootErrSts: %08x %08x %08x\n",
data->rootErrorStatus, data->uncorrErrorStatus,
data->corrErrorStatus);
if (data->tlpHdr1 || data->tlpHdr2 ||
data->tlpHdr3 || data->tlpHdr4)
- pr_info(" RootErrLog: %08x %08x %08x %08x\n",
+ pr_info("RootErrLog: %08x %08x %08x %08x\n",
data->tlpHdr1, data->tlpHdr2,
data->tlpHdr3, data->tlpHdr4);
if (data->sourceId || data->errorClass ||
data->correlator)
- pr_info(" RootErrLog1: %08x %016llx %016llx\n",
+ pr_info("RootErrLog1: %08x %016llx %016llx\n",
data->sourceId, data->errorClass,
data->correlator);
if (data->p7iocPlssr || data->p7iocCsr)
- pr_info(" PhbSts: %016llx %016llx\n",
+ pr_info("PhbSts: %016llx %016llx\n",
data->p7iocPlssr, data->p7iocCsr);
- if (data->lemFir || data->lemErrorMask ||
- data->lemWOF)
- pr_info(" Lem: %016llx %016llx %016llx\n",
+ if (data->lemFir)
+ pr_info("Lem: %016llx %016llx %016llx\n",
data->lemFir, data->lemErrorMask,
data->lemWOF);
- if (data->phbErrorStatus || data->phbFirstErrorStatus ||
- data->phbErrorLog0 || data->phbErrorLog1)
- pr_info(" PhbErr: %016llx %016llx %016llx %016llx\n",
+ if (data->phbErrorStatus)
+ pr_info("PhbErr: %016llx %016llx %016llx %016llx\n",
data->phbErrorStatus, data->phbFirstErrorStatus,
data->phbErrorLog0, data->phbErrorLog1);
- if (data->mmioErrorStatus || data->mmioFirstErrorStatus ||
- data->mmioErrorLog0 || data->mmioErrorLog1)
- pr_info(" OutErr: %016llx %016llx %016llx %016llx\n",
+ if (data->mmioErrorStatus)
+ pr_info("OutErr: %016llx %016llx %016llx %016llx\n",
data->mmioErrorStatus, data->mmioFirstErrorStatus,
data->mmioErrorLog0, data->mmioErrorLog1);
- if (data->dma0ErrorStatus || data->dma0FirstErrorStatus ||
- data->dma0ErrorLog0 || data->dma0ErrorLog1)
- pr_info(" InAErr: %016llx %016llx %016llx %016llx\n",
+ if (data->dma0ErrorStatus)
+ pr_info("InAErr: %016llx %016llx %016llx %016llx\n",
data->dma0ErrorStatus, data->dma0FirstErrorStatus,
data->dma0ErrorLog0, data->dma0ErrorLog1);
- if (data->dma1ErrorStatus || data->dma1FirstErrorStatus ||
- data->dma1ErrorLog0 || data->dma1ErrorLog1)
- pr_info(" InBErr: %016llx %016llx %016llx %016llx\n",
+ if (data->dma1ErrorStatus)
+ pr_info("InBErr: %016llx %016llx %016llx %016llx\n",
data->dma1ErrorStatus, data->dma1FirstErrorStatus,
data->dma1ErrorLog0, data->dma1ErrorLog1);
@@ -198,7 +193,7 @@
(data->pestB[i] >> 63) == 0)
continue;
- pr_info(" PE[%3d] A/B: %016llx %016llx\n",
+ pr_info("PE[%3d] A/B: %016llx %016llx\n",
i, data->pestA[i], data->pestB[i]);
}
}
@@ -210,69 +205,63 @@
int i;
data = (struct OpalIoPhb3ErrorData*)common;
- pr_info("PHB3 PHB#%d Diag-data (Version: %d)\n\n",
+ pr_info("PHB3 PHB#%d Diag-data (Version: %d)\n",
hose->global_number, common->version);
if (data->brdgCtl)
- pr_info(" brdgCtl: %08x\n",
+ pr_info("brdgCtl: %08x\n",
data->brdgCtl);
if (data->portStatusReg || data->rootCmplxStatus ||
data->busAgentStatus)
- pr_info(" UtlSts: %08x %08x %08x\n",
+ pr_info("UtlSts: %08x %08x %08x\n",
data->portStatusReg, data->rootCmplxStatus,
data->busAgentStatus);
if (data->deviceStatus || data->slotStatus ||
data->linkStatus || data->devCmdStatus ||
data->devSecStatus)
- pr_info(" RootSts: %08x %08x %08x %08x %08x\n",
+ pr_info("RootSts: %08x %08x %08x %08x %08x\n",
data->deviceStatus, data->slotStatus,
data->linkStatus, data->devCmdStatus,
data->devSecStatus);
if (data->rootErrorStatus || data->uncorrErrorStatus ||
data->corrErrorStatus)
- pr_info(" RootErrSts: %08x %08x %08x\n",
+ pr_info("RootErrSts: %08x %08x %08x\n",
data->rootErrorStatus, data->uncorrErrorStatus,
data->corrErrorStatus);
if (data->tlpHdr1 || data->tlpHdr2 ||
data->tlpHdr3 || data->tlpHdr4)
- pr_info(" RootErrLog: %08x %08x %08x %08x\n",
+ pr_info("RootErrLog: %08x %08x %08x %08x\n",
data->tlpHdr1, data->tlpHdr2,
data->tlpHdr3, data->tlpHdr4);
if (data->sourceId || data->errorClass ||
data->correlator)
- pr_info(" RootErrLog1: %08x %016llx %016llx\n",
+ pr_info("RootErrLog1: %08x %016llx %016llx\n",
data->sourceId, data->errorClass,
data->correlator);
- if (data->nFir || data->nFirMask ||
- data->nFirWOF)
- pr_info(" nFir: %016llx %016llx %016llx\n",
+ if (data->nFir)
+ pr_info("nFir: %016llx %016llx %016llx\n",
data->nFir, data->nFirMask,
data->nFirWOF);
if (data->phbPlssr || data->phbCsr)
- pr_info(" PhbSts: %016llx %016llx\n",
+ pr_info("PhbSts: %016llx %016llx\n",
data->phbPlssr, data->phbCsr);
- if (data->lemFir || data->lemErrorMask ||
- data->lemWOF)
- pr_info(" Lem: %016llx %016llx %016llx\n",
+ if (data->lemFir)
+ pr_info("Lem: %016llx %016llx %016llx\n",
data->lemFir, data->lemErrorMask,
data->lemWOF);
- if (data->phbErrorStatus || data->phbFirstErrorStatus ||
- data->phbErrorLog0 || data->phbErrorLog1)
- pr_info(" PhbErr: %016llx %016llx %016llx %016llx\n",
+ if (data->phbErrorStatus)
+ pr_info("PhbErr: %016llx %016llx %016llx %016llx\n",
data->phbErrorStatus, data->phbFirstErrorStatus,
data->phbErrorLog0, data->phbErrorLog1);
- if (data->mmioErrorStatus || data->mmioFirstErrorStatus ||
- data->mmioErrorLog0 || data->mmioErrorLog1)
- pr_info(" OutErr: %016llx %016llx %016llx %016llx\n",
+ if (data->mmioErrorStatus)
+ pr_info("OutErr: %016llx %016llx %016llx %016llx\n",
data->mmioErrorStatus, data->mmioFirstErrorStatus,
data->mmioErrorLog0, data->mmioErrorLog1);
- if (data->dma0ErrorStatus || data->dma0FirstErrorStatus ||
- data->dma0ErrorLog0 || data->dma0ErrorLog1)
- pr_info(" InAErr: %016llx %016llx %016llx %016llx\n",
+ if (data->dma0ErrorStatus)
+ pr_info("InAErr: %016llx %016llx %016llx %016llx\n",
data->dma0ErrorStatus, data->dma0FirstErrorStatus,
data->dma0ErrorLog0, data->dma0ErrorLog1);
- if (data->dma1ErrorStatus || data->dma1FirstErrorStatus ||
- data->dma1ErrorLog0 || data->dma1ErrorLog1)
- pr_info(" InBErr: %016llx %016llx %016llx %016llx\n",
+ if (data->dma1ErrorStatus)
+ pr_info("InBErr: %016llx %016llx %016llx %016llx\n",
data->dma1ErrorStatus, data->dma1FirstErrorStatus,
data->dma1ErrorLog0, data->dma1ErrorLog1);
@@ -281,7 +270,7 @@
(data->pestB[i] >> 63) == 0)
continue;
- pr_info(" PE[%3d] A/B: %016llx %016llx\n",
+ pr_info("PE[%3d] A/B: %016llx %016llx\n",
i, data->pestA[i], data->pestB[i]);
}
}
@@ -384,9 +373,6 @@
struct pci_dn *pdn = PCI_DN(dn);
struct pnv_phb *phb = pdn->phb->private_data;
u32 bdfn = (pdn->busno << 8) | pdn->devfn;
-#ifdef CONFIG_EEH
- struct eeh_pe *phb_pe = NULL;
-#endif
s64 rc;
switch (size) {
@@ -412,31 +398,9 @@
default:
return PCIBIOS_FUNC_NOT_SUPPORTED;
}
+
cfg_dbg("%s: bus: %x devfn: %x +%x/%x -> %08x\n",
__func__, pdn->busno, pdn->devfn, where, size, *val);
-
- /*
- * Check if the specified PE has been put into frozen
- * state. On the other hand, we needn't do that while
- * the PHB has been put into frozen state because of
- * PHB-fatal errors.
- */
-#ifdef CONFIG_EEH
- phb_pe = eeh_phb_pe_get(pdn->phb);
- if (phb_pe && (phb_pe->state & EEH_PE_ISOLATED))
- return PCIBIOS_SUCCESSFUL;
-
- if (phb->eeh_state & PNV_EEH_STATE_ENABLED) {
- if (*val == EEH_IO_ERROR_VALUE(size) &&
- eeh_dev_check_failure(of_node_to_eeh_dev(dn)))
- return PCIBIOS_DEVICE_NOT_FOUND;
- } else {
- pnv_pci_config_check_eeh(phb, dn);
- }
-#else
- pnv_pci_config_check_eeh(phb, dn);
-#endif
-
return PCIBIOS_SUCCESSFUL;
}
@@ -463,33 +427,74 @@
return PCIBIOS_FUNC_NOT_SUPPORTED;
}
- /* Check if the PHB got frozen due to an error (no response) */
-#ifdef CONFIG_EEH
- if (!(phb->eeh_state & PNV_EEH_STATE_ENABLED))
- pnv_pci_config_check_eeh(phb, dn);
-#else
- pnv_pci_config_check_eeh(phb, dn);
-#endif
-
return PCIBIOS_SUCCESSFUL;
}
+#if CONFIG_EEH
+static bool pnv_pci_cfg_check(struct pci_controller *hose,
+ struct device_node *dn)
+{
+ struct eeh_dev *edev = NULL;
+ struct pnv_phb *phb = hose->private_data;
+
+ /* EEH not enabled ? */
+ if (!(phb->flags & PNV_PHB_FLAG_EEH))
+ return true;
+
+ /* PE reset or device removed ? */
+ edev = of_node_to_eeh_dev(dn);
+ if (edev) {
+ if (edev->pe &&
+ (edev->pe->state & EEH_PE_RESET))
+ return false;
+
+ if (edev->mode & EEH_DEV_REMOVED)
+ return false;
+ }
+
+ return true;
+}
+#else
+static inline pnv_pci_cfg_check(struct pci_controller *hose,
+ struct device_node *dn)
+{
+ return true;
+}
+#endif /* CONFIG_EEH */
+
static int pnv_pci_read_config(struct pci_bus *bus,
unsigned int devfn,
int where, int size, u32 *val)
{
struct device_node *dn, *busdn = pci_bus_to_OF_node(bus);
struct pci_dn *pdn;
-
- for (dn = busdn->child; dn; dn = dn->sibling) {
- pdn = PCI_DN(dn);
- if (pdn && pdn->devfn == devfn)
- return pnv_pci_cfg_read(dn, where, size, val);
- }
+ struct pnv_phb *phb;
+ bool found = false;
+ int ret;
*val = 0xFFFFFFFF;
- return PCIBIOS_DEVICE_NOT_FOUND;
+ for (dn = busdn->child; dn; dn = dn->sibling) {
+ pdn = PCI_DN(dn);
+ if (pdn && pdn->devfn == devfn) {
+ phb = pdn->phb->private_data;
+ found = true;
+ break;
+ }
+ }
+ if (!found || !pnv_pci_cfg_check(pdn->phb, dn))
+ return PCIBIOS_DEVICE_NOT_FOUND;
+
+ ret = pnv_pci_cfg_read(dn, where, size, val);
+ if (phb->flags & PNV_PHB_FLAG_EEH) {
+ if (*val == EEH_IO_ERROR_VALUE(size) &&
+ eeh_dev_check_failure(of_node_to_eeh_dev(dn)))
+ return PCIBIOS_DEVICE_NOT_FOUND;
+ } else {
+ pnv_pci_config_check_eeh(phb, dn);
+ }
+
+ return ret;
}
static int pnv_pci_write_config(struct pci_bus *bus,
@@ -498,14 +503,27 @@
{
struct device_node *dn, *busdn = pci_bus_to_OF_node(bus);
struct pci_dn *pdn;
+ struct pnv_phb *phb;
+ bool found = false;
+ int ret;
for (dn = busdn->child; dn; dn = dn->sibling) {
pdn = PCI_DN(dn);
- if (pdn && pdn->devfn == devfn)
- return pnv_pci_cfg_write(dn, where, size, val);
+ if (pdn && pdn->devfn == devfn) {
+ phb = pdn->phb->private_data;
+ found = true;
+ break;
+ }
}
- return PCIBIOS_DEVICE_NOT_FOUND;
+ if (!found || !pnv_pci_cfg_check(pdn->phb, dn))
+ return PCIBIOS_DEVICE_NOT_FOUND;
+
+ ret = pnv_pci_cfg_write(dn, where, size, val);
+ if (!(phb->flags & PNV_PHB_FLAG_EEH))
+ pnv_pci_config_check_eeh(phb, dn);
+
+ return ret;
}
struct pci_ops pnv_pci_ops = {
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index cde1694..676232c 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -81,28 +81,27 @@
int (*configure_bridge)(struct eeh_pe *pe);
int (*next_error)(struct eeh_pe **pe);
};
-
-#define PNV_EEH_STATE_ENABLED (1 << 0) /* EEH enabled */
-#define PNV_EEH_STATE_REMOVED (1 << 1) /* PHB removed */
-
#endif /* CONFIG_EEH */
+#define PNV_PHB_FLAG_EEH (1 << 0)
+
struct pnv_phb {
struct pci_controller *hose;
enum pnv_phb_type type;
enum pnv_phb_model model;
u64 hub_id;
u64 opal_id;
+ int flags;
void __iomem *regs;
int initialized;
spinlock_t lock;
#ifdef CONFIG_EEH
struct pnv_eeh_ops *eeh_ops;
- int eeh_state;
#endif
#ifdef CONFIG_DEBUG_FS
+ int has_dbgfs;
struct dentry *dbgfs;
#endif
@@ -205,5 +204,7 @@
extern void pnv_pci_init_ioda2_phb(struct device_node *np);
extern void pnv_pci_ioda_tce_invalidate(struct iommu_table *tbl,
__be64 *startp, __be64 *endp, bool rm);
+extern void pnv_pci_reset_secondary_bus(struct pci_dev *dev);
+extern int ioda_eeh_phb_reset(struct pci_controller *hose, int option);
#endif /* __POWERNV_PCI_H */
diff --git a/arch/powerpc/platforms/powernv/powernv.h b/arch/powerpc/platforms/powernv/powernv.h
index 0051e10..75501bf 100644
--- a/arch/powerpc/platforms/powernv/powernv.h
+++ b/arch/powerpc/platforms/powernv/powernv.h
@@ -25,4 +25,6 @@
extern void pnv_lpc_init(void);
+bool cpu_core_split_required(void);
+
#endif /* _POWERNV_H */
diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c
index 8723d32..8c16a5f 100644
--- a/arch/powerpc/platforms/powernv/setup.c
+++ b/arch/powerpc/platforms/powernv/setup.c
@@ -27,6 +27,7 @@
#include <linux/interrupt.h>
#include <linux/bug.h>
#include <linux/pci.h>
+#include <linux/cpufreq.h>
#include <asm/machdep.h>
#include <asm/firmware.h>
@@ -98,11 +99,32 @@
of_node_put(root);
}
+static void pnv_prepare_going_down(void)
+{
+ /*
+ * Disable all notifiers from OPAL, we can't
+ * service interrupts anymore anyway
+ */
+ opal_notifier_disable();
+
+ /* Soft disable interrupts */
+ local_irq_disable();
+
+ /*
+ * Return secondary CPUs to firwmare if a flash update
+ * is pending otherwise we will get all sort of error
+ * messages about CPU being stuck etc.. This will also
+ * have the side effect of hard disabling interrupts so
+ * past this point, the kernel is effectively dead.
+ */
+ opal_flash_term_callback();
+}
+
static void __noreturn pnv_restart(char *cmd)
{
long rc = OPAL_BUSY;
- opal_notifier_disable();
+ pnv_prepare_going_down();
while (rc == OPAL_BUSY || rc == OPAL_BUSY_EVENT) {
rc = opal_cec_reboot();
@@ -119,7 +141,7 @@
{
long rc = OPAL_BUSY;
- opal_notifier_disable();
+ pnv_prepare_going_down();
while (rc == OPAL_BUSY || rc == OPAL_BUSY_EVENT) {
rc = opal_cec_power_down(0);
@@ -222,6 +244,13 @@
}
#endif /* CONFIG_KEXEC */
+#ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
+static unsigned long pnv_memory_block_size(void)
+{
+ return 256UL * 1024 * 1024;
+}
+#endif
+
static void __init pnv_setup_machdep_opal(void)
{
ppc_md.get_boot_time = opal_get_boot_time;
@@ -269,6 +298,25 @@
return 1;
}
+/*
+ * Returns the cpu frequency for 'cpu' in Hz. This is used by
+ * /proc/cpuinfo
+ */
+unsigned long pnv_get_proc_freq(unsigned int cpu)
+{
+ unsigned long ret_freq;
+
+ ret_freq = cpufreq_quick_get(cpu) * 1000ul;
+
+ /*
+ * If the backend cpufreq driver does not exist,
+ * then fallback to old way of reporting the clockrate.
+ */
+ if (!ret_freq)
+ ret_freq = ppc_proc_freq;
+ return ret_freq;
+}
+
define_machine(powernv) {
.name = "PowerNV",
.probe = pnv_probe,
@@ -276,6 +324,7 @@
.setup_arch = pnv_setup_arch,
.init_IRQ = pnv_init_IRQ,
.show_cpuinfo = pnv_show_cpuinfo,
+ .get_proc_freq = pnv_get_proc_freq,
.progress = pnv_progress,
.machine_shutdown = pnv_shutdown,
.power_save = power7_idle,
@@ -284,4 +333,7 @@
#ifdef CONFIG_KEXEC
.kexec_cpu_down = pnv_kexec_cpu_down,
#endif
+#ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
+ .memory_block_size = pnv_memory_block_size,
+#endif
};
diff --git a/arch/powerpc/platforms/powernv/smp.c b/arch/powerpc/platforms/powernv/smp.c
index bf5fcd4..0062a43 100644
--- a/arch/powerpc/platforms/powernv/smp.c
+++ b/arch/powerpc/platforms/powernv/smp.c
@@ -31,6 +31,7 @@
#include <asm/xics.h>
#include <asm/opal.h>
#include <asm/runlatch.h>
+#include <asm/code-patching.h>
#include "powernv.h"
@@ -50,8 +51,8 @@
int pnv_smp_kick_cpu(int nr)
{
unsigned int pcpu = get_hard_smp_processor_id(nr);
- unsigned long start_here = __pa(*((unsigned long *)
- generic_secondary_smp_init));
+ unsigned long start_here =
+ __pa(ppc_function_entry(generic_secondary_smp_init));
long rc;
BUG_ON(nr < 0 || nr >= NR_CPUS);
@@ -158,17 +159,19 @@
mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) & ~(u64)LPCR_PECE1);
while (!generic_check_cpu_restart(cpu)) {
ppc64_runlatch_off();
- power7_nap();
+ power7_nap(1);
ppc64_runlatch_on();
- if (!generic_check_cpu_restart(cpu)) {
+
+ /* Reenable IRQs briefly to clear the IPI that woke us */
+ local_irq_enable();
+ local_irq_disable();
+ mb();
+
+ if (cpu_core_split_required())
+ continue;
+
+ if (!generic_check_cpu_restart(cpu))
DBG("CPU%d Unexpected exit while offline !\n", cpu);
- /* We may be getting an IPI, so we re-enable
- * interrupts to process it, it will be ignored
- * since we aren't online (hopefully)
- */
- local_irq_enable();
- local_irq_disable();
- }
}
mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) | LPCR_PECE1);
DBG("CPU%d coming online...\n", cpu);
diff --git a/arch/powerpc/platforms/powernv/subcore-asm.S b/arch/powerpc/platforms/powernv/subcore-asm.S
new file mode 100644
index 0000000..39bb24a
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/subcore-asm.S
@@ -0,0 +1,95 @@
+/*
+ * Copyright 2013, Michael (Ellerman|Neuling), IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <asm/asm-offsets.h>
+#include <asm/ppc_asm.h>
+#include <asm/reg.h>
+
+#include "subcore.h"
+
+
+_GLOBAL(split_core_secondary_loop)
+ /*
+ * r3 = u8 *state, used throughout the routine
+ * r4 = temp
+ * r5 = temp
+ * ..
+ * r12 = MSR
+ */
+ mfmsr r12
+
+ /* Disable interrupts so SRR0/1 don't get trashed */
+ li r4,0
+ ori r4,r4,MSR_EE|MSR_SE|MSR_BE|MSR_RI
+ andc r4,r12,r4
+ sync
+ mtmsrd r4
+
+ /* Switch to real mode and leave interrupts off */
+ li r5, MSR_IR|MSR_DR
+ andc r5, r4, r5
+
+ LOAD_REG_ADDR(r4, real_mode)
+
+ mtspr SPRN_SRR0,r4
+ mtspr SPRN_SRR1,r5
+ rfid
+ b . /* prevent speculative execution */
+
+real_mode:
+ /* Grab values from unsplit SPRs */
+ mfspr r6, SPRN_LDBAR
+ mfspr r7, SPRN_PMMAR
+ mfspr r8, SPRN_PMCR
+ mfspr r9, SPRN_RPR
+ mfspr r10, SPRN_SDR1
+
+ /* Order reading the SPRs vs telling the primary we are ready to split */
+ sync
+
+ /* Tell thread 0 we are in real mode */
+ li r4, SYNC_STEP_REAL_MODE
+ stb r4, 0(r3)
+
+ li r5, (HID0_POWER8_4LPARMODE | HID0_POWER8_2LPARMODE)@highest
+ sldi r5, r5, 48
+
+ /* Loop until we see the split happen in HID0 */
+1: mfspr r4, SPRN_HID0
+ and. r4, r4, r5
+ beq 1b
+
+ /*
+ * We only need to initialise the below regs once for each subcore,
+ * but it's simpler and harmless to do it on each thread.
+ */
+
+ /* Make sure various SPRS have sane values */
+ li r4, 0
+ mtspr SPRN_LPID, r4
+ mtspr SPRN_PCR, r4
+ mtspr SPRN_HDEC, r4
+
+ /* Restore SPR values now we are split */
+ mtspr SPRN_LDBAR, r6
+ mtspr SPRN_PMMAR, r7
+ mtspr SPRN_PMCR, r8
+ mtspr SPRN_RPR, r9
+ mtspr SPRN_SDR1, r10
+
+ LOAD_REG_ADDR(r5, virtual_mode)
+
+ /* Get out of real mode */
+ mtspr SPRN_SRR0,r5
+ mtspr SPRN_SRR1,r12
+ rfid
+ b . /* prevent speculative execution */
+
+virtual_mode:
+ blr
diff --git a/arch/powerpc/platforms/powernv/subcore.c b/arch/powerpc/platforms/powernv/subcore.c
new file mode 100644
index 0000000..894ecb3
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/subcore.c
@@ -0,0 +1,392 @@
+/*
+ * Copyright 2013, Michael (Ellerman|Neuling), IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#define pr_fmt(fmt) "powernv: " fmt
+
+#include <linux/kernel.h>
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
+#include <linux/device.h>
+#include <linux/gfp.h>
+#include <linux/smp.h>
+#include <linux/stop_machine.h>
+
+#include <asm/cputhreads.h>
+#include <asm/kvm_ppc.h>
+#include <asm/machdep.h>
+#include <asm/opal.h>
+#include <asm/smp.h>
+
+#include "subcore.h"
+
+
+/*
+ * Split/unsplit procedure:
+ *
+ * A core can be in one of three states, unsplit, 2-way split, and 4-way split.
+ *
+ * The mapping to subcores_per_core is simple:
+ *
+ * State | subcores_per_core
+ * ------------|------------------
+ * Unsplit | 1
+ * 2-way split | 2
+ * 4-way split | 4
+ *
+ * The core is split along thread boundaries, the mapping between subcores and
+ * threads is as follows:
+ *
+ * Unsplit:
+ * ----------------------------
+ * Subcore | 0 |
+ * ----------------------------
+ * Thread | 0 1 2 3 4 5 6 7 |
+ * ----------------------------
+ *
+ * 2-way split:
+ * -------------------------------------
+ * Subcore | 0 | 1 |
+ * -------------------------------------
+ * Thread | 0 1 2 3 | 4 5 6 7 |
+ * -------------------------------------
+ *
+ * 4-way split:
+ * -----------------------------------------
+ * Subcore | 0 | 1 | 2 | 3 |
+ * -----------------------------------------
+ * Thread | 0 1 | 2 3 | 4 5 | 6 7 |
+ * -----------------------------------------
+ *
+ *
+ * Transitions
+ * -----------
+ *
+ * It is not possible to transition between either of the split states, the
+ * core must first be unsplit. The legal transitions are:
+ *
+ * ----------- ---------------
+ * | | <----> | 2-way split |
+ * | | ---------------
+ * | Unsplit |
+ * | | ---------------
+ * | | <----> | 4-way split |
+ * ----------- ---------------
+ *
+ * Unsplitting
+ * -----------
+ *
+ * Unsplitting is the simpler procedure. It requires thread 0 to request the
+ * unsplit while all other threads NAP.
+ *
+ * Thread 0 clears HID0_POWER8_DYNLPARDIS (Dynamic LPAR Disable). This tells
+ * the hardware that if all threads except 0 are napping, the hardware should
+ * unsplit the core.
+ *
+ * Non-zero threads are sent to a NAP loop, they don't exit the loop until they
+ * see the core unsplit.
+ *
+ * Core 0 spins waiting for the hardware to see all the other threads napping
+ * and perform the unsplit.
+ *
+ * Once thread 0 sees the unsplit, it IPIs the secondary threads to wake them
+ * out of NAP. They will then see the core unsplit and exit the NAP loop.
+ *
+ * Splitting
+ * ---------
+ *
+ * The basic splitting procedure is fairly straight forward. However it is
+ * complicated by the fact that after the split occurs, the newly created
+ * subcores are not in a fully initialised state.
+ *
+ * Most notably the subcores do not have the correct value for SDR1, which
+ * means they must not be running in virtual mode when the split occurs. The
+ * subcores have separate timebases SPRs but these are pre-synchronised by
+ * opal.
+ *
+ * To begin with secondary threads are sent to an assembly routine. There they
+ * switch to real mode, so they are immune to the uninitialised SDR1 value.
+ * Once in real mode they indicate that they are in real mode, and spin waiting
+ * to see the core split.
+ *
+ * Thread 0 waits to see that all secondaries are in real mode, and then begins
+ * the splitting procedure. It firstly sets HID0_POWER8_DYNLPARDIS, which
+ * prevents the hardware from unsplitting. Then it sets the appropriate HID bit
+ * to request the split, and spins waiting to see that the split has happened.
+ *
+ * Concurrently the secondaries will notice the split. When they do they set up
+ * their SPRs, notably SDR1, and then they can return to virtual mode and exit
+ * the procedure.
+ */
+
+/* Initialised at boot by subcore_init() */
+static int subcores_per_core;
+
+/*
+ * Used to communicate to offline cpus that we want them to pop out of the
+ * offline loop and do a split or unsplit.
+ *
+ * 0 - no split happening
+ * 1 - unsplit in progress
+ * 2 - split to 2 in progress
+ * 4 - split to 4 in progress
+ */
+static int new_split_mode;
+
+static cpumask_var_t cpu_offline_mask;
+
+struct split_state {
+ u8 step;
+ u8 master;
+};
+
+static DEFINE_PER_CPU(struct split_state, split_state);
+
+static void wait_for_sync_step(int step)
+{
+ int i, cpu = smp_processor_id();
+
+ for (i = cpu + 1; i < cpu + threads_per_core; i++)
+ while(per_cpu(split_state, i).step < step)
+ barrier();
+
+ /* Order the wait loop vs any subsequent loads/stores. */
+ mb();
+}
+
+static void unsplit_core(void)
+{
+ u64 hid0, mask;
+ int i, cpu;
+
+ mask = HID0_POWER8_2LPARMODE | HID0_POWER8_4LPARMODE;
+
+ cpu = smp_processor_id();
+ if (cpu_thread_in_core(cpu) != 0) {
+ while (mfspr(SPRN_HID0) & mask)
+ power7_nap(0);
+
+ per_cpu(split_state, cpu).step = SYNC_STEP_UNSPLIT;
+ return;
+ }
+
+ hid0 = mfspr(SPRN_HID0);
+ hid0 &= ~HID0_POWER8_DYNLPARDIS;
+ mtspr(SPRN_HID0, hid0);
+
+ while (mfspr(SPRN_HID0) & mask)
+ cpu_relax();
+
+ /* Wake secondaries out of NAP */
+ for (i = cpu + 1; i < cpu + threads_per_core; i++)
+ smp_send_reschedule(i);
+
+ wait_for_sync_step(SYNC_STEP_UNSPLIT);
+}
+
+static void split_core(int new_mode)
+{
+ struct { u64 value; u64 mask; } split_parms[2] = {
+ { HID0_POWER8_1TO2LPAR, HID0_POWER8_2LPARMODE },
+ { HID0_POWER8_1TO4LPAR, HID0_POWER8_4LPARMODE }
+ };
+ int i, cpu;
+ u64 hid0;
+
+ /* Convert new_mode (2 or 4) into an index into our parms array */
+ i = (new_mode >> 1) - 1;
+ BUG_ON(i < 0 || i > 1);
+
+ cpu = smp_processor_id();
+ if (cpu_thread_in_core(cpu) != 0) {
+ split_core_secondary_loop(&per_cpu(split_state, cpu).step);
+ return;
+ }
+
+ wait_for_sync_step(SYNC_STEP_REAL_MODE);
+
+ /* Write new mode */
+ hid0 = mfspr(SPRN_HID0);
+ hid0 |= HID0_POWER8_DYNLPARDIS | split_parms[i].value;
+ mtspr(SPRN_HID0, hid0);
+
+ /* Wait for it to happen */
+ while (!(mfspr(SPRN_HID0) & split_parms[i].mask))
+ cpu_relax();
+}
+
+static void cpu_do_split(int new_mode)
+{
+ /*
+ * At boot subcores_per_core will be 0, so we will always unsplit at
+ * boot. In the usual case where the core is already unsplit it's a
+ * nop, and this just ensures the kernel's notion of the mode is
+ * consistent with the hardware.
+ */
+ if (subcores_per_core != 1)
+ unsplit_core();
+
+ if (new_mode != 1)
+ split_core(new_mode);
+
+ mb();
+ per_cpu(split_state, smp_processor_id()).step = SYNC_STEP_FINISHED;
+}
+
+bool cpu_core_split_required(void)
+{
+ smp_rmb();
+
+ if (!new_split_mode)
+ return false;
+
+ cpu_do_split(new_split_mode);
+
+ return true;
+}
+
+static int cpu_update_split_mode(void *data)
+{
+ int cpu, new_mode = *(int *)data;
+
+ if (this_cpu_ptr(&split_state)->master) {
+ new_split_mode = new_mode;
+ smp_wmb();
+
+ cpumask_andnot(cpu_offline_mask, cpu_present_mask,
+ cpu_online_mask);
+
+ /* This should work even though the cpu is offline */
+ for_each_cpu(cpu, cpu_offline_mask)
+ smp_send_reschedule(cpu);
+ }
+
+ cpu_do_split(new_mode);
+
+ if (this_cpu_ptr(&split_state)->master) {
+ /* Wait for all cpus to finish before we touch subcores_per_core */
+ for_each_present_cpu(cpu) {
+ if (cpu >= setup_max_cpus)
+ break;
+
+ while(per_cpu(split_state, cpu).step < SYNC_STEP_FINISHED)
+ barrier();
+ }
+
+ new_split_mode = 0;
+
+ /* Make the new mode public */
+ subcores_per_core = new_mode;
+ threads_per_subcore = threads_per_core / subcores_per_core;
+
+ /* Make sure the new mode is written before we exit */
+ mb();
+ }
+
+ return 0;
+}
+
+static int set_subcores_per_core(int new_mode)
+{
+ struct split_state *state;
+ int cpu;
+
+ if (kvm_hv_mode_active()) {
+ pr_err("Unable to change split core mode while KVM active.\n");
+ return -EBUSY;
+ }
+
+ /*
+ * We are only called at boot, or from the sysfs write. If that ever
+ * changes we'll need a lock here.
+ */
+ BUG_ON(new_mode < 1 || new_mode > 4 || new_mode == 3);
+
+ for_each_present_cpu(cpu) {
+ state = &per_cpu(split_state, cpu);
+ state->step = SYNC_STEP_INITIAL;
+ state->master = 0;
+ }
+
+ get_online_cpus();
+
+ /* This cpu will update the globals before exiting stop machine */
+ this_cpu_ptr(&split_state)->master = 1;
+
+ /* Ensure state is consistent before we call the other cpus */
+ mb();
+
+ stop_machine(cpu_update_split_mode, &new_mode, cpu_online_mask);
+
+ put_online_cpus();
+
+ return 0;
+}
+
+static ssize_t __used store_subcores_per_core(struct device *dev,
+ struct device_attribute *attr, const char *buf,
+ size_t count)
+{
+ unsigned long val;
+ int rc;
+
+ /* We are serialised by the attribute lock */
+
+ rc = sscanf(buf, "%lx", &val);
+ if (rc != 1)
+ return -EINVAL;
+
+ switch (val) {
+ case 1:
+ case 2:
+ case 4:
+ if (subcores_per_core == val)
+ /* Nothing to do */
+ goto out;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ rc = set_subcores_per_core(val);
+ if (rc)
+ return rc;
+
+out:
+ return count;
+}
+
+static ssize_t show_subcores_per_core(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ return sprintf(buf, "%x\n", subcores_per_core);
+}
+
+static DEVICE_ATTR(subcores_per_core, 0644,
+ show_subcores_per_core, store_subcores_per_core);
+
+static int subcore_init(void)
+{
+ if (!cpu_has_feature(CPU_FTR_ARCH_207S))
+ return 0;
+
+ /*
+ * We need all threads in a core to be present to split/unsplit so
+ * continue only if max_cpus are aligned to threads_per_core.
+ */
+ if (setup_max_cpus % threads_per_core)
+ return 0;
+
+ BUG_ON(!alloc_cpumask_var(&cpu_offline_mask, GFP_KERNEL));
+
+ set_subcores_per_core(1);
+
+ return device_create_file(cpu_subsys.dev_root,
+ &dev_attr_subcores_per_core);
+}
+machine_device_initcall(powernv, subcore_init);
diff --git a/arch/powerpc/platforms/powernv/subcore.h b/arch/powerpc/platforms/powernv/subcore.h
new file mode 100644
index 0000000..148abc9
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/subcore.h
@@ -0,0 +1,18 @@
+/*
+ * Copyright 2013, Michael Ellerman, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+/* These are ordered and tested with <= */
+#define SYNC_STEP_INITIAL 0
+#define SYNC_STEP_UNSPLIT 1 /* Set by secondary when it sees unsplit */
+#define SYNC_STEP_REAL_MODE 2 /* Set by secondary when in real mode */
+#define SYNC_STEP_FINISHED 3 /* Set by secondary when split/unsplit is done */
+
+#ifndef __ASSEMBLY__
+void split_core_secondary_loop(u8 *state);
+#endif
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c b/arch/powerpc/platforms/pseries/eeh_pseries.c
index 8a8f047..0bec0c0 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -175,6 +175,36 @@
return 0;
}
+static int pseries_eeh_find_ecap(struct device_node *dn, int cap)
+{
+ struct pci_dn *pdn = PCI_DN(dn);
+ struct eeh_dev *edev = of_node_to_eeh_dev(dn);
+ u32 header;
+ int pos = 256;
+ int ttl = (4096 - 256) / 8;
+
+ if (!edev || !edev->pcie_cap)
+ return 0;
+ if (rtas_read_config(pdn, pos, 4, &header) != PCIBIOS_SUCCESSFUL)
+ return 0;
+ else if (!header)
+ return 0;
+
+ while (ttl-- > 0) {
+ if (PCI_EXT_CAP_ID(header) == cap && pos)
+ return pos;
+
+ pos = PCI_EXT_CAP_NEXT(header);
+ if (pos < 256)
+ break;
+
+ if (rtas_read_config(pdn, pos, 4, &header) != PCIBIOS_SUCCESSFUL)
+ break;
+ }
+
+ return 0;
+}
+
/**
* pseries_eeh_of_probe - EEH probe on the given device
* @dn: OF node
@@ -220,7 +250,9 @@
* or PCIe switch downstream port.
*/
edev->class_code = class_code;
+ edev->pcix_cap = pseries_eeh_find_cap(dn, PCI_CAP_ID_PCIX);
edev->pcie_cap = pseries_eeh_find_cap(dn, PCI_CAP_ID_EXP);
+ edev->aer_cap = pseries_eeh_find_ecap(dn, PCI_EXT_CAP_ID_ERR);
edev->mode &= 0xFFFFFF00;
if ((edev->class_code >> 8) == PCI_CLASS_BRIDGE_PCI) {
edev->mode |= EEH_DEV_BRIDGE;
@@ -464,6 +496,7 @@
} else {
result = EEH_STATE_NOT_SUPPORT;
}
+ break;
default:
result = EEH_STATE_NOT_SUPPORT;
}
@@ -499,11 +532,19 @@
/* If fundamental-reset not supported, try hot-reset */
if (option == EEH_RESET_FUNDAMENTAL &&
ret == -8) {
+ option = EEH_RESET_HOT;
ret = rtas_call(ibm_set_slot_reset, 4, 1, NULL,
config_addr, BUID_HI(pe->phb->buid),
- BUID_LO(pe->phb->buid), EEH_RESET_HOT);
+ BUID_LO(pe->phb->buid), option);
}
+ /* We need reset hold or settlement delay */
+ if (option == EEH_RESET_FUNDAMENTAL ||
+ option == EEH_RESET_HOT)
+ msleep(EEH_PE_RST_HOLD_TIME);
+ else
+ msleep(EEH_PE_RST_SETTLE_TIME);
+
return ret;
}
diff --git a/arch/powerpc/platforms/pseries/hotplug-memory.c b/arch/powerpc/platforms/pseries/hotplug-memory.c
index 7f75c94..7995135 100644
--- a/arch/powerpc/platforms/pseries/hotplug-memory.c
+++ b/arch/powerpc/platforms/pseries/hotplug-memory.c
@@ -21,7 +21,7 @@
#include <asm/prom.h>
#include <asm/sparsemem.h>
-static unsigned long get_memblock_size(void)
+unsigned long pseries_memory_block_size(void)
{
struct device_node *np;
unsigned int memblock_size = MIN_MEMORY_BLOCK_SIZE;
@@ -64,17 +64,6 @@
return memblock_size;
}
-/* WARNING: This is going to override the generic definition whenever
- * pseries is built-in regardless of what platform is active at boot
- * time. This is fine for now as this is the only "option" and it
- * should work everywhere. If not, we'll have to turn this into a
- * ppc_md. callback
- */
-unsigned long memory_block_size_bytes(void)
-{
- return get_memblock_size();
-}
-
#ifdef CONFIG_MEMORY_HOTREMOVE
static int pseries_remove_memory(u64 start, u64 size)
{
@@ -105,7 +94,7 @@
if (!pfn_valid(start_pfn))
goto out;
- block_sz = memory_block_size_bytes();
+ block_sz = pseries_memory_block_size();
sections_per_block = block_sz / MIN_MEMORY_BLOCK_SIZE;
nid = memory_add_physaddr_to_nid(base);
@@ -201,7 +190,7 @@
u32 *p;
int i, rc = -EINVAL;
- memblock_size = get_memblock_size();
+ memblock_size = pseries_memory_block_size();
if (!memblock_size)
return -EINVAL;
diff --git a/arch/powerpc/platforms/pseries/hvCall.S b/arch/powerpc/platforms/pseries/hvCall.S
index 444fe77..99ecf0a 100644
--- a/arch/powerpc/platforms/pseries/hvCall.S
+++ b/arch/powerpc/platforms/pseries/hvCall.S
@@ -49,7 +49,7 @@
std r0,16(r1); \
addi r4,r1,STK_PARAM(FIRST_REG); \
stdu r1,-STACK_FRAME_OVERHEAD(r1); \
- bl .__trace_hcall_entry; \
+ bl __trace_hcall_entry; \
addi r1,r1,STACK_FRAME_OVERHEAD; \
ld r0,16(r1); \
ld r3,STK_PARAM(R3)(r1); \
@@ -83,7 +83,7 @@
mr r3,r6; \
std r0,16(r1); \
stdu r1,-STACK_FRAME_OVERHEAD(r1); \
- bl .__trace_hcall_exit; \
+ bl __trace_hcall_exit; \
addi r1,r1,STACK_FRAME_OVERHEAD; \
ld r0,16(r1); \
ld r3,STK_PARAM(R3)(r1); \
@@ -106,7 +106,7 @@
.text
-_GLOBAL(plpar_hcall_norets)
+_GLOBAL_TOC(plpar_hcall_norets)
HMT_MEDIUM
mfcr r0
@@ -122,7 +122,7 @@
mtcrf 0xff,r0
blr /* return r3 = status */
-_GLOBAL(plpar_hcall)
+_GLOBAL_TOC(plpar_hcall)
HMT_MEDIUM
mfcr r0
@@ -188,7 +188,7 @@
blr /* return r3 = status */
-_GLOBAL(plpar_hcall9)
+_GLOBAL_TOC(plpar_hcall9)
HMT_MEDIUM
mfcr r0
diff --git a/arch/powerpc/platforms/pseries/pseries.h b/arch/powerpc/platforms/pseries/pseries.h
index 9921953..361add6 100644
--- a/arch/powerpc/platforms/pseries/pseries.h
+++ b/arch/powerpc/platforms/pseries/pseries.h
@@ -64,4 +64,6 @@
struct pci_host_bridge;
int pseries_root_bridge_prepare(struct pci_host_bridge *bridge);
+unsigned long pseries_memory_block_size(void);
+
#endif /* _PSERIES_PSERIES_H */
diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index 099d2df..f2f40e6 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -510,7 +510,11 @@
static int __init pSeries_init_panel(void)
{
/* Manually leave the kernel version on the panel. */
+#ifdef __BIG_ENDIAN__
ppc_md.progress("Linux ppc64\n", 0);
+#else
+ ppc_md.progress("Linux ppc64le\n", 0);
+#endif
ppc_md.progress(init_utsname()->version, 0);
return 0;
@@ -806,4 +810,7 @@
#ifdef CONFIG_KEXEC
.machine_kexec = pSeries_machine_kexec,
#endif
+#ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
+ .memory_block_size = pseries_memory_block_size,
+#endif
};
diff --git a/arch/powerpc/platforms/pseries/smp.c b/arch/powerpc/platforms/pseries/smp.c
index 24f58cb..a3555b1 100644
--- a/arch/powerpc/platforms/pseries/smp.c
+++ b/arch/powerpc/platforms/pseries/smp.c
@@ -44,6 +44,7 @@
#include <asm/xics.h>
#include <asm/dbell.h>
#include <asm/plpar_wrappers.h>
+#include <asm/code-patching.h>
#include "pseries.h"
#include "offline_states.h"
@@ -96,8 +97,8 @@
static inline int smp_startup_cpu(unsigned int lcpu)
{
int status;
- unsigned long start_here = __pa((u32)*((unsigned long *)
- generic_secondary_smp_init));
+ unsigned long start_here =
+ __pa(ppc_function_entry(generic_secondary_smp_init));
unsigned int pcpu;
int start_cpu;
diff --git a/arch/powerpc/platforms/wsp/scom_smp.c b/arch/powerpc/platforms/wsp/scom_smp.c
index 268bc89..8c79ce0 100644
--- a/arch/powerpc/platforms/wsp/scom_smp.c
+++ b/arch/powerpc/platforms/wsp/scom_smp.c
@@ -20,6 +20,7 @@
#include <asm/reg_a2.h>
#include <asm/scom.h>
#include <asm/udbg.h>
+#include <asm/code-patching.h>
#include "wsp.h"
@@ -405,7 +406,7 @@
goto fail;
}
- start_here = *(unsigned long *)(core_setup ? generic_secondary_smp_init
+ start_here = ppc_function_entry(core_setup ? generic_secondary_smp_init
: generic_secondary_thread_init);
pr_devel("CPU%d entry point at 0x%lx...\n", lcpu, start_here);
diff --git a/arch/powerpc/sysdev/Kconfig b/arch/powerpc/sysdev/Kconfig
index 7baa70d..a19332a3 100644
--- a/arch/powerpc/sysdev/Kconfig
+++ b/arch/powerpc/sysdev/Kconfig
@@ -7,6 +7,12 @@
depends on PCI && 4xx
default n
+config PPC4xx_HSTA_MSI
+ bool
+ depends on PCI_MSI
+ depends on PCI && 4xx
+ default n
+
config PPC4xx_MSI
bool
depends on PCI_MSI
diff --git a/arch/powerpc/sysdev/Makefile b/arch/powerpc/sysdev/Makefile
index afbcc37..f7cb2a1 100644
--- a/arch/powerpc/sysdev/Makefile
+++ b/arch/powerpc/sysdev/Makefile
@@ -45,6 +45,7 @@
ifeq ($(CONFIG_PCI),y)
obj-$(CONFIG_4xx) += ppc4xx_pci.o
endif
+obj-$(CONFIG_PPC4xx_HSTA_MSI) += ppc4xx_hsta_msi.o
obj-$(CONFIG_PPC4xx_MSI) += ppc4xx_msi.o
obj-$(CONFIG_PPC4xx_CPM) += ppc4xx_cpm.o
obj-$(CONFIG_PPC4xx_GPIO) += ppc4xx_gpio.o
diff --git a/arch/powerpc/sysdev/fsl_pci.c b/arch/powerpc/sysdev/fsl_pci.c
index 3f415e2..4bd091a 100644
--- a/arch/powerpc/sysdev/fsl_pci.c
+++ b/arch/powerpc/sysdev/fsl_pci.c
@@ -1150,8 +1150,7 @@
pci = hose->private_data;
/* Enable PTOD, ENL23D & EXL23D */
- out_be32(&pci->pex_pme_mes_disr, 0);
- setbits32(&pci->pex_pme_mes_disr,
+ clrbits32(&pci->pex_pme_mes_disr,
PME_DISR_EN_PTOD | PME_DISR_EN_ENL23D | PME_DISR_EN_EXL23D);
out_be32(&pci->pex_pme_mes_ier, 0);
diff --git a/arch/powerpc/sysdev/fsl_rio.c b/arch/powerpc/sysdev/fsl_rio.c
index cf2b084..c04b718 100644
--- a/arch/powerpc/sysdev/fsl_rio.c
+++ b/arch/powerpc/sysdev/fsl_rio.c
@@ -391,8 +391,10 @@
ops->get_inb_message = fsl_get_inb_message;
rmu_node = of_parse_phandle(dev->dev.of_node, "fsl,srio-rmu-handle", 0);
- if (!rmu_node)
+ if (!rmu_node) {
+ dev_err(&dev->dev, "No valid fsl,srio-rmu-handle property\n");
goto err_rmu;
+ }
rc = of_address_to_resource(rmu_node, 0, &rmu_regs);
if (rc) {
dev_err(&dev->dev, "Can't get %s property 'reg'\n",
@@ -413,6 +415,7 @@
/*set up doobell node*/
np = of_find_compatible_node(NULL, NULL, "fsl,srio-dbell-unit");
if (!np) {
+ dev_err(&dev->dev, "No fsl,srio-dbell-unit node\n");
rc = -ENODEV;
goto err_dbell;
}
@@ -441,6 +444,7 @@
/*set up port write node*/
np = of_find_compatible_node(NULL, NULL, "fsl,srio-port-write-unit");
if (!np) {
+ dev_err(&dev->dev, "No fsl,srio-port-write-unit node\n");
rc = -ENODEV;
goto err_pw;
}
@@ -633,14 +637,18 @@
return 0;
err:
kfree(pw);
+ pw = NULL;
err_pw:
kfree(dbell);
+ dbell = NULL;
err_dbell:
iounmap(rmu_regs_win);
+ rmu_regs_win = NULL;
err_rmu:
kfree(ops);
err_ops:
iounmap(rio_regs_win);
+ rio_regs_win = NULL;
err_rio_regs:
return rc;
}
diff --git a/arch/powerpc/sysdev/fsl_rmu.c b/arch/powerpc/sysdev/fsl_rmu.c
index 00e224a..b48197a 100644
--- a/arch/powerpc/sysdev/fsl_rmu.c
+++ b/arch/powerpc/sysdev/fsl_rmu.c
@@ -881,9 +881,9 @@
rc = request_irq(IRQ_RIO_RX(mport), fsl_rio_rx_handler, 0,
"msg_rx", (void *)mport);
if (rc < 0) {
- dma_free_coherent(priv->dev, RIO_MSG_BUFFER_SIZE,
- rmu->msg_tx_ring.virt_buffer[i],
- rmu->msg_tx_ring.phys_buffer[i]);
+ dma_free_coherent(priv->dev,
+ rmu->msg_rx_ring.size * RIO_MAX_MSG_SIZE,
+ rmu->msg_rx_ring.virt, rmu->msg_rx_ring.phys);
goto out;
}
diff --git a/arch/powerpc/sysdev/mpic.c b/arch/powerpc/sysdev/mpic.c
index 8209744..be33c97 100644
--- a/arch/powerpc/sysdev/mpic.c
+++ b/arch/powerpc/sysdev/mpic.c
@@ -1588,10 +1588,6 @@
num_timers = 8;
}
- /* FSL mpic error interrupt intialization */
- if (mpic->flags & MPIC_FSL_HAS_EIMR)
- mpic_err_int_init(mpic, MPIC_FSL_ERR_INT);
-
/* Initialize timers to our reserved vectors and mask them for now */
for (i = 0; i < num_timers; i++) {
unsigned int offset = mpic_tm_offset(mpic, i);
@@ -1675,6 +1671,10 @@
irq_set_chained_handler(virq, &mpic_cascade);
}
}
+
+ /* FSL mpic error interrupt intialization */
+ if (mpic->flags & MPIC_FSL_HAS_EIMR)
+ mpic_err_int_init(mpic, MPIC_FSL_ERR_INT);
}
void __init mpic_set_clk_ratio(struct mpic *mpic, u32 clock_ratio)
diff --git a/arch/powerpc/sysdev/ppc4xx_hsta_msi.c b/arch/powerpc/sysdev/ppc4xx_hsta_msi.c
new file mode 100644
index 0000000..11c8884
--- /dev/null
+++ b/arch/powerpc/sysdev/ppc4xx_hsta_msi.c
@@ -0,0 +1,215 @@
+/*
+ * MSI support for PPC4xx SoCs using High Speed Transfer Assist (HSTA) for
+ * generation of the interrupt.
+ *
+ * Copyright © 2013 Alistair Popple <alistair@popple.id.au> IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/interrupt.h>
+#include <linux/msi.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/pci.h>
+#include <linux/semaphore.h>
+#include <asm/msi_bitmap.h>
+
+struct ppc4xx_hsta_msi {
+ struct device *dev;
+
+ /* The ioremapped HSTA MSI IO space */
+ u32 __iomem *data;
+
+ /* Physical address of HSTA MSI IO space */
+ u64 address;
+ struct msi_bitmap bmp;
+
+ /* An array mapping offsets to hardware IRQs */
+ int *irq_map;
+
+ /* Number of hwirqs supported */
+ int irq_count;
+};
+static struct ppc4xx_hsta_msi ppc4xx_hsta_msi;
+
+static int hsta_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
+{
+ struct msi_msg msg;
+ struct msi_desc *entry;
+ int irq, hwirq;
+ u64 addr;
+
+ list_for_each_entry(entry, &dev->msi_list, list) {
+ irq = msi_bitmap_alloc_hwirqs(&ppc4xx_hsta_msi.bmp, 1);
+ if (irq < 0) {
+ pr_debug("%s: Failed to allocate msi interrupt\n",
+ __func__);
+ return irq;
+ }
+
+ hwirq = ppc4xx_hsta_msi.irq_map[irq];
+ if (hwirq == NO_IRQ) {
+ pr_err("%s: Failed mapping irq %d\n", __func__, irq);
+ return -EINVAL;
+ }
+
+ /*
+ * HSTA generates interrupts on writes to 128-bit aligned
+ * addresses.
+ */
+ addr = ppc4xx_hsta_msi.address + irq*0x10;
+ msg.address_hi = upper_32_bits(addr);
+ msg.address_lo = lower_32_bits(addr);
+
+ /* Data is not used by the HSTA. */
+ msg.data = 0;
+
+ pr_debug("%s: Setup irq %d (0x%0llx)\n", __func__, hwirq,
+ (((u64) msg.address_hi) << 32) | msg.address_lo);
+
+ if (irq_set_msi_desc(hwirq, entry)) {
+ pr_err(
+ "%s: Invalid hwirq %d specified in device tree\n",
+ __func__, hwirq);
+ msi_bitmap_free_hwirqs(&ppc4xx_hsta_msi.bmp, irq, 1);
+ return -EINVAL;
+ }
+ write_msi_msg(hwirq, &msg);
+ }
+
+ return 0;
+}
+
+static int hsta_find_hwirq_offset(int hwirq)
+{
+ int irq;
+
+ /* Find the offset given the hwirq */
+ for (irq = 0; irq < ppc4xx_hsta_msi.irq_count; irq++)
+ if (ppc4xx_hsta_msi.irq_map[irq] == hwirq)
+ return irq;
+
+ return -EINVAL;
+}
+
+static void hsta_teardown_msi_irqs(struct pci_dev *dev)
+{
+ struct msi_desc *entry;
+ int irq;
+
+ list_for_each_entry(entry, &dev->msi_list, list) {
+ if (entry->irq == NO_IRQ)
+ continue;
+
+ irq = hsta_find_hwirq_offset(entry->irq);
+
+ /* entry->irq should always be in irq_map */
+ BUG_ON(irq < 0);
+ irq_set_msi_desc(entry->irq, NULL);
+ msi_bitmap_free_hwirqs(&ppc4xx_hsta_msi.bmp, irq, 1);
+ pr_debug("%s: Teardown IRQ %u (index %u)\n", __func__,
+ entry->irq, irq);
+ }
+}
+
+static int hsta_msi_check_device(struct pci_dev *pdev, int nvec, int type)
+{
+ /* We don't support MSI-X */
+ if (type == PCI_CAP_ID_MSIX) {
+ pr_debug("%s: MSI-X not supported.\n", __func__);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int hsta_msi_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct resource *mem;
+ int irq, ret, irq_count;
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (IS_ERR(mem)) {
+ dev_err(dev, "Unable to get mmio space\n");
+ return -EINVAL;
+ }
+
+ irq_count = of_irq_count(dev->of_node);
+ if (!irq_count) {
+ dev_err(dev, "Unable to find IRQ range\n");
+ return -EINVAL;
+ }
+
+ ppc4xx_hsta_msi.dev = dev;
+ ppc4xx_hsta_msi.address = mem->start;
+ ppc4xx_hsta_msi.data = ioremap(mem->start, resource_size(mem));
+ ppc4xx_hsta_msi.irq_count = irq_count;
+ if (IS_ERR(ppc4xx_hsta_msi.data)) {
+ dev_err(dev, "Unable to map memory\n");
+ return -ENOMEM;
+ }
+
+ ret = msi_bitmap_alloc(&ppc4xx_hsta_msi.bmp, irq_count, dev->of_node);
+ if (ret)
+ goto out;
+
+ ppc4xx_hsta_msi.irq_map = kmalloc(sizeof(int) * irq_count, GFP_KERNEL);
+ if (IS_ERR(ppc4xx_hsta_msi.irq_map)) {
+ ret = -ENOMEM;
+ goto out1;
+ }
+
+ /* Setup a mapping from irq offsets to hardware irq numbers */
+ for (irq = 0; irq < irq_count; irq++) {
+ ppc4xx_hsta_msi.irq_map[irq] =
+ irq_of_parse_and_map(dev->of_node, irq);
+ if (ppc4xx_hsta_msi.irq_map[irq] == NO_IRQ) {
+ dev_err(dev, "Unable to map IRQ\n");
+ ret = -EINVAL;
+ goto out2;
+ }
+ }
+
+ ppc_md.setup_msi_irqs = hsta_setup_msi_irqs;
+ ppc_md.teardown_msi_irqs = hsta_teardown_msi_irqs;
+ ppc_md.msi_check_device = hsta_msi_check_device;
+ return 0;
+
+out2:
+ kfree(ppc4xx_hsta_msi.irq_map);
+
+out1:
+ msi_bitmap_free(&ppc4xx_hsta_msi.bmp);
+
+out:
+ iounmap(ppc4xx_hsta_msi.data);
+ return ret;
+}
+
+static const struct of_device_id hsta_msi_ids[] = {
+ {
+ .compatible = "ibm,hsta-msi",
+ },
+ {}
+};
+
+static struct platform_driver hsta_msi_driver = {
+ .probe = hsta_msi_probe,
+ .driver = {
+ .name = "hsta-msi",
+ .owner = THIS_MODULE,
+ .of_match_table = hsta_msi_ids,
+ },
+};
+
+static int hsta_msi_init(void)
+{
+ return platform_driver_register(&hsta_msi_driver);
+}
+subsys_initcall(hsta_msi_init);
diff --git a/arch/powerpc/sysdev/ppc4xx_pci.c b/arch/powerpc/sysdev/ppc4xx_pci.c
index 4914fd3..df6e2fc 100644
--- a/arch/powerpc/sysdev/ppc4xx_pci.c
+++ b/arch/powerpc/sysdev/ppc4xx_pci.c
@@ -176,8 +176,12 @@
return -ENXIO;
}
- /* Check that we are fully contained within 32 bits space */
- if (res->end > 0xffffffff) {
+ /* Check that we are fully contained within 32 bits space if we are not
+ * running on a 460sx or 476fpe which have 64 bit bus addresses.
+ */
+ if (res->end > 0xffffffff &&
+ !(of_device_is_compatible(hose->dn, "ibm,plb-pciex-460sx")
+ || of_device_is_compatible(hose->dn, "ibm,plb-pciex-476fpe"))) {
printk(KERN_ERR "%s: dma-ranges outside of 32 bits space\n",
hose->dn->full_name);
return -ENXIO;
@@ -1440,7 +1444,8 @@
ppc4xx_pciex_hwops = &ppc405ex_pcie_hwops;
#endif
#ifdef CONFIG_476FPE
- if (of_device_is_compatible(np, "ibm,plb-pciex-476fpe"))
+ if (of_device_is_compatible(np, "ibm,plb-pciex-476fpe")
+ || of_device_is_compatible(np, "ibm,plb-pciex-476gtr"))
ppc4xx_pciex_hwops = &ppc_476fpe_pcie_hwops;
#endif
if (ppc4xx_pciex_hwops == NULL) {
@@ -1751,7 +1756,10 @@
dcr_write(port->dcrs, DCRO_PEGPL_OMR1MSKL,
sa | DCRO_PEGPL_460SX_OMR1MSKL_UOT
| DCRO_PEGPL_OMRxMSKL_VAL);
- else if (of_device_is_compatible(port->node, "ibm,plb-pciex-476fpe"))
+ else if (of_device_is_compatible(
+ port->node, "ibm,plb-pciex-476fpe") ||
+ of_device_is_compatible(
+ port->node, "ibm,plb-pciex-476gtr"))
dcr_write(port->dcrs, DCRO_PEGPL_OMR1MSKL,
sa | DCRO_PEGPL_476FPE_OMR1MSKL_UOT
| DCRO_PEGPL_OMRxMSKL_VAL);
@@ -1881,7 +1889,10 @@
sa |= PCI_BASE_ADDRESS_MEM_PREFETCH;
if (of_device_is_compatible(port->node, "ibm,plb-pciex-460sx") ||
- of_device_is_compatible(port->node, "ibm,plb-pciex-476fpe"))
+ of_device_is_compatible(
+ port->node, "ibm,plb-pciex-476fpe") ||
+ of_device_is_compatible(
+ port->node, "ibm,plb-pciex-476gtr"))
sa |= PCI_BASE_ADDRESS_MEM_TYPE_64;
out_le32(mbase + PECFG_BAR0HMPA, RES_TO_U32_HIGH(sa));
diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
index 08504e7..d199bfa 100644
--- a/arch/powerpc/xmon/xmon.c
+++ b/arch/powerpc/xmon/xmon.c
@@ -419,7 +419,7 @@
get_output_lock();
excprint(regs);
if (bp) {
- printf("cpu 0x%x stopped at breakpoint 0x%x (",
+ printf("cpu 0x%x stopped at breakpoint 0x%lx (",
cpu, BP_NUM(bp));
xmon_print_symbol(regs->nip, " ", ")\n");
}
@@ -513,7 +513,7 @@
excprint(regs);
bp = at_breakpoint(regs->nip);
if (bp) {
- printf("Stopped at breakpoint %x (", BP_NUM(bp));
+ printf("Stopped at breakpoint %lx (", BP_NUM(bp));
xmon_print_symbol(regs->nip, " ", ")\n");
}
if (unrecoverable_excp(regs))
@@ -759,7 +759,7 @@
brk.address = dabr.address;
brk.type = (dabr.enabled & HW_BRK_TYPE_DABR) | HW_BRK_TYPE_PRIV_ALL;
brk.len = 8;
- set_breakpoint(&brk);
+ __set_breakpoint(&brk);
}
if (iabr && cpu_has_feature(CPU_FTR_IABR))
mtspr(SPRN_IABR, iabr->address
@@ -997,14 +997,14 @@
last_cpu = cpu;
} else {
if (last_cpu != first_cpu)
- printf("-%lx", last_cpu);
+ printf("-0x%lx", last_cpu);
last_cpu = first_cpu = cpu;
- printf(" %lx", cpu);
+ printf(" 0x%lx", cpu);
}
}
}
if (last_cpu != first_cpu)
- printf("-%lx", last_cpu);
+ printf("-0x%lx", last_cpu);
printf("\n");
return 0;
}
@@ -1024,7 +1024,7 @@
/* take control back */
mb();
xmon_owner = smp_processor_id();
- printf("cpu %u didn't take control\n", cpu);
+ printf("cpu 0x%x didn't take control\n", cpu);
return 0;
}
barrier();
@@ -1086,7 +1086,7 @@
fcs = 0xffff;
for (i = 0; i < ncsum; ++i) {
if (mread(adrs+i, &v, 1) == 0) {
- printf("csum stopped at %x\n", adrs+i);
+ printf("csum stopped at "REG"\n", adrs+i);
break;
}
fcs = FCS(fcs, v);
@@ -1202,12 +1202,12 @@
/* assume a breakpoint address */
bp = at_breakpoint(a);
if (bp == NULL) {
- printf("No breakpoint at %x\n", a);
+ printf("No breakpoint at %lx\n", a);
break;
}
}
- printf("Cleared breakpoint %x (", BP_NUM(bp));
+ printf("Cleared breakpoint %lx (", BP_NUM(bp));
xmon_print_symbol(bp->address, " ", ")\n");
bp->enabled = 0;
break;
@@ -1746,7 +1746,7 @@
__delay(200);
n = size;
} else {
- printf("*** Error writing address %x\n", adrs + n);
+ printf("*** Error writing address "REG"\n", adrs + n);
}
catch_memory_errors = 0;
return n;
@@ -2435,7 +2435,7 @@
ret = func(args[0], args[1], args[2], args[3],
args[4], args[5], args[6], args[7]);
sync();
- printf("return value is %x\n", ret);
+ printf("return value is 0x%lx\n", ret);
} else {
printf("*** %x exception occurred\n", fault_except);
}
@@ -2700,7 +2700,7 @@
unsigned long esid,vsid,valid;
unsigned long llp;
- printf("SLB contents of cpu %x\n", smp_processor_id());
+ printf("SLB contents of cpu 0x%x\n", smp_processor_id());
for (i = 0; i < mmu_slb_size; i++) {
asm volatile("slbmfee %0,%1" : "=r" (esid) : "r" (i));
@@ -2732,7 +2732,7 @@
int i;
unsigned long *tmp = (unsigned long *)local_paca->stab_addr;
- printf("Segment table contents of cpu %x\n", smp_processor_id());
+ printf("Segment table contents of cpu 0x%x\n", smp_processor_id());
for (i = 0; i < PAGE_SIZE/16; i++) {
unsigned long a, b;
diff --git a/arch/sh/include/asm/ftrace.h b/arch/sh/include/asm/ftrace.h
index 13e9966..e79fb6e 100644
--- a/arch/sh/include/asm/ftrace.h
+++ b/arch/sh/include/asm/ftrace.h
@@ -40,15 +40,7 @@
/* arch/sh/kernel/return_address.c */
extern void *return_address(unsigned int);
-#define HAVE_ARCH_CALLER_ADDR
-
-#define CALLER_ADDR0 ((unsigned long)__builtin_return_address(0))
-#define CALLER_ADDR1 ((unsigned long)return_address(1))
-#define CALLER_ADDR2 ((unsigned long)return_address(2))
-#define CALLER_ADDR3 ((unsigned long)return_address(3))
-#define CALLER_ADDR4 ((unsigned long)return_address(4))
-#define CALLER_ADDR5 ((unsigned long)return_address(5))
-#define CALLER_ADDR6 ((unsigned long)return_address(6))
+#define ftrace_return_address(n) return_address(n)
#endif /* __ASSEMBLY__ */
diff --git a/arch/tile/kernel/proc.c b/arch/tile/kernel/proc.c
index 681100c..6829a95 100644
--- a/arch/tile/kernel/proc.c
+++ b/arch/tile/kernel/proc.c
@@ -113,7 +113,7 @@
* Support /proc/sys/tile directory
*/
-static ctl_table unaligned_subtable[] = {
+static struct ctl_table unaligned_subtable[] = {
{
.procname = "enabled",
.data = &unaligned_fixup,
@@ -138,7 +138,7 @@
{}
};
-static ctl_table unaligned_table[] = {
+static struct ctl_table unaligned_table[] = {
{
.procname = "unaligned_fixup",
.mode = 0555,
diff --git a/arch/x86/boot/header.S b/arch/x86/boot/header.S
index 0ca9a5c..84c2234 100644
--- a/arch/x86/boot/header.S
+++ b/arch/x86/boot/header.S
@@ -375,8 +375,7 @@
# define XLF0 0
#endif
-#if defined(CONFIG_RELOCATABLE) && defined(CONFIG_X86_64) && \
- !defined(CONFIG_EFI_MIXED)
+#if defined(CONFIG_RELOCATABLE) && defined(CONFIG_X86_64)
/* kernel/boot_param/ramdisk could be loaded above 4g */
# define XLF1 XLF_CAN_BE_LOADED_ABOVE_4G
#else
diff --git a/arch/x86/crypto/ghash-clmulni-intel_asm.S b/arch/x86/crypto/ghash-clmulni-intel_asm.S
index 185fad4..5d1e007 100644
--- a/arch/x86/crypto/ghash-clmulni-intel_asm.S
+++ b/arch/x86/crypto/ghash-clmulni-intel_asm.S
@@ -92,7 +92,7 @@
ret
ENDPROC(__clmul_gf128mul_ble)
-/* void clmul_ghash_mul(char *dst, const be128 *shash) */
+/* void clmul_ghash_mul(char *dst, const u128 *shash) */
ENTRY(clmul_ghash_mul)
movups (%rdi), DATA
movups (%rsi), SHASH
@@ -106,7 +106,7 @@
/*
* void clmul_ghash_update(char *dst, const char *src, unsigned int srclen,
- * const be128 *shash);
+ * const u128 *shash);
*/
ENTRY(clmul_ghash_update)
cmp $16, %rdx
diff --git a/arch/x86/crypto/ghash-clmulni-intel_glue.c b/arch/x86/crypto/ghash-clmulni-intel_glue.c
index d785cf2..88bb7ba 100644
--- a/arch/x86/crypto/ghash-clmulni-intel_glue.c
+++ b/arch/x86/crypto/ghash-clmulni-intel_glue.c
@@ -25,17 +25,17 @@
#define GHASH_BLOCK_SIZE 16
#define GHASH_DIGEST_SIZE 16
-void clmul_ghash_mul(char *dst, const be128 *shash);
+void clmul_ghash_mul(char *dst, const u128 *shash);
void clmul_ghash_update(char *dst, const char *src, unsigned int srclen,
- const be128 *shash);
+ const u128 *shash);
struct ghash_async_ctx {
struct cryptd_ahash *cryptd_tfm;
};
struct ghash_ctx {
- be128 shash;
+ u128 shash;
};
struct ghash_desc_ctx {
@@ -68,11 +68,11 @@
a = be64_to_cpu(x->a);
b = be64_to_cpu(x->b);
- ctx->shash.a = (__be64)((b << 1) | (a >> 63));
- ctx->shash.b = (__be64)((a << 1) | (b >> 63));
+ ctx->shash.a = (b << 1) | (a >> 63);
+ ctx->shash.b = (a << 1) | (b >> 63);
if (a >> 63)
- ctx->shash.b ^= cpu_to_be64(0xc2);
+ ctx->shash.b ^= ((u64)0xc2) << 56;
return 0;
}
diff --git a/arch/x86/include/asm/signal.h b/arch/x86/include/asm/signal.h
index 35e67a4..31eab86 100644
--- a/arch/x86/include/asm/signal.h
+++ b/arch/x86/include/asm/signal.h
@@ -92,12 +92,6 @@
? __const_sigismember((set), (sig)) \
: __gen_sigismember((set), (sig)))
-static inline int sigfindinword(unsigned long word)
-{
- asm("bsfl %1,%0" : "=r"(word) : "rm"(word) : "cc");
- return word;
-}
-
struct pt_regs;
#else /* __i386__ */
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 491ef3e..047f9ff 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -26,6 +26,7 @@
obj-y += probe_roms.o
obj-$(CONFIG_X86_32) += i386_ksyms_32.o
obj-$(CONFIG_X86_64) += sys_x86_64.o x8664_ksyms_64.o
+obj-$(CONFIG_X86_64) += mcount_64.o
obj-y += syscall_$(BITS).o vsyscall_gtod.o
obj-$(CONFIG_X86_64) += vsyscall_64.o
obj-$(CONFIG_X86_64) += vsyscall_emu_64.o
diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S
index 9698798..48a2644 100644
--- a/arch/x86/kernel/entry_64.S
+++ b/arch/x86/kernel/entry_64.S
@@ -53,7 +53,6 @@
#include <asm/page_types.h>
#include <asm/irqflags.h>
#include <asm/paravirt.h>
-#include <asm/ftrace.h>
#include <asm/percpu.h>
#include <asm/asm.h>
#include <asm/context_tracking.h>
@@ -70,209 +69,6 @@
.code64
.section .entry.text, "ax"
-#ifdef CONFIG_FUNCTION_TRACER
-
-#ifdef CC_USING_FENTRY
-# define function_hook __fentry__
-#else
-# define function_hook mcount
-#endif
-
-#ifdef CONFIG_DYNAMIC_FTRACE
-
-ENTRY(function_hook)
- retq
-END(function_hook)
-
-/* skip is set if stack has been adjusted */
-.macro ftrace_caller_setup skip=0
- MCOUNT_SAVE_FRAME \skip
-
- /* Load the ftrace_ops into the 3rd parameter */
- movq function_trace_op(%rip), %rdx
-
- /* Load ip into the first parameter */
- movq RIP(%rsp), %rdi
- subq $MCOUNT_INSN_SIZE, %rdi
- /* Load the parent_ip into the second parameter */
-#ifdef CC_USING_FENTRY
- movq SS+16(%rsp), %rsi
-#else
- movq 8(%rbp), %rsi
-#endif
-.endm
-
-ENTRY(ftrace_caller)
- /* Check if tracing was disabled (quick check) */
- cmpl $0, function_trace_stop
- jne ftrace_stub
-
- ftrace_caller_setup
- /* regs go into 4th parameter (but make it NULL) */
- movq $0, %rcx
-
-GLOBAL(ftrace_call)
- call ftrace_stub
-
- MCOUNT_RESTORE_FRAME
-ftrace_return:
-
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
-GLOBAL(ftrace_graph_call)
- jmp ftrace_stub
-#endif
-
-GLOBAL(ftrace_stub)
- retq
-END(ftrace_caller)
-
-ENTRY(ftrace_regs_caller)
- /* Save the current flags before compare (in SS location)*/
- pushfq
-
- /* Check if tracing was disabled (quick check) */
- cmpl $0, function_trace_stop
- jne ftrace_restore_flags
-
- /* skip=8 to skip flags saved in SS */
- ftrace_caller_setup 8
-
- /* Save the rest of pt_regs */
- movq %r15, R15(%rsp)
- movq %r14, R14(%rsp)
- movq %r13, R13(%rsp)
- movq %r12, R12(%rsp)
- movq %r11, R11(%rsp)
- movq %r10, R10(%rsp)
- movq %rbp, RBP(%rsp)
- movq %rbx, RBX(%rsp)
- /* Copy saved flags */
- movq SS(%rsp), %rcx
- movq %rcx, EFLAGS(%rsp)
- /* Kernel segments */
- movq $__KERNEL_DS, %rcx
- movq %rcx, SS(%rsp)
- movq $__KERNEL_CS, %rcx
- movq %rcx, CS(%rsp)
- /* Stack - skipping return address */
- leaq SS+16(%rsp), %rcx
- movq %rcx, RSP(%rsp)
-
- /* regs go into 4th parameter */
- leaq (%rsp), %rcx
-
-GLOBAL(ftrace_regs_call)
- call ftrace_stub
-
- /* Copy flags back to SS, to restore them */
- movq EFLAGS(%rsp), %rax
- movq %rax, SS(%rsp)
-
- /* Handlers can change the RIP */
- movq RIP(%rsp), %rax
- movq %rax, SS+8(%rsp)
-
- /* restore the rest of pt_regs */
- movq R15(%rsp), %r15
- movq R14(%rsp), %r14
- movq R13(%rsp), %r13
- movq R12(%rsp), %r12
- movq R10(%rsp), %r10
- movq RBP(%rsp), %rbp
- movq RBX(%rsp), %rbx
-
- /* skip=8 to skip flags saved in SS */
- MCOUNT_RESTORE_FRAME 8
-
- /* Restore flags */
- popfq
-
- jmp ftrace_return
-ftrace_restore_flags:
- popfq
- jmp ftrace_stub
-
-END(ftrace_regs_caller)
-
-
-#else /* ! CONFIG_DYNAMIC_FTRACE */
-
-ENTRY(function_hook)
- cmpl $0, function_trace_stop
- jne ftrace_stub
-
- cmpq $ftrace_stub, ftrace_trace_function
- jnz trace
-
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
- cmpq $ftrace_stub, ftrace_graph_return
- jnz ftrace_graph_caller
-
- cmpq $ftrace_graph_entry_stub, ftrace_graph_entry
- jnz ftrace_graph_caller
-#endif
-
-GLOBAL(ftrace_stub)
- retq
-
-trace:
- MCOUNT_SAVE_FRAME
-
- movq RIP(%rsp), %rdi
-#ifdef CC_USING_FENTRY
- movq SS+16(%rsp), %rsi
-#else
- movq 8(%rbp), %rsi
-#endif
- subq $MCOUNT_INSN_SIZE, %rdi
-
- call *ftrace_trace_function
-
- MCOUNT_RESTORE_FRAME
-
- jmp ftrace_stub
-END(function_hook)
-#endif /* CONFIG_DYNAMIC_FTRACE */
-#endif /* CONFIG_FUNCTION_TRACER */
-
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
-ENTRY(ftrace_graph_caller)
- MCOUNT_SAVE_FRAME
-
-#ifdef CC_USING_FENTRY
- leaq SS+16(%rsp), %rdi
- movq $0, %rdx /* No framepointers needed */
-#else
- leaq 8(%rbp), %rdi
- movq (%rbp), %rdx
-#endif
- movq RIP(%rsp), %rsi
- subq $MCOUNT_INSN_SIZE, %rsi
-
- call prepare_ftrace_return
-
- MCOUNT_RESTORE_FRAME
-
- retq
-END(ftrace_graph_caller)
-
-GLOBAL(return_to_handler)
- subq $24, %rsp
-
- /* Save the return values */
- movq %rax, (%rsp)
- movq %rdx, 8(%rsp)
- movq %rbp, %rdi
-
- call ftrace_return_to_handler
-
- movq %rax, %rdi
- movq 8(%rsp), %rdx
- movq (%rsp), %rax
- addq $24, %rsp
- jmp *%rdi
-#endif
-
#ifndef CONFIG_PREEMPT
#define retint_kernel retint_restore_args
diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c
index 52819e8..cbc4a91 100644
--- a/arch/x86/kernel/ftrace.c
+++ b/arch/x86/kernel/ftrace.c
@@ -297,16 +297,7 @@
static int ftrace_write(unsigned long ip, const char *val, int size)
{
- /*
- * On x86_64, kernel text mappings are mapped read-only with
- * CONFIG_DEBUG_RODATA. So we use the kernel identity mapping instead
- * of the kernel text mapping to modify the kernel text.
- *
- * For 32bit kernels, these mappings are same and we can use
- * kernel identity mapping to modify code.
- */
- if (within(ip, (unsigned long)_text, (unsigned long)_etext))
- ip = (unsigned long)__va(__pa_symbol(ip));
+ ip = text_ip_addr(ip);
if (probe_kernel_write((void *)ip, val, size))
return -EPERM;
@@ -349,40 +340,14 @@
return add_break(rec->ip, old);
}
-/*
- * If the record has the FTRACE_FL_REGS set, that means that it
- * wants to convert to a callback that saves all regs. If FTRACE_FL_REGS
- * is not not set, then it wants to convert to the normal callback.
- */
-static unsigned long get_ftrace_addr(struct dyn_ftrace *rec)
-{
- if (rec->flags & FTRACE_FL_REGS)
- return (unsigned long)FTRACE_REGS_ADDR;
- else
- return (unsigned long)FTRACE_ADDR;
-}
-
-/*
- * The FTRACE_FL_REGS_EN is set when the record already points to
- * a function that saves all the regs. Basically the '_EN' version
- * represents the current state of the function.
- */
-static unsigned long get_ftrace_old_addr(struct dyn_ftrace *rec)
-{
- if (rec->flags & FTRACE_FL_REGS_EN)
- return (unsigned long)FTRACE_REGS_ADDR;
- else
- return (unsigned long)FTRACE_ADDR;
-}
-
static int add_breakpoints(struct dyn_ftrace *rec, int enable)
{
unsigned long ftrace_addr;
int ret;
- ret = ftrace_test_record(rec, enable);
+ ftrace_addr = ftrace_get_addr_curr(rec);
- ftrace_addr = get_ftrace_addr(rec);
+ ret = ftrace_test_record(rec, enable);
switch (ret) {
case FTRACE_UPDATE_IGNORE:
@@ -392,10 +357,7 @@
/* converting nop to call */
return add_brk_on_nop(rec);
- case FTRACE_UPDATE_MODIFY_CALL_REGS:
case FTRACE_UPDATE_MODIFY_CALL:
- ftrace_addr = get_ftrace_old_addr(rec);
- /* fall through */
case FTRACE_UPDATE_MAKE_NOP:
/* converting a call to a nop */
return add_brk_on_call(rec, ftrace_addr);
@@ -440,14 +402,14 @@
* If not, don't touch the breakpoint, we make just create
* a disaster.
*/
- ftrace_addr = get_ftrace_addr(rec);
+ ftrace_addr = ftrace_get_addr_new(rec);
nop = ftrace_call_replace(ip, ftrace_addr);
if (memcmp(&ins[1], &nop[1], MCOUNT_INSN_SIZE - 1) == 0)
goto update;
/* Check both ftrace_addr and ftrace_old_addr */
- ftrace_addr = get_ftrace_old_addr(rec);
+ ftrace_addr = ftrace_get_addr_curr(rec);
nop = ftrace_call_replace(ip, ftrace_addr);
if (memcmp(&ins[1], &nop[1], MCOUNT_INSN_SIZE - 1) != 0)
@@ -491,13 +453,12 @@
ret = ftrace_test_record(rec, enable);
- ftrace_addr = get_ftrace_addr(rec);
+ ftrace_addr = ftrace_get_addr_new(rec);
switch (ret) {
case FTRACE_UPDATE_IGNORE:
return 0;
- case FTRACE_UPDATE_MODIFY_CALL_REGS:
case FTRACE_UPDATE_MODIFY_CALL:
case FTRACE_UPDATE_MAKE_CALL:
/* converting nop to call */
@@ -538,13 +499,12 @@
ret = ftrace_update_record(rec, enable);
- ftrace_addr = get_ftrace_addr(rec);
+ ftrace_addr = ftrace_get_addr_new(rec);
switch (ret) {
case FTRACE_UPDATE_IGNORE:
return 0;
- case FTRACE_UPDATE_MODIFY_CALL_REGS:
case FTRACE_UPDATE_MODIFY_CALL:
case FTRACE_UPDATE_MAKE_CALL:
/* converting nop to call */
@@ -621,8 +581,8 @@
return;
remove_breakpoints:
+ pr_warn("Failed on %s (%d):\n", report, count);
ftrace_bug(ret, rec ? rec->ip : 0);
- printk(KERN_WARNING "Failed on %s (%d):\n", report, count);
for_ftrace_rec_iter(iter) {
rec = ftrace_rec_iter_record(iter);
/*
diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c
index 283a76a..11ccfb0 100644
--- a/arch/x86/kernel/irq.c
+++ b/arch/x86/kernel/irq.c
@@ -17,6 +17,7 @@
#include <asm/idle.h>
#include <asm/mce.h>
#include <asm/hw_irq.h>
+#include <asm/desc.h>
#define CREATE_TRACE_POINTS
#include <asm/trace/irq_vectors.h>
@@ -334,10 +335,17 @@
for_each_online_cpu(cpu) {
if (cpu == this_cpu)
continue;
- for (vector = FIRST_EXTERNAL_VECTOR; vector < NR_VECTORS;
- vector++) {
- if (per_cpu(vector_irq, cpu)[vector] < 0)
- count++;
+ /*
+ * We scan from FIRST_EXTERNAL_VECTOR to first system
+ * vector. If the vector is marked in the used vectors
+ * bitmap or an irq is assigned to it, we don't count
+ * it as available.
+ */
+ for (vector = FIRST_EXTERNAL_VECTOR;
+ vector < first_system_vector; vector++) {
+ if (!test_bit(vector, used_vectors) &&
+ per_cpu(vector_irq, cpu)[vector] < 0)
+ count++;
}
}
diff --git a/arch/x86/kernel/mcount_64.S b/arch/x86/kernel/mcount_64.S
new file mode 100644
index 0000000..c050a01
--- /dev/null
+++ b/arch/x86/kernel/mcount_64.S
@@ -0,0 +1,217 @@
+/*
+ * linux/arch/x86_64/mcount_64.S
+ *
+ * Copyright (C) 2014 Steven Rostedt, Red Hat Inc
+ */
+
+#include <linux/linkage.h>
+#include <asm/ptrace.h>
+#include <asm/ftrace.h>
+
+
+ .code64
+ .section .entry.text, "ax"
+
+
+#ifdef CONFIG_FUNCTION_TRACER
+
+#ifdef CC_USING_FENTRY
+# define function_hook __fentry__
+#else
+# define function_hook mcount
+#endif
+
+#ifdef CONFIG_DYNAMIC_FTRACE
+
+ENTRY(function_hook)
+ retq
+END(function_hook)
+
+/* skip is set if stack has been adjusted */
+.macro ftrace_caller_setup skip=0
+ MCOUNT_SAVE_FRAME \skip
+
+ /* Load the ftrace_ops into the 3rd parameter */
+ movq function_trace_op(%rip), %rdx
+
+ /* Load ip into the first parameter */
+ movq RIP(%rsp), %rdi
+ subq $MCOUNT_INSN_SIZE, %rdi
+ /* Load the parent_ip into the second parameter */
+#ifdef CC_USING_FENTRY
+ movq SS+16(%rsp), %rsi
+#else
+ movq 8(%rbp), %rsi
+#endif
+.endm
+
+ENTRY(ftrace_caller)
+ /* Check if tracing was disabled (quick check) */
+ cmpl $0, function_trace_stop
+ jne ftrace_stub
+
+ ftrace_caller_setup
+ /* regs go into 4th parameter (but make it NULL) */
+ movq $0, %rcx
+
+GLOBAL(ftrace_call)
+ call ftrace_stub
+
+ MCOUNT_RESTORE_FRAME
+ftrace_return:
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+GLOBAL(ftrace_graph_call)
+ jmp ftrace_stub
+#endif
+
+GLOBAL(ftrace_stub)
+ retq
+END(ftrace_caller)
+
+ENTRY(ftrace_regs_caller)
+ /* Save the current flags before compare (in SS location)*/
+ pushfq
+
+ /* Check if tracing was disabled (quick check) */
+ cmpl $0, function_trace_stop
+ jne ftrace_restore_flags
+
+ /* skip=8 to skip flags saved in SS */
+ ftrace_caller_setup 8
+
+ /* Save the rest of pt_regs */
+ movq %r15, R15(%rsp)
+ movq %r14, R14(%rsp)
+ movq %r13, R13(%rsp)
+ movq %r12, R12(%rsp)
+ movq %r11, R11(%rsp)
+ movq %r10, R10(%rsp)
+ movq %rbp, RBP(%rsp)
+ movq %rbx, RBX(%rsp)
+ /* Copy saved flags */
+ movq SS(%rsp), %rcx
+ movq %rcx, EFLAGS(%rsp)
+ /* Kernel segments */
+ movq $__KERNEL_DS, %rcx
+ movq %rcx, SS(%rsp)
+ movq $__KERNEL_CS, %rcx
+ movq %rcx, CS(%rsp)
+ /* Stack - skipping return address */
+ leaq SS+16(%rsp), %rcx
+ movq %rcx, RSP(%rsp)
+
+ /* regs go into 4th parameter */
+ leaq (%rsp), %rcx
+
+GLOBAL(ftrace_regs_call)
+ call ftrace_stub
+
+ /* Copy flags back to SS, to restore them */
+ movq EFLAGS(%rsp), %rax
+ movq %rax, SS(%rsp)
+
+ /* Handlers can change the RIP */
+ movq RIP(%rsp), %rax
+ movq %rax, SS+8(%rsp)
+
+ /* restore the rest of pt_regs */
+ movq R15(%rsp), %r15
+ movq R14(%rsp), %r14
+ movq R13(%rsp), %r13
+ movq R12(%rsp), %r12
+ movq R10(%rsp), %r10
+ movq RBP(%rsp), %rbp
+ movq RBX(%rsp), %rbx
+
+ /* skip=8 to skip flags saved in SS */
+ MCOUNT_RESTORE_FRAME 8
+
+ /* Restore flags */
+ popfq
+
+ jmp ftrace_return
+ftrace_restore_flags:
+ popfq
+ jmp ftrace_stub
+
+END(ftrace_regs_caller)
+
+
+#else /* ! CONFIG_DYNAMIC_FTRACE */
+
+ENTRY(function_hook)
+ cmpl $0, function_trace_stop
+ jne ftrace_stub
+
+ cmpq $ftrace_stub, ftrace_trace_function
+ jnz trace
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+ cmpq $ftrace_stub, ftrace_graph_return
+ jnz ftrace_graph_caller
+
+ cmpq $ftrace_graph_entry_stub, ftrace_graph_entry
+ jnz ftrace_graph_caller
+#endif
+
+GLOBAL(ftrace_stub)
+ retq
+
+trace:
+ MCOUNT_SAVE_FRAME
+
+ movq RIP(%rsp), %rdi
+#ifdef CC_USING_FENTRY
+ movq SS+16(%rsp), %rsi
+#else
+ movq 8(%rbp), %rsi
+#endif
+ subq $MCOUNT_INSN_SIZE, %rdi
+
+ call *ftrace_trace_function
+
+ MCOUNT_RESTORE_FRAME
+
+ jmp ftrace_stub
+END(function_hook)
+#endif /* CONFIG_DYNAMIC_FTRACE */
+#endif /* CONFIG_FUNCTION_TRACER */
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+ENTRY(ftrace_graph_caller)
+ MCOUNT_SAVE_FRAME
+
+#ifdef CC_USING_FENTRY
+ leaq SS+16(%rsp), %rdi
+ movq $0, %rdx /* No framepointers needed */
+#else
+ leaq 8(%rbp), %rdi
+ movq (%rbp), %rdx
+#endif
+ movq RIP(%rsp), %rsi
+ subq $MCOUNT_INSN_SIZE, %rsi
+
+ call prepare_ftrace_return
+
+ MCOUNT_RESTORE_FRAME
+
+ retq
+END(ftrace_graph_caller)
+
+GLOBAL(return_to_handler)
+ subq $24, %rsp
+
+ /* Save the return values */
+ movq %rax, (%rsp)
+ movq %rdx, 8(%rsp)
+ movq %rbp, %rdi
+
+ call ftrace_return_to_handler
+
+ movq %rax, %rdi
+ movq 8(%rsp), %rdx
+ movq (%rsp), %rax
+ addq $24, %rsp
+ jmp *%rdi
+#endif
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index 5d93ac1b..5492798 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -866,9 +866,6 @@
/* was set by cpu_init() */
cpumask_clear_cpu(cpu, cpu_initialized_mask);
-
- set_cpu_present(cpu, false);
- per_cpu(x86_cpu_to_apicid, cpu) = BAD_APICID;
}
/* mark "stuck" area as not stuck */
@@ -928,7 +925,7 @@
err = do_boot_cpu(apicid, cpu, tidle);
if (err) {
- pr_debug("do_boot_cpu failed %d\n", err);
+ pr_err("do_boot_cpu failed(%d) to wakeup CPU#%u\n", err, cpu);
return -EIO;
}
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index 835b248..87fc96b 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -917,6 +917,9 @@
void *tmp, *p, *q = NULL;
int count = 0;
+ if (efi_enabled(EFI_OLD_MEMMAP))
+ return;
+
for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) {
md = p;
diff --git a/arch/x86/platform/intel-mid/device_libs/Makefile b/arch/x86/platform/intel-mid/device_libs/Makefile
index 097e7a7..af9307f 100644
--- a/arch/x86/platform/intel-mid/device_libs/Makefile
+++ b/arch/x86/platform/intel-mid/device_libs/Makefile
@@ -20,3 +20,4 @@
obj-$(subst m,y,$(CONFIG_SERIAL_MRST_MAX3110)) += platform_max3111.o
# MISC Devices
obj-$(subst m,y,$(CONFIG_KEYBOARD_GPIO)) += platform_gpio_keys.o
+obj-$(subst m,y,$(CONFIG_INTEL_MID_WATCHDOG)) += platform_wdt.o
diff --git a/arch/x86/platform/intel-mid/device_libs/platform_wdt.c b/arch/x86/platform/intel-mid/device_libs/platform_wdt.c
new file mode 100644
index 0000000..973cf3b
--- /dev/null
+++ b/arch/x86/platform/intel-mid/device_libs/platform_wdt.c
@@ -0,0 +1,72 @@
+/*
+ * platform_wdt.c: Watchdog platform library file
+ *
+ * (C) Copyright 2014 Intel Corporation
+ * Author: David Cohen <david.a.cohen@linux.intel.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2
+ * of the License.
+ */
+
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/intel-mid_wdt.h>
+#include <asm/intel-mid.h>
+#include <asm/io_apic.h>
+
+#define TANGIER_EXT_TIMER0_MSI 15
+
+static struct platform_device wdt_dev = {
+ .name = "intel_mid_wdt",
+ .id = -1,
+};
+
+static int tangier_probe(struct platform_device *pdev)
+{
+ int ioapic;
+ int irq;
+ struct intel_mid_wdt_pdata *pdata = pdev->dev.platform_data;
+ struct io_apic_irq_attr irq_attr = { 0 };
+
+ if (!pdata)
+ return -EINVAL;
+
+ irq = pdata->irq;
+ ioapic = mp_find_ioapic(irq);
+ if (ioapic >= 0) {
+ int ret;
+ irq_attr.ioapic = ioapic;
+ irq_attr.ioapic_pin = irq;
+ irq_attr.trigger = 1;
+ /* irq_attr.polarity = 0; -> Active high */
+ ret = io_apic_set_pci_routing(NULL, irq, &irq_attr);
+ if (ret)
+ return ret;
+ } else {
+ dev_warn(&pdev->dev, "cannot find interrupt %d in ioapic\n",
+ irq);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static struct intel_mid_wdt_pdata tangier_pdata = {
+ .irq = TANGIER_EXT_TIMER0_MSI,
+ .probe = tangier_probe,
+};
+
+static int __init register_mid_wdt(void)
+{
+ if (intel_mid_identify_cpu() == INTEL_MID_CPU_CHIP_TANGIER) {
+ wdt_dev.dev.platform_data = &tangier_pdata;
+ return platform_device_register(&wdt_dev);
+ }
+
+ return -ENODEV;
+}
+
+rootfs_initcall(register_mid_wdt);
diff --git a/arch/x86/vdso/Makefile b/arch/x86/vdso/Makefile
index 895d4b1..9769df0 100644
--- a/arch/x86/vdso/Makefile
+++ b/arch/x86/vdso/Makefile
@@ -59,6 +59,7 @@
$(obj)/vdso64.so.dbg: $(src)/vdso.lds $(vobjs) FORCE
$(call if_changed,vdso)
+HOST_EXTRACFLAGS += -I$(srctree)/tools/include
hostprogs-y += vdso2c
quiet_cmd_vdso2c = VDSO2C $@
diff --git a/arch/x86/vdso/vdso2c.c b/arch/x86/vdso/vdso2c.c
index deabaf5..450ac6e 100644
--- a/arch/x86/vdso/vdso2c.c
+++ b/arch/x86/vdso/vdso2c.c
@@ -11,6 +11,8 @@
#include <sys/mman.h>
#include <sys/types.h>
+#include <tools/le_byteshift.h>
+
#include <linux/elf.h>
#include <linux/types.h>
@@ -56,12 +58,12 @@
*/
#define GLE(x, bits, ifnot) \
__builtin_choose_expr( \
- (sizeof(x) == bits/8), \
- (__typeof__(x))le##bits##toh(x), ifnot)
+ (sizeof(*(x)) == bits/8), \
+ (__typeof__(*(x)))get_unaligned_le##bits(x), ifnot)
-extern void bad_get_le(uint64_t);
+extern void bad_get_le(void);
#define LAST_LE(x) \
- __builtin_choose_expr(sizeof(x) == 1, (x), bad_get_le(x))
+ __builtin_choose_expr(sizeof(*(x)) == 1, *(x), bad_get_le())
#define GET_LE(x) \
GLE(x, 64, GLE(x, 32, GLE(x, 16, LAST_LE(x))))
diff --git a/arch/x86/vdso/vdso2c.h b/arch/x86/vdso/vdso2c.h
index d1e99e1..8a07463 100644
--- a/arch/x86/vdso/vdso2c.h
+++ b/arch/x86/vdso/vdso2c.h
@@ -18,27 +18,27 @@
const char *secstrings;
uint64_t syms[NSYMS] = {};
- Elf_Phdr *pt = (Elf_Phdr *)(addr + GET_LE(hdr->e_phoff));
+ Elf_Phdr *pt = (Elf_Phdr *)(addr + GET_LE(&hdr->e_phoff));
/* Walk the segment table. */
- for (i = 0; i < GET_LE(hdr->e_phnum); i++) {
- if (GET_LE(pt[i].p_type) == PT_LOAD) {
+ for (i = 0; i < GET_LE(&hdr->e_phnum); i++) {
+ if (GET_LE(&pt[i].p_type) == PT_LOAD) {
if (found_load)
fail("multiple PT_LOAD segs\n");
- if (GET_LE(pt[i].p_offset) != 0 ||
- GET_LE(pt[i].p_vaddr) != 0)
+ if (GET_LE(&pt[i].p_offset) != 0 ||
+ GET_LE(&pt[i].p_vaddr) != 0)
fail("PT_LOAD in wrong place\n");
- if (GET_LE(pt[i].p_memsz) != GET_LE(pt[i].p_filesz))
+ if (GET_LE(&pt[i].p_memsz) != GET_LE(&pt[i].p_filesz))
fail("cannot handle memsz != filesz\n");
- load_size = GET_LE(pt[i].p_memsz);
+ load_size = GET_LE(&pt[i].p_memsz);
found_load = 1;
- } else if (GET_LE(pt[i].p_type) == PT_DYNAMIC) {
- dyn = addr + GET_LE(pt[i].p_offset);
- dyn_end = addr + GET_LE(pt[i].p_offset) +
- GET_LE(pt[i].p_memsz);
+ } else if (GET_LE(&pt[i].p_type) == PT_DYNAMIC) {
+ dyn = addr + GET_LE(&pt[i].p_offset);
+ dyn_end = addr + GET_LE(&pt[i].p_offset) +
+ GET_LE(&pt[i].p_memsz);
}
}
if (!found_load)
@@ -47,24 +47,24 @@
/* Walk the dynamic table */
for (i = 0; dyn + i < dyn_end &&
- GET_LE(dyn[i].d_tag) != DT_NULL; i++) {
- typeof(dyn[i].d_tag) tag = GET_LE(dyn[i].d_tag);
+ GET_LE(&dyn[i].d_tag) != DT_NULL; i++) {
+ typeof(dyn[i].d_tag) tag = GET_LE(&dyn[i].d_tag);
if (tag == DT_REL || tag == DT_RELSZ ||
tag == DT_RELENT || tag == DT_TEXTREL)
fail("vdso image contains dynamic relocations\n");
}
/* Walk the section table */
- secstrings_hdr = addr + GET_LE(hdr->e_shoff) +
- GET_LE(hdr->e_shentsize)*GET_LE(hdr->e_shstrndx);
- secstrings = addr + GET_LE(secstrings_hdr->sh_offset);
- for (i = 0; i < GET_LE(hdr->e_shnum); i++) {
- Elf_Shdr *sh = addr + GET_LE(hdr->e_shoff) +
- GET_LE(hdr->e_shentsize) * i;
- if (GET_LE(sh->sh_type) == SHT_SYMTAB)
+ secstrings_hdr = addr + GET_LE(&hdr->e_shoff) +
+ GET_LE(&hdr->e_shentsize)*GET_LE(&hdr->e_shstrndx);
+ secstrings = addr + GET_LE(&secstrings_hdr->sh_offset);
+ for (i = 0; i < GET_LE(&hdr->e_shnum); i++) {
+ Elf_Shdr *sh = addr + GET_LE(&hdr->e_shoff) +
+ GET_LE(&hdr->e_shentsize) * i;
+ if (GET_LE(&sh->sh_type) == SHT_SYMTAB)
symtab_hdr = sh;
- if (!strcmp(secstrings + GET_LE(sh->sh_name),
+ if (!strcmp(secstrings + GET_LE(&sh->sh_name),
".altinstructions"))
alt_sec = sh;
}
@@ -72,25 +72,25 @@
if (!symtab_hdr)
fail("no symbol table\n");
- strtab_hdr = addr + GET_LE(hdr->e_shoff) +
- GET_LE(hdr->e_shentsize) * GET_LE(symtab_hdr->sh_link);
+ strtab_hdr = addr + GET_LE(&hdr->e_shoff) +
+ GET_LE(&hdr->e_shentsize) * GET_LE(&symtab_hdr->sh_link);
/* Walk the symbol table */
for (i = 0;
- i < GET_LE(symtab_hdr->sh_size) / GET_LE(symtab_hdr->sh_entsize);
+ i < GET_LE(&symtab_hdr->sh_size) / GET_LE(&symtab_hdr->sh_entsize);
i++) {
int k;
- Elf_Sym *sym = addr + GET_LE(symtab_hdr->sh_offset) +
- GET_LE(symtab_hdr->sh_entsize) * i;
- const char *name = addr + GET_LE(strtab_hdr->sh_offset) +
- GET_LE(sym->st_name);
+ Elf_Sym *sym = addr + GET_LE(&symtab_hdr->sh_offset) +
+ GET_LE(&symtab_hdr->sh_entsize) * i;
+ const char *name = addr + GET_LE(&strtab_hdr->sh_offset) +
+ GET_LE(&sym->st_name);
for (k = 0; k < NSYMS; k++) {
if (!strcmp(name, required_syms[k])) {
if (syms[k]) {
fail("duplicate symbol %s\n",
required_syms[k]);
}
- syms[k] = GET_LE(sym->st_value);
+ syms[k] = GET_LE(&sym->st_value);
}
}
}
@@ -150,9 +150,9 @@
fprintf(outfile, "\t},\n");
if (alt_sec) {
fprintf(outfile, "\t.alt = %lu,\n",
- (unsigned long)GET_LE(alt_sec->sh_offset));
+ (unsigned long)GET_LE(&alt_sec->sh_offset));
fprintf(outfile, "\t.alt_len = %lu,\n",
- (unsigned long)GET_LE(alt_sec->sh_size));
+ (unsigned long)GET_LE(&alt_sec->sh_size));
}
for (i = 0; i < NSYMS; i++) {
if (syms[i])
diff --git a/arch/xtensa/include/asm/ftrace.h b/arch/xtensa/include/asm/ftrace.h
index 736b9d2..6c6d9a9 100644
--- a/arch/xtensa/include/asm/ftrace.h
+++ b/arch/xtensa/include/asm/ftrace.h
@@ -12,24 +12,18 @@
#include <asm/processor.h>
-#define HAVE_ARCH_CALLER_ADDR
#ifndef __ASSEMBLY__
-#define CALLER_ADDR0 ({ unsigned long a0, a1; \
+#define ftrace_return_address0 ({ unsigned long a0, a1; \
__asm__ __volatile__ ( \
"mov %0, a0\n" \
"mov %1, a1\n" \
: "=r"(a0), "=r"(a1)); \
MAKE_PC_FROM_RA(a0, a1); })
+
#ifdef CONFIG_FRAME_POINTER
extern unsigned long return_address(unsigned level);
-#define CALLER_ADDR1 return_address(1)
-#define CALLER_ADDR2 return_address(2)
-#define CALLER_ADDR3 return_address(3)
-#else /* CONFIG_FRAME_POINTER */
-#define CALLER_ADDR1 (0)
-#define CALLER_ADDR2 (0)
-#define CALLER_ADDR3 (0)
-#endif /* CONFIG_FRAME_POINTER */
+#define ftrace_return_address(n) return_address(n)
+#endif
#endif /* __ASSEMBLY__ */
#ifdef CONFIG_FUNCTION_TRACER
diff --git a/block/bio.c b/block/bio.c
index 2d64488..8c2e55e 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1977,7 +1977,7 @@
/* associate blkcg if exists */
rcu_read_lock();
css = task_css(current, blkio_cgrp_id);
- if (css && css_tryget(css))
+ if (css && css_tryget_online(css))
bio->bi_css = css;
rcu_read_unlock();
diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
index 5aa9b8b..069bc20 100644
--- a/block/blk-cgroup.c
+++ b/block/blk-cgroup.c
@@ -185,7 +185,7 @@
lockdep_assert_held(q->queue_lock);
/* blkg holds a reference to blkcg */
- if (!css_tryget(&blkcg->css)) {
+ if (!css_tryget_online(&blkcg->css)) {
ret = -EINVAL;
goto err_free_blkg;
}
diff --git a/block/blk-cgroup.h b/block/blk-cgroup.h
index 5480ade..cbb7f94 100644
--- a/block/blk-cgroup.h
+++ b/block/blk-cgroup.h
@@ -204,7 +204,7 @@
*/
static inline struct blkcg *blkcg_parent(struct blkcg *blkcg)
{
- return css_to_blkcg(css_parent(&blkcg->css));
+ return css_to_blkcg(blkcg->css.parent);
}
/**
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 9353b46..3fdb21a 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -1346,10 +1346,10 @@
return 0;
}
-static int tg_set_conf(struct cgroup_subsys_state *css, struct cftype *cft,
- const char *buf, bool is_u64)
+static ssize_t tg_set_conf(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off, bool is_u64)
{
- struct blkcg *blkcg = css_to_blkcg(css);
+ struct blkcg *blkcg = css_to_blkcg(of_css(of));
struct blkg_conf_ctx ctx;
struct throtl_grp *tg;
struct throtl_service_queue *sq;
@@ -1368,9 +1368,9 @@
ctx.v = -1;
if (is_u64)
- *(u64 *)((void *)tg + cft->private) = ctx.v;
+ *(u64 *)((void *)tg + of_cft(of)->private) = ctx.v;
else
- *(unsigned int *)((void *)tg + cft->private) = ctx.v;
+ *(unsigned int *)((void *)tg + of_cft(of)->private) = ctx.v;
throtl_log(&tg->service_queue,
"limit change rbps=%llu wbps=%llu riops=%u wiops=%u",
@@ -1404,19 +1404,19 @@
}
blkg_conf_finish(&ctx);
- return 0;
+ return nbytes;
}
-static int tg_set_conf_u64(struct cgroup_subsys_state *css, struct cftype *cft,
- char *buf)
+static ssize_t tg_set_conf_u64(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
- return tg_set_conf(css, cft, buf, true);
+ return tg_set_conf(of, buf, nbytes, off, true);
}
-static int tg_set_conf_uint(struct cgroup_subsys_state *css, struct cftype *cft,
- char *buf)
+static ssize_t tg_set_conf_uint(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
- return tg_set_conf(css, cft, buf, false);
+ return tg_set_conf(of, buf, nbytes, off, false);
}
static struct cftype throtl_files[] = {
@@ -1424,25 +1424,25 @@
.name = "throttle.read_bps_device",
.private = offsetof(struct throtl_grp, bps[READ]),
.seq_show = tg_print_conf_u64,
- .write_string = tg_set_conf_u64,
+ .write = tg_set_conf_u64,
},
{
.name = "throttle.write_bps_device",
.private = offsetof(struct throtl_grp, bps[WRITE]),
.seq_show = tg_print_conf_u64,
- .write_string = tg_set_conf_u64,
+ .write = tg_set_conf_u64,
},
{
.name = "throttle.read_iops_device",
.private = offsetof(struct throtl_grp, iops[READ]),
.seq_show = tg_print_conf_uint,
- .write_string = tg_set_conf_uint,
+ .write = tg_set_conf_uint,
},
{
.name = "throttle.write_iops_device",
.private = offsetof(struct throtl_grp, iops[WRITE]),
.seq_show = tg_print_conf_uint,
- .write_string = tg_set_conf_uint,
+ .write = tg_set_conf_uint,
},
{
.name = "throttle.io_service_bytes",
diff --git a/block/bounce.c b/block/bounce.c
index 523918b..ab21ba2 100644
--- a/block/bounce.c
+++ b/block/bounce.c
@@ -3,6 +3,8 @@
* - Split from highmem.c
*/
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/mm.h>
#include <linux/export.h>
#include <linux/swap.h>
@@ -15,6 +17,7 @@
#include <linux/hash.h>
#include <linux/highmem.h>
#include <linux/bootmem.h>
+#include <linux/printk.h>
#include <asm/tlbflush.h>
#include <trace/events/block.h>
@@ -34,7 +37,7 @@
page_pool = mempool_create_page_pool(POOL_SIZE, 0);
BUG_ON(!page_pool);
- printk("bounce pool size: %d pages\n", POOL_SIZE);
+ pr_info("pool size: %d pages\n", POOL_SIZE);
return 0;
}
@@ -86,7 +89,7 @@
mempool_free_pages, (void *) 0);
BUG_ON(!isa_page_pool);
- printk("isa bounce pool size: %d pages\n", ISA_POOL_SIZE);
+ pr_info("isa pool size: %d pages\n", ISA_POOL_SIZE);
return 0;
}
diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c
index 22dffeb..cadc378 100644
--- a/block/cfq-iosched.c
+++ b/block/cfq-iosched.c
@@ -1670,11 +1670,11 @@
return 0;
}
-static int __cfqg_set_weight_device(struct cgroup_subsys_state *css,
- struct cftype *cft, const char *buf,
- bool is_leaf_weight)
+static ssize_t __cfqg_set_weight_device(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off,
+ bool is_leaf_weight)
{
- struct blkcg *blkcg = css_to_blkcg(css);
+ struct blkcg *blkcg = css_to_blkcg(of_css(of));
struct blkg_conf_ctx ctx;
struct cfq_group *cfqg;
int ret;
@@ -1697,19 +1697,19 @@
}
blkg_conf_finish(&ctx);
- return ret;
+ return ret ?: nbytes;
}
-static int cfqg_set_weight_device(struct cgroup_subsys_state *css,
- struct cftype *cft, char *buf)
+static ssize_t cfqg_set_weight_device(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
- return __cfqg_set_weight_device(css, cft, buf, false);
+ return __cfqg_set_weight_device(of, buf, nbytes, off, false);
}
-static int cfqg_set_leaf_weight_device(struct cgroup_subsys_state *css,
- struct cftype *cft, char *buf)
+static ssize_t cfqg_set_leaf_weight_device(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
- return __cfqg_set_weight_device(css, cft, buf, true);
+ return __cfqg_set_weight_device(of, buf, nbytes, off, true);
}
static int __cfq_set_weight(struct cgroup_subsys_state *css, struct cftype *cft,
@@ -1837,7 +1837,7 @@
.name = "weight_device",
.flags = CFTYPE_ONLY_ON_ROOT,
.seq_show = cfqg_print_leaf_weight_device,
- .write_string = cfqg_set_leaf_weight_device,
+ .write = cfqg_set_leaf_weight_device,
},
{
.name = "weight",
@@ -1851,7 +1851,7 @@
.name = "weight_device",
.flags = CFTYPE_NOT_ON_ROOT,
.seq_show = cfqg_print_weight_device,
- .write_string = cfqg_set_weight_device,
+ .write = cfqg_set_weight_device,
},
{
.name = "weight",
@@ -1863,7 +1863,7 @@
{
.name = "leaf_weight_device",
.seq_show = cfqg_print_leaf_weight_device,
- .write_string = cfqg_set_leaf_weight_device,
+ .write = cfqg_set_leaf_weight_device,
},
{
.name = "leaf_weight",
diff --git a/crypto/ahash.c b/crypto/ahash.c
index 6e72233..f2a5d8f 100644
--- a/crypto/ahash.c
+++ b/crypto/ahash.c
@@ -15,6 +15,7 @@
#include <crypto/internal/hash.h>
#include <crypto/scatterwalk.h>
+#include <linux/bug.h>
#include <linux/err.h>
#include <linux/kernel.h>
#include <linux/module.h>
@@ -46,7 +47,10 @@
unsigned int nbytes = min(walk->entrylen,
((unsigned int)(PAGE_SIZE)) - offset);
- walk->data = kmap_atomic(walk->pg);
+ if (walk->flags & CRYPTO_ALG_ASYNC)
+ walk->data = kmap(walk->pg);
+ else
+ walk->data = kmap_atomic(walk->pg);
walk->data += offset;
if (offset & alignmask) {
@@ -93,8 +97,16 @@
return nbytes;
}
- kunmap_atomic(walk->data);
- crypto_yield(walk->flags);
+ if (walk->flags & CRYPTO_ALG_ASYNC)
+ kunmap(walk->pg);
+ else {
+ kunmap_atomic(walk->data);
+ /*
+ * The may sleep test only makes sense for sync users.
+ * Async users don't need to sleep here anyway.
+ */
+ crypto_yield(walk->flags);
+ }
if (err)
return err;
@@ -124,12 +136,31 @@
walk->alignmask = crypto_ahash_alignmask(crypto_ahash_reqtfm(req));
walk->sg = req->src;
- walk->flags = req->base.flags;
+ walk->flags = req->base.flags & CRYPTO_TFM_REQ_MASK;
return hash_walk_new_entry(walk);
}
EXPORT_SYMBOL_GPL(crypto_hash_walk_first);
+int crypto_ahash_walk_first(struct ahash_request *req,
+ struct crypto_hash_walk *walk)
+{
+ walk->total = req->nbytes;
+
+ if (!walk->total)
+ return 0;
+
+ walk->alignmask = crypto_ahash_alignmask(crypto_ahash_reqtfm(req));
+ walk->sg = req->src;
+ walk->flags = req->base.flags & CRYPTO_TFM_REQ_MASK;
+ walk->flags |= CRYPTO_ALG_ASYNC;
+
+ BUILD_BUG_ON(CRYPTO_TFM_REQ_MASK & CRYPTO_ALG_ASYNC);
+
+ return hash_walk_new_entry(walk);
+}
+EXPORT_SYMBOL_GPL(crypto_ahash_walk_first);
+
int crypto_hash_walk_first_compat(struct hash_desc *hdesc,
struct crypto_hash_walk *walk,
struct scatterlist *sg, unsigned int len)
@@ -141,7 +172,7 @@
walk->alignmask = crypto_hash_alignmask(hdesc->tfm);
walk->sg = sg;
- walk->flags = hdesc->flags;
+ walk->flags = hdesc->flags & CRYPTO_TFM_REQ_MASK;
return hash_walk_new_entry(walk);
}
diff --git a/crypto/crypto_user.c b/crypto/crypto_user.c
index 43665d0..e2a34fe 100644
--- a/crypto/crypto_user.c
+++ b/crypto/crypto_user.c
@@ -265,6 +265,9 @@
struct nlattr *priority = attrs[CRYPTOCFGA_PRIORITY_VAL];
LIST_HEAD(list);
+ if (!netlink_capable(skb, CAP_NET_ADMIN))
+ return -EPERM;
+
if (!null_terminated(p->cru_name) || !null_terminated(p->cru_driver_name))
return -EINVAL;
@@ -295,6 +298,9 @@
struct crypto_alg *alg;
struct crypto_user_alg *p = nlmsg_data(nlh);
+ if (!netlink_capable(skb, CAP_NET_ADMIN))
+ return -EPERM;
+
if (!null_terminated(p->cru_name) || !null_terminated(p->cru_driver_name))
return -EINVAL;
@@ -379,6 +385,9 @@
struct crypto_user_alg *p = nlmsg_data(nlh);
struct nlattr *priority = attrs[CRYPTOCFGA_PRIORITY_VAL];
+ if (!netlink_capable(skb, CAP_NET_ADMIN))
+ return -EPERM;
+
if (!null_terminated(p->cru_name) || !null_terminated(p->cru_driver_name))
return -EINVAL;
@@ -466,9 +475,6 @@
type -= CRYPTO_MSG_BASE;
link = &crypto_dispatch[type];
- if (!netlink_capable(skb, CAP_NET_ADMIN))
- return -EPERM;
-
if ((type == (CRYPTO_MSG_GETALG - CRYPTO_MSG_BASE) &&
(nlh->nlmsg_flags & NLM_F_DUMP))) {
struct crypto_alg *alg;
diff --git a/crypto/shash.c b/crypto/shash.c
index 929058a..47c7139 100644
--- a/crypto/shash.c
+++ b/crypto/shash.c
@@ -67,7 +67,8 @@
static inline unsigned int shash_align_buffer_size(unsigned len,
unsigned long mask)
{
- return len + (mask & ~(__alignof__(u8 __attribute__ ((aligned))) - 1));
+ typedef u8 __attribute__ ((aligned)) u8_aligned;
+ return len + (mask & ~(__alignof__(u8_aligned) - 1));
}
static int shash_update_unaligned(struct shash_desc *desc, const u8 *data,
diff --git a/crypto/tcrypt.c b/crypto/tcrypt.c
index 870be7b..ba247cf 100644
--- a/crypto/tcrypt.c
+++ b/crypto/tcrypt.c
@@ -282,6 +282,11 @@
unsigned int *b_size;
unsigned int iv_len;
+ if (aad_size >= PAGE_SIZE) {
+ pr_err("associate data length (%u) too big\n", aad_size);
+ return;
+ }
+
if (enc == ENCRYPT)
e = "encryption";
else
@@ -308,14 +313,14 @@
if (IS_ERR(tfm)) {
pr_err("alg: aead: Failed to load transform for %s: %ld\n", algo,
PTR_ERR(tfm));
- return;
+ goto out_notfm;
}
req = aead_request_alloc(tfm, GFP_KERNEL);
if (!req) {
pr_err("alg: aead: Failed to allocate request for %s\n",
algo);
- goto out;
+ goto out_noreq;
}
i = 0;
@@ -323,14 +328,7 @@
b_size = aead_sizes;
do {
assoc = axbuf[0];
-
- if (aad_size < PAGE_SIZE)
- memset(assoc, 0xff, aad_size);
- else {
- pr_err("associate data length (%u) too big\n",
- aad_size);
- goto out_nosg;
- }
+ memset(assoc, 0xff, aad_size);
sg_init_one(&asg[0], assoc, aad_size);
if ((*keysize + *b_size) > TVMEMSIZE * PAGE_SIZE) {
@@ -392,7 +390,10 @@
} while (*keysize);
out:
+ aead_request_free(req);
+out_noreq:
crypto_free_aead(tfm);
+out_notfm:
kfree(sg);
out_nosg:
testmgr_free_buf(xoutbuf);
@@ -1518,7 +1519,36 @@
case 157:
ret += tcrypt_test("authenc(hmac(sha1),ecb(cipher_null))");
break;
-
+ case 181:
+ ret += tcrypt_test("authenc(hmac(sha1),cbc(des))");
+ break;
+ case 182:
+ ret += tcrypt_test("authenc(hmac(sha1),cbc(des3_ede))");
+ break;
+ case 183:
+ ret += tcrypt_test("authenc(hmac(sha224),cbc(des))");
+ break;
+ case 184:
+ ret += tcrypt_test("authenc(hmac(sha224),cbc(des3_ede))");
+ break;
+ case 185:
+ ret += tcrypt_test("authenc(hmac(sha256),cbc(des))");
+ break;
+ case 186:
+ ret += tcrypt_test("authenc(hmac(sha256),cbc(des3_ede))");
+ break;
+ case 187:
+ ret += tcrypt_test("authenc(hmac(sha384),cbc(des))");
+ break;
+ case 188:
+ ret += tcrypt_test("authenc(hmac(sha384),cbc(des3_ede))");
+ break;
+ case 189:
+ ret += tcrypt_test("authenc(hmac(sha512),cbc(des))");
+ break;
+ case 190:
+ ret += tcrypt_test("authenc(hmac(sha512),cbc(des3_ede))");
+ break;
case 200:
test_cipher_speed("ecb(aes)", ENCRYPT, sec, NULL, 0,
speed_template_16_24_32);
diff --git a/crypto/testmgr.c b/crypto/testmgr.c
index dc3cf35..498649a 100644
--- a/crypto/testmgr.c
+++ b/crypto/testmgr.c
@@ -414,16 +414,18 @@
void *input;
void *output;
void *assoc;
- char iv[MAX_IVLEN];
+ char *iv;
char *xbuf[XBUFSIZE];
char *xoutbuf[XBUFSIZE];
char *axbuf[XBUFSIZE];
+ iv = kzalloc(MAX_IVLEN, GFP_KERNEL);
+ if (!iv)
+ return ret;
if (testmgr_alloc_buf(xbuf))
goto out_noxbuf;
if (testmgr_alloc_buf(axbuf))
goto out_noaxbuf;
-
if (diff_dst && testmgr_alloc_buf(xoutbuf))
goto out_nooutbuf;
@@ -767,6 +769,7 @@
out_noaxbuf:
testmgr_free_buf(xbuf);
out_noxbuf:
+ kfree(iv);
return ret;
}
@@ -1831,8 +1834,38 @@
.suite = {
.aead = {
.enc = {
- .vecs = hmac_sha1_aes_cbc_enc_tv_template,
- .count = HMAC_SHA1_AES_CBC_ENC_TEST_VECTORS
+ .vecs =
+ hmac_sha1_aes_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA1_AES_CBC_ENC_TEST_VEC
+ }
+ }
+ }
+ }, {
+ .alg = "authenc(hmac(sha1),cbc(des))",
+ .test = alg_test_aead,
+ .fips_allowed = 1,
+ .suite = {
+ .aead = {
+ .enc = {
+ .vecs =
+ hmac_sha1_des_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA1_DES_CBC_ENC_TEST_VEC
+ }
+ }
+ }
+ }, {
+ .alg = "authenc(hmac(sha1),cbc(des3_ede))",
+ .test = alg_test_aead,
+ .fips_allowed = 1,
+ .suite = {
+ .aead = {
+ .enc = {
+ .vecs =
+ hmac_sha1_des3_ede_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA1_DES3_EDE_CBC_ENC_TEST_VEC
}
}
}
@@ -1843,12 +1876,44 @@
.suite = {
.aead = {
.enc = {
- .vecs = hmac_sha1_ecb_cipher_null_enc_tv_template,
- .count = HMAC_SHA1_ECB_CIPHER_NULL_ENC_TEST_VECTORS
+ .vecs =
+ hmac_sha1_ecb_cipher_null_enc_tv_temp,
+ .count =
+ HMAC_SHA1_ECB_CIPHER_NULL_ENC_TEST_VEC
},
.dec = {
- .vecs = hmac_sha1_ecb_cipher_null_dec_tv_template,
- .count = HMAC_SHA1_ECB_CIPHER_NULL_DEC_TEST_VECTORS
+ .vecs =
+ hmac_sha1_ecb_cipher_null_dec_tv_temp,
+ .count =
+ HMAC_SHA1_ECB_CIPHER_NULL_DEC_TEST_VEC
+ }
+ }
+ }
+ }, {
+ .alg = "authenc(hmac(sha224),cbc(des))",
+ .test = alg_test_aead,
+ .fips_allowed = 1,
+ .suite = {
+ .aead = {
+ .enc = {
+ .vecs =
+ hmac_sha224_des_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA224_DES_CBC_ENC_TEST_VEC
+ }
+ }
+ }
+ }, {
+ .alg = "authenc(hmac(sha224),cbc(des3_ede))",
+ .test = alg_test_aead,
+ .fips_allowed = 1,
+ .suite = {
+ .aead = {
+ .enc = {
+ .vecs =
+ hmac_sha224_des3_ede_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA224_DES3_EDE_CBC_ENC_TEST_VEC
}
}
}
@@ -1859,8 +1924,66 @@
.suite = {
.aead = {
.enc = {
- .vecs = hmac_sha256_aes_cbc_enc_tv_template,
- .count = HMAC_SHA256_AES_CBC_ENC_TEST_VECTORS
+ .vecs =
+ hmac_sha256_aes_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA256_AES_CBC_ENC_TEST_VEC
+ }
+ }
+ }
+ }, {
+ .alg = "authenc(hmac(sha256),cbc(des))",
+ .test = alg_test_aead,
+ .fips_allowed = 1,
+ .suite = {
+ .aead = {
+ .enc = {
+ .vecs =
+ hmac_sha256_des_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA256_DES_CBC_ENC_TEST_VEC
+ }
+ }
+ }
+ }, {
+ .alg = "authenc(hmac(sha256),cbc(des3_ede))",
+ .test = alg_test_aead,
+ .fips_allowed = 1,
+ .suite = {
+ .aead = {
+ .enc = {
+ .vecs =
+ hmac_sha256_des3_ede_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA256_DES3_EDE_CBC_ENC_TEST_VEC
+ }
+ }
+ }
+ }, {
+ .alg = "authenc(hmac(sha384),cbc(des))",
+ .test = alg_test_aead,
+ .fips_allowed = 1,
+ .suite = {
+ .aead = {
+ .enc = {
+ .vecs =
+ hmac_sha384_des_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA384_DES_CBC_ENC_TEST_VEC
+ }
+ }
+ }
+ }, {
+ .alg = "authenc(hmac(sha384),cbc(des3_ede))",
+ .test = alg_test_aead,
+ .fips_allowed = 1,
+ .suite = {
+ .aead = {
+ .enc = {
+ .vecs =
+ hmac_sha384_des3_ede_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA384_DES3_EDE_CBC_ENC_TEST_VEC
}
}
}
@@ -1871,8 +1994,38 @@
.suite = {
.aead = {
.enc = {
- .vecs = hmac_sha512_aes_cbc_enc_tv_template,
- .count = HMAC_SHA512_AES_CBC_ENC_TEST_VECTORS
+ .vecs =
+ hmac_sha512_aes_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA512_AES_CBC_ENC_TEST_VEC
+ }
+ }
+ }
+ }, {
+ .alg = "authenc(hmac(sha512),cbc(des))",
+ .test = alg_test_aead,
+ .fips_allowed = 1,
+ .suite = {
+ .aead = {
+ .enc = {
+ .vecs =
+ hmac_sha512_des_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA512_DES_CBC_ENC_TEST_VEC
+ }
+ }
+ }
+ }, {
+ .alg = "authenc(hmac(sha512),cbc(des3_ede))",
+ .test = alg_test_aead,
+ .fips_allowed = 1,
+ .suite = {
+ .aead = {
+ .enc = {
+ .vecs =
+ hmac_sha512_des3_ede_cbc_enc_tv_temp,
+ .count =
+ HMAC_SHA512_DES3_EDE_CBC_ENC_TEST_VEC
}
}
}
@@ -3273,8 +3426,8 @@
panic("%s: %s alg self test failed in fips mode!\n", driver, alg);
if (fips_enabled && !rc)
- printk(KERN_INFO "alg: self-tests for %s (%s) passed\n",
- driver, alg);
+ pr_info(KERN_INFO "alg: self-tests for %s (%s) passed\n",
+ driver, alg);
return rc;
diff --git a/crypto/testmgr.h b/crypto/testmgr.h
index 3db83db..69d0dd8e 100644
--- a/crypto/testmgr.h
+++ b/crypto/testmgr.h
@@ -487,10 +487,15 @@
* SHA1 test vectors from from FIPS PUB 180-1
* Long vector from CAVS 5.0
*/
-#define SHA1_TEST_VECTORS 3
+#define SHA1_TEST_VECTORS 6
static struct hash_testvec sha1_tv_template[] = {
{
+ .plaintext = "",
+ .psize = 0,
+ .digest = "\xda\x39\xa3\xee\x5e\x6b\x4b\x0d\x32\x55"
+ "\xbf\xef\x95\x60\x18\x90\xaf\xd8\x07\x09",
+ }, {
.plaintext = "abc",
.psize = 3,
.digest = "\xa9\x99\x3e\x36\x47\x06\x81\x6a\xba\x3e"
@@ -529,6 +534,144 @@
"\x45\x9c\x02\xb6\x9b\x4a\xa8\xf5\x82\x17",
.np = 4,
.tap = { 63, 64, 31, 5 }
+ }, {
+ .plaintext = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-",
+ .psize = 64,
+ .digest = "\xc8\x71\xf6\x9a\x63\xcc\xa9\x84\x84\x82"
+ "\x64\xe7\x79\x95\x5d\xd7\x19\x41\x7c\x91",
+ }, {
+ .plaintext = "\x08\x9f\x13\xaa\x41\xd8\x4c\xe3"
+ "\x7a\x11\x85\x1c\xb3\x27\xbe\x55"
+ "\xec\x60\xf7\x8e\x02\x99\x30\xc7"
+ "\x3b\xd2\x69\x00\x74\x0b\xa2\x16"
+ "\xad\x44\xdb\x4f\xe6\x7d\x14\x88"
+ "\x1f\xb6\x2a\xc1\x58\xef\x63\xfa"
+ "\x91\x05\x9c\x33\xca\x3e\xd5\x6c"
+ "\x03\x77\x0e\xa5\x19\xb0\x47\xde"
+ "\x52\xe9\x80\x17\x8b\x22\xb9\x2d"
+ "\xc4\x5b\xf2\x66\xfd\x94\x08\x9f"
+ "\x36\xcd\x41\xd8\x6f\x06\x7a\x11"
+ "\xa8\x1c\xb3\x4a\xe1\x55\xec\x83"
+ "\x1a\x8e\x25\xbc\x30\xc7\x5e\xf5"
+ "\x69\x00\x97\x0b\xa2\x39\xd0\x44"
+ "\xdb\x72\x09\x7d\x14\xab\x1f\xb6"
+ "\x4d\xe4\x58\xef\x86\x1d\x91\x28"
+ "\xbf\x33\xca\x61\xf8\x6c\x03\x9a"
+ "\x0e\xa5\x3c\xd3\x47\xde\x75\x0c"
+ "\x80\x17\xae\x22\xb9\x50\xe7\x5b"
+ "\xf2\x89\x20\x94\x2b\xc2\x36\xcd"
+ "\x64\xfb\x6f\x06\x9d\x11\xa8\x3f"
+ "\xd6\x4a\xe1\x78\x0f\x83\x1a\xb1"
+ "\x25\xbc\x53\xea\x5e\xf5\x8c\x00"
+ "\x97\x2e\xc5\x39\xd0\x67\xfe\x72"
+ "\x09\xa0\x14\xab\x42\xd9\x4d\xe4"
+ "\x7b\x12\x86\x1d\xb4\x28\xbf\x56"
+ "\xed\x61\xf8\x8f\x03\x9a\x31\xc8"
+ "\x3c\xd3\x6a\x01\x75\x0c\xa3\x17"
+ "\xae\x45\xdc\x50\xe7\x7e\x15\x89"
+ "\x20\xb7\x2b\xc2\x59\xf0\x64\xfb"
+ "\x92\x06\x9d\x34\xcb\x3f\xd6\x6d"
+ "\x04\x78\x0f\xa6\x1a\xb1\x48\xdf"
+ "\x53\xea\x81\x18\x8c\x23\xba\x2e"
+ "\xc5\x5c\xf3\x67\xfe\x95\x09\xa0"
+ "\x37\xce\x42\xd9\x70\x07\x7b\x12"
+ "\xa9\x1d\xb4\x4b\xe2\x56\xed\x84"
+ "\x1b\x8f\x26\xbd\x31\xc8\x5f\xf6"
+ "\x6a\x01\x98\x0c\xa3\x3a\xd1\x45"
+ "\xdc\x73\x0a\x7e\x15\xac\x20\xb7"
+ "\x4e\xe5\x59\xf0\x87\x1e\x92\x29"
+ "\xc0\x34\xcb\x62\xf9\x6d\x04\x9b"
+ "\x0f\xa6\x3d\xd4\x48\xdf\x76\x0d"
+ "\x81\x18\xaf\x23\xba\x51\xe8\x5c"
+ "\xf3\x8a\x21\x95\x2c\xc3\x37\xce"
+ "\x65\xfc\x70\x07\x9e\x12\xa9\x40"
+ "\xd7\x4b\xe2\x79\x10\x84\x1b\xb2"
+ "\x26\xbd\x54\xeb\x5f\xf6\x8d\x01"
+ "\x98\x2f\xc6\x3a\xd1\x68\xff\x73"
+ "\x0a\xa1\x15\xac\x43\xda\x4e\xe5"
+ "\x7c\x13\x87\x1e\xb5\x29\xc0\x57"
+ "\xee\x62\xf9\x90\x04\x9b\x32\xc9"
+ "\x3d\xd4\x6b\x02\x76\x0d\xa4\x18"
+ "\xaf\x46\xdd\x51\xe8\x7f\x16\x8a"
+ "\x21\xb8\x2c\xc3\x5a\xf1\x65\xfc"
+ "\x93\x07\x9e\x35\xcc\x40\xd7\x6e"
+ "\x05\x79\x10\xa7\x1b\xb2\x49\xe0"
+ "\x54\xeb\x82\x19\x8d\x24\xbb\x2f"
+ "\xc6\x5d\xf4\x68\xff\x96\x0a\xa1"
+ "\x38\xcf\x43\xda\x71\x08\x7c\x13"
+ "\xaa\x1e\xb5\x4c\xe3\x57\xee\x85"
+ "\x1c\x90\x27\xbe\x32\xc9\x60\xf7"
+ "\x6b\x02\x99\x0d\xa4\x3b\xd2\x46"
+ "\xdd\x74\x0b\x7f\x16\xad\x21\xb8"
+ "\x4f\xe6\x5a\xf1\x88\x1f\x93\x2a"
+ "\xc1\x35\xcc\x63\xfa\x6e\x05\x9c"
+ "\x10\xa7\x3e\xd5\x49\xe0\x77\x0e"
+ "\x82\x19\xb0\x24\xbb\x52\xe9\x5d"
+ "\xf4\x8b\x22\x96\x2d\xc4\x38\xcf"
+ "\x66\xfd\x71\x08\x9f\x13\xaa\x41"
+ "\xd8\x4c\xe3\x7a\x11\x85\x1c\xb3"
+ "\x27\xbe\x55\xec\x60\xf7\x8e\x02"
+ "\x99\x30\xc7\x3b\xd2\x69\x00\x74"
+ "\x0b\xa2\x16\xad\x44\xdb\x4f\xe6"
+ "\x7d\x14\x88\x1f\xb6\x2a\xc1\x58"
+ "\xef\x63\xfa\x91\x05\x9c\x33\xca"
+ "\x3e\xd5\x6c\x03\x77\x0e\xa5\x19"
+ "\xb0\x47\xde\x52\xe9\x80\x17\x8b"
+ "\x22\xb9\x2d\xc4\x5b\xf2\x66\xfd"
+ "\x94\x08\x9f\x36\xcd\x41\xd8\x6f"
+ "\x06\x7a\x11\xa8\x1c\xb3\x4a\xe1"
+ "\x55\xec\x83\x1a\x8e\x25\xbc\x30"
+ "\xc7\x5e\xf5\x69\x00\x97\x0b\xa2"
+ "\x39\xd0\x44\xdb\x72\x09\x7d\x14"
+ "\xab\x1f\xb6\x4d\xe4\x58\xef\x86"
+ "\x1d\x91\x28\xbf\x33\xca\x61\xf8"
+ "\x6c\x03\x9a\x0e\xa5\x3c\xd3\x47"
+ "\xde\x75\x0c\x80\x17\xae\x22\xb9"
+ "\x50\xe7\x5b\xf2\x89\x20\x94\x2b"
+ "\xc2\x36\xcd\x64\xfb\x6f\x06\x9d"
+ "\x11\xa8\x3f\xd6\x4a\xe1\x78\x0f"
+ "\x83\x1a\xb1\x25\xbc\x53\xea\x5e"
+ "\xf5\x8c\x00\x97\x2e\xc5\x39\xd0"
+ "\x67\xfe\x72\x09\xa0\x14\xab\x42"
+ "\xd9\x4d\xe4\x7b\x12\x86\x1d\xb4"
+ "\x28\xbf\x56\xed\x61\xf8\x8f\x03"
+ "\x9a\x31\xc8\x3c\xd3\x6a\x01\x75"
+ "\x0c\xa3\x17\xae\x45\xdc\x50\xe7"
+ "\x7e\x15\x89\x20\xb7\x2b\xc2\x59"
+ "\xf0\x64\xfb\x92\x06\x9d\x34\xcb"
+ "\x3f\xd6\x6d\x04\x78\x0f\xa6\x1a"
+ "\xb1\x48\xdf\x53\xea\x81\x18\x8c"
+ "\x23\xba\x2e\xc5\x5c\xf3\x67\xfe"
+ "\x95\x09\xa0\x37\xce\x42\xd9\x70"
+ "\x07\x7b\x12\xa9\x1d\xb4\x4b\xe2"
+ "\x56\xed\x84\x1b\x8f\x26\xbd\x31"
+ "\xc8\x5f\xf6\x6a\x01\x98\x0c\xa3"
+ "\x3a\xd1\x45\xdc\x73\x0a\x7e\x15"
+ "\xac\x20\xb7\x4e\xe5\x59\xf0\x87"
+ "\x1e\x92\x29\xc0\x34\xcb\x62\xf9"
+ "\x6d\x04\x9b\x0f\xa6\x3d\xd4\x48"
+ "\xdf\x76\x0d\x81\x18\xaf\x23\xba"
+ "\x51\xe8\x5c\xf3\x8a\x21\x95\x2c"
+ "\xc3\x37\xce\x65\xfc\x70\x07\x9e"
+ "\x12\xa9\x40\xd7\x4b\xe2\x79\x10"
+ "\x84\x1b\xb2\x26\xbd\x54\xeb\x5f"
+ "\xf6\x8d\x01\x98\x2f\xc6\x3a\xd1"
+ "\x68\xff\x73\x0a\xa1\x15\xac\x43"
+ "\xda\x4e\xe5\x7c\x13\x87\x1e\xb5"
+ "\x29\xc0\x57\xee\x62\xf9\x90\x04"
+ "\x9b\x32\xc9\x3d\xd4\x6b\x02\x76"
+ "\x0d\xa4\x18\xaf\x46\xdd\x51\xe8"
+ "\x7f\x16\x8a\x21\xb8\x2c\xc3\x5a"
+ "\xf1\x65\xfc\x93\x07\x9e\x35\xcc"
+ "\x40\xd7\x6e\x05\x79\x10\xa7\x1b"
+ "\xb2\x49\xe0\x54\xeb\x82\x19\x8d"
+ "\x24\xbb\x2f\xc6\x5d\xf4\x68\xff"
+ "\x96\x0a\xa1\x38\xcf\x43\xda\x71"
+ "\x08\x7c\x13\xaa\x1e\xb5\x4c",
+ .psize = 1023,
+ .digest = "\xb8\xe3\x54\xed\xc5\xfc\xef\xa4"
+ "\x55\x73\x4a\x81\x99\xe4\x47\x2a"
+ "\x30\xd6\xc9\x85",
}
};
@@ -536,10 +679,17 @@
/*
* SHA224 test vectors from from FIPS PUB 180-2
*/
-#define SHA224_TEST_VECTORS 2
+#define SHA224_TEST_VECTORS 5
static struct hash_testvec sha224_tv_template[] = {
{
+ .plaintext = "",
+ .psize = 0,
+ .digest = "\xd1\x4a\x02\x8c\x2a\x3a\x2b\xc9"
+ "\x47\x61\x02\xbb\x28\x82\x34\xc4"
+ "\x15\xa2\xb0\x1f\x82\x8e\xa6\x2a"
+ "\xc5\xb3\xe4\x2f",
+ }, {
.plaintext = "abc",
.psize = 3,
.digest = "\x23\x09\x7D\x22\x34\x05\xD8\x22"
@@ -556,16 +706,164 @@
"\x52\x52\x25\x25",
.np = 2,
.tap = { 28, 28 }
+ }, {
+ .plaintext = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-",
+ .psize = 64,
+ .digest = "\xc4\xdb\x2b\x3a\x58\xc3\x99\x01"
+ "\x42\xfd\x10\x92\xaa\x4e\x04\x08"
+ "\x58\xbb\xbb\xe8\xf8\x14\xa7\x0c"
+ "\xef\x3b\xcb\x0e",
+ }, {
+ .plaintext = "\x08\x9f\x13\xaa\x41\xd8\x4c\xe3"
+ "\x7a\x11\x85\x1c\xb3\x27\xbe\x55"
+ "\xec\x60\xf7\x8e\x02\x99\x30\xc7"
+ "\x3b\xd2\x69\x00\x74\x0b\xa2\x16"
+ "\xad\x44\xdb\x4f\xe6\x7d\x14\x88"
+ "\x1f\xb6\x2a\xc1\x58\xef\x63\xfa"
+ "\x91\x05\x9c\x33\xca\x3e\xd5\x6c"
+ "\x03\x77\x0e\xa5\x19\xb0\x47\xde"
+ "\x52\xe9\x80\x17\x8b\x22\xb9\x2d"
+ "\xc4\x5b\xf2\x66\xfd\x94\x08\x9f"
+ "\x36\xcd\x41\xd8\x6f\x06\x7a\x11"
+ "\xa8\x1c\xb3\x4a\xe1\x55\xec\x83"
+ "\x1a\x8e\x25\xbc\x30\xc7\x5e\xf5"
+ "\x69\x00\x97\x0b\xa2\x39\xd0\x44"
+ "\xdb\x72\x09\x7d\x14\xab\x1f\xb6"
+ "\x4d\xe4\x58\xef\x86\x1d\x91\x28"
+ "\xbf\x33\xca\x61\xf8\x6c\x03\x9a"
+ "\x0e\xa5\x3c\xd3\x47\xde\x75\x0c"
+ "\x80\x17\xae\x22\xb9\x50\xe7\x5b"
+ "\xf2\x89\x20\x94\x2b\xc2\x36\xcd"
+ "\x64\xfb\x6f\x06\x9d\x11\xa8\x3f"
+ "\xd6\x4a\xe1\x78\x0f\x83\x1a\xb1"
+ "\x25\xbc\x53\xea\x5e\xf5\x8c\x00"
+ "\x97\x2e\xc5\x39\xd0\x67\xfe\x72"
+ "\x09\xa0\x14\xab\x42\xd9\x4d\xe4"
+ "\x7b\x12\x86\x1d\xb4\x28\xbf\x56"
+ "\xed\x61\xf8\x8f\x03\x9a\x31\xc8"
+ "\x3c\xd3\x6a\x01\x75\x0c\xa3\x17"
+ "\xae\x45\xdc\x50\xe7\x7e\x15\x89"
+ "\x20\xb7\x2b\xc2\x59\xf0\x64\xfb"
+ "\x92\x06\x9d\x34\xcb\x3f\xd6\x6d"
+ "\x04\x78\x0f\xa6\x1a\xb1\x48\xdf"
+ "\x53\xea\x81\x18\x8c\x23\xba\x2e"
+ "\xc5\x5c\xf3\x67\xfe\x95\x09\xa0"
+ "\x37\xce\x42\xd9\x70\x07\x7b\x12"
+ "\xa9\x1d\xb4\x4b\xe2\x56\xed\x84"
+ "\x1b\x8f\x26\xbd\x31\xc8\x5f\xf6"
+ "\x6a\x01\x98\x0c\xa3\x3a\xd1\x45"
+ "\xdc\x73\x0a\x7e\x15\xac\x20\xb7"
+ "\x4e\xe5\x59\xf0\x87\x1e\x92\x29"
+ "\xc0\x34\xcb\x62\xf9\x6d\x04\x9b"
+ "\x0f\xa6\x3d\xd4\x48\xdf\x76\x0d"
+ "\x81\x18\xaf\x23\xba\x51\xe8\x5c"
+ "\xf3\x8a\x21\x95\x2c\xc3\x37\xce"
+ "\x65\xfc\x70\x07\x9e\x12\xa9\x40"
+ "\xd7\x4b\xe2\x79\x10\x84\x1b\xb2"
+ "\x26\xbd\x54\xeb\x5f\xf6\x8d\x01"
+ "\x98\x2f\xc6\x3a\xd1\x68\xff\x73"
+ "\x0a\xa1\x15\xac\x43\xda\x4e\xe5"
+ "\x7c\x13\x87\x1e\xb5\x29\xc0\x57"
+ "\xee\x62\xf9\x90\x04\x9b\x32\xc9"
+ "\x3d\xd4\x6b\x02\x76\x0d\xa4\x18"
+ "\xaf\x46\xdd\x51\xe8\x7f\x16\x8a"
+ "\x21\xb8\x2c\xc3\x5a\xf1\x65\xfc"
+ "\x93\x07\x9e\x35\xcc\x40\xd7\x6e"
+ "\x05\x79\x10\xa7\x1b\xb2\x49\xe0"
+ "\x54\xeb\x82\x19\x8d\x24\xbb\x2f"
+ "\xc6\x5d\xf4\x68\xff\x96\x0a\xa1"
+ "\x38\xcf\x43\xda\x71\x08\x7c\x13"
+ "\xaa\x1e\xb5\x4c\xe3\x57\xee\x85"
+ "\x1c\x90\x27\xbe\x32\xc9\x60\xf7"
+ "\x6b\x02\x99\x0d\xa4\x3b\xd2\x46"
+ "\xdd\x74\x0b\x7f\x16\xad\x21\xb8"
+ "\x4f\xe6\x5a\xf1\x88\x1f\x93\x2a"
+ "\xc1\x35\xcc\x63\xfa\x6e\x05\x9c"
+ "\x10\xa7\x3e\xd5\x49\xe0\x77\x0e"
+ "\x82\x19\xb0\x24\xbb\x52\xe9\x5d"
+ "\xf4\x8b\x22\x96\x2d\xc4\x38\xcf"
+ "\x66\xfd\x71\x08\x9f\x13\xaa\x41"
+ "\xd8\x4c\xe3\x7a\x11\x85\x1c\xb3"
+ "\x27\xbe\x55\xec\x60\xf7\x8e\x02"
+ "\x99\x30\xc7\x3b\xd2\x69\x00\x74"
+ "\x0b\xa2\x16\xad\x44\xdb\x4f\xe6"
+ "\x7d\x14\x88\x1f\xb6\x2a\xc1\x58"
+ "\xef\x63\xfa\x91\x05\x9c\x33\xca"
+ "\x3e\xd5\x6c\x03\x77\x0e\xa5\x19"
+ "\xb0\x47\xde\x52\xe9\x80\x17\x8b"
+ "\x22\xb9\x2d\xc4\x5b\xf2\x66\xfd"
+ "\x94\x08\x9f\x36\xcd\x41\xd8\x6f"
+ "\x06\x7a\x11\xa8\x1c\xb3\x4a\xe1"
+ "\x55\xec\x83\x1a\x8e\x25\xbc\x30"
+ "\xc7\x5e\xf5\x69\x00\x97\x0b\xa2"
+ "\x39\xd0\x44\xdb\x72\x09\x7d\x14"
+ "\xab\x1f\xb6\x4d\xe4\x58\xef\x86"
+ "\x1d\x91\x28\xbf\x33\xca\x61\xf8"
+ "\x6c\x03\x9a\x0e\xa5\x3c\xd3\x47"
+ "\xde\x75\x0c\x80\x17\xae\x22\xb9"
+ "\x50\xe7\x5b\xf2\x89\x20\x94\x2b"
+ "\xc2\x36\xcd\x64\xfb\x6f\x06\x9d"
+ "\x11\xa8\x3f\xd6\x4a\xe1\x78\x0f"
+ "\x83\x1a\xb1\x25\xbc\x53\xea\x5e"
+ "\xf5\x8c\x00\x97\x2e\xc5\x39\xd0"
+ "\x67\xfe\x72\x09\xa0\x14\xab\x42"
+ "\xd9\x4d\xe4\x7b\x12\x86\x1d\xb4"
+ "\x28\xbf\x56\xed\x61\xf8\x8f\x03"
+ "\x9a\x31\xc8\x3c\xd3\x6a\x01\x75"
+ "\x0c\xa3\x17\xae\x45\xdc\x50\xe7"
+ "\x7e\x15\x89\x20\xb7\x2b\xc2\x59"
+ "\xf0\x64\xfb\x92\x06\x9d\x34\xcb"
+ "\x3f\xd6\x6d\x04\x78\x0f\xa6\x1a"
+ "\xb1\x48\xdf\x53\xea\x81\x18\x8c"
+ "\x23\xba\x2e\xc5\x5c\xf3\x67\xfe"
+ "\x95\x09\xa0\x37\xce\x42\xd9\x70"
+ "\x07\x7b\x12\xa9\x1d\xb4\x4b\xe2"
+ "\x56\xed\x84\x1b\x8f\x26\xbd\x31"
+ "\xc8\x5f\xf6\x6a\x01\x98\x0c\xa3"
+ "\x3a\xd1\x45\xdc\x73\x0a\x7e\x15"
+ "\xac\x20\xb7\x4e\xe5\x59\xf0\x87"
+ "\x1e\x92\x29\xc0\x34\xcb\x62\xf9"
+ "\x6d\x04\x9b\x0f\xa6\x3d\xd4\x48"
+ "\xdf\x76\x0d\x81\x18\xaf\x23\xba"
+ "\x51\xe8\x5c\xf3\x8a\x21\x95\x2c"
+ "\xc3\x37\xce\x65\xfc\x70\x07\x9e"
+ "\x12\xa9\x40\xd7\x4b\xe2\x79\x10"
+ "\x84\x1b\xb2\x26\xbd\x54\xeb\x5f"
+ "\xf6\x8d\x01\x98\x2f\xc6\x3a\xd1"
+ "\x68\xff\x73\x0a\xa1\x15\xac\x43"
+ "\xda\x4e\xe5\x7c\x13\x87\x1e\xb5"
+ "\x29\xc0\x57\xee\x62\xf9\x90\x04"
+ "\x9b\x32\xc9\x3d\xd4\x6b\x02\x76"
+ "\x0d\xa4\x18\xaf\x46\xdd\x51\xe8"
+ "\x7f\x16\x8a\x21\xb8\x2c\xc3\x5a"
+ "\xf1\x65\xfc\x93\x07\x9e\x35\xcc"
+ "\x40\xd7\x6e\x05\x79\x10\xa7\x1b"
+ "\xb2\x49\xe0\x54\xeb\x82\x19\x8d"
+ "\x24\xbb\x2f\xc6\x5d\xf4\x68\xff"
+ "\x96\x0a\xa1\x38\xcf\x43\xda\x71"
+ "\x08\x7c\x13\xaa\x1e\xb5\x4c",
+ .psize = 1023,
+ .digest = "\x98\x43\x07\x63\x75\xe0\xa7\x1c"
+ "\x78\xb1\x8b\xfd\x04\xf5\x2d\x91"
+ "\x20\x48\xa4\x28\xff\x55\xb1\xd3"
+ "\xe6\xf9\x4f\xcc",
}
};
/*
* SHA256 test vectors from from NIST
*/
-#define SHA256_TEST_VECTORS 2
+#define SHA256_TEST_VECTORS 5
static struct hash_testvec sha256_tv_template[] = {
{
+ .plaintext = "",
+ .psize = 0,
+ .digest = "\xe3\xb0\xc4\x42\x98\xfc\x1c\x14"
+ "\x9a\xfb\xf4\xc8\x99\x6f\xb9\x24"
+ "\x27\xae\x41\xe4\x64\x9b\x93\x4c"
+ "\xa4\x95\x99\x1b\x78\x52\xb8\x55",
+ }, {
.plaintext = "abc",
.psize = 3,
.digest = "\xba\x78\x16\xbf\x8f\x01\xcf\xea"
@@ -581,16 +879,166 @@
"\xf6\xec\xed\xd4\x19\xdb\x06\xc1",
.np = 2,
.tap = { 28, 28 }
- },
+ }, {
+ .plaintext = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-",
+ .psize = 64,
+ .digest = "\xb5\xfe\xad\x56\x7d\xff\xcb\xa4"
+ "\x2c\x32\x29\x32\x19\xbb\xfb\xfa"
+ "\xd6\xff\x94\xa3\x72\x91\x85\x66"
+ "\x3b\xa7\x87\x77\x58\xa3\x40\x3a",
+ }, {
+ .plaintext = "\x08\x9f\x13\xaa\x41\xd8\x4c\xe3"
+ "\x7a\x11\x85\x1c\xb3\x27\xbe\x55"
+ "\xec\x60\xf7\x8e\x02\x99\x30\xc7"
+ "\x3b\xd2\x69\x00\x74\x0b\xa2\x16"
+ "\xad\x44\xdb\x4f\xe6\x7d\x14\x88"
+ "\x1f\xb6\x2a\xc1\x58\xef\x63\xfa"
+ "\x91\x05\x9c\x33\xca\x3e\xd5\x6c"
+ "\x03\x77\x0e\xa5\x19\xb0\x47\xde"
+ "\x52\xe9\x80\x17\x8b\x22\xb9\x2d"
+ "\xc4\x5b\xf2\x66\xfd\x94\x08\x9f"
+ "\x36\xcd\x41\xd8\x6f\x06\x7a\x11"
+ "\xa8\x1c\xb3\x4a\xe1\x55\xec\x83"
+ "\x1a\x8e\x25\xbc\x30\xc7\x5e\xf5"
+ "\x69\x00\x97\x0b\xa2\x39\xd0\x44"
+ "\xdb\x72\x09\x7d\x14\xab\x1f\xb6"
+ "\x4d\xe4\x58\xef\x86\x1d\x91\x28"
+ "\xbf\x33\xca\x61\xf8\x6c\x03\x9a"
+ "\x0e\xa5\x3c\xd3\x47\xde\x75\x0c"
+ "\x80\x17\xae\x22\xb9\x50\xe7\x5b"
+ "\xf2\x89\x20\x94\x2b\xc2\x36\xcd"
+ "\x64\xfb\x6f\x06\x9d\x11\xa8\x3f"
+ "\xd6\x4a\xe1\x78\x0f\x83\x1a\xb1"
+ "\x25\xbc\x53\xea\x5e\xf5\x8c\x00"
+ "\x97\x2e\xc5\x39\xd0\x67\xfe\x72"
+ "\x09\xa0\x14\xab\x42\xd9\x4d\xe4"
+ "\x7b\x12\x86\x1d\xb4\x28\xbf\x56"
+ "\xed\x61\xf8\x8f\x03\x9a\x31\xc8"
+ "\x3c\xd3\x6a\x01\x75\x0c\xa3\x17"
+ "\xae\x45\xdc\x50\xe7\x7e\x15\x89"
+ "\x20\xb7\x2b\xc2\x59\xf0\x64\xfb"
+ "\x92\x06\x9d\x34\xcb\x3f\xd6\x6d"
+ "\x04\x78\x0f\xa6\x1a\xb1\x48\xdf"
+ "\x53\xea\x81\x18\x8c\x23\xba\x2e"
+ "\xc5\x5c\xf3\x67\xfe\x95\x09\xa0"
+ "\x37\xce\x42\xd9\x70\x07\x7b\x12"
+ "\xa9\x1d\xb4\x4b\xe2\x56\xed\x84"
+ "\x1b\x8f\x26\xbd\x31\xc8\x5f\xf6"
+ "\x6a\x01\x98\x0c\xa3\x3a\xd1\x45"
+ "\xdc\x73\x0a\x7e\x15\xac\x20\xb7"
+ "\x4e\xe5\x59\xf0\x87\x1e\x92\x29"
+ "\xc0\x34\xcb\x62\xf9\x6d\x04\x9b"
+ "\x0f\xa6\x3d\xd4\x48\xdf\x76\x0d"
+ "\x81\x18\xaf\x23\xba\x51\xe8\x5c"
+ "\xf3\x8a\x21\x95\x2c\xc3\x37\xce"
+ "\x65\xfc\x70\x07\x9e\x12\xa9\x40"
+ "\xd7\x4b\xe2\x79\x10\x84\x1b\xb2"
+ "\x26\xbd\x54\xeb\x5f\xf6\x8d\x01"
+ "\x98\x2f\xc6\x3a\xd1\x68\xff\x73"
+ "\x0a\xa1\x15\xac\x43\xda\x4e\xe5"
+ "\x7c\x13\x87\x1e\xb5\x29\xc0\x57"
+ "\xee\x62\xf9\x90\x04\x9b\x32\xc9"
+ "\x3d\xd4\x6b\x02\x76\x0d\xa4\x18"
+ "\xaf\x46\xdd\x51\xe8\x7f\x16\x8a"
+ "\x21\xb8\x2c\xc3\x5a\xf1\x65\xfc"
+ "\x93\x07\x9e\x35\xcc\x40\xd7\x6e"
+ "\x05\x79\x10\xa7\x1b\xb2\x49\xe0"
+ "\x54\xeb\x82\x19\x8d\x24\xbb\x2f"
+ "\xc6\x5d\xf4\x68\xff\x96\x0a\xa1"
+ "\x38\xcf\x43\xda\x71\x08\x7c\x13"
+ "\xaa\x1e\xb5\x4c\xe3\x57\xee\x85"
+ "\x1c\x90\x27\xbe\x32\xc9\x60\xf7"
+ "\x6b\x02\x99\x0d\xa4\x3b\xd2\x46"
+ "\xdd\x74\x0b\x7f\x16\xad\x21\xb8"
+ "\x4f\xe6\x5a\xf1\x88\x1f\x93\x2a"
+ "\xc1\x35\xcc\x63\xfa\x6e\x05\x9c"
+ "\x10\xa7\x3e\xd5\x49\xe0\x77\x0e"
+ "\x82\x19\xb0\x24\xbb\x52\xe9\x5d"
+ "\xf4\x8b\x22\x96\x2d\xc4\x38\xcf"
+ "\x66\xfd\x71\x08\x9f\x13\xaa\x41"
+ "\xd8\x4c\xe3\x7a\x11\x85\x1c\xb3"
+ "\x27\xbe\x55\xec\x60\xf7\x8e\x02"
+ "\x99\x30\xc7\x3b\xd2\x69\x00\x74"
+ "\x0b\xa2\x16\xad\x44\xdb\x4f\xe6"
+ "\x7d\x14\x88\x1f\xb6\x2a\xc1\x58"
+ "\xef\x63\xfa\x91\x05\x9c\x33\xca"
+ "\x3e\xd5\x6c\x03\x77\x0e\xa5\x19"
+ "\xb0\x47\xde\x52\xe9\x80\x17\x8b"
+ "\x22\xb9\x2d\xc4\x5b\xf2\x66\xfd"
+ "\x94\x08\x9f\x36\xcd\x41\xd8\x6f"
+ "\x06\x7a\x11\xa8\x1c\xb3\x4a\xe1"
+ "\x55\xec\x83\x1a\x8e\x25\xbc\x30"
+ "\xc7\x5e\xf5\x69\x00\x97\x0b\xa2"
+ "\x39\xd0\x44\xdb\x72\x09\x7d\x14"
+ "\xab\x1f\xb6\x4d\xe4\x58\xef\x86"
+ "\x1d\x91\x28\xbf\x33\xca\x61\xf8"
+ "\x6c\x03\x9a\x0e\xa5\x3c\xd3\x47"
+ "\xde\x75\x0c\x80\x17\xae\x22\xb9"
+ "\x50\xe7\x5b\xf2\x89\x20\x94\x2b"
+ "\xc2\x36\xcd\x64\xfb\x6f\x06\x9d"
+ "\x11\xa8\x3f\xd6\x4a\xe1\x78\x0f"
+ "\x83\x1a\xb1\x25\xbc\x53\xea\x5e"
+ "\xf5\x8c\x00\x97\x2e\xc5\x39\xd0"
+ "\x67\xfe\x72\x09\xa0\x14\xab\x42"
+ "\xd9\x4d\xe4\x7b\x12\x86\x1d\xb4"
+ "\x28\xbf\x56\xed\x61\xf8\x8f\x03"
+ "\x9a\x31\xc8\x3c\xd3\x6a\x01\x75"
+ "\x0c\xa3\x17\xae\x45\xdc\x50\xe7"
+ "\x7e\x15\x89\x20\xb7\x2b\xc2\x59"
+ "\xf0\x64\xfb\x92\x06\x9d\x34\xcb"
+ "\x3f\xd6\x6d\x04\x78\x0f\xa6\x1a"
+ "\xb1\x48\xdf\x53\xea\x81\x18\x8c"
+ "\x23\xba\x2e\xc5\x5c\xf3\x67\xfe"
+ "\x95\x09\xa0\x37\xce\x42\xd9\x70"
+ "\x07\x7b\x12\xa9\x1d\xb4\x4b\xe2"
+ "\x56\xed\x84\x1b\x8f\x26\xbd\x31"
+ "\xc8\x5f\xf6\x6a\x01\x98\x0c\xa3"
+ "\x3a\xd1\x45\xdc\x73\x0a\x7e\x15"
+ "\xac\x20\xb7\x4e\xe5\x59\xf0\x87"
+ "\x1e\x92\x29\xc0\x34\xcb\x62\xf9"
+ "\x6d\x04\x9b\x0f\xa6\x3d\xd4\x48"
+ "\xdf\x76\x0d\x81\x18\xaf\x23\xba"
+ "\x51\xe8\x5c\xf3\x8a\x21\x95\x2c"
+ "\xc3\x37\xce\x65\xfc\x70\x07\x9e"
+ "\x12\xa9\x40\xd7\x4b\xe2\x79\x10"
+ "\x84\x1b\xb2\x26\xbd\x54\xeb\x5f"
+ "\xf6\x8d\x01\x98\x2f\xc6\x3a\xd1"
+ "\x68\xff\x73\x0a\xa1\x15\xac\x43"
+ "\xda\x4e\xe5\x7c\x13\x87\x1e\xb5"
+ "\x29\xc0\x57\xee\x62\xf9\x90\x04"
+ "\x9b\x32\xc9\x3d\xd4\x6b\x02\x76"
+ "\x0d\xa4\x18\xaf\x46\xdd\x51\xe8"
+ "\x7f\x16\x8a\x21\xb8\x2c\xc3\x5a"
+ "\xf1\x65\xfc\x93\x07\x9e\x35\xcc"
+ "\x40\xd7\x6e\x05\x79\x10\xa7\x1b"
+ "\xb2\x49\xe0\x54\xeb\x82\x19\x8d"
+ "\x24\xbb\x2f\xc6\x5d\xf4\x68\xff"
+ "\x96\x0a\xa1\x38\xcf\x43\xda\x71"
+ "\x08\x7c\x13\xaa\x1e\xb5\x4c",
+ .psize = 1023,
+ .digest = "\xc5\xce\x0c\xca\x01\x4f\x53\x3a"
+ "\x32\x32\x17\xcc\xd4\x6a\x71\xa9"
+ "\xf3\xed\x50\x10\x64\x8e\x06\xbe"
+ "\x9b\x4a\xa6\xbb\x05\x89\x59\x51",
+ }
};
/*
* SHA384 test vectors from from NIST and kerneli
*/
-#define SHA384_TEST_VECTORS 4
+#define SHA384_TEST_VECTORS 6
static struct hash_testvec sha384_tv_template[] = {
{
+ .plaintext = "",
+ .psize = 0,
+ .digest = "\x38\xb0\x60\xa7\x51\xac\x96\x38"
+ "\x4c\xd9\x32\x7e\xb1\xb1\xe3\x6a"
+ "\x21\xfd\xb7\x11\x14\xbe\x07\x43"
+ "\x4c\x0c\xc7\xbf\x63\xf6\xe1\xda"
+ "\x27\x4e\xde\xbf\xe7\x6f\x65\xfb"
+ "\xd5\x1a\xd2\xf1\x48\x98\xb9\x5b",
+ }, {
.plaintext= "abc",
.psize = 3,
.digest = "\xcb\x00\x75\x3f\x45\xa3\x5e\x8b"
@@ -630,16 +1078,163 @@
"\xc9\x38\xe2\xd1\x99\xe8\xbe\xa4",
.np = 4,
.tap = { 26, 26, 26, 26 }
- },
+ }, {
+ .plaintext = "\x08\x9f\x13\xaa\x41\xd8\x4c\xe3"
+ "\x7a\x11\x85\x1c\xb3\x27\xbe\x55"
+ "\xec\x60\xf7\x8e\x02\x99\x30\xc7"
+ "\x3b\xd2\x69\x00\x74\x0b\xa2\x16"
+ "\xad\x44\xdb\x4f\xe6\x7d\x14\x88"
+ "\x1f\xb6\x2a\xc1\x58\xef\x63\xfa"
+ "\x91\x05\x9c\x33\xca\x3e\xd5\x6c"
+ "\x03\x77\x0e\xa5\x19\xb0\x47\xde"
+ "\x52\xe9\x80\x17\x8b\x22\xb9\x2d"
+ "\xc4\x5b\xf2\x66\xfd\x94\x08\x9f"
+ "\x36\xcd\x41\xd8\x6f\x06\x7a\x11"
+ "\xa8\x1c\xb3\x4a\xe1\x55\xec\x83"
+ "\x1a\x8e\x25\xbc\x30\xc7\x5e\xf5"
+ "\x69\x00\x97\x0b\xa2\x39\xd0\x44"
+ "\xdb\x72\x09\x7d\x14\xab\x1f\xb6"
+ "\x4d\xe4\x58\xef\x86\x1d\x91\x28"
+ "\xbf\x33\xca\x61\xf8\x6c\x03\x9a"
+ "\x0e\xa5\x3c\xd3\x47\xde\x75\x0c"
+ "\x80\x17\xae\x22\xb9\x50\xe7\x5b"
+ "\xf2\x89\x20\x94\x2b\xc2\x36\xcd"
+ "\x64\xfb\x6f\x06\x9d\x11\xa8\x3f"
+ "\xd6\x4a\xe1\x78\x0f\x83\x1a\xb1"
+ "\x25\xbc\x53\xea\x5e\xf5\x8c\x00"
+ "\x97\x2e\xc5\x39\xd0\x67\xfe\x72"
+ "\x09\xa0\x14\xab\x42\xd9\x4d\xe4"
+ "\x7b\x12\x86\x1d\xb4\x28\xbf\x56"
+ "\xed\x61\xf8\x8f\x03\x9a\x31\xc8"
+ "\x3c\xd3\x6a\x01\x75\x0c\xa3\x17"
+ "\xae\x45\xdc\x50\xe7\x7e\x15\x89"
+ "\x20\xb7\x2b\xc2\x59\xf0\x64\xfb"
+ "\x92\x06\x9d\x34\xcb\x3f\xd6\x6d"
+ "\x04\x78\x0f\xa6\x1a\xb1\x48\xdf"
+ "\x53\xea\x81\x18\x8c\x23\xba\x2e"
+ "\xc5\x5c\xf3\x67\xfe\x95\x09\xa0"
+ "\x37\xce\x42\xd9\x70\x07\x7b\x12"
+ "\xa9\x1d\xb4\x4b\xe2\x56\xed\x84"
+ "\x1b\x8f\x26\xbd\x31\xc8\x5f\xf6"
+ "\x6a\x01\x98\x0c\xa3\x3a\xd1\x45"
+ "\xdc\x73\x0a\x7e\x15\xac\x20\xb7"
+ "\x4e\xe5\x59\xf0\x87\x1e\x92\x29"
+ "\xc0\x34\xcb\x62\xf9\x6d\x04\x9b"
+ "\x0f\xa6\x3d\xd4\x48\xdf\x76\x0d"
+ "\x81\x18\xaf\x23\xba\x51\xe8\x5c"
+ "\xf3\x8a\x21\x95\x2c\xc3\x37\xce"
+ "\x65\xfc\x70\x07\x9e\x12\xa9\x40"
+ "\xd7\x4b\xe2\x79\x10\x84\x1b\xb2"
+ "\x26\xbd\x54\xeb\x5f\xf6\x8d\x01"
+ "\x98\x2f\xc6\x3a\xd1\x68\xff\x73"
+ "\x0a\xa1\x15\xac\x43\xda\x4e\xe5"
+ "\x7c\x13\x87\x1e\xb5\x29\xc0\x57"
+ "\xee\x62\xf9\x90\x04\x9b\x32\xc9"
+ "\x3d\xd4\x6b\x02\x76\x0d\xa4\x18"
+ "\xaf\x46\xdd\x51\xe8\x7f\x16\x8a"
+ "\x21\xb8\x2c\xc3\x5a\xf1\x65\xfc"
+ "\x93\x07\x9e\x35\xcc\x40\xd7\x6e"
+ "\x05\x79\x10\xa7\x1b\xb2\x49\xe0"
+ "\x54\xeb\x82\x19\x8d\x24\xbb\x2f"
+ "\xc6\x5d\xf4\x68\xff\x96\x0a\xa1"
+ "\x38\xcf\x43\xda\x71\x08\x7c\x13"
+ "\xaa\x1e\xb5\x4c\xe3\x57\xee\x85"
+ "\x1c\x90\x27\xbe\x32\xc9\x60\xf7"
+ "\x6b\x02\x99\x0d\xa4\x3b\xd2\x46"
+ "\xdd\x74\x0b\x7f\x16\xad\x21\xb8"
+ "\x4f\xe6\x5a\xf1\x88\x1f\x93\x2a"
+ "\xc1\x35\xcc\x63\xfa\x6e\x05\x9c"
+ "\x10\xa7\x3e\xd5\x49\xe0\x77\x0e"
+ "\x82\x19\xb0\x24\xbb\x52\xe9\x5d"
+ "\xf4\x8b\x22\x96\x2d\xc4\x38\xcf"
+ "\x66\xfd\x71\x08\x9f\x13\xaa\x41"
+ "\xd8\x4c\xe3\x7a\x11\x85\x1c\xb3"
+ "\x27\xbe\x55\xec\x60\xf7\x8e\x02"
+ "\x99\x30\xc7\x3b\xd2\x69\x00\x74"
+ "\x0b\xa2\x16\xad\x44\xdb\x4f\xe6"
+ "\x7d\x14\x88\x1f\xb6\x2a\xc1\x58"
+ "\xef\x63\xfa\x91\x05\x9c\x33\xca"
+ "\x3e\xd5\x6c\x03\x77\x0e\xa5\x19"
+ "\xb0\x47\xde\x52\xe9\x80\x17\x8b"
+ "\x22\xb9\x2d\xc4\x5b\xf2\x66\xfd"
+ "\x94\x08\x9f\x36\xcd\x41\xd8\x6f"
+ "\x06\x7a\x11\xa8\x1c\xb3\x4a\xe1"
+ "\x55\xec\x83\x1a\x8e\x25\xbc\x30"
+ "\xc7\x5e\xf5\x69\x00\x97\x0b\xa2"
+ "\x39\xd0\x44\xdb\x72\x09\x7d\x14"
+ "\xab\x1f\xb6\x4d\xe4\x58\xef\x86"
+ "\x1d\x91\x28\xbf\x33\xca\x61\xf8"
+ "\x6c\x03\x9a\x0e\xa5\x3c\xd3\x47"
+ "\xde\x75\x0c\x80\x17\xae\x22\xb9"
+ "\x50\xe7\x5b\xf2\x89\x20\x94\x2b"
+ "\xc2\x36\xcd\x64\xfb\x6f\x06\x9d"
+ "\x11\xa8\x3f\xd6\x4a\xe1\x78\x0f"
+ "\x83\x1a\xb1\x25\xbc\x53\xea\x5e"
+ "\xf5\x8c\x00\x97\x2e\xc5\x39\xd0"
+ "\x67\xfe\x72\x09\xa0\x14\xab\x42"
+ "\xd9\x4d\xe4\x7b\x12\x86\x1d\xb4"
+ "\x28\xbf\x56\xed\x61\xf8\x8f\x03"
+ "\x9a\x31\xc8\x3c\xd3\x6a\x01\x75"
+ "\x0c\xa3\x17\xae\x45\xdc\x50\xe7"
+ "\x7e\x15\x89\x20\xb7\x2b\xc2\x59"
+ "\xf0\x64\xfb\x92\x06\x9d\x34\xcb"
+ "\x3f\xd6\x6d\x04\x78\x0f\xa6\x1a"
+ "\xb1\x48\xdf\x53\xea\x81\x18\x8c"
+ "\x23\xba\x2e\xc5\x5c\xf3\x67\xfe"
+ "\x95\x09\xa0\x37\xce\x42\xd9\x70"
+ "\x07\x7b\x12\xa9\x1d\xb4\x4b\xe2"
+ "\x56\xed\x84\x1b\x8f\x26\xbd\x31"
+ "\xc8\x5f\xf6\x6a\x01\x98\x0c\xa3"
+ "\x3a\xd1\x45\xdc\x73\x0a\x7e\x15"
+ "\xac\x20\xb7\x4e\xe5\x59\xf0\x87"
+ "\x1e\x92\x29\xc0\x34\xcb\x62\xf9"
+ "\x6d\x04\x9b\x0f\xa6\x3d\xd4\x48"
+ "\xdf\x76\x0d\x81\x18\xaf\x23\xba"
+ "\x51\xe8\x5c\xf3\x8a\x21\x95\x2c"
+ "\xc3\x37\xce\x65\xfc\x70\x07\x9e"
+ "\x12\xa9\x40\xd7\x4b\xe2\x79\x10"
+ "\x84\x1b\xb2\x26\xbd\x54\xeb\x5f"
+ "\xf6\x8d\x01\x98\x2f\xc6\x3a\xd1"
+ "\x68\xff\x73\x0a\xa1\x15\xac\x43"
+ "\xda\x4e\xe5\x7c\x13\x87\x1e\xb5"
+ "\x29\xc0\x57\xee\x62\xf9\x90\x04"
+ "\x9b\x32\xc9\x3d\xd4\x6b\x02\x76"
+ "\x0d\xa4\x18\xaf\x46\xdd\x51\xe8"
+ "\x7f\x16\x8a\x21\xb8\x2c\xc3\x5a"
+ "\xf1\x65\xfc\x93\x07\x9e\x35\xcc"
+ "\x40\xd7\x6e\x05\x79\x10\xa7\x1b"
+ "\xb2\x49\xe0\x54\xeb\x82\x19\x8d"
+ "\x24\xbb\x2f\xc6\x5d\xf4\x68\xff"
+ "\x96\x0a\xa1\x38\xcf\x43\xda\x71"
+ "\x08\x7c\x13\xaa\x1e\xb5\x4c",
+ .psize = 1023,
+ .digest = "\x4d\x97\x23\xc8\xea\x7a\x7c\x15"
+ "\xb8\xff\x97\x9c\xf5\x13\x4f\x31"
+ "\xde\x67\xf7\x24\x73\xcd\x70\x1c"
+ "\x03\x4a\xba\x8a\x87\x49\xfe\xdc"
+ "\x75\x29\x62\x83\xae\x3f\x17\xab"
+ "\xfd\x10\x4d\x8e\x17\x1c\x1f\xca",
+ }
};
/*
* SHA512 test vectors from from NIST and kerneli
*/
-#define SHA512_TEST_VECTORS 4
+#define SHA512_TEST_VECTORS 6
static struct hash_testvec sha512_tv_template[] = {
{
+ .plaintext = "",
+ .psize = 0,
+ .digest = "\xcf\x83\xe1\x35\x7e\xef\xb8\xbd"
+ "\xf1\x54\x28\x50\xd6\x6d\x80\x07"
+ "\xd6\x20\xe4\x05\x0b\x57\x15\xdc"
+ "\x83\xf4\xa9\x21\xd3\x6c\xe9\xce"
+ "\x47\xd0\xd1\x3c\x5d\x85\xf2\xb0"
+ "\xff\x83\x18\xd2\x87\x7e\xec\x2f"
+ "\x63\xb9\x31\xbd\x47\x41\x7a\x81"
+ "\xa5\x38\x32\x7a\xf9\x27\xda\x3e",
+ }, {
.plaintext = "abc",
.psize = 3,
.digest = "\xdd\xaf\x35\xa1\x93\x61\x7a\xba"
@@ -687,7 +1282,145 @@
"\xed\xb4\x19\x87\x23\x28\x50\xc9",
.np = 4,
.tap = { 26, 26, 26, 26 }
- },
+ }, {
+ .plaintext = "\x08\x9f\x13\xaa\x41\xd8\x4c\xe3"
+ "\x7a\x11\x85\x1c\xb3\x27\xbe\x55"
+ "\xec\x60\xf7\x8e\x02\x99\x30\xc7"
+ "\x3b\xd2\x69\x00\x74\x0b\xa2\x16"
+ "\xad\x44\xdb\x4f\xe6\x7d\x14\x88"
+ "\x1f\xb6\x2a\xc1\x58\xef\x63\xfa"
+ "\x91\x05\x9c\x33\xca\x3e\xd5\x6c"
+ "\x03\x77\x0e\xa5\x19\xb0\x47\xde"
+ "\x52\xe9\x80\x17\x8b\x22\xb9\x2d"
+ "\xc4\x5b\xf2\x66\xfd\x94\x08\x9f"
+ "\x36\xcd\x41\xd8\x6f\x06\x7a\x11"
+ "\xa8\x1c\xb3\x4a\xe1\x55\xec\x83"
+ "\x1a\x8e\x25\xbc\x30\xc7\x5e\xf5"
+ "\x69\x00\x97\x0b\xa2\x39\xd0\x44"
+ "\xdb\x72\x09\x7d\x14\xab\x1f\xb6"
+ "\x4d\xe4\x58\xef\x86\x1d\x91\x28"
+ "\xbf\x33\xca\x61\xf8\x6c\x03\x9a"
+ "\x0e\xa5\x3c\xd3\x47\xde\x75\x0c"
+ "\x80\x17\xae\x22\xb9\x50\xe7\x5b"
+ "\xf2\x89\x20\x94\x2b\xc2\x36\xcd"
+ "\x64\xfb\x6f\x06\x9d\x11\xa8\x3f"
+ "\xd6\x4a\xe1\x78\x0f\x83\x1a\xb1"
+ "\x25\xbc\x53\xea\x5e\xf5\x8c\x00"
+ "\x97\x2e\xc5\x39\xd0\x67\xfe\x72"
+ "\x09\xa0\x14\xab\x42\xd9\x4d\xe4"
+ "\x7b\x12\x86\x1d\xb4\x28\xbf\x56"
+ "\xed\x61\xf8\x8f\x03\x9a\x31\xc8"
+ "\x3c\xd3\x6a\x01\x75\x0c\xa3\x17"
+ "\xae\x45\xdc\x50\xe7\x7e\x15\x89"
+ "\x20\xb7\x2b\xc2\x59\xf0\x64\xfb"
+ "\x92\x06\x9d\x34\xcb\x3f\xd6\x6d"
+ "\x04\x78\x0f\xa6\x1a\xb1\x48\xdf"
+ "\x53\xea\x81\x18\x8c\x23\xba\x2e"
+ "\xc5\x5c\xf3\x67\xfe\x95\x09\xa0"
+ "\x37\xce\x42\xd9\x70\x07\x7b\x12"
+ "\xa9\x1d\xb4\x4b\xe2\x56\xed\x84"
+ "\x1b\x8f\x26\xbd\x31\xc8\x5f\xf6"
+ "\x6a\x01\x98\x0c\xa3\x3a\xd1\x45"
+ "\xdc\x73\x0a\x7e\x15\xac\x20\xb7"
+ "\x4e\xe5\x59\xf0\x87\x1e\x92\x29"
+ "\xc0\x34\xcb\x62\xf9\x6d\x04\x9b"
+ "\x0f\xa6\x3d\xd4\x48\xdf\x76\x0d"
+ "\x81\x18\xaf\x23\xba\x51\xe8\x5c"
+ "\xf3\x8a\x21\x95\x2c\xc3\x37\xce"
+ "\x65\xfc\x70\x07\x9e\x12\xa9\x40"
+ "\xd7\x4b\xe2\x79\x10\x84\x1b\xb2"
+ "\x26\xbd\x54\xeb\x5f\xf6\x8d\x01"
+ "\x98\x2f\xc6\x3a\xd1\x68\xff\x73"
+ "\x0a\xa1\x15\xac\x43\xda\x4e\xe5"
+ "\x7c\x13\x87\x1e\xb5\x29\xc0\x57"
+ "\xee\x62\xf9\x90\x04\x9b\x32\xc9"
+ "\x3d\xd4\x6b\x02\x76\x0d\xa4\x18"
+ "\xaf\x46\xdd\x51\xe8\x7f\x16\x8a"
+ "\x21\xb8\x2c\xc3\x5a\xf1\x65\xfc"
+ "\x93\x07\x9e\x35\xcc\x40\xd7\x6e"
+ "\x05\x79\x10\xa7\x1b\xb2\x49\xe0"
+ "\x54\xeb\x82\x19\x8d\x24\xbb\x2f"
+ "\xc6\x5d\xf4\x68\xff\x96\x0a\xa1"
+ "\x38\xcf\x43\xda\x71\x08\x7c\x13"
+ "\xaa\x1e\xb5\x4c\xe3\x57\xee\x85"
+ "\x1c\x90\x27\xbe\x32\xc9\x60\xf7"
+ "\x6b\x02\x99\x0d\xa4\x3b\xd2\x46"
+ "\xdd\x74\x0b\x7f\x16\xad\x21\xb8"
+ "\x4f\xe6\x5a\xf1\x88\x1f\x93\x2a"
+ "\xc1\x35\xcc\x63\xfa\x6e\x05\x9c"
+ "\x10\xa7\x3e\xd5\x49\xe0\x77\x0e"
+ "\x82\x19\xb0\x24\xbb\x52\xe9\x5d"
+ "\xf4\x8b\x22\x96\x2d\xc4\x38\xcf"
+ "\x66\xfd\x71\x08\x9f\x13\xaa\x41"
+ "\xd8\x4c\xe3\x7a\x11\x85\x1c\xb3"
+ "\x27\xbe\x55\xec\x60\xf7\x8e\x02"
+ "\x99\x30\xc7\x3b\xd2\x69\x00\x74"
+ "\x0b\xa2\x16\xad\x44\xdb\x4f\xe6"
+ "\x7d\x14\x88\x1f\xb6\x2a\xc1\x58"
+ "\xef\x63\xfa\x91\x05\x9c\x33\xca"
+ "\x3e\xd5\x6c\x03\x77\x0e\xa5\x19"
+ "\xb0\x47\xde\x52\xe9\x80\x17\x8b"
+ "\x22\xb9\x2d\xc4\x5b\xf2\x66\xfd"
+ "\x94\x08\x9f\x36\xcd\x41\xd8\x6f"
+ "\x06\x7a\x11\xa8\x1c\xb3\x4a\xe1"
+ "\x55\xec\x83\x1a\x8e\x25\xbc\x30"
+ "\xc7\x5e\xf5\x69\x00\x97\x0b\xa2"
+ "\x39\xd0\x44\xdb\x72\x09\x7d\x14"
+ "\xab\x1f\xb6\x4d\xe4\x58\xef\x86"
+ "\x1d\x91\x28\xbf\x33\xca\x61\xf8"
+ "\x6c\x03\x9a\x0e\xa5\x3c\xd3\x47"
+ "\xde\x75\x0c\x80\x17\xae\x22\xb9"
+ "\x50\xe7\x5b\xf2\x89\x20\x94\x2b"
+ "\xc2\x36\xcd\x64\xfb\x6f\x06\x9d"
+ "\x11\xa8\x3f\xd6\x4a\xe1\x78\x0f"
+ "\x83\x1a\xb1\x25\xbc\x53\xea\x5e"
+ "\xf5\x8c\x00\x97\x2e\xc5\x39\xd0"
+ "\x67\xfe\x72\x09\xa0\x14\xab\x42"
+ "\xd9\x4d\xe4\x7b\x12\x86\x1d\xb4"
+ "\x28\xbf\x56\xed\x61\xf8\x8f\x03"
+ "\x9a\x31\xc8\x3c\xd3\x6a\x01\x75"
+ "\x0c\xa3\x17\xae\x45\xdc\x50\xe7"
+ "\x7e\x15\x89\x20\xb7\x2b\xc2\x59"
+ "\xf0\x64\xfb\x92\x06\x9d\x34\xcb"
+ "\x3f\xd6\x6d\x04\x78\x0f\xa6\x1a"
+ "\xb1\x48\xdf\x53\xea\x81\x18\x8c"
+ "\x23\xba\x2e\xc5\x5c\xf3\x67\xfe"
+ "\x95\x09\xa0\x37\xce\x42\xd9\x70"
+ "\x07\x7b\x12\xa9\x1d\xb4\x4b\xe2"
+ "\x56\xed\x84\x1b\x8f\x26\xbd\x31"
+ "\xc8\x5f\xf6\x6a\x01\x98\x0c\xa3"
+ "\x3a\xd1\x45\xdc\x73\x0a\x7e\x15"
+ "\xac\x20\xb7\x4e\xe5\x59\xf0\x87"
+ "\x1e\x92\x29\xc0\x34\xcb\x62\xf9"
+ "\x6d\x04\x9b\x0f\xa6\x3d\xd4\x48"
+ "\xdf\x76\x0d\x81\x18\xaf\x23\xba"
+ "\x51\xe8\x5c\xf3\x8a\x21\x95\x2c"
+ "\xc3\x37\xce\x65\xfc\x70\x07\x9e"
+ "\x12\xa9\x40\xd7\x4b\xe2\x79\x10"
+ "\x84\x1b\xb2\x26\xbd\x54\xeb\x5f"
+ "\xf6\x8d\x01\x98\x2f\xc6\x3a\xd1"
+ "\x68\xff\x73\x0a\xa1\x15\xac\x43"
+ "\xda\x4e\xe5\x7c\x13\x87\x1e\xb5"
+ "\x29\xc0\x57\xee\x62\xf9\x90\x04"
+ "\x9b\x32\xc9\x3d\xd4\x6b\x02\x76"
+ "\x0d\xa4\x18\xaf\x46\xdd\x51\xe8"
+ "\x7f\x16\x8a\x21\xb8\x2c\xc3\x5a"
+ "\xf1\x65\xfc\x93\x07\x9e\x35\xcc"
+ "\x40\xd7\x6e\x05\x79\x10\xa7\x1b"
+ "\xb2\x49\xe0\x54\xeb\x82\x19\x8d"
+ "\x24\xbb\x2f\xc6\x5d\xf4\x68\xff"
+ "\x96\x0a\xa1\x38\xcf\x43\xda\x71"
+ "\x08\x7c\x13\xaa\x1e\xb5\x4c",
+ .psize = 1023,
+ .digest = "\x76\xc9\xd4\x91\x7a\x5f\x0f\xaa"
+ "\x13\x39\xf3\x01\x7a\xfa\xe5\x41"
+ "\x5f\x0b\xf8\xeb\x32\xfc\xbf\xb0"
+ "\xfa\x8c\xcd\x17\x83\xe2\xfa\xeb"
+ "\x1c\x19\xde\xe2\x75\xdc\x34\x64"
+ "\x5f\x35\x9c\x61\x2f\x10\xf9\xec"
+ "\x59\xca\x9d\xcc\x25\x0c\x43\xba"
+ "\x85\xa8\xf8\xfe\xb5\x24\xb2\xee",
+ }
};
@@ -12823,11 +13556,11 @@
#define AES_CBC_DEC_TEST_VECTORS 5
#define HMAC_MD5_ECB_CIPHER_NULL_ENC_TEST_VECTORS 2
#define HMAC_MD5_ECB_CIPHER_NULL_DEC_TEST_VECTORS 2
-#define HMAC_SHA1_ECB_CIPHER_NULL_ENC_TEST_VECTORS 2
-#define HMAC_SHA1_ECB_CIPHER_NULL_DEC_TEST_VECTORS 2
-#define HMAC_SHA1_AES_CBC_ENC_TEST_VECTORS 7
-#define HMAC_SHA256_AES_CBC_ENC_TEST_VECTORS 7
-#define HMAC_SHA512_AES_CBC_ENC_TEST_VECTORS 7
+#define HMAC_SHA1_ECB_CIPHER_NULL_ENC_TEST_VEC 2
+#define HMAC_SHA1_ECB_CIPHER_NULL_DEC_TEST_VEC 2
+#define HMAC_SHA1_AES_CBC_ENC_TEST_VEC 7
+#define HMAC_SHA256_AES_CBC_ENC_TEST_VEC 7
+#define HMAC_SHA512_AES_CBC_ENC_TEST_VEC 7
#define AES_LRW_ENC_TEST_VECTORS 8
#define AES_LRW_DEC_TEST_VECTORS 8
#define AES_XTS_ENC_TEST_VECTORS 5
@@ -12844,7 +13577,7 @@
#define AES_GCM_4106_DEC_TEST_VECTORS 7
#define AES_GCM_4543_ENC_TEST_VECTORS 1
#define AES_GCM_4543_DEC_TEST_VECTORS 2
-#define AES_CCM_ENC_TEST_VECTORS 7
+#define AES_CCM_ENC_TEST_VECTORS 8
#define AES_CCM_DEC_TEST_VECTORS 7
#define AES_CCM_4309_ENC_TEST_VECTORS 7
#define AES_CCM_4309_DEC_TEST_VECTORS 10
@@ -13715,7 +14448,7 @@
},
};
-static struct aead_testvec hmac_sha1_aes_cbc_enc_tv_template[] = {
+static struct aead_testvec hmac_sha1_aes_cbc_enc_tv_temp[] = {
{ /* RFC 3602 Case 1 */
#ifdef __LITTLE_ENDIAN
.key = "\x08\x00" /* rta length */
@@ -13964,7 +14697,7 @@
},
};
-static struct aead_testvec hmac_sha1_ecb_cipher_null_enc_tv_template[] = {
+static struct aead_testvec hmac_sha1_ecb_cipher_null_enc_tv_temp[] = {
{ /* Input data from RFC 2410 Case 1 */
#ifdef __LITTLE_ENDIAN
.key = "\x08\x00" /* rta length */
@@ -14010,7 +14743,7 @@
},
};
-static struct aead_testvec hmac_sha1_ecb_cipher_null_dec_tv_template[] = {
+static struct aead_testvec hmac_sha1_ecb_cipher_null_dec_tv_temp[] = {
{
#ifdef __LITTLE_ENDIAN
.key = "\x08\x00" /* rta length */
@@ -14056,7 +14789,7 @@
},
};
-static struct aead_testvec hmac_sha256_aes_cbc_enc_tv_template[] = {
+static struct aead_testvec hmac_sha256_aes_cbc_enc_tv_temp[] = {
{ /* RFC 3602 Case 1 */
#ifdef __LITTLE_ENDIAN
.key = "\x08\x00" /* rta length */
@@ -14319,7 +15052,7 @@
},
};
-static struct aead_testvec hmac_sha512_aes_cbc_enc_tv_template[] = {
+static struct aead_testvec hmac_sha512_aes_cbc_enc_tv_temp[] = {
{ /* RFC 3602 Case 1 */
#ifdef __LITTLE_ENDIAN
.key = "\x08\x00" /* rta length */
@@ -14638,6 +15371,652 @@
},
};
+#define HMAC_SHA1_DES_CBC_ENC_TEST_VEC 1
+
+static struct aead_testvec hmac_sha1_des_cbc_enc_tv_temp[] = {
+ { /*Generated with cryptopp*/
+#ifdef __LITTLE_ENDIAN
+ .key = "\x08\x00" /* rta length */
+ "\x01\x00" /* rta type */
+#else
+ .key = "\x00\x08" /* rta length */
+ "\x00\x01" /* rta type */
+#endif
+ "\x00\x00\x00\x08" /* enc key length */
+ "\x11\x22\x33\x44\x55\x66\x77\x88"
+ "\x99\xaa\xbb\xcc\xdd\xee\xff\x11"
+ "\x22\x33\x44\x55"
+ "\xE9\xC0\xFF\x2E\x76\x0B\x64\x24",
+ .klen = 8 + 20 + 8,
+ .iv = "\x7D\x33\x88\x93\x0F\x93\xB2\x42",
+ .assoc = "\x00\x00\x43\x21\x00\x00\x00\x01",
+ .alen = 8,
+ .input = "\x6f\x54\x20\x6f\x61\x4d\x79\x6e"
+ "\x53\x20\x63\x65\x65\x72\x73\x74"
+ "\x54\x20\x6f\x6f\x4d\x20\x6e\x61"
+ "\x20\x79\x65\x53\x72\x63\x74\x65"
+ "\x20\x73\x6f\x54\x20\x6f\x61\x4d"
+ "\x79\x6e\x53\x20\x63\x65\x65\x72"
+ "\x73\x74\x54\x20\x6f\x6f\x4d\x20"
+ "\x6e\x61\x20\x79\x65\x53\x72\x63"
+ "\x74\x65\x20\x73\x6f\x54\x20\x6f"
+ "\x61\x4d\x79\x6e\x53\x20\x63\x65"
+ "\x65\x72\x73\x74\x54\x20\x6f\x6f"
+ "\x4d\x20\x6e\x61\x20\x79\x65\x53"
+ "\x72\x63\x74\x65\x20\x73\x6f\x54"
+ "\x20\x6f\x61\x4d\x79\x6e\x53\x20"
+ "\x63\x65\x65\x72\x73\x74\x54\x20"
+ "\x6f\x6f\x4d\x20\x6e\x61\x0a\x79",
+ .ilen = 128,
+ .result = "\x70\xd6\xde\x64\x87\x17\xf1\xe8"
+ "\x54\x31\x85\x37\xed\x6b\x01\x8d"
+ "\xe3\xcc\xe0\x1d\x5e\xf3\xfe\xf1"
+ "\x41\xaa\x33\x91\xa7\x7d\x99\x88"
+ "\x4d\x85\x6e\x2f\xa3\x69\xf5\x82"
+ "\x3a\x6f\x25\xcb\x7d\x58\x1f\x9b"
+ "\xaa\x9c\x11\xd5\x76\x67\xce\xde"
+ "\x56\xd7\x5a\x80\x69\xea\x3a\x02"
+ "\xf0\xc7\x7c\xe3\xcb\x40\xe5\x52"
+ "\xd1\x10\x92\x78\x0b\x8e\x5b\xf1"
+ "\xe3\x26\x1f\xe1\x15\x41\xc7\xba"
+ "\x99\xdb\x08\x51\x1c\xd3\x01\xf4"
+ "\x87\x47\x39\xb8\xd2\xdd\xbd\xfb"
+ "\x66\x13\xdf\x1c\x01\x44\xf0\x7a"
+ "\x1a\x6b\x13\xf5\xd5\x0b\xb8\xba"
+ "\x53\xba\xe1\x76\xe3\x82\x07\x86"
+ "\x95\x16\x20\x09\xf5\x95\x19\xfd"
+ "\x3c\xc7\xe0\x42\xc0\x14\x69\xfa"
+ "\x5c\x44\xa9\x37",
+ .rlen = 128 + 20,
+ },
+};
+
+#define HMAC_SHA224_DES_CBC_ENC_TEST_VEC 1
+
+static struct aead_testvec hmac_sha224_des_cbc_enc_tv_temp[] = {
+ { /*Generated with cryptopp*/
+#ifdef __LITTLE_ENDIAN
+ .key = "\x08\x00" /* rta length */
+ "\x01\x00" /* rta type */
+#else
+ .key = "\x00\x08" /* rta length */
+ "\x00\x01" /* rta type */
+#endif
+ "\x00\x00\x00\x08" /* enc key length */
+ "\x11\x22\x33\x44\x55\x66\x77\x88"
+ "\x99\xaa\xbb\xcc\xdd\xee\xff\x11"
+ "\x22\x33\x44\x55\x66\x77\x88\x99"
+ "\xE9\xC0\xFF\x2E\x76\x0B\x64\x24",
+ .klen = 8 + 24 + 8,
+ .iv = "\x7D\x33\x88\x93\x0F\x93\xB2\x42",
+ .assoc = "\x00\x00\x43\x21\x00\x00\x00\x01",
+ .alen = 8,
+ .input = "\x6f\x54\x20\x6f\x61\x4d\x79\x6e"
+ "\x53\x20\x63\x65\x65\x72\x73\x74"
+ "\x54\x20\x6f\x6f\x4d\x20\x6e\x61"
+ "\x20\x79\x65\x53\x72\x63\x74\x65"
+ "\x20\x73\x6f\x54\x20\x6f\x61\x4d"
+ "\x79\x6e\x53\x20\x63\x65\x65\x72"
+ "\x73\x74\x54\x20\x6f\x6f\x4d\x20"
+ "\x6e\x61\x20\x79\x65\x53\x72\x63"
+ "\x74\x65\x20\x73\x6f\x54\x20\x6f"
+ "\x61\x4d\x79\x6e\x53\x20\x63\x65"
+ "\x65\x72\x73\x74\x54\x20\x6f\x6f"
+ "\x4d\x20\x6e\x61\x20\x79\x65\x53"
+ "\x72\x63\x74\x65\x20\x73\x6f\x54"
+ "\x20\x6f\x61\x4d\x79\x6e\x53\x20"
+ "\x63\x65\x65\x72\x73\x74\x54\x20"
+ "\x6f\x6f\x4d\x20\x6e\x61\x0a\x79",
+ .ilen = 128,
+ .result = "\x70\xd6\xde\x64\x87\x17\xf1\xe8"
+ "\x54\x31\x85\x37\xed\x6b\x01\x8d"
+ "\xe3\xcc\xe0\x1d\x5e\xf3\xfe\xf1"
+ "\x41\xaa\x33\x91\xa7\x7d\x99\x88"
+ "\x4d\x85\x6e\x2f\xa3\x69\xf5\x82"
+ "\x3a\x6f\x25\xcb\x7d\x58\x1f\x9b"
+ "\xaa\x9c\x11\xd5\x76\x67\xce\xde"
+ "\x56\xd7\x5a\x80\x69\xea\x3a\x02"
+ "\xf0\xc7\x7c\xe3\xcb\x40\xe5\x52"
+ "\xd1\x10\x92\x78\x0b\x8e\x5b\xf1"
+ "\xe3\x26\x1f\xe1\x15\x41\xc7\xba"
+ "\x99\xdb\x08\x51\x1c\xd3\x01\xf4"
+ "\x87\x47\x39\xb8\xd2\xdd\xbd\xfb"
+ "\x66\x13\xdf\x1c\x01\x44\xf0\x7a"
+ "\x1a\x6b\x13\xf5\xd5\x0b\xb8\xba"
+ "\x53\xba\xe1\x76\xe3\x82\x07\x86"
+ "\x9c\x2d\x7e\xee\x20\x34\x55\x0a"
+ "\xce\xb5\x4e\x64\x53\xe7\xbf\x91"
+ "\xab\xd4\xd9\xda\xc9\x12\xae\xf7",
+ .rlen = 128 + 24,
+ },
+};
+
+#define HMAC_SHA256_DES_CBC_ENC_TEST_VEC 1
+
+static struct aead_testvec hmac_sha256_des_cbc_enc_tv_temp[] = {
+ { /*Generated with cryptopp*/
+#ifdef __LITTLE_ENDIAN
+ .key = "\x08\x00" /* rta length */
+ "\x01\x00" /* rta type */
+#else
+ .key = "\x00\x08" /* rta length */
+ "\x00\x01" /* rta type */
+#endif
+ "\x00\x00\x00\x08" /* enc key length */
+ "\x11\x22\x33\x44\x55\x66\x77\x88"
+ "\x99\xaa\xbb\xcc\xdd\xee\xff\x11"
+ "\x22\x33\x44\x55\x66\x77\x88\x99"
+ "\xaa\xbb\xcc\xdd\xee\xff\x11\x22"
+ "\xE9\xC0\xFF\x2E\x76\x0B\x64\x24",
+ .klen = 8 + 32 + 8,
+ .iv = "\x7D\x33\x88\x93\x0F\x93\xB2\x42",
+ .assoc = "\x00\x00\x43\x21\x00\x00\x00\x01",
+ .alen = 8,
+ .input = "\x6f\x54\x20\x6f\x61\x4d\x79\x6e"
+ "\x53\x20\x63\x65\x65\x72\x73\x74"
+ "\x54\x20\x6f\x6f\x4d\x20\x6e\x61"
+ "\x20\x79\x65\x53\x72\x63\x74\x65"
+ "\x20\x73\x6f\x54\x20\x6f\x61\x4d"
+ "\x79\x6e\x53\x20\x63\x65\x65\x72"
+ "\x73\x74\x54\x20\x6f\x6f\x4d\x20"
+ "\x6e\x61\x20\x79\x65\x53\x72\x63"
+ "\x74\x65\x20\x73\x6f\x54\x20\x6f"
+ "\x61\x4d\x79\x6e\x53\x20\x63\x65"
+ "\x65\x72\x73\x74\x54\x20\x6f\x6f"
+ "\x4d\x20\x6e\x61\x20\x79\x65\x53"
+ "\x72\x63\x74\x65\x20\x73\x6f\x54"
+ "\x20\x6f\x61\x4d\x79\x6e\x53\x20"
+ "\x63\x65\x65\x72\x73\x74\x54\x20"
+ "\x6f\x6f\x4d\x20\x6e\x61\x0a\x79",
+ .ilen = 128,
+ .result = "\x70\xd6\xde\x64\x87\x17\xf1\xe8"
+ "\x54\x31\x85\x37\xed\x6b\x01\x8d"
+ "\xe3\xcc\xe0\x1d\x5e\xf3\xfe\xf1"
+ "\x41\xaa\x33\x91\xa7\x7d\x99\x88"
+ "\x4d\x85\x6e\x2f\xa3\x69\xf5\x82"
+ "\x3a\x6f\x25\xcb\x7d\x58\x1f\x9b"
+ "\xaa\x9c\x11\xd5\x76\x67\xce\xde"
+ "\x56\xd7\x5a\x80\x69\xea\x3a\x02"
+ "\xf0\xc7\x7c\xe3\xcb\x40\xe5\x52"
+ "\xd1\x10\x92\x78\x0b\x8e\x5b\xf1"
+ "\xe3\x26\x1f\xe1\x15\x41\xc7\xba"
+ "\x99\xdb\x08\x51\x1c\xd3\x01\xf4"
+ "\x87\x47\x39\xb8\xd2\xdd\xbd\xfb"
+ "\x66\x13\xdf\x1c\x01\x44\xf0\x7a"
+ "\x1a\x6b\x13\xf5\xd5\x0b\xb8\xba"
+ "\x53\xba\xe1\x76\xe3\x82\x07\x86"
+ "\xc6\x58\xa1\x60\x70\x91\x39\x36"
+ "\x50\xf6\x5d\xab\x4b\x51\x4e\x5e"
+ "\xde\x63\xde\x76\x52\xde\x9f\xba"
+ "\x90\xcf\x15\xf2\xbb\x6e\x84\x00",
+ .rlen = 128 + 32,
+ },
+};
+
+#define HMAC_SHA384_DES_CBC_ENC_TEST_VEC 1
+
+static struct aead_testvec hmac_sha384_des_cbc_enc_tv_temp[] = {
+ { /*Generated with cryptopp*/
+#ifdef __LITTLE_ENDIAN
+ .key = "\x08\x00" /* rta length */
+ "\x01\x00" /* rta type */
+#else
+ .key = "\x00\x08" /* rta length */
+ "\x00\x01" /* rta type */
+#endif
+ "\x00\x00\x00\x08" /* enc key length */
+ "\x11\x22\x33\x44\x55\x66\x77\x88"
+ "\x99\xaa\xbb\xcc\xdd\xee\xff\x11"
+ "\x22\x33\x44\x55\x66\x77\x88\x99"
+ "\xaa\xbb\xcc\xdd\xee\xff\x11\x22"
+ "\x33\x44\x55\x66\x77\x88\x99\xaa"
+ "\xbb\xcc\xdd\xee\xff\x11\x22\x33"
+ "\xE9\xC0\xFF\x2E\x76\x0B\x64\x24",
+ .klen = 8 + 48 + 8,
+ .iv = "\x7D\x33\x88\x93\x0F\x93\xB2\x42",
+ .assoc = "\x00\x00\x43\x21\x00\x00\x00\x01",
+ .alen = 8,
+ .input = "\x6f\x54\x20\x6f\x61\x4d\x79\x6e"
+ "\x53\x20\x63\x65\x65\x72\x73\x74"
+ "\x54\x20\x6f\x6f\x4d\x20\x6e\x61"
+ "\x20\x79\x65\x53\x72\x63\x74\x65"
+ "\x20\x73\x6f\x54\x20\x6f\x61\x4d"
+ "\x79\x6e\x53\x20\x63\x65\x65\x72"
+ "\x73\x74\x54\x20\x6f\x6f\x4d\x20"
+ "\x6e\x61\x20\x79\x65\x53\x72\x63"
+ "\x74\x65\x20\x73\x6f\x54\x20\x6f"
+ "\x61\x4d\x79\x6e\x53\x20\x63\x65"
+ "\x65\x72\x73\x74\x54\x20\x6f\x6f"
+ "\x4d\x20\x6e\x61\x20\x79\x65\x53"
+ "\x72\x63\x74\x65\x20\x73\x6f\x54"
+ "\x20\x6f\x61\x4d\x79\x6e\x53\x20"
+ "\x63\x65\x65\x72\x73\x74\x54\x20"
+ "\x6f\x6f\x4d\x20\x6e\x61\x0a\x79",
+ .ilen = 128,
+ .result = "\x70\xd6\xde\x64\x87\x17\xf1\xe8"
+ "\x54\x31\x85\x37\xed\x6b\x01\x8d"
+ "\xe3\xcc\xe0\x1d\x5e\xf3\xfe\xf1"
+ "\x41\xaa\x33\x91\xa7\x7d\x99\x88"
+ "\x4d\x85\x6e\x2f\xa3\x69\xf5\x82"
+ "\x3a\x6f\x25\xcb\x7d\x58\x1f\x9b"
+ "\xaa\x9c\x11\xd5\x76\x67\xce\xde"
+ "\x56\xd7\x5a\x80\x69\xea\x3a\x02"
+ "\xf0\xc7\x7c\xe3\xcb\x40\xe5\x52"
+ "\xd1\x10\x92\x78\x0b\x8e\x5b\xf1"
+ "\xe3\x26\x1f\xe1\x15\x41\xc7\xba"
+ "\x99\xdb\x08\x51\x1c\xd3\x01\xf4"
+ "\x87\x47\x39\xb8\xd2\xdd\xbd\xfb"
+ "\x66\x13\xdf\x1c\x01\x44\xf0\x7a"
+ "\x1a\x6b\x13\xf5\xd5\x0b\xb8\xba"
+ "\x53\xba\xe1\x76\xe3\x82\x07\x86"
+ "\xa8\x8e\x9c\x74\x8c\x2b\x99\xa0"
+ "\xc8\x8c\xef\x25\x07\x83\x11\x3a"
+ "\x31\x8d\xbe\x3b\x6a\xd7\x96\xfe"
+ "\x5e\x67\xb5\x74\xe7\xe7\x85\x61"
+ "\x6a\x95\x26\x75\xcc\x53\x89\xf3"
+ "\x74\xc9\x2a\x76\x20\xa2\x64\x62",
+ .rlen = 128 + 48,
+ },
+};
+
+#define HMAC_SHA512_DES_CBC_ENC_TEST_VEC 1
+
+static struct aead_testvec hmac_sha512_des_cbc_enc_tv_temp[] = {
+ { /*Generated with cryptopp*/
+#ifdef __LITTLE_ENDIAN
+ .key = "\x08\x00" /* rta length */
+ "\x01\x00" /* rta type */
+#else
+ .key = "\x00\x08" /* rta length */
+ "\x00\x01" /* rta type */
+#endif
+ "\x00\x00\x00\x08" /* enc key length */
+ "\x11\x22\x33\x44\x55\x66\x77\x88"
+ "\x99\xaa\xbb\xcc\xdd\xee\xff\x11"
+ "\x22\x33\x44\x55\x66\x77\x88\x99"
+ "\xaa\xbb\xcc\xdd\xee\xff\x11\x22"
+ "\x33\x44\x55\x66\x77\x88\x99\xaa"
+ "\xbb\xcc\xdd\xee\xff\x11\x22\x33"
+ "\x44\x55\x66\x77\x88\x99\xaa\xbb"
+ "\xcc\xdd\xee\xff\x11\x22\x33\x44"
+ "\xE9\xC0\xFF\x2E\x76\x0B\x64\x24",
+ .klen = 8 + 64 + 8,
+ .iv = "\x7D\x33\x88\x93\x0F\x93\xB2\x42",
+ .assoc = "\x00\x00\x43\x21\x00\x00\x00\x01",
+ .alen = 8,
+ .input = "\x6f\x54\x20\x6f\x61\x4d\x79\x6e"
+ "\x53\x20\x63\x65\x65\x72\x73\x74"
+ "\x54\x20\x6f\x6f\x4d\x20\x6e\x61"
+ "\x20\x79\x65\x53\x72\x63\x74\x65"
+ "\x20\x73\x6f\x54\x20\x6f\x61\x4d"
+ "\x79\x6e\x53\x20\x63\x65\x65\x72"
+ "\x73\x74\x54\x20\x6f\x6f\x4d\x20"
+ "\x6e\x61\x20\x79\x65\x53\x72\x63"
+ "\x74\x65\x20\x73\x6f\x54\x20\x6f"
+ "\x61\x4d\x79\x6e\x53\x20\x63\x65"
+ "\x65\x72\x73\x74\x54\x20\x6f\x6f"
+ "\x4d\x20\x6e\x61\x20\x79\x65\x53"
+ "\x72\x63\x74\x65\x20\x73\x6f\x54"
+ "\x20\x6f\x61\x4d\x79\x6e\x53\x20"
+ "\x63\x65\x65\x72\x73\x74\x54\x20"
+ "\x6f\x6f\x4d\x20\x6e\x61\x0a\x79",
+ .ilen = 128,
+ .result = "\x70\xd6\xde\x64\x87\x17\xf1\xe8"
+ "\x54\x31\x85\x37\xed\x6b\x01\x8d"
+ "\xe3\xcc\xe0\x1d\x5e\xf3\xfe\xf1"
+ "\x41\xaa\x33\x91\xa7\x7d\x99\x88"
+ "\x4d\x85\x6e\x2f\xa3\x69\xf5\x82"
+ "\x3a\x6f\x25\xcb\x7d\x58\x1f\x9b"
+ "\xaa\x9c\x11\xd5\x76\x67\xce\xde"
+ "\x56\xd7\x5a\x80\x69\xea\x3a\x02"
+ "\xf0\xc7\x7c\xe3\xcb\x40\xe5\x52"
+ "\xd1\x10\x92\x78\x0b\x8e\x5b\xf1"
+ "\xe3\x26\x1f\xe1\x15\x41\xc7\xba"
+ "\x99\xdb\x08\x51\x1c\xd3\x01\xf4"
+ "\x87\x47\x39\xb8\xd2\xdd\xbd\xfb"
+ "\x66\x13\xdf\x1c\x01\x44\xf0\x7a"
+ "\x1a\x6b\x13\xf5\xd5\x0b\xb8\xba"
+ "\x53\xba\xe1\x76\xe3\x82\x07\x86"
+ "\xc6\x2c\x73\x88\xb0\x9d\x5f\x3e"
+ "\x5b\x78\xca\x0e\xab\x8a\xa3\xbb"
+ "\xd9\x1d\xc3\xe3\x05\xac\x76\xfb"
+ "\x58\x83\xda\x67\xfb\x21\x24\xa2"
+ "\xb1\xa7\xd7\x66\xa6\x8d\xa6\x93"
+ "\x97\xe2\xe3\xb8\xaa\x48\x85\xee"
+ "\x8c\xf6\x07\x95\x1f\xa6\x6c\x96"
+ "\x99\xc7\x5c\x8d\xd8\xb5\x68\x7b",
+ .rlen = 128 + 64,
+ },
+};
+
+#define HMAC_SHA1_DES3_EDE_CBC_ENC_TEST_VEC 1
+
+static struct aead_testvec hmac_sha1_des3_ede_cbc_enc_tv_temp[] = {
+ { /*Generated with cryptopp*/
+#ifdef __LITTLE_ENDIAN
+ .key = "\x08\x00" /* rta length */
+ "\x01\x00" /* rta type */
+#else
+ .key = "\x00\x08" /* rta length */
+ "\x00\x01" /* rta type */
+#endif
+ "\x00\x00\x00\x18" /* enc key length */
+ "\x11\x22\x33\x44\x55\x66\x77\x88"
+ "\x99\xaa\xbb\xcc\xdd\xee\xff\x11"
+ "\x22\x33\x44\x55"
+ "\xE9\xC0\xFF\x2E\x76\x0B\x64\x24"
+ "\x44\x4D\x99\x5A\x12\xD6\x40\xC0"
+ "\xEA\xC2\x84\xE8\x14\x95\xDB\xE8",
+ .klen = 8 + 20 + 24,
+ .iv = "\x7D\x33\x88\x93\x0F\x93\xB2\x42",
+ .assoc = "\x00\x00\x43\x21\x00\x00\x00\x01",
+ .alen = 8,
+ .input = "\x6f\x54\x20\x6f\x61\x4d\x79\x6e"
+ "\x53\x20\x63\x65\x65\x72\x73\x74"
+ "\x54\x20\x6f\x6f\x4d\x20\x6e\x61"
+ "\x20\x79\x65\x53\x72\x63\x74\x65"
+ "\x20\x73\x6f\x54\x20\x6f\x61\x4d"
+ "\x79\x6e\x53\x20\x63\x65\x65\x72"
+ "\x73\x74\x54\x20\x6f\x6f\x4d\x20"
+ "\x6e\x61\x20\x79\x65\x53\x72\x63"
+ "\x74\x65\x20\x73\x6f\x54\x20\x6f"
+ "\x61\x4d\x79\x6e\x53\x20\x63\x65"
+ "\x65\x72\x73\x74\x54\x20\x6f\x6f"
+ "\x4d\x20\x6e\x61\x20\x79\x65\x53"
+ "\x72\x63\x74\x65\x20\x73\x6f\x54"
+ "\x20\x6f\x61\x4d\x79\x6e\x53\x20"
+ "\x63\x65\x65\x72\x73\x74\x54\x20"
+ "\x6f\x6f\x4d\x20\x6e\x61\x0a\x79",
+ .ilen = 128,
+ .result = "\x0e\x2d\xb6\x97\x3c\x56\x33\xf4"
+ "\x67\x17\x21\xc7\x6e\x8a\xd5\x49"
+ "\x74\xb3\x49\x05\xc5\x1c\xd0\xed"
+ "\x12\x56\x5c\x53\x96\xb6\x00\x7d"
+ "\x90\x48\xfc\xf5\x8d\x29\x39\xcc"
+ "\x8a\xd5\x35\x18\x36\x23\x4e\xd7"
+ "\x76\xd1\xda\x0c\x94\x67\xbb\x04"
+ "\x8b\xf2\x03\x6c\xa8\xcf\xb6\xea"
+ "\x22\x64\x47\xaa\x8f\x75\x13\xbf"
+ "\x9f\xc2\xc3\xf0\xc9\x56\xc5\x7a"
+ "\x71\x63\x2e\x89\x7b\x1e\x12\xca"
+ "\xe2\x5f\xaf\xd8\xa4\xf8\xc9\x7a"
+ "\xd6\xf9\x21\x31\x62\x44\x45\xa6"
+ "\xd6\xbc\x5a\xd3\x2d\x54\x43\xcc"
+ "\x9d\xde\xa5\x70\xe9\x42\x45\x8a"
+ "\x6b\xfa\xb1\x91\x13\xb0\xd9\x19"
+ "\x67\x6d\xb1\xf5\xb8\x10\xdc\xc6"
+ "\x75\x86\x96\x6b\xb1\xc5\xe4\xcf"
+ "\xd1\x60\x91\xb3",
+ .rlen = 128 + 20,
+ },
+};
+
+#define HMAC_SHA224_DES3_EDE_CBC_ENC_TEST_VEC 1
+
+static struct aead_testvec hmac_sha224_des3_ede_cbc_enc_tv_temp[] = {
+ { /*Generated with cryptopp*/
+#ifdef __LITTLE_ENDIAN
+ .key = "\x08\x00" /* rta length */
+ "\x01\x00" /* rta type */
+#else
+ .key = "\x00\x08" /* rta length */
+ "\x00\x01" /* rta type */
+#endif
+ "\x00\x00\x00\x18" /* enc key length */
+ "\x11\x22\x33\x44\x55\x66\x77\x88"
+ "\x99\xaa\xbb\xcc\xdd\xee\xff\x11"
+ "\x22\x33\x44\x55\x66\x77\x88\x99"
+ "\xE9\xC0\xFF\x2E\x76\x0B\x64\x24"
+ "\x44\x4D\x99\x5A\x12\xD6\x40\xC0"
+ "\xEA\xC2\x84\xE8\x14\x95\xDB\xE8",
+ .klen = 8 + 24 + 24,
+ .iv = "\x7D\x33\x88\x93\x0F\x93\xB2\x42",
+ .assoc = "\x00\x00\x43\x21\x00\x00\x00\x01",
+ .alen = 8,
+ .input = "\x6f\x54\x20\x6f\x61\x4d\x79\x6e"
+ "\x53\x20\x63\x65\x65\x72\x73\x74"
+ "\x54\x20\x6f\x6f\x4d\x20\x6e\x61"
+ "\x20\x79\x65\x53\x72\x63\x74\x65"
+ "\x20\x73\x6f\x54\x20\x6f\x61\x4d"
+ "\x79\x6e\x53\x20\x63\x65\x65\x72"
+ "\x73\x74\x54\x20\x6f\x6f\x4d\x20"
+ "\x6e\x61\x20\x79\x65\x53\x72\x63"
+ "\x74\x65\x20\x73\x6f\x54\x20\x6f"
+ "\x61\x4d\x79\x6e\x53\x20\x63\x65"
+ "\x65\x72\x73\x74\x54\x20\x6f\x6f"
+ "\x4d\x20\x6e\x61\x20\x79\x65\x53"
+ "\x72\x63\x74\x65\x20\x73\x6f\x54"
+ "\x20\x6f\x61\x4d\x79\x6e\x53\x20"
+ "\x63\x65\x65\x72\x73\x74\x54\x20"
+ "\x6f\x6f\x4d\x20\x6e\x61\x0a\x79",
+ .ilen = 128,
+ .result = "\x0e\x2d\xb6\x97\x3c\x56\x33\xf4"
+ "\x67\x17\x21\xc7\x6e\x8a\xd5\x49"
+ "\x74\xb3\x49\x05\xc5\x1c\xd0\xed"
+ "\x12\x56\x5c\x53\x96\xb6\x00\x7d"
+ "\x90\x48\xfc\xf5\x8d\x29\x39\xcc"
+ "\x8a\xd5\x35\x18\x36\x23\x4e\xd7"
+ "\x76\xd1\xda\x0c\x94\x67\xbb\x04"
+ "\x8b\xf2\x03\x6c\xa8\xcf\xb6\xea"
+ "\x22\x64\x47\xaa\x8f\x75\x13\xbf"
+ "\x9f\xc2\xc3\xf0\xc9\x56\xc5\x7a"
+ "\x71\x63\x2e\x89\x7b\x1e\x12\xca"
+ "\xe2\x5f\xaf\xd8\xa4\xf8\xc9\x7a"
+ "\xd6\xf9\x21\x31\x62\x44\x45\xa6"
+ "\xd6\xbc\x5a\xd3\x2d\x54\x43\xcc"
+ "\x9d\xde\xa5\x70\xe9\x42\x45\x8a"
+ "\x6b\xfa\xb1\x91\x13\xb0\xd9\x19"
+ "\x15\x24\x7f\x5a\x45\x4a\x66\xce"
+ "\x2b\x0b\x93\x99\x2f\x9d\x0c\x6c"
+ "\x56\x1f\xe1\xa6\x41\xb2\x4c\xd0",
+ .rlen = 128 + 24,
+ },
+};
+
+#define HMAC_SHA256_DES3_EDE_CBC_ENC_TEST_VEC 1
+
+static struct aead_testvec hmac_sha256_des3_ede_cbc_enc_tv_temp[] = {
+ { /*Generated with cryptopp*/
+#ifdef __LITTLE_ENDIAN
+ .key = "\x08\x00" /* rta length */
+ "\x01\x00" /* rta type */
+#else
+ .key = "\x00\x08" /* rta length */
+ "\x00\x01" /* rta type */
+#endif
+ "\x00\x00\x00\x18" /* enc key length */
+ "\x11\x22\x33\x44\x55\x66\x77\x88"
+ "\x99\xaa\xbb\xcc\xdd\xee\xff\x11"
+ "\x22\x33\x44\x55\x66\x77\x88\x99"
+ "\xaa\xbb\xcc\xdd\xee\xff\x11\x22"
+ "\xE9\xC0\xFF\x2E\x76\x0B\x64\x24"
+ "\x44\x4D\x99\x5A\x12\xD6\x40\xC0"
+ "\xEA\xC2\x84\xE8\x14\x95\xDB\xE8",
+ .klen = 8 + 32 + 24,
+ .iv = "\x7D\x33\x88\x93\x0F\x93\xB2\x42",
+ .assoc = "\x00\x00\x43\x21\x00\x00\x00\x01",
+ .alen = 8,
+ .input = "\x6f\x54\x20\x6f\x61\x4d\x79\x6e"
+ "\x53\x20\x63\x65\x65\x72\x73\x74"
+ "\x54\x20\x6f\x6f\x4d\x20\x6e\x61"
+ "\x20\x79\x65\x53\x72\x63\x74\x65"
+ "\x20\x73\x6f\x54\x20\x6f\x61\x4d"
+ "\x79\x6e\x53\x20\x63\x65\x65\x72"
+ "\x73\x74\x54\x20\x6f\x6f\x4d\x20"
+ "\x6e\x61\x20\x79\x65\x53\x72\x63"
+ "\x74\x65\x20\x73\x6f\x54\x20\x6f"
+ "\x61\x4d\x79\x6e\x53\x20\x63\x65"
+ "\x65\x72\x73\x74\x54\x20\x6f\x6f"
+ "\x4d\x20\x6e\x61\x20\x79\x65\x53"
+ "\x72\x63\x74\x65\x20\x73\x6f\x54"
+ "\x20\x6f\x61\x4d\x79\x6e\x53\x20"
+ "\x63\x65\x65\x72\x73\x74\x54\x20"
+ "\x6f\x6f\x4d\x20\x6e\x61\x0a\x79",
+ .ilen = 128,
+ .result = "\x0e\x2d\xb6\x97\x3c\x56\x33\xf4"
+ "\x67\x17\x21\xc7\x6e\x8a\xd5\x49"
+ "\x74\xb3\x49\x05\xc5\x1c\xd0\xed"
+ "\x12\x56\x5c\x53\x96\xb6\x00\x7d"
+ "\x90\x48\xfc\xf5\x8d\x29\x39\xcc"
+ "\x8a\xd5\x35\x18\x36\x23\x4e\xd7"
+ "\x76\xd1\xda\x0c\x94\x67\xbb\x04"
+ "\x8b\xf2\x03\x6c\xa8\xcf\xb6\xea"
+ "\x22\x64\x47\xaa\x8f\x75\x13\xbf"
+ "\x9f\xc2\xc3\xf0\xc9\x56\xc5\x7a"
+ "\x71\x63\x2e\x89\x7b\x1e\x12\xca"
+ "\xe2\x5f\xaf\xd8\xa4\xf8\xc9\x7a"
+ "\xd6\xf9\x21\x31\x62\x44\x45\xa6"
+ "\xd6\xbc\x5a\xd3\x2d\x54\x43\xcc"
+ "\x9d\xde\xa5\x70\xe9\x42\x45\x8a"
+ "\x6b\xfa\xb1\x91\x13\xb0\xd9\x19"
+ "\x73\xb0\xea\x9f\xe8\x18\x80\xd6"
+ "\x56\x38\x44\xc0\xdb\xe3\x4f\x71"
+ "\xf7\xce\xd1\xd3\xf8\xbd\x3e\x4f"
+ "\xca\x43\x95\xdf\x80\x61\x81\xa9",
+ .rlen = 128 + 32,
+ },
+};
+
+#define HMAC_SHA384_DES3_EDE_CBC_ENC_TEST_VEC 1
+
+static struct aead_testvec hmac_sha384_des3_ede_cbc_enc_tv_temp[] = {
+ { /*Generated with cryptopp*/
+#ifdef __LITTLE_ENDIAN
+ .key = "\x08\x00" /* rta length */
+ "\x01\x00" /* rta type */
+#else
+ .key = "\x00\x08" /* rta length */
+ "\x00\x01" /* rta type */
+#endif
+ "\x00\x00\x00\x18" /* enc key length */
+ "\x11\x22\x33\x44\x55\x66\x77\x88"
+ "\x99\xaa\xbb\xcc\xdd\xee\xff\x11"
+ "\x22\x33\x44\x55\x66\x77\x88\x99"
+ "\xaa\xbb\xcc\xdd\xee\xff\x11\x22"
+ "\x33\x44\x55\x66\x77\x88\x99\xaa"
+ "\xbb\xcc\xdd\xee\xff\x11\x22\x33"
+ "\xE9\xC0\xFF\x2E\x76\x0B\x64\x24"
+ "\x44\x4D\x99\x5A\x12\xD6\x40\xC0"
+ "\xEA\xC2\x84\xE8\x14\x95\xDB\xE8",
+ .klen = 8 + 48 + 24,
+ .iv = "\x7D\x33\x88\x93\x0F\x93\xB2\x42",
+ .assoc = "\x00\x00\x43\x21\x00\x00\x00\x01",
+ .alen = 8,
+ .input = "\x6f\x54\x20\x6f\x61\x4d\x79\x6e"
+ "\x53\x20\x63\x65\x65\x72\x73\x74"
+ "\x54\x20\x6f\x6f\x4d\x20\x6e\x61"
+ "\x20\x79\x65\x53\x72\x63\x74\x65"
+ "\x20\x73\x6f\x54\x20\x6f\x61\x4d"
+ "\x79\x6e\x53\x20\x63\x65\x65\x72"
+ "\x73\x74\x54\x20\x6f\x6f\x4d\x20"
+ "\x6e\x61\x20\x79\x65\x53\x72\x63"
+ "\x74\x65\x20\x73\x6f\x54\x20\x6f"
+ "\x61\x4d\x79\x6e\x53\x20\x63\x65"
+ "\x65\x72\x73\x74\x54\x20\x6f\x6f"
+ "\x4d\x20\x6e\x61\x20\x79\x65\x53"
+ "\x72\x63\x74\x65\x20\x73\x6f\x54"
+ "\x20\x6f\x61\x4d\x79\x6e\x53\x20"
+ "\x63\x65\x65\x72\x73\x74\x54\x20"
+ "\x6f\x6f\x4d\x20\x6e\x61\x0a\x79",
+ .ilen = 128,
+ .result = "\x0e\x2d\xb6\x97\x3c\x56\x33\xf4"
+ "\x67\x17\x21\xc7\x6e\x8a\xd5\x49"
+ "\x74\xb3\x49\x05\xc5\x1c\xd0\xed"
+ "\x12\x56\x5c\x53\x96\xb6\x00\x7d"
+ "\x90\x48\xfc\xf5\x8d\x29\x39\xcc"
+ "\x8a\xd5\x35\x18\x36\x23\x4e\xd7"
+ "\x76\xd1\xda\x0c\x94\x67\xbb\x04"
+ "\x8b\xf2\x03\x6c\xa8\xcf\xb6\xea"
+ "\x22\x64\x47\xaa\x8f\x75\x13\xbf"
+ "\x9f\xc2\xc3\xf0\xc9\x56\xc5\x7a"
+ "\x71\x63\x2e\x89\x7b\x1e\x12\xca"
+ "\xe2\x5f\xaf\xd8\xa4\xf8\xc9\x7a"
+ "\xd6\xf9\x21\x31\x62\x44\x45\xa6"
+ "\xd6\xbc\x5a\xd3\x2d\x54\x43\xcc"
+ "\x9d\xde\xa5\x70\xe9\x42\x45\x8a"
+ "\x6b\xfa\xb1\x91\x13\xb0\xd9\x19"
+ "\x6d\x77\xfc\x80\x9d\x8a\x9c\xb7"
+ "\x70\xe7\x93\xbf\x73\xe6\x9f\x83"
+ "\x99\x62\x23\xe6\x5b\xd0\xda\x18"
+ "\xa4\x32\x8a\x0b\x46\xd7\xf0\x39"
+ "\x36\x5d\x13\x2f\x86\x10\x78\xd6"
+ "\xd6\xbe\x5c\xb9\x15\x89\xf9\x1b",
+ .rlen = 128 + 48,
+ },
+};
+
+#define HMAC_SHA512_DES3_EDE_CBC_ENC_TEST_VEC 1
+
+static struct aead_testvec hmac_sha512_des3_ede_cbc_enc_tv_temp[] = {
+ { /*Generated with cryptopp*/
+#ifdef __LITTLE_ENDIAN
+ .key = "\x08\x00" /* rta length */
+ "\x01\x00" /* rta type */
+#else
+ .key = "\x00\x08" /* rta length */
+ "\x00\x01" /* rta type */
+#endif
+ "\x00\x00\x00\x18" /* enc key length */
+ "\x11\x22\x33\x44\x55\x66\x77\x88"
+ "\x99\xaa\xbb\xcc\xdd\xee\xff\x11"
+ "\x22\x33\x44\x55\x66\x77\x88\x99"
+ "\xaa\xbb\xcc\xdd\xee\xff\x11\x22"
+ "\x33\x44\x55\x66\x77\x88\x99\xaa"
+ "\xbb\xcc\xdd\xee\xff\x11\x22\x33"
+ "\x44\x55\x66\x77\x88\x99\xaa\xbb"
+ "\xcc\xdd\xee\xff\x11\x22\x33\x44"
+ "\xE9\xC0\xFF\x2E\x76\x0B\x64\x24"
+ "\x44\x4D\x99\x5A\x12\xD6\x40\xC0"
+ "\xEA\xC2\x84\xE8\x14\x95\xDB\xE8",
+ .klen = 8 + 64 + 24,
+ .iv = "\x7D\x33\x88\x93\x0F\x93\xB2\x42",
+ .assoc = "\x00\x00\x43\x21\x00\x00\x00\x01",
+ .alen = 8,
+ .input = "\x6f\x54\x20\x6f\x61\x4d\x79\x6e"
+ "\x53\x20\x63\x65\x65\x72\x73\x74"
+ "\x54\x20\x6f\x6f\x4d\x20\x6e\x61"
+ "\x20\x79\x65\x53\x72\x63\x74\x65"
+ "\x20\x73\x6f\x54\x20\x6f\x61\x4d"
+ "\x79\x6e\x53\x20\x63\x65\x65\x72"
+ "\x73\x74\x54\x20\x6f\x6f\x4d\x20"
+ "\x6e\x61\x20\x79\x65\x53\x72\x63"
+ "\x74\x65\x20\x73\x6f\x54\x20\x6f"
+ "\x61\x4d\x79\x6e\x53\x20\x63\x65"
+ "\x65\x72\x73\x74\x54\x20\x6f\x6f"
+ "\x4d\x20\x6e\x61\x20\x79\x65\x53"
+ "\x72\x63\x74\x65\x20\x73\x6f\x54"
+ "\x20\x6f\x61\x4d\x79\x6e\x53\x20"
+ "\x63\x65\x65\x72\x73\x74\x54\x20"
+ "\x6f\x6f\x4d\x20\x6e\x61\x0a\x79",
+ .ilen = 128,
+ .result = "\x0e\x2d\xb6\x97\x3c\x56\x33\xf4"
+ "\x67\x17\x21\xc7\x6e\x8a\xd5\x49"
+ "\x74\xb3\x49\x05\xc5\x1c\xd0\xed"
+ "\x12\x56\x5c\x53\x96\xb6\x00\x7d"
+ "\x90\x48\xfc\xf5\x8d\x29\x39\xcc"
+ "\x8a\xd5\x35\x18\x36\x23\x4e\xd7"
+ "\x76\xd1\xda\x0c\x94\x67\xbb\x04"
+ "\x8b\xf2\x03\x6c\xa8\xcf\xb6\xea"
+ "\x22\x64\x47\xaa\x8f\x75\x13\xbf"
+ "\x9f\xc2\xc3\xf0\xc9\x56\xc5\x7a"
+ "\x71\x63\x2e\x89\x7b\x1e\x12\xca"
+ "\xe2\x5f\xaf\xd8\xa4\xf8\xc9\x7a"
+ "\xd6\xf9\x21\x31\x62\x44\x45\xa6"
+ "\xd6\xbc\x5a\xd3\x2d\x54\x43\xcc"
+ "\x9d\xde\xa5\x70\xe9\x42\x45\x8a"
+ "\x6b\xfa\xb1\x91\x13\xb0\xd9\x19"
+ "\x41\xb5\x1f\xbb\xbd\x4e\xb8\x32"
+ "\x22\x86\x4e\x57\x1b\x2a\xd8\x6e"
+ "\xa9\xfb\xc8\xf3\xbf\x2d\xae\x2b"
+ "\x3b\xbc\x41\xe8\x38\xbb\xf1\x60"
+ "\x4c\x68\xa9\x4e\x8c\x73\xa7\xc0"
+ "\x2a\x74\xd4\x65\x12\xcb\x55\xf2"
+ "\xd5\x02\x6d\xe6\xaf\xc9\x2f\xf2"
+ "\x57\xaa\x85\xf7\xf3\x6a\xcb\xdb",
+ .rlen = 128 + 64,
+ },
+};
+
static struct cipher_testvec aes_lrw_enc_tv_template[] = {
/* from http://grouper.ieee.org/groups/1619/email/pdf00017.pdf */
{ /* LRW-32-AES 1 */
@@ -18746,7 +20125,29 @@
"\x7c\xf9\xbe\xc2\x40\x88\x97\xc6"
"\xba",
.rlen = 33,
- },
+ }, {
+ /*
+ * This is the same vector as aes_ccm_rfc4309_enc_tv_template[0]
+ * below but rewritten to use the ccm algorithm directly.
+ */
+ .key = "\x83\xac\x54\x66\xc2\xeb\xe5\x05"
+ "\x2e\x01\xd1\xfc\x5d\x82\x66\x2e",
+ .klen = 16,
+ .iv = "\x03\x96\xac\x59\x30\x07\xa1\xe2\xa2\xc7\x55\x24\0\0\0\0",
+ .alen = 0,
+ .input = "\x19\xc8\x81\xf6\xe9\x86\xff\x93"
+ "\x0b\x78\x67\xe5\xbb\xb7\xfc\x6e"
+ "\x83\x77\xb3\xa6\x0c\x8c\x9f\x9c"
+ "\x35\x2e\xad\xe0\x62\xf9\x91\xa1",
+ .ilen = 32,
+ .result = "\xab\x6f\xe1\x69\x1d\x19\x99\xa8"
+ "\x92\xa0\xc4\x6f\x7e\xe2\x8b\xb1"
+ "\x70\xbb\x8c\xa6\x4c\x6e\x97\x8a"
+ "\x57\x2b\xbe\x5d\x98\xa6\xb1\x32"
+ "\xda\x24\xea\xd9\xa1\x39\x98\xfd"
+ "\xa4\xbe\xd9\xf2\x1a\x6d\x22\xa8",
+ .rlen = 48,
+ }
};
static struct aead_testvec aes_ccm_dec_tv_template[] = {
diff --git a/drivers/ata/Kconfig b/drivers/ata/Kconfig
index 0033faf..7671dba 100644
--- a/drivers/ata/Kconfig
+++ b/drivers/ata/Kconfig
@@ -123,6 +123,15 @@
If unsure, say N.
+config AHCI_MVEBU
+ tristate "Marvell EBU AHCI SATA support"
+ depends on ARCH_MVEBU
+ help
+ This option enables support for the Marvebu EBU SoC's
+ onboard AHCI SATA.
+
+ If unsure, say N.
+
config AHCI_SUNXI
tristate "Allwinner sunxi AHCI SATA support"
depends on ARCH_SUNXI
diff --git a/drivers/ata/Makefile b/drivers/ata/Makefile
index 44c8016..5a02aee 100644
--- a/drivers/ata/Makefile
+++ b/drivers/ata/Makefile
@@ -12,6 +12,7 @@
obj-$(CONFIG_SATA_HIGHBANK) += sata_highbank.o libahci.o
obj-$(CONFIG_AHCI_DA850) += ahci_da850.o libahci.o libahci_platform.o
obj-$(CONFIG_AHCI_IMX) += ahci_imx.o libahci.o libahci_platform.o
+obj-$(CONFIG_AHCI_MVEBU) += ahci_mvebu.o libahci.o libahci_platform.o
obj-$(CONFIG_AHCI_SUNXI) += ahci_sunxi.o libahci.o libahci_platform.o
obj-$(CONFIG_AHCI_ST) += ahci_st.o libahci.o libahci_platform.o
obj-$(CONFIG_AHCI_XGENE) += ahci_xgene.o libahci.o libahci_platform.o
diff --git a/drivers/ata/acard-ahci.c b/drivers/ata/acard-ahci.c
index b51605a..0cd7c7a 100644
--- a/drivers/ata/acard-ahci.c
+++ b/drivers/ata/acard-ahci.c
@@ -77,7 +77,7 @@
static int acard_ahci_port_start(struct ata_port *ap);
static int acard_ahci_init_one(struct pci_dev *pdev, const struct pci_device_id *ent);
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int acard_ahci_pci_device_suspend(struct pci_dev *pdev, pm_message_t mesg);
static int acard_ahci_pci_device_resume(struct pci_dev *pdev);
#endif
@@ -118,13 +118,13 @@
.id_table = acard_ahci_pci_tbl,
.probe = acard_ahci_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = acard_ahci_pci_device_suspend,
.resume = acard_ahci_pci_device_resume,
#endif
};
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int acard_ahci_pci_device_suspend(struct pci_dev *pdev, pm_message_t mesg)
{
struct ata_host *host = pci_get_drvdata(pdev);
diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c
index 6070781..dae5607 100644
--- a/drivers/ata/ahci.c
+++ b/drivers/ata/ahci.c
@@ -445,10 +445,14 @@
.driver_data = board_ahci_yes_fbs }, /* 88se9172 */
{ PCI_DEVICE(PCI_VENDOR_ID_MARVELL_EXT, 0x9192),
.driver_data = board_ahci_yes_fbs }, /* 88se9172 on some Gigabyte */
+ { PCI_DEVICE(PCI_VENDOR_ID_MARVELL_EXT, 0x91a0),
+ .driver_data = board_ahci_yes_fbs },
{ PCI_DEVICE(PCI_VENDOR_ID_MARVELL_EXT, 0x91a3),
.driver_data = board_ahci_yes_fbs },
{ PCI_DEVICE(PCI_VENDOR_ID_MARVELL_EXT, 0x9230),
.driver_data = board_ahci_yes_fbs },
+ { PCI_DEVICE(PCI_VENDOR_ID_TTI, 0x0642),
+ .driver_data = board_ahci_yes_fbs },
/* Promise */
{ PCI_VDEVICE(PROMISE, 0x3f20), board_ahci }, /* PDC42819 */
diff --git a/drivers/ata/ahci.h b/drivers/ata/ahci.h
index af63c75..05882e4 100644
--- a/drivers/ata/ahci.h
+++ b/drivers/ata/ahci.h
@@ -237,6 +237,7 @@
error-handling stage) */
AHCI_HFLAG_MULTI_MSI = (1 << 16), /* multiple PCI MSIs */
AHCI_HFLAG_NO_DEVSLP = (1 << 17), /* no device sleep */
+ AHCI_HFLAG_NO_FBS = (1 << 18), /* no FBS */
/* ap->flags bits */
diff --git a/drivers/ata/ahci_da850.c b/drivers/ata/ahci_da850.c
index 2c83613..2b77d53 100644
--- a/drivers/ata/ahci_da850.c
+++ b/drivers/ata/ahci_da850.c
@@ -85,7 +85,8 @@
da850_sata_init(dev, pwrdn_reg, hpriv->mmio);
- rc = ahci_platform_init_host(pdev, hpriv, &ahci_da850_port_info, 0, 0);
+ rc = ahci_platform_init_host(pdev, hpriv, &ahci_da850_port_info,
+ 0, 0, 0);
if (rc)
goto disable_resources;
diff --git a/drivers/ata/ahci_imx.c b/drivers/ata/ahci_imx.c
index 8befeb6..3a90152 100644
--- a/drivers/ata/ahci_imx.c
+++ b/drivers/ata/ahci_imx.c
@@ -432,7 +432,8 @@
reg_val = clk_get_rate(imxpriv->ahb_clk) / 1000;
writel(reg_val, hpriv->mmio + IMX_TIMER1MS);
- ret = ahci_platform_init_host(pdev, hpriv, &ahci_imx_port_info, 0, 0);
+ ret = ahci_platform_init_host(pdev, hpriv, &ahci_imx_port_info,
+ 0, 0, 0);
if (ret)
imx_sata_disable(hpriv);
diff --git a/drivers/ata/ahci_mvebu.c b/drivers/ata/ahci_mvebu.c
new file mode 100644
index 0000000..fd3dfd7
--- /dev/null
+++ b/drivers/ata/ahci_mvebu.c
@@ -0,0 +1,128 @@
+/*
+ * AHCI glue platform driver for Marvell EBU SOCs
+ *
+ * Copyright (C) 2014 Marvell
+ *
+ * Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+ * Marcin Wojtas <mw@semihalf.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/ahci_platform.h>
+#include <linux/kernel.h>
+#include <linux/mbus.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include "ahci.h"
+
+#define AHCI_VENDOR_SPECIFIC_0_ADDR 0xa0
+#define AHCI_VENDOR_SPECIFIC_0_DATA 0xa4
+
+#define AHCI_WINDOW_CTRL(win) (0x60 + ((win) << 4))
+#define AHCI_WINDOW_BASE(win) (0x64 + ((win) << 4))
+#define AHCI_WINDOW_SIZE(win) (0x68 + ((win) << 4))
+
+static void ahci_mvebu_mbus_config(struct ahci_host_priv *hpriv,
+ const struct mbus_dram_target_info *dram)
+{
+ int i;
+
+ for (i = 0; i < 4; i++) {
+ writel(0, hpriv->mmio + AHCI_WINDOW_CTRL(i));
+ writel(0, hpriv->mmio + AHCI_WINDOW_BASE(i));
+ writel(0, hpriv->mmio + AHCI_WINDOW_SIZE(i));
+ }
+
+ for (i = 0; i < dram->num_cs; i++) {
+ const struct mbus_dram_window *cs = dram->cs + i;
+
+ writel((cs->mbus_attr << 8) |
+ (dram->mbus_dram_target_id << 4) | 1,
+ hpriv->mmio + AHCI_WINDOW_CTRL(i));
+ writel(cs->base, hpriv->mmio + AHCI_WINDOW_BASE(i));
+ writel(((cs->size - 1) & 0xffff0000),
+ hpriv->mmio + AHCI_WINDOW_SIZE(i));
+ }
+}
+
+static void ahci_mvebu_regret_option(struct ahci_host_priv *hpriv)
+{
+ /*
+ * Enable the regret bit to allow the SATA unit to regret a
+ * request that didn't receive an acknowlegde and avoid a
+ * deadlock
+ */
+ writel(0x4, hpriv->mmio + AHCI_VENDOR_SPECIFIC_0_ADDR);
+ writel(0x80, hpriv->mmio + AHCI_VENDOR_SPECIFIC_0_DATA);
+}
+
+static const struct ata_port_info ahci_mvebu_port_info = {
+ .flags = AHCI_FLAG_COMMON,
+ .pio_mask = ATA_PIO4,
+ .udma_mask = ATA_UDMA6,
+ .port_ops = &ahci_platform_ops,
+};
+
+static int ahci_mvebu_probe(struct platform_device *pdev)
+{
+ struct ahci_host_priv *hpriv;
+ const struct mbus_dram_target_info *dram;
+ int rc;
+
+ hpriv = ahci_platform_get_resources(pdev);
+ if (IS_ERR(hpriv))
+ return PTR_ERR(hpriv);
+
+ rc = ahci_platform_enable_resources(hpriv);
+ if (rc)
+ return rc;
+
+ dram = mv_mbus_dram_info();
+ if (!dram)
+ return -ENODEV;
+
+ ahci_mvebu_mbus_config(hpriv, dram);
+ ahci_mvebu_regret_option(hpriv);
+
+ rc = ahci_platform_init_host(pdev, hpriv, &ahci_mvebu_port_info,
+ 0, 0, 0);
+ if (rc)
+ goto disable_resources;
+
+ return 0;
+
+disable_resources:
+ ahci_platform_disable_resources(hpriv);
+ return rc;
+}
+
+static const struct of_device_id ahci_mvebu_of_match[] = {
+ { .compatible = "marvell,armada-380-ahci", },
+ { },
+};
+MODULE_DEVICE_TABLE(of, ahci_mvebu_of_match);
+
+/*
+ * We currently don't provide power management related operations,
+ * since there is no suspend/resume support at the platform level for
+ * Armada 38x for the moment.
+ */
+static struct platform_driver ahci_mvebu_driver = {
+ .probe = ahci_mvebu_probe,
+ .remove = ata_platform_remove_one,
+ .driver = {
+ .name = "ahci-mvebu",
+ .owner = THIS_MODULE,
+ .of_match_table = ahci_mvebu_of_match,
+ },
+};
+module_platform_driver(ahci_mvebu_driver);
+
+MODULE_DESCRIPTION("Marvell EBU AHCI SATA driver");
+MODULE_AUTHOR("Thomas Petazzoni <thomas.petazzoni@free-electrons.com>, Marcin Wojtas <mw@semihalf.com>");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("platform:ahci_mvebu");
diff --git a/drivers/ata/ahci_platform.c b/drivers/ata/ahci_platform.c
index ef67e79..ebe505c 100644
--- a/drivers/ata/ahci_platform.c
+++ b/drivers/ata/ahci_platform.c
@@ -16,6 +16,7 @@
#include <linux/module.h>
#include <linux/pm.h>
#include <linux/device.h>
+#include <linux/of_device.h>
#include <linux/platform_device.h>
#include <linux/libata.h>
#include <linux/ahci_platform.h>
@@ -33,6 +34,7 @@
struct device *dev = &pdev->dev;
struct ahci_platform_data *pdata = dev_get_platdata(dev);
struct ahci_host_priv *hpriv;
+ unsigned long hflags = 0;
int rc;
hpriv = ahci_platform_get_resources(pdev);
@@ -55,7 +57,11 @@
goto disable_resources;
}
- rc = ahci_platform_init_host(pdev, hpriv, &ahci_port_info, 0, 0);
+ if (of_device_is_compatible(dev->of_node, "hisilicon,hisi-ahci"))
+ hflags |= AHCI_HFLAG_NO_FBS;
+
+ rc = ahci_platform_init_host(pdev, hpriv, &ahci_port_info,
+ hflags, 0, 0);
if (rc)
goto pdata_exit;
@@ -76,6 +82,7 @@
{ .compatible = "snps,exynos5440-ahci", },
{ .compatible = "ibm,476gtr-ahci", },
{ .compatible = "snps,dwc-ahci", },
+ { .compatible = "hisilicon,hisi-ahci", },
{},
};
MODULE_DEVICE_TABLE(of, ahci_of_match);
diff --git a/drivers/ata/ahci_st.c b/drivers/ata/ahci_st.c
index 6332222..2595598 100644
--- a/drivers/ata/ahci_st.c
+++ b/drivers/ata/ahci_st.c
@@ -166,7 +166,7 @@
if (err)
return err;
- err = ahci_platform_init_host(pdev, hpriv, &st_ahci_port_info, 0, 0);
+ err = ahci_platform_init_host(pdev, hpriv, &st_ahci_port_info, 0, 0, 0);
if (err) {
ahci_platform_disable_resources(hpriv);
return err;
diff --git a/drivers/ata/ahci_sunxi.c b/drivers/ata/ahci_sunxi.c
index 42d3f64..02002f1 100644
--- a/drivers/ata/ahci_sunxi.c
+++ b/drivers/ata/ahci_sunxi.c
@@ -157,8 +157,6 @@
}
static const struct ata_port_info ahci_sunxi_port_info = {
- AHCI_HFLAGS(AHCI_HFLAG_32BIT_ONLY | AHCI_HFLAG_NO_MSI |
- AHCI_HFLAG_NO_PMP | AHCI_HFLAG_YES_NCQ),
.flags = AHCI_FLAG_COMMON | ATA_FLAG_NCQ,
.pio_mask = ATA_PIO4,
.udma_mask = ATA_UDMA6,
@@ -169,6 +167,7 @@
{
struct device *dev = &pdev->dev;
struct ahci_host_priv *hpriv;
+ unsigned long hflags;
int rc;
hpriv = ahci_platform_get_resources(pdev);
@@ -185,7 +184,11 @@
if (rc)
goto disable_resources;
- rc = ahci_platform_init_host(pdev, hpriv, &ahci_sunxi_port_info, 0, 0);
+ hflags = AHCI_HFLAG_32BIT_ONLY | AHCI_HFLAG_NO_MSI |
+ AHCI_HFLAG_NO_PMP | AHCI_HFLAG_YES_NCQ;
+
+ rc = ahci_platform_init_host(pdev, hpriv, &ahci_sunxi_port_info,
+ hflags, 0, 0);
if (rc)
goto disable_resources;
diff --git a/drivers/ata/ahci_xgene.c b/drivers/ata/ahci_xgene.c
index 77c89bf..042a9bb 100644
--- a/drivers/ata/ahci_xgene.c
+++ b/drivers/ata/ahci_xgene.c
@@ -303,7 +303,6 @@
};
static const struct ata_port_info xgene_ahci_port_info = {
- AHCI_HFLAGS(AHCI_HFLAG_NO_PMP | AHCI_HFLAG_YES_NCQ),
.flags = AHCI_FLAG_COMMON | ATA_FLAG_NCQ,
.pio_mask = ATA_PIO4,
.udma_mask = ATA_UDMA6,
@@ -382,6 +381,7 @@
struct ahci_host_priv *hpriv;
struct xgene_ahci_context *ctx;
struct resource *res;
+ unsigned long hflags;
int rc;
hpriv = ahci_platform_get_resources(pdev);
@@ -450,7 +450,10 @@
goto disable_resources;
}
- rc = ahci_platform_init_host(pdev, hpriv, &xgene_ahci_port_info, 0, 0);
+ hflags = AHCI_HFLAG_NO_PMP | AHCI_HFLAG_YES_NCQ;
+
+ rc = ahci_platform_init_host(pdev, hpriv, &xgene_ahci_port_info,
+ hflags, 0, 0);
if (rc)
goto disable_resources;
diff --git a/drivers/ata/ata_generic.c b/drivers/ata/ata_generic.c
index 9498a7d..9ff545c 100644
--- a/drivers/ata/ata_generic.c
+++ b/drivers/ata/ata_generic.c
@@ -241,7 +241,7 @@
.id_table = ata_generic,
.probe = ata_generic_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/ata_piix.c b/drivers/ata/ata_piix.c
index 6334c8d..893e30e 100644
--- a/drivers/ata/ata_piix.c
+++ b/drivers/ata/ata_piix.c
@@ -830,7 +830,7 @@
return ap->ops->bmdma_status(ap) & ATA_DMA_INTR;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int piix_broken_suspend(void)
{
static const struct dmi_system_id sysids[] = {
@@ -1767,7 +1767,7 @@
.id_table = piix_pci_tbl,
.probe = piix_init_one,
.remove = piix_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = piix_pci_device_suspend,
.resume = piix_pci_device_resume,
#endif
diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c
index b986145..40ea583 100644
--- a/drivers/ata/libahci.c
+++ b/drivers/ata/libahci.c
@@ -464,6 +464,11 @@
cap |= HOST_CAP_FBS;
}
+ if ((cap & HOST_CAP_FBS) && (hpriv->flags & AHCI_HFLAG_NO_FBS)) {
+ dev_info(dev, "controller can't do FBS, turning off CAP_FBS\n");
+ cap &= ~HOST_CAP_FBS;
+ }
+
if (force_port_map && port_map != force_port_map) {
dev_info(dev, "forcing port_map 0x%x -> 0x%x\n",
port_map, force_port_map);
diff --git a/drivers/ata/libahci_platform.c b/drivers/ata/libahci_platform.c
index 7cb3a85..3a5b4ed 100644
--- a/drivers/ata/libahci_platform.c
+++ b/drivers/ata/libahci_platform.c
@@ -283,6 +283,7 @@
* @pdev: platform device pointer for the host
* @hpriv: ahci-host private data for the host
* @pi_template: template for the ata_port_info to use
+ * @host_flags: ahci host flags used in ahci_host_priv
* @force_port_map: param passed to ahci_save_initial_config
* @mask_port_map: param passed to ahci_save_initial_config
*
@@ -296,6 +297,7 @@
int ahci_platform_init_host(struct platform_device *pdev,
struct ahci_host_priv *hpriv,
const struct ata_port_info *pi_template,
+ unsigned long host_flags,
unsigned int force_port_map,
unsigned int mask_port_map)
{
@@ -312,7 +314,8 @@
}
/* prepare host */
- hpriv->flags |= (unsigned long)pi.private_data;
+ pi.private_data = (void *)host_flags;
+ hpriv->flags |= host_flags;
ahci_save_initial_config(dev, hpriv, force_port_map, mask_port_map);
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index ea83828..18d97d5 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -4224,10 +4224,10 @@
{ "PIONEER DVD-RW DVR-216D", NULL, ATA_HORKAGE_NOSETXFER },
/* devices that don't properly handle queued TRIM commands */
- { "Micron_M500*", "MU0[1-4]*", ATA_HORKAGE_NO_NCQ_TRIM, },
- { "Crucial_CT???M500SSD*", "MU0[1-4]*", ATA_HORKAGE_NO_NCQ_TRIM, },
- { "Micron_M550*", NULL, ATA_HORKAGE_NO_NCQ_TRIM, },
- { "Crucial_CT???M550SSD*", NULL, ATA_HORKAGE_NO_NCQ_TRIM, },
+ { "Micron_M500*", NULL, ATA_HORKAGE_NO_NCQ_TRIM, },
+ { "Crucial_CT???M500SSD*", NULL, ATA_HORKAGE_NO_NCQ_TRIM, },
+ { "Micron_M550*", NULL, ATA_HORKAGE_NO_NCQ_TRIM, },
+ { "Crucial_CT???M550SSD*", NULL, ATA_HORKAGE_NO_NCQ_TRIM, },
/*
* Some WD SATA-I drives spin up and down erratically when the link
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index ef8567d..72691fd 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1993,7 +1993,11 @@
memcpy(rbuf, hdr, sizeof(hdr));
memcpy(&rbuf[8], "ATA ", 8);
ata_id_string(args->id, &rbuf[16], ATA_ID_PROD, 16);
- ata_id_string(args->id, &rbuf[32], ATA_ID_FW_REV, 4);
+
+ /* From SAT, use last 2 words from fw rev unless they are spaces */
+ ata_id_string(args->id, &rbuf[32], ATA_ID_FW_REV + 2, 4);
+ if (strncmp(&rbuf[32], " ", 4) == 0)
+ ata_id_string(args->id, &rbuf[32], ATA_ID_FW_REV, 4);
if (rbuf[32] == 0 || rbuf[32] == ' ')
memcpy(&rbuf[32], "n/a ", 4);
diff --git a/drivers/ata/libata-sff.c b/drivers/ata/libata-sff.c
index b603720..1121153 100644
--- a/drivers/ata/libata-sff.c
+++ b/drivers/ata/libata-sff.c
@@ -2433,15 +2433,6 @@
mask = (1 << 2) | (1 << 0);
if ((tmp8 & mask) != mask)
legacy_mode = 1;
-#if defined(CONFIG_NO_ATA_LEGACY)
- /* Some platforms with PCI limits cannot address compat
- port space. In that case we punt if their firmware has
- left a device in compatibility mode */
- if (legacy_mode) {
- printk(KERN_ERR "ata: Compatibility mode ATA is not supported on this platform, skipping.\n");
- return -EOPNOTSUPP;
- }
-#endif
}
if (!devres_open_group(dev, NULL, GFP_KERNEL))
diff --git a/drivers/ata/pata_acpi.c b/drivers/ata/pata_acpi.c
index 5108b87..b70fce2 100644
--- a/drivers/ata/pata_acpi.c
+++ b/drivers/ata/pata_acpi.c
@@ -265,7 +265,7 @@
.id_table = pacpi_pci_tbl,
.probe = pacpi_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_ali.c b/drivers/ata/pata_ali.c
index 1b7b2cc..d19cd88 100644
--- a/drivers/ata/pata_ali.c
+++ b/drivers/ata/pata_ali.c
@@ -589,7 +589,7 @@
return ata_pci_bmdma_init_one(pdev, ppi, &ali_sht, NULL, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int ali_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -616,7 +616,7 @@
.id_table = ali,
.probe = ali_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ali_reinit_one,
#endif
diff --git a/drivers/ata/pata_amd.c b/drivers/ata/pata_amd.c
index 1206fa6..8d4d959 100644
--- a/drivers/ata/pata_amd.c
+++ b/drivers/ata/pata_amd.c
@@ -574,7 +574,7 @@
return ata_pci_bmdma_init_one(pdev, ppi, &amd_sht, hpriv, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int amd_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -625,7 +625,7 @@
.id_table = amd,
.probe = amd_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = amd_reinit_one,
#endif
diff --git a/drivers/ata/pata_artop.c b/drivers/ata/pata_artop.c
index 3aa4e65..96c05c9 100644
--- a/drivers/ata/pata_artop.c
+++ b/drivers/ata/pata_artop.c
@@ -422,7 +422,7 @@
{ } /* terminate list */
};
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int atp8xx_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -444,7 +444,7 @@
.id_table = artop_pci_tbl,
.probe = artop_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = atp8xx_reinit_one,
#endif
diff --git a/drivers/ata/pata_atiixp.c b/drivers/ata/pata_atiixp.c
index 30fa4ca..970f776 100644
--- a/drivers/ata/pata_atiixp.c
+++ b/drivers/ata/pata_atiixp.c
@@ -298,7 +298,7 @@
.id_table = atiixp,
.probe = atiixp_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.resume = ata_pci_device_resume,
.suspend = ata_pci_device_suspend,
#endif
diff --git a/drivers/ata/pata_atp867x.c b/drivers/ata/pata_atp867x.c
index 7e73a0f..a705cfc 100644
--- a/drivers/ata/pata_atp867x.c
+++ b/drivers/ata/pata_atp867x.c
@@ -530,7 +530,7 @@
return rc;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int atp867x_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -558,7 +558,7 @@
.id_table = atp867x_pci_tbl,
.probe = atp867x_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = atp867x_reinit_one,
#endif
diff --git a/drivers/ata/pata_bf54x.c b/drivers/ata/pata_bf54x.c
index ba0d8a2..03f2f2bc 100644
--- a/drivers/ata/pata_bf54x.c
+++ b/drivers/ata/pata_bf54x.c
@@ -1619,7 +1619,7 @@
return 0;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int bfin_atapi_suspend(struct platform_device *pdev, pm_message_t state)
{
struct ata_host *host = platform_get_drvdata(pdev);
diff --git a/drivers/ata/pata_cmd640.c b/drivers/ata/pata_cmd640.c
index 57f1be6..c47caa8 100644
--- a/drivers/ata/pata_cmd640.c
+++ b/drivers/ata/pata_cmd640.c
@@ -231,7 +231,7 @@
return ata_pci_sff_init_one(pdev, ppi, &cmd640_sht, NULL, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int cmd640_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -256,7 +256,7 @@
.id_table = cmd640,
.probe = cmd640_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = cmd640_reinit_one,
#endif
diff --git a/drivers/ata/pata_cmd64x.c b/drivers/ata/pata_cmd64x.c
index 6bca350..13ca588 100644
--- a/drivers/ata/pata_cmd64x.c
+++ b/drivers/ata/pata_cmd64x.c
@@ -487,7 +487,7 @@
return ata_pci_bmdma_init_one(pdev, ppi, &cmd64x_sht, NULL, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int cmd64x_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -518,7 +518,7 @@
.id_table = cmd64x,
.probe = cmd64x_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = cmd64x_reinit_one,
#endif
diff --git a/drivers/ata/pata_cs5520.c b/drivers/ata/pata_cs5520.c
index bcde4b7..d65cb9d 100644
--- a/drivers/ata/pata_cs5520.c
+++ b/drivers/ata/pata_cs5520.c
@@ -229,7 +229,7 @@
return ata_host_register(host, &cs5520_sht);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
/**
* cs5520_reinit_one - device resume
* @pdev: PCI device
@@ -278,7 +278,7 @@
pci_save_state(pdev);
return 0;
}
-#endif /* CONFIG_PM */
+#endif /* CONFIG_PM_SLEEP */
/* For now keep DMA off. We can set it for all but A rev CS5510 once the
core ATA code can handle it */
@@ -295,7 +295,7 @@
.id_table = pata_cs5520,
.probe = cs5520_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = cs5520_pci_device_suspend,
.resume = cs5520_reinit_one,
#endif
diff --git a/drivers/ata/pata_cs5530.c b/drivers/ata/pata_cs5530.c
index 8afe854..48ae4b4 100644
--- a/drivers/ata/pata_cs5530.c
+++ b/drivers/ata/pata_cs5530.c
@@ -326,7 +326,7 @@
return ata_pci_bmdma_init_one(pdev, ppi, &cs5530_sht, NULL, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int cs5530_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -343,7 +343,7 @@
ata_host_resume(host);
return 0;
}
-#endif /* CONFIG_PM */
+#endif /* CONFIG_PM_SLEEP */
static const struct pci_device_id cs5530[] = {
{ PCI_VDEVICE(CYRIX, PCI_DEVICE_ID_CYRIX_5530_IDE), },
@@ -356,7 +356,7 @@
.id_table = cs5530,
.probe = cs5530_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = cs5530_reinit_one,
#endif
diff --git a/drivers/ata/pata_cs5535.c b/drivers/ata/pata_cs5535.c
index 2c0986f..97584e8 100644
--- a/drivers/ata/pata_cs5535.c
+++ b/drivers/ata/pata_cs5535.c
@@ -200,7 +200,7 @@
.id_table = cs5535,
.probe = cs5535_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_cs5536.c b/drivers/ata/pata_cs5536.c
index 32ddcae..6c15a55 100644
--- a/drivers/ata/pata_cs5536.c
+++ b/drivers/ata/pata_cs5536.c
@@ -297,7 +297,7 @@
.id_table = cs5536,
.probe = cs5536_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_cypress.c b/drivers/ata/pata_cypress.c
index 3435bd6..7930184 100644
--- a/drivers/ata/pata_cypress.c
+++ b/drivers/ata/pata_cypress.c
@@ -151,7 +151,7 @@
.id_table = cy82c693,
.probe = cy82c693_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_efar.c b/drivers/ata/pata_efar.c
index f440892..4a57a6f 100644
--- a/drivers/ata/pata_efar.c
+++ b/drivers/ata/pata_efar.c
@@ -288,7 +288,7 @@
.id_table = efar_pci_tbl,
.probe = efar_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_ep93xx.c b/drivers/ata/pata_ep93xx.c
index cad9d45..6ad5c07 100644
--- a/drivers/ata/pata_ep93xx.c
+++ b/drivers/ata/pata_ep93xx.c
@@ -708,8 +708,8 @@
struct dma_chan *channel = qc->dma_dir == DMA_TO_DEVICE
? drv_data->dma_tx_channel : drv_data->dma_rx_channel;
- txd = channel->device->device_prep_slave_sg(channel, qc->sg,
- qc->n_elem, qc->dma_dir, DMA_CTRL_ACK, NULL);
+ txd = dmaengine_prep_slave_sg(channel, qc->sg, qc->n_elem, qc->dma_dir,
+ DMA_CTRL_ACK);
if (!txd) {
dev_err(qc->ap->dev, "failed to prepare slave for sg dma\n");
return;
diff --git a/drivers/ata/pata_hpt366.c b/drivers/ata/pata_hpt366.c
index 8e76f79..cbc3de7 100644
--- a/drivers/ata/pata_hpt366.c
+++ b/drivers/ata/pata_hpt366.c
@@ -386,7 +386,7 @@
return ata_pci_bmdma_init_one(dev, ppi, &hpt36x_sht, hpriv, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int hpt36x_reinit_one(struct pci_dev *dev)
{
struct ata_host *host = pci_get_drvdata(dev);
@@ -411,7 +411,7 @@
.id_table = hpt36x,
.probe = hpt36x_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = hpt36x_reinit_one,
#endif
diff --git a/drivers/ata/pata_hpt3x3.c b/drivers/ata/pata_hpt3x3.c
index 255c5aa..d019cdd 100644
--- a/drivers/ata/pata_hpt3x3.c
+++ b/drivers/ata/pata_hpt3x3.c
@@ -249,7 +249,7 @@
IRQF_SHARED, &hpt3x3_sht);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int hpt3x3_reinit_one(struct pci_dev *dev)
{
struct ata_host *host = pci_get_drvdata(dev);
@@ -277,7 +277,7 @@
.id_table = hpt3x3,
.probe = hpt3x3_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = hpt3x3_reinit_one,
#endif
diff --git a/drivers/ata/pata_imx.c b/drivers/ata/pata_imx.c
index e0872db..af42457 100644
--- a/drivers/ata/pata_imx.c
+++ b/drivers/ata/pata_imx.c
@@ -185,7 +185,7 @@
return 0;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int pata_imx_suspend(struct device *dev)
{
struct ata_host *host = dev_get_drvdata(dev);
@@ -244,7 +244,7 @@
.name = DRV_NAME,
.of_match_table = imx_pata_dt_ids,
.owner = THIS_MODULE,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.pm = &pata_imx_pm_ops,
#endif
},
diff --git a/drivers/ata/pata_it8213.c b/drivers/ata/pata_it8213.c
index 81369d1..4f97d1e 100644
--- a/drivers/ata/pata_it8213.c
+++ b/drivers/ata/pata_it8213.c
@@ -283,7 +283,7 @@
.id_table = it8213_pci_tbl,
.probe = it8213_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_it821x.c b/drivers/ata/pata_it821x.c
index dc3d787..a5088ec 100644
--- a/drivers/ata/pata_it821x.c
+++ b/drivers/ata/pata_it821x.c
@@ -935,7 +935,7 @@
return ata_pci_bmdma_init_one(pdev, ppi, &it821x_sht, NULL, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int it821x_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -965,7 +965,7 @@
.id_table = it821x,
.probe = it821x_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = it821x_reinit_one,
#endif
diff --git a/drivers/ata/pata_jmicron.c b/drivers/ata/pata_jmicron.c
index b1cfa02..4d1a5d2 100644
--- a/drivers/ata/pata_jmicron.c
+++ b/drivers/ata/pata_jmicron.c
@@ -157,7 +157,7 @@
.id_table = jmicron_pci_tbl,
.probe = jmicron_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_macio.c b/drivers/ata/pata_macio.c
index c28d064..a02f76f 100644
--- a/drivers/ata/pata_macio.c
+++ b/drivers/ata/pata_macio.c
@@ -845,8 +845,7 @@
return 0;
}
-#ifdef CONFIG_PM
-
+#ifdef CONFIG_PM_SLEEP
static int pata_macio_do_suspend(struct pata_macio_priv *priv, pm_message_t mesg)
{
int rc;
@@ -907,8 +906,7 @@
return 0;
}
-
-#endif /* CONFIG_PM */
+#endif /* CONFIG_PM_SLEEP */
static struct scsi_host_template pata_macio_sht = {
ATA_BASE_SHT(DRV_NAME),
@@ -1208,8 +1206,7 @@
return 0;
}
-#ifdef CONFIG_PM
-
+#ifdef CONFIG_PM_SLEEP
static int pata_macio_suspend(struct macio_dev *mdev, pm_message_t mesg)
{
struct ata_host *host = macio_get_drvdata(mdev);
@@ -1223,8 +1220,7 @@
return pata_macio_do_resume(host->private_data);
}
-
-#endif /* CONFIG_PM */
+#endif /* CONFIG_PM_SLEEP */
#ifdef CONFIG_PMAC_MEDIABAY
static void pata_macio_mb_event(struct macio_dev* mdev, int mb_state)
@@ -1316,8 +1312,7 @@
ata_host_detach(host);
}
-#ifdef CONFIG_PM
-
+#ifdef CONFIG_PM_SLEEP
static int pata_macio_pci_suspend(struct pci_dev *pdev, pm_message_t mesg)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -1331,8 +1326,7 @@
return pata_macio_do_resume(host->private_data);
}
-
-#endif /* CONFIG_PM */
+#endif /* CONFIG_PM_SLEEP */
static struct of_device_id pata_macio_match[] =
{
@@ -1360,7 +1354,7 @@
},
.probe = pata_macio_attach,
.remove = pata_macio_detach,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = pata_macio_suspend,
.resume = pata_macio_resume,
#endif
@@ -1383,7 +1377,7 @@
.id_table = pata_macio_pci_match,
.probe = pata_macio_pci_attach,
.remove = pata_macio_pci_detach,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = pata_macio_pci_suspend,
.resume = pata_macio_pci_resume,
#endif
diff --git a/drivers/ata/pata_marvell.c b/drivers/ata/pata_marvell.c
index 6bad3df..ae9feb1 100644
--- a/drivers/ata/pata_marvell.c
+++ b/drivers/ata/pata_marvell.c
@@ -171,7 +171,7 @@
.id_table = marvell_pci_tbl,
.probe = marvell_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_mpc52xx.c b/drivers/ata/pata_mpc52xx.c
index 0024ced..ccd1c83 100644
--- a/drivers/ata/pata_mpc52xx.c
+++ b/drivers/ata/pata_mpc52xx.c
@@ -819,9 +819,7 @@
return 0;
}
-
-#ifdef CONFIG_PM
-
+#ifdef CONFIG_PM_SLEEP
static int
mpc52xx_ata_suspend(struct platform_device *op, pm_message_t state)
{
@@ -847,10 +845,8 @@
return 0;
}
-
#endif
-
static struct of_device_id mpc52xx_ata_of_match[] = {
{ .compatible = "fsl,mpc5200-ata", },
{ .compatible = "mpc5200-ata", },
@@ -861,7 +857,7 @@
static struct platform_driver mpc52xx_ata_of_platform_driver = {
.probe = mpc52xx_ata_probe,
.remove = mpc52xx_ata_remove,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = mpc52xx_ata_suspend,
.resume = mpc52xx_ata_resume,
#endif
diff --git a/drivers/ata/pata_mpiix.c b/drivers/ata/pata_mpiix.c
index f39a537..202b4d6 100644
--- a/drivers/ata/pata_mpiix.c
+++ b/drivers/ata/pata_mpiix.c
@@ -223,7 +223,7 @@
.id_table = mpiix,
.probe = mpiix_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_netcell.c b/drivers/ata/pata_netcell.c
index e3b9709..0ea1833 100644
--- a/drivers/ata/pata_netcell.c
+++ b/drivers/ata/pata_netcell.c
@@ -92,7 +92,7 @@
.id_table = netcell_pci_tbl,
.probe = netcell_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_ninja32.c b/drivers/ata/pata_ninja32.c
index 56201a6..efb272d 100644
--- a/drivers/ata/pata_ninja32.c
+++ b/drivers/ata/pata_ninja32.c
@@ -152,8 +152,7 @@
IRQF_SHARED, &ninja32_sht);
}
-#ifdef CONFIG_PM
-
+#ifdef CONFIG_PM_SLEEP
static int ninja32_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -183,7 +182,7 @@
.id_table = ninja32,
.probe = ninja32_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ninja32_reinit_one,
#endif
diff --git a/drivers/ata/pata_ns87410.c b/drivers/ata/pata_ns87410.c
index 6154c3e..200e1eb 100644
--- a/drivers/ata/pata_ns87410.c
+++ b/drivers/ata/pata_ns87410.c
@@ -161,7 +161,7 @@
.id_table = ns87410,
.probe = ns87410_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_ns87415.c b/drivers/ata/pata_ns87415.c
index d44df7c..84c6b22 100644
--- a/drivers/ata/pata_ns87415.c
+++ b/drivers/ata/pata_ns87415.c
@@ -385,7 +385,7 @@
{ } /* terminate list */
};
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int ns87415_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -407,7 +407,7 @@
.id_table = ns87415_pci_tbl,
.probe = ns87415_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ns87415_reinit_one,
#endif
diff --git a/drivers/ata/pata_octeon_cf.c b/drivers/ata/pata_octeon_cf.c
index 83c4ddb..2a97d3a 100644
--- a/drivers/ata/pata_octeon_cf.c
+++ b/drivers/ata/pata_octeon_cf.c
@@ -865,7 +865,7 @@
if (node == NULL)
return -EINVAL;
- cf_port = kzalloc(sizeof(*cf_port), GFP_KERNEL);
+ cf_port = devm_kzalloc(&pdev->dev, sizeof(*cf_port), GFP_KERNEL);
if (!cf_port)
return -ENOMEM;
@@ -881,10 +881,9 @@
n_size = of_n_size_cells(node);
reg_prop = of_find_property(node, "reg", ®_len);
- if (!reg_prop || reg_len < sizeof(__be32)) {
- rv = -EINVAL;
- goto free_cf_port;
- }
+ if (!reg_prop || reg_len < sizeof(__be32))
+ return -EINVAL;
+
cs_num = reg_prop->value;
cf_port->cs0 = be32_to_cpup(cs_num);
@@ -901,16 +900,13 @@
res_dma = platform_get_resource(dma_dev, IORESOURCE_MEM, 0);
if (!res_dma) {
of_node_put(dma_node);
- rv = -EINVAL;
- goto free_cf_port;
+ return -EINVAL;
}
cf_port->dma_base = (u64)devm_ioremap_nocache(&pdev->dev, res_dma->start,
resource_size(res_dma));
-
if (!cf_port->dma_base) {
of_node_put(dma_node);
- rv = -EINVAL;
- goto free_cf_port;
+ return -EINVAL;
}
irq_handler = octeon_cf_interrupt;
@@ -921,41 +917,34 @@
of_node_put(dma_node);
}
res_cs1 = platform_get_resource(pdev, IORESOURCE_MEM, 1);
- if (!res_cs1) {
- rv = -EINVAL;
- goto free_cf_port;
- }
+ if (!res_cs1)
+ return -EINVAL;
+
cs1 = devm_ioremap_nocache(&pdev->dev, res_cs1->start,
resource_size(res_cs1));
-
if (!cs1)
- goto free_cf_port;
+ return rv;
- if (reg_len < (n_addr + n_size + 1) * sizeof(__be32)) {
- rv = -EINVAL;
- goto free_cf_port;
- }
+ if (reg_len < (n_addr + n_size + 1) * sizeof(__be32))
+ return -EINVAL;
+
cs_num += n_addr + n_size;
cf_port->cs1 = be32_to_cpup(cs_num);
}
res_cs0 = platform_get_resource(pdev, IORESOURCE_MEM, 0);
-
- if (!res_cs0) {
- rv = -EINVAL;
- goto free_cf_port;
- }
+ if (!res_cs0)
+ return -EINVAL;
cs0 = devm_ioremap_nocache(&pdev->dev, res_cs0->start,
resource_size(res_cs0));
-
if (!cs0)
- goto free_cf_port;
+ return rv;
/* allocate host */
host = ata_host_alloc(&pdev->dev, 1);
if (!host)
- goto free_cf_port;
+ return rv;
ap = host->ports[0];
ap->private_data = cf_port;
@@ -1020,17 +1009,12 @@
ata_port_desc(ap, "cmd %p ctl %p", base, ap->ioaddr.ctl_addr);
-
dev_info(&pdev->dev, "version " DRV_VERSION" %d bit%s.\n",
is_16bit ? 16 : 8,
cf_port->is_true_ide ? ", True IDE" : "");
return ata_host_activate(host, irq, irq_handler,
IRQF_SHARED, &octeon_cf_sht);
-
-free_cf_port:
- kfree(cf_port);
- return rv;
}
static void octeon_cf_shutdown(struct device *dev)
diff --git a/drivers/ata/pata_oldpiix.c b/drivers/ata/pata_oldpiix.c
index 319b644..b9bf78b 100644
--- a/drivers/ata/pata_oldpiix.c
+++ b/drivers/ata/pata_oldpiix.c
@@ -258,7 +258,7 @@
.id_table = oldpiix_pci_tbl,
.probe = oldpiix_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_opti.c b/drivers/ata/pata_opti.c
index fb042e05..3a944a0 100644
--- a/drivers/ata/pata_opti.c
+++ b/drivers/ata/pata_opti.c
@@ -184,7 +184,7 @@
.id_table = opti,
.probe = opti_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_optidma.c b/drivers/ata/pata_optidma.c
index bb71ea2..bdec7ef 100644
--- a/drivers/ata/pata_optidma.c
+++ b/drivers/ata/pata_optidma.c
@@ -440,7 +440,7 @@
.id_table = optidma,
.probe = optidma_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_pdc2027x.c b/drivers/ata/pata_pdc2027x.c
index 1151f23..4d06a5c 100644
--- a/drivers/ata/pata_pdc2027x.c
+++ b/drivers/ata/pata_pdc2027x.c
@@ -62,7 +62,7 @@
};
static int pdc2027x_init_one(struct pci_dev *pdev, const struct pci_device_id *ent);
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int pdc2027x_reinit_one(struct pci_dev *pdev);
#endif
static int pdc2027x_prereset(struct ata_link *link, unsigned long deadline);
@@ -128,7 +128,7 @@
.id_table = pdc2027x_pci_tbl,
.probe = pdc2027x_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = pdc2027x_reinit_one,
#endif
@@ -761,7 +761,7 @@
IRQF_SHARED, &pdc2027x_sht);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int pdc2027x_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
diff --git a/drivers/ata/pata_pdc202xx_old.c b/drivers/ata/pata_pdc202xx_old.c
index defa050..9001991d 100644
--- a/drivers/ata/pata_pdc202xx_old.c
+++ b/drivers/ata/pata_pdc202xx_old.c
@@ -377,7 +377,7 @@
.id_table = pdc202xx,
.probe = pdc202xx_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_piccolo.c b/drivers/ata/pata_piccolo.c
index 0b46be1..35cb0e2 100644
--- a/drivers/ata/pata_piccolo.c
+++ b/drivers/ata/pata_piccolo.c
@@ -110,7 +110,7 @@
.id_table = ata_tosh,
.probe = ata_tosh_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_radisys.c b/drivers/ata/pata_radisys.c
index be3f102..a3f1123d 100644
--- a/drivers/ata/pata_radisys.c
+++ b/drivers/ata/pata_radisys.c
@@ -237,7 +237,7 @@
.id_table = radisys_pci_tbl,
.probe = radisys_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_rdc.c b/drivers/ata/pata_rdc.c
index 521b213..9ce5952 100644
--- a/drivers/ata/pata_rdc.c
+++ b/drivers/ata/pata_rdc.c
@@ -382,7 +382,7 @@
.id_table = rdc_pci_tbl,
.probe = rdc_init_one,
.remove = rdc_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_rz1000.c b/drivers/ata/pata_rz1000.c
index caedc90..b3ec18c 100644
--- a/drivers/ata/pata_rz1000.c
+++ b/drivers/ata/pata_rz1000.c
@@ -101,7 +101,7 @@
return -ENODEV;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int rz1000_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -133,7 +133,7 @@
.id_table = pata_rz1000,
.probe = rz1000_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = rz1000_reinit_one,
#endif
diff --git a/drivers/ata/pata_samsung_cf.c b/drivers/ata/pata_samsung_cf.c
index 0610e78..fb52883 100644
--- a/drivers/ata/pata_samsung_cf.c
+++ b/drivers/ata/pata_samsung_cf.c
@@ -619,7 +619,7 @@
return 0;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int pata_s3c_suspend(struct device *dev)
{
struct platform_device *pdev = to_platform_device(dev);
@@ -670,7 +670,7 @@
.driver = {
.name = DRV_NAME,
.owner = THIS_MODULE,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.pm = &pata_s3c_pm_ops,
#endif
},
diff --git a/drivers/ata/pata_sc1200.c b/drivers/ata/pata_sc1200.c
index 96a232f..c71de5d 100644
--- a/drivers/ata/pata_sc1200.c
+++ b/drivers/ata/pata_sc1200.c
@@ -254,7 +254,7 @@
.id_table = sc1200,
.probe = sc1200_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_scc.c b/drivers/ata/pata_scc.c
index f1f5b5a..4e006d7 100644
--- a/drivers/ata/pata_scc.c
+++ b/drivers/ata/pata_scc.c
@@ -1096,7 +1096,7 @@
.id_table = scc_pci_tbl,
.probe = scc_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_sch.c b/drivers/ata/pata_sch.c
index 5a1cde0..b920c34 100644
--- a/drivers/ata/pata_sch.c
+++ b/drivers/ata/pata_sch.c
@@ -64,7 +64,7 @@
.id_table = sch_pci_tbl,
.probe = sch_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_serverworks.c b/drivers/ata/pata_serverworks.c
index e27f31f..fc5f31d 100644
--- a/drivers/ata/pata_serverworks.c
+++ b/drivers/ata/pata_serverworks.c
@@ -436,7 +436,7 @@
return ata_pci_bmdma_init_one(pdev, ppi, &serverworks_sht, NULL, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int serverworks_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -468,7 +468,7 @@
.id_table = serverworks,
.probe = serverworks_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = serverworks_reinit_one,
#endif
diff --git a/drivers/ata/pata_sil680.c b/drivers/ata/pata_sil680.c
index 73fe362..f597edc 100644
--- a/drivers/ata/pata_sil680.c
+++ b/drivers/ata/pata_sil680.c
@@ -403,7 +403,7 @@
return ata_pci_bmdma_init_one(pdev, ppi, &sil680_sht, NULL, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int sil680_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -429,7 +429,7 @@
.id_table = sil680,
.probe = sil680_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = sil680_reinit_one,
#endif
diff --git a/drivers/ata/pata_sis.c b/drivers/ata/pata_sis.c
index 78d913aa..626f989 100644
--- a/drivers/ata/pata_sis.c
+++ b/drivers/ata/pata_sis.c
@@ -869,7 +869,7 @@
return ata_pci_bmdma_init_one(pdev, ppi, &sis_sht, chipset, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int sis_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -899,7 +899,7 @@
.id_table = sis_pci_tbl,
.probe = sis_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = sis_reinit_one,
#endif
diff --git a/drivers/ata/pata_sl82c105.c b/drivers/ata/pata_sl82c105.c
index 900f0e4..4935f61f 100644
--- a/drivers/ata/pata_sl82c105.c
+++ b/drivers/ata/pata_sl82c105.c
@@ -337,7 +337,7 @@
return ata_pci_bmdma_init_one(dev, ppi, &sl82c105_sht, NULL, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int sl82c105_reinit_one(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -365,7 +365,7 @@
.id_table = sl82c105,
.probe = sl82c105_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = sl82c105_reinit_one,
#endif
diff --git a/drivers/ata/pata_triflex.c b/drivers/ata/pata_triflex.c
index 7bc78e2..d9364af 100644
--- a/drivers/ata/pata_triflex.c
+++ b/drivers/ata/pata_triflex.c
@@ -207,7 +207,7 @@
{ },
};
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int triflex_ata_pci_device_suspend(struct pci_dev *pdev, pm_message_t mesg)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -233,7 +233,7 @@
.id_table = triflex,
.probe = triflex_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = triflex_ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/pata_via.c b/drivers/ata/pata_via.c
index f6c9632..1ca6bca 100644
--- a/drivers/ata/pata_via.c
+++ b/drivers/ata/pata_via.c
@@ -659,7 +659,7 @@
return ata_pci_bmdma_init_one(pdev, ppi, &via_sht, (void *)config, 0);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
/**
* via_reinit_one - reinit after resume
* @pdev; PCI device
@@ -704,7 +704,7 @@
.id_table = via,
.probe = via_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = via_reinit_one,
#endif
diff --git a/drivers/ata/sata_fsl.c b/drivers/ata/sata_fsl.c
index fb0b40a..616a6d2 100644
--- a/drivers/ata/sata_fsl.c
+++ b/drivers/ata/sata_fsl.c
@@ -774,20 +774,6 @@
VPRINTK("HControl = 0x%x\n", ioread32(hcr_base + HCONTROL));
VPRINTK("CHBA = 0x%x\n", ioread32(hcr_base + CHBA));
-#ifdef CONFIG_MPC8315_DS
- /*
- * Workaround for 8315DS board 3gbps link-up issue,
- * currently limit SATA port to GEN1 speed
- */
- sata_fsl_scr_read(&ap->link, SCR_CONTROL, &temp);
- temp &= ~(0xF << 4);
- temp |= (0x1 << 4);
- sata_fsl_scr_write(&ap->link, SCR_CONTROL, temp);
-
- sata_fsl_scr_read(&ap->link, SCR_CONTROL, &temp);
- dev_warn(dev, "scr_control, speed limited to %x\n", temp);
-#endif
-
return 0;
}
@@ -1588,7 +1574,7 @@
return 0;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int sata_fsl_suspend(struct platform_device *op, pm_message_t state)
{
struct ata_host *host = platform_get_drvdata(op);
@@ -1644,7 +1630,7 @@
},
.probe = sata_fsl_probe,
.remove = sata_fsl_remove,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = sata_fsl_suspend,
.resume = sata_fsl_resume,
#endif
diff --git a/drivers/ata/sata_inic162x.c b/drivers/ata/sata_inic162x.c
index 5c54d95..06982782 100644
--- a/drivers/ata/sata_inic162x.c
+++ b/drivers/ata/sata_inic162x.c
@@ -785,7 +785,7 @@
return 0;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int inic_pci_device_resume(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -898,7 +898,7 @@
static struct pci_driver inic_pci_driver = {
.name = DRV_NAME,
.id_table = inic_pci_tbl,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = inic_pci_device_resume,
#endif
diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
index 05c8a44..391cfda 100644
--- a/drivers/ata/sata_mv.c
+++ b/drivers/ata/sata_mv.c
@@ -4222,7 +4222,7 @@
return 0;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int mv_platform_suspend(struct platform_device *pdev, pm_message_t state)
{
struct ata_host *host = platform_get_drvdata(pdev);
@@ -4289,7 +4289,7 @@
#ifdef CONFIG_PCI
static int mv_pci_init_one(struct pci_dev *pdev,
const struct pci_device_id *ent);
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int mv_pci_device_resume(struct pci_dev *pdev);
#endif
@@ -4299,7 +4299,7 @@
.id_table = mv_pci_tbl,
.probe = mv_pci_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = mv_pci_device_resume,
#endif
@@ -4457,7 +4457,7 @@
IS_GEN_I(hpriv) ? &mv5_sht : &mv6_sht);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int mv_pci_device_resume(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
diff --git a/drivers/ata/sata_nv.c b/drivers/ata/sata_nv.c
index ba5f271..cdf99fa 100644
--- a/drivers/ata/sata_nv.c
+++ b/drivers/ata/sata_nv.c
@@ -295,7 +295,7 @@
#define NV_ADMA_CHECK_INTR(GCTL, PORT) ((GCTL) & (1 << (19 + (12 * (PORT)))))
static int nv_init_one(struct pci_dev *pdev, const struct pci_device_id *ent);
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int nv_pci_device_resume(struct pci_dev *pdev);
#endif
static void nv_ck804_host_stop(struct ata_host *host);
@@ -379,7 +379,7 @@
.name = DRV_NAME,
.id_table = nv_pci_tbl,
.probe = nv_init_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = nv_pci_device_resume,
#endif
@@ -2431,7 +2431,7 @@
return ata_pci_sff_activate_host(host, ipriv->irq_handler, ipriv->sht);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int nv_pci_device_resume(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
diff --git a/drivers/ata/sata_rcar.c b/drivers/ata/sata_rcar.c
index 2b25bd8..61eb6d7 100644
--- a/drivers/ata/sata_rcar.c
+++ b/drivers/ata/sata_rcar.c
@@ -937,7 +937,7 @@
return 0;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int sata_rcar_suspend(struct device *dev)
{
struct ata_host *host = dev_get_drvdata(dev);
@@ -991,7 +991,7 @@
.name = DRV_NAME,
.owner = THIS_MODULE,
.of_match_table = sata_rcar_match,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.pm = &sata_rcar_pm_ops,
#endif
},
diff --git a/drivers/ata/sata_sil.c b/drivers/ata/sata_sil.c
index 3062f86..40b76b2 100644
--- a/drivers/ata/sata_sil.c
+++ b/drivers/ata/sata_sil.c
@@ -112,7 +112,7 @@
};
static int sil_init_one(struct pci_dev *pdev, const struct pci_device_id *ent);
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int sil_pci_device_resume(struct pci_dev *pdev);
#endif
static void sil_dev_config(struct ata_device *dev);
@@ -166,7 +166,7 @@
.id_table = sil_pci_tbl,
.probe = sil_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = sil_pci_device_resume,
#endif
@@ -802,7 +802,7 @@
&sil_sht);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int sil_pci_device_resume(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
diff --git a/drivers/ata/sata_sil24.c b/drivers/ata/sata_sil24.c
index aa1051b..0534890 100644
--- a/drivers/ata/sata_sil24.c
+++ b/drivers/ata/sata_sil24.c
@@ -353,8 +353,10 @@
static void sil24_post_internal_cmd(struct ata_queued_cmd *qc);
static int sil24_port_start(struct ata_port *ap);
static int sil24_init_one(struct pci_dev *pdev, const struct pci_device_id *ent);
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int sil24_pci_device_resume(struct pci_dev *pdev);
+#endif
+#ifdef CONFIG_PM
static int sil24_port_resume(struct ata_port *ap);
#endif
@@ -375,7 +377,7 @@
.id_table = sil24_pci_tbl,
.probe = sil24_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = sil24_pci_device_resume,
#endif
@@ -1350,7 +1352,7 @@
&sil24_sht);
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int sil24_pci_device_resume(struct pci_dev *pdev)
{
struct ata_host *host = pci_get_drvdata(pdev);
@@ -1370,7 +1372,9 @@
return 0;
}
+#endif
+#ifdef CONFIG_PM
static int sil24_port_resume(struct ata_port *ap)
{
sil24_config_pmp(ap, ap->nr_pmp_links);
diff --git a/drivers/ata/sata_sis.c b/drivers/ata/sata_sis.c
index b513428..d1637ac 100644
--- a/drivers/ata/sata_sis.c
+++ b/drivers/ata/sata_sis.c
@@ -82,7 +82,7 @@
.id_table = sis_pci_tbl,
.probe = sis_init_one,
.remove = ata_pci_remove_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/ata/sata_via.c b/drivers/ata/sata_via.c
index f72e842..47bf894 100644
--- a/drivers/ata/sata_via.c
+++ b/drivers/ata/sata_via.c
@@ -103,7 +103,7 @@
.name = DRV_NAME,
.id_table = svia_pci_tbl,
.probe = svia_init_one,
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
.suspend = ata_pci_device_suspend,
.resume = ata_pci_device_resume,
#endif
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 56a027d..fb31b8e 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -243,14 +243,11 @@
struct nbd_request request;
unsigned long size = blk_rq_bytes(req);
+ memset(&request, 0, sizeof(request));
request.magic = htonl(NBD_REQUEST_MAGIC);
request.type = htonl(nbd_cmd(req));
- if (nbd_cmd(req) == NBD_CMD_FLUSH) {
- /* Other values are reserved for FLUSH requests. */
- request.from = 0;
- request.len = 0;
- } else {
+ if (nbd_cmd(req) != NBD_CMD_FLUSH && nbd_cmd(req) != NBD_CMD_DISC) {
request.from = cpu_to_be64((u64)blk_rq_pos(req) << 9);
request.len = htonl(size);
}
diff --git a/drivers/cdrom/cdrom.c b/drivers/cdrom/cdrom.c
index 0f40c95..898b84b 100644
--- a/drivers/cdrom/cdrom.c
+++ b/drivers/cdrom/cdrom.c
@@ -3470,7 +3470,7 @@
return 0;
}
-static int cdrom_sysctl_info(ctl_table *ctl, int write,
+static int cdrom_sysctl_info(struct ctl_table *ctl, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
int pos;
@@ -3583,7 +3583,7 @@
mutex_unlock(&cdrom_mutex);
}
-static int cdrom_sysctl_handler(ctl_table *ctl, int write,
+static int cdrom_sysctl_handler(struct ctl_table *ctl, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
int ret;
@@ -3609,7 +3609,7 @@
}
/* Place files in /proc/sys/dev/cdrom */
-static ctl_table cdrom_table[] = {
+static struct ctl_table cdrom_table[] = {
{
.procname = "info",
.data = &cdrom_sysctl_settings.info,
@@ -3655,7 +3655,7 @@
{ }
};
-static ctl_table cdrom_cdrom_table[] = {
+static struct ctl_table cdrom_cdrom_table[] = {
{
.procname = "cdrom",
.maxlen = 0,
@@ -3666,7 +3666,7 @@
};
/* Make sure that /proc/sys/dev is there */
-static ctl_table cdrom_root_table[] = {
+static struct ctl_table cdrom_root_table[] = {
{
.procname = "dev",
.maxlen = 0,
diff --git a/drivers/char/hw_random/Kconfig b/drivers/char/hw_random/Kconfig
index 244759b..836b061 100644
--- a/drivers/char/hw_random/Kconfig
+++ b/drivers/char/hw_random/Kconfig
@@ -2,7 +2,7 @@
# Hardware Random Number Generator (RNG) configuration
#
-config HW_RANDOM
+menuconfig HW_RANDOM
tristate "Hardware Random Number Generator Core support"
default m
---help---
@@ -20,9 +20,11 @@
If unsure, say Y.
+if HW_RANDOM
+
config HW_RANDOM_TIMERIOMEM
tristate "Timer IOMEM HW Random Number Generator support"
- depends on HW_RANDOM && HAS_IOMEM
+ depends on HAS_IOMEM
---help---
This driver provides kernel-side support for a generic Random
Number Generator used by reading a 'dumb' iomem address that
@@ -36,7 +38,7 @@
config HW_RANDOM_INTEL
tristate "Intel HW Random Number Generator support"
- depends on HW_RANDOM && (X86 || IA64) && PCI
+ depends on (X86 || IA64) && PCI
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -49,7 +51,7 @@
config HW_RANDOM_AMD
tristate "AMD HW Random Number Generator support"
- depends on HW_RANDOM && (X86 || PPC_MAPLE) && PCI
+ depends on (X86 || PPC_MAPLE) && PCI
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -62,8 +64,8 @@
config HW_RANDOM_ATMEL
tristate "Atmel Random Number Generator support"
- depends on HW_RANDOM && HAVE_CLK
- default (HW_RANDOM && ARCH_AT91)
+ depends on ARCH_AT91 && HAVE_CLK
+ default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
Generator hardware found on Atmel AT91 devices.
@@ -75,7 +77,7 @@
config HW_RANDOM_BCM63XX
tristate "Broadcom BCM63xx Random Number Generator support"
- depends on HW_RANDOM && BCM63XX
+ depends on BCM63XX
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -88,7 +90,7 @@
config HW_RANDOM_BCM2835
tristate "Broadcom BCM2835 Random Number Generator support"
- depends on HW_RANDOM && ARCH_BCM2835
+ depends on ARCH_BCM2835
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -101,7 +103,7 @@
config HW_RANDOM_GEODE
tristate "AMD Geode HW Random Number Generator support"
- depends on HW_RANDOM && X86_32 && PCI
+ depends on X86_32 && PCI
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -114,7 +116,7 @@
config HW_RANDOM_N2RNG
tristate "Niagara2 Random Number Generator support"
- depends on HW_RANDOM && SPARC64
+ depends on SPARC64
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -127,7 +129,7 @@
config HW_RANDOM_VIA
tristate "VIA HW Random Number Generator support"
- depends on HW_RANDOM && X86
+ depends on X86
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -140,7 +142,7 @@
config HW_RANDOM_IXP4XX
tristate "Intel IXP4xx NPU HW Pseudo-Random Number Generator support"
- depends on HW_RANDOM && ARCH_IXP4XX
+ depends on ARCH_IXP4XX
default HW_RANDOM
---help---
This driver provides kernel-side support for the Pseudo-Random
@@ -153,7 +155,7 @@
config HW_RANDOM_OMAP
tristate "OMAP Random Number Generator support"
- depends on HW_RANDOM && (ARCH_OMAP16XX || ARCH_OMAP2PLUS)
+ depends on ARCH_OMAP16XX || ARCH_OMAP2PLUS
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -167,7 +169,7 @@
config HW_RANDOM_OMAP3_ROM
tristate "OMAP3 ROM Random Number Generator support"
- depends on HW_RANDOM && ARCH_OMAP3
+ depends on ARCH_OMAP3
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -180,7 +182,7 @@
config HW_RANDOM_OCTEON
tristate "Octeon Random Number Generator support"
- depends on HW_RANDOM && CAVIUM_OCTEON_SOC
+ depends on CAVIUM_OCTEON_SOC
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -193,7 +195,7 @@
config HW_RANDOM_PASEMI
tristate "PA Semi HW Random Number Generator support"
- depends on HW_RANDOM && PPC_PASEMI
+ depends on PPC_PASEMI
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -206,7 +208,7 @@
config HW_RANDOM_VIRTIO
tristate "VirtIO Random Number Generator support"
- depends on HW_RANDOM && VIRTIO
+ depends on VIRTIO
---help---
This driver provides kernel-side support for the virtual Random Number
Generator hardware.
@@ -216,7 +218,7 @@
config HW_RANDOM_TX4939
tristate "TX4939 Random Number Generator support"
- depends on HW_RANDOM && SOC_TX4939
+ depends on SOC_TX4939
default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
@@ -229,7 +231,8 @@
config HW_RANDOM_MXC_RNGA
tristate "Freescale i.MX RNGA Random Number Generator"
- depends on HW_RANDOM && ARCH_HAS_RNGA
+ depends on ARCH_HAS_RNGA
+ default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
Generator hardware found on Freescale i.MX processors.
@@ -241,7 +244,8 @@
config HW_RANDOM_NOMADIK
tristate "ST-Ericsson Nomadik Random Number Generator support"
- depends on HW_RANDOM && ARCH_NOMADIK
+ depends on ARCH_NOMADIK
+ default HW_RANDOM
---help---
This driver provides kernel-side support for the Random Number
Generator hardware found on ST-Ericsson SoCs (8815 and 8500).
@@ -251,21 +255,10 @@
If unsure, say Y.
-config HW_RANDOM_PICOXCELL
- tristate "Picochip picoXcell true random number generator support"
- depends on HW_RANDOM && ARCH_PICOXCELL && PICOXCELL_PC3X3
- ---help---
- This driver provides kernel-side support for the Random Number
- Generator hardware found on Picochip PC3x3 and later devices.
-
- To compile this driver as a module, choose M here: the
- module will be called picoxcell-rng.
-
- If unsure, say Y.
-
config HW_RANDOM_PPC4XX
tristate "PowerPC 4xx generic true random number generator support"
- depends on HW_RANDOM && PPC && 4xx
+ depends on PPC && 4xx
+ default HW_RANDOM
---help---
This driver provides the kernel-side support for the TRNG hardware
found in the security function of some PowerPC 4xx SoCs.
@@ -275,6 +268,73 @@
If unsure, say N.
+config HW_RANDOM_PSERIES
+ tristate "pSeries HW Random Number Generator support"
+ depends on PPC64 && IBMVIO
+ default HW_RANDOM
+ ---help---
+ This driver provides kernel-side support for the Random Number
+ Generator hardware found on POWER7+ machines and above
+
+ To compile this driver as a module, choose M here: the
+ module will be called pseries-rng.
+
+ If unsure, say Y.
+
+config HW_RANDOM_POWERNV
+ tristate "PowerNV Random Number Generator support"
+ depends on PPC_POWERNV
+ default HW_RANDOM
+ ---help---
+ This is the driver for Random Number Generator hardware found
+ in POWER7+ and above machines for PowerNV platform.
+
+ To compile this driver as a module, choose M here: the
+ module will be called powernv-rng.
+
+ If unsure, say Y.
+
+config HW_RANDOM_EXYNOS
+ tristate "EXYNOS HW random number generator support"
+ depends on ARCH_EXYNOS
+ default HW_RANDOM
+ ---help---
+ This driver provides kernel-side support for the Random Number
+ Generator hardware found on EXYNOS SOCs.
+
+ To compile this driver as a module, choose M here: the
+ module will be called exynos-rng.
+
+ If unsure, say Y.
+
+config HW_RANDOM_TPM
+ tristate "TPM HW Random Number Generator support"
+ depends on TCG_TPM
+ default HW_RANDOM
+ ---help---
+ This driver provides kernel-side support for the Random Number
+ Generator in the Trusted Platform Module
+
+ To compile this driver as a module, choose M here: the
+ module will be called tpm-rng.
+
+ If unsure, say Y.
+
+config HW_RANDOM_MSM
+ tristate "Qualcomm SoCs Random Number Generator support"
+ depends on HW_RANDOM && ARCH_QCOM
+ default HW_RANDOM
+ ---help---
+ This driver provides kernel-side support for the Random Number
+ Generator hardware found on Qualcomm SoCs.
+
+ To compile this driver as a module, choose M here. the
+ module will be called msm-rng.
+
+ If unsure, say Y.
+
+endif # HW_RANDOM
+
config UML_RANDOM
depends on UML
tristate "Hardware random number generator"
@@ -289,66 +349,3 @@
(check your distro, or download from
http://sourceforge.net/projects/gkernel/). rngd periodically reads
/dev/hwrng and injects the entropy into /dev/random.
-
-config HW_RANDOM_PSERIES
- tristate "pSeries HW Random Number Generator support"
- depends on HW_RANDOM && PPC64 && IBMVIO
- default HW_RANDOM
- ---help---
- This driver provides kernel-side support for the Random Number
- Generator hardware found on POWER7+ machines and above
-
- To compile this driver as a module, choose M here: the
- module will be called pseries-rng.
-
- If unsure, say Y.
-
-config HW_RANDOM_POWERNV
- tristate "PowerNV Random Number Generator support"
- depends on HW_RANDOM && PPC_POWERNV
- default HW_RANDOM
- ---help---
- This is the driver for Random Number Generator hardware found
- in POWER7+ and above machines for PowerNV platform.
-
- To compile this driver as a module, choose M here: the
- module will be called powernv-rng.
-
- If unsure, say Y.
-
-config HW_RANDOM_EXYNOS
- tristate "EXYNOS HW random number generator support"
- depends on HW_RANDOM && HAS_IOMEM && HAVE_CLK
- ---help---
- This driver provides kernel-side support for the Random Number
- Generator hardware found on EXYNOS SOCs.
-
- To compile this driver as a module, choose M here: the
- module will be called exynos-rng.
-
- If unsure, say Y.
-
-config HW_RANDOM_TPM
- tristate "TPM HW Random Number Generator support"
- depends on HW_RANDOM && TCG_TPM
- default HW_RANDOM
- ---help---
- This driver provides kernel-side support for the Random Number
- Generator in the Trusted Platform Module
-
- To compile this driver as a module, choose M here: the
- module will be called tpm-rng.
-
- If unsure, say Y.
-
-config HW_RANDOM_MSM
- tristate "Qualcomm SoCs Random Number Generator support"
- depends on HW_RANDOM && ARCH_QCOM
- ---help---
- This driver provides kernel-side support for the Random Number
- Generator hardware found on Qualcomm SoCs.
-
- To compile this driver as a module, choose M here. the
- module will be called msm-rng.
-
- If unsure, say Y.
diff --git a/drivers/char/hw_random/Makefile b/drivers/char/hw_random/Makefile
index 3ae7755..199ed283 100644
--- a/drivers/char/hw_random/Makefile
+++ b/drivers/char/hw_random/Makefile
@@ -22,7 +22,6 @@
obj-$(CONFIG_HW_RANDOM_MXC_RNGA) += mxc-rnga.o
obj-$(CONFIG_HW_RANDOM_OCTEON) += octeon-rng.o
obj-$(CONFIG_HW_RANDOM_NOMADIK) += nomadik-rng.o
-obj-$(CONFIG_HW_RANDOM_PICOXCELL) += picoxcell-rng.o
obj-$(CONFIG_HW_RANDOM_PPC4XX) += ppc4xx-rng.o
obj-$(CONFIG_HW_RANDOM_PSERIES) += pseries-rng.o
obj-$(CONFIG_HW_RANDOM_POWERNV) += powernv-rng.o
diff --git a/drivers/char/hw_random/n2-drv.c b/drivers/char/hw_random/n2-drv.c
index 432232e..292a588 100644
--- a/drivers/char/hw_random/n2-drv.c
+++ b/drivers/char/hw_random/n2-drv.c
@@ -632,7 +632,7 @@
multi_capable = (match->data != NULL);
n2rng_driver_version();
- np = kzalloc(sizeof(*np), GFP_KERNEL);
+ np = devm_kzalloc(&op->dev, sizeof(*np), GFP_KERNEL);
if (!np)
goto out;
np->op = op;
@@ -653,7 +653,7 @@
&np->hvapi_minor)) {
dev_err(&op->dev, "Cannot register suitable "
"HVAPI version.\n");
- goto out_free;
+ goto out;
}
}
@@ -676,15 +676,16 @@
dev_info(&op->dev, "Registered RNG HVAPI major %lu minor %lu\n",
np->hvapi_major, np->hvapi_minor);
- np->units = kzalloc(sizeof(struct n2rng_unit) * np->num_units,
- GFP_KERNEL);
+ np->units = devm_kzalloc(&op->dev,
+ sizeof(struct n2rng_unit) * np->num_units,
+ GFP_KERNEL);
err = -ENOMEM;
if (!np->units)
goto out_hvapi_unregister;
err = n2rng_init_control(np);
if (err)
- goto out_free_units;
+ goto out_hvapi_unregister;
dev_info(&op->dev, "Found %s RNG, units: %d\n",
((np->flags & N2RNG_FLAG_MULTI) ?
@@ -697,7 +698,7 @@
err = hwrng_register(&np->hwrng);
if (err)
- goto out_free_units;
+ goto out_hvapi_unregister;
platform_set_drvdata(op, np);
@@ -705,15 +706,9 @@
return 0;
-out_free_units:
- kfree(np->units);
- np->units = NULL;
-
out_hvapi_unregister:
sun4v_hvapi_unregister(HV_GRP_RNG);
-out_free:
- kfree(np);
out:
return err;
}
@@ -730,11 +725,6 @@
sun4v_hvapi_unregister(HV_GRP_RNG);
- kfree(np->units);
- np->units = NULL;
-
- kfree(np);
-
return 0;
}
diff --git a/drivers/char/hw_random/omap-rng.c b/drivers/char/hw_random/omap-rng.c
index 9b89ff48..f66ea25 100644
--- a/drivers/char/hw_random/omap-rng.c
+++ b/drivers/char/hw_random/omap-rng.c
@@ -369,10 +369,8 @@
int ret;
priv = devm_kzalloc(dev, sizeof(struct omap_rng_dev), GFP_KERNEL);
- if (!priv) {
- dev_err(&pdev->dev, "could not allocate memory\n");
+ if (!priv)
return -ENOMEM;
- };
omap_rng_ops.priv = (unsigned long)priv;
platform_set_drvdata(pdev, priv);
diff --git a/drivers/char/hw_random/picoxcell-rng.c b/drivers/char/hw_random/picoxcell-rng.c
deleted file mode 100644
index eab5448..0000000
--- a/drivers/char/hw_random/picoxcell-rng.c
+++ /dev/null
@@ -1,181 +0,0 @@
-/*
- * Copyright (c) 2010-2011 Picochip Ltd., Jamie Iles
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * All enquiries to support@picochip.com
- */
-#include <linux/clk.h>
-#include <linux/delay.h>
-#include <linux/err.h>
-#include <linux/hw_random.h>
-#include <linux/io.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/platform_device.h>
-
-#define DATA_REG_OFFSET 0x0200
-#define CSR_REG_OFFSET 0x0278
-#define CSR_OUT_EMPTY_MASK (1 << 24)
-#define CSR_FAULT_MASK (1 << 1)
-#define TRNG_BLOCK_RESET_MASK (1 << 0)
-#define TAI_REG_OFFSET 0x0380
-
-/*
- * The maximum amount of time in microseconds to spend waiting for data if the
- * core wants us to wait. The TRNG should generate 32 bits every 320ns so a
- * timeout of 20us seems reasonable. The TRNG does builtin tests of the data
- * for randomness so we can't always assume there is data present.
- */
-#define PICO_TRNG_TIMEOUT 20
-
-static void __iomem *rng_base;
-static struct clk *rng_clk;
-static struct device *rng_dev;
-
-static inline u32 picoxcell_trng_read_csr(void)
-{
- return __raw_readl(rng_base + CSR_REG_OFFSET);
-}
-
-static inline bool picoxcell_trng_is_empty(void)
-{
- return picoxcell_trng_read_csr() & CSR_OUT_EMPTY_MASK;
-}
-
-/*
- * Take the random number generator out of reset and make sure the interrupts
- * are masked. We shouldn't need to get large amounts of random bytes so just
- * poll the status register. The hardware generates 32 bits every 320ns so we
- * shouldn't have to wait long enough to warrant waiting for an IRQ.
- */
-static void picoxcell_trng_start(void)
-{
- __raw_writel(0, rng_base + TAI_REG_OFFSET);
- __raw_writel(0, rng_base + CSR_REG_OFFSET);
-}
-
-static void picoxcell_trng_reset(void)
-{
- __raw_writel(TRNG_BLOCK_RESET_MASK, rng_base + CSR_REG_OFFSET);
- __raw_writel(TRNG_BLOCK_RESET_MASK, rng_base + TAI_REG_OFFSET);
- picoxcell_trng_start();
-}
-
-/*
- * Get some random data from the random number generator. The hw_random core
- * layer provides us with locking.
- */
-static int picoxcell_trng_read(struct hwrng *rng, void *buf, size_t max,
- bool wait)
-{
- int i;
-
- /* Wait for some data to become available. */
- for (i = 0; i < PICO_TRNG_TIMEOUT && picoxcell_trng_is_empty(); ++i) {
- if (!wait)
- return 0;
-
- udelay(1);
- }
-
- if (picoxcell_trng_read_csr() & CSR_FAULT_MASK) {
- dev_err(rng_dev, "fault detected, resetting TRNG\n");
- picoxcell_trng_reset();
- return -EIO;
- }
-
- if (i == PICO_TRNG_TIMEOUT)
- return 0;
-
- *(u32 *)buf = __raw_readl(rng_base + DATA_REG_OFFSET);
- return sizeof(u32);
-}
-
-static struct hwrng picoxcell_trng = {
- .name = "picoxcell",
- .read = picoxcell_trng_read,
-};
-
-static int picoxcell_trng_probe(struct platform_device *pdev)
-{
- int ret;
- struct resource *mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
-
- rng_base = devm_ioremap_resource(&pdev->dev, mem);
- if (IS_ERR(rng_base))
- return PTR_ERR(rng_base);
-
- rng_clk = devm_clk_get(&pdev->dev, NULL);
- if (IS_ERR(rng_clk)) {
- dev_warn(&pdev->dev, "no clk\n");
- return PTR_ERR(rng_clk);
- }
-
- ret = clk_enable(rng_clk);
- if (ret) {
- dev_warn(&pdev->dev, "unable to enable clk\n");
- return ret;
- }
-
- picoxcell_trng_start();
- ret = hwrng_register(&picoxcell_trng);
- if (ret)
- goto err_register;
-
- rng_dev = &pdev->dev;
- dev_info(&pdev->dev, "pixoxcell random number generator active\n");
-
- return 0;
-
-err_register:
- clk_disable(rng_clk);
- return ret;
-}
-
-static int picoxcell_trng_remove(struct platform_device *pdev)
-{
- hwrng_unregister(&picoxcell_trng);
- clk_disable(rng_clk);
-
- return 0;
-}
-
-#ifdef CONFIG_PM
-static int picoxcell_trng_suspend(struct device *dev)
-{
- clk_disable(rng_clk);
-
- return 0;
-}
-
-static int picoxcell_trng_resume(struct device *dev)
-{
- return clk_enable(rng_clk);
-}
-
-static const struct dev_pm_ops picoxcell_trng_pm_ops = {
- .suspend = picoxcell_trng_suspend,
- .resume = picoxcell_trng_resume,
-};
-#endif /* CONFIG_PM */
-
-static struct platform_driver picoxcell_trng_driver = {
- .probe = picoxcell_trng_probe,
- .remove = picoxcell_trng_remove,
- .driver = {
- .name = "picoxcell-trng",
- .owner = THIS_MODULE,
-#ifdef CONFIG_PM
- .pm = &picoxcell_trng_pm_ops,
-#endif /* CONFIG_PM */
- },
-};
-
-module_platform_driver(picoxcell_trng_driver);
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Jamie Iles");
-MODULE_DESCRIPTION("Picochip picoXcell TRNG driver");
diff --git a/drivers/char/hw_random/timeriomem-rng.c b/drivers/char/hw_random/timeriomem-rng.c
index 439ff8b..b6ab9ac 100644
--- a/drivers/char/hw_random/timeriomem-rng.c
+++ b/drivers/char/hw_random/timeriomem-rng.c
@@ -120,10 +120,8 @@
/* Allocate memory for the device structure (and zero it) */
priv = devm_kzalloc(&pdev->dev,
sizeof(struct timeriomem_rng_private_data), GFP_KERNEL);
- if (!priv) {
- dev_err(&pdev->dev, "failed to allocate device structure.\n");
+ if (!priv)
return -ENOMEM;
- }
platform_set_drvdata(pdev, priv);
diff --git a/drivers/char/random.c b/drivers/char/random.c
index 06cea7f..4ad71ef 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1582,10 +1582,10 @@
/*
* Return entropy available scaled to integral bits
*/
-static int proc_do_entropy(ctl_table *table, int write,
+static int proc_do_entropy(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
- ctl_table fake_table;
+ struct ctl_table fake_table;
int entropy_count;
entropy_count = *(int *)table->data >> ENTROPY_SHIFT;
diff --git a/drivers/clk/Kconfig b/drivers/clk/Kconfig
index 3a21964..9f9c5ae 100644
--- a/drivers/clk/Kconfig
+++ b/drivers/clk/Kconfig
@@ -58,12 +58,12 @@
clock generators.
config COMMON_CLK_S2MPS11
- tristate "Clock driver for S2MPS11/S5M8767 MFD"
+ tristate "Clock driver for S2MPS1X/S5M8767 MFD"
depends on MFD_SEC_CORE
---help---
- This driver supports S2MPS11/S5M8767 crystal oscillator clock. These
- multi-function devices have 3 fixed-rate oscillators, clocked at
- 32KHz each.
+ This driver supports S2MPS11/S2MPS14/S5M8767 crystal oscillator
+ clock. These multi-function devices have two (S2MPS14) or three
+ (S2MPS11, S5M8767) fixed-rate oscillators, clocked at 32KHz each.
config CLK_TWL6040
tristate "External McPDM functional clock from twl6040"
diff --git a/drivers/clk/Makefile b/drivers/clk/Makefile
index 50b2a7e..567f102 100644
--- a/drivers/clk/Makefile
+++ b/drivers/clk/Makefile
@@ -13,6 +13,7 @@
# hardware specific clock types
# please keep this section sorted lexicographically by file/directory path name
obj-$(CONFIG_COMMON_CLK_AXI_CLKGEN) += clk-axi-clkgen.o
+obj-$(CONFIG_ARCH_AXXIA) += clk-axm5516.o
obj-$(CONFIG_ARCH_BCM2835) += clk-bcm2835.o
obj-$(CONFIG_ARCH_EFM32) += clk-efm32gg.o
obj-$(CONFIG_ARCH_HIGHBANK) += clk-highbank.o
@@ -32,8 +33,10 @@
obj-$(CONFIG_COMMON_CLK_XGENE) += clk-xgene.o
obj-$(CONFIG_COMMON_CLK_AT91) += at91/
obj-$(CONFIG_ARCH_BCM_MOBILE) += bcm/
+obj-$(CONFIG_ARCH_BERLIN) += berlin/
obj-$(CONFIG_ARCH_HI3xxx) += hisilicon/
obj-$(CONFIG_ARCH_HIP04) += hisilicon/
+obj-$(CONFIG_ARCH_HIX5HD2) += hisilicon/
obj-$(CONFIG_COMMON_CLK_KEYSTONE) += keystone/
ifeq ($(CONFIG_COMMON_CLK), y)
obj-$(CONFIG_ARCH_MMP) += mmp/
diff --git a/drivers/clk/bcm/Kconfig b/drivers/clk/bcm/Kconfig
index a7262fb..75506e5 100644
--- a/drivers/clk/bcm/Kconfig
+++ b/drivers/clk/bcm/Kconfig
@@ -6,4 +6,4 @@
help
Enable common clock framework support for Broadcom SoCs
using "Kona" style clock control units, including those
- in the BCM281xx family.
+ in the BCM281xx and BCM21664 families.
diff --git a/drivers/clk/bcm/Makefile b/drivers/clk/bcm/Makefile
index cf93359..6297d05 100644
--- a/drivers/clk/bcm/Makefile
+++ b/drivers/clk/bcm/Makefile
@@ -1,3 +1,4 @@
obj-$(CONFIG_CLK_BCM_KONA) += clk-kona.o
obj-$(CONFIG_CLK_BCM_KONA) += clk-kona-setup.o
obj-$(CONFIG_CLK_BCM_KONA) += clk-bcm281xx.o
+obj-$(CONFIG_CLK_BCM_KONA) += clk-bcm21664.o
diff --git a/drivers/clk/bcm/clk-bcm21664.c b/drivers/clk/bcm/clk-bcm21664.c
new file mode 100644
index 0000000..eeae4ca
--- /dev/null
+++ b/drivers/clk/bcm/clk-bcm21664.c
@@ -0,0 +1,290 @@
+/*
+ * Copyright (C) 2014 Broadcom Corporation
+ * Copyright 2014 Linaro Limited
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "clk-kona.h"
+#include "dt-bindings/clock/bcm21664.h"
+
+#define BCM21664_CCU_COMMON(_name, _capname) \
+ KONA_CCU_COMMON(BCM21664, _name, _capname)
+
+/* Root CCU */
+
+static struct peri_clk_data frac_1m_data = {
+ .gate = HW_SW_GATE(0x214, 16, 0, 1),
+ .clocks = CLOCKS("ref_crystal"),
+};
+
+static struct ccu_data root_ccu_data = {
+ BCM21664_CCU_COMMON(root, ROOT),
+ /* no policy control */
+ .kona_clks = {
+ [BCM21664_ROOT_CCU_FRAC_1M] =
+ KONA_CLK(root, frac_1m, peri),
+ [BCM21664_ROOT_CCU_CLOCK_COUNT] = LAST_KONA_CLK,
+ },
+};
+
+/* AON CCU */
+
+static struct peri_clk_data hub_timer_data = {
+ .gate = HW_SW_GATE(0x0414, 16, 0, 1),
+ .hyst = HYST(0x0414, 8, 9),
+ .clocks = CLOCKS("bbl_32k",
+ "frac_1m",
+ "dft_19_5m"),
+ .sel = SELECTOR(0x0a10, 0, 2),
+ .trig = TRIGGER(0x0a40, 4),
+};
+
+static struct ccu_data aon_ccu_data = {
+ BCM21664_CCU_COMMON(aon, AON),
+ .policy = {
+ .enable = CCU_LVM_EN(0x0034, 0),
+ .control = CCU_POLICY_CTL(0x000c, 0, 1, 2),
+ },
+ .kona_clks = {
+ [BCM21664_AON_CCU_HUB_TIMER] =
+ KONA_CLK(aon, hub_timer, peri),
+ [BCM21664_AON_CCU_CLOCK_COUNT] = LAST_KONA_CLK,
+ },
+};
+
+/* Master CCU */
+
+static struct peri_clk_data sdio1_data = {
+ .gate = HW_SW_GATE(0x0358, 18, 2, 3),
+ .clocks = CLOCKS("ref_crystal",
+ "var_52m",
+ "ref_52m",
+ "var_96m",
+ "ref_96m"),
+ .sel = SELECTOR(0x0a28, 0, 3),
+ .div = DIVIDER(0x0a28, 4, 14),
+ .trig = TRIGGER(0x0afc, 9),
+};
+
+static struct peri_clk_data sdio2_data = {
+ .gate = HW_SW_GATE(0x035c, 18, 2, 3),
+ .clocks = CLOCKS("ref_crystal",
+ "var_52m",
+ "ref_52m",
+ "var_96m",
+ "ref_96m"),
+ .sel = SELECTOR(0x0a2c, 0, 3),
+ .div = DIVIDER(0x0a2c, 4, 14),
+ .trig = TRIGGER(0x0afc, 10),
+};
+
+static struct peri_clk_data sdio3_data = {
+ .gate = HW_SW_GATE(0x0364, 18, 2, 3),
+ .clocks = CLOCKS("ref_crystal",
+ "var_52m",
+ "ref_52m",
+ "var_96m",
+ "ref_96m"),
+ .sel = SELECTOR(0x0a34, 0, 3),
+ .div = DIVIDER(0x0a34, 4, 14),
+ .trig = TRIGGER(0x0afc, 12),
+};
+
+static struct peri_clk_data sdio4_data = {
+ .gate = HW_SW_GATE(0x0360, 18, 2, 3),
+ .clocks = CLOCKS("ref_crystal",
+ "var_52m",
+ "ref_52m",
+ "var_96m",
+ "ref_96m"),
+ .sel = SELECTOR(0x0a30, 0, 3),
+ .div = DIVIDER(0x0a30, 4, 14),
+ .trig = TRIGGER(0x0afc, 11),
+};
+
+static struct peri_clk_data sdio1_sleep_data = {
+ .clocks = CLOCKS("ref_32k"), /* Verify */
+ .gate = HW_SW_GATE(0x0358, 18, 2, 3),
+};
+
+static struct peri_clk_data sdio2_sleep_data = {
+ .clocks = CLOCKS("ref_32k"), /* Verify */
+ .gate = HW_SW_GATE(0x035c, 18, 2, 3),
+};
+
+static struct peri_clk_data sdio3_sleep_data = {
+ .clocks = CLOCKS("ref_32k"), /* Verify */
+ .gate = HW_SW_GATE(0x0364, 18, 2, 3),
+};
+
+static struct peri_clk_data sdio4_sleep_data = {
+ .clocks = CLOCKS("ref_32k"), /* Verify */
+ .gate = HW_SW_GATE(0x0360, 18, 2, 3),
+};
+
+static struct ccu_data master_ccu_data = {
+ BCM21664_CCU_COMMON(master, MASTER),
+ .policy = {
+ .enable = CCU_LVM_EN(0x0034, 0),
+ .control = CCU_POLICY_CTL(0x000c, 0, 1, 2),
+ },
+ .kona_clks = {
+ [BCM21664_MASTER_CCU_SDIO1] =
+ KONA_CLK(master, sdio1, peri),
+ [BCM21664_MASTER_CCU_SDIO2] =
+ KONA_CLK(master, sdio2, peri),
+ [BCM21664_MASTER_CCU_SDIO3] =
+ KONA_CLK(master, sdio3, peri),
+ [BCM21664_MASTER_CCU_SDIO4] =
+ KONA_CLK(master, sdio4, peri),
+ [BCM21664_MASTER_CCU_SDIO1_SLEEP] =
+ KONA_CLK(master, sdio1_sleep, peri),
+ [BCM21664_MASTER_CCU_SDIO2_SLEEP] =
+ KONA_CLK(master, sdio2_sleep, peri),
+ [BCM21664_MASTER_CCU_SDIO3_SLEEP] =
+ KONA_CLK(master, sdio3_sleep, peri),
+ [BCM21664_MASTER_CCU_SDIO4_SLEEP] =
+ KONA_CLK(master, sdio4_sleep, peri),
+ [BCM21664_MASTER_CCU_CLOCK_COUNT] = LAST_KONA_CLK,
+ },
+};
+
+/* Slave CCU */
+
+static struct peri_clk_data uartb_data = {
+ .gate = HW_SW_GATE(0x0400, 18, 2, 3),
+ .clocks = CLOCKS("ref_crystal",
+ "var_156m",
+ "ref_156m"),
+ .sel = SELECTOR(0x0a10, 0, 2),
+ .div = FRAC_DIVIDER(0x0a10, 4, 12, 8),
+ .trig = TRIGGER(0x0afc, 2),
+};
+
+static struct peri_clk_data uartb2_data = {
+ .gate = HW_SW_GATE(0x0404, 18, 2, 3),
+ .clocks = CLOCKS("ref_crystal",
+ "var_156m",
+ "ref_156m"),
+ .sel = SELECTOR(0x0a14, 0, 2),
+ .div = FRAC_DIVIDER(0x0a14, 4, 12, 8),
+ .trig = TRIGGER(0x0afc, 3),
+};
+
+static struct peri_clk_data uartb3_data = {
+ .gate = HW_SW_GATE(0x0408, 18, 2, 3),
+ .clocks = CLOCKS("ref_crystal",
+ "var_156m",
+ "ref_156m"),
+ .sel = SELECTOR(0x0a18, 0, 2),
+ .div = FRAC_DIVIDER(0x0a18, 4, 12, 8),
+ .trig = TRIGGER(0x0afc, 4),
+};
+
+static struct peri_clk_data bsc1_data = {
+ .gate = HW_SW_GATE(0x0458, 18, 2, 3),
+ .clocks = CLOCKS("ref_crystal",
+ "var_104m",
+ "ref_104m",
+ "var_13m",
+ "ref_13m"),
+ .sel = SELECTOR(0x0a64, 0, 3),
+ .trig = TRIGGER(0x0afc, 23),
+};
+
+static struct peri_clk_data bsc2_data = {
+ .gate = HW_SW_GATE(0x045c, 18, 2, 3),
+ .clocks = CLOCKS("ref_crystal",
+ "var_104m",
+ "ref_104m",
+ "var_13m",
+ "ref_13m"),
+ .sel = SELECTOR(0x0a68, 0, 3),
+ .trig = TRIGGER(0x0afc, 24),
+};
+
+static struct peri_clk_data bsc3_data = {
+ .gate = HW_SW_GATE(0x0470, 18, 2, 3),
+ .clocks = CLOCKS("ref_crystal",
+ "var_104m",
+ "ref_104m",
+ "var_13m",
+ "ref_13m"),
+ .sel = SELECTOR(0x0a7c, 0, 3),
+ .trig = TRIGGER(0x0afc, 18),
+};
+
+static struct peri_clk_data bsc4_data = {
+ .gate = HW_SW_GATE(0x0474, 18, 2, 3),
+ .clocks = CLOCKS("ref_crystal",
+ "var_104m",
+ "ref_104m",
+ "var_13m",
+ "ref_13m"),
+ .sel = SELECTOR(0x0a80, 0, 3),
+ .trig = TRIGGER(0x0afc, 19),
+};
+
+static struct ccu_data slave_ccu_data = {
+ BCM21664_CCU_COMMON(slave, SLAVE),
+ .policy = {
+ .enable = CCU_LVM_EN(0x0034, 0),
+ .control = CCU_POLICY_CTL(0x000c, 0, 1, 2),
+ },
+ .kona_clks = {
+ [BCM21664_SLAVE_CCU_UARTB] =
+ KONA_CLK(slave, uartb, peri),
+ [BCM21664_SLAVE_CCU_UARTB2] =
+ KONA_CLK(slave, uartb2, peri),
+ [BCM21664_SLAVE_CCU_UARTB3] =
+ KONA_CLK(slave, uartb3, peri),
+ [BCM21664_SLAVE_CCU_BSC1] =
+ KONA_CLK(slave, bsc1, peri),
+ [BCM21664_SLAVE_CCU_BSC2] =
+ KONA_CLK(slave, bsc2, peri),
+ [BCM21664_SLAVE_CCU_BSC3] =
+ KONA_CLK(slave, bsc3, peri),
+ [BCM21664_SLAVE_CCU_BSC4] =
+ KONA_CLK(slave, bsc4, peri),
+ [BCM21664_SLAVE_CCU_CLOCK_COUNT] = LAST_KONA_CLK,
+ },
+};
+
+/* Device tree match table callback functions */
+
+static void __init kona_dt_root_ccu_setup(struct device_node *node)
+{
+ kona_dt_ccu_setup(&root_ccu_data, node);
+}
+
+static void __init kona_dt_aon_ccu_setup(struct device_node *node)
+{
+ kona_dt_ccu_setup(&aon_ccu_data, node);
+}
+
+static void __init kona_dt_master_ccu_setup(struct device_node *node)
+{
+ kona_dt_ccu_setup(&master_ccu_data, node);
+}
+
+static void __init kona_dt_slave_ccu_setup(struct device_node *node)
+{
+ kona_dt_ccu_setup(&slave_ccu_data, node);
+}
+
+CLK_OF_DECLARE(bcm21664_root_ccu, BCM21664_DT_ROOT_CCU_COMPAT,
+ kona_dt_root_ccu_setup);
+CLK_OF_DECLARE(bcm21664_aon_ccu, BCM21664_DT_AON_CCU_COMPAT,
+ kona_dt_aon_ccu_setup);
+CLK_OF_DECLARE(bcm21664_master_ccu, BCM21664_DT_MASTER_CCU_COMPAT,
+ kona_dt_master_ccu_setup);
+CLK_OF_DECLARE(bcm21664_slave_ccu, BCM21664_DT_SLAVE_CCU_COMPAT,
+ kona_dt_slave_ccu_setup);
diff --git a/drivers/clk/bcm/clk-bcm281xx.c b/drivers/clk/bcm/clk-bcm281xx.c
index 3c66de6..502a487 100644
--- a/drivers/clk/bcm/clk-bcm281xx.c
+++ b/drivers/clk/bcm/clk-bcm281xx.c
@@ -15,14 +15,10 @@
#include "clk-kona.h"
#include "dt-bindings/clock/bcm281xx.h"
-/* bcm11351 CCU device tree "compatible" strings */
-#define BCM11351_DT_ROOT_CCU_COMPAT "brcm,bcm11351-root-ccu"
-#define BCM11351_DT_AON_CCU_COMPAT "brcm,bcm11351-aon-ccu"
-#define BCM11351_DT_HUB_CCU_COMPAT "brcm,bcm11351-hub-ccu"
-#define BCM11351_DT_MASTER_CCU_COMPAT "brcm,bcm11351-master-ccu"
-#define BCM11351_DT_SLAVE_CCU_COMPAT "brcm,bcm11351-slave-ccu"
+#define BCM281XX_CCU_COMMON(_name, _ucase_name) \
+ KONA_CCU_COMMON(BCM281XX, _name, _ucase_name)
-/* Root CCU clocks */
+/* Root CCU */
static struct peri_clk_data frac_1m_data = {
.gate = HW_SW_GATE(0x214, 16, 0, 1),
@@ -31,7 +27,16 @@
.clocks = CLOCKS("ref_crystal"),
};
-/* AON CCU clocks */
+static struct ccu_data root_ccu_data = {
+ BCM281XX_CCU_COMMON(root, ROOT),
+ .kona_clks = {
+ [BCM281XX_ROOT_CCU_FRAC_1M] =
+ KONA_CLK(root, frac_1m, peri),
+ [BCM281XX_ROOT_CCU_CLOCK_COUNT] = LAST_KONA_CLK,
+ },
+};
+
+/* AON CCU */
static struct peri_clk_data hub_timer_data = {
.gate = HW_SW_GATE(0x0414, 16, 0, 1),
@@ -60,7 +65,20 @@
.trig = TRIGGER(0x0a40, 2),
};
-/* Hub CCU clocks */
+static struct ccu_data aon_ccu_data = {
+ BCM281XX_CCU_COMMON(aon, AON),
+ .kona_clks = {
+ [BCM281XX_AON_CCU_HUB_TIMER] =
+ KONA_CLK(aon, hub_timer, peri),
+ [BCM281XX_AON_CCU_PMU_BSC] =
+ KONA_CLK(aon, pmu_bsc, peri),
+ [BCM281XX_AON_CCU_PMU_BSC_VAR] =
+ KONA_CLK(aon, pmu_bsc_var, peri),
+ [BCM281XX_AON_CCU_CLOCK_COUNT] = LAST_KONA_CLK,
+ },
+};
+
+/* Hub CCU */
static struct peri_clk_data tmon_1m_data = {
.gate = HW_SW_GATE(0x04a4, 18, 2, 3),
@@ -70,7 +88,16 @@
.trig = TRIGGER(0x0e84, 1),
};
-/* Master CCU clocks */
+static struct ccu_data hub_ccu_data = {
+ BCM281XX_CCU_COMMON(hub, HUB),
+ .kona_clks = {
+ [BCM281XX_HUB_CCU_TMON_1M] =
+ KONA_CLK(hub, tmon_1m, peri),
+ [BCM281XX_HUB_CCU_CLOCK_COUNT] = LAST_KONA_CLK,
+ },
+};
+
+/* Master CCU */
static struct peri_clk_data sdio1_data = {
.gate = HW_SW_GATE(0x0358, 18, 2, 3),
@@ -153,7 +180,28 @@
.trig = TRIGGER(0x0afc, 5),
};
-/* Slave CCU clocks */
+static struct ccu_data master_ccu_data = {
+ BCM281XX_CCU_COMMON(master, MASTER),
+ .kona_clks = {
+ [BCM281XX_MASTER_CCU_SDIO1] =
+ KONA_CLK(master, sdio1, peri),
+ [BCM281XX_MASTER_CCU_SDIO2] =
+ KONA_CLK(master, sdio2, peri),
+ [BCM281XX_MASTER_CCU_SDIO3] =
+ KONA_CLK(master, sdio3, peri),
+ [BCM281XX_MASTER_CCU_SDIO4] =
+ KONA_CLK(master, sdio4, peri),
+ [BCM281XX_MASTER_CCU_USB_IC] =
+ KONA_CLK(master, usb_ic, peri),
+ [BCM281XX_MASTER_CCU_HSIC2_48M] =
+ KONA_CLK(master, hsic2_48m, peri),
+ [BCM281XX_MASTER_CCU_HSIC2_12M] =
+ KONA_CLK(master, hsic2_12m, peri),
+ [BCM281XX_MASTER_CCU_CLOCK_COUNT] = LAST_KONA_CLK,
+ },
+};
+
+/* Slave CCU */
static struct peri_clk_data uartb_data = {
.gate = HW_SW_GATE(0x0400, 18, 2, 3),
@@ -261,156 +309,67 @@
.trig = TRIGGER(0x0afc, 15),
};
-/*
- * CCU setup routines
- *
- * These are called from kona_dt_ccu_setup() to initialize the array
- * of clocks provided by the CCU. Once allocated, the entries in
- * the array are initialized by calling kona_clk_setup() with the
- * initialization data for each clock. They return 0 if successful
- * or an error code otherwise.
- */
-static int __init bcm281xx_root_ccu_clks_setup(struct ccu_data *ccu)
-{
- struct clk **clks;
- size_t count = BCM281XX_ROOT_CCU_CLOCK_COUNT;
-
- clks = kzalloc(count * sizeof(*clks), GFP_KERNEL);
- if (!clks) {
- pr_err("%s: failed to allocate root clocks\n", __func__);
- return -ENOMEM;
- }
- ccu->data.clks = clks;
- ccu->data.clk_num = count;
-
- PERI_CLK_SETUP(clks, ccu, BCM281XX_ROOT_CCU_FRAC_1M, frac_1m);
-
- return 0;
-}
-
-static int __init bcm281xx_aon_ccu_clks_setup(struct ccu_data *ccu)
-{
- struct clk **clks;
- size_t count = BCM281XX_AON_CCU_CLOCK_COUNT;
-
- clks = kzalloc(count * sizeof(*clks), GFP_KERNEL);
- if (!clks) {
- pr_err("%s: failed to allocate aon clocks\n", __func__);
- return -ENOMEM;
- }
- ccu->data.clks = clks;
- ccu->data.clk_num = count;
-
- PERI_CLK_SETUP(clks, ccu, BCM281XX_AON_CCU_HUB_TIMER, hub_timer);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_AON_CCU_PMU_BSC, pmu_bsc);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_AON_CCU_PMU_BSC_VAR, pmu_bsc_var);
-
- return 0;
-}
-
-static int __init bcm281xx_hub_ccu_clks_setup(struct ccu_data *ccu)
-{
- struct clk **clks;
- size_t count = BCM281XX_HUB_CCU_CLOCK_COUNT;
-
- clks = kzalloc(count * sizeof(*clks), GFP_KERNEL);
- if (!clks) {
- pr_err("%s: failed to allocate hub clocks\n", __func__);
- return -ENOMEM;
- }
- ccu->data.clks = clks;
- ccu->data.clk_num = count;
-
- PERI_CLK_SETUP(clks, ccu, BCM281XX_HUB_CCU_TMON_1M, tmon_1m);
-
- return 0;
-}
-
-static int __init bcm281xx_master_ccu_clks_setup(struct ccu_data *ccu)
-{
- struct clk **clks;
- size_t count = BCM281XX_MASTER_CCU_CLOCK_COUNT;
-
- clks = kzalloc(count * sizeof(*clks), GFP_KERNEL);
- if (!clks) {
- pr_err("%s: failed to allocate master clocks\n", __func__);
- return -ENOMEM;
- }
- ccu->data.clks = clks;
- ccu->data.clk_num = count;
-
- PERI_CLK_SETUP(clks, ccu, BCM281XX_MASTER_CCU_SDIO1, sdio1);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_MASTER_CCU_SDIO2, sdio2);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_MASTER_CCU_SDIO3, sdio3);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_MASTER_CCU_SDIO4, sdio4);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_MASTER_CCU_USB_IC, usb_ic);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_MASTER_CCU_HSIC2_48M, hsic2_48m);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_MASTER_CCU_HSIC2_12M, hsic2_12m);
-
- return 0;
-}
-
-static int __init bcm281xx_slave_ccu_clks_setup(struct ccu_data *ccu)
-{
- struct clk **clks;
- size_t count = BCM281XX_SLAVE_CCU_CLOCK_COUNT;
-
- clks = kzalloc(count * sizeof(*clks), GFP_KERNEL);
- if (!clks) {
- pr_err("%s: failed to allocate slave clocks\n", __func__);
- return -ENOMEM;
- }
- ccu->data.clks = clks;
- ccu->data.clk_num = count;
-
- PERI_CLK_SETUP(clks, ccu, BCM281XX_SLAVE_CCU_UARTB, uartb);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_SLAVE_CCU_UARTB2, uartb2);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_SLAVE_CCU_UARTB3, uartb3);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_SLAVE_CCU_UARTB4, uartb4);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_SLAVE_CCU_SSP0, ssp0);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_SLAVE_CCU_SSP2, ssp2);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_SLAVE_CCU_BSC1, bsc1);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_SLAVE_CCU_BSC2, bsc2);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_SLAVE_CCU_BSC3, bsc3);
- PERI_CLK_SETUP(clks, ccu, BCM281XX_SLAVE_CCU_PWM, pwm);
-
- return 0;
-}
+static struct ccu_data slave_ccu_data = {
+ BCM281XX_CCU_COMMON(slave, SLAVE),
+ .kona_clks = {
+ [BCM281XX_SLAVE_CCU_UARTB] =
+ KONA_CLK(slave, uartb, peri),
+ [BCM281XX_SLAVE_CCU_UARTB2] =
+ KONA_CLK(slave, uartb2, peri),
+ [BCM281XX_SLAVE_CCU_UARTB3] =
+ KONA_CLK(slave, uartb3, peri),
+ [BCM281XX_SLAVE_CCU_UARTB4] =
+ KONA_CLK(slave, uartb4, peri),
+ [BCM281XX_SLAVE_CCU_SSP0] =
+ KONA_CLK(slave, ssp0, peri),
+ [BCM281XX_SLAVE_CCU_SSP2] =
+ KONA_CLK(slave, ssp2, peri),
+ [BCM281XX_SLAVE_CCU_BSC1] =
+ KONA_CLK(slave, bsc1, peri),
+ [BCM281XX_SLAVE_CCU_BSC2] =
+ KONA_CLK(slave, bsc2, peri),
+ [BCM281XX_SLAVE_CCU_BSC3] =
+ KONA_CLK(slave, bsc3, peri),
+ [BCM281XX_SLAVE_CCU_PWM] =
+ KONA_CLK(slave, pwm, peri),
+ [BCM281XX_SLAVE_CCU_CLOCK_COUNT] = LAST_KONA_CLK,
+ },
+};
/* Device tree match table callback functions */
static void __init kona_dt_root_ccu_setup(struct device_node *node)
{
- kona_dt_ccu_setup(node, bcm281xx_root_ccu_clks_setup);
+ kona_dt_ccu_setup(&root_ccu_data, node);
}
static void __init kona_dt_aon_ccu_setup(struct device_node *node)
{
- kona_dt_ccu_setup(node, bcm281xx_aon_ccu_clks_setup);
+ kona_dt_ccu_setup(&aon_ccu_data, node);
}
static void __init kona_dt_hub_ccu_setup(struct device_node *node)
{
- kona_dt_ccu_setup(node, bcm281xx_hub_ccu_clks_setup);
+ kona_dt_ccu_setup(&hub_ccu_data, node);
}
static void __init kona_dt_master_ccu_setup(struct device_node *node)
{
- kona_dt_ccu_setup(node, bcm281xx_master_ccu_clks_setup);
+ kona_dt_ccu_setup(&master_ccu_data, node);
}
static void __init kona_dt_slave_ccu_setup(struct device_node *node)
{
- kona_dt_ccu_setup(node, bcm281xx_slave_ccu_clks_setup);
+ kona_dt_ccu_setup(&slave_ccu_data, node);
}
-CLK_OF_DECLARE(bcm11351_root_ccu, BCM11351_DT_ROOT_CCU_COMPAT,
+CLK_OF_DECLARE(bcm281xx_root_ccu, BCM281XX_DT_ROOT_CCU_COMPAT,
kona_dt_root_ccu_setup);
-CLK_OF_DECLARE(bcm11351_aon_ccu, BCM11351_DT_AON_CCU_COMPAT,
+CLK_OF_DECLARE(bcm281xx_aon_ccu, BCM281XX_DT_AON_CCU_COMPAT,
kona_dt_aon_ccu_setup);
-CLK_OF_DECLARE(bcm11351_hub_ccu, BCM11351_DT_HUB_CCU_COMPAT,
+CLK_OF_DECLARE(bcm281xx_hub_ccu, BCM281XX_DT_HUB_CCU_COMPAT,
kona_dt_hub_ccu_setup);
-CLK_OF_DECLARE(bcm11351_master_ccu, BCM11351_DT_MASTER_CCU_COMPAT,
+CLK_OF_DECLARE(bcm281xx_master_ccu, BCM281XX_DT_MASTER_CCU_COMPAT,
kona_dt_master_ccu_setup);
-CLK_OF_DECLARE(bcm11351_slave_ccu, BCM11351_DT_SLAVE_CCU_COMPAT,
+CLK_OF_DECLARE(bcm281xx_slave_ccu, BCM281XX_DT_SLAVE_CCU_COMPAT,
kona_dt_slave_ccu_setup);
diff --git a/drivers/clk/bcm/clk-kona-setup.c b/drivers/clk/bcm/clk-kona-setup.c
index 54a0652..e5aeded 100644
--- a/drivers/clk/bcm/clk-kona-setup.c
+++ b/drivers/clk/bcm/clk-kona-setup.c
@@ -25,6 +25,31 @@
/* Validity checking */
+static bool ccu_data_offsets_valid(struct ccu_data *ccu)
+{
+ struct ccu_policy *ccu_policy = &ccu->policy;
+ u32 limit;
+
+ limit = ccu->range - sizeof(u32);
+ limit = round_down(limit, sizeof(u32));
+ if (ccu_policy_exists(ccu_policy)) {
+ if (ccu_policy->enable.offset > limit) {
+ pr_err("%s: bad policy enable offset for %s "
+ "(%u > %u)\n", __func__,
+ ccu->name, ccu_policy->enable.offset, limit);
+ return false;
+ }
+ if (ccu_policy->control.offset > limit) {
+ pr_err("%s: bad policy control offset for %s "
+ "(%u > %u)\n", __func__,
+ ccu->name, ccu_policy->control.offset, limit);
+ return false;
+ }
+ }
+
+ return true;
+}
+
static bool clk_requires_trigger(struct kona_clk *bcm_clk)
{
struct peri_clk_data *peri = bcm_clk->u.peri;
@@ -54,7 +79,9 @@
static bool peri_clk_data_offsets_valid(struct kona_clk *bcm_clk)
{
struct peri_clk_data *peri;
+ struct bcm_clk_policy *policy;
struct bcm_clk_gate *gate;
+ struct bcm_clk_hyst *hyst;
struct bcm_clk_div *div;
struct bcm_clk_sel *sel;
struct bcm_clk_trig *trig;
@@ -64,19 +91,41 @@
BUG_ON(bcm_clk->type != bcm_clk_peri);
peri = bcm_clk->u.peri;
- name = bcm_clk->name;
+ name = bcm_clk->init_data.name;
range = bcm_clk->ccu->range;
limit = range - sizeof(u32);
limit = round_down(limit, sizeof(u32));
+ policy = &peri->policy;
+ if (policy_exists(policy)) {
+ if (policy->offset > limit) {
+ pr_err("%s: bad policy offset for %s (%u > %u)\n",
+ __func__, name, policy->offset, limit);
+ return false;
+ }
+ }
+
gate = &peri->gate;
+ hyst = &peri->hyst;
if (gate_exists(gate)) {
if (gate->offset > limit) {
pr_err("%s: bad gate offset for %s (%u > %u)\n",
__func__, name, gate->offset, limit);
return false;
}
+
+ if (hyst_exists(hyst)) {
+ if (hyst->offset > limit) {
+ pr_err("%s: bad hysteresis offset for %s "
+ "(%u > %u)\n", __func__,
+ name, hyst->offset, limit);
+ return false;
+ }
+ }
+ } else if (hyst_exists(hyst)) {
+ pr_err("%s: hysteresis but no gate for %s\n", __func__, name);
+ return false;
}
div = &peri->div;
@@ -167,6 +216,36 @@
return true;
}
+static bool
+ccu_policy_valid(struct ccu_policy *ccu_policy, const char *ccu_name)
+{
+ struct bcm_lvm_en *enable = &ccu_policy->enable;
+ struct bcm_policy_ctl *control;
+
+ if (!bit_posn_valid(enable->bit, "policy enable", ccu_name))
+ return false;
+
+ control = &ccu_policy->control;
+ if (!bit_posn_valid(control->go_bit, "policy control GO", ccu_name))
+ return false;
+
+ if (!bit_posn_valid(control->atl_bit, "policy control ATL", ccu_name))
+ return false;
+
+ if (!bit_posn_valid(control->ac_bit, "policy control AC", ccu_name))
+ return false;
+
+ return true;
+}
+
+static bool policy_valid(struct bcm_clk_policy *policy, const char *clock_name)
+{
+ if (!bit_posn_valid(policy->bit, "policy", clock_name))
+ return false;
+
+ return true;
+}
+
/*
* All gates, if defined, have a status bit, and for hardware-only
* gates, that's it. Gates that can be software controlled also
@@ -196,6 +275,17 @@
return true;
}
+static bool hyst_valid(struct bcm_clk_hyst *hyst, const char *clock_name)
+{
+ if (!bit_posn_valid(hyst->en_bit, "hysteresis enable", clock_name))
+ return false;
+
+ if (!bit_posn_valid(hyst->val_bit, "hysteresis value", clock_name))
+ return false;
+
+ return true;
+}
+
/*
* A selector bitfield must be valid. Its parent_sel array must
* also be reasonable for the field.
@@ -312,7 +402,9 @@
peri_clk_data_valid(struct kona_clk *bcm_clk)
{
struct peri_clk_data *peri;
+ struct bcm_clk_policy *policy;
struct bcm_clk_gate *gate;
+ struct bcm_clk_hyst *hyst;
struct bcm_clk_sel *sel;
struct bcm_clk_div *div;
struct bcm_clk_div *pre_div;
@@ -330,11 +422,20 @@
return false;
peri = bcm_clk->u.peri;
- name = bcm_clk->name;
+ name = bcm_clk->init_data.name;
+
+ policy = &peri->policy;
+ if (policy_exists(policy) && !policy_valid(policy, name))
+ return false;
+
gate = &peri->gate;
if (gate_exists(gate) && !gate_valid(gate, "gate", name))
return false;
+ hyst = &peri->hyst;
+ if (hyst_exists(hyst) && !hyst_valid(hyst, name))
+ return false;
+
sel = &peri->sel;
if (selector_exists(sel)) {
if (!sel_valid(sel, "selector", name))
@@ -567,7 +668,6 @@
struct clk_init_data *init_data)
{
clk_sel_teardown(&data->sel, init_data);
- init_data->ops = NULL;
}
/*
@@ -576,10 +676,9 @@
* that can be assigned if the clock has one or more parent clocks
* associated with it.
*/
-static int peri_clk_setup(struct ccu_data *ccu, struct peri_clk_data *data,
- struct clk_init_data *init_data)
+static int
+peri_clk_setup(struct peri_clk_data *data, struct clk_init_data *init_data)
{
- init_data->ops = &kona_peri_clk_ops;
init_data->flags = CLK_IGNORE_UNUSED;
return clk_sel_setup(data->clocks, &data->sel, init_data);
@@ -617,39 +716,26 @@
bcm_clk_teardown(bcm_clk);
}
-struct clk *kona_clk_setup(struct ccu_data *ccu, const char *name,
- enum bcm_clk_type type, void *data)
+struct clk *kona_clk_setup(struct kona_clk *bcm_clk)
{
- struct kona_clk *bcm_clk;
- struct clk_init_data *init_data;
+ struct clk_init_data *init_data = &bcm_clk->init_data;
struct clk *clk = NULL;
- bcm_clk = kzalloc(sizeof(*bcm_clk), GFP_KERNEL);
- if (!bcm_clk) {
- pr_err("%s: failed to allocate bcm_clk for %s\n", __func__,
- name);
- return NULL;
- }
- bcm_clk->ccu = ccu;
- bcm_clk->name = name;
-
- init_data = &bcm_clk->init_data;
- init_data->name = name;
- switch (type) {
+ switch (bcm_clk->type) {
case bcm_clk_peri:
- if (peri_clk_setup(ccu, data, init_data))
- goto out_free;
+ if (peri_clk_setup(bcm_clk->u.data, init_data))
+ return NULL;
break;
default:
- data = NULL;
- break;
+ pr_err("%s: clock type %d invalid for %s\n", __func__,
+ (int)bcm_clk->type, init_data->name);
+ return NULL;
}
- bcm_clk->type = type;
- bcm_clk->u.data = data;
/* Make sure everything makes sense before we set it up */
if (!kona_clk_valid(bcm_clk)) {
- pr_err("%s: clock data invalid for %s\n", __func__, name);
+ pr_err("%s: clock data invalid for %s\n", __func__,
+ init_data->name);
goto out_teardown;
}
@@ -657,7 +743,7 @@
clk = clk_register(NULL, &bcm_clk->hw);
if (IS_ERR(clk)) {
pr_err("%s: error registering clock %s (%ld)\n", __func__,
- name, PTR_ERR(clk));
+ init_data->name, PTR_ERR(clk));
goto out_teardown;
}
BUG_ON(!clk);
@@ -665,8 +751,6 @@
return clk;
out_teardown:
bcm_clk_teardown(bcm_clk);
-out_free:
- kfree(bcm_clk);
return NULL;
}
@@ -675,50 +759,64 @@
{
u32 i;
- for (i = 0; i < ccu->data.clk_num; i++)
- kona_clk_teardown(ccu->data.clks[i]);
- kfree(ccu->data.clks);
+ for (i = 0; i < ccu->clk_data.clk_num; i++)
+ kona_clk_teardown(ccu->clk_data.clks[i]);
+ kfree(ccu->clk_data.clks);
}
static void kona_ccu_teardown(struct ccu_data *ccu)
{
- if (!ccu)
- return;
-
+ kfree(ccu->clk_data.clks);
+ ccu->clk_data.clks = NULL;
if (!ccu->base)
- goto done;
+ return;
of_clk_del_provider(ccu->node); /* safe if never added */
ccu_clks_teardown(ccu);
list_del(&ccu->links);
of_node_put(ccu->node);
+ ccu->node = NULL;
iounmap(ccu->base);
-done:
- kfree(ccu->name);
- kfree(ccu);
+ ccu->base = NULL;
+}
+
+static bool ccu_data_valid(struct ccu_data *ccu)
+{
+ struct ccu_policy *ccu_policy;
+
+ if (!ccu_data_offsets_valid(ccu))
+ return false;
+
+ ccu_policy = &ccu->policy;
+ if (ccu_policy_exists(ccu_policy))
+ if (!ccu_policy_valid(ccu_policy, ccu->name))
+ return false;
+
+ return true;
}
/*
* Set up a CCU. Call the provided ccu_clks_setup callback to
* initialize the array of clocks provided by the CCU.
*/
-void __init kona_dt_ccu_setup(struct device_node *node,
- int (*ccu_clks_setup)(struct ccu_data *))
+void __init kona_dt_ccu_setup(struct ccu_data *ccu,
+ struct device_node *node)
{
- struct ccu_data *ccu;
struct resource res = { 0 };
resource_size_t range;
+ unsigned int i;
int ret;
- ccu = kzalloc(sizeof(*ccu), GFP_KERNEL);
- if (ccu)
- ccu->name = kstrdup(node->name, GFP_KERNEL);
- if (!ccu || !ccu->name) {
- pr_err("%s: unable to allocate CCU struct for %s\n",
- __func__, node->name);
- kfree(ccu);
+ if (ccu->clk_data.clk_num) {
+ size_t size;
- return;
+ size = ccu->clk_data.clk_num * sizeof(*ccu->clk_data.clks);
+ ccu->clk_data.clks = kzalloc(size, GFP_KERNEL);
+ if (!ccu->clk_data.clks) {
+ pr_err("%s: unable to allocate %u clocks for %s\n",
+ __func__, ccu->clk_data.clk_num, node->name);
+ return;
+ }
}
ret = of_address_to_resource(node, 0, &res);
@@ -736,24 +834,33 @@
}
ccu->range = (u32)range;
+
+ if (!ccu_data_valid(ccu)) {
+ pr_err("%s: ccu data not valid for %s\n", __func__, node->name);
+ goto out_err;
+ }
+
ccu->base = ioremap(res.start, ccu->range);
if (!ccu->base) {
pr_err("%s: unable to map CCU registers for %s\n", __func__,
node->name);
goto out_err;
}
-
- spin_lock_init(&ccu->lock);
- INIT_LIST_HEAD(&ccu->links);
ccu->node = of_node_get(node);
-
list_add_tail(&ccu->links, &ccu_list);
- /* Set up clocks array (in ccu->data) */
- if (ccu_clks_setup(ccu))
- goto out_err;
+ /*
+ * Set up each defined kona clock and save the result in
+ * the clock framework clock array (in ccu->data). Then
+ * register as a provider for these clocks.
+ */
+ for (i = 0; i < ccu->clk_data.clk_num; i++) {
+ if (!ccu->kona_clks[i].ccu)
+ continue;
+ ccu->clk_data.clks[i] = kona_clk_setup(&ccu->kona_clks[i]);
+ }
- ret = of_clk_add_provider(node, of_clk_src_onecell_get, &ccu->data);
+ ret = of_clk_add_provider(node, of_clk_src_onecell_get, &ccu->clk_data);
if (ret) {
pr_err("%s: error adding ccu %s as provider (%d)\n", __func__,
node->name, ret);
diff --git a/drivers/clk/bcm/clk-kona.c b/drivers/clk/bcm/clk-kona.c
index db11a87..95af2e6 100644
--- a/drivers/clk/bcm/clk-kona.c
+++ b/drivers/clk/bcm/clk-kona.c
@@ -16,6 +16,14 @@
#include <linux/delay.h>
+/*
+ * "Policies" affect the frequencies of bus clocks provided by a
+ * CCU. (I believe these polices are named "Deep Sleep", "Economy",
+ * "Normal", and "Turbo".) A lower policy number has lower power
+ * consumption, and policy 2 is the default.
+ */
+#define CCU_POLICY_COUNT 4
+
#define CCU_ACCESS_PASSWORD 0xA5A500
#define CLK_GATE_DELAY_LOOP 2000
@@ -207,9 +215,154 @@
return true;
udelay(1);
}
+ pr_warn("%s: %s/0x%04x bit %u was never %s\n", __func__,
+ ccu->name, reg_offset, bit, want ? "set" : "clear");
+
return false;
}
+/* Policy operations */
+
+static bool __ccu_policy_engine_start(struct ccu_data *ccu, bool sync)
+{
+ struct bcm_policy_ctl *control = &ccu->policy.control;
+ u32 offset;
+ u32 go_bit;
+ u32 mask;
+ bool ret;
+
+ /* If we don't need to control policy for this CCU, we're done. */
+ if (!policy_ctl_exists(control))
+ return true;
+
+ offset = control->offset;
+ go_bit = control->go_bit;
+
+ /* Ensure we're not busy before we start */
+ ret = __ccu_wait_bit(ccu, offset, go_bit, false);
+ if (!ret) {
+ pr_err("%s: ccu %s policy engine wouldn't go idle\n",
+ __func__, ccu->name);
+ return false;
+ }
+
+ /*
+ * If it's a synchronous request, we'll wait for the voltage
+ * and frequency of the active load to stabilize before
+ * returning. To do this we select the active load by
+ * setting the ATL bit.
+ *
+ * An asynchronous request instead ramps the voltage in the
+ * background, and when that process stabilizes, the target
+ * load is copied to the active load and the CCU frequency
+ * is switched. We do this by selecting the target load
+ * (ATL bit clear) and setting the request auto-copy (AC bit
+ * set).
+ *
+ * Note, we do NOT read-modify-write this register.
+ */
+ mask = (u32)1 << go_bit;
+ if (sync)
+ mask |= 1 << control->atl_bit;
+ else
+ mask |= 1 << control->ac_bit;
+ __ccu_write(ccu, offset, mask);
+
+ /* Wait for indication that operation is complete. */
+ ret = __ccu_wait_bit(ccu, offset, go_bit, false);
+ if (!ret)
+ pr_err("%s: ccu %s policy engine never started\n",
+ __func__, ccu->name);
+
+ return ret;
+}
+
+static bool __ccu_policy_engine_stop(struct ccu_data *ccu)
+{
+ struct bcm_lvm_en *enable = &ccu->policy.enable;
+ u32 offset;
+ u32 enable_bit;
+ bool ret;
+
+ /* If we don't need to control policy for this CCU, we're done. */
+ if (!policy_lvm_en_exists(enable))
+ return true;
+
+ /* Ensure we're not busy before we start */
+ offset = enable->offset;
+ enable_bit = enable->bit;
+ ret = __ccu_wait_bit(ccu, offset, enable_bit, false);
+ if (!ret) {
+ pr_err("%s: ccu %s policy engine already stopped\n",
+ __func__, ccu->name);
+ return false;
+ }
+
+ /* Now set the bit to stop the engine (NO read-modify-write) */
+ __ccu_write(ccu, offset, (u32)1 << enable_bit);
+
+ /* Wait for indication that it has stopped. */
+ ret = __ccu_wait_bit(ccu, offset, enable_bit, false);
+ if (!ret)
+ pr_err("%s: ccu %s policy engine never stopped\n",
+ __func__, ccu->name);
+
+ return ret;
+}
+
+/*
+ * A CCU has four operating conditions ("policies"), and some clocks
+ * can be disabled or enabled based on which policy is currently in
+ * effect. Such clocks have a bit in a "policy mask" register for
+ * each policy indicating whether the clock is enabled for that
+ * policy or not. The bit position for a clock is the same for all
+ * four registers, and the 32-bit registers are at consecutive
+ * addresses.
+ */
+static bool policy_init(struct ccu_data *ccu, struct bcm_clk_policy *policy)
+{
+ u32 offset;
+ u32 mask;
+ int i;
+ bool ret;
+
+ if (!policy_exists(policy))
+ return true;
+
+ /*
+ * We need to stop the CCU policy engine to allow update
+ * of our policy bits.
+ */
+ if (!__ccu_policy_engine_stop(ccu)) {
+ pr_err("%s: unable to stop CCU %s policy engine\n",
+ __func__, ccu->name);
+ return false;
+ }
+
+ /*
+ * For now, if a clock defines its policy bit we just mark
+ * it "enabled" for all four policies.
+ */
+ offset = policy->offset;
+ mask = (u32)1 << policy->bit;
+ for (i = 0; i < CCU_POLICY_COUNT; i++) {
+ u32 reg_val;
+
+ reg_val = __ccu_read(ccu, offset);
+ reg_val |= mask;
+ __ccu_write(ccu, offset, reg_val);
+ offset += sizeof(u32);
+ }
+
+ /* We're done updating; fire up the policy engine again. */
+ ret = __ccu_policy_engine_start(ccu, true);
+ if (!ret)
+ pr_err("%s: unable to restart CCU %s policy engine\n",
+ __func__, ccu->name);
+
+ return ret;
+}
+
/* Gate operations */
/* Determine whether a clock is gated. CCU lock must be held. */
@@ -374,6 +527,35 @@
return -EIO;
}
+/* Hysteresis operations */
+
+/*
+ * If a clock gate requires a turn-off delay it will have
+ * "hysteresis" register bits defined. The first, if set, enables
+ * the delay; and if enabled, the second bit determines whether the
+ * delay is "low" or "high" (1 means high). For now, if it's
+ * defined for a clock, we set it.
+ */
+static bool hyst_init(struct ccu_data *ccu, struct bcm_clk_hyst *hyst)
+{
+ u32 offset;
+ u32 reg_val;
+ u32 mask;
+
+ if (!hyst_exists(hyst))
+ return true;
+
+ offset = hyst->offset;
+ mask = (u32)1 << hyst->en_bit;
+ mask |= (u32)1 << hyst->val_bit;
+
+ reg_val = __ccu_read(ccu, offset);
+ reg_val |= mask;
+ __ccu_write(ccu, offset, reg_val);
+
+ return true;
+}
+
/* Trigger operations */
/*
@@ -806,7 +988,7 @@
struct kona_clk *bcm_clk = to_kona_clk(hw);
struct bcm_clk_gate *gate = &bcm_clk->u.peri->gate;
- return clk_gate(bcm_clk->ccu, bcm_clk->name, gate, true);
+ return clk_gate(bcm_clk->ccu, bcm_clk->init_data.name, gate, true);
}
static void kona_peri_clk_disable(struct clk_hw *hw)
@@ -814,7 +996,7 @@
struct kona_clk *bcm_clk = to_kona_clk(hw);
struct bcm_clk_gate *gate = &bcm_clk->u.peri->gate;
- (void)clk_gate(bcm_clk->ccu, bcm_clk->name, gate, false);
+ (void)clk_gate(bcm_clk->ccu, bcm_clk->init_data.name, gate, false);
}
static int kona_peri_clk_is_enabled(struct clk_hw *hw)
@@ -849,6 +1031,58 @@
rate ? rate : 1, *parent_rate, NULL);
}
+static long kona_peri_clk_determine_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *best_parent_rate, struct clk **best_parent)
+{
+ struct kona_clk *bcm_clk = to_kona_clk(hw);
+ struct clk *clk = hw->clk;
+ struct clk *current_parent;
+ unsigned long parent_rate;
+ unsigned long best_delta;
+ unsigned long best_rate;
+ u32 parent_count;
+ u32 which;
+
+ /*
+ * If there is no other parent to choose, use the current one.
+ * Note: We don't honor (or use) CLK_SET_RATE_NO_REPARENT.
+ */
+ WARN_ON_ONCE(bcm_clk->init_data.flags & CLK_SET_RATE_NO_REPARENT);
+ parent_count = (u32)bcm_clk->init_data.num_parents;
+ if (parent_count < 2)
+ return kona_peri_clk_round_rate(hw, rate, best_parent_rate);
+
+ /* Unless we can do better, stick with current parent */
+ current_parent = clk_get_parent(clk);
+ parent_rate = __clk_get_rate(current_parent);
+ best_rate = kona_peri_clk_round_rate(hw, rate, &parent_rate);
+ best_delta = abs(best_rate - rate);
+
+ /* Check whether any other parent clock can produce a better result */
+ for (which = 0; which < parent_count; which++) {
+ struct clk *parent = clk_get_parent_by_index(clk, which);
+ unsigned long delta;
+ unsigned long other_rate;
+
+ BUG_ON(!parent);
+ if (parent == current_parent)
+ continue;
+
+ /* We don't support CLK_SET_RATE_PARENT */
+ parent_rate = __clk_get_rate(parent);
+ other_rate = kona_peri_clk_round_rate(hw, rate, &parent_rate);
+ delta = abs(other_rate - rate);
+ if (delta < best_delta) {
+ best_delta = delta;
+ best_rate = other_rate;
+ *best_parent = parent;
+ *best_parent_rate = parent_rate;
+ }
+ }
+
+ return best_rate;
+}
+
static int kona_peri_clk_set_parent(struct clk_hw *hw, u8 index)
{
struct kona_clk *bcm_clk = to_kona_clk(hw);
@@ -872,12 +1106,13 @@
ret = selector_write(bcm_clk->ccu, &data->gate, sel, trig, index);
if (ret == -ENXIO) {
- pr_err("%s: gating failure for %s\n", __func__, bcm_clk->name);
+ pr_err("%s: gating failure for %s\n", __func__,
+ bcm_clk->init_data.name);
ret = -EIO; /* Don't proliferate weird errors */
} else if (ret == -EIO) {
pr_err("%s: %strigger failed for %s\n", __func__,
trig == &data->pre_trig ? "pre-" : "",
- bcm_clk->name);
+ bcm_clk->init_data.name);
}
return ret;
@@ -936,10 +1171,12 @@
ret = divider_write(bcm_clk->ccu, &data->gate, &data->div,
&data->trig, scaled_div);
if (ret == -ENXIO) {
- pr_err("%s: gating failure for %s\n", __func__, bcm_clk->name);
+ pr_err("%s: gating failure for %s\n", __func__,
+ bcm_clk->init_data.name);
ret = -EIO; /* Don't proliferate weird errors */
} else if (ret == -EIO) {
- pr_err("%s: trigger failed for %s\n", __func__, bcm_clk->name);
+ pr_err("%s: trigger failed for %s\n", __func__,
+ bcm_clk->init_data.name);
}
return ret;
@@ -950,7 +1187,7 @@
.disable = kona_peri_clk_disable,
.is_enabled = kona_peri_clk_is_enabled,
.recalc_rate = kona_peri_clk_recalc_rate,
- .round_rate = kona_peri_clk_round_rate,
+ .determine_rate = kona_peri_clk_determine_rate,
.set_parent = kona_peri_clk_set_parent,
.get_parent = kona_peri_clk_get_parent,
.set_rate = kona_peri_clk_set_rate,
@@ -961,15 +1198,24 @@
{
struct ccu_data *ccu = bcm_clk->ccu;
struct peri_clk_data *peri = bcm_clk->u.peri;
- const char *name = bcm_clk->name;
+ const char *name = bcm_clk->init_data.name;
struct bcm_clk_trig *trig;
BUG_ON(bcm_clk->type != bcm_clk_peri);
+ if (!policy_init(ccu, &peri->policy)) {
+ pr_err("%s: error initializing policy for %s\n",
+ __func__, name);
+ return false;
+ }
if (!gate_init(ccu, &peri->gate)) {
pr_err("%s: error initializing gate for %s\n", __func__, name);
return false;
}
+ if (!hyst_init(ccu, &peri->hyst)) {
+ pr_err("%s: error initializing hyst for %s\n", __func__, name);
+ return false;
+ }
if (!div_init(ccu, &peri->gate, &peri->div, &peri->trig)) {
pr_err("%s: error initializing divider for %s\n", __func__,
name);
@@ -1014,13 +1260,13 @@
{
unsigned long flags;
unsigned int which;
- struct clk **clks = ccu->data.clks;
+ struct clk **clks = ccu->clk_data.clks;
bool success = true;
flags = ccu_lock(ccu);
__ccu_write_enable(ccu);
- for (which = 0; which < ccu->data.clk_num; which++) {
+ for (which = 0; which < ccu->clk_data.clk_num; which++) {
struct kona_clk *bcm_clk;
if (!clks[which])
diff --git a/drivers/clk/bcm/clk-kona.h b/drivers/clk/bcm/clk-kona.h
index dee6909..2537b30 100644
--- a/drivers/clk/bcm/clk-kona.h
+++ b/drivers/clk/bcm/clk-kona.h
@@ -43,8 +43,14 @@
#define FLAG_FLIP(obj, type, flag) ((obj)->flags ^= FLAG(type, flag))
#define FLAG_TEST(obj, type, flag) (!!((obj)->flags & FLAG(type, flag)))
+/* CCU field state tests */
+
+#define ccu_policy_exists(ccu_policy) ((ccu_policy)->enable.offset != 0)
+
/* Clock field state tests */
+#define policy_exists(policy) ((policy)->offset != 0)
+
#define gate_exists(gate) FLAG_TEST(gate, GATE, EXISTS)
#define gate_is_enabled(gate) FLAG_TEST(gate, GATE, ENABLED)
#define gate_is_hw_controllable(gate) FLAG_TEST(gate, GATE, HW)
@@ -54,6 +60,8 @@
#define gate_flip_enabled(gate) FLAG_FLIP(gate, GATE, ENABLED)
+#define hyst_exists(hyst) ((hyst)->offset != 0)
+
#define divider_exists(div) FLAG_TEST(div, DIV, EXISTS)
#define divider_is_fixed(div) FLAG_TEST(div, DIV, FIXED)
#define divider_has_fraction(div) (!divider_is_fixed(div) && \
@@ -62,6 +70,9 @@
#define selector_exists(sel) ((sel)->width != 0)
#define trigger_exists(trig) FLAG_TEST(trig, TRIG, EXISTS)
+#define policy_lvm_en_exists(enable) ((enable)->offset != 0)
+#define policy_ctl_exists(control) ((control)->offset != 0)
+
/* Clock type, used to tell common block what it's part of */
enum bcm_clk_type {
bcm_clk_none, /* undefined clock type */
@@ -71,25 +82,26 @@
};
/*
- * Each CCU defines a mapped area of memory containing registers
- * used to manage clocks implemented by the CCU. Access to memory
- * within the CCU's space is serialized by a spinlock. Before any
- * (other) address can be written, a special access "password" value
- * must be written to its WR_ACCESS register (located at the base
- * address of the range). We keep track of the name of each CCU as
- * it is set up, and maintain them in a list.
+ * CCU policy control for clocks. Clocks can be enabled or disabled
+ * based on the CCU policy in effect. One bit in each policy mask
+ * register (one per CCU policy) represents whether the clock is
+ * enabled when that policy is effect or not. The CCU policy engine
+ * must be stopped to update these bits, and must be restarted again
+ * afterward.
*/
-struct ccu_data {
- void __iomem *base; /* base of mapped address space */
- spinlock_t lock; /* serialization lock */
- bool write_enabled; /* write access is currently enabled */
- struct list_head links; /* for ccu_list */
- struct device_node *node;
- struct clk_onecell_data data;
- const char *name;
- u32 range; /* byte range of address space */
+struct bcm_clk_policy {
+ u32 offset; /* first policy mask register offset */
+ u32 bit; /* bit used in all mask registers */
};
+/* Policy initialization macro */
+
+#define POLICY(_offset, _bit) \
+ { \
+ .offset = (_offset), \
+ .bit = (_bit), \
+ }
+
/*
* Gating control and status is managed by a 32-bit gate register.
*
@@ -195,6 +207,22 @@
.flags = FLAG(GATE, HW)|FLAG(GATE, EXISTS), \
}
+/* Gate hysteresis for clocks */
+struct bcm_clk_hyst {
+ u32 offset; /* hyst register offset (normally CLKGATE) */
+ u32 en_bit; /* bit used to enable hysteresis */
+ u32 val_bit; /* if enabled: 0 = low delay; 1 = high delay */
+};
+
+/* Hysteresis initialization macro */
+
+#define HYST(_offset, _en_bit, _val_bit) \
+ { \
+ .offset = (_offset), \
+ .en_bit = (_en_bit), \
+ .val_bit = (_val_bit), \
+ }
+
/*
* Each clock can have zero, one, or two dividers which change the
* output rate of the clock. Each divider can be either fixed or
@@ -360,7 +388,9 @@
}
struct peri_clk_data {
+ struct bcm_clk_policy policy;
struct bcm_clk_gate gate;
+ struct bcm_clk_hyst hyst;
struct bcm_clk_trig pre_trig;
struct bcm_clk_div pre_div;
struct bcm_clk_trig trig;
@@ -373,8 +403,7 @@
struct kona_clk {
struct clk_hw hw;
- struct clk_init_data init_data;
- const char *name; /* name of this clock */
+ struct clk_init_data init_data; /* includes name of this clock */
struct ccu_data *ccu; /* ccu this clock is associated with */
enum bcm_clk_type type;
union {
@@ -385,15 +414,93 @@
#define to_kona_clk(_hw) \
container_of(_hw, struct kona_clk, hw)
+/* Initialization macro for an entry in a CCU's kona_clks[] array. */
+#define KONA_CLK(_ccu_name, _clk_name, _type) \
+ { \
+ .init_data = { \
+ .name = #_clk_name, \
+ .ops = &kona_ ## _type ## _clk_ops, \
+ }, \
+ .ccu = &_ccu_name ## _ccu_data, \
+ .type = bcm_clk_ ## _type, \
+ .u.data = &_clk_name ## _data, \
+ }
+#define LAST_KONA_CLK { .type = bcm_clk_none }
+
+/*
+ * CCU policy control. To enable software update of the policy
+ * tables the CCU policy engine must be stopped by setting the
+ * software update enable bit (LVM_EN). After an update the engine
+ * is restarted using the GO bit and either the GO_ATL or GO_AC bit.
+ */
+struct bcm_lvm_en {
+ u32 offset; /* LVM_EN register offset */
+ u32 bit; /* POLICY_CONFIG_EN bit in register */
+};
+
+/* Policy enable initialization macro */
+#define CCU_LVM_EN(_offset, _bit) \
+ { \
+ .offset = (_offset), \
+ .bit = (_bit), \
+ }
+
+struct bcm_policy_ctl {
+ u32 offset; /* POLICY_CTL register offset */
+ u32 go_bit;
+ u32 atl_bit; /* GO, GO_ATL, and GO_AC bits */
+ u32 ac_bit;
+};
+
+/* Policy control initialization macro */
+#define CCU_POLICY_CTL(_offset, _go_bit, _ac_bit, _atl_bit) \
+ { \
+ .offset = (_offset), \
+ .go_bit = (_go_bit), \
+ .ac_bit = (_ac_bit), \
+ .atl_bit = (_atl_bit), \
+ }
+
+struct ccu_policy {
+ struct bcm_lvm_en enable;
+ struct bcm_policy_ctl control;
+};
+
+/*
+ * Each CCU defines a mapped area of memory containing registers
+ * used to manage clocks implemented by the CCU. Access to memory
+ * within the CCU's space is serialized by a spinlock. Before any
+ * (other) address can be written, a special access "password" value
+ * must be written to its WR_ACCESS register (located at the base
+ * address of the range). We keep track of the name of each CCU as
+ * it is set up, and maintain them in a list.
+ */
+struct ccu_data {
+ void __iomem *base; /* base of mapped address space */
+ spinlock_t lock; /* serialization lock */
+ bool write_enabled; /* write access is currently enabled */
+ struct ccu_policy policy;
+ struct list_head links; /* for ccu_list */
+ struct device_node *node;
+ struct clk_onecell_data clk_data;
+ const char *name;
+ u32 range; /* byte range of address space */
+ struct kona_clk kona_clks[]; /* must be last */
+};
+
+/* Initialization for common fields in a Kona ccu_data structure */
+#define KONA_CCU_COMMON(_prefix, _name, _ccuname) \
+ .name = #_name "_ccu", \
+ .lock = __SPIN_LOCK_UNLOCKED(_name ## _ccu_data.lock), \
+ .links = LIST_HEAD_INIT(_name ## _ccu_data.links), \
+ .clk_data = { \
+ .clk_num = _prefix ## _ ## _ccuname ## _CCU_CLOCK_COUNT, \
+ }
+
/* Exported globals */
extern struct clk_ops kona_peri_clk_ops;
-/* Help functions */
-
-#define PERI_CLK_SETUP(clks, ccu, id, name) \
- clks[id] = kona_clk_setup(ccu, #name, bcm_clk_peri, &name ## _data)
-
/* Externally visible functions */
extern u64 do_div_round_closest(u64 dividend, unsigned long divisor);
@@ -401,10 +508,9 @@
extern u64 scaled_div_build(struct bcm_clk_div *div, u32 div_value,
u32 billionths);
-extern struct clk *kona_clk_setup(struct ccu_data *ccu, const char *name,
- enum bcm_clk_type type, void *data);
-extern void __init kona_dt_ccu_setup(struct device_node *node,
- int (*ccu_clks_setup)(struct ccu_data *));
+extern struct clk *kona_clk_setup(struct kona_clk *bcm_clk);
+extern void __init kona_dt_ccu_setup(struct ccu_data *ccu,
+ struct device_node *node);
extern bool __init kona_ccu_init(struct ccu_data *ccu);
#endif /* _CLK_KONA_H */
diff --git a/drivers/clk/berlin/Makefile b/drivers/clk/berlin/Makefile
new file mode 100644
index 0000000..2a36ab710
--- /dev/null
+++ b/drivers/clk/berlin/Makefile
@@ -0,0 +1,4 @@
+obj-y += berlin2-avpll.o berlin2-pll.o berlin2-div.o
+obj-$(CONFIG_MACH_BERLIN_BG2) += bg2.o
+obj-$(CONFIG_MACH_BERLIN_BG2CD) += bg2.o
+obj-$(CONFIG_MACH_BERLIN_BG2Q) += bg2q.o
diff --git a/drivers/clk/berlin/berlin2-avpll.c b/drivers/clk/berlin/berlin2-avpll.c
new file mode 100644
index 0000000..fd0f26c
--- /dev/null
+++ b/drivers/clk/berlin/berlin2-avpll.c
@@ -0,0 +1,393 @@
+/*
+ * Copyright (c) 2014 Marvell Technology Group Ltd.
+ *
+ * Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
+ * Alexandre Belloni <alexandre.belloni@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <linux/clk-provider.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+
+#include "berlin2-avpll.h"
+
+/*
+ * Berlin2 SoCs comprise up to two PLLs called AVPLL built upon a
+ * VCO with 8 channels each, channel 8 is the odd-one-out and does
+ * not provide mul/div.
+ *
+ * Unfortunately, its registers are not named but just numbered. To
+ * get in at least some kind of structure, we split each AVPLL into
+ * the VCOs and each channel into separate clock drivers.
+ *
+ * Also, here and there the VCO registers are a bit different with
+ * respect to bit shifts. Make sure to add a comment for those.
+ */
+#define NUM_CHANNELS 8
+
+#define AVPLL_CTRL(x) ((x) * 0x4)
+
+#define VCO_CTRL0 AVPLL_CTRL(0)
+/* BG2/BG2CDs VCO_B has an additional shift of 4 for its VCO_CTRL0 reg */
+#define VCO_RESET BIT(0)
+#define VCO_POWERUP BIT(1)
+#define VCO_INTERPOL_SHIFT 2
+#define VCO_INTERPOL_MASK (0xf << VCO_INTERPOL_SHIFT)
+#define VCO_REG1V45_SEL_SHIFT 6
+#define VCO_REG1V45_SEL(x) ((x) << VCO_REG1V45_SEL_SHIFT)
+#define VCO_REG1V45_SEL_1V40 VCO_REG1V45_SEL(0)
+#define VCO_REG1V45_SEL_1V45 VCO_REG1V45_SEL(1)
+#define VCO_REG1V45_SEL_1V50 VCO_REG1V45_SEL(2)
+#define VCO_REG1V45_SEL_1V55 VCO_REG1V45_SEL(3)
+#define VCO_REG1V45_SEL_MASK VCO_REG1V45_SEL(3)
+#define VCO_REG0V9_SEL_SHIFT 8
+#define VCO_REG0V9_SEL_MASK (0xf << VCO_REG0V9_SEL_SHIFT)
+#define VCO_VTHCAL_SHIFT 12
+#define VCO_VTHCAL(x) ((x) << VCO_VTHCAL_SHIFT)
+#define VCO_VTHCAL_0V90 VCO_VTHCAL(0)
+#define VCO_VTHCAL_0V95 VCO_VTHCAL(1)
+#define VCO_VTHCAL_1V00 VCO_VTHCAL(2)
+#define VCO_VTHCAL_1V05 VCO_VTHCAL(3)
+#define VCO_VTHCAL_MASK VCO_VTHCAL(3)
+#define VCO_KVCOEXT_SHIFT 14
+#define VCO_KVCOEXT_MASK (0x3 << VCO_KVCOEXT_SHIFT)
+#define VCO_KVCOEXT_ENABLE BIT(17)
+#define VCO_V2IEXT_SHIFT 18
+#define VCO_V2IEXT_MASK (0xf << VCO_V2IEXT_SHIFT)
+#define VCO_V2IEXT_ENABLE BIT(22)
+#define VCO_SPEED_SHIFT 23
+#define VCO_SPEED(x) ((x) << VCO_SPEED_SHIFT)
+#define VCO_SPEED_1G08_1G21 VCO_SPEED(0)
+#define VCO_SPEED_1G21_1G40 VCO_SPEED(1)
+#define VCO_SPEED_1G40_1G61 VCO_SPEED(2)
+#define VCO_SPEED_1G61_1G86 VCO_SPEED(3)
+#define VCO_SPEED_1G86_2G00 VCO_SPEED(4)
+#define VCO_SPEED_2G00_2G22 VCO_SPEED(5)
+#define VCO_SPEED_2G22 VCO_SPEED(6)
+#define VCO_SPEED_MASK VCO_SPEED(0x7)
+#define VCO_CLKDET_ENABLE BIT(26)
+#define VCO_CTRL1 AVPLL_CTRL(1)
+#define VCO_REFDIV_SHIFT 0
+#define VCO_REFDIV(x) ((x) << VCO_REFDIV_SHIFT)
+#define VCO_REFDIV_1 VCO_REFDIV(0)
+#define VCO_REFDIV_2 VCO_REFDIV(1)
+#define VCO_REFDIV_4 VCO_REFDIV(2)
+#define VCO_REFDIV_3 VCO_REFDIV(3)
+#define VCO_REFDIV_MASK VCO_REFDIV(0x3f)
+#define VCO_FBDIV_SHIFT 6
+#define VCO_FBDIV(x) ((x) << VCO_FBDIV_SHIFT)
+#define VCO_FBDIV_MASK VCO_FBDIV(0xff)
+#define VCO_ICP_SHIFT 14
+/* PLL Charge Pump Current = 10uA * (x + 1) */
+#define VCO_ICP(x) ((x) << VCO_ICP_SHIFT)
+#define VCO_ICP_MASK VCO_ICP(0xf)
+#define VCO_LOAD_CAP BIT(18)
+#define VCO_CALIBRATION_START BIT(19)
+#define VCO_FREQOFFSETn(x) AVPLL_CTRL(3 + (x))
+#define VCO_FREQOFFSET_MASK 0x7ffff
+#define VCO_CTRL10 AVPLL_CTRL(10)
+#define VCO_POWERUP_CH1 BIT(20)
+#define VCO_CTRL11 AVPLL_CTRL(11)
+#define VCO_CTRL12 AVPLL_CTRL(12)
+#define VCO_CTRL13 AVPLL_CTRL(13)
+#define VCO_CTRL14 AVPLL_CTRL(14)
+#define VCO_CTRL15 AVPLL_CTRL(15)
+#define VCO_SYNC1n(x) AVPLL_CTRL(15 + (x))
+#define VCO_SYNC1_MASK 0x1ffff
+#define VCO_SYNC2n(x) AVPLL_CTRL(23 + (x))
+#define VCO_SYNC2_MASK 0x1ffff
+#define VCO_CTRL30 AVPLL_CTRL(30)
+#define VCO_DPLL_CH1_ENABLE BIT(17)
+
+struct berlin2_avpll_vco {
+ struct clk_hw hw;
+ void __iomem *base;
+ u8 flags;
+};
+
+#define to_avpll_vco(hw) container_of(hw, struct berlin2_avpll_vco, hw)
+
+static int berlin2_avpll_vco_is_enabled(struct clk_hw *hw)
+{
+ struct berlin2_avpll_vco *vco = to_avpll_vco(hw);
+ u32 reg;
+
+ reg = readl_relaxed(vco->base + VCO_CTRL0);
+ if (vco->flags & BERLIN2_AVPLL_BIT_QUIRK)
+ reg >>= 4;
+
+ return !!(reg & VCO_POWERUP);
+}
+
+static int berlin2_avpll_vco_enable(struct clk_hw *hw)
+{
+ struct berlin2_avpll_vco *vco = to_avpll_vco(hw);
+ u32 reg;
+
+ reg = readl_relaxed(vco->base + VCO_CTRL0);
+ if (vco->flags & BERLIN2_AVPLL_BIT_QUIRK)
+ reg |= VCO_POWERUP << 4;
+ else
+ reg |= VCO_POWERUP;
+ writel_relaxed(reg, vco->base + VCO_CTRL0);
+
+ return 0;
+}
+
+static void berlin2_avpll_vco_disable(struct clk_hw *hw)
+{
+ struct berlin2_avpll_vco *vco = to_avpll_vco(hw);
+ u32 reg;
+
+ reg = readl_relaxed(vco->base + VCO_CTRL0);
+ if (vco->flags & BERLIN2_AVPLL_BIT_QUIRK)
+ reg &= ~(VCO_POWERUP << 4);
+ else
+ reg &= ~VCO_POWERUP;
+ writel_relaxed(reg, vco->base + VCO_CTRL0);
+}
+
+static u8 vco_refdiv[] = { 1, 2, 4, 3 };
+
+static unsigned long
+berlin2_avpll_vco_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
+{
+ struct berlin2_avpll_vco *vco = to_avpll_vco(hw);
+ u32 reg, refdiv, fbdiv;
+ u64 freq = parent_rate;
+
+ /* AVPLL VCO frequency: Fvco = (Fref / refdiv) * fbdiv */
+ reg = readl_relaxed(vco->base + VCO_CTRL1);
+ refdiv = (reg & VCO_REFDIV_MASK) >> VCO_REFDIV_SHIFT;
+ refdiv = vco_refdiv[refdiv];
+ fbdiv = (reg & VCO_FBDIV_MASK) >> VCO_FBDIV_SHIFT;
+ freq *= fbdiv;
+ do_div(freq, refdiv);
+
+ return (unsigned long)freq;
+}
+
+static const struct clk_ops berlin2_avpll_vco_ops = {
+ .is_enabled = berlin2_avpll_vco_is_enabled,
+ .enable = berlin2_avpll_vco_enable,
+ .disable = berlin2_avpll_vco_disable,
+ .recalc_rate = berlin2_avpll_vco_recalc_rate,
+};
+
+struct clk * __init berlin2_avpll_vco_register(void __iomem *base,
+ const char *name, const char *parent_name,
+ u8 vco_flags, unsigned long flags)
+{
+ struct berlin2_avpll_vco *vco;
+ struct clk_init_data init;
+
+ vco = kzalloc(sizeof(*vco), GFP_KERNEL);
+ if (!vco)
+ return ERR_PTR(-ENOMEM);
+
+ vco->base = base;
+ vco->flags = vco_flags;
+ vco->hw.init = &init;
+ init.name = name;
+ init.ops = &berlin2_avpll_vco_ops;
+ init.parent_names = &parent_name;
+ init.num_parents = 1;
+ init.flags = flags;
+
+ return clk_register(NULL, &vco->hw);
+}
+
+struct berlin2_avpll_channel {
+ struct clk_hw hw;
+ void __iomem *base;
+ u8 flags;
+ u8 index;
+};
+
+#define to_avpll_channel(hw) container_of(hw, struct berlin2_avpll_channel, hw)
+
+static int berlin2_avpll_channel_is_enabled(struct clk_hw *hw)
+{
+ struct berlin2_avpll_channel *ch = to_avpll_channel(hw);
+ u32 reg;
+
+ if (ch->index == 7)
+ return 1;
+
+ reg = readl_relaxed(ch->base + VCO_CTRL10);
+ reg &= VCO_POWERUP_CH1 << ch->index;
+
+ return !!reg;
+}
+
+static int berlin2_avpll_channel_enable(struct clk_hw *hw)
+{
+ struct berlin2_avpll_channel *ch = to_avpll_channel(hw);
+ u32 reg;
+
+ reg = readl_relaxed(ch->base + VCO_CTRL10);
+ reg |= VCO_POWERUP_CH1 << ch->index;
+ writel_relaxed(reg, ch->base + VCO_CTRL10);
+
+ return 0;
+}
+
+static void berlin2_avpll_channel_disable(struct clk_hw *hw)
+{
+ struct berlin2_avpll_channel *ch = to_avpll_channel(hw);
+ u32 reg;
+
+ reg = readl_relaxed(ch->base + VCO_CTRL10);
+ reg &= ~(VCO_POWERUP_CH1 << ch->index);
+ writel_relaxed(reg, ch->base + VCO_CTRL10);
+}
+
+static const u8 div_hdmi[] = { 1, 2, 4, 6 };
+static const u8 div_av1[] = { 1, 2, 5, 5 };
+
+static unsigned long
+berlin2_avpll_channel_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
+{
+ struct berlin2_avpll_channel *ch = to_avpll_channel(hw);
+ u32 reg, div_av2, div_av3, divider = 1;
+ u64 freq = parent_rate;
+
+ reg = readl_relaxed(ch->base + VCO_CTRL30);
+ if ((reg & (VCO_DPLL_CH1_ENABLE << ch->index)) == 0)
+ goto skip_div;
+
+ /*
+ * Fch = (Fref * sync2) /
+ * (sync1 * div_hdmi * div_av1 * div_av2 * div_av3)
+ */
+
+ reg = readl_relaxed(ch->base + VCO_SYNC1n(ch->index));
+ /* BG2/BG2CDs SYNC1 reg on AVPLL_B channel 1 is shifted by 4 */
+ if (ch->flags & BERLIN2_AVPLL_BIT_QUIRK && ch->index == 0)
+ reg >>= 4;
+ divider = reg & VCO_SYNC1_MASK;
+
+ reg = readl_relaxed(ch->base + VCO_SYNC2n(ch->index));
+ freq *= reg & VCO_SYNC2_MASK;
+
+ /* Channel 8 has no dividers */
+ if (ch->index == 7)
+ goto skip_div;
+
+ /*
+ * HDMI divider start at VCO_CTRL11, bit 7; MSB is enable, lower 2 bit
+ * determine divider.
+ */
+ reg = readl_relaxed(ch->base + VCO_CTRL11) >> 7;
+ reg = (reg >> (ch->index * 3));
+ if (reg & BIT(2))
+ divider *= div_hdmi[reg & 0x3];
+
+ /*
+ * AV1 divider start at VCO_CTRL11, bit 28; MSB is enable, lower 2 bit
+ * determine divider.
+ */
+ if (ch->index == 0) {
+ reg = readl_relaxed(ch->base + VCO_CTRL11);
+ reg >>= 28;
+ } else {
+ reg = readl_relaxed(ch->base + VCO_CTRL12);
+ reg >>= (ch->index-1) * 3;
+ }
+ if (reg & BIT(2))
+ divider *= div_av1[reg & 0x3];
+
+ /*
+ * AV2 divider start at VCO_CTRL12, bit 18; each 7 bits wide,
+ * zero is not a valid value.
+ */
+ if (ch->index < 2) {
+ reg = readl_relaxed(ch->base + VCO_CTRL12);
+ reg >>= 18 + (ch->index * 7);
+ } else if (ch->index < 7) {
+ reg = readl_relaxed(ch->base + VCO_CTRL13);
+ reg >>= (ch->index - 2) * 7;
+ } else {
+ reg = readl_relaxed(ch->base + VCO_CTRL14);
+ }
+ div_av2 = reg & 0x7f;
+ if (div_av2)
+ divider *= div_av2;
+
+ /*
+ * AV3 divider start at VCO_CTRL14, bit 7; each 4 bits wide.
+ * AV2/AV3 form a fractional divider, where only specfic values for AV3
+ * are allowed. AV3 != 0 divides by AV2/2, AV3=0 is bypass.
+ */
+ if (ch->index < 6) {
+ reg = readl_relaxed(ch->base + VCO_CTRL14);
+ reg >>= 7 + (ch->index * 4);
+ } else {
+ reg = readl_relaxed(ch->base + VCO_CTRL15);
+ }
+ div_av3 = reg & 0xf;
+ if (div_av2 && div_av3)
+ freq *= 2;
+
+skip_div:
+ do_div(freq, divider);
+ return (unsigned long)freq;
+}
+
+static const struct clk_ops berlin2_avpll_channel_ops = {
+ .is_enabled = berlin2_avpll_channel_is_enabled,
+ .enable = berlin2_avpll_channel_enable,
+ .disable = berlin2_avpll_channel_disable,
+ .recalc_rate = berlin2_avpll_channel_recalc_rate,
+};
+
+/*
+ * Another nice quirk:
+ * On some production SoCs, AVPLL channels are scrambled with respect
+ * to the channel numbering in the registers but still referenced by
+ * their original channel numbers. We deal with it by having a flag
+ * and a translation table for the index.
+ */
+static const u8 quirk_index[] __initconst = { 0, 6, 5, 4, 3, 2, 1, 7 };
+
+struct clk * __init berlin2_avpll_channel_register(void __iomem *base,
+ const char *name, u8 index, const char *parent_name,
+ u8 ch_flags, unsigned long flags)
+{
+ struct berlin2_avpll_channel *ch;
+ struct clk_init_data init;
+
+ ch = kzalloc(sizeof(*ch), GFP_KERNEL);
+ if (!ch)
+ return ERR_PTR(-ENOMEM);
+
+ ch->base = base;
+ if (ch_flags & BERLIN2_AVPLL_SCRAMBLE_QUIRK)
+ ch->index = quirk_index[index];
+ else
+ ch->index = index;
+
+ ch->flags = ch_flags;
+ ch->hw.init = &init;
+ init.name = name;
+ init.ops = &berlin2_avpll_channel_ops;
+ init.parent_names = &parent_name;
+ init.num_parents = 1;
+ init.flags = flags;
+
+ return clk_register(NULL, &ch->hw);
+}
diff --git a/drivers/clk/berlin/berlin2-avpll.h b/drivers/clk/berlin/berlin2-avpll.h
new file mode 100644
index 0000000..a37f506
--- /dev/null
+++ b/drivers/clk/berlin/berlin2-avpll.h
@@ -0,0 +1,36 @@
+/*
+ * Copyright (c) 2014 Marvell Technology Group Ltd.
+ *
+ * Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
+ * Alexandre Belloni <alexandre.belloni@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __BERLIN2_AVPLL_H
+#define __BERLIN2_AVPLL_H
+
+struct clk;
+
+#define BERLIN2_AVPLL_BIT_QUIRK BIT(0)
+#define BERLIN2_AVPLL_SCRAMBLE_QUIRK BIT(1)
+
+struct clk * __init
+berlin2_avpll_vco_register(void __iomem *base, const char *name,
+ const char *parent_name, u8 vco_flags, unsigned long flags);
+
+struct clk * __init
+berlin2_avpll_channel_register(void __iomem *base, const char *name,
+ u8 index, const char *parent_name, u8 ch_flags,
+ unsigned long flags);
+
+#endif /* __BERLIN2_AVPLL_H */
diff --git a/drivers/clk/berlin/berlin2-div.c b/drivers/clk/berlin/berlin2-div.c
new file mode 100644
index 0000000..81ff97f
--- /dev/null
+++ b/drivers/clk/berlin/berlin2-div.c
@@ -0,0 +1,265 @@
+/*
+ * Copyright (c) 2014 Marvell Technology Group Ltd.
+ *
+ * Alexandre Belloni <alexandre.belloni@free-electrons.com>
+ * Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <linux/bitops.h>
+#include <linux/clk-provider.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include "berlin2-div.h"
+
+/*
+ * Clock dividers in Berlin2 SoCs comprise a complex cell to select
+ * input pll and divider. The virtual structure as it is used in Marvell
+ * BSP code can be seen as:
+ *
+ * +---+
+ * pll0 --------------->| 0 | +---+
+ * +---+ |(B)|--+--------------->| 0 | +---+
+ * pll1.0 -->| 0 | +-->| 1 | | +--------+ |(E)|----->| 0 | +---+
+ * pll1.1 -->| 1 | | +---+ +-->|(C) 1:M |-->| 1 | |(F)|-->|(G)|->
+ * ... -->|(A)|--+ | +--------+ +---+ +-->| 1 | +---+
+ * ... -->| | +-->|(D) 1:3 |----------+ +---+
+ * pll1.N -->| N | +---------
+ * +---+
+ *
+ * (A) input pll clock mux controlled by <PllSelect[1:n]>
+ * (B) input pll bypass mux controlled by <PllSwitch>
+ * (C) programmable clock divider controlled by <Select[1:n]>
+ * (D) constant div-by-3 clock divider
+ * (E) programmable clock divider bypass controlled by <Switch>
+ * (F) constant div-by-3 clock mux controlled by <D3Switch>
+ * (G) clock gate controlled by <Enable>
+ *
+ * For whatever reason, above control signals come in two flavors:
+ * - single register dividers with all bits in one register
+ * - shared register dividers with bits spread over multiple registers
+ * (including signals for the same cell spread over consecutive registers)
+ *
+ * Also, clock gate and pll mux is not available on every div cell, so
+ * we have to deal with those, too. We reuse common clock composite driver
+ * for it.
+ */
+
+#define PLL_SELECT_MASK 0x7
+#define DIV_SELECT_MASK 0x7
+
+struct berlin2_div {
+ struct clk_hw hw;
+ void __iomem *base;
+ struct berlin2_div_map map;
+ spinlock_t *lock;
+};
+
+#define to_berlin2_div(hw) container_of(hw, struct berlin2_div, hw)
+
+static u8 clk_div[] = { 1, 2, 4, 6, 8, 12, 1, 1 };
+
+static int berlin2_div_is_enabled(struct clk_hw *hw)
+{
+ struct berlin2_div *div = to_berlin2_div(hw);
+ struct berlin2_div_map *map = &div->map;
+ u32 reg;
+
+ if (div->lock)
+ spin_lock(div->lock);
+
+ reg = readl_relaxed(div->base + map->gate_offs);
+ reg >>= map->gate_shift;
+
+ if (div->lock)
+ spin_unlock(div->lock);
+
+ return (reg & 0x1);
+}
+
+static int berlin2_div_enable(struct clk_hw *hw)
+{
+ struct berlin2_div *div = to_berlin2_div(hw);
+ struct berlin2_div_map *map = &div->map;
+ u32 reg;
+
+ if (div->lock)
+ spin_lock(div->lock);
+
+ reg = readl_relaxed(div->base + map->gate_offs);
+ reg |= BIT(map->gate_shift);
+ writel_relaxed(reg, div->base + map->gate_offs);
+
+ if (div->lock)
+ spin_unlock(div->lock);
+
+ return 0;
+}
+
+static void berlin2_div_disable(struct clk_hw *hw)
+{
+ struct berlin2_div *div = to_berlin2_div(hw);
+ struct berlin2_div_map *map = &div->map;
+ u32 reg;
+
+ if (div->lock)
+ spin_lock(div->lock);
+
+ reg = readl_relaxed(div->base + map->gate_offs);
+ reg &= ~BIT(map->gate_shift);
+ writel_relaxed(reg, div->base + map->gate_offs);
+
+ if (div->lock)
+ spin_unlock(div->lock);
+}
+
+static int berlin2_div_set_parent(struct clk_hw *hw, u8 index)
+{
+ struct berlin2_div *div = to_berlin2_div(hw);
+ struct berlin2_div_map *map = &div->map;
+ u32 reg;
+
+ if (div->lock)
+ spin_lock(div->lock);
+
+ /* index == 0 is PLL_SWITCH */
+ reg = readl_relaxed(div->base + map->pll_switch_offs);
+ if (index == 0)
+ reg &= ~BIT(map->pll_switch_shift);
+ else
+ reg |= BIT(map->pll_switch_shift);
+ writel_relaxed(reg, div->base + map->pll_switch_offs);
+
+ /* index > 0 is PLL_SELECT */
+ if (index > 0) {
+ reg = readl_relaxed(div->base + map->pll_select_offs);
+ reg &= ~(PLL_SELECT_MASK << map->pll_select_shift);
+ reg |= (index - 1) << map->pll_select_shift;
+ writel_relaxed(reg, div->base + map->pll_select_offs);
+ }
+
+ if (div->lock)
+ spin_unlock(div->lock);
+
+ return 0;
+}
+
+static u8 berlin2_div_get_parent(struct clk_hw *hw)
+{
+ struct berlin2_div *div = to_berlin2_div(hw);
+ struct berlin2_div_map *map = &div->map;
+ u32 reg;
+ u8 index = 0;
+
+ if (div->lock)
+ spin_lock(div->lock);
+
+ /* PLL_SWITCH == 0 is index 0 */
+ reg = readl_relaxed(div->base + map->pll_switch_offs);
+ reg &= BIT(map->pll_switch_shift);
+ if (reg) {
+ reg = readl_relaxed(div->base + map->pll_select_offs);
+ reg >>= map->pll_select_shift;
+ reg &= PLL_SELECT_MASK;
+ index = 1 + reg;
+ }
+
+ if (div->lock)
+ spin_unlock(div->lock);
+
+ return index;
+}
+
+static unsigned long berlin2_div_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct berlin2_div *div = to_berlin2_div(hw);
+ struct berlin2_div_map *map = &div->map;
+ u32 divsw, div3sw, divider = 1;
+
+ if (div->lock)
+ spin_lock(div->lock);
+
+ divsw = readl_relaxed(div->base + map->div_switch_offs) &
+ (1 << map->div_switch_shift);
+ div3sw = readl_relaxed(div->base + map->div3_switch_offs) &
+ (1 << map->div3_switch_shift);
+
+ /* constant divide-by-3 (dominant) */
+ if (div3sw != 0) {
+ divider = 3;
+ /* divider can be bypassed with DIV_SWITCH == 0 */
+ } else if (divsw == 0) {
+ divider = 1;
+ /* clock divider determined by DIV_SELECT */
+ } else {
+ u32 reg;
+ reg = readl_relaxed(div->base + map->div_select_offs);
+ reg >>= map->div_select_shift;
+ reg &= DIV_SELECT_MASK;
+ divider = clk_div[reg];
+ }
+
+ if (div->lock)
+ spin_unlock(div->lock);
+
+ return parent_rate / divider;
+}
+
+static const struct clk_ops berlin2_div_rate_ops = {
+ .recalc_rate = berlin2_div_recalc_rate,
+};
+
+static const struct clk_ops berlin2_div_gate_ops = {
+ .is_enabled = berlin2_div_is_enabled,
+ .enable = berlin2_div_enable,
+ .disable = berlin2_div_disable,
+};
+
+static const struct clk_ops berlin2_div_mux_ops = {
+ .set_parent = berlin2_div_set_parent,
+ .get_parent = berlin2_div_get_parent,
+};
+
+struct clk * __init
+berlin2_div_register(const struct berlin2_div_map *map,
+ void __iomem *base, const char *name, u8 div_flags,
+ const char **parent_names, int num_parents,
+ unsigned long flags, spinlock_t *lock)
+{
+ const struct clk_ops *mux_ops = &berlin2_div_mux_ops;
+ const struct clk_ops *rate_ops = &berlin2_div_rate_ops;
+ const struct clk_ops *gate_ops = &berlin2_div_gate_ops;
+ struct berlin2_div *div;
+
+ div = kzalloc(sizeof(*div), GFP_KERNEL);
+ if (!div)
+ return ERR_PTR(-ENOMEM);
+
+ /* copy div_map to allow __initconst */
+ memcpy(&div->map, map, sizeof(*map));
+ div->base = base;
+ div->lock = lock;
+
+ if ((div_flags & BERLIN2_DIV_HAS_GATE) == 0)
+ gate_ops = NULL;
+ if ((div_flags & BERLIN2_DIV_HAS_MUX) == 0)
+ mux_ops = NULL;
+
+ return clk_register_composite(NULL, name, parent_names, num_parents,
+ &div->hw, mux_ops, &div->hw, rate_ops,
+ &div->hw, gate_ops, flags);
+}
diff --git a/drivers/clk/berlin/berlin2-div.h b/drivers/clk/berlin/berlin2-div.h
new file mode 100644
index 0000000..15e3384
--- /dev/null
+++ b/drivers/clk/berlin/berlin2-div.h
@@ -0,0 +1,89 @@
+/*
+ * Copyright (c) 2014 Marvell Technology Group Ltd.
+ *
+ * Alexandre Belloni <alexandre.belloni@free-electrons.com>
+ * Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __BERLIN2_DIV_H
+#define __BERLIN2_DIV_H
+
+struct clk;
+
+#define BERLIN2_DIV_HAS_GATE BIT(0)
+#define BERLIN2_DIV_HAS_MUX BIT(1)
+
+#define BERLIN2_PLL_SELECT(_off, _sh) \
+ .pll_select_offs = _off, \
+ .pll_select_shift = _sh
+
+#define BERLIN2_PLL_SWITCH(_off, _sh) \
+ .pll_switch_offs = _off, \
+ .pll_switch_shift = _sh
+
+#define BERLIN2_DIV_SELECT(_off, _sh) \
+ .div_select_offs = _off, \
+ .div_select_shift = _sh
+
+#define BERLIN2_DIV_SWITCH(_off, _sh) \
+ .div_switch_offs = _off, \
+ .div_switch_shift = _sh
+
+#define BERLIN2_DIV_D3SWITCH(_off, _sh) \
+ .div3_switch_offs = _off, \
+ .div3_switch_shift = _sh
+
+#define BERLIN2_DIV_GATE(_off, _sh) \
+ .gate_offs = _off, \
+ .gate_shift = _sh
+
+#define BERLIN2_SINGLE_DIV(_off) \
+ BERLIN2_DIV_GATE(_off, 0), \
+ BERLIN2_PLL_SELECT(_off, 1), \
+ BERLIN2_PLL_SWITCH(_off, 4), \
+ BERLIN2_DIV_SWITCH(_off, 5), \
+ BERLIN2_DIV_D3SWITCH(_off, 6), \
+ BERLIN2_DIV_SELECT(_off, 7)
+
+struct berlin2_div_map {
+ u16 pll_select_offs;
+ u16 pll_switch_offs;
+ u16 div_select_offs;
+ u16 div_switch_offs;
+ u16 div3_switch_offs;
+ u16 gate_offs;
+ u8 pll_select_shift;
+ u8 pll_switch_shift;
+ u8 div_select_shift;
+ u8 div_switch_shift;
+ u8 div3_switch_shift;
+ u8 gate_shift;
+};
+
+struct berlin2_div_data {
+ const char *name;
+ const u8 *parent_ids;
+ int num_parents;
+ unsigned long flags;
+ struct berlin2_div_map map;
+ u8 div_flags;
+};
+
+struct clk * __init
+berlin2_div_register(const struct berlin2_div_map *map,
+ void __iomem *base, const char *name, u8 div_flags,
+ const char **parent_names, int num_parents,
+ unsigned long flags, spinlock_t *lock);
+
+#endif /* __BERLIN2_DIV_H */
diff --git a/drivers/clk/berlin/berlin2-pll.c b/drivers/clk/berlin/berlin2-pll.c
new file mode 100644
index 0000000..bdc506b
--- /dev/null
+++ b/drivers/clk/berlin/berlin2-pll.c
@@ -0,0 +1,117 @@
+/*
+ * Copyright (c) 2014 Marvell Technology Group Ltd.
+ *
+ * Alexandre Belloni <alexandre.belloni@free-electrons.com>
+ * Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <linux/clk-provider.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+#include <asm/div64.h>
+
+#include "berlin2-div.h"
+
+struct berlin2_pll_map {
+ const u8 vcodiv[16];
+ u8 mult;
+ u8 fbdiv_shift;
+ u8 rfdiv_shift;
+ u8 divsel_shift;
+};
+
+struct berlin2_pll {
+ struct clk_hw hw;
+ void __iomem *base;
+ struct berlin2_pll_map map;
+};
+
+#define to_berlin2_pll(hw) container_of(hw, struct berlin2_pll, hw)
+
+#define SPLL_CTRL0 0x00
+#define SPLL_CTRL1 0x04
+#define SPLL_CTRL2 0x08
+#define SPLL_CTRL3 0x0c
+#define SPLL_CTRL4 0x10
+
+#define FBDIV_MASK 0x1ff
+#define RFDIV_MASK 0x1f
+#define DIVSEL_MASK 0xf
+
+/*
+ * The output frequency formula for the pll is:
+ * clkout = fbdiv / refdiv * parent / vcodiv
+ */
+static unsigned long
+berlin2_pll_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
+{
+ struct berlin2_pll *pll = to_berlin2_pll(hw);
+ struct berlin2_pll_map *map = &pll->map;
+ u32 val, fbdiv, rfdiv, vcodivsel, vcodiv;
+ u64 rate = parent_rate;
+
+ val = readl_relaxed(pll->base + SPLL_CTRL0);
+ fbdiv = (val >> map->fbdiv_shift) & FBDIV_MASK;
+ rfdiv = (val >> map->rfdiv_shift) & RFDIV_MASK;
+ if (rfdiv == 0) {
+ pr_warn("%s has zero rfdiv\n", __clk_get_name(hw->clk));
+ rfdiv = 1;
+ }
+
+ val = readl_relaxed(pll->base + SPLL_CTRL1);
+ vcodivsel = (val >> map->divsel_shift) & DIVSEL_MASK;
+ vcodiv = map->vcodiv[vcodivsel];
+ if (vcodiv == 0) {
+ pr_warn("%s has zero vcodiv (index %d)\n",
+ __clk_get_name(hw->clk), vcodivsel);
+ vcodiv = 1;
+ }
+
+ rate *= fbdiv * map->mult;
+ do_div(rate, rfdiv * vcodiv);
+
+ return (unsigned long)rate;
+}
+
+static const struct clk_ops berlin2_pll_ops = {
+ .recalc_rate = berlin2_pll_recalc_rate,
+};
+
+struct clk * __init
+berlin2_pll_register(const struct berlin2_pll_map *map,
+ void __iomem *base, const char *name,
+ const char *parent_name, unsigned long flags)
+{
+ struct clk_init_data init;
+ struct berlin2_pll *pll;
+
+ pll = kzalloc(sizeof(*pll), GFP_KERNEL);
+ if (!pll)
+ return ERR_PTR(-ENOMEM);
+
+ /* copy pll_map to allow __initconst */
+ memcpy(&pll->map, map, sizeof(*map));
+ pll->base = base;
+ pll->hw.init = &init;
+ init.name = name;
+ init.ops = &berlin2_pll_ops;
+ init.parent_names = &parent_name;
+ init.num_parents = 1;
+ init.flags = flags;
+
+ return clk_register(NULL, &pll->hw);
+}
diff --git a/drivers/clk/berlin/berlin2-pll.h b/drivers/clk/berlin/berlin2-pll.h
new file mode 100644
index 0000000..8831ce2
--- /dev/null
+++ b/drivers/clk/berlin/berlin2-pll.h
@@ -0,0 +1,37 @@
+/*
+ * Copyright (c) 2014 Marvell Technology Group Ltd.
+ *
+ * Alexandre Belloni <alexandre.belloni@free-electrons.com>
+ * Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __BERLIN2_PLL_H
+#define __BERLIN2_PLL_H
+
+struct clk;
+
+struct berlin2_pll_map {
+ const u8 vcodiv[16];
+ u8 mult;
+ u8 fbdiv_shift;
+ u8 rfdiv_shift;
+ u8 divsel_shift;
+};
+
+struct clk * __init
+berlin2_pll_register(const struct berlin2_pll_map *map,
+ void __iomem *base, const char *name,
+ const char *parent_name, unsigned long flags);
+
+#endif /* __BERLIN2_PLL_H */
diff --git a/drivers/clk/berlin/bg2.c b/drivers/clk/berlin/bg2.c
new file mode 100644
index 0000000..515fb13
--- /dev/null
+++ b/drivers/clk/berlin/bg2.c
@@ -0,0 +1,691 @@
+/*
+ * Copyright (c) 2014 Marvell Technology Group Ltd.
+ *
+ * Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
+ * Alexandre Belloni <alexandre.belloni@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/kernel.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+
+#include <dt-bindings/clock/berlin2.h>
+
+#include "berlin2-avpll.h"
+#include "berlin2-div.h"
+#include "berlin2-pll.h"
+#include "common.h"
+
+#define REG_PINMUX0 0x0000
+#define REG_PINMUX1 0x0004
+#define REG_SYSPLLCTL0 0x0014
+#define REG_SYSPLLCTL4 0x0024
+#define REG_MEMPLLCTL0 0x0028
+#define REG_MEMPLLCTL4 0x0038
+#define REG_CPUPLLCTL0 0x003c
+#define REG_CPUPLLCTL4 0x004c
+#define REG_AVPLLCTL0 0x0050
+#define REG_AVPLLCTL31 0x00cc
+#define REG_AVPLLCTL62 0x0148
+#define REG_PLLSTATUS 0x014c
+#define REG_CLKENABLE 0x0150
+#define REG_CLKSELECT0 0x0154
+#define REG_CLKSELECT1 0x0158
+#define REG_CLKSELECT2 0x015c
+#define REG_CLKSELECT3 0x0160
+#define REG_CLKSWITCH0 0x0164
+#define REG_CLKSWITCH1 0x0168
+#define REG_RESET_TRIGGER 0x0178
+#define REG_RESET_STATUS0 0x017c
+#define REG_RESET_STATUS1 0x0180
+#define REG_SW_GENERIC0 0x0184
+#define REG_SW_GENERIC3 0x0190
+#define REG_PRODUCTID 0x01cc
+#define REG_PRODUCTID_EXT 0x01d0
+#define REG_GFX3DCORE_CLKCTL 0x022c
+#define REG_GFX3DSYS_CLKCTL 0x0230
+#define REG_ARC_CLKCTL 0x0234
+#define REG_VIP_CLKCTL 0x0238
+#define REG_SDIO0XIN_CLKCTL 0x023c
+#define REG_SDIO1XIN_CLKCTL 0x0240
+#define REG_GFX3DEXTRA_CLKCTL 0x0244
+#define REG_GFX3D_RESET 0x0248
+#define REG_GC360_CLKCTL 0x024c
+#define REG_SDIO_DLLMST_CLKCTL 0x0250
+
+/*
+ * BG2/BG2CD SoCs have the following audio/video I/O units:
+ *
+ * audiohd: HDMI TX audio
+ * audio0: 7.1ch TX
+ * audio1: 2ch TX
+ * audio2: 2ch RX
+ * audio3: SPDIF TX
+ * video0: HDMI video
+ * video1: Secondary video
+ * video2: SD auxiliary video
+ *
+ * There are no external audio clocks (ACLKI0, ACLKI1) and
+ * only one external video clock (VCLKI0).
+ *
+ * Currently missing bits and pieces:
+ * - audio_fast_pll is unknown
+ * - audiohd_pll is unknown
+ * - video0_pll is unknown
+ * - audio[023], audiohd parent pll is assumed to be audio_fast_pll
+ *
+ */
+
+#define MAX_CLKS 41
+static struct clk *clks[MAX_CLKS];
+static struct clk_onecell_data clk_data;
+static DEFINE_SPINLOCK(lock);
+static void __iomem *gbase;
+
+enum {
+ REFCLK, VIDEO_EXT0,
+ SYSPLL, MEMPLL, CPUPLL,
+ AVPLL_A1, AVPLL_A2, AVPLL_A3, AVPLL_A4,
+ AVPLL_A5, AVPLL_A6, AVPLL_A7, AVPLL_A8,
+ AVPLL_B1, AVPLL_B2, AVPLL_B3, AVPLL_B4,
+ AVPLL_B5, AVPLL_B6, AVPLL_B7, AVPLL_B8,
+ AUDIO1_PLL, AUDIO_FAST_PLL,
+ VIDEO0_PLL, VIDEO0_IN,
+ VIDEO1_PLL, VIDEO1_IN,
+ VIDEO2_PLL, VIDEO2_IN,
+};
+
+static const char *clk_names[] = {
+ [REFCLK] = "refclk",
+ [VIDEO_EXT0] = "video_ext0",
+ [SYSPLL] = "syspll",
+ [MEMPLL] = "mempll",
+ [CPUPLL] = "cpupll",
+ [AVPLL_A1] = "avpll_a1",
+ [AVPLL_A2] = "avpll_a2",
+ [AVPLL_A3] = "avpll_a3",
+ [AVPLL_A4] = "avpll_a4",
+ [AVPLL_A5] = "avpll_a5",
+ [AVPLL_A6] = "avpll_a6",
+ [AVPLL_A7] = "avpll_a7",
+ [AVPLL_A8] = "avpll_a8",
+ [AVPLL_B1] = "avpll_b1",
+ [AVPLL_B2] = "avpll_b2",
+ [AVPLL_B3] = "avpll_b3",
+ [AVPLL_B4] = "avpll_b4",
+ [AVPLL_B5] = "avpll_b5",
+ [AVPLL_B6] = "avpll_b6",
+ [AVPLL_B7] = "avpll_b7",
+ [AVPLL_B8] = "avpll_b8",
+ [AUDIO1_PLL] = "audio1_pll",
+ [AUDIO_FAST_PLL] = "audio_fast_pll",
+ [VIDEO0_PLL] = "video0_pll",
+ [VIDEO0_IN] = "video0_in",
+ [VIDEO1_PLL] = "video1_pll",
+ [VIDEO1_IN] = "video1_in",
+ [VIDEO2_PLL] = "video2_pll",
+ [VIDEO2_IN] = "video2_in",
+};
+
+static const struct berlin2_pll_map bg2_pll_map __initconst = {
+ .vcodiv = {10, 15, 20, 25, 30, 40, 50, 60, 80},
+ .mult = 10,
+ .fbdiv_shift = 6,
+ .rfdiv_shift = 1,
+ .divsel_shift = 7,
+};
+
+static const u8 default_parent_ids[] = {
+ SYSPLL, AVPLL_B4, AVPLL_A5, AVPLL_B6, AVPLL_B7, SYSPLL
+};
+
+static const struct berlin2_div_data bg2_divs[] __initconst = {
+ {
+ .name = "sys",
+ .parent_ids = (const u8 []){
+ SYSPLL, AVPLL_B4, AVPLL_B5, AVPLL_B6, AVPLL_B7, SYSPLL
+ },
+ .num_parents = 6,
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 0),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT0, 0),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT0, 3),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 3),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 4),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 5),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = CLK_IGNORE_UNUSED,
+ },
+ {
+ .name = "cpu",
+ .parent_ids = (const u8 []){
+ CPUPLL, MEMPLL, MEMPLL, MEMPLL, MEMPLL
+ },
+ .num_parents = 5,
+ .map = {
+ BERLIN2_PLL_SELECT(REG_CLKSELECT0, 6),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT0, 9),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 6),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 7),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 8),
+ },
+ .div_flags = BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "drmfigo",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 16),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT0, 17),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT0, 20),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 12),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 13),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 14),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "cfg",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 1),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT0, 23),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT0, 26),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 15),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 16),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 17),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "gfx",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 4),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT0, 29),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT1, 0),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 18),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 19),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 20),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "zsp",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 5),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT1, 3),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT1, 6),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 21),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 22),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 23),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "perif",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 6),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT1, 9),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT1, 12),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 24),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 25),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 26),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = CLK_IGNORE_UNUSED,
+ },
+ {
+ .name = "pcube",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 2),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT1, 15),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT1, 18),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 27),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 28),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 29),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "vscope",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 3),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT1, 21),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT1, 24),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 30),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 31),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH1, 0),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "nfc_ecc",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 18),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT1, 27),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT2, 0),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH1, 1),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH1, 2),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH1, 3),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "vpp",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 21),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT2, 3),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT2, 6),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH1, 4),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH1, 5),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH1, 6),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "app",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 20),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT2, 9),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT2, 12),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH1, 7),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH1, 8),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH1, 9),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "audio0",
+ .parent_ids = (const u8 []){ AUDIO_FAST_PLL },
+ .num_parents = 1,
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 22),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT2, 17),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH1, 10),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH1, 11),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE,
+ .flags = 0,
+ },
+ {
+ .name = "audio2",
+ .parent_ids = (const u8 []){ AUDIO_FAST_PLL },
+ .num_parents = 1,
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 24),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT2, 20),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH1, 14),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH1, 15),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE,
+ .flags = 0,
+ },
+ {
+ .name = "audio3",
+ .parent_ids = (const u8 []){ AUDIO_FAST_PLL },
+ .num_parents = 1,
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 25),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT2, 23),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH1, 16),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH1, 17),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE,
+ .flags = 0,
+ },
+ {
+ .name = "audio1",
+ .parent_ids = (const u8 []){ AUDIO1_PLL },
+ .num_parents = 1,
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 23),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT3, 0),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH1, 12),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH1, 13),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE,
+ .flags = 0,
+ },
+ {
+ .name = "gfx3d_core",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_SINGLE_DIV(REG_GFX3DCORE_CLKCTL),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "gfx3d_sys",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_SINGLE_DIV(REG_GFX3DSYS_CLKCTL),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "arc",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_SINGLE_DIV(REG_ARC_CLKCTL),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "vip",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_SINGLE_DIV(REG_VIP_CLKCTL),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "sdio0xin",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_SINGLE_DIV(REG_SDIO0XIN_CLKCTL),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "sdio1xin",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_SINGLE_DIV(REG_SDIO1XIN_CLKCTL),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "gfx3d_extra",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_SINGLE_DIV(REG_GFX3DEXTRA_CLKCTL),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "gc360",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_SINGLE_DIV(REG_GC360_CLKCTL),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "sdio_dllmst",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_SINGLE_DIV(REG_SDIO_DLLMST_CLKCTL),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+};
+
+static const struct berlin2_gate_data bg2_gates[] __initconst = {
+ { "geth0", "perif", 7 },
+ { "geth1", "perif", 8 },
+ { "sata", "perif", 9 },
+ { "ahbapb", "perif", 10, CLK_IGNORE_UNUSED },
+ { "usb0", "perif", 11 },
+ { "usb1", "perif", 12 },
+ { "pbridge", "perif", 13, CLK_IGNORE_UNUSED },
+ { "sdio0", "perif", 14, CLK_IGNORE_UNUSED },
+ { "sdio1", "perif", 15, CLK_IGNORE_UNUSED },
+ { "nfc", "perif", 17 },
+ { "smemc", "perif", 19 },
+ { "audiohd", "audiohd_pll", 26 },
+ { "video0", "video0_in", 27 },
+ { "video1", "video1_in", 28 },
+ { "video2", "video2_in", 29 },
+};
+
+static void __init berlin2_clock_setup(struct device_node *np)
+{
+ const char *parent_names[9];
+ struct clk *clk;
+ u8 avpll_flags = 0;
+ int n;
+
+ gbase = of_iomap(np, 0);
+ if (!gbase)
+ return;
+
+ /* overwrite default clock names with DT provided ones */
+ clk = of_clk_get_by_name(np, clk_names[REFCLK]);
+ if (!IS_ERR(clk)) {
+ clk_names[REFCLK] = __clk_get_name(clk);
+ clk_put(clk);
+ }
+
+ clk = of_clk_get_by_name(np, clk_names[VIDEO_EXT0]);
+ if (!IS_ERR(clk)) {
+ clk_names[VIDEO_EXT0] = __clk_get_name(clk);
+ clk_put(clk);
+ }
+
+ /* simple register PLLs */
+ clk = berlin2_pll_register(&bg2_pll_map, gbase + REG_SYSPLLCTL0,
+ clk_names[SYSPLL], clk_names[REFCLK], 0);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+
+ clk = berlin2_pll_register(&bg2_pll_map, gbase + REG_MEMPLLCTL0,
+ clk_names[MEMPLL], clk_names[REFCLK], 0);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+
+ clk = berlin2_pll_register(&bg2_pll_map, gbase + REG_CPUPLLCTL0,
+ clk_names[CPUPLL], clk_names[REFCLK], 0);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+
+ if (of_device_is_compatible(np, "marvell,berlin2-global-register"))
+ avpll_flags |= BERLIN2_AVPLL_SCRAMBLE_QUIRK;
+
+ /* audio/video VCOs */
+ clk = berlin2_avpll_vco_register(gbase + REG_AVPLLCTL0, "avpll_vcoA",
+ clk_names[REFCLK], avpll_flags, 0);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+
+ for (n = 0; n < 8; n++) {
+ clk = berlin2_avpll_channel_register(gbase + REG_AVPLLCTL0,
+ clk_names[AVPLL_A1 + n], n, "avpll_vcoA",
+ avpll_flags, 0);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+ }
+
+ clk = berlin2_avpll_vco_register(gbase + REG_AVPLLCTL31, "avpll_vcoB",
+ clk_names[REFCLK], BERLIN2_AVPLL_BIT_QUIRK |
+ avpll_flags, 0);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+
+ for (n = 0; n < 8; n++) {
+ clk = berlin2_avpll_channel_register(gbase + REG_AVPLLCTL31,
+ clk_names[AVPLL_B1 + n], n, "avpll_vcoB",
+ BERLIN2_AVPLL_BIT_QUIRK | avpll_flags, 0);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+ }
+
+ /* reference clock bypass switches */
+ parent_names[0] = clk_names[SYSPLL];
+ parent_names[1] = clk_names[REFCLK];
+ clk = clk_register_mux(NULL, "syspll_byp", parent_names, 2,
+ 0, gbase + REG_CLKSWITCH0, 0, 1, 0, &lock);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+ clk_names[SYSPLL] = __clk_get_name(clk);
+
+ parent_names[0] = clk_names[MEMPLL];
+ parent_names[1] = clk_names[REFCLK];
+ clk = clk_register_mux(NULL, "mempll_byp", parent_names, 2,
+ 0, gbase + REG_CLKSWITCH0, 1, 1, 0, &lock);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+ clk_names[MEMPLL] = __clk_get_name(clk);
+
+ parent_names[0] = clk_names[CPUPLL];
+ parent_names[1] = clk_names[REFCLK];
+ clk = clk_register_mux(NULL, "cpupll_byp", parent_names, 2,
+ 0, gbase + REG_CLKSWITCH0, 2, 1, 0, &lock);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+ clk_names[CPUPLL] = __clk_get_name(clk);
+
+ /* clock muxes */
+ parent_names[0] = clk_names[AVPLL_B3];
+ parent_names[1] = clk_names[AVPLL_A3];
+ clk = clk_register_mux(NULL, clk_names[AUDIO1_PLL], parent_names, 2,
+ 0, gbase + REG_CLKSELECT2, 29, 1, 0, &lock);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+
+ parent_names[0] = clk_names[VIDEO0_PLL];
+ parent_names[1] = clk_names[VIDEO_EXT0];
+ clk = clk_register_mux(NULL, clk_names[VIDEO0_IN], parent_names, 2,
+ 0, gbase + REG_CLKSELECT3, 4, 1, 0, &lock);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+
+ parent_names[0] = clk_names[VIDEO1_PLL];
+ parent_names[1] = clk_names[VIDEO_EXT0];
+ clk = clk_register_mux(NULL, clk_names[VIDEO1_IN], parent_names, 2,
+ 0, gbase + REG_CLKSELECT3, 6, 1, 0, &lock);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+
+ parent_names[0] = clk_names[AVPLL_A2];
+ parent_names[1] = clk_names[AVPLL_B2];
+ clk = clk_register_mux(NULL, clk_names[VIDEO1_PLL], parent_names, 2,
+ 0, gbase + REG_CLKSELECT3, 7, 1, 0, &lock);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+
+ parent_names[0] = clk_names[VIDEO2_PLL];
+ parent_names[1] = clk_names[VIDEO_EXT0];
+ clk = clk_register_mux(NULL, clk_names[VIDEO2_IN], parent_names, 2,
+ 0, gbase + REG_CLKSELECT3, 9, 1, 0, &lock);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+
+ parent_names[0] = clk_names[AVPLL_B1];
+ parent_names[1] = clk_names[AVPLL_A5];
+ clk = clk_register_mux(NULL, clk_names[VIDEO2_PLL], parent_names, 2,
+ 0, gbase + REG_CLKSELECT3, 10, 1, 0, &lock);
+ if (IS_ERR(clk))
+ goto bg2_fail;
+
+ /* clock divider cells */
+ for (n = 0; n < ARRAY_SIZE(bg2_divs); n++) {
+ const struct berlin2_div_data *dd = &bg2_divs[n];
+ int k;
+
+ for (k = 0; k < dd->num_parents; k++)
+ parent_names[k] = clk_names[dd->parent_ids[k]];
+
+ clks[CLKID_SYS + n] = berlin2_div_register(&dd->map, gbase,
+ dd->name, dd->div_flags, parent_names,
+ dd->num_parents, dd->flags, &lock);
+ }
+
+ /* clock gate cells */
+ for (n = 0; n < ARRAY_SIZE(bg2_gates); n++) {
+ const struct berlin2_gate_data *gd = &bg2_gates[n];
+
+ clks[CLKID_GETH0 + n] = clk_register_gate(NULL, gd->name,
+ gd->parent_name, gd->flags, gbase + REG_CLKENABLE,
+ gd->bit_idx, 0, &lock);
+ }
+
+ /* twdclk is derived from cpu/3 */
+ clks[CLKID_TWD] =
+ clk_register_fixed_factor(NULL, "twd", "cpu", 0, 1, 3);
+
+ /* check for errors on leaf clocks */
+ for (n = 0; n < MAX_CLKS; n++) {
+ if (!IS_ERR(clks[n]))
+ continue;
+
+ pr_err("%s: Unable to register leaf clock %d\n",
+ np->full_name, n);
+ goto bg2_fail;
+ }
+
+ /* register clk-provider */
+ clk_data.clks = clks;
+ clk_data.clk_num = MAX_CLKS;
+ of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data);
+
+ return;
+
+bg2_fail:
+ iounmap(gbase);
+}
+CLK_OF_DECLARE(berlin2_clock, "marvell,berlin2-chip-ctrl",
+ berlin2_clock_setup);
+CLK_OF_DECLARE(berlin2cd_clock, "marvell,berlin2cd-chip-ctrl",
+ berlin2_clock_setup);
diff --git a/drivers/clk/berlin/bg2q.c b/drivers/clk/berlin/bg2q.c
new file mode 100644
index 0000000..21784e4
--- /dev/null
+++ b/drivers/clk/berlin/bg2q.c
@@ -0,0 +1,389 @@
+/*
+ * Copyright (c) 2014 Marvell Technology Group Ltd.
+ *
+ * Alexandre Belloni <alexandre.belloni@free-electrons.com>
+ * Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/kernel.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+
+#include <dt-bindings/clock/berlin2q.h>
+
+#include "berlin2-div.h"
+#include "berlin2-pll.h"
+#include "common.h"
+
+#define REG_PINMUX0 0x0018
+#define REG_PINMUX5 0x002c
+#define REG_SYSPLLCTL0 0x0030
+#define REG_SYSPLLCTL4 0x0040
+#define REG_CLKENABLE 0x00e8
+#define REG_CLKSELECT0 0x00ec
+#define REG_CLKSELECT1 0x00f0
+#define REG_CLKSELECT2 0x00f4
+#define REG_CLKSWITCH0 0x00f8
+#define REG_CLKSWITCH1 0x00fc
+#define REG_SW_GENERIC0 0x0110
+#define REG_SW_GENERIC3 0x011c
+#define REG_SDIO0XIN_CLKCTL 0x0158
+#define REG_SDIO1XIN_CLKCTL 0x015c
+
+#define MAX_CLKS 27
+static struct clk *clks[MAX_CLKS];
+static struct clk_onecell_data clk_data;
+static DEFINE_SPINLOCK(lock);
+static void __iomem *gbase;
+static void __iomem *cpupll_base;
+
+enum {
+ REFCLK,
+ SYSPLL, CPUPLL,
+ AVPLL_B1, AVPLL_B2, AVPLL_B3, AVPLL_B4,
+ AVPLL_B5, AVPLL_B6, AVPLL_B7, AVPLL_B8,
+};
+
+static const char *clk_names[] = {
+ [REFCLK] = "refclk",
+ [SYSPLL] = "syspll",
+ [CPUPLL] = "cpupll",
+ [AVPLL_B1] = "avpll_b1",
+ [AVPLL_B2] = "avpll_b2",
+ [AVPLL_B3] = "avpll_b3",
+ [AVPLL_B4] = "avpll_b4",
+ [AVPLL_B5] = "avpll_b5",
+ [AVPLL_B6] = "avpll_b6",
+ [AVPLL_B7] = "avpll_b7",
+ [AVPLL_B8] = "avpll_b8",
+};
+
+static const struct berlin2_pll_map bg2q_pll_map __initconst = {
+ .vcodiv = {1, 0, 2, 0, 3, 4, 0, 6, 8},
+ .mult = 1,
+ .fbdiv_shift = 7,
+ .rfdiv_shift = 2,
+ .divsel_shift = 9,
+};
+
+static const u8 default_parent_ids[] = {
+ SYSPLL, AVPLL_B4, AVPLL_B5, AVPLL_B6, AVPLL_B7, SYSPLL
+};
+
+static const struct berlin2_div_data bg2q_divs[] __initconst = {
+ {
+ .name = "sys",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 0),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT0, 0),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT0, 3),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 3),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 4),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 5),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = CLK_IGNORE_UNUSED,
+ },
+ {
+ .name = "drmfigo",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 17),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT0, 6),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT0, 9),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 6),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 7),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 8),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "cfg",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 1),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT0, 12),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT0, 15),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 9),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 10),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 11),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "gfx2d",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 4),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT0, 18),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT0, 21),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 12),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 13),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 14),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "zsp",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 6),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT0, 24),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT0, 27),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 15),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 16),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 17),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "perif",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 7),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT1, 0),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT1, 3),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 18),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 19),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 20),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = CLK_IGNORE_UNUSED,
+ },
+ {
+ .name = "pcube",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 2),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT1, 6),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT1, 9),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 21),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 22),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 23),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "vscope",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 3),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT1, 12),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT1, 15),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 24),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 25),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 26),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "nfc_ecc",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 19),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT1, 18),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT1, 21),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 27),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 28),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH0, 29),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "vpp",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 21),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT1, 24),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT1, 27),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH0, 30),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH0, 31),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH1, 0),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "app",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_DIV_GATE(REG_CLKENABLE, 20),
+ BERLIN2_PLL_SELECT(REG_CLKSELECT2, 0),
+ BERLIN2_DIV_SELECT(REG_CLKSELECT2, 3),
+ BERLIN2_PLL_SWITCH(REG_CLKSWITCH1, 1),
+ BERLIN2_DIV_SWITCH(REG_CLKSWITCH1, 2),
+ BERLIN2_DIV_D3SWITCH(REG_CLKSWITCH1, 3),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "sdio0xin",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_SINGLE_DIV(REG_SDIO0XIN_CLKCTL),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+ {
+ .name = "sdio1xin",
+ .parent_ids = default_parent_ids,
+ .num_parents = ARRAY_SIZE(default_parent_ids),
+ .map = {
+ BERLIN2_SINGLE_DIV(REG_SDIO1XIN_CLKCTL),
+ },
+ .div_flags = BERLIN2_DIV_HAS_GATE | BERLIN2_DIV_HAS_MUX,
+ .flags = 0,
+ },
+};
+
+static const struct berlin2_gate_data bg2q_gates[] __initconst = {
+ { "gfx2daxi", "perif", 5 },
+ { "geth0", "perif", 8 },
+ { "sata", "perif", 9 },
+ { "ahbapb", "perif", 10, CLK_IGNORE_UNUSED },
+ { "usb0", "perif", 11 },
+ { "usb1", "perif", 12 },
+ { "usb2", "perif", 13 },
+ { "usb3", "perif", 14 },
+ { "pbridge", "perif", 15, CLK_IGNORE_UNUSED },
+ { "sdio", "perif", 16, CLK_IGNORE_UNUSED },
+ { "nfc", "perif", 18 },
+ { "smemc", "perif", 19 },
+ { "pcie", "perif", 22 },
+};
+
+static void __init berlin2q_clock_setup(struct device_node *np)
+{
+ const char *parent_names[9];
+ struct clk *clk;
+ int n;
+
+ gbase = of_iomap(np, 0);
+ if (!gbase) {
+ pr_err("%s: Unable to map global base\n", np->full_name);
+ return;
+ }
+
+ /* BG2Q CPU PLL is not part of global registers */
+ cpupll_base = of_iomap(np, 1);
+ if (!cpupll_base) {
+ pr_err("%s: Unable to map cpupll base\n", np->full_name);
+ iounmap(gbase);
+ return;
+ }
+
+ /* overwrite default clock names with DT provided ones */
+ clk = of_clk_get_by_name(np, clk_names[REFCLK]);
+ if (!IS_ERR(clk)) {
+ clk_names[REFCLK] = __clk_get_name(clk);
+ clk_put(clk);
+ }
+
+ /* simple register PLLs */
+ clk = berlin2_pll_register(&bg2q_pll_map, gbase + REG_SYSPLLCTL0,
+ clk_names[SYSPLL], clk_names[REFCLK], 0);
+ if (IS_ERR(clk))
+ goto bg2q_fail;
+
+ clk = berlin2_pll_register(&bg2q_pll_map, cpupll_base,
+ clk_names[CPUPLL], clk_names[REFCLK], 0);
+ if (IS_ERR(clk))
+ goto bg2q_fail;
+
+ /* TODO: add BG2Q AVPLL */
+
+ /*
+ * TODO: add reference clock bypass switches:
+ * memPLLSWBypass, cpuPLLSWBypass, and sysPLLSWBypass
+ */
+
+ /* clock divider cells */
+ for (n = 0; n < ARRAY_SIZE(bg2q_divs); n++) {
+ const struct berlin2_div_data *dd = &bg2q_divs[n];
+ int k;
+
+ for (k = 0; k < dd->num_parents; k++)
+ parent_names[k] = clk_names[dd->parent_ids[k]];
+
+ clks[CLKID_SYS + n] = berlin2_div_register(&dd->map, gbase,
+ dd->name, dd->div_flags, parent_names,
+ dd->num_parents, dd->flags, &lock);
+ }
+
+ /* clock gate cells */
+ for (n = 0; n < ARRAY_SIZE(bg2q_gates); n++) {
+ const struct berlin2_gate_data *gd = &bg2q_gates[n];
+
+ clks[CLKID_GFX2DAXI + n] = clk_register_gate(NULL, gd->name,
+ gd->parent_name, gd->flags, gbase + REG_CLKENABLE,
+ gd->bit_idx, 0, &lock);
+ }
+
+ /*
+ * twdclk is derived from cpu/3
+ * TODO: use cpupll until cpuclk is not available
+ */
+ clks[CLKID_TWD] =
+ clk_register_fixed_factor(NULL, "twd", clk_names[CPUPLL],
+ 0, 1, 3);
+
+ /* check for errors on leaf clocks */
+ for (n = 0; n < MAX_CLKS; n++) {
+ if (!IS_ERR(clks[n]))
+ continue;
+
+ pr_err("%s: Unable to register leaf clock %d\n",
+ np->full_name, n);
+ goto bg2q_fail;
+ }
+
+ /* register clk-provider */
+ clk_data.clks = clks;
+ clk_data.clk_num = MAX_CLKS;
+ of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data);
+
+ return;
+
+bg2q_fail:
+ iounmap(cpupll_base);
+ iounmap(gbase);
+}
+CLK_OF_DECLARE(berlin2q_clock, "marvell,berlin2q-chip-ctrl",
+ berlin2q_clock_setup);
diff --git a/drivers/clk/berlin/common.h b/drivers/clk/berlin/common.h
new file mode 100644
index 0000000..bc68a14
--- /dev/null
+++ b/drivers/clk/berlin/common.h
@@ -0,0 +1,29 @@
+/*
+ * Copyright (c) 2014 Marvell Technology Group Ltd.
+ *
+ * Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
+ * Alexandre Belloni <alexandre.belloni@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __BERLIN2_COMMON_H
+#define __BERLIN2_COMMON_H
+
+struct berlin2_gate_data {
+ const char *name;
+ const char *parent_name;
+ u8 bit_idx;
+ unsigned long flags;
+};
+
+#endif /* BERLIN2_COMMON_H */
diff --git a/drivers/clk/clk-axm5516.c b/drivers/clk/clk-axm5516.c
new file mode 100644
index 0000000..d2f1e11
--- /dev/null
+++ b/drivers/clk/clk-axm5516.c
@@ -0,0 +1,615 @@
+/*
+ * drivers/clk/clk-axm5516.c
+ *
+ * Provides clock implementations for three different types of clock devices on
+ * the Axxia device: PLL clock, a clock divider and a clock mux.
+ *
+ * Copyright (C) 2014 LSI Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ */
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/clk-provider.h>
+#include <linux/regmap.h>
+#include <dt-bindings/clock/lsi,axm5516-clks.h>
+
+
+/**
+ * struct axxia_clk - Common struct to all Axxia clocks.
+ * @hw: clk_hw for the common clk framework
+ * @regmap: Regmap for the clock control registers
+ */
+struct axxia_clk {
+ struct clk_hw hw;
+ struct regmap *regmap;
+};
+#define to_axxia_clk(_hw) container_of(_hw, struct axxia_clk, hw)
+
+/**
+ * struct axxia_pllclk - Axxia PLL generated clock.
+ * @aclk: Common struct
+ * @reg: Offset into regmap for PLL control register
+ */
+struct axxia_pllclk {
+ struct axxia_clk aclk;
+ u32 reg;
+};
+#define to_axxia_pllclk(_aclk) container_of(_aclk, struct axxia_pllclk, aclk)
+
+/**
+ * axxia_pllclk_recalc - Calculate the PLL generated clock rate given the
+ * parent clock rate.
+ */
+static unsigned long
+axxia_pllclk_recalc(struct clk_hw *hw, unsigned long parent_rate)
+{
+ struct axxia_clk *aclk = to_axxia_clk(hw);
+ struct axxia_pllclk *pll = to_axxia_pllclk(aclk);
+ unsigned long rate, fbdiv, refdiv, postdiv;
+ u32 control;
+
+ regmap_read(aclk->regmap, pll->reg, &control);
+ postdiv = ((control >> 0) & 0xf) + 1;
+ fbdiv = ((control >> 4) & 0xfff) + 3;
+ refdiv = ((control >> 16) & 0x1f) + 1;
+ rate = (parent_rate / (refdiv * postdiv)) * fbdiv;
+
+ return rate;
+}
+
+static const struct clk_ops axxia_pllclk_ops = {
+ .recalc_rate = axxia_pllclk_recalc,
+};
+
+/**
+ * struct axxia_divclk - Axxia clock divider
+ * @aclk: Common struct
+ * @reg: Offset into regmap for PLL control register
+ * @shift: Bit position for divider value
+ * @width: Number of bits in divider value
+ */
+struct axxia_divclk {
+ struct axxia_clk aclk;
+ u32 reg;
+ u32 shift;
+ u32 width;
+};
+#define to_axxia_divclk(_aclk) container_of(_aclk, struct axxia_divclk, aclk)
+
+/**
+ * axxia_divclk_recalc_rate - Calculate clock divider output rage
+ */
+static unsigned long
+axxia_divclk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
+{
+ struct axxia_clk *aclk = to_axxia_clk(hw);
+ struct axxia_divclk *divclk = to_axxia_divclk(aclk);
+ u32 ctrl, div;
+
+ regmap_read(aclk->regmap, divclk->reg, &ctrl);
+ div = 1 + ((ctrl >> divclk->shift) & ((1 << divclk->width)-1));
+
+ return parent_rate / div;
+}
+
+static const struct clk_ops axxia_divclk_ops = {
+ .recalc_rate = axxia_divclk_recalc_rate,
+};
+
+/**
+ * struct axxia_clkmux - Axxia clock mux
+ * @aclk: Common struct
+ * @reg: Offset into regmap for PLL control register
+ * @shift: Bit position for selection value
+ * @width: Number of bits in selection value
+ */
+struct axxia_clkmux {
+ struct axxia_clk aclk;
+ u32 reg;
+ u32 shift;
+ u32 width;
+};
+#define to_axxia_clkmux(_aclk) container_of(_aclk, struct axxia_clkmux, aclk)
+
+/**
+ * axxia_clkmux_get_parent - Return the index of selected parent clock
+ */
+static u8 axxia_clkmux_get_parent(struct clk_hw *hw)
+{
+ struct axxia_clk *aclk = to_axxia_clk(hw);
+ struct axxia_clkmux *mux = to_axxia_clkmux(aclk);
+ u32 ctrl, parent;
+
+ regmap_read(aclk->regmap, mux->reg, &ctrl);
+ parent = (ctrl >> mux->shift) & ((1 << mux->width) - 1);
+
+ return (u8) parent;
+}
+
+static const struct clk_ops axxia_clkmux_ops = {
+ .get_parent = axxia_clkmux_get_parent,
+};
+
+
+/*
+ * PLLs
+ */
+
+static struct axxia_pllclk clk_fab_pll = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_fab_pll",
+ .parent_names = (const char *[]){
+ "clk_ref0"
+ },
+ .num_parents = 1,
+ .ops = &axxia_pllclk_ops,
+ },
+ .reg = 0x01800,
+};
+
+static struct axxia_pllclk clk_cpu_pll = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_cpu_pll",
+ .parent_names = (const char *[]){
+ "clk_ref0"
+ },
+ .num_parents = 1,
+ .ops = &axxia_pllclk_ops,
+ },
+ .reg = 0x02000,
+};
+
+static struct axxia_pllclk clk_sys_pll = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_sys_pll",
+ .parent_names = (const char *[]){
+ "clk_ref0"
+ },
+ .num_parents = 1,
+ .ops = &axxia_pllclk_ops,
+ },
+ .reg = 0x02800,
+};
+
+static struct axxia_pllclk clk_sm0_pll = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_sm0_pll",
+ .parent_names = (const char *[]){
+ "clk_ref2"
+ },
+ .num_parents = 1,
+ .ops = &axxia_pllclk_ops,
+ },
+ .reg = 0x03000,
+};
+
+static struct axxia_pllclk clk_sm1_pll = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_sm1_pll",
+ .parent_names = (const char *[]){
+ "clk_ref1"
+ },
+ .num_parents = 1,
+ .ops = &axxia_pllclk_ops,
+ },
+ .reg = 0x03800,
+};
+
+/*
+ * Clock dividers
+ */
+
+static struct axxia_divclk clk_cpu0_div = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_cpu0_div",
+ .parent_names = (const char *[]){
+ "clk_cpu_pll"
+ },
+ .num_parents = 1,
+ .ops = &axxia_divclk_ops,
+ },
+ .reg = 0x10008,
+ .shift = 0,
+ .width = 4,
+};
+
+static struct axxia_divclk clk_cpu1_div = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_cpu1_div",
+ .parent_names = (const char *[]){
+ "clk_cpu_pll"
+ },
+ .num_parents = 1,
+ .ops = &axxia_divclk_ops,
+ },
+ .reg = 0x10008,
+ .shift = 4,
+ .width = 4,
+};
+
+static struct axxia_divclk clk_cpu2_div = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_cpu2_div",
+ .parent_names = (const char *[]){
+ "clk_cpu_pll"
+ },
+ .num_parents = 1,
+ .ops = &axxia_divclk_ops,
+ },
+ .reg = 0x10008,
+ .shift = 8,
+ .width = 4,
+};
+
+static struct axxia_divclk clk_cpu3_div = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_cpu3_div",
+ .parent_names = (const char *[]){
+ "clk_cpu_pll"
+ },
+ .num_parents = 1,
+ .ops = &axxia_divclk_ops,
+ },
+ .reg = 0x10008,
+ .shift = 12,
+ .width = 4,
+};
+
+static struct axxia_divclk clk_nrcp_div = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_nrcp_div",
+ .parent_names = (const char *[]){
+ "clk_sys_pll"
+ },
+ .num_parents = 1,
+ .ops = &axxia_divclk_ops,
+ },
+ .reg = 0x1000c,
+ .shift = 0,
+ .width = 4,
+};
+
+static struct axxia_divclk clk_sys_div = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_sys_div",
+ .parent_names = (const char *[]){
+ "clk_sys_pll"
+ },
+ .num_parents = 1,
+ .ops = &axxia_divclk_ops,
+ },
+ .reg = 0x1000c,
+ .shift = 4,
+ .width = 4,
+};
+
+static struct axxia_divclk clk_fab_div = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_fab_div",
+ .parent_names = (const char *[]){
+ "clk_fab_pll"
+ },
+ .num_parents = 1,
+ .ops = &axxia_divclk_ops,
+ },
+ .reg = 0x1000c,
+ .shift = 8,
+ .width = 4,
+};
+
+static struct axxia_divclk clk_per_div = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_per_div",
+ .parent_names = (const char *[]){
+ "clk_sm1_pll"
+ },
+ .num_parents = 1,
+ .flags = CLK_IS_BASIC,
+ .ops = &axxia_divclk_ops,
+ },
+ .reg = 0x1000c,
+ .shift = 12,
+ .width = 4,
+};
+
+static struct axxia_divclk clk_mmc_div = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_mmc_div",
+ .parent_names = (const char *[]){
+ "clk_sm1_pll"
+ },
+ .num_parents = 1,
+ .flags = CLK_IS_BASIC,
+ .ops = &axxia_divclk_ops,
+ },
+ .reg = 0x1000c,
+ .shift = 16,
+ .width = 4,
+};
+
+/*
+ * Clock MUXes
+ */
+
+static struct axxia_clkmux clk_cpu0_mux = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_cpu0",
+ .parent_names = (const char *[]){
+ "clk_ref0",
+ "clk_cpu_pll",
+ "clk_cpu0_div",
+ "clk_cpu0_div"
+ },
+ .num_parents = 4,
+ .ops = &axxia_clkmux_ops,
+ },
+ .reg = 0x10000,
+ .shift = 0,
+ .width = 2,
+};
+
+static struct axxia_clkmux clk_cpu1_mux = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_cpu1",
+ .parent_names = (const char *[]){
+ "clk_ref0",
+ "clk_cpu_pll",
+ "clk_cpu1_div",
+ "clk_cpu1_div"
+ },
+ .num_parents = 4,
+ .ops = &axxia_clkmux_ops,
+ },
+ .reg = 0x10000,
+ .shift = 2,
+ .width = 2,
+};
+
+static struct axxia_clkmux clk_cpu2_mux = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_cpu2",
+ .parent_names = (const char *[]){
+ "clk_ref0",
+ "clk_cpu_pll",
+ "clk_cpu2_div",
+ "clk_cpu2_div"
+ },
+ .num_parents = 4,
+ .ops = &axxia_clkmux_ops,
+ },
+ .reg = 0x10000,
+ .shift = 4,
+ .width = 2,
+};
+
+static struct axxia_clkmux clk_cpu3_mux = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_cpu3",
+ .parent_names = (const char *[]){
+ "clk_ref0",
+ "clk_cpu_pll",
+ "clk_cpu3_div",
+ "clk_cpu3_div"
+ },
+ .num_parents = 4,
+ .ops = &axxia_clkmux_ops,
+ },
+ .reg = 0x10000,
+ .shift = 6,
+ .width = 2,
+};
+
+static struct axxia_clkmux clk_nrcp_mux = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_nrcp",
+ .parent_names = (const char *[]){
+ "clk_ref0",
+ "clk_sys_pll",
+ "clk_nrcp_div",
+ "clk_nrcp_div"
+ },
+ .num_parents = 4,
+ .ops = &axxia_clkmux_ops,
+ },
+ .reg = 0x10004,
+ .shift = 0,
+ .width = 2,
+};
+
+static struct axxia_clkmux clk_sys_mux = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_sys",
+ .parent_names = (const char *[]){
+ "clk_ref0",
+ "clk_sys_pll",
+ "clk_sys_div",
+ "clk_sys_div"
+ },
+ .num_parents = 4,
+ .ops = &axxia_clkmux_ops,
+ },
+ .reg = 0x10004,
+ .shift = 2,
+ .width = 2,
+};
+
+static struct axxia_clkmux clk_fab_mux = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_fab",
+ .parent_names = (const char *[]){
+ "clk_ref0",
+ "clk_fab_pll",
+ "clk_fab_div",
+ "clk_fab_div"
+ },
+ .num_parents = 4,
+ .ops = &axxia_clkmux_ops,
+ },
+ .reg = 0x10004,
+ .shift = 4,
+ .width = 2,
+};
+
+static struct axxia_clkmux clk_per_mux = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_per",
+ .parent_names = (const char *[]){
+ "clk_ref1",
+ "clk_per_div"
+ },
+ .num_parents = 2,
+ .ops = &axxia_clkmux_ops,
+ },
+ .reg = 0x10004,
+ .shift = 6,
+ .width = 1,
+};
+
+static struct axxia_clkmux clk_mmc_mux = {
+ .aclk.hw.init = &(struct clk_init_data){
+ .name = "clk_mmc",
+ .parent_names = (const char *[]){
+ "clk_ref1",
+ "clk_mmc_div"
+ },
+ .num_parents = 2,
+ .ops = &axxia_clkmux_ops,
+ },
+ .reg = 0x10004,
+ .shift = 9,
+ .width = 1,
+};
+
+/* Table of all supported clocks indexed by the clock identifiers from the
+ * device tree binding
+ */
+static struct axxia_clk *axmclk_clocks[] = {
+ [AXXIA_CLK_FAB_PLL] = &clk_fab_pll.aclk,
+ [AXXIA_CLK_CPU_PLL] = &clk_cpu_pll.aclk,
+ [AXXIA_CLK_SYS_PLL] = &clk_sys_pll.aclk,
+ [AXXIA_CLK_SM0_PLL] = &clk_sm0_pll.aclk,
+ [AXXIA_CLK_SM1_PLL] = &clk_sm1_pll.aclk,
+ [AXXIA_CLK_FAB_DIV] = &clk_fab_div.aclk,
+ [AXXIA_CLK_SYS_DIV] = &clk_sys_div.aclk,
+ [AXXIA_CLK_NRCP_DIV] = &clk_nrcp_div.aclk,
+ [AXXIA_CLK_CPU0_DIV] = &clk_cpu0_div.aclk,
+ [AXXIA_CLK_CPU1_DIV] = &clk_cpu1_div.aclk,
+ [AXXIA_CLK_CPU2_DIV] = &clk_cpu2_div.aclk,
+ [AXXIA_CLK_CPU3_DIV] = &clk_cpu3_div.aclk,
+ [AXXIA_CLK_PER_DIV] = &clk_per_div.aclk,
+ [AXXIA_CLK_MMC_DIV] = &clk_mmc_div.aclk,
+ [AXXIA_CLK_FAB] = &clk_fab_mux.aclk,
+ [AXXIA_CLK_SYS] = &clk_sys_mux.aclk,
+ [AXXIA_CLK_NRCP] = &clk_nrcp_mux.aclk,
+ [AXXIA_CLK_CPU0] = &clk_cpu0_mux.aclk,
+ [AXXIA_CLK_CPU1] = &clk_cpu1_mux.aclk,
+ [AXXIA_CLK_CPU2] = &clk_cpu2_mux.aclk,
+ [AXXIA_CLK_CPU3] = &clk_cpu3_mux.aclk,
+ [AXXIA_CLK_PER] = &clk_per_mux.aclk,
+ [AXXIA_CLK_MMC] = &clk_mmc_mux.aclk,
+};
+
+static const struct regmap_config axmclk_regmap_config = {
+ .reg_bits = 32,
+ .reg_stride = 4,
+ .val_bits = 32,
+ .max_register = 0x1fffc,
+ .fast_io = true,
+};
+
+static const struct of_device_id axmclk_match_table[] = {
+ { .compatible = "lsi,axm5516-clks" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, axmclk_match_table);
+
+struct axmclk_priv {
+ struct clk_onecell_data onecell;
+ struct clk *clks[];
+};
+
+static int axmclk_probe(struct platform_device *pdev)
+{
+ void __iomem *base;
+ struct resource *res;
+ int i, ret;
+ struct device *dev = &pdev->dev;
+ struct clk *clk;
+ struct regmap *regmap;
+ size_t num_clks;
+ struct axmclk_priv *priv;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+
+ regmap = devm_regmap_init_mmio(dev, base, &axmclk_regmap_config);
+ if (IS_ERR(regmap))
+ return PTR_ERR(regmap);
+
+ num_clks = ARRAY_SIZE(axmclk_clocks);
+ pr_info("axmclk: supporting %u clocks\n", num_clks);
+ priv = devm_kzalloc(dev, sizeof(*priv) + sizeof(*priv->clks) * num_clks,
+ GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->onecell.clks = priv->clks;
+ priv->onecell.clk_num = num_clks;
+
+ /* Update each entry with the allocated regmap and register the clock
+ * with the common clock framework
+ */
+ for (i = 0; i < num_clks; i++) {
+ axmclk_clocks[i]->regmap = regmap;
+ clk = devm_clk_register(dev, &axmclk_clocks[i]->hw);
+ if (IS_ERR(clk))
+ return PTR_ERR(clk);
+ priv->clks[i] = clk;
+ }
+
+ ret = of_clk_add_provider(dev->of_node,
+ of_clk_src_onecell_get, &priv->onecell);
+
+ return ret;
+}
+
+static int axmclk_remove(struct platform_device *pdev)
+{
+ of_clk_del_provider(pdev->dev.of_node);
+ return 0;
+}
+
+static struct platform_driver axmclk_driver = {
+ .probe = axmclk_probe,
+ .remove = axmclk_remove,
+ .driver = {
+ .name = "clk-axm5516",
+ .owner = THIS_MODULE,
+ .of_match_table = axmclk_match_table,
+ },
+};
+
+static int __init axmclk_init(void)
+{
+ return platform_driver_register(&axmclk_driver);
+}
+core_initcall(axmclk_init);
+
+static void __exit axmclk_exit(void)
+{
+ platform_driver_unregister(&axmclk_driver);
+}
+module_exit(axmclk_exit);
+
+MODULE_DESCRIPTION("AXM5516 clock driver");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:clk-axm5516");
diff --git a/drivers/clk/clk-divider.c b/drivers/clk/clk-divider.c
index 3fbee45..18a9de2 100644
--- a/drivers/clk/clk-divider.c
+++ b/drivers/clk/clk-divider.c
@@ -43,6 +43,17 @@
return maxdiv;
}
+static unsigned int _get_table_mindiv(const struct clk_div_table *table)
+{
+ unsigned int mindiv = UINT_MAX;
+ const struct clk_div_table *clkt;
+
+ for (clkt = table; clkt->div; clkt++)
+ if (clkt->div < mindiv)
+ mindiv = clkt->div;
+ return mindiv;
+}
+
static unsigned int _get_maxdiv(struct clk_divider *divider)
{
if (divider->flags & CLK_DIVIDER_ONE_BASED)
@@ -162,6 +173,24 @@
return up;
}
+static int _round_down_table(const struct clk_div_table *table, int div)
+{
+ const struct clk_div_table *clkt;
+ int down = _get_table_mindiv(table);
+
+ for (clkt = table; clkt->div; clkt++) {
+ if (clkt->div == div)
+ return clkt->div;
+ else if (clkt->div > div)
+ continue;
+
+ if ((div - clkt->div) < (div - down))
+ down = clkt->div;
+ }
+
+ return down;
+}
+
static int _div_round_up(struct clk_divider *divider,
unsigned long parent_rate, unsigned long rate)
{
@@ -175,6 +204,54 @@
return div;
}
+static int _div_round_closest(struct clk_divider *divider,
+ unsigned long parent_rate, unsigned long rate)
+{
+ int up, down, div;
+
+ up = down = div = DIV_ROUND_CLOSEST(parent_rate, rate);
+
+ if (divider->flags & CLK_DIVIDER_POWER_OF_TWO) {
+ up = __roundup_pow_of_two(div);
+ down = __rounddown_pow_of_two(div);
+ } else if (divider->table) {
+ up = _round_up_table(divider->table, div);
+ down = _round_down_table(divider->table, div);
+ }
+
+ return (up - div) <= (div - down) ? up : down;
+}
+
+static int _div_round(struct clk_divider *divider, unsigned long parent_rate,
+ unsigned long rate)
+{
+ if (divider->flags & CLK_DIVIDER_ROUND_CLOSEST)
+ return _div_round_closest(divider, parent_rate, rate);
+
+ return _div_round_up(divider, parent_rate, rate);
+}
+
+static bool _is_best_div(struct clk_divider *divider,
+ unsigned long rate, unsigned long now, unsigned long best)
+{
+ if (divider->flags & CLK_DIVIDER_ROUND_CLOSEST)
+ return abs(rate - now) < abs(rate - best);
+
+ return now <= rate && now > best;
+}
+
+static int _next_div(struct clk_divider *divider, int div)
+{
+ div++;
+
+ if (divider->flags & CLK_DIVIDER_POWER_OF_TWO)
+ return __roundup_pow_of_two(div);
+ if (divider->table)
+ return _round_up_table(divider->table, div);
+
+ return div;
+}
+
static int clk_divider_bestdiv(struct clk_hw *hw, unsigned long rate,
unsigned long *best_parent_rate)
{
@@ -190,7 +267,7 @@
if (!(__clk_get_flags(hw->clk) & CLK_SET_RATE_PARENT)) {
parent_rate = *best_parent_rate;
- bestdiv = _div_round_up(divider, parent_rate, rate);
+ bestdiv = _div_round(divider, parent_rate, rate);
bestdiv = bestdiv == 0 ? 1 : bestdiv;
bestdiv = bestdiv > maxdiv ? maxdiv : bestdiv;
return bestdiv;
@@ -202,7 +279,7 @@
*/
maxdiv = min(ULONG_MAX / rate, maxdiv);
- for (i = 1; i <= maxdiv; i++) {
+ for (i = 1; i <= maxdiv; i = _next_div(divider, i)) {
if (!_is_valid_div(divider, i))
continue;
if (rate * i == parent_rate_saved) {
@@ -217,7 +294,7 @@
parent_rate = __clk_round_rate(__clk_get_parent(hw->clk),
MULT_ROUND_UP(rate, i));
now = DIV_ROUND_UP(parent_rate, i);
- if (now <= rate && now > best) {
+ if (_is_best_div(divider, rate, now, best)) {
bestdiv = i;
best = now;
*best_parent_rate = parent_rate;
@@ -284,6 +361,11 @@
};
EXPORT_SYMBOL_GPL(clk_divider_ops);
+const struct clk_ops clk_divider_ro_ops = {
+ .recalc_rate = clk_divider_recalc_rate,
+};
+EXPORT_SYMBOL_GPL(clk_divider_ro_ops);
+
static struct clk *_register_divider(struct device *dev, const char *name,
const char *parent_name, unsigned long flags,
void __iomem *reg, u8 shift, u8 width,
@@ -309,7 +391,10 @@
}
init.name = name;
- init.ops = &clk_divider_ops;
+ if (clk_divider_flags & CLK_DIVIDER_READ_ONLY)
+ init.ops = &clk_divider_ro_ops;
+ else
+ init.ops = &clk_divider_ops;
init.flags = flags | CLK_IS_BASIC;
init.parent_names = (parent_name ? &parent_name: NULL);
init.num_parents = (parent_name ? 1 : 0);
diff --git a/drivers/clk/clk-s2mps11.c b/drivers/clk/clk-s2mps11.c
index f2f62a1..9b7b585 100644
--- a/drivers/clk/clk-s2mps11.c
+++ b/drivers/clk/clk-s2mps11.c
@@ -1,7 +1,7 @@
/*
* clk-s2mps11.c - Clock driver for S2MPS11.
*
- * Copyright (C) 2013 Samsung Electornics
+ * Copyright (C) 2013,2014 Samsung Electornics
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
@@ -13,10 +13,6 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
*/
#include <linux/module.h>
@@ -27,6 +23,7 @@
#include <linux/clk-provider.h>
#include <linux/platform_device.h>
#include <linux/mfd/samsung/s2mps11.h>
+#include <linux/mfd/samsung/s2mps14.h>
#include <linux/mfd/samsung/s5m8767.h>
#include <linux/mfd/samsung/core.h>
@@ -44,6 +41,7 @@
struct s2mps11_clk {
struct sec_pmic_dev *iodev;
+ struct device_node *clk_np;
struct clk_hw hw;
struct clk *clk;
struct clk_lookup *lookup;
@@ -125,7 +123,21 @@
},
};
-static struct device_node *s2mps11_clk_parse_dt(struct platform_device *pdev)
+static struct clk_init_data s2mps14_clks_init[S2MPS11_CLKS_NUM] = {
+ [S2MPS11_CLK_AP] = {
+ .name = "s2mps14_ap",
+ .ops = &s2mps11_clk_ops,
+ .flags = CLK_IS_ROOT,
+ },
+ [S2MPS11_CLK_BT] = {
+ .name = "s2mps14_bt",
+ .ops = &s2mps11_clk_ops,
+ .flags = CLK_IS_ROOT,
+ },
+};
+
+static struct device_node *s2mps11_clk_parse_dt(struct platform_device *pdev,
+ struct clk_init_data *clks_init)
{
struct sec_pmic_dev *iodev = dev_get_drvdata(pdev->dev.parent);
struct device_node *clk_np;
@@ -140,14 +152,12 @@
return ERR_PTR(-EINVAL);
}
- clk_table = devm_kzalloc(&pdev->dev, sizeof(struct clk *) *
- S2MPS11_CLKS_NUM, GFP_KERNEL);
- if (!clk_table)
- return ERR_PTR(-ENOMEM);
-
- for (i = 0; i < S2MPS11_CLKS_NUM; i++)
+ for (i = 0; i < S2MPS11_CLKS_NUM; i++) {
+ if (!clks_init[i].name)
+ continue; /* Skip clocks not present in some devices */
of_property_read_string_index(clk_np, "clock-output-names", i,
- &s2mps11_clks_init[i].name);
+ &clks_init[i].name);
+ }
return clk_np;
}
@@ -156,8 +166,8 @@
{
struct sec_pmic_dev *iodev = dev_get_drvdata(pdev->dev.parent);
struct s2mps11_clk *s2mps11_clks, *s2mps11_clk;
- struct device_node *clk_np = NULL;
unsigned int s2mps11_reg;
+ struct clk_init_data *clks_init;
int i, ret = 0;
u32 val;
@@ -168,25 +178,39 @@
s2mps11_clk = s2mps11_clks;
- clk_np = s2mps11_clk_parse_dt(pdev);
- if (IS_ERR(clk_np))
- return PTR_ERR(clk_np);
+ clk_table = devm_kzalloc(&pdev->dev, sizeof(struct clk *) *
+ S2MPS11_CLKS_NUM, GFP_KERNEL);
+ if (!clk_table)
+ return -ENOMEM;
switch(platform_get_device_id(pdev)->driver_data) {
case S2MPS11X:
s2mps11_reg = S2MPS11_REG_RTC_CTRL;
+ clks_init = s2mps11_clks_init;
+ break;
+ case S2MPS14X:
+ s2mps11_reg = S2MPS14_REG_RTCCTRL;
+ clks_init = s2mps14_clks_init;
break;
case S5M8767X:
s2mps11_reg = S5M8767_REG_CTRL1;
+ clks_init = s2mps11_clks_init;
break;
default:
dev_err(&pdev->dev, "Invalid device type\n");
return -EINVAL;
};
+ /* Store clocks of_node in first element of s2mps11_clks array */
+ s2mps11_clks->clk_np = s2mps11_clk_parse_dt(pdev, clks_init);
+ if (IS_ERR(s2mps11_clks->clk_np))
+ return PTR_ERR(s2mps11_clks->clk_np);
+
for (i = 0; i < S2MPS11_CLKS_NUM; i++, s2mps11_clk++) {
+ if (!clks_init[i].name)
+ continue; /* Skip clocks not present in some devices */
s2mps11_clk->iodev = iodev;
- s2mps11_clk->hw.init = &s2mps11_clks_init[i];
+ s2mps11_clk->hw.init = &clks_init[i];
s2mps11_clk->mask = 1 << i;
s2mps11_clk->reg = s2mps11_reg;
@@ -219,15 +243,18 @@
clkdev_add(s2mps11_clk->lookup);
}
- if (clk_table) {
- for (i = 0; i < S2MPS11_CLKS_NUM; i++)
- clk_table[i] = s2mps11_clks[i].clk;
-
- clk_data.clks = clk_table;
- clk_data.clk_num = S2MPS11_CLKS_NUM;
- of_clk_add_provider(clk_np, of_clk_src_onecell_get, &clk_data);
+ for (i = 0; i < S2MPS11_CLKS_NUM; i++) {
+ /* Skip clocks not present on S2MPS14 */
+ if (!clks_init[i].name)
+ continue;
+ clk_table[i] = s2mps11_clks[i].clk;
}
+ clk_data.clks = clk_table;
+ clk_data.clk_num = S2MPS11_CLKS_NUM;
+ of_clk_add_provider(s2mps11_clks->clk_np, of_clk_src_onecell_get,
+ &clk_data);
+
platform_set_drvdata(pdev, s2mps11_clks);
return ret;
@@ -250,14 +277,23 @@
struct s2mps11_clk *s2mps11_clks = platform_get_drvdata(pdev);
int i;
- for (i = 0; i < S2MPS11_CLKS_NUM; i++)
+ of_clk_del_provider(s2mps11_clks[0].clk_np);
+ /* Drop the reference obtained in s2mps11_clk_parse_dt */
+ of_node_put(s2mps11_clks[0].clk_np);
+
+ for (i = 0; i < S2MPS11_CLKS_NUM; i++) {
+ /* Skip clocks not present on S2MPS14 */
+ if (!s2mps11_clks[i].lookup)
+ continue;
clkdev_drop(s2mps11_clks[i].lookup);
+ }
return 0;
}
static const struct platform_device_id s2mps11_clk_id[] = {
{ "s2mps11-clk", S2MPS11X},
+ { "s2mps14-clk", S2MPS14X},
{ "s5m8767-clk", S5M8767X},
{ },
};
diff --git a/drivers/clk/clk-si570.c b/drivers/clk/clk-si570.c
index 4bbbe32..fc167b3 100644
--- a/drivers/clk/clk-si570.c
+++ b/drivers/clk/clk-si570.c
@@ -526,6 +526,6 @@
module_i2c_driver(si570_driver);
MODULE_AUTHOR("Guenter Roeck <guenter.roeck@ericsson.com>");
-MODULE_AUTHOR("Soeren Brinkmann <soren.brinkmann@xilinx.com");
+MODULE_AUTHOR("Soeren Brinkmann <soren.brinkmann@xilinx.com>");
MODULE_DESCRIPTION("Si570 driver");
MODULE_LICENSE("GPL");
diff --git a/drivers/clk/clk-u300.c b/drivers/clk/clk-u300.c
index 3efbdd0..406bfc1 100644
--- a/drivers/clk/clk-u300.c
+++ b/drivers/clk/clk-u300.c
@@ -1168,6 +1168,7 @@
.compatible = "stericsson,u300-syscon-mclk",
.data = of_u300_syscon_mclk_init,
},
+ {}
};
diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c
index 7cf2c09..8b73ede 100644
--- a/drivers/clk/clk.c
+++ b/drivers/clk/clk.c
@@ -106,12 +106,11 @@
if (!c)
return;
- seq_printf(s, "%*s%-*s %-11d %-12d %-10lu %-11lu",
+ seq_printf(s, "%*s%-*s %11d %12d %11lu %10lu\n",
level * 3 + 1, "",
30 - level * 3, c->name,
c->enable_count, c->prepare_count, clk_get_rate(c),
clk_get_accuracy(c));
- seq_printf(s, "\n");
}
static void clk_summary_show_subtree(struct seq_file *s, struct clk *c,
@@ -132,8 +131,8 @@
{
struct clk *c;
- seq_printf(s, " clock enable_cnt prepare_cnt rate accuracy\n");
- seq_printf(s, "---------------------------------------------------------------------------------\n");
+ seq_puts(s, " clock enable_cnt prepare_cnt rate accuracy\n");
+ seq_puts(s, "--------------------------------------------------------------------------------\n");
clk_prepare_lock();
@@ -822,6 +821,9 @@
*/
void clk_unprepare(struct clk *clk)
{
+ if (IS_ERR_OR_NULL(clk))
+ return;
+
clk_prepare_lock();
__clk_unprepare(clk);
clk_prepare_unlock();
@@ -883,9 +885,6 @@
if (!clk)
return;
- if (WARN_ON(IS_ERR(clk)))
- return;
-
if (WARN_ON(clk->enable_count == 0))
return;
@@ -914,6 +913,9 @@
{
unsigned long flags;
+ if (IS_ERR_OR_NULL(clk))
+ return;
+
flags = clk_enable_lock();
__clk_disable(clk);
clk_enable_unlock(flags);
@@ -1004,6 +1006,7 @@
else
return clk->rate;
}
+EXPORT_SYMBOL_GPL(__clk_round_rate);
/**
* clk_round_rate - round the given rate for a clk
@@ -1115,6 +1118,13 @@
}
EXPORT_SYMBOL_GPL(clk_get_accuracy);
+static unsigned long clk_recalc(struct clk *clk, unsigned long parent_rate)
+{
+ if (clk->ops->recalc_rate)
+ return clk->ops->recalc_rate(clk->hw, parent_rate);
+ return parent_rate;
+}
+
/**
* __clk_recalc_rates
* @clk: first clk in the subtree
@@ -1140,10 +1150,7 @@
if (clk->parent)
parent_rate = clk->parent->rate;
- if (clk->ops->recalc_rate)
- clk->rate = clk->ops->recalc_rate(clk->hw, parent_rate);
- else
- clk->rate = parent_rate;
+ clk->rate = clk_recalc(clk, parent_rate);
/*
* ignore NOTIFY_STOP and NOTIFY_BAD return values for POST_RATE_CHANGE
@@ -1334,10 +1341,7 @@
unsigned long new_rate;
int ret = NOTIFY_DONE;
- if (clk->ops->recalc_rate)
- new_rate = clk->ops->recalc_rate(clk->hw, parent_rate);
- else
- new_rate = parent_rate;
+ new_rate = clk_recalc(clk, parent_rate);
/* abort rate change if a driver returns NOTIFY_BAD or NOTIFY_STOP */
if (clk->notifier_count)
@@ -1373,10 +1377,7 @@
new_parent->new_child = clk;
hlist_for_each_entry(child, &clk->children, child_node) {
- if (child->ops->recalc_rate)
- child->new_rate = child->ops->recalc_rate(child->hw, new_rate);
- else
- child->new_rate = new_rate;
+ child->new_rate = clk_recalc(child, new_rate);
clk_calc_subtree(child, child->new_rate, NULL, 0);
}
}
@@ -1524,10 +1525,7 @@
if (!skip_set_rate && clk->ops->set_rate)
clk->ops->set_rate(clk->hw, clk->new_rate, best_parent_rate);
- if (clk->ops->recalc_rate)
- clk->rate = clk->ops->recalc_rate(clk->hw, best_parent_rate);
- else
- clk->rate = best_parent_rate;
+ clk->rate = clk_recalc(clk, best_parent_rate);
if (clk->notifier_count && old_rate != clk->rate)
__clk_notify(clk, POST_RATE_CHANGE, old_rate, clk->rate);
@@ -1716,9 +1714,6 @@
if (!clk)
return 0;
- if (!clk->ops)
- return -EINVAL;
-
/* verify ops for for multi-parent clks */
if ((clk->num_parents > 1) && (!clk->ops->set_parent))
return -ENOSYS;
diff --git a/drivers/clk/clk.h b/drivers/clk/clk.h
index 795cc9f..c798138 100644
--- a/drivers/clk/clk.h
+++ b/drivers/clk/clk.h
@@ -10,6 +10,7 @@
*/
#if defined(CONFIG_OF) && defined(CONFIG_COMMON_CLK)
+struct clk *of_clk_get_by_clkspec(struct of_phandle_args *clkspec);
struct clk *__of_clk_get_from_provider(struct of_phandle_args *clkspec);
void of_clk_lock(void);
void of_clk_unlock(void);
diff --git a/drivers/clk/clkdev.c b/drivers/clk/clkdev.c
index a360b2e..f890b90 100644
--- a/drivers/clk/clkdev.c
+++ b/drivers/clk/clkdev.c
@@ -27,6 +27,32 @@
static DEFINE_MUTEX(clocks_mutex);
#if defined(CONFIG_OF) && defined(CONFIG_COMMON_CLK)
+
+/**
+ * of_clk_get_by_clkspec() - Lookup a clock form a clock provider
+ * @clkspec: pointer to a clock specifier data structure
+ *
+ * This function looks up a struct clk from the registered list of clock
+ * providers, an input is a clock specifier data structure as returned
+ * from the of_parse_phandle_with_args() function call.
+ */
+struct clk *of_clk_get_by_clkspec(struct of_phandle_args *clkspec)
+{
+ struct clk *clk;
+
+ if (!clkspec)
+ return ERR_PTR(-EINVAL);
+
+ of_clk_lock();
+ clk = __of_clk_get_from_provider(clkspec);
+
+ if (!IS_ERR(clk) && !__clk_get(clk))
+ clk = ERR_PTR(-ENOENT);
+
+ of_clk_unlock();
+ return clk;
+}
+
struct clk *of_clk_get(struct device_node *np, int index)
{
struct of_phandle_args clkspec;
@@ -41,13 +67,7 @@
if (rc)
return ERR_PTR(rc);
- of_clk_lock();
- clk = __of_clk_get_from_provider(&clkspec);
-
- if (!IS_ERR(clk) && !__clk_get(clk))
- clk = ERR_PTR(-ENOENT);
-
- of_clk_unlock();
+ clk = of_clk_get_by_clkspec(&clkspec);
of_node_put(clkspec.np);
return clk;
}
diff --git a/drivers/clk/hisilicon/Makefile b/drivers/clk/hisilicon/Makefile
index 40b33c6..038c02f 100644
--- a/drivers/clk/hisilicon/Makefile
+++ b/drivers/clk/hisilicon/Makefile
@@ -6,3 +6,4 @@
obj-$(CONFIG_ARCH_HI3xxx) += clk-hi3620.o
obj-$(CONFIG_ARCH_HIP04) += clk-hip04.o
+obj-$(CONFIG_ARCH_HIX5HD2) += clk-hix5hd2.o
diff --git a/drivers/clk/hisilicon/clk-hix5hd2.c b/drivers/clk/hisilicon/clk-hix5hd2.c
new file mode 100644
index 0000000..e5fcfb4
--- /dev/null
+++ b/drivers/clk/hisilicon/clk-hix5hd2.c
@@ -0,0 +1,101 @@
+/*
+ * Copyright (c) 2014 Linaro Ltd.
+ * Copyright (c) 2014 Hisilicon Limited.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ */
+
+#include <linux/of_address.h>
+#include <dt-bindings/clock/hix5hd2-clock.h>
+#include "clk.h"
+
+static struct hisi_fixed_rate_clock hix5hd2_fixed_rate_clks[] __initdata = {
+ { HIX5HD2_FIXED_1200M, "1200m", NULL, CLK_IS_ROOT, 1200000000, },
+ { HIX5HD2_FIXED_400M, "400m", NULL, CLK_IS_ROOT, 400000000, },
+ { HIX5HD2_FIXED_48M, "48m", NULL, CLK_IS_ROOT, 48000000, },
+ { HIX5HD2_FIXED_24M, "24m", NULL, CLK_IS_ROOT, 24000000, },
+ { HIX5HD2_FIXED_600M, "600m", NULL, CLK_IS_ROOT, 600000000, },
+ { HIX5HD2_FIXED_300M, "300m", NULL, CLK_IS_ROOT, 300000000, },
+ { HIX5HD2_FIXED_75M, "75m", NULL, CLK_IS_ROOT, 75000000, },
+ { HIX5HD2_FIXED_200M, "200m", NULL, CLK_IS_ROOT, 200000000, },
+ { HIX5HD2_FIXED_100M, "100m", NULL, CLK_IS_ROOT, 100000000, },
+ { HIX5HD2_FIXED_40M, "40m", NULL, CLK_IS_ROOT, 40000000, },
+ { HIX5HD2_FIXED_150M, "150m", NULL, CLK_IS_ROOT, 150000000, },
+ { HIX5HD2_FIXED_1728M, "1728m", NULL, CLK_IS_ROOT, 1728000000, },
+ { HIX5HD2_FIXED_28P8M, "28p8m", NULL, CLK_IS_ROOT, 28000000, },
+ { HIX5HD2_FIXED_432M, "432m", NULL, CLK_IS_ROOT, 432000000, },
+ { HIX5HD2_FIXED_345P6M, "345p6m", NULL, CLK_IS_ROOT, 345000000, },
+ { HIX5HD2_FIXED_288M, "288m", NULL, CLK_IS_ROOT, 288000000, },
+ { HIX5HD2_FIXED_60M, "60m", NULL, CLK_IS_ROOT, 60000000, },
+ { HIX5HD2_FIXED_750M, "750m", NULL, CLK_IS_ROOT, 750000000, },
+ { HIX5HD2_FIXED_500M, "500m", NULL, CLK_IS_ROOT, 500000000, },
+ { HIX5HD2_FIXED_54M, "54m", NULL, CLK_IS_ROOT, 54000000, },
+ { HIX5HD2_FIXED_27M, "27m", NULL, CLK_IS_ROOT, 27000000, },
+ { HIX5HD2_FIXED_1500M, "1500m", NULL, CLK_IS_ROOT, 1500000000, },
+ { HIX5HD2_FIXED_375M, "375m", NULL, CLK_IS_ROOT, 375000000, },
+ { HIX5HD2_FIXED_187M, "187m", NULL, CLK_IS_ROOT, 187000000, },
+ { HIX5HD2_FIXED_250M, "250m", NULL, CLK_IS_ROOT, 250000000, },
+ { HIX5HD2_FIXED_125M, "125m", NULL, CLK_IS_ROOT, 125000000, },
+ { HIX5HD2_FIXED_2P02M, "2m", NULL, CLK_IS_ROOT, 2000000, },
+ { HIX5HD2_FIXED_50M, "50m", NULL, CLK_IS_ROOT, 50000000, },
+ { HIX5HD2_FIXED_25M, "25m", NULL, CLK_IS_ROOT, 25000000, },
+ { HIX5HD2_FIXED_83M, "83m", NULL, CLK_IS_ROOT, 83333333, },
+};
+
+static const char *sfc_mux_p[] __initconst = {
+ "24m", "150m", "200m", "100m", "75m", };
+static u32 sfc_mux_table[] = {0, 4, 5, 6, 7};
+
+static const char *sdio1_mux_p[] __initconst = {
+ "75m", "100m", "50m", "15m", };
+static u32 sdio1_mux_table[] = {0, 1, 2, 3};
+
+static const char *fephy_mux_p[] __initconst = { "25m", "125m"};
+static u32 fephy_mux_table[] = {0, 1};
+
+
+static struct hisi_mux_clock hix5hd2_mux_clks[] __initdata = {
+ { HIX5HD2_SFC_MUX, "sfc_mux", sfc_mux_p, ARRAY_SIZE(sfc_mux_p),
+ CLK_SET_RATE_PARENT, 0x5c, 8, 3, 0, sfc_mux_table, },
+ { HIX5HD2_MMC_MUX, "mmc_mux", sdio1_mux_p, ARRAY_SIZE(sdio1_mux_p),
+ CLK_SET_RATE_PARENT, 0xa0, 8, 2, 0, sdio1_mux_table, },
+ { HIX5HD2_FEPHY_MUX, "fephy_mux",
+ fephy_mux_p, ARRAY_SIZE(fephy_mux_p),
+ CLK_SET_RATE_PARENT, 0x120, 8, 2, 0, fephy_mux_table, },
+};
+
+static struct hisi_gate_clock hix5hd2_gate_clks[] __initdata = {
+ /*sfc*/
+ { HIX5HD2_SFC_CLK, "clk_sfc", "sfc_mux",
+ CLK_SET_RATE_PARENT, 0x5c, 0, 0, },
+ { HIX5HD2_SFC_RST, "rst_sfc", "clk_sfc",
+ CLK_SET_RATE_PARENT, 0x5c, 4, CLK_GATE_SET_TO_DISABLE, },
+ /*sdio1*/
+ { HIX5HD2_MMC_BIU_CLK, "clk_mmc_biu", "200m",
+ CLK_SET_RATE_PARENT, 0xa0, 0, 0, },
+ { HIX5HD2_MMC_CIU_CLK, "clk_mmc_ciu", "mmc_mux",
+ CLK_SET_RATE_PARENT, 0xa0, 1, 0, },
+ { HIX5HD2_MMC_CIU_RST, "rst_mmc_ciu", "clk_mmc_ciu",
+ CLK_SET_RATE_PARENT, 0xa0, 4, CLK_GATE_SET_TO_DISABLE, },
+};
+
+static void __init hix5hd2_clk_init(struct device_node *np)
+{
+ struct hisi_clock_data *clk_data;
+
+ clk_data = hisi_clk_init(np, HIX5HD2_NR_CLKS);
+ if (!clk_data)
+ return;
+
+ hisi_clk_register_fixed_rate(hix5hd2_fixed_rate_clks,
+ ARRAY_SIZE(hix5hd2_fixed_rate_clks),
+ clk_data);
+ hisi_clk_register_mux(hix5hd2_mux_clks, ARRAY_SIZE(hix5hd2_mux_clks),
+ clk_data);
+ hisi_clk_register_gate(hix5hd2_gate_clks,
+ ARRAY_SIZE(hix5hd2_gate_clks), clk_data);
+}
+
+CLK_OF_DECLARE(hix5hd2_clk, "hisilicon,hix5hd2-clock", hix5hd2_clk_init);
diff --git a/drivers/clk/hisilicon/clk.c b/drivers/clk/hisilicon/clk.c
index 276f672..a078e84 100644
--- a/drivers/clk/hisilicon/clk.c
+++ b/drivers/clk/hisilicon/clk.c
@@ -127,11 +127,14 @@
int i;
for (i = 0; i < nums; i++) {
- clk = clk_register_mux(NULL, clks[i].name, clks[i].parent_names,
- clks[i].num_parents, clks[i].flags,
- base + clks[i].offset, clks[i].shift,
- clks[i].width, clks[i].mux_flags,
- &hisi_clk_lock);
+ u32 mask = BIT(clks[i].width) - 1;
+
+ clk = clk_register_mux_table(NULL, clks[i].name,
+ clks[i].parent_names,
+ clks[i].num_parents, clks[i].flags,
+ base + clks[i].offset, clks[i].shift,
+ mask, clks[i].mux_flags,
+ clks[i].table, &hisi_clk_lock);
if (IS_ERR(clk)) {
pr_err("%s: failed to register clock %s\n",
__func__, clks[i].name);
@@ -174,6 +177,34 @@
}
}
+void __init hisi_clk_register_gate(struct hisi_gate_clock *clks,
+ int nums, struct hisi_clock_data *data)
+{
+ struct clk *clk;
+ void __iomem *base = data->base;
+ int i;
+
+ for (i = 0; i < nums; i++) {
+ clk = clk_register_gate(NULL, clks[i].name,
+ clks[i].parent_name,
+ clks[i].flags,
+ base + clks[i].offset,
+ clks[i].bit_idx,
+ clks[i].gate_flags,
+ &hisi_clk_lock);
+ if (IS_ERR(clk)) {
+ pr_err("%s: failed to register clock %s\n",
+ __func__, clks[i].name);
+ continue;
+ }
+
+ if (clks[i].alias)
+ clk_register_clkdev(clk, clks[i].alias, NULL);
+
+ data->clk_data.clks[clks[i].id] = clk;
+ }
+}
+
void __init hisi_clk_register_gate_sep(struct hisi_gate_clock *clks,
int nums, struct hisi_clock_data *data)
{
diff --git a/drivers/clk/hisilicon/clk.h b/drivers/clk/hisilicon/clk.h
index 43fa5da..31083ff 100644
--- a/drivers/clk/hisilicon/clk.h
+++ b/drivers/clk/hisilicon/clk.h
@@ -62,6 +62,7 @@
u8 shift;
u8 width;
u8 mux_flags;
+ u32 *table;
const char *alias;
};
@@ -103,6 +104,8 @@
struct hisi_clock_data *);
void __init hisi_clk_register_divider(struct hisi_divider_clock *,
int, struct hisi_clock_data *);
+void __init hisi_clk_register_gate(struct hisi_gate_clock *,
+ int, struct hisi_clock_data *);
void __init hisi_clk_register_gate_sep(struct hisi_gate_clock *,
int, struct hisi_clock_data *);
#endif /* __HISI_CLK_H */
diff --git a/drivers/clk/mvebu/Kconfig b/drivers/clk/mvebu/Kconfig
index 693f7be..3b34dba 100644
--- a/drivers/clk/mvebu/Kconfig
+++ b/drivers/clk/mvebu/Kconfig
@@ -34,3 +34,7 @@
config KIRKWOOD_CLK
bool
select MVEBU_CLK_COMMON
+
+config ORION_CLK
+ bool
+ select MVEBU_CLK_COMMON
diff --git a/drivers/clk/mvebu/Makefile b/drivers/clk/mvebu/Makefile
index 4c66162f..a9a56fc 100644
--- a/drivers/clk/mvebu/Makefile
+++ b/drivers/clk/mvebu/Makefile
@@ -8,3 +8,4 @@
obj-$(CONFIG_ARMADA_XP_CLK) += armada-xp.o
obj-$(CONFIG_DOVE_CLK) += dove.o
obj-$(CONFIG_KIRKWOOD_CLK) += kirkwood.o
+obj-$(CONFIG_ORION_CLK) += orion.o
diff --git a/drivers/clk/mvebu/orion.c b/drivers/clk/mvebu/orion.c
new file mode 100644
index 0000000..fd12956
--- /dev/null
+++ b/drivers/clk/mvebu/orion.c
@@ -0,0 +1,210 @@
+/*
+ * Marvell Orion SoC clocks
+ *
+ * Copyright (C) 2014 Thomas Petazzoni
+ *
+ * Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/kernel.h>
+#include <linux/clk-provider.h>
+#include <linux/io.h>
+#include <linux/of.h>
+#include "common.h"
+
+static const struct coreclk_ratio orion_coreclk_ratios[] __initconst = {
+ { .id = 0, .name = "ddrclk", }
+};
+
+/*
+ * Orion 5182
+ */
+
+#define SAR_MV88F5182_TCLK_FREQ 8
+#define SAR_MV88F5182_TCLK_FREQ_MASK 0x3
+
+static u32 __init mv88f5182_get_tclk_freq(void __iomem *sar)
+{
+ u32 opt = (readl(sar) >> SAR_MV88F5182_TCLK_FREQ) &
+ SAR_MV88F5182_TCLK_FREQ_MASK;
+ if (opt == 1)
+ return 150000000;
+ else if (opt == 2)
+ return 166666667;
+ else
+ return 0;
+}
+
+#define SAR_MV88F5182_CPU_FREQ 4
+#define SAR_MV88F5182_CPU_FREQ_MASK 0xf
+
+static u32 __init mv88f5182_get_cpu_freq(void __iomem *sar)
+{
+ u32 opt = (readl(sar) >> SAR_MV88F5182_CPU_FREQ) &
+ SAR_MV88F5182_CPU_FREQ_MASK;
+ if (opt == 0)
+ return 333333333;
+ else if (opt == 1 || opt == 2)
+ return 400000000;
+ else if (opt == 3)
+ return 500000000;
+ else
+ return 0;
+}
+
+static void __init mv88f5182_get_clk_ratio(void __iomem *sar, int id,
+ int *mult, int *div)
+{
+ u32 opt = (readl(sar) >> SAR_MV88F5182_CPU_FREQ) &
+ SAR_MV88F5182_CPU_FREQ_MASK;
+ if (opt == 0 || opt == 1) {
+ *mult = 1;
+ *div = 2;
+ } else if (opt == 2 || opt == 3) {
+ *mult = 1;
+ *div = 3;
+ } else {
+ *mult = 0;
+ *div = 1;
+ }
+}
+
+static const struct coreclk_soc_desc mv88f5182_coreclks = {
+ .get_tclk_freq = mv88f5182_get_tclk_freq,
+ .get_cpu_freq = mv88f5182_get_cpu_freq,
+ .get_clk_ratio = mv88f5182_get_clk_ratio,
+ .ratios = orion_coreclk_ratios,
+ .num_ratios = ARRAY_SIZE(orion_coreclk_ratios),
+};
+
+static void __init mv88f5182_clk_init(struct device_node *np)
+{
+ return mvebu_coreclk_setup(np, &mv88f5182_coreclks);
+}
+
+CLK_OF_DECLARE(mv88f5182_clk, "marvell,mv88f5182-core-clock", mv88f5182_clk_init);
+
+/*
+ * Orion 5281
+ */
+
+static u32 __init mv88f5281_get_tclk_freq(void __iomem *sar)
+{
+ /* On 5281, tclk is always 166 Mhz */
+ return 166666667;
+}
+
+#define SAR_MV88F5281_CPU_FREQ 4
+#define SAR_MV88F5281_CPU_FREQ_MASK 0xf
+
+static u32 __init mv88f5281_get_cpu_freq(void __iomem *sar)
+{
+ u32 opt = (readl(sar) >> SAR_MV88F5281_CPU_FREQ) &
+ SAR_MV88F5281_CPU_FREQ_MASK;
+ if (opt == 1 || opt == 2)
+ return 400000000;
+ else if (opt == 3)
+ return 500000000;
+ else
+ return 0;
+}
+
+static void __init mv88f5281_get_clk_ratio(void __iomem *sar, int id,
+ int *mult, int *div)
+{
+ u32 opt = (readl(sar) >> SAR_MV88F5281_CPU_FREQ) &
+ SAR_MV88F5281_CPU_FREQ_MASK;
+ if (opt == 1) {
+ *mult = 1;
+ *div = 2;
+ } else if (opt == 2 || opt == 3) {
+ *mult = 1;
+ *div = 3;
+ } else {
+ *mult = 0;
+ *div = 1;
+ }
+}
+
+static const struct coreclk_soc_desc mv88f5281_coreclks = {
+ .get_tclk_freq = mv88f5281_get_tclk_freq,
+ .get_cpu_freq = mv88f5281_get_cpu_freq,
+ .get_clk_ratio = mv88f5281_get_clk_ratio,
+ .ratios = orion_coreclk_ratios,
+ .num_ratios = ARRAY_SIZE(orion_coreclk_ratios),
+};
+
+static void __init mv88f5281_clk_init(struct device_node *np)
+{
+ return mvebu_coreclk_setup(np, &mv88f5281_coreclks);
+}
+
+CLK_OF_DECLARE(mv88f5281_clk, "marvell,mv88f5281-core-clock", mv88f5281_clk_init);
+
+/*
+ * Orion 6183
+ */
+
+#define SAR_MV88F6183_TCLK_FREQ 9
+#define SAR_MV88F6183_TCLK_FREQ_MASK 0x1
+
+static u32 __init mv88f6183_get_tclk_freq(void __iomem *sar)
+{
+ u32 opt = (readl(sar) >> SAR_MV88F6183_TCLK_FREQ) &
+ SAR_MV88F6183_TCLK_FREQ_MASK;
+ if (opt == 0)
+ return 133333333;
+ else if (opt == 1)
+ return 166666667;
+ else
+ return 0;
+}
+
+#define SAR_MV88F6183_CPU_FREQ 1
+#define SAR_MV88F6183_CPU_FREQ_MASK 0x3f
+
+static u32 __init mv88f6183_get_cpu_freq(void __iomem *sar)
+{
+ u32 opt = (readl(sar) >> SAR_MV88F6183_CPU_FREQ) &
+ SAR_MV88F6183_CPU_FREQ_MASK;
+ if (opt == 9)
+ return 333333333;
+ else if (opt == 17)
+ return 400000000;
+ else
+ return 0;
+}
+
+static void __init mv88f6183_get_clk_ratio(void __iomem *sar, int id,
+ int *mult, int *div)
+{
+ u32 opt = (readl(sar) >> SAR_MV88F6183_CPU_FREQ) &
+ SAR_MV88F6183_CPU_FREQ_MASK;
+ if (opt == 9 || opt == 17) {
+ *mult = 1;
+ *div = 2;
+ } else {
+ *mult = 0;
+ *div = 1;
+ }
+}
+
+static const struct coreclk_soc_desc mv88f6183_coreclks = {
+ .get_tclk_freq = mv88f6183_get_tclk_freq,
+ .get_cpu_freq = mv88f6183_get_cpu_freq,
+ .get_clk_ratio = mv88f6183_get_clk_ratio,
+ .ratios = orion_coreclk_ratios,
+ .num_ratios = ARRAY_SIZE(orion_coreclk_ratios),
+};
+
+
+static void __init mv88f6183_clk_init(struct device_node *np)
+{
+ return mvebu_coreclk_setup(np, &mv88f6183_coreclks);
+}
+
+CLK_OF_DECLARE(mv88f6183_clk, "marvell,mv88f6183-core-clock", mv88f6183_clk_init);
diff --git a/drivers/clk/qcom/Kconfig b/drivers/clk/qcom/Kconfig
index 995bcfa..7f696b7 100644
--- a/drivers/clk/qcom/Kconfig
+++ b/drivers/clk/qcom/Kconfig
@@ -13,10 +13,10 @@
i2c, USB, SD/eMMC, etc.
config MSM_GCC_8960
- tristate "MSM8960 Global Clock Controller"
+ tristate "APQ8064/MSM8960 Global Clock Controller"
depends on COMMON_CLK_QCOM
help
- Support for the global clock controller on msm8960 devices.
+ Support for the global clock controller on apq8064/msm8960 devices.
Say Y if you want to use peripheral devices such as UART, SPI,
i2c, USB, SD/eMMC, SATA, PCIe, etc.
diff --git a/drivers/clk/qcom/Makefile b/drivers/clk/qcom/Makefile
index f60db2e..689e05b 100644
--- a/drivers/clk/qcom/Makefile
+++ b/drivers/clk/qcom/Makefile
@@ -1,5 +1,6 @@
obj-$(CONFIG_COMMON_CLK_QCOM) += clk-qcom.o
+clk-qcom-y += common.o
clk-qcom-y += clk-regmap.o
clk-qcom-y += clk-pll.o
clk-qcom-y += clk-rcg.o
diff --git a/drivers/clk/qcom/clk-rcg.h b/drivers/clk/qcom/clk-rcg.h
index 1d6b6de..b9ec11d 100644
--- a/drivers/clk/qcom/clk-rcg.h
+++ b/drivers/clk/qcom/clk-rcg.h
@@ -155,5 +155,8 @@
#define to_clk_rcg2(_hw) container_of(to_clk_regmap(_hw), struct clk_rcg2, clkr)
extern const struct clk_ops clk_rcg2_ops;
+extern const struct clk_ops clk_edp_pixel_ops;
+extern const struct clk_ops clk_byte_ops;
+extern const struct clk_ops clk_pixel_ops;
#endif
diff --git a/drivers/clk/qcom/clk-rcg2.c b/drivers/clk/qcom/clk-rcg2.c
index 00f878a..cd185d5 100644
--- a/drivers/clk/qcom/clk-rcg2.c
+++ b/drivers/clk/qcom/clk-rcg2.c
@@ -19,6 +19,7 @@
#include <linux/clk-provider.h>
#include <linux/delay.h>
#include <linux/regmap.h>
+#include <linux/math64.h>
#include <asm/div64.h>
@@ -55,7 +56,7 @@
if (ret)
return ret;
- return (cmd & CMD_ROOT_OFF) != 0;
+ return (cmd & CMD_ROOT_OFF) == 0;
}
static u8 clk_rcg2_get_parent(struct clk_hw *hw)
@@ -181,7 +182,8 @@
if (rate <= f->freq)
return f;
- return NULL;
+ /* Default to our fastest rate */
+ return f - 1;
}
static long _freq_tbl_determine_rate(struct clk_hw *hw,
@@ -224,31 +226,25 @@
return _freq_tbl_determine_rate(hw, rcg->freq_tbl, rate, p_rate, p);
}
-static int __clk_rcg2_set_rate(struct clk_hw *hw, unsigned long rate)
+static int clk_rcg2_configure(struct clk_rcg2 *rcg, const struct freq_tbl *f)
{
- struct clk_rcg2 *rcg = to_clk_rcg2(hw);
- const struct freq_tbl *f;
u32 cfg, mask;
int ret;
- f = find_freq(rcg->freq_tbl, rate);
- if (!f)
- return -EINVAL;
-
if (rcg->mnd_width && f->n) {
mask = BIT(rcg->mnd_width) - 1;
- ret = regmap_update_bits(rcg->clkr.regmap, rcg->cmd_rcgr + M_REG,
- mask, f->m);
+ ret = regmap_update_bits(rcg->clkr.regmap,
+ rcg->cmd_rcgr + M_REG, mask, f->m);
if (ret)
return ret;
- ret = regmap_update_bits(rcg->clkr.regmap, rcg->cmd_rcgr + N_REG,
- mask, ~(f->n - f->m));
+ ret = regmap_update_bits(rcg->clkr.regmap,
+ rcg->cmd_rcgr + N_REG, mask, ~(f->n - f->m));
if (ret)
return ret;
- ret = regmap_update_bits(rcg->clkr.regmap, rcg->cmd_rcgr + D_REG,
- mask, ~f->n);
+ ret = regmap_update_bits(rcg->clkr.regmap,
+ rcg->cmd_rcgr + D_REG, mask, ~f->n);
if (ret)
return ret;
}
@@ -259,14 +255,26 @@
cfg |= rcg->parent_map[f->src] << CFG_SRC_SEL_SHIFT;
if (rcg->mnd_width && f->n)
cfg |= CFG_MODE_DUAL_EDGE;
- ret = regmap_update_bits(rcg->clkr.regmap, rcg->cmd_rcgr + CFG_REG, mask,
- cfg);
+ ret = regmap_update_bits(rcg->clkr.regmap,
+ rcg->cmd_rcgr + CFG_REG, mask, cfg);
if (ret)
return ret;
return update_config(rcg);
}
+static int __clk_rcg2_set_rate(struct clk_hw *hw, unsigned long rate)
+{
+ struct clk_rcg2 *rcg = to_clk_rcg2(hw);
+ const struct freq_tbl *f;
+
+ f = find_freq(rcg->freq_tbl, rate);
+ if (!f)
+ return -EINVAL;
+
+ return clk_rcg2_configure(rcg, f);
+}
+
static int clk_rcg2_set_rate(struct clk_hw *hw, unsigned long rate,
unsigned long parent_rate)
{
@@ -289,3 +297,265 @@
.set_rate_and_parent = clk_rcg2_set_rate_and_parent,
};
EXPORT_SYMBOL_GPL(clk_rcg2_ops);
+
+struct frac_entry {
+ int num;
+ int den;
+};
+
+static const struct frac_entry frac_table_675m[] = { /* link rate of 270M */
+ { 52, 295 }, /* 119 M */
+ { 11, 57 }, /* 130.25 M */
+ { 63, 307 }, /* 138.50 M */
+ { 11, 50 }, /* 148.50 M */
+ { 47, 206 }, /* 154 M */
+ { 31, 100 }, /* 205.25 M */
+ { 107, 269 }, /* 268.50 M */
+ { },
+};
+
+static struct frac_entry frac_table_810m[] = { /* Link rate of 162M */
+ { 31, 211 }, /* 119 M */
+ { 32, 199 }, /* 130.25 M */
+ { 63, 307 }, /* 138.50 M */
+ { 11, 60 }, /* 148.50 M */
+ { 50, 263 }, /* 154 M */
+ { 31, 120 }, /* 205.25 M */
+ { 119, 359 }, /* 268.50 M */
+ { },
+};
+
+static int clk_edp_pixel_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct clk_rcg2 *rcg = to_clk_rcg2(hw);
+ struct freq_tbl f = *rcg->freq_tbl;
+ const struct frac_entry *frac;
+ int delta = 100000;
+ s64 src_rate = parent_rate;
+ s64 request;
+ u32 mask = BIT(rcg->hid_width) - 1;
+ u32 hid_div;
+
+ if (src_rate == 810000000)
+ frac = frac_table_810m;
+ else
+ frac = frac_table_675m;
+
+ for (; frac->num; frac++) {
+ request = rate;
+ request *= frac->den;
+ request = div_s64(request, frac->num);
+ if ((src_rate < (request - delta)) ||
+ (src_rate > (request + delta)))
+ continue;
+
+ regmap_read(rcg->clkr.regmap, rcg->cmd_rcgr + CFG_REG,
+ &hid_div);
+ f.pre_div = hid_div;
+ f.pre_div >>= CFG_SRC_DIV_SHIFT;
+ f.pre_div &= mask;
+ f.m = frac->num;
+ f.n = frac->den;
+
+ return clk_rcg2_configure(rcg, &f);
+ }
+
+ return -EINVAL;
+}
+
+static int clk_edp_pixel_set_rate_and_parent(struct clk_hw *hw,
+ unsigned long rate, unsigned long parent_rate, u8 index)
+{
+ /* Parent index is set statically in frequency table */
+ return clk_edp_pixel_set_rate(hw, rate, parent_rate);
+}
+
+static long clk_edp_pixel_determine_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *p_rate, struct clk **p)
+{
+ struct clk_rcg2 *rcg = to_clk_rcg2(hw);
+ const struct freq_tbl *f = rcg->freq_tbl;
+ const struct frac_entry *frac;
+ int delta = 100000;
+ s64 src_rate = *p_rate;
+ s64 request;
+ u32 mask = BIT(rcg->hid_width) - 1;
+ u32 hid_div;
+
+ /* Force the correct parent */
+ *p = clk_get_parent_by_index(hw->clk, f->src);
+
+ if (src_rate == 810000000)
+ frac = frac_table_810m;
+ else
+ frac = frac_table_675m;
+
+ for (; frac->num; frac++) {
+ request = rate;
+ request *= frac->den;
+ request = div_s64(request, frac->num);
+ if ((src_rate < (request - delta)) ||
+ (src_rate > (request + delta)))
+ continue;
+
+ regmap_read(rcg->clkr.regmap, rcg->cmd_rcgr + CFG_REG,
+ &hid_div);
+ hid_div >>= CFG_SRC_DIV_SHIFT;
+ hid_div &= mask;
+
+ return calc_rate(src_rate, frac->num, frac->den, !!frac->den,
+ hid_div);
+ }
+
+ return -EINVAL;
+}
+
+const struct clk_ops clk_edp_pixel_ops = {
+ .is_enabled = clk_rcg2_is_enabled,
+ .get_parent = clk_rcg2_get_parent,
+ .set_parent = clk_rcg2_set_parent,
+ .recalc_rate = clk_rcg2_recalc_rate,
+ .set_rate = clk_edp_pixel_set_rate,
+ .set_rate_and_parent = clk_edp_pixel_set_rate_and_parent,
+ .determine_rate = clk_edp_pixel_determine_rate,
+};
+EXPORT_SYMBOL_GPL(clk_edp_pixel_ops);
+
+static long clk_byte_determine_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *p_rate, struct clk **p)
+{
+ struct clk_rcg2 *rcg = to_clk_rcg2(hw);
+ const struct freq_tbl *f = rcg->freq_tbl;
+ unsigned long parent_rate, div;
+ u32 mask = BIT(rcg->hid_width) - 1;
+
+ if (rate == 0)
+ return -EINVAL;
+
+ *p = clk_get_parent_by_index(hw->clk, f->src);
+ *p_rate = parent_rate = __clk_round_rate(*p, rate);
+
+ div = DIV_ROUND_UP((2 * parent_rate), rate) - 1;
+ div = min_t(u32, div, mask);
+
+ return calc_rate(parent_rate, 0, 0, 0, div);
+}
+
+static int clk_byte_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct clk_rcg2 *rcg = to_clk_rcg2(hw);
+ struct freq_tbl f = *rcg->freq_tbl;
+ unsigned long div;
+ u32 mask = BIT(rcg->hid_width) - 1;
+
+ div = DIV_ROUND_UP((2 * parent_rate), rate) - 1;
+ div = min_t(u32, div, mask);
+
+ f.pre_div = div;
+
+ return clk_rcg2_configure(rcg, &f);
+}
+
+static int clk_byte_set_rate_and_parent(struct clk_hw *hw,
+ unsigned long rate, unsigned long parent_rate, u8 index)
+{
+ /* Parent index is set statically in frequency table */
+ return clk_byte_set_rate(hw, rate, parent_rate);
+}
+
+const struct clk_ops clk_byte_ops = {
+ .is_enabled = clk_rcg2_is_enabled,
+ .get_parent = clk_rcg2_get_parent,
+ .set_parent = clk_rcg2_set_parent,
+ .recalc_rate = clk_rcg2_recalc_rate,
+ .set_rate = clk_byte_set_rate,
+ .set_rate_and_parent = clk_byte_set_rate_and_parent,
+ .determine_rate = clk_byte_determine_rate,
+};
+EXPORT_SYMBOL_GPL(clk_byte_ops);
+
+static const struct frac_entry frac_table_pixel[] = {
+ { 3, 8 },
+ { 2, 9 },
+ { 4, 9 },
+ { 1, 1 },
+ { }
+};
+
+static long clk_pixel_determine_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *p_rate, struct clk **p)
+{
+ struct clk_rcg2 *rcg = to_clk_rcg2(hw);
+ unsigned long request, src_rate;
+ int delta = 100000;
+ const struct freq_tbl *f = rcg->freq_tbl;
+ const struct frac_entry *frac = frac_table_pixel;
+ struct clk *parent = *p = clk_get_parent_by_index(hw->clk, f->src);
+
+ for (; frac->num; frac++) {
+ request = (rate * frac->den) / frac->num;
+
+ src_rate = __clk_round_rate(parent, request);
+ if ((src_rate < (request - delta)) ||
+ (src_rate > (request + delta)))
+ continue;
+
+ *p_rate = src_rate;
+ return (src_rate * frac->num) / frac->den;
+ }
+
+ return -EINVAL;
+}
+
+static int clk_pixel_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct clk_rcg2 *rcg = to_clk_rcg2(hw);
+ struct freq_tbl f = *rcg->freq_tbl;
+ const struct frac_entry *frac = frac_table_pixel;
+ unsigned long request, src_rate;
+ int delta = 100000;
+ u32 mask = BIT(rcg->hid_width) - 1;
+ u32 hid_div;
+ struct clk *parent = clk_get_parent_by_index(hw->clk, f.src);
+
+ for (; frac->num; frac++) {
+ request = (rate * frac->den) / frac->num;
+
+ src_rate = __clk_round_rate(parent, request);
+ if ((src_rate < (request - delta)) ||
+ (src_rate > (request + delta)))
+ continue;
+
+ regmap_read(rcg->clkr.regmap, rcg->cmd_rcgr + CFG_REG,
+ &hid_div);
+ f.pre_div = hid_div;
+ f.pre_div >>= CFG_SRC_DIV_SHIFT;
+ f.pre_div &= mask;
+ f.m = frac->num;
+ f.n = frac->den;
+
+ return clk_rcg2_configure(rcg, &f);
+ }
+ return -EINVAL;
+}
+
+static int clk_pixel_set_rate_and_parent(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate, u8 index)
+{
+ /* Parent index is set statically in frequency table */
+ return clk_pixel_set_rate(hw, rate, parent_rate);
+}
+
+const struct clk_ops clk_pixel_ops = {
+ .is_enabled = clk_rcg2_is_enabled,
+ .get_parent = clk_rcg2_get_parent,
+ .set_parent = clk_rcg2_set_parent,
+ .recalc_rate = clk_rcg2_recalc_rate,
+ .set_rate = clk_pixel_set_rate,
+ .set_rate_and_parent = clk_pixel_set_rate_and_parent,
+ .determine_rate = clk_pixel_determine_rate,
+};
+EXPORT_SYMBOL_GPL(clk_pixel_ops);
diff --git a/drivers/clk/qcom/common.c b/drivers/clk/qcom/common.c
new file mode 100644
index 0000000..9b5a1cf
--- /dev/null
+++ b/drivers/clk/qcom/common.c
@@ -0,0 +1,101 @@
+/*
+ * Copyright (c) 2013-2014, The Linux Foundation. All rights reserved.
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/export.h>
+#include <linux/regmap.h>
+#include <linux/platform_device.h>
+#include <linux/clk-provider.h>
+#include <linux/reset-controller.h>
+
+#include "common.h"
+#include "clk-regmap.h"
+#include "reset.h"
+
+struct qcom_cc {
+ struct qcom_reset_controller reset;
+ struct clk_onecell_data data;
+ struct clk *clks[];
+};
+
+int qcom_cc_probe(struct platform_device *pdev, const struct qcom_cc_desc *desc)
+{
+ void __iomem *base;
+ struct resource *res;
+ int i, ret;
+ struct device *dev = &pdev->dev;
+ struct clk *clk;
+ struct clk_onecell_data *data;
+ struct clk **clks;
+ struct regmap *regmap;
+ struct qcom_reset_controller *reset;
+ struct qcom_cc *cc;
+ size_t num_clks = desc->num_clks;
+ struct clk_regmap **rclks = desc->clks;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+
+ regmap = devm_regmap_init_mmio(dev, base, desc->config);
+ if (IS_ERR(regmap))
+ return PTR_ERR(regmap);
+
+ cc = devm_kzalloc(dev, sizeof(*cc) + sizeof(*clks) * num_clks,
+ GFP_KERNEL);
+ if (!cc)
+ return -ENOMEM;
+
+ clks = cc->clks;
+ data = &cc->data;
+ data->clks = clks;
+ data->clk_num = num_clks;
+
+ for (i = 0; i < num_clks; i++) {
+ if (!rclks[i]) {
+ clks[i] = ERR_PTR(-ENOENT);
+ continue;
+ }
+ clk = devm_clk_register_regmap(dev, rclks[i]);
+ if (IS_ERR(clk))
+ return PTR_ERR(clk);
+ clks[i] = clk;
+ }
+
+ ret = of_clk_add_provider(dev->of_node, of_clk_src_onecell_get, data);
+ if (ret)
+ return ret;
+
+ reset = &cc->reset;
+ reset->rcdev.of_node = dev->of_node;
+ reset->rcdev.ops = &qcom_reset_ops;
+ reset->rcdev.owner = dev->driver->owner;
+ reset->rcdev.nr_resets = desc->num_resets;
+ reset->regmap = regmap;
+ reset->reset_map = desc->resets;
+ platform_set_drvdata(pdev, &reset->rcdev);
+
+ ret = reset_controller_register(&reset->rcdev);
+ if (ret)
+ of_clk_del_provider(dev->of_node);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(qcom_cc_probe);
+
+void qcom_cc_remove(struct platform_device *pdev)
+{
+ of_clk_del_provider(pdev->dev.of_node);
+ reset_controller_unregister(platform_get_drvdata(pdev));
+}
+EXPORT_SYMBOL_GPL(qcom_cc_remove);
diff --git a/drivers/clk/qcom/common.h b/drivers/clk/qcom/common.h
new file mode 100644
index 0000000..2c3cfc8
--- /dev/null
+++ b/drivers/clk/qcom/common.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright (c) 2014, The Linux Foundation. All rights reserved.
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#ifndef __QCOM_CLK_COMMON_H__
+#define __QCOM_CLK_COMMON_H__
+
+struct platform_device;
+struct regmap_config;
+struct clk_regmap;
+struct qcom_reset_map;
+
+struct qcom_cc_desc {
+ const struct regmap_config *config;
+ struct clk_regmap **clks;
+ size_t num_clks;
+ const struct qcom_reset_map *resets;
+ size_t num_resets;
+};
+
+extern int qcom_cc_probe(struct platform_device *pdev,
+ const struct qcom_cc_desc *desc);
+
+extern void qcom_cc_remove(struct platform_device *pdev);
+
+#endif
diff --git a/drivers/clk/qcom/gcc-msm8660.c b/drivers/clk/qcom/gcc-msm8660.c
index bc0b7f1..0c4b727 100644
--- a/drivers/clk/qcom/gcc-msm8660.c
+++ b/drivers/clk/qcom/gcc-msm8660.c
@@ -25,6 +25,7 @@
#include <dt-bindings/clock/qcom,gcc-msm8660.h>
#include <dt-bindings/reset/qcom,gcc-msm8660.h>
+#include "common.h"
#include "clk-regmap.h"
#include "clk-pll.h"
#include "clk-rcg.h"
@@ -2701,51 +2702,24 @@
.fast_io = true,
};
+static const struct qcom_cc_desc gcc_msm8660_desc = {
+ .config = &gcc_msm8660_regmap_config,
+ .clks = gcc_msm8660_clks,
+ .num_clks = ARRAY_SIZE(gcc_msm8660_clks),
+ .resets = gcc_msm8660_resets,
+ .num_resets = ARRAY_SIZE(gcc_msm8660_resets),
+};
+
static const struct of_device_id gcc_msm8660_match_table[] = {
{ .compatible = "qcom,gcc-msm8660" },
{ }
};
MODULE_DEVICE_TABLE(of, gcc_msm8660_match_table);
-struct qcom_cc {
- struct qcom_reset_controller reset;
- struct clk_onecell_data data;
- struct clk *clks[];
-};
-
static int gcc_msm8660_probe(struct platform_device *pdev)
{
- void __iomem *base;
- struct resource *res;
- int i, ret;
- struct device *dev = &pdev->dev;
struct clk *clk;
- struct clk_onecell_data *data;
- struct clk **clks;
- struct regmap *regmap;
- size_t num_clks;
- struct qcom_reset_controller *reset;
- struct qcom_cc *cc;
-
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- base = devm_ioremap_resource(dev, res);
- if (IS_ERR(base))
- return PTR_ERR(base);
-
- regmap = devm_regmap_init_mmio(dev, base, &gcc_msm8660_regmap_config);
- if (IS_ERR(regmap))
- return PTR_ERR(regmap);
-
- num_clks = ARRAY_SIZE(gcc_msm8660_clks);
- cc = devm_kzalloc(dev, sizeof(*cc) + sizeof(*clks) * num_clks,
- GFP_KERNEL);
- if (!cc)
- return -ENOMEM;
-
- clks = cc->clks;
- data = &cc->data;
- data->clks = clks;
- data->clk_num = num_clks;
+ struct device *dev = &pdev->dev;
/* Temporary until RPM clocks supported */
clk = clk_register_fixed_rate(dev, "cxo", NULL, CLK_IS_ROOT, 19200000);
@@ -2756,39 +2730,12 @@
if (IS_ERR(clk))
return PTR_ERR(clk);
- for (i = 0; i < num_clks; i++) {
- if (!gcc_msm8660_clks[i])
- continue;
- clk = devm_clk_register_regmap(dev, gcc_msm8660_clks[i]);
- if (IS_ERR(clk))
- return PTR_ERR(clk);
- clks[i] = clk;
- }
-
- ret = of_clk_add_provider(dev->of_node, of_clk_src_onecell_get, data);
- if (ret)
- return ret;
-
- reset = &cc->reset;
- reset->rcdev.of_node = dev->of_node;
- reset->rcdev.ops = &qcom_reset_ops,
- reset->rcdev.owner = THIS_MODULE,
- reset->rcdev.nr_resets = ARRAY_SIZE(gcc_msm8660_resets),
- reset->regmap = regmap;
- reset->reset_map = gcc_msm8660_resets,
- platform_set_drvdata(pdev, &reset->rcdev);
-
- ret = reset_controller_register(&reset->rcdev);
- if (ret)
- of_clk_del_provider(dev->of_node);
-
- return ret;
+ return qcom_cc_probe(pdev, &gcc_msm8660_desc);
}
static int gcc_msm8660_remove(struct platform_device *pdev)
{
- of_clk_del_provider(pdev->dev.of_node);
- reset_controller_unregister(platform_get_drvdata(pdev));
+ qcom_cc_remove(pdev);
return 0;
}
diff --git a/drivers/clk/qcom/gcc-msm8960.c b/drivers/clk/qcom/gcc-msm8960.c
index fd446ab..f4ffd91 100644
--- a/drivers/clk/qcom/gcc-msm8960.c
+++ b/drivers/clk/qcom/gcc-msm8960.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2013, The Linux Foundation. All rights reserved.
+ * Copyright (c) 2013-2014, The Linux Foundation. All rights reserved.
*
* This software is licensed under the terms of the GNU General Public
* License version 2, as published by the Free Software Foundation, and
@@ -25,6 +25,7 @@
#include <dt-bindings/clock/qcom,gcc-msm8960.h>
#include <dt-bindings/reset/qcom,gcc-msm8960.h>
+#include "common.h"
#include "clk-regmap.h"
#include "clk-pll.h"
#include "clk-rcg.h"
@@ -2809,7 +2810,7 @@
[PPSS_PROC_RESET] = { 0x2594, 1 },
[PPSS_RESET] = { 0x2594},
[DMA_BAM_RESET] = { 0x25c0, 7 },
- [SIC_TIC_RESET] = { 0x2600, 7 },
+ [SPS_TIC_H_RESET] = { 0x2600, 7 },
[SLIMBUS_H_RESET] = { 0x2620, 7 },
[SFAB_CFPB_M_RESET] = { 0x2680, 7 },
[SFAB_CFPB_S_RESET] = { 0x26c0, 7 },
@@ -2822,7 +2823,7 @@
[SFAB_SFPB_M_RESET] = { 0x2780, 7 },
[SFAB_SFPB_S_RESET] = { 0x27a0, 7 },
[RPM_PROC_RESET] = { 0x27c0, 7 },
- [PMIC_SSBI2_RESET] = { 0x270c, 12 },
+ [PMIC_SSBI2_RESET] = { 0x280c, 12 },
[SDC1_RESET] = { 0x2830 },
[SDC2_RESET] = { 0x2850 },
[SDC3_RESET] = { 0x2870 },
@@ -2867,6 +2868,16 @@
[RIVA_RESET] = { 0x35e0 },
};
+static struct clk_regmap *gcc_apq8064_clks[] = {
+ [PLL8] = &pll8.clkr,
+ [PLL8_VOTE] = &pll8_vote,
+ [GSBI7_UART_SRC] = &gsbi7_uart_src.clkr,
+ [GSBI7_UART_CLK] = &gsbi7_uart_clk.clkr,
+ [GSBI7_QUP_SRC] = &gsbi7_qup_src.clkr,
+ [GSBI7_QUP_CLK] = &gsbi7_qup_clk.clkr,
+ [GSBI7_H_CLK] = &gsbi7_h_clk.clkr,
+};
+
static const struct regmap_config gcc_msm8960_regmap_config = {
.reg_bits = 32,
.reg_stride = 4,
@@ -2875,51 +2886,38 @@
.fast_io = true,
};
+static const struct qcom_cc_desc gcc_msm8960_desc = {
+ .config = &gcc_msm8960_regmap_config,
+ .clks = gcc_msm8960_clks,
+ .num_clks = ARRAY_SIZE(gcc_msm8960_clks),
+ .resets = gcc_msm8960_resets,
+ .num_resets = ARRAY_SIZE(gcc_msm8960_resets),
+};
+
+static const struct qcom_cc_desc gcc_apq8064_desc = {
+ .config = &gcc_msm8960_regmap_config,
+ .clks = gcc_apq8064_clks,
+ .num_clks = ARRAY_SIZE(gcc_apq8064_clks),
+ .resets = gcc_msm8960_resets,
+ .num_resets = ARRAY_SIZE(gcc_msm8960_resets),
+};
+
static const struct of_device_id gcc_msm8960_match_table[] = {
- { .compatible = "qcom,gcc-msm8960" },
+ { .compatible = "qcom,gcc-msm8960", .data = &gcc_msm8960_desc },
+ { .compatible = "qcom,gcc-apq8064", .data = &gcc_apq8064_desc },
{ }
};
MODULE_DEVICE_TABLE(of, gcc_msm8960_match_table);
-struct qcom_cc {
- struct qcom_reset_controller reset;
- struct clk_onecell_data data;
- struct clk *clks[];
-};
-
static int gcc_msm8960_probe(struct platform_device *pdev)
{
- void __iomem *base;
- struct resource *res;
- int i, ret;
- struct device *dev = &pdev->dev;
struct clk *clk;
- struct clk_onecell_data *data;
- struct clk **clks;
- struct regmap *regmap;
- size_t num_clks;
- struct qcom_reset_controller *reset;
- struct qcom_cc *cc;
+ struct device *dev = &pdev->dev;
+ const struct of_device_id *match;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- base = devm_ioremap_resource(dev, res);
- if (IS_ERR(base))
- return PTR_ERR(base);
-
- regmap = devm_regmap_init_mmio(dev, base, &gcc_msm8960_regmap_config);
- if (IS_ERR(regmap))
- return PTR_ERR(regmap);
-
- num_clks = ARRAY_SIZE(gcc_msm8960_clks);
- cc = devm_kzalloc(dev, sizeof(*cc) + sizeof(*clks) * num_clks,
- GFP_KERNEL);
- if (!cc)
- return -ENOMEM;
-
- clks = cc->clks;
- data = &cc->data;
- data->clks = clks;
- data->clk_num = num_clks;
+ match = of_match_device(gcc_msm8960_match_table, &pdev->dev);
+ if (!match)
+ return -EINVAL;
/* Temporary until RPM clocks supported */
clk = clk_register_fixed_rate(dev, "cxo", NULL, CLK_IS_ROOT, 19200000);
@@ -2930,39 +2928,12 @@
if (IS_ERR(clk))
return PTR_ERR(clk);
- for (i = 0; i < num_clks; i++) {
- if (!gcc_msm8960_clks[i])
- continue;
- clk = devm_clk_register_regmap(dev, gcc_msm8960_clks[i]);
- if (IS_ERR(clk))
- return PTR_ERR(clk);
- clks[i] = clk;
- }
-
- ret = of_clk_add_provider(dev->of_node, of_clk_src_onecell_get, data);
- if (ret)
- return ret;
-
- reset = &cc->reset;
- reset->rcdev.of_node = dev->of_node;
- reset->rcdev.ops = &qcom_reset_ops,
- reset->rcdev.owner = THIS_MODULE,
- reset->rcdev.nr_resets = ARRAY_SIZE(gcc_msm8960_resets),
- reset->regmap = regmap;
- reset->reset_map = gcc_msm8960_resets,
- platform_set_drvdata(pdev, &reset->rcdev);
-
- ret = reset_controller_register(&reset->rcdev);
- if (ret)
- of_clk_del_provider(dev->of_node);
-
- return ret;
+ return qcom_cc_probe(pdev, match->data);
}
static int gcc_msm8960_remove(struct platform_device *pdev)
{
- of_clk_del_provider(pdev->dev.of_node);
- reset_controller_unregister(platform_get_drvdata(pdev));
+ qcom_cc_remove(pdev);
return 0;
}
diff --git a/drivers/clk/qcom/gcc-msm8974.c b/drivers/clk/qcom/gcc-msm8974.c
index 51d457e..7af7c18 100644
--- a/drivers/clk/qcom/gcc-msm8974.c
+++ b/drivers/clk/qcom/gcc-msm8974.c
@@ -25,6 +25,7 @@
#include <dt-bindings/clock/qcom,gcc-msm8974.h>
#include <dt-bindings/reset/qcom,gcc-msm8974.h>
+#include "common.h"
#include "clk-regmap.h"
#include "clk-pll.h"
#include "clk-rcg.h"
@@ -34,6 +35,7 @@
#define P_XO 0
#define P_GPLL0 1
#define P_GPLL1 1
+#define P_GPLL4 2
static const u8 gcc_xo_gpll0_map[] = {
[P_XO] = 0,
@@ -45,6 +47,18 @@
"gpll0_vote",
};
+static const u8 gcc_xo_gpll0_gpll4_map[] = {
+ [P_XO] = 0,
+ [P_GPLL0] = 1,
+ [P_GPLL4] = 5,
+};
+
+static const char *gcc_xo_gpll0_gpll4[] = {
+ "xo",
+ "gpll0_vote",
+ "gpll4_vote",
+};
+
#define F(f, s, h, m, n) { (f), (s), (2 * (h) - 1), (m), (n) }
static struct clk_pll gpll0 = {
@@ -137,6 +151,33 @@
},
};
+static struct clk_pll gpll4 = {
+ .l_reg = 0x1dc4,
+ .m_reg = 0x1dc8,
+ .n_reg = 0x1dcc,
+ .config_reg = 0x1dd4,
+ .mode_reg = 0x1dc0,
+ .status_reg = 0x1ddc,
+ .status_bit = 17,
+ .clkr.hw.init = &(struct clk_init_data){
+ .name = "gpll4",
+ .parent_names = (const char *[]){ "xo" },
+ .num_parents = 1,
+ .ops = &clk_pll_ops,
+ },
+};
+
+static struct clk_regmap gpll4_vote = {
+ .enable_reg = 0x1480,
+ .enable_mask = BIT(4),
+ .hw.init = &(struct clk_init_data){
+ .name = "gpll4_vote",
+ .parent_names = (const char *[]){ "gpll4" },
+ .num_parents = 1,
+ .ops = &clk_pll_vote_ops,
+ },
+};
+
static const struct freq_tbl ftbl_gcc_usb30_master_clk[] = {
F(125000000, P_GPLL0, 1, 5, 24),
{ }
@@ -811,18 +852,33 @@
{ }
};
+static const struct freq_tbl ftbl_gcc_sdcc1_apps_clk_pro[] = {
+ F(144000, P_XO, 16, 3, 25),
+ F(400000, P_XO, 12, 1, 4),
+ F(20000000, P_GPLL0, 15, 1, 2),
+ F(25000000, P_GPLL0, 12, 1, 2),
+ F(50000000, P_GPLL0, 12, 0, 0),
+ F(100000000, P_GPLL0, 6, 0, 0),
+ F(192000000, P_GPLL4, 4, 0, 0),
+ F(200000000, P_GPLL0, 3, 0, 0),
+ F(384000000, P_GPLL4, 2, 0, 0),
+ { }
+};
+
+static struct clk_init_data sdcc1_apps_clk_src_init = {
+ .name = "sdcc1_apps_clk_src",
+ .parent_names = gcc_xo_gpll0,
+ .num_parents = 2,
+ .ops = &clk_rcg2_ops,
+};
+
static struct clk_rcg2 sdcc1_apps_clk_src = {
.cmd_rcgr = 0x04d0,
.mnd_width = 8,
.hid_width = 5,
.parent_map = gcc_xo_gpll0_map,
.freq_tbl = ftbl_gcc_sdcc1_4_apps_clk,
- .clkr.hw.init = &(struct clk_init_data){
- .name = "sdcc1_apps_clk_src",
- .parent_names = gcc_xo_gpll0,
- .num_parents = 2,
- .ops = &clk_rcg2_ops,
- },
+ .clkr.hw.init = &sdcc1_apps_clk_src_init,
};
static struct clk_rcg2 sdcc2_apps_clk_src = {
@@ -1340,7 +1396,7 @@
};
static struct clk_branch gcc_blsp2_ahb_clk = {
- .halt_reg = 0x05c4,
+ .halt_reg = 0x0944,
.halt_check = BRANCH_HALT_VOTED,
.clkr = {
.enable_reg = 0x1484,
@@ -1994,6 +2050,38 @@
},
};
+static struct clk_branch gcc_sdcc1_cdccal_ff_clk = {
+ .halt_reg = 0x04e8,
+ .clkr = {
+ .enable_reg = 0x04e8,
+ .enable_mask = BIT(0),
+ .hw.init = &(struct clk_init_data){
+ .name = "gcc_sdcc1_cdccal_ff_clk",
+ .parent_names = (const char *[]){
+ "xo"
+ },
+ .num_parents = 1,
+ .ops = &clk_branch2_ops,
+ },
+ },
+};
+
+static struct clk_branch gcc_sdcc1_cdccal_sleep_clk = {
+ .halt_reg = 0x04e4,
+ .clkr = {
+ .enable_reg = 0x04e4,
+ .enable_mask = BIT(0),
+ .hw.init = &(struct clk_init_data){
+ .name = "gcc_sdcc1_cdccal_sleep_clk",
+ .parent_names = (const char *[]){
+ "sleep_clk_src"
+ },
+ .num_parents = 1,
+ .ops = &clk_branch2_ops,
+ },
+ },
+};
+
static struct clk_branch gcc_sdcc2_ahb_clk = {
.halt_reg = 0x0508,
.clkr = {
@@ -2483,6 +2571,10 @@
[GCC_USB_HSIC_IO_CAL_SLEEP_CLK] = &gcc_usb_hsic_io_cal_sleep_clk.clkr,
[GCC_USB_HSIC_SYSTEM_CLK] = &gcc_usb_hsic_system_clk.clkr,
[GCC_MMSS_GPLL0_CLK_SRC] = &gcc_mmss_gpll0_clk_src,
+ [GPLL4] = NULL,
+ [GPLL4_VOTE] = NULL,
+ [GCC_SDCC1_CDCCAL_SLEEP_CLK] = NULL,
+ [GCC_SDCC1_CDCCAL_FF_CLK] = NULL,
};
static const struct qcom_reset_map gcc_msm8974_resets[] = {
@@ -2574,51 +2666,51 @@
.fast_io = true,
};
+static const struct qcom_cc_desc gcc_msm8974_desc = {
+ .config = &gcc_msm8974_regmap_config,
+ .clks = gcc_msm8974_clocks,
+ .num_clks = ARRAY_SIZE(gcc_msm8974_clocks),
+ .resets = gcc_msm8974_resets,
+ .num_resets = ARRAY_SIZE(gcc_msm8974_resets),
+};
+
static const struct of_device_id gcc_msm8974_match_table[] = {
{ .compatible = "qcom,gcc-msm8974" },
+ { .compatible = "qcom,gcc-msm8974pro" , .data = (void *)1UL },
+ { .compatible = "qcom,gcc-msm8974pro-ac", .data = (void *)1UL },
{ }
};
MODULE_DEVICE_TABLE(of, gcc_msm8974_match_table);
-struct qcom_cc {
- struct qcom_reset_controller reset;
- struct clk_onecell_data data;
- struct clk *clks[];
-};
+static void msm8974_pro_clock_override(void)
+{
+ sdcc1_apps_clk_src_init.parent_names = gcc_xo_gpll0_gpll4;
+ sdcc1_apps_clk_src_init.num_parents = 3;
+ sdcc1_apps_clk_src.freq_tbl = ftbl_gcc_sdcc1_apps_clk_pro;
+ sdcc1_apps_clk_src.parent_map = gcc_xo_gpll0_gpll4_map;
+
+ gcc_msm8974_clocks[GPLL4] = &gpll4.clkr;
+ gcc_msm8974_clocks[GPLL4_VOTE] = &gpll4_vote;
+ gcc_msm8974_clocks[GCC_SDCC1_CDCCAL_SLEEP_CLK] =
+ &gcc_sdcc1_cdccal_sleep_clk.clkr;
+ gcc_msm8974_clocks[GCC_SDCC1_CDCCAL_FF_CLK] =
+ &gcc_sdcc1_cdccal_ff_clk.clkr;
+}
static int gcc_msm8974_probe(struct platform_device *pdev)
{
- void __iomem *base;
- struct resource *res;
- int i, ret;
- struct device *dev = &pdev->dev;
struct clk *clk;
- struct clk_onecell_data *data;
- struct clk **clks;
- struct regmap *regmap;
- size_t num_clks;
- struct qcom_reset_controller *reset;
- struct qcom_cc *cc;
+ struct device *dev = &pdev->dev;
+ bool pro;
+ const struct of_device_id *id;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- base = devm_ioremap_resource(dev, res);
- if (IS_ERR(base))
- return PTR_ERR(base);
+ id = of_match_device(gcc_msm8974_match_table, dev);
+ if (!id)
+ return -ENODEV;
+ pro = !!(id->data);
- regmap = devm_regmap_init_mmio(dev, base, &gcc_msm8974_regmap_config);
- if (IS_ERR(regmap))
- return PTR_ERR(regmap);
-
- num_clks = ARRAY_SIZE(gcc_msm8974_clocks);
- cc = devm_kzalloc(dev, sizeof(*cc) + sizeof(*clks) * num_clks,
- GFP_KERNEL);
- if (!cc)
- return -ENOMEM;
-
- clks = cc->clks;
- data = &cc->data;
- data->clks = clks;
- data->clk_num = num_clks;
+ if (pro)
+ msm8974_pro_clock_override();
/* Temporary until RPM clocks supported */
clk = clk_register_fixed_rate(dev, "xo", NULL, CLK_IS_ROOT, 19200000);
@@ -2631,39 +2723,12 @@
if (IS_ERR(clk))
return PTR_ERR(clk);
- for (i = 0; i < num_clks; i++) {
- if (!gcc_msm8974_clocks[i])
- continue;
- clk = devm_clk_register_regmap(dev, gcc_msm8974_clocks[i]);
- if (IS_ERR(clk))
- return PTR_ERR(clk);
- clks[i] = clk;
- }
-
- ret = of_clk_add_provider(dev->of_node, of_clk_src_onecell_get, data);
- if (ret)
- return ret;
-
- reset = &cc->reset;
- reset->rcdev.of_node = dev->of_node;
- reset->rcdev.ops = &qcom_reset_ops,
- reset->rcdev.owner = THIS_MODULE,
- reset->rcdev.nr_resets = ARRAY_SIZE(gcc_msm8974_resets),
- reset->regmap = regmap;
- reset->reset_map = gcc_msm8974_resets,
- platform_set_drvdata(pdev, &reset->rcdev);
-
- ret = reset_controller_register(&reset->rcdev);
- if (ret)
- of_clk_del_provider(dev->of_node);
-
- return ret;
+ return qcom_cc_probe(pdev, &gcc_msm8974_desc);
}
static int gcc_msm8974_remove(struct platform_device *pdev)
{
- of_clk_del_provider(pdev->dev.of_node);
- reset_controller_unregister(platform_get_drvdata(pdev));
+ qcom_cc_remove(pdev);
return 0;
}
diff --git a/drivers/clk/qcom/mmcc-msm8960.c b/drivers/clk/qcom/mmcc-msm8960.c
index f9b59c7..12f3c0b 100644
--- a/drivers/clk/qcom/mmcc-msm8960.c
+++ b/drivers/clk/qcom/mmcc-msm8960.c
@@ -26,6 +26,7 @@
#include <dt-bindings/clock/qcom,mmcc-msm8960.h>
#include <dt-bindings/reset/qcom,mmcc-msm8960.h>
+#include "common.h"
#include "clk-regmap.h"
#include "clk-pll.h"
#include "clk-rcg.h"
@@ -2222,85 +2223,28 @@
.fast_io = true,
};
+static const struct qcom_cc_desc mmcc_msm8960_desc = {
+ .config = &mmcc_msm8960_regmap_config,
+ .clks = mmcc_msm8960_clks,
+ .num_clks = ARRAY_SIZE(mmcc_msm8960_clks),
+ .resets = mmcc_msm8960_resets,
+ .num_resets = ARRAY_SIZE(mmcc_msm8960_resets),
+};
+
static const struct of_device_id mmcc_msm8960_match_table[] = {
{ .compatible = "qcom,mmcc-msm8960" },
{ }
};
MODULE_DEVICE_TABLE(of, mmcc_msm8960_match_table);
-struct qcom_cc {
- struct qcom_reset_controller reset;
- struct clk_onecell_data data;
- struct clk *clks[];
-};
-
static int mmcc_msm8960_probe(struct platform_device *pdev)
{
- void __iomem *base;
- struct resource *res;
- int i, ret;
- struct device *dev = &pdev->dev;
- struct clk *clk;
- struct clk_onecell_data *data;
- struct clk **clks;
- struct regmap *regmap;
- size_t num_clks;
- struct qcom_reset_controller *reset;
- struct qcom_cc *cc;
-
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- base = devm_ioremap_resource(dev, res);
- if (IS_ERR(base))
- return PTR_ERR(base);
-
- regmap = devm_regmap_init_mmio(dev, base, &mmcc_msm8960_regmap_config);
- if (IS_ERR(regmap))
- return PTR_ERR(regmap);
-
- num_clks = ARRAY_SIZE(mmcc_msm8960_clks);
- cc = devm_kzalloc(dev, sizeof(*cc) + sizeof(*clks) * num_clks,
- GFP_KERNEL);
- if (!cc)
- return -ENOMEM;
-
- clks = cc->clks;
- data = &cc->data;
- data->clks = clks;
- data->clk_num = num_clks;
-
- for (i = 0; i < num_clks; i++) {
- if (!mmcc_msm8960_clks[i])
- continue;
- clk = devm_clk_register_regmap(dev, mmcc_msm8960_clks[i]);
- if (IS_ERR(clk))
- return PTR_ERR(clk);
- clks[i] = clk;
- }
-
- ret = of_clk_add_provider(dev->of_node, of_clk_src_onecell_get, data);
- if (ret)
- return ret;
-
- reset = &cc->reset;
- reset->rcdev.of_node = dev->of_node;
- reset->rcdev.ops = &qcom_reset_ops,
- reset->rcdev.owner = THIS_MODULE,
- reset->rcdev.nr_resets = ARRAY_SIZE(mmcc_msm8960_resets),
- reset->regmap = regmap;
- reset->reset_map = mmcc_msm8960_resets,
- platform_set_drvdata(pdev, &reset->rcdev);
-
- ret = reset_controller_register(&reset->rcdev);
- if (ret)
- of_clk_del_provider(dev->of_node);
-
- return ret;
+ return qcom_cc_probe(pdev, &mmcc_msm8960_desc);
}
static int mmcc_msm8960_remove(struct platform_device *pdev)
{
- of_clk_del_provider(pdev->dev.of_node);
- reset_controller_unregister(platform_get_drvdata(pdev));
+ qcom_cc_remove(pdev);
return 0;
}
diff --git a/drivers/clk/qcom/mmcc-msm8974.c b/drivers/clk/qcom/mmcc-msm8974.c
index c957745..c65b905 100644
--- a/drivers/clk/qcom/mmcc-msm8974.c
+++ b/drivers/clk/qcom/mmcc-msm8974.c
@@ -25,6 +25,7 @@
#include <dt-bindings/clock/qcom,mmcc-msm8974.h>
#include <dt-bindings/reset/qcom,mmcc-msm8974.h>
+#include "common.h"
#include "clk-regmap.h"
#include "clk-pll.h"
#include "clk-rcg.h"
@@ -40,9 +41,11 @@
#define P_EDPVCO 3
#define P_GPLL1 4
#define P_DSI0PLL 4
+#define P_DSI0PLL_BYTE 4
#define P_MMPLL2 4
#define P_MMPLL3 4
#define P_DSI1PLL 5
+#define P_DSI1PLL_BYTE 5
static const u8 mmcc_xo_mmpll0_mmpll1_gpll0_map[] = {
[P_XO] = 0,
@@ -160,6 +163,24 @@
"dsi1pll",
};
+static const u8 mmcc_xo_dsibyte_hdmi_edp_gpll0_map[] = {
+ [P_XO] = 0,
+ [P_EDPLINK] = 4,
+ [P_HDMIPLL] = 3,
+ [P_GPLL0] = 5,
+ [P_DSI0PLL_BYTE] = 1,
+ [P_DSI1PLL_BYTE] = 2,
+};
+
+static const char *mmcc_xo_dsibyte_hdmi_edp_gpll0[] = {
+ "xo",
+ "edp_link_clk",
+ "hdmipll",
+ "gpll0_vote",
+ "dsi0pllbyte",
+ "dsi1pllbyte",
+};
+
#define F(f, s, h, m, n) { (f), (s), (2 * (h) - 1), (m), (n) }
static struct clk_pll mmpll0 = {
@@ -169,6 +190,7 @@
.config_reg = 0x0014,
.mode_reg = 0x0000,
.status_reg = 0x001c,
+ .status_bit = 17,
.clkr.hw.init = &(struct clk_init_data){
.name = "mmpll0",
.parent_names = (const char *[]){ "xo" },
@@ -192,9 +214,10 @@
.l_reg = 0x0044,
.m_reg = 0x0048,
.n_reg = 0x004c,
- .config_reg = 0x0054,
+ .config_reg = 0x0050,
.mode_reg = 0x0040,
.status_reg = 0x005c,
+ .status_bit = 17,
.clkr.hw.init = &(struct clk_init_data){
.name = "mmpll1",
.parent_names = (const char *[]){ "xo" },
@@ -218,7 +241,7 @@
.l_reg = 0x4104,
.m_reg = 0x4108,
.n_reg = 0x410c,
- .config_reg = 0x4114,
+ .config_reg = 0x4110,
.mode_reg = 0x4100,
.status_reg = 0x411c,
.clkr.hw.init = &(struct clk_init_data){
@@ -233,9 +256,10 @@
.l_reg = 0x0084,
.m_reg = 0x0088,
.n_reg = 0x008c,
- .config_reg = 0x0094,
+ .config_reg = 0x0090,
.mode_reg = 0x0080,
.status_reg = 0x009c,
+ .status_bit = 17,
.clkr.hw.init = &(struct clk_init_data){
.name = "mmpll3",
.parent_names = (const char *[]){ "xo" },
@@ -496,15 +520,8 @@
},
};
-static struct freq_tbl ftbl_mdss_pclk0_clk[] = {
- F(125000000, P_DSI0PLL, 2, 0, 0),
- F(250000000, P_DSI0PLL, 1, 0, 0),
- { }
-};
-
-static struct freq_tbl ftbl_mdss_pclk1_clk[] = {
- F(125000000, P_DSI1PLL, 2, 0, 0),
- F(250000000, P_DSI1PLL, 1, 0, 0),
+static struct freq_tbl pixel_freq_tbl[] = {
+ { .src = P_DSI0PLL },
{ }
};
@@ -513,12 +530,13 @@
.mnd_width = 8,
.hid_width = 5,
.parent_map = mmcc_xo_dsi_hdmi_edp_gpll0_map,
- .freq_tbl = ftbl_mdss_pclk0_clk,
+ .freq_tbl = pixel_freq_tbl,
.clkr.hw.init = &(struct clk_init_data){
.name = "pclk0_clk_src",
.parent_names = mmcc_xo_dsi_hdmi_edp_gpll0,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_pixel_ops,
+ .flags = CLK_SET_RATE_PARENT,
},
};
@@ -527,12 +545,13 @@
.mnd_width = 8,
.hid_width = 5,
.parent_map = mmcc_xo_dsi_hdmi_edp_gpll0_map,
- .freq_tbl = ftbl_mdss_pclk1_clk,
+ .freq_tbl = pixel_freq_tbl,
.clkr.hw.init = &(struct clk_init_data){
.name = "pclk1_clk_src",
.parent_names = mmcc_xo_dsi_hdmi_edp_gpll0,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_pixel_ops,
+ .flags = CLK_SET_RATE_PARENT,
},
};
@@ -750,41 +769,36 @@
},
};
-static struct freq_tbl ftbl_mdss_byte0_clk[] = {
- F(93750000, P_DSI0PLL, 8, 0, 0),
- F(187500000, P_DSI0PLL, 4, 0, 0),
- { }
-};
-
-static struct freq_tbl ftbl_mdss_byte1_clk[] = {
- F(93750000, P_DSI1PLL, 8, 0, 0),
- F(187500000, P_DSI1PLL, 4, 0, 0),
+static struct freq_tbl byte_freq_tbl[] = {
+ { .src = P_DSI0PLL_BYTE },
{ }
};
static struct clk_rcg2 byte0_clk_src = {
.cmd_rcgr = 0x2120,
.hid_width = 5,
- .parent_map = mmcc_xo_dsi_hdmi_edp_gpll0_map,
- .freq_tbl = ftbl_mdss_byte0_clk,
+ .parent_map = mmcc_xo_dsibyte_hdmi_edp_gpll0_map,
+ .freq_tbl = byte_freq_tbl,
.clkr.hw.init = &(struct clk_init_data){
.name = "byte0_clk_src",
- .parent_names = mmcc_xo_dsi_hdmi_edp_gpll0,
+ .parent_names = mmcc_xo_dsibyte_hdmi_edp_gpll0,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_byte_ops,
+ .flags = CLK_SET_RATE_PARENT,
},
};
static struct clk_rcg2 byte1_clk_src = {
.cmd_rcgr = 0x2140,
.hid_width = 5,
- .parent_map = mmcc_xo_dsi_hdmi_edp_gpll0_map,
- .freq_tbl = ftbl_mdss_byte1_clk,
+ .parent_map = mmcc_xo_dsibyte_hdmi_edp_gpll0_map,
+ .freq_tbl = byte_freq_tbl,
.clkr.hw.init = &(struct clk_init_data){
.name = "byte1_clk_src",
- .parent_names = mmcc_xo_dsi_hdmi_edp_gpll0,
+ .parent_names = mmcc_xo_dsibyte_hdmi_edp_gpll0,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_byte_ops,
+ .flags = CLK_SET_RATE_PARENT,
},
};
@@ -822,12 +836,12 @@
.parent_names = mmcc_xo_dsi_hdmi_edp_gpll0,
.num_parents = 6,
.ops = &clk_rcg2_ops,
+ .flags = CLK_SET_RATE_PARENT,
},
};
-static struct freq_tbl ftbl_mdss_edppixel_clk[] = {
- F(175000000, P_EDPVCO, 2, 0, 0),
- F(350000000, P_EDPVCO, 11, 0, 0),
+static struct freq_tbl edp_pixel_freq_tbl[] = {
+ { .src = P_EDPVCO },
{ }
};
@@ -836,12 +850,12 @@
.mnd_width = 8,
.hid_width = 5,
.parent_map = mmcc_xo_dsi_hdmi_edp_map,
- .freq_tbl = ftbl_mdss_edppixel_clk,
+ .freq_tbl = edp_pixel_freq_tbl,
.clkr.hw.init = &(struct clk_init_data){
.name = "edppixel_clk_src",
.parent_names = mmcc_xo_dsi_hdmi_edp,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_edp_pixel_ops,
},
};
@@ -853,11 +867,11 @@
static struct clk_rcg2 esc0_clk_src = {
.cmd_rcgr = 0x2160,
.hid_width = 5,
- .parent_map = mmcc_xo_dsi_hdmi_edp_gpll0_map,
+ .parent_map = mmcc_xo_dsibyte_hdmi_edp_gpll0_map,
.freq_tbl = ftbl_mdss_esc0_1_clk,
.clkr.hw.init = &(struct clk_init_data){
.name = "esc0_clk_src",
- .parent_names = mmcc_xo_dsi_hdmi_edp_gpll0,
+ .parent_names = mmcc_xo_dsibyte_hdmi_edp_gpll0,
.num_parents = 6,
.ops = &clk_rcg2_ops,
},
@@ -866,26 +880,18 @@
static struct clk_rcg2 esc1_clk_src = {
.cmd_rcgr = 0x2180,
.hid_width = 5,
- .parent_map = mmcc_xo_dsi_hdmi_edp_gpll0_map,
+ .parent_map = mmcc_xo_dsibyte_hdmi_edp_gpll0_map,
.freq_tbl = ftbl_mdss_esc0_1_clk,
.clkr.hw.init = &(struct clk_init_data){
.name = "esc1_clk_src",
- .parent_names = mmcc_xo_dsi_hdmi_edp_gpll0,
+ .parent_names = mmcc_xo_dsibyte_hdmi_edp_gpll0,
.num_parents = 6,
.ops = &clk_rcg2_ops,
},
};
-static struct freq_tbl ftbl_mdss_extpclk_clk[] = {
- F(25200000, P_HDMIPLL, 1, 0, 0),
- F(27000000, P_HDMIPLL, 1, 0, 0),
- F(27030000, P_HDMIPLL, 1, 0, 0),
- F(65000000, P_HDMIPLL, 1, 0, 0),
- F(74250000, P_HDMIPLL, 1, 0, 0),
- F(108000000, P_HDMIPLL, 1, 0, 0),
- F(148500000, P_HDMIPLL, 1, 0, 0),
- F(268500000, P_HDMIPLL, 1, 0, 0),
- F(297000000, P_HDMIPLL, 1, 0, 0),
+static struct freq_tbl extpclk_freq_tbl[] = {
+ { .src = P_HDMIPLL },
{ }
};
@@ -893,12 +899,13 @@
.cmd_rcgr = 0x2060,
.hid_width = 5,
.parent_map = mmcc_xo_dsi_hdmi_edp_gpll0_map,
- .freq_tbl = ftbl_mdss_extpclk_clk,
+ .freq_tbl = extpclk_freq_tbl,
.clkr.hw.init = &(struct clk_init_data){
.name = "extpclk_clk_src",
.parent_names = mmcc_xo_dsi_hdmi_edp_gpll0,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_byte_ops,
+ .flags = CLK_SET_RATE_PARENT,
},
};
@@ -2318,7 +2325,7 @@
.vco_val = 0x0,
.vco_mask = 0x3 << 20,
.pre_div_val = 0x0,
- .pre_div_mask = 0x3 << 12,
+ .pre_div_mask = 0x7 << 12,
.post_div_val = 0x0,
.post_div_mask = 0x3 << 8,
.mn_ena_mask = BIT(24),
@@ -2332,7 +2339,7 @@
.vco_val = 0x0,
.vco_mask = 0x3 << 20,
.pre_div_val = 0x0,
- .pre_div_mask = 0x3 << 12,
+ .pre_div_mask = 0x7 << 12,
.post_div_val = 0x0,
.post_div_mask = 0x3 << 8,
.mn_ena_mask = BIT(24),
@@ -2524,88 +2531,39 @@
.fast_io = true,
};
+static const struct qcom_cc_desc mmcc_msm8974_desc = {
+ .config = &mmcc_msm8974_regmap_config,
+ .clks = mmcc_msm8974_clocks,
+ .num_clks = ARRAY_SIZE(mmcc_msm8974_clocks),
+ .resets = mmcc_msm8974_resets,
+ .num_resets = ARRAY_SIZE(mmcc_msm8974_resets),
+};
+
static const struct of_device_id mmcc_msm8974_match_table[] = {
{ .compatible = "qcom,mmcc-msm8974" },
{ }
};
MODULE_DEVICE_TABLE(of, mmcc_msm8974_match_table);
-struct qcom_cc {
- struct qcom_reset_controller reset;
- struct clk_onecell_data data;
- struct clk *clks[];
-};
-
static int mmcc_msm8974_probe(struct platform_device *pdev)
{
- void __iomem *base;
- struct resource *res;
- int i, ret;
- struct device *dev = &pdev->dev;
- struct clk *clk;
- struct clk_onecell_data *data;
- struct clk **clks;
+ int ret;
struct regmap *regmap;
- size_t num_clks;
- struct qcom_reset_controller *reset;
- struct qcom_cc *cc;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- base = devm_ioremap_resource(dev, res);
- if (IS_ERR(base))
- return PTR_ERR(base);
-
- regmap = devm_regmap_init_mmio(dev, base, &mmcc_msm8974_regmap_config);
- if (IS_ERR(regmap))
- return PTR_ERR(regmap);
-
- num_clks = ARRAY_SIZE(mmcc_msm8974_clocks);
- cc = devm_kzalloc(dev, sizeof(*cc) + sizeof(*clks) * num_clks,
- GFP_KERNEL);
- if (!cc)
- return -ENOMEM;
-
- clks = cc->clks;
- data = &cc->data;
- data->clks = clks;
- data->clk_num = num_clks;
-
- clk_pll_configure_sr_hpm_lp(&mmpll1, regmap, &mmpll1_config, true);
- clk_pll_configure_sr_hpm_lp(&mmpll3, regmap, &mmpll3_config, false);
-
- for (i = 0; i < num_clks; i++) {
- if (!mmcc_msm8974_clocks[i])
- continue;
- clk = devm_clk_register_regmap(dev, mmcc_msm8974_clocks[i]);
- if (IS_ERR(clk))
- return PTR_ERR(clk);
- clks[i] = clk;
- }
-
- ret = of_clk_add_provider(dev->of_node, of_clk_src_onecell_get, data);
+ ret = qcom_cc_probe(pdev, &mmcc_msm8974_desc);
if (ret)
return ret;
- reset = &cc->reset;
- reset->rcdev.of_node = dev->of_node;
- reset->rcdev.ops = &qcom_reset_ops,
- reset->rcdev.owner = THIS_MODULE,
- reset->rcdev.nr_resets = ARRAY_SIZE(mmcc_msm8974_resets),
- reset->regmap = regmap;
- reset->reset_map = mmcc_msm8974_resets,
- platform_set_drvdata(pdev, &reset->rcdev);
+ regmap = dev_get_regmap(&pdev->dev, NULL);
+ clk_pll_configure_sr_hpm_lp(&mmpll1, regmap, &mmpll1_config, true);
+ clk_pll_configure_sr_hpm_lp(&mmpll3, regmap, &mmpll3_config, false);
- ret = reset_controller_register(&reset->rcdev);
- if (ret)
- of_clk_del_provider(dev->of_node);
-
- return ret;
+ return 0;
}
static int mmcc_msm8974_remove(struct platform_device *pdev)
{
- of_clk_del_provider(pdev->dev.of_node);
- reset_controller_unregister(platform_get_drvdata(pdev));
+ qcom_cc_remove(pdev);
return 0;
}
diff --git a/drivers/clk/samsung/clk-exynos4.c b/drivers/clk/samsung/clk-exynos4.c
index c4df294..4f150c9 100644
--- a/drivers/clk/samsung/clk-exynos4.c
+++ b/drivers/clk/samsung/clk-exynos4.c
@@ -324,7 +324,7 @@
.resume = exynos4_clk_resume,
};
-static void exynos4_clk_sleep_init(void)
+static void __init exynos4_clk_sleep_init(void)
{
exynos4_save_common = samsung_clk_alloc_reg_dump(exynos4_clk_regs,
ARRAY_SIZE(exynos4_clk_regs));
@@ -359,7 +359,7 @@
__func__);
}
#else
-static void exynos4_clk_sleep_init(void) {}
+static void __init exynos4_clk_sleep_init(void) {}
#endif
/* list of all parent clock list */
diff --git a/drivers/clk/shmobile/Makefile b/drivers/clk/shmobile/Makefile
index 5404cb9..e002923 100644
--- a/drivers/clk/shmobile/Makefile
+++ b/drivers/clk/shmobile/Makefile
@@ -1,5 +1,7 @@
obj-$(CONFIG_ARCH_EMEV2) += clk-emev2.o
obj-$(CONFIG_ARCH_R7S72100) += clk-rz.o
+obj-$(CONFIG_ARCH_R8A7740) += clk-r8a7740.o
+obj-$(CONFIG_ARCH_R8A7779) += clk-r8a7779.o
obj-$(CONFIG_ARCH_R8A7790) += clk-rcar-gen2.o
obj-$(CONFIG_ARCH_R8A7791) += clk-rcar-gen2.o
obj-$(CONFIG_ARCH_SHMOBILE_MULTI) += clk-div6.o
diff --git a/drivers/clk/shmobile/clk-mstp.c b/drivers/clk/shmobile/clk-mstp.c
index 1f6324e..2d2fe77 100644
--- a/drivers/clk/shmobile/clk-mstp.c
+++ b/drivers/clk/shmobile/clk-mstp.c
@@ -112,7 +112,7 @@
else
value = clk_readl(group->smstpcr);
- return !!(value & BIT(clock->bit_index));
+ return !(value & BIT(clock->bit_index));
}
static const struct clk_ops cpg_mstp_clock_ops = {
diff --git a/drivers/clk/shmobile/clk-r8a7740.c b/drivers/clk/shmobile/clk-r8a7740.c
new file mode 100644
index 0000000..1e2eaae
--- /dev/null
+++ b/drivers/clk/shmobile/clk-r8a7740.c
@@ -0,0 +1,199 @@
+/*
+ * r8a7740 Core CPG Clocks
+ *
+ * Copyright (C) 2014 Ulrich Hecht
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/clkdev.h>
+#include <linux/clk/shmobile.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/spinlock.h>
+
+struct r8a7740_cpg {
+ struct clk_onecell_data data;
+ spinlock_t lock;
+ void __iomem *reg;
+};
+
+#define CPG_FRQCRA 0x00
+#define CPG_FRQCRB 0x04
+#define CPG_PLLC2CR 0x2c
+#define CPG_USBCKCR 0x8c
+#define CPG_FRQCRC 0xe0
+
+#define CLK_ENABLE_ON_INIT BIT(0)
+
+struct div4_clk {
+ const char *name;
+ unsigned int reg;
+ unsigned int shift;
+ int flags;
+};
+
+static struct div4_clk div4_clks[] = {
+ { "i", CPG_FRQCRA, 20, CLK_ENABLE_ON_INIT },
+ { "zg", CPG_FRQCRA, 16, CLK_ENABLE_ON_INIT },
+ { "b", CPG_FRQCRA, 8, CLK_ENABLE_ON_INIT },
+ { "m1", CPG_FRQCRA, 4, CLK_ENABLE_ON_INIT },
+ { "hp", CPG_FRQCRB, 4, 0 },
+ { "hpp", CPG_FRQCRC, 20, 0 },
+ { "usbp", CPG_FRQCRC, 16, 0 },
+ { "s", CPG_FRQCRC, 12, 0 },
+ { "zb", CPG_FRQCRC, 8, 0 },
+ { "m3", CPG_FRQCRC, 4, 0 },
+ { "cp", CPG_FRQCRC, 0, 0 },
+ { NULL, 0, 0, 0 },
+};
+
+static const struct clk_div_table div4_div_table[] = {
+ { 0, 2 }, { 1, 3 }, { 2, 4 }, { 3, 6 }, { 4, 8 }, { 5, 12 },
+ { 6, 16 }, { 7, 18 }, { 8, 24 }, { 9, 32 }, { 10, 36 }, { 11, 48 },
+ { 13, 72 }, { 14, 96 }, { 0, 0 }
+};
+
+static u32 cpg_mode __initdata;
+
+static struct clk * __init
+r8a7740_cpg_register_clock(struct device_node *np, struct r8a7740_cpg *cpg,
+ const char *name)
+{
+ const struct clk_div_table *table = NULL;
+ const char *parent_name;
+ unsigned int shift, reg;
+ unsigned int mult = 1;
+ unsigned int div = 1;
+
+ if (!strcmp(name, "r")) {
+ switch (cpg_mode & (BIT(2) | BIT(1))) {
+ case BIT(1) | BIT(2):
+ /* extal1 */
+ parent_name = of_clk_get_parent_name(np, 0);
+ div = 2048;
+ break;
+ case BIT(2):
+ /* extal1 */
+ parent_name = of_clk_get_parent_name(np, 0);
+ div = 1024;
+ break;
+ default:
+ /* extalr */
+ parent_name = of_clk_get_parent_name(np, 2);
+ break;
+ }
+ } else if (!strcmp(name, "system")) {
+ parent_name = of_clk_get_parent_name(np, 0);
+ if (cpg_mode & BIT(1))
+ div = 2;
+ } else if (!strcmp(name, "pllc0")) {
+ /* PLLC0/1 are configurable multiplier clocks. Register them as
+ * fixed factor clocks for now as there's no generic multiplier
+ * clock implementation and we currently have no need to change
+ * the multiplier value.
+ */
+ u32 value = clk_readl(cpg->reg + CPG_FRQCRC);
+ parent_name = "system";
+ mult = ((value >> 24) & 0x7f) + 1;
+ } else if (!strcmp(name, "pllc1")) {
+ u32 value = clk_readl(cpg->reg + CPG_FRQCRA);
+ parent_name = "system";
+ mult = ((value >> 24) & 0x7f) + 1;
+ div = 2;
+ } else if (!strcmp(name, "pllc2")) {
+ u32 value = clk_readl(cpg->reg + CPG_PLLC2CR);
+ parent_name = "system";
+ mult = ((value >> 24) & 0x3f) + 1;
+ } else if (!strcmp(name, "usb24s")) {
+ u32 value = clk_readl(cpg->reg + CPG_USBCKCR);
+ if (value & BIT(7))
+ /* extal2 */
+ parent_name = of_clk_get_parent_name(np, 1);
+ else
+ parent_name = "system";
+ if (!(value & BIT(6)))
+ div = 2;
+ } else {
+ struct div4_clk *c;
+ for (c = div4_clks; c->name; c++) {
+ if (!strcmp(name, c->name)) {
+ parent_name = "pllc1";
+ table = div4_div_table;
+ reg = c->reg;
+ shift = c->shift;
+ break;
+ }
+ }
+ if (!c->name)
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (!table) {
+ return clk_register_fixed_factor(NULL, name, parent_name, 0,
+ mult, div);
+ } else {
+ return clk_register_divider_table(NULL, name, parent_name, 0,
+ cpg->reg + reg, shift, 4, 0,
+ table, &cpg->lock);
+ }
+}
+
+static void __init r8a7740_cpg_clocks_init(struct device_node *np)
+{
+ struct r8a7740_cpg *cpg;
+ struct clk **clks;
+ unsigned int i;
+ int num_clks;
+
+ if (of_property_read_u32(np, "renesas,mode", &cpg_mode))
+ pr_warn("%s: missing renesas,mode property\n", __func__);
+
+ num_clks = of_property_count_strings(np, "clock-output-names");
+ if (num_clks < 0) {
+ pr_err("%s: failed to count clocks\n", __func__);
+ return;
+ }
+
+ cpg = kzalloc(sizeof(*cpg), GFP_KERNEL);
+ clks = kzalloc(num_clks * sizeof(*clks), GFP_KERNEL);
+ if (cpg == NULL || clks == NULL) {
+ /* We're leaking memory on purpose, there's no point in cleaning
+ * up as the system won't boot anyway.
+ */
+ return;
+ }
+
+ spin_lock_init(&cpg->lock);
+
+ cpg->data.clks = clks;
+ cpg->data.clk_num = num_clks;
+
+ cpg->reg = of_iomap(np, 0);
+ if (WARN_ON(cpg->reg == NULL))
+ return;
+
+ for (i = 0; i < num_clks; ++i) {
+ const char *name;
+ struct clk *clk;
+
+ of_property_read_string_index(np, "clock-output-names", i,
+ &name);
+
+ clk = r8a7740_cpg_register_clock(np, cpg, name);
+ if (IS_ERR(clk))
+ pr_err("%s: failed to register %s %s clock (%ld)\n",
+ __func__, np->name, name, PTR_ERR(clk));
+ else
+ cpg->data.clks[i] = clk;
+ }
+
+ of_clk_add_provider(np, of_clk_src_onecell_get, &cpg->data);
+}
+CLK_OF_DECLARE(r8a7740_cpg_clks, "renesas,r8a7740-cpg-clocks",
+ r8a7740_cpg_clocks_init);
diff --git a/drivers/clk/shmobile/clk-r8a7779.c b/drivers/clk/shmobile/clk-r8a7779.c
new file mode 100644
index 0000000..652ecac
--- /dev/null
+++ b/drivers/clk/shmobile/clk-r8a7779.c
@@ -0,0 +1,180 @@
+/*
+ * r8a7779 Core CPG Clocks
+ *
+ * Copyright (C) 2013, 2014 Horms Solutions Ltd.
+ *
+ * Contact: Simon Horman <horms@verge.net.au>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/clkdev.h>
+#include <linux/clk/shmobile.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/spinlock.h>
+
+#include <dt-bindings/clock/r8a7779-clock.h>
+
+#define CPG_NUM_CLOCKS (R8A7779_CLK_OUT + 1)
+
+struct r8a7779_cpg {
+ struct clk_onecell_data data;
+ spinlock_t lock;
+ void __iomem *reg;
+};
+
+/* -----------------------------------------------------------------------------
+ * CPG Clock Data
+ */
+
+/*
+ * MD1 = 1 MD1 = 0
+ * (PLLA = 1500) (PLLA = 1600)
+ * (MHz) (MHz)
+ *------------------------------------------------+--------------------
+ * clkz 1000 (2/3) 800 (1/2)
+ * clkzs 250 (1/6) 200 (1/8)
+ * clki 750 (1/2) 800 (1/2)
+ * clks 250 (1/6) 200 (1/8)
+ * clks1 125 (1/12) 100 (1/16)
+ * clks3 187.5 (1/8) 200 (1/8)
+ * clks4 93.7 (1/16) 100 (1/16)
+ * clkp 62.5 (1/24) 50 (1/32)
+ * clkg 62.5 (1/24) 66.6 (1/24)
+ * clkb, CLKOUT
+ * (MD2 = 0) 62.5 (1/24) 66.6 (1/24)
+ * (MD2 = 1) 41.6 (1/36) 50 (1/32)
+ */
+
+#define CPG_CLK_CONFIG_INDEX(md) (((md) & (BIT(2)|BIT(1))) >> 1)
+
+struct cpg_clk_config {
+ unsigned int z_mult;
+ unsigned int z_div;
+ unsigned int zs_and_s_div;
+ unsigned int s1_div;
+ unsigned int p_div;
+ unsigned int b_and_out_div;
+};
+
+static const struct cpg_clk_config cpg_clk_configs[4] __initconst = {
+ { 1, 2, 8, 16, 32, 24 },
+ { 2, 3, 6, 12, 24, 24 },
+ { 1, 2, 8, 16, 32, 32 },
+ { 2, 3, 6, 12, 24, 36 },
+};
+
+/*
+ * MD PLLA Ratio
+ * 12 11
+ *------------------------
+ * 0 0 x42
+ * 0 1 x48
+ * 1 0 x56
+ * 1 1 x64
+ */
+
+#define CPG_PLLA_MULT_INDEX(md) (((md) & (BIT(12)|BIT(11))) >> 11)
+
+static const unsigned int cpg_plla_mult[4] __initconst = { 42, 48, 56, 64 };
+
+/* -----------------------------------------------------------------------------
+ * Initialization
+ */
+
+static u32 cpg_mode __initdata;
+
+static struct clk * __init
+r8a7779_cpg_register_clock(struct device_node *np, struct r8a7779_cpg *cpg,
+ const struct cpg_clk_config *config,
+ unsigned int plla_mult, const char *name)
+{
+ const char *parent_name = "plla";
+ unsigned int mult = 1;
+ unsigned int div = 1;
+
+ if (!strcmp(name, "plla")) {
+ parent_name = of_clk_get_parent_name(np, 0);
+ mult = plla_mult;
+ } else if (!strcmp(name, "z")) {
+ div = config->z_div;
+ mult = config->z_mult;
+ } else if (!strcmp(name, "zs") || !strcmp(name, "s")) {
+ div = config->zs_and_s_div;
+ } else if (!strcmp(name, "s1")) {
+ div = config->s1_div;
+ } else if (!strcmp(name, "p")) {
+ div = config->p_div;
+ } else if (!strcmp(name, "b") || !strcmp(name, "out")) {
+ div = config->b_and_out_div;
+ } else {
+ return ERR_PTR(-EINVAL);
+ }
+
+ return clk_register_fixed_factor(NULL, name, parent_name, 0, mult, div);
+}
+
+static void __init r8a7779_cpg_clocks_init(struct device_node *np)
+{
+ const struct cpg_clk_config *config;
+ struct r8a7779_cpg *cpg;
+ struct clk **clks;
+ unsigned int i, plla_mult;
+ int num_clks;
+
+ num_clks = of_property_count_strings(np, "clock-output-names");
+ if (num_clks < 0) {
+ pr_err("%s: failed to count clocks\n", __func__);
+ return;
+ }
+
+ cpg = kzalloc(sizeof(*cpg), GFP_KERNEL);
+ clks = kzalloc(CPG_NUM_CLOCKS * sizeof(*clks), GFP_KERNEL);
+ if (cpg == NULL || clks == NULL) {
+ /* We're leaking memory on purpose, there's no point in cleaning
+ * up as the system won't boot anyway.
+ */
+ return;
+ }
+
+ spin_lock_init(&cpg->lock);
+
+ cpg->data.clks = clks;
+ cpg->data.clk_num = num_clks;
+
+ config = &cpg_clk_configs[CPG_CLK_CONFIG_INDEX(cpg_mode)];
+ plla_mult = cpg_plla_mult[CPG_PLLA_MULT_INDEX(cpg_mode)];
+
+ for (i = 0; i < num_clks; ++i) {
+ const char *name;
+ struct clk *clk;
+
+ of_property_read_string_index(np, "clock-output-names", i,
+ &name);
+
+ clk = r8a7779_cpg_register_clock(np, cpg, config,
+ plla_mult, name);
+ if (IS_ERR(clk))
+ pr_err("%s: failed to register %s %s clock (%ld)\n",
+ __func__, np->name, name, PTR_ERR(clk));
+ else
+ cpg->data.clks[i] = clk;
+ }
+
+ of_clk_add_provider(np, of_clk_src_onecell_get, &cpg->data);
+}
+CLK_OF_DECLARE(r8a7779_cpg_clks, "renesas,r8a7779-cpg-clocks",
+ r8a7779_cpg_clocks_init);
+
+void __init r8a7779_clocks_init(u32 mode)
+{
+ cpg_mode = mode;
+
+ of_clk_init(NULL);
+}
diff --git a/drivers/clk/socfpga/clk-gate.c b/drivers/clk/socfpga/clk-gate.c
index 501d513..dd3a78c 100644
--- a/drivers/clk/socfpga/clk-gate.c
+++ b/drivers/clk/socfpga/clk-gate.c
@@ -32,7 +32,6 @@
#define SOCFPGA_MMC_CLK "sdmmc_clk"
#define SOCFPGA_GPIO_DB_CLK_OFFSET 0xA8
-#define div_mask(width) ((1 << (width)) - 1)
#define streq(a, b) (strcmp((a), (b)) == 0)
#define to_socfpga_gate_clk(p) container_of(p, struct socfpga_gate_clk, hw.hw)
diff --git a/drivers/clk/socfpga/clk-periph.c b/drivers/clk/socfpga/clk-periph.c
index 81623a3..46531c3 100644
--- a/drivers/clk/socfpga/clk-periph.c
+++ b/drivers/clk/socfpga/clk-periph.c
@@ -29,12 +29,18 @@
unsigned long parent_rate)
{
struct socfpga_periph_clk *socfpgaclk = to_socfpga_periph_clk(hwclk);
- u32 div;
+ u32 div, val;
- if (socfpgaclk->fixed_div)
+ if (socfpgaclk->fixed_div) {
div = socfpgaclk->fixed_div;
- else
+ } else {
+ if (socfpgaclk->div_reg) {
+ val = readl(socfpgaclk->div_reg) >> socfpgaclk->shift;
+ val &= div_mask(socfpgaclk->width);
+ parent_rate /= (val + 1);
+ }
div = ((readl(socfpgaclk->hw.reg) & 0x1ff) + 1);
+ }
return parent_rate / div;
}
@@ -54,6 +60,7 @@
struct clk_init_data init;
int rc;
u32 fixed_div;
+ u32 div_reg[3];
of_property_read_u32(node, "reg", ®);
@@ -63,6 +70,15 @@
periph_clk->hw.reg = clk_mgr_base_addr + reg;
+ rc = of_property_read_u32_array(node, "div-reg", div_reg, 3);
+ if (!rc) {
+ periph_clk->div_reg = clk_mgr_base_addr + div_reg[0];
+ periph_clk->shift = div_reg[1];
+ periph_clk->width = div_reg[2];
+ } else {
+ periph_clk->div_reg = 0;
+ }
+
rc = of_property_read_u32(node, "fixed-divider", &fixed_div);
if (rc)
periph_clk->fixed_div = 0;
diff --git a/drivers/clk/socfpga/clk.h b/drivers/clk/socfpga/clk.h
index d2e5401..d291f60 100644
--- a/drivers/clk/socfpga/clk.h
+++ b/drivers/clk/socfpga/clk.h
@@ -27,6 +27,7 @@
#define CLKMGR_PERPLL_SRC 0xAC
#define SOCFPGA_MAX_PARENTS 3
+#define div_mask(width) ((1 << (width)) - 1)
extern void __iomem *clk_mgr_base_addr;
@@ -52,6 +53,9 @@
struct clk_gate hw;
char *parent_name;
u32 fixed_div;
+ void __iomem *div_reg;
+ u32 width; /* only valid if div_reg != 0 */
+ u32 shift; /* only valid if div_reg != 0 */
};
#endif /* SOCFPGA_CLK_H */
diff --git a/drivers/clk/st/clkgen-pll.c b/drivers/clk/st/clkgen-pll.c
index a886702..d8b9b1a 100644
--- a/drivers/clk/st/clkgen-pll.c
+++ b/drivers/clk/st/clkgen-pll.c
@@ -655,6 +655,7 @@
.compatible = "st,stih416-gpu-pll-c32",
.data = &st_pll1200c32_gpu_416,
},
+ {}
};
static void __init clkgengpu_c32_pll_setup(struct device_node *np)
diff --git a/drivers/clk/sunxi/clk-factors.c b/drivers/clk/sunxi/clk-factors.c
index 9e232644..3806d97 100644
--- a/drivers/clk/sunxi/clk-factors.c
+++ b/drivers/clk/sunxi/clk-factors.c
@@ -77,6 +77,41 @@
return rate;
}
+static long clk_factors_determine_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *best_parent_rate,
+ struct clk **best_parent_p)
+{
+ struct clk *clk = hw->clk, *parent, *best_parent = NULL;
+ int i, num_parents;
+ unsigned long parent_rate, best = 0, child_rate, best_child_rate = 0;
+
+ /* find the parent that can help provide the fastest rate <= rate */
+ num_parents = __clk_get_num_parents(clk);
+ for (i = 0; i < num_parents; i++) {
+ parent = clk_get_parent_by_index(clk, i);
+ if (!parent)
+ continue;
+ if (__clk_get_flags(clk) & CLK_SET_RATE_PARENT)
+ parent_rate = __clk_round_rate(parent, rate);
+ else
+ parent_rate = __clk_get_rate(parent);
+
+ child_rate = clk_factors_round_rate(hw, rate, &parent_rate);
+
+ if (child_rate <= rate && child_rate > best_child_rate) {
+ best_parent = parent;
+ best = parent_rate;
+ best_child_rate = child_rate;
+ }
+ }
+
+ if (best_parent)
+ *best_parent_p = best_parent;
+ *best_parent_rate = best;
+
+ return best_child_rate;
+}
+
static int clk_factors_set_rate(struct clk_hw *hw, unsigned long rate,
unsigned long parent_rate)
{
@@ -113,6 +148,7 @@
}
const struct clk_ops clk_factors_ops = {
+ .determine_rate = clk_factors_determine_rate,
.recalc_rate = clk_factors_recalc_rate,
.round_rate = clk_factors_round_rate,
.set_rate = clk_factors_set_rate,
diff --git a/drivers/clk/sunxi/clk-sunxi.c b/drivers/clk/sunxi/clk-sunxi.c
index 9eddf22..4264834 100644
--- a/drivers/clk/sunxi/clk-sunxi.c
+++ b/drivers/clk/sunxi/clk-sunxi.c
@@ -507,6 +507,43 @@
/**
+ * clk_sunxi_mmc_phase_control() - configures MMC clock phase control
+ */
+
+void clk_sunxi_mmc_phase_control(struct clk *clk, u8 sample, u8 output)
+{
+ #define to_clk_composite(_hw) container_of(_hw, struct clk_composite, hw)
+ #define to_clk_factors(_hw) container_of(_hw, struct clk_factors, hw)
+
+ struct clk_hw *hw = __clk_get_hw(clk);
+ struct clk_composite *composite = to_clk_composite(hw);
+ struct clk_hw *rate_hw = composite->rate_hw;
+ struct clk_factors *factors = to_clk_factors(rate_hw);
+ unsigned long flags = 0;
+ u32 reg;
+
+ if (factors->lock)
+ spin_lock_irqsave(factors->lock, flags);
+
+ reg = readl(factors->reg);
+
+ /* set sample clock phase control */
+ reg &= ~(0x7 << 20);
+ reg |= ((sample & 0x7) << 20);
+
+ /* set output clock phase control */
+ reg &= ~(0x7 << 8);
+ reg |= ((output & 0x7) << 8);
+
+ writel(reg, factors->reg);
+
+ if (factors->lock)
+ spin_unlock_irqrestore(factors->lock, flags);
+}
+EXPORT_SYMBOL(clk_sunxi_mmc_phase_control);
+
+
+/**
* sunxi_factors_clk_setup() - Setup function for factor clocks
*/
diff --git a/drivers/clk/tegra/clk-id.h b/drivers/clk/tegra/clk-id.h
index c39613c..0011d54 100644
--- a/drivers/clk/tegra/clk-id.h
+++ b/drivers/clk/tegra/clk-id.h
@@ -233,6 +233,7 @@
tegra_clk_xusb_hs_src,
tegra_clk_xusb_ss,
tegra_clk_xusb_ss_src,
+ tegra_clk_xusb_ss_div2,
tegra_clk_max,
};
diff --git a/drivers/clk/tegra/clk-pll.c b/drivers/clk/tegra/clk-pll.c
index 6aad8ab..637b62c 100644
--- a/drivers/clk/tegra/clk-pll.c
+++ b/drivers/clk/tegra/clk-pll.c
@@ -96,10 +96,20 @@
(PLLE_SS_MAX_VAL | PLLE_SS_INC_VAL | PLLE_SS_INCINTRV_VAL)
#define PLLE_AUX_PLLP_SEL BIT(2)
+#define PLLE_AUX_USE_LOCKDET BIT(3)
#define PLLE_AUX_ENABLE_SWCTL BIT(4)
+#define PLLE_AUX_SS_SWCTL BIT(6)
#define PLLE_AUX_SEQ_ENABLE BIT(24)
+#define PLLE_AUX_SEQ_START_STATE BIT(25)
#define PLLE_AUX_PLLRE_SEL BIT(28)
+#define XUSBIO_PLL_CFG0 0x51c
+#define XUSBIO_PLL_CFG0_PADPLL_RESET_SWCTL BIT(0)
+#define XUSBIO_PLL_CFG0_CLK_ENABLE_SWCTL BIT(2)
+#define XUSBIO_PLL_CFG0_PADPLL_USE_LOCKDET BIT(6)
+#define XUSBIO_PLL_CFG0_SEQ_ENABLE BIT(24)
+#define XUSBIO_PLL_CFG0_SEQ_START_STATE BIT(25)
+
#define PLLE_MISC_PLLE_PTS BIT(8)
#define PLLE_MISC_IDDQ_SW_VALUE BIT(13)
#define PLLE_MISC_IDDQ_SW_CTRL BIT(14)
@@ -1328,7 +1338,28 @@
pll_writel(val, PLLE_SS_CTRL, pll);
udelay(1);
- /* TODO: enable hw control of xusb brick pll */
+ /* Enable hw control of xusb brick pll */
+ val = pll_readl_misc(pll);
+ val &= ~PLLE_MISC_IDDQ_SW_CTRL;
+ pll_writel_misc(val, pll);
+
+ val = pll_readl(pll->params->aux_reg, pll);
+ val |= (PLLE_AUX_USE_LOCKDET | PLLE_AUX_SEQ_START_STATE);
+ val &= ~(PLLE_AUX_ENABLE_SWCTL | PLLE_AUX_SS_SWCTL);
+ pll_writel(val, pll->params->aux_reg, pll);
+ udelay(1);
+ val |= PLLE_AUX_SEQ_ENABLE;
+ pll_writel(val, pll->params->aux_reg, pll);
+
+ val = pll_readl(XUSBIO_PLL_CFG0, pll);
+ val |= (XUSBIO_PLL_CFG0_PADPLL_USE_LOCKDET |
+ XUSBIO_PLL_CFG0_SEQ_START_STATE);
+ val &= ~(XUSBIO_PLL_CFG0_CLK_ENABLE_SWCTL |
+ XUSBIO_PLL_CFG0_PADPLL_RESET_SWCTL);
+ pll_writel(val, XUSBIO_PLL_CFG0, pll);
+ udelay(1);
+ val |= XUSBIO_PLL_CFG0_SEQ_ENABLE;
+ pll_writel(val, XUSBIO_PLL_CFG0, pll);
out:
if (pll->lock)
diff --git a/drivers/clk/tegra/clk-tegra-periph.c b/drivers/clk/tegra/clk-tegra-periph.c
index 1fa5c3f..adf6b81 100644
--- a/drivers/clk/tegra/clk-tegra-periph.c
+++ b/drivers/clk/tegra/clk-tegra-periph.c
@@ -329,7 +329,9 @@
static const char *mux_clkm_48M_pllp_480M[] = {
"clk_m", "pll_u_48M", "pll_p", "pll_u_480M"
};
-#define mux_clkm_48M_pllp_480M_idx NULL
+static u32 mux_clkm_48M_pllp_480M_idx[] = {
+ [0] = 0, [1] = 2, [2] = 4, [3] = 6,
+};
static const char *mux_clkm_pllre_clk32_480M_pllc_ref[] = {
"clk_m", "pll_re_out", "clk_32k", "pll_u_480M", "pll_c", "pll_ref"
@@ -338,6 +340,11 @@
[0] = 0, [1] = 1, [2] = 3, [3] = 3, [4] = 4, [5] = 7,
};
+static const char *mux_ss_60M[] = {
+ "xusb_ss_div2", "pll_u_60M"
+};
+#define mux_ss_60M_idx NULL
+
static const char *mux_d_audio_clk[] = {
"pll_a_out0", "pll_p", "clk_m", "spdif_in_sync", "i2s0_sync",
"i2s1_sync", "i2s2_sync", "i2s3_sync", "i2s4_sync", "vimclk_sync",
@@ -499,6 +506,7 @@
XUSB("xusb_falcon_src", mux_clkm_pllp_pllc_pllre, CLK_SOURCE_XUSB_FALCON_SRC, 143, TEGRA_PERIPH_NO_RESET, tegra_clk_xusb_falcon_src),
XUSB("xusb_fs_src", mux_clkm_48M_pllp_480M, CLK_SOURCE_XUSB_FS_SRC, 143, TEGRA_PERIPH_NO_RESET, tegra_clk_xusb_fs_src),
XUSB("xusb_ss_src", mux_clkm_pllre_clk32_480M_pllc_ref, CLK_SOURCE_XUSB_SS_SRC, 143, TEGRA_PERIPH_NO_RESET, tegra_clk_xusb_ss_src),
+ NODIV("xusb_hs_src", mux_ss_60M, CLK_SOURCE_XUSB_SS_SRC, 25, MASK(1), 143, TEGRA_PERIPH_NO_RESET, tegra_clk_xusb_hs_src, NULL),
XUSB("xusb_dev_src", mux_clkm_pllp_pllc_pllre, CLK_SOURCE_XUSB_DEV_SRC, 95, TEGRA_PERIPH_ON_APB | TEGRA_PERIPH_NO_RESET, tegra_clk_xusb_dev_src),
};
diff --git a/drivers/clk/tegra/clk-tegra114.c b/drivers/clk/tegra/clk-tegra114.c
index 80431f0..b9c8ba2 100644
--- a/drivers/clk/tegra/clk-tegra114.c
+++ b/drivers/clk/tegra/clk-tegra114.c
@@ -142,7 +142,6 @@
#define UTMIPLL_HW_PWRDN_CFG0_IDDQ_SWCTL BIT(0)
#define CLK_SOURCE_CSITE 0x1d4
-#define CLK_SOURCE_XUSB_SS_SRC 0x610
#define CLK_SOURCE_EMC 0x19c
/* PLLM override registers */
@@ -834,6 +833,7 @@
[tegra_clk_xusb_falcon_src] = { .dt_id = TEGRA114_CLK_XUSB_FALCON_SRC, .present = true },
[tegra_clk_xusb_fs_src] = { .dt_id = TEGRA114_CLK_XUSB_FS_SRC, .present = true },
[tegra_clk_xusb_ss_src] = { .dt_id = TEGRA114_CLK_XUSB_SS_SRC, .present = true },
+ [tegra_clk_xusb_ss_div2] = { .dt_id = TEGRA114_CLK_XUSB_SS_DIV2, .present = true},
[tegra_clk_xusb_dev_src] = { .dt_id = TEGRA114_CLK_XUSB_DEV_SRC, .present = true },
[tegra_clk_xusb_dev] = { .dt_id = TEGRA114_CLK_XUSB_DEV, .present = true },
[tegra_clk_xusb_hs_src] = { .dt_id = TEGRA114_CLK_XUSB_HS_SRC, .present = true },
@@ -1182,16 +1182,11 @@
void __iomem *pmc_base)
{
struct clk *clk;
- u32 val;
- /* xusb_hs_src */
- val = readl(clk_base + CLK_SOURCE_XUSB_SS_SRC);
- val |= BIT(25); /* always select PLLU_60M */
- writel(val, clk_base + CLK_SOURCE_XUSB_SS_SRC);
-
- clk = clk_register_fixed_factor(NULL, "xusb_hs_src", "pll_u_60M", 0,
- 1, 1);
- clks[TEGRA114_CLK_XUSB_HS_SRC] = clk;
+ /* xusb_ss_div2 */
+ clk = clk_register_fixed_factor(NULL, "xusb_ss_div2", "xusb_ss_src", 0,
+ 1, 2);
+ clks[TEGRA114_CLK_XUSB_SS_DIV2] = clk;
/* dsia mux */
clk = clk_register_mux(NULL, "dsia_mux", mux_plld_out0_plld2_out0,
@@ -1301,7 +1296,12 @@
{TEGRA114_CLK_GR3D, TEGRA114_CLK_PLL_C2, 300000000, 0},
{TEGRA114_CLK_DSIALP, TEGRA114_CLK_PLL_P, 68000000, 0},
{TEGRA114_CLK_DSIBLP, TEGRA114_CLK_PLL_P, 68000000, 0},
-
+ {TEGRA114_CLK_PLL_RE_VCO, TEGRA114_CLK_CLK_MAX, 612000000, 0},
+ {TEGRA114_CLK_XUSB_SS_SRC, TEGRA114_CLK_PLL_RE_OUT, 122400000, 0},
+ {TEGRA114_CLK_XUSB_FS_SRC, TEGRA114_CLK_PLL_U_48M, 48000000, 0},
+ {TEGRA114_CLK_XUSB_HS_SRC, TEGRA114_CLK_XUSB_SS_DIV2, 61200000, 0},
+ {TEGRA114_CLK_XUSB_FALCON_SRC, TEGRA114_CLK_PLL_P, 204000000, 0},
+ {TEGRA114_CLK_XUSB_HOST_SRC, TEGRA114_CLK_PLL_P, 102000000, 0},
/* This MUST be the last entry. */
{TEGRA114_CLK_CLK_MAX, TEGRA114_CLK_CLK_MAX, 0, 0},
};
diff --git a/drivers/clk/tegra/clk-tegra124.c b/drivers/clk/tegra/clk-tegra124.c
index cc37c34..80efe51 100644
--- a/drivers/clk/tegra/clk-tegra124.c
+++ b/drivers/clk/tegra/clk-tegra124.c
@@ -30,7 +30,6 @@
#define CLK_SOURCE_CSITE 0x1d4
#define CLK_SOURCE_EMC 0x19c
-#define CLK_SOURCE_XUSB_SS_SRC 0x610
#define PLLC_BASE 0x80
#define PLLC_OUT 0x84
@@ -925,6 +924,7 @@
[tegra_clk_xusb_falcon_src] = { .dt_id = TEGRA124_CLK_XUSB_FALCON_SRC, .present = true },
[tegra_clk_xusb_fs_src] = { .dt_id = TEGRA124_CLK_XUSB_FS_SRC, .present = true },
[tegra_clk_xusb_ss_src] = { .dt_id = TEGRA124_CLK_XUSB_SS_SRC, .present = true },
+ [tegra_clk_xusb_ss_div2] = { .dt_id = TEGRA124_CLK_XUSB_SS_DIV2, .present = true },
[tegra_clk_xusb_dev_src] = { .dt_id = TEGRA124_CLK_XUSB_DEV_SRC, .present = true },
[tegra_clk_xusb_dev] = { .dt_id = TEGRA124_CLK_XUSB_DEV, .present = true },
[tegra_clk_xusb_hs_src] = { .dt_id = TEGRA124_CLK_XUSB_HS_SRC, .present = true },
@@ -1105,16 +1105,11 @@
void __iomem *pmc_base)
{
struct clk *clk;
- u32 val;
- /* xusb_hs_src */
- val = readl(clk_base + CLK_SOURCE_XUSB_SS_SRC);
- val |= BIT(25); /* always select PLLU_60M */
- writel(val, clk_base + CLK_SOURCE_XUSB_SS_SRC);
-
- clk = clk_register_fixed_factor(NULL, "xusb_hs_src", "pll_u_60M", 0,
- 1, 1);
- clks[TEGRA124_CLK_XUSB_HS_SRC] = clk;
+ /* xusb_ss_div2 */
+ clk = clk_register_fixed_factor(NULL, "xusb_ss_div2", "xusb_ss_src", 0,
+ 1, 2);
+ clks[TEGRA124_CLK_XUSB_SS_DIV2] = clk;
/* dsia mux */
clk = clk_register_mux(NULL, "dsia_mux", mux_plld_out0_plld2_out0,
@@ -1368,6 +1363,12 @@
{TEGRA124_CLK_SBC4, TEGRA124_CLK_PLL_P, 12000000, 1},
{TEGRA124_CLK_TSEC, TEGRA124_CLK_PLL_C3, 0, 0},
{TEGRA124_CLK_MSENC, TEGRA124_CLK_PLL_C3, 0, 0},
+ {TEGRA124_CLK_PLL_RE_VCO, TEGRA124_CLK_CLK_MAX, 672000000, 0},
+ {TEGRA124_CLK_XUSB_SS_SRC, TEGRA124_CLK_PLL_U_480M, 120000000, 0},
+ {TEGRA124_CLK_XUSB_FS_SRC, TEGRA124_CLK_PLL_U_48M, 48000000, 0},
+ {TEGRA124_CLK_XUSB_HS_SRC, TEGRA124_CLK_PLL_U_60M, 60000000, 0},
+ {TEGRA124_CLK_XUSB_FALCON_SRC, TEGRA124_CLK_PLL_RE_OUT, 224000000, 0},
+ {TEGRA124_CLK_XUSB_HOST_SRC, TEGRA124_CLK_PLL_RE_OUT, 112000000, 0},
/* This MUST be the last entry. */
{TEGRA124_CLK_CLK_MAX, TEGRA124_CLK_CLK_MAX, 0, 0},
};
diff --git a/drivers/clk/versatile/clk-icst.c b/drivers/clk/versatile/clk-icst.c
index a820b0cf..bc96f10 100644
--- a/drivers/clk/versatile/clk-icst.c
+++ b/drivers/clk/versatile/clk-icst.c
@@ -140,6 +140,7 @@
pclone = kmemdup(desc->params, sizeof(*pclone), GFP_KERNEL);
if (!pclone) {
+ kfree(icst);
pr_err("could not clone ICST params\n");
return ERR_PTR(-ENOMEM);
}
@@ -160,3 +161,4 @@
return clk;
}
+EXPORT_SYMBOL_GPL(icst_clk_register);
diff --git a/drivers/clk/versatile/clk-impd1.c b/drivers/clk/versatile/clk-impd1.c
index 31b44f0..1cc1330 100644
--- a/drivers/clk/versatile/clk-impd1.c
+++ b/drivers/clk/versatile/clk-impd1.c
@@ -20,6 +20,8 @@
#define IMPD1_LOCK 0x08
struct impd1_clk {
+ char *pclkname;
+ struct clk *pclk;
char *vco1name;
struct clk *vco1clk;
char *vco2name;
@@ -31,7 +33,7 @@
struct clk *spiclk;
char *scname;
struct clk *scclk;
- struct clk_lookup *clks[6];
+ struct clk_lookup *clks[15];
};
/* One entry for each connected IM-PD1 LM */
@@ -86,6 +88,7 @@
{
struct impd1_clk *imc;
struct clk *clk;
+ struct clk *pclk;
int i;
if (id > 3) {
@@ -94,11 +97,18 @@
}
imc = &impd1_clks[id];
+ /* Register the fixed rate PCLK */
+ imc->pclkname = kasprintf(GFP_KERNEL, "lm%x-pclk", id);
+ pclk = clk_register_fixed_rate(NULL, imc->pclkname, NULL,
+ CLK_IS_ROOT, 0);
+ imc->pclk = pclk;
+
imc->vco1name = kasprintf(GFP_KERNEL, "lm%x-vco1", id);
clk = icst_clk_register(NULL, &impd1_icst1_desc, imc->vco1name, NULL,
base);
imc->vco1clk = clk;
- imc->clks[0] = clkdev_alloc(clk, NULL, "lm%x:01000", id);
+ imc->clks[0] = clkdev_alloc(pclk, "apb_pclk", "lm%x:01000", id);
+ imc->clks[1] = clkdev_alloc(clk, NULL, "lm%x:01000", id);
/* VCO2 is also called "CLK2" */
imc->vco2name = kasprintf(GFP_KERNEL, "lm%x-vco2", id);
@@ -107,32 +117,43 @@
imc->vco2clk = clk;
/* MMCI uses CLK2 right off */
- imc->clks[1] = clkdev_alloc(clk, NULL, "lm%x:00700", id);
+ imc->clks[2] = clkdev_alloc(pclk, "apb_pclk", "lm%x:00700", id);
+ imc->clks[3] = clkdev_alloc(clk, NULL, "lm%x:00700", id);
/* UART reference clock divides CLK2 by a fixed factor 4 */
imc->uartname = kasprintf(GFP_KERNEL, "lm%x-uartclk", id);
clk = clk_register_fixed_factor(NULL, imc->uartname, imc->vco2name,
CLK_IGNORE_UNUSED, 1, 4);
imc->uartclk = clk;
- imc->clks[2] = clkdev_alloc(clk, NULL, "lm%x:00100", id);
- imc->clks[3] = clkdev_alloc(clk, NULL, "lm%x:00200", id);
+ imc->clks[4] = clkdev_alloc(pclk, "apb_pclk", "lm%x:00100", id);
+ imc->clks[5] = clkdev_alloc(clk, NULL, "lm%x:00100", id);
+ imc->clks[6] = clkdev_alloc(pclk, "apb_pclk", "lm%x:00200", id);
+ imc->clks[7] = clkdev_alloc(clk, NULL, "lm%x:00200", id);
/* SPI PL022 clock divides CLK2 by a fixed factor 64 */
imc->spiname = kasprintf(GFP_KERNEL, "lm%x-spiclk", id);
clk = clk_register_fixed_factor(NULL, imc->spiname, imc->vco2name,
CLK_IGNORE_UNUSED, 1, 64);
- imc->clks[4] = clkdev_alloc(clk, NULL, "lm%x:00300", id);
+ imc->clks[8] = clkdev_alloc(pclk, "apb_pclk", "lm%x:00300", id);
+ imc->clks[9] = clkdev_alloc(clk, NULL, "lm%x:00300", id);
+
+ /* The GPIO blocks and AACI have only PCLK */
+ imc->clks[10] = clkdev_alloc(pclk, "apb_pclk", "lm%x:00400", id);
+ imc->clks[11] = clkdev_alloc(pclk, "apb_pclk", "lm%x:00500", id);
+ imc->clks[12] = clkdev_alloc(pclk, "apb_pclk", "lm%x:00800", id);
/* Smart Card clock divides CLK2 by a fixed factor 4 */
imc->scname = kasprintf(GFP_KERNEL, "lm%x-scclk", id);
clk = clk_register_fixed_factor(NULL, imc->scname, imc->vco2name,
CLK_IGNORE_UNUSED, 1, 4);
imc->scclk = clk;
- imc->clks[5] = clkdev_alloc(clk, NULL, "lm%x:00600", id);
+ imc->clks[13] = clkdev_alloc(pclk, "apb_pclk", "lm%x:00600", id);
+ imc->clks[14] = clkdev_alloc(clk, NULL, "lm%x:00600", id);
for (i = 0; i < ARRAY_SIZE(imc->clks); i++)
clkdev_add(imc->clks[i]);
}
+EXPORT_SYMBOL_GPL(integrator_impd1_clk_init);
void integrator_impd1_clk_exit(unsigned int id)
{
@@ -149,9 +170,12 @@
clk_unregister(imc->uartclk);
clk_unregister(imc->vco2clk);
clk_unregister(imc->vco1clk);
+ clk_unregister(imc->pclk);
kfree(imc->scname);
kfree(imc->spiname);
kfree(imc->uartname);
kfree(imc->vco2name);
kfree(imc->vco1name);
+ kfree(imc->pclkname);
}
+EXPORT_SYMBOL_GPL(integrator_impd1_clk_exit);
diff --git a/drivers/clk/zynq/clkc.c b/drivers/clk/zynq/clkc.c
index 52c09af..246cf12 100644
--- a/drivers/clk/zynq/clkc.c
+++ b/drivers/clk/zynq/clkc.c
@@ -53,6 +53,9 @@
#define NUM_MIO_PINS 54
+#define DBG_CLK_CTRL_CLKACT_TRC BIT(0)
+#define DBG_CLK_CTRL_CPU_1XCLKACT BIT(1)
+
enum zynq_clk {
armpll, ddrpll, iopll,
cpu_6or4x, cpu_3or2x, cpu_2x, cpu_1x,
@@ -499,6 +502,15 @@
clk_output_name[cpu_1x], 0, SLCR_DBG_CLK_CTRL, 1, 0,
&dbgclk_lock);
+ /* leave debug clocks in the state the bootloader set them up to */
+ tmp = clk_readl(SLCR_DBG_CLK_CTRL);
+ if (tmp & DBG_CLK_CTRL_CLKACT_TRC)
+ if (clk_prepare_enable(clks[dbg_trc]))
+ pr_warn("%s: trace clk enable failed\n", __func__);
+ if (tmp & DBG_CLK_CTRL_CPU_1XCLKACT)
+ if (clk_prepare_enable(clks[dbg_apb]))
+ pr_warn("%s: debug APB clk enable failed\n", __func__);
+
/* One gated clock for all APER clocks. */
clks[dma] = clk_register_gate(NULL, clk_output_name[dma],
clk_output_name[cpu_2x], 0, SLCR_APER_CLK_CTRL, 0, 0,
diff --git a/drivers/cpuidle/Kconfig b/drivers/cpuidle/Kconfig
index f04e25f..1b96fb9 100644
--- a/drivers/cpuidle/Kconfig
+++ b/drivers/cpuidle/Kconfig
@@ -35,6 +35,11 @@
source "drivers/cpuidle/Kconfig.arm"
endmenu
+menu "MIPS CPU Idle Drivers"
+depends on MIPS
+source "drivers/cpuidle/Kconfig.mips"
+endmenu
+
menu "POWERPC CPU Idle Drivers"
depends on PPC
source "drivers/cpuidle/Kconfig.powerpc"
diff --git a/drivers/cpuidle/Kconfig.mips b/drivers/cpuidle/Kconfig.mips
new file mode 100644
index 0000000..0e70ee2
--- /dev/null
+++ b/drivers/cpuidle/Kconfig.mips
@@ -0,0 +1,17 @@
+#
+# MIPS CPU Idle Drivers
+#
+config MIPS_CPS_CPUIDLE
+ bool "CPU Idle driver for MIPS CPS platforms"
+ depends on CPU_IDLE
+ depends on SYS_SUPPORTS_MIPS_CPS
+ select ARCH_NEEDS_CPU_IDLE_COUPLED if MIPS_MT
+ select GENERIC_CLOCKEVENTS_BROADCAST if SMP
+ select MIPS_CPS_PM
+ default y
+ help
+ Select this option to enable processor idle state management
+ through cpuidle for systems built around the MIPS Coherent
+ Processing System (CPS) architecture. In order to make use of
+ the deepest idle states you will need to ensure that you are
+ also using the CONFIG_MIPS_CPS SMP implementation.
diff --git a/drivers/cpuidle/Makefile b/drivers/cpuidle/Makefile
index 9b5b2b5..d8bb1ff 100644
--- a/drivers/cpuidle/Makefile
+++ b/drivers/cpuidle/Makefile
@@ -18,6 +18,10 @@
obj-$(CONFIG_ARM_EXYNOS_CPUIDLE) += cpuidle-exynos.o
###############################################################################
+# MIPS drivers
+obj-$(CONFIG_MIPS_CPS_CPUIDLE) += cpuidle-cps.o
+
+###############################################################################
# POWERPC drivers
obj-$(CONFIG_PSERIES_CPUIDLE) += cpuidle-pseries.o
obj-$(CONFIG_POWERNV_CPUIDLE) += cpuidle-powernv.o
diff --git a/drivers/cpuidle/cpuidle-cps.c b/drivers/cpuidle/cpuidle-cps.c
new file mode 100644
index 0000000..fc7b627
--- /dev/null
+++ b/drivers/cpuidle/cpuidle-cps.c
@@ -0,0 +1,186 @@
+/*
+ * Copyright (C) 2014 Imagination Technologies
+ * Author: Paul Burton <paul.burton@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/cpu_pm.h>
+#include <linux/cpuidle.h>
+#include <linux/init.h>
+
+#include <asm/idle.h>
+#include <asm/pm-cps.h>
+
+/* Enumeration of the various idle states this driver may enter */
+enum cps_idle_state {
+ STATE_WAIT = 0, /* MIPS wait instruction, coherent */
+ STATE_NC_WAIT, /* MIPS wait instruction, non-coherent */
+ STATE_CLOCK_GATED, /* Core clock gated */
+ STATE_POWER_GATED, /* Core power gated */
+ STATE_COUNT
+};
+
+static int cps_nc_enter(struct cpuidle_device *dev,
+ struct cpuidle_driver *drv, int index)
+{
+ enum cps_pm_state pm_state;
+ int err;
+
+ /*
+ * At least one core must remain powered up & clocked in order for the
+ * system to have any hope of functioning.
+ *
+ * TODO: don't treat core 0 specially, just prevent the final core
+ * TODO: remap interrupt affinity temporarily
+ */
+ if (!cpu_data[dev->cpu].core && (index > STATE_NC_WAIT))
+ index = STATE_NC_WAIT;
+
+ /* Select the appropriate cps_pm_state */
+ switch (index) {
+ case STATE_NC_WAIT:
+ pm_state = CPS_PM_NC_WAIT;
+ break;
+ case STATE_CLOCK_GATED:
+ pm_state = CPS_PM_CLOCK_GATED;
+ break;
+ case STATE_POWER_GATED:
+ pm_state = CPS_PM_POWER_GATED;
+ break;
+ default:
+ BUG();
+ return -EINVAL;
+ }
+
+ /* Notify listeners the CPU is about to power down */
+ if ((pm_state == CPS_PM_POWER_GATED) && cpu_pm_enter())
+ return -EINTR;
+
+ /* Enter that state */
+ err = cps_pm_enter_state(pm_state);
+
+ /* Notify listeners the CPU is back up */
+ if (pm_state == CPS_PM_POWER_GATED)
+ cpu_pm_exit();
+
+ return err ?: index;
+}
+
+static struct cpuidle_driver cps_driver = {
+ .name = "cpc_cpuidle",
+ .owner = THIS_MODULE,
+ .states = {
+ [STATE_WAIT] = MIPS_CPUIDLE_WAIT_STATE,
+ [STATE_NC_WAIT] = {
+ .enter = cps_nc_enter,
+ .exit_latency = 200,
+ .target_residency = 450,
+ .flags = CPUIDLE_FLAG_TIME_VALID,
+ .name = "nc-wait",
+ .desc = "non-coherent MIPS wait",
+ },
+ [STATE_CLOCK_GATED] = {
+ .enter = cps_nc_enter,
+ .exit_latency = 300,
+ .target_residency = 700,
+ .flags = CPUIDLE_FLAG_TIME_VALID |
+ CPUIDLE_FLAG_TIMER_STOP,
+ .name = "clock-gated",
+ .desc = "core clock gated",
+ },
+ [STATE_POWER_GATED] = {
+ .enter = cps_nc_enter,
+ .exit_latency = 600,
+ .target_residency = 1000,
+ .flags = CPUIDLE_FLAG_TIME_VALID |
+ CPUIDLE_FLAG_TIMER_STOP,
+ .name = "power-gated",
+ .desc = "core power gated",
+ },
+ },
+ .state_count = STATE_COUNT,
+ .safe_state_index = 0,
+};
+
+static void __init cps_cpuidle_unregister(void)
+{
+ int cpu;
+ struct cpuidle_device *device;
+
+ for_each_possible_cpu(cpu) {
+ device = &per_cpu(cpuidle_dev, cpu);
+ cpuidle_unregister_device(device);
+ }
+
+ cpuidle_unregister_driver(&cps_driver);
+}
+
+static int __init cps_cpuidle_init(void)
+{
+ int err, cpu, core, i;
+ struct cpuidle_device *device;
+
+ /* Detect supported states */
+ if (!cps_pm_support_state(CPS_PM_POWER_GATED))
+ cps_driver.state_count = STATE_CLOCK_GATED + 1;
+ if (!cps_pm_support_state(CPS_PM_CLOCK_GATED))
+ cps_driver.state_count = STATE_NC_WAIT + 1;
+ if (!cps_pm_support_state(CPS_PM_NC_WAIT))
+ cps_driver.state_count = STATE_WAIT + 1;
+
+ /* Inform the user if some states are unavailable */
+ if (cps_driver.state_count < STATE_COUNT) {
+ pr_info("cpuidle-cps: limited to ");
+ switch (cps_driver.state_count - 1) {
+ case STATE_WAIT:
+ pr_cont("coherent wait\n");
+ break;
+ case STATE_NC_WAIT:
+ pr_cont("non-coherent wait\n");
+ break;
+ case STATE_CLOCK_GATED:
+ pr_cont("clock gating\n");
+ break;
+ }
+ }
+
+ /*
+ * Set the coupled flag on the appropriate states if this system
+ * requires it.
+ */
+ if (coupled_coherence)
+ for (i = STATE_NC_WAIT; i < cps_driver.state_count; i++)
+ cps_driver.states[i].flags |= CPUIDLE_FLAG_COUPLED;
+
+ err = cpuidle_register_driver(&cps_driver);
+ if (err) {
+ pr_err("Failed to register CPS cpuidle driver\n");
+ return err;
+ }
+
+ for_each_possible_cpu(cpu) {
+ core = cpu_data[cpu].core;
+ device = &per_cpu(cpuidle_dev, cpu);
+ device->cpu = cpu;
+#ifdef CONFIG_MIPS_MT
+ cpumask_copy(&device->coupled_cpus, &cpu_sibling_map[cpu]);
+#endif
+
+ err = cpuidle_register_device(device);
+ if (err) {
+ pr_err("Failed to register CPU%d cpuidle device\n",
+ cpu);
+ goto err_out;
+ }
+ }
+
+ return 0;
+err_out:
+ cps_cpuidle_unregister();
+ return err;
+}
+device_initcall(cps_cpuidle_init);
diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index 03ccdb0..f066fa2 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig
@@ -301,14 +301,14 @@
found in some Freescale i.MX chips.
config CRYPTO_DEV_S5P
- tristate "Support for Samsung S5PV210 crypto accelerator"
- depends on ARCH_S5PV210
+ tristate "Support for Samsung S5PV210/Exynos crypto accelerator"
+ depends on ARCH_S5PV210 || ARCH_EXYNOS
select CRYPTO_AES
select CRYPTO_ALGAPI
select CRYPTO_BLKCIPHER
help
This option allows you to have support for S5P crypto acceleration.
- Select this to offload Samsung S5PV210 or S5PC110 from AES
+ Select this to offload Samsung S5PV210 or S5PC110, Exynos from AES
algorithms execution.
config CRYPTO_DEV_NX
diff --git a/drivers/crypto/atmel-aes.c b/drivers/crypto/atmel-aes.c
index d7c9e31..a083474 100644
--- a/drivers/crypto/atmel-aes.c
+++ b/drivers/crypto/atmel-aes.c
@@ -716,6 +716,12 @@
return -EINVAL;
}
ctx->block_size = CFB32_BLOCK_SIZE;
+ } else if (mode & AES_FLAGS_CFB64) {
+ if (!IS_ALIGNED(req->nbytes, CFB64_BLOCK_SIZE)) {
+ pr_err("request size is not exact amount of CFB64 blocks\n");
+ return -EINVAL;
+ }
+ ctx->block_size = CFB64_BLOCK_SIZE;
} else {
if (!IS_ALIGNED(req->nbytes, AES_BLOCK_SIZE)) {
pr_err("request size is not exact amount of AES blocks\n");
@@ -1069,7 +1075,7 @@
.cra_driver_name = "atmel-cfb8-aes",
.cra_priority = 100,
.cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER | CRYPTO_ALG_ASYNC,
- .cra_blocksize = CFB64_BLOCK_SIZE,
+ .cra_blocksize = CFB8_BLOCK_SIZE,
.cra_ctxsize = sizeof(struct atmel_aes_ctx),
.cra_alignmask = 0x0,
.cra_type = &crypto_ablkcipher_type,
diff --git a/drivers/crypto/bfin_crc.c b/drivers/crypto/bfin_crc.c
index c9ff298..b099e33 100644
--- a/drivers/crypto/bfin_crc.c
+++ b/drivers/crypto/bfin_crc.c
@@ -29,10 +29,11 @@
#include <crypto/hash.h>
#include <crypto/internal/hash.h>
-#include <asm/blackfin.h>
-#include <asm/bfin_crc.h>
#include <asm/dma.h>
#include <asm/portmux.h>
+#include <asm/io.h>
+
+#include "bfin_crc.h"
#define CRC_CCRYPTO_QUEUE_LENGTH 5
@@ -54,12 +55,13 @@
int irq;
int dma_ch;
u32 poly;
- volatile struct crc_register *regs;
+ struct crc_register *regs;
struct ahash_request *req; /* current request in operation */
struct dma_desc_array *sg_cpu; /* virt addr of sg dma descriptors */
dma_addr_t sg_dma; /* phy addr of sg dma descriptors */
u8 *sg_mid_buf;
+ dma_addr_t sg_mid_dma; /* phy addr of sg mid buffer */
struct tasklet_struct done_task;
struct crypto_queue queue; /* waiting requests */
@@ -132,13 +134,13 @@
static int bfin_crypto_crc_init_hw(struct bfin_crypto_crc *crc, u32 key)
{
- crc->regs->datacntrld = 0;
- crc->regs->control = MODE_CALC_CRC << OPMODE_OFFSET;
- crc->regs->curresult = key;
+ writel(0, &crc->regs->datacntrld);
+ writel(MODE_CALC_CRC << OPMODE_OFFSET, &crc->regs->control);
+ writel(key, &crc->regs->curresult);
/* setup CRC interrupts */
- crc->regs->status = CMPERRI | DCNTEXPI;
- crc->regs->intrenset = CMPERRI | DCNTEXPI;
+ writel(CMPERRI | DCNTEXPI, &crc->regs->status);
+ writel(CMPERRI | DCNTEXPI, &crc->regs->intrenset);
return 0;
}
@@ -194,7 +196,6 @@
dma_map_sg(crc->dev, ctx->sg, ctx->sg_nents, DMA_TO_DEVICE);
for_each_sg(ctx->sg, sg, ctx->sg_nents, j) {
- dma_config = DMAFLOW_ARRAY | RESTART | NDSIZE_3 | DMAEN | PSIZE_32;
dma_addr = sg_dma_address(sg);
/* deduce extra bytes in last sg */
if (sg_is_last(sg))
@@ -207,12 +208,29 @@
bytes in current sg buffer. Move addr of current
sg and deduce the length of current sg.
*/
- memcpy(crc->sg_mid_buf +((i-1) << 2) + mid_dma_count,
- (void *)dma_addr,
+ memcpy(crc->sg_mid_buf +(i << 2) + mid_dma_count,
+ sg_virt(sg),
CHKSUM_DIGEST_SIZE - mid_dma_count);
dma_addr += CHKSUM_DIGEST_SIZE - mid_dma_count;
dma_count -= CHKSUM_DIGEST_SIZE - mid_dma_count;
+
+ dma_config = DMAFLOW_ARRAY | RESTART | NDSIZE_3 |
+ DMAEN | PSIZE_32 | WDSIZE_32;
+
+ /* setup new dma descriptor for next middle dma */
+ crc->sg_cpu[i].start_addr = crc->sg_mid_dma + (i << 2);
+ crc->sg_cpu[i].cfg = dma_config;
+ crc->sg_cpu[i].x_count = 1;
+ crc->sg_cpu[i].x_modify = CHKSUM_DIGEST_SIZE;
+ dev_dbg(crc->dev, "%d: crc_dma: start_addr:0x%lx, "
+ "cfg:0x%lx, x_count:0x%lx, x_modify:0x%lx\n",
+ i, crc->sg_cpu[i].start_addr,
+ crc->sg_cpu[i].cfg, crc->sg_cpu[i].x_count,
+ crc->sg_cpu[i].x_modify);
+ i++;
}
+
+ dma_config = DMAFLOW_ARRAY | RESTART | NDSIZE_3 | DMAEN | PSIZE_32;
/* chop current sg dma len to multiple of 32 bits */
mid_dma_count = dma_count % 4;
dma_count &= ~0x3;
@@ -243,24 +261,9 @@
if (mid_dma_count) {
/* copy extra bytes to next middle dma buffer */
- dma_config = DMAFLOW_ARRAY | RESTART | NDSIZE_3 |
- DMAEN | PSIZE_32 | WDSIZE_32;
memcpy(crc->sg_mid_buf + (i << 2),
- (void *)(dma_addr + (dma_count << 2)),
+ (u8*)sg_virt(sg) + (dma_count << 2),
mid_dma_count);
- /* setup new dma descriptor for next middle dma */
- crc->sg_cpu[i].start_addr = dma_map_single(crc->dev,
- crc->sg_mid_buf + (i << 2),
- CHKSUM_DIGEST_SIZE, DMA_TO_DEVICE);
- crc->sg_cpu[i].cfg = dma_config;
- crc->sg_cpu[i].x_count = 1;
- crc->sg_cpu[i].x_modify = CHKSUM_DIGEST_SIZE;
- dev_dbg(crc->dev, "%d: crc_dma: start_addr:0x%lx, "
- "cfg:0x%lx, x_count:0x%lx, x_modify:0x%lx\n",
- i, crc->sg_cpu[i].start_addr,
- crc->sg_cpu[i].cfg, crc->sg_cpu[i].x_count,
- crc->sg_cpu[i].x_modify);
- i++;
}
}
@@ -303,6 +306,7 @@
int nsg, i, j;
unsigned int nextlen;
unsigned long flags;
+ u32 reg;
spin_lock_irqsave(&crc->lock, flags);
if (req)
@@ -402,13 +406,14 @@
ctx->sg_buflen += CHKSUM_DIGEST_SIZE;
/* set CRC data count before start DMA */
- crc->regs->datacnt = ctx->sg_buflen >> 2;
+ writel(ctx->sg_buflen >> 2, &crc->regs->datacnt);
/* setup and enable CRC DMA */
bfin_crypto_crc_config_dma(crc);
/* finally kick off CRC operation */
- crc->regs->control |= BLKEN;
+ reg = readl(&crc->regs->control);
+ writel(reg | BLKEN, &crc->regs->control);
return -EINPROGRESS;
}
@@ -529,14 +534,17 @@
static irqreturn_t bfin_crypto_crc_handler(int irq, void *dev_id)
{
struct bfin_crypto_crc *crc = dev_id;
+ u32 reg;
- if (crc->regs->status & DCNTEXP) {
- crc->regs->status = DCNTEXP;
+ if (readl(&crc->regs->status) & DCNTEXP) {
+ writel(DCNTEXP, &crc->regs->status);
/* prepare results */
- put_unaligned_le32(crc->regs->result, crc->req->result);
+ put_unaligned_le32(readl(&crc->regs->result),
+ crc->req->result);
- crc->regs->control &= ~BLKEN;
+ reg = readl(&crc->regs->control);
+ writel(reg & ~BLKEN, &crc->regs->control);
crc->busy = 0;
if (crc->req->base.complete)
@@ -560,7 +568,7 @@
struct bfin_crypto_crc *crc = platform_get_drvdata(pdev);
int i = 100000;
- while ((crc->regs->control & BLKEN) && --i)
+ while ((readl(&crc->regs->control) & BLKEN) && --i)
cpu_relax();
if (i == 0)
@@ -647,29 +655,32 @@
* 1 last + 1 next dma descriptors
*/
crc->sg_mid_buf = (u8 *)(crc->sg_cpu + ((CRC_MAX_DMA_DESC + 1) << 1));
+ crc->sg_mid_dma = crc->sg_dma + sizeof(struct dma_desc_array)
+ * ((CRC_MAX_DMA_DESC + 1) << 1);
- crc->regs->control = 0;
- crc->regs->poly = crc->poly = (u32)pdev->dev.platform_data;
+ writel(0, &crc->regs->control);
+ crc->poly = (u32)pdev->dev.platform_data;
+ writel(crc->poly, &crc->regs->poly);
- while (!(crc->regs->status & LUTDONE) && (--timeout) > 0)
+ while (!(readl(&crc->regs->status) & LUTDONE) && (--timeout) > 0)
cpu_relax();
if (timeout == 0)
dev_info(&pdev->dev, "init crc poly timeout\n");
+ platform_set_drvdata(pdev, crc);
+
spin_lock(&crc_list.lock);
list_add(&crc->list, &crc_list.dev_list);
spin_unlock(&crc_list.lock);
- platform_set_drvdata(pdev, crc);
-
- ret = crypto_register_ahash(&algs);
- if (ret) {
- spin_lock(&crc_list.lock);
- list_del(&crc->list);
- spin_unlock(&crc_list.lock);
- dev_err(&pdev->dev, "Cann't register crypto ahash device\n");
- goto out_error_dma;
+ if (list_is_singular(&crc_list.dev_list)) {
+ ret = crypto_register_ahash(&algs);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "Can't register crypto ahash device\n");
+ goto out_error_dma;
+ }
}
dev_info(&pdev->dev, "initialized\n");
diff --git a/drivers/crypto/bfin_crc.h b/drivers/crypto/bfin_crc.h
new file mode 100644
index 0000000..75cef4dc
--- /dev/null
+++ b/drivers/crypto/bfin_crc.h
@@ -0,0 +1,125 @@
+/*
+ * bfin_crc.h - interface to Blackfin CRC controllers
+ *
+ * Copyright 2012 Analog Devices Inc.
+ *
+ * Licensed under the GPL-2 or later.
+ */
+
+#ifndef __BFIN_CRC_H__
+#define __BFIN_CRC_H__
+
+/* Function driver which use hardware crc must initialize the structure */
+struct crc_info {
+ /* Input data address */
+ unsigned char *in_addr;
+ /* Output data address */
+ unsigned char *out_addr;
+ /* Input or output bytes */
+ unsigned long datasize;
+ union {
+ /* CRC to compare with that of input buffer */
+ unsigned long crc_compare;
+ /* Value to compare with input data */
+ unsigned long val_verify;
+ /* Value to fill */
+ unsigned long val_fill;
+ };
+ /* Value to program the 32b CRC Polynomial */
+ unsigned long crc_poly;
+ union {
+ /* CRC calculated from the input data */
+ unsigned long crc_result;
+ /* First failed position to verify input data */
+ unsigned long pos_verify;
+ };
+ /* CRC mirror flags */
+ unsigned int bitmirr:1;
+ unsigned int bytmirr:1;
+ unsigned int w16swp:1;
+ unsigned int fdsel:1;
+ unsigned int rsltmirr:1;
+ unsigned int polymirr:1;
+ unsigned int cmpmirr:1;
+};
+
+/* Userspace interface */
+#define CRC_IOC_MAGIC 'C'
+#define CRC_IOC_CALC_CRC _IOWR('C', 0x01, unsigned int)
+#define CRC_IOC_MEMCPY_CRC _IOWR('C', 0x02, unsigned int)
+#define CRC_IOC_VERIFY_VAL _IOWR('C', 0x03, unsigned int)
+#define CRC_IOC_FILL_VAL _IOWR('C', 0x04, unsigned int)
+
+
+#ifdef __KERNEL__
+
+#include <linux/types.h>
+#include <linux/spinlock.h>
+#include <linux/miscdevice.h>
+
+struct crc_register {
+ u32 control;
+ u32 datacnt;
+ u32 datacntrld;
+ u32 __pad_1[2];
+ u32 compare;
+ u32 fillval;
+ u32 datafifo;
+ u32 intren;
+ u32 intrenset;
+ u32 intrenclr;
+ u32 poly;
+ u32 __pad_2[4];
+ u32 status;
+ u32 datacntcap;
+ u32 __pad_3;
+ u32 result;
+ u32 curresult;
+ u32 __pad_4[3];
+ u32 revid;
+};
+
+/* CRC_STATUS Masks */
+#define CMPERR 0x00000002 /* Compare error */
+#define DCNTEXP 0x00000010 /* datacnt register expired */
+#define IBR 0x00010000 /* Input buffer ready */
+#define OBR 0x00020000 /* Output buffer ready */
+#define IRR 0x00040000 /* Immediate result readt */
+#define LUTDONE 0x00080000 /* Look-up table generation done */
+#define FSTAT 0x00700000 /* FIFO status */
+#define MAX_FIFO 4 /* Max fifo size */
+
+/* CRC_CONTROL Masks */
+#define BLKEN 0x00000001 /* Block enable */
+#define OPMODE 0x000000F0 /* Operation mode */
+#define OPMODE_OFFSET 4 /* Operation mode mask offset*/
+#define MODE_DMACPY_CRC 1 /* MTM CRC compute and compare */
+#define MODE_DATA_FILL 2 /* MTM data fill */
+#define MODE_CALC_CRC 3 /* MSM CRC compute and compare */
+#define MODE_DATA_VERIFY 4 /* MSM data verify */
+#define AUTOCLRZ 0x00000100 /* Auto clear to zero */
+#define AUTOCLRF 0x00000200 /* Auto clear to one */
+#define OBRSTALL 0x00001000 /* Stall on output buffer ready */
+#define IRRSTALL 0x00002000 /* Stall on immediate result ready */
+#define BITMIRR 0x00010000 /* Mirror bits within each byte of 32-bit input data */
+#define BITMIRR_OFFSET 16 /* Mirror bits offset */
+#define BYTMIRR 0x00020000 /* Mirror bytes of 32-bit input data */
+#define BYTMIRR_OFFSET 17 /* Mirror bytes offset */
+#define W16SWP 0x00040000 /* Mirror uppper and lower 16-bit word of 32-bit input data */
+#define W16SWP_OFFSET 18 /* Mirror 16-bit word offset */
+#define FDSEL 0x00080000 /* FIFO is written after input data is mirrored */
+#define FDSEL_OFFSET 19 /* Mirror FIFO offset */
+#define RSLTMIRR 0x00100000 /* CRC result registers are mirrored. */
+#define RSLTMIRR_OFFSET 20 /* Mirror CRC result offset. */
+#define POLYMIRR 0x00200000 /* CRC poly register is mirrored. */
+#define POLYMIRR_OFFSET 21 /* Mirror CRC poly offset. */
+#define CMPMIRR 0x00400000 /* CRC compare register is mirrored. */
+#define CMPMIRR_OFFSET 22 /* Mirror CRC compare offset. */
+
+/* CRC_INTREN Masks */
+#define CMPERRI 0x02 /* CRC_ERROR_INTR */
+#define DCNTEXPI 0x10 /* CRC_STATUS_INTR */
+
+#endif
+
+#endif
diff --git a/drivers/crypto/caam/caamalg.c b/drivers/crypto/caam/caamalg.c
index 5f89125..c09ce1f 100644
--- a/drivers/crypto/caam/caamalg.c
+++ b/drivers/crypto/caam/caamalg.c
@@ -303,6 +303,7 @@
* Job Descriptor and Shared Descriptors
* must all fit into the 64-word Descriptor h/w Buffer
*/
+ keys_fit_inline = false;
if (DESC_AEAD_NULL_DEC_LEN + DESC_JOB_IO_LEN +
ctx->split_key_pad_len <= CAAM_DESC_BYTES_MAX)
keys_fit_inline = true;
@@ -472,6 +473,7 @@
* Job Descriptor and Shared Descriptors
* must all fit into the 64-word Descriptor h/w Buffer
*/
+ keys_fit_inline = false;
if (DESC_AEAD_DEC_LEN + DESC_JOB_IO_LEN +
ctx->split_key_pad_len + ctx->enckeylen <=
CAAM_DESC_BYTES_MAX)
@@ -527,6 +529,7 @@
* Job Descriptor and Shared Descriptors
* must all fit into the 64-word Descriptor h/w Buffer
*/
+ keys_fit_inline = false;
if (DESC_AEAD_GIVENC_LEN + DESC_JOB_IO_LEN +
ctx->split_key_pad_len + ctx->enckeylen <=
CAAM_DESC_BYTES_MAX)
@@ -918,11 +921,8 @@
edesc = (struct aead_edesc *)((char *)desc -
offsetof(struct aead_edesc, hw_desc));
- if (err) {
- char tmp[CAAM_ERROR_STR_MAX];
-
- dev_err(jrdev, "%08x: %s\n", err, caam_jr_strstatus(tmp, err));
- }
+ if (err)
+ caam_jr_strstatus(jrdev, err);
aead_unmap(jrdev, edesc, req);
@@ -969,11 +969,8 @@
req->cryptlen - ctx->authsize, 1);
#endif
- if (err) {
- char tmp[CAAM_ERROR_STR_MAX];
-
- dev_err(jrdev, "%08x: %s\n", err, caam_jr_strstatus(tmp, err));
- }
+ if (err)
+ caam_jr_strstatus(jrdev, err);
aead_unmap(jrdev, edesc, req);
@@ -1018,11 +1015,8 @@
edesc = (struct ablkcipher_edesc *)((char *)desc -
offsetof(struct ablkcipher_edesc, hw_desc));
- if (err) {
- char tmp[CAAM_ERROR_STR_MAX];
-
- dev_err(jrdev, "%08x: %s\n", err, caam_jr_strstatus(tmp, err));
- }
+ if (err)
+ caam_jr_strstatus(jrdev, err);
#ifdef DEBUG
print_hex_dump(KERN_ERR, "dstiv @"__stringify(__LINE__)": ",
@@ -1053,11 +1047,8 @@
edesc = (struct ablkcipher_edesc *)((char *)desc -
offsetof(struct ablkcipher_edesc, hw_desc));
- if (err) {
- char tmp[CAAM_ERROR_STR_MAX];
-
- dev_err(jrdev, "%08x: %s\n", err, caam_jr_strstatus(tmp, err));
- }
+ if (err)
+ caam_jr_strstatus(jrdev, err);
#ifdef DEBUG
print_hex_dump(KERN_ERR, "dstiv @"__stringify(__LINE__)": ",
diff --git a/drivers/crypto/caam/caamhash.c b/drivers/crypto/caam/caamhash.c
index 0378328..0d9284e 100644
--- a/drivers/crypto/caam/caamhash.c
+++ b/drivers/crypto/caam/caamhash.c
@@ -545,7 +545,8 @@
DMA_TO_DEVICE);
if (dma_mapping_error(jrdev, ctx->key_dma)) {
dev_err(jrdev, "unable to map key i/o memory\n");
- return -ENOMEM;
+ ret = -ENOMEM;
+ goto map_err;
}
#ifdef DEBUG
print_hex_dump(KERN_ERR, "ctx.key@"__stringify(__LINE__)": ",
@@ -559,6 +560,7 @@
DMA_TO_DEVICE);
}
+map_err:
kfree(hashed_key);
return ret;
badkey:
@@ -631,11 +633,8 @@
edesc = (struct ahash_edesc *)((char *)desc -
offsetof(struct ahash_edesc, hw_desc));
- if (err) {
- char tmp[CAAM_ERROR_STR_MAX];
-
- dev_err(jrdev, "%08x: %s\n", err, caam_jr_strstatus(tmp, err));
- }
+ if (err)
+ caam_jr_strstatus(jrdev, err);
ahash_unmap(jrdev, edesc, req, digestsize);
kfree(edesc);
@@ -669,11 +668,8 @@
edesc = (struct ahash_edesc *)((char *)desc -
offsetof(struct ahash_edesc, hw_desc));
- if (err) {
- char tmp[CAAM_ERROR_STR_MAX];
-
- dev_err(jrdev, "%08x: %s\n", err, caam_jr_strstatus(tmp, err));
- }
+ if (err)
+ caam_jr_strstatus(jrdev, err);
ahash_unmap_ctx(jrdev, edesc, req, ctx->ctx_len, DMA_BIDIRECTIONAL);
kfree(edesc);
@@ -707,11 +703,8 @@
edesc = (struct ahash_edesc *)((char *)desc -
offsetof(struct ahash_edesc, hw_desc));
- if (err) {
- char tmp[CAAM_ERROR_STR_MAX];
-
- dev_err(jrdev, "%08x: %s\n", err, caam_jr_strstatus(tmp, err));
- }
+ if (err)
+ caam_jr_strstatus(jrdev, err);
ahash_unmap_ctx(jrdev, edesc, req, digestsize, DMA_FROM_DEVICE);
kfree(edesc);
@@ -745,11 +738,8 @@
edesc = (struct ahash_edesc *)((char *)desc -
offsetof(struct ahash_edesc, hw_desc));
- if (err) {
- char tmp[CAAM_ERROR_STR_MAX];
-
- dev_err(jrdev, "%08x: %s\n", err, caam_jr_strstatus(tmp, err));
- }
+ if (err)
+ caam_jr_strstatus(jrdev, err);
ahash_unmap_ctx(jrdev, edesc, req, ctx->ctx_len, DMA_TO_DEVICE);
kfree(edesc);
diff --git a/drivers/crypto/caam/caamrng.c b/drivers/crypto/caam/caamrng.c
index 3529b540..8c07d31 100644
--- a/drivers/crypto/caam/caamrng.c
+++ b/drivers/crypto/caam/caamrng.c
@@ -103,11 +103,8 @@
bd = (struct buf_data *)((char *)desc -
offsetof(struct buf_data, hw_desc));
- if (err) {
- char tmp[CAAM_ERROR_STR_MAX];
-
- dev_err(jrdev, "%08x: %s\n", err, caam_jr_strstatus(tmp, err));
- }
+ if (err)
+ caam_jr_strstatus(jrdev, err);
atomic_set(&bd->empty, BUF_NOT_EMPTY);
complete(&bd->filled);
diff --git a/drivers/crypto/caam/error.c b/drivers/crypto/caam/error.c
index 0eabd81..6531054 100644
--- a/drivers/crypto/caam/error.c
+++ b/drivers/crypto/caam/error.c
@@ -11,247 +11,208 @@
#include "jr.h"
#include "error.h"
-#define SPRINTFCAT(str, format, param, max_alloc) \
-{ \
- char *tmp; \
- \
- tmp = kmalloc(sizeof(format) + max_alloc, GFP_ATOMIC); \
- if (likely(tmp)) { \
- sprintf(tmp, format, param); \
- strcat(str, tmp); \
- kfree(tmp); \
- } else { \
- strcat(str, "kmalloc failure in SPRINTFCAT"); \
- } \
-}
+static const struct {
+ u8 value;
+ const char *error_text;
+} desc_error_list[] = {
+ { 0x00, "No error." },
+ { 0x01, "SGT Length Error. The descriptor is trying to read more data than is contained in the SGT table." },
+ { 0x02, "SGT Null Entry Error." },
+ { 0x03, "Job Ring Control Error. There is a bad value in the Job Ring Control register." },
+ { 0x04, "Invalid Descriptor Command. The Descriptor Command field is invalid." },
+ { 0x05, "Reserved." },
+ { 0x06, "Invalid KEY Command" },
+ { 0x07, "Invalid LOAD Command" },
+ { 0x08, "Invalid STORE Command" },
+ { 0x09, "Invalid OPERATION Command" },
+ { 0x0A, "Invalid FIFO LOAD Command" },
+ { 0x0B, "Invalid FIFO STORE Command" },
+ { 0x0C, "Invalid MOVE/MOVE_LEN Command" },
+ { 0x0D, "Invalid JUMP Command. A nonlocal JUMP Command is invalid because the target is not a Job Header Command, or the jump is from a Trusted Descriptor to a Job Descriptor, or because the target Descriptor contains a Shared Descriptor." },
+ { 0x0E, "Invalid MATH Command" },
+ { 0x0F, "Invalid SIGNATURE Command" },
+ { 0x10, "Invalid Sequence Command. A SEQ IN PTR OR SEQ OUT PTR Command is invalid or a SEQ KEY, SEQ LOAD, SEQ FIFO LOAD, or SEQ FIFO STORE decremented the input or output sequence length below 0. This error may result if a built-in PROTOCOL Command has encountered a malformed PDU." },
+ { 0x11, "Skip data type invalid. The type must be 0xE or 0xF."},
+ { 0x12, "Shared Descriptor Header Error" },
+ { 0x13, "Header Error. Invalid length or parity, or certain other problems." },
+ { 0x14, "Burster Error. Burster has gotten to an illegal state" },
+ { 0x15, "Context Register Length Error. The descriptor is trying to read or write past the end of the Context Register. A SEQ LOAD or SEQ STORE with the VLF bit set was executed with too large a length in the variable length register (VSOL for SEQ STORE or VSIL for SEQ LOAD)." },
+ { 0x16, "DMA Error" },
+ { 0x17, "Reserved." },
+ { 0x1A, "Job failed due to JR reset" },
+ { 0x1B, "Job failed due to Fail Mode" },
+ { 0x1C, "DECO Watchdog timer timeout error" },
+ { 0x1D, "DECO tried to copy a key from another DECO but the other DECO's Key Registers were locked" },
+ { 0x1E, "DECO attempted to copy data from a DECO that had an unmasked Descriptor error" },
+ { 0x1F, "LIODN error. DECO was trying to share from itself or from another DECO but the two Non-SEQ LIODN values didn't match or the 'shared from' DECO's Descriptor required that the SEQ LIODNs be the same and they aren't." },
+ { 0x20, "DECO has completed a reset initiated via the DRR register" },
+ { 0x21, "Nonce error. When using EKT (CCM) key encryption option in the FIFO STORE Command, the Nonce counter reached its maximum value and this encryption mode can no longer be used." },
+ { 0x22, "Meta data is too large (> 511 bytes) for TLS decap (input frame; block ciphers) and IPsec decap (output frame, when doing the next header byte update) and DCRC (output frame)." },
+ { 0x23, "Read Input Frame error" },
+ { 0x24, "JDKEK, TDKEK or TDSK not loaded error" },
+ { 0x80, "DNR (do not run) error" },
+ { 0x81, "undefined protocol command" },
+ { 0x82, "invalid setting in PDB" },
+ { 0x83, "Anti-replay LATE error" },
+ { 0x84, "Anti-replay REPLAY error" },
+ { 0x85, "Sequence number overflow" },
+ { 0x86, "Sigver invalid signature" },
+ { 0x87, "DSA Sign Illegal test descriptor" },
+ { 0x88, "Protocol Format Error - A protocol has seen an error in the format of data received. When running RSA, this means that formatting with random padding was used, and did not follow the form: 0x00, 0x02, 8-to-N bytes of non-zero pad, 0x00, F data." },
+ { 0x89, "Protocol Size Error - A protocol has seen an error in size. When running RSA, pdb size N < (size of F) when no formatting is used; or pdb size N < (F + 11) when formatting is used." },
+ { 0xC1, "Blob Command error: Undefined mode" },
+ { 0xC2, "Blob Command error: Secure Memory Blob mode error" },
+ { 0xC4, "Blob Command error: Black Blob key or input size error" },
+ { 0xC5, "Blob Command error: Invalid key destination" },
+ { 0xC8, "Blob Command error: Trusted/Secure mode error" },
+ { 0xF0, "IPsec TTL or hop limit field either came in as 0, or was decremented to 0" },
+ { 0xF1, "3GPP HFN matches or exceeds the Threshold" },
+};
-static void report_jump_idx(u32 status, char *outstr)
+static const char * const cha_id_list[] = {
+ "",
+ "AES",
+ "DES",
+ "ARC4",
+ "MDHA",
+ "RNG",
+ "SNOW f8",
+ "Kasumi f8/9",
+ "PKHA",
+ "CRCA",
+ "SNOW f9",
+ "ZUCE",
+ "ZUCA",
+};
+
+static const char * const err_id_list[] = {
+ "No error.",
+ "Mode error.",
+ "Data size error.",
+ "Key size error.",
+ "PKHA A memory size error.",
+ "PKHA B memory size error.",
+ "Data arrived out of sequence error.",
+ "PKHA divide-by-zero error.",
+ "PKHA modulus even error.",
+ "DES key parity error.",
+ "ICV check failed.",
+ "Hardware error.",
+ "Unsupported CCM AAD size.",
+ "Class 1 CHA is not reset",
+ "Invalid CHA combination was selected",
+ "Invalid CHA selected.",
+};
+
+static const char * const rng_err_id_list[] = {
+ "",
+ "",
+ "",
+ "Instantiate",
+ "Not instantiated",
+ "Test instantiate",
+ "Prediction resistance",
+ "Prediction resistance and test request",
+ "Uninstantiate",
+ "Secure key generation",
+};
+
+static void report_ccb_status(struct device *jrdev, const u32 status,
+ const char *error)
{
- u8 idx = (status & JRSTA_DECOERR_INDEX_MASK) >>
- JRSTA_DECOERR_INDEX_SHIFT;
-
- if (status & JRSTA_DECOERR_JUMP)
- strcat(outstr, "jump tgt desc idx ");
- else
- strcat(outstr, "desc idx ");
-
- SPRINTFCAT(outstr, "%d: ", idx, sizeof("255"));
-}
-
-static void report_ccb_status(u32 status, char *outstr)
-{
- static const char * const cha_id_list[] = {
- "",
- "AES",
- "DES",
- "ARC4",
- "MDHA",
- "RNG",
- "SNOW f8",
- "Kasumi f8/9",
- "PKHA",
- "CRCA",
- "SNOW f9",
- "ZUCE",
- "ZUCA",
- };
- static const char * const err_id_list[] = {
- "No error.",
- "Mode error.",
- "Data size error.",
- "Key size error.",
- "PKHA A memory size error.",
- "PKHA B memory size error.",
- "Data arrived out of sequence error.",
- "PKHA divide-by-zero error.",
- "PKHA modulus even error.",
- "DES key parity error.",
- "ICV check failed.",
- "Hardware error.",
- "Unsupported CCM AAD size.",
- "Class 1 CHA is not reset",
- "Invalid CHA combination was selected",
- "Invalid CHA selected.",
- };
- static const char * const rng_err_id_list[] = {
- "",
- "",
- "",
- "Instantiate",
- "Not instantiated",
- "Test instantiate",
- "Prediction resistance",
- "Prediction resistance and test request",
- "Uninstantiate",
- "Secure key generation",
- };
u8 cha_id = (status & JRSTA_CCBERR_CHAID_MASK) >>
JRSTA_CCBERR_CHAID_SHIFT;
u8 err_id = status & JRSTA_CCBERR_ERRID_MASK;
+ u8 idx = (status & JRSTA_DECOERR_INDEX_MASK) >>
+ JRSTA_DECOERR_INDEX_SHIFT;
+ char *idx_str;
+ const char *cha_str = "unidentified cha_id value 0x";
+ char cha_err_code[3] = { 0 };
+ const char *err_str = "unidentified err_id value 0x";
+ char err_err_code[3] = { 0 };
- report_jump_idx(status, outstr);
+ if (status & JRSTA_DECOERR_JUMP)
+ idx_str = "jump tgt desc idx";
+ else
+ idx_str = "desc idx";
- if (cha_id < ARRAY_SIZE(cha_id_list)) {
- SPRINTFCAT(outstr, "%s: ", cha_id_list[cha_id],
- strlen(cha_id_list[cha_id]));
- } else {
- SPRINTFCAT(outstr, "unidentified cha_id value 0x%02x: ",
- cha_id, sizeof("ff"));
- }
+ if (cha_id < ARRAY_SIZE(cha_id_list))
+ cha_str = cha_id_list[cha_id];
+ else
+ snprintf(cha_err_code, sizeof(cha_err_code), "%02x", cha_id);
if ((cha_id << JRSTA_CCBERR_CHAID_SHIFT) == JRSTA_CCBERR_CHAID_RNG &&
err_id < ARRAY_SIZE(rng_err_id_list) &&
strlen(rng_err_id_list[err_id])) {
/* RNG-only error */
- SPRINTFCAT(outstr, "%s", rng_err_id_list[err_id],
- strlen(rng_err_id_list[err_id]));
- } else if (err_id < ARRAY_SIZE(err_id_list)) {
- SPRINTFCAT(outstr, "%s", err_id_list[err_id],
- strlen(err_id_list[err_id]));
- } else {
- SPRINTFCAT(outstr, "unidentified err_id value 0x%02x",
- err_id, sizeof("ff"));
- }
+ err_str = rng_err_id_list[err_id];
+ } else if (err_id < ARRAY_SIZE(err_id_list))
+ err_str = err_id_list[err_id];
+ else
+ snprintf(err_err_code, sizeof(err_err_code), "%02x", err_id);
+
+ dev_err(jrdev, "%08x: %s: %s %d: %s%s: %s%s\n",
+ status, error, idx_str, idx,
+ cha_str, cha_err_code,
+ err_str, err_err_code);
}
-static void report_jump_status(u32 status, char *outstr)
+static void report_jump_status(struct device *jrdev, const u32 status,
+ const char *error)
{
- SPRINTFCAT(outstr, "%s() not implemented", __func__, sizeof(__func__));
+ dev_err(jrdev, "%08x: %s: %s() not implemented\n",
+ status, error, __func__);
}
-static void report_deco_status(u32 status, char *outstr)
+static void report_deco_status(struct device *jrdev, const u32 status,
+ const char *error)
{
- static const struct {
- u8 value;
- char *error_text;
- } desc_error_list[] = {
- { 0x00, "No error." },
- { 0x01, "SGT Length Error. The descriptor is trying to read "
- "more data than is contained in the SGT table." },
- { 0x02, "SGT Null Entry Error." },
- { 0x03, "Job Ring Control Error. There is a bad value in the "
- "Job Ring Control register." },
- { 0x04, "Invalid Descriptor Command. The Descriptor Command "
- "field is invalid." },
- { 0x05, "Reserved." },
- { 0x06, "Invalid KEY Command" },
- { 0x07, "Invalid LOAD Command" },
- { 0x08, "Invalid STORE Command" },
- { 0x09, "Invalid OPERATION Command" },
- { 0x0A, "Invalid FIFO LOAD Command" },
- { 0x0B, "Invalid FIFO STORE Command" },
- { 0x0C, "Invalid MOVE/MOVE_LEN Command" },
- { 0x0D, "Invalid JUMP Command. A nonlocal JUMP Command is "
- "invalid because the target is not a Job Header "
- "Command, or the jump is from a Trusted Descriptor to "
- "a Job Descriptor, or because the target Descriptor "
- "contains a Shared Descriptor." },
- { 0x0E, "Invalid MATH Command" },
- { 0x0F, "Invalid SIGNATURE Command" },
- { 0x10, "Invalid Sequence Command. A SEQ IN PTR OR SEQ OUT PTR "
- "Command is invalid or a SEQ KEY, SEQ LOAD, SEQ FIFO "
- "LOAD, or SEQ FIFO STORE decremented the input or "
- "output sequence length below 0. This error may result "
- "if a built-in PROTOCOL Command has encountered a "
- "malformed PDU." },
- { 0x11, "Skip data type invalid. The type must be 0xE or 0xF."},
- { 0x12, "Shared Descriptor Header Error" },
- { 0x13, "Header Error. Invalid length or parity, or certain "
- "other problems." },
- { 0x14, "Burster Error. Burster has gotten to an illegal "
- "state" },
- { 0x15, "Context Register Length Error. The descriptor is "
- "trying to read or write past the end of the Context "
- "Register. A SEQ LOAD or SEQ STORE with the VLF bit "
- "set was executed with too large a length in the "
- "variable length register (VSOL for SEQ STORE or VSIL "
- "for SEQ LOAD)." },
- { 0x16, "DMA Error" },
- { 0x17, "Reserved." },
- { 0x1A, "Job failed due to JR reset" },
- { 0x1B, "Job failed due to Fail Mode" },
- { 0x1C, "DECO Watchdog timer timeout error" },
- { 0x1D, "DECO tried to copy a key from another DECO but the "
- "other DECO's Key Registers were locked" },
- { 0x1E, "DECO attempted to copy data from a DECO that had an "
- "unmasked Descriptor error" },
- { 0x1F, "LIODN error. DECO was trying to share from itself or "
- "from another DECO but the two Non-SEQ LIODN values "
- "didn't match or the 'shared from' DECO's Descriptor "
- "required that the SEQ LIODNs be the same and they "
- "aren't." },
- { 0x20, "DECO has completed a reset initiated via the DRR "
- "register" },
- { 0x21, "Nonce error. When using EKT (CCM) key encryption "
- "option in the FIFO STORE Command, the Nonce counter "
- "reached its maximum value and this encryption mode "
- "can no longer be used." },
- { 0x22, "Meta data is too large (> 511 bytes) for TLS decap "
- "(input frame; block ciphers) and IPsec decap (output "
- "frame, when doing the next header byte update) and "
- "DCRC (output frame)." },
- { 0x23, "Read Input Frame error" },
- { 0x24, "JDKEK, TDKEK or TDSK not loaded error" },
- { 0x80, "DNR (do not run) error" },
- { 0x81, "undefined protocol command" },
- { 0x82, "invalid setting in PDB" },
- { 0x83, "Anti-replay LATE error" },
- { 0x84, "Anti-replay REPLAY error" },
- { 0x85, "Sequence number overflow" },
- { 0x86, "Sigver invalid signature" },
- { 0x87, "DSA Sign Illegal test descriptor" },
- { 0x88, "Protocol Format Error - A protocol has seen an error "
- "in the format of data received. When running RSA, "
- "this means that formatting with random padding was "
- "used, and did not follow the form: 0x00, 0x02, 8-to-N "
- "bytes of non-zero pad, 0x00, F data." },
- { 0x89, "Protocol Size Error - A protocol has seen an error in "
- "size. When running RSA, pdb size N < (size of F) when "
- "no formatting is used; or pdb size N < (F + 11) when "
- "formatting is used." },
- { 0xC1, "Blob Command error: Undefined mode" },
- { 0xC2, "Blob Command error: Secure Memory Blob mode error" },
- { 0xC4, "Blob Command error: Black Blob key or input size "
- "error" },
- { 0xC5, "Blob Command error: Invalid key destination" },
- { 0xC8, "Blob Command error: Trusted/Secure mode error" },
- { 0xF0, "IPsec TTL or hop limit field either came in as 0, "
- "or was decremented to 0" },
- { 0xF1, "3GPP HFN matches or exceeds the Threshold" },
- };
- u8 desc_error = status & JRSTA_DECOERR_ERROR_MASK;
+ u8 err_id = status & JRSTA_DECOERR_ERROR_MASK;
+ u8 idx = (status & JRSTA_DECOERR_INDEX_MASK) >>
+ JRSTA_DECOERR_INDEX_SHIFT;
+ char *idx_str;
+ const char *err_str = "unidentified error value 0x";
+ char err_err_code[3] = { 0 };
int i;
- report_jump_idx(status, outstr);
+ if (status & JRSTA_DECOERR_JUMP)
+ idx_str = "jump tgt desc idx";
+ else
+ idx_str = "desc idx";
for (i = 0; i < ARRAY_SIZE(desc_error_list); i++)
- if (desc_error_list[i].value == desc_error)
+ if (desc_error_list[i].value == err_id)
break;
- if (i != ARRAY_SIZE(desc_error_list) && desc_error_list[i].error_text) {
- SPRINTFCAT(outstr, "%s", desc_error_list[i].error_text,
- strlen(desc_error_list[i].error_text));
- } else {
- SPRINTFCAT(outstr, "unidentified error value 0x%02x",
- desc_error, sizeof("ff"));
- }
+ if (i != ARRAY_SIZE(desc_error_list) && desc_error_list[i].error_text)
+ err_str = desc_error_list[i].error_text;
+ else
+ snprintf(err_err_code, sizeof(err_err_code), "%02x", err_id);
+
+ dev_err(jrdev, "%08x: %s: %s %d: %s%s\n",
+ status, error, idx_str, idx, err_str, err_err_code);
}
-static void report_jr_status(u32 status, char *outstr)
+static void report_jr_status(struct device *jrdev, const u32 status,
+ const char *error)
{
- SPRINTFCAT(outstr, "%s() not implemented", __func__, sizeof(__func__));
+ dev_err(jrdev, "%08x: %s: %s() not implemented\n",
+ status, error, __func__);
}
-static void report_cond_code_status(u32 status, char *outstr)
+static void report_cond_code_status(struct device *jrdev, const u32 status,
+ const char *error)
{
- SPRINTFCAT(outstr, "%s() not implemented", __func__, sizeof(__func__));
+ dev_err(jrdev, "%08x: %s: %s() not implemented\n",
+ status, error, __func__);
}
-char *caam_jr_strstatus(char *outstr, u32 status)
+void caam_jr_strstatus(struct device *jrdev, u32 status)
{
static const struct stat_src {
- void (*report_ssed)(u32 status, char *outstr);
- char *error;
+ void (*report_ssed)(struct device *jrdev, const u32 status,
+ const char *error);
+ const char *error;
} status_src[] = {
{ NULL, "No error" },
{ NULL, NULL },
@@ -263,12 +224,16 @@
{ report_cond_code_status, "Condition Code" },
};
u32 ssrc = status >> JRSTA_SSRC_SHIFT;
+ const char *error = status_src[ssrc].error;
- sprintf(outstr, "%s: ", status_src[ssrc].error);
-
- if (status_src[ssrc].report_ssed)
- status_src[ssrc].report_ssed(status, outstr);
-
- return outstr;
+ /*
+ * If there is no further error handling function, just
+ * print the error code, error string and exit. Otherwise
+ * call the handler function.
+ */
+ if (!status_src[ssrc].report_ssed)
+ dev_err(jrdev, "%08x: %s: \n", status, status_src[ssrc].error);
+ else
+ status_src[ssrc].report_ssed(jrdev, status, error);
}
EXPORT_SYMBOL(caam_jr_strstatus);
diff --git a/drivers/crypto/caam/error.h b/drivers/crypto/caam/error.h
index 02c7baa..b6350b0 100644
--- a/drivers/crypto/caam/error.h
+++ b/drivers/crypto/caam/error.h
@@ -7,5 +7,5 @@
#ifndef CAAM_ERROR_H
#define CAAM_ERROR_H
#define CAAM_ERROR_STR_MAX 302
-extern char *caam_jr_strstatus(char *outstr, u32 status);
+void caam_jr_strstatus(struct device *jrdev, u32 status);
#endif /* CAAM_ERROR_H */
diff --git a/drivers/crypto/caam/key_gen.c b/drivers/crypto/caam/key_gen.c
index ea2e406..871703c 100644
--- a/drivers/crypto/caam/key_gen.c
+++ b/drivers/crypto/caam/key_gen.c
@@ -19,11 +19,8 @@
dev_err(dev, "%s %d: err 0x%x\n", __func__, __LINE__, err);
#endif
- if (err) {
- char tmp[CAAM_ERROR_STR_MAX];
-
- dev_err(dev, "%08x: %s\n", err, caam_jr_strstatus(tmp, err));
- }
+ if (err)
+ caam_jr_strstatus(dev, err);
res->err = err;
diff --git a/drivers/crypto/ccp/ccp-crypto-aes-xts.c b/drivers/crypto/ccp/ccp-crypto-aes-xts.c
index 0237ab5..0cc5594 100644
--- a/drivers/crypto/ccp/ccp-crypto-aes-xts.c
+++ b/drivers/crypto/ccp/ccp-crypto-aes-xts.c
@@ -191,12 +191,12 @@
ctx->complete = ccp_aes_xts_complete;
ctx->u.aes.key_len = 0;
- fallback_tfm = crypto_alloc_ablkcipher(tfm->__crt_alg->cra_name, 0,
+ fallback_tfm = crypto_alloc_ablkcipher(crypto_tfm_alg_name(tfm), 0,
CRYPTO_ALG_ASYNC |
CRYPTO_ALG_NEED_FALLBACK);
if (IS_ERR(fallback_tfm)) {
pr_warn("could not load fallback driver %s\n",
- tfm->__crt_alg->cra_name);
+ crypto_tfm_alg_name(tfm));
return PTR_ERR(fallback_tfm);
}
ctx->u.aes.tfm_ablkcipher = fallback_tfm;
diff --git a/drivers/crypto/ccp/ccp-pci.c b/drivers/crypto/ccp/ccp-pci.c
index 93319f9..0d74623 100644
--- a/drivers/crypto/ccp/ccp-pci.c
+++ b/drivers/crypto/ccp/ccp-pci.c
@@ -48,12 +48,11 @@
for (v = 0; v < ARRAY_SIZE(msix_entry); v++)
msix_entry[v].entry = v;
- while ((ret = pci_enable_msix(pdev, msix_entry, v)) > 0)
- v = ret;
- if (ret)
+ ret = pci_enable_msix_range(pdev, msix_entry, 1, v);
+ if (ret < 0)
return ret;
- ccp_pci->msix_count = v;
+ ccp_pci->msix_count = ret;
for (v = 0; v < ccp_pci->msix_count; v++) {
/* Set the interrupt names and request the irqs */
snprintf(ccp_pci->msix[v].name, name_len, "ccp-%u", v);
diff --git a/drivers/crypto/geode-aes.c b/drivers/crypto/geode-aes.c
index 0c9ff49..fe538e5 100644
--- a/drivers/crypto/geode-aes.c
+++ b/drivers/crypto/geode-aes.c
@@ -226,7 +226,7 @@
op->dst = (void *) out;
op->mode = AES_MODE_ECB;
op->flags = 0;
- op->len = AES_MIN_BLOCK_SIZE;
+ op->len = AES_BLOCK_SIZE;
op->dir = AES_DIR_ENCRYPT;
geode_aes_crypt(op);
@@ -247,7 +247,7 @@
op->dst = (void *) out;
op->mode = AES_MODE_ECB;
op->flags = 0;
- op->len = AES_MIN_BLOCK_SIZE;
+ op->len = AES_BLOCK_SIZE;
op->dir = AES_DIR_DECRYPT;
geode_aes_crypt(op);
@@ -255,7 +255,7 @@
static int fallback_init_cip(struct crypto_tfm *tfm)
{
- const char *name = tfm->__crt_alg->cra_name;
+ const char *name = crypto_tfm_alg_name(tfm);
struct geode_aes_op *op = crypto_tfm_ctx(tfm);
op->fallback.cip = crypto_alloc_cipher(name, 0,
@@ -286,7 +286,7 @@
CRYPTO_ALG_NEED_FALLBACK,
.cra_init = fallback_init_cip,
.cra_exit = fallback_exit_cip,
- .cra_blocksize = AES_MIN_BLOCK_SIZE,
+ .cra_blocksize = AES_BLOCK_SIZE,
.cra_ctxsize = sizeof(struct geode_aes_op),
.cra_module = THIS_MODULE,
.cra_u = {
@@ -320,7 +320,7 @@
op->src = walk.src.virt.addr,
op->dst = walk.dst.virt.addr;
op->mode = AES_MODE_CBC;
- op->len = nbytes - (nbytes % AES_MIN_BLOCK_SIZE);
+ op->len = nbytes - (nbytes % AES_BLOCK_SIZE);
op->dir = AES_DIR_DECRYPT;
ret = geode_aes_crypt(op);
@@ -352,7 +352,7 @@
op->src = walk.src.virt.addr,
op->dst = walk.dst.virt.addr;
op->mode = AES_MODE_CBC;
- op->len = nbytes - (nbytes % AES_MIN_BLOCK_SIZE);
+ op->len = nbytes - (nbytes % AES_BLOCK_SIZE);
op->dir = AES_DIR_ENCRYPT;
ret = geode_aes_crypt(op);
@@ -365,7 +365,7 @@
static int fallback_init_blk(struct crypto_tfm *tfm)
{
- const char *name = tfm->__crt_alg->cra_name;
+ const char *name = crypto_tfm_alg_name(tfm);
struct geode_aes_op *op = crypto_tfm_ctx(tfm);
op->fallback.blk = crypto_alloc_blkcipher(name, 0,
@@ -396,7 +396,7 @@
CRYPTO_ALG_NEED_FALLBACK,
.cra_init = fallback_init_blk,
.cra_exit = fallback_exit_blk,
- .cra_blocksize = AES_MIN_BLOCK_SIZE,
+ .cra_blocksize = AES_BLOCK_SIZE,
.cra_ctxsize = sizeof(struct geode_aes_op),
.cra_alignmask = 15,
.cra_type = &crypto_blkcipher_type,
@@ -408,7 +408,7 @@
.setkey = geode_setkey_blk,
.encrypt = geode_cbc_encrypt,
.decrypt = geode_cbc_decrypt,
- .ivsize = AES_IV_LENGTH,
+ .ivsize = AES_BLOCK_SIZE,
}
}
};
@@ -432,7 +432,7 @@
op->src = walk.src.virt.addr,
op->dst = walk.dst.virt.addr;
op->mode = AES_MODE_ECB;
- op->len = nbytes - (nbytes % AES_MIN_BLOCK_SIZE);
+ op->len = nbytes - (nbytes % AES_BLOCK_SIZE);
op->dir = AES_DIR_DECRYPT;
ret = geode_aes_crypt(op);
@@ -462,7 +462,7 @@
op->src = walk.src.virt.addr,
op->dst = walk.dst.virt.addr;
op->mode = AES_MODE_ECB;
- op->len = nbytes - (nbytes % AES_MIN_BLOCK_SIZE);
+ op->len = nbytes - (nbytes % AES_BLOCK_SIZE);
op->dir = AES_DIR_ENCRYPT;
ret = geode_aes_crypt(op);
@@ -482,7 +482,7 @@
CRYPTO_ALG_NEED_FALLBACK,
.cra_init = fallback_init_blk,
.cra_exit = fallback_exit_blk,
- .cra_blocksize = AES_MIN_BLOCK_SIZE,
+ .cra_blocksize = AES_BLOCK_SIZE,
.cra_ctxsize = sizeof(struct geode_aes_op),
.cra_alignmask = 15,
.cra_type = &crypto_blkcipher_type,
@@ -547,7 +547,7 @@
if (ret)
goto eecb;
- printk(KERN_NOTICE "geode-aes: GEODE AES engine enabled.\n");
+ dev_notice(&dev->dev, "GEODE AES engine enabled.\n");
return 0;
eecb:
@@ -565,7 +565,7 @@
eenable:
pci_disable_device(dev);
- printk(KERN_ERR "geode-aes: GEODE AES initialization failed.\n");
+ dev_err(&dev->dev, "GEODE AES initialization failed.\n");
return ret;
}
diff --git a/drivers/crypto/geode-aes.h b/drivers/crypto/geode-aes.h
index f1855b5..f442ca9 100644
--- a/drivers/crypto/geode-aes.h
+++ b/drivers/crypto/geode-aes.h
@@ -10,10 +10,6 @@
#define _GEODE_AES_H_
/* driver logic flags */
-#define AES_IV_LENGTH 16
-#define AES_KEY_LENGTH 16
-#define AES_MIN_BLOCK_SIZE 16
-
#define AES_MODE_ECB 0
#define AES_MODE_CBC 1
@@ -64,7 +60,7 @@
u32 flags;
int len;
- u8 key[AES_KEY_LENGTH];
+ u8 key[AES_KEYSIZE_128];
u8 *iv;
union {
diff --git a/drivers/crypto/mv_cesa.c b/drivers/crypto/mv_cesa.c
index 8d1e6f8..29d0ee5 100644
--- a/drivers/crypto/mv_cesa.c
+++ b/drivers/crypto/mv_cesa.c
@@ -622,8 +622,8 @@
}
if (async_req) {
- if (async_req->tfm->__crt_alg->cra_type !=
- &crypto_ahash_type) {
+ if (crypto_tfm_alg_type(async_req->tfm) !=
+ CRYPTO_ALG_TYPE_AHASH) {
struct ablkcipher_request *req =
ablkcipher_request_cast(async_req);
mv_start_new_crypt_req(req);
@@ -843,7 +843,7 @@
static int mv_cra_hash_init(struct crypto_tfm *tfm, const char *base_hash_name,
enum hash_op op, int count_add)
{
- const char *fallback_driver_name = tfm->__crt_alg->cra_name;
+ const char *fallback_driver_name = crypto_tfm_alg_name(tfm);
struct mv_tfm_hash_ctx *ctx = crypto_tfm_ctx(tfm);
struct crypto_shash *fallback_tfm = NULL;
struct crypto_shash *base_hash = NULL;
diff --git a/drivers/crypto/mxs-dcp.c b/drivers/crypto/mxs-dcp.c
index 7bbe0ab..b5f7e6d 100644
--- a/drivers/crypto/mxs-dcp.c
+++ b/drivers/crypto/mxs-dcp.c
@@ -104,7 +104,6 @@
* design of Linux Crypto API.
*/
static struct dcp *global_sdcp;
-static DEFINE_MUTEX(global_mutex);
/* DCP register layout. */
#define MXS_DCP_CTRL 0x00
@@ -482,7 +481,7 @@
static int mxs_dcp_aes_fallback_init(struct crypto_tfm *tfm)
{
- const char *name = tfm->__crt_alg->cra_name;
+ const char *name = crypto_tfm_alg_name(tfm);
const uint32_t flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK;
struct dcp_async_ctx *actx = crypto_tfm_ctx(tfm);
struct crypto_ablkcipher *blk;
@@ -907,60 +906,49 @@
struct resource *iores;
int dcp_vmi_irq, dcp_irq;
- mutex_lock(&global_mutex);
if (global_sdcp) {
dev_err(dev, "Only one DCP instance allowed!\n");
- ret = -ENODEV;
- goto err_mutex;
+ return -ENODEV;
}
iores = platform_get_resource(pdev, IORESOURCE_MEM, 0);
dcp_vmi_irq = platform_get_irq(pdev, 0);
- if (dcp_vmi_irq < 0) {
- ret = dcp_vmi_irq;
- goto err_mutex;
- }
+ if (dcp_vmi_irq < 0)
+ return dcp_vmi_irq;
dcp_irq = platform_get_irq(pdev, 1);
- if (dcp_irq < 0) {
- ret = dcp_irq;
- goto err_mutex;
- }
+ if (dcp_irq < 0)
+ return dcp_irq;
sdcp = devm_kzalloc(dev, sizeof(*sdcp), GFP_KERNEL);
- if (!sdcp) {
- ret = -ENOMEM;
- goto err_mutex;
- }
+ if (!sdcp)
+ return -ENOMEM;
sdcp->dev = dev;
sdcp->base = devm_ioremap_resource(dev, iores);
- if (IS_ERR(sdcp->base)) {
- ret = PTR_ERR(sdcp->base);
- goto err_mutex;
- }
+ if (IS_ERR(sdcp->base))
+ return PTR_ERR(sdcp->base);
+
ret = devm_request_irq(dev, dcp_vmi_irq, mxs_dcp_irq, 0,
"dcp-vmi-irq", sdcp);
if (ret) {
dev_err(dev, "Failed to claim DCP VMI IRQ!\n");
- goto err_mutex;
+ return ret;
}
ret = devm_request_irq(dev, dcp_irq, mxs_dcp_irq, 0,
"dcp-irq", sdcp);
if (ret) {
dev_err(dev, "Failed to claim DCP IRQ!\n");
- goto err_mutex;
+ return ret;
}
/* Allocate coherent helper block. */
sdcp->coh = devm_kzalloc(dev, sizeof(*sdcp->coh) + DCP_ALIGNMENT,
GFP_KERNEL);
- if (!sdcp->coh) {
- ret = -ENOMEM;
- goto err_mutex;
- }
+ if (!sdcp->coh)
+ return -ENOMEM;
/* Re-align the structure so it fits the DCP constraints. */
sdcp->coh = PTR_ALIGN(sdcp->coh, DCP_ALIGNMENT);
@@ -968,7 +956,7 @@
/* Restart the DCP block. */
ret = stmp_reset_block(sdcp->base);
if (ret)
- goto err_mutex;
+ return ret;
/* Initialize control register. */
writel(MXS_DCP_CTRL_GATHER_RESIDUAL_WRITES |
@@ -1006,8 +994,7 @@
NULL, "mxs_dcp_chan/sha");
if (IS_ERR(sdcp->thread[DCP_CHAN_HASH_SHA])) {
dev_err(dev, "Error starting SHA thread!\n");
- ret = PTR_ERR(sdcp->thread[DCP_CHAN_HASH_SHA]);
- goto err_mutex;
+ return PTR_ERR(sdcp->thread[DCP_CHAN_HASH_SHA]);
}
sdcp->thread[DCP_CHAN_CRYPTO] = kthread_run(dcp_chan_thread_aes,
@@ -1064,9 +1051,6 @@
err_destroy_sha_thread:
kthread_stop(sdcp->thread[DCP_CHAN_HASH_SHA]);
-
-err_mutex:
- mutex_unlock(&global_mutex);
return ret;
}
@@ -1088,9 +1072,7 @@
platform_set_drvdata(pdev, NULL);
- mutex_lock(&global_mutex);
global_sdcp = NULL;
- mutex_unlock(&global_mutex);
return 0;
}
diff --git a/drivers/crypto/n2_core.c b/drivers/crypto/n2_core.c
index e1f0ab4..7263c10 100644
--- a/drivers/crypto/n2_core.c
+++ b/drivers/crypto/n2_core.c
@@ -356,7 +356,7 @@
static int n2_hash_cra_init(struct crypto_tfm *tfm)
{
- const char *fallback_driver_name = tfm->__crt_alg->cra_name;
+ const char *fallback_driver_name = crypto_tfm_alg_name(tfm);
struct crypto_ahash *ahash = __crypto_ahash_cast(tfm);
struct n2_hash_ctx *ctx = crypto_ahash_ctx(ahash);
struct crypto_ahash *fallback_tfm;
@@ -391,7 +391,7 @@
static int n2_hmac_cra_init(struct crypto_tfm *tfm)
{
- const char *fallback_driver_name = tfm->__crt_alg->cra_name;
+ const char *fallback_driver_name = crypto_tfm_alg_name(tfm);
struct crypto_ahash *ahash = __crypto_ahash_cast(tfm);
struct n2_hmac_ctx *ctx = crypto_ahash_ctx(ahash);
struct n2_hmac_alg *n2alg = n2_hmac_alg(tfm);
diff --git a/drivers/crypto/nx/nx-842.c b/drivers/crypto/nx/nx-842.c
index 5ce8b57..502edf0 100644
--- a/drivers/crypto/nx/nx-842.c
+++ b/drivers/crypto/nx/nx-842.c
@@ -1229,7 +1229,7 @@
old_devdata = rcu_dereference_check(devdata,
lockdep_is_held(&devdata_mutex));
of_reconfig_notifier_unregister(&nx842_of_nb);
- rcu_assign_pointer(devdata, NULL);
+ RCU_INIT_POINTER(devdata, NULL);
spin_unlock_irqrestore(&devdata_mutex, flags);
synchronize_rcu();
dev_set_drvdata(&viodev->dev, NULL);
@@ -1280,7 +1280,7 @@
spin_lock_irqsave(&devdata_mutex, flags);
old_devdata = rcu_dereference_check(devdata,
lockdep_is_held(&devdata_mutex));
- rcu_assign_pointer(devdata, NULL);
+ RCU_INIT_POINTER(devdata, NULL);
spin_unlock_irqrestore(&devdata_mutex, flags);
synchronize_rcu();
if (old_devdata)
diff --git a/drivers/crypto/omap-des.c b/drivers/crypto/omap-des.c
index ec5f131..b8bc84b 100644
--- a/drivers/crypto/omap-des.c
+++ b/drivers/crypto/omap-des.c
@@ -223,12 +223,19 @@
static int omap_des_hw_init(struct omap_des_dev *dd)
{
+ int err;
+
/*
* clocks are enabled when request starts and disabled when finished.
* It may be long delays between requests.
* Device might go to off mode to save power.
*/
- pm_runtime_get_sync(dd->dev);
+ err = pm_runtime_get_sync(dd->dev);
+ if (err < 0) {
+ pm_runtime_put_noidle(dd->dev);
+ dev_err(dd->dev, "%s: failed to get_sync(%d)\n", __func__, err);
+ return err;
+ }
if (!(dd->flags & FLAGS_INIT)) {
dd->flags |= FLAGS_INIT;
@@ -1074,16 +1081,20 @@
if (err)
goto err_res;
- dd->io_base = devm_request_and_ioremap(dev, res);
- if (!dd->io_base) {
- dev_err(dev, "can't ioremap\n");
- err = -ENOMEM;
+ dd->io_base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(dd->io_base)) {
+ err = PTR_ERR(dd->io_base);
goto err_res;
}
dd->phys_base = res->start;
pm_runtime_enable(dev);
- pm_runtime_get_sync(dev);
+ err = pm_runtime_get_sync(dev);
+ if (err < 0) {
+ pm_runtime_put_noidle(dev);
+ dev_err(dd->dev, "%s: failed to get_sync(%d)\n", __func__, err);
+ goto err_get;
+ }
omap_des_dma_stop(dd);
@@ -1148,6 +1159,7 @@
err_irq:
tasklet_kill(&dd->done_task);
tasklet_kill(&dd->queue_task);
+err_get:
pm_runtime_disable(dev);
err_res:
dd = NULL;
@@ -1191,7 +1203,14 @@
static int omap_des_resume(struct device *dev)
{
- pm_runtime_get_sync(dev);
+ int err;
+
+ err = pm_runtime_get_sync(dev);
+ if (err < 0) {
+ pm_runtime_put_noidle(dev);
+ dev_err(dev, "%s: failed to get_sync(%d)\n", __func__, err);
+ return err;
+ }
return 0;
}
#endif
diff --git a/drivers/crypto/padlock-sha.c b/drivers/crypto/padlock-sha.c
index 9266c0e..bace885 100644
--- a/drivers/crypto/padlock-sha.c
+++ b/drivers/crypto/padlock-sha.c
@@ -211,7 +211,7 @@
static int padlock_cra_init(struct crypto_tfm *tfm)
{
struct crypto_shash *hash = __crypto_shash_cast(tfm);
- const char *fallback_driver_name = tfm->__crt_alg->cra_name;
+ const char *fallback_driver_name = crypto_tfm_alg_name(tfm);
struct padlock_sha_ctx *ctx = crypto_tfm_ctx(tfm);
struct crypto_shash *fallback_tfm;
int err = -ENOMEM;
diff --git a/drivers/crypto/s5p-sss.c b/drivers/crypto/s5p-sss.c
index be45762..4197ad9 100644
--- a/drivers/crypto/s5p-sss.c
+++ b/drivers/crypto/s5p-sss.c
@@ -22,6 +22,7 @@
#include <linux/scatterlist.h>
#include <linux/dma-mapping.h>
#include <linux/io.h>
+#include <linux/of.h>
#include <linux/crypto.h>
#include <linux/interrupt.h>
@@ -29,9 +30,6 @@
#include <crypto/aes.h>
#include <crypto/ctr.h>
-#include <plat/cpu.h>
-#include <mach/dma.h>
-
#define _SBF(s, v) ((v) << (s))
#define _BIT(b) _SBF(b, 1)
@@ -105,7 +103,7 @@
#define SSS_REG_FCPKDMAO 0x005C
/* AES registers */
-#define SSS_REG_AES_CONTROL 0x4000
+#define SSS_REG_AES_CONTROL 0x00
#define SSS_AES_BYTESWAP_DI _BIT(11)
#define SSS_AES_BYTESWAP_DO _BIT(10)
#define SSS_AES_BYTESWAP_IV _BIT(9)
@@ -121,21 +119,25 @@
#define SSS_AES_CHAIN_MODE_CTR _SBF(1, 0x02)
#define SSS_AES_MODE_DECRYPT _BIT(0)
-#define SSS_REG_AES_STATUS 0x4004
+#define SSS_REG_AES_STATUS 0x04
#define SSS_AES_BUSY _BIT(2)
#define SSS_AES_INPUT_READY _BIT(1)
#define SSS_AES_OUTPUT_READY _BIT(0)
-#define SSS_REG_AES_IN_DATA(s) (0x4010 + (s << 2))
-#define SSS_REG_AES_OUT_DATA(s) (0x4020 + (s << 2))
-#define SSS_REG_AES_IV_DATA(s) (0x4030 + (s << 2))
-#define SSS_REG_AES_CNT_DATA(s) (0x4040 + (s << 2))
-#define SSS_REG_AES_KEY_DATA(s) (0x4080 + (s << 2))
+#define SSS_REG_AES_IN_DATA(s) (0x10 + (s << 2))
+#define SSS_REG_AES_OUT_DATA(s) (0x20 + (s << 2))
+#define SSS_REG_AES_IV_DATA(s) (0x30 + (s << 2))
+#define SSS_REG_AES_CNT_DATA(s) (0x40 + (s << 2))
+#define SSS_REG_AES_KEY_DATA(s) (0x80 + (s << 2))
#define SSS_REG(dev, reg) ((dev)->ioaddr + (SSS_REG_##reg))
#define SSS_READ(dev, reg) __raw_readl(SSS_REG(dev, reg))
#define SSS_WRITE(dev, reg, val) __raw_writel((val), SSS_REG(dev, reg))
+#define SSS_AES_REG(dev, reg) ((dev)->aes_ioaddr + SSS_REG_##reg)
+#define SSS_AES_WRITE(dev, reg, val) __raw_writel((val), \
+ SSS_AES_REG(dev, reg))
+
/* HW engine modes */
#define FLAGS_AES_DECRYPT _BIT(0)
#define FLAGS_AES_MODE_MASK _SBF(1, 0x03)
@@ -145,6 +147,20 @@
#define AES_KEY_LEN 16
#define CRYPTO_QUEUE_LEN 1
+/**
+ * struct samsung_aes_variant - platform specific SSS driver data
+ * @has_hash_irq: true if SSS module uses hash interrupt, false otherwise
+ * @aes_offset: AES register offset from SSS module's base.
+ *
+ * Specifies platform specific configuration of SSS module.
+ * Note: A structure for driver specific platform data is used for future
+ * expansion of its usage.
+ */
+struct samsung_aes_variant {
+ bool has_hash_irq;
+ unsigned int aes_offset;
+};
+
struct s5p_aes_reqctx {
unsigned long mode;
};
@@ -161,6 +177,7 @@
struct device *dev;
struct clk *clk;
void __iomem *ioaddr;
+ void __iomem *aes_ioaddr;
int irq_hash;
int irq_fc;
@@ -173,10 +190,48 @@
struct crypto_queue queue;
bool busy;
spinlock_t lock;
+
+ struct samsung_aes_variant *variant;
};
static struct s5p_aes_dev *s5p_dev;
+static const struct samsung_aes_variant s5p_aes_data = {
+ .has_hash_irq = true,
+ .aes_offset = 0x4000,
+};
+
+static const struct samsung_aes_variant exynos_aes_data = {
+ .has_hash_irq = false,
+ .aes_offset = 0x200,
+};
+
+static const struct of_device_id s5p_sss_dt_match[] = {
+ {
+ .compatible = "samsung,s5pv210-secss",
+ .data = &s5p_aes_data,
+ },
+ {
+ .compatible = "samsung,exynos4210-secss",
+ .data = &exynos_aes_data,
+ },
+ { },
+};
+MODULE_DEVICE_TABLE(of, s5p_sss_dt_match);
+
+static inline struct samsung_aes_variant *find_s5p_sss_version
+ (struct platform_device *pdev)
+{
+ if (IS_ENABLED(CONFIG_OF) && (pdev->dev.of_node)) {
+ const struct of_device_id *match;
+ match = of_match_node(s5p_sss_dt_match,
+ pdev->dev.of_node);
+ return (struct samsung_aes_variant *)match->data;
+ }
+ return (struct samsung_aes_variant *)
+ platform_get_device_id(pdev)->driver_data;
+}
+
static void s5p_set_dma_indata(struct s5p_aes_dev *dev, struct scatterlist *sg)
{
SSS_WRITE(dev, FCBRDMAS, sg_dma_address(sg));
@@ -272,8 +327,12 @@
}
s5p_set_dma_outdata(dev, dev->sg_dst);
- } else
+ } else {
s5p_aes_complete(dev, err);
+
+ dev->busy = true;
+ tasklet_schedule(&dev->tasklet);
+ }
}
static void s5p_aes_rx(struct s5p_aes_dev *dev)
@@ -322,14 +381,15 @@
{
void __iomem *keystart;
- memcpy(dev->ioaddr + SSS_REG_AES_IV_DATA(0), iv, 0x10);
+ if (iv)
+ memcpy(dev->aes_ioaddr + SSS_REG_AES_IV_DATA(0), iv, 0x10);
if (keylen == AES_KEYSIZE_256)
- keystart = dev->ioaddr + SSS_REG_AES_KEY_DATA(0);
+ keystart = dev->aes_ioaddr + SSS_REG_AES_KEY_DATA(0);
else if (keylen == AES_KEYSIZE_192)
- keystart = dev->ioaddr + SSS_REG_AES_KEY_DATA(2);
+ keystart = dev->aes_ioaddr + SSS_REG_AES_KEY_DATA(2);
else
- keystart = dev->ioaddr + SSS_REG_AES_KEY_DATA(4);
+ keystart = dev->aes_ioaddr + SSS_REG_AES_KEY_DATA(4);
memcpy(keystart, key, keylen);
}
@@ -379,7 +439,7 @@
if (err)
goto outdata_error;
- SSS_WRITE(dev, AES_CONTROL, aes_control);
+ SSS_AES_WRITE(dev, AES_CONTROL, aes_control);
s5p_set_aes(dev, dev->ctx->aes_key, req->info, dev->ctx->keylen);
s5p_set_dma_indata(dev, req->src);
@@ -410,10 +470,13 @@
spin_lock_irqsave(&dev->lock, flags);
backlog = crypto_get_backlog(&dev->queue);
async_req = crypto_dequeue_request(&dev->queue);
- spin_unlock_irqrestore(&dev->lock, flags);
- if (!async_req)
+ if (!async_req) {
+ dev->busy = false;
+ spin_unlock_irqrestore(&dev->lock, flags);
return;
+ }
+ spin_unlock_irqrestore(&dev->lock, flags);
if (backlog)
backlog->complete(backlog, -EINPROGRESS);
@@ -432,14 +495,13 @@
int err;
spin_lock_irqsave(&dev->lock, flags);
+ err = ablkcipher_enqueue_request(&dev->queue, req);
if (dev->busy) {
- err = -EAGAIN;
spin_unlock_irqrestore(&dev->lock, flags);
goto exit;
}
dev->busy = true;
- err = ablkcipher_enqueue_request(&dev->queue, req);
spin_unlock_irqrestore(&dev->lock, flags);
tasklet_schedule(&dev->tasklet);
@@ -564,6 +626,7 @@
struct s5p_aes_dev *pdata;
struct device *dev = &pdev->dev;
struct resource *res;
+ struct samsung_aes_variant *variant;
if (s5p_dev)
return -EEXIST;
@@ -577,30 +640,25 @@
if (IS_ERR(pdata->ioaddr))
return PTR_ERR(pdata->ioaddr);
+ variant = find_s5p_sss_version(pdev);
+
pdata->clk = devm_clk_get(dev, "secss");
if (IS_ERR(pdata->clk)) {
dev_err(dev, "failed to find secss clock source\n");
return -ENOENT;
}
- clk_enable(pdata->clk);
+ err = clk_prepare_enable(pdata->clk);
+ if (err < 0) {
+ dev_err(dev, "Enabling SSS clk failed, err %d\n", err);
+ return err;
+ }
spin_lock_init(&pdata->lock);
- pdata->irq_hash = platform_get_irq_byname(pdev, "hash");
- if (pdata->irq_hash < 0) {
- err = pdata->irq_hash;
- dev_warn(dev, "hash interrupt is not available.\n");
- goto err_irq;
- }
- err = devm_request_irq(dev, pdata->irq_hash, s5p_aes_interrupt,
- IRQF_SHARED, pdev->name, pdev);
- if (err < 0) {
- dev_warn(dev, "hash interrupt is not available.\n");
- goto err_irq;
- }
+ pdata->aes_ioaddr = pdata->ioaddr + variant->aes_offset;
- pdata->irq_fc = platform_get_irq_byname(pdev, "feed control");
+ pdata->irq_fc = platform_get_irq(pdev, 0);
if (pdata->irq_fc < 0) {
err = pdata->irq_fc;
dev_warn(dev, "feed control interrupt is not available.\n");
@@ -613,6 +671,23 @@
goto err_irq;
}
+ if (variant->has_hash_irq) {
+ pdata->irq_hash = platform_get_irq(pdev, 1);
+ if (pdata->irq_hash < 0) {
+ err = pdata->irq_hash;
+ dev_warn(dev, "hash interrupt is not available.\n");
+ goto err_irq;
+ }
+ err = devm_request_irq(dev, pdata->irq_hash, s5p_aes_interrupt,
+ IRQF_SHARED, pdev->name, pdev);
+ if (err < 0) {
+ dev_warn(dev, "hash interrupt is not available.\n");
+ goto err_irq;
+ }
+ }
+
+ pdata->busy = false;
+ pdata->variant = variant;
pdata->dev = dev;
platform_set_drvdata(pdev, pdata);
s5p_dev = pdata;
@@ -639,7 +714,7 @@
tasklet_kill(&pdata->tasklet);
err_irq:
- clk_disable(pdata->clk);
+ clk_disable_unprepare(pdata->clk);
s5p_dev = NULL;
@@ -659,7 +734,7 @@
tasklet_kill(&pdata->tasklet);
- clk_disable(pdata->clk);
+ clk_disable_unprepare(pdata->clk);
s5p_dev = NULL;
@@ -672,6 +747,7 @@
.driver = {
.owner = THIS_MODULE,
.name = "s5p-secss",
+ .of_match_table = s5p_sss_dt_match,
},
};
diff --git a/drivers/crypto/sahara.c b/drivers/crypto/sahara.c
index 07a5987..164e1ec 100644
--- a/drivers/crypto/sahara.c
+++ b/drivers/crypto/sahara.c
@@ -728,7 +728,7 @@
static int sahara_aes_cra_init(struct crypto_tfm *tfm)
{
- const char *name = tfm->__crt_alg->cra_name;
+ const char *name = crypto_tfm_alg_name(tfm);
struct sahara_ctx *ctx = crypto_tfm_ctx(tfm);
ctx->fallback = crypto_alloc_ablkcipher(name, 0,
diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
index 5c58638..1eca7b9 100644
--- a/drivers/dma/Kconfig
+++ b/drivers/dma/Kconfig
@@ -234,7 +234,7 @@
config PCH_DMA
tristate "Intel EG20T PCH / LAPIS Semicon IOH(ML7213/ML7223/ML7831) DMA"
- depends on PCI && X86
+ depends on PCI && (X86_32 || COMPILE_TEST)
select DMA_ENGINE
help
Enable support for Intel EG20T PCH DMA engine.
@@ -269,7 +269,7 @@
select DMA_ENGINE
help
Support the MXS DMA engine. This engine including APBH-DMA
- and APBX-DMA is integrated into Freescale i.MX23/28 chips.
+ and APBX-DMA is integrated into Freescale i.MX23/28/MX6Q/MX6DL chips.
config EP93XX_DMA
bool "Cirrus Logic EP93xx DMA support"
@@ -361,6 +361,20 @@
multiplexing capability for DMA request sources(slot).
This module can be found on Freescale Vybrid and LS-1 SoCs.
+config XILINX_VDMA
+ tristate "Xilinx AXI VDMA Engine"
+ depends on (ARCH_ZYNQ || MICROBLAZE)
+ select DMA_ENGINE
+ help
+ Enable support for Xilinx AXI VDMA Soft IP.
+
+ This engine provides high-bandwidth direct memory access
+ between memory and AXI4-Stream video type target
+ peripherals including peripherals which support AXI4-
+ Stream Video Protocol. It has two stream interfaces/
+ channels, Memory Mapped to Stream (MM2S) and Stream to
+ Memory Mapped (S2MM) for the data transfers.
+
config DMA_ENGINE
bool
diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile
index 5150c82..c779e1e 100644
--- a/drivers/dma/Makefile
+++ b/drivers/dma/Makefile
@@ -46,3 +46,4 @@
obj-$(CONFIG_MOXART_DMA) += moxart-dma.o
obj-$(CONFIG_FSL_EDMA) += fsl-edma.o
obj-$(CONFIG_QCOM_BAM_DMA) += qcom_bam_dma.o
+obj-y += xilinx/
diff --git a/drivers/dma/dw/core.c b/drivers/dma/dw/core.c
index 7a74076..a27ded5 100644
--- a/drivers/dma/dw/core.c
+++ b/drivers/dma/dw/core.c
@@ -1493,6 +1493,13 @@
dw->regs = chip->regs;
chip->dw = dw;
+ dw->clk = devm_clk_get(chip->dev, "hclk");
+ if (IS_ERR(dw->clk))
+ return PTR_ERR(dw->clk);
+ err = clk_prepare_enable(dw->clk);
+ if (err)
+ return err;
+
dw_params = dma_read_byaddr(chip->regs, DW_PARAMS);
autocfg = dw_params >> DW_PARAMS_EN & 0x1;
@@ -1500,15 +1507,19 @@
if (!pdata && autocfg) {
pdata = devm_kzalloc(chip->dev, sizeof(*pdata), GFP_KERNEL);
- if (!pdata)
- return -ENOMEM;
+ if (!pdata) {
+ err = -ENOMEM;
+ goto err_pdata;
+ }
/* Fill platform data with the default values */
pdata->is_private = true;
pdata->chan_allocation_order = CHAN_ALLOCATION_ASCENDING;
pdata->chan_priority = CHAN_PRIORITY_ASCENDING;
- } else if (!pdata || pdata->nr_channels > DW_DMA_MAX_NR_CHANNELS)
- return -EINVAL;
+ } else if (!pdata || pdata->nr_channels > DW_DMA_MAX_NR_CHANNELS) {
+ err = -EINVAL;
+ goto err_pdata;
+ }
if (autocfg)
nr_channels = (dw_params >> DW_PARAMS_NR_CHAN & 0x7) + 1;
@@ -1517,13 +1528,10 @@
dw->chan = devm_kcalloc(chip->dev, nr_channels, sizeof(*dw->chan),
GFP_KERNEL);
- if (!dw->chan)
- return -ENOMEM;
-
- dw->clk = devm_clk_get(chip->dev, "hclk");
- if (IS_ERR(dw->clk))
- return PTR_ERR(dw->clk);
- clk_prepare_enable(dw->clk);
+ if (!dw->chan) {
+ err = -ENOMEM;
+ goto err_pdata;
+ }
/* Get hardware configuration parameters */
if (autocfg) {
@@ -1553,7 +1561,8 @@
sizeof(struct dw_desc), 4, 0);
if (!dw->desc_pool) {
dev_err(chip->dev, "No memory for descriptors dma pool\n");
- return -ENOMEM;
+ err = -ENOMEM;
+ goto err_pdata;
}
tasklet_init(&dw->tasklet, dw_dma_tasklet, (unsigned long)dw);
@@ -1561,7 +1570,7 @@
err = request_irq(chip->irq, dw_dma_interrupt, IRQF_SHARED,
"dw_dmac", dw);
if (err)
- return err;
+ goto err_pdata;
INIT_LIST_HEAD(&dw->dma.channels);
for (i = 0; i < nr_channels; i++) {
@@ -1650,12 +1659,20 @@
dma_writel(dw, CFG, DW_CFG_DMA_EN);
+ err = dma_async_device_register(&dw->dma);
+ if (err)
+ goto err_dma_register;
+
dev_info(chip->dev, "DesignWare DMA Controller, %d channels\n",
nr_channels);
- dma_async_device_register(&dw->dma);
-
return 0;
+
+err_dma_register:
+ free_irq(chip->irq, dw);
+err_pdata:
+ clk_disable_unprepare(dw->clk);
+ return err;
}
EXPORT_SYMBOL_GPL(dw_dma_probe);
@@ -1676,6 +1693,8 @@
channel_clear_bit(dw, CH_EN, dwc->mask);
}
+ clk_disable_unprepare(dw->clk);
+
return 0;
}
EXPORT_SYMBOL_GPL(dw_dma_remove);
diff --git a/drivers/dma/dw/pci.c b/drivers/dma/dw/pci.c
index fec59f1..39e30c3 100644
--- a/drivers/dma/dw/pci.c
+++ b/drivers/dma/dw/pci.c
@@ -93,19 +93,13 @@
return dw_dma_resume(chip);
};
-#else /* !CONFIG_PM_SLEEP */
-
-#define dw_pci_suspend_late NULL
-#define dw_pci_resume_early NULL
-
-#endif /* !CONFIG_PM_SLEEP */
+#endif /* CONFIG_PM_SLEEP */
static const struct dev_pm_ops dw_pci_dev_pm_ops = {
- .suspend_late = dw_pci_suspend_late,
- .resume_early = dw_pci_resume_early,
+ SET_LATE_SYSTEM_SLEEP_PM_OPS(dw_pci_suspend_late, dw_pci_resume_early)
};
-static DEFINE_PCI_DEVICE_TABLE(dw_pci_id_table) = {
+static const struct pci_device_id dw_pci_id_table[] = {
/* Medfield */
{ PCI_VDEVICE(INTEL, 0x0827), (kernel_ulong_t)&dw_pci_pdata },
{ PCI_VDEVICE(INTEL, 0x0830), (kernel_ulong_t)&dw_pci_pdata },
diff --git a/drivers/dma/dw/platform.c b/drivers/dma/dw/platform.c
index 453822c..c5b339a 100644
--- a/drivers/dma/dw/platform.c
+++ b/drivers/dma/dw/platform.c
@@ -256,7 +256,7 @@
#ifdef CONFIG_PM_SLEEP
-static int dw_suspend_noirq(struct device *dev)
+static int dw_suspend_late(struct device *dev)
{
struct platform_device *pdev = to_platform_device(dev);
struct dw_dma_chip *chip = platform_get_drvdata(pdev);
@@ -264,7 +264,7 @@
return dw_dma_suspend(chip);
}
-static int dw_resume_noirq(struct device *dev)
+static int dw_resume_early(struct device *dev)
{
struct platform_device *pdev = to_platform_device(dev);
struct dw_dma_chip *chip = platform_get_drvdata(pdev);
@@ -272,20 +272,10 @@
return dw_dma_resume(chip);
}
-#else /* !CONFIG_PM_SLEEP */
-
-#define dw_suspend_noirq NULL
-#define dw_resume_noirq NULL
-
-#endif /* !CONFIG_PM_SLEEP */
+#endif /* CONFIG_PM_SLEEP */
static const struct dev_pm_ops dw_dev_pm_ops = {
- .suspend_noirq = dw_suspend_noirq,
- .resume_noirq = dw_resume_noirq,
- .freeze_noirq = dw_suspend_noirq,
- .thaw_noirq = dw_resume_noirq,
- .restore_noirq = dw_resume_noirq,
- .poweroff_noirq = dw_suspend_noirq,
+ SET_LATE_SYSTEM_SLEEP_PM_OPS(dw_suspend_late, dw_resume_early)
};
static struct platform_driver dw_driver = {
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index f157c6f..e0fec68 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -61,6 +61,16 @@
return DMA_IN(chan, &chan->regs->sr, 32);
}
+static void set_mr(struct fsldma_chan *chan, u32 val)
+{
+ DMA_OUT(chan, &chan->regs->mr, val, 32);
+}
+
+static u32 get_mr(struct fsldma_chan *chan)
+{
+ return DMA_IN(chan, &chan->regs->mr, 32);
+}
+
static void set_cdar(struct fsldma_chan *chan, dma_addr_t addr)
{
DMA_OUT(chan, &chan->regs->cdar, addr | FSL_DMA_SNEN, 64);
@@ -71,6 +81,11 @@
return DMA_IN(chan, &chan->regs->cdar, 64) & ~FSL_DMA_SNEN;
}
+static void set_bcr(struct fsldma_chan *chan, u32 val)
+{
+ DMA_OUT(chan, &chan->regs->bcr, val, 32);
+}
+
static u32 get_bcr(struct fsldma_chan *chan)
{
return DMA_IN(chan, &chan->regs->bcr, 32);
@@ -135,7 +150,7 @@
static void dma_init(struct fsldma_chan *chan)
{
/* Reset the channel */
- DMA_OUT(chan, &chan->regs->mr, 0, 32);
+ set_mr(chan, 0);
switch (chan->feature & FSL_DMA_IP_MASK) {
case FSL_DMA_IP_85XX:
@@ -144,16 +159,15 @@
* EOLNIE - End of links interrupt enable
* BWC - Bandwidth sharing among channels
*/
- DMA_OUT(chan, &chan->regs->mr, FSL_DMA_MR_BWC
- | FSL_DMA_MR_EIE | FSL_DMA_MR_EOLNIE, 32);
+ set_mr(chan, FSL_DMA_MR_BWC | FSL_DMA_MR_EIE
+ | FSL_DMA_MR_EOLNIE);
break;
case FSL_DMA_IP_83XX:
/* Set the channel to below modes:
* EOTIE - End-of-transfer interrupt enable
* PRC_RM - PCI read multiple
*/
- DMA_OUT(chan, &chan->regs->mr, FSL_DMA_MR_EOTIE
- | FSL_DMA_MR_PRC_RM, 32);
+ set_mr(chan, FSL_DMA_MR_EOTIE | FSL_DMA_MR_PRC_RM);
break;
}
}
@@ -175,10 +189,10 @@
{
u32 mode;
- mode = DMA_IN(chan, &chan->regs->mr, 32);
+ mode = get_mr(chan);
if (chan->feature & FSL_DMA_CHAN_PAUSE_EXT) {
- DMA_OUT(chan, &chan->regs->bcr, 0, 32);
+ set_bcr(chan, 0);
mode |= FSL_DMA_MR_EMP_EN;
} else {
mode &= ~FSL_DMA_MR_EMP_EN;
@@ -191,7 +205,7 @@
mode |= FSL_DMA_MR_CS;
}
- DMA_OUT(chan, &chan->regs->mr, mode, 32);
+ set_mr(chan, mode);
}
static void dma_halt(struct fsldma_chan *chan)
@@ -200,7 +214,7 @@
int i;
/* read the mode register */
- mode = DMA_IN(chan, &chan->regs->mr, 32);
+ mode = get_mr(chan);
/*
* The 85xx controller supports channel abort, which will stop
@@ -209,14 +223,14 @@
*/
if ((chan->feature & FSL_DMA_IP_MASK) == FSL_DMA_IP_85XX) {
mode |= FSL_DMA_MR_CA;
- DMA_OUT(chan, &chan->regs->mr, mode, 32);
+ set_mr(chan, mode);
mode &= ~FSL_DMA_MR_CA;
}
/* stop the DMA controller */
mode &= ~(FSL_DMA_MR_CS | FSL_DMA_MR_EMS_EN);
- DMA_OUT(chan, &chan->regs->mr, mode, 32);
+ set_mr(chan, mode);
/* wait for the DMA controller to become idle */
for (i = 0; i < 100; i++) {
@@ -245,7 +259,7 @@
{
u32 mode;
- mode = DMA_IN(chan, &chan->regs->mr, 32);
+ mode = get_mr(chan);
switch (size) {
case 0:
@@ -259,7 +273,7 @@
break;
}
- DMA_OUT(chan, &chan->regs->mr, mode, 32);
+ set_mr(chan, mode);
}
/**
@@ -277,7 +291,7 @@
{
u32 mode;
- mode = DMA_IN(chan, &chan->regs->mr, 32);
+ mode = get_mr(chan);
switch (size) {
case 0:
@@ -291,7 +305,7 @@
break;
}
- DMA_OUT(chan, &chan->regs->mr, mode, 32);
+ set_mr(chan, mode);
}
/**
@@ -312,10 +326,10 @@
BUG_ON(size > 1024);
- mode = DMA_IN(chan, &chan->regs->mr, 32);
+ mode = get_mr(chan);
mode |= (__ilog2(size) << 24) & 0x0f000000;
- DMA_OUT(chan, &chan->regs->mr, mode, 32);
+ set_mr(chan, mode);
}
/**
@@ -404,6 +418,19 @@
}
/**
+ * fsl_dma_free_descriptor - Free descriptor from channel's DMA pool.
+ * @chan : Freescale DMA channel
+ * @desc: descriptor to be freed
+ */
+static void fsl_dma_free_descriptor(struct fsldma_chan *chan,
+ struct fsl_desc_sw *desc)
+{
+ list_del(&desc->node);
+ chan_dbg(chan, "LD %p free\n", desc);
+ dma_pool_free(chan->desc_pool, desc, desc->async_tx.phys);
+}
+
+/**
* fsl_dma_alloc_descriptor - Allocate descriptor from channel's DMA pool.
* @chan : Freescale DMA channel
*
@@ -426,14 +453,107 @@
desc->async_tx.tx_submit = fsl_dma_tx_submit;
desc->async_tx.phys = pdesc;
-#ifdef FSL_DMA_LD_DEBUG
chan_dbg(chan, "LD %p allocated\n", desc);
-#endif
return desc;
}
/**
+ * fsl_chan_xfer_ld_queue - transfer any pending transactions
+ * @chan : Freescale DMA channel
+ *
+ * HARDWARE STATE: idle
+ * LOCKING: must hold chan->desc_lock
+ */
+static void fsl_chan_xfer_ld_queue(struct fsldma_chan *chan)
+{
+ struct fsl_desc_sw *desc;
+
+ /*
+ * If the list of pending descriptors is empty, then we
+ * don't need to do any work at all
+ */
+ if (list_empty(&chan->ld_pending)) {
+ chan_dbg(chan, "no pending LDs\n");
+ return;
+ }
+
+ /*
+ * The DMA controller is not idle, which means that the interrupt
+ * handler will start any queued transactions when it runs after
+ * this transaction finishes
+ */
+ if (!chan->idle) {
+ chan_dbg(chan, "DMA controller still busy\n");
+ return;
+ }
+
+ /*
+ * If there are some link descriptors which have not been
+ * transferred, we need to start the controller
+ */
+
+ /*
+ * Move all elements from the queue of pending transactions
+ * onto the list of running transactions
+ */
+ chan_dbg(chan, "idle, starting controller\n");
+ desc = list_first_entry(&chan->ld_pending, struct fsl_desc_sw, node);
+ list_splice_tail_init(&chan->ld_pending, &chan->ld_running);
+
+ /*
+ * The 85xx DMA controller doesn't clear the channel start bit
+ * automatically at the end of a transfer. Therefore we must clear
+ * it in software before starting the transfer.
+ */
+ if ((chan->feature & FSL_DMA_IP_MASK) == FSL_DMA_IP_85XX) {
+ u32 mode;
+
+ mode = get_mr(chan);
+ mode &= ~FSL_DMA_MR_CS;
+ set_mr(chan, mode);
+ }
+
+ /*
+ * Program the descriptor's address into the DMA controller,
+ * then start the DMA transaction
+ */
+ set_cdar(chan, desc->async_tx.phys);
+ get_cdar(chan);
+
+ dma_start(chan);
+ chan->idle = false;
+}
+
+/**
+ * fsldma_cleanup_descriptor - cleanup and free a single link descriptor
+ * @chan: Freescale DMA channel
+ * @desc: descriptor to cleanup and free
+ *
+ * This function is used on a descriptor which has been executed by the DMA
+ * controller. It will run any callbacks, submit any dependencies, and then
+ * free the descriptor.
+ */
+static void fsldma_cleanup_descriptor(struct fsldma_chan *chan,
+ struct fsl_desc_sw *desc)
+{
+ struct dma_async_tx_descriptor *txd = &desc->async_tx;
+
+ /* Run the link descriptor callback function */
+ if (txd->callback) {
+ chan_dbg(chan, "LD %p callback\n", desc);
+ txd->callback(txd->callback_param);
+ }
+
+ /* Run any dependencies */
+ dma_run_dependencies(txd);
+
+ dma_descriptor_unmap(txd);
+ chan_dbg(chan, "LD %p free\n", desc);
+ dma_pool_free(chan->desc_pool, desc, txd->phys);
+}
+
+/**
* fsl_dma_alloc_chan_resources - Allocate resources for DMA channel.
* @chan : Freescale DMA channel
*
@@ -477,13 +597,8 @@
{
struct fsl_desc_sw *desc, *_desc;
- list_for_each_entry_safe(desc, _desc, list, node) {
- list_del(&desc->node);
-#ifdef FSL_DMA_LD_DEBUG
- chan_dbg(chan, "LD %p free\n", desc);
-#endif
- dma_pool_free(chan->desc_pool, desc, desc->async_tx.phys);
- }
+ list_for_each_entry_safe(desc, _desc, list, node)
+ fsl_dma_free_descriptor(chan, desc);
}
static void fsldma_free_desc_list_reverse(struct fsldma_chan *chan,
@@ -491,13 +606,8 @@
{
struct fsl_desc_sw *desc, *_desc;
- list_for_each_entry_safe_reverse(desc, _desc, list, node) {
- list_del(&desc->node);
-#ifdef FSL_DMA_LD_DEBUG
- chan_dbg(chan, "LD %p free\n", desc);
-#endif
- dma_pool_free(chan->desc_pool, desc, desc->async_tx.phys);
- }
+ list_for_each_entry_safe_reverse(desc, _desc, list, node)
+ fsl_dma_free_descriptor(chan, desc);
}
/**
@@ -520,35 +630,6 @@
}
static struct dma_async_tx_descriptor *
-fsl_dma_prep_interrupt(struct dma_chan *dchan, unsigned long flags)
-{
- struct fsldma_chan *chan;
- struct fsl_desc_sw *new;
-
- if (!dchan)
- return NULL;
-
- chan = to_fsl_chan(dchan);
-
- new = fsl_dma_alloc_descriptor(chan);
- if (!new) {
- chan_err(chan, "%s\n", msg_ld_oom);
- return NULL;
- }
-
- new->async_tx.cookie = -EBUSY;
- new->async_tx.flags = flags;
-
- /* Insert the link descriptor to the LD ring */
- list_add_tail(&new->node, &new->tx_list);
-
- /* Set End-of-link to the last link descriptor of new list */
- set_ld_eol(chan, new);
-
- return &new->async_tx;
-}
-
-static struct dma_async_tx_descriptor *
fsl_dma_prep_memcpy(struct dma_chan *dchan,
dma_addr_t dma_dst, dma_addr_t dma_src,
size_t len, unsigned long flags)
@@ -817,105 +898,6 @@
}
/**
- * fsldma_cleanup_descriptor - cleanup and free a single link descriptor
- * @chan: Freescale DMA channel
- * @desc: descriptor to cleanup and free
- *
- * This function is used on a descriptor which has been executed by the DMA
- * controller. It will run any callbacks, submit any dependencies, and then
- * free the descriptor.
- */
-static void fsldma_cleanup_descriptor(struct fsldma_chan *chan,
- struct fsl_desc_sw *desc)
-{
- struct dma_async_tx_descriptor *txd = &desc->async_tx;
-
- /* Run the link descriptor callback function */
- if (txd->callback) {
-#ifdef FSL_DMA_LD_DEBUG
- chan_dbg(chan, "LD %p callback\n", desc);
-#endif
- txd->callback(txd->callback_param);
- }
-
- /* Run any dependencies */
- dma_run_dependencies(txd);
-
- dma_descriptor_unmap(txd);
-#ifdef FSL_DMA_LD_DEBUG
- chan_dbg(chan, "LD %p free\n", desc);
-#endif
- dma_pool_free(chan->desc_pool, desc, txd->phys);
-}
-
-/**
- * fsl_chan_xfer_ld_queue - transfer any pending transactions
- * @chan : Freescale DMA channel
- *
- * HARDWARE STATE: idle
- * LOCKING: must hold chan->desc_lock
- */
-static void fsl_chan_xfer_ld_queue(struct fsldma_chan *chan)
-{
- struct fsl_desc_sw *desc;
-
- /*
- * If the list of pending descriptors is empty, then we
- * don't need to do any work at all
- */
- if (list_empty(&chan->ld_pending)) {
- chan_dbg(chan, "no pending LDs\n");
- return;
- }
-
- /*
- * The DMA controller is not idle, which means that the interrupt
- * handler will start any queued transactions when it runs after
- * this transaction finishes
- */
- if (!chan->idle) {
- chan_dbg(chan, "DMA controller still busy\n");
- return;
- }
-
- /*
- * If there are some link descriptors which have not been
- * transferred, we need to start the controller
- */
-
- /*
- * Move all elements from the queue of pending transactions
- * onto the list of running transactions
- */
- chan_dbg(chan, "idle, starting controller\n");
- desc = list_first_entry(&chan->ld_pending, struct fsl_desc_sw, node);
- list_splice_tail_init(&chan->ld_pending, &chan->ld_running);
-
- /*
- * The 85xx DMA controller doesn't clear the channel start bit
- * automatically at the end of a transfer. Therefore we must clear
- * it in software before starting the transfer.
- */
- if ((chan->feature & FSL_DMA_IP_MASK) == FSL_DMA_IP_85XX) {
- u32 mode;
-
- mode = DMA_IN(chan, &chan->regs->mr, 32);
- mode &= ~FSL_DMA_MR_CS;
- DMA_OUT(chan, &chan->regs->mr, mode, 32);
- }
-
- /*
- * Program the descriptor's address into the DMA controller,
- * then start the DMA transaction
- */
- set_cdar(chan, desc->async_tx.phys);
- get_cdar(chan);
-
- dma_start(chan);
- chan->idle = false;
-}
-
-/**
* fsl_dma_memcpy_issue_pending - Issue the DMA start command
* @chan : Freescale DMA channel
*/
@@ -1304,12 +1286,10 @@
fdev->irq = irq_of_parse_and_map(op->dev.of_node, 0);
dma_cap_set(DMA_MEMCPY, fdev->common.cap_mask);
- dma_cap_set(DMA_INTERRUPT, fdev->common.cap_mask);
dma_cap_set(DMA_SG, fdev->common.cap_mask);
dma_cap_set(DMA_SLAVE, fdev->common.cap_mask);
fdev->common.device_alloc_chan_resources = fsl_dma_alloc_chan_resources;
fdev->common.device_free_chan_resources = fsl_dma_free_chan_resources;
- fdev->common.device_prep_dma_interrupt = fsl_dma_prep_interrupt;
fdev->common.device_prep_dma_memcpy = fsl_dma_prep_memcpy;
fdev->common.device_prep_dma_sg = fsl_dma_prep_sg;
fdev->common.device_tx_status = fsl_tx_status;
diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c
index 19041ce..1287146 100644
--- a/drivers/dma/imx-sdma.c
+++ b/drivers/dma/imx-sdma.c
@@ -607,8 +607,6 @@
if (bd->mode.status & BD_RROR)
sdmac->status = DMA_ERROR;
- else
- sdmac->status = DMA_IN_PROGRESS;
bd->mode.status |= BD_DONE;
sdmac->buf_tail++;
diff --git a/drivers/dma/mmp_pdma.c b/drivers/dma/mmp_pdma.c
index bf02e7b..a7b186d 100644
--- a/drivers/dma/mmp_pdma.c
+++ b/drivers/dma/mmp_pdma.c
@@ -29,8 +29,8 @@
#define DALGN 0x00a0
#define DINT 0x00f0
#define DDADR 0x0200
-#define DSADR 0x0204
-#define DTADR 0x0208
+#define DSADR(n) (0x0204 + ((n) << 4))
+#define DTADR(n) (0x0208 + ((n) << 4))
#define DCMD 0x020c
#define DCSR_RUN BIT(31) /* Run Bit (read / write) */
@@ -277,7 +277,7 @@
return;
/* clear the channel mapping in DRCMR */
- reg = DRCMR(pchan->phy->vchan->drcmr);
+ reg = DRCMR(pchan->drcmr);
writel(0, pchan->phy->base + reg);
spin_lock_irqsave(&pdev->phy_lock, flags);
@@ -748,11 +748,92 @@
return 0;
}
+static unsigned int mmp_pdma_residue(struct mmp_pdma_chan *chan,
+ dma_cookie_t cookie)
+{
+ struct mmp_pdma_desc_sw *sw;
+ u32 curr, residue = 0;
+ bool passed = false;
+ bool cyclic = chan->cyclic_first != NULL;
+
+ /*
+ * If the channel does not have a phy pointer anymore, it has already
+ * been completed. Therefore, its residue is 0.
+ */
+ if (!chan->phy)
+ return 0;
+
+ if (chan->dir == DMA_DEV_TO_MEM)
+ curr = readl(chan->phy->base + DTADR(chan->phy->idx));
+ else
+ curr = readl(chan->phy->base + DSADR(chan->phy->idx));
+
+ list_for_each_entry(sw, &chan->chain_running, node) {
+ u32 start, end, len;
+
+ if (chan->dir == DMA_DEV_TO_MEM)
+ start = sw->desc.dtadr;
+ else
+ start = sw->desc.dsadr;
+
+ len = sw->desc.dcmd & DCMD_LENGTH;
+ end = start + len;
+
+ /*
+ * 'passed' will be latched once we found the descriptor which
+ * lies inside the boundaries of the curr pointer. All
+ * descriptors that occur in the list _after_ we found that
+ * partially handled descriptor are still to be processed and
+ * are hence added to the residual bytes counter.
+ */
+
+ if (passed) {
+ residue += len;
+ } else if (curr >= start && curr <= end) {
+ residue += end - curr;
+ passed = true;
+ }
+
+ /*
+ * Descriptors that have the ENDIRQEN bit set mark the end of a
+ * transaction chain, and the cookie assigned with it has been
+ * returned previously from mmp_pdma_tx_submit().
+ *
+ * In case we have multiple transactions in the running chain,
+ * and the cookie does not match the one the user asked us
+ * about, reset the state variables and start over.
+ *
+ * This logic does not apply to cyclic transactions, where all
+ * descriptors have the ENDIRQEN bit set, and for which we
+ * can't have multiple transactions on one channel anyway.
+ */
+ if (cyclic || !(sw->desc.dcmd & DCMD_ENDIRQEN))
+ continue;
+
+ if (sw->async_tx.cookie == cookie) {
+ return residue;
+ } else {
+ residue = 0;
+ passed = false;
+ }
+ }
+
+ /* We should only get here in case of cyclic transactions */
+ return residue;
+}
+
static enum dma_status mmp_pdma_tx_status(struct dma_chan *dchan,
dma_cookie_t cookie,
struct dma_tx_state *txstate)
{
- return dma_cookie_status(dchan, cookie, txstate);
+ struct mmp_pdma_chan *chan = to_mmp_pdma_chan(dchan);
+ enum dma_status ret;
+
+ ret = dma_cookie_status(dchan, cookie, txstate);
+ if (likely(ret != DMA_ERROR))
+ dma_set_residue(txstate, mmp_pdma_residue(chan, cookie));
+
+ return ret;
}
/**
@@ -858,8 +939,7 @@
struct mmp_pdma_chan *chan;
int ret;
- chan = devm_kzalloc(pdev->dev, sizeof(struct mmp_pdma_chan),
- GFP_KERNEL);
+ chan = devm_kzalloc(pdev->dev, sizeof(*chan), GFP_KERNEL);
if (chan == NULL)
return -ENOMEM;
@@ -946,8 +1026,7 @@
irq_num++;
}
- pdev->phy = devm_kcalloc(pdev->dev,
- dma_channels, sizeof(struct mmp_pdma_chan),
+ pdev->phy = devm_kcalloc(pdev->dev, dma_channels, sizeof(*pdev->phy),
GFP_KERNEL);
if (pdev->phy == NULL)
return -ENOMEM;
diff --git a/drivers/dma/mpc512x_dma.c b/drivers/dma/mpc512x_dma.c
index 448750d..2ad4373 100644
--- a/drivers/dma/mpc512x_dma.c
+++ b/drivers/dma/mpc512x_dma.c
@@ -2,6 +2,7 @@
* Copyright (C) Freescale Semicondutor, Inc. 2007, 2008.
* Copyright (C) Semihalf 2009
* Copyright (C) Ilya Yanok, Emcraft Systems 2010
+ * Copyright (C) Alexander Popov, Promcontroller 2014
*
* Written by Piotr Ziecik <kosmo@semihalf.com>. Hardware description
* (defines, structures and comments) was taken from MPC5121 DMA driver
@@ -29,8 +30,18 @@
*/
/*
- * This is initial version of MPC5121 DMA driver. Only memory to memory
- * transfers are supported (tested using dmatest module).
+ * MPC512x and MPC8308 DMA driver. It supports
+ * memory to memory data transfers (tested using dmatest module) and
+ * data transfers between memory and peripheral I/O memory
+ * by means of slave scatter/gather with these limitations:
+ * - chunked transfers (described by s/g lists with more than one item)
+ * are refused as long as proper support for scatter/gather is missing;
+ * - transfers on MPC8308 always start from software as this SoC appears
+ * not to have external request lines for peripheral flow control;
+ * - only peripheral devices with 4-byte FIFO access register are supported;
+ * - minimal memory <-> I/O memory transfer chunk is 4 bytes and consequently
+ * source and destination addresses must be 4-byte aligned
+ * and transfer size must be aligned on (4 * maxburst) boundary;
*/
#include <linux/module.h>
@@ -52,9 +63,17 @@
#define MPC_DMA_DESCRIPTORS 64
/* Macro definitions */
-#define MPC_DMA_CHANNELS 64
#define MPC_DMA_TCD_OFFSET 0x1000
+/*
+ * Maximum channel counts for individual hardware variants
+ * and the maximum channel count over all supported controllers,
+ * used for data structure size
+ */
+#define MPC8308_DMACHAN_MAX 16
+#define MPC512x_DMACHAN_MAX 64
+#define MPC_DMA_CHANNELS 64
+
/* Arbitration mode of group and channel */
#define MPC_DMA_DMACR_EDCG (1 << 31)
#define MPC_DMA_DMACR_ERGA (1 << 3)
@@ -181,6 +200,7 @@
dma_addr_t tcd_paddr;
int error;
struct list_head node;
+ int will_access_peripheral;
};
struct mpc_dma_chan {
@@ -193,6 +213,12 @@
struct mpc_dma_tcd *tcd;
dma_addr_t tcd_paddr;
+ /* Settings for access to peripheral FIFO */
+ dma_addr_t src_per_paddr;
+ u32 src_tcd_nunits;
+ dma_addr_t dst_per_paddr;
+ u32 dst_tcd_nunits;
+
/* Lock for this structure */
spinlock_t lock;
};
@@ -243,8 +269,23 @@
struct mpc_dma_desc *mdesc;
int cid = mchan->chan.chan_id;
- /* Move all queued descriptors to active list */
- list_splice_tail_init(&mchan->queued, &mchan->active);
+ while (!list_empty(&mchan->queued)) {
+ mdesc = list_first_entry(&mchan->queued,
+ struct mpc_dma_desc, node);
+ /*
+ * Grab either several mem-to-mem transfer descriptors
+ * or one peripheral transfer descriptor,
+ * don't mix mem-to-mem and peripheral transfer descriptors
+ * within the same 'active' list.
+ */
+ if (mdesc->will_access_peripheral) {
+ if (list_empty(&mchan->active))
+ list_move_tail(&mdesc->node, &mchan->active);
+ break;
+ } else {
+ list_move_tail(&mdesc->node, &mchan->active);
+ }
+ }
/* Chain descriptors into one transaction */
list_for_each_entry(mdesc, &mchan->active, node) {
@@ -270,7 +311,17 @@
if (first != prev)
mdma->tcd[cid].e_sg = 1;
- out_8(&mdma->regs->dmassrt, cid);
+
+ if (mdma->is_mpc8308) {
+ /* MPC8308, no request lines, software initiated start */
+ out_8(&mdma->regs->dmassrt, cid);
+ } else if (first->will_access_peripheral) {
+ /* Peripherals involved, start by external request signal */
+ out_8(&mdma->regs->dmaserq, cid);
+ } else {
+ /* Memory to memory transfer, software initiated start */
+ out_8(&mdma->regs->dmassrt, cid);
+ }
}
/* Handle interrupt on one half of DMA controller (32 channels) */
@@ -588,6 +639,7 @@
}
mdesc->error = 0;
+ mdesc->will_access_peripheral = 0;
tcd = mdesc->tcd;
/* Prepare Transfer Control Descriptor for this transaction */
@@ -635,6 +687,193 @@
return &mdesc->desc;
}
+static struct dma_async_tx_descriptor *
+mpc_dma_prep_slave_sg(struct dma_chan *chan, struct scatterlist *sgl,
+ unsigned int sg_len, enum dma_transfer_direction direction,
+ unsigned long flags, void *context)
+{
+ struct mpc_dma *mdma = dma_chan_to_mpc_dma(chan);
+ struct mpc_dma_chan *mchan = dma_chan_to_mpc_dma_chan(chan);
+ struct mpc_dma_desc *mdesc = NULL;
+ dma_addr_t per_paddr;
+ u32 tcd_nunits;
+ struct mpc_dma_tcd *tcd;
+ unsigned long iflags;
+ struct scatterlist *sg;
+ size_t len;
+ int iter, i;
+
+ /* Currently there is no proper support for scatter/gather */
+ if (sg_len != 1)
+ return NULL;
+
+ if (!is_slave_direction(direction))
+ return NULL;
+
+ for_each_sg(sgl, sg, sg_len, i) {
+ spin_lock_irqsave(&mchan->lock, iflags);
+
+ mdesc = list_first_entry(&mchan->free,
+ struct mpc_dma_desc, node);
+ if (!mdesc) {
+ spin_unlock_irqrestore(&mchan->lock, iflags);
+ /* Try to free completed descriptors */
+ mpc_dma_process_completed(mdma);
+ return NULL;
+ }
+
+ list_del(&mdesc->node);
+
+ if (direction == DMA_DEV_TO_MEM) {
+ per_paddr = mchan->src_per_paddr;
+ tcd_nunits = mchan->src_tcd_nunits;
+ } else {
+ per_paddr = mchan->dst_per_paddr;
+ tcd_nunits = mchan->dst_tcd_nunits;
+ }
+
+ spin_unlock_irqrestore(&mchan->lock, iflags);
+
+ if (per_paddr == 0 || tcd_nunits == 0)
+ goto err_prep;
+
+ mdesc->error = 0;
+ mdesc->will_access_peripheral = 1;
+
+ /* Prepare Transfer Control Descriptor for this transaction */
+ tcd = mdesc->tcd;
+
+ memset(tcd, 0, sizeof(struct mpc_dma_tcd));
+
+ if (!IS_ALIGNED(sg_dma_address(sg), 4))
+ goto err_prep;
+
+ if (direction == DMA_DEV_TO_MEM) {
+ tcd->saddr = per_paddr;
+ tcd->daddr = sg_dma_address(sg);
+ tcd->soff = 0;
+ tcd->doff = 4;
+ } else {
+ tcd->saddr = sg_dma_address(sg);
+ tcd->daddr = per_paddr;
+ tcd->soff = 4;
+ tcd->doff = 0;
+ }
+
+ tcd->ssize = MPC_DMA_TSIZE_4;
+ tcd->dsize = MPC_DMA_TSIZE_4;
+
+ len = sg_dma_len(sg);
+ tcd->nbytes = tcd_nunits * 4;
+ if (!IS_ALIGNED(len, tcd->nbytes))
+ goto err_prep;
+
+ iter = len / tcd->nbytes;
+ if (iter >= 1 << 15) {
+ /* len is too big */
+ goto err_prep;
+ }
+ /* citer_linkch contains the high bits of iter */
+ tcd->biter = iter & 0x1ff;
+ tcd->biter_linkch = iter >> 9;
+ tcd->citer = tcd->biter;
+ tcd->citer_linkch = tcd->biter_linkch;
+
+ tcd->e_sg = 0;
+ tcd->d_req = 1;
+
+ /* Place descriptor in prepared list */
+ spin_lock_irqsave(&mchan->lock, iflags);
+ list_add_tail(&mdesc->node, &mchan->prepared);
+ spin_unlock_irqrestore(&mchan->lock, iflags);
+ }
+
+ return &mdesc->desc;
+
+err_prep:
+ /* Put the descriptor back */
+ spin_lock_irqsave(&mchan->lock, iflags);
+ list_add_tail(&mdesc->node, &mchan->free);
+ spin_unlock_irqrestore(&mchan->lock, iflags);
+
+ return NULL;
+}
+
+static int mpc_dma_device_control(struct dma_chan *chan, enum dma_ctrl_cmd cmd,
+ unsigned long arg)
+{
+ struct mpc_dma_chan *mchan;
+ struct mpc_dma *mdma;
+ struct dma_slave_config *cfg;
+ unsigned long flags;
+
+ mchan = dma_chan_to_mpc_dma_chan(chan);
+ switch (cmd) {
+ case DMA_TERMINATE_ALL:
+ /* Disable channel requests */
+ mdma = dma_chan_to_mpc_dma(chan);
+
+ spin_lock_irqsave(&mchan->lock, flags);
+
+ out_8(&mdma->regs->dmacerq, chan->chan_id);
+ list_splice_tail_init(&mchan->prepared, &mchan->free);
+ list_splice_tail_init(&mchan->queued, &mchan->free);
+ list_splice_tail_init(&mchan->active, &mchan->free);
+
+ spin_unlock_irqrestore(&mchan->lock, flags);
+
+ return 0;
+
+ case DMA_SLAVE_CONFIG:
+ /*
+ * Software constraints:
+ * - only transfers between a peripheral device and
+ * memory are supported;
+ * - only peripheral devices with 4-byte FIFO access register
+ * are supported;
+ * - minimal transfer chunk is 4 bytes and consequently
+ * source and destination addresses must be 4-byte aligned
+ * and transfer size must be aligned on (4 * maxburst)
+ * boundary;
+ * - during the transfer RAM address is being incremented by
+ * the size of minimal transfer chunk;
+ * - peripheral port's address is constant during the transfer.
+ */
+
+ cfg = (void *)arg;
+
+ if (cfg->src_addr_width != DMA_SLAVE_BUSWIDTH_4_BYTES ||
+ cfg->dst_addr_width != DMA_SLAVE_BUSWIDTH_4_BYTES ||
+ !IS_ALIGNED(cfg->src_addr, 4) ||
+ !IS_ALIGNED(cfg->dst_addr, 4)) {
+ return -EINVAL;
+ }
+
+ spin_lock_irqsave(&mchan->lock, flags);
+
+ mchan->src_per_paddr = cfg->src_addr;
+ mchan->src_tcd_nunits = cfg->src_maxburst;
+ mchan->dst_per_paddr = cfg->dst_addr;
+ mchan->dst_tcd_nunits = cfg->dst_maxburst;
+
+ /* Apply defaults */
+ if (mchan->src_tcd_nunits == 0)
+ mchan->src_tcd_nunits = 1;
+ if (mchan->dst_tcd_nunits == 0)
+ mchan->dst_tcd_nunits = 1;
+
+ spin_unlock_irqrestore(&mchan->lock, flags);
+
+ return 0;
+
+ default:
+ /* Unknown command */
+ break;
+ }
+
+ return -ENXIO;
+}
+
static int mpc_dma_probe(struct platform_device *op)
{
struct device_node *dn = op->dev.of_node;
@@ -649,13 +888,15 @@
mdma = devm_kzalloc(dev, sizeof(struct mpc_dma), GFP_KERNEL);
if (!mdma) {
dev_err(dev, "Memory exhausted!\n");
- return -ENOMEM;
+ retval = -ENOMEM;
+ goto err;
}
mdma->irq = irq_of_parse_and_map(dn, 0);
if (mdma->irq == NO_IRQ) {
dev_err(dev, "Error mapping IRQ!\n");
- return -EINVAL;
+ retval = -EINVAL;
+ goto err;
}
if (of_device_is_compatible(dn, "fsl,mpc8308-dma")) {
@@ -663,14 +904,15 @@
mdma->irq2 = irq_of_parse_and_map(dn, 1);
if (mdma->irq2 == NO_IRQ) {
dev_err(dev, "Error mapping IRQ!\n");
- return -EINVAL;
+ retval = -EINVAL;
+ goto err_dispose1;
}
}
retval = of_address_to_resource(dn, 0, &res);
if (retval) {
dev_err(dev, "Error parsing memory region!\n");
- return retval;
+ goto err_dispose2;
}
regs_start = res.start;
@@ -678,31 +920,34 @@
if (!devm_request_mem_region(dev, regs_start, regs_size, DRV_NAME)) {
dev_err(dev, "Error requesting memory region!\n");
- return -EBUSY;
+ retval = -EBUSY;
+ goto err_dispose2;
}
mdma->regs = devm_ioremap(dev, regs_start, regs_size);
if (!mdma->regs) {
dev_err(dev, "Error mapping memory region!\n");
- return -ENOMEM;
+ retval = -ENOMEM;
+ goto err_dispose2;
}
mdma->tcd = (struct mpc_dma_tcd *)((u8 *)(mdma->regs)
+ MPC_DMA_TCD_OFFSET);
- retval = devm_request_irq(dev, mdma->irq, &mpc_dma_irq, 0, DRV_NAME,
- mdma);
+ retval = request_irq(mdma->irq, &mpc_dma_irq, 0, DRV_NAME, mdma);
if (retval) {
dev_err(dev, "Error requesting IRQ!\n");
- return -EINVAL;
+ retval = -EINVAL;
+ goto err_dispose2;
}
if (mdma->is_mpc8308) {
- retval = devm_request_irq(dev, mdma->irq2, &mpc_dma_irq, 0,
- DRV_NAME, mdma);
+ retval = request_irq(mdma->irq2, &mpc_dma_irq, 0,
+ DRV_NAME, mdma);
if (retval) {
dev_err(dev, "Error requesting IRQ2!\n");
- return -EINVAL;
+ retval = -EINVAL;
+ goto err_free1;
}
}
@@ -710,18 +955,21 @@
dma = &mdma->dma;
dma->dev = dev;
- if (!mdma->is_mpc8308)
- dma->chancnt = MPC_DMA_CHANNELS;
+ if (mdma->is_mpc8308)
+ dma->chancnt = MPC8308_DMACHAN_MAX;
else
- dma->chancnt = 16; /* MPC8308 DMA has only 16 channels */
+ dma->chancnt = MPC512x_DMACHAN_MAX;
dma->device_alloc_chan_resources = mpc_dma_alloc_chan_resources;
dma->device_free_chan_resources = mpc_dma_free_chan_resources;
dma->device_issue_pending = mpc_dma_issue_pending;
dma->device_tx_status = mpc_dma_tx_status;
dma->device_prep_dma_memcpy = mpc_dma_prep_memcpy;
+ dma->device_prep_slave_sg = mpc_dma_prep_slave_sg;
+ dma->device_control = mpc_dma_device_control;
INIT_LIST_HEAD(&dma->channels);
dma_cap_set(DMA_MEMCPY, dma->cap_mask);
+ dma_cap_set(DMA_SLAVE, dma->cap_mask);
for (i = 0; i < dma->chancnt; i++) {
mchan = &mdma->channels[i];
@@ -747,7 +995,19 @@
* - Round-robin group arbitration,
* - Round-robin channel arbitration.
*/
- if (!mdma->is_mpc8308) {
+ if (mdma->is_mpc8308) {
+ /* MPC8308 has 16 channels and lacks some registers */
+ out_be32(&mdma->regs->dmacr, MPC_DMA_DMACR_ERCA);
+
+ /* enable snooping */
+ out_be32(&mdma->regs->dmagpor, MPC_DMA_DMAGPOR_SNOOP_ENABLE);
+ /* Disable error interrupts */
+ out_be32(&mdma->regs->dmaeeil, 0);
+
+ /* Clear interrupts status */
+ out_be32(&mdma->regs->dmaintl, 0xFFFF);
+ out_be32(&mdma->regs->dmaerrl, 0xFFFF);
+ } else {
out_be32(&mdma->regs->dmacr, MPC_DMA_DMACR_EDCG |
MPC_DMA_DMACR_ERGA | MPC_DMA_DMACR_ERCA);
@@ -768,29 +1028,28 @@
/* Route interrupts to IPIC */
out_be32(&mdma->regs->dmaihsa, 0);
out_be32(&mdma->regs->dmailsa, 0);
- } else {
- /* MPC8308 has 16 channels and lacks some registers */
- out_be32(&mdma->regs->dmacr, MPC_DMA_DMACR_ERCA);
-
- /* enable snooping */
- out_be32(&mdma->regs->dmagpor, MPC_DMA_DMAGPOR_SNOOP_ENABLE);
- /* Disable error interrupts */
- out_be32(&mdma->regs->dmaeeil, 0);
-
- /* Clear interrupts status */
- out_be32(&mdma->regs->dmaintl, 0xFFFF);
- out_be32(&mdma->regs->dmaerrl, 0xFFFF);
}
/* Register DMA engine */
dev_set_drvdata(dev, mdma);
retval = dma_async_device_register(dma);
- if (retval) {
- devm_free_irq(dev, mdma->irq, mdma);
- irq_dispose_mapping(mdma->irq);
- }
+ if (retval)
+ goto err_free2;
return retval;
+
+err_free2:
+ if (mdma->is_mpc8308)
+ free_irq(mdma->irq2, mdma);
+err_free1:
+ free_irq(mdma->irq, mdma);
+err_dispose2:
+ if (mdma->is_mpc8308)
+ irq_dispose_mapping(mdma->irq2);
+err_dispose1:
+ irq_dispose_mapping(mdma->irq);
+err:
+ return retval;
}
static int mpc_dma_remove(struct platform_device *op)
@@ -799,7 +1058,11 @@
struct mpc_dma *mdma = dev_get_drvdata(dev);
dma_async_device_unregister(&mdma->dma);
- devm_free_irq(dev, mdma->irq, mdma);
+ if (mdma->is_mpc8308) {
+ free_irq(mdma->irq2, mdma);
+ irq_dispose_mapping(mdma->irq2);
+ }
+ free_irq(mdma->irq, mdma);
irq_dispose_mapping(mdma->irq);
return 0;
@@ -807,6 +1070,7 @@
static struct of_device_id mpc_dma_match[] = {
{ .compatible = "fsl,mpc5121-dma", },
+ { .compatible = "fsl,mpc8308-dma", },
{},
};
diff --git a/drivers/dma/pch_dma.c b/drivers/dma/pch_dma.c
index 05fa548..9f9ca9f 100644
--- a/drivers/dma/pch_dma.c
+++ b/drivers/dma/pch_dma.c
@@ -21,6 +21,7 @@
#include <linux/dma-mapping.h>
#include <linux/init.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/pch_dma.h>
@@ -996,7 +997,7 @@
#define PCI_DEVICE_ID_ML7831_DMA1_8CH 0x8810
#define PCI_DEVICE_ID_ML7831_DMA2_4CH 0x8815
-DEFINE_PCI_DEVICE_TABLE(pch_dma_id_table) = {
+const struct pci_device_id pch_dma_id_table[] = {
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_EG20T_PCH_DMA_8CH), 8 },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_EG20T_PCH_DMA_4CH), 4 },
{ PCI_VDEVICE(ROHM, PCI_DEVICE_ID_ML7213_DMA1_8CH), 8}, /* UART Video */
diff --git a/drivers/dma/s3c24xx-dma.c b/drivers/dma/s3c24xx-dma.c
index b209a0f..012520c 100644
--- a/drivers/dma/s3c24xx-dma.c
+++ b/drivers/dma/s3c24xx-dma.c
@@ -164,6 +164,7 @@
* @disrcc: value for source control register
* @didstc: value for destination control register
* @dcon: base value for dcon register
+ * @cyclic: indicate cyclic transfer
*/
struct s3c24xx_txd {
struct virt_dma_desc vd;
@@ -173,6 +174,7 @@
u32 disrcc;
u32 didstc;
u32 dcon;
+ bool cyclic;
};
struct s3c24xx_dma_chan;
@@ -669,8 +671,10 @@
/* when more sg's are in this txd, start the next one */
if (!list_is_last(txd->at, &txd->dsg_list)) {
txd->at = txd->at->next;
+ if (txd->cyclic)
+ vchan_cyclic_callback(&txd->vd);
s3c24xx_dma_start_next_sg(s3cchan, txd);
- } else {
+ } else if (!txd->cyclic) {
s3cchan->at = NULL;
vchan_cookie_complete(&txd->vd);
@@ -682,6 +686,12 @@
s3c24xx_dma_start_next_txd(s3cchan);
else
s3c24xx_dma_phy_free(s3cchan);
+ } else {
+ vchan_cyclic_callback(&txd->vd);
+
+ /* Cyclic: reset at beginning */
+ txd->at = txd->dsg_list.next;
+ s3c24xx_dma_start_next_sg(s3cchan, txd);
}
}
spin_unlock(&s3cchan->vc.lock);
@@ -877,6 +887,104 @@
return vchan_tx_prep(&s3cchan->vc, &txd->vd, flags);
}
+static struct dma_async_tx_descriptor *s3c24xx_dma_prep_dma_cyclic(
+ struct dma_chan *chan, dma_addr_t addr, size_t size, size_t period,
+ enum dma_transfer_direction direction, unsigned long flags,
+ void *context)
+{
+ struct s3c24xx_dma_chan *s3cchan = to_s3c24xx_dma_chan(chan);
+ struct s3c24xx_dma_engine *s3cdma = s3cchan->host;
+ const struct s3c24xx_dma_platdata *pdata = s3cdma->pdata;
+ struct s3c24xx_dma_channel *cdata = &pdata->channels[s3cchan->id];
+ struct s3c24xx_txd *txd;
+ struct s3c24xx_sg *dsg;
+ unsigned sg_len;
+ dma_addr_t slave_addr;
+ u32 hwcfg = 0;
+ int i;
+
+ dev_dbg(&s3cdma->pdev->dev,
+ "prepare cyclic transaction of %zu bytes with period %zu from %s\n",
+ size, period, s3cchan->name);
+
+ if (!is_slave_direction(direction)) {
+ dev_err(&s3cdma->pdev->dev,
+ "direction %d unsupported\n", direction);
+ return NULL;
+ }
+
+ txd = s3c24xx_dma_get_txd();
+ if (!txd)
+ return NULL;
+
+ txd->cyclic = 1;
+
+ if (cdata->handshake)
+ txd->dcon |= S3C24XX_DCON_HANDSHAKE;
+
+ switch (cdata->bus) {
+ case S3C24XX_DMA_APB:
+ txd->dcon |= S3C24XX_DCON_SYNC_PCLK;
+ hwcfg |= S3C24XX_DISRCC_LOC_APB;
+ break;
+ case S3C24XX_DMA_AHB:
+ txd->dcon |= S3C24XX_DCON_SYNC_HCLK;
+ hwcfg |= S3C24XX_DISRCC_LOC_AHB;
+ break;
+ }
+
+ /*
+ * Always assume our peripheral desintation is a fixed
+ * address in memory.
+ */
+ hwcfg |= S3C24XX_DISRCC_INC_FIXED;
+
+ /*
+ * Individual dma operations are requested by the slave,
+ * so serve only single atomic operations (S3C24XX_DCON_SERV_SINGLE).
+ */
+ txd->dcon |= S3C24XX_DCON_SERV_SINGLE;
+
+ if (direction == DMA_MEM_TO_DEV) {
+ txd->disrcc = S3C24XX_DISRCC_LOC_AHB |
+ S3C24XX_DISRCC_INC_INCREMENT;
+ txd->didstc = hwcfg;
+ slave_addr = s3cchan->cfg.dst_addr;
+ txd->width = s3cchan->cfg.dst_addr_width;
+ } else {
+ txd->disrcc = hwcfg;
+ txd->didstc = S3C24XX_DIDSTC_LOC_AHB |
+ S3C24XX_DIDSTC_INC_INCREMENT;
+ slave_addr = s3cchan->cfg.src_addr;
+ txd->width = s3cchan->cfg.src_addr_width;
+ }
+
+ sg_len = size / period;
+
+ for (i = 0; i < sg_len; i++) {
+ dsg = kzalloc(sizeof(*dsg), GFP_NOWAIT);
+ if (!dsg) {
+ s3c24xx_dma_free_txd(txd);
+ return NULL;
+ }
+ list_add_tail(&dsg->node, &txd->dsg_list);
+
+ dsg->len = period;
+ /* Check last period length */
+ if (i == sg_len - 1)
+ dsg->len = size - period * i;
+ if (direction == DMA_MEM_TO_DEV) {
+ dsg->src_addr = addr + period * i;
+ dsg->dst_addr = slave_addr;
+ } else { /* DMA_DEV_TO_MEM */
+ dsg->src_addr = slave_addr;
+ dsg->dst_addr = addr + period * i;
+ }
+ }
+
+ return vchan_tx_prep(&s3cchan->vc, &txd->vd, flags);
+}
+
static struct dma_async_tx_descriptor *s3c24xx_dma_prep_slave_sg(
struct dma_chan *chan, struct scatterlist *sgl,
unsigned int sg_len, enum dma_transfer_direction direction,
@@ -961,7 +1069,6 @@
dsg->src_addr = slave_addr;
dsg->dst_addr = sg_dma_address(sg);
}
- break;
}
return vchan_tx_prep(&s3cchan->vc, &txd->vd, flags);
@@ -1198,6 +1305,7 @@
/* Initialize slave engine for SoC internal dedicated peripherals */
dma_cap_set(DMA_SLAVE, s3cdma->slave.cap_mask);
+ dma_cap_set(DMA_CYCLIC, s3cdma->slave.cap_mask);
dma_cap_set(DMA_PRIVATE, s3cdma->slave.cap_mask);
s3cdma->slave.dev = &pdev->dev;
s3cdma->slave.device_alloc_chan_resources =
@@ -1207,6 +1315,7 @@
s3cdma->slave.device_tx_status = s3c24xx_dma_tx_status;
s3cdma->slave.device_issue_pending = s3c24xx_dma_issue_pending;
s3cdma->slave.device_prep_slave_sg = s3c24xx_dma_prep_slave_sg;
+ s3cdma->slave.device_prep_dma_cyclic = s3c24xx_dma_prep_dma_cyclic;
s3cdma->slave.device_control = s3c24xx_dma_control;
/* Register as many memcpy channels as there are physical channels */
diff --git a/drivers/dma/sh/Kconfig b/drivers/dma/sh/Kconfig
index b4c8138..0f719816 100644
--- a/drivers/dma/sh/Kconfig
+++ b/drivers/dma/sh/Kconfig
@@ -4,7 +4,7 @@
config SH_DMAE_BASE
bool "Renesas SuperH DMA Engine support"
- depends on (SUPERH && SH_DMA) || (ARM && ARCH_SHMOBILE)
+ depends on (SUPERH && SH_DMA) || ARCH_SHMOBILE || COMPILE_TEST
depends on !SH_DMA_API
default y
select DMA_ENGINE
diff --git a/drivers/dma/sh/rcar-hpbdma.c b/drivers/dma/sh/rcar-hpbdma.c
index 3083d90..b212d94 100644
--- a/drivers/dma/sh/rcar-hpbdma.c
+++ b/drivers/dma/sh/rcar-hpbdma.c
@@ -18,6 +18,7 @@
#include <linux/dmaengine.h>
#include <linux/delay.h>
+#include <linux/err.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/module.h>
diff --git a/drivers/dma/sh/shdma-base.c b/drivers/dma/sh/shdma-base.c
index 5239677..b35007e 100644
--- a/drivers/dma/sh/shdma-base.c
+++ b/drivers/dma/sh/shdma-base.c
@@ -73,8 +73,7 @@
static dma_cookie_t shdma_tx_submit(struct dma_async_tx_descriptor *tx)
{
struct shdma_desc *chunk, *c, *desc =
- container_of(tx, struct shdma_desc, async_tx),
- *last = desc;
+ container_of(tx, struct shdma_desc, async_tx);
struct shdma_chan *schan = to_shdma_chan(tx->chan);
dma_async_tx_callback callback = tx->callback;
dma_cookie_t cookie;
@@ -98,19 +97,20 @@
&chunk->node == &schan->ld_free))
break;
chunk->mark = DESC_SUBMITTED;
- /* Callback goes to the last chunk */
- chunk->async_tx.callback = NULL;
+ if (chunk->chunks == 1) {
+ chunk->async_tx.callback = callback;
+ chunk->async_tx.callback_param = tx->callback_param;
+ } else {
+ /* Callback goes to the last chunk */
+ chunk->async_tx.callback = NULL;
+ }
chunk->cookie = cookie;
list_move_tail(&chunk->node, &schan->ld_queue);
- last = chunk;
dev_dbg(schan->dev, "submit #%d@%p on %d\n",
- tx->cookie, &last->async_tx, schan->id);
+ tx->cookie, &chunk->async_tx, schan->id);
}
- last->async_tx.callback = callback;
- last->async_tx.callback_param = tx->callback_param;
-
if (power_up) {
int ret;
schan->pm_state = SHDMA_PM_BUSY;
@@ -304,6 +304,7 @@
dma_async_tx_callback callback = NULL;
void *param = NULL;
unsigned long flags;
+ LIST_HEAD(cyclic_list);
spin_lock_irqsave(&schan->chan_lock, flags);
list_for_each_entry_safe(desc, _desc, &schan->ld_queue, node) {
@@ -369,10 +370,16 @@
if (((desc->mark == DESC_COMPLETED ||
desc->mark == DESC_WAITING) &&
async_tx_test_ack(&desc->async_tx)) || all) {
- /* Remove from ld_queue list */
- desc->mark = DESC_IDLE;
- list_move(&desc->node, &schan->ld_free);
+ if (all || !desc->cyclic) {
+ /* Remove from ld_queue list */
+ desc->mark = DESC_IDLE;
+ list_move(&desc->node, &schan->ld_free);
+ } else {
+ /* reuse as cyclic */
+ desc->mark = DESC_SUBMITTED;
+ list_move_tail(&desc->node, &cyclic_list);
+ }
if (list_empty(&schan->ld_queue)) {
dev_dbg(schan->dev, "Bring down channel %d\n", schan->id);
@@ -389,6 +396,8 @@
*/
schan->dma_chan.completed_cookie = schan->dma_chan.cookie;
+ list_splice_tail(&cyclic_list, &schan->ld_queue);
+
spin_unlock_irqrestore(&schan->chan_lock, flags);
if (callback)
@@ -521,7 +530,7 @@
*/
static struct dma_async_tx_descriptor *shdma_prep_sg(struct shdma_chan *schan,
struct scatterlist *sgl, unsigned int sg_len, dma_addr_t *addr,
- enum dma_transfer_direction direction, unsigned long flags)
+ enum dma_transfer_direction direction, unsigned long flags, bool cyclic)
{
struct scatterlist *sg;
struct shdma_desc *first = NULL, *new = NULL /* compiler... */;
@@ -569,7 +578,11 @@
if (!new)
goto err_get_desc;
- new->chunks = chunks--;
+ new->cyclic = cyclic;
+ if (cyclic)
+ new->chunks = 1;
+ else
+ new->chunks = chunks--;
list_add_tail(&new->node, &tx_list);
} while (len);
}
@@ -612,7 +625,8 @@
sg_dma_address(&sg) = dma_src;
sg_dma_len(&sg) = len;
- return shdma_prep_sg(schan, &sg, 1, &dma_dest, DMA_MEM_TO_MEM, flags);
+ return shdma_prep_sg(schan, &sg, 1, &dma_dest, DMA_MEM_TO_MEM,
+ flags, false);
}
static struct dma_async_tx_descriptor *shdma_prep_slave_sg(
@@ -640,7 +654,58 @@
slave_addr = ops->slave_addr(schan);
return shdma_prep_sg(schan, sgl, sg_len, &slave_addr,
- direction, flags);
+ direction, flags, false);
+}
+
+#define SHDMA_MAX_SG_LEN 32
+
+static struct dma_async_tx_descriptor *shdma_prep_dma_cyclic(
+ struct dma_chan *chan, dma_addr_t buf_addr, size_t buf_len,
+ size_t period_len, enum dma_transfer_direction direction,
+ unsigned long flags, void *context)
+{
+ struct shdma_chan *schan = to_shdma_chan(chan);
+ struct shdma_dev *sdev = to_shdma_dev(schan->dma_chan.device);
+ const struct shdma_ops *ops = sdev->ops;
+ unsigned int sg_len = buf_len / period_len;
+ int slave_id = schan->slave_id;
+ dma_addr_t slave_addr;
+ struct scatterlist sgl[SHDMA_MAX_SG_LEN];
+ int i;
+
+ if (!chan)
+ return NULL;
+
+ BUG_ON(!schan->desc_num);
+
+ if (sg_len > SHDMA_MAX_SG_LEN) {
+ dev_err(schan->dev, "sg length %d exceds limit %d",
+ sg_len, SHDMA_MAX_SG_LEN);
+ return NULL;
+ }
+
+ /* Someone calling slave DMA on a generic channel? */
+ if (slave_id < 0 || (buf_len < period_len)) {
+ dev_warn(schan->dev,
+ "%s: bad parameter: buf_len=%zu, period_len=%zu, id=%d\n",
+ __func__, buf_len, period_len, slave_id);
+ return NULL;
+ }
+
+ slave_addr = ops->slave_addr(schan);
+
+ sg_init_table(sgl, sg_len);
+ for (i = 0; i < sg_len; i++) {
+ dma_addr_t src = buf_addr + (period_len * i);
+
+ sg_set_page(&sgl[i], pfn_to_page(PFN_DOWN(src)), period_len,
+ offset_in_page(src));
+ sg_dma_address(&sgl[i]) = src;
+ sg_dma_len(&sgl[i]) = period_len;
+ }
+
+ return shdma_prep_sg(schan, sgl, sg_len, &slave_addr,
+ direction, flags, true);
}
static int shdma_control(struct dma_chan *chan, enum dma_ctrl_cmd cmd,
@@ -915,6 +980,7 @@
/* Compulsory for DMA_SLAVE fields */
dma_dev->device_prep_slave_sg = shdma_prep_slave_sg;
+ dma_dev->device_prep_dma_cyclic = shdma_prep_dma_cyclic;
dma_dev->device_control = shdma_control;
dma_dev->dev = dev;
diff --git a/drivers/dma/sh/shdmac.c b/drivers/dma/sh/shdmac.c
index dda7e75..146d5df 100644
--- a/drivers/dma/sh/shdmac.c
+++ b/drivers/dma/sh/shdmac.c
@@ -18,21 +18,22 @@
*
*/
+#include <linux/delay.h>
+#include <linux/dmaengine.h>
+#include <linux/err.h>
#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/kdebug.h>
#include <linux/module.h>
+#include <linux/notifier.h>
#include <linux/of.h>
#include <linux/of_device.h>
-#include <linux/slab.h>
-#include <linux/interrupt.h>
-#include <linux/dmaengine.h>
-#include <linux/delay.h>
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
-#include <linux/sh_dma.h>
-#include <linux/notifier.h>
-#include <linux/kdebug.h>
-#include <linux/spinlock.h>
#include <linux/rculist.h>
+#include <linux/sh_dma.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
#include "../dmaengine.h"
#include "shdma.h"
diff --git a/drivers/dma/sh/sudmac.c b/drivers/dma/sh/sudmac.c
index 4e7df43..3ce1039 100644
--- a/drivers/dma/sh/sudmac.c
+++ b/drivers/dma/sh/sudmac.c
@@ -14,12 +14,13 @@
* published by the Free Software Foundation.
*/
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/slab.h>
-#include <linux/interrupt.h>
#include <linux/dmaengine.h>
+#include <linux/err.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/sudmac.h>
struct sudmac_chan {
diff --git a/drivers/dma/ste_dma40.c b/drivers/dma/ste_dma40.c
index bf18c78..c798445 100644
--- a/drivers/dma/ste_dma40.c
+++ b/drivers/dma/ste_dma40.c
@@ -556,7 +556,6 @@
* later
* @reg_val_backup_chan: Backup data for standard channel parameter registers.
* @gcc_pwr_off_mask: Mask to maintain the channels that can be turned off.
- * @initialized: true if the dma has been initialized
* @gen_dmac: the struct for generic registers values to represent u8500/8540
* DMA controller
*/
@@ -594,7 +593,6 @@
u32 reg_val_backup_v4[BACKUP_REGS_SZ_MAX];
u32 *reg_val_backup_chan;
u16 gcc_pwr_off_mask;
- bool initialized;
struct d40_gen_dmac gen_dmac;
};
@@ -1056,62 +1054,6 @@
return len;
}
-
-#ifdef CONFIG_PM
-static void dma40_backup(void __iomem *baseaddr, u32 *backup,
- u32 *regaddr, int num, bool save)
-{
- int i;
-
- for (i = 0; i < num; i++) {
- void __iomem *addr = baseaddr + regaddr[i];
-
- if (save)
- backup[i] = readl_relaxed(addr);
- else
- writel_relaxed(backup[i], addr);
- }
-}
-
-static void d40_save_restore_registers(struct d40_base *base, bool save)
-{
- int i;
-
- /* Save/Restore channel specific registers */
- for (i = 0; i < base->num_phy_chans; i++) {
- void __iomem *addr;
- int idx;
-
- if (base->phy_res[i].reserved)
- continue;
-
- addr = base->virtbase + D40_DREG_PCBASE + i * D40_DREG_PCDELTA;
- idx = i * ARRAY_SIZE(d40_backup_regs_chan);
-
- dma40_backup(addr, &base->reg_val_backup_chan[idx],
- d40_backup_regs_chan,
- ARRAY_SIZE(d40_backup_regs_chan),
- save);
- }
-
- /* Save/Restore global registers */
- dma40_backup(base->virtbase, base->reg_val_backup,
- d40_backup_regs, ARRAY_SIZE(d40_backup_regs),
- save);
-
- /* Save/Restore registers only existing on dma40 v3 and later */
- if (base->gen_dmac.backup)
- dma40_backup(base->virtbase, base->reg_val_backup_v4,
- base->gen_dmac.backup,
- base->gen_dmac.backup_size,
- save);
-}
-#else
-static void d40_save_restore_registers(struct d40_base *base, bool save)
-{
-}
-#endif
-
static int __d40_execute_command_phy(struct d40_chan *d40c,
enum d40_command command)
{
@@ -1495,8 +1437,8 @@
if (!d40c->busy)
return 0;
- pm_runtime_get_sync(d40c->base->dev);
spin_lock_irqsave(&d40c->lock, flags);
+ pm_runtime_get_sync(d40c->base->dev);
res = d40_channel_execute_command(d40c, D40_DMA_SUSPEND_REQ);
@@ -2998,16 +2940,86 @@
}
/* Suspend resume functionality */
-#ifdef CONFIG_PM
-static int dma40_pm_suspend(struct device *dev)
+#ifdef CONFIG_PM_SLEEP
+static int dma40_suspend(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct d40_base *base = platform_get_drvdata(pdev);
+ int ret;
+
+ ret = pm_runtime_force_suspend(dev);
+ if (ret)
+ return ret;
+
+ if (base->lcpa_regulator)
+ ret = regulator_disable(base->lcpa_regulator);
+ return ret;
+}
+
+static int dma40_resume(struct device *dev)
{
struct platform_device *pdev = to_platform_device(dev);
struct d40_base *base = platform_get_drvdata(pdev);
int ret = 0;
- if (base->lcpa_regulator)
- ret = regulator_disable(base->lcpa_regulator);
- return ret;
+ if (base->lcpa_regulator) {
+ ret = regulator_enable(base->lcpa_regulator);
+ if (ret)
+ return ret;
+ }
+
+ return pm_runtime_force_resume(dev);
+}
+#endif
+
+#ifdef CONFIG_PM
+static void dma40_backup(void __iomem *baseaddr, u32 *backup,
+ u32 *regaddr, int num, bool save)
+{
+ int i;
+
+ for (i = 0; i < num; i++) {
+ void __iomem *addr = baseaddr + regaddr[i];
+
+ if (save)
+ backup[i] = readl_relaxed(addr);
+ else
+ writel_relaxed(backup[i], addr);
+ }
+}
+
+static void d40_save_restore_registers(struct d40_base *base, bool save)
+{
+ int i;
+
+ /* Save/Restore channel specific registers */
+ for (i = 0; i < base->num_phy_chans; i++) {
+ void __iomem *addr;
+ int idx;
+
+ if (base->phy_res[i].reserved)
+ continue;
+
+ addr = base->virtbase + D40_DREG_PCBASE + i * D40_DREG_PCDELTA;
+ idx = i * ARRAY_SIZE(d40_backup_regs_chan);
+
+ dma40_backup(addr, &base->reg_val_backup_chan[idx],
+ d40_backup_regs_chan,
+ ARRAY_SIZE(d40_backup_regs_chan),
+ save);
+ }
+
+ /* Save/Restore global registers */
+ dma40_backup(base->virtbase, base->reg_val_backup,
+ d40_backup_regs, ARRAY_SIZE(d40_backup_regs),
+ save);
+
+ /* Save/Restore registers only existing on dma40 v3 and later */
+ if (base->gen_dmac.backup)
+ dma40_backup(base->virtbase, base->reg_val_backup_v4,
+ base->gen_dmac.backup,
+ base->gen_dmac.backup_size,
+ save);
}
static int dma40_runtime_suspend(struct device *dev)
@@ -3030,36 +3042,20 @@
struct platform_device *pdev = to_platform_device(dev);
struct d40_base *base = platform_get_drvdata(pdev);
- if (base->initialized)
- d40_save_restore_registers(base, false);
+ d40_save_restore_registers(base, false);
writel_relaxed(D40_DREG_GCC_ENABLE_ALL,
base->virtbase + D40_DREG_GCC);
return 0;
}
-
-static int dma40_resume(struct device *dev)
-{
- struct platform_device *pdev = to_platform_device(dev);
- struct d40_base *base = platform_get_drvdata(pdev);
- int ret = 0;
-
- if (base->lcpa_regulator)
- ret = regulator_enable(base->lcpa_regulator);
-
- return ret;
-}
+#endif
static const struct dev_pm_ops dma40_pm_ops = {
- .suspend = dma40_pm_suspend,
- .runtime_suspend = dma40_runtime_suspend,
- .runtime_resume = dma40_runtime_resume,
- .resume = dma40_resume,
+ SET_LATE_SYSTEM_SLEEP_PM_OPS(dma40_suspend, dma40_resume)
+ SET_PM_RUNTIME_PM_OPS(dma40_runtime_suspend,
+ dma40_runtime_resume,
+ NULL)
};
-#define DMA40_PM_OPS (&dma40_pm_ops)
-#else
-#define DMA40_PM_OPS NULL
-#endif
/* Initialization functions. */
@@ -3645,12 +3641,6 @@
goto failure;
}
- pm_runtime_irq_safe(base->dev);
- pm_runtime_set_autosuspend_delay(base->dev, DMA40_AUTOSUSPEND_DELAY);
- pm_runtime_use_autosuspend(base->dev);
- pm_runtime_enable(base->dev);
- pm_runtime_resume(base->dev);
-
if (base->plat_data->use_esram_lcla) {
base->lcpa_regulator = regulator_get(base->dev, "lcla_esram");
@@ -3671,7 +3661,15 @@
}
}
- base->initialized = true;
+ writel_relaxed(D40_DREG_GCC_ENABLE_ALL, base->virtbase + D40_DREG_GCC);
+
+ pm_runtime_irq_safe(base->dev);
+ pm_runtime_set_autosuspend_delay(base->dev, DMA40_AUTOSUSPEND_DELAY);
+ pm_runtime_use_autosuspend(base->dev);
+ pm_runtime_mark_last_busy(base->dev);
+ pm_runtime_set_active(base->dev);
+ pm_runtime_enable(base->dev);
+
ret = d40_dmaengine_init(base, num_reserved_chans);
if (ret)
goto failure;
@@ -3754,7 +3752,7 @@
.driver = {
.owner = THIS_MODULE,
.name = D40_NAME,
- .pm = DMA40_PM_OPS,
+ .pm = &dma40_pm_ops,
.of_match_table = d40_match,
},
};
diff --git a/drivers/dma/xilinx/Makefile b/drivers/dma/xilinx/Makefile
new file mode 100644
index 0000000..3c4e9f2
--- /dev/null
+++ b/drivers/dma/xilinx/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_XILINX_VDMA) += xilinx_vdma.o
diff --git a/drivers/dma/xilinx/xilinx_vdma.c b/drivers/dma/xilinx/xilinx_vdma.c
new file mode 100644
index 0000000..42a13e8
--- /dev/null
+++ b/drivers/dma/xilinx/xilinx_vdma.c
@@ -0,0 +1,1379 @@
+/*
+ * DMA driver for Xilinx Video DMA Engine
+ *
+ * Copyright (C) 2010-2014 Xilinx, Inc. All rights reserved.
+ *
+ * Based on the Freescale DMA driver.
+ *
+ * Description:
+ * The AXI Video Direct Memory Access (AXI VDMA) core is a soft Xilinx IP
+ * core that provides high-bandwidth direct memory access between memory
+ * and AXI4-Stream type video target peripherals. The core provides efficient
+ * two dimensional DMA operations with independent asynchronous read (S2MM)
+ * and write (MM2S) channel operation. It can be configured to have either
+ * one channel or two channels. If configured as two channels, one is to
+ * transmit to the video device (MM2S) and another is to receive from the
+ * video device (S2MM). Initialization, status, interrupt and management
+ * registers are accessed through an AXI4-Lite slave interface.
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include <linux/amba/xilinx_dma.h>
+#include <linux/bitops.h>
+#include <linux/dmapool.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_dma.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/slab.h>
+
+#include "../dmaengine.h"
+
+/* Register/Descriptor Offsets */
+#define XILINX_VDMA_MM2S_CTRL_OFFSET 0x0000
+#define XILINX_VDMA_S2MM_CTRL_OFFSET 0x0030
+#define XILINX_VDMA_MM2S_DESC_OFFSET 0x0050
+#define XILINX_VDMA_S2MM_DESC_OFFSET 0x00a0
+
+/* Control Registers */
+#define XILINX_VDMA_REG_DMACR 0x0000
+#define XILINX_VDMA_DMACR_DELAY_MAX 0xff
+#define XILINX_VDMA_DMACR_DELAY_SHIFT 24
+#define XILINX_VDMA_DMACR_FRAME_COUNT_MAX 0xff
+#define XILINX_VDMA_DMACR_FRAME_COUNT_SHIFT 16
+#define XILINX_VDMA_DMACR_ERR_IRQ BIT(14)
+#define XILINX_VDMA_DMACR_DLY_CNT_IRQ BIT(13)
+#define XILINX_VDMA_DMACR_FRM_CNT_IRQ BIT(12)
+#define XILINX_VDMA_DMACR_MASTER_SHIFT 8
+#define XILINX_VDMA_DMACR_FSYNCSRC_SHIFT 5
+#define XILINX_VDMA_DMACR_FRAMECNT_EN BIT(4)
+#define XILINX_VDMA_DMACR_GENLOCK_EN BIT(3)
+#define XILINX_VDMA_DMACR_RESET BIT(2)
+#define XILINX_VDMA_DMACR_CIRC_EN BIT(1)
+#define XILINX_VDMA_DMACR_RUNSTOP BIT(0)
+#define XILINX_VDMA_DMACR_FSYNCSRC_MASK GENMASK(6, 5)
+
+#define XILINX_VDMA_REG_DMASR 0x0004
+#define XILINX_VDMA_DMASR_EOL_LATE_ERR BIT(15)
+#define XILINX_VDMA_DMASR_ERR_IRQ BIT(14)
+#define XILINX_VDMA_DMASR_DLY_CNT_IRQ BIT(13)
+#define XILINX_VDMA_DMASR_FRM_CNT_IRQ BIT(12)
+#define XILINX_VDMA_DMASR_SOF_LATE_ERR BIT(11)
+#define XILINX_VDMA_DMASR_SG_DEC_ERR BIT(10)
+#define XILINX_VDMA_DMASR_SG_SLV_ERR BIT(9)
+#define XILINX_VDMA_DMASR_EOF_EARLY_ERR BIT(8)
+#define XILINX_VDMA_DMASR_SOF_EARLY_ERR BIT(7)
+#define XILINX_VDMA_DMASR_DMA_DEC_ERR BIT(6)
+#define XILINX_VDMA_DMASR_DMA_SLAVE_ERR BIT(5)
+#define XILINX_VDMA_DMASR_DMA_INT_ERR BIT(4)
+#define XILINX_VDMA_DMASR_IDLE BIT(1)
+#define XILINX_VDMA_DMASR_HALTED BIT(0)
+#define XILINX_VDMA_DMASR_DELAY_MASK GENMASK(31, 24)
+#define XILINX_VDMA_DMASR_FRAME_COUNT_MASK GENMASK(23, 16)
+
+#define XILINX_VDMA_REG_CURDESC 0x0008
+#define XILINX_VDMA_REG_TAILDESC 0x0010
+#define XILINX_VDMA_REG_REG_INDEX 0x0014
+#define XILINX_VDMA_REG_FRMSTORE 0x0018
+#define XILINX_VDMA_REG_THRESHOLD 0x001c
+#define XILINX_VDMA_REG_FRMPTR_STS 0x0024
+#define XILINX_VDMA_REG_PARK_PTR 0x0028
+#define XILINX_VDMA_PARK_PTR_WR_REF_SHIFT 8
+#define XILINX_VDMA_PARK_PTR_RD_REF_SHIFT 0
+#define XILINX_VDMA_REG_VDMA_VERSION 0x002c
+
+/* Register Direct Mode Registers */
+#define XILINX_VDMA_REG_VSIZE 0x0000
+#define XILINX_VDMA_REG_HSIZE 0x0004
+
+#define XILINX_VDMA_REG_FRMDLY_STRIDE 0x0008
+#define XILINX_VDMA_FRMDLY_STRIDE_FRMDLY_SHIFT 24
+#define XILINX_VDMA_FRMDLY_STRIDE_STRIDE_SHIFT 0
+
+#define XILINX_VDMA_REG_START_ADDRESS(n) (0x000c + 4 * (n))
+
+/* HW specific definitions */
+#define XILINX_VDMA_MAX_CHANS_PER_DEVICE 0x2
+
+#define XILINX_VDMA_DMAXR_ALL_IRQ_MASK \
+ (XILINX_VDMA_DMASR_FRM_CNT_IRQ | \
+ XILINX_VDMA_DMASR_DLY_CNT_IRQ | \
+ XILINX_VDMA_DMASR_ERR_IRQ)
+
+#define XILINX_VDMA_DMASR_ALL_ERR_MASK \
+ (XILINX_VDMA_DMASR_EOL_LATE_ERR | \
+ XILINX_VDMA_DMASR_SOF_LATE_ERR | \
+ XILINX_VDMA_DMASR_SG_DEC_ERR | \
+ XILINX_VDMA_DMASR_SG_SLV_ERR | \
+ XILINX_VDMA_DMASR_EOF_EARLY_ERR | \
+ XILINX_VDMA_DMASR_SOF_EARLY_ERR | \
+ XILINX_VDMA_DMASR_DMA_DEC_ERR | \
+ XILINX_VDMA_DMASR_DMA_SLAVE_ERR | \
+ XILINX_VDMA_DMASR_DMA_INT_ERR)
+
+/*
+ * Recoverable errors are DMA Internal error, SOF Early, EOF Early
+ * and SOF Late. They are only recoverable when C_FLUSH_ON_FSYNC
+ * is enabled in the h/w system.
+ */
+#define XILINX_VDMA_DMASR_ERR_RECOVER_MASK \
+ (XILINX_VDMA_DMASR_SOF_LATE_ERR | \
+ XILINX_VDMA_DMASR_EOF_EARLY_ERR | \
+ XILINX_VDMA_DMASR_SOF_EARLY_ERR | \
+ XILINX_VDMA_DMASR_DMA_INT_ERR)
+
+/* Axi VDMA Flush on Fsync bits */
+#define XILINX_VDMA_FLUSH_S2MM 3
+#define XILINX_VDMA_FLUSH_MM2S 2
+#define XILINX_VDMA_FLUSH_BOTH 1
+
+/* Delay loop counter to prevent hardware failure */
+#define XILINX_VDMA_LOOP_COUNT 1000000
+
+/**
+ * struct xilinx_vdma_desc_hw - Hardware Descriptor
+ * @next_desc: Next Descriptor Pointer @0x00
+ * @pad1: Reserved @0x04
+ * @buf_addr: Buffer address @0x08
+ * @pad2: Reserved @0x0C
+ * @vsize: Vertical Size @0x10
+ * @hsize: Horizontal Size @0x14
+ * @stride: Number of bytes between the first
+ * pixels of each horizontal line @0x18
+ */
+struct xilinx_vdma_desc_hw {
+ u32 next_desc;
+ u32 pad1;
+ u32 buf_addr;
+ u32 pad2;
+ u32 vsize;
+ u32 hsize;
+ u32 stride;
+} __aligned(64);
+
+/**
+ * struct xilinx_vdma_tx_segment - Descriptor segment
+ * @hw: Hardware descriptor
+ * @node: Node in the descriptor segments list
+ * @phys: Physical address of segment
+ */
+struct xilinx_vdma_tx_segment {
+ struct xilinx_vdma_desc_hw hw;
+ struct list_head node;
+ dma_addr_t phys;
+} __aligned(64);
+
+/**
+ * struct xilinx_vdma_tx_descriptor - Per Transaction structure
+ * @async_tx: Async transaction descriptor
+ * @segments: TX segments list
+ * @node: Node in the channel descriptors list
+ */
+struct xilinx_vdma_tx_descriptor {
+ struct dma_async_tx_descriptor async_tx;
+ struct list_head segments;
+ struct list_head node;
+};
+
+/**
+ * struct xilinx_vdma_chan - Driver specific VDMA channel structure
+ * @xdev: Driver specific device structure
+ * @ctrl_offset: Control registers offset
+ * @desc_offset: TX descriptor registers offset
+ * @lock: Descriptor operation lock
+ * @pending_list: Descriptors waiting
+ * @active_desc: Active descriptor
+ * @allocated_desc: Allocated descriptor
+ * @done_list: Complete descriptors
+ * @common: DMA common channel
+ * @desc_pool: Descriptors pool
+ * @dev: The dma device
+ * @irq: Channel IRQ
+ * @id: Channel ID
+ * @direction: Transfer direction
+ * @num_frms: Number of frames
+ * @has_sg: Support scatter transfers
+ * @genlock: Support genlock mode
+ * @err: Channel has errors
+ * @tasklet: Cleanup work after irq
+ * @config: Device configuration info
+ * @flush_on_fsync: Flush on Frame sync
+ */
+struct xilinx_vdma_chan {
+ struct xilinx_vdma_device *xdev;
+ u32 ctrl_offset;
+ u32 desc_offset;
+ spinlock_t lock;
+ struct list_head pending_list;
+ struct xilinx_vdma_tx_descriptor *active_desc;
+ struct xilinx_vdma_tx_descriptor *allocated_desc;
+ struct list_head done_list;
+ struct dma_chan common;
+ struct dma_pool *desc_pool;
+ struct device *dev;
+ int irq;
+ int id;
+ enum dma_transfer_direction direction;
+ int num_frms;
+ bool has_sg;
+ bool genlock;
+ bool err;
+ struct tasklet_struct tasklet;
+ struct xilinx_vdma_config config;
+ bool flush_on_fsync;
+};
+
+/**
+ * struct xilinx_vdma_device - VDMA device structure
+ * @regs: I/O mapped base address
+ * @dev: Device Structure
+ * @common: DMA device structure
+ * @chan: Driver specific VDMA channel
+ * @has_sg: Specifies whether Scatter-Gather is present or not
+ * @flush_on_fsync: Flush on frame sync
+ */
+struct xilinx_vdma_device {
+ void __iomem *regs;
+ struct device *dev;
+ struct dma_device common;
+ struct xilinx_vdma_chan *chan[XILINX_VDMA_MAX_CHANS_PER_DEVICE];
+ bool has_sg;
+ u32 flush_on_fsync;
+};
+
+/* Macros */
+#define to_xilinx_chan(chan) \
+ container_of(chan, struct xilinx_vdma_chan, common)
+#define to_vdma_tx_descriptor(tx) \
+ container_of(tx, struct xilinx_vdma_tx_descriptor, async_tx)
+
+/* IO accessors */
+static inline u32 vdma_read(struct xilinx_vdma_chan *chan, u32 reg)
+{
+ return ioread32(chan->xdev->regs + reg);
+}
+
+static inline void vdma_write(struct xilinx_vdma_chan *chan, u32 reg, u32 value)
+{
+ iowrite32(value, chan->xdev->regs + reg);
+}
+
+static inline void vdma_desc_write(struct xilinx_vdma_chan *chan, u32 reg,
+ u32 value)
+{
+ vdma_write(chan, chan->desc_offset + reg, value);
+}
+
+static inline u32 vdma_ctrl_read(struct xilinx_vdma_chan *chan, u32 reg)
+{
+ return vdma_read(chan, chan->ctrl_offset + reg);
+}
+
+static inline void vdma_ctrl_write(struct xilinx_vdma_chan *chan, u32 reg,
+ u32 value)
+{
+ vdma_write(chan, chan->ctrl_offset + reg, value);
+}
+
+static inline void vdma_ctrl_clr(struct xilinx_vdma_chan *chan, u32 reg,
+ u32 clr)
+{
+ vdma_ctrl_write(chan, reg, vdma_ctrl_read(chan, reg) & ~clr);
+}
+
+static inline void vdma_ctrl_set(struct xilinx_vdma_chan *chan, u32 reg,
+ u32 set)
+{
+ vdma_ctrl_write(chan, reg, vdma_ctrl_read(chan, reg) | set);
+}
+
+/* -----------------------------------------------------------------------------
+ * Descriptors and segments alloc and free
+ */
+
+/**
+ * xilinx_vdma_alloc_tx_segment - Allocate transaction segment
+ * @chan: Driver specific VDMA channel
+ *
+ * Return: The allocated segment on success and NULL on failure.
+ */
+static struct xilinx_vdma_tx_segment *
+xilinx_vdma_alloc_tx_segment(struct xilinx_vdma_chan *chan)
+{
+ struct xilinx_vdma_tx_segment *segment;
+ dma_addr_t phys;
+
+ segment = dma_pool_alloc(chan->desc_pool, GFP_ATOMIC, &phys);
+ if (!segment)
+ return NULL;
+
+ memset(segment, 0, sizeof(*segment));
+ segment->phys = phys;
+
+ return segment;
+}
+
+/**
+ * xilinx_vdma_free_tx_segment - Free transaction segment
+ * @chan: Driver specific VDMA channel
+ * @segment: VDMA transaction segment
+ */
+static void xilinx_vdma_free_tx_segment(struct xilinx_vdma_chan *chan,
+ struct xilinx_vdma_tx_segment *segment)
+{
+ dma_pool_free(chan->desc_pool, segment, segment->phys);
+}
+
+/**
+ * xilinx_vdma_tx_descriptor - Allocate transaction descriptor
+ * @chan: Driver specific VDMA channel
+ *
+ * Return: The allocated descriptor on success and NULL on failure.
+ */
+static struct xilinx_vdma_tx_descriptor *
+xilinx_vdma_alloc_tx_descriptor(struct xilinx_vdma_chan *chan)
+{
+ struct xilinx_vdma_tx_descriptor *desc;
+ unsigned long flags;
+
+ if (chan->allocated_desc)
+ return chan->allocated_desc;
+
+ desc = kzalloc(sizeof(*desc), GFP_KERNEL);
+ if (!desc)
+ return NULL;
+
+ spin_lock_irqsave(&chan->lock, flags);
+ chan->allocated_desc = desc;
+ spin_unlock_irqrestore(&chan->lock, flags);
+
+ INIT_LIST_HEAD(&desc->segments);
+
+ return desc;
+}
+
+/**
+ * xilinx_vdma_free_tx_descriptor - Free transaction descriptor
+ * @chan: Driver specific VDMA channel
+ * @desc: VDMA transaction descriptor
+ */
+static void
+xilinx_vdma_free_tx_descriptor(struct xilinx_vdma_chan *chan,
+ struct xilinx_vdma_tx_descriptor *desc)
+{
+ struct xilinx_vdma_tx_segment *segment, *next;
+
+ if (!desc)
+ return;
+
+ list_for_each_entry_safe(segment, next, &desc->segments, node) {
+ list_del(&segment->node);
+ xilinx_vdma_free_tx_segment(chan, segment);
+ }
+
+ kfree(desc);
+}
+
+/* Required functions */
+
+/**
+ * xilinx_vdma_free_desc_list - Free descriptors list
+ * @chan: Driver specific VDMA channel
+ * @list: List to parse and delete the descriptor
+ */
+static void xilinx_vdma_free_desc_list(struct xilinx_vdma_chan *chan,
+ struct list_head *list)
+{
+ struct xilinx_vdma_tx_descriptor *desc, *next;
+
+ list_for_each_entry_safe(desc, next, list, node) {
+ list_del(&desc->node);
+ xilinx_vdma_free_tx_descriptor(chan, desc);
+ }
+}
+
+/**
+ * xilinx_vdma_free_descriptors - Free channel descriptors
+ * @chan: Driver specific VDMA channel
+ */
+static void xilinx_vdma_free_descriptors(struct xilinx_vdma_chan *chan)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&chan->lock, flags);
+
+ xilinx_vdma_free_desc_list(chan, &chan->pending_list);
+ xilinx_vdma_free_desc_list(chan, &chan->done_list);
+
+ xilinx_vdma_free_tx_descriptor(chan, chan->active_desc);
+ chan->active_desc = NULL;
+
+ spin_unlock_irqrestore(&chan->lock, flags);
+}
+
+/**
+ * xilinx_vdma_free_chan_resources - Free channel resources
+ * @dchan: DMA channel
+ */
+static void xilinx_vdma_free_chan_resources(struct dma_chan *dchan)
+{
+ struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
+
+ dev_dbg(chan->dev, "Free all channel resources.\n");
+
+ xilinx_vdma_free_descriptors(chan);
+ dma_pool_destroy(chan->desc_pool);
+ chan->desc_pool = NULL;
+}
+
+/**
+ * xilinx_vdma_chan_desc_cleanup - Clean channel descriptors
+ * @chan: Driver specific VDMA channel
+ */
+static void xilinx_vdma_chan_desc_cleanup(struct xilinx_vdma_chan *chan)
+{
+ struct xilinx_vdma_tx_descriptor *desc, *next;
+ unsigned long flags;
+
+ spin_lock_irqsave(&chan->lock, flags);
+
+ list_for_each_entry_safe(desc, next, &chan->done_list, node) {
+ dma_async_tx_callback callback;
+ void *callback_param;
+
+ /* Remove from the list of running transactions */
+ list_del(&desc->node);
+
+ /* Run the link descriptor callback function */
+ callback = desc->async_tx.callback;
+ callback_param = desc->async_tx.callback_param;
+ if (callback) {
+ spin_unlock_irqrestore(&chan->lock, flags);
+ callback(callback_param);
+ spin_lock_irqsave(&chan->lock, flags);
+ }
+
+ /* Run any dependencies, then free the descriptor */
+ dma_run_dependencies(&desc->async_tx);
+ xilinx_vdma_free_tx_descriptor(chan, desc);
+ }
+
+ spin_unlock_irqrestore(&chan->lock, flags);
+}
+
+/**
+ * xilinx_vdma_do_tasklet - Schedule completion tasklet
+ * @data: Pointer to the Xilinx VDMA channel structure
+ */
+static void xilinx_vdma_do_tasklet(unsigned long data)
+{
+ struct xilinx_vdma_chan *chan = (struct xilinx_vdma_chan *)data;
+
+ xilinx_vdma_chan_desc_cleanup(chan);
+}
+
+/**
+ * xilinx_vdma_alloc_chan_resources - Allocate channel resources
+ * @dchan: DMA channel
+ *
+ * Return: '0' on success and failure value on error
+ */
+static int xilinx_vdma_alloc_chan_resources(struct dma_chan *dchan)
+{
+ struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
+
+ /* Has this channel already been allocated? */
+ if (chan->desc_pool)
+ return 0;
+
+ /*
+ * We need the descriptor to be aligned to 64bytes
+ * for meeting Xilinx VDMA specification requirement.
+ */
+ chan->desc_pool = dma_pool_create("xilinx_vdma_desc_pool",
+ chan->dev,
+ sizeof(struct xilinx_vdma_tx_segment),
+ __alignof__(struct xilinx_vdma_tx_segment), 0);
+ if (!chan->desc_pool) {
+ dev_err(chan->dev,
+ "unable to allocate channel %d descriptor pool\n",
+ chan->id);
+ return -ENOMEM;
+ }
+
+ dma_cookie_init(dchan);
+ return 0;
+}
+
+/**
+ * xilinx_vdma_tx_status - Get VDMA transaction status
+ * @dchan: DMA channel
+ * @cookie: Transaction identifier
+ * @txstate: Transaction state
+ *
+ * Return: DMA transaction status
+ */
+static enum dma_status xilinx_vdma_tx_status(struct dma_chan *dchan,
+ dma_cookie_t cookie,
+ struct dma_tx_state *txstate)
+{
+ return dma_cookie_status(dchan, cookie, txstate);
+}
+
+/**
+ * xilinx_vdma_is_running - Check if VDMA channel is running
+ * @chan: Driver specific VDMA channel
+ *
+ * Return: '1' if running, '0' if not.
+ */
+static bool xilinx_vdma_is_running(struct xilinx_vdma_chan *chan)
+{
+ return !(vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR) &
+ XILINX_VDMA_DMASR_HALTED) &&
+ (vdma_ctrl_read(chan, XILINX_VDMA_REG_DMACR) &
+ XILINX_VDMA_DMACR_RUNSTOP);
+}
+
+/**
+ * xilinx_vdma_is_idle - Check if VDMA channel is idle
+ * @chan: Driver specific VDMA channel
+ *
+ * Return: '1' if idle, '0' if not.
+ */
+static bool xilinx_vdma_is_idle(struct xilinx_vdma_chan *chan)
+{
+ return vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR) &
+ XILINX_VDMA_DMASR_IDLE;
+}
+
+/**
+ * xilinx_vdma_halt - Halt VDMA channel
+ * @chan: Driver specific VDMA channel
+ */
+static void xilinx_vdma_halt(struct xilinx_vdma_chan *chan)
+{
+ int loop = XILINX_VDMA_LOOP_COUNT;
+
+ vdma_ctrl_clr(chan, XILINX_VDMA_REG_DMACR, XILINX_VDMA_DMACR_RUNSTOP);
+
+ /* Wait for the hardware to halt */
+ do {
+ if (vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR) &
+ XILINX_VDMA_DMASR_HALTED)
+ break;
+ } while (loop--);
+
+ if (!loop) {
+ dev_err(chan->dev, "Cannot stop channel %p: %x\n",
+ chan, vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR));
+ chan->err = true;
+ }
+
+ return;
+}
+
+/**
+ * xilinx_vdma_start - Start VDMA channel
+ * @chan: Driver specific VDMA channel
+ */
+static void xilinx_vdma_start(struct xilinx_vdma_chan *chan)
+{
+ int loop = XILINX_VDMA_LOOP_COUNT;
+
+ vdma_ctrl_set(chan, XILINX_VDMA_REG_DMACR, XILINX_VDMA_DMACR_RUNSTOP);
+
+ /* Wait for the hardware to start */
+ do {
+ if (!(vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR) &
+ XILINX_VDMA_DMASR_HALTED))
+ break;
+ } while (loop--);
+
+ if (!loop) {
+ dev_err(chan->dev, "Cannot start channel %p: %x\n",
+ chan, vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR));
+
+ chan->err = true;
+ }
+
+ return;
+}
+
+/**
+ * xilinx_vdma_start_transfer - Starts VDMA transfer
+ * @chan: Driver specific channel struct pointer
+ */
+static void xilinx_vdma_start_transfer(struct xilinx_vdma_chan *chan)
+{
+ struct xilinx_vdma_config *config = &chan->config;
+ struct xilinx_vdma_tx_descriptor *desc;
+ unsigned long flags;
+ u32 reg;
+ struct xilinx_vdma_tx_segment *head, *tail = NULL;
+
+ if (chan->err)
+ return;
+
+ spin_lock_irqsave(&chan->lock, flags);
+
+ /* There's already an active descriptor, bail out. */
+ if (chan->active_desc)
+ goto out_unlock;
+
+ if (list_empty(&chan->pending_list))
+ goto out_unlock;
+
+ desc = list_first_entry(&chan->pending_list,
+ struct xilinx_vdma_tx_descriptor, node);
+
+ /* If it is SG mode and hardware is busy, cannot submit */
+ if (chan->has_sg && xilinx_vdma_is_running(chan) &&
+ !xilinx_vdma_is_idle(chan)) {
+ dev_dbg(chan->dev, "DMA controller still busy\n");
+ goto out_unlock;
+ }
+
+ /*
+ * If hardware is idle, then all descriptors on the running lists are
+ * done, start new transfers
+ */
+ if (chan->has_sg) {
+ head = list_first_entry(&desc->segments,
+ struct xilinx_vdma_tx_segment, node);
+ tail = list_entry(desc->segments.prev,
+ struct xilinx_vdma_tx_segment, node);
+
+ vdma_ctrl_write(chan, XILINX_VDMA_REG_CURDESC, head->phys);
+ }
+
+ /* Configure the hardware using info in the config structure */
+ reg = vdma_ctrl_read(chan, XILINX_VDMA_REG_DMACR);
+
+ if (config->frm_cnt_en)
+ reg |= XILINX_VDMA_DMACR_FRAMECNT_EN;
+ else
+ reg &= ~XILINX_VDMA_DMACR_FRAMECNT_EN;
+
+ /*
+ * With SG, start with circular mode, so that BDs can be fetched.
+ * In direct register mode, if not parking, enable circular mode
+ */
+ if (chan->has_sg || !config->park)
+ reg |= XILINX_VDMA_DMACR_CIRC_EN;
+
+ if (config->park)
+ reg &= ~XILINX_VDMA_DMACR_CIRC_EN;
+
+ vdma_ctrl_write(chan, XILINX_VDMA_REG_DMACR, reg);
+
+ if (config->park && (config->park_frm >= 0) &&
+ (config->park_frm < chan->num_frms)) {
+ if (chan->direction == DMA_MEM_TO_DEV)
+ vdma_write(chan, XILINX_VDMA_REG_PARK_PTR,
+ config->park_frm <<
+ XILINX_VDMA_PARK_PTR_RD_REF_SHIFT);
+ else
+ vdma_write(chan, XILINX_VDMA_REG_PARK_PTR,
+ config->park_frm <<
+ XILINX_VDMA_PARK_PTR_WR_REF_SHIFT);
+ }
+
+ /* Start the hardware */
+ xilinx_vdma_start(chan);
+
+ if (chan->err)
+ goto out_unlock;
+
+ /* Start the transfer */
+ if (chan->has_sg) {
+ vdma_ctrl_write(chan, XILINX_VDMA_REG_TAILDESC, tail->phys);
+ } else {
+ struct xilinx_vdma_tx_segment *segment, *last = NULL;
+ int i = 0;
+
+ list_for_each_entry(segment, &desc->segments, node) {
+ vdma_desc_write(chan,
+ XILINX_VDMA_REG_START_ADDRESS(i++),
+ segment->hw.buf_addr);
+ last = segment;
+ }
+
+ if (!last)
+ goto out_unlock;
+
+ /* HW expects these parameters to be same for one transaction */
+ vdma_desc_write(chan, XILINX_VDMA_REG_HSIZE, last->hw.hsize);
+ vdma_desc_write(chan, XILINX_VDMA_REG_FRMDLY_STRIDE,
+ last->hw.stride);
+ vdma_desc_write(chan, XILINX_VDMA_REG_VSIZE, last->hw.vsize);
+ }
+
+ list_del(&desc->node);
+ chan->active_desc = desc;
+
+out_unlock:
+ spin_unlock_irqrestore(&chan->lock, flags);
+}
+
+/**
+ * xilinx_vdma_issue_pending - Issue pending transactions
+ * @dchan: DMA channel
+ */
+static void xilinx_vdma_issue_pending(struct dma_chan *dchan)
+{
+ struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
+
+ xilinx_vdma_start_transfer(chan);
+}
+
+/**
+ * xilinx_vdma_complete_descriptor - Mark the active descriptor as complete
+ * @chan : xilinx DMA channel
+ *
+ * CONTEXT: hardirq
+ */
+static void xilinx_vdma_complete_descriptor(struct xilinx_vdma_chan *chan)
+{
+ struct xilinx_vdma_tx_descriptor *desc;
+ unsigned long flags;
+
+ spin_lock_irqsave(&chan->lock, flags);
+
+ desc = chan->active_desc;
+ if (!desc) {
+ dev_dbg(chan->dev, "no running descriptors\n");
+ goto out_unlock;
+ }
+
+ dma_cookie_complete(&desc->async_tx);
+ list_add_tail(&desc->node, &chan->done_list);
+
+ chan->active_desc = NULL;
+
+out_unlock:
+ spin_unlock_irqrestore(&chan->lock, flags);
+}
+
+/**
+ * xilinx_vdma_reset - Reset VDMA channel
+ * @chan: Driver specific VDMA channel
+ *
+ * Return: '0' on success and failure value on error
+ */
+static int xilinx_vdma_reset(struct xilinx_vdma_chan *chan)
+{
+ int loop = XILINX_VDMA_LOOP_COUNT;
+ u32 tmp;
+
+ vdma_ctrl_set(chan, XILINX_VDMA_REG_DMACR, XILINX_VDMA_DMACR_RESET);
+
+ tmp = vdma_ctrl_read(chan, XILINX_VDMA_REG_DMACR) &
+ XILINX_VDMA_DMACR_RESET;
+
+ /* Wait for the hardware to finish reset */
+ do {
+ tmp = vdma_ctrl_read(chan, XILINX_VDMA_REG_DMACR) &
+ XILINX_VDMA_DMACR_RESET;
+ } while (loop-- && tmp);
+
+ if (!loop) {
+ dev_err(chan->dev, "reset timeout, cr %x, sr %x\n",
+ vdma_ctrl_read(chan, XILINX_VDMA_REG_DMACR),
+ vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR));
+ return -ETIMEDOUT;
+ }
+
+ chan->err = false;
+
+ return 0;
+}
+
+/**
+ * xilinx_vdma_chan_reset - Reset VDMA channel and enable interrupts
+ * @chan: Driver specific VDMA channel
+ *
+ * Return: '0' on success and failure value on error
+ */
+static int xilinx_vdma_chan_reset(struct xilinx_vdma_chan *chan)
+{
+ int err;
+
+ /* Reset VDMA */
+ err = xilinx_vdma_reset(chan);
+ if (err)
+ return err;
+
+ /* Enable interrupts */
+ vdma_ctrl_set(chan, XILINX_VDMA_REG_DMACR,
+ XILINX_VDMA_DMAXR_ALL_IRQ_MASK);
+
+ return 0;
+}
+
+/**
+ * xilinx_vdma_irq_handler - VDMA Interrupt handler
+ * @irq: IRQ number
+ * @data: Pointer to the Xilinx VDMA channel structure
+ *
+ * Return: IRQ_HANDLED/IRQ_NONE
+ */
+static irqreturn_t xilinx_vdma_irq_handler(int irq, void *data)
+{
+ struct xilinx_vdma_chan *chan = data;
+ u32 status;
+
+ /* Read the status and ack the interrupts. */
+ status = vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR);
+ if (!(status & XILINX_VDMA_DMAXR_ALL_IRQ_MASK))
+ return IRQ_NONE;
+
+ vdma_ctrl_write(chan, XILINX_VDMA_REG_DMASR,
+ status & XILINX_VDMA_DMAXR_ALL_IRQ_MASK);
+
+ if (status & XILINX_VDMA_DMASR_ERR_IRQ) {
+ /*
+ * An error occurred. If C_FLUSH_ON_FSYNC is enabled and the
+ * error is recoverable, ignore it. Otherwise flag the error.
+ *
+ * Only recoverable errors can be cleared in the DMASR register,
+ * make sure not to write to other error bits to 1.
+ */
+ u32 errors = status & XILINX_VDMA_DMASR_ALL_ERR_MASK;
+ vdma_ctrl_write(chan, XILINX_VDMA_REG_DMASR,
+ errors & XILINX_VDMA_DMASR_ERR_RECOVER_MASK);
+
+ if (!chan->flush_on_fsync ||
+ (errors & ~XILINX_VDMA_DMASR_ERR_RECOVER_MASK)) {
+ dev_err(chan->dev,
+ "Channel %p has errors %x, cdr %x tdr %x\n",
+ chan, errors,
+ vdma_ctrl_read(chan, XILINX_VDMA_REG_CURDESC),
+ vdma_ctrl_read(chan, XILINX_VDMA_REG_TAILDESC));
+ chan->err = true;
+ }
+ }
+
+ if (status & XILINX_VDMA_DMASR_DLY_CNT_IRQ) {
+ /*
+ * Device takes too long to do the transfer when user requires
+ * responsiveness.
+ */
+ dev_dbg(chan->dev, "Inter-packet latency too long\n");
+ }
+
+ if (status & XILINX_VDMA_DMASR_FRM_CNT_IRQ) {
+ xilinx_vdma_complete_descriptor(chan);
+ xilinx_vdma_start_transfer(chan);
+ }
+
+ tasklet_schedule(&chan->tasklet);
+ return IRQ_HANDLED;
+}
+
+/**
+ * xilinx_vdma_tx_submit - Submit DMA transaction
+ * @tx: Async transaction descriptor
+ *
+ * Return: cookie value on success and failure value on error
+ */
+static dma_cookie_t xilinx_vdma_tx_submit(struct dma_async_tx_descriptor *tx)
+{
+ struct xilinx_vdma_tx_descriptor *desc = to_vdma_tx_descriptor(tx);
+ struct xilinx_vdma_chan *chan = to_xilinx_chan(tx->chan);
+ dma_cookie_t cookie;
+ unsigned long flags;
+ int err;
+
+ if (chan->err) {
+ /*
+ * If reset fails, need to hard reset the system.
+ * Channel is no longer functional
+ */
+ err = xilinx_vdma_chan_reset(chan);
+ if (err < 0)
+ return err;
+ }
+
+ spin_lock_irqsave(&chan->lock, flags);
+
+ cookie = dma_cookie_assign(tx);
+
+ /* Append the transaction to the pending transactions queue. */
+ list_add_tail(&desc->node, &chan->pending_list);
+
+ /* Free the allocated desc */
+ chan->allocated_desc = NULL;
+
+ spin_unlock_irqrestore(&chan->lock, flags);
+
+ return cookie;
+}
+
+/**
+ * xilinx_vdma_dma_prep_interleaved - prepare a descriptor for a
+ * DMA_SLAVE transaction
+ * @dchan: DMA channel
+ * @xt: Interleaved template pointer
+ * @flags: transfer ack flags
+ *
+ * Return: Async transaction descriptor on success and NULL on failure
+ */
+static struct dma_async_tx_descriptor *
+xilinx_vdma_dma_prep_interleaved(struct dma_chan *dchan,
+ struct dma_interleaved_template *xt,
+ unsigned long flags)
+{
+ struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
+ struct xilinx_vdma_tx_descriptor *desc;
+ struct xilinx_vdma_tx_segment *segment, *prev = NULL;
+ struct xilinx_vdma_desc_hw *hw;
+
+ if (!is_slave_direction(xt->dir))
+ return NULL;
+
+ if (!xt->numf || !xt->sgl[0].size)
+ return NULL;
+
+ /* Allocate a transaction descriptor. */
+ desc = xilinx_vdma_alloc_tx_descriptor(chan);
+ if (!desc)
+ return NULL;
+
+ dma_async_tx_descriptor_init(&desc->async_tx, &chan->common);
+ desc->async_tx.tx_submit = xilinx_vdma_tx_submit;
+ async_tx_ack(&desc->async_tx);
+
+ /* Allocate the link descriptor from DMA pool */
+ segment = xilinx_vdma_alloc_tx_segment(chan);
+ if (!segment)
+ goto error;
+
+ /* Fill in the hardware descriptor */
+ hw = &segment->hw;
+ hw->vsize = xt->numf;
+ hw->hsize = xt->sgl[0].size;
+ hw->stride = xt->sgl[0].icg <<
+ XILINX_VDMA_FRMDLY_STRIDE_STRIDE_SHIFT;
+ hw->stride |= chan->config.frm_dly <<
+ XILINX_VDMA_FRMDLY_STRIDE_FRMDLY_SHIFT;
+
+ if (xt->dir != DMA_MEM_TO_DEV)
+ hw->buf_addr = xt->dst_start;
+ else
+ hw->buf_addr = xt->src_start;
+
+ /* Link the previous next descriptor to current */
+ prev = list_last_entry(&desc->segments,
+ struct xilinx_vdma_tx_segment, node);
+ prev->hw.next_desc = segment->phys;
+
+ /* Insert the segment into the descriptor segments list. */
+ list_add_tail(&segment->node, &desc->segments);
+
+ prev = segment;
+
+ /* Link the last hardware descriptor with the first. */
+ segment = list_first_entry(&desc->segments,
+ struct xilinx_vdma_tx_segment, node);
+ prev->hw.next_desc = segment->phys;
+
+ return &desc->async_tx;
+
+error:
+ xilinx_vdma_free_tx_descriptor(chan, desc);
+ return NULL;
+}
+
+/**
+ * xilinx_vdma_terminate_all - Halt the channel and free descriptors
+ * @chan: Driver specific VDMA Channel pointer
+ */
+static void xilinx_vdma_terminate_all(struct xilinx_vdma_chan *chan)
+{
+ /* Halt the DMA engine */
+ xilinx_vdma_halt(chan);
+
+ /* Remove and free all of the descriptors in the lists */
+ xilinx_vdma_free_descriptors(chan);
+}
+
+/**
+ * xilinx_vdma_channel_set_config - Configure VDMA channel
+ * Run-time configuration for Axi VDMA, supports:
+ * . halt the channel
+ * . configure interrupt coalescing and inter-packet delay threshold
+ * . start/stop parking
+ * . enable genlock
+ *
+ * @dchan: DMA channel
+ * @cfg: VDMA device configuration pointer
+ *
+ * Return: '0' on success and failure value on error
+ */
+int xilinx_vdma_channel_set_config(struct dma_chan *dchan,
+ struct xilinx_vdma_config *cfg)
+{
+ struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
+ u32 dmacr;
+
+ if (cfg->reset)
+ return xilinx_vdma_chan_reset(chan);
+
+ dmacr = vdma_ctrl_read(chan, XILINX_VDMA_REG_DMACR);
+
+ chan->config.frm_dly = cfg->frm_dly;
+ chan->config.park = cfg->park;
+
+ /* genlock settings */
+ chan->config.gen_lock = cfg->gen_lock;
+ chan->config.master = cfg->master;
+
+ if (cfg->gen_lock && chan->genlock) {
+ dmacr |= XILINX_VDMA_DMACR_GENLOCK_EN;
+ dmacr |= cfg->master << XILINX_VDMA_DMACR_MASTER_SHIFT;
+ }
+
+ chan->config.frm_cnt_en = cfg->frm_cnt_en;
+ if (cfg->park)
+ chan->config.park_frm = cfg->park_frm;
+ else
+ chan->config.park_frm = -1;
+
+ chan->config.coalesc = cfg->coalesc;
+ chan->config.delay = cfg->delay;
+
+ if (cfg->coalesc <= XILINX_VDMA_DMACR_FRAME_COUNT_MAX) {
+ dmacr |= cfg->coalesc << XILINX_VDMA_DMACR_FRAME_COUNT_SHIFT;
+ chan->config.coalesc = cfg->coalesc;
+ }
+
+ if (cfg->delay <= XILINX_VDMA_DMACR_DELAY_MAX) {
+ dmacr |= cfg->delay << XILINX_VDMA_DMACR_DELAY_SHIFT;
+ chan->config.delay = cfg->delay;
+ }
+
+ /* FSync Source selection */
+ dmacr &= ~XILINX_VDMA_DMACR_FSYNCSRC_MASK;
+ dmacr |= cfg->ext_fsync << XILINX_VDMA_DMACR_FSYNCSRC_SHIFT;
+
+ vdma_ctrl_write(chan, XILINX_VDMA_REG_DMACR, dmacr);
+
+ return 0;
+}
+EXPORT_SYMBOL(xilinx_vdma_channel_set_config);
+
+/**
+ * xilinx_vdma_device_control - Configure DMA channel of the device
+ * @dchan: DMA Channel pointer
+ * @cmd: DMA control command
+ * @arg: Channel configuration
+ *
+ * Return: '0' on success and failure value on error
+ */
+static int xilinx_vdma_device_control(struct dma_chan *dchan,
+ enum dma_ctrl_cmd cmd, unsigned long arg)
+{
+ struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
+
+ if (cmd != DMA_TERMINATE_ALL)
+ return -ENXIO;
+
+ xilinx_vdma_terminate_all(chan);
+
+ return 0;
+}
+
+/* -----------------------------------------------------------------------------
+ * Probe and remove
+ */
+
+/**
+ * xilinx_vdma_chan_remove - Per Channel remove function
+ * @chan: Driver specific VDMA channel
+ */
+static void xilinx_vdma_chan_remove(struct xilinx_vdma_chan *chan)
+{
+ /* Disable all interrupts */
+ vdma_ctrl_clr(chan, XILINX_VDMA_REG_DMACR,
+ XILINX_VDMA_DMAXR_ALL_IRQ_MASK);
+
+ if (chan->irq > 0)
+ free_irq(chan->irq, chan);
+
+ tasklet_kill(&chan->tasklet);
+
+ list_del(&chan->common.device_node);
+}
+
+/**
+ * xilinx_vdma_chan_probe - Per Channel Probing
+ * It get channel features from the device tree entry and
+ * initialize special channel handling routines
+ *
+ * @xdev: Driver specific device structure
+ * @node: Device node
+ *
+ * Return: '0' on success and failure value on error
+ */
+static int xilinx_vdma_chan_probe(struct xilinx_vdma_device *xdev,
+ struct device_node *node)
+{
+ struct xilinx_vdma_chan *chan;
+ bool has_dre = false;
+ u32 value, width;
+ int err;
+
+ /* Allocate and initialize the channel structure */
+ chan = devm_kzalloc(xdev->dev, sizeof(*chan), GFP_KERNEL);
+ if (!chan)
+ return -ENOMEM;
+
+ chan->dev = xdev->dev;
+ chan->xdev = xdev;
+ chan->has_sg = xdev->has_sg;
+
+ spin_lock_init(&chan->lock);
+ INIT_LIST_HEAD(&chan->pending_list);
+ INIT_LIST_HEAD(&chan->done_list);
+
+ /* Retrieve the channel properties from the device tree */
+ has_dre = of_property_read_bool(node, "xlnx,include-dre");
+
+ chan->genlock = of_property_read_bool(node, "xlnx,genlock-mode");
+
+ err = of_property_read_u32(node, "xlnx,datawidth", &value);
+ if (err) {
+ dev_err(xdev->dev, "missing xlnx,datawidth property\n");
+ return err;
+ }
+ width = value >> 3; /* Convert bits to bytes */
+
+ /* If data width is greater than 8 bytes, DRE is not in hw */
+ if (width > 8)
+ has_dre = false;
+
+ if (!has_dre)
+ xdev->common.copy_align = fls(width - 1);
+
+ if (of_device_is_compatible(node, "xlnx,axi-vdma-mm2s-channel")) {
+ chan->direction = DMA_MEM_TO_DEV;
+ chan->id = 0;
+
+ chan->ctrl_offset = XILINX_VDMA_MM2S_CTRL_OFFSET;
+ chan->desc_offset = XILINX_VDMA_MM2S_DESC_OFFSET;
+
+ if (xdev->flush_on_fsync == XILINX_VDMA_FLUSH_BOTH ||
+ xdev->flush_on_fsync == XILINX_VDMA_FLUSH_MM2S)
+ chan->flush_on_fsync = true;
+ } else if (of_device_is_compatible(node,
+ "xlnx,axi-vdma-s2mm-channel")) {
+ chan->direction = DMA_DEV_TO_MEM;
+ chan->id = 1;
+
+ chan->ctrl_offset = XILINX_VDMA_S2MM_CTRL_OFFSET;
+ chan->desc_offset = XILINX_VDMA_S2MM_DESC_OFFSET;
+
+ if (xdev->flush_on_fsync == XILINX_VDMA_FLUSH_BOTH ||
+ xdev->flush_on_fsync == XILINX_VDMA_FLUSH_S2MM)
+ chan->flush_on_fsync = true;
+ } else {
+ dev_err(xdev->dev, "Invalid channel compatible node\n");
+ return -EINVAL;
+ }
+
+ /* Request the interrupt */
+ chan->irq = irq_of_parse_and_map(node, 0);
+ err = request_irq(chan->irq, xilinx_vdma_irq_handler, IRQF_SHARED,
+ "xilinx-vdma-controller", chan);
+ if (err) {
+ dev_err(xdev->dev, "unable to request IRQ %d\n", chan->irq);
+ return err;
+ }
+
+ /* Initialize the tasklet */
+ tasklet_init(&chan->tasklet, xilinx_vdma_do_tasklet,
+ (unsigned long)chan);
+
+ /*
+ * Initialize the DMA channel and add it to the DMA engine channels
+ * list.
+ */
+ chan->common.device = &xdev->common;
+
+ list_add_tail(&chan->common.device_node, &xdev->common.channels);
+ xdev->chan[chan->id] = chan;
+
+ /* Reset the channel */
+ err = xilinx_vdma_chan_reset(chan);
+ if (err < 0) {
+ dev_err(xdev->dev, "Reset channel failed\n");
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * of_dma_xilinx_xlate - Translation function
+ * @dma_spec: Pointer to DMA specifier as found in the device tree
+ * @ofdma: Pointer to DMA controller data
+ *
+ * Return: DMA channel pointer on success and NULL on error
+ */
+static struct dma_chan *of_dma_xilinx_xlate(struct of_phandle_args *dma_spec,
+ struct of_dma *ofdma)
+{
+ struct xilinx_vdma_device *xdev = ofdma->of_dma_data;
+ int chan_id = dma_spec->args[0];
+
+ if (chan_id >= XILINX_VDMA_MAX_CHANS_PER_DEVICE)
+ return NULL;
+
+ return dma_get_slave_channel(&xdev->chan[chan_id]->common);
+}
+
+/**
+ * xilinx_vdma_probe - Driver probe function
+ * @pdev: Pointer to the platform_device structure
+ *
+ * Return: '0' on success and failure value on error
+ */
+static int xilinx_vdma_probe(struct platform_device *pdev)
+{
+ struct device_node *node = pdev->dev.of_node;
+ struct xilinx_vdma_device *xdev;
+ struct device_node *child;
+ struct resource *io;
+ u32 num_frames;
+ int i, err;
+
+ /* Allocate and initialize the DMA engine structure */
+ xdev = devm_kzalloc(&pdev->dev, sizeof(*xdev), GFP_KERNEL);
+ if (!xdev)
+ return -ENOMEM;
+
+ xdev->dev = &pdev->dev;
+
+ /* Request and map I/O memory */
+ io = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ xdev->regs = devm_ioremap_resource(&pdev->dev, io);
+ if (IS_ERR(xdev->regs))
+ return PTR_ERR(xdev->regs);
+
+ /* Retrieve the DMA engine properties from the device tree */
+ xdev->has_sg = of_property_read_bool(node, "xlnx,include-sg");
+
+ err = of_property_read_u32(node, "xlnx,num-fstores", &num_frames);
+ if (err < 0) {
+ dev_err(xdev->dev, "missing xlnx,num-fstores property\n");
+ return err;
+ }
+
+ err = of_property_read_u32(node, "xlnx,flush-fsync",
+ &xdev->flush_on_fsync);
+ if (err < 0)
+ dev_warn(xdev->dev, "missing xlnx,flush-fsync property\n");
+
+ /* Initialize the DMA engine */
+ xdev->common.dev = &pdev->dev;
+
+ INIT_LIST_HEAD(&xdev->common.channels);
+ dma_cap_set(DMA_SLAVE, xdev->common.cap_mask);
+ dma_cap_set(DMA_PRIVATE, xdev->common.cap_mask);
+
+ xdev->common.device_alloc_chan_resources =
+ xilinx_vdma_alloc_chan_resources;
+ xdev->common.device_free_chan_resources =
+ xilinx_vdma_free_chan_resources;
+ xdev->common.device_prep_interleaved_dma =
+ xilinx_vdma_dma_prep_interleaved;
+ xdev->common.device_control = xilinx_vdma_device_control;
+ xdev->common.device_tx_status = xilinx_vdma_tx_status;
+ xdev->common.device_issue_pending = xilinx_vdma_issue_pending;
+
+ platform_set_drvdata(pdev, xdev);
+
+ /* Initialize the channels */
+ for_each_child_of_node(node, child) {
+ err = xilinx_vdma_chan_probe(xdev, child);
+ if (err < 0)
+ goto error;
+ }
+
+ for (i = 0; i < XILINX_VDMA_MAX_CHANS_PER_DEVICE; i++)
+ if (xdev->chan[i])
+ xdev->chan[i]->num_frms = num_frames;
+
+ /* Register the DMA engine with the core */
+ dma_async_device_register(&xdev->common);
+
+ err = of_dma_controller_register(node, of_dma_xilinx_xlate,
+ xdev);
+ if (err < 0) {
+ dev_err(&pdev->dev, "Unable to register DMA to DT\n");
+ dma_async_device_unregister(&xdev->common);
+ goto error;
+ }
+
+ dev_info(&pdev->dev, "Xilinx AXI VDMA Engine Driver Probed!!\n");
+
+ return 0;
+
+error:
+ for (i = 0; i < XILINX_VDMA_MAX_CHANS_PER_DEVICE; i++)
+ if (xdev->chan[i])
+ xilinx_vdma_chan_remove(xdev->chan[i]);
+
+ return err;
+}
+
+/**
+ * xilinx_vdma_remove - Driver remove function
+ * @pdev: Pointer to the platform_device structure
+ *
+ * Return: Always '0'
+ */
+static int xilinx_vdma_remove(struct platform_device *pdev)
+{
+ struct xilinx_vdma_device *xdev = platform_get_drvdata(pdev);
+ int i;
+
+ of_dma_controller_free(pdev->dev.of_node);
+
+ dma_async_device_unregister(&xdev->common);
+
+ for (i = 0; i < XILINX_VDMA_MAX_CHANS_PER_DEVICE; i++)
+ if (xdev->chan[i])
+ xilinx_vdma_chan_remove(xdev->chan[i]);
+
+ return 0;
+}
+
+static const struct of_device_id xilinx_vdma_of_ids[] = {
+ { .compatible = "xlnx,axi-vdma-1.00.a",},
+ {}
+};
+
+static struct platform_driver xilinx_vdma_driver = {
+ .driver = {
+ .name = "xilinx-vdma",
+ .owner = THIS_MODULE,
+ .of_match_table = xilinx_vdma_of_ids,
+ },
+ .probe = xilinx_vdma_probe,
+ .remove = xilinx_vdma_remove,
+};
+
+module_platform_driver(xilinx_vdma_driver);
+
+MODULE_AUTHOR("Xilinx, Inc.");
+MODULE_DESCRIPTION("Xilinx VDMA driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/firmware/efi/Kconfig b/drivers/firmware/efi/Kconfig
index 1e75f48..d420ae2d 100644
--- a/drivers/firmware/efi/Kconfig
+++ b/drivers/firmware/efi/Kconfig
@@ -47,6 +47,13 @@
See also Documentation/ABI/testing/sysfs-firmware-efi-runtime-map.
+config EFI_PARAMS_FROM_FDT
+ bool
+ help
+ Select this config option from the architecture Kconfig if
+ the EFI runtime support gets system table address, memory
+ map address, and other parameters from the device tree.
+
endmenu
config UEFI_CPER
diff --git a/drivers/firmware/efi/arm-stub.c b/drivers/firmware/efi/arm-stub.c
new file mode 100644
index 0000000..41114ce
--- /dev/null
+++ b/drivers/firmware/efi/arm-stub.c
@@ -0,0 +1,278 @@
+/*
+ * EFI stub implementation that is shared by arm and arm64 architectures.
+ * This should be #included by the EFI stub implementation files.
+ *
+ * Copyright (C) 2013,2014 Linaro Limited
+ * Roy Franz <roy.franz@linaro.org
+ * Copyright (C) 2013 Red Hat, Inc.
+ * Mark Salter <msalter@redhat.com>
+ *
+ * This file is part of the Linux kernel, and is made available under the
+ * terms of the GNU General Public License version 2.
+ *
+ */
+
+static int __init efi_secureboot_enabled(efi_system_table_t *sys_table_arg)
+{
+ static efi_guid_t const var_guid __initconst = EFI_GLOBAL_VARIABLE_GUID;
+ static efi_char16_t const var_name[] __initconst = {
+ 'S', 'e', 'c', 'u', 'r', 'e', 'B', 'o', 'o', 't', 0 };
+
+ efi_get_variable_t *f_getvar = sys_table_arg->runtime->get_variable;
+ unsigned long size = sizeof(u8);
+ efi_status_t status;
+ u8 val;
+
+ status = f_getvar((efi_char16_t *)var_name, (efi_guid_t *)&var_guid,
+ NULL, &size, &val);
+
+ switch (status) {
+ case EFI_SUCCESS:
+ return val;
+ case EFI_NOT_FOUND:
+ return 0;
+ default:
+ return 1;
+ }
+}
+
+static efi_status_t efi_open_volume(efi_system_table_t *sys_table_arg,
+ void *__image, void **__fh)
+{
+ efi_file_io_interface_t *io;
+ efi_loaded_image_t *image = __image;
+ efi_file_handle_t *fh;
+ efi_guid_t fs_proto = EFI_FILE_SYSTEM_GUID;
+ efi_status_t status;
+ void *handle = (void *)(unsigned long)image->device_handle;
+
+ status = sys_table_arg->boottime->handle_protocol(handle,
+ &fs_proto, (void **)&io);
+ if (status != EFI_SUCCESS) {
+ efi_printk(sys_table_arg, "Failed to handle fs_proto\n");
+ return status;
+ }
+
+ status = io->open_volume(io, &fh);
+ if (status != EFI_SUCCESS)
+ efi_printk(sys_table_arg, "Failed to open volume\n");
+
+ *__fh = fh;
+ return status;
+}
+static efi_status_t efi_file_close(void *handle)
+{
+ efi_file_handle_t *fh = handle;
+
+ return fh->close(handle);
+}
+
+static efi_status_t
+efi_file_read(void *handle, unsigned long *size, void *addr)
+{
+ efi_file_handle_t *fh = handle;
+
+ return fh->read(handle, size, addr);
+}
+
+
+static efi_status_t
+efi_file_size(efi_system_table_t *sys_table_arg, void *__fh,
+ efi_char16_t *filename_16, void **handle, u64 *file_sz)
+{
+ efi_file_handle_t *h, *fh = __fh;
+ efi_file_info_t *info;
+ efi_status_t status;
+ efi_guid_t info_guid = EFI_FILE_INFO_ID;
+ unsigned long info_sz;
+
+ status = fh->open(fh, &h, filename_16, EFI_FILE_MODE_READ, (u64)0);
+ if (status != EFI_SUCCESS) {
+ efi_printk(sys_table_arg, "Failed to open file: ");
+ efi_char16_printk(sys_table_arg, filename_16);
+ efi_printk(sys_table_arg, "\n");
+ return status;
+ }
+
+ *handle = h;
+
+ info_sz = 0;
+ status = h->get_info(h, &info_guid, &info_sz, NULL);
+ if (status != EFI_BUFFER_TOO_SMALL) {
+ efi_printk(sys_table_arg, "Failed to get file info size\n");
+ return status;
+ }
+
+grow:
+ status = sys_table_arg->boottime->allocate_pool(EFI_LOADER_DATA,
+ info_sz, (void **)&info);
+ if (status != EFI_SUCCESS) {
+ efi_printk(sys_table_arg, "Failed to alloc mem for file info\n");
+ return status;
+ }
+
+ status = h->get_info(h, &info_guid, &info_sz,
+ info);
+ if (status == EFI_BUFFER_TOO_SMALL) {
+ sys_table_arg->boottime->free_pool(info);
+ goto grow;
+ }
+
+ *file_sz = info->file_size;
+ sys_table_arg->boottime->free_pool(info);
+
+ if (status != EFI_SUCCESS)
+ efi_printk(sys_table_arg, "Failed to get initrd info\n");
+
+ return status;
+}
+
+
+
+static void efi_char16_printk(efi_system_table_t *sys_table_arg,
+ efi_char16_t *str)
+{
+ struct efi_simple_text_output_protocol *out;
+
+ out = (struct efi_simple_text_output_protocol *)sys_table_arg->con_out;
+ out->output_string(out, str);
+}
+
+
+/*
+ * This function handles the architcture specific differences between arm and
+ * arm64 regarding where the kernel image must be loaded and any memory that
+ * must be reserved. On failure it is required to free all
+ * all allocations it has made.
+ */
+static efi_status_t handle_kernel_image(efi_system_table_t *sys_table,
+ unsigned long *image_addr,
+ unsigned long *image_size,
+ unsigned long *reserve_addr,
+ unsigned long *reserve_size,
+ unsigned long dram_base,
+ efi_loaded_image_t *image);
+/*
+ * EFI entry point for the arm/arm64 EFI stubs. This is the entrypoint
+ * that is described in the PE/COFF header. Most of the code is the same
+ * for both archictectures, with the arch-specific code provided in the
+ * handle_kernel_image() function.
+ */
+unsigned long __init efi_entry(void *handle, efi_system_table_t *sys_table,
+ unsigned long *image_addr)
+{
+ efi_loaded_image_t *image;
+ efi_status_t status;
+ unsigned long image_size = 0;
+ unsigned long dram_base;
+ /* addr/point and size pairs for memory management*/
+ unsigned long initrd_addr;
+ u64 initrd_size = 0;
+ unsigned long fdt_addr = 0; /* Original DTB */
+ u64 fdt_size = 0; /* We don't get size from configuration table */
+ char *cmdline_ptr = NULL;
+ int cmdline_size = 0;
+ unsigned long new_fdt_addr;
+ efi_guid_t loaded_image_proto = LOADED_IMAGE_PROTOCOL_GUID;
+ unsigned long reserve_addr = 0;
+ unsigned long reserve_size = 0;
+
+ /* Check if we were booted by the EFI firmware */
+ if (sys_table->hdr.signature != EFI_SYSTEM_TABLE_SIGNATURE)
+ goto fail;
+
+ pr_efi(sys_table, "Booting Linux Kernel...\n");
+
+ /*
+ * Get a handle to the loaded image protocol. This is used to get
+ * information about the running image, such as size and the command
+ * line.
+ */
+ status = sys_table->boottime->handle_protocol(handle,
+ &loaded_image_proto, (void *)&image);
+ if (status != EFI_SUCCESS) {
+ pr_efi_err(sys_table, "Failed to get loaded image protocol\n");
+ goto fail;
+ }
+
+ dram_base = get_dram_base(sys_table);
+ if (dram_base == EFI_ERROR) {
+ pr_efi_err(sys_table, "Failed to find DRAM base\n");
+ goto fail;
+ }
+ status = handle_kernel_image(sys_table, image_addr, &image_size,
+ &reserve_addr,
+ &reserve_size,
+ dram_base, image);
+ if (status != EFI_SUCCESS) {
+ pr_efi_err(sys_table, "Failed to relocate kernel\n");
+ goto fail;
+ }
+
+ /*
+ * Get the command line from EFI, using the LOADED_IMAGE
+ * protocol. We are going to copy the command line into the
+ * device tree, so this can be allocated anywhere.
+ */
+ cmdline_ptr = efi_convert_cmdline(sys_table, image, &cmdline_size);
+ if (!cmdline_ptr) {
+ pr_efi_err(sys_table, "getting command line via LOADED_IMAGE_PROTOCOL\n");
+ goto fail_free_image;
+ }
+
+ /*
+ * Unauthenticated device tree data is a security hazard, so
+ * ignore 'dtb=' unless UEFI Secure Boot is disabled.
+ */
+ if (efi_secureboot_enabled(sys_table)) {
+ pr_efi(sys_table, "UEFI Secure Boot is enabled.\n");
+ } else {
+ status = handle_cmdline_files(sys_table, image, cmdline_ptr,
+ "dtb=",
+ ~0UL, (unsigned long *)&fdt_addr,
+ (unsigned long *)&fdt_size);
+
+ if (status != EFI_SUCCESS) {
+ pr_efi_err(sys_table, "Failed to load device tree!\n");
+ goto fail_free_cmdline;
+ }
+ }
+ if (!fdt_addr)
+ /* Look for a device tree configuration table entry. */
+ fdt_addr = (uintptr_t)get_fdt(sys_table);
+
+ status = handle_cmdline_files(sys_table, image, cmdline_ptr,
+ "initrd=", dram_base + SZ_512M,
+ (unsigned long *)&initrd_addr,
+ (unsigned long *)&initrd_size);
+ if (status != EFI_SUCCESS)
+ pr_efi_err(sys_table, "Failed initrd from command line!\n");
+
+ new_fdt_addr = fdt_addr;
+ status = allocate_new_fdt_and_exit_boot(sys_table, handle,
+ &new_fdt_addr, dram_base + MAX_FDT_OFFSET,
+ initrd_addr, initrd_size, cmdline_ptr,
+ fdt_addr, fdt_size);
+
+ /*
+ * If all went well, we need to return the FDT address to the
+ * calling function so it can be passed to kernel as part of
+ * the kernel boot protocol.
+ */
+ if (status == EFI_SUCCESS)
+ return new_fdt_addr;
+
+ pr_efi_err(sys_table, "Failed to update FDT and exit boot services\n");
+
+ efi_free(sys_table, initrd_size, initrd_addr);
+ efi_free(sys_table, fdt_size, fdt_addr);
+
+fail_free_cmdline:
+ efi_free(sys_table, cmdline_size, (unsigned long)cmdline_ptr);
+
+fail_free_image:
+ efi_free(sys_table, image_size, *image_addr);
+ efi_free(sys_table, reserve_size, reserve_addr);
+fail:
+ return EFI_ERROR;
+}
diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index af20f17..cd36deb 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -20,6 +20,8 @@
#include <linux/init.h>
#include <linux/device.h>
#include <linux/efi.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
#include <linux/io.h>
struct efi __read_mostly efi = {
@@ -318,3 +320,80 @@
return 0;
}
+
+#ifdef CONFIG_EFI_PARAMS_FROM_FDT
+
+#define UEFI_PARAM(name, prop, field) \
+ { \
+ { name }, \
+ { prop }, \
+ offsetof(struct efi_fdt_params, field), \
+ FIELD_SIZEOF(struct efi_fdt_params, field) \
+ }
+
+static __initdata struct {
+ const char name[32];
+ const char propname[32];
+ int offset;
+ int size;
+} dt_params[] = {
+ UEFI_PARAM("System Table", "linux,uefi-system-table", system_table),
+ UEFI_PARAM("MemMap Address", "linux,uefi-mmap-start", mmap),
+ UEFI_PARAM("MemMap Size", "linux,uefi-mmap-size", mmap_size),
+ UEFI_PARAM("MemMap Desc. Size", "linux,uefi-mmap-desc-size", desc_size),
+ UEFI_PARAM("MemMap Desc. Version", "linux,uefi-mmap-desc-ver", desc_ver)
+};
+
+struct param_info {
+ int verbose;
+ void *params;
+};
+
+static int __init fdt_find_uefi_params(unsigned long node, const char *uname,
+ int depth, void *data)
+{
+ struct param_info *info = data;
+ void *prop, *dest;
+ unsigned long len;
+ u64 val;
+ int i;
+
+ if (depth != 1 ||
+ (strcmp(uname, "chosen") != 0 && strcmp(uname, "chosen@0") != 0))
+ return 0;
+
+ pr_info("Getting parameters from FDT:\n");
+
+ for (i = 0; i < ARRAY_SIZE(dt_params); i++) {
+ prop = of_get_flat_dt_prop(node, dt_params[i].propname, &len);
+ if (!prop) {
+ pr_err("Can't find %s in device tree!\n",
+ dt_params[i].name);
+ return 0;
+ }
+ dest = info->params + dt_params[i].offset;
+
+ val = of_read_number(prop, len / sizeof(u32));
+
+ if (dt_params[i].size == sizeof(u32))
+ *(u32 *)dest = val;
+ else
+ *(u64 *)dest = val;
+
+ if (info->verbose)
+ pr_info(" %s: 0x%0*llx\n", dt_params[i].name,
+ dt_params[i].size * 2, val);
+ }
+ return 1;
+}
+
+int __init efi_get_fdt_params(struct efi_fdt_params *params, int verbose)
+{
+ struct param_info info;
+
+ info.verbose = verbose;
+ info.params = params;
+
+ return of_scan_flat_dt(fdt_find_uefi_params, &info);
+}
+#endif /* CONFIG_EFI_PARAMS_FROM_FDT */
diff --git a/drivers/firmware/efi/fdt.c b/drivers/firmware/efi/fdt.c
new file mode 100644
index 0000000..5c6a8e8
--- /dev/null
+++ b/drivers/firmware/efi/fdt.c
@@ -0,0 +1,285 @@
+/*
+ * FDT related Helper functions used by the EFI stub on multiple
+ * architectures. This should be #included by the EFI stub
+ * implementation files.
+ *
+ * Copyright 2013 Linaro Limited; author Roy Franz
+ *
+ * This file is part of the Linux kernel, and is made available
+ * under the terms of the GNU General Public License version 2.
+ *
+ */
+
+static efi_status_t update_fdt(efi_system_table_t *sys_table, void *orig_fdt,
+ unsigned long orig_fdt_size,
+ void *fdt, int new_fdt_size, char *cmdline_ptr,
+ u64 initrd_addr, u64 initrd_size,
+ efi_memory_desc_t *memory_map,
+ unsigned long map_size, unsigned long desc_size,
+ u32 desc_ver)
+{
+ int node, prev;
+ int status;
+ u32 fdt_val32;
+ u64 fdt_val64;
+
+ /*
+ * Copy definition of linux_banner here. Since this code is
+ * built as part of the decompressor for ARM v7, pulling
+ * in version.c where linux_banner is defined for the
+ * kernel brings other kernel dependencies with it.
+ */
+ const char linux_banner[] =
+ "Linux version " UTS_RELEASE " (" LINUX_COMPILE_BY "@"
+ LINUX_COMPILE_HOST ") (" LINUX_COMPILER ") " UTS_VERSION "\n";
+
+ /* Do some checks on provided FDT, if it exists*/
+ if (orig_fdt) {
+ if (fdt_check_header(orig_fdt)) {
+ pr_efi_err(sys_table, "Device Tree header not valid!\n");
+ return EFI_LOAD_ERROR;
+ }
+ /*
+ * We don't get the size of the FDT if we get if from a
+ * configuration table.
+ */
+ if (orig_fdt_size && fdt_totalsize(orig_fdt) > orig_fdt_size) {
+ pr_efi_err(sys_table, "Truncated device tree! foo!\n");
+ return EFI_LOAD_ERROR;
+ }
+ }
+
+ if (orig_fdt)
+ status = fdt_open_into(orig_fdt, fdt, new_fdt_size);
+ else
+ status = fdt_create_empty_tree(fdt, new_fdt_size);
+
+ if (status != 0)
+ goto fdt_set_fail;
+
+ /*
+ * Delete any memory nodes present. We must delete nodes which
+ * early_init_dt_scan_memory may try to use.
+ */
+ prev = 0;
+ for (;;) {
+ const char *type, *name;
+ int len;
+
+ node = fdt_next_node(fdt, prev, NULL);
+ if (node < 0)
+ break;
+
+ type = fdt_getprop(fdt, node, "device_type", &len);
+ if (type && strncmp(type, "memory", len) == 0) {
+ fdt_del_node(fdt, node);
+ continue;
+ }
+
+ prev = node;
+ }
+
+ node = fdt_subnode_offset(fdt, 0, "chosen");
+ if (node < 0) {
+ node = fdt_add_subnode(fdt, 0, "chosen");
+ if (node < 0) {
+ status = node; /* node is error code when negative */
+ goto fdt_set_fail;
+ }
+ }
+
+ if ((cmdline_ptr != NULL) && (strlen(cmdline_ptr) > 0)) {
+ status = fdt_setprop(fdt, node, "bootargs", cmdline_ptr,
+ strlen(cmdline_ptr) + 1);
+ if (status)
+ goto fdt_set_fail;
+ }
+
+ /* Set initrd address/end in device tree, if present */
+ if (initrd_size != 0) {
+ u64 initrd_image_end;
+ u64 initrd_image_start = cpu_to_fdt64(initrd_addr);
+
+ status = fdt_setprop(fdt, node, "linux,initrd-start",
+ &initrd_image_start, sizeof(u64));
+ if (status)
+ goto fdt_set_fail;
+ initrd_image_end = cpu_to_fdt64(initrd_addr + initrd_size);
+ status = fdt_setprop(fdt, node, "linux,initrd-end",
+ &initrd_image_end, sizeof(u64));
+ if (status)
+ goto fdt_set_fail;
+ }
+
+ /* Add FDT entries for EFI runtime services in chosen node. */
+ node = fdt_subnode_offset(fdt, 0, "chosen");
+ fdt_val64 = cpu_to_fdt64((u64)(unsigned long)sys_table);
+ status = fdt_setprop(fdt, node, "linux,uefi-system-table",
+ &fdt_val64, sizeof(fdt_val64));
+ if (status)
+ goto fdt_set_fail;
+
+ fdt_val64 = cpu_to_fdt64((u64)(unsigned long)memory_map);
+ status = fdt_setprop(fdt, node, "linux,uefi-mmap-start",
+ &fdt_val64, sizeof(fdt_val64));
+ if (status)
+ goto fdt_set_fail;
+
+ fdt_val32 = cpu_to_fdt32(map_size);
+ status = fdt_setprop(fdt, node, "linux,uefi-mmap-size",
+ &fdt_val32, sizeof(fdt_val32));
+ if (status)
+ goto fdt_set_fail;
+
+ fdt_val32 = cpu_to_fdt32(desc_size);
+ status = fdt_setprop(fdt, node, "linux,uefi-mmap-desc-size",
+ &fdt_val32, sizeof(fdt_val32));
+ if (status)
+ goto fdt_set_fail;
+
+ fdt_val32 = cpu_to_fdt32(desc_ver);
+ status = fdt_setprop(fdt, node, "linux,uefi-mmap-desc-ver",
+ &fdt_val32, sizeof(fdt_val32));
+ if (status)
+ goto fdt_set_fail;
+
+ /*
+ * Add kernel version banner so stub/kernel match can be
+ * verified.
+ */
+ status = fdt_setprop_string(fdt, node, "linux,uefi-stub-kern-ver",
+ linux_banner);
+ if (status)
+ goto fdt_set_fail;
+
+ return EFI_SUCCESS;
+
+fdt_set_fail:
+ if (status == -FDT_ERR_NOSPACE)
+ return EFI_BUFFER_TOO_SMALL;
+
+ return EFI_LOAD_ERROR;
+}
+
+#ifndef EFI_FDT_ALIGN
+#define EFI_FDT_ALIGN EFI_PAGE_SIZE
+#endif
+
+/*
+ * Allocate memory for a new FDT, then add EFI, commandline, and
+ * initrd related fields to the FDT. This routine increases the
+ * FDT allocation size until the allocated memory is large
+ * enough. EFI allocations are in EFI_PAGE_SIZE granules,
+ * which are fixed at 4K bytes, so in most cases the first
+ * allocation should succeed.
+ * EFI boot services are exited at the end of this function.
+ * There must be no allocations between the get_memory_map()
+ * call and the exit_boot_services() call, so the exiting of
+ * boot services is very tightly tied to the creation of the FDT
+ * with the final memory map in it.
+ */
+
+efi_status_t allocate_new_fdt_and_exit_boot(efi_system_table_t *sys_table,
+ void *handle,
+ unsigned long *new_fdt_addr,
+ unsigned long max_addr,
+ u64 initrd_addr, u64 initrd_size,
+ char *cmdline_ptr,
+ unsigned long fdt_addr,
+ unsigned long fdt_size)
+{
+ unsigned long map_size, desc_size;
+ u32 desc_ver;
+ unsigned long mmap_key;
+ efi_memory_desc_t *memory_map;
+ unsigned long new_fdt_size;
+ efi_status_t status;
+
+ /*
+ * Estimate size of new FDT, and allocate memory for it. We
+ * will allocate a bigger buffer if this ends up being too
+ * small, so a rough guess is OK here.
+ */
+ new_fdt_size = fdt_size + EFI_PAGE_SIZE;
+ while (1) {
+ status = efi_high_alloc(sys_table, new_fdt_size, EFI_FDT_ALIGN,
+ new_fdt_addr, max_addr);
+ if (status != EFI_SUCCESS) {
+ pr_efi_err(sys_table, "Unable to allocate memory for new device tree.\n");
+ goto fail;
+ }
+
+ /*
+ * Now that we have done our final memory allocation (and free)
+ * we can get the memory map key needed for
+ * exit_boot_services().
+ */
+ status = efi_get_memory_map(sys_table, &memory_map, &map_size,
+ &desc_size, &desc_ver, &mmap_key);
+ if (status != EFI_SUCCESS)
+ goto fail_free_new_fdt;
+
+ status = update_fdt(sys_table,
+ (void *)fdt_addr, fdt_size,
+ (void *)*new_fdt_addr, new_fdt_size,
+ cmdline_ptr, initrd_addr, initrd_size,
+ memory_map, map_size, desc_size, desc_ver);
+
+ /* Succeeding the first time is the expected case. */
+ if (status == EFI_SUCCESS)
+ break;
+
+ if (status == EFI_BUFFER_TOO_SMALL) {
+ /*
+ * We need to allocate more space for the new
+ * device tree, so free existing buffer that is
+ * too small. Also free memory map, as we will need
+ * to get new one that reflects the free/alloc we do
+ * on the device tree buffer.
+ */
+ efi_free(sys_table, new_fdt_size, *new_fdt_addr);
+ sys_table->boottime->free_pool(memory_map);
+ new_fdt_size += EFI_PAGE_SIZE;
+ } else {
+ pr_efi_err(sys_table, "Unable to constuct new device tree.\n");
+ goto fail_free_mmap;
+ }
+ }
+
+ /* Now we are ready to exit_boot_services.*/
+ status = sys_table->boottime->exit_boot_services(handle, mmap_key);
+
+
+ if (status == EFI_SUCCESS)
+ return status;
+
+ pr_efi_err(sys_table, "Exit boot services failed.\n");
+
+fail_free_mmap:
+ sys_table->boottime->free_pool(memory_map);
+
+fail_free_new_fdt:
+ efi_free(sys_table, new_fdt_size, *new_fdt_addr);
+
+fail:
+ return EFI_LOAD_ERROR;
+}
+
+static void *get_fdt(efi_system_table_t *sys_table)
+{
+ efi_guid_t fdt_guid = DEVICE_TREE_GUID;
+ efi_config_table_t *tables;
+ void *fdt;
+ int i;
+
+ tables = (efi_config_table_t *) sys_table->tables;
+ fdt = NULL;
+
+ for (i = 0; i < sys_table->nr_tables; i++)
+ if (efi_guidcmp(tables[i].guid, fdt_guid) == 0) {
+ fdt = (void *) tables[i].table;
+ break;
+ }
+
+ return fdt;
+}
diff --git a/drivers/gpio/gpio-stmpe.c b/drivers/gpio/gpio-stmpe.c
index 2776a09..628b5849 100644
--- a/drivers/gpio/gpio-stmpe.c
+++ b/drivers/gpio/gpio-stmpe.c
@@ -23,7 +23,8 @@
enum { REG_RE, REG_FE, REG_IE };
#define CACHE_NR_REGS 3
-#define CACHE_NR_BANKS (STMPE_NR_GPIOS / 8)
+/* No variant has more than 24 GPIOs */
+#define CACHE_NR_BANKS (24 / 8)
struct stmpe_gpio {
struct gpio_chip chip;
@@ -31,8 +32,6 @@
struct device *dev;
struct mutex irq_lock;
struct irq_domain *domain;
-
- int irq_base;
unsigned norequest_mask;
/* Caches of interrupt control registers for bus_lock */
@@ -311,13 +310,8 @@
static int stmpe_gpio_irq_init(struct stmpe_gpio *stmpe_gpio,
struct device_node *np)
{
- int base = 0;
-
- if (!np)
- base = stmpe_gpio->irq_base;
-
stmpe_gpio->domain = irq_domain_add_simple(np,
- stmpe_gpio->chip.ngpio, base,
+ stmpe_gpio->chip.ngpio, 0,
&stmpe_gpio_irq_simple_ops, stmpe_gpio);
if (!stmpe_gpio->domain) {
dev_err(stmpe_gpio->dev, "failed to create irqdomain\n");
@@ -354,7 +348,7 @@
#ifdef CONFIG_OF
stmpe_gpio->chip.of_node = np;
#endif
- stmpe_gpio->chip.base = pdata ? pdata->gpio_base : -1;
+ stmpe_gpio->chip.base = -1;
if (pdata)
stmpe_gpio->norequest_mask = pdata->norequest_mask;
@@ -362,9 +356,7 @@
of_property_read_u32(np, "st,norequest-mask",
&stmpe_gpio->norequest_mask);
- if (irq >= 0)
- stmpe_gpio->irq_base = stmpe->irq_base + STMPE_INT_GPIO(0);
- else
+ if (irq < 0)
dev_info(&pdev->dev,
"device configured in no-irq mode; "
"irqs are not available\n");
diff --git a/drivers/gpu/drm/drm_crtc_helper.c b/drivers/gpu/drm/drm_crtc_helper.c
index df281b5..872ba11 100644
--- a/drivers/gpu/drm/drm_crtc_helper.c
+++ b/drivers/gpu/drm/drm_crtc_helper.c
@@ -29,6 +29,7 @@
* Jesse Barnes <jesse.barnes@intel.com>
*/
+#include <linux/kernel.h>
#include <linux/export.h>
#include <linux/moduleparam.h>
@@ -88,7 +89,13 @@
struct drm_connector *connector;
struct drm_device *dev = encoder->dev;
- WARN_ON(!mutex_is_locked(&dev->mode_config.mutex));
+ /*
+ * We can expect this mutex to be locked if we are not panicking.
+ * Locking is currently fubar in the panic handler.
+ */
+ if (!oops_in_progress)
+ WARN_ON(!mutex_is_locked(&dev->mode_config.mutex));
+
list_for_each_entry(connector, &dev->mode_config.connector_list, head)
if (connector->encoder == encoder)
return true;
@@ -112,7 +119,13 @@
struct drm_encoder *encoder;
struct drm_device *dev = crtc->dev;
- WARN_ON(!mutex_is_locked(&dev->mode_config.mutex));
+ /*
+ * We can expect this mutex to be locked if we are not panicking.
+ * Locking is currently fubar in the panic handler.
+ */
+ if (!oops_in_progress)
+ WARN_ON(!mutex_is_locked(&dev->mode_config.mutex));
+
list_for_each_entry(encoder, &dev->mode_config.encoder_list, head)
if (encoder->crtc == crtc && drm_helper_encoder_in_use(encoder))
return true;
diff --git a/drivers/gpu/drm/radeon/atombios_crtc.c b/drivers/gpu/drm/radeon/atombios_crtc.c
index c31c12b..e911898 100644
--- a/drivers/gpu/drm/radeon/atombios_crtc.c
+++ b/drivers/gpu/drm/radeon/atombios_crtc.c
@@ -270,8 +270,6 @@
switch (mode) {
case DRM_MODE_DPMS_ON:
radeon_crtc->enabled = true;
- /* adjust pm to dpms changes BEFORE enabling crtcs */
- radeon_pm_compute_clocks(rdev);
atombios_enable_crtc(crtc, ATOM_ENABLE);
if (ASIC_IS_DCE3(rdev) && !ASIC_IS_DCE6(rdev))
atombios_enable_crtc_memreq(crtc, ATOM_ENABLE);
@@ -289,10 +287,10 @@
atombios_enable_crtc_memreq(crtc, ATOM_DISABLE);
atombios_enable_crtc(crtc, ATOM_DISABLE);
radeon_crtc->enabled = false;
- /* adjust pm to dpms changes AFTER disabling crtcs */
- radeon_pm_compute_clocks(rdev);
break;
}
+ /* adjust pm to dpms */
+ radeon_pm_compute_clocks(rdev);
}
static void
diff --git a/drivers/gpu/drm/radeon/radeon_asic.c b/drivers/gpu/drm/radeon/radeon_asic.c
index be20e62..e5f0177 100644
--- a/drivers/gpu/drm/radeon/radeon_asic.c
+++ b/drivers/gpu/drm/radeon/radeon_asic.c
@@ -2049,8 +2049,8 @@
.blit_ring_index = RADEON_RING_TYPE_GFX_INDEX,
.dma = &cik_copy_dma,
.dma_ring_index = R600_RING_TYPE_DMA_INDEX,
- .copy = &cik_copy_dma,
- .copy_ring_index = R600_RING_TYPE_DMA_INDEX,
+ .copy = &cik_copy_cpdma,
+ .copy_ring_index = RADEON_RING_TYPE_GFX_INDEX,
},
.surface = {
.set_reg = r600_set_surface_reg,
diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
index 1467140..2cd144c 100644
--- a/drivers/gpu/drm/radeon/radeon_device.c
+++ b/drivers/gpu/drm/radeon/radeon_device.c
@@ -1558,6 +1558,10 @@
drm_kms_helper_poll_enable(dev);
+ /* set the power state here in case we are a PX system or headless */
+ if ((rdev->pm.pm_method == PM_METHOD_DPM) && rdev->pm.dpm_enabled)
+ radeon_pm_compute_clocks(rdev);
+
if (fbcon) {
radeon_fbdev_set_suspend(rdev, 0);
console_unlock();
diff --git a/drivers/gpu/drm/radeon/radeon_pm.c b/drivers/gpu/drm/radeon/radeon_pm.c
index 53d6e1b..2bdae61 100644
--- a/drivers/gpu/drm/radeon/radeon_pm.c
+++ b/drivers/gpu/drm/radeon/radeon_pm.c
@@ -1104,7 +1104,6 @@
if (ret)
goto dpm_resume_fail;
rdev->pm.dpm_enabled = true;
- radeon_pm_compute_clocks(rdev);
return;
dpm_resume_fail:
diff --git a/drivers/gpu/drm/radeon/radeon_vm.c b/drivers/gpu/drm/radeon/radeon_vm.c
index 1f42669..c11b71d 100644
--- a/drivers/gpu/drm/radeon/radeon_vm.c
+++ b/drivers/gpu/drm/radeon/radeon_vm.c
@@ -132,7 +132,7 @@
struct radeon_cs_reloc *list;
unsigned i, idx;
- list = kmalloc_array(vm->max_pde_used + 1,
+ list = kmalloc_array(vm->max_pde_used + 2,
sizeof(struct radeon_cs_reloc), GFP_KERNEL);
if (!list)
return NULL;
@@ -585,7 +585,8 @@
{
static const uint32_t incr = RADEON_VM_PTE_COUNT * 8;
- uint64_t pd_addr = radeon_bo_gpu_offset(vm->page_directory);
+ struct radeon_bo *pd = vm->page_directory;
+ uint64_t pd_addr = radeon_bo_gpu_offset(pd);
uint64_t last_pde = ~0, last_pt = ~0;
unsigned count = 0, pt_idx, ndw;
struct radeon_ib ib;
@@ -642,6 +643,7 @@
incr, R600_PTE_VALID);
if (ib.length_dw != 0) {
+ radeon_semaphore_sync_to(ib.semaphore, pd->tbo.sync_obj);
radeon_semaphore_sync_to(ib.semaphore, vm->last_id_use);
r = radeon_ib_schedule(rdev, &ib, NULL);
if (r) {
@@ -689,15 +691,18 @@
/* walk over the address space and update the page tables */
for (addr = start; addr < end; ) {
uint64_t pt_idx = addr >> RADEON_VM_BLOCK_SIZE;
+ struct radeon_bo *pt = vm->page_tables[pt_idx].bo;
unsigned nptes;
uint64_t pte;
+ radeon_semaphore_sync_to(ib->semaphore, pt->tbo.sync_obj);
+
if ((addr & ~mask) == (end & ~mask))
nptes = end - addr;
else
nptes = RADEON_VM_PTE_COUNT - (addr & mask);
- pte = radeon_bo_gpu_offset(vm->page_tables[pt_idx].bo);
+ pte = radeon_bo_gpu_offset(pt);
pte += (addr & mask) * 8;
if ((last_pte + 8 * count) != pte) {
diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig
index c94db1c..620d100 100644
--- a/drivers/i2c/busses/Kconfig
+++ b/drivers/i2c/busses/Kconfig
@@ -449,7 +449,7 @@
config I2C_EG20T
tristate "Intel EG20T PCH/LAPIS Semicon IOH(ML7213/ML7223/ML7831) I2C"
- depends on PCI
+ depends on PCI && (X86_32 || COMPILE_TEST)
help
This driver is for PCH(Platform controller Hub) I2C of EG20T which
is an IOH(Input/Output Hub) for x86 embedded processor.
@@ -570,13 +570,6 @@
I2C interface from ST-Ericsson's Nomadik and Ux500 architectures,
as well as the STA2X11 PCIe I/O HUB.
-config I2C_NUC900
- tristate "NUC900 I2C Driver"
- depends on ARCH_W90X900
- help
- Say Y here to include support for I2C controller in the
- Winbond/Nuvoton NUC900 based System-on-Chip devices.
-
config I2C_OCORES
tristate "OpenCores I2C Controller"
help
@@ -993,6 +986,15 @@
help
Supports the SiByte SOC on-chip I2C interfaces (2 channels).
+config I2C_CROS_EC_TUNNEL
+ tristate "ChromeOS EC tunnel I2C bus"
+ depends on MFD_CROS_EC
+ help
+ If you say yes here you get an I2C bus that will tunnel i2c commands
+ through to the other side of the ChromeOS EC to the i2c bus
+ connected there. This will work whatever the interface used to
+ talk to the EC (SPI, I2C or LPC).
+
config SCx200_I2C
tristate "NatSemi SCx200 I2C using GPIO pins (DEPRECATED)"
depends on SCx200_GPIO
diff --git a/drivers/i2c/busses/Makefile b/drivers/i2c/busses/Makefile
index 18d18ff..298692cc 100644
--- a/drivers/i2c/busses/Makefile
+++ b/drivers/i2c/busses/Makefile
@@ -55,7 +55,6 @@
obj-$(CONFIG_I2C_MV64XXX) += i2c-mv64xxx.o
obj-$(CONFIG_I2C_MXS) += i2c-mxs.o
obj-$(CONFIG_I2C_NOMADIK) += i2c-nomadik.o
-obj-$(CONFIG_I2C_NUC900) += i2c-nuc900.o
obj-$(CONFIG_I2C_OCORES) += i2c-ocores.o
obj-$(CONFIG_I2C_OMAP) += i2c-omap.o
obj-$(CONFIG_I2C_PASEMI) += i2c-pasemi.o
@@ -95,6 +94,7 @@
# Other I2C/SMBus bus drivers
obj-$(CONFIG_I2C_ACORN) += i2c-acorn.o
obj-$(CONFIG_I2C_BCM_KONA) += i2c-bcm-kona.o
+obj-$(CONFIG_I2C_CROS_EC_TUNNEL) += i2c-cros-ec-tunnel.o
obj-$(CONFIG_I2C_ELEKTOR) += i2c-elektor.o
obj-$(CONFIG_I2C_PCA_ISA) += i2c-pca-isa.o
obj-$(CONFIG_I2C_SIBYTE) += i2c-sibyte.o
diff --git a/drivers/i2c/busses/i2c-ali1563.c b/drivers/i2c/busses/i2c-ali1563.c
index 98a1c977..15517d7 100644
--- a/drivers/i2c/busses/i2c-ali1563.c
+++ b/drivers/i2c/busses/i2c-ali1563.c
@@ -63,7 +63,7 @@
static struct pci_driver ali1563_pci_driver;
static unsigned short ali1563_smba;
-static int ali1563_transaction(struct i2c_adapter * a, int size)
+static int ali1563_transaction(struct i2c_adapter *a, int size)
{
u32 data;
int timeout;
@@ -78,7 +78,7 @@
data = inb_p(SMB_HST_STS);
if (data & HST_STS_BAD) {
dev_err(&a->dev, "ali1563: Trying to reset busy device\n");
- outb_p(data | HST_STS_BAD,SMB_HST_STS);
+ outb_p(data | HST_STS_BAD, SMB_HST_STS);
data = inb_p(SMB_HST_STS);
if (data & HST_STS_BAD)
return -EBUSY;
@@ -102,10 +102,10 @@
if (!timeout) {
dev_err(&a->dev, "Timeout - Trying to KILL transaction!\n");
/* Issue 'kill' to host controller */
- outb_p(HST_CNTL2_KILL,SMB_HST_CNTL2);
+ outb_p(HST_CNTL2_KILL, SMB_HST_CNTL2);
data = inb_p(SMB_HST_STS);
status = -ETIMEDOUT;
- }
+ }
/* device error - no response, ignore the autodetection case */
if (data & HST_STS_DEVERR) {
@@ -117,18 +117,18 @@
if (data & HST_STS_BUSERR) {
dev_err(&a->dev, "Bus collision!\n");
/* Issue timeout, hoping it helps */
- outb_p(HST_CNTL1_TIMEOUT,SMB_HST_CNTL1);
+ outb_p(HST_CNTL1_TIMEOUT, SMB_HST_CNTL1);
}
if (data & HST_STS_FAIL) {
dev_err(&a->dev, "Cleaning fail after KILL!\n");
- outb_p(0x0,SMB_HST_CNTL2);
+ outb_p(0x0, SMB_HST_CNTL2);
}
return status;
}
-static int ali1563_block_start(struct i2c_adapter * a)
+static int ali1563_block_start(struct i2c_adapter *a)
{
u32 data;
int timeout;
@@ -142,8 +142,8 @@
data = inb_p(SMB_HST_STS);
if (data & HST_STS_BAD) {
- dev_warn(&a->dev,"ali1563: Trying to reset busy device\n");
- outb_p(data | HST_STS_BAD,SMB_HST_STS);
+ dev_warn(&a->dev, "ali1563: Trying to reset busy device\n");
+ outb_p(data | HST_STS_BAD, SMB_HST_STS);
data = inb_p(SMB_HST_STS);
if (data & HST_STS_BAD)
return -EBUSY;
@@ -184,13 +184,14 @@
return status;
}
-static int ali1563_block(struct i2c_adapter * a, union i2c_smbus_data * data, u8 rw)
+static int ali1563_block(struct i2c_adapter *a,
+ union i2c_smbus_data *data, u8 rw)
{
int i, len;
int error = 0;
/* Do we need this? */
- outb_p(HST_CNTL1_LAST,SMB_HST_CNTL1);
+ outb_p(HST_CNTL1_LAST, SMB_HST_CNTL1);
if (rw == I2C_SMBUS_WRITE) {
len = data->block[0];
@@ -198,8 +199,8 @@
len = 1;
else if (len > 32)
len = 32;
- outb_p(len,SMB_HST_DAT0);
- outb_p(data->block[1],SMB_BLK_DAT);
+ outb_p(len, SMB_HST_DAT0);
+ outb_p(data->block[1], SMB_BLK_DAT);
} else
len = 32;
@@ -208,10 +209,12 @@
for (i = 0; i < len; i++) {
if (rw == I2C_SMBUS_WRITE) {
outb_p(data->block[i + 1], SMB_BLK_DAT);
- if ((error = ali1563_block_start(a)))
+ error = ali1563_block_start(a);
+ if (error)
break;
} else {
- if ((error = ali1563_block_start(a)))
+ error = ali1563_block_start(a);
+ if (error)
break;
if (i == 0) {
len = inb_p(SMB_HST_DAT0);
@@ -224,25 +227,26 @@
}
}
/* Do we need this? */
- outb_p(HST_CNTL1_LAST,SMB_HST_CNTL1);
+ outb_p(HST_CNTL1_LAST, SMB_HST_CNTL1);
return error;
}
-static s32 ali1563_access(struct i2c_adapter * a, u16 addr,
+static s32 ali1563_access(struct i2c_adapter *a, u16 addr,
unsigned short flags, char rw, u8 cmd,
- int size, union i2c_smbus_data * data)
+ int size, union i2c_smbus_data *data)
{
int error = 0;
int timeout;
u32 reg;
for (timeout = ALI1563_MAX_TIMEOUT; timeout; timeout--) {
- if (!(reg = inb_p(SMB_HST_STS) & HST_STS_BUSY))
+ reg = inb_p(SMB_HST_STS);
+ if (!(reg & HST_STS_BUSY))
break;
}
if (!timeout)
- dev_warn(&a->dev,"SMBus not idle. HST_STS = %02x\n",reg);
- outb_p(0xff,SMB_HST_STS);
+ dev_warn(&a->dev, "SMBus not idle. HST_STS = %02x\n", reg);
+ outb_p(0xff, SMB_HST_STS);
/* Map the size to what the chip understands */
switch (size) {
@@ -268,13 +272,14 @@
}
outb_p(((addr & 0x7f) << 1) | (rw & 0x01), SMB_HST_ADD);
- outb_p((inb_p(SMB_HST_CNTL2) & ~HST_CNTL2_SIZEMASK) | (size << 3), SMB_HST_CNTL2);
+ outb_p((inb_p(SMB_HST_CNTL2) & ~HST_CNTL2_SIZEMASK) |
+ (size << 3), SMB_HST_CNTL2);
/* Write the command register */
- switch(size) {
+ switch (size) {
case HST_CNTL2_BYTE:
- if (rw== I2C_SMBUS_WRITE)
+ if (rw == I2C_SMBUS_WRITE)
/* Beware it uses DAT0 register and not CMD! */
outb_p(cmd, SMB_HST_DAT0);
break;
@@ -292,11 +297,12 @@
break;
case HST_CNTL2_BLOCK:
outb_p(cmd, SMB_HST_CMD);
- error = ali1563_block(a,data,rw);
+ error = ali1563_block(a, data, rw);
goto Done;
}
- if ((error = ali1563_transaction(a, size)))
+ error = ali1563_transaction(a, size);
+ if (error)
goto Done;
if ((rw == I2C_SMBUS_WRITE) || (size == HST_CNTL2_QUICK))
@@ -317,7 +323,7 @@
return error;
}
-static u32 ali1563_func(struct i2c_adapter * a)
+static u32 ali1563_func(struct i2c_adapter *a)
{
return I2C_FUNC_SMBUS_QUICK | I2C_FUNC_SMBUS_BYTE |
I2C_FUNC_SMBUS_BYTE_DATA | I2C_FUNC_SMBUS_WORD_DATA |
@@ -329,13 +335,13 @@
{
u16 ctrl;
- pci_read_config_word(dev,ALI1563_SMBBA,&ctrl);
+ pci_read_config_word(dev, ALI1563_SMBBA, &ctrl);
/* SMB I/O Base in high 12 bits and must be aligned with the
* size of the I/O space. */
ali1563_smba = ctrl & ~(ALI1563_SMB_IOSIZE - 1);
if (!ali1563_smba) {
- dev_warn(&dev->dev,"ali1563_smba Uninitialized\n");
+ dev_warn(&dev->dev, "ali1563_smba Uninitialized\n");
goto Err;
}
@@ -350,8 +356,8 @@
ctrl | ALI1563_SMB_IOEN);
pci_read_config_word(dev, ALI1563_SMBBA, &ctrl);
if (!(ctrl & ALI1563_SMB_IOEN)) {
- dev_err(&dev->dev, "I/O space still not enabled, "
- "giving up\n");
+ dev_err(&dev->dev,
+ "I/O space still not enabled, giving up\n");
goto Err;
}
}
@@ -375,7 +381,7 @@
static void ali1563_shutdown(struct pci_dev *dev)
{
- release_region(ali1563_smba,ALI1563_SMB_IOSIZE);
+ release_region(ali1563_smba, ALI1563_SMB_IOSIZE);
}
static const struct i2c_algorithm ali1563_algorithm = {
@@ -394,12 +400,14 @@
{
int error;
- if ((error = ali1563_setup(dev)))
+ error = ali1563_setup(dev);
+ if (error)
goto exit;
ali1563_adapter.dev.parent = &dev->dev;
snprintf(ali1563_adapter.name, sizeof(ali1563_adapter.name),
"SMBus ALi 1563 Adapter @ %04x", ali1563_smba);
- if ((error = i2c_add_adapter(&ali1563_adapter)))
+ error = i2c_add_adapter(&ali1563_adapter);
+ if (error)
goto exit_shutdown;
return 0;
@@ -421,12 +429,12 @@
{},
};
-MODULE_DEVICE_TABLE (pci, ali1563_id_table);
+MODULE_DEVICE_TABLE(pci, ali1563_id_table);
static struct pci_driver ali1563_pci_driver = {
- .name = "ali1563_smbus",
+ .name = "ali1563_smbus",
.id_table = ali1563_id_table,
- .probe = ali1563_probe,
+ .probe = ali1563_probe,
.remove = ali1563_remove,
};
diff --git a/drivers/i2c/busses/i2c-bcm2835.c b/drivers/i2c/busses/i2c-bcm2835.c
index c607195..214ff97 100644
--- a/drivers/i2c/busses/i2c-bcm2835.c
+++ b/drivers/i2c/busses/i2c-bcm2835.c
@@ -225,10 +225,8 @@
struct i2c_adapter *adap;
i2c_dev = devm_kzalloc(&pdev->dev, sizeof(*i2c_dev), GFP_KERNEL);
- if (!i2c_dev) {
- dev_err(&pdev->dev, "Cannot allocate i2c_dev\n");
+ if (!i2c_dev)
return -ENOMEM;
- }
platform_set_drvdata(pdev, i2c_dev);
i2c_dev->dev = &pdev->dev;
init_completion(&i2c_dev->completion);
diff --git a/drivers/i2c/busses/i2c-bfin-twi.c b/drivers/i2c/busses/i2c-bfin-twi.c
index e6d5162..3e271e7 100644
--- a/drivers/i2c/busses/i2c-bfin-twi.c
+++ b/drivers/i2c/busses/i2c-bfin-twi.c
@@ -620,35 +620,27 @@
int rc;
unsigned int clkhilow;
- iface = kzalloc(sizeof(struct bfin_twi_iface), GFP_KERNEL);
+ iface = devm_kzalloc(&pdev->dev, sizeof(struct bfin_twi_iface),
+ GFP_KERNEL);
if (!iface) {
dev_err(&pdev->dev, "Cannot allocate memory\n");
- rc = -ENOMEM;
- goto out_error_nomem;
+ return -ENOMEM;
}
spin_lock_init(&(iface->lock));
/* Find and map our resources */
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (res == NULL) {
- dev_err(&pdev->dev, "Cannot get IORESOURCE_MEM\n");
- rc = -ENOENT;
- goto out_error_get_res;
- }
-
- iface->regs_base = ioremap(res->start, resource_size(res));
- if (iface->regs_base == NULL) {
+ iface->regs_base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(iface->regs_base)) {
dev_err(&pdev->dev, "Cannot map IO\n");
- rc = -ENXIO;
- goto out_error_ioremap;
+ return PTR_ERR(iface->regs_base);
}
iface->irq = platform_get_irq(pdev, 0);
if (iface->irq < 0) {
dev_err(&pdev->dev, "No IRQ specified\n");
- rc = -ENOENT;
- goto out_error_no_irq;
+ return -ENOENT;
}
p_adap = &iface->adap;
@@ -666,15 +658,15 @@
"i2c-bfin-twi");
if (rc) {
dev_err(&pdev->dev, "Can't setup pin mux!\n");
- goto out_error_pin_mux;
+ return -EBUSY;
}
- rc = request_irq(iface->irq, bfin_twi_interrupt_entry,
+ rc = devm_request_irq(&pdev->dev, iface->irq, bfin_twi_interrupt_entry,
0, pdev->name, iface);
if (rc) {
dev_err(&pdev->dev, "Can't get IRQ %d !\n", iface->irq);
rc = -ENODEV;
- goto out_error_req_irq;
+ goto out_error;
}
/* Set TWI internal clock as 10MHz */
@@ -695,7 +687,7 @@
rc = i2c_add_numbered_adapter(p_adap);
if (rc < 0) {
dev_err(&pdev->dev, "Can't add i2c adapter!\n");
- goto out_error_add_adapter;
+ goto out_error;
}
platform_set_drvdata(pdev, iface);
@@ -705,17 +697,8 @@
return 0;
-out_error_add_adapter:
- free_irq(iface->irq, iface);
-out_error_req_irq:
-out_error_no_irq:
+out_error:
peripheral_free_list(dev_get_platdata(&pdev->dev));
-out_error_pin_mux:
- iounmap(iface->regs_base);
-out_error_ioremap:
-out_error_get_res:
- kfree(iface);
-out_error_nomem:
return rc;
}
@@ -724,10 +707,7 @@
struct bfin_twi_iface *iface = platform_get_drvdata(pdev);
i2c_del_adapter(&(iface->adap));
- free_irq(iface->irq, iface);
peripheral_free_list(dev_get_platdata(&pdev->dev));
- iounmap(iface->regs_base);
- kfree(iface);
return 0;
}
diff --git a/drivers/i2c/busses/i2c-cros-ec-tunnel.c b/drivers/i2c/busses/i2c-cros-ec-tunnel.c
new file mode 100644
index 0000000..8e7a714
--- /dev/null
+++ b/drivers/i2c/busses/i2c-cros-ec-tunnel.c
@@ -0,0 +1,318 @@
+/*
+ * Copyright (C) 2013 Google, Inc
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * Expose an I2C passthrough to the ChromeOS EC.
+ */
+
+#include <linux/module.h>
+#include <linux/i2c.h>
+#include <linux/mfd/cros_ec.h>
+#include <linux/mfd/cros_ec_commands.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+
+/**
+ * struct ec_i2c_device - Driver data for I2C tunnel
+ *
+ * @dev: Device node
+ * @adap: I2C adapter
+ * @ec: Pointer to EC device
+ * @remote_bus: The EC bus number we tunnel to on the other side.
+ * @request_buf: Buffer for transmitting data; we expect most transfers to fit.
+ * @response_buf: Buffer for receiving data; we expect most transfers to fit.
+ */
+
+struct ec_i2c_device {
+ struct device *dev;
+ struct i2c_adapter adap;
+ struct cros_ec_device *ec;
+
+ u16 remote_bus;
+
+ u8 request_buf[256];
+ u8 response_buf[256];
+};
+
+/**
+ * ec_i2c_count_message - Count bytes needed for ec_i2c_construct_message
+ *
+ * @i2c_msgs: The i2c messages to read
+ * @num: The number of i2c messages.
+ *
+ * Returns the number of bytes the messages will take up.
+ */
+static int ec_i2c_count_message(const struct i2c_msg i2c_msgs[], int num)
+{
+ int i;
+ int size;
+
+ size = sizeof(struct ec_params_i2c_passthru);
+ size += num * sizeof(struct ec_params_i2c_passthru_msg);
+ for (i = 0; i < num; i++)
+ if (!(i2c_msgs[i].flags & I2C_M_RD))
+ size += i2c_msgs[i].len;
+
+ return size;
+}
+
+/**
+ * ec_i2c_construct_message - construct a message to go to the EC
+ *
+ * This function effectively stuffs the standard i2c_msg format of Linux into
+ * a format that the EC understands.
+ *
+ * @buf: The buffer to fill. We assume that the buffer is big enough.
+ * @i2c_msgs: The i2c messages to read.
+ * @num: The number of i2c messages.
+ * @bus_num: The remote bus number we want to talk to.
+ *
+ * Returns 0 or a negative error number.
+ */
+static int ec_i2c_construct_message(u8 *buf, const struct i2c_msg i2c_msgs[],
+ int num, u16 bus_num)
+{
+ struct ec_params_i2c_passthru *params;
+ u8 *out_data;
+ int i;
+
+ out_data = buf + sizeof(struct ec_params_i2c_passthru) +
+ num * sizeof(struct ec_params_i2c_passthru_msg);
+
+ params = (struct ec_params_i2c_passthru *)buf;
+ params->port = bus_num;
+ params->num_msgs = num;
+ for (i = 0; i < num; i++) {
+ const struct i2c_msg *i2c_msg = &i2c_msgs[i];
+ struct ec_params_i2c_passthru_msg *msg = ¶ms->msg[i];
+
+ msg->len = i2c_msg->len;
+ msg->addr_flags = i2c_msg->addr;
+
+ if (i2c_msg->flags & I2C_M_TEN)
+ msg->addr_flags |= EC_I2C_FLAG_10BIT;
+
+ if (i2c_msg->flags & I2C_M_RD) {
+ msg->addr_flags |= EC_I2C_FLAG_READ;
+ } else {
+ memcpy(out_data, i2c_msg->buf, msg->len);
+ out_data += msg->len;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * ec_i2c_count_response - Count bytes needed for ec_i2c_parse_response
+ *
+ * @i2c_msgs: The i2c messages to to fill up.
+ * @num: The number of i2c messages expected.
+ *
+ * Returns the number of response bytes expeced.
+ */
+static int ec_i2c_count_response(struct i2c_msg i2c_msgs[], int num)
+{
+ int size;
+ int i;
+
+ size = sizeof(struct ec_response_i2c_passthru);
+ for (i = 0; i < num; i++)
+ if (i2c_msgs[i].flags & I2C_M_RD)
+ size += i2c_msgs[i].len;
+
+ return size;
+}
+
+/**
+ * ec_i2c_parse_response - Parse a response from the EC
+ *
+ * We'll take the EC's response and copy it back into msgs.
+ *
+ * @buf: The buffer to parse.
+ * @i2c_msgs: The i2c messages to to fill up.
+ * @num: The number of i2c messages; will be modified to include the actual
+ * number received.
+ *
+ * Returns 0 or a negative error number.
+ */
+static int ec_i2c_parse_response(const u8 *buf, struct i2c_msg i2c_msgs[],
+ int *num)
+{
+ const struct ec_response_i2c_passthru *resp;
+ const u8 *in_data;
+ int i;
+
+ in_data = buf + sizeof(struct ec_response_i2c_passthru);
+
+ resp = (const struct ec_response_i2c_passthru *)buf;
+ if (resp->i2c_status & EC_I2C_STATUS_TIMEOUT)
+ return -ETIMEDOUT;
+ else if (resp->i2c_status & EC_I2C_STATUS_ERROR)
+ return -EREMOTEIO;
+
+ /* Other side could send us back fewer messages, but not more */
+ if (resp->num_msgs > *num)
+ return -EPROTO;
+ *num = resp->num_msgs;
+
+ for (i = 0; i < *num; i++) {
+ struct i2c_msg *i2c_msg = &i2c_msgs[i];
+
+ if (i2c_msgs[i].flags & I2C_M_RD) {
+ memcpy(i2c_msg->buf, in_data, i2c_msg->len);
+ in_data += i2c_msg->len;
+ }
+ }
+
+ return 0;
+}
+
+static int ec_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg i2c_msgs[],
+ int num)
+{
+ struct ec_i2c_device *bus = adap->algo_data;
+ struct device *dev = bus->dev;
+ const u16 bus_num = bus->remote_bus;
+ int request_len;
+ int response_len;
+ u8 *request = NULL;
+ u8 *response = NULL;
+ int result;
+
+ request_len = ec_i2c_count_message(i2c_msgs, num);
+ if (request_len < 0) {
+ dev_warn(dev, "Error constructing message %d\n", request_len);
+ result = request_len;
+ goto exit;
+ }
+ response_len = ec_i2c_count_response(i2c_msgs, num);
+ if (response_len < 0) {
+ /* Unexpected; no errors should come when NULL response */
+ dev_warn(dev, "Error preparing response %d\n", response_len);
+ result = response_len;
+ goto exit;
+ }
+
+ if (request_len <= ARRAY_SIZE(bus->request_buf)) {
+ request = bus->request_buf;
+ } else {
+ request = kzalloc(request_len, GFP_KERNEL);
+ if (request == NULL) {
+ result = -ENOMEM;
+ goto exit;
+ }
+ }
+ if (response_len <= ARRAY_SIZE(bus->response_buf)) {
+ response = bus->response_buf;
+ } else {
+ response = kzalloc(response_len, GFP_KERNEL);
+ if (response == NULL) {
+ result = -ENOMEM;
+ goto exit;
+ }
+ }
+
+ ec_i2c_construct_message(request, i2c_msgs, num, bus_num);
+ result = bus->ec->command_sendrecv(bus->ec, EC_CMD_I2C_PASSTHRU,
+ request, request_len,
+ response, response_len);
+ if (result)
+ goto exit;
+
+ result = ec_i2c_parse_response(response, i2c_msgs, &num);
+ if (result < 0)
+ goto exit;
+
+ /* Indicate success by saying how many messages were sent */
+ result = num;
+exit:
+ if (request != bus->request_buf)
+ kfree(request);
+ if (response != bus->response_buf)
+ kfree(response);
+
+ return result;
+}
+
+static u32 ec_i2c_functionality(struct i2c_adapter *adap)
+{
+ return I2C_FUNC_I2C | I2C_FUNC_SMBUS_EMUL;
+}
+
+static const struct i2c_algorithm ec_i2c_algorithm = {
+ .master_xfer = ec_i2c_xfer,
+ .functionality = ec_i2c_functionality,
+};
+
+static int ec_i2c_probe(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct cros_ec_device *ec = dev_get_drvdata(pdev->dev.parent);
+ struct device *dev = &pdev->dev;
+ struct ec_i2c_device *bus = NULL;
+ u32 remote_bus;
+ int err;
+
+ if (!ec->command_sendrecv) {
+ dev_err(dev, "Missing sendrecv\n");
+ return -EINVAL;
+ }
+
+ bus = devm_kzalloc(dev, sizeof(*bus), GFP_KERNEL);
+ if (bus == NULL)
+ return -ENOMEM;
+
+ err = of_property_read_u32(np, "google,remote-bus", &remote_bus);
+ if (err) {
+ dev_err(dev, "Couldn't read remote-bus property\n");
+ return err;
+ }
+ bus->remote_bus = remote_bus;
+
+ bus->ec = ec;
+ bus->dev = dev;
+
+ bus->adap.owner = THIS_MODULE;
+ strlcpy(bus->adap.name, "cros-ec-i2c-tunnel", sizeof(bus->adap.name));
+ bus->adap.algo = &ec_i2c_algorithm;
+ bus->adap.algo_data = bus;
+ bus->adap.dev.parent = &pdev->dev;
+ bus->adap.dev.of_node = np;
+
+ err = i2c_add_adapter(&bus->adap);
+ if (err) {
+ dev_err(dev, "cannot register i2c adapter\n");
+ return err;
+ }
+ platform_set_drvdata(pdev, bus);
+
+ return err;
+}
+
+static int ec_i2c_remove(struct platform_device *dev)
+{
+ struct ec_i2c_device *bus = platform_get_drvdata(dev);
+
+ i2c_del_adapter(&bus->adap);
+
+ return 0;
+}
+
+static struct platform_driver ec_i2c_tunnel_driver = {
+ .probe = ec_i2c_probe,
+ .remove = ec_i2c_remove,
+ .driver = {
+ .name = "cros-ec-i2c-tunnel",
+ },
+};
+
+module_platform_driver(ec_i2c_tunnel_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("EC I2C tunnel driver");
+MODULE_ALIAS("platform:cros-ec-i2c-tunnel");
diff --git a/drivers/i2c/busses/i2c-designware-pcidrv.c b/drivers/i2c/busses/i2c-designware-pcidrv.c
index 85056c2..3356f7a 100644
--- a/drivers/i2c/busses/i2c-designware-pcidrv.c
+++ b/drivers/i2c/busses/i2c-designware-pcidrv.c
@@ -56,6 +56,7 @@
medfield_5,
baytrail,
+ haswell,
};
struct dw_scl_sda_cfg {
@@ -95,6 +96,15 @@
.sda_hold = 0x6,
};
+/* Haswell HCNT/LCNT/SDA hold time */
+static struct dw_scl_sda_cfg hsw_config = {
+ .ss_hcnt = 0x01b0,
+ .fs_hcnt = 0x48,
+ .ss_lcnt = 0x01fb,
+ .fs_lcnt = 0xa0,
+ .sda_hold = 0x9,
+};
+
static struct dw_pci_controller dw_pci_controllers[] = {
[moorestown_0] = {
.bus_num = 0,
@@ -168,6 +178,15 @@
.functionality = I2C_FUNC_10BIT_ADDR,
.scl_sda_cfg = &byt_config,
},
+ [haswell] = {
+ .bus_num = -1,
+ .bus_cfg = INTEL_MID_STD_CFG | DW_IC_CON_SPEED_FAST,
+ .tx_fifo_depth = 32,
+ .rx_fifo_depth = 32,
+ .clk_khz = 100000,
+ .functionality = I2C_FUNC_10BIT_ADDR,
+ .scl_sda_cfg = &hsw_config,
+ },
};
static struct i2c_algorithm i2c_dw_algo = {
.master_xfer = i2c_dw_xfer,
@@ -328,6 +347,9 @@
{ PCI_VDEVICE(INTEL, 0x0F45), baytrail },
{ PCI_VDEVICE(INTEL, 0x0F46), baytrail },
{ PCI_VDEVICE(INTEL, 0x0F47), baytrail },
+ /* Haswell */
+ { PCI_VDEVICE(INTEL, 0x9c61), haswell },
+ { PCI_VDEVICE(INTEL, 0x9c62), haswell },
{ 0,}
};
MODULE_DEVICE_TABLE(pci, i2_designware_pci_ids);
diff --git a/drivers/i2c/busses/i2c-designware-platdrv.c b/drivers/i2c/busses/i2c-designware-platdrv.c
index 9c7802614..402ec39 100644
--- a/drivers/i2c/busses/i2c-designware-platdrv.c
+++ b/drivers/i2c/busses/i2c-designware-platdrv.c
@@ -247,12 +247,13 @@
MODULE_DEVICE_TABLE(of, dw_i2c_of_match);
#endif
-#ifdef CONFIG_PM_SLEEP
+#ifdef CONFIG_PM
static int dw_i2c_suspend(struct device *dev)
{
struct platform_device *pdev = to_platform_device(dev);
struct dw_i2c_dev *i_dev = platform_get_drvdata(pdev);
+ i2c_dw_disable(i_dev);
clk_disable_unprepare(i_dev->clk);
return 0;
@@ -268,13 +269,11 @@
return 0;
}
-
-static SIMPLE_DEV_PM_OPS(dw_i2c_dev_pm_ops, dw_i2c_suspend, dw_i2c_resume);
-#define DW_I2C_DEV_PM_OPS (&dw_i2c_dev_pm_ops)
-#else
-#define DW_I2C_DEV_PM_OPS NULL
#endif
+static UNIVERSAL_DEV_PM_OPS(dw_i2c_dev_pm_ops, dw_i2c_suspend,
+ dw_i2c_resume, NULL);
+
/* work with hotplug and coldplug */
MODULE_ALIAS("platform:i2c_designware");
@@ -286,7 +285,7 @@
.owner = THIS_MODULE,
.of_match_table = of_match_ptr(dw_i2c_of_match),
.acpi_match_table = ACPI_PTR(dw_i2c_acpi_match),
- .pm = DW_I2C_DEV_PM_OPS,
+ .pm = &dw_i2c_dev_pm_ops,
},
};
diff --git a/drivers/i2c/busses/i2c-diolan-u2c.c b/drivers/i2c/busses/i2c-diolan-u2c.c
index 721f7eb..b19a310 100644
--- a/drivers/i2c/busses/i2c-diolan-u2c.c
+++ b/drivers/i2c/busses/i2c-diolan-u2c.c
@@ -455,7 +455,6 @@
/* allocate memory for our device state and initialize it */
dev = kzalloc(sizeof(*dev), GFP_KERNEL);
if (dev == NULL) {
- dev_err(&interface->dev, "no memory for device state\n");
ret = -ENOMEM;
goto error;
}
diff --git a/drivers/i2c/busses/i2c-efm32.c b/drivers/i2c/busses/i2c-efm32.c
index 777ed40..f7eccd6 100644
--- a/drivers/i2c/busses/i2c-efm32.c
+++ b/drivers/i2c/busses/i2c-efm32.c
@@ -320,10 +320,8 @@
return -EINVAL;
ddata = devm_kzalloc(&pdev->dev, sizeof(*ddata), GFP_KERNEL);
- if (!ddata) {
- dev_dbg(&pdev->dev, "failed to allocate private data\n");
+ if (!ddata)
return -ENOMEM;
- }
platform_set_drvdata(pdev, ddata);
init_completion(&ddata->done);
diff --git a/drivers/i2c/busses/i2c-eg20t.c b/drivers/i2c/busses/i2c-eg20t.c
index ff775ac..a44ea13 100644
--- a/drivers/i2c/busses/i2c-eg20t.c
+++ b/drivers/i2c/busses/i2c-eg20t.c
@@ -751,10 +751,8 @@
pch_pci_dbg(pdev, "Entered.\n");
adap_info = kzalloc((sizeof(struct adapter_info)), GFP_KERNEL);
- if (adap_info == NULL) {
- pch_pci_err(pdev, "Memory allocation FAILED\n");
+ if (adap_info == NULL)
return -ENOMEM;
- }
ret = pci_enable_device(pdev);
if (ret) {
diff --git a/drivers/i2c/busses/i2c-exynos5.c b/drivers/i2c/busses/i2c-exynos5.c
index 00af0a0..63d2292 100644
--- a/drivers/i2c/busses/i2c-exynos5.c
+++ b/drivers/i2c/busses/i2c-exynos5.c
@@ -76,12 +76,6 @@
#define HSI2C_RXFIFO_TRIGGER_LEVEL(x) ((x) << 4)
#define HSI2C_TXFIFO_TRIGGER_LEVEL(x) ((x) << 16)
-/* As per user manual FIFO max depth is 64bytes */
-#define HSI2C_FIFO_MAX 0x40
-/* default trigger levels for Tx and Rx FIFOs */
-#define HSI2C_DEF_TXFIFO_LVL (HSI2C_FIFO_MAX - 0x30)
-#define HSI2C_DEF_RXFIFO_LVL (HSI2C_FIFO_MAX - 0x10)
-
/* I2C_TRAILING_CTL Register bits */
#define HSI2C_TRAILING_COUNT (0xf)
@@ -183,14 +177,54 @@
* 2. Fast speed upto 1Mbps
*/
int speed_mode;
+
+ /* Version of HS-I2C Hardware */
+ struct exynos_hsi2c_variant *variant;
+};
+
+/**
+ * struct exynos_hsi2c_variant - platform specific HSI2C driver data
+ * @fifo_depth: the fifo depth supported by the HSI2C module
+ *
+ * Specifies platform specific configuration of HSI2C module.
+ * Note: A structure for driver specific platform data is used for future
+ * expansion of its usage.
+ */
+struct exynos_hsi2c_variant {
+ unsigned int fifo_depth;
+};
+
+static const struct exynos_hsi2c_variant exynos5250_hsi2c_data = {
+ .fifo_depth = 64,
+};
+
+static const struct exynos_hsi2c_variant exynos5260_hsi2c_data = {
+ .fifo_depth = 16,
};
static const struct of_device_id exynos5_i2c_match[] = {
- { .compatible = "samsung,exynos5-hsi2c" },
- {},
+ {
+ .compatible = "samsung,exynos5-hsi2c",
+ .data = &exynos5250_hsi2c_data
+ }, {
+ .compatible = "samsung,exynos5250-hsi2c",
+ .data = &exynos5250_hsi2c_data
+ }, {
+ .compatible = "samsung,exynos5260-hsi2c",
+ .data = &exynos5260_hsi2c_data
+ }, {},
};
MODULE_DEVICE_TABLE(of, exynos5_i2c_match);
+static inline struct exynos_hsi2c_variant *exynos5_i2c_get_variant
+ (struct platform_device *pdev)
+{
+ const struct of_device_id *match;
+
+ match = of_match_node(exynos5_i2c_match, pdev->dev.of_node);
+ return (struct exynos_hsi2c_variant *)match->data;
+}
+
static void exynos5_i2c_clr_pend_irq(struct exynos5_i2c *i2c)
{
writel(readl(i2c->regs + HSI2C_INT_STATUS),
@@ -415,7 +449,7 @@
fifo_status = readl(i2c->regs + HSI2C_FIFO_STATUS);
fifo_level = HSI2C_TX_FIFO_LVL(fifo_status);
- len = HSI2C_FIFO_MAX - fifo_level;
+ len = i2c->variant->fifo_depth - fifo_level;
if (len > (i2c->msg->len - i2c->msg_ptr))
len = i2c->msg->len - i2c->msg_ptr;
@@ -483,6 +517,7 @@
u32 i2c_auto_conf = 0;
u32 fifo_ctl;
unsigned long flags;
+ unsigned short trig_lvl;
i2c_ctl = readl(i2c->regs + HSI2C_CTL);
i2c_ctl &= ~(HSI2C_TXCHON | HSI2C_RXCHON);
@@ -493,13 +528,19 @@
i2c_auto_conf = HSI2C_READ_WRITE;
- fifo_ctl |= HSI2C_RXFIFO_TRIGGER_LEVEL(HSI2C_DEF_TXFIFO_LVL);
+ trig_lvl = (i2c->msg->len > i2c->variant->fifo_depth) ?
+ (i2c->variant->fifo_depth * 3 / 4) : i2c->msg->len;
+ fifo_ctl |= HSI2C_RXFIFO_TRIGGER_LEVEL(trig_lvl);
+
int_en |= (HSI2C_INT_RX_ALMOSTFULL_EN |
HSI2C_INT_TRAILING_EN);
} else {
i2c_ctl |= HSI2C_TXCHON;
- fifo_ctl |= HSI2C_TXFIFO_TRIGGER_LEVEL(HSI2C_DEF_RXFIFO_LVL);
+ trig_lvl = (i2c->msg->len > i2c->variant->fifo_depth) ?
+ (i2c->variant->fifo_depth * 1 / 4) : i2c->msg->len;
+ fifo_ctl |= HSI2C_TXFIFO_TRIGGER_LEVEL(trig_lvl);
+
int_en |= HSI2C_INT_TX_ALMOSTEMPTY_EN;
}
@@ -621,10 +662,8 @@
int ret;
i2c = devm_kzalloc(&pdev->dev, sizeof(struct exynos5_i2c), GFP_KERNEL);
- if (!i2c) {
- dev_err(&pdev->dev, "no memory for state\n");
+ if (!i2c)
return -ENOMEM;
- }
if (of_property_read_u32(np, "clock-frequency", &op_clock)) {
i2c->speed_mode = HSI2C_FAST_SPD;
@@ -691,7 +730,9 @@
if (ret)
goto err_clk;
- exynos5_i2c_init(i2c);
+ i2c->variant = exynos5_i2c_get_variant(pdev);
+
+ exynos5_i2c_reset(i2c);
ret = i2c_add_adapter(&i2c->adap);
if (ret < 0) {
diff --git a/drivers/i2c/busses/i2c-gpio.c b/drivers/i2c/busses/i2c-gpio.c
index 02d2d4a..71a45b2 100644
--- a/drivers/i2c/busses/i2c-gpio.c
+++ b/drivers/i2c/busses/i2c-gpio.c
@@ -147,24 +147,22 @@
scl_pin = pdata->scl_pin;
}
- ret = gpio_request(sda_pin, "sda");
+ ret = devm_gpio_request(&pdev->dev, sda_pin, "sda");
if (ret) {
if (ret == -EINVAL)
ret = -EPROBE_DEFER; /* Try again later */
- goto err_request_sda;
+ return ret;
}
- ret = gpio_request(scl_pin, "scl");
+ ret = devm_gpio_request(&pdev->dev, scl_pin, "scl");
if (ret) {
if (ret == -EINVAL)
ret = -EPROBE_DEFER; /* Try again later */
- goto err_request_scl;
+ return ret;
}
priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
- if (!priv) {
- ret = -ENOMEM;
- goto err_add_bus;
- }
+ if (!priv)
+ return -ENOMEM;
adap = &priv->adap;
bit_data = &priv->bit_data;
pdata = &priv->pdata;
@@ -225,7 +223,7 @@
adap->nr = pdev->id;
ret = i2c_bit_add_numbered_bus(adap);
if (ret)
- goto err_add_bus;
+ return ret;
platform_set_drvdata(pdev, priv);
@@ -235,13 +233,6 @@
? ", no clock stretching" : "");
return 0;
-
-err_add_bus:
- gpio_free(scl_pin);
-err_request_scl:
- gpio_free(sda_pin);
-err_request_sda:
- return ret;
}
static int i2c_gpio_remove(struct platform_device *pdev)
@@ -255,8 +246,6 @@
pdata = &priv->pdata;
i2c_del_adapter(adap);
- gpio_free(pdata->scl_pin);
- gpio_free(pdata->sda_pin);
return 0;
}
diff --git a/drivers/i2c/busses/i2c-imx.c b/drivers/i2c/busses/i2c-imx.c
index db895fb..aa8bc14 100644
--- a/drivers/i2c/busses/i2c-imx.c
+++ b/drivers/i2c/busses/i2c-imx.c
@@ -183,6 +183,8 @@
unsigned int disable_delay;
int stopped;
unsigned int ifdr; /* IMX_I2C_IFDR */
+ unsigned int cur_clk;
+ unsigned int bitrate;
const struct imx_i2c_hwdata *hwdata;
};
@@ -305,6 +307,48 @@
return 0;
}
+static void i2c_imx_set_clk(struct imx_i2c_struct *i2c_imx)
+{
+ struct imx_i2c_clk_pair *i2c_clk_div = i2c_imx->hwdata->clk_div;
+ unsigned int i2c_clk_rate;
+ unsigned int div;
+ int i;
+
+ /* Divider value calculation */
+ i2c_clk_rate = clk_get_rate(i2c_imx->clk);
+ if (i2c_imx->cur_clk == i2c_clk_rate)
+ return;
+ else
+ i2c_imx->cur_clk = i2c_clk_rate;
+
+ div = (i2c_clk_rate + i2c_imx->bitrate - 1) / i2c_imx->bitrate;
+ if (div < i2c_clk_div[0].div)
+ i = 0;
+ else if (div > i2c_clk_div[i2c_imx->hwdata->ndivs - 1].div)
+ i = i2c_imx->hwdata->ndivs - 1;
+ else
+ for (i = 0; i2c_clk_div[i].div < div; i++);
+
+ /* Store divider value */
+ i2c_imx->ifdr = i2c_clk_div[i].val;
+
+ /*
+ * There dummy delay is calculated.
+ * It should be about one I2C clock period long.
+ * This delay is used in I2C bus disable function
+ * to fix chip hardware bug.
+ */
+ i2c_imx->disable_delay = (500000U * i2c_clk_div[i].div
+ + (i2c_clk_rate / 2) - 1) / (i2c_clk_rate / 2);
+
+#ifdef CONFIG_I2C_DEBUG_BUS
+ dev_dbg(&i2c_imx->adapter.dev, "I2C_CLK=%d, REQ DIV=%d\n",
+ i2c_clk_rate, div);
+ dev_dbg(&i2c_imx->adapter.dev, "IFDR[IC]=0x%x, REAL DIV=%d\n",
+ i2c_clk_div[i].val, i2c_clk_div[i].div);
+#endif
+}
+
static int i2c_imx_start(struct imx_i2c_struct *i2c_imx)
{
unsigned int temp = 0;
@@ -312,6 +356,8 @@
dev_dbg(&i2c_imx->adapter.dev, "<%s>\n", __func__);
+ i2c_imx_set_clk(i2c_imx);
+
result = clk_prepare_enable(i2c_imx->clk);
if (result)
return result;
@@ -367,45 +413,6 @@
clk_disable_unprepare(i2c_imx->clk);
}
-static void i2c_imx_set_clk(struct imx_i2c_struct *i2c_imx,
- unsigned int rate)
-{
- struct imx_i2c_clk_pair *i2c_clk_div = i2c_imx->hwdata->clk_div;
- unsigned int i2c_clk_rate;
- unsigned int div;
- int i;
-
- /* Divider value calculation */
- i2c_clk_rate = clk_get_rate(i2c_imx->clk);
- div = (i2c_clk_rate + rate - 1) / rate;
- if (div < i2c_clk_div[0].div)
- i = 0;
- else if (div > i2c_clk_div[i2c_imx->hwdata->ndivs - 1].div)
- i = i2c_imx->hwdata->ndivs - 1;
- else
- for (i = 0; i2c_clk_div[i].div < div; i++);
-
- /* Store divider value */
- i2c_imx->ifdr = i2c_clk_div[i].val;
-
- /*
- * There dummy delay is calculated.
- * It should be about one I2C clock period long.
- * This delay is used in I2C bus disable function
- * to fix chip hardware bug.
- */
- i2c_imx->disable_delay = (500000U * i2c_clk_div[i].div
- + (i2c_clk_rate / 2) - 1) / (i2c_clk_rate / 2);
-
- /* dev_dbg() can't be used, because adapter is not yet registered */
-#ifdef CONFIG_I2C_DEBUG_BUS
- dev_dbg(&i2c_imx->adapter.dev, "<%s> I2C_CLK=%d, REQ DIV=%d\n",
- __func__, i2c_clk_rate, div);
- dev_dbg(&i2c_imx->adapter.dev, "<%s> IFDR[IC]=0x%x, REAL DIV=%d\n",
- __func__, i2c_clk_div[i].val, i2c_clk_div[i].div);
-#endif
-}
-
static irqreturn_t i2c_imx_isr(int irq, void *dev_id)
{
struct imx_i2c_struct *i2c_imx = dev_id;
@@ -458,10 +465,11 @@
return 0;
}
-static int i2c_imx_read(struct imx_i2c_struct *i2c_imx, struct i2c_msg *msgs)
+static int i2c_imx_read(struct imx_i2c_struct *i2c_imx, struct i2c_msg *msgs, bool is_lastmsg)
{
int i, result;
unsigned int temp;
+ int block_data = msgs->flags & I2C_M_RECV_LEN;
dev_dbg(&i2c_imx->adapter.dev,
"<%s> write slave address: addr=0x%x\n",
@@ -481,7 +489,12 @@
/* setup bus to read data */
temp = imx_i2c_read_reg(i2c_imx, IMX_I2C_I2CR);
temp &= ~I2CR_MTX;
- if (msgs->len - 1)
+
+ /*
+ * Reset the I2CR_TXAK flag initially for SMBus block read since the
+ * length is unknown
+ */
+ if ((msgs->len - 1) || block_data)
temp &= ~I2CR_TXAK;
imx_i2c_write_reg(temp, i2c_imx, IMX_I2C_I2CR);
imx_i2c_read_reg(i2c_imx, IMX_I2C_I2DR); /* dummy read */
@@ -490,19 +503,49 @@
/* read data */
for (i = 0; i < msgs->len; i++) {
+ u8 len = 0;
result = i2c_imx_trx_complete(i2c_imx);
if (result)
return result;
- if (i == (msgs->len - 1)) {
- /* It must generate STOP before read I2DR to prevent
- controller from generating another clock cycle */
+ /*
+ * First byte is the length of remaining packet
+ * in the SMBus block data read. Add it to
+ * msgs->len.
+ */
+ if ((!i) && block_data) {
+ len = imx_i2c_read_reg(i2c_imx, IMX_I2C_I2DR);
+ if ((len == 0) || (len > I2C_SMBUS_BLOCK_MAX))
+ return -EPROTO;
dev_dbg(&i2c_imx->adapter.dev,
- "<%s> clear MSTA\n", __func__);
- temp = imx_i2c_read_reg(i2c_imx, IMX_I2C_I2CR);
- temp &= ~(I2CR_MSTA | I2CR_MTX);
- imx_i2c_write_reg(temp, i2c_imx, IMX_I2C_I2CR);
- i2c_imx_bus_busy(i2c_imx, 0);
- i2c_imx->stopped = 1;
+ "<%s> read length: 0x%X\n",
+ __func__, len);
+ msgs->len += len;
+ }
+ if (i == (msgs->len - 1)) {
+ if (is_lastmsg) {
+ /*
+ * It must generate STOP before read I2DR to prevent
+ * controller from generating another clock cycle
+ */
+ dev_dbg(&i2c_imx->adapter.dev,
+ "<%s> clear MSTA\n", __func__);
+ temp = imx_i2c_read_reg(i2c_imx, IMX_I2C_I2CR);
+ temp &= ~(I2CR_MSTA | I2CR_MTX);
+ imx_i2c_write_reg(temp, i2c_imx, IMX_I2C_I2CR);
+ i2c_imx_bus_busy(i2c_imx, 0);
+ i2c_imx->stopped = 1;
+ } else {
+ /*
+ * For i2c master receiver repeat restart operation like:
+ * read -> repeat MSTA -> read/write
+ * The controller must set MTX before read the last byte in
+ * the first read operation, otherwise the first read cost
+ * one extra clock cycle.
+ */
+ temp = readb(i2c_imx->base + IMX_I2C_I2CR);
+ temp |= I2CR_MTX;
+ writeb(temp, i2c_imx->base + IMX_I2C_I2CR);
+ }
} else if (i == (msgs->len - 2)) {
dev_dbg(&i2c_imx->adapter.dev,
"<%s> set TXAK\n", __func__);
@@ -510,7 +553,10 @@
temp |= I2CR_TXAK;
imx_i2c_write_reg(temp, i2c_imx, IMX_I2C_I2CR);
}
- msgs->buf[i] = imx_i2c_read_reg(i2c_imx, IMX_I2C_I2DR);
+ if ((!i) && block_data)
+ msgs->buf[0] = len;
+ else
+ msgs->buf[i] = imx_i2c_read_reg(i2c_imx, IMX_I2C_I2DR);
dev_dbg(&i2c_imx->adapter.dev,
"<%s> read byte: B%d=0x%X\n",
__func__, i, msgs->buf[i]);
@@ -523,6 +569,7 @@
{
unsigned int i, temp;
int result;
+ bool is_lastmsg = false;
struct imx_i2c_struct *i2c_imx = i2c_get_adapdata(adapter);
dev_dbg(&i2c_imx->adapter.dev, "<%s>\n", __func__);
@@ -534,6 +581,9 @@
/* read/write data */
for (i = 0; i < num; i++) {
+ if (i == num - 1)
+ is_lastmsg = true;
+
if (i) {
dev_dbg(&i2c_imx->adapter.dev,
"<%s> repeated start\n", __func__);
@@ -564,7 +614,7 @@
(temp & I2SR_RXAK ? 1 : 0));
#endif
if (msgs[i].flags & I2C_M_RD)
- result = i2c_imx_read(i2c_imx, &msgs[i]);
+ result = i2c_imx_read(i2c_imx, &msgs[i], is_lastmsg);
else
result = i2c_imx_write(i2c_imx, &msgs[i]);
if (result)
@@ -583,7 +633,8 @@
static u32 i2c_imx_func(struct i2c_adapter *adapter)
{
- return I2C_FUNC_I2C | I2C_FUNC_SMBUS_EMUL;
+ return I2C_FUNC_I2C | I2C_FUNC_SMBUS_EMUL
+ | I2C_FUNC_SMBUS_READ_BLOCK_DATA;
}
static struct i2c_algorithm i2c_imx_algo = {
@@ -600,7 +651,6 @@
struct imxi2c_platform_data *pdata = dev_get_platdata(&pdev->dev);
void __iomem *base;
int irq, ret;
- u32 bitrate;
dev_dbg(&pdev->dev, "<%s>\n", __func__);
@@ -617,10 +667,8 @@
i2c_imx = devm_kzalloc(&pdev->dev, sizeof(struct imx_i2c_struct),
GFP_KERNEL);
- if (!i2c_imx) {
- dev_err(&pdev->dev, "can't allocate interface\n");
+ if (!i2c_imx)
return -ENOMEM;
- }
if (of_id)
i2c_imx->hwdata = of_id->data;
@@ -664,12 +712,11 @@
i2c_set_adapdata(&i2c_imx->adapter, i2c_imx);
/* Set up clock divider */
- bitrate = IMX_I2C_BIT_RATE;
+ i2c_imx->bitrate = IMX_I2C_BIT_RATE;
ret = of_property_read_u32(pdev->dev.of_node,
- "clock-frequency", &bitrate);
+ "clock-frequency", &i2c_imx->bitrate);
if (ret < 0 && pdata && pdata->bitrate)
- bitrate = pdata->bitrate;
- i2c_imx_set_clk(i2c_imx, bitrate);
+ i2c_imx->bitrate = pdata->bitrate;
/* Set up chip registers to defaults */
imx_i2c_write_reg(i2c_imx->hwdata->i2cr_ien_opcode ^ I2CR_IEN,
diff --git a/drivers/i2c/busses/i2c-mpc.c b/drivers/i2c/busses/i2c-mpc.c
index f539163..6a32aa0 100644
--- a/drivers/i2c/busses/i2c-mpc.c
+++ b/drivers/i2c/busses/i2c-mpc.c
@@ -115,7 +115,7 @@
for (k = 9; k; k--) {
writeccr(i2c, 0);
writeccr(i2c, CCR_MSTA | CCR_MTX | CCR_MEN);
- udelay(delay_val);
+ readb(i2c->base + MPC_I2C_DR);
writeccr(i2c, CCR_MEN);
udelay(delay_val << 1);
}
diff --git a/drivers/i2c/busses/i2c-mv64xxx.c b/drivers/i2c/busses/i2c-mv64xxx.c
index 540ea69..9f4b775 100644
--- a/drivers/i2c/busses/i2c-mv64xxx.c
+++ b/drivers/i2c/busses/i2c-mv64xxx.c
@@ -681,7 +681,7 @@
*****************************************************************************
*/
static const struct of_device_id mv64xxx_i2c_of_match_table[] = {
- { .compatible = "allwinner,sun4i-i2c", .data = &mv64xxx_i2c_regs_sun4i},
+ { .compatible = "allwinner,sun4i-a10-i2c", .data = &mv64xxx_i2c_regs_sun4i},
{ .compatible = "allwinner,sun6i-a31-i2c", .data = &mv64xxx_i2c_regs_sun4i},
{ .compatible = "marvell,mv64xxx-i2c", .data = &mv64xxx_i2c_regs_mv64xxx},
{ .compatible = "marvell,mv78230-i2c", .data = &mv64xxx_i2c_regs_mv64xxx},
diff --git a/drivers/i2c/busses/i2c-nomadik.c b/drivers/i2c/busses/i2c-nomadik.c
index 32c85e9..0e55d85 100644
--- a/drivers/i2c/busses/i2c-nomadik.c
+++ b/drivers/i2c/busses/i2c-nomadik.c
@@ -879,19 +879,19 @@
#ifdef CONFIG_PM_SLEEP
static int nmk_i2c_suspend_late(struct device *dev)
{
- pinctrl_pm_select_sleep_state(dev);
+ int ret;
+ ret = pm_runtime_force_suspend(dev);
+ if (ret)
+ return ret;
+
+ pinctrl_pm_select_sleep_state(dev);
return 0;
}
static int nmk_i2c_resume_early(struct device *dev)
{
- /* First go to the default state */
- pinctrl_pm_select_default_state(dev);
- /* Then let's idle the pins until the next transfer happens */
- pinctrl_pm_select_idle_state(dev);
-
- return 0;
+ return pm_runtime_force_resume(dev);
}
#endif
diff --git a/drivers/i2c/busses/i2c-nuc900.c b/drivers/i2c/busses/i2c-nuc900.c
deleted file mode 100644
index 36394d7..0000000
--- a/drivers/i2c/busses/i2c-nuc900.c
+++ /dev/null
@@ -1,709 +0,0 @@
-/*
- * linux/drivers/i2c/busses/i2c-nuc900.c
- *
- * Copyright (c) 2010 Nuvoton technology corporation.
- *
- * This driver based on S3C2410 I2C driver of Ben Dooks <ben-Y5A6D6n0/KfQXOPxS62xeg@public.gmane.org>.
- * Written by Wan ZongShun <mcuos.com-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation;version 2 of the License.
- *
- */
-
-#include <linux/kernel.h>
-#include <linux/module.h>
-
-#include <linux/i2c.h>
-#include <linux/init.h>
-#include <linux/time.h>
-#include <linux/interrupt.h>
-#include <linux/delay.h>
-#include <linux/errno.h>
-#include <linux/err.h>
-#include <linux/platform_device.h>
-#include <linux/clk.h>
-#include <linux/cpufreq.h>
-#include <linux/slab.h>
-#include <linux/io.h>
-
-#include <mach/mfp.h>
-#include <linux/platform_data/i2c-nuc900.h>
-
-/* nuc900 i2c registers offset */
-
-#define CSR 0x00
-#define DIVIDER 0x04
-#define CMDR 0x08
-#define SWR 0x0C
-#define RXR 0x10
-#define TXR 0x14
-
-/* nuc900 i2c CSR register bits */
-
-#define IRQEN 0x003
-#define I2CBUSY 0x400
-#define I2CSTART 0x018
-#define IRQFLAG 0x004
-#define ARBIT_LOST 0x200
-#define SLAVE_ACK 0x800
-
-/* nuc900 i2c CMDR register bits */
-
-#define I2C_CMD_START 0x10
-#define I2C_CMD_STOP 0x08
-#define I2C_CMD_READ 0x04
-#define I2C_CMD_WRITE 0x02
-#define I2C_CMD_NACK 0x01
-
-/* i2c controller state */
-
-enum nuc900_i2c_state {
- STATE_IDLE,
- STATE_START,
- STATE_READ,
- STATE_WRITE,
- STATE_STOP
-};
-
-/* i2c controller private data */
-
-struct nuc900_i2c {
- spinlock_t lock;
- wait_queue_head_t wait;
-
- struct i2c_msg *msg;
- unsigned int msg_num;
- unsigned int msg_idx;
- unsigned int msg_ptr;
- unsigned int irq;
-
- enum nuc900_i2c_state state;
-
- void __iomem *regs;
- struct clk *clk;
- struct device *dev;
- struct resource *ioarea;
- struct i2c_adapter adap;
-};
-
-/* nuc900_i2c_master_complete
- *
- * complete the message and wake up the caller, using the given return code,
- * or zero to mean ok.
-*/
-
-static inline void nuc900_i2c_master_complete(struct nuc900_i2c *i2c, int ret)
-{
- dev_dbg(i2c->dev, "master_complete %d\n", ret);
-
- i2c->msg_ptr = 0;
- i2c->msg = NULL;
- i2c->msg_idx++;
- i2c->msg_num = 0;
- if (ret)
- i2c->msg_idx = ret;
-
- wake_up(&i2c->wait);
-}
-
-/* irq enable/disable functions */
-
-static inline void nuc900_i2c_disable_irq(struct nuc900_i2c *i2c)
-{
- unsigned long tmp;
-
- tmp = readl(i2c->regs + CSR);
- writel(tmp & ~IRQEN, i2c->regs + CSR);
-}
-
-static inline void nuc900_i2c_enable_irq(struct nuc900_i2c *i2c)
-{
- unsigned long tmp;
-
- tmp = readl(i2c->regs + CSR);
- writel(tmp | IRQEN, i2c->regs + CSR);
-}
-
-
-/* nuc900_i2c_message_start
- *
- * put the start of a message onto the bus
-*/
-
-static void nuc900_i2c_message_start(struct nuc900_i2c *i2c,
- struct i2c_msg *msg)
-{
- unsigned int addr = (msg->addr & 0x7f) << 1;
-
- if (msg->flags & I2C_M_RD)
- addr |= 0x1;
- writel(addr & 0xff, i2c->regs + TXR);
- writel(I2C_CMD_START | I2C_CMD_WRITE, i2c->regs + CMDR);
-}
-
-static inline void nuc900_i2c_stop(struct nuc900_i2c *i2c, int ret)
-{
-
- dev_dbg(i2c->dev, "STOP\n");
-
- /* stop the transfer */
- i2c->state = STATE_STOP;
- writel(I2C_CMD_STOP, i2c->regs + CMDR);
-
- nuc900_i2c_master_complete(i2c, ret);
- nuc900_i2c_disable_irq(i2c);
-}
-
-/* helper functions to determine the current state in the set of
- * messages we are sending
-*/
-
-/* is_lastmsg()
- *
- * returns TRUE if the current message is the last in the set
-*/
-
-static inline int is_lastmsg(struct nuc900_i2c *i2c)
-{
- return i2c->msg_idx >= (i2c->msg_num - 1);
-}
-
-/* is_msglast
- *
- * returns TRUE if we this is the last byte in the current message
-*/
-
-static inline int is_msglast(struct nuc900_i2c *i2c)
-{
- return i2c->msg_ptr == i2c->msg->len-1;
-}
-
-/* is_msgend
- *
- * returns TRUE if we reached the end of the current message
-*/
-
-static inline int is_msgend(struct nuc900_i2c *i2c)
-{
- return i2c->msg_ptr >= i2c->msg->len;
-}
-
-/* i2c_nuc900_irq_nextbyte
- *
- * process an interrupt and work out what to do
- */
-
-static void i2c_nuc900_irq_nextbyte(struct nuc900_i2c *i2c,
- unsigned long iicstat)
-{
- unsigned char byte;
-
- switch (i2c->state) {
-
- case STATE_IDLE:
- dev_err(i2c->dev, "%s: called in STATE_IDLE\n", __func__);
- break;
-
- case STATE_STOP:
- dev_err(i2c->dev, "%s: called in STATE_STOP\n", __func__);
- nuc900_i2c_disable_irq(i2c);
- break;
-
- case STATE_START:
- /* last thing we did was send a start condition on the
- * bus, or started a new i2c message
- */
-
- if (iicstat & SLAVE_ACK &&
- !(i2c->msg->flags & I2C_M_IGNORE_NAK)) {
- /* ack was not received... */
-
- dev_dbg(i2c->dev, "ack was not received\n");
- nuc900_i2c_stop(i2c, -ENXIO);
- break;
- }
-
- if (i2c->msg->flags & I2C_M_RD)
- i2c->state = STATE_READ;
- else
- i2c->state = STATE_WRITE;
-
- /* terminate the transfer if there is nothing to do
- * as this is used by the i2c probe to find devices.
- */
-
- if (is_lastmsg(i2c) && i2c->msg->len == 0) {
- nuc900_i2c_stop(i2c, 0);
- break;
- }
-
- if (i2c->state == STATE_READ)
- goto prepare_read;
-
- /* fall through to the write state, as we will need to
- * send a byte as well
- */
-
- case STATE_WRITE:
- /* we are writing data to the device... check for the
- * end of the message, and if so, work out what to do
- */
-
- if (!(i2c->msg->flags & I2C_M_IGNORE_NAK)) {
- if (iicstat & SLAVE_ACK) {
- dev_dbg(i2c->dev, "WRITE: No Ack\n");
-
- nuc900_i2c_stop(i2c, -ECONNREFUSED);
- break;
- }
- }
-
-retry_write:
-
- if (!is_msgend(i2c)) {
- byte = i2c->msg->buf[i2c->msg_ptr++];
- writeb(byte, i2c->regs + TXR);
- writel(I2C_CMD_WRITE, i2c->regs + CMDR);
-
- } else if (!is_lastmsg(i2c)) {
- /* we need to go to the next i2c message */
-
- dev_dbg(i2c->dev, "WRITE: Next Message\n");
-
- i2c->msg_ptr = 0;
- i2c->msg_idx++;
- i2c->msg++;
-
- /* check to see if we need to do another message */
- if (i2c->msg->flags & I2C_M_NOSTART) {
-
- if (i2c->msg->flags & I2C_M_RD) {
- /* cannot do this, the controller
- * forces us to send a new START
- * when we change direction
- */
-
- nuc900_i2c_stop(i2c, -EINVAL);
- }
-
- goto retry_write;
- } else {
- /* send the new start */
- nuc900_i2c_message_start(i2c, i2c->msg);
- i2c->state = STATE_START;
- }
-
- } else {
- /* send stop */
-
- nuc900_i2c_stop(i2c, 0);
- }
- break;
-
- case STATE_READ:
- /* we have a byte of data in the data register, do
- * something with it, and then work out whether we are
- * going to do any more read/write
- */
-
- byte = readb(i2c->regs + RXR);
- i2c->msg->buf[i2c->msg_ptr++] = byte;
-
-prepare_read:
- if (is_msglast(i2c)) {
- /* last byte of buffer */
-
- if (is_lastmsg(i2c))
- writel(I2C_CMD_READ | I2C_CMD_NACK,
- i2c->regs + CMDR);
-
- } else if (is_msgend(i2c)) {
- /* ok, we've read the entire buffer, see if there
- * is anything else we need to do
- */
-
- if (is_lastmsg(i2c)) {
- /* last message, send stop and complete */
- dev_dbg(i2c->dev, "READ: Send Stop\n");
-
- nuc900_i2c_stop(i2c, 0);
- } else {
- /* go to the next transfer */
- dev_dbg(i2c->dev, "READ: Next Transfer\n");
-
- i2c->msg_ptr = 0;
- i2c->msg_idx++;
- i2c->msg++;
-
- writel(I2C_CMD_READ, i2c->regs + CMDR);
- }
-
- } else {
- writel(I2C_CMD_READ, i2c->regs + CMDR);
- }
-
- break;
- }
-}
-
-/* nuc900_i2c_irq
- *
- * top level IRQ servicing routine
-*/
-
-static irqreturn_t nuc900_i2c_irq(int irqno, void *dev_id)
-{
- struct nuc900_i2c *i2c = dev_id;
- unsigned long status;
-
- status = readl(i2c->regs + CSR);
- writel(status | IRQFLAG, i2c->regs + CSR);
-
- if (status & ARBIT_LOST) {
- /* deal with arbitration loss */
- dev_err(i2c->dev, "deal with arbitration loss\n");
- goto out;
- }
-
- if (i2c->state == STATE_IDLE) {
- dev_dbg(i2c->dev, "IRQ: error i2c->state == IDLE\n");
- goto out;
- }
-
- /* pretty much this leaves us with the fact that we've
- * transmitted or received whatever byte we last sent
- */
-
- i2c_nuc900_irq_nextbyte(i2c, status);
-
- out:
- return IRQ_HANDLED;
-}
-
-
-/* nuc900_i2c_set_master
- *
- * get the i2c bus for a master transaction
-*/
-
-static int nuc900_i2c_set_master(struct nuc900_i2c *i2c)
-{
- int timeout = 400;
-
- while (timeout-- > 0) {
- if (((readl(i2c->regs + SWR) & I2CSTART) == I2CSTART) &&
- ((readl(i2c->regs + CSR) & I2CBUSY) == 0)) {
- return 0;
- }
-
- msleep(1);
- }
-
- return -ETIMEDOUT;
-}
-
-/* nuc900_i2c_doxfer
- *
- * this starts an i2c transfer
-*/
-
-static int nuc900_i2c_doxfer(struct nuc900_i2c *i2c,
- struct i2c_msg *msgs, int num)
-{
- unsigned long iicstat, timeout;
- int spins = 20;
- int ret;
-
- ret = nuc900_i2c_set_master(i2c);
- if (ret != 0) {
- dev_err(i2c->dev, "cannot get bus (error %d)\n", ret);
- ret = -EAGAIN;
- goto out;
- }
-
- spin_lock_irq(&i2c->lock);
-
- i2c->msg = msgs;
- i2c->msg_num = num;
- i2c->msg_ptr = 0;
- i2c->msg_idx = 0;
- i2c->state = STATE_START;
-
- nuc900_i2c_message_start(i2c, msgs);
- spin_unlock_irq(&i2c->lock);
-
- timeout = wait_event_timeout(i2c->wait, i2c->msg_num == 0, HZ * 5);
-
- ret = i2c->msg_idx;
-
- /* having these next two as dev_err() makes life very
- * noisy when doing an i2cdetect
- */
-
- if (timeout == 0)
- dev_dbg(i2c->dev, "timeout\n");
- else if (ret != num)
- dev_dbg(i2c->dev, "incomplete xfer (%d)\n", ret);
-
- /* ensure the stop has been through the bus */
-
- dev_dbg(i2c->dev, "waiting for bus idle\n");
-
- /* first, try busy waiting briefly */
- do {
- iicstat = readl(i2c->regs + CSR);
- } while ((iicstat & I2CBUSY) && --spins);
-
- /* if that timed out sleep */
- if (!spins) {
- msleep(1);
- iicstat = readl(i2c->regs + CSR);
- }
-
- if (iicstat & I2CBUSY)
- dev_warn(i2c->dev, "timeout waiting for bus idle\n");
-
- out:
- return ret;
-}
-
-/* nuc900_i2c_xfer
- *
- * first port of call from the i2c bus code when an message needs
- * transferring across the i2c bus.
-*/
-
-static int nuc900_i2c_xfer(struct i2c_adapter *adap,
- struct i2c_msg *msgs, int num)
-{
- struct nuc900_i2c *i2c = (struct nuc900_i2c *)adap->algo_data;
- int retry;
- int ret;
-
- nuc900_i2c_enable_irq(i2c);
-
- for (retry = 0; retry < adap->retries; retry++) {
-
- ret = nuc900_i2c_doxfer(i2c, msgs, num);
-
- if (ret != -EAGAIN)
- return ret;
-
- dev_dbg(i2c->dev, "Retrying transmission (%d)\n", retry);
-
- udelay(100);
- }
-
- return -EREMOTEIO;
-}
-
-/* declare our i2c functionality */
-static u32 nuc900_i2c_func(struct i2c_adapter *adap)
-{
- return I2C_FUNC_I2C | I2C_FUNC_SMBUS_EMUL | I2C_FUNC_NOSTART |
- I2C_FUNC_PROTOCOL_MANGLING;
-}
-
-/* i2c bus registration info */
-
-static const struct i2c_algorithm nuc900_i2c_algorithm = {
- .master_xfer = nuc900_i2c_xfer,
- .functionality = nuc900_i2c_func,
-};
-
-/* nuc900_i2c_probe
- *
- * called by the bus driver when a suitable device is found
-*/
-
-static int nuc900_i2c_probe(struct platform_device *pdev)
-{
- struct nuc900_i2c *i2c;
- struct nuc900_platform_i2c *pdata;
- struct resource *res;
- int ret;
-
- pdata = dev_get_platdata(&pdev->dev);
- if (!pdata) {
- dev_err(&pdev->dev, "no platform data\n");
- return -EINVAL;
- }
-
- i2c = kzalloc(sizeof(struct nuc900_i2c), GFP_KERNEL);
- if (!i2c) {
- dev_err(&pdev->dev, "no memory for state\n");
- return -ENOMEM;
- }
-
- strlcpy(i2c->adap.name, "nuc900-i2c0", sizeof(i2c->adap.name));
- i2c->adap.owner = THIS_MODULE;
- i2c->adap.algo = &nuc900_i2c_algorithm;
- i2c->adap.retries = 2;
- i2c->adap.class = I2C_CLASS_HWMON | I2C_CLASS_SPD;
-
- spin_lock_init(&i2c->lock);
- init_waitqueue_head(&i2c->wait);
-
- /* find the clock and enable it */
-
- i2c->dev = &pdev->dev;
- i2c->clk = clk_get(&pdev->dev, NULL);
- if (IS_ERR(i2c->clk)) {
- dev_err(&pdev->dev, "cannot get clock\n");
- ret = -ENOENT;
- goto err_noclk;
- }
-
- dev_dbg(&pdev->dev, "clock source %p\n", i2c->clk);
-
- clk_enable(i2c->clk);
-
- /* map the registers */
-
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (res == NULL) {
- dev_err(&pdev->dev, "cannot find IO resource\n");
- ret = -ENOENT;
- goto err_clk;
- }
-
- i2c->ioarea = request_mem_region(res->start, resource_size(res),
- pdev->name);
-
- if (i2c->ioarea == NULL) {
- dev_err(&pdev->dev, "cannot request IO\n");
- ret = -ENXIO;
- goto err_clk;
- }
-
- i2c->regs = ioremap(res->start, resource_size(res));
-
- if (i2c->regs == NULL) {
- dev_err(&pdev->dev, "cannot map IO\n");
- ret = -ENXIO;
- goto err_ioarea;
- }
-
- dev_dbg(&pdev->dev, "registers %p (%p, %p)\n",
- i2c->regs, i2c->ioarea, res);
-
- /* setup info block for the i2c core */
-
- i2c->adap.algo_data = i2c;
- i2c->adap.dev.parent = &pdev->dev;
-
- mfp_set_groupg(&pdev->dev, NULL);
-
- clk_get_rate(i2c->clk);
-
- ret = (i2c->clk.apbfreq)/(pdata->bus_freq * 5) - 1;
- writel(ret & 0xffff, i2c->regs + DIVIDER);
-
- /* find the IRQ for this unit (note, this relies on the init call to
- * ensure no current IRQs pending
- */
-
- i2c->irq = ret = platform_get_irq(pdev, 0);
- if (ret <= 0) {
- dev_err(&pdev->dev, "cannot find IRQ\n");
- goto err_iomap;
- }
-
- ret = request_irq(i2c->irq, nuc900_i2c_irq, IRQF_SHARED,
- dev_name(&pdev->dev), i2c);
-
- if (ret != 0) {
- dev_err(&pdev->dev, "cannot claim IRQ %d\n", i2c->irq);
- goto err_iomap;
- }
-
- /* Note, previous versions of the driver used i2c_add_adapter()
- * to add the bus at any number. We now pass the bus number via
- * the platform data, so if unset it will now default to always
- * being bus 0.
- */
-
- i2c->adap.nr = pdata->bus_num;
-
- ret = i2c_add_numbered_adapter(&i2c->adap);
- if (ret < 0) {
- dev_err(&pdev->dev, "failed to add bus to i2c core\n");
- goto err_irq;
- }
-
- platform_set_drvdata(pdev, i2c);
-
- dev_info(&pdev->dev, "%s: NUC900 I2C adapter\n",
- dev_name(&i2c->adap.dev));
- return 0;
-
- err_irq:
- free_irq(i2c->irq, i2c);
-
- err_iomap:
- iounmap(i2c->regs);
-
- err_ioarea:
- release_resource(i2c->ioarea);
- kfree(i2c->ioarea);
-
- err_clk:
- clk_disable(i2c->clk);
- clk_put(i2c->clk);
-
- err_noclk:
- kfree(i2c);
- return ret;
-}
-
-/* nuc900_i2c_remove
- *
- * called when device is removed from the bus
-*/
-
-static int nuc900_i2c_remove(struct platform_device *pdev)
-{
- struct nuc900_i2c *i2c = platform_get_drvdata(pdev);
-
- i2c_del_adapter(&i2c->adap);
- free_irq(i2c->irq, i2c);
-
- clk_disable(i2c->clk);
- clk_put(i2c->clk);
-
- iounmap(i2c->regs);
-
- release_resource(i2c->ioarea);
- kfree(i2c->ioarea);
- kfree(i2c);
-
- return 0;
-}
-
-static struct platform_driver nuc900_i2c_driver = {
- .probe = nuc900_i2c_probe,
- .remove = nuc900_i2c_remove,
- .driver = {
- .owner = THIS_MODULE,
- .name = "nuc900-i2c0",
- },
-};
-
-static int __init i2c_adap_nuc900_init(void)
-{
- return platform_driver_register(&nuc900_i2c_driver);
-}
-
-static void __exit i2c_adap_nuc900_exit(void)
-{
- platform_driver_unregister(&nuc900_i2c_driver);
-}
-subsys_initcall(i2c_adap_nuc900_init);
-module_exit(i2c_adap_nuc900_exit);
-
-MODULE_DESCRIPTION("NUC900 I2C Bus driver");
-MODULE_AUTHOR("Wan ZongShun, <mcuos.com-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>");
-MODULE_LICENSE("GPL");
-MODULE_ALIAS("platform:nuc900-i2c0");
diff --git a/drivers/i2c/busses/i2c-ocores.c b/drivers/i2c/busses/i2c-ocores.c
index 1f6369f..0e10cc6 100644
--- a/drivers/i2c/busses/i2c-ocores.c
+++ b/drivers/i2c/busses/i2c-ocores.c
@@ -250,7 +250,7 @@
.algo = &ocores_algorithm,
};
-static struct of_device_id ocores_i2c_match[] = {
+static const struct of_device_id ocores_i2c_match[] = {
{
.compatible = "opencores,i2c-ocores",
.data = (void *)TYPE_OCORES,
diff --git a/drivers/i2c/busses/i2c-omap.c b/drivers/i2c/busses/i2c-omap.c
index 85f8eac..b182793 100644
--- a/drivers/i2c/busses/i2c-omap.c
+++ b/drivers/i2c/busses/i2c-omap.c
@@ -1114,10 +1114,8 @@
}
dev = devm_kzalloc(&pdev->dev, sizeof(struct omap_i2c_dev), GFP_KERNEL);
- if (!dev) {
- dev_err(&pdev->dev, "Menory allocation failed\n");
+ if (!dev)
return -ENOMEM;
- }
mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
dev->base = devm_ioremap_resource(&pdev->dev, mem);
diff --git a/drivers/i2c/busses/i2c-pxa.c b/drivers/i2c/busses/i2c-pxa.c
index bbe6dfb..be671f7 100644
--- a/drivers/i2c/busses/i2c-pxa.c
+++ b/drivers/i2c/busses/i2c-pxa.c
@@ -1084,7 +1084,7 @@
.functionality = i2c_pxa_functionality,
};
-static struct of_device_id i2c_pxa_dt_ids[] = {
+static const struct of_device_id i2c_pxa_dt_ids[] = {
{ .compatible = "mrvl,pxa-i2c", .data = (void *)REGS_PXA2XX },
{ .compatible = "mrvl,pwri2c", .data = (void *)REGS_PXA3XX },
{ .compatible = "mrvl,mmp-twsi", .data = (void *)REGS_PXA2XX },
diff --git a/drivers/i2c/busses/i2c-rcar.c b/drivers/i2c/busses/i2c-rcar.c
index 06d47aaf..8994059 100644
--- a/drivers/i2c/busses/i2c-rcar.c
+++ b/drivers/i2c/busses/i2c-rcar.c
@@ -1,7 +1,9 @@
/*
- * drivers/i2c/busses/i2c-rcar.c
+ * Driver for the Renesas RCar I2C unit
*
- * Copyright (C) 2012 Renesas Solutions Corp.
+ * Copyright (C) 2014 Wolfram Sang <wsa@sang-engineering.com>
+ *
+ * Copyright (C) 2012-14 Renesas Solutions Corp.
* Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
*
* This file is based on the drivers/i2c/busses/i2c-sh7760.c
@@ -12,16 +14,12 @@
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License
+ * the Free Software Foundation; version 2 of the License.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/clk.h>
#include <linux/delay.h>
@@ -36,7 +34,6 @@
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
#include <linux/slab.h>
-#include <linux/spinlock.h>
/* register offsets */
#define ICSCR 0x00 /* slave ctrl */
@@ -60,7 +57,7 @@
#define FSB (1 << 1) /* force stop bit */
#define ESG (1 << 0) /* en startbit gen */
-/* ICMSR */
+/* ICMSR (also for ICMIE) */
#define MNR (1 << 6) /* nack received */
#define MAL (1 << 5) /* arbitration lost */
#define MST (1 << 4) /* sent a stop */
@@ -69,32 +66,18 @@
#define MDR (1 << 1)
#define MAT (1 << 0) /* slave addr xfer done */
-/* ICMIE */
-#define MNRE (1 << 6) /* nack irq en */
-#define MALE (1 << 5) /* arblos irq en */
-#define MSTE (1 << 4) /* stop irq en */
-#define MDEE (1 << 3)
-#define MDTE (1 << 2)
-#define MDRE (1 << 1)
-#define MATE (1 << 0) /* address sent irq en */
+#define RCAR_BUS_PHASE_START (MDBS | MIE | ESG)
+#define RCAR_BUS_PHASE_DATA (MDBS | MIE)
+#define RCAR_BUS_PHASE_STOP (MDBS | MIE | FSB)
-enum {
- RCAR_BUS_PHASE_ADDR,
- RCAR_BUS_PHASE_DATA,
- RCAR_BUS_PHASE_STOP,
-};
+#define RCAR_IRQ_SEND (MNR | MAL | MST | MAT | MDE)
+#define RCAR_IRQ_RECV (MNR | MAL | MST | MAT | MDR)
+#define RCAR_IRQ_STOP (MST)
-enum {
- RCAR_IRQ_CLOSE,
- RCAR_IRQ_OPEN_FOR_SEND,
- RCAR_IRQ_OPEN_FOR_RECV,
- RCAR_IRQ_OPEN_FOR_STOP,
-};
+#define RCAR_IRQ_ACK_SEND (~(MAT | MDE))
+#define RCAR_IRQ_ACK_RECV (~(MAT | MDR))
-/*
- * flags
- */
#define ID_LAST_MSG (1 << 0)
#define ID_IOERROR (1 << 1)
#define ID_DONE (1 << 2)
@@ -112,14 +95,12 @@
struct i2c_msg *msg;
struct clk *clk;
- spinlock_t lock;
wait_queue_head_t wait;
int pos;
- int irq;
u32 icccr;
u32 flags;
- enum rcar_i2c_type devtype;
+ enum rcar_i2c_type devtype;
};
#define rcar_i2c_priv_to_dev(p) ((p)->adap.dev.parent)
@@ -130,9 +111,7 @@
#define LOOP_TIMEOUT 1024
-/*
- * basic functions
- */
+
static void rcar_i2c_write(struct rcar_i2c_priv *priv, int reg, u32 val)
{
writel(val, priv->io + reg);
@@ -161,36 +140,6 @@
rcar_i2c_write(priv, ICMAR, 0);
}
-static void rcar_i2c_irq_mask(struct rcar_i2c_priv *priv, int open)
-{
- u32 val = MNRE | MALE | MSTE | MATE; /* default */
-
- switch (open) {
- case RCAR_IRQ_OPEN_FOR_SEND:
- val |= MDEE; /* default + send */
- break;
- case RCAR_IRQ_OPEN_FOR_RECV:
- val |= MDRE; /* default + read */
- break;
- case RCAR_IRQ_OPEN_FOR_STOP:
- val = MSTE; /* stop irq only */
- break;
- case RCAR_IRQ_CLOSE:
- default:
- val = 0; /* all close */
- break;
- }
- rcar_i2c_write(priv, ICMIER, val);
-}
-
-static void rcar_i2c_set_addr(struct rcar_i2c_priv *priv, u32 recv)
-{
- rcar_i2c_write(priv, ICMAR, (priv->msg->addr << 1) | recv);
-}
-
-/*
- * bus control functions
- */
static int rcar_i2c_bus_barrier(struct rcar_i2c_priv *priv)
{
int i;
@@ -205,24 +154,6 @@
return -EBUSY;
}
-static void rcar_i2c_bus_phase(struct rcar_i2c_priv *priv, int phase)
-{
- switch (phase) {
- case RCAR_BUS_PHASE_ADDR:
- rcar_i2c_write(priv, ICMCR, MDBS | MIE | ESG);
- break;
- case RCAR_BUS_PHASE_DATA:
- rcar_i2c_write(priv, ICMCR, MDBS | MIE);
- break;
- case RCAR_BUS_PHASE_STOP:
- rcar_i2c_write(priv, ICMCR, MDBS | MIE | FSB);
- break;
- }
-}
-
-/*
- * clock function
- */
static int rcar_i2c_clock_calculate(struct rcar_i2c_priv *priv,
u32 bus_speed,
struct device *dev)
@@ -312,60 +243,18 @@
return 0;
}
-static void rcar_i2c_clock_start(struct rcar_i2c_priv *priv)
+static int rcar_i2c_prepare_msg(struct rcar_i2c_priv *priv)
{
- rcar_i2c_write(priv, ICCCR, priv->icccr);
-}
+ int read = !!rcar_i2c_is_recv(priv);
-/*
- * status functions
- */
-static u32 rcar_i2c_status_get(struct rcar_i2c_priv *priv)
-{
- return rcar_i2c_read(priv, ICMSR);
-}
-
-#define rcar_i2c_status_clear(priv) rcar_i2c_status_bit_clear(priv, 0xffffffff)
-static void rcar_i2c_status_bit_clear(struct rcar_i2c_priv *priv, u32 bit)
-{
- rcar_i2c_write(priv, ICMSR, ~bit);
-}
-
-/*
- * recv/send functions
- */
-static int rcar_i2c_recv(struct rcar_i2c_priv *priv)
-{
- rcar_i2c_set_addr(priv, 1);
- rcar_i2c_status_clear(priv);
- rcar_i2c_bus_phase(priv, RCAR_BUS_PHASE_ADDR);
- rcar_i2c_irq_mask(priv, RCAR_IRQ_OPEN_FOR_RECV);
+ rcar_i2c_write(priv, ICMAR, (priv->msg->addr << 1) | read);
+ rcar_i2c_write(priv, ICMSR, 0);
+ rcar_i2c_write(priv, ICMCR, RCAR_BUS_PHASE_START);
+ rcar_i2c_write(priv, ICMIER, read ? RCAR_IRQ_RECV : RCAR_IRQ_SEND);
return 0;
}
-static int rcar_i2c_send(struct rcar_i2c_priv *priv)
-{
- int ret;
-
- /*
- * It should check bus status when send case
- */
- ret = rcar_i2c_bus_barrier(priv);
- if (ret < 0)
- return ret;
-
- rcar_i2c_set_addr(priv, 0);
- rcar_i2c_status_clear(priv);
- rcar_i2c_bus_phase(priv, RCAR_BUS_PHASE_ADDR);
- rcar_i2c_irq_mask(priv, RCAR_IRQ_OPEN_FOR_SEND);
-
- return 0;
-}
-
-#define rcar_i2c_send_restart(priv) rcar_i2c_status_bit_clear(priv, (MAT | MDE))
-#define rcar_i2c_recv_restart(priv) rcar_i2c_status_bit_clear(priv, (MAT | MDR))
-
/*
* interrupt functions
*/
@@ -386,7 +275,7 @@
* goto data phase.
*/
if (msr & MAT)
- rcar_i2c_bus_phase(priv, RCAR_BUS_PHASE_DATA);
+ rcar_i2c_write(priv, ICMCR, RCAR_BUS_PHASE_DATA);
if (priv->pos < msg->len) {
/*
@@ -414,7 +303,7 @@
* prepare stop condition here.
* ID_DONE will be set on STOP irq.
*/
- rcar_i2c_bus_phase(priv, RCAR_BUS_PHASE_STOP);
+ rcar_i2c_write(priv, ICMCR, RCAR_BUS_PHASE_STOP);
else
/*
* If current msg is _NOT_ last msg,
@@ -425,7 +314,7 @@
return ID_DONE;
}
- rcar_i2c_send_restart(priv);
+ rcar_i2c_write(priv, ICMSR, RCAR_IRQ_ACK_SEND);
return 0;
}
@@ -462,11 +351,11 @@
* otherwise, go to DATA phase.
*/
if (priv->pos + 1 >= msg->len)
- rcar_i2c_bus_phase(priv, RCAR_BUS_PHASE_STOP);
+ rcar_i2c_write(priv, ICMCR, RCAR_BUS_PHASE_STOP);
else
- rcar_i2c_bus_phase(priv, RCAR_BUS_PHASE_DATA);
+ rcar_i2c_write(priv, ICMCR, RCAR_BUS_PHASE_DATA);
- rcar_i2c_recv_restart(priv);
+ rcar_i2c_write(priv, ICMSR, RCAR_IRQ_ACK_RECV);
return 0;
}
@@ -474,53 +363,31 @@
static irqreturn_t rcar_i2c_irq(int irq, void *ptr)
{
struct rcar_i2c_priv *priv = ptr;
- struct device *dev = rcar_i2c_priv_to_dev(priv);
u32 msr;
- /*-------------- spin lock -----------------*/
- spin_lock(&priv->lock);
+ msr = rcar_i2c_read(priv, ICMSR);
- msr = rcar_i2c_status_get(priv);
-
- /*
- * Arbitration lost
- */
+ /* Arbitration lost */
if (msr & MAL) {
- /*
- * CAUTION
- *
- * When arbitration lost, device become _slave_ mode.
- */
- dev_dbg(dev, "Arbitration Lost\n");
rcar_i2c_flags_set(priv, (ID_DONE | ID_ARBLOST));
goto out;
}
- /*
- * Stop
- */
+ /* Stop */
if (msr & MST) {
- dev_dbg(dev, "Stop\n");
rcar_i2c_flags_set(priv, ID_DONE);
goto out;
}
- /*
- * Nack
- */
+ /* Nack */
if (msr & MNR) {
- dev_dbg(dev, "Nack\n");
-
/* go to stop phase */
- rcar_i2c_bus_phase(priv, RCAR_BUS_PHASE_STOP);
- rcar_i2c_irq_mask(priv, RCAR_IRQ_OPEN_FOR_STOP);
+ rcar_i2c_write(priv, ICMCR, RCAR_BUS_PHASE_STOP);
+ rcar_i2c_write(priv, ICMIER, RCAR_IRQ_STOP);
rcar_i2c_flags_set(priv, ID_NACK);
goto out;
}
- /*
- * recv/send
- */
if (rcar_i2c_is_recv(priv))
rcar_i2c_flags_set(priv, rcar_i2c_irq_recv(priv, msr));
else
@@ -528,14 +395,11 @@
out:
if (rcar_i2c_flags_has(priv, ID_DONE)) {
- rcar_i2c_irq_mask(priv, RCAR_IRQ_CLOSE);
- rcar_i2c_status_clear(priv);
+ rcar_i2c_write(priv, ICMIER, 0);
+ rcar_i2c_write(priv, ICMSR, 0);
wake_up(&priv->wait);
}
- spin_unlock(&priv->lock);
- /*-------------- spin unlock -----------------*/
-
return IRQ_HANDLED;
}
@@ -545,21 +409,18 @@
{
struct rcar_i2c_priv *priv = i2c_get_adapdata(adap);
struct device *dev = rcar_i2c_priv_to_dev(priv);
- unsigned long flags;
int i, ret, timeout;
pm_runtime_get_sync(dev);
- /*-------------- spin lock -----------------*/
- spin_lock_irqsave(&priv->lock, flags);
-
rcar_i2c_init(priv);
- rcar_i2c_clock_start(priv);
+ /* start clock */
+ rcar_i2c_write(priv, ICCCR, priv->icccr);
- spin_unlock_irqrestore(&priv->lock, flags);
- /*-------------- spin unlock -----------------*/
+ ret = rcar_i2c_bus_barrier(priv);
+ if (ret < 0)
+ goto out;
- ret = -EINVAL;
for (i = 0; i < num; i++) {
/* This HW can't send STOP after address phase */
if (msgs[i].len == 0) {
@@ -567,9 +428,6 @@
break;
}
- /*-------------- spin lock -----------------*/
- spin_lock_irqsave(&priv->lock, flags);
-
/* init each data */
priv->msg = &msgs[i];
priv->pos = 0;
@@ -577,21 +435,11 @@
if (priv->msg == &msgs[num - 1])
rcar_i2c_flags_set(priv, ID_LAST_MSG);
- /* start send/recv */
- if (rcar_i2c_is_recv(priv))
- ret = rcar_i2c_recv(priv);
- else
- ret = rcar_i2c_send(priv);
-
- spin_unlock_irqrestore(&priv->lock, flags);
- /*-------------- spin unlock -----------------*/
+ ret = rcar_i2c_prepare_msg(priv);
if (ret < 0)
break;
- /*
- * wait result
- */
timeout = wait_event_timeout(priv->wait,
rcar_i2c_flags_has(priv, ID_DONE),
5 * HZ);
@@ -600,9 +448,6 @@
break;
}
- /*
- * error handling
- */
if (rcar_i2c_flags_has(priv, ID_NACK)) {
ret = -ENXIO;
break;
@@ -620,7 +465,7 @@
ret = i + 1; /* The number of transfer */
}
-
+out:
pm_runtime_put(dev);
if (ret < 0 && ret != -ENXIO)
@@ -646,6 +491,9 @@
{ .compatible = "renesas,i2c-r8a7779", .data = (void *)I2C_RCAR_GEN1 },
{ .compatible = "renesas,i2c-r8a7790", .data = (void *)I2C_RCAR_GEN2 },
{ .compatible = "renesas,i2c-r8a7791", .data = (void *)I2C_RCAR_GEN2 },
+ { .compatible = "renesas,i2c-r8a7792", .data = (void *)I2C_RCAR_GEN2 },
+ { .compatible = "renesas,i2c-r8a7793", .data = (void *)I2C_RCAR_GEN2 },
+ { .compatible = "renesas,i2c-r8a7794", .data = (void *)I2C_RCAR_GEN2 },
{},
};
MODULE_DEVICE_TABLE(of, rcar_i2c_dt_ids);
@@ -658,13 +506,11 @@
struct resource *res;
struct device *dev = &pdev->dev;
u32 bus_speed;
- int ret;
+ int irq, ret;
priv = devm_kzalloc(dev, sizeof(struct rcar_i2c_priv), GFP_KERNEL);
- if (!priv) {
- dev_err(dev, "no mem for private data\n");
+ if (!priv)
return -ENOMEM;
- }
priv->clk = devm_clk_get(dev, NULL);
if (IS_ERR(priv->clk)) {
@@ -692,9 +538,8 @@
if (IS_ERR(priv->io))
return PTR_ERR(priv->io);
- priv->irq = platform_get_irq(pdev, 0);
+ irq = platform_get_irq(pdev, 0);
init_waitqueue_head(&priv->wait);
- spin_lock_init(&priv->lock);
adap = &priv->adap;
adap->nr = pdev->id;
@@ -706,10 +551,10 @@
i2c_set_adapdata(adap, priv);
strlcpy(adap->name, pdev->name, sizeof(adap->name));
- ret = devm_request_irq(dev, priv->irq, rcar_i2c_irq, 0,
+ ret = devm_request_irq(dev, irq, rcar_i2c_irq, 0,
dev_name(dev), priv);
if (ret < 0) {
- dev_err(dev, "cannot get irq %d\n", priv->irq);
+ dev_err(dev, "cannot get irq %d\n", irq);
return ret;
}
@@ -759,6 +604,6 @@
module_platform_driver(rcar_i2c_driver);
-MODULE_LICENSE("GPL");
+MODULE_LICENSE("GPL v2");
MODULE_DESCRIPTION("Renesas R-Car I2C bus driver");
MODULE_AUTHOR("Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>");
diff --git a/drivers/i2c/busses/i2c-riic.c b/drivers/i2c/busses/i2c-riic.c
index 9e1f8ba..af3b3d0 100644
--- a/drivers/i2c/busses/i2c-riic.c
+++ b/drivers/i2c/busses/i2c-riic.c
@@ -404,7 +404,7 @@
return 0;
}
-static struct of_device_id riic_i2c_dt_ids[] = {
+static const struct of_device_id riic_i2c_dt_ids[] = {
{ .compatible = "renesas,riic-rz" },
{ /* Sentinel */ },
};
diff --git a/drivers/i2c/busses/i2c-s3c2410.c b/drivers/i2c/busses/i2c-s3c2410.c
index bb3a996..e828a1d 100644
--- a/drivers/i2c/busses/i2c-s3c2410.c
+++ b/drivers/i2c/busses/i2c-s3c2410.c
@@ -1114,16 +1114,12 @@
}
i2c = devm_kzalloc(&pdev->dev, sizeof(struct s3c24xx_i2c), GFP_KERNEL);
- if (!i2c) {
- dev_err(&pdev->dev, "no memory for state\n");
+ if (!i2c)
return -ENOMEM;
- }
i2c->pdata = devm_kzalloc(&pdev->dev, sizeof(*pdata), GFP_KERNEL);
- if (!i2c->pdata) {
- dev_err(&pdev->dev, "no memory for platform data\n");
+ if (!i2c->pdata)
return -ENOMEM;
- }
i2c->quirks = s3c24xx_get_device_quirks(pdev);
if (pdata)
diff --git a/drivers/i2c/busses/i2c-sh_mobile.c b/drivers/i2c/busses/i2c-sh_mobile.c
index 1d79585..8b5e79c 100644
--- a/drivers/i2c/busses/i2c-sh_mobile.c
+++ b/drivers/i2c/busses/i2c-sh_mobile.c
@@ -32,6 +32,7 @@
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/slab.h>
+#include <linux/of_device.h>
#include <linux/i2c/i2c-sh_mobile.h>
/* Transmit operation: */
@@ -139,6 +140,10 @@
bool send_stop;
};
+struct sh_mobile_dt_config {
+ int clks_per_count;
+};
+
#define IIC_FLAG_HAS_ICIC67 (1 << 0)
#define STANDARD_MODE 100000
@@ -194,7 +199,7 @@
iic_wr(pd, offs, (iic_rd(pd, offs) | set) & ~clr);
}
-static u32 sh_mobile_i2c_iccl(unsigned long count_khz, u32 tLOW, u32 tf, int offset)
+static u32 sh_mobile_i2c_iccl(unsigned long count_khz, u32 tLOW, u32 tf)
{
/*
* Conditional expression:
@@ -206,10 +211,10 @@
* account the fall time of SCL signal (tf). Default tf value
* should be 0.3 us, for safety.
*/
- return (((count_khz * (tLOW + tf)) + 5000) / 10000) + offset;
+ return (((count_khz * (tLOW + tf)) + 5000) / 10000);
}
-static u32 sh_mobile_i2c_icch(unsigned long count_khz, u32 tHIGH, u32 tf, int offset)
+static u32 sh_mobile_i2c_icch(unsigned long count_khz, u32 tHIGH, u32 tf)
{
/*
* Conditional expression:
@@ -225,52 +230,58 @@
* to take into account the fall time of SDA signal (tf) at START
* condition, in order to meet both tHIGH and tHD;STA specs.
*/
- return (((count_khz * (tHIGH + tf)) + 5000) / 10000) + offset;
+ return (((count_khz * (tHIGH + tf)) + 5000) / 10000);
}
-static void sh_mobile_i2c_init(struct sh_mobile_i2c_data *pd)
+static int sh_mobile_i2c_init(struct sh_mobile_i2c_data *pd)
{
unsigned long i2c_clk_khz;
u32 tHIGH, tLOW, tf;
- int offset;
+ uint16_t max_val;
/* Get clock rate after clock is enabled */
clk_prepare_enable(pd->clk);
i2c_clk_khz = clk_get_rate(pd->clk) / 1000;
+ clk_disable_unprepare(pd->clk);
i2c_clk_khz /= pd->clks_per_count;
if (pd->bus_speed == STANDARD_MODE) {
tLOW = 47; /* tLOW = 4.7 us */
tHIGH = 40; /* tHD;STA = tHIGH = 4.0 us */
tf = 3; /* tf = 0.3 us */
- offset = 0; /* No offset */
} else if (pd->bus_speed == FAST_MODE) {
tLOW = 13; /* tLOW = 1.3 us */
tHIGH = 6; /* tHD;STA = tHIGH = 0.6 us */
tf = 3; /* tf = 0.3 us */
- offset = 0; /* No offset */
} else {
dev_err(pd->dev, "unrecognized bus speed %lu Hz\n",
pd->bus_speed);
- goto out;
+ return -EINVAL;
}
- pd->iccl = sh_mobile_i2c_iccl(i2c_clk_khz, tLOW, tf, offset);
+ pd->iccl = sh_mobile_i2c_iccl(i2c_clk_khz, tLOW, tf);
+ pd->icch = sh_mobile_i2c_icch(i2c_clk_khz, tHIGH, tf);
+
+ max_val = pd->flags & IIC_FLAG_HAS_ICIC67 ? 0x1ff : 0xff;
+ if (pd->iccl > max_val || pd->icch > max_val) {
+ dev_err(pd->dev, "timing values out of range: L/H=0x%x/0x%x\n",
+ pd->iccl, pd->icch);
+ return -EINVAL;
+ }
+
/* one more bit of ICCL in ICIC */
- if ((pd->iccl > 0xff) && (pd->flags & IIC_FLAG_HAS_ICIC67))
+ if (pd->iccl & 0x100)
pd->icic |= ICIC_ICCLB8;
else
pd->icic &= ~ICIC_ICCLB8;
- pd->icch = sh_mobile_i2c_icch(i2c_clk_khz, tHIGH, tf, offset);
/* one more bit of ICCH in ICIC */
- if ((pd->icch > 0xff) && (pd->flags & IIC_FLAG_HAS_ICIC67))
+ if (pd->icch & 0x100)
pd->icic |= ICIC_ICCHB8;
else
pd->icic &= ~ICIC_ICCHB8;
-out:
- clk_disable_unprepare(pd->clk);
+ return 0;
}
static void activate_ch(struct sh_mobile_i2c_data *pd)
@@ -316,7 +327,7 @@
switch (op) {
case OP_START: /* issue start and trigger DTE interrupt */
- iic_wr(pd, ICCR, 0x94);
+ iic_wr(pd, ICCR, ICCR_ICE | ICCR_TRS | ICCR_BBSY);
break;
case OP_TX_FIRST: /* disable DTE interrupt and write data */
iic_wr(pd, ICIC, ICIC_WAITE | ICIC_ALE | ICIC_TACKE);
@@ -327,10 +338,11 @@
break;
case OP_TX_STOP: /* write data and issue a stop afterwards */
iic_wr(pd, ICDR, data);
- iic_wr(pd, ICCR, pd->send_stop ? 0x90 : 0x94);
+ iic_wr(pd, ICCR, pd->send_stop ? ICCR_ICE | ICCR_TRS
+ : ICCR_ICE | ICCR_TRS | ICCR_BBSY);
break;
case OP_TX_TO_RX: /* select read mode */
- iic_wr(pd, ICCR, 0x81);
+ iic_wr(pd, ICCR, ICCR_ICE | ICCR_SCP);
break;
case OP_RX: /* just read data */
ret = iic_rd(pd, ICDR);
@@ -338,13 +350,13 @@
case OP_RX_STOP: /* enable DTE interrupt, issue stop */
iic_wr(pd, ICIC,
ICIC_DTEE | ICIC_WAITE | ICIC_ALE | ICIC_TACKE);
- iic_wr(pd, ICCR, 0xc0);
+ iic_wr(pd, ICCR, ICCR_ICE | ICCR_RACK);
break;
case OP_RX_STOP_DATA: /* enable DTE interrupt, read data, issue stop */
iic_wr(pd, ICIC,
ICIC_DTEE | ICIC_WAITE | ICIC_ALE | ICIC_TACKE);
ret = iic_rd(pd, ICDR);
- iic_wr(pd, ICCR, 0xc0);
+ iic_wr(pd, ICCR, ICCR_ICE | ICCR_RACK);
break;
}
@@ -479,7 +491,7 @@
{
if (usr_msg->len == 0 && (usr_msg->flags & I2C_M_RD)) {
dev_err(pd->dev, "Unsupported zero length i2c read\n");
- return -EIO;
+ return -EOPNOTSUPP;
}
if (do_init) {
@@ -514,17 +526,12 @@
break;
if (val & ICSR_TACK)
- return -EIO;
+ return -ENXIO;
udelay(10);
}
- if (!i) {
- dev_warn(pd->dev, "Timeout polling for DTE!\n");
- return -ETIMEDOUT;
- }
-
- return 0;
+ return i ? 0 : -ETIMEDOUT;
}
static int poll_busy(struct sh_mobile_i2c_data *pd)
@@ -542,20 +549,18 @@
*/
if (!(val & ICSR_BUSY)) {
/* handle missing acknowledge and arbitration lost */
- if ((val | pd->sr) & (ICSR_TACK | ICSR_AL))
- return -EIO;
+ val |= pd->sr;
+ if (val & ICSR_TACK)
+ return -ENXIO;
+ if (val & ICSR_AL)
+ return -EAGAIN;
break;
}
udelay(10);
}
- if (!i) {
- dev_err(pd->dev, "Polling timed out\n");
- return -ETIMEDOUT;
- }
-
- return 0;
+ return i ? 0 : -ETIMEDOUT;
}
static int sh_mobile_i2c_xfer(struct i2c_adapter *adapter,
@@ -617,42 +622,44 @@
.master_xfer = sh_mobile_i2c_xfer,
};
-static int sh_mobile_i2c_hook_irqs(struct platform_device *dev, int hook)
+static const struct sh_mobile_dt_config default_dt_config = {
+ .clks_per_count = 1,
+};
+
+static const struct sh_mobile_dt_config rcar_gen2_dt_config = {
+ .clks_per_count = 2,
+};
+
+static const struct of_device_id sh_mobile_i2c_dt_ids[] = {
+ { .compatible = "renesas,rmobile-iic", .data = &default_dt_config },
+ { .compatible = "renesas,iic-r8a7790", .data = &rcar_gen2_dt_config },
+ { .compatible = "renesas,iic-r8a7791", .data = &rcar_gen2_dt_config },
+ { .compatible = "renesas,iic-r8a7792", .data = &rcar_gen2_dt_config },
+ { .compatible = "renesas,iic-r8a7793", .data = &rcar_gen2_dt_config },
+ { .compatible = "renesas,iic-r8a7794", .data = &rcar_gen2_dt_config },
+ {},
+};
+MODULE_DEVICE_TABLE(of, sh_mobile_i2c_dt_ids);
+
+static int sh_mobile_i2c_hook_irqs(struct platform_device *dev)
{
struct resource *res;
- int ret = -ENXIO;
- int n, k = 0;
+ resource_size_t n;
+ int k = 0, ret;
while ((res = platform_get_resource(dev, IORESOURCE_IRQ, k))) {
- for (n = res->start; hook && n <= res->end; n++) {
- if (request_irq(n, sh_mobile_i2c_isr, 0,
- dev_name(&dev->dev), dev)) {
- for (n--; n >= res->start; n--)
- free_irq(n, dev);
-
- goto rollback;
+ for (n = res->start; n <= res->end; n++) {
+ ret = devm_request_irq(&dev->dev, n, sh_mobile_i2c_isr,
+ 0, dev_name(&dev->dev), dev);
+ if (ret) {
+ dev_err(&dev->dev, "cannot request IRQ %pa\n", &n);
+ return ret;
}
}
k++;
}
- if (hook)
- return k > 0 ? 0 : -ENOENT;
-
- ret = 0;
-
- rollback:
- k--;
-
- while (k >= 0) {
- res = platform_get_resource(dev, IORESOURCE_IRQ, k);
- for (n = res->start; n <= res->end; n++)
- free_irq(n, dev);
-
- k--;
- }
-
- return ret;
+ return k > 0 ? 0 : -ENOENT;
}
static int sh_mobile_i2c_probe(struct platform_device *dev)
@@ -661,62 +668,64 @@
struct sh_mobile_i2c_data *pd;
struct i2c_adapter *adap;
struct resource *res;
- int size;
int ret;
+ u32 bus_speed;
- pd = kzalloc(sizeof(struct sh_mobile_i2c_data), GFP_KERNEL);
- if (pd == NULL) {
- dev_err(&dev->dev, "cannot allocate private data\n");
+ pd = devm_kzalloc(&dev->dev, sizeof(struct sh_mobile_i2c_data), GFP_KERNEL);
+ if (!pd)
return -ENOMEM;
- }
- pd->clk = clk_get(&dev->dev, NULL);
+ pd->clk = devm_clk_get(&dev->dev, NULL);
if (IS_ERR(pd->clk)) {
dev_err(&dev->dev, "cannot get clock\n");
- ret = PTR_ERR(pd->clk);
- goto err;
+ return PTR_ERR(pd->clk);
}
- ret = sh_mobile_i2c_hook_irqs(dev, 1);
- if (ret) {
- dev_err(&dev->dev, "cannot request IRQ\n");
- goto err_clk;
- }
+ ret = sh_mobile_i2c_hook_irqs(dev);
+ if (ret)
+ return ret;
pd->dev = &dev->dev;
platform_set_drvdata(dev, pd);
res = platform_get_resource(dev, IORESOURCE_MEM, 0);
- if (res == NULL) {
- dev_err(&dev->dev, "cannot find IO resource\n");
- ret = -ENOENT;
- goto err_irq;
- }
- size = resource_size(res);
-
- pd->reg = ioremap(res->start, size);
- if (pd->reg == NULL) {
- dev_err(&dev->dev, "cannot map IO\n");
- ret = -ENXIO;
- goto err_irq;
- }
+ pd->reg = devm_ioremap_resource(&dev->dev, res);
+ if (IS_ERR(pd->reg))
+ return PTR_ERR(pd->reg);
/* Use platform data bus speed or STANDARD_MODE */
- pd->bus_speed = STANDARD_MODE;
- if (pdata && pdata->bus_speed)
- pd->bus_speed = pdata->bus_speed;
+ ret = of_property_read_u32(dev->dev.of_node, "clock-frequency", &bus_speed);
+ pd->bus_speed = ret ? STANDARD_MODE : bus_speed;
+
pd->clks_per_count = 1;
- if (pdata && pdata->clks_per_count)
- pd->clks_per_count = pdata->clks_per_count;
+
+ if (dev->dev.of_node) {
+ const struct of_device_id *match;
+
+ match = of_match_device(sh_mobile_i2c_dt_ids, &dev->dev);
+ if (match) {
+ const struct sh_mobile_dt_config *config;
+
+ config = match->data;
+ pd->clks_per_count = config->clks_per_count;
+ }
+ } else {
+ if (pdata && pdata->bus_speed)
+ pd->bus_speed = pdata->bus_speed;
+ if (pdata && pdata->clks_per_count)
+ pd->clks_per_count = pdata->clks_per_count;
+ }
/* The IIC blocks on SH-Mobile ARM processors
* come with two new bits in ICIC.
*/
- if (size > 0x17)
+ if (resource_size(res) > 0x17)
pd->flags |= IIC_FLAG_HAS_ICIC67;
- sh_mobile_i2c_init(pd);
+ ret = sh_mobile_i2c_init(pd);
+ if (ret)
+ return ret;
/* Enable Runtime PM for this device.
*
@@ -750,24 +759,14 @@
ret = i2c_add_numbered_adapter(adap);
if (ret < 0) {
dev_err(&dev->dev, "cannot add numbered adapter\n");
- goto err_all;
+ return ret;
}
dev_info(&dev->dev,
- "I2C adapter %d with bus speed %lu Hz (L/H=%x/%x)\n",
+ "I2C adapter %d with bus speed %lu Hz (L/H=0x%x/0x%x)\n",
adap->nr, pd->bus_speed, pd->iccl, pd->icch);
return 0;
-
- err_all:
- iounmap(pd->reg);
- err_irq:
- sh_mobile_i2c_hook_irqs(dev, 0);
- err_clk:
- clk_put(pd->clk);
- err:
- kfree(pd);
- return ret;
}
static int sh_mobile_i2c_remove(struct platform_device *dev)
@@ -775,11 +774,7 @@
struct sh_mobile_i2c_data *pd = platform_get_drvdata(dev);
i2c_del_adapter(&pd->adap);
- iounmap(pd->reg);
- sh_mobile_i2c_hook_irqs(dev, 0);
- clk_put(pd->clk);
pm_runtime_disable(&dev->dev);
- kfree(pd);
return 0;
}
@@ -800,12 +795,6 @@
.runtime_resume = sh_mobile_i2c_runtime_nop,
};
-static const struct of_device_id sh_mobile_i2c_dt_ids[] = {
- { .compatible = "renesas,rmobile-iic", },
- {},
-};
-MODULE_DEVICE_TABLE(of, sh_mobile_i2c_dt_ids);
-
static struct platform_driver sh_mobile_i2c_driver = {
.driver = {
.name = "i2c-sh_mobile",
diff --git a/drivers/i2c/busses/i2c-simtec.c b/drivers/i2c/busses/i2c-simtec.c
index 294c80f..964e5c6 100644
--- a/drivers/i2c/busses/i2c-simtec.c
+++ b/drivers/i2c/busses/i2c-simtec.c
@@ -77,10 +77,8 @@
int ret;
pd = kzalloc(sizeof(struct simtec_i2c_data), GFP_KERNEL);
- if (pd == NULL) {
- dev_err(&dev->dev, "cannot allocate private data\n");
+ if (pd == NULL)
return -ENOMEM;
- }
platform_set_drvdata(dev, pd);
diff --git a/drivers/i2c/busses/i2c-sirf.c b/drivers/i2c/busses/i2c-sirf.c
index 8e3be7e..a3216de 100644
--- a/drivers/i2c/busses/i2c-sirf.c
+++ b/drivers/i2c/busses/i2c-sirf.c
@@ -307,7 +307,6 @@
siic = devm_kzalloc(&pdev->dev, sizeof(*siic), GFP_KERNEL);
if (!siic) {
- dev_err(&pdev->dev, "Can't allocate driver data\n");
err = -ENOMEM;
goto out;
}
diff --git a/drivers/i2c/busses/i2c-st.c b/drivers/i2c/busses/i2c-st.c
index 8720161..95b94767 100644
--- a/drivers/i2c/busses/i2c-st.c
+++ b/drivers/i2c/busses/i2c-st.c
@@ -847,7 +847,7 @@
return 0;
}
-static struct of_device_id st_i2c_match[] = {
+static const struct of_device_id st_i2c_match[] = {
{ .compatible = "st,comms-ssc-i2c", },
{ .compatible = "st,comms-ssc4-i2c", },
{},
diff --git a/drivers/i2c/busses/i2c-stu300.c b/drivers/i2c/busses/i2c-stu300.c
index 29b1fb7..fefb1c1 100644
--- a/drivers/i2c/busses/i2c-stu300.c
+++ b/drivers/i2c/busses/i2c-stu300.c
@@ -868,10 +868,8 @@
int ret = 0;
dev = devm_kzalloc(&pdev->dev, sizeof(struct stu300_dev), GFP_KERNEL);
- if (!dev) {
- dev_err(&pdev->dev, "could not allocate device struct\n");
+ if (!dev)
return -ENOMEM;
- }
bus_nr = pdev->id;
dev->clk = devm_clk_get(&pdev->dev, NULL);
diff --git a/drivers/i2c/busses/i2c-tegra.c b/drivers/i2c/busses/i2c-tegra.c
index 00f04cb..f1bb2fc 100644
--- a/drivers/i2c/busses/i2c-tegra.c
+++ b/drivers/i2c/busses/i2c-tegra.c
@@ -732,10 +732,8 @@
}
i2c_dev = devm_kzalloc(&pdev->dev, sizeof(*i2c_dev), GFP_KERNEL);
- if (!i2c_dev) {
- dev_err(&pdev->dev, "Could not allocate struct tegra_i2c_dev");
+ if (!i2c_dev)
return -ENOMEM;
- }
i2c_dev->base = base;
i2c_dev->div_clk = div_clk;
diff --git a/drivers/i2c/busses/i2c-wmt.c b/drivers/i2c/busses/i2c-wmt.c
index 2c8a3e4..f80a38c2 100644
--- a/drivers/i2c/busses/i2c-wmt.c
+++ b/drivers/i2c/busses/i2c-wmt.c
@@ -379,10 +379,8 @@
u32 clk_rate;
i2c_dev = devm_kzalloc(&pdev->dev, sizeof(*i2c_dev), GFP_KERNEL);
- if (!i2c_dev) {
- dev_err(&pdev->dev, "device memory allocation failed\n");
+ if (!i2c_dev)
return -ENOMEM;
- }
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
i2c_dev->base = devm_ioremap_resource(&pdev->dev, res);
@@ -454,7 +452,7 @@
return 0;
}
-static struct of_device_id wmt_i2c_dt_ids[] = {
+static const struct of_device_id wmt_i2c_dt_ids[] = {
{ .compatible = "wm,wm8505-i2c" },
{ /* Sentinel */ },
};
diff --git a/drivers/i2c/busses/scx200_acb.c b/drivers/i2c/busses/scx200_acb.c
index cb66f95..ff3f574 100644
--- a/drivers/i2c/busses/scx200_acb.c
+++ b/drivers/i2c/busses/scx200_acb.c
@@ -431,10 +431,8 @@
struct i2c_adapter *adapter;
iface = kzalloc(sizeof(*iface), GFP_KERNEL);
- if (!iface) {
- pr_err("can't allocate memory\n");
+ if (!iface)
return NULL;
- }
adapter = &iface->adapter;
i2c_set_adapdata(adapter, iface);
diff --git a/drivers/i2c/muxes/i2c-mux-pca954x.c b/drivers/i2c/muxes/i2c-mux-pca954x.c
index 550bd36..9bd4212 100644
--- a/drivers/i2c/muxes/i2c-mux-pca954x.c
+++ b/drivers/i2c/muxes/i2c-mux-pca954x.c
@@ -36,12 +36,11 @@
*/
#include <linux/device.h>
-#include <linux/gpio.h>
+#include <linux/gpio/consumer.h>
#include <linux/i2c.h>
#include <linux/i2c-mux.h>
#include <linux/i2c/pca954x.h>
#include <linux/module.h>
-#include <linux/of_gpio.h>
#include <linux/slab.h>
#define PCA954X_MAX_NCHANS 8
@@ -186,7 +185,7 @@
{
struct i2c_adapter *adap = to_i2c_adapter(client->dev.parent);
struct pca954x_platform_data *pdata = dev_get_platdata(&client->dev);
- struct device_node *np = client->dev.of_node;
+ struct gpio_desc *gpio;
int num, force, class;
struct pca954x *data;
int ret;
@@ -200,21 +199,10 @@
i2c_set_clientdata(client, data);
- if (IS_ENABLED(CONFIG_OF) && np) {
- enum of_gpio_flags flags;
- int gpio;
-
- /* Get the mux out of reset if a reset GPIO is specified. */
- gpio = of_get_named_gpio_flags(np, "reset-gpio", 0, &flags);
- if (gpio_is_valid(gpio)) {
- ret = devm_gpio_request_one(&client->dev, gpio,
- flags & OF_GPIO_ACTIVE_LOW ?
- GPIOF_OUT_INIT_HIGH : GPIOF_OUT_INIT_LOW,
- "pca954x reset");
- if (ret < 0)
- return ret;
- }
- }
+ /* Get the mux out of reset if a reset GPIO is specified. */
+ gpio = devm_gpiod_get(&client->dev, "reset");
+ if (!IS_ERR(gpio))
+ gpiod_direction_output(gpio, 0);
/* Write the mux register at addr to verify
* that the mux is in fact present. This also
diff --git a/drivers/infiniband/Makefile b/drivers/infiniband/Makefile
index bf508b5..dc21836 100644
--- a/drivers/infiniband/Makefile
+++ b/drivers/infiniband/Makefile
@@ -1,18 +1,3 @@
obj-$(CONFIG_INFINIBAND) += core/
-obj-$(CONFIG_INFINIBAND_MTHCA) += hw/mthca/
-obj-$(CONFIG_INFINIBAND_IPATH) += hw/ipath/
-obj-$(CONFIG_INFINIBAND_QIB) += hw/qib/
-obj-$(CONFIG_INFINIBAND_EHCA) += hw/ehca/
-obj-$(CONFIG_INFINIBAND_AMSO1100) += hw/amso1100/
-obj-$(CONFIG_INFINIBAND_CXGB3) += hw/cxgb3/
-obj-$(CONFIG_INFINIBAND_CXGB4) += hw/cxgb4/
-obj-$(CONFIG_MLX4_INFINIBAND) += hw/mlx4/
-obj-$(CONFIG_MLX5_INFINIBAND) += hw/mlx5/
-obj-$(CONFIG_INFINIBAND_NES) += hw/nes/
-obj-$(CONFIG_INFINIBAND_OCRDMA) += hw/ocrdma/
-obj-$(CONFIG_INFINIBAND_USNIC) += hw/usnic/
-obj-$(CONFIG_INFINIBAND_IPOIB) += ulp/ipoib/
-obj-$(CONFIG_INFINIBAND_SRP) += ulp/srp/
-obj-$(CONFIG_INFINIBAND_SRPT) += ulp/srpt/
-obj-$(CONFIG_INFINIBAND_ISER) += ulp/iser/
-obj-$(CONFIG_INFINIBAND_ISERT) += ulp/isert/
+obj-$(CONFIG_INFINIBAND) += hw/
+obj-$(CONFIG_INFINIBAND) += ulp/
diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
index 3ab3865..ffd0af6 100644
--- a/drivers/infiniband/core/Makefile
+++ b/drivers/infiniband/core/Makefile
@@ -18,7 +18,7 @@
ib_cm-y := cm.o
-iw_cm-y := iwcm.o
+iw_cm-y := iwcm.o iwpm_util.o iwpm_msg.o
rdma_cm-y := cma.o
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 42c3058..d570030 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -3607,7 +3607,8 @@
id_stats = ibnl_put_msg(skb, &nlh, cb->nlh->nlmsg_seq,
sizeof *id_stats, RDMA_NL_RDMA_CM,
- RDMA_NL_RDMA_CM_ID_STATS);
+ RDMA_NL_RDMA_CM_ID_STATS,
+ NLM_F_MULTI);
if (!id_stats)
goto out;
diff --git a/drivers/infiniband/core/iwpm_msg.c b/drivers/infiniband/core/iwpm_msg.c
new file mode 100644
index 0000000..b85ddbc
--- /dev/null
+++ b/drivers/infiniband/core/iwpm_msg.c
@@ -0,0 +1,685 @@
+/*
+ * Copyright (c) 2014 Intel Corporation. All rights reserved.
+ * Copyright (c) 2014 Chelsio, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "iwpm_util.h"
+
+static const char iwpm_ulib_name[] = "iWarpPortMapperUser";
+static int iwpm_ulib_version = 3;
+static int iwpm_user_pid = IWPM_PID_UNDEFINED;
+static atomic_t echo_nlmsg_seq;
+
+int iwpm_valid_pid(void)
+{
+ return iwpm_user_pid > 0;
+}
+EXPORT_SYMBOL(iwpm_valid_pid);
+
+/*
+ * iwpm_register_pid - Send a netlink query to user space
+ * for the iwarp port mapper pid
+ *
+ * nlmsg attributes:
+ * [IWPM_NLA_REG_PID_SEQ]
+ * [IWPM_NLA_REG_IF_NAME]
+ * [IWPM_NLA_REG_IBDEV_NAME]
+ * [IWPM_NLA_REG_ULIB_NAME]
+ */
+int iwpm_register_pid(struct iwpm_dev_data *pm_msg, u8 nl_client)
+{
+ struct sk_buff *skb = NULL;
+ struct iwpm_nlmsg_request *nlmsg_request = NULL;
+ struct nlmsghdr *nlh;
+ u32 msg_seq;
+ const char *err_str = "";
+ int ret = -EINVAL;
+
+ if (!iwpm_valid_client(nl_client)) {
+ err_str = "Invalid port mapper client";
+ goto pid_query_error;
+ }
+ if (iwpm_registered_client(nl_client))
+ return 0;
+ skb = iwpm_create_nlmsg(RDMA_NL_IWPM_REG_PID, &nlh, nl_client);
+ if (!skb) {
+ err_str = "Unable to create a nlmsg";
+ goto pid_query_error;
+ }
+ nlh->nlmsg_seq = iwpm_get_nlmsg_seq();
+ nlmsg_request = iwpm_get_nlmsg_request(nlh->nlmsg_seq, nl_client, GFP_KERNEL);
+ if (!nlmsg_request) {
+ err_str = "Unable to allocate netlink request";
+ goto pid_query_error;
+ }
+ msg_seq = atomic_read(&echo_nlmsg_seq);
+
+ /* fill in the pid request message */
+ err_str = "Unable to put attribute of the nlmsg";
+ ret = ibnl_put_attr(skb, nlh, sizeof(u32), &msg_seq, IWPM_NLA_REG_PID_SEQ);
+ if (ret)
+ goto pid_query_error;
+ ret = ibnl_put_attr(skb, nlh, IWPM_IFNAME_SIZE,
+ pm_msg->if_name, IWPM_NLA_REG_IF_NAME);
+ if (ret)
+ goto pid_query_error;
+ ret = ibnl_put_attr(skb, nlh, IWPM_DEVNAME_SIZE,
+ pm_msg->dev_name, IWPM_NLA_REG_IBDEV_NAME);
+ if (ret)
+ goto pid_query_error;
+ ret = ibnl_put_attr(skb, nlh, IWPM_ULIBNAME_SIZE,
+ (char *)iwpm_ulib_name, IWPM_NLA_REG_ULIB_NAME);
+ if (ret)
+ goto pid_query_error;
+
+ pr_debug("%s: Multicasting a nlmsg (dev = %s ifname = %s iwpm = %s)\n",
+ __func__, pm_msg->dev_name, pm_msg->if_name, iwpm_ulib_name);
+
+ ret = ibnl_multicast(skb, nlh, RDMA_NL_GROUP_IWPM, GFP_KERNEL);
+ if (ret) {
+ skb = NULL; /* skb is freed in the netlink send-op handling */
+ iwpm_set_registered(nl_client, 1);
+ iwpm_user_pid = IWPM_PID_UNAVAILABLE;
+ err_str = "Unable to send a nlmsg";
+ goto pid_query_error;
+ }
+ nlmsg_request->req_buffer = pm_msg;
+ ret = iwpm_wait_complete_req(nlmsg_request);
+ return ret;
+pid_query_error:
+ pr_info("%s: %s (client = %d)\n", __func__, err_str, nl_client);
+ if (skb)
+ dev_kfree_skb(skb);
+ if (nlmsg_request)
+ iwpm_free_nlmsg_request(&nlmsg_request->kref);
+ return ret;
+}
+EXPORT_SYMBOL(iwpm_register_pid);
+
+/*
+ * iwpm_add_mapping - Send a netlink add mapping message
+ * to the port mapper
+ * nlmsg attributes:
+ * [IWPM_NLA_MANAGE_MAPPING_SEQ]
+ * [IWPM_NLA_MANAGE_ADDR]
+ */
+int iwpm_add_mapping(struct iwpm_sa_data *pm_msg, u8 nl_client)
+{
+ struct sk_buff *skb = NULL;
+ struct iwpm_nlmsg_request *nlmsg_request = NULL;
+ struct nlmsghdr *nlh;
+ u32 msg_seq;
+ const char *err_str = "";
+ int ret = -EINVAL;
+
+ if (!iwpm_valid_client(nl_client)) {
+ err_str = "Invalid port mapper client";
+ goto add_mapping_error;
+ }
+ if (!iwpm_registered_client(nl_client)) {
+ err_str = "Unregistered port mapper client";
+ goto add_mapping_error;
+ }
+ if (!iwpm_valid_pid())
+ return 0;
+ skb = iwpm_create_nlmsg(RDMA_NL_IWPM_ADD_MAPPING, &nlh, nl_client);
+ if (!skb) {
+ err_str = "Unable to create a nlmsg";
+ goto add_mapping_error;
+ }
+ nlh->nlmsg_seq = iwpm_get_nlmsg_seq();
+ nlmsg_request = iwpm_get_nlmsg_request(nlh->nlmsg_seq, nl_client, GFP_KERNEL);
+ if (!nlmsg_request) {
+ err_str = "Unable to allocate netlink request";
+ goto add_mapping_error;
+ }
+ msg_seq = atomic_read(&echo_nlmsg_seq);
+ /* fill in the add mapping message */
+ err_str = "Unable to put attribute of the nlmsg";
+ ret = ibnl_put_attr(skb, nlh, sizeof(u32), &msg_seq,
+ IWPM_NLA_MANAGE_MAPPING_SEQ);
+ if (ret)
+ goto add_mapping_error;
+ ret = ibnl_put_attr(skb, nlh, sizeof(struct sockaddr_storage),
+ &pm_msg->loc_addr, IWPM_NLA_MANAGE_ADDR);
+ if (ret)
+ goto add_mapping_error;
+ nlmsg_request->req_buffer = pm_msg;
+
+ ret = ibnl_unicast(skb, nlh, iwpm_user_pid);
+ if (ret) {
+ skb = NULL; /* skb is freed in the netlink send-op handling */
+ iwpm_user_pid = IWPM_PID_UNDEFINED;
+ err_str = "Unable to send a nlmsg";
+ goto add_mapping_error;
+ }
+ ret = iwpm_wait_complete_req(nlmsg_request);
+ return ret;
+add_mapping_error:
+ pr_info("%s: %s (client = %d)\n", __func__, err_str, nl_client);
+ if (skb)
+ dev_kfree_skb(skb);
+ if (nlmsg_request)
+ iwpm_free_nlmsg_request(&nlmsg_request->kref);
+ return ret;
+}
+EXPORT_SYMBOL(iwpm_add_mapping);
+
+/*
+ * iwpm_add_and_query_mapping - Send a netlink add and query
+ * mapping message to the port mapper
+ * nlmsg attributes:
+ * [IWPM_NLA_QUERY_MAPPING_SEQ]
+ * [IWPM_NLA_QUERY_LOCAL_ADDR]
+ * [IWPM_NLA_QUERY_REMOTE_ADDR]
+ */
+int iwpm_add_and_query_mapping(struct iwpm_sa_data *pm_msg, u8 nl_client)
+{
+ struct sk_buff *skb = NULL;
+ struct iwpm_nlmsg_request *nlmsg_request = NULL;
+ struct nlmsghdr *nlh;
+ u32 msg_seq;
+ const char *err_str = "";
+ int ret = -EINVAL;
+
+ if (!iwpm_valid_client(nl_client)) {
+ err_str = "Invalid port mapper client";
+ goto query_mapping_error;
+ }
+ if (!iwpm_registered_client(nl_client)) {
+ err_str = "Unregistered port mapper client";
+ goto query_mapping_error;
+ }
+ if (!iwpm_valid_pid())
+ return 0;
+ ret = -ENOMEM;
+ skb = iwpm_create_nlmsg(RDMA_NL_IWPM_QUERY_MAPPING, &nlh, nl_client);
+ if (!skb) {
+ err_str = "Unable to create a nlmsg";
+ goto query_mapping_error;
+ }
+ nlh->nlmsg_seq = iwpm_get_nlmsg_seq();
+ nlmsg_request = iwpm_get_nlmsg_request(nlh->nlmsg_seq,
+ nl_client, GFP_KERNEL);
+ if (!nlmsg_request) {
+ err_str = "Unable to allocate netlink request";
+ goto query_mapping_error;
+ }
+ msg_seq = atomic_read(&echo_nlmsg_seq);
+
+ /* fill in the query message */
+ err_str = "Unable to put attribute of the nlmsg";
+ ret = ibnl_put_attr(skb, nlh, sizeof(u32), &msg_seq,
+ IWPM_NLA_QUERY_MAPPING_SEQ);
+ if (ret)
+ goto query_mapping_error;
+ ret = ibnl_put_attr(skb, nlh, sizeof(struct sockaddr_storage),
+ &pm_msg->loc_addr, IWPM_NLA_QUERY_LOCAL_ADDR);
+ if (ret)
+ goto query_mapping_error;
+ ret = ibnl_put_attr(skb, nlh, sizeof(struct sockaddr_storage),
+ &pm_msg->rem_addr, IWPM_NLA_QUERY_REMOTE_ADDR);
+ if (ret)
+ goto query_mapping_error;
+ nlmsg_request->req_buffer = pm_msg;
+
+ ret = ibnl_unicast(skb, nlh, iwpm_user_pid);
+ if (ret) {
+ skb = NULL; /* skb is freed in the netlink send-op handling */
+ err_str = "Unable to send a nlmsg";
+ goto query_mapping_error;
+ }
+ ret = iwpm_wait_complete_req(nlmsg_request);
+ return ret;
+query_mapping_error:
+ pr_info("%s: %s (client = %d)\n", __func__, err_str, nl_client);
+ if (skb)
+ dev_kfree_skb(skb);
+ if (nlmsg_request)
+ iwpm_free_nlmsg_request(&nlmsg_request->kref);
+ return ret;
+}
+EXPORT_SYMBOL(iwpm_add_and_query_mapping);
+
+/*
+ * iwpm_remove_mapping - Send a netlink remove mapping message
+ * to the port mapper
+ * nlmsg attributes:
+ * [IWPM_NLA_MANAGE_MAPPING_SEQ]
+ * [IWPM_NLA_MANAGE_ADDR]
+ */
+int iwpm_remove_mapping(struct sockaddr_storage *local_addr, u8 nl_client)
+{
+ struct sk_buff *skb = NULL;
+ struct nlmsghdr *nlh;
+ u32 msg_seq;
+ const char *err_str = "";
+ int ret = -EINVAL;
+
+ if (!iwpm_valid_client(nl_client)) {
+ err_str = "Invalid port mapper client";
+ goto remove_mapping_error;
+ }
+ if (!iwpm_registered_client(nl_client)) {
+ err_str = "Unregistered port mapper client";
+ goto remove_mapping_error;
+ }
+ if (!iwpm_valid_pid())
+ return 0;
+ skb = iwpm_create_nlmsg(RDMA_NL_IWPM_REMOVE_MAPPING, &nlh, nl_client);
+ if (!skb) {
+ ret = -ENOMEM;
+ err_str = "Unable to create a nlmsg";
+ goto remove_mapping_error;
+ }
+ msg_seq = atomic_read(&echo_nlmsg_seq);
+ nlh->nlmsg_seq = iwpm_get_nlmsg_seq();
+ err_str = "Unable to put attribute of the nlmsg";
+ ret = ibnl_put_attr(skb, nlh, sizeof(u32), &msg_seq,
+ IWPM_NLA_MANAGE_MAPPING_SEQ);
+ if (ret)
+ goto remove_mapping_error;
+ ret = ibnl_put_attr(skb, nlh, sizeof(struct sockaddr_storage),
+ local_addr, IWPM_NLA_MANAGE_ADDR);
+ if (ret)
+ goto remove_mapping_error;
+
+ ret = ibnl_unicast(skb, nlh, iwpm_user_pid);
+ if (ret) {
+ skb = NULL; /* skb is freed in the netlink send-op handling */
+ iwpm_user_pid = IWPM_PID_UNDEFINED;
+ err_str = "Unable to send a nlmsg";
+ goto remove_mapping_error;
+ }
+ iwpm_print_sockaddr(local_addr,
+ "remove_mapping: Local sockaddr:");
+ return 0;
+remove_mapping_error:
+ pr_info("%s: %s (client = %d)\n", __func__, err_str, nl_client);
+ if (skb)
+ dev_kfree_skb_any(skb);
+ return ret;
+}
+EXPORT_SYMBOL(iwpm_remove_mapping);
+
+/* netlink attribute policy for the received response to register pid request */
+static const struct nla_policy resp_reg_policy[IWPM_NLA_RREG_PID_MAX] = {
+ [IWPM_NLA_RREG_PID_SEQ] = { .type = NLA_U32 },
+ [IWPM_NLA_RREG_IBDEV_NAME] = { .type = NLA_STRING,
+ .len = IWPM_DEVNAME_SIZE - 1 },
+ [IWPM_NLA_RREG_ULIB_NAME] = { .type = NLA_STRING,
+ .len = IWPM_ULIBNAME_SIZE - 1 },
+ [IWPM_NLA_RREG_ULIB_VER] = { .type = NLA_U16 },
+ [IWPM_NLA_RREG_PID_ERR] = { .type = NLA_U16 }
+};
+
+/*
+ * iwpm_register_pid_cb - Process a port mapper response to
+ * iwpm_register_pid()
+ */
+int iwpm_register_pid_cb(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ struct iwpm_nlmsg_request *nlmsg_request = NULL;
+ struct nlattr *nltb[IWPM_NLA_RREG_PID_MAX];
+ struct iwpm_dev_data *pm_msg;
+ char *dev_name, *iwpm_name;
+ u32 msg_seq;
+ u8 nl_client;
+ u16 iwpm_version;
+ const char *msg_type = "Register Pid response";
+
+ if (iwpm_parse_nlmsg(cb, IWPM_NLA_RREG_PID_MAX,
+ resp_reg_policy, nltb, msg_type))
+ return -EINVAL;
+
+ msg_seq = nla_get_u32(nltb[IWPM_NLA_RREG_PID_SEQ]);
+ nlmsg_request = iwpm_find_nlmsg_request(msg_seq);
+ if (!nlmsg_request) {
+ pr_info("%s: Could not find a matching request (seq = %u)\n",
+ __func__, msg_seq);
+ return -EINVAL;
+ }
+ pm_msg = nlmsg_request->req_buffer;
+ nl_client = nlmsg_request->nl_client;
+ dev_name = (char *)nla_data(nltb[IWPM_NLA_RREG_IBDEV_NAME]);
+ iwpm_name = (char *)nla_data(nltb[IWPM_NLA_RREG_ULIB_NAME]);
+ iwpm_version = nla_get_u16(nltb[IWPM_NLA_RREG_ULIB_VER]);
+
+ /* check device name, ulib name and version */
+ if (strcmp(pm_msg->dev_name, dev_name) ||
+ strcmp(iwpm_ulib_name, iwpm_name) ||
+ iwpm_version != iwpm_ulib_version) {
+
+ pr_info("%s: Incorrect info (dev = %s name = %s version = %d)\n",
+ __func__, dev_name, iwpm_name, iwpm_version);
+ nlmsg_request->err_code = IWPM_USER_LIB_INFO_ERR;
+ goto register_pid_response_exit;
+ }
+ iwpm_user_pid = cb->nlh->nlmsg_pid;
+ atomic_set(&echo_nlmsg_seq, cb->nlh->nlmsg_seq);
+ pr_debug("%s: iWarp Port Mapper (pid = %d) is available!\n",
+ __func__, iwpm_user_pid);
+ if (iwpm_valid_client(nl_client))
+ iwpm_set_registered(nl_client, 1);
+register_pid_response_exit:
+ nlmsg_request->request_done = 1;
+ /* always for found nlmsg_request */
+ kref_put(&nlmsg_request->kref, iwpm_free_nlmsg_request);
+ barrier();
+ wake_up(&nlmsg_request->waitq);
+ return 0;
+}
+EXPORT_SYMBOL(iwpm_register_pid_cb);
+
+/* netlink attribute policy for the received response to add mapping request */
+static const struct nla_policy resp_add_policy[IWPM_NLA_RMANAGE_MAPPING_MAX] = {
+ [IWPM_NLA_MANAGE_MAPPING_SEQ] = { .type = NLA_U32 },
+ [IWPM_NLA_MANAGE_ADDR] = { .len = sizeof(struct sockaddr_storage) },
+ [IWPM_NLA_MANAGE_MAPPED_LOC_ADDR] = { .len = sizeof(struct sockaddr_storage) },
+ [IWPM_NLA_RMANAGE_MAPPING_ERR] = { .type = NLA_U16 }
+};
+
+/*
+ * iwpm_add_mapping_cb - Process a port mapper response to
+ * iwpm_add_mapping()
+ */
+int iwpm_add_mapping_cb(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ struct iwpm_sa_data *pm_msg;
+ struct iwpm_nlmsg_request *nlmsg_request = NULL;
+ struct nlattr *nltb[IWPM_NLA_RMANAGE_MAPPING_MAX];
+ struct sockaddr_storage *local_sockaddr;
+ struct sockaddr_storage *mapped_sockaddr;
+ const char *msg_type;
+ u32 msg_seq;
+
+ msg_type = "Add Mapping response";
+ if (iwpm_parse_nlmsg(cb, IWPM_NLA_RMANAGE_MAPPING_MAX,
+ resp_add_policy, nltb, msg_type))
+ return -EINVAL;
+
+ atomic_set(&echo_nlmsg_seq, cb->nlh->nlmsg_seq);
+
+ msg_seq = nla_get_u32(nltb[IWPM_NLA_MANAGE_MAPPING_SEQ]);
+ nlmsg_request = iwpm_find_nlmsg_request(msg_seq);
+ if (!nlmsg_request) {
+ pr_info("%s: Could not find a matching request (seq = %u)\n",
+ __func__, msg_seq);
+ return -EINVAL;
+ }
+ pm_msg = nlmsg_request->req_buffer;
+ local_sockaddr = (struct sockaddr_storage *)
+ nla_data(nltb[IWPM_NLA_MANAGE_ADDR]);
+ mapped_sockaddr = (struct sockaddr_storage *)
+ nla_data(nltb[IWPM_NLA_MANAGE_MAPPED_LOC_ADDR]);
+
+ if (iwpm_compare_sockaddr(local_sockaddr, &pm_msg->loc_addr)) {
+ nlmsg_request->err_code = IWPM_USER_LIB_INFO_ERR;
+ goto add_mapping_response_exit;
+ }
+ if (mapped_sockaddr->ss_family != local_sockaddr->ss_family) {
+ pr_info("%s: Sockaddr family doesn't match the requested one\n",
+ __func__);
+ nlmsg_request->err_code = IWPM_USER_LIB_INFO_ERR;
+ goto add_mapping_response_exit;
+ }
+ memcpy(&pm_msg->mapped_loc_addr, mapped_sockaddr,
+ sizeof(*mapped_sockaddr));
+ iwpm_print_sockaddr(&pm_msg->loc_addr,
+ "add_mapping: Local sockaddr:");
+ iwpm_print_sockaddr(&pm_msg->mapped_loc_addr,
+ "add_mapping: Mapped local sockaddr:");
+
+add_mapping_response_exit:
+ nlmsg_request->request_done = 1;
+ /* always for found request */
+ kref_put(&nlmsg_request->kref, iwpm_free_nlmsg_request);
+ barrier();
+ wake_up(&nlmsg_request->waitq);
+ return 0;
+}
+EXPORT_SYMBOL(iwpm_add_mapping_cb);
+
+/* netlink attribute policy for the response to add and query mapping request */
+static const struct nla_policy resp_query_policy[IWPM_NLA_RQUERY_MAPPING_MAX] = {
+ [IWPM_NLA_QUERY_MAPPING_SEQ] = { .type = NLA_U32 },
+ [IWPM_NLA_QUERY_LOCAL_ADDR] = { .len = sizeof(struct sockaddr_storage) },
+ [IWPM_NLA_QUERY_REMOTE_ADDR] = { .len = sizeof(struct sockaddr_storage) },
+ [IWPM_NLA_RQUERY_MAPPED_LOC_ADDR] = { .len = sizeof(struct sockaddr_storage) },
+ [IWPM_NLA_RQUERY_MAPPED_REM_ADDR] = { .len = sizeof(struct sockaddr_storage) },
+ [IWPM_NLA_RQUERY_MAPPING_ERR] = { .type = NLA_U16 }
+};
+
+/*
+ * iwpm_add_and_query_mapping_cb - Process a port mapper response to
+ * iwpm_add_and_query_mapping()
+ */
+int iwpm_add_and_query_mapping_cb(struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ struct iwpm_sa_data *pm_msg;
+ struct iwpm_nlmsg_request *nlmsg_request = NULL;
+ struct nlattr *nltb[IWPM_NLA_RQUERY_MAPPING_MAX];
+ struct sockaddr_storage *local_sockaddr, *remote_sockaddr;
+ struct sockaddr_storage *mapped_loc_sockaddr, *mapped_rem_sockaddr;
+ const char *msg_type;
+ u32 msg_seq;
+ u16 err_code;
+
+ msg_type = "Query Mapping response";
+ if (iwpm_parse_nlmsg(cb, IWPM_NLA_RQUERY_MAPPING_MAX,
+ resp_query_policy, nltb, msg_type))
+ return -EINVAL;
+ atomic_set(&echo_nlmsg_seq, cb->nlh->nlmsg_seq);
+
+ msg_seq = nla_get_u32(nltb[IWPM_NLA_QUERY_MAPPING_SEQ]);
+ nlmsg_request = iwpm_find_nlmsg_request(msg_seq);
+ if (!nlmsg_request) {
+ pr_info("%s: Could not find a matching request (seq = %u)\n",
+ __func__, msg_seq);
+ return -EINVAL;
+ }
+ pm_msg = nlmsg_request->req_buffer;
+ local_sockaddr = (struct sockaddr_storage *)
+ nla_data(nltb[IWPM_NLA_QUERY_LOCAL_ADDR]);
+ remote_sockaddr = (struct sockaddr_storage *)
+ nla_data(nltb[IWPM_NLA_QUERY_REMOTE_ADDR]);
+ mapped_loc_sockaddr = (struct sockaddr_storage *)
+ nla_data(nltb[IWPM_NLA_RQUERY_MAPPED_LOC_ADDR]);
+ mapped_rem_sockaddr = (struct sockaddr_storage *)
+ nla_data(nltb[IWPM_NLA_RQUERY_MAPPED_REM_ADDR]);
+
+ err_code = nla_get_u16(nltb[IWPM_NLA_RQUERY_MAPPING_ERR]);
+ if (err_code == IWPM_REMOTE_QUERY_REJECT) {
+ pr_info("%s: Received a Reject (pid = %u, echo seq = %u)\n",
+ __func__, cb->nlh->nlmsg_pid, msg_seq);
+ nlmsg_request->err_code = IWPM_REMOTE_QUERY_REJECT;
+ }
+ if (iwpm_compare_sockaddr(local_sockaddr, &pm_msg->loc_addr) ||
+ iwpm_compare_sockaddr(remote_sockaddr, &pm_msg->rem_addr)) {
+ pr_info("%s: Incorrect local sockaddr\n", __func__);
+ nlmsg_request->err_code = IWPM_USER_LIB_INFO_ERR;
+ goto query_mapping_response_exit;
+ }
+ if (mapped_loc_sockaddr->ss_family != local_sockaddr->ss_family ||
+ mapped_rem_sockaddr->ss_family != remote_sockaddr->ss_family) {
+ pr_info("%s: Sockaddr family doesn't match the requested one\n",
+ __func__);
+ nlmsg_request->err_code = IWPM_USER_LIB_INFO_ERR;
+ goto query_mapping_response_exit;
+ }
+ memcpy(&pm_msg->mapped_loc_addr, mapped_loc_sockaddr,
+ sizeof(*mapped_loc_sockaddr));
+ memcpy(&pm_msg->mapped_rem_addr, mapped_rem_sockaddr,
+ sizeof(*mapped_rem_sockaddr));
+
+ iwpm_print_sockaddr(&pm_msg->loc_addr,
+ "query_mapping: Local sockaddr:");
+ iwpm_print_sockaddr(&pm_msg->mapped_loc_addr,
+ "query_mapping: Mapped local sockaddr:");
+ iwpm_print_sockaddr(&pm_msg->rem_addr,
+ "query_mapping: Remote sockaddr:");
+ iwpm_print_sockaddr(&pm_msg->mapped_rem_addr,
+ "query_mapping: Mapped remote sockaddr:");
+query_mapping_response_exit:
+ nlmsg_request->request_done = 1;
+ /* always for found request */
+ kref_put(&nlmsg_request->kref, iwpm_free_nlmsg_request);
+ barrier();
+ wake_up(&nlmsg_request->waitq);
+ return 0;
+}
+EXPORT_SYMBOL(iwpm_add_and_query_mapping_cb);
+
+/* netlink attribute policy for the received request for mapping info */
+static const struct nla_policy resp_mapinfo_policy[IWPM_NLA_MAPINFO_REQ_MAX] = {
+ [IWPM_NLA_MAPINFO_ULIB_NAME] = { .type = NLA_STRING,
+ .len = IWPM_ULIBNAME_SIZE - 1 },
+ [IWPM_NLA_MAPINFO_ULIB_VER] = { .type = NLA_U16 }
+};
+
+/*
+ * iwpm_mapping_info_cb - Process a port mapper request for mapping info
+ */
+int iwpm_mapping_info_cb(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ struct nlattr *nltb[IWPM_NLA_MAPINFO_REQ_MAX];
+ const char *msg_type = "Mapping Info response";
+ int iwpm_pid;
+ u8 nl_client;
+ char *iwpm_name;
+ u16 iwpm_version;
+ int ret = -EINVAL;
+
+ if (iwpm_parse_nlmsg(cb, IWPM_NLA_MAPINFO_REQ_MAX,
+ resp_mapinfo_policy, nltb, msg_type)) {
+ pr_info("%s: Unable to parse nlmsg\n", __func__);
+ return ret;
+ }
+ iwpm_name = (char *)nla_data(nltb[IWPM_NLA_MAPINFO_ULIB_NAME]);
+ iwpm_version = nla_get_u16(nltb[IWPM_NLA_MAPINFO_ULIB_VER]);
+ if (strcmp(iwpm_ulib_name, iwpm_name) ||
+ iwpm_version != iwpm_ulib_version) {
+ pr_info("%s: Invalid port mapper name = %s version = %d\n",
+ __func__, iwpm_name, iwpm_version);
+ return ret;
+ }
+ nl_client = RDMA_NL_GET_CLIENT(cb->nlh->nlmsg_type);
+ if (!iwpm_valid_client(nl_client)) {
+ pr_info("%s: Invalid port mapper client = %d\n",
+ __func__, nl_client);
+ return ret;
+ }
+ iwpm_set_registered(nl_client, 0);
+ atomic_set(&echo_nlmsg_seq, cb->nlh->nlmsg_seq);
+ if (!iwpm_mapinfo_available())
+ return 0;
+ iwpm_pid = cb->nlh->nlmsg_pid;
+ pr_debug("%s: iWarp Port Mapper (pid = %d) is available!\n",
+ __func__, iwpm_pid);
+ ret = iwpm_send_mapinfo(nl_client, iwpm_pid);
+ return ret;
+}
+EXPORT_SYMBOL(iwpm_mapping_info_cb);
+
+/* netlink attribute policy for the received mapping info ack */
+static const struct nla_policy ack_mapinfo_policy[IWPM_NLA_MAPINFO_NUM_MAX] = {
+ [IWPM_NLA_MAPINFO_SEQ] = { .type = NLA_U32 },
+ [IWPM_NLA_MAPINFO_SEND_NUM] = { .type = NLA_U32 },
+ [IWPM_NLA_MAPINFO_ACK_NUM] = { .type = NLA_U32 }
+};
+
+/*
+ * iwpm_ack_mapping_info_cb - Process a port mapper ack for
+ * the provided mapping info records
+ */
+int iwpm_ack_mapping_info_cb(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ struct nlattr *nltb[IWPM_NLA_MAPINFO_NUM_MAX];
+ u32 mapinfo_send, mapinfo_ack;
+ const char *msg_type = "Mapping Info Ack";
+
+ if (iwpm_parse_nlmsg(cb, IWPM_NLA_MAPINFO_NUM_MAX,
+ ack_mapinfo_policy, nltb, msg_type))
+ return -EINVAL;
+ mapinfo_send = nla_get_u32(nltb[IWPM_NLA_MAPINFO_SEND_NUM]);
+ mapinfo_ack = nla_get_u32(nltb[IWPM_NLA_MAPINFO_ACK_NUM]);
+ if (mapinfo_ack != mapinfo_send)
+ pr_info("%s: Invalid mapinfo number (sent = %u ack-ed = %u)\n",
+ __func__, mapinfo_send, mapinfo_ack);
+ atomic_set(&echo_nlmsg_seq, cb->nlh->nlmsg_seq);
+ return 0;
+}
+EXPORT_SYMBOL(iwpm_ack_mapping_info_cb);
+
+/* netlink attribute policy for the received port mapper error message */
+static const struct nla_policy map_error_policy[IWPM_NLA_ERR_MAX] = {
+ [IWPM_NLA_ERR_SEQ] = { .type = NLA_U32 },
+ [IWPM_NLA_ERR_CODE] = { .type = NLA_U16 },
+};
+
+/*
+ * iwpm_mapping_error_cb - Process a port mapper error message
+ */
+int iwpm_mapping_error_cb(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ struct iwpm_nlmsg_request *nlmsg_request = NULL;
+ int nl_client = RDMA_NL_GET_CLIENT(cb->nlh->nlmsg_type);
+ struct nlattr *nltb[IWPM_NLA_ERR_MAX];
+ u32 msg_seq;
+ u16 err_code;
+ const char *msg_type = "Mapping Error Msg";
+
+ if (iwpm_parse_nlmsg(cb, IWPM_NLA_ERR_MAX,
+ map_error_policy, nltb, msg_type))
+ return -EINVAL;
+
+ msg_seq = nla_get_u32(nltb[IWPM_NLA_ERR_SEQ]);
+ err_code = nla_get_u16(nltb[IWPM_NLA_ERR_CODE]);
+ pr_info("%s: Received msg seq = %u err code = %u client = %d\n",
+ __func__, msg_seq, err_code, nl_client);
+ /* look for nlmsg_request */
+ nlmsg_request = iwpm_find_nlmsg_request(msg_seq);
+ if (!nlmsg_request) {
+ /* not all errors have associated requests */
+ pr_debug("Could not find matching req (seq = %u)\n", msg_seq);
+ return 0;
+ }
+ atomic_set(&echo_nlmsg_seq, cb->nlh->nlmsg_seq);
+ nlmsg_request->err_code = err_code;
+ nlmsg_request->request_done = 1;
+ /* always for found request */
+ kref_put(&nlmsg_request->kref, iwpm_free_nlmsg_request);
+ barrier();
+ wake_up(&nlmsg_request->waitq);
+ return 0;
+}
+EXPORT_SYMBOL(iwpm_mapping_error_cb);
diff --git a/drivers/infiniband/core/iwpm_util.c b/drivers/infiniband/core/iwpm_util.c
new file mode 100644
index 0000000..69e9f84
--- /dev/null
+++ b/drivers/infiniband/core/iwpm_util.c
@@ -0,0 +1,607 @@
+/*
+ * Copyright (c) 2014 Chelsio, Inc. All rights reserved.
+ * Copyright (c) 2014 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "iwpm_util.h"
+
+#define IWPM_HASH_BUCKET_SIZE 512
+#define IWPM_HASH_BUCKET_MASK (IWPM_HASH_BUCKET_SIZE - 1)
+
+static LIST_HEAD(iwpm_nlmsg_req_list);
+static DEFINE_SPINLOCK(iwpm_nlmsg_req_lock);
+
+static struct hlist_head *iwpm_hash_bucket;
+static DEFINE_SPINLOCK(iwpm_mapinfo_lock);
+
+static DEFINE_MUTEX(iwpm_admin_lock);
+static struct iwpm_admin_data iwpm_admin;
+
+int iwpm_init(u8 nl_client)
+{
+ if (iwpm_valid_client(nl_client))
+ return -EINVAL;
+ mutex_lock(&iwpm_admin_lock);
+ if (atomic_read(&iwpm_admin.refcount) == 0) {
+ iwpm_hash_bucket = kzalloc(IWPM_HASH_BUCKET_SIZE *
+ sizeof(struct hlist_head), GFP_KERNEL);
+ if (!iwpm_hash_bucket) {
+ mutex_unlock(&iwpm_admin_lock);
+ pr_err("%s Unable to create mapinfo hash table\n", __func__);
+ return -ENOMEM;
+ }
+ }
+ atomic_inc(&iwpm_admin.refcount);
+ mutex_unlock(&iwpm_admin_lock);
+ iwpm_set_valid(nl_client, 1);
+ return 0;
+}
+EXPORT_SYMBOL(iwpm_init);
+
+static void free_hash_bucket(void);
+
+int iwpm_exit(u8 nl_client)
+{
+
+ if (!iwpm_valid_client(nl_client))
+ return -EINVAL;
+ mutex_lock(&iwpm_admin_lock);
+ if (atomic_read(&iwpm_admin.refcount) == 0) {
+ mutex_unlock(&iwpm_admin_lock);
+ pr_err("%s Incorrect usage - negative refcount\n", __func__);
+ return -EINVAL;
+ }
+ if (atomic_dec_and_test(&iwpm_admin.refcount)) {
+ free_hash_bucket();
+ pr_debug("%s: Mapinfo hash table is destroyed\n", __func__);
+ }
+ mutex_unlock(&iwpm_admin_lock);
+ iwpm_set_valid(nl_client, 0);
+ return 0;
+}
+EXPORT_SYMBOL(iwpm_exit);
+
+static struct hlist_head *get_hash_bucket_head(struct sockaddr_storage *,
+ struct sockaddr_storage *);
+
+int iwpm_create_mapinfo(struct sockaddr_storage *local_sockaddr,
+ struct sockaddr_storage *mapped_sockaddr,
+ u8 nl_client)
+{
+ struct hlist_head *hash_bucket_head;
+ struct iwpm_mapping_info *map_info;
+ unsigned long flags;
+
+ if (!iwpm_valid_client(nl_client))
+ return -EINVAL;
+ map_info = kzalloc(sizeof(struct iwpm_mapping_info), GFP_KERNEL);
+ if (!map_info) {
+ pr_err("%s: Unable to allocate a mapping info\n", __func__);
+ return -ENOMEM;
+ }
+ memcpy(&map_info->local_sockaddr, local_sockaddr,
+ sizeof(struct sockaddr_storage));
+ memcpy(&map_info->mapped_sockaddr, mapped_sockaddr,
+ sizeof(struct sockaddr_storage));
+ map_info->nl_client = nl_client;
+
+ spin_lock_irqsave(&iwpm_mapinfo_lock, flags);
+ if (iwpm_hash_bucket) {
+ hash_bucket_head = get_hash_bucket_head(
+ &map_info->local_sockaddr,
+ &map_info->mapped_sockaddr);
+ hlist_add_head(&map_info->hlist_node, hash_bucket_head);
+ }
+ spin_unlock_irqrestore(&iwpm_mapinfo_lock, flags);
+ return 0;
+}
+EXPORT_SYMBOL(iwpm_create_mapinfo);
+
+int iwpm_remove_mapinfo(struct sockaddr_storage *local_sockaddr,
+ struct sockaddr_storage *mapped_local_addr)
+{
+ struct hlist_node *tmp_hlist_node;
+ struct hlist_head *hash_bucket_head;
+ struct iwpm_mapping_info *map_info = NULL;
+ unsigned long flags;
+ int ret = -EINVAL;
+
+ spin_lock_irqsave(&iwpm_mapinfo_lock, flags);
+ if (iwpm_hash_bucket) {
+ hash_bucket_head = get_hash_bucket_head(
+ local_sockaddr,
+ mapped_local_addr);
+ hlist_for_each_entry_safe(map_info, tmp_hlist_node,
+ hash_bucket_head, hlist_node) {
+
+ if (!iwpm_compare_sockaddr(&map_info->mapped_sockaddr,
+ mapped_local_addr)) {
+
+ hlist_del_init(&map_info->hlist_node);
+ kfree(map_info);
+ ret = 0;
+ break;
+ }
+ }
+ }
+ spin_unlock_irqrestore(&iwpm_mapinfo_lock, flags);
+ return ret;
+}
+EXPORT_SYMBOL(iwpm_remove_mapinfo);
+
+static void free_hash_bucket(void)
+{
+ struct hlist_node *tmp_hlist_node;
+ struct iwpm_mapping_info *map_info;
+ unsigned long flags;
+ int i;
+
+ /* remove all the mapinfo data from the list */
+ spin_lock_irqsave(&iwpm_mapinfo_lock, flags);
+ for (i = 0; i < IWPM_HASH_BUCKET_SIZE; i++) {
+ hlist_for_each_entry_safe(map_info, tmp_hlist_node,
+ &iwpm_hash_bucket[i], hlist_node) {
+
+ hlist_del_init(&map_info->hlist_node);
+ kfree(map_info);
+ }
+ }
+ /* free the hash list */
+ kfree(iwpm_hash_bucket);
+ iwpm_hash_bucket = NULL;
+ spin_unlock_irqrestore(&iwpm_mapinfo_lock, flags);
+}
+
+struct iwpm_nlmsg_request *iwpm_get_nlmsg_request(__u32 nlmsg_seq,
+ u8 nl_client, gfp_t gfp)
+{
+ struct iwpm_nlmsg_request *nlmsg_request = NULL;
+ unsigned long flags;
+
+ nlmsg_request = kzalloc(sizeof(struct iwpm_nlmsg_request), gfp);
+ if (!nlmsg_request) {
+ pr_err("%s Unable to allocate a nlmsg_request\n", __func__);
+ return NULL;
+ }
+ spin_lock_irqsave(&iwpm_nlmsg_req_lock, flags);
+ list_add_tail(&nlmsg_request->inprocess_list, &iwpm_nlmsg_req_list);
+ spin_unlock_irqrestore(&iwpm_nlmsg_req_lock, flags);
+
+ kref_init(&nlmsg_request->kref);
+ kref_get(&nlmsg_request->kref);
+ nlmsg_request->nlmsg_seq = nlmsg_seq;
+ nlmsg_request->nl_client = nl_client;
+ nlmsg_request->request_done = 0;
+ nlmsg_request->err_code = 0;
+ return nlmsg_request;
+}
+
+void iwpm_free_nlmsg_request(struct kref *kref)
+{
+ struct iwpm_nlmsg_request *nlmsg_request;
+ unsigned long flags;
+
+ nlmsg_request = container_of(kref, struct iwpm_nlmsg_request, kref);
+
+ spin_lock_irqsave(&iwpm_nlmsg_req_lock, flags);
+ list_del_init(&nlmsg_request->inprocess_list);
+ spin_unlock_irqrestore(&iwpm_nlmsg_req_lock, flags);
+
+ if (!nlmsg_request->request_done)
+ pr_debug("%s Freeing incomplete nlmsg request (seq = %u).\n",
+ __func__, nlmsg_request->nlmsg_seq);
+ kfree(nlmsg_request);
+}
+
+struct iwpm_nlmsg_request *iwpm_find_nlmsg_request(__u32 echo_seq)
+{
+ struct iwpm_nlmsg_request *nlmsg_request;
+ struct iwpm_nlmsg_request *found_request = NULL;
+ unsigned long flags;
+
+ spin_lock_irqsave(&iwpm_nlmsg_req_lock, flags);
+ list_for_each_entry(nlmsg_request, &iwpm_nlmsg_req_list,
+ inprocess_list) {
+ if (nlmsg_request->nlmsg_seq == echo_seq) {
+ found_request = nlmsg_request;
+ kref_get(&nlmsg_request->kref);
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&iwpm_nlmsg_req_lock, flags);
+ return found_request;
+}
+
+int iwpm_wait_complete_req(struct iwpm_nlmsg_request *nlmsg_request)
+{
+ int ret;
+ init_waitqueue_head(&nlmsg_request->waitq);
+
+ ret = wait_event_timeout(nlmsg_request->waitq,
+ (nlmsg_request->request_done != 0), IWPM_NL_TIMEOUT);
+ if (!ret) {
+ ret = -EINVAL;
+ pr_info("%s: Timeout %d sec for netlink request (seq = %u)\n",
+ __func__, (IWPM_NL_TIMEOUT/HZ), nlmsg_request->nlmsg_seq);
+ } else {
+ ret = nlmsg_request->err_code;
+ }
+ kref_put(&nlmsg_request->kref, iwpm_free_nlmsg_request);
+ return ret;
+}
+
+int iwpm_get_nlmsg_seq(void)
+{
+ return atomic_inc_return(&iwpm_admin.nlmsg_seq);
+}
+
+int iwpm_valid_client(u8 nl_client)
+{
+ if (nl_client >= RDMA_NL_NUM_CLIENTS)
+ return 0;
+ return iwpm_admin.client_list[nl_client];
+}
+
+void iwpm_set_valid(u8 nl_client, int valid)
+{
+ if (nl_client >= RDMA_NL_NUM_CLIENTS)
+ return;
+ iwpm_admin.client_list[nl_client] = valid;
+}
+
+/* valid client */
+int iwpm_registered_client(u8 nl_client)
+{
+ return iwpm_admin.reg_list[nl_client];
+}
+
+/* valid client */
+void iwpm_set_registered(u8 nl_client, int reg)
+{
+ iwpm_admin.reg_list[nl_client] = reg;
+}
+
+int iwpm_compare_sockaddr(struct sockaddr_storage *a_sockaddr,
+ struct sockaddr_storage *b_sockaddr)
+{
+ if (a_sockaddr->ss_family != b_sockaddr->ss_family)
+ return 1;
+ if (a_sockaddr->ss_family == AF_INET) {
+ struct sockaddr_in *a4_sockaddr =
+ (struct sockaddr_in *)a_sockaddr;
+ struct sockaddr_in *b4_sockaddr =
+ (struct sockaddr_in *)b_sockaddr;
+ if (!memcmp(&a4_sockaddr->sin_addr,
+ &b4_sockaddr->sin_addr, sizeof(struct in_addr))
+ && a4_sockaddr->sin_port == b4_sockaddr->sin_port)
+ return 0;
+
+ } else if (a_sockaddr->ss_family == AF_INET6) {
+ struct sockaddr_in6 *a6_sockaddr =
+ (struct sockaddr_in6 *)a_sockaddr;
+ struct sockaddr_in6 *b6_sockaddr =
+ (struct sockaddr_in6 *)b_sockaddr;
+ if (!memcmp(&a6_sockaddr->sin6_addr,
+ &b6_sockaddr->sin6_addr, sizeof(struct in6_addr))
+ && a6_sockaddr->sin6_port == b6_sockaddr->sin6_port)
+ return 0;
+
+ } else {
+ pr_err("%s: Invalid sockaddr family\n", __func__);
+ }
+ return 1;
+}
+
+struct sk_buff *iwpm_create_nlmsg(u32 nl_op, struct nlmsghdr **nlh,
+ int nl_client)
+{
+ struct sk_buff *skb = NULL;
+
+ skb = dev_alloc_skb(NLMSG_GOODSIZE);
+ if (!skb) {
+ pr_err("%s Unable to allocate skb\n", __func__);
+ goto create_nlmsg_exit;
+ }
+ if (!(ibnl_put_msg(skb, nlh, 0, 0, nl_client, nl_op,
+ NLM_F_REQUEST))) {
+ pr_warn("%s: Unable to put the nlmsg header\n", __func__);
+ dev_kfree_skb(skb);
+ skb = NULL;
+ }
+create_nlmsg_exit:
+ return skb;
+}
+
+int iwpm_parse_nlmsg(struct netlink_callback *cb, int policy_max,
+ const struct nla_policy *nlmsg_policy,
+ struct nlattr *nltb[], const char *msg_type)
+{
+ int nlh_len = 0;
+ int ret;
+ const char *err_str = "";
+
+ ret = nlmsg_validate(cb->nlh, nlh_len, policy_max-1, nlmsg_policy);
+ if (ret) {
+ err_str = "Invalid attribute";
+ goto parse_nlmsg_error;
+ }
+ ret = nlmsg_parse(cb->nlh, nlh_len, nltb, policy_max-1, nlmsg_policy);
+ if (ret) {
+ err_str = "Unable to parse the nlmsg";
+ goto parse_nlmsg_error;
+ }
+ ret = iwpm_validate_nlmsg_attr(nltb, policy_max);
+ if (ret) {
+ err_str = "Invalid NULL attribute";
+ goto parse_nlmsg_error;
+ }
+ return 0;
+parse_nlmsg_error:
+ pr_warn("%s: %s (msg type %s ret = %d)\n",
+ __func__, err_str, msg_type, ret);
+ return ret;
+}
+
+void iwpm_print_sockaddr(struct sockaddr_storage *sockaddr, char *msg)
+{
+ struct sockaddr_in6 *sockaddr_v6;
+ struct sockaddr_in *sockaddr_v4;
+
+ switch (sockaddr->ss_family) {
+ case AF_INET:
+ sockaddr_v4 = (struct sockaddr_in *)sockaddr;
+ pr_debug("%s IPV4 %pI4: %u(0x%04X)\n",
+ msg, &sockaddr_v4->sin_addr,
+ ntohs(sockaddr_v4->sin_port),
+ ntohs(sockaddr_v4->sin_port));
+ break;
+ case AF_INET6:
+ sockaddr_v6 = (struct sockaddr_in6 *)sockaddr;
+ pr_debug("%s IPV6 %pI6: %u(0x%04X)\n",
+ msg, &sockaddr_v6->sin6_addr,
+ ntohs(sockaddr_v6->sin6_port),
+ ntohs(sockaddr_v6->sin6_port));
+ break;
+ default:
+ break;
+ }
+}
+
+static u32 iwpm_ipv6_jhash(struct sockaddr_in6 *ipv6_sockaddr)
+{
+ u32 ipv6_hash = jhash(&ipv6_sockaddr->sin6_addr, sizeof(struct in6_addr), 0);
+ u32 hash = jhash_2words(ipv6_hash, (__force u32) ipv6_sockaddr->sin6_port, 0);
+ return hash;
+}
+
+static u32 iwpm_ipv4_jhash(struct sockaddr_in *ipv4_sockaddr)
+{
+ u32 ipv4_hash = jhash(&ipv4_sockaddr->sin_addr, sizeof(struct in_addr), 0);
+ u32 hash = jhash_2words(ipv4_hash, (__force u32) ipv4_sockaddr->sin_port, 0);
+ return hash;
+}
+
+static struct hlist_head *get_hash_bucket_head(struct sockaddr_storage
+ *local_sockaddr,
+ struct sockaddr_storage
+ *mapped_sockaddr)
+{
+ u32 local_hash, mapped_hash, hash;
+
+ if (local_sockaddr->ss_family == AF_INET) {
+ local_hash = iwpm_ipv4_jhash((struct sockaddr_in *) local_sockaddr);
+ mapped_hash = iwpm_ipv4_jhash((struct sockaddr_in *) mapped_sockaddr);
+
+ } else if (local_sockaddr->ss_family == AF_INET6) {
+ local_hash = iwpm_ipv6_jhash((struct sockaddr_in6 *) local_sockaddr);
+ mapped_hash = iwpm_ipv6_jhash((struct sockaddr_in6 *) mapped_sockaddr);
+ } else {
+ pr_err("%s: Invalid sockaddr family\n", __func__);
+ return NULL;
+ }
+
+ if (local_hash == mapped_hash) /* if port mapper isn't available */
+ hash = local_hash;
+ else
+ hash = jhash_2words(local_hash, mapped_hash, 0);
+
+ return &iwpm_hash_bucket[hash & IWPM_HASH_BUCKET_MASK];
+}
+
+static int send_mapinfo_num(u32 mapping_num, u8 nl_client, int iwpm_pid)
+{
+ struct sk_buff *skb = NULL;
+ struct nlmsghdr *nlh;
+ u32 msg_seq;
+ const char *err_str = "";
+ int ret = -EINVAL;
+
+ skb = iwpm_create_nlmsg(RDMA_NL_IWPM_MAPINFO_NUM, &nlh, nl_client);
+ if (!skb) {
+ err_str = "Unable to create a nlmsg";
+ goto mapinfo_num_error;
+ }
+ nlh->nlmsg_seq = iwpm_get_nlmsg_seq();
+ msg_seq = 0;
+ err_str = "Unable to put attribute of mapinfo number nlmsg";
+ ret = ibnl_put_attr(skb, nlh, sizeof(u32), &msg_seq, IWPM_NLA_MAPINFO_SEQ);
+ if (ret)
+ goto mapinfo_num_error;
+ ret = ibnl_put_attr(skb, nlh, sizeof(u32),
+ &mapping_num, IWPM_NLA_MAPINFO_SEND_NUM);
+ if (ret)
+ goto mapinfo_num_error;
+ ret = ibnl_unicast(skb, nlh, iwpm_pid);
+ if (ret) {
+ skb = NULL;
+ err_str = "Unable to send a nlmsg";
+ goto mapinfo_num_error;
+ }
+ pr_debug("%s: Sent mapping number = %d\n", __func__, mapping_num);
+ return 0;
+mapinfo_num_error:
+ pr_info("%s: %s\n", __func__, err_str);
+ if (skb)
+ dev_kfree_skb(skb);
+ return ret;
+}
+
+static int send_nlmsg_done(struct sk_buff *skb, u8 nl_client, int iwpm_pid)
+{
+ struct nlmsghdr *nlh = NULL;
+ int ret = 0;
+
+ if (!skb)
+ return ret;
+ if (!(ibnl_put_msg(skb, &nlh, 0, 0, nl_client,
+ RDMA_NL_IWPM_MAPINFO, NLM_F_MULTI))) {
+ pr_warn("%s Unable to put NLMSG_DONE\n", __func__);
+ return -ENOMEM;
+ }
+ nlh->nlmsg_type = NLMSG_DONE;
+ ret = ibnl_unicast(skb, (struct nlmsghdr *)skb->data, iwpm_pid);
+ if (ret)
+ pr_warn("%s Unable to send a nlmsg\n", __func__);
+ return ret;
+}
+
+int iwpm_send_mapinfo(u8 nl_client, int iwpm_pid)
+{
+ struct iwpm_mapping_info *map_info;
+ struct sk_buff *skb = NULL;
+ struct nlmsghdr *nlh;
+ int skb_num = 0, mapping_num = 0;
+ int i = 0, nlmsg_bytes = 0;
+ unsigned long flags;
+ const char *err_str = "";
+ int ret;
+
+ skb = dev_alloc_skb(NLMSG_GOODSIZE);
+ if (!skb) {
+ ret = -ENOMEM;
+ err_str = "Unable to allocate skb";
+ goto send_mapping_info_exit;
+ }
+ skb_num++;
+ spin_lock_irqsave(&iwpm_mapinfo_lock, flags);
+ for (i = 0; i < IWPM_HASH_BUCKET_SIZE; i++) {
+ hlist_for_each_entry(map_info, &iwpm_hash_bucket[i],
+ hlist_node) {
+ if (map_info->nl_client != nl_client)
+ continue;
+ nlh = NULL;
+ if (!(ibnl_put_msg(skb, &nlh, 0, 0, nl_client,
+ RDMA_NL_IWPM_MAPINFO, NLM_F_MULTI))) {
+ ret = -ENOMEM;
+ err_str = "Unable to put the nlmsg header";
+ goto send_mapping_info_unlock;
+ }
+ err_str = "Unable to put attribute of the nlmsg";
+ ret = ibnl_put_attr(skb, nlh,
+ sizeof(struct sockaddr_storage),
+ &map_info->local_sockaddr,
+ IWPM_NLA_MAPINFO_LOCAL_ADDR);
+ if (ret)
+ goto send_mapping_info_unlock;
+
+ ret = ibnl_put_attr(skb, nlh,
+ sizeof(struct sockaddr_storage),
+ &map_info->mapped_sockaddr,
+ IWPM_NLA_MAPINFO_MAPPED_ADDR);
+ if (ret)
+ goto send_mapping_info_unlock;
+
+ iwpm_print_sockaddr(&map_info->local_sockaddr,
+ "send_mapping_info: Local sockaddr:");
+ iwpm_print_sockaddr(&map_info->mapped_sockaddr,
+ "send_mapping_info: Mapped local sockaddr:");
+ mapping_num++;
+ nlmsg_bytes += nlh->nlmsg_len;
+
+ /* check if all mappings can fit in one skb */
+ if (NLMSG_GOODSIZE - nlmsg_bytes < nlh->nlmsg_len * 2) {
+ /* and leave room for NLMSG_DONE */
+ nlmsg_bytes = 0;
+ skb_num++;
+ spin_unlock_irqrestore(&iwpm_mapinfo_lock,
+ flags);
+ /* send the skb */
+ ret = send_nlmsg_done(skb, nl_client, iwpm_pid);
+ skb = NULL;
+ if (ret) {
+ err_str = "Unable to send map info";
+ goto send_mapping_info_exit;
+ }
+ if (skb_num == IWPM_MAPINFO_SKB_COUNT) {
+ ret = -ENOMEM;
+ err_str = "Insufficient skbs for map info";
+ goto send_mapping_info_exit;
+ }
+ skb = dev_alloc_skb(NLMSG_GOODSIZE);
+ if (!skb) {
+ ret = -ENOMEM;
+ err_str = "Unable to allocate skb";
+ goto send_mapping_info_exit;
+ }
+ spin_lock_irqsave(&iwpm_mapinfo_lock, flags);
+ }
+ }
+ }
+send_mapping_info_unlock:
+ spin_unlock_irqrestore(&iwpm_mapinfo_lock, flags);
+send_mapping_info_exit:
+ if (ret) {
+ pr_warn("%s: %s (ret = %d)\n", __func__, err_str, ret);
+ if (skb)
+ dev_kfree_skb(skb);
+ return ret;
+ }
+ send_nlmsg_done(skb, nl_client, iwpm_pid);
+ return send_mapinfo_num(mapping_num, nl_client, iwpm_pid);
+}
+
+int iwpm_mapinfo_available(void)
+{
+ unsigned long flags;
+ int full_bucket = 0, i = 0;
+
+ spin_lock_irqsave(&iwpm_mapinfo_lock, flags);
+ if (iwpm_hash_bucket) {
+ for (i = 0; i < IWPM_HASH_BUCKET_SIZE; i++) {
+ if (!hlist_empty(&iwpm_hash_bucket[i])) {
+ full_bucket = 1;
+ break;
+ }
+ }
+ }
+ spin_unlock_irqrestore(&iwpm_mapinfo_lock, flags);
+ return full_bucket;
+}
diff --git a/drivers/infiniband/core/iwpm_util.h b/drivers/infiniband/core/iwpm_util.h
new file mode 100644
index 0000000..9777c86
--- /dev/null
+++ b/drivers/infiniband/core/iwpm_util.h
@@ -0,0 +1,238 @@
+/*
+ * Copyright (c) 2014 Intel Corporation. All rights reserved.
+ * Copyright (c) 2014 Chelsio, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+#ifndef _IWPM_UTIL_H
+#define _IWPM_UTIL_H
+
+#include <linux/module.h>
+#include <linux/io.h>
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/spinlock.h>
+#include <linux/kernel.h>
+#include <linux/netdevice.h>
+#include <linux/delay.h>
+#include <linux/workqueue.h>
+#include <linux/mutex.h>
+#include <linux/jhash.h>
+#include <linux/kref.h>
+#include <net/netlink.h>
+#include <linux/errno.h>
+#include <rdma/iw_portmap.h>
+#include <rdma/rdma_netlink.h>
+
+
+#define IWPM_NL_RETRANS 3
+#define IWPM_NL_TIMEOUT (10*HZ)
+#define IWPM_MAPINFO_SKB_COUNT 20
+
+#define IWPM_PID_UNDEFINED -1
+#define IWPM_PID_UNAVAILABLE -2
+
+struct iwpm_nlmsg_request {
+ struct list_head inprocess_list;
+ __u32 nlmsg_seq;
+ void *req_buffer;
+ u8 nl_client;
+ u8 request_done;
+ u16 err_code;
+ wait_queue_head_t waitq;
+ struct kref kref;
+};
+
+struct iwpm_mapping_info {
+ struct hlist_node hlist_node;
+ struct sockaddr_storage local_sockaddr;
+ struct sockaddr_storage mapped_sockaddr;
+ u8 nl_client;
+};
+
+struct iwpm_admin_data {
+ atomic_t refcount;
+ atomic_t nlmsg_seq;
+ int client_list[RDMA_NL_NUM_CLIENTS];
+ int reg_list[RDMA_NL_NUM_CLIENTS];
+};
+
+/**
+ * iwpm_get_nlmsg_request - Allocate and initialize netlink message request
+ * @nlmsg_seq: Sequence number of the netlink message
+ * @nl_client: The index of the netlink client
+ * @gfp: Indicates how the memory for the request should be allocated
+ *
+ * Returns the newly allocated netlink request object if successful,
+ * otherwise returns NULL
+ */
+struct iwpm_nlmsg_request *iwpm_get_nlmsg_request(__u32 nlmsg_seq,
+ u8 nl_client, gfp_t gfp);
+
+/**
+ * iwpm_free_nlmsg_request - Deallocate netlink message request
+ * @kref: Holds reference of netlink message request
+ */
+void iwpm_free_nlmsg_request(struct kref *kref);
+
+/**
+ * iwpm_find_nlmsg_request - Find netlink message request in the request list
+ * @echo_seq: Sequence number of the netlink request to find
+ *
+ * Returns the found netlink message request,
+ * if not found, returns NULL
+ */
+struct iwpm_nlmsg_request *iwpm_find_nlmsg_request(__u32 echo_seq);
+
+/**
+ * iwpm_wait_complete_req - Block while servicing the netlink request
+ * @nlmsg_request: Netlink message request to service
+ *
+ * Wakes up, after the request is completed or expired
+ * Returns 0 if the request is complete without error
+ */
+int iwpm_wait_complete_req(struct iwpm_nlmsg_request *nlmsg_request);
+
+/**
+ * iwpm_get_nlmsg_seq - Get the sequence number for a netlink
+ * message to send to the port mapper
+ *
+ * Returns the sequence number for the netlink message.
+ */
+int iwpm_get_nlmsg_seq(void);
+
+/**
+ * iwpm_valid_client - Check if the port mapper client is valid
+ * @nl_client: The index of the netlink client
+ *
+ * Valid clients need to call iwpm_init() before using
+ * the port mapper
+ */
+int iwpm_valid_client(u8 nl_client);
+
+/**
+ * iwpm_set_valid - Set the port mapper client to valid or not
+ * @nl_client: The index of the netlink client
+ * @valid: 1 if valid or 0 if invalid
+ */
+void iwpm_set_valid(u8 nl_client, int valid);
+
+/**
+ * iwpm_registered_client - Check if the port mapper client is registered
+ * @nl_client: The index of the netlink client
+ *
+ * Call iwpm_register_pid() to register a client
+ */
+int iwpm_registered_client(u8 nl_client);
+
+/**
+ * iwpm_set_registered - Set the port mapper client to registered or not
+ * @nl_client: The index of the netlink client
+ * @reg: 1 if registered or 0 if not
+ */
+void iwpm_set_registered(u8 nl_client, int reg);
+
+/**
+ * iwpm_send_mapinfo - Send local and mapped IPv4/IPv6 address info of
+ * a client to the user space port mapper
+ * @nl_client: The index of the netlink client
+ * @iwpm_pid: The pid of the user space port mapper
+ *
+ * If successful, returns the number of sent mapping info records
+ */
+int iwpm_send_mapinfo(u8 nl_client, int iwpm_pid);
+
+/**
+ * iwpm_mapinfo_available - Check if any mapping info records is available
+ * in the hash table
+ *
+ * Returns 1 if mapping information is available, otherwise returns 0
+ */
+int iwpm_mapinfo_available(void);
+
+/**
+ * iwpm_compare_sockaddr - Compare two sockaddr storage structs
+ *
+ * Returns 0 if they are holding the same ip/tcp address info,
+ * otherwise returns 1
+ */
+int iwpm_compare_sockaddr(struct sockaddr_storage *a_sockaddr,
+ struct sockaddr_storage *b_sockaddr);
+
+/**
+ * iwpm_validate_nlmsg_attr - Check for NULL netlink attributes
+ * @nltb: Holds address of each netlink message attributes
+ * @nla_count: Number of netlink message attributes
+ *
+ * Returns error if any of the nla_count attributes is NULL
+ */
+static inline int iwpm_validate_nlmsg_attr(struct nlattr *nltb[],
+ int nla_count)
+{
+ int i;
+ for (i = 1; i < nla_count; i++) {
+ if (!nltb[i])
+ return -EINVAL;
+ }
+ return 0;
+}
+
+/**
+ * iwpm_create_nlmsg - Allocate skb and form a netlink message
+ * @nl_op: Netlink message opcode
+ * @nlh: Holds address of the netlink message header in skb
+ * @nl_client: The index of the netlink client
+ *
+ * Returns the newly allcated skb, or NULL if the tailroom of the skb
+ * is insufficient to store the message header and payload
+ */
+struct sk_buff *iwpm_create_nlmsg(u32 nl_op, struct nlmsghdr **nlh,
+ int nl_client);
+
+/**
+ * iwpm_parse_nlmsg - Validate and parse the received netlink message
+ * @cb: Netlink callback structure
+ * @policy_max: Maximum attribute type to be expected
+ * @nlmsg_policy: Validation policy
+ * @nltb: Array to store policy_max parsed elements
+ * @msg_type: Type of netlink message
+ *
+ * Returns 0 on success or a negative error code
+ */
+int iwpm_parse_nlmsg(struct netlink_callback *cb, int policy_max,
+ const struct nla_policy *nlmsg_policy,
+ struct nlattr *nltb[], const char *msg_type);
+
+/**
+ * iwpm_print_sockaddr - Print IPv4/IPv6 address and TCP port
+ * @sockaddr: Socket address to print
+ * @msg: Message to print
+ */
+void iwpm_print_sockaddr(struct sockaddr_storage *sockaddr, char *msg);
+#endif
diff --git a/drivers/infiniband/core/netlink.c b/drivers/infiniband/core/netlink.c
index a1e9cba..23dd5a5 100644
--- a/drivers/infiniband/core/netlink.c
+++ b/drivers/infiniband/core/netlink.c
@@ -103,13 +103,13 @@
EXPORT_SYMBOL(ibnl_remove_client);
void *ibnl_put_msg(struct sk_buff *skb, struct nlmsghdr **nlh, int seq,
- int len, int client, int op)
+ int len, int client, int op, int flags)
{
unsigned char *prev_tail;
prev_tail = skb_tail_pointer(skb);
*nlh = nlmsg_put(skb, 0, seq, RDMA_NL_GET_TYPE(client, op),
- len, NLM_F_MULTI);
+ len, flags);
if (!*nlh)
goto out_nlmsg_trim;
(*nlh)->nlmsg_len = skb_tail_pointer(skb) - prev_tail;
@@ -172,6 +172,20 @@
mutex_unlock(&ibnl_mutex);
}
+int ibnl_unicast(struct sk_buff *skb, struct nlmsghdr *nlh,
+ __u32 pid)
+{
+ return nlmsg_unicast(nls, skb, pid);
+}
+EXPORT_SYMBOL(ibnl_unicast);
+
+int ibnl_multicast(struct sk_buff *skb, struct nlmsghdr *nlh,
+ unsigned int group, gfp_t flags)
+{
+ return nlmsg_multicast(nls, skb, 0, group, flags);
+}
+EXPORT_SYMBOL(ibnl_multicast);
+
int __init ibnl_init(void)
{
struct netlink_kernel_cfg cfg = {
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index f820958..233eaf5 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -618,7 +618,7 @@
static int send_mad(struct ib_sa_query *query, int timeout_ms, gfp_t gfp_mask)
{
- bool preload = gfp_mask & __GFP_WAIT;
+ bool preload = !!(gfp_mask & __GFP_WAIT);
unsigned long flags;
int ret, id;
diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c
index 7d3292c..cbd0383 100644
--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -429,15 +429,19 @@
struct attribute *a;
int i;
- for (i = 0; (a = p->gid_group.attrs[i]); ++i)
- kfree(a);
+ if (p->gid_group.attrs) {
+ for (i = 0; (a = p->gid_group.attrs[i]); ++i)
+ kfree(a);
- kfree(p->gid_group.attrs);
+ kfree(p->gid_group.attrs);
+ }
- for (i = 0; (a = p->pkey_group.attrs[i]); ++i)
- kfree(a);
+ if (p->pkey_group.attrs) {
+ for (i = 0; (a = p->pkey_group.attrs[i]); ++i)
+ kfree(a);
- kfree(p->pkey_group.attrs);
+ kfree(p->pkey_group.attrs);
+ }
kfree(p);
}
@@ -534,10 +538,12 @@
p->port_num = port_num;
ret = kobject_init_and_add(&p->kobj, &port_type,
- kobject_get(device->ports_parent),
+ device->ports_parent,
"%d", port_num);
- if (ret)
- goto err_put;
+ if (ret) {
+ kfree(p);
+ return ret;
+ }
ret = sysfs_create_group(&p->kobj, &pma_group);
if (ret)
@@ -585,6 +591,7 @@
kfree(p->pkey_group.attrs[i]);
kfree(p->pkey_group.attrs);
+ p->pkey_group.attrs = NULL;
err_remove_gid:
sysfs_remove_group(&p->kobj, &p->gid_group);
@@ -594,13 +601,13 @@
kfree(p->gid_group.attrs[i]);
kfree(p->gid_group.attrs);
+ p->gid_group.attrs = NULL;
err_remove_pma:
sysfs_remove_group(&p->kobj, &pma_group);
err_put:
- kobject_put(device->ports_parent);
- kfree(p);
+ kobject_put(&p->kobj);
return ret;
}
@@ -809,6 +816,22 @@
.attrs = iw_proto_stats_attrs,
};
+static void free_port_list_attributes(struct ib_device *device)
+{
+ struct kobject *p, *t;
+
+ list_for_each_entry_safe(p, t, &device->port_list, entry) {
+ struct ib_port *port = container_of(p, struct ib_port, kobj);
+ list_del(&p->entry);
+ sysfs_remove_group(p, &pma_group);
+ sysfs_remove_group(p, &port->pkey_group);
+ sysfs_remove_group(p, &port->gid_group);
+ kobject_put(p);
+ }
+
+ kobject_put(device->ports_parent);
+}
+
int ib_device_register_sysfs(struct ib_device *device,
int (*port_callback)(struct ib_device *,
u8, struct kobject *))
@@ -835,7 +858,7 @@
}
device->ports_parent = kobject_create_and_add("ports",
- kobject_get(&class_dev->kobj));
+ &class_dev->kobj);
if (!device->ports_parent) {
ret = -ENOMEM;
goto err_put;
@@ -862,21 +885,7 @@
return 0;
err_put:
- {
- struct kobject *p, *t;
- struct ib_port *port;
-
- list_for_each_entry_safe(p, t, &device->port_list, entry) {
- list_del(&p->entry);
- port = container_of(p, struct ib_port, kobj);
- sysfs_remove_group(p, &pma_group);
- sysfs_remove_group(p, &port->pkey_group);
- sysfs_remove_group(p, &port->gid_group);
- kobject_put(p);
- }
- }
-
- kobject_put(&class_dev->kobj);
+ free_port_list_attributes(device);
err_unregister:
device_unregister(class_dev);
@@ -887,22 +896,18 @@
void ib_device_unregister_sysfs(struct ib_device *device)
{
- struct kobject *p, *t;
- struct ib_port *port;
-
/* Hold kobject until ib_dealloc_device() */
- kobject_get(&device->dev.kobj);
+ struct kobject *kobj_dev = kobject_get(&device->dev.kobj);
+ int i;
- list_for_each_entry_safe(p, t, &device->port_list, entry) {
- list_del(&p->entry);
- port = container_of(p, struct ib_port, kobj);
- sysfs_remove_group(p, &pma_group);
- sysfs_remove_group(p, &port->pkey_group);
- sysfs_remove_group(p, &port->gid_group);
- kobject_put(p);
- }
+ if (device->node_type == RDMA_NODE_RNIC && device->get_protocol_stats)
+ sysfs_remove_group(kobj_dev, &iw_stats_group);
- kobject_put(device->ports_parent);
+ free_port_list_attributes(device);
+
+ for (i = 0; i < ARRAY_SIZE(ib_class_attributes); ++i)
+ device_remove_file(&device->dev, ib_class_attributes[i]);
+
device_unregister(&device->dev);
}
diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c
index f0d588f..1acb991 100644
--- a/drivers/infiniband/core/user_mad.c
+++ b/drivers/infiniband/core/user_mad.c
@@ -98,7 +98,7 @@
struct ib_umad_device {
int start_port, end_port;
- struct kref ref;
+ struct kobject kobj;
struct ib_umad_port port[0];
};
@@ -134,14 +134,18 @@
static void ib_umad_add_one(struct ib_device *device);
static void ib_umad_remove_one(struct ib_device *device);
-static void ib_umad_release_dev(struct kref *ref)
+static void ib_umad_release_dev(struct kobject *kobj)
{
struct ib_umad_device *dev =
- container_of(ref, struct ib_umad_device, ref);
+ container_of(kobj, struct ib_umad_device, kobj);
kfree(dev);
}
+static struct kobj_type ib_umad_dev_ktype = {
+ .release = ib_umad_release_dev,
+};
+
static int hdr_size(struct ib_umad_file *file)
{
return file->use_pkey_index ? sizeof (struct ib_user_mad_hdr) :
@@ -780,27 +784,19 @@
{
struct ib_umad_port *port;
struct ib_umad_file *file;
- int ret;
+ int ret = -ENXIO;
port = container_of(inode->i_cdev, struct ib_umad_port, cdev);
- if (port)
- kref_get(&port->umad_dev->ref);
- else
- return -ENXIO;
mutex_lock(&port->file_mutex);
- if (!port->ib_dev) {
- ret = -ENXIO;
+ if (!port->ib_dev)
goto out;
- }
+ ret = -ENOMEM;
file = kzalloc(sizeof *file, GFP_KERNEL);
- if (!file) {
- kref_put(&port->umad_dev->ref, ib_umad_release_dev);
- ret = -ENOMEM;
+ if (!file)
goto out;
- }
mutex_init(&file->mutex);
spin_lock_init(&file->send_lock);
@@ -814,6 +810,13 @@
list_add_tail(&file->port_list, &port->file_list);
ret = nonseekable_open(inode, filp);
+ if (ret) {
+ list_del(&file->port_list);
+ kfree(file);
+ goto out;
+ }
+
+ kobject_get(&port->umad_dev->kobj);
out:
mutex_unlock(&port->file_mutex);
@@ -852,7 +855,7 @@
mutex_unlock(&file->port->file_mutex);
kfree(file);
- kref_put(&dev->ref, ib_umad_release_dev);
+ kobject_put(&dev->kobj);
return 0;
}
@@ -880,10 +883,6 @@
int ret;
port = container_of(inode->i_cdev, struct ib_umad_port, sm_cdev);
- if (port)
- kref_get(&port->umad_dev->ref);
- else
- return -ENXIO;
if (filp->f_flags & O_NONBLOCK) {
if (down_trylock(&port->sm_sem)) {
@@ -898,17 +897,27 @@
}
ret = ib_modify_port(port->ib_dev, port->port_num, 0, &props);
- if (ret) {
- up(&port->sm_sem);
- goto fail;
- }
+ if (ret)
+ goto err_up_sem;
filp->private_data = port;
- return nonseekable_open(inode, filp);
+ ret = nonseekable_open(inode, filp);
+ if (ret)
+ goto err_clr_sm_cap;
+
+ kobject_get(&port->umad_dev->kobj);
+
+ return 0;
+
+err_clr_sm_cap:
+ swap(props.set_port_cap_mask, props.clr_port_cap_mask);
+ ib_modify_port(port->ib_dev, port->port_num, 0, &props);
+
+err_up_sem:
+ up(&port->sm_sem);
fail:
- kref_put(&port->umad_dev->ref, ib_umad_release_dev);
return ret;
}
@@ -927,7 +936,7 @@
up(&port->sm_sem);
- kref_put(&port->umad_dev->ref, ib_umad_release_dev);
+ kobject_put(&port->umad_dev->kobj);
return ret;
}
@@ -995,6 +1004,7 @@
}
static int ib_umad_init_port(struct ib_device *device, int port_num,
+ struct ib_umad_device *umad_dev,
struct ib_umad_port *port)
{
int devnum;
@@ -1027,6 +1037,7 @@
cdev_init(&port->cdev, &umad_fops);
port->cdev.owner = THIS_MODULE;
+ port->cdev.kobj.parent = &umad_dev->kobj;
kobject_set_name(&port->cdev.kobj, "umad%d", port->dev_num);
if (cdev_add(&port->cdev, base, 1))
goto err_cdev;
@@ -1045,6 +1056,7 @@
base += IB_UMAD_MAX_PORTS;
cdev_init(&port->sm_cdev, &umad_sm_fops);
port->sm_cdev.owner = THIS_MODULE;
+ port->sm_cdev.kobj.parent = &umad_dev->kobj;
kobject_set_name(&port->sm_cdev.kobj, "issm%d", port->dev_num);
if (cdev_add(&port->sm_cdev, base, 1))
goto err_sm_cdev;
@@ -1138,7 +1150,7 @@
if (!umad_dev)
return;
- kref_init(&umad_dev->ref);
+ kobject_init(&umad_dev->kobj, &ib_umad_dev_ktype);
umad_dev->start_port = s;
umad_dev->end_port = e;
@@ -1146,7 +1158,8 @@
for (i = s; i <= e; ++i) {
umad_dev->port[i - s].umad_dev = umad_dev;
- if (ib_umad_init_port(device, i, &umad_dev->port[i - s]))
+ if (ib_umad_init_port(device, i, umad_dev,
+ &umad_dev->port[i - s]))
goto err;
}
@@ -1158,7 +1171,7 @@
while (--i >= s)
ib_umad_kill_port(&umad_dev->port[i - s]);
- kref_put(&umad_dev->ref, ib_umad_release_dev);
+ kobject_put(&umad_dev->kobj);
}
static void ib_umad_remove_one(struct ib_device *device)
@@ -1172,7 +1185,7 @@
for (i = 0; i <= umad_dev->end_port - umad_dev->start_port; ++i)
ib_umad_kill_port(&umad_dev->port[i]);
- kref_put(&umad_dev->ref, ib_umad_release_dev);
+ kobject_put(&umad_dev->kobj);
}
static char *umad_devnode(struct device *dev, umode_t *mode)
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 92525f8..c2b89cc 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -48,7 +48,7 @@
#include "core_priv.h"
-int ib_rate_to_mult(enum ib_rate rate)
+__attribute_const__ int ib_rate_to_mult(enum ib_rate rate)
{
switch (rate) {
case IB_RATE_2_5_GBPS: return 1;
@@ -65,7 +65,7 @@
}
EXPORT_SYMBOL(ib_rate_to_mult);
-enum ib_rate mult_to_ib_rate(int mult)
+__attribute_const__ enum ib_rate mult_to_ib_rate(int mult)
{
switch (mult) {
case 1: return IB_RATE_2_5_GBPS;
@@ -82,7 +82,7 @@
}
EXPORT_SYMBOL(mult_to_ib_rate);
-int ib_rate_to_mbps(enum ib_rate rate)
+__attribute_const__ int ib_rate_to_mbps(enum ib_rate rate)
{
switch (rate) {
case IB_RATE_2_5_GBPS: return 2500;
@@ -107,7 +107,7 @@
}
EXPORT_SYMBOL(ib_rate_to_mbps);
-enum rdma_transport_type
+__attribute_const__ enum rdma_transport_type
rdma_node_get_transport(enum rdma_node_type node_type)
{
switch (node_type) {
diff --git a/drivers/infiniband/hw/Makefile b/drivers/infiniband/hw/Makefile
new file mode 100644
index 0000000..e900b03
--- /dev/null
+++ b/drivers/infiniband/hw/Makefile
@@ -0,0 +1,12 @@
+obj-$(CONFIG_INFINIBAND_MTHCA) += mthca/
+obj-$(CONFIG_INFINIBAND_IPATH) += ipath/
+obj-$(CONFIG_INFINIBAND_QIB) += qib/
+obj-$(CONFIG_INFINIBAND_EHCA) += ehca/
+obj-$(CONFIG_INFINIBAND_AMSO1100) += amso1100/
+obj-$(CONFIG_INFINIBAND_CXGB3) += cxgb3/
+obj-$(CONFIG_INFINIBAND_CXGB4) += cxgb4/
+obj-$(CONFIG_MLX4_INFINIBAND) += mlx4/
+obj-$(CONFIG_MLX5_INFINIBAND) += mlx5/
+obj-$(CONFIG_INFINIBAND_NES) += nes/
+obj-$(CONFIG_INFINIBAND_OCRDMA) += ocrdma/
+obj-$(CONFIG_INFINIBAND_USNIC) += usnic/
diff --git a/drivers/infiniband/hw/cxgb3/cxio_hal.c b/drivers/infiniband/hw/cxgb3/cxio_hal.c
index c3f5aca..de1c61b4 100644
--- a/drivers/infiniband/hw/cxgb3/cxio_hal.c
+++ b/drivers/infiniband/hw/cxgb3/cxio_hal.c
@@ -735,14 +735,12 @@
((perm & TPT_MW_BIND) ? F_TPT_MW_BIND_ENABLE : 0) |
V_TPT_ADDR_TYPE((zbva ? TPT_ZBTO : TPT_VATO)) |
V_TPT_PAGE_SIZE(page_size));
- tpt.rsvd_pbl_addr = reset_tpt_entry ? 0 :
- cpu_to_be32(V_TPT_PBL_ADDR(PBL_OFF(rdev_p, pbl_addr)>>3));
+ tpt.rsvd_pbl_addr = cpu_to_be32(V_TPT_PBL_ADDR(PBL_OFF(rdev_p, pbl_addr)>>3));
tpt.len = cpu_to_be32(len);
tpt.va_hi = cpu_to_be32((u32) (to >> 32));
tpt.va_low_or_fbo = cpu_to_be32((u32) (to & 0xFFFFFFFFULL));
tpt.rsvd_bind_cnt_or_pstag = 0;
- tpt.rsvd_pbl_size = reset_tpt_entry ? 0 :
- cpu_to_be32(V_TPT_PBL_SIZE(pbl_size >> 2));
+ tpt.rsvd_pbl_size = cpu_to_be32(V_TPT_PBL_SIZE(pbl_size >> 2));
}
err = cxio_hal_ctrl_qp_write_mem(rdev_p,
stag_idx +
diff --git a/drivers/infiniband/hw/cxgb3/iwch_cm.c b/drivers/infiniband/hw/cxgb3/iwch_cm.c
index 095bb04..cb78b1e9 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_cm.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_cm.c
@@ -418,6 +418,7 @@
skb->priority = CPL_PRIORITY_DATA;
set_arp_failure_handler(skb, abort_arp_failure);
req = (struct cpl_abort_req *) skb_put(skb, sizeof(*req));
+ memset(req, 0, sizeof(*req));
req->wr.wr_hi = htonl(V_WR_OP(FW_WROPCODE_OFLD_HOST_ABORT_CON_REQ));
req->wr.wr_lo = htonl(V_WR_TID(ep->hwtid));
OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_ABORT_REQ, ep->hwtid));
diff --git a/drivers/infiniband/hw/cxgb4/cm.c b/drivers/infiniband/hw/cxgb4/cm.c
index 1f863a9..96d7131a 100644
--- a/drivers/infiniband/hw/cxgb4/cm.c
+++ b/drivers/infiniband/hw/cxgb4/cm.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2009-2010 Chelsio, Inc. All rights reserved.
+ * Copyright (c) 2009-2014 Chelsio, Inc. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
@@ -47,6 +47,8 @@
#include <net/ip6_route.h>
#include <net/addrconf.h>
+#include <rdma/ib_addr.h>
+
#include "iw_cxgb4.h"
static char *states[] = {
@@ -294,6 +296,12 @@
dst_release(ep->dst);
cxgb4_l2t_release(ep->l2t);
}
+ if (test_bit(RELEASE_MAPINFO, &ep->com.flags)) {
+ print_addr(&ep->com, __func__, "remove_mapinfo/mapping");
+ iwpm_remove_mapinfo(&ep->com.local_addr,
+ &ep->com.mapped_local_addr);
+ iwpm_remove_mapping(&ep->com.local_addr, RDMA_NL_C4IW);
+ }
kfree(ep);
}
@@ -341,10 +349,7 @@
static struct net_device *get_real_dev(struct net_device *egress_dev)
{
- struct net_device *phys_dev = egress_dev;
- if (egress_dev->priv_flags & IFF_802_1Q_VLAN)
- phys_dev = vlan_dev_real_dev(egress_dev);
- return phys_dev;
+ return rdma_vlan_dev_real_dev(egress_dev) ? : egress_dev;
}
static int our_interface(struct c4iw_dev *dev, struct net_device *egress_dev)
@@ -528,6 +533,38 @@
return c4iw_l2t_send(&ep->com.dev->rdev, skb, ep->l2t);
}
+/*
+ * c4iw_form_pm_msg - Form a port mapper message with mapping info
+ */
+static void c4iw_form_pm_msg(struct c4iw_ep *ep,
+ struct iwpm_sa_data *pm_msg)
+{
+ memcpy(&pm_msg->loc_addr, &ep->com.local_addr,
+ sizeof(ep->com.local_addr));
+ memcpy(&pm_msg->rem_addr, &ep->com.remote_addr,
+ sizeof(ep->com.remote_addr));
+}
+
+/*
+ * c4iw_form_reg_msg - Form a port mapper message with dev info
+ */
+static void c4iw_form_reg_msg(struct c4iw_dev *dev,
+ struct iwpm_dev_data *pm_msg)
+{
+ memcpy(pm_msg->dev_name, dev->ibdev.name, IWPM_DEVNAME_SIZE);
+ memcpy(pm_msg->if_name, dev->rdev.lldi.ports[0]->name,
+ IWPM_IFNAME_SIZE);
+}
+
+static void c4iw_record_pm_msg(struct c4iw_ep *ep,
+ struct iwpm_sa_data *pm_msg)
+{
+ memcpy(&ep->com.mapped_local_addr, &pm_msg->mapped_loc_addr,
+ sizeof(ep->com.mapped_local_addr));
+ memcpy(&ep->com.mapped_remote_addr, &pm_msg->mapped_rem_addr,
+ sizeof(ep->com.mapped_remote_addr));
+}
+
static int send_connect(struct c4iw_ep *ep)
{
struct cpl_act_open_req *req;
@@ -546,10 +583,14 @@
int sizev6 = is_t4(ep->com.dev->rdev.lldi.adapter_type) ?
sizeof(struct cpl_act_open_req6) :
sizeof(struct cpl_t5_act_open_req6);
- struct sockaddr_in *la = (struct sockaddr_in *)&ep->com.local_addr;
- struct sockaddr_in *ra = (struct sockaddr_in *)&ep->com.remote_addr;
- struct sockaddr_in6 *la6 = (struct sockaddr_in6 *)&ep->com.local_addr;
- struct sockaddr_in6 *ra6 = (struct sockaddr_in6 *)&ep->com.remote_addr;
+ struct sockaddr_in *la = (struct sockaddr_in *)
+ &ep->com.mapped_local_addr;
+ struct sockaddr_in *ra = (struct sockaddr_in *)
+ &ep->com.mapped_remote_addr;
+ struct sockaddr_in6 *la6 = (struct sockaddr_in6 *)
+ &ep->com.mapped_local_addr;
+ struct sockaddr_in6 *ra6 = (struct sockaddr_in6 *)
+ &ep->com.mapped_remote_addr;
wrlen = (ep->com.remote_addr.ss_family == AF_INET) ?
roundup(sizev4, 16) :
@@ -1627,10 +1668,10 @@
req->le.filter = cpu_to_be32(cxgb4_select_ntuple(
ep->com.dev->rdev.lldi.ports[0],
ep->l2t));
- sin = (struct sockaddr_in *)&ep->com.local_addr;
+ sin = (struct sockaddr_in *)&ep->com.mapped_local_addr;
req->le.lport = sin->sin_port;
req->le.u.ipv4.lip = sin->sin_addr.s_addr;
- sin = (struct sockaddr_in *)&ep->com.remote_addr;
+ sin = (struct sockaddr_in *)&ep->com.mapped_remote_addr;
req->le.pport = sin->sin_port;
req->le.u.ipv4.pip = sin->sin_addr.s_addr;
req->tcb.t_state_to_astid =
@@ -1746,16 +1787,16 @@
if (!ep->l2t)
goto out;
ep->mtu = dst_mtu(dst);
- ep->tx_chan = cxgb4_port_chan(n->dev);
- ep->smac_idx = (cxgb4_port_viid(n->dev) & 0x7F) << 1;
+ ep->tx_chan = cxgb4_port_chan(pdev);
+ ep->smac_idx = (cxgb4_port_viid(pdev) & 0x7F) << 1;
step = cdev->rdev.lldi.ntxq /
cdev->rdev.lldi.nchan;
- ep->txq_idx = cxgb4_port_idx(n->dev) * step;
- ep->ctrlq_idx = cxgb4_port_idx(n->dev);
+ ep->txq_idx = cxgb4_port_idx(pdev) * step;
+ ep->ctrlq_idx = cxgb4_port_idx(pdev);
step = cdev->rdev.lldi.nrxq /
cdev->rdev.lldi.nchan;
ep->rss_qid = cdev->rdev.lldi.rxq_ids[
- cxgb4_port_idx(n->dev) * step];
+ cxgb4_port_idx(pdev) * step];
if (clear_mpa_v1) {
ep->retry_with_mpa_v1 = 0;
@@ -1870,10 +1911,10 @@
struct sockaddr_in6 *ra6;
ep = lookup_atid(t, atid);
- la = (struct sockaddr_in *)&ep->com.local_addr;
- ra = (struct sockaddr_in *)&ep->com.remote_addr;
- la6 = (struct sockaddr_in6 *)&ep->com.local_addr;
- ra6 = (struct sockaddr_in6 *)&ep->com.remote_addr;
+ la = (struct sockaddr_in *)&ep->com.mapped_local_addr;
+ ra = (struct sockaddr_in *)&ep->com.mapped_remote_addr;
+ la6 = (struct sockaddr_in6 *)&ep->com.mapped_local_addr;
+ ra6 = (struct sockaddr_in6 *)&ep->com.mapped_remote_addr;
PDBG("%s ep %p atid %u status %u errno %d\n", __func__, ep, atid,
status, status2errno(status));
@@ -2730,13 +2771,15 @@
struct c4iw_dev *dev = to_c4iw_dev(cm_id->device);
struct c4iw_ep *ep;
int err = 0;
- struct sockaddr_in *laddr = (struct sockaddr_in *)&cm_id->local_addr;
- struct sockaddr_in *raddr = (struct sockaddr_in *)&cm_id->remote_addr;
- struct sockaddr_in6 *laddr6 = (struct sockaddr_in6 *)&cm_id->local_addr;
- struct sockaddr_in6 *raddr6 = (struct sockaddr_in6 *)
- &cm_id->remote_addr;
+ struct sockaddr_in *laddr;
+ struct sockaddr_in *raddr;
+ struct sockaddr_in6 *laddr6;
+ struct sockaddr_in6 *raddr6;
+ struct iwpm_dev_data pm_reg_msg;
+ struct iwpm_sa_data pm_msg;
__u8 *ra;
int iptype;
+ int iwpm_err = 0;
if ((conn_param->ord > c4iw_max_read_depth) ||
(conn_param->ird > c4iw_max_read_depth)) {
@@ -2767,7 +2810,7 @@
if (!ep->com.qp) {
PDBG("%s qpn 0x%x not found!\n", __func__, conn_param->qpn);
err = -EINVAL;
- goto fail2;
+ goto fail1;
}
ref_qp(ep);
PDBG("%s qpn 0x%x qp %p cm_id %p\n", __func__, conn_param->qpn,
@@ -2780,10 +2823,50 @@
if (ep->atid == -1) {
printk(KERN_ERR MOD "%s - cannot alloc atid.\n", __func__);
err = -ENOMEM;
- goto fail2;
+ goto fail1;
}
insert_handle(dev, &dev->atid_idr, ep, ep->atid);
+ memcpy(&ep->com.local_addr, &cm_id->local_addr,
+ sizeof(ep->com.local_addr));
+ memcpy(&ep->com.remote_addr, &cm_id->remote_addr,
+ sizeof(ep->com.remote_addr));
+
+ /* No port mapper available, go with the specified peer information */
+ memcpy(&ep->com.mapped_local_addr, &cm_id->local_addr,
+ sizeof(ep->com.mapped_local_addr));
+ memcpy(&ep->com.mapped_remote_addr, &cm_id->remote_addr,
+ sizeof(ep->com.mapped_remote_addr));
+
+ c4iw_form_reg_msg(dev, &pm_reg_msg);
+ iwpm_err = iwpm_register_pid(&pm_reg_msg, RDMA_NL_C4IW);
+ if (iwpm_err) {
+ PDBG("%s: Port Mapper reg pid fail (err = %d).\n",
+ __func__, iwpm_err);
+ }
+ if (iwpm_valid_pid() && !iwpm_err) {
+ c4iw_form_pm_msg(ep, &pm_msg);
+ iwpm_err = iwpm_add_and_query_mapping(&pm_msg, RDMA_NL_C4IW);
+ if (iwpm_err)
+ PDBG("%s: Port Mapper query fail (err = %d).\n",
+ __func__, iwpm_err);
+ else
+ c4iw_record_pm_msg(ep, &pm_msg);
+ }
+ if (iwpm_create_mapinfo(&ep->com.local_addr,
+ &ep->com.mapped_local_addr, RDMA_NL_C4IW)) {
+ iwpm_remove_mapping(&ep->com.local_addr, RDMA_NL_C4IW);
+ err = -ENOMEM;
+ goto fail1;
+ }
+ print_addr(&ep->com, __func__, "add_query/create_mapinfo");
+ set_bit(RELEASE_MAPINFO, &ep->com.flags);
+
+ laddr = (struct sockaddr_in *)&ep->com.mapped_local_addr;
+ raddr = (struct sockaddr_in *)&ep->com.mapped_remote_addr;
+ laddr6 = (struct sockaddr_in6 *)&ep->com.mapped_local_addr;
+ raddr6 = (struct sockaddr_in6 *) &ep->com.mapped_remote_addr;
+
if (cm_id->remote_addr.ss_family == AF_INET) {
iptype = 4;
ra = (__u8 *)&raddr->sin_addr;
@@ -2794,7 +2877,7 @@
if ((__force int)raddr->sin_addr.s_addr == INADDR_ANY) {
err = pick_local_ipaddrs(dev, cm_id);
if (err)
- goto fail2;
+ goto fail1;
}
/* find a route */
@@ -2814,7 +2897,7 @@
if (ipv6_addr_type(&raddr6->sin6_addr) == IPV6_ADDR_ANY) {
err = pick_local_ip6addrs(dev, cm_id);
if (err)
- goto fail2;
+ goto fail1;
}
/* find a route */
@@ -2830,13 +2913,13 @@
if (!ep->dst) {
printk(KERN_ERR MOD "%s - cannot find route.\n", __func__);
err = -EHOSTUNREACH;
- goto fail3;
+ goto fail2;
}
err = import_ep(ep, iptype, ra, ep->dst, ep->com.dev, true);
if (err) {
printk(KERN_ERR MOD "%s - cannot alloc l2e.\n", __func__);
- goto fail4;
+ goto fail3;
}
PDBG("%s txq_idx %u tx_chan %u smac_idx %u rss_qid %u l2t_idx %u\n",
@@ -2845,10 +2928,6 @@
state_set(&ep->com, CONNECTING);
ep->tos = 0;
- memcpy(&ep->com.local_addr, &cm_id->local_addr,
- sizeof(ep->com.local_addr));
- memcpy(&ep->com.remote_addr, &cm_id->remote_addr,
- sizeof(ep->com.remote_addr));
/* send connect request to rnic */
err = send_connect(ep);
@@ -2856,12 +2935,12 @@
goto out;
cxgb4_l2t_release(ep->l2t);
-fail4:
- dst_release(ep->dst);
fail3:
+ dst_release(ep->dst);
+fail2:
remove_handle(ep->com.dev, &ep->com.dev->atid_idr, ep->atid);
cxgb4_free_atid(ep->com.dev->rdev.lldi.tids, ep->atid);
-fail2:
+fail1:
cm_id->rem_ref(cm_id);
c4iw_put_ep(&ep->com);
out:
@@ -2871,7 +2950,8 @@
static int create_server6(struct c4iw_dev *dev, struct c4iw_listen_ep *ep)
{
int err;
- struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)&ep->com.local_addr;
+ struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)
+ &ep->com.mapped_local_addr;
c4iw_init_wr_wait(&ep->com.wr_wait);
err = cxgb4_create_server6(ep->com.dev->rdev.lldi.ports[0],
@@ -2892,7 +2972,8 @@
static int create_server4(struct c4iw_dev *dev, struct c4iw_listen_ep *ep)
{
int err;
- struct sockaddr_in *sin = (struct sockaddr_in *)&ep->com.local_addr;
+ struct sockaddr_in *sin = (struct sockaddr_in *)
+ &ep->com.mapped_local_addr;
if (dev->rdev.lldi.enable_fw_ofld_conn) {
do {
@@ -2927,6 +3008,9 @@
int err = 0;
struct c4iw_dev *dev = to_c4iw_dev(cm_id->device);
struct c4iw_listen_ep *ep;
+ struct iwpm_dev_data pm_reg_msg;
+ struct iwpm_sa_data pm_msg;
+ int iwpm_err = 0;
might_sleep();
@@ -2961,6 +3045,37 @@
goto fail2;
}
insert_handle(dev, &dev->stid_idr, ep, ep->stid);
+
+ /* No port mapper available, go with the specified info */
+ memcpy(&ep->com.mapped_local_addr, &cm_id->local_addr,
+ sizeof(ep->com.mapped_local_addr));
+
+ c4iw_form_reg_msg(dev, &pm_reg_msg);
+ iwpm_err = iwpm_register_pid(&pm_reg_msg, RDMA_NL_C4IW);
+ if (iwpm_err) {
+ PDBG("%s: Port Mapper reg pid fail (err = %d).\n",
+ __func__, iwpm_err);
+ }
+ if (iwpm_valid_pid() && !iwpm_err) {
+ memcpy(&pm_msg.loc_addr, &ep->com.local_addr,
+ sizeof(ep->com.local_addr));
+ iwpm_err = iwpm_add_mapping(&pm_msg, RDMA_NL_C4IW);
+ if (iwpm_err)
+ PDBG("%s: Port Mapper query fail (err = %d).\n",
+ __func__, iwpm_err);
+ else
+ memcpy(&ep->com.mapped_local_addr,
+ &pm_msg.mapped_loc_addr,
+ sizeof(ep->com.mapped_local_addr));
+ }
+ if (iwpm_create_mapinfo(&ep->com.local_addr,
+ &ep->com.mapped_local_addr, RDMA_NL_C4IW)) {
+ err = -ENOMEM;
+ goto fail3;
+ }
+ print_addr(&ep->com, __func__, "add_mapping/create_mapinfo");
+
+ set_bit(RELEASE_MAPINFO, &ep->com.flags);
state_set(&ep->com, LISTEN);
if (ep->com.local_addr.ss_family == AF_INET)
err = create_server4(dev, ep);
@@ -2970,6 +3085,8 @@
cm_id->provider_data = ep;
goto out;
}
+
+fail3:
cxgb4_free_stid(ep->com.dev->rdev.lldi.tids, ep->stid,
ep->com.local_addr.ss_family);
fail2:
diff --git a/drivers/infiniband/hw/cxgb4/cq.c b/drivers/infiniband/hw/cxgb4/cq.c
index cfaa56a..7151a02 100644
--- a/drivers/infiniband/hw/cxgb4/cq.c
+++ b/drivers/infiniband/hw/cxgb4/cq.c
@@ -940,7 +940,6 @@
if (!mm2)
goto err4;
- memset(&uresp, 0, sizeof(uresp));
uresp.qid_mask = rhp->rdev.cqmask;
uresp.cqid = chp->cq.cqid;
uresp.size = chp->cq.size;
@@ -951,7 +950,8 @@
uresp.gts_key = ucontext->key;
ucontext->key += PAGE_SIZE;
spin_unlock(&ucontext->mmap_lock);
- ret = ib_copy_to_udata(udata, &uresp, sizeof uresp);
+ ret = ib_copy_to_udata(udata, &uresp,
+ sizeof(uresp) - sizeof(uresp.reserved));
if (ret)
goto err5;
diff --git a/drivers/infiniband/hw/cxgb4/device.c b/drivers/infiniband/hw/cxgb4/device.c
index f4fa50a..dd93aad 100644
--- a/drivers/infiniband/hw/cxgb4/device.c
+++ b/drivers/infiniband/hw/cxgb4/device.c
@@ -77,6 +77,16 @@
int pos;
};
+/* registered cxgb4 netlink callbacks */
+static struct ibnl_client_cbs c4iw_nl_cb_table[] = {
+ [RDMA_NL_IWPM_REG_PID] = {.dump = iwpm_register_pid_cb},
+ [RDMA_NL_IWPM_ADD_MAPPING] = {.dump = iwpm_add_mapping_cb},
+ [RDMA_NL_IWPM_QUERY_MAPPING] = {.dump = iwpm_add_and_query_mapping_cb},
+ [RDMA_NL_IWPM_HANDLE_ERR] = {.dump = iwpm_mapping_error_cb},
+ [RDMA_NL_IWPM_MAPINFO] = {.dump = iwpm_mapping_info_cb},
+ [RDMA_NL_IWPM_MAPINFO_NUM] = {.dump = iwpm_ack_mapping_info_cb}
+};
+
static int count_idrs(int id, void *p, void *data)
{
int *countp = data;
@@ -113,35 +123,49 @@
&qp->ep->com.local_addr;
struct sockaddr_in *rsin = (struct sockaddr_in *)
&qp->ep->com.remote_addr;
+ struct sockaddr_in *mapped_lsin = (struct sockaddr_in *)
+ &qp->ep->com.mapped_local_addr;
+ struct sockaddr_in *mapped_rsin = (struct sockaddr_in *)
+ &qp->ep->com.mapped_remote_addr;
cc = snprintf(qpd->buf + qpd->pos, space,
"rc qp sq id %u rq id %u state %u "
"onchip %u ep tid %u state %u "
- "%pI4:%u->%pI4:%u\n",
+ "%pI4:%u/%u->%pI4:%u/%u\n",
qp->wq.sq.qid, qp->wq.rq.qid,
(int)qp->attr.state,
qp->wq.sq.flags & T4_SQ_ONCHIP,
qp->ep->hwtid, (int)qp->ep->com.state,
&lsin->sin_addr, ntohs(lsin->sin_port),
- &rsin->sin_addr, ntohs(rsin->sin_port));
+ ntohs(mapped_lsin->sin_port),
+ &rsin->sin_addr, ntohs(rsin->sin_port),
+ ntohs(mapped_rsin->sin_port));
} else {
struct sockaddr_in6 *lsin6 = (struct sockaddr_in6 *)
&qp->ep->com.local_addr;
struct sockaddr_in6 *rsin6 = (struct sockaddr_in6 *)
&qp->ep->com.remote_addr;
+ struct sockaddr_in6 *mapped_lsin6 =
+ (struct sockaddr_in6 *)
+ &qp->ep->com.mapped_local_addr;
+ struct sockaddr_in6 *mapped_rsin6 =
+ (struct sockaddr_in6 *)
+ &qp->ep->com.mapped_remote_addr;
cc = snprintf(qpd->buf + qpd->pos, space,
"rc qp sq id %u rq id %u state %u "
"onchip %u ep tid %u state %u "
- "%pI6:%u->%pI6:%u\n",
+ "%pI6:%u/%u->%pI6:%u/%u\n",
qp->wq.sq.qid, qp->wq.rq.qid,
(int)qp->attr.state,
qp->wq.sq.flags & T4_SQ_ONCHIP,
qp->ep->hwtid, (int)qp->ep->com.state,
&lsin6->sin6_addr,
ntohs(lsin6->sin6_port),
+ ntohs(mapped_lsin6->sin6_port),
&rsin6->sin6_addr,
- ntohs(rsin6->sin6_port));
+ ntohs(rsin6->sin6_port),
+ ntohs(mapped_rsin6->sin6_port));
}
} else
cc = snprintf(qpd->buf + qpd->pos, space,
@@ -386,31 +410,43 @@
&ep->com.local_addr;
struct sockaddr_in *rsin = (struct sockaddr_in *)
&ep->com.remote_addr;
+ struct sockaddr_in *mapped_lsin = (struct sockaddr_in *)
+ &ep->com.mapped_local_addr;
+ struct sockaddr_in *mapped_rsin = (struct sockaddr_in *)
+ &ep->com.mapped_remote_addr;
cc = snprintf(epd->buf + epd->pos, space,
"ep %p cm_id %p qp %p state %d flags 0x%lx "
"history 0x%lx hwtid %d atid %d "
- "%pI4:%d <-> %pI4:%d\n",
+ "%pI4:%d/%d <-> %pI4:%d/%d\n",
ep, ep->com.cm_id, ep->com.qp,
(int)ep->com.state, ep->com.flags,
ep->com.history, ep->hwtid, ep->atid,
&lsin->sin_addr, ntohs(lsin->sin_port),
- &rsin->sin_addr, ntohs(rsin->sin_port));
+ ntohs(mapped_lsin->sin_port),
+ &rsin->sin_addr, ntohs(rsin->sin_port),
+ ntohs(mapped_rsin->sin_port));
} else {
struct sockaddr_in6 *lsin6 = (struct sockaddr_in6 *)
&ep->com.local_addr;
struct sockaddr_in6 *rsin6 = (struct sockaddr_in6 *)
&ep->com.remote_addr;
+ struct sockaddr_in6 *mapped_lsin6 = (struct sockaddr_in6 *)
+ &ep->com.mapped_local_addr;
+ struct sockaddr_in6 *mapped_rsin6 = (struct sockaddr_in6 *)
+ &ep->com.mapped_remote_addr;
cc = snprintf(epd->buf + epd->pos, space,
"ep %p cm_id %p qp %p state %d flags 0x%lx "
"history 0x%lx hwtid %d atid %d "
- "%pI6:%d <-> %pI6:%d\n",
+ "%pI6:%d/%d <-> %pI6:%d/%d\n",
ep, ep->com.cm_id, ep->com.qp,
(int)ep->com.state, ep->com.flags,
ep->com.history, ep->hwtid, ep->atid,
&lsin6->sin6_addr, ntohs(lsin6->sin6_port),
- &rsin6->sin6_addr, ntohs(rsin6->sin6_port));
+ ntohs(mapped_lsin6->sin6_port),
+ &rsin6->sin6_addr, ntohs(rsin6->sin6_port),
+ ntohs(mapped_rsin6->sin6_port));
}
if (cc < space)
epd->pos += cc;
@@ -431,23 +467,29 @@
if (ep->com.local_addr.ss_family == AF_INET) {
struct sockaddr_in *lsin = (struct sockaddr_in *)
&ep->com.local_addr;
+ struct sockaddr_in *mapped_lsin = (struct sockaddr_in *)
+ &ep->com.mapped_local_addr;
cc = snprintf(epd->buf + epd->pos, space,
"ep %p cm_id %p state %d flags 0x%lx stid %d "
- "backlog %d %pI4:%d\n",
+ "backlog %d %pI4:%d/%d\n",
ep, ep->com.cm_id, (int)ep->com.state,
ep->com.flags, ep->stid, ep->backlog,
- &lsin->sin_addr, ntohs(lsin->sin_port));
+ &lsin->sin_addr, ntohs(lsin->sin_port),
+ ntohs(mapped_lsin->sin_port));
} else {
struct sockaddr_in6 *lsin6 = (struct sockaddr_in6 *)
&ep->com.local_addr;
+ struct sockaddr_in6 *mapped_lsin6 = (struct sockaddr_in6 *)
+ &ep->com.mapped_local_addr;
cc = snprintf(epd->buf + epd->pos, space,
"ep %p cm_id %p state %d flags 0x%lx stid %d "
- "backlog %d %pI6:%d\n",
+ "backlog %d %pI6:%d/%d\n",
ep, ep->com.cm_id, (int)ep->com.state,
ep->com.flags, ep->stid, ep->backlog,
- &lsin6->sin6_addr, ntohs(lsin6->sin6_port));
+ &lsin6->sin6_addr, ntohs(lsin6->sin6_port),
+ ntohs(mapped_lsin6->sin6_port));
}
if (cc < space)
epd->pos += cc;
@@ -687,6 +729,7 @@
if (ctx->dev->rdev.oc_mw_kva)
iounmap(ctx->dev->rdev.oc_mw_kva);
ib_dealloc_device(&ctx->dev->ibdev);
+ iwpm_exit(RDMA_NL_C4IW);
ctx->dev = NULL;
}
@@ -736,6 +779,7 @@
pci_resource_len(devp->rdev.lldi.pdev, 2));
if (!devp->rdev.bar2_kva) {
pr_err(MOD "Unable to ioremap BAR2\n");
+ ib_dealloc_device(&devp->ibdev);
return ERR_PTR(-EINVAL);
}
} else if (ocqp_supported(infop)) {
@@ -747,6 +791,7 @@
devp->rdev.lldi.vr->ocq.size);
if (!devp->rdev.oc_mw_kva) {
pr_err(MOD "Unable to ioremap onchip mem\n");
+ ib_dealloc_device(&devp->ibdev);
return ERR_PTR(-EINVAL);
}
}
@@ -780,6 +825,14 @@
c4iw_debugfs_root);
setup_debugfs(devp);
}
+
+ ret = iwpm_init(RDMA_NL_C4IW);
+ if (ret) {
+ pr_err("port mapper initialization failed with %d\n", ret);
+ ib_dealloc_device(&devp->ibdev);
+ return ERR_PTR(ret);
+ }
+
return devp;
}
@@ -1274,6 +1327,11 @@
printk(KERN_WARNING MOD
"could not create debugfs entry, continuing\n");
+ if (ibnl_add_client(RDMA_NL_C4IW, RDMA_NL_IWPM_NUM_OPS,
+ c4iw_nl_cb_table))
+ pr_err("%s[%u]: Failed to add netlink callback\n"
+ , __func__, __LINE__);
+
cxgb4_register_uld(CXGB4_ULD_RDMA, &c4iw_uld_info);
return 0;
@@ -1291,6 +1349,7 @@
}
mutex_unlock(&dev_mutex);
cxgb4_unregister_uld(CXGB4_ULD_RDMA);
+ ibnl_remove_client(RDMA_NL_C4IW);
c4iw_cm_term();
debugfs_remove_recursive(c4iw_debugfs_root);
}
diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
index 7474b49..6f533fb 100644
--- a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
+++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
@@ -52,6 +52,8 @@
#include <rdma/ib_verbs.h>
#include <rdma/iw_cm.h>
+#include <rdma/rdma_netlink.h>
+#include <rdma/iw_portmap.h>
#include "cxgb4.h"
#include "cxgb4_uld.h"
@@ -728,6 +730,7 @@
CLOSE_SENT = 3,
TIMEOUT = 4,
QP_REFERENCED = 5,
+ RELEASE_MAPINFO = 6,
};
enum c4iw_ep_history {
@@ -764,6 +767,8 @@
struct mutex mutex;
struct sockaddr_storage local_addr;
struct sockaddr_storage remote_addr;
+ struct sockaddr_storage mapped_local_addr;
+ struct sockaddr_storage mapped_remote_addr;
struct c4iw_wr_wait wr_wait;
unsigned long flags;
unsigned long history;
@@ -807,6 +812,45 @@
unsigned int retry_count;
};
+static inline void print_addr(struct c4iw_ep_common *epc, const char *func,
+ const char *msg)
+{
+
+#define SINA(a) (&(((struct sockaddr_in *)(a))->sin_addr.s_addr))
+#define SINP(a) ntohs(((struct sockaddr_in *)(a))->sin_port)
+#define SIN6A(a) (&(((struct sockaddr_in6 *)(a))->sin6_addr))
+#define SIN6P(a) ntohs(((struct sockaddr_in6 *)(a))->sin6_port)
+
+ if (c4iw_debug) {
+ switch (epc->local_addr.ss_family) {
+ case AF_INET:
+ PDBG("%s %s %pI4:%u/%u <-> %pI4:%u/%u\n",
+ func, msg, SINA(&epc->local_addr),
+ SINP(&epc->local_addr),
+ SINP(&epc->mapped_local_addr),
+ SINA(&epc->remote_addr),
+ SINP(&epc->remote_addr),
+ SINP(&epc->mapped_remote_addr));
+ break;
+ case AF_INET6:
+ PDBG("%s %s %pI6:%u/%u <-> %pI6:%u/%u\n",
+ func, msg, SIN6A(&epc->local_addr),
+ SIN6P(&epc->local_addr),
+ SIN6P(&epc->mapped_local_addr),
+ SIN6A(&epc->remote_addr),
+ SIN6P(&epc->remote_addr),
+ SIN6P(&epc->mapped_remote_addr));
+ break;
+ default:
+ break;
+ }
+ }
+#undef SINA
+#undef SINP
+#undef SIN6A
+#undef SIN6P
+}
+
static inline struct c4iw_ep *to_ep(struct iw_cm_id *cm_id)
{
return cm_id->provider_data;
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index a94a3e1..c777e22 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -122,7 +122,7 @@
INIT_LIST_HEAD(&context->mmaps);
spin_lock_init(&context->mmap_lock);
- if (udata->outlen < sizeof(uresp)) {
+ if (udata->outlen < sizeof(uresp) - sizeof(uresp.reserved)) {
if (!warned++)
pr_err(MOD "Warning - downlevel libcxgb4 (non-fatal), device status page disabled.");
rhp->rdev.flags |= T4_STATUS_PAGE_DISABLED;
@@ -140,7 +140,8 @@
context->key += PAGE_SIZE;
spin_unlock(&context->mmap_lock);
- ret = ib_copy_to_udata(udata, &uresp, sizeof(uresp));
+ ret = ib_copy_to_udata(udata, &uresp,
+ sizeof(uresp) - sizeof(uresp.reserved));
if (ret)
goto err_mm;
diff --git a/drivers/infiniband/hw/cxgb4/user.h b/drivers/infiniband/hw/cxgb4/user.h
index 11ccd27..cbd0ce1 100644
--- a/drivers/infiniband/hw/cxgb4/user.h
+++ b/drivers/infiniband/hw/cxgb4/user.h
@@ -48,6 +48,7 @@
__u32 cqid;
__u32 size;
__u32 qid_mask;
+ __u32 reserved; /* explicit padding (optional for i386) */
};
@@ -74,5 +75,6 @@
struct c4iw_alloc_ucontext_resp {
__u64 status_page_key;
__u32 status_page_size;
+ __u32 reserved; /* explicit padding (optional for i386) */
};
#endif
diff --git a/drivers/infiniband/hw/ipath/ipath_diag.c b/drivers/infiniband/hw/ipath/ipath_diag.c
index e2f9a51..45802e97 100644
--- a/drivers/infiniband/hw/ipath/ipath_diag.c
+++ b/drivers/infiniband/hw/ipath/ipath_diag.c
@@ -346,6 +346,10 @@
ret = -EFAULT;
goto bail;
}
+ dp.len = odp.len;
+ dp.unit = odp.unit;
+ dp.data = odp.data;
+ dp.pbc_wd = 0;
} else {
ret = -EINVAL;
goto bail;
diff --git a/drivers/infiniband/hw/ipath/ipath_intr.c b/drivers/infiniband/hw/ipath/ipath_intr.c
index 26dfbc8..01ba792 100644
--- a/drivers/infiniband/hw/ipath/ipath_intr.c
+++ b/drivers/infiniband/hw/ipath/ipath_intr.c
@@ -70,7 +70,7 @@
if (sbuf[0] || sbuf[1] || (piobcnt > 128 && (sbuf[2] || sbuf[3]))) {
int i;
if (ipath_debug & (__IPATH_PKTDBG|__IPATH_DBG) &&
- dd->ipath_lastcancel > jiffies) {
+ time_after(dd->ipath_lastcancel, jiffies)) {
__IPATH_DBG_WHICH(__IPATH_PKTDBG|__IPATH_DBG,
"SendbufErrs %lx %lx", sbuf[0],
sbuf[1]);
@@ -755,7 +755,7 @@
/* likely due to cancel; so suppress message unless verbose */
if ((errs & (INFINIPATH_E_SPKTLEN | INFINIPATH_E_SPIOARMLAUNCH)) &&
- dd->ipath_lastcancel > jiffies) {
+ time_after(dd->ipath_lastcancel, jiffies)) {
/* armlaunch takes precedence; it often causes both. */
ipath_cdbg(VERBOSE,
"Suppressed %s error (%llx) after sendbuf cancel\n",
diff --git a/drivers/infiniband/hw/ipath/ipath_sdma.c b/drivers/infiniband/hw/ipath/ipath_sdma.c
index 98ac18e..17a5177 100644
--- a/drivers/infiniband/hw/ipath/ipath_sdma.c
+++ b/drivers/infiniband/hw/ipath/ipath_sdma.c
@@ -247,7 +247,7 @@
/* ipath_sdma_abort() is done, waiting for interrupt */
if (status == IPATH_SDMA_ABORT_DISARMED) {
- if (jiffies < dd->ipath_sdma_abort_intr_timeout)
+ if (time_before(jiffies, dd->ipath_sdma_abort_intr_timeout))
goto resched_noprint;
/* give up, intr got lost somewhere */
ipath_dbg("give up waiting for SDMADISABLED intr\n");
@@ -341,7 +341,7 @@
* JAG - this is bad to just have default be a loop without
* state change
*/
- if (jiffies > dd->ipath_sdma_abort_jiffies) {
+ if (time_after(jiffies, dd->ipath_sdma_abort_jiffies)) {
ipath_dbg("looping with status 0x%08lx\n",
dd->ipath_sdma_status);
dd->ipath_sdma_abort_jiffies = jiffies + 5 * HZ;
diff --git a/drivers/infiniband/hw/mlx4/ah.c b/drivers/infiniband/hw/mlx4/ah.c
index 170dca6..2d8c339 100644
--- a/drivers/infiniband/hw/mlx4/ah.c
+++ b/drivers/infiniband/hw/mlx4/ah.c
@@ -73,7 +73,7 @@
{
struct mlx4_ib_dev *ibdev = to_mdev(pd->device);
struct mlx4_dev *dev = ibdev->dev;
- int is_mcast;
+ int is_mcast = 0;
struct in6_addr in6;
u16 vlan_tag;
diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c
index 5f64081..1066eec 100644
--- a/drivers/infiniband/hw/mlx4/cq.c
+++ b/drivers/infiniband/hw/mlx4/cq.c
@@ -102,7 +102,7 @@
int err;
err = mlx4_buf_alloc(dev->dev, nent * dev->dev->caps.cqe_size,
- PAGE_SIZE * 2, &buf->buf);
+ PAGE_SIZE * 2, &buf->buf, GFP_KERNEL);
if (err)
goto out;
@@ -113,7 +113,7 @@
if (err)
goto err_buf;
- err = mlx4_buf_write_mtt(dev->dev, &buf->mtt, &buf->buf);
+ err = mlx4_buf_write_mtt(dev->dev, &buf->mtt, &buf->buf, GFP_KERNEL);
if (err)
goto err_mtt;
@@ -209,7 +209,7 @@
uar = &to_mucontext(context)->uar;
} else {
- err = mlx4_db_alloc(dev->dev, &cq->db, 1);
+ err = mlx4_db_alloc(dev->dev, &cq->db, 1, GFP_KERNEL);
if (err)
goto err_cq;
diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c
index fd36ec6..287ad05 100644
--- a/drivers/infiniband/hw/mlx4/mad.c
+++ b/drivers/infiniband/hw/mlx4/mad.c
@@ -478,10 +478,6 @@
if (!tun_ctx || tun_ctx->state != DEMUX_PV_STATE_ACTIVE)
return -EAGAIN;
- /* QP0 forwarding only for Dom0 */
- if (!dest_qpt && (mlx4_master_func_num(dev->dev) != slave))
- return -EINVAL;
-
if (!dest_qpt)
tun_qp = &tun_ctx->qp[0];
else
@@ -667,6 +663,21 @@
}
/* Class-specific handling */
switch (mad->mad_hdr.mgmt_class) {
+ case IB_MGMT_CLASS_SUBN_LID_ROUTED:
+ case IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE:
+ /* 255 indicates the dom0 */
+ if (slave != 255 && slave != mlx4_master_func_num(dev->dev)) {
+ if (!mlx4_vf_smi_enabled(dev->dev, slave, port))
+ return -EPERM;
+ /* for a VF. drop unsolicited MADs */
+ if (!(mad->mad_hdr.method & IB_MGMT_METHOD_RESP)) {
+ mlx4_ib_warn(ibdev, "demux QP0. rejecting unsolicited mad for slave %d class 0x%x, method 0x%x\n",
+ slave, mad->mad_hdr.mgmt_class,
+ mad->mad_hdr.method);
+ return -EINVAL;
+ }
+ }
+ break;
case IB_MGMT_CLASS_SUBN_ADM:
if (mlx4_ib_demux_sa_handler(ibdev, port, slave,
(struct ib_sa_mad *) mad))
@@ -1165,10 +1176,6 @@
if (!sqp_ctx || sqp_ctx->state != DEMUX_PV_STATE_ACTIVE)
return -EAGAIN;
- /* QP0 forwarding only for Dom0 */
- if (dest_qpt == IB_QPT_SMI && (mlx4_master_func_num(dev->dev) != slave))
- return -EINVAL;
-
if (dest_qpt == IB_QPT_SMI) {
src_qpnum = 0;
sqp = &sqp_ctx->qp[0];
@@ -1285,11 +1292,6 @@
"belongs to another slave\n", wc->src_qp);
return;
}
- if (slave != mlx4_master_func_num(dev->dev) && !(wc->src_qp & 0x2)) {
- mlx4_ib_warn(ctx->ib_dev, "can't multiplex bad sqp:%d: "
- "non-master trying to send QP0 packets\n", wc->src_qp);
- return;
- }
/* Map transaction ID */
ib_dma_sync_single_for_cpu(ctx->ib_dev, tun_qp->ring[wr_ix].map,
@@ -1317,6 +1319,12 @@
/* Class-specific handling */
switch (tunnel->mad.mad_hdr.mgmt_class) {
+ case IB_MGMT_CLASS_SUBN_LID_ROUTED:
+ case IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE:
+ if (slave != mlx4_master_func_num(dev->dev) &&
+ !mlx4_vf_smi_enabled(dev->dev, slave, ctx->port))
+ return;
+ break;
case IB_MGMT_CLASS_SUBN_ADM:
if (mlx4_ib_multiplex_sa_handler(ctx->ib_dev, ctx->port, slave,
(struct ib_sa_mad *) &tunnel->mad))
@@ -1749,9 +1757,9 @@
return -EEXIST;
ctx->state = DEMUX_PV_STATE_STARTING;
- /* have QP0 only on port owner, and only if link layer is IB */
- if (ctx->slave == mlx4_master_func_num(to_mdev(ctx->ib_dev)->dev) &&
- rdma_port_get_link_layer(ibdev, ctx->port) == IB_LINK_LAYER_INFINIBAND)
+ /* have QP0 only if link layer is IB */
+ if (rdma_port_get_link_layer(ibdev, ctx->port) ==
+ IB_LINK_LAYER_INFINIBAND)
ctx->has_smi = 1;
if (ctx->has_smi) {
diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c
index 199c789..0f7027e 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -545,12 +545,11 @@
return 0;
}
-static int mlx4_SET_PORT(struct mlx4_ib_dev *dev, u8 port, int reset_qkey_viols,
- u32 cap_mask)
+static int mlx4_ib_SET_PORT(struct mlx4_ib_dev *dev, u8 port, int reset_qkey_viols,
+ u32 cap_mask)
{
struct mlx4_cmd_mailbox *mailbox;
int err;
- u8 is_eth = dev->dev->caps.port_type[port] == MLX4_PORT_TYPE_ETH;
mailbox = mlx4_alloc_cmd_mailbox(dev->dev);
if (IS_ERR(mailbox))
@@ -564,8 +563,8 @@
((__be32 *) mailbox->buf)[1] = cpu_to_be32(cap_mask);
}
- err = mlx4_cmd(dev->dev, mailbox->dma, port, is_eth, MLX4_CMD_SET_PORT,
- MLX4_CMD_TIME_CLASS_B, MLX4_CMD_NATIVE);
+ err = mlx4_cmd(dev->dev, mailbox->dma, port, 0, MLX4_CMD_SET_PORT,
+ MLX4_CMD_TIME_CLASS_B, MLX4_CMD_WRAPPED);
mlx4_free_cmd_mailbox(dev->dev, mailbox);
return err;
@@ -574,11 +573,20 @@
static int mlx4_ib_modify_port(struct ib_device *ibdev, u8 port, int mask,
struct ib_port_modify *props)
{
+ struct mlx4_ib_dev *mdev = to_mdev(ibdev);
+ u8 is_eth = mdev->dev->caps.port_type[port] == MLX4_PORT_TYPE_ETH;
struct ib_port_attr attr;
u32 cap_mask;
int err;
- mutex_lock(&to_mdev(ibdev)->cap_mask_mutex);
+ /* return OK if this is RoCE. CM calls ib_modify_port() regardless
+ * of whether port link layer is ETH or IB. For ETH ports, qkey
+ * violations and port capabilities are not meaningful.
+ */
+ if (is_eth)
+ return 0;
+
+ mutex_lock(&mdev->cap_mask_mutex);
err = mlx4_ib_query_port(ibdev, port, &attr);
if (err)
@@ -587,9 +595,9 @@
cap_mask = (attr.port_cap_flags | props->set_port_cap_mask) &
~props->clr_port_cap_mask;
- err = mlx4_SET_PORT(to_mdev(ibdev), port,
- !!(mask & IB_PORT_RESET_QKEY_CNTR),
- cap_mask);
+ err = mlx4_ib_SET_PORT(mdev, port,
+ !!(mask & IB_PORT_RESET_QKEY_CNTR),
+ cap_mask);
out:
mutex_unlock(&to_mdev(ibdev)->cap_mask_mutex);
diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h
index 66b0b7d..369da3c 100644
--- a/drivers/infiniband/hw/mlx4/mlx4_ib.h
+++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h
@@ -156,6 +156,7 @@
MLX4_IB_QP_LSO = IB_QP_CREATE_IPOIB_UD_LSO,
MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK = IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK,
MLX4_IB_QP_NETIF = IB_QP_CREATE_NETIF_QP,
+ MLX4_IB_QP_CREATE_USE_GFP_NOIO = IB_QP_CREATE_USE_GFP_NOIO,
MLX4_IB_SRIOV_TUNNEL_QP = 1 << 30,
MLX4_IB_SRIOV_SQP = 1 << 31,
};
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index dc57482..6778045 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -608,9 +608,20 @@
return !attr->srq;
}
+static int qp0_enabled_vf(struct mlx4_dev *dev, int qpn)
+{
+ int i;
+ for (i = 0; i < dev->caps.num_ports; i++) {
+ if (qpn == dev->caps.qp0_proxy[i])
+ return !!dev->caps.qp0_qkey[i];
+ }
+ return 0;
+}
+
static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd,
struct ib_qp_init_attr *init_attr,
- struct ib_udata *udata, int sqpn, struct mlx4_ib_qp **caller_qp)
+ struct ib_udata *udata, int sqpn, struct mlx4_ib_qp **caller_qp,
+ gfp_t gfp)
{
int qpn;
int err;
@@ -625,10 +636,13 @@
!(init_attr->create_flags & MLX4_IB_SRIOV_SQP))) {
if (init_attr->qp_type == IB_QPT_GSI)
qp_type = MLX4_IB_QPT_PROXY_GSI;
- else if (mlx4_is_master(dev->dev))
- qp_type = MLX4_IB_QPT_PROXY_SMI_OWNER;
- else
- qp_type = MLX4_IB_QPT_PROXY_SMI;
+ else {
+ if (mlx4_is_master(dev->dev) ||
+ qp0_enabled_vf(dev->dev, sqpn))
+ qp_type = MLX4_IB_QPT_PROXY_SMI_OWNER;
+ else
+ qp_type = MLX4_IB_QPT_PROXY_SMI;
+ }
}
qpn = sqpn;
/* add extra sg entry for tunneling */
@@ -643,7 +657,9 @@
return -EINVAL;
if (tnl_init->proxy_qp_type == IB_QPT_GSI)
qp_type = MLX4_IB_QPT_TUN_GSI;
- else if (tnl_init->slave == mlx4_master_func_num(dev->dev))
+ else if (tnl_init->slave == mlx4_master_func_num(dev->dev) ||
+ mlx4_vf_smi_enabled(dev->dev, tnl_init->slave,
+ tnl_init->port))
qp_type = MLX4_IB_QPT_TUN_SMI_OWNER;
else
qp_type = MLX4_IB_QPT_TUN_SMI;
@@ -658,14 +674,14 @@
if (qp_type == MLX4_IB_QPT_SMI || qp_type == MLX4_IB_QPT_GSI ||
(qp_type & (MLX4_IB_QPT_PROXY_SMI | MLX4_IB_QPT_PROXY_SMI_OWNER |
MLX4_IB_QPT_PROXY_GSI | MLX4_IB_QPT_TUN_SMI_OWNER))) {
- sqp = kzalloc(sizeof (struct mlx4_ib_sqp), GFP_KERNEL);
+ sqp = kzalloc(sizeof (struct mlx4_ib_sqp), gfp);
if (!sqp)
return -ENOMEM;
qp = &sqp->qp;
qp->pri.vid = 0xFFFF;
qp->alt.vid = 0xFFFF;
} else {
- qp = kzalloc(sizeof (struct mlx4_ib_qp), GFP_KERNEL);
+ qp = kzalloc(sizeof (struct mlx4_ib_qp), gfp);
if (!qp)
return -ENOMEM;
qp->pri.vid = 0xFFFF;
@@ -748,14 +764,14 @@
goto err;
if (qp_has_rq(init_attr)) {
- err = mlx4_db_alloc(dev->dev, &qp->db, 0);
+ err = mlx4_db_alloc(dev->dev, &qp->db, 0, gfp);
if (err)
goto err;
*qp->db.db = 0;
}
- if (mlx4_buf_alloc(dev->dev, qp->buf_size, PAGE_SIZE * 2, &qp->buf)) {
+ if (mlx4_buf_alloc(dev->dev, qp->buf_size, PAGE_SIZE * 2, &qp->buf, gfp)) {
err = -ENOMEM;
goto err_db;
}
@@ -765,13 +781,12 @@
if (err)
goto err_buf;
- err = mlx4_buf_write_mtt(dev->dev, &qp->mtt, &qp->buf);
+ err = mlx4_buf_write_mtt(dev->dev, &qp->mtt, &qp->buf, gfp);
if (err)
goto err_mtt;
- qp->sq.wrid = kmalloc(qp->sq.wqe_cnt * sizeof (u64), GFP_KERNEL);
- qp->rq.wrid = kmalloc(qp->rq.wqe_cnt * sizeof (u64), GFP_KERNEL);
-
+ qp->sq.wrid = kmalloc(qp->sq.wqe_cnt * sizeof (u64), gfp);
+ qp->rq.wrid = kmalloc(qp->rq.wqe_cnt * sizeof (u64), gfp);
if (!qp->sq.wrid || !qp->rq.wrid) {
err = -ENOMEM;
goto err_wrid;
@@ -801,7 +816,7 @@
goto err_proxy;
}
- err = mlx4_qp_alloc(dev->dev, qpn, &qp->mqp);
+ err = mlx4_qp_alloc(dev->dev, qpn, &qp->mqp, gfp);
if (err)
goto err_qpn;
@@ -1040,7 +1055,10 @@
struct mlx4_ib_qp *qp = NULL;
int err;
u16 xrcdn = 0;
+ gfp_t gfp;
+ gfp = (init_attr->create_flags & MLX4_IB_QP_CREATE_USE_GFP_NOIO) ?
+ GFP_NOIO : GFP_KERNEL;
/*
* We only support LSO, vendor flag1, and multicast loopback blocking,
* and only for kernel UD QPs.
@@ -1049,7 +1067,8 @@
MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK |
MLX4_IB_SRIOV_TUNNEL_QP |
MLX4_IB_SRIOV_SQP |
- MLX4_IB_QP_NETIF))
+ MLX4_IB_QP_NETIF |
+ MLX4_IB_QP_CREATE_USE_GFP_NOIO))
return ERR_PTR(-EINVAL);
if (init_attr->create_flags & IB_QP_CREATE_NETIF_QP) {
@@ -1059,7 +1078,7 @@
if (init_attr->create_flags &&
(udata ||
- ((init_attr->create_flags & ~MLX4_IB_SRIOV_SQP) &&
+ ((init_attr->create_flags & ~(MLX4_IB_SRIOV_SQP | MLX4_IB_QP_CREATE_USE_GFP_NOIO)) &&
init_attr->qp_type != IB_QPT_UD) ||
((init_attr->create_flags & MLX4_IB_SRIOV_SQP) &&
init_attr->qp_type > IB_QPT_GSI)))
@@ -1079,7 +1098,7 @@
case IB_QPT_RC:
case IB_QPT_UC:
case IB_QPT_RAW_PACKET:
- qp = kzalloc(sizeof *qp, GFP_KERNEL);
+ qp = kzalloc(sizeof *qp, gfp);
if (!qp)
return ERR_PTR(-ENOMEM);
qp->pri.vid = 0xFFFF;
@@ -1088,7 +1107,7 @@
case IB_QPT_UD:
{
err = create_qp_common(to_mdev(pd->device), pd, init_attr,
- udata, 0, &qp);
+ udata, 0, &qp, gfp);
if (err)
return ERR_PTR(err);
@@ -1106,7 +1125,7 @@
err = create_qp_common(to_mdev(pd->device), pd, init_attr, udata,
get_sqp_num(to_mdev(pd->device), init_attr),
- &qp);
+ &qp, gfp);
if (err)
return ERR_PTR(err);
@@ -1938,6 +1957,19 @@
return err;
}
+static int vf_get_qp0_qkey(struct mlx4_dev *dev, int qpn, u32 *qkey)
+{
+ int i;
+ for (i = 0; i < dev->caps.num_ports; i++) {
+ if (qpn == dev->caps.qp0_proxy[i] ||
+ qpn == dev->caps.qp0_tunnel[i]) {
+ *qkey = dev->caps.qp0_qkey[i];
+ return 0;
+ }
+ }
+ return -EINVAL;
+}
+
static int build_sriov_qp0_header(struct mlx4_ib_sqp *sqp,
struct ib_send_wr *wr,
void *wqe, unsigned *mlx_seg_len)
@@ -1995,8 +2027,13 @@
cpu_to_be32(mdev->dev->caps.qp0_tunnel[sqp->qp.port - 1]);
sqp->ud_header.bth.psn = cpu_to_be32((sqp->send_psn++) & ((1 << 24) - 1));
- if (mlx4_get_parav_qkey(mdev->dev, sqp->qp.mqp.qpn, &qkey))
- return -EINVAL;
+ if (mlx4_is_master(mdev->dev)) {
+ if (mlx4_get_parav_qkey(mdev->dev, sqp->qp.mqp.qpn, &qkey))
+ return -EINVAL;
+ } else {
+ if (vf_get_qp0_qkey(mdev->dev, sqp->qp.mqp.qpn, &qkey))
+ return -EINVAL;
+ }
sqp->ud_header.deth.qkey = cpu_to_be32(qkey);
sqp->ud_header.deth.source_qpn = cpu_to_be32(sqp->qp.mqp.qpn);
@@ -2378,7 +2415,8 @@
static void set_tunnel_datagram_seg(struct mlx4_ib_dev *dev,
struct mlx4_wqe_datagram_seg *dseg,
- struct ib_send_wr *wr, enum ib_qp_type qpt)
+ struct ib_send_wr *wr,
+ enum mlx4_ib_qp_type qpt)
{
union mlx4_ext_av *av = &to_mah(wr->wr.ud.ah)->av;
struct mlx4_av sqp_av = {0};
@@ -2391,8 +2429,10 @@
cpu_to_be32(0xf0000000);
memcpy(dseg->av, &sqp_av, sizeof (struct mlx4_av));
- /* This function used only for sending on QP1 proxies */
- dseg->dqpn = cpu_to_be32(dev->dev->caps.qp1_tunnel[port - 1]);
+ if (qpt == MLX4_IB_QPT_PROXY_GSI)
+ dseg->dqpn = cpu_to_be32(dev->dev->caps.qp1_tunnel[port - 1]);
+ else
+ dseg->dqpn = cpu_to_be32(dev->dev->caps.qp0_tunnel[port - 1]);
/* Use QKEY from the QP context, which is set by master */
dseg->qkey = cpu_to_be32(IB_QP_SET_QKEY);
}
@@ -2687,11 +2727,6 @@
break;
case MLX4_IB_QPT_PROXY_SMI_OWNER:
- if (unlikely(!mlx4_is_master(to_mdev(ibqp->device)->dev))) {
- err = -ENOSYS;
- *bad_wr = wr;
- goto out;
- }
err = build_sriov_qp0_header(to_msqp(qp), wr, ctrl, &seglen);
if (unlikely(err)) {
*bad_wr = wr;
@@ -2708,16 +2743,13 @@
size += seglen / 16;
break;
case MLX4_IB_QPT_PROXY_SMI:
- /* don't allow QP0 sends on guests */
- err = -ENOSYS;
- *bad_wr = wr;
- goto out;
case MLX4_IB_QPT_PROXY_GSI:
/* If we are tunneling special qps, this is a UD qp.
* In this case we first add a UD segment targeting
* the tunnel qp, and then add a header with address
* information */
- set_tunnel_datagram_seg(to_mdev(ibqp->device), wqe, wr, ibqp->qp_type);
+ set_tunnel_datagram_seg(to_mdev(ibqp->device), wqe, wr,
+ qp->mlx4_ib_qp_type);
wqe += sizeof (struct mlx4_wqe_datagram_seg);
size += sizeof (struct mlx4_wqe_datagram_seg) / 16;
build_tunnel_header(wr, wqe, &seglen);
diff --git a/drivers/infiniband/hw/mlx4/srq.c b/drivers/infiniband/hw/mlx4/srq.c
index 60c5fb0..62d9285 100644
--- a/drivers/infiniband/hw/mlx4/srq.c
+++ b/drivers/infiniband/hw/mlx4/srq.c
@@ -134,13 +134,14 @@
if (err)
goto err_mtt;
} else {
- err = mlx4_db_alloc(dev->dev, &srq->db, 0);
+ err = mlx4_db_alloc(dev->dev, &srq->db, 0, GFP_KERNEL);
if (err)
goto err_srq;
*srq->db.db = 0;
- if (mlx4_buf_alloc(dev->dev, buf_size, PAGE_SIZE * 2, &srq->buf)) {
+ if (mlx4_buf_alloc(dev->dev, buf_size, PAGE_SIZE * 2, &srq->buf,
+ GFP_KERNEL)) {
err = -ENOMEM;
goto err_db;
}
@@ -165,7 +166,7 @@
if (err)
goto err_buf;
- err = mlx4_buf_write_mtt(dev->dev, &srq->mtt, &srq->buf);
+ err = mlx4_buf_write_mtt(dev->dev, &srq->mtt, &srq->buf, GFP_KERNEL);
if (err)
goto err_mtt;
diff --git a/drivers/infiniband/hw/mlx4/sysfs.c b/drivers/infiniband/hw/mlx4/sysfs.c
index 5a38e43e..cb4c66e 100644
--- a/drivers/infiniband/hw/mlx4/sysfs.c
+++ b/drivers/infiniband/hw/mlx4/sysfs.c
@@ -389,8 +389,10 @@
struct mlx4_ib_dev *dev;
struct attribute_group pkey_group;
struct attribute_group gid_group;
- u8 port_num;
+ struct device_attribute enable_smi_admin;
+ struct device_attribute smi_enabled;
int slave;
+ u8 port_num;
};
@@ -558,6 +560,101 @@
return NULL;
}
+static ssize_t sysfs_show_smi_enabled(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct mlx4_port *p =
+ container_of(attr, struct mlx4_port, smi_enabled);
+ ssize_t len = 0;
+
+ if (mlx4_vf_smi_enabled(p->dev->dev, p->slave, p->port_num))
+ len = sprintf(buf, "%d\n", 1);
+ else
+ len = sprintf(buf, "%d\n", 0);
+
+ return len;
+}
+
+static ssize_t sysfs_show_enable_smi_admin(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct mlx4_port *p =
+ container_of(attr, struct mlx4_port, enable_smi_admin);
+ ssize_t len = 0;
+
+ if (mlx4_vf_get_enable_smi_admin(p->dev->dev, p->slave, p->port_num))
+ len = sprintf(buf, "%d\n", 1);
+ else
+ len = sprintf(buf, "%d\n", 0);
+
+ return len;
+}
+
+static ssize_t sysfs_store_enable_smi_admin(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct mlx4_port *p =
+ container_of(attr, struct mlx4_port, enable_smi_admin);
+ int enable;
+
+ if (sscanf(buf, "%i", &enable) != 1 ||
+ enable < 0 || enable > 1)
+ return -EINVAL;
+
+ if (mlx4_vf_set_enable_smi_admin(p->dev->dev, p->slave, p->port_num, enable))
+ return -EINVAL;
+ return count;
+}
+
+static int add_vf_smi_entries(struct mlx4_port *p)
+{
+ int is_eth = rdma_port_get_link_layer(&p->dev->ib_dev, p->port_num) ==
+ IB_LINK_LAYER_ETHERNET;
+ int ret;
+
+ /* do not display entries if eth transport, or if master */
+ if (is_eth || p->slave == mlx4_master_func_num(p->dev->dev))
+ return 0;
+
+ sysfs_attr_init(&p->smi_enabled.attr);
+ p->smi_enabled.show = sysfs_show_smi_enabled;
+ p->smi_enabled.store = NULL;
+ p->smi_enabled.attr.name = "smi_enabled";
+ p->smi_enabled.attr.mode = 0444;
+ ret = sysfs_create_file(&p->kobj, &p->smi_enabled.attr);
+ if (ret) {
+ pr_err("failed to create smi_enabled\n");
+ return ret;
+ }
+
+ sysfs_attr_init(&p->enable_smi_admin.attr);
+ p->enable_smi_admin.show = sysfs_show_enable_smi_admin;
+ p->enable_smi_admin.store = sysfs_store_enable_smi_admin;
+ p->enable_smi_admin.attr.name = "enable_smi_admin";
+ p->enable_smi_admin.attr.mode = 0644;
+ ret = sysfs_create_file(&p->kobj, &p->enable_smi_admin.attr);
+ if (ret) {
+ pr_err("failed to create enable_smi_admin\n");
+ sysfs_remove_file(&p->kobj, &p->smi_enabled.attr);
+ return ret;
+ }
+ return 0;
+}
+
+static void remove_vf_smi_entries(struct mlx4_port *p)
+{
+ int is_eth = rdma_port_get_link_layer(&p->dev->ib_dev, p->port_num) ==
+ IB_LINK_LAYER_ETHERNET;
+
+ if (is_eth || p->slave == mlx4_master_func_num(p->dev->dev))
+ return;
+
+ sysfs_remove_file(&p->kobj, &p->smi_enabled.attr);
+ sysfs_remove_file(&p->kobj, &p->enable_smi_admin.attr);
+}
+
static int add_port(struct mlx4_ib_dev *dev, int port_num, int slave)
{
struct mlx4_port *p;
@@ -602,6 +699,10 @@
if (ret)
goto err_free_gid;
+ ret = add_vf_smi_entries(p);
+ if (ret)
+ goto err_free_gid;
+
list_add_tail(&p->kobj.entry, &dev->pkeys.pkey_port_list[slave]);
return 0;
@@ -669,6 +770,7 @@
mport = container_of(p, struct mlx4_port, kobj);
sysfs_remove_group(p, &mport->pkey_group);
sysfs_remove_group(p, &mport->gid_group);
+ remove_vf_smi_entries(mport);
kobject_put(p);
}
kobject_put(dev->dev_ports_parent[slave]);
@@ -713,6 +815,7 @@
port = container_of(p, struct mlx4_port, kobj);
sysfs_remove_group(p, &port->pkey_group);
sysfs_remove_group(p, &port->gid_group);
+ remove_vf_smi_entries(port);
kobject_put(p);
kobject_put(device->dev_ports_parent[slave]);
}
diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 62bb6b49d..8ae4f89 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -32,6 +32,7 @@
#include <linux/kref.h>
#include <rdma/ib_umem.h>
+#include <rdma/ib_user_verbs.h>
#include "mlx5_ib.h"
#include "user.h"
@@ -602,14 +603,24 @@
int *cqe_size, int *index, int *inlen)
{
struct mlx5_ib_create_cq ucmd;
+ size_t ucmdlen;
int page_shift;
int npages;
int ncont;
int err;
- if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd)))
+ ucmdlen =
+ (udata->inlen - sizeof(struct ib_uverbs_cmd_hdr) <
+ sizeof(ucmd)) ? (sizeof(ucmd) -
+ sizeof(ucmd.reserved)) : sizeof(ucmd);
+
+ if (ib_copy_from_udata(&ucmd, udata, ucmdlen))
return -EFAULT;
+ if (ucmdlen == sizeof(ucmd) &&
+ ucmd.reserved != 0)
+ return -EINVAL;
+
if (ucmd.cqe_size != 64 && ucmd.cqe_size != 128)
return -EINVAL;
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 5054158..f2ccf1a 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -264,8 +264,6 @@
__be64 *pas;
dma_addr_t dma;
int npages;
- struct completion done;
- enum ib_wc_status status;
struct mlx5_ib_dev *dev;
struct mlx5_create_mkey_mbox_out out;
struct mlx5_core_sig_ctx *sig;
@@ -277,6 +275,17 @@
dma_addr_t map;
};
+struct mlx5_ib_umr_context {
+ enum ib_wc_status status;
+ struct completion done;
+};
+
+static inline void mlx5_ib_init_umr_context(struct mlx5_ib_umr_context *context)
+{
+ context->status = -1;
+ init_completion(&context->done);
+}
+
struct umr_common {
struct ib_pd *pd;
struct ib_cq *cq;
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 81392b2..afa873b 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -73,6 +73,8 @@
struct mlx5_cache_ent *ent = &cache->ent[c];
u8 key;
unsigned long flags;
+ struct mlx5_mr_table *table = &dev->mdev.priv.mr_table;
+ int err;
spin_lock_irqsave(&ent->lock, flags);
ent->pending--;
@@ -107,6 +109,13 @@
ent->cur++;
ent->size++;
spin_unlock_irqrestore(&ent->lock, flags);
+
+ write_lock_irqsave(&table->lock, flags);
+ err = radix_tree_insert(&table->tree, mlx5_base_mkey(mr->mmr.key),
+ &mr->mmr);
+ if (err)
+ pr_err("Error inserting to mr tree. 0x%x\n", -err);
+ write_unlock_irqrestore(&table->lock, flags);
}
static int add_keys(struct mlx5_ib_dev *dev, int c, int num)
@@ -699,7 +708,7 @@
void mlx5_umr_cq_handler(struct ib_cq *cq, void *cq_context)
{
- struct mlx5_ib_mr *mr;
+ struct mlx5_ib_umr_context *context;
struct ib_wc wc;
int err;
@@ -712,9 +721,9 @@
if (err == 0)
break;
- mr = (struct mlx5_ib_mr *)(unsigned long)wc.wr_id;
- mr->status = wc.status;
- complete(&mr->done);
+ context = (struct mlx5_ib_umr_context *) (unsigned long) wc.wr_id;
+ context->status = wc.status;
+ complete(&context->done);
}
ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
}
@@ -726,11 +735,12 @@
struct mlx5_ib_dev *dev = to_mdev(pd->device);
struct device *ddev = dev->ib_dev.dma_device;
struct umr_common *umrc = &dev->umrc;
+ struct mlx5_ib_umr_context umr_context;
struct ib_send_wr wr, *bad;
struct mlx5_ib_mr *mr;
struct ib_sge sg;
int size = sizeof(u64) * npages;
- int err;
+ int err = 0;
int i;
for (i = 0; i < 1; i++) {
@@ -751,7 +761,7 @@
mr->pas = kmalloc(size + MLX5_UMR_ALIGN - 1, GFP_KERNEL);
if (!mr->pas) {
err = -ENOMEM;
- goto error;
+ goto free_mr;
}
mlx5_ib_populate_pas(dev, umem, page_shift,
@@ -760,44 +770,46 @@
mr->dma = dma_map_single(ddev, mr_align(mr->pas, MLX5_UMR_ALIGN), size,
DMA_TO_DEVICE);
if (dma_mapping_error(ddev, mr->dma)) {
- kfree(mr->pas);
err = -ENOMEM;
- goto error;
+ goto free_pas;
}
memset(&wr, 0, sizeof(wr));
- wr.wr_id = (u64)(unsigned long)mr;
+ wr.wr_id = (u64)(unsigned long)&umr_context;
prep_umr_reg_wqe(pd, &wr, &sg, mr->dma, npages, mr->mmr.key, page_shift, virt_addr, len, access_flags);
- /* We serialize polls so one process does not kidnap another's
- * completion. This is not a problem since wr is completed in
- * around 1 usec
- */
+ mlx5_ib_init_umr_context(&umr_context);
down(&umrc->sem);
- init_completion(&mr->done);
err = ib_post_send(umrc->qp, &wr, &bad);
if (err) {
mlx5_ib_warn(dev, "post send failed, err %d\n", err);
- up(&umrc->sem);
- goto error;
+ goto unmap_dma;
+ } else {
+ wait_for_completion(&umr_context.done);
+ if (umr_context.status != IB_WC_SUCCESS) {
+ mlx5_ib_warn(dev, "reg umr failed\n");
+ err = -EFAULT;
+ }
}
- wait_for_completion(&mr->done);
- up(&umrc->sem);
+ mr->mmr.iova = virt_addr;
+ mr->mmr.size = len;
+ mr->mmr.pd = to_mpd(pd)->pdn;
+
+unmap_dma:
+ up(&umrc->sem);
dma_unmap_single(ddev, mr->dma, size, DMA_TO_DEVICE);
+
+free_pas:
kfree(mr->pas);
- if (mr->status != IB_WC_SUCCESS) {
- mlx5_ib_warn(dev, "reg umr failed\n");
- err = -EFAULT;
- goto error;
+free_mr:
+ if (err) {
+ free_cached_mr(dev, mr);
+ return ERR_PTR(err);
}
return mr;
-
-error:
- free_cached_mr(dev, mr);
- return ERR_PTR(err);
}
static struct mlx5_ib_mr *reg_create(struct ib_pd *pd, u64 virt_addr,
@@ -926,24 +938,26 @@
static int unreg_umr(struct mlx5_ib_dev *dev, struct mlx5_ib_mr *mr)
{
struct umr_common *umrc = &dev->umrc;
+ struct mlx5_ib_umr_context umr_context;
struct ib_send_wr wr, *bad;
int err;
memset(&wr, 0, sizeof(wr));
- wr.wr_id = (u64)(unsigned long)mr;
+ wr.wr_id = (u64)(unsigned long)&umr_context;
prep_umr_unreg_wqe(dev, &wr, mr->mmr.key);
+ mlx5_ib_init_umr_context(&umr_context);
down(&umrc->sem);
- init_completion(&mr->done);
err = ib_post_send(umrc->qp, &wr, &bad);
if (err) {
up(&umrc->sem);
mlx5_ib_dbg(dev, "err %d\n", err);
goto error;
+ } else {
+ wait_for_completion(&umr_context.done);
+ up(&umrc->sem);
}
- wait_for_completion(&mr->done);
- up(&umrc->sem);
- if (mr->status != IB_WC_SUCCESS) {
+ if (umr_context.status != IB_WC_SUCCESS) {
mlx5_ib_warn(dev, "unreg umr failed\n");
err = -EFAULT;
goto error;
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index dc930ed..d13ddf1 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -574,6 +574,10 @@
uar_index = uuarn_to_uar_index(&context->uuari, uuarn);
mlx5_ib_dbg(dev, "uuarn 0x%x, uar_index 0x%x\n", uuarn, uar_index);
+ qp->rq.offset = 0;
+ qp->sq.wqe_shift = ilog2(MLX5_SEND_WQE_BB);
+ qp->sq.offset = qp->rq.wqe_cnt << qp->rq.wqe_shift;
+
err = set_user_buf_size(dev, qp, &ucmd);
if (err)
goto err_uuar;
@@ -2078,6 +2082,7 @@
struct ib_sig_domain *wire = &sig_attrs->wire;
int ret, selector;
+ memset(bsf, 0, sizeof(*bsf));
switch (sig_attrs->mem.sig_type) {
case IB_SIG_TYPE_T10_DIF:
if (sig_attrs->wire.sig_type != IB_SIG_TYPE_T10_DIF)
@@ -2090,9 +2095,11 @@
/* Same block structure */
basic->bsf_size_sbs = 1 << 4;
if (mem->sig.dif.bg_type == wire->sig.dif.bg_type)
- basic->wire.copy_byte_mask = 0xff;
- else
- basic->wire.copy_byte_mask = 0x3f;
+ basic->wire.copy_byte_mask |= 0xc0;
+ if (mem->sig.dif.app_tag == wire->sig.dif.app_tag)
+ basic->wire.copy_byte_mask |= 0x30;
+ if (mem->sig.dif.ref_tag == wire->sig.dif.ref_tag)
+ basic->wire.copy_byte_mask |= 0x0f;
} else
basic->wire.bs_selector = bs_selector(wire->sig.dif.pi_interval);
@@ -2131,9 +2138,13 @@
int ret;
int wqe_size;
- if (!wr->wr.sig_handover.prot) {
+ if (!wr->wr.sig_handover.prot ||
+ (data_key == wr->wr.sig_handover.prot->lkey &&
+ data_va == wr->wr.sig_handover.prot->addr &&
+ data_len == wr->wr.sig_handover.prot->length)) {
/**
* Source domain doesn't contain signature information
+ * or data and protection are interleaved in memory.
* So need construct:
* ------------------
* | data_klm |
@@ -2187,23 +2198,13 @@
data_sentry->bcount = cpu_to_be16(block_size);
data_sentry->key = cpu_to_be32(data_key);
data_sentry->va = cpu_to_be64(data_va);
+ data_sentry->stride = cpu_to_be16(block_size);
+
prot_sentry->bcount = cpu_to_be16(prot_size);
prot_sentry->key = cpu_to_be32(prot_key);
+ prot_sentry->va = cpu_to_be64(prot_va);
+ prot_sentry->stride = cpu_to_be16(prot_size);
- if (prot_key == data_key && prot_va == data_va) {
- /**
- * The data and protection are interleaved
- * in a single memory region
- **/
- prot_sentry->va = cpu_to_be64(data_va + block_size);
- prot_sentry->stride = cpu_to_be16(block_size + prot_size);
- data_sentry->stride = prot_sentry->stride;
- } else {
- /* The data and protection are two different buffers */
- prot_sentry->va = cpu_to_be64(prot_va);
- data_sentry->stride = cpu_to_be16(block_size);
- prot_sentry->stride = cpu_to_be16(prot_size);
- }
wqe_size = ALIGN(sizeof(*sblock_ctrl) + sizeof(*data_sentry) +
sizeof(*prot_sentry), 64);
}
@@ -2275,7 +2276,10 @@
/* length of the protected region, data + protection */
region_len = wr->sg_list->length;
- if (wr->wr.sig_handover.prot)
+ if (wr->wr.sig_handover.prot &&
+ (wr->wr.sig_handover.prot->lkey != wr->sg_list->lkey ||
+ wr->wr.sig_handover.prot->addr != wr->sg_list->addr ||
+ wr->wr.sig_handover.prot->length != wr->sg_list->length))
region_len += wr->wr.sig_handover.prot->length;
/**
diff --git a/drivers/infiniband/hw/mlx5/srq.c b/drivers/infiniband/hw/mlx5/srq.c
index 210b3ea..384af6d 100644
--- a/drivers/infiniband/hw/mlx5/srq.c
+++ b/drivers/infiniband/hw/mlx5/srq.c
@@ -35,6 +35,7 @@
#include <linux/mlx5/srq.h>
#include <linux/slab.h>
#include <rdma/ib_umem.h>
+#include <rdma/ib_user_verbs.h>
#include "mlx5_ib.h"
#include "user.h"
@@ -78,16 +79,27 @@
{
struct mlx5_ib_dev *dev = to_mdev(pd->device);
struct mlx5_ib_create_srq ucmd;
+ size_t ucmdlen;
int err;
int npages;
int page_shift;
int ncont;
u32 offset;
- if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd))) {
+ ucmdlen =
+ (udata->inlen - sizeof(struct ib_uverbs_cmd_hdr) <
+ sizeof(ucmd)) ? (sizeof(ucmd) -
+ sizeof(ucmd.reserved)) : sizeof(ucmd);
+
+ if (ib_copy_from_udata(&ucmd, udata, ucmdlen)) {
mlx5_ib_dbg(dev, "failed copy udata\n");
return -EFAULT;
}
+
+ if (ucmdlen == sizeof(ucmd) &&
+ ucmd.reserved != 0)
+ return -EINVAL;
+
srq->wq_sig = !!(ucmd.flags & MLX5_SRQ_FLAG_SIGNATURE);
srq->umem = ib_umem_get(pd->uobject->context, ucmd.buf_addr, buf_size,
diff --git a/drivers/infiniband/hw/mlx5/user.h b/drivers/infiniband/hw/mlx5/user.h
index 0f4f8e4..d0ba264 100644
--- a/drivers/infiniband/hw/mlx5/user.h
+++ b/drivers/infiniband/hw/mlx5/user.h
@@ -91,6 +91,7 @@
__u64 buf_addr;
__u64 db_addr;
__u32 cqe_size;
+ __u32 reserved; /* explicit padding (optional on i386) */
};
struct mlx5_ib_create_cq_resp {
@@ -109,6 +110,7 @@
__u64 buf_addr;
__u64 db_addr;
__u32 flags;
+ __u32 reserved; /* explicit padding (optional on i386) */
};
struct mlx5_ib_create_srq_resp {
diff --git a/drivers/infiniband/hw/nes/nes.c b/drivers/infiniband/hw/nes/nes.c
index 353c7b0..3b2a6dc 100644
--- a/drivers/infiniband/hw/nes/nes.c
+++ b/drivers/infiniband/hw/nes/nes.c
@@ -68,7 +68,6 @@
int max_mtu = 9000;
int interrupt_mod_interval = 0;
-
/* Interoperability */
int mpa_version = 1;
module_param(mpa_version, int, 0644);
@@ -112,6 +111,16 @@
MODULE_DEVICE_TABLE(pci, nes_pci_table);
+/* registered nes netlink callbacks */
+static struct ibnl_client_cbs nes_nl_cb_table[] = {
+ [RDMA_NL_IWPM_REG_PID] = {.dump = iwpm_register_pid_cb},
+ [RDMA_NL_IWPM_ADD_MAPPING] = {.dump = iwpm_add_mapping_cb},
+ [RDMA_NL_IWPM_QUERY_MAPPING] = {.dump = iwpm_add_and_query_mapping_cb},
+ [RDMA_NL_IWPM_HANDLE_ERR] = {.dump = iwpm_mapping_error_cb},
+ [RDMA_NL_IWPM_MAPINFO] = {.dump = iwpm_mapping_info_cb},
+ [RDMA_NL_IWPM_MAPINFO_NUM] = {.dump = iwpm_ack_mapping_info_cb}
+};
+
static int nes_inetaddr_event(struct notifier_block *, unsigned long, void *);
static int nes_net_event(struct notifier_block *, unsigned long, void *);
static int nes_notifiers_registered;
@@ -672,6 +681,17 @@
}
nes_notifiers_registered++;
+ if (ibnl_add_client(RDMA_NL_NES, RDMA_NL_IWPM_NUM_OPS, nes_nl_cb_table))
+ printk(KERN_ERR PFX "%s[%u]: Failed to add netlink callback\n",
+ __func__, __LINE__);
+
+ ret = iwpm_init(RDMA_NL_NES);
+ if (ret) {
+ printk(KERN_ERR PFX "%s: port mapper initialization failed\n",
+ pci_name(pcidev));
+ goto bail7;
+ }
+
INIT_DELAYED_WORK(&nesdev->work, nes_recheck_link_status);
/* Initialize network devices */
@@ -710,6 +730,7 @@
nes_debug(NES_DBG_INIT, "netdev_count=%d, nesadapter->netdev_count=%d\n",
nesdev->netdev_count, nesdev->nesadapter->netdev_count);
+ ibnl_remove_client(RDMA_NL_NES);
nes_notifiers_registered--;
if (nes_notifiers_registered == 0) {
@@ -773,6 +794,8 @@
nesdev->nesadapter->netdev_count--;
}
}
+ ibnl_remove_client(RDMA_NL_NES);
+ iwpm_exit(RDMA_NL_NES);
nes_notifiers_registered--;
if (nes_notifiers_registered == 0) {
diff --git a/drivers/infiniband/hw/nes/nes.h b/drivers/infiniband/hw/nes/nes.h
index 33cc589..bd9d132 100644
--- a/drivers/infiniband/hw/nes/nes.h
+++ b/drivers/infiniband/hw/nes/nes.h
@@ -51,6 +51,8 @@
#include <rdma/ib_pack.h>
#include <rdma/rdma_cm.h>
#include <rdma/iw_cm.h>
+#include <rdma/rdma_netlink.h>
+#include <rdma/iw_portmap.h>
#define NES_SEND_FIRST_WRITE
@@ -130,6 +132,7 @@
#define NES_DBG_IW_TX 0x00040000
#define NES_DBG_SHUTDOWN 0x00080000
#define NES_DBG_PAU 0x00100000
+#define NES_DBG_NLMSG 0x00200000
#define NES_DBG_RSVD1 0x10000000
#define NES_DBG_RSVD2 0x20000000
#define NES_DBG_RSVD3 0x40000000
diff --git a/drivers/infiniband/hw/nes/nes_cm.c b/drivers/infiniband/hw/nes/nes_cm.c
index dfa9df4..6f09a72 100644
--- a/drivers/infiniband/hw/nes/nes_cm.c
+++ b/drivers/infiniband/hw/nes/nes_cm.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2006 - 2011 Intel Corporation. All rights reserved.
+ * Copyright (c) 2006 - 2014 Intel Corporation. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
@@ -59,6 +59,7 @@
#include <net/route.h>
#include <net/ip_fib.h>
#include <net/tcp.h>
+#include <linux/fcntl.h>
#include "nes.h"
@@ -166,7 +167,6 @@
{
return rem_ref_cm_node(cm_node->cm_core, cm_node);
}
-
/**
* create_event
*/
@@ -482,11 +482,11 @@
iph->ttl = 0x40;
iph->protocol = 0x06; /* IPPROTO_TCP */
- iph->saddr = htonl(cm_node->loc_addr);
- iph->daddr = htonl(cm_node->rem_addr);
+ iph->saddr = htonl(cm_node->mapped_loc_addr);
+ iph->daddr = htonl(cm_node->mapped_rem_addr);
- tcph->source = htons(cm_node->loc_port);
- tcph->dest = htons(cm_node->rem_port);
+ tcph->source = htons(cm_node->mapped_loc_port);
+ tcph->dest = htons(cm_node->mapped_rem_port);
tcph->seq = htonl(cm_node->tcp_cntxt.loc_seq_num);
if (flags & SET_ACK) {
@@ -525,6 +525,100 @@
cm_packets_created++;
}
+/*
+ * nes_create_sockaddr - Record ip addr and tcp port in a sockaddr struct
+ */
+static void nes_create_sockaddr(__be32 ip_addr, __be16 port,
+ struct sockaddr_storage *addr)
+{
+ struct sockaddr_in *nes_sockaddr = (struct sockaddr_in *)addr;
+ nes_sockaddr->sin_family = AF_INET;
+ memcpy(&nes_sockaddr->sin_addr.s_addr, &ip_addr, sizeof(__be32));
+ nes_sockaddr->sin_port = port;
+}
+
+/*
+ * nes_create_mapinfo - Create a mapinfo object in the port mapper data base
+ */
+static int nes_create_mapinfo(struct nes_cm_info *cm_info)
+{
+ struct sockaddr_storage local_sockaddr;
+ struct sockaddr_storage mapped_sockaddr;
+
+ nes_create_sockaddr(htonl(cm_info->loc_addr), htons(cm_info->loc_port),
+ &local_sockaddr);
+ nes_create_sockaddr(htonl(cm_info->mapped_loc_addr),
+ htons(cm_info->mapped_loc_port), &mapped_sockaddr);
+
+ return iwpm_create_mapinfo(&local_sockaddr,
+ &mapped_sockaddr, RDMA_NL_NES);
+}
+
+/*
+ * nes_remove_mapinfo - Remove a mapinfo object from the port mapper data base
+ * and send a remove mapping op message to
+ * the userspace port mapper
+ */
+static int nes_remove_mapinfo(u32 loc_addr, u16 loc_port,
+ u32 mapped_loc_addr, u16 mapped_loc_port)
+{
+ struct sockaddr_storage local_sockaddr;
+ struct sockaddr_storage mapped_sockaddr;
+
+ nes_create_sockaddr(htonl(loc_addr), htons(loc_port), &local_sockaddr);
+ nes_create_sockaddr(htonl(mapped_loc_addr), htons(mapped_loc_port),
+ &mapped_sockaddr);
+
+ iwpm_remove_mapinfo(&local_sockaddr, &mapped_sockaddr);
+ return iwpm_remove_mapping(&local_sockaddr, RDMA_NL_NES);
+}
+
+/*
+ * nes_form_pm_msg - Form a port mapper message with mapping info
+ */
+static void nes_form_pm_msg(struct nes_cm_info *cm_info,
+ struct iwpm_sa_data *pm_msg)
+{
+ nes_create_sockaddr(htonl(cm_info->loc_addr), htons(cm_info->loc_port),
+ &pm_msg->loc_addr);
+ nes_create_sockaddr(htonl(cm_info->rem_addr), htons(cm_info->rem_port),
+ &pm_msg->rem_addr);
+}
+
+/*
+ * nes_form_reg_msg - Form a port mapper message with dev info
+ */
+static void nes_form_reg_msg(struct nes_vnic *nesvnic,
+ struct iwpm_dev_data *pm_msg)
+{
+ memcpy(pm_msg->dev_name, nesvnic->nesibdev->ibdev.name,
+ IWPM_DEVNAME_SIZE);
+ memcpy(pm_msg->if_name, nesvnic->netdev->name, IWPM_IFNAME_SIZE);
+}
+
+/*
+ * nes_record_pm_msg - Save the received mapping info
+ */
+static void nes_record_pm_msg(struct nes_cm_info *cm_info,
+ struct iwpm_sa_data *pm_msg)
+{
+ struct sockaddr_in *mapped_loc_addr =
+ (struct sockaddr_in *)&pm_msg->mapped_loc_addr;
+ struct sockaddr_in *mapped_rem_addr =
+ (struct sockaddr_in *)&pm_msg->mapped_rem_addr;
+
+ if (mapped_loc_addr->sin_family == AF_INET) {
+ cm_info->mapped_loc_addr =
+ ntohl(mapped_loc_addr->sin_addr.s_addr);
+ cm_info->mapped_loc_port = ntohs(mapped_loc_addr->sin_port);
+ }
+ if (mapped_rem_addr->sin_family == AF_INET) {
+ cm_info->mapped_rem_addr =
+ ntohl(mapped_rem_addr->sin_addr.s_addr);
+ cm_info->mapped_rem_port = ntohs(mapped_rem_addr->sin_port);
+ }
+}
+
/**
* print_core - dump a cm core
*/
@@ -1147,8 +1241,11 @@
loc_addr, loc_port,
cm_node->rem_addr, cm_node->rem_port,
rem_addr, rem_port);
- if ((cm_node->loc_addr == loc_addr) && (cm_node->loc_port == loc_port) &&
- (cm_node->rem_addr == rem_addr) && (cm_node->rem_port == rem_port)) {
+ if ((cm_node->mapped_loc_addr == loc_addr) &&
+ (cm_node->mapped_loc_port == loc_port) &&
+ (cm_node->mapped_rem_addr == rem_addr) &&
+ (cm_node->mapped_rem_port == rem_port)) {
+
add_ref_cm_node(cm_node);
spin_unlock_irqrestore(&cm_core->ht_lock, flags);
return cm_node;
@@ -1165,18 +1262,28 @@
* find_listener - find a cm node listening on this addr-port pair
*/
static struct nes_cm_listener *find_listener(struct nes_cm_core *cm_core,
- nes_addr_t dst_addr, u16 dst_port, enum nes_cm_listener_state listener_state)
+ nes_addr_t dst_addr, u16 dst_port,
+ enum nes_cm_listener_state listener_state, int local)
{
unsigned long flags;
struct nes_cm_listener *listen_node;
+ nes_addr_t listen_addr;
+ u16 listen_port;
/* walk list and find cm_node associated with this session ID */
spin_lock_irqsave(&cm_core->listen_list_lock, flags);
list_for_each_entry(listen_node, &cm_core->listen_list.list, list) {
+ if (local) {
+ listen_addr = listen_node->loc_addr;
+ listen_port = listen_node->loc_port;
+ } else {
+ listen_addr = listen_node->mapped_loc_addr;
+ listen_port = listen_node->mapped_loc_port;
+ }
/* compare node pair, return node handle if a match */
- if (((listen_node->loc_addr == dst_addr) ||
- listen_node->loc_addr == 0x00000000) &&
- (listen_node->loc_port == dst_port) &&
+ if (((listen_addr == dst_addr) ||
+ listen_addr == 0x00000000) &&
+ (listen_port == dst_port) &&
(listener_state & listen_node->listener_state)) {
atomic_inc(&listen_node->ref_count);
spin_unlock_irqrestore(&cm_core->listen_list_lock, flags);
@@ -1189,7 +1296,6 @@
return NULL;
}
-
/**
* add_hte_node - add a cm node to the hash table
*/
@@ -1310,9 +1416,20 @@
spin_unlock_irqrestore(&cm_core->listen_list_lock, flags);
- if (listener->nesvnic)
- nes_manage_apbvt(listener->nesvnic, listener->loc_port,
- PCI_FUNC(listener->nesvnic->nesdev->pcidev->devfn), NES_MANAGE_APBVT_DEL);
+ if (listener->nesvnic) {
+ nes_manage_apbvt(listener->nesvnic,
+ listener->mapped_loc_port,
+ PCI_FUNC(listener->nesvnic->nesdev->pcidev->devfn),
+ NES_MANAGE_APBVT_DEL);
+
+ nes_remove_mapinfo(listener->loc_addr,
+ listener->loc_port,
+ listener->mapped_loc_addr,
+ listener->mapped_loc_port);
+ nes_debug(NES_DBG_NLMSG,
+ "Delete APBVT mapped_loc_port = %04X\n",
+ listener->mapped_loc_port);
+ }
nes_debug(NES_DBG_CM, "destroying listener (%p)\n", listener);
@@ -1454,6 +1571,11 @@
cm_node->loc_port = cm_info->loc_port;
cm_node->rem_port = cm_info->rem_port;
+ cm_node->mapped_loc_addr = cm_info->mapped_loc_addr;
+ cm_node->mapped_rem_addr = cm_info->mapped_rem_addr;
+ cm_node->mapped_loc_port = cm_info->mapped_loc_port;
+ cm_node->mapped_rem_port = cm_info->mapped_rem_port;
+
cm_node->mpa_frame_rev = mpa_version;
cm_node->send_rdma0_op = SEND_RDMA_READ_ZERO;
cm_node->mpav2_ird_ord = 0;
@@ -1500,8 +1622,10 @@
cm_node->loopbackpartner = NULL;
/* get the mac addr for the remote node */
- oldarpindex = nes_arp_table(nesdev, cm_node->rem_addr, NULL, NES_ARP_RESOLVE);
- arpindex = nes_addr_resolve_neigh(nesvnic, cm_info->rem_addr, oldarpindex);
+ oldarpindex = nes_arp_table(nesdev, cm_node->mapped_rem_addr,
+ NULL, NES_ARP_RESOLVE);
+ arpindex = nes_addr_resolve_neigh(nesvnic,
+ cm_node->mapped_rem_addr, oldarpindex);
if (arpindex < 0) {
kfree(cm_node);
return NULL;
@@ -1563,11 +1687,14 @@
mini_cm_dec_refcnt_listen(cm_core, cm_node->listener, 0);
} else {
if (cm_node->apbvt_set && cm_node->nesvnic) {
- nes_manage_apbvt(cm_node->nesvnic, cm_node->loc_port,
- PCI_FUNC(
- cm_node->nesvnic->nesdev->pcidev->devfn),
+ nes_manage_apbvt(cm_node->nesvnic, cm_node->mapped_loc_port,
+ PCI_FUNC(cm_node->nesvnic->nesdev->pcidev->devfn),
NES_MANAGE_APBVT_DEL);
}
+ nes_debug(NES_DBG_NLMSG, "Delete APBVT mapped_loc_port = %04X\n",
+ cm_node->mapped_loc_port);
+ nes_remove_mapinfo(cm_node->loc_addr, cm_node->loc_port,
+ cm_node->mapped_loc_addr, cm_node->mapped_loc_port);
}
atomic_dec(&cm_core->node_cnt);
@@ -2235,17 +2362,21 @@
* mini_cm_listen - create a listen node with params
*/
static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *cm_core,
- struct nes_vnic *nesvnic, struct nes_cm_info *cm_info)
+ struct nes_vnic *nesvnic, struct nes_cm_info *cm_info)
{
struct nes_cm_listener *listener;
+ struct iwpm_dev_data pm_reg_msg;
+ struct iwpm_sa_data pm_msg;
unsigned long flags;
+ int iwpm_err = 0;
nes_debug(NES_DBG_CM, "Search for 0x%08x : 0x%04x\n",
cm_info->loc_addr, cm_info->loc_port);
/* cannot have multiple matching listeners */
- listener = find_listener(cm_core, htonl(cm_info->loc_addr),
- htons(cm_info->loc_port), NES_CM_LISTENER_EITHER_STATE);
+ listener = find_listener(cm_core, cm_info->loc_addr, cm_info->loc_port,
+ NES_CM_LISTENER_EITHER_STATE, 1);
+
if (listener && listener->listener_state == NES_CM_LISTENER_ACTIVE_STATE) {
/* find automatically incs ref count ??? */
atomic_dec(&listener->ref_count);
@@ -2254,6 +2385,22 @@
}
if (!listener) {
+ nes_form_reg_msg(nesvnic, &pm_reg_msg);
+ iwpm_err = iwpm_register_pid(&pm_reg_msg, RDMA_NL_NES);
+ if (iwpm_err) {
+ nes_debug(NES_DBG_NLMSG,
+ "Port Mapper reg pid fail (err = %d).\n", iwpm_err);
+ }
+ if (iwpm_valid_pid() && !iwpm_err) {
+ nes_form_pm_msg(cm_info, &pm_msg);
+ iwpm_err = iwpm_add_mapping(&pm_msg, RDMA_NL_NES);
+ if (iwpm_err)
+ nes_debug(NES_DBG_NLMSG,
+ "Port Mapper query fail (err = %d).\n", iwpm_err);
+ else
+ nes_record_pm_msg(cm_info, &pm_msg);
+ }
+
/* create a CM listen node (1/2 node to compare incoming traffic to) */
listener = kzalloc(sizeof(*listener), GFP_ATOMIC);
if (!listener) {
@@ -2261,8 +2408,10 @@
return NULL;
}
- listener->loc_addr = htonl(cm_info->loc_addr);
- listener->loc_port = htons(cm_info->loc_port);
+ listener->loc_addr = cm_info->loc_addr;
+ listener->loc_port = cm_info->loc_port;
+ listener->mapped_loc_addr = cm_info->mapped_loc_addr;
+ listener->mapped_loc_port = cm_info->mapped_loc_port;
listener->reused_node = 0;
atomic_set(&listener->ref_count, 1);
@@ -2324,14 +2473,18 @@
if (cm_info->loc_addr == cm_info->rem_addr) {
loopbackremotelistener = find_listener(cm_core,
- ntohl(nesvnic->local_ipaddr), cm_node->rem_port,
- NES_CM_LISTENER_ACTIVE_STATE);
+ cm_node->mapped_loc_addr, cm_node->mapped_rem_port,
+ NES_CM_LISTENER_ACTIVE_STATE, 0);
if (loopbackremotelistener == NULL) {
create_event(cm_node, NES_CM_EVENT_ABORTED);
} else {
loopback_cm_info = *cm_info;
loopback_cm_info.loc_port = cm_info->rem_port;
loopback_cm_info.rem_port = cm_info->loc_port;
+ loopback_cm_info.mapped_loc_port =
+ cm_info->mapped_rem_port;
+ loopback_cm_info.mapped_rem_port =
+ cm_info->mapped_loc_port;
loopback_cm_info.cm_id = loopbackremotelistener->cm_id;
loopbackremotenode = make_cm_node(cm_core, nesvnic,
&loopback_cm_info, loopbackremotelistener);
@@ -2560,6 +2713,12 @@
nfo.rem_addr = ntohl(iph->saddr);
nfo.rem_port = ntohs(tcph->source);
+ /* If port mapper is available these should be mapped address info */
+ nfo.mapped_loc_addr = ntohl(iph->daddr);
+ nfo.mapped_loc_port = ntohs(tcph->dest);
+ nfo.mapped_rem_addr = ntohl(iph->saddr);
+ nfo.mapped_rem_port = ntohs(tcph->source);
+
tmp_daddr = cpu_to_be32(iph->daddr);
tmp_saddr = cpu_to_be32(iph->saddr);
@@ -2568,8 +2727,8 @@
do {
cm_node = find_node(cm_core,
- nfo.rem_port, nfo.rem_addr,
- nfo.loc_port, nfo.loc_addr);
+ nfo.mapped_rem_port, nfo.mapped_rem_addr,
+ nfo.mapped_loc_port, nfo.mapped_loc_addr);
if (!cm_node) {
/* Only type of packet accepted are for */
@@ -2578,9 +2737,9 @@
skb_handled = 0;
break;
}
- listener = find_listener(cm_core, nfo.loc_addr,
- nfo.loc_port,
- NES_CM_LISTENER_ACTIVE_STATE);
+ listener = find_listener(cm_core, nfo.mapped_loc_addr,
+ nfo.mapped_loc_port,
+ NES_CM_LISTENER_ACTIVE_STATE, 0);
if (!listener) {
nfo.cm_id = NULL;
nfo.conn_type = 0;
@@ -3184,10 +3343,12 @@
nes_cm_init_tsa_conn(nesqp, cm_node);
- nesqp->nesqp_context->tcpPorts[0] = cpu_to_le16(ntohs(laddr->sin_port));
- nesqp->nesqp_context->tcpPorts[1] = cpu_to_le16(ntohs(raddr->sin_port));
+ nesqp->nesqp_context->tcpPorts[0] =
+ cpu_to_le16(cm_node->mapped_loc_port);
+ nesqp->nesqp_context->tcpPorts[1] =
+ cpu_to_le16(cm_node->mapped_rem_port);
- nesqp->nesqp_context->ip0 = cpu_to_le32(ntohl(raddr->sin_addr.s_addr));
+ nesqp->nesqp_context->ip0 = cpu_to_le32(cm_node->mapped_rem_addr);
nesqp->nesqp_context->misc2 |= cpu_to_le32(
(u32)PCI_FUNC(nesdev->pcidev->devfn) <<
@@ -3211,9 +3372,9 @@
memset(&nes_quad, 0, sizeof(nes_quad));
nes_quad.DstIpAdrIndex =
cpu_to_le32((u32)PCI_FUNC(nesdev->pcidev->devfn) << 24);
- nes_quad.SrcIpadr = raddr->sin_addr.s_addr;
- nes_quad.TcpPorts[0] = raddr->sin_port;
- nes_quad.TcpPorts[1] = laddr->sin_port;
+ nes_quad.SrcIpadr = htonl(cm_node->mapped_rem_addr);
+ nes_quad.TcpPorts[0] = htons(cm_node->mapped_rem_port);
+ nes_quad.TcpPorts[1] = htons(cm_node->mapped_loc_port);
/* Produce hash key */
crc_value = get_crc_value(&nes_quad);
@@ -3315,6 +3476,9 @@
int apbvt_set = 0;
struct sockaddr_in *laddr = (struct sockaddr_in *)&cm_id->local_addr;
struct sockaddr_in *raddr = (struct sockaddr_in *)&cm_id->remote_addr;
+ struct iwpm_dev_data pm_reg_msg;
+ struct iwpm_sa_data pm_msg;
+ int iwpm_err = 0;
if (cm_id->remote_addr.ss_family != AF_INET)
return -ENOSYS;
@@ -3352,20 +3516,44 @@
nes_debug(NES_DBG_CM, "mpa private data len =%u\n",
conn_param->private_data_len);
+ /* set up the connection params for the node */
+ cm_info.loc_addr = ntohl(laddr->sin_addr.s_addr);
+ cm_info.loc_port = ntohs(laddr->sin_port);
+ cm_info.rem_addr = ntohl(raddr->sin_addr.s_addr);
+ cm_info.rem_port = ntohs(raddr->sin_port);
+ cm_info.cm_id = cm_id;
+ cm_info.conn_type = NES_CM_IWARP_CONN_TYPE;
+
+ /* No port mapper available, go with the specified peer information */
+ cm_info.mapped_loc_addr = cm_info.loc_addr;
+ cm_info.mapped_loc_port = cm_info.loc_port;
+ cm_info.mapped_rem_addr = cm_info.rem_addr;
+ cm_info.mapped_rem_port = cm_info.rem_port;
+
+ nes_form_reg_msg(nesvnic, &pm_reg_msg);
+ iwpm_err = iwpm_register_pid(&pm_reg_msg, RDMA_NL_NES);
+ if (iwpm_err) {
+ nes_debug(NES_DBG_NLMSG,
+ "Port Mapper reg pid fail (err = %d).\n", iwpm_err);
+ }
+ if (iwpm_valid_pid() && !iwpm_err) {
+ nes_form_pm_msg(&cm_info, &pm_msg);
+ iwpm_err = iwpm_add_and_query_mapping(&pm_msg, RDMA_NL_NES);
+ if (iwpm_err)
+ nes_debug(NES_DBG_NLMSG,
+ "Port Mapper query fail (err = %d).\n", iwpm_err);
+ else
+ nes_record_pm_msg(&cm_info, &pm_msg);
+ }
+
if (laddr->sin_addr.s_addr != raddr->sin_addr.s_addr) {
- nes_manage_apbvt(nesvnic, ntohs(laddr->sin_port),
- PCI_FUNC(nesdev->pcidev->devfn),
- NES_MANAGE_APBVT_ADD);
+ nes_manage_apbvt(nesvnic, cm_info.mapped_loc_port,
+ PCI_FUNC(nesdev->pcidev->devfn), NES_MANAGE_APBVT_ADD);
apbvt_set = 1;
}
- /* set up the connection params for the node */
- cm_info.loc_addr = htonl(laddr->sin_addr.s_addr);
- cm_info.loc_port = htons(laddr->sin_port);
- cm_info.rem_addr = htonl(raddr->sin_addr.s_addr);
- cm_info.rem_port = htons(raddr->sin_port);
- cm_info.cm_id = cm_id;
- cm_info.conn_type = NES_CM_IWARP_CONN_TYPE;
+ if (nes_create_mapinfo(&cm_info))
+ return -ENOMEM;
cm_id->add_ref(cm_id);
@@ -3375,10 +3563,14 @@
&cm_info);
if (!cm_node) {
if (apbvt_set)
- nes_manage_apbvt(nesvnic, ntohs(laddr->sin_port),
+ nes_manage_apbvt(nesvnic, cm_info.mapped_loc_port,
PCI_FUNC(nesdev->pcidev->devfn),
NES_MANAGE_APBVT_DEL);
+ nes_debug(NES_DBG_NLMSG, "Delete mapped_loc_port = %04X\n",
+ cm_info.mapped_loc_port);
+ nes_remove_mapinfo(cm_info.loc_addr, cm_info.loc_port,
+ cm_info.mapped_loc_addr, cm_info.mapped_loc_port);
cm_id->rem_ref(cm_id);
return -ENOMEM;
}
@@ -3424,13 +3616,16 @@
nesvnic->local_ipaddr, laddr->sin_addr.s_addr);
/* setup listen params in our api call struct */
- cm_info.loc_addr = nesvnic->local_ipaddr;
- cm_info.loc_port = laddr->sin_port;
+ cm_info.loc_addr = ntohl(nesvnic->local_ipaddr);
+ cm_info.loc_port = ntohs(laddr->sin_port);
cm_info.backlog = backlog;
cm_info.cm_id = cm_id;
cm_info.conn_type = NES_CM_IWARP_CONN_TYPE;
+ /* No port mapper available, go with the specified info */
+ cm_info.mapped_loc_addr = cm_info.loc_addr;
+ cm_info.mapped_loc_port = cm_info.loc_port;
cm_node = g_cm_core->api->listen(g_cm_core, nesvnic, &cm_info);
if (!cm_node) {
@@ -3442,7 +3637,10 @@
cm_id->provider_data = cm_node;
if (!cm_node->reused_node) {
- err = nes_manage_apbvt(nesvnic, ntohs(laddr->sin_port),
+ if (nes_create_mapinfo(&cm_info))
+ return -ENOMEM;
+
+ err = nes_manage_apbvt(nesvnic, cm_node->mapped_loc_port,
PCI_FUNC(nesvnic->nesdev->pcidev->devfn),
NES_MANAGE_APBVT_ADD);
if (err) {
@@ -3567,9 +3765,11 @@
nes_cm_init_tsa_conn(nesqp, cm_node);
/* set the QP tsa context */
- nesqp->nesqp_context->tcpPorts[0] = cpu_to_le16(ntohs(laddr->sin_port));
- nesqp->nesqp_context->tcpPorts[1] = cpu_to_le16(ntohs(raddr->sin_port));
- nesqp->nesqp_context->ip0 = cpu_to_le32(ntohl(raddr->sin_addr.s_addr));
+ nesqp->nesqp_context->tcpPorts[0] =
+ cpu_to_le16(cm_node->mapped_loc_port);
+ nesqp->nesqp_context->tcpPorts[1] =
+ cpu_to_le16(cm_node->mapped_rem_port);
+ nesqp->nesqp_context->ip0 = cpu_to_le32(cm_node->mapped_rem_addr);
nesqp->nesqp_context->misc2 |= cpu_to_le32(
(u32)PCI_FUNC(nesdev->pcidev->devfn) <<
@@ -3599,9 +3799,9 @@
nes_quad.DstIpAdrIndex =
cpu_to_le32((u32)PCI_FUNC(nesdev->pcidev->devfn) << 24);
- nes_quad.SrcIpadr = raddr->sin_addr.s_addr;
- nes_quad.TcpPorts[0] = raddr->sin_port;
- nes_quad.TcpPorts[1] = laddr->sin_port;
+ nes_quad.SrcIpadr = htonl(cm_node->mapped_rem_addr);
+ nes_quad.TcpPorts[0] = htons(cm_node->mapped_rem_port);
+ nes_quad.TcpPorts[1] = htons(cm_node->mapped_loc_port);
/* Produce hash key */
crc_value = get_crc_value(&nes_quad);
@@ -3629,7 +3829,7 @@
cm_event.ird = cm_node->ird_size;
cm_event.ord = cm_node->ord_size;
- cm_event_laddr->sin_addr.s_addr = event->cm_info.rem_addr;
+ cm_event_laddr->sin_addr.s_addr = htonl(event->cm_info.rem_addr);
ret = cm_id->event_handler(cm_id, &cm_event);
nes_debug(NES_DBG_CM, "OFA CM event_handler returned, ret=%d\n", ret);
diff --git a/drivers/infiniband/hw/nes/nes_cm.h b/drivers/infiniband/hw/nes/nes_cm.h
index 522c99c..f522cf6 100644
--- a/drivers/infiniband/hw/nes/nes_cm.h
+++ b/drivers/infiniband/hw/nes/nes_cm.h
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2006 - 2011 Intel Corporation. All rights reserved.
+ * Copyright (c) 2006 - 2014 Intel Corporation. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
@@ -293,8 +293,8 @@
struct list_head list;
struct nes_cm_core *cm_core;
u8 loc_mac[ETH_ALEN];
- nes_addr_t loc_addr;
- u16 loc_port;
+ nes_addr_t loc_addr, mapped_loc_addr;
+ u16 loc_port, mapped_loc_port;
struct iw_cm_id *cm_id;
enum nes_cm_conn_type conn_type;
atomic_t ref_count;
@@ -308,7 +308,9 @@
/* per connection node and node state information */
struct nes_cm_node {
nes_addr_t loc_addr, rem_addr;
+ nes_addr_t mapped_loc_addr, mapped_rem_addr;
u16 loc_port, rem_port;
+ u16 mapped_loc_port, mapped_rem_port;
u8 loc_mac[ETH_ALEN];
u8 rem_mac[ETH_ALEN];
@@ -364,6 +366,10 @@
u16 rem_port;
nes_addr_t loc_addr;
nes_addr_t rem_addr;
+ u16 mapped_loc_port;
+ u16 mapped_rem_port;
+ nes_addr_t mapped_loc_addr;
+ nes_addr_t mapped_rem_addr;
enum nes_cm_conn_type conn_type;
int backlog;
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_stats.c b/drivers/infiniband/hw/ocrdma/ocrdma_stats.c
index 6c54106..41a9aec 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_stats.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_stats.c
@@ -510,16 +510,9 @@
return status;
}
-static int ocrdma_debugfs_open(struct inode *inode, struct file *file)
-{
- if (inode->i_private)
- file->private_data = inode->i_private;
- return 0;
-}
-
static const struct file_operations ocrdma_dbg_ops = {
.owner = THIS_MODULE,
- .open = ocrdma_debugfs_open,
+ .open = simple_open,
.read = ocrdma_dbgfs_ops_read,
};
diff --git a/drivers/infiniband/hw/qib/qib_init.c b/drivers/infiniband/hw/qib/qib_init.c
index 5b7aeb2..8d3c78dd 100644
--- a/drivers/infiniband/hw/qib/qib_init.c
+++ b/drivers/infiniband/hw/qib/qib_init.c
@@ -1272,7 +1272,7 @@
* Do all the generic driver unit- and chip-independent memory
* allocation and initialization.
*/
-static int __init qlogic_ib_init(void)
+static int __init qib_ib_init(void)
{
int ret;
@@ -1316,12 +1316,12 @@
return ret;
}
-module_init(qlogic_ib_init);
+module_init(qib_ib_init);
/*
* Do the non-unit driver cleanup, memory free, etc. at unload.
*/
-static void __exit qlogic_ib_cleanup(void)
+static void __exit qib_ib_cleanup(void)
{
int ret;
@@ -1346,7 +1346,7 @@
qib_dev_cleanup();
}
-module_exit(qlogic_ib_cleanup);
+module_exit(qib_ib_cleanup);
/* this can only be called after a successful initialization */
static void cleanup_device_data(struct qib_devdata *dd)
diff --git a/drivers/infiniband/hw/qib/qib_mad.c b/drivers/infiniband/hw/qib/qib_mad.c
index edad991..22c720e 100644
--- a/drivers/infiniband/hw/qib/qib_mad.c
+++ b/drivers/infiniband/hw/qib/qib_mad.c
@@ -1028,7 +1028,7 @@
event.event = IB_EVENT_PKEY_CHANGE;
event.device = &dd->verbs_dev.ibdev;
- event.element.port_num = 1;
+ event.element.port_num = port;
ib_dispatch_event(&event);
}
return 0;
diff --git a/drivers/infiniband/hw/qib/qib_qp.c b/drivers/infiniband/hw/qib/qib_qp.c
index 0cad0c4..7fcc150 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -985,7 +985,8 @@
struct ib_qp *ret;
if (init_attr->cap.max_send_sge > ib_qib_max_sges ||
- init_attr->cap.max_send_wr > ib_qib_max_qp_wrs) {
+ init_attr->cap.max_send_wr > ib_qib_max_qp_wrs ||
+ init_attr->create_flags) {
ret = ERR_PTR(-EINVAL);
goto bail;
}
diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
index d48d2c0..53bd6a2 100644
--- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
+++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
@@ -466,6 +466,9 @@
ucontext = to_uucontext(pd->uobject->context);
us_ibdev = to_usdev(pd->device);
+ if (init_attr->create_flags)
+ return ERR_PTR(-EINVAL);
+
err = ib_copy_from_udata(&cmd, udata, sizeof(cmd));
if (err) {
usnic_err("%s: cannot copy udata for create_qp\n",
diff --git a/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.c b/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.c
index d135ad9..3a4288e 100644
--- a/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.c
+++ b/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.c
@@ -1,3 +1,21 @@
+/*
+ * Copyright (c) 2014, Cisco Systems, Inc. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+
#include <linux/init.h>
#include <linux/list.h>
#include <linux/slab.h>
diff --git a/drivers/infiniband/ulp/Makefile b/drivers/infiniband/ulp/Makefile
new file mode 100644
index 0000000..f3c7dcf
--- /dev/null
+++ b/drivers/infiniband/ulp/Makefile
@@ -0,0 +1,5 @@
+obj-$(CONFIG_INFINIBAND_IPOIB) += ipoib/
+obj-$(CONFIG_INFINIBAND_SRP) += srp/
+obj-$(CONFIG_INFINIBAND_SRPT) += srpt/
+obj-$(CONFIG_INFINIBAND_ISER) += iser/
+obj-$(CONFIG_INFINIBAND_ISERT) += isert/
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
index 1377f85..933efce 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
@@ -1030,10 +1030,20 @@
.cap.max_send_sge = 1,
.sq_sig_type = IB_SIGNAL_ALL_WR,
.qp_type = IB_QPT_RC,
- .qp_context = tx
+ .qp_context = tx,
+ .create_flags = IB_QP_CREATE_USE_GFP_NOIO
};
- return ib_create_qp(priv->pd, &attr);
+ struct ib_qp *tx_qp;
+
+ tx_qp = ib_create_qp(priv->pd, &attr);
+ if (PTR_ERR(tx_qp) == -EINVAL) {
+ ipoib_warn(priv, "can't use GFP_NOIO for QPs on device %s, using GFP_KERNEL\n",
+ priv->ca->name);
+ attr.create_flags &= ~IB_QP_CREATE_USE_GFP_NOIO;
+ tx_qp = ib_create_qp(priv->pd, &attr);
+ }
+ return tx_qp;
}
static int ipoib_cm_send_req(struct net_device *dev,
@@ -1104,12 +1114,14 @@
struct ipoib_dev_priv *priv = netdev_priv(p->dev);
int ret;
- p->tx_ring = vzalloc(ipoib_sendq_size * sizeof *p->tx_ring);
+ p->tx_ring = __vmalloc(ipoib_sendq_size * sizeof *p->tx_ring,
+ GFP_NOIO, PAGE_KERNEL);
if (!p->tx_ring) {
ipoib_warn(priv, "failed to allocate tx ring\n");
ret = -ENOMEM;
goto err_tx;
}
+ memset(p->tx_ring, 0, ipoib_sendq_size * sizeof *p->tx_ring);
p->qp = ipoib_cm_create_tx_qp(p->dev, p);
if (IS_ERR(p->qp)) {
diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.c b/drivers/infiniband/ulp/iser/iscsi_iser.c
index 25f195e..eb797395 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.c
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.c
@@ -99,6 +99,7 @@
module_param_named(pi_guard, iser_pi_guard, int, 0644);
MODULE_PARM_DESC(pi_guard, "T10-PI guard_type, 0:CRC|1:IP_CSUM (default:CRC)");
+static struct workqueue_struct *release_wq;
struct iser_global ig;
void
@@ -337,24 +338,6 @@
return cls_conn;
}
-static void
-iscsi_iser_conn_destroy(struct iscsi_cls_conn *cls_conn)
-{
- struct iscsi_conn *conn = cls_conn->dd_data;
- struct iser_conn *ib_conn = conn->dd_data;
-
- iscsi_conn_teardown(cls_conn);
- /*
- * Userspace will normally call the stop callback and
- * already have freed the ib_conn, but if it goofed up then
- * we free it here.
- */
- if (ib_conn) {
- ib_conn->iscsi_conn = NULL;
- iser_conn_put(ib_conn, 1); /* deref iscsi/ib conn unbinding */
- }
-}
-
static int
iscsi_iser_conn_bind(struct iscsi_cls_session *cls_session,
struct iscsi_cls_conn *cls_conn, uint64_t transport_eph,
@@ -392,29 +375,39 @@
conn->dd_data = ib_conn;
ib_conn->iscsi_conn = conn;
- iser_conn_get(ib_conn); /* ref iscsi/ib conn binding */
return 0;
}
+static int
+iscsi_iser_conn_start(struct iscsi_cls_conn *cls_conn)
+{
+ struct iscsi_conn *iscsi_conn;
+ struct iser_conn *ib_conn;
+
+ iscsi_conn = cls_conn->dd_data;
+ ib_conn = iscsi_conn->dd_data;
+ reinit_completion(&ib_conn->stop_completion);
+
+ return iscsi_conn_start(cls_conn);
+}
+
static void
iscsi_iser_conn_stop(struct iscsi_cls_conn *cls_conn, int flag)
{
struct iscsi_conn *conn = cls_conn->dd_data;
struct iser_conn *ib_conn = conn->dd_data;
+ iser_dbg("stopping iscsi_conn: %p, ib_conn: %p\n", conn, ib_conn);
+ iscsi_conn_stop(cls_conn, flag);
+
/*
* Userspace may have goofed up and not bound the connection or
* might have only partially setup the connection.
*/
if (ib_conn) {
- iscsi_conn_stop(cls_conn, flag);
- /*
- * There is no unbind event so the stop callback
- * must release the ref from the bind.
- */
- iser_conn_put(ib_conn, 1); /* deref iscsi/ib conn unbinding */
+ conn->dd_data = NULL;
+ complete(&ib_conn->stop_completion);
}
- conn->dd_data = NULL;
}
static void iscsi_iser_session_destroy(struct iscsi_cls_session *cls_session)
@@ -515,28 +508,28 @@
case ISCSI_PARAM_HDRDGST_EN:
sscanf(buf, "%d", &value);
if (value) {
- iser_err("DataDigest wasn't negotiated to None");
+ iser_err("DataDigest wasn't negotiated to None\n");
return -EPROTO;
}
break;
case ISCSI_PARAM_DATADGST_EN:
sscanf(buf, "%d", &value);
if (value) {
- iser_err("DataDigest wasn't negotiated to None");
+ iser_err("DataDigest wasn't negotiated to None\n");
return -EPROTO;
}
break;
case ISCSI_PARAM_IFMARKER_EN:
sscanf(buf, "%d", &value);
if (value) {
- iser_err("IFMarker wasn't negotiated to No");
+ iser_err("IFMarker wasn't negotiated to No\n");
return -EPROTO;
}
break;
case ISCSI_PARAM_OFMARKER_EN:
sscanf(buf, "%d", &value);
if (value) {
- iser_err("OFMarker wasn't negotiated to No");
+ iser_err("OFMarker wasn't negotiated to No\n");
return -EPROTO;
}
break;
@@ -652,19 +645,20 @@
struct iser_conn *ib_conn;
ib_conn = ep->dd_data;
- if (ib_conn->iscsi_conn)
- /*
- * Must suspend xmit path if the ep is bound to the
- * iscsi_conn, so we know we are not accessing the ib_conn
- * when we free it.
- *
- * This may not be bound if the ep poll failed.
- */
- iscsi_suspend_tx(ib_conn->iscsi_conn);
-
-
- iser_info("ib conn %p state %d\n", ib_conn, ib_conn->state);
+ iser_info("ep %p ib conn %p state %d\n", ep, ib_conn, ib_conn->state);
iser_conn_terminate(ib_conn);
+
+ /*
+ * if iser_conn and iscsi_conn are bound, we must wait iscsi_conn_stop
+ * call and ISER_CONN_DOWN state before freeing the iser resources.
+ * otherwise we are safe to free resources immediately.
+ */
+ if (ib_conn->iscsi_conn) {
+ INIT_WORK(&ib_conn->release_work, iser_release_work);
+ queue_work(release_wq, &ib_conn->release_work);
+ } else {
+ iser_conn_release(ib_conn);
+ }
}
static umode_t iser_attr_is_visible(int param_type, int param)
@@ -748,13 +742,13 @@
/* connection management */
.create_conn = iscsi_iser_conn_create,
.bind_conn = iscsi_iser_conn_bind,
- .destroy_conn = iscsi_iser_conn_destroy,
+ .destroy_conn = iscsi_conn_teardown,
.attr_is_visible = iser_attr_is_visible,
.set_param = iscsi_iser_set_param,
.get_conn_param = iscsi_conn_get_param,
.get_ep_param = iscsi_iser_get_ep_param,
.get_session_param = iscsi_session_get_param,
- .start_conn = iscsi_conn_start,
+ .start_conn = iscsi_iser_conn_start,
.stop_conn = iscsi_iser_conn_stop,
/* iscsi host params */
.get_host_param = iscsi_host_get_param,
@@ -801,6 +795,12 @@
mutex_init(&ig.connlist_mutex);
INIT_LIST_HEAD(&ig.connlist);
+ release_wq = alloc_workqueue("release workqueue", 0, 0);
+ if (!release_wq) {
+ iser_err("failed to allocate release workqueue\n");
+ return -ENOMEM;
+ }
+
iscsi_iser_scsi_transport = iscsi_register_transport(
&iscsi_iser_transport);
if (!iscsi_iser_scsi_transport) {
@@ -819,7 +819,24 @@
static void __exit iser_exit(void)
{
+ struct iser_conn *ib_conn, *n;
+ int connlist_empty;
+
iser_dbg("Removing iSER datamover...\n");
+ destroy_workqueue(release_wq);
+
+ mutex_lock(&ig.connlist_mutex);
+ connlist_empty = list_empty(&ig.connlist);
+ mutex_unlock(&ig.connlist_mutex);
+
+ if (!connlist_empty) {
+ iser_err("Error cleanup stage completed but we still have iser "
+ "connections, destroying them anyway.\n");
+ list_for_each_entry_safe(ib_conn, n, &ig.connlist, conn_list) {
+ iser_conn_release(ib_conn);
+ }
+ }
+
iscsi_unregister_transport(&iscsi_iser_transport);
kmem_cache_destroy(ig.desc_cache);
}
diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.h b/drivers/infiniband/ulp/iser/iscsi_iser.h
index 324129f..97cd385 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.h
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.h
@@ -69,7 +69,7 @@
#define DRV_NAME "iser"
#define PFX DRV_NAME ": "
-#define DRV_VER "1.3"
+#define DRV_VER "1.4"
#define iser_dbg(fmt, arg...) \
do { \
@@ -333,6 +333,8 @@
int post_recv_buf_count; /* posted rx count */
atomic_t post_send_buf_count; /* posted tx count */
char name[ISER_OBJECT_NAME_SIZE];
+ struct work_struct release_work;
+ struct completion stop_completion;
struct list_head conn_list; /* entry in ig conn list */
char *login_buf;
@@ -417,12 +419,12 @@
void iser_conn_init(struct iser_conn *ib_conn);
-void iser_conn_get(struct iser_conn *ib_conn);
-
-int iser_conn_put(struct iser_conn *ib_conn, int destroy_cma_id_allowed);
+void iser_conn_release(struct iser_conn *ib_conn);
void iser_conn_terminate(struct iser_conn *ib_conn);
+void iser_release_work(struct work_struct *work);
+
void iser_rcv_completion(struct iser_rx_desc *desc,
unsigned long dto_xfer_len,
struct iser_conn *ib_conn);
diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index 32849f2..ea01075 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -581,14 +581,30 @@
return ret;
}
+void iser_release_work(struct work_struct *work)
+{
+ struct iser_conn *ib_conn;
+
+ ib_conn = container_of(work, struct iser_conn, release_work);
+
+ /* wait for .conn_stop callback */
+ wait_for_completion(&ib_conn->stop_completion);
+
+ /* wait for the qp`s post send and post receive buffers to empty */
+ wait_event_interruptible(ib_conn->wait,
+ ib_conn->state == ISER_CONN_DOWN);
+
+ iser_conn_release(ib_conn);
+}
+
/**
* Frees all conn objects and deallocs conn descriptor
*/
-static void iser_conn_release(struct iser_conn *ib_conn, int can_destroy_id)
+void iser_conn_release(struct iser_conn *ib_conn)
{
struct iser_device *device = ib_conn->device;
- BUG_ON(ib_conn->state != ISER_CONN_DOWN);
+ BUG_ON(ib_conn->state == ISER_CONN_UP);
mutex_lock(&ig.connlist_mutex);
list_del(&ib_conn->conn_list);
@@ -600,27 +616,13 @@
if (device != NULL)
iser_device_try_release(device);
/* if cma handler context, the caller actually destroy the id */
- if (ib_conn->cma_id != NULL && can_destroy_id) {
+ if (ib_conn->cma_id != NULL) {
rdma_destroy_id(ib_conn->cma_id);
ib_conn->cma_id = NULL;
}
iscsi_destroy_endpoint(ib_conn->ep);
}
-void iser_conn_get(struct iser_conn *ib_conn)
-{
- atomic_inc(&ib_conn->refcount);
-}
-
-int iser_conn_put(struct iser_conn *ib_conn, int can_destroy_id)
-{
- if (atomic_dec_and_test(&ib_conn->refcount)) {
- iser_conn_release(ib_conn, can_destroy_id);
- return 1;
- }
- return 0;
-}
-
/**
* triggers start of the disconnect procedures and wait for them to be done
*/
@@ -638,24 +640,19 @@
if (err)
iser_err("Failed to disconnect, conn: 0x%p err %d\n",
ib_conn,err);
-
- wait_event_interruptible(ib_conn->wait,
- ib_conn->state == ISER_CONN_DOWN);
-
- iser_conn_put(ib_conn, 1); /* deref ib conn deallocate */
}
-static int iser_connect_error(struct rdma_cm_id *cma_id)
+static void iser_connect_error(struct rdma_cm_id *cma_id)
{
struct iser_conn *ib_conn;
+
ib_conn = (struct iser_conn *)cma_id->context;
ib_conn->state = ISER_CONN_DOWN;
wake_up_interruptible(&ib_conn->wait);
- return iser_conn_put(ib_conn, 0); /* deref ib conn's cma id */
}
-static int iser_addr_handler(struct rdma_cm_id *cma_id)
+static void iser_addr_handler(struct rdma_cm_id *cma_id)
{
struct iser_device *device;
struct iser_conn *ib_conn;
@@ -664,7 +661,8 @@
device = iser_device_find_by_ib_device(cma_id);
if (!device) {
iser_err("device lookup/creation failed\n");
- return iser_connect_error(cma_id);
+ iser_connect_error(cma_id);
+ return;
}
ib_conn = (struct iser_conn *)cma_id->context;
@@ -686,13 +684,12 @@
ret = rdma_resolve_route(cma_id, 1000);
if (ret) {
iser_err("resolve route failed: %d\n", ret);
- return iser_connect_error(cma_id);
+ iser_connect_error(cma_id);
+ return;
}
-
- return 0;
}
-static int iser_route_handler(struct rdma_cm_id *cma_id)
+static void iser_route_handler(struct rdma_cm_id *cma_id)
{
struct rdma_conn_param conn_param;
int ret;
@@ -720,9 +717,9 @@
goto failure;
}
- return 0;
+ return;
failure:
- return iser_connect_error(cma_id);
+ iser_connect_error(cma_id);
}
static void iser_connected_handler(struct rdma_cm_id *cma_id)
@@ -735,14 +732,13 @@
iser_info("remote qpn:%x my qpn:%x\n", attr.dest_qp_num, cma_id->qp->qp_num);
ib_conn = (struct iser_conn *)cma_id->context;
- ib_conn->state = ISER_CONN_UP;
- wake_up_interruptible(&ib_conn->wait);
+ if (iser_conn_state_comp_exch(ib_conn, ISER_CONN_PENDING, ISER_CONN_UP))
+ wake_up_interruptible(&ib_conn->wait);
}
-static int iser_disconnected_handler(struct rdma_cm_id *cma_id)
+static void iser_disconnected_handler(struct rdma_cm_id *cma_id)
{
struct iser_conn *ib_conn;
- int ret;
ib_conn = (struct iser_conn *)cma_id->context;
@@ -762,24 +758,19 @@
ib_conn->state = ISER_CONN_DOWN;
wake_up_interruptible(&ib_conn->wait);
}
-
- ret = iser_conn_put(ib_conn, 0); /* deref ib conn's cma id */
- return ret;
}
static int iser_cma_handler(struct rdma_cm_id *cma_id, struct rdma_cm_event *event)
{
- int ret = 0;
-
iser_info("event %d status %d conn %p id %p\n",
event->event, event->status, cma_id->context, cma_id);
switch (event->event) {
case RDMA_CM_EVENT_ADDR_RESOLVED:
- ret = iser_addr_handler(cma_id);
+ iser_addr_handler(cma_id);
break;
case RDMA_CM_EVENT_ROUTE_RESOLVED:
- ret = iser_route_handler(cma_id);
+ iser_route_handler(cma_id);
break;
case RDMA_CM_EVENT_ESTABLISHED:
iser_connected_handler(cma_id);
@@ -789,18 +780,18 @@
case RDMA_CM_EVENT_CONNECT_ERROR:
case RDMA_CM_EVENT_UNREACHABLE:
case RDMA_CM_EVENT_REJECTED:
- ret = iser_connect_error(cma_id);
+ iser_connect_error(cma_id);
break;
case RDMA_CM_EVENT_DISCONNECTED:
case RDMA_CM_EVENT_DEVICE_REMOVAL:
case RDMA_CM_EVENT_ADDR_CHANGE:
- ret = iser_disconnected_handler(cma_id);
+ iser_disconnected_handler(cma_id);
break;
default:
iser_err("Unexpected RDMA CM event (%d)\n", event->event);
break;
}
- return ret;
+ return 0;
}
void iser_conn_init(struct iser_conn *ib_conn)
@@ -809,7 +800,7 @@
init_waitqueue_head(&ib_conn->wait);
ib_conn->post_recv_buf_count = 0;
atomic_set(&ib_conn->post_send_buf_count, 0);
- atomic_set(&ib_conn->refcount, 1); /* ref ib conn allocation */
+ init_completion(&ib_conn->stop_completion);
INIT_LIST_HEAD(&ib_conn->conn_list);
spin_lock_init(&ib_conn->lock);
}
@@ -837,7 +828,6 @@
ib_conn->state = ISER_CONN_PENDING;
- iser_conn_get(ib_conn); /* ref ib conn's cma id */
ib_conn->cma_id = rdma_create_id(iser_cma_handler,
(void *)ib_conn,
RDMA_PS_TCP, IB_QPT_RC);
@@ -874,9 +864,8 @@
ib_conn->cma_id = NULL;
addr_failure:
ib_conn->state = ISER_CONN_DOWN;
- iser_conn_put(ib_conn, 1); /* deref ib conn's cma id */
connect_failure:
- iser_conn_put(ib_conn, 1); /* deref ib conn deallocate */
+ iser_conn_release(ib_conn);
return err;
}
diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c
index a171046..b9d6474 100644
--- a/drivers/infiniband/ulp/isert/ib_isert.c
+++ b/drivers/infiniband/ulp/isert/ib_isert.c
@@ -1210,6 +1210,8 @@
if (!rc && dump_payload == false && unsol_data)
iscsit_set_unsoliticed_dataout(cmd);
+ else if (dump_payload && imm_data)
+ target_put_sess_cmd(conn->sess->se_sess, &cmd->se_cmd);
return 0;
}
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index 66a908b..e3c2c5b 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -30,7 +30,7 @@
* SOFTWARE.
*/
-#define pr_fmt(fmt) PFX fmt
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/module.h>
#include <linux/init.h>
@@ -66,6 +66,8 @@
static unsigned int cmd_sg_entries;
static unsigned int indirect_sg_entries;
static bool allow_ext_sg;
+static bool prefer_fr;
+static bool register_always;
static int topspin_workarounds = 1;
module_param(srp_sg_tablesize, uint, 0444);
@@ -87,6 +89,14 @@
MODULE_PARM_DESC(topspin_workarounds,
"Enable workarounds for Topspin/Cisco SRP target bugs if != 0");
+module_param(prefer_fr, bool, 0444);
+MODULE_PARM_DESC(prefer_fr,
+"Whether to use fast registration if both FMR and fast registration are supported");
+
+module_param(register_always, bool, 0444);
+MODULE_PARM_DESC(register_always,
+ "Use memory registration even for contiguous memory regions");
+
static struct kernel_param_ops srp_tmo_ops;
static int srp_reconnect_delay = 10;
@@ -288,28 +298,174 @@
return 0;
}
+static struct ib_fmr_pool *srp_alloc_fmr_pool(struct srp_target_port *target)
+{
+ struct srp_device *dev = target->srp_host->srp_dev;
+ struct ib_fmr_pool_param fmr_param;
+
+ memset(&fmr_param, 0, sizeof(fmr_param));
+ fmr_param.pool_size = target->scsi_host->can_queue;
+ fmr_param.dirty_watermark = fmr_param.pool_size / 4;
+ fmr_param.cache = 1;
+ fmr_param.max_pages_per_fmr = dev->max_pages_per_mr;
+ fmr_param.page_shift = ilog2(dev->mr_page_size);
+ fmr_param.access = (IB_ACCESS_LOCAL_WRITE |
+ IB_ACCESS_REMOTE_WRITE |
+ IB_ACCESS_REMOTE_READ);
+
+ return ib_create_fmr_pool(dev->pd, &fmr_param);
+}
+
+/**
+ * srp_destroy_fr_pool() - free the resources owned by a pool
+ * @pool: Fast registration pool to be destroyed.
+ */
+static void srp_destroy_fr_pool(struct srp_fr_pool *pool)
+{
+ int i;
+ struct srp_fr_desc *d;
+
+ if (!pool)
+ return;
+
+ for (i = 0, d = &pool->desc[0]; i < pool->size; i++, d++) {
+ if (d->frpl)
+ ib_free_fast_reg_page_list(d->frpl);
+ if (d->mr)
+ ib_dereg_mr(d->mr);
+ }
+ kfree(pool);
+}
+
+/**
+ * srp_create_fr_pool() - allocate and initialize a pool for fast registration
+ * @device: IB device to allocate fast registration descriptors for.
+ * @pd: Protection domain associated with the FR descriptors.
+ * @pool_size: Number of descriptors to allocate.
+ * @max_page_list_len: Maximum fast registration work request page list length.
+ */
+static struct srp_fr_pool *srp_create_fr_pool(struct ib_device *device,
+ struct ib_pd *pd, int pool_size,
+ int max_page_list_len)
+{
+ struct srp_fr_pool *pool;
+ struct srp_fr_desc *d;
+ struct ib_mr *mr;
+ struct ib_fast_reg_page_list *frpl;
+ int i, ret = -EINVAL;
+
+ if (pool_size <= 0)
+ goto err;
+ ret = -ENOMEM;
+ pool = kzalloc(sizeof(struct srp_fr_pool) +
+ pool_size * sizeof(struct srp_fr_desc), GFP_KERNEL);
+ if (!pool)
+ goto err;
+ pool->size = pool_size;
+ pool->max_page_list_len = max_page_list_len;
+ spin_lock_init(&pool->lock);
+ INIT_LIST_HEAD(&pool->free_list);
+
+ for (i = 0, d = &pool->desc[0]; i < pool->size; i++, d++) {
+ mr = ib_alloc_fast_reg_mr(pd, max_page_list_len);
+ if (IS_ERR(mr)) {
+ ret = PTR_ERR(mr);
+ goto destroy_pool;
+ }
+ d->mr = mr;
+ frpl = ib_alloc_fast_reg_page_list(device, max_page_list_len);
+ if (IS_ERR(frpl)) {
+ ret = PTR_ERR(frpl);
+ goto destroy_pool;
+ }
+ d->frpl = frpl;
+ list_add_tail(&d->entry, &pool->free_list);
+ }
+
+out:
+ return pool;
+
+destroy_pool:
+ srp_destroy_fr_pool(pool);
+
+err:
+ pool = ERR_PTR(ret);
+ goto out;
+}
+
+/**
+ * srp_fr_pool_get() - obtain a descriptor suitable for fast registration
+ * @pool: Pool to obtain descriptor from.
+ */
+static struct srp_fr_desc *srp_fr_pool_get(struct srp_fr_pool *pool)
+{
+ struct srp_fr_desc *d = NULL;
+ unsigned long flags;
+
+ spin_lock_irqsave(&pool->lock, flags);
+ if (!list_empty(&pool->free_list)) {
+ d = list_first_entry(&pool->free_list, typeof(*d), entry);
+ list_del(&d->entry);
+ }
+ spin_unlock_irqrestore(&pool->lock, flags);
+
+ return d;
+}
+
+/**
+ * srp_fr_pool_put() - put an FR descriptor back in the free list
+ * @pool: Pool the descriptor was allocated from.
+ * @desc: Pointer to an array of fast registration descriptor pointers.
+ * @n: Number of descriptors to put back.
+ *
+ * Note: The caller must already have queued an invalidation request for
+ * desc->mr->rkey before calling this function.
+ */
+static void srp_fr_pool_put(struct srp_fr_pool *pool, struct srp_fr_desc **desc,
+ int n)
+{
+ unsigned long flags;
+ int i;
+
+ spin_lock_irqsave(&pool->lock, flags);
+ for (i = 0; i < n; i++)
+ list_add(&desc[i]->entry, &pool->free_list);
+ spin_unlock_irqrestore(&pool->lock, flags);
+}
+
+static struct srp_fr_pool *srp_alloc_fr_pool(struct srp_target_port *target)
+{
+ struct srp_device *dev = target->srp_host->srp_dev;
+
+ return srp_create_fr_pool(dev->dev, dev->pd,
+ target->scsi_host->can_queue,
+ dev->max_pages_per_mr);
+}
+
static int srp_create_target_ib(struct srp_target_port *target)
{
+ struct srp_device *dev = target->srp_host->srp_dev;
struct ib_qp_init_attr *init_attr;
struct ib_cq *recv_cq, *send_cq;
struct ib_qp *qp;
+ struct ib_fmr_pool *fmr_pool = NULL;
+ struct srp_fr_pool *fr_pool = NULL;
+ const int m = 1 + dev->use_fast_reg;
int ret;
init_attr = kzalloc(sizeof *init_attr, GFP_KERNEL);
if (!init_attr)
return -ENOMEM;
- recv_cq = ib_create_cq(target->srp_host->srp_dev->dev,
- srp_recv_completion, NULL, target,
+ recv_cq = ib_create_cq(dev->dev, srp_recv_completion, NULL, target,
target->queue_size, target->comp_vector);
if (IS_ERR(recv_cq)) {
ret = PTR_ERR(recv_cq);
goto err;
}
- send_cq = ib_create_cq(target->srp_host->srp_dev->dev,
- srp_send_completion, NULL, target,
- target->queue_size, target->comp_vector);
+ send_cq = ib_create_cq(dev->dev, srp_send_completion, NULL, target,
+ m * target->queue_size, target->comp_vector);
if (IS_ERR(send_cq)) {
ret = PTR_ERR(send_cq);
goto err_recv_cq;
@@ -318,16 +474,16 @@
ib_req_notify_cq(recv_cq, IB_CQ_NEXT_COMP);
init_attr->event_handler = srp_qp_event;
- init_attr->cap.max_send_wr = target->queue_size;
+ init_attr->cap.max_send_wr = m * target->queue_size;
init_attr->cap.max_recv_wr = target->queue_size;
init_attr->cap.max_recv_sge = 1;
init_attr->cap.max_send_sge = 1;
- init_attr->sq_sig_type = IB_SIGNAL_ALL_WR;
+ init_attr->sq_sig_type = IB_SIGNAL_REQ_WR;
init_attr->qp_type = IB_QPT_RC;
init_attr->send_cq = send_cq;
init_attr->recv_cq = recv_cq;
- qp = ib_create_qp(target->srp_host->srp_dev->pd, init_attr);
+ qp = ib_create_qp(dev->pd, init_attr);
if (IS_ERR(qp)) {
ret = PTR_ERR(qp);
goto err_send_cq;
@@ -337,6 +493,30 @@
if (ret)
goto err_qp;
+ if (dev->use_fast_reg && dev->has_fr) {
+ fr_pool = srp_alloc_fr_pool(target);
+ if (IS_ERR(fr_pool)) {
+ ret = PTR_ERR(fr_pool);
+ shost_printk(KERN_WARNING, target->scsi_host, PFX
+ "FR pool allocation failed (%d)\n", ret);
+ goto err_qp;
+ }
+ if (target->fr_pool)
+ srp_destroy_fr_pool(target->fr_pool);
+ target->fr_pool = fr_pool;
+ } else if (!dev->use_fast_reg && dev->has_fmr) {
+ fmr_pool = srp_alloc_fmr_pool(target);
+ if (IS_ERR(fmr_pool)) {
+ ret = PTR_ERR(fmr_pool);
+ shost_printk(KERN_WARNING, target->scsi_host, PFX
+ "FMR pool allocation failed (%d)\n", ret);
+ goto err_qp;
+ }
+ if (target->fmr_pool)
+ ib_destroy_fmr_pool(target->fmr_pool);
+ target->fmr_pool = fmr_pool;
+ }
+
if (target->qp)
ib_destroy_qp(target->qp);
if (target->recv_cq)
@@ -371,8 +551,16 @@
*/
static void srp_free_target_ib(struct srp_target_port *target)
{
+ struct srp_device *dev = target->srp_host->srp_dev;
int i;
+ if (dev->use_fast_reg) {
+ if (target->fr_pool)
+ srp_destroy_fr_pool(target->fr_pool);
+ } else {
+ if (target->fmr_pool)
+ ib_destroy_fmr_pool(target->fmr_pool);
+ }
ib_destroy_qp(target->qp);
ib_destroy_cq(target->send_cq);
ib_destroy_cq(target->recv_cq);
@@ -577,7 +765,8 @@
static void srp_free_req_data(struct srp_target_port *target)
{
- struct ib_device *ibdev = target->srp_host->srp_dev->dev;
+ struct srp_device *dev = target->srp_host->srp_dev;
+ struct ib_device *ibdev = dev->dev;
struct srp_request *req;
int i;
@@ -586,7 +775,10 @@
for (i = 0; i < target->req_ring_size; ++i) {
req = &target->req_ring[i];
- kfree(req->fmr_list);
+ if (dev->use_fast_reg)
+ kfree(req->fr_list);
+ else
+ kfree(req->fmr_list);
kfree(req->map_page);
if (req->indirect_dma_addr) {
ib_dma_unmap_single(ibdev, req->indirect_dma_addr,
@@ -605,6 +797,7 @@
struct srp_device *srp_dev = target->srp_host->srp_dev;
struct ib_device *ibdev = srp_dev->dev;
struct srp_request *req;
+ void *mr_list;
dma_addr_t dma_addr;
int i, ret = -ENOMEM;
@@ -617,12 +810,20 @@
for (i = 0; i < target->req_ring_size; ++i) {
req = &target->req_ring[i];
- req->fmr_list = kmalloc(target->cmd_sg_cnt * sizeof(void *),
- GFP_KERNEL);
- req->map_page = kmalloc(SRP_FMR_SIZE * sizeof(void *),
- GFP_KERNEL);
+ mr_list = kmalloc(target->cmd_sg_cnt * sizeof(void *),
+ GFP_KERNEL);
+ if (!mr_list)
+ goto out;
+ if (srp_dev->use_fast_reg)
+ req->fr_list = mr_list;
+ else
+ req->fmr_list = mr_list;
+ req->map_page = kmalloc(srp_dev->max_pages_per_mr *
+ sizeof(void *), GFP_KERNEL);
+ if (!req->map_page)
+ goto out;
req->indirect_desc = kmalloc(target->indirect_size, GFP_KERNEL);
- if (!req->fmr_list || !req->map_page || !req->indirect_desc)
+ if (!req->indirect_desc)
goto out;
dma_addr = ib_dma_map_single(ibdev, req->indirect_desc,
@@ -759,21 +960,56 @@
}
}
+static int srp_inv_rkey(struct srp_target_port *target, u32 rkey)
+{
+ struct ib_send_wr *bad_wr;
+ struct ib_send_wr wr = {
+ .opcode = IB_WR_LOCAL_INV,
+ .wr_id = LOCAL_INV_WR_ID_MASK,
+ .next = NULL,
+ .num_sge = 0,
+ .send_flags = 0,
+ .ex.invalidate_rkey = rkey,
+ };
+
+ return ib_post_send(target->qp, &wr, &bad_wr);
+}
+
static void srp_unmap_data(struct scsi_cmnd *scmnd,
struct srp_target_port *target,
struct srp_request *req)
{
- struct ib_device *ibdev = target->srp_host->srp_dev->dev;
- struct ib_pool_fmr **pfmr;
+ struct srp_device *dev = target->srp_host->srp_dev;
+ struct ib_device *ibdev = dev->dev;
+ int i, res;
if (!scsi_sglist(scmnd) ||
(scmnd->sc_data_direction != DMA_TO_DEVICE &&
scmnd->sc_data_direction != DMA_FROM_DEVICE))
return;
- pfmr = req->fmr_list;
- while (req->nfmr--)
- ib_fmr_pool_unmap(*pfmr++);
+ if (dev->use_fast_reg) {
+ struct srp_fr_desc **pfr;
+
+ for (i = req->nmdesc, pfr = req->fr_list; i > 0; i--, pfr++) {
+ res = srp_inv_rkey(target, (*pfr)->mr->rkey);
+ if (res < 0) {
+ shost_printk(KERN_ERR, target->scsi_host, PFX
+ "Queueing INV WR for rkey %#x failed (%d)\n",
+ (*pfr)->mr->rkey, res);
+ queue_work(system_long_wq,
+ &target->tl_err_work);
+ }
+ }
+ if (req->nmdesc)
+ srp_fr_pool_put(target->fr_pool, req->fr_list,
+ req->nmdesc);
+ } else {
+ struct ib_pool_fmr **pfmr;
+
+ for (i = req->nmdesc, pfmr = req->fmr_list; i > 0; i--, pfmr++)
+ ib_fmr_pool_unmap(*pfmr);
+ }
ib_dma_unmap_sg(ibdev, scsi_sglist(scmnd), scsi_sg_count(scmnd),
scmnd->sc_data_direction);
@@ -813,6 +1049,10 @@
/**
* srp_free_req() - Unmap data and add request to the free request list.
+ * @target: SRP target port.
+ * @req: Request to be freed.
+ * @scmnd: SCSI command associated with @req.
+ * @req_lim_delta: Amount to be added to @target->req_lim.
*/
static void srp_free_req(struct srp_target_port *target,
struct srp_request *req, struct scsi_cmnd *scmnd,
@@ -882,21 +1122,19 @@
* callbacks will have finished before a new QP is allocated.
*/
ret = srp_new_cm_id(target);
- /*
- * Whether or not creating a new CM ID succeeded, create a new
- * QP. This guarantees that all completion callback function
- * invocations have finished before request resetting starts.
- */
- if (ret == 0)
- ret = srp_create_target_ib(target);
- else
- srp_create_target_ib(target);
for (i = 0; i < target->req_ring_size; ++i) {
struct srp_request *req = &target->req_ring[i];
srp_finish_req(target, req, NULL, DID_RESET << 16);
}
+ /*
+ * Whether or not creating a new CM ID succeeded, create a new
+ * QP. This guarantees that all callback functions for the old QP have
+ * finished before any send requests are posted on the new QP.
+ */
+ ret += srp_create_target_ib(target);
+
INIT_LIST_HEAD(&target->free_tx);
for (i = 0; i < target->queue_size; ++i)
list_add(&target->tx_ring[i]->list, &target->free_tx);
@@ -928,33 +1166,87 @@
static int srp_map_finish_fmr(struct srp_map_state *state,
struct srp_target_port *target)
{
- struct srp_device *dev = target->srp_host->srp_dev;
struct ib_pool_fmr *fmr;
u64 io_addr = 0;
- if (!state->npages)
- return 0;
-
- if (state->npages == 1) {
- srp_map_desc(state, state->base_dma_addr, state->fmr_len,
- target->rkey);
- state->npages = state->fmr_len = 0;
- return 0;
- }
-
- fmr = ib_fmr_pool_map_phys(dev->fmr_pool, state->pages,
+ fmr = ib_fmr_pool_map_phys(target->fmr_pool, state->pages,
state->npages, io_addr);
if (IS_ERR(fmr))
return PTR_ERR(fmr);
*state->next_fmr++ = fmr;
- state->nfmr++;
+ state->nmdesc++;
- srp_map_desc(state, 0, state->fmr_len, fmr->fmr->rkey);
- state->npages = state->fmr_len = 0;
+ srp_map_desc(state, 0, state->dma_len, fmr->fmr->rkey);
+
return 0;
}
+static int srp_map_finish_fr(struct srp_map_state *state,
+ struct srp_target_port *target)
+{
+ struct srp_device *dev = target->srp_host->srp_dev;
+ struct ib_send_wr *bad_wr;
+ struct ib_send_wr wr;
+ struct srp_fr_desc *desc;
+ u32 rkey;
+
+ desc = srp_fr_pool_get(target->fr_pool);
+ if (!desc)
+ return -ENOMEM;
+
+ rkey = ib_inc_rkey(desc->mr->rkey);
+ ib_update_fast_reg_key(desc->mr, rkey);
+
+ memcpy(desc->frpl->page_list, state->pages,
+ sizeof(state->pages[0]) * state->npages);
+
+ memset(&wr, 0, sizeof(wr));
+ wr.opcode = IB_WR_FAST_REG_MR;
+ wr.wr_id = FAST_REG_WR_ID_MASK;
+ wr.wr.fast_reg.iova_start = state->base_dma_addr;
+ wr.wr.fast_reg.page_list = desc->frpl;
+ wr.wr.fast_reg.page_list_len = state->npages;
+ wr.wr.fast_reg.page_shift = ilog2(dev->mr_page_size);
+ wr.wr.fast_reg.length = state->dma_len;
+ wr.wr.fast_reg.access_flags = (IB_ACCESS_LOCAL_WRITE |
+ IB_ACCESS_REMOTE_READ |
+ IB_ACCESS_REMOTE_WRITE);
+ wr.wr.fast_reg.rkey = desc->mr->lkey;
+
+ *state->next_fr++ = desc;
+ state->nmdesc++;
+
+ srp_map_desc(state, state->base_dma_addr, state->dma_len,
+ desc->mr->rkey);
+
+ return ib_post_send(target->qp, &wr, &bad_wr);
+}
+
+static int srp_finish_mapping(struct srp_map_state *state,
+ struct srp_target_port *target)
+{
+ int ret = 0;
+
+ if (state->npages == 0)
+ return 0;
+
+ if (state->npages == 1 && !register_always)
+ srp_map_desc(state, state->base_dma_addr, state->dma_len,
+ target->rkey);
+ else
+ ret = target->srp_host->srp_dev->use_fast_reg ?
+ srp_map_finish_fr(state, target) :
+ srp_map_finish_fmr(state, target);
+
+ if (ret == 0) {
+ state->npages = 0;
+ state->dma_len = 0;
+ }
+
+ return ret;
+}
+
static void srp_map_update_start(struct srp_map_state *state,
struct scatterlist *sg, int sg_index,
dma_addr_t dma_addr)
@@ -967,7 +1259,7 @@
static int srp_map_sg_entry(struct srp_map_state *state,
struct srp_target_port *target,
struct scatterlist *sg, int sg_index,
- int use_fmr)
+ bool use_mr)
{
struct srp_device *dev = target->srp_host->srp_dev;
struct ib_device *ibdev = dev->dev;
@@ -979,23 +1271,25 @@
if (!dma_len)
return 0;
- if (use_fmr == SRP_MAP_NO_FMR) {
- /* Once we're in direct map mode for a request, we don't
- * go back to FMR mode, so no need to update anything
+ if (!use_mr) {
+ /*
+ * Once we're in direct map mode for a request, we don't
+ * go back to FMR or FR mode, so no need to update anything
* other than the descriptor.
*/
srp_map_desc(state, dma_addr, dma_len, target->rkey);
return 0;
}
- /* If we start at an offset into the FMR page, don't merge into
- * the current FMR. Finish it out, and use the kernel's MR for this
- * sg entry. This is to avoid potential bugs on some SRP targets
- * that were never quite defined, but went away when the initiator
- * avoided using FMR on such page fragments.
+ /*
+ * Since not all RDMA HW drivers support non-zero page offsets for
+ * FMR, if we start at an offset into a page, don't merge into the
+ * current FMR mapping. Finish it out, and use the kernel's MR for
+ * this sg entry.
*/
- if (dma_addr & ~dev->fmr_page_mask || dma_len > dev->fmr_max_size) {
- ret = srp_map_finish_fmr(state, target);
+ if ((!dev->use_fast_reg && dma_addr & ~dev->mr_page_mask) ||
+ dma_len > dev->mr_max_size) {
+ ret = srp_finish_mapping(state, target);
if (ret)
return ret;
@@ -1004,52 +1298,106 @@
return 0;
}
- /* If this is the first sg to go into the FMR, save our position.
- * We need to know the first unmapped entry, its index, and the
- * first unmapped address within that entry to be able to restart
- * mapping after an error.
+ /*
+ * If this is the first sg that will be mapped via FMR or via FR, save
+ * our position. We need to know the first unmapped entry, its index,
+ * and the first unmapped address within that entry to be able to
+ * restart mapping after an error.
*/
if (!state->unmapped_sg)
srp_map_update_start(state, sg, sg_index, dma_addr);
while (dma_len) {
- if (state->npages == SRP_FMR_SIZE) {
- ret = srp_map_finish_fmr(state, target);
+ unsigned offset = dma_addr & ~dev->mr_page_mask;
+ if (state->npages == dev->max_pages_per_mr || offset != 0) {
+ ret = srp_finish_mapping(state, target);
if (ret)
return ret;
srp_map_update_start(state, sg, sg_index, dma_addr);
}
- len = min_t(unsigned int, dma_len, dev->fmr_page_size);
+ len = min_t(unsigned int, dma_len, dev->mr_page_size - offset);
if (!state->npages)
state->base_dma_addr = dma_addr;
- state->pages[state->npages++] = dma_addr;
- state->fmr_len += len;
+ state->pages[state->npages++] = dma_addr & dev->mr_page_mask;
+ state->dma_len += len;
dma_addr += len;
dma_len -= len;
}
- /* If the last entry of the FMR wasn't a full page, then we need to
+ /*
+ * If the last entry of the MR wasn't a full page, then we need to
* close it out and start a new one -- we can only merge at page
* boundries.
*/
ret = 0;
- if (len != dev->fmr_page_size) {
- ret = srp_map_finish_fmr(state, target);
+ if (len != dev->mr_page_size) {
+ ret = srp_finish_mapping(state, target);
if (!ret)
srp_map_update_start(state, NULL, 0, 0);
}
return ret;
}
+static int srp_map_sg(struct srp_map_state *state,
+ struct srp_target_port *target, struct srp_request *req,
+ struct scatterlist *scat, int count)
+{
+ struct srp_device *dev = target->srp_host->srp_dev;
+ struct ib_device *ibdev = dev->dev;
+ struct scatterlist *sg;
+ int i;
+ bool use_mr;
+
+ state->desc = req->indirect_desc;
+ state->pages = req->map_page;
+ if (dev->use_fast_reg) {
+ state->next_fr = req->fr_list;
+ use_mr = !!target->fr_pool;
+ } else {
+ state->next_fmr = req->fmr_list;
+ use_mr = !!target->fmr_pool;
+ }
+
+ for_each_sg(scat, sg, count, i) {
+ if (srp_map_sg_entry(state, target, sg, i, use_mr)) {
+ /*
+ * Memory registration failed, so backtrack to the
+ * first unmapped entry and continue on without using
+ * memory registration.
+ */
+ dma_addr_t dma_addr;
+ unsigned int dma_len;
+
+backtrack:
+ sg = state->unmapped_sg;
+ i = state->unmapped_index;
+
+ dma_addr = ib_sg_dma_address(ibdev, sg);
+ dma_len = ib_sg_dma_len(ibdev, sg);
+ dma_len -= (state->unmapped_addr - dma_addr);
+ dma_addr = state->unmapped_addr;
+ use_mr = false;
+ srp_map_desc(state, dma_addr, dma_len, target->rkey);
+ }
+ }
+
+ if (use_mr && srp_finish_mapping(state, target))
+ goto backtrack;
+
+ req->nmdesc = state->nmdesc;
+
+ return 0;
+}
+
static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_target_port *target,
struct srp_request *req)
{
- struct scatterlist *scat, *sg;
+ struct scatterlist *scat;
struct srp_cmd *cmd = req->cmd->buf;
- int i, len, nents, count, use_fmr;
+ int len, nents, count;
struct srp_device *dev;
struct ib_device *ibdev;
struct srp_map_state state;
@@ -1081,7 +1429,7 @@
fmt = SRP_DATA_DESC_DIRECT;
len = sizeof (struct srp_cmd) + sizeof (struct srp_direct_buf);
- if (count == 1) {
+ if (count == 1 && !register_always) {
/*
* The midlayer only generated a single gather/scatter
* entry, or DMA mapping coalesced everything to a
@@ -1094,13 +1442,13 @@
buf->key = cpu_to_be32(target->rkey);
buf->len = cpu_to_be32(ib_sg_dma_len(ibdev, scat));
- req->nfmr = 0;
+ req->nmdesc = 0;
goto map_complete;
}
- /* We have more than one scatter/gather entry, so build our indirect
- * descriptor table, trying to merge as many entries with FMR as we
- * can.
+ /*
+ * We have more than one scatter/gather entry, so build our indirect
+ * descriptor table, trying to merge as many entries as we can.
*/
indirect_hdr = (void *) cmd->add_data;
@@ -1108,35 +1456,7 @@
target->indirect_size, DMA_TO_DEVICE);
memset(&state, 0, sizeof(state));
- state.desc = req->indirect_desc;
- state.pages = req->map_page;
- state.next_fmr = req->fmr_list;
-
- use_fmr = dev->fmr_pool ? SRP_MAP_ALLOW_FMR : SRP_MAP_NO_FMR;
-
- for_each_sg(scat, sg, count, i) {
- if (srp_map_sg_entry(&state, target, sg, i, use_fmr)) {
- /* FMR mapping failed, so backtrack to the first
- * unmapped entry and continue on without using FMR.
- */
- dma_addr_t dma_addr;
- unsigned int dma_len;
-
-backtrack:
- sg = state.unmapped_sg;
- i = state.unmapped_index;
-
- dma_addr = ib_sg_dma_address(ibdev, sg);
- dma_len = ib_sg_dma_len(ibdev, sg);
- dma_len -= (state.unmapped_addr - dma_addr);
- dma_addr = state.unmapped_addr;
- use_fmr = SRP_MAP_NO_FMR;
- srp_map_desc(&state, dma_addr, dma_len, target->rkey);
- }
- }
-
- if (use_fmr == SRP_MAP_ALLOW_FMR && srp_map_finish_fmr(&state, target))
- goto backtrack;
+ srp_map_sg(&state, target, req, scat, count);
/* We've mapped the request, now pull as much of the indirect
* descriptor table as we can into the command buffer. If this
@@ -1144,9 +1464,9 @@
* guaranteed to fit into the command, as the SCSI layer won't
* give us more S/G entries than we allow.
*/
- req->nfmr = state.nfmr;
if (state.ndesc == 1) {
- /* FMR mapping was able to collapse this to one entry,
+ /*
+ * Memory registration collapsed the sg-list into one entry,
* so use a direct descriptor.
*/
struct srp_direct_buf *buf = (void *) cmd->add_data;
@@ -1455,6 +1775,7 @@
/**
* srp_tl_err_work() - handle a transport layer error
+ * @work: Work structure embedded in an SRP target port.
*
* Note: This function may get invoked before the rport has been created,
* hence the target->rport test.
@@ -1468,14 +1789,24 @@
srp_start_tl_fail_timers(target->rport);
}
-static void srp_handle_qp_err(enum ib_wc_status wc_status, bool send_err,
- struct srp_target_port *target)
+static void srp_handle_qp_err(u64 wr_id, enum ib_wc_status wc_status,
+ bool send_err, struct srp_target_port *target)
{
if (target->connected && !target->qp_in_error) {
- shost_printk(KERN_ERR, target->scsi_host,
- PFX "failed %s status %d\n",
- send_err ? "send" : "receive",
- wc_status);
+ if (wr_id & LOCAL_INV_WR_ID_MASK) {
+ shost_printk(KERN_ERR, target->scsi_host, PFX
+ "LOCAL_INV failed with status %d\n",
+ wc_status);
+ } else if (wr_id & FAST_REG_WR_ID_MASK) {
+ shost_printk(KERN_ERR, target->scsi_host, PFX
+ "FAST_REG_MR failed status %d\n",
+ wc_status);
+ } else {
+ shost_printk(KERN_ERR, target->scsi_host,
+ PFX "failed %s status %d for iu %p\n",
+ send_err ? "send" : "receive",
+ wc_status, (void *)(uintptr_t)wr_id);
+ }
queue_work(system_long_wq, &target->tl_err_work);
}
target->qp_in_error = true;
@@ -1491,7 +1822,7 @@
if (likely(wc.status == IB_WC_SUCCESS)) {
srp_handle_recv(target, &wc);
} else {
- srp_handle_qp_err(wc.status, false, target);
+ srp_handle_qp_err(wc.wr_id, wc.status, false, target);
}
}
}
@@ -1507,7 +1838,7 @@
iu = (struct srp_iu *) (uintptr_t) wc.wr_id;
list_add(&iu->list, &target->free_tx);
} else {
- srp_handle_qp_err(wc.status, true, target);
+ srp_handle_qp_err(wc.wr_id, wc.status, true, target);
}
}
}
@@ -1521,7 +1852,7 @@
struct srp_cmd *cmd;
struct ib_device *dev;
unsigned long flags;
- int len, result;
+ int len, ret;
const bool in_scsi_eh = !in_interrupt() && current == shost->ehandler;
/*
@@ -1533,12 +1864,9 @@
if (in_scsi_eh)
mutex_lock(&rport->mutex);
- result = srp_chkready(target->rport);
- if (unlikely(result)) {
- scmnd->result = result;
- scmnd->scsi_done(scmnd);
- goto unlock_rport;
- }
+ scmnd->result = srp_chkready(target->rport);
+ if (unlikely(scmnd->result))
+ goto err;
spin_lock_irqsave(&target->lock, flags);
iu = __srp_get_tx_iu(target, SRP_IU_CMD);
@@ -1553,7 +1881,6 @@
ib_dma_sync_single_for_cpu(dev, iu->dma, target->max_iu_len,
DMA_TO_DEVICE);
- scmnd->result = 0;
scmnd->host_scribble = (void *) req;
cmd = iu->buf;
@@ -1570,7 +1897,15 @@
len = srp_map_data(scmnd, target, req);
if (len < 0) {
shost_printk(KERN_ERR, target->scsi_host,
- PFX "Failed to map data\n");
+ PFX "Failed to map data (%d)\n", len);
+ /*
+ * If we ran out of memory descriptors (-ENOMEM) because an
+ * application is queuing many requests with more than
+ * max_pages_per_mr sg-list elements, tell the SCSI mid-layer
+ * to reduce queue depth temporarily.
+ */
+ scmnd->result = len == -ENOMEM ?
+ DID_OK << 16 | QUEUE_FULL << 1 : DID_ERROR << 16;
goto err_iu;
}
@@ -1582,11 +1917,13 @@
goto err_unmap;
}
+ ret = 0;
+
unlock_rport:
if (in_scsi_eh)
mutex_unlock(&rport->mutex);
- return 0;
+ return ret;
err_unmap:
srp_unmap_data(scmnd, target, req);
@@ -1594,16 +1931,27 @@
err_iu:
srp_put_tx_iu(target, iu, SRP_IU_CMD);
+ /*
+ * Avoid that the loops that iterate over the request ring can
+ * encounter a dangling SCSI command pointer.
+ */
+ req->scmnd = NULL;
+
spin_lock_irqsave(&target->lock, flags);
list_add(&req->list, &target->free_reqs);
err_unlock:
spin_unlock_irqrestore(&target->lock, flags);
- if (in_scsi_eh)
- mutex_unlock(&rport->mutex);
+err:
+ if (scmnd->result) {
+ scmnd->scsi_done(scmnd);
+ ret = 0;
+ } else {
+ ret = SCSI_MLQUEUE_HOST_BUSY;
+ }
- return SCSI_MLQUEUE_HOST_BUSY;
+ goto unlock_rport;
}
/*
@@ -2310,6 +2658,8 @@
/**
* srp_conn_unique() - check whether the connection to a target is unique
+ * @host: SRP host.
+ * @target: SRP target port.
*/
static bool srp_conn_unique(struct srp_host *host,
struct srp_target_port *target)
@@ -2605,7 +2955,8 @@
container_of(dev, struct srp_host, dev);
struct Scsi_Host *target_host;
struct srp_target_port *target;
- struct ib_device *ibdev = host->srp_dev->dev;
+ struct srp_device *srp_dev = host->srp_dev;
+ struct ib_device *ibdev = srp_dev->dev;
int ret;
target_host = scsi_host_alloc(&srp_template,
@@ -2650,9 +3001,9 @@
goto err;
}
- if (!host->srp_dev->fmr_pool && !target->allow_ext_sg &&
- target->cmd_sg_cnt < target->sg_tablesize) {
- pr_warn("No FMR pool and no external indirect descriptors, limiting sg_tablesize to cmd_sg_cnt\n");
+ if (!srp_dev->has_fmr && !srp_dev->has_fr && !target->allow_ext_sg &&
+ target->cmd_sg_cnt < target->sg_tablesize) {
+ pr_warn("No MR pool and no external indirect descriptors, limiting sg_tablesize to cmd_sg_cnt\n");
target->sg_tablesize = target->cmd_sg_cnt;
}
@@ -2790,9 +3141,9 @@
{
struct srp_device *srp_dev;
struct ib_device_attr *dev_attr;
- struct ib_fmr_pool_param fmr_param;
struct srp_host *host;
- int max_pages_per_fmr, fmr_page_shift, s, e, p;
+ int mr_page_shift, s, e, p;
+ u64 max_pages_per_mr;
dev_attr = kmalloc(sizeof *dev_attr, GFP_KERNEL);
if (!dev_attr)
@@ -2807,15 +3158,39 @@
if (!srp_dev)
goto free_attr;
+ srp_dev->has_fmr = (device->alloc_fmr && device->dealloc_fmr &&
+ device->map_phys_fmr && device->unmap_fmr);
+ srp_dev->has_fr = (dev_attr->device_cap_flags &
+ IB_DEVICE_MEM_MGT_EXTENSIONS);
+ if (!srp_dev->has_fmr && !srp_dev->has_fr)
+ dev_warn(&device->dev, "neither FMR nor FR is supported\n");
+
+ srp_dev->use_fast_reg = (srp_dev->has_fr &&
+ (!srp_dev->has_fmr || prefer_fr));
+
/*
* Use the smallest page size supported by the HCA, down to a
* minimum of 4096 bytes. We're unlikely to build large sglists
* out of smaller entries.
*/
- fmr_page_shift = max(12, ffs(dev_attr->page_size_cap) - 1);
- srp_dev->fmr_page_size = 1 << fmr_page_shift;
- srp_dev->fmr_page_mask = ~((u64) srp_dev->fmr_page_size - 1);
- srp_dev->fmr_max_size = srp_dev->fmr_page_size * SRP_FMR_SIZE;
+ mr_page_shift = max(12, ffs(dev_attr->page_size_cap) - 1);
+ srp_dev->mr_page_size = 1 << mr_page_shift;
+ srp_dev->mr_page_mask = ~((u64) srp_dev->mr_page_size - 1);
+ max_pages_per_mr = dev_attr->max_mr_size;
+ do_div(max_pages_per_mr, srp_dev->mr_page_size);
+ srp_dev->max_pages_per_mr = min_t(u64, SRP_MAX_PAGES_PER_MR,
+ max_pages_per_mr);
+ if (srp_dev->use_fast_reg) {
+ srp_dev->max_pages_per_mr =
+ min_t(u32, srp_dev->max_pages_per_mr,
+ dev_attr->max_fast_reg_page_list_len);
+ }
+ srp_dev->mr_max_size = srp_dev->mr_page_size *
+ srp_dev->max_pages_per_mr;
+ pr_debug("%s: mr_page_shift = %d, dev_attr->max_mr_size = %#llx, dev_attr->max_fast_reg_page_list_len = %u, max_pages_per_mr = %d, mr_max_size = %#x\n",
+ device->name, mr_page_shift, dev_attr->max_mr_size,
+ dev_attr->max_fast_reg_page_list_len,
+ srp_dev->max_pages_per_mr, srp_dev->mr_max_size);
INIT_LIST_HEAD(&srp_dev->dev_list);
@@ -2831,27 +3206,6 @@
if (IS_ERR(srp_dev->mr))
goto err_pd;
- for (max_pages_per_fmr = SRP_FMR_SIZE;
- max_pages_per_fmr >= SRP_FMR_MIN_SIZE;
- max_pages_per_fmr /= 2, srp_dev->fmr_max_size /= 2) {
- memset(&fmr_param, 0, sizeof fmr_param);
- fmr_param.pool_size = SRP_FMR_POOL_SIZE;
- fmr_param.dirty_watermark = SRP_FMR_DIRTY_SIZE;
- fmr_param.cache = 1;
- fmr_param.max_pages_per_fmr = max_pages_per_fmr;
- fmr_param.page_shift = fmr_page_shift;
- fmr_param.access = (IB_ACCESS_LOCAL_WRITE |
- IB_ACCESS_REMOTE_WRITE |
- IB_ACCESS_REMOTE_READ);
-
- srp_dev->fmr_pool = ib_create_fmr_pool(srp_dev->pd, &fmr_param);
- if (!IS_ERR(srp_dev->fmr_pool))
- break;
- }
-
- if (IS_ERR(srp_dev->fmr_pool))
- srp_dev->fmr_pool = NULL;
-
if (device->node_type == RDMA_NODE_IB_SWITCH) {
s = 0;
e = 0;
@@ -2914,8 +3268,6 @@
kfree(host);
}
- if (srp_dev->fmr_pool)
- ib_destroy_fmr_pool(srp_dev->fmr_pool);
ib_dereg_mr(srp_dev->mr);
ib_dealloc_pd(srp_dev->pd);
diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h
index aad27b7..e46ecb1 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.h
+++ b/drivers/infiniband/ulp/srp/ib_srp.h
@@ -66,13 +66,10 @@
SRP_TAG_NO_REQ = ~0U,
SRP_TAG_TSK_MGMT = 1U << 31,
- SRP_FMR_SIZE = 512,
- SRP_FMR_MIN_SIZE = 128,
- SRP_FMR_POOL_SIZE = 1024,
- SRP_FMR_DIRTY_SIZE = SRP_FMR_POOL_SIZE / 4,
+ SRP_MAX_PAGES_PER_MR = 512,
- SRP_MAP_ALLOW_FMR = 0,
- SRP_MAP_NO_FMR = 1,
+ LOCAL_INV_WR_ID_MASK = 1,
+ FAST_REG_WR_ID_MASK = 2,
};
enum srp_target_state {
@@ -86,15 +83,24 @@
SRP_IU_RSP,
};
+/*
+ * @mr_page_mask: HCA memory registration page mask.
+ * @mr_page_size: HCA memory registration page size.
+ * @mr_max_size: Maximum size in bytes of a single FMR / FR registration
+ * request.
+ */
struct srp_device {
struct list_head dev_list;
struct ib_device *dev;
struct ib_pd *pd;
struct ib_mr *mr;
- struct ib_fmr_pool *fmr_pool;
- u64 fmr_page_mask;
- int fmr_page_size;
- int fmr_max_size;
+ u64 mr_page_mask;
+ int mr_page_size;
+ int mr_max_size;
+ int max_pages_per_mr;
+ bool has_fmr;
+ bool has_fr;
+ bool use_fast_reg;
};
struct srp_host {
@@ -112,11 +118,14 @@
struct list_head list;
struct scsi_cmnd *scmnd;
struct srp_iu *cmd;
- struct ib_pool_fmr **fmr_list;
+ union {
+ struct ib_pool_fmr **fmr_list;
+ struct srp_fr_desc **fr_list;
+ };
u64 *map_page;
struct srp_direct_buf *indirect_desc;
dma_addr_t indirect_dma_addr;
- short nfmr;
+ short nmdesc;
short index;
};
@@ -131,6 +140,10 @@
struct ib_cq *send_cq ____cacheline_aligned_in_smp;
struct ib_cq *recv_cq;
struct ib_qp *qp;
+ union {
+ struct ib_fmr_pool *fmr_pool;
+ struct srp_fr_pool *fr_pool;
+ };
u32 lkey;
u32 rkey;
enum srp_target_state state;
@@ -197,15 +210,66 @@
enum dma_data_direction direction;
};
+/**
+ * struct srp_fr_desc - fast registration work request arguments
+ * @entry: Entry in srp_fr_pool.free_list.
+ * @mr: Memory region.
+ * @frpl: Fast registration page list.
+ */
+struct srp_fr_desc {
+ struct list_head entry;
+ struct ib_mr *mr;
+ struct ib_fast_reg_page_list *frpl;
+};
+
+/**
+ * struct srp_fr_pool - pool of fast registration descriptors
+ *
+ * An entry is available for allocation if and only if it occurs in @free_list.
+ *
+ * @size: Number of descriptors in this pool.
+ * @max_page_list_len: Maximum fast registration work request page list length.
+ * @lock: Protects free_list.
+ * @free_list: List of free descriptors.
+ * @desc: Fast registration descriptor pool.
+ */
+struct srp_fr_pool {
+ int size;
+ int max_page_list_len;
+ spinlock_t lock;
+ struct list_head free_list;
+ struct srp_fr_desc desc[0];
+};
+
+/**
+ * struct srp_map_state - per-request DMA memory mapping state
+ * @desc: Pointer to the element of the SRP buffer descriptor array
+ * that is being filled in.
+ * @pages: Array with DMA addresses of pages being considered for
+ * memory registration.
+ * @base_dma_addr: DMA address of the first page that has not yet been mapped.
+ * @dma_len: Number of bytes that will be registered with the next
+ * FMR or FR memory registration call.
+ * @total_len: Total number of bytes in the sg-list being mapped.
+ * @npages: Number of page addresses in the pages[] array.
+ * @nmdesc: Number of FMR or FR memory descriptors used for mapping.
+ * @ndesc: Number of SRP buffer descriptors that have been filled in.
+ * @unmapped_sg: First element of the sg-list that is mapped via FMR or FR.
+ * @unmapped_index: Index of the first element mapped via FMR or FR.
+ * @unmapped_addr: DMA address of the first element mapped via FMR or FR.
+ */
struct srp_map_state {
- struct ib_pool_fmr **next_fmr;
+ union {
+ struct ib_pool_fmr **next_fmr;
+ struct srp_fr_desc **next_fr;
+ };
struct srp_direct_buf *desc;
u64 *pages;
dma_addr_t base_dma_addr;
- u32 fmr_len;
+ u32 dma_len;
u32 total_len;
unsigned int npages;
- unsigned int nfmr;
+ unsigned int nmdesc;
unsigned int ndesc;
struct scatterlist *unmapped_sg;
int unmapped_index;
diff --git a/drivers/input/evdev.c b/drivers/input/evdev.c
index ce953d8..fd325ec 100644
--- a/drivers/input/evdev.c
+++ b/drivers/input/evdev.c
@@ -629,12 +629,10 @@
return copy_to_user(p, str, len) ? -EFAULT : len;
}
-#define OLD_KEY_MAX 0x1ff
static int handle_eviocgbit(struct input_dev *dev,
unsigned int type, unsigned int size,
void __user *p, int compat_mode)
{
- static unsigned long keymax_warn_time;
unsigned long *bits;
int len;
@@ -652,24 +650,8 @@
default: return -EINVAL;
}
- /*
- * Work around bugs in userspace programs that like to do
- * EVIOCGBIT(EV_KEY, KEY_MAX) and not realize that 'len'
- * should be in bytes, not in bits.
- */
- if (type == EV_KEY && size == OLD_KEY_MAX) {
- len = OLD_KEY_MAX;
- if (printk_timed_ratelimit(&keymax_warn_time, 10 * 1000))
- pr_warning("(EVIOCGBIT): Suspicious buffer size %u, "
- "limiting output to %zu bytes. See "
- "http://userweb.kernel.org/~dtor/eviocgbit-bug.html\n",
- OLD_KEY_MAX,
- BITS_TO_LONGS(OLD_KEY_MAX) * sizeof(long));
- }
-
return bits_to_user(bits, len, size, p, compat_mode);
}
-#undef OLD_KEY_MAX
static int evdev_handle_get_keycode(struct input_dev *dev, void __user *p)
{
diff --git a/drivers/input/input-polldev.c b/drivers/input/input-polldev.c
index 7f161d9..3664f81 100644
--- a/drivers/input/input-polldev.c
+++ b/drivers/input/input-polldev.c
@@ -147,6 +147,11 @@
.attrs = sysfs_attrs
};
+static const struct attribute_group *input_polldev_attribute_groups[] = {
+ &input_polldev_attribute_group,
+ NULL
+};
+
/**
* input_allocate_polled_device - allocate memory for polled device
*
@@ -171,6 +176,91 @@
}
EXPORT_SYMBOL(input_allocate_polled_device);
+struct input_polled_devres {
+ struct input_polled_dev *polldev;
+};
+
+static int devm_input_polldev_match(struct device *dev, void *res, void *data)
+{
+ struct input_polled_devres *devres = res;
+
+ return devres->polldev == data;
+}
+
+static void devm_input_polldev_release(struct device *dev, void *res)
+{
+ struct input_polled_devres *devres = res;
+ struct input_polled_dev *polldev = devres->polldev;
+
+ dev_dbg(dev, "%s: dropping reference/freeing %s\n",
+ __func__, dev_name(&polldev->input->dev));
+
+ input_put_device(polldev->input);
+ kfree(polldev);
+}
+
+static void devm_input_polldev_unregister(struct device *dev, void *res)
+{
+ struct input_polled_devres *devres = res;
+ struct input_polled_dev *polldev = devres->polldev;
+
+ dev_dbg(dev, "%s: unregistering device %s\n",
+ __func__, dev_name(&polldev->input->dev));
+ input_unregister_device(polldev->input);
+
+ /*
+ * Note that we are still holding extra reference to the input
+ * device so it will stick around until devm_input_polldev_release()
+ * is called.
+ */
+}
+
+/**
+ * devm_input_allocate_polled_device - allocate managed polled device
+ * @dev: device owning the polled device being created
+ *
+ * Returns prepared &struct input_polled_dev or %NULL.
+ *
+ * Managed polled input devices do not need to be explicitly unregistered
+ * or freed as it will be done automatically when owner device unbinds
+ * from * its driver (or binding fails). Once such managed polled device
+ * is allocated, it is ready to be set up and registered in the same
+ * fashion as regular polled input devices (using
+ * input_register_polled_device() function).
+ *
+ * If you want to manually unregister and free such managed polled devices,
+ * it can be still done by calling input_unregister_polled_device() and
+ * input_free_polled_device(), although it is rarely needed.
+ *
+ * NOTE: the owner device is set up as parent of input device and users
+ * should not override it.
+ */
+struct input_polled_dev *devm_input_allocate_polled_device(struct device *dev)
+{
+ struct input_polled_dev *polldev;
+ struct input_polled_devres *devres;
+
+ devres = devres_alloc(devm_input_polldev_release, sizeof(*devres),
+ GFP_KERNEL);
+ if (!devres)
+ return NULL;
+
+ polldev = input_allocate_polled_device();
+ if (!polldev) {
+ devres_free(devres);
+ return NULL;
+ }
+
+ polldev->input->dev.parent = dev;
+ polldev->devres_managed = true;
+
+ devres->polldev = polldev;
+ devres_add(dev, devres);
+
+ return polldev;
+}
+EXPORT_SYMBOL(devm_input_allocate_polled_device);
+
/**
* input_free_polled_device - free memory allocated for polled device
* @dev: device to free
@@ -181,7 +271,12 @@
void input_free_polled_device(struct input_polled_dev *dev)
{
if (dev) {
- input_free_device(dev->input);
+ if (dev->devres_managed)
+ WARN_ON(devres_destroy(dev->input->dev.parent,
+ devm_input_polldev_release,
+ devm_input_polldev_match,
+ dev));
+ input_put_device(dev->input);
kfree(dev);
}
}
@@ -199,26 +294,35 @@
*/
int input_register_polled_device(struct input_polled_dev *dev)
{
+ struct input_polled_devres *devres = NULL;
struct input_dev *input = dev->input;
int error;
+ if (dev->devres_managed) {
+ devres = devres_alloc(devm_input_polldev_unregister,
+ sizeof(*devres), GFP_KERNEL);
+ if (!devres)
+ return -ENOMEM;
+
+ devres->polldev = dev;
+ }
+
input_set_drvdata(input, dev);
INIT_DELAYED_WORK(&dev->work, input_polled_device_work);
+
if (!dev->poll_interval)
dev->poll_interval = 500;
if (!dev->poll_interval_max)
dev->poll_interval_max = dev->poll_interval;
+
input->open = input_open_polled_device;
input->close = input_close_polled_device;
- error = input_register_device(input);
- if (error)
- return error;
+ input->dev.groups = input_polldev_attribute_groups;
- error = sysfs_create_group(&input->dev.kobj,
- &input_polldev_attribute_group);
+ error = input_register_device(input);
if (error) {
- input_unregister_device(input);
+ devres_free(devres);
return error;
}
@@ -231,6 +335,12 @@
*/
input_get_device(input);
+ if (dev->devres_managed) {
+ dev_dbg(input->dev.parent, "%s: registering %s with devres.\n",
+ __func__, dev_name(&input->dev));
+ devres_add(input->dev.parent, devres);
+ }
+
return 0;
}
EXPORT_SYMBOL(input_register_polled_device);
@@ -245,8 +355,11 @@
*/
void input_unregister_polled_device(struct input_polled_dev *dev)
{
- sysfs_remove_group(&dev->input->dev.kobj,
- &input_polldev_attribute_group);
+ if (dev->devres_managed)
+ WARN_ON(devres_destroy(dev->input->dev.parent,
+ devm_input_polldev_unregister,
+ devm_input_polldev_match,
+ dev));
input_unregister_device(dev->input);
}
diff --git a/drivers/input/keyboard/Kconfig b/drivers/input/keyboard/Kconfig
index ffc7ad3..f7e79b4 100644
--- a/drivers/input/keyboard/Kconfig
+++ b/drivers/input/keyboard/Kconfig
@@ -524,6 +524,17 @@
To compile this driver as a module, choose M here: the
module will be called stowaway.
+config KEYBOARD_ST_KEYSCAN
+ tristate "STMicroelectronics keyscan support"
+ depends on ARCH_STI || COMPILE_TEST
+ select INPUT_MATRIXKMAP
+ help
+ Say Y here if you want to use a keypad attached to the keyscan block
+ on some STMicroelectronics SoC devices.
+
+ To compile this driver as a module, choose M here: the
+ module will be called st-keyscan.
+
config KEYBOARD_SUNKBD
tristate "Sun Type 4 and Type 5 keyboard"
select SERIO
@@ -578,7 +589,7 @@
config KEYBOARD_OMAP4
tristate "TI OMAP4+ keypad support"
- depends on ARCH_OMAP2PLUS
+ depends on OF || ARCH_OMAP2PLUS
select INPUT_MATRIXKMAP
help
Say Y here if you want to use the OMAP4+ keypad.
diff --git a/drivers/input/keyboard/Makefile b/drivers/input/keyboard/Makefile
index 11cff7b8..7504ae1 100644
--- a/drivers/input/keyboard/Makefile
+++ b/drivers/input/keyboard/Makefile
@@ -51,6 +51,7 @@
obj-$(CONFIG_KEYBOARD_SPEAR) += spear-keyboard.o
obj-$(CONFIG_KEYBOARD_STMPE) += stmpe-keypad.o
obj-$(CONFIG_KEYBOARD_STOWAWAY) += stowaway.o
+obj-$(CONFIG_KEYBOARD_ST_KEYSCAN) += st-keyscan.o
obj-$(CONFIG_KEYBOARD_SUNKBD) += sunkbd.o
obj-$(CONFIG_KEYBOARD_TC3589X) += tc3589x-keypad.o
obj-$(CONFIG_KEYBOARD_TEGRA) += tegra-kbc.o
diff --git a/drivers/input/keyboard/adp5520-keys.c b/drivers/input/keyboard/adp5520-keys.c
index 4cc14c2..7f4a8b5 100644
--- a/drivers/input/keyboard/adp5520-keys.c
+++ b/drivers/input/keyboard/adp5520-keys.c
@@ -12,6 +12,7 @@
#include <linux/input.h>
#include <linux/mfd/adp5520.h>
#include <linux/slab.h>
+#include <linux/device.h>
struct adp5520_keys {
struct input_dev *input;
@@ -81,7 +82,7 @@
return -EINVAL;
}
- if (pdata == NULL) {
+ if (!pdata) {
dev_err(&pdev->dev, "missing platform data\n");
return -EINVAL;
}
@@ -89,17 +90,15 @@
if (!(pdata->rows_en_mask && pdata->cols_en_mask))
return -EINVAL;
- dev = kzalloc(sizeof(*dev), GFP_KERNEL);
- if (dev == NULL) {
+ dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+ if (!dev) {
dev_err(&pdev->dev, "failed to alloc memory\n");
return -ENOMEM;
}
- input = input_allocate_device();
- if (!input) {
- ret = -ENOMEM;
- goto err;
- }
+ input = devm_input_allocate_device(&pdev->dev);
+ if (!input)
+ return -ENOMEM;
dev->master = pdev->dev.parent;
dev->input = input;
@@ -135,7 +134,7 @@
ret = input_register_device(input);
if (ret) {
dev_err(&pdev->dev, "unable to register input device\n");
- goto err;
+ return ret;
}
en_mask = pdata->rows_en_mask | pdata->cols_en_mask;
@@ -157,8 +156,7 @@
if (ret) {
dev_err(&pdev->dev, "failed to write\n");
- ret = -EIO;
- goto err1;
+ return -EIO;
}
dev->notifier.notifier_call = adp5520_keys_notifier;
@@ -166,19 +164,11 @@
ADP5520_KP_IEN | ADP5520_KR_IEN);
if (ret) {
dev_err(&pdev->dev, "failed to register notifier\n");
- goto err1;
+ return ret;
}
platform_set_drvdata(pdev, dev);
return 0;
-
-err1:
- input_unregister_device(input);
- input = NULL;
-err:
- input_free_device(input);
- kfree(dev);
- return ret;
}
static int adp5520_keys_remove(struct platform_device *pdev)
@@ -188,8 +178,6 @@
adp5520_unregister_notifier(dev->master, &dev->notifier,
ADP5520_KP_IEN | ADP5520_KR_IEN);
- input_unregister_device(dev->input);
- kfree(dev);
return 0;
}
diff --git a/drivers/input/keyboard/clps711x-keypad.c b/drivers/input/keyboard/clps711x-keypad.c
index 3955aec..552b65c 100644
--- a/drivers/input/keyboard/clps711x-keypad.c
+++ b/drivers/input/keyboard/clps711x-keypad.c
@@ -185,7 +185,7 @@
return 0;
}
-static struct of_device_id clps711x_keypad_of_match[] = {
+static const struct of_device_id clps711x_keypad_of_match[] = {
{ .compatible = "cirrus,clps711x-keypad", },
{ }
};
diff --git a/drivers/input/keyboard/gpio_keys.c b/drivers/input/keyboard/gpio_keys.c
index 2db1324..8c98e97 100644
--- a/drivers/input/keyboard/gpio_keys.c
+++ b/drivers/input/keyboard/gpio_keys.c
@@ -424,6 +424,16 @@
return IRQ_HANDLED;
}
+static void gpio_keys_quiesce_key(void *data)
+{
+ struct gpio_button_data *bdata = data;
+
+ if (bdata->timer_debounce)
+ del_timer_sync(&bdata->timer);
+
+ cancel_work_sync(&bdata->work);
+}
+
static int gpio_keys_setup_key(struct platform_device *pdev,
struct input_dev *input,
struct gpio_button_data *bdata,
@@ -433,7 +443,8 @@
struct device *dev = &pdev->dev;
irq_handler_t isr;
unsigned long irqflags;
- int irq, error;
+ int irq;
+ int error;
bdata->input = input;
bdata->button = button;
@@ -441,7 +452,8 @@
if (gpio_is_valid(button->gpio)) {
- error = gpio_request_one(button->gpio, GPIOF_IN, desc);
+ error = devm_gpio_request_one(&pdev->dev, button->gpio,
+ GPIOF_IN, desc);
if (error < 0) {
dev_err(dev, "Failed to request GPIO %d, error %d\n",
button->gpio, error);
@@ -463,7 +475,7 @@
dev_err(dev,
"Unable to get irq number for GPIO %d, error %d\n",
button->gpio, error);
- goto fail;
+ return error;
}
bdata->irq = irq;
@@ -497,26 +509,33 @@
input_set_capability(input, button->type ?: EV_KEY, button->code);
/*
+ * Install custom action to cancel debounce timer and
+ * workqueue item.
+ */
+ error = devm_add_action(&pdev->dev, gpio_keys_quiesce_key, bdata);
+ if (error) {
+ dev_err(&pdev->dev,
+ "failed to register quiesce action, error: %d\n",
+ error);
+ return error;
+ }
+
+ /*
* If platform has specified that the button can be disabled,
* we don't want it to share the interrupt line.
*/
if (!button->can_disable)
irqflags |= IRQF_SHARED;
- error = request_any_context_irq(bdata->irq, isr, irqflags, desc, bdata);
+ error = devm_request_any_context_irq(&pdev->dev, bdata->irq,
+ isr, irqflags, desc, bdata);
if (error < 0) {
dev_err(dev, "Unable to claim irq %d; error %d\n",
bdata->irq, error);
- goto fail;
+ return error;
}
return 0;
-
-fail:
- if (gpio_is_valid(button->gpio))
- gpio_free(button->gpio);
-
- return error;
}
static void gpio_keys_report_state(struct gpio_keys_drvdata *ddata)
@@ -578,23 +597,18 @@
int i;
node = dev->of_node;
- if (!node) {
- error = -ENODEV;
- goto err_out;
- }
+ if (!node)
+ return ERR_PTR(-ENODEV);
nbuttons = of_get_child_count(node);
- if (nbuttons == 0) {
- error = -ENODEV;
- goto err_out;
- }
+ if (nbuttons == 0)
+ return ERR_PTR(-ENODEV);
- pdata = kzalloc(sizeof(*pdata) + nbuttons * (sizeof *button),
- GFP_KERNEL);
- if (!pdata) {
- error = -ENOMEM;
- goto err_out;
- }
+ pdata = devm_kzalloc(dev,
+ sizeof(*pdata) + nbuttons * sizeof(*button),
+ GFP_KERNEL);
+ if (!pdata)
+ return ERR_PTR(-ENOMEM);
pdata->buttons = (struct gpio_keys_button *)(pdata + 1);
pdata->nbuttons = nbuttons;
@@ -619,7 +633,7 @@
dev_err(dev,
"Failed to get gpio flags, error: %d\n",
error);
- goto err_free_pdata;
+ return ERR_PTR(error);
}
button = &pdata->buttons[i++];
@@ -630,8 +644,7 @@
if (of_property_read_u32(pp, "linux,code", &button->code)) {
dev_err(dev, "Button without keycode: 0x%x\n",
button->gpio);
- error = -EINVAL;
- goto err_free_pdata;
+ return ERR_PTR(-EINVAL);
}
button->desc = of_get_property(pp, "label", NULL);
@@ -646,20 +659,13 @@
button->debounce_interval = 5;
}
- if (pdata->nbuttons == 0) {
- error = -EINVAL;
- goto err_free_pdata;
- }
+ if (pdata->nbuttons == 0)
+ return ERR_PTR(-EINVAL);
return pdata;
-
-err_free_pdata:
- kfree(pdata);
-err_out:
- return ERR_PTR(error);
}
-static struct of_device_id gpio_keys_of_match[] = {
+static const struct of_device_id gpio_keys_of_match[] = {
{ .compatible = "gpio-keys", },
{ },
};
@@ -675,22 +681,13 @@
#endif
-static void gpio_remove_key(struct gpio_button_data *bdata)
-{
- free_irq(bdata->irq, bdata);
- if (bdata->timer_debounce)
- del_timer_sync(&bdata->timer);
- cancel_work_sync(&bdata->work);
- if (gpio_is_valid(bdata->button->gpio))
- gpio_free(bdata->button->gpio);
-}
-
static int gpio_keys_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
const struct gpio_keys_platform_data *pdata = dev_get_platdata(dev);
struct gpio_keys_drvdata *ddata;
struct input_dev *input;
+ size_t size;
int i, error;
int wakeup = 0;
@@ -700,14 +697,18 @@
return PTR_ERR(pdata);
}
- ddata = kzalloc(sizeof(struct gpio_keys_drvdata) +
- pdata->nbuttons * sizeof(struct gpio_button_data),
- GFP_KERNEL);
- input = input_allocate_device();
- if (!ddata || !input) {
+ size = sizeof(struct gpio_keys_drvdata) +
+ pdata->nbuttons * sizeof(struct gpio_button_data);
+ ddata = devm_kzalloc(dev, size, GFP_KERNEL);
+ if (!ddata) {
dev_err(dev, "failed to allocate state\n");
- error = -ENOMEM;
- goto fail1;
+ return -ENOMEM;
+ }
+
+ input = devm_input_allocate_device(dev);
+ if (!input) {
+ dev_err(dev, "failed to allocate input device\n");
+ return -ENOMEM;
}
ddata->pdata = pdata;
@@ -738,7 +739,7 @@
error = gpio_keys_setup_key(pdev, input, bdata, button);
if (error)
- goto fail2;
+ return error;
if (button->wakeup)
wakeup = 1;
@@ -748,57 +749,31 @@
if (error) {
dev_err(dev, "Unable to export keys/switches, error: %d\n",
error);
- goto fail2;
+ return error;
}
error = input_register_device(input);
if (error) {
dev_err(dev, "Unable to register input device, error: %d\n",
error);
- goto fail3;
+ goto err_remove_group;
}
device_init_wakeup(&pdev->dev, wakeup);
return 0;
- fail3:
+err_remove_group:
sysfs_remove_group(&pdev->dev.kobj, &gpio_keys_attr_group);
- fail2:
- while (--i >= 0)
- gpio_remove_key(&ddata->data[i]);
-
- fail1:
- input_free_device(input);
- kfree(ddata);
- /* If we have no platform data, we allocated pdata dynamically. */
- if (!dev_get_platdata(&pdev->dev))
- kfree(pdata);
-
return error;
}
static int gpio_keys_remove(struct platform_device *pdev)
{
- struct gpio_keys_drvdata *ddata = platform_get_drvdata(pdev);
- struct input_dev *input = ddata->input;
- int i;
-
sysfs_remove_group(&pdev->dev.kobj, &gpio_keys_attr_group);
device_init_wakeup(&pdev->dev, 0);
- for (i = 0; i < ddata->pdata->nbuttons; i++)
- gpio_remove_key(&ddata->data[i]);
-
- input_unregister_device(input);
-
- /* If we have no platform data, we allocated pdata dynamically. */
- if (!dev_get_platdata(&pdev->dev))
- kfree(ddata->pdata);
-
- kfree(ddata);
-
return 0;
}
diff --git a/drivers/input/keyboard/gpio_keys_polled.c b/drivers/input/keyboard/gpio_keys_polled.c
index e571e19..432d363 100644
--- a/drivers/input/keyboard/gpio_keys_polled.c
+++ b/drivers/input/keyboard/gpio_keys_polled.c
@@ -120,12 +120,10 @@
if (nbuttons == 0)
return NULL;
- pdata = kzalloc(sizeof(*pdata) + nbuttons * (sizeof *button),
- GFP_KERNEL);
- if (!pdata) {
- error = -ENOMEM;
- goto err_out;
- }
+ pdata = devm_kzalloc(dev, sizeof(*pdata) + nbuttons * sizeof(*button),
+ GFP_KERNEL);
+ if (!pdata)
+ return ERR_PTR(-ENOMEM);
pdata->buttons = (struct gpio_keys_button *)(pdata + 1);
pdata->nbuttons = nbuttons;
@@ -151,7 +149,7 @@
dev_err(dev,
"Failed to get gpio flags, error: %d\n",
error);
- goto err_free_pdata;
+ return ERR_PTR(error);
}
button = &pdata->buttons[i++];
@@ -162,8 +160,7 @@
if (of_property_read_u32(pp, "linux,code", &button->code)) {
dev_err(dev, "Button without keycode: 0x%x\n",
button->gpio);
- error = -EINVAL;
- goto err_free_pdata;
+ return ERR_PTR(-EINVAL);
}
button->desc = of_get_property(pp, "label", NULL);
@@ -178,20 +175,13 @@
button->debounce_interval = 5;
}
- if (pdata->nbuttons == 0) {
- error = -EINVAL;
- goto err_free_pdata;
- }
+ if (pdata->nbuttons == 0)
+ return ERR_PTR(-EINVAL);
return pdata;
-
-err_free_pdata:
- kfree(pdata);
-err_out:
- return ERR_PTR(error);
}
-static struct of_device_id gpio_keys_polled_of_match[] = {
+static const struct of_device_id gpio_keys_polled_of_match[] = {
{ .compatible = "gpio-keys-polled", },
{ },
};
@@ -213,6 +203,7 @@
struct gpio_keys_polled_dev *bdev;
struct input_polled_dev *poll_dev;
struct input_dev *input;
+ size_t size;
int error;
int i;
@@ -228,24 +219,21 @@
if (!pdata->poll_interval) {
dev_err(dev, "missing poll_interval value\n");
- error = -EINVAL;
- goto err_free_pdata;
+ return -EINVAL;
}
- bdev = kzalloc(sizeof(struct gpio_keys_polled_dev) +
- pdata->nbuttons * sizeof(struct gpio_keys_button_data),
- GFP_KERNEL);
+ size = sizeof(struct gpio_keys_polled_dev) +
+ pdata->nbuttons * sizeof(struct gpio_keys_button_data);
+ bdev = devm_kzalloc(&pdev->dev, size, GFP_KERNEL);
if (!bdev) {
dev_err(dev, "no memory for private data\n");
- error = -ENOMEM;
- goto err_free_pdata;
+ return -ENOMEM;
}
- poll_dev = input_allocate_polled_device();
+ poll_dev = devm_input_allocate_polled_device(&pdev->dev);
if (!poll_dev) {
dev_err(dev, "no memory for polled device\n");
- error = -ENOMEM;
- goto err_free_bdev;
+ return -ENOMEM;
}
poll_dev->private = bdev;
@@ -258,7 +246,6 @@
input->name = pdev->name;
input->phys = DRV_NAME"/input0";
- input->dev.parent = &pdev->dev;
input->id.bustype = BUS_HOST;
input->id.vendor = 0x0001;
@@ -277,16 +264,15 @@
if (button->wakeup) {
dev_err(dev, DRV_NAME " does not support wakeup\n");
- error = -EINVAL;
- goto err_free_gpio;
+ return -EINVAL;
}
- error = gpio_request_one(gpio, GPIOF_IN,
- button->desc ?: DRV_NAME);
+ error = devm_gpio_request_one(&pdev->dev, gpio, GPIOF_IN,
+ button->desc ? : DRV_NAME);
if (error) {
dev_err(dev, "unable to claim gpio %u, err=%d\n",
gpio, error);
- goto err_free_gpio;
+ return error;
}
bdata->can_sleep = gpio_cansleep(gpio);
@@ -306,7 +292,7 @@
if (error) {
dev_err(dev, "unable to register polled device, err=%d\n",
error);
- goto err_free_gpio;
+ return error;
}
/* report initial state of the buttons */
@@ -315,52 +301,10 @@
&bdev->data[i]);
return 0;
-
-err_free_gpio:
- while (--i >= 0)
- gpio_free(pdata->buttons[i].gpio);
-
- input_free_polled_device(poll_dev);
-
-err_free_bdev:
- kfree(bdev);
-
-err_free_pdata:
- /* If we have no platform_data, we allocated pdata dynamically. */
- if (!dev_get_platdata(&pdev->dev))
- kfree(pdata);
-
- return error;
-}
-
-static int gpio_keys_polled_remove(struct platform_device *pdev)
-{
- struct gpio_keys_polled_dev *bdev = platform_get_drvdata(pdev);
- const struct gpio_keys_platform_data *pdata = bdev->pdata;
- int i;
-
- input_unregister_polled_device(bdev->poll_dev);
-
- for (i = 0; i < pdata->nbuttons; i++)
- gpio_free(pdata->buttons[i].gpio);
-
- input_free_polled_device(bdev->poll_dev);
-
- /*
- * If we had no platform_data, we allocated pdata dynamically and
- * must free it here.
- */
- if (!dev_get_platdata(&pdev->dev))
- kfree(pdata);
-
- kfree(bdev);
-
- return 0;
}
static struct platform_driver gpio_keys_polled_driver = {
.probe = gpio_keys_polled_probe,
- .remove = gpio_keys_polled_remove,
.driver = {
.name = DRV_NAME,
.owner = THIS_MODULE,
diff --git a/drivers/input/keyboard/imx_keypad.c b/drivers/input/keyboard/imx_keypad.c
index 97ec335..8280cb1 100644
--- a/drivers/input/keyboard/imx_keypad.c
+++ b/drivers/input/keyboard/imx_keypad.c
@@ -415,7 +415,7 @@
}
#ifdef CONFIG_OF
-static struct of_device_id imx_keypad_of_match[] = {
+static const struct of_device_id imx_keypad_of_match[] = {
{ .compatible = "fsl,imx21-kpp", },
{ /* sentinel */ }
};
diff --git a/drivers/input/keyboard/jornada680_kbd.c b/drivers/input/keyboard/jornada680_kbd.c
index 69b1f00..0ba4428 100644
--- a/drivers/input/keyboard/jornada680_kbd.c
+++ b/drivers/input/keyboard/jornada680_kbd.c
@@ -16,6 +16,7 @@
* published by the Free Software Foundation.
*/
+#include <linux/device.h>
#include <linux/input.h>
#include <linux/input-polldev.h>
#include <linux/interrupt.h>
@@ -185,14 +186,15 @@
struct input_dev *input_dev;
int i, error;
- jornadakbd = kzalloc(sizeof(struct jornadakbd), GFP_KERNEL);
+ jornadakbd = devm_kzalloc(&pdev->dev, sizeof(struct jornadakbd),
+ GFP_KERNEL);
if (!jornadakbd)
return -ENOMEM;
- poll_dev = input_allocate_polled_device();
+ poll_dev = devm_input_allocate_polled_device(&pdev->dev);
if (!poll_dev) {
- error = -ENOMEM;
- goto failed;
+ dev_err(&pdev->dev, "failed to allocate polled input device\n");
+ return -ENOMEM;
}
platform_set_drvdata(pdev, jornadakbd);
@@ -224,27 +226,10 @@
input_set_capability(input_dev, EV_MSC, MSC_SCAN);
error = input_register_polled_device(jornadakbd->poll_dev);
- if (error)
- goto failed;
-
- return 0;
-
- failed:
- printk(KERN_ERR "Jornadakbd: failed to register driver, error: %d\n",
- error);
- input_free_polled_device(poll_dev);
- kfree(jornadakbd);
- return error;
-
-}
-
-static int jornada680kbd_remove(struct platform_device *pdev)
-{
- struct jornadakbd *jornadakbd = platform_get_drvdata(pdev);
-
- input_unregister_polled_device(jornadakbd->poll_dev);
- input_free_polled_device(jornadakbd->poll_dev);
- kfree(jornadakbd);
+ if (error) {
+ dev_err(&pdev->dev, "failed to register polled input device\n");
+ return error;
+ }
return 0;
}
@@ -255,7 +240,6 @@
.owner = THIS_MODULE,
},
.probe = jornada680kbd_probe,
- .remove = jornada680kbd_remove,
};
module_platform_driver(jornada680kbd_driver);
diff --git a/drivers/input/keyboard/mcs_touchkey.c b/drivers/input/keyboard/mcs_touchkey.c
index 1da8e0b..375b05c 100644
--- a/drivers/input/keyboard/mcs_touchkey.c
+++ b/drivers/input/keyboard/mcs_touchkey.c
@@ -147,7 +147,7 @@
}
dev_info(&client->dev, "Firmware version: %d\n", fw_ver);
- input_dev->name = "MELPAS MCS Touchkey";
+ input_dev->name = "MELFAS MCS Touchkey";
input_dev->id.bustype = BUS_I2C;
input_dev->dev.parent = &client->dev;
input_dev->evbit[0] = BIT_MASK(EV_KEY);
diff --git a/drivers/input/keyboard/omap4-keypad.c b/drivers/input/keyboard/omap4-keypad.c
index 0400b3f..024b7bd 100644
--- a/drivers/input/keyboard/omap4-keypad.c
+++ b/drivers/input/keyboard/omap4-keypad.c
@@ -28,11 +28,10 @@
#include <linux/io.h>
#include <linux/of.h>
#include <linux/input.h>
+#include <linux/input/matrix_keypad.h>
#include <linux/slab.h>
#include <linux/pm_runtime.h>
-#include <linux/platform_data/omap4-keypad.h>
-
/* OMAP4 registers */
#define OMAP4_KBD_REVISION 0x00
#define OMAP4_KBD_SYSCONFIG 0x10
@@ -218,7 +217,6 @@
pm_runtime_put_sync(input->dev.parent);
}
-#ifdef CONFIG_OF
static int omap4_keypad_parse_dt(struct device *dev,
struct omap4_keypad *keypad_data)
{
@@ -235,20 +233,9 @@
return 0;
}
-#else
-static inline int omap4_keypad_parse_dt(struct device *dev,
- struct omap4_keypad *keypad_data)
-{
- return -ENOSYS;
-}
-#endif
static int omap4_keypad_probe(struct platform_device *pdev)
{
- const struct omap4_keypad_platform_data *pdata =
- dev_get_platdata(&pdev->dev);
- const struct matrix_keymap_data *keymap_data =
- pdata ? pdata->keymap_data : NULL;
struct omap4_keypad *keypad_data;
struct input_dev *input_dev;
struct resource *res;
@@ -277,14 +264,9 @@
keypad_data->irq = irq;
- if (pdata) {
- keypad_data->rows = pdata->rows;
- keypad_data->cols = pdata->cols;
- } else {
- error = omap4_keypad_parse_dt(&pdev->dev, keypad_data);
- if (error)
- return error;
- }
+ error = omap4_keypad_parse_dt(&pdev->dev, keypad_data);
+ if (error)
+ return error;
res = request_mem_region(res->start, resource_size(res), pdev->name);
if (!res) {
@@ -363,7 +345,7 @@
goto err_free_input;
}
- error = matrix_keypad_build_keymap(keymap_data, NULL,
+ error = matrix_keypad_build_keymap(NULL, NULL,
keypad_data->rows, keypad_data->cols,
keypad_data->keymap, input_dev);
if (error) {
@@ -434,13 +416,11 @@
return 0;
}
-#ifdef CONFIG_OF
static const struct of_device_id omap_keypad_dt_match[] = {
{ .compatible = "ti,omap4-keypad" },
{},
};
MODULE_DEVICE_TABLE(of, omap_keypad_dt_match);
-#endif
#ifdef CONFIG_PM_SLEEP
static int omap4_keypad_suspend(struct device *dev)
@@ -482,7 +462,7 @@
.name = "omap4-keypad",
.owner = THIS_MODULE,
.pm = &omap4_keypad_pm_ops,
- .of_match_table = of_match_ptr(omap_keypad_dt_match),
+ .of_match_table = omap_keypad_dt_match,
},
};
module_platform_driver(omap4_keypad_driver);
diff --git a/drivers/input/keyboard/st-keyscan.c b/drivers/input/keyboard/st-keyscan.c
new file mode 100644
index 0000000..758b487
--- /dev/null
+++ b/drivers/input/keyboard/st-keyscan.c
@@ -0,0 +1,274 @@
+/*
+ * STMicroelectronics Key Scanning driver
+ *
+ * Copyright (c) 2014 STMicroelectonics Ltd.
+ * Author: Stuart Menefy <stuart.menefy@st.com>
+ *
+ * Based on sh_keysc.c, copyright 2008 Magnus Damm
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/module.h>
+#include <linux/interrupt.h>
+#include <linux/platform_device.h>
+#include <linux/clk.h>
+#include <linux/io.h>
+#include <linux/input/matrix_keypad.h>
+
+#define ST_KEYSCAN_MAXKEYS 16
+
+#define KEYSCAN_CONFIG_OFF 0x0
+#define KEYSCAN_CONFIG_ENABLE 0x1
+#define KEYSCAN_DEBOUNCE_TIME_OFF 0x4
+#define KEYSCAN_MATRIX_STATE_OFF 0x8
+#define KEYSCAN_MATRIX_DIM_OFF 0xc
+#define KEYSCAN_MATRIX_DIM_X_SHIFT 0x0
+#define KEYSCAN_MATRIX_DIM_Y_SHIFT 0x2
+
+struct st_keyscan {
+ void __iomem *base;
+ int irq;
+ struct clk *clk;
+ struct input_dev *input_dev;
+ unsigned long last_state;
+ unsigned int n_rows;
+ unsigned int n_cols;
+ unsigned int debounce_us;
+};
+
+static irqreturn_t keyscan_isr(int irq, void *dev_id)
+{
+ struct st_keyscan *keypad = dev_id;
+ unsigned short *keycode = keypad->input_dev->keycode;
+ unsigned long state, change;
+ int bit_nr;
+
+ state = readl(keypad->base + KEYSCAN_MATRIX_STATE_OFF) & 0xffff;
+ change = keypad->last_state ^ state;
+ keypad->last_state = state;
+
+ for_each_set_bit(bit_nr, &change, BITS_PER_LONG)
+ input_report_key(keypad->input_dev,
+ keycode[bit_nr], state & BIT(bit_nr));
+
+ input_sync(keypad->input_dev);
+
+ return IRQ_HANDLED;
+}
+
+static int keyscan_start(struct st_keyscan *keypad)
+{
+ int error;
+
+ error = clk_enable(keypad->clk);
+ if (error)
+ return error;
+
+ writel(keypad->debounce_us * (clk_get_rate(keypad->clk) / 1000000),
+ keypad->base + KEYSCAN_DEBOUNCE_TIME_OFF);
+
+ writel(((keypad->n_cols - 1) << KEYSCAN_MATRIX_DIM_X_SHIFT) |
+ ((keypad->n_rows - 1) << KEYSCAN_MATRIX_DIM_Y_SHIFT),
+ keypad->base + KEYSCAN_MATRIX_DIM_OFF);
+
+ writel(KEYSCAN_CONFIG_ENABLE, keypad->base + KEYSCAN_CONFIG_OFF);
+
+ return 0;
+}
+
+static void keyscan_stop(struct st_keyscan *keypad)
+{
+ writel(0, keypad->base + KEYSCAN_CONFIG_OFF);
+
+ clk_disable(keypad->clk);
+}
+
+static int keyscan_open(struct input_dev *dev)
+{
+ struct st_keyscan *keypad = input_get_drvdata(dev);
+
+ return keyscan_start(keypad);
+}
+
+static void keyscan_close(struct input_dev *dev)
+{
+ struct st_keyscan *keypad = input_get_drvdata(dev);
+
+ keyscan_stop(keypad);
+}
+
+static int keypad_matrix_key_parse_dt(struct st_keyscan *keypad_data)
+{
+ struct device *dev = keypad_data->input_dev->dev.parent;
+ struct device_node *np = dev->of_node;
+ int error;
+
+ error = matrix_keypad_parse_of_params(dev, &keypad_data->n_rows,
+ &keypad_data->n_cols);
+ if (error) {
+ dev_err(dev, "failed to parse keypad params\n");
+ return error;
+ }
+
+ of_property_read_u32(np, "st,debounce-us", &keypad_data->debounce_us);
+
+ dev_dbg(dev, "n_rows=%d n_col=%d debounce=%d\n",
+ keypad_data->n_rows, keypad_data->n_cols,
+ keypad_data->debounce_us);
+
+ return 0;
+}
+
+static int keyscan_probe(struct platform_device *pdev)
+{
+ struct st_keyscan *keypad_data;
+ struct input_dev *input_dev;
+ struct resource *res;
+ int error;
+
+ if (!pdev->dev.of_node) {
+ dev_err(&pdev->dev, "no DT data present\n");
+ return -EINVAL;
+ }
+
+ keypad_data = devm_kzalloc(&pdev->dev, sizeof(*keypad_data),
+ GFP_KERNEL);
+ if (!keypad_data)
+ return -ENOMEM;
+
+ input_dev = devm_input_allocate_device(&pdev->dev);
+ if (!input_dev) {
+ dev_err(&pdev->dev, "failed to allocate the input device\n");
+ return -ENOMEM;
+ }
+
+ input_dev->name = pdev->name;
+ input_dev->phys = "keyscan-keys/input0";
+ input_dev->dev.parent = &pdev->dev;
+ input_dev->open = keyscan_open;
+ input_dev->close = keyscan_close;
+
+ input_dev->id.bustype = BUS_HOST;
+
+ error = keypad_matrix_key_parse_dt(keypad_data);
+ if (error)
+ return error;
+
+ error = matrix_keypad_build_keymap(NULL, NULL,
+ keypad_data->n_rows,
+ keypad_data->n_cols,
+ NULL, input_dev);
+ if (error) {
+ dev_err(&pdev->dev, "failed to build keymap\n");
+ return error;
+ }
+
+ input_set_drvdata(input_dev, keypad_data);
+
+ keypad_data->input_dev = input_dev;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ keypad_data->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(keypad_data->base))
+ return PTR_ERR(keypad_data->base);
+
+ keypad_data->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(keypad_data->clk)) {
+ dev_err(&pdev->dev, "cannot get clock\n");
+ return PTR_ERR(keypad_data->clk);
+ }
+
+ error = clk_enable(keypad_data->clk);
+ if (error) {
+ dev_err(&pdev->dev, "failed to enable clock\n");
+ return error;
+ }
+
+ keyscan_stop(keypad_data);
+
+ keypad_data->irq = platform_get_irq(pdev, 0);
+ if (keypad_data->irq < 0) {
+ dev_err(&pdev->dev, "no IRQ specified\n");
+ return -EINVAL;
+ }
+
+ error = devm_request_irq(&pdev->dev, keypad_data->irq, keyscan_isr, 0,
+ pdev->name, keypad_data);
+ if (error) {
+ dev_err(&pdev->dev, "failed to request IRQ\n");
+ return error;
+ }
+
+ error = input_register_device(input_dev);
+ if (error) {
+ dev_err(&pdev->dev, "failed to register input device\n");
+ return error;
+ }
+
+ platform_set_drvdata(pdev, keypad_data);
+
+ device_set_wakeup_capable(&pdev->dev, 1);
+
+ return 0;
+}
+
+static int keyscan_suspend(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct st_keyscan *keypad = platform_get_drvdata(pdev);
+ struct input_dev *input = keypad->input_dev;
+
+ mutex_lock(&input->mutex);
+
+ if (device_may_wakeup(dev))
+ enable_irq_wake(keypad->irq);
+ else if (input->users)
+ keyscan_stop(keypad);
+
+ mutex_unlock(&input->mutex);
+ return 0;
+}
+
+static int keyscan_resume(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct st_keyscan *keypad = platform_get_drvdata(pdev);
+ struct input_dev *input = keypad->input_dev;
+ int retval = 0;
+
+ mutex_lock(&input->mutex);
+
+ if (device_may_wakeup(dev))
+ disable_irq_wake(keypad->irq);
+ else if (input->users)
+ retval = keyscan_start(keypad);
+
+ mutex_unlock(&input->mutex);
+ return retval;
+}
+
+static SIMPLE_DEV_PM_OPS(keyscan_dev_pm_ops, keyscan_suspend, keyscan_resume);
+
+static const struct of_device_id keyscan_of_match[] = {
+ { .compatible = "st,sti-keyscan" },
+ { },
+};
+MODULE_DEVICE_TABLE(of, keyscan_of_match);
+
+static struct platform_driver keyscan_device_driver = {
+ .probe = keyscan_probe,
+ .driver = {
+ .name = "st-keyscan",
+ .pm = &keyscan_dev_pm_ops,
+ .of_match_table = of_match_ptr(keyscan_of_match),
+ }
+};
+
+module_platform_driver(keyscan_device_driver);
+
+MODULE_AUTHOR("Stuart Menefy <stuart.menefy@st.com>");
+MODULE_DESCRIPTION("STMicroelectronics keyscan device driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/input/keyboard/tc3589x-keypad.c b/drivers/input/keyboard/tc3589x-keypad.c
index 74494a3..ad7abae 100644
--- a/drivers/input/keyboard/tc3589x-keypad.c
+++ b/drivers/input/keyboard/tc3589x-keypad.c
@@ -296,6 +296,65 @@
tc3589x_keypad_disable(keypad);
}
+#ifdef CONFIG_OF
+static const struct tc3589x_keypad_platform_data *
+tc3589x_keypad_of_probe(struct device *dev)
+{
+ struct device_node *np = dev->of_node;
+ struct tc3589x_keypad_platform_data *plat;
+ u32 cols, rows;
+ u32 debounce_ms;
+ int proplen;
+
+ if (!np)
+ return ERR_PTR(-ENODEV);
+
+ plat = devm_kzalloc(dev, sizeof(*plat), GFP_KERNEL);
+ if (!plat)
+ return ERR_PTR(-ENOMEM);
+
+ of_property_read_u32(np, "keypad,num-columns", &cols);
+ of_property_read_u32(np, "keypad,num-rows", &rows);
+ plat->kcol = (u8) cols;
+ plat->krow = (u8) rows;
+ if (!plat->krow || !plat->kcol ||
+ plat->krow > TC_KPD_ROWS || plat->kcol > TC_KPD_COLUMNS) {
+ dev_err(dev,
+ "keypad columns/rows not properly specified (%ux%u)\n",
+ plat->kcol, plat->krow);
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (!of_get_property(np, "linux,keymap", &proplen)) {
+ dev_err(dev, "property linux,keymap not found\n");
+ return ERR_PTR(-ENOENT);
+ }
+
+ plat->no_autorepeat = of_property_read_bool(np, "linux,no-autorepeat");
+ plat->enable_wakeup = of_property_read_bool(np, "linux,wakeup");
+
+ /* The custom delay format is ms/16 */
+ of_property_read_u32(np, "debounce-delay-ms", &debounce_ms);
+ if (debounce_ms)
+ plat->debounce_period = debounce_ms * 16;
+ else
+ plat->debounce_period = TC_KPD_DEBOUNCE_PERIOD;
+
+ plat->settle_time = TC_KPD_SETTLE_TIME;
+ /* FIXME: should be property of the IRQ resource? */
+ plat->irqtype = IRQF_TRIGGER_FALLING;
+
+ return plat;
+}
+#else
+static inline const struct tc3589x_keypad_platform_data *
+tc3589x_keypad_of_probe(struct device *dev)
+{
+ return ERR_PTR(-ENODEV);
+}
+#endif
+
+
static int tc3589x_keypad_probe(struct platform_device *pdev)
{
struct tc3589x *tc3589x = dev_get_drvdata(pdev->dev.parent);
@@ -306,8 +365,11 @@
plat = tc3589x->pdata->keypad;
if (!plat) {
- dev_err(&pdev->dev, "invalid keypad platform data\n");
- return -EINVAL;
+ plat = tc3589x_keypad_of_probe(&pdev->dev);
+ if (IS_ERR(plat)) {
+ dev_err(&pdev->dev, "invalid keypad platform data\n");
+ return PTR_ERR(plat);
+ }
}
irq = platform_get_irq(pdev, 0);
diff --git a/drivers/input/misc/88pm860x_onkey.c b/drivers/input/misc/88pm860x_onkey.c
index abd8453..220ce0f 100644
--- a/drivers/input/misc/88pm860x_onkey.c
+++ b/drivers/input/misc/88pm860x_onkey.c
@@ -26,6 +26,7 @@
#include <linux/interrupt.h>
#include <linux/mfd/88pm860x.h>
#include <linux/slab.h>
+#include <linux/device.h>
#define PM8607_WAKEUP 0x0b
@@ -68,7 +69,8 @@
return -EINVAL;
}
- info = kzalloc(sizeof(struct pm860x_onkey_info), GFP_KERNEL);
+ info = devm_kzalloc(&pdev->dev, sizeof(struct pm860x_onkey_info),
+ GFP_KERNEL);
if (!info)
return -ENOMEM;
info->chip = chip;
@@ -76,11 +78,10 @@
info->dev = &pdev->dev;
info->irq = irq;
- info->idev = input_allocate_device();
+ info->idev = devm_input_allocate_device(&pdev->dev);
if (!info->idev) {
dev_err(chip->dev, "Failed to allocate input dev\n");
- ret = -ENOMEM;
- goto out;
+ return -ENOMEM;
}
info->idev->name = "88pm860x_on";
@@ -93,42 +94,22 @@
ret = input_register_device(info->idev);
if (ret) {
dev_err(chip->dev, "Can't register input device: %d\n", ret);
- goto out_reg;
+ return ret;
}
- ret = request_threaded_irq(info->irq, NULL, pm860x_onkey_handler,
- IRQF_ONESHOT, "onkey", info);
+ ret = devm_request_threaded_irq(&pdev->dev, info->irq, NULL,
+ pm860x_onkey_handler, IRQF_ONESHOT,
+ "onkey", info);
if (ret < 0) {
dev_err(chip->dev, "Failed to request IRQ: #%d: %d\n",
info->irq, ret);
- goto out_irq;
+ return ret;
}
platform_set_drvdata(pdev, info);
device_init_wakeup(&pdev->dev, 1);
return 0;
-
-out_irq:
- input_unregister_device(info->idev);
- kfree(info);
- return ret;
-
-out_reg:
- input_free_device(info->idev);
-out:
- kfree(info);
- return ret;
-}
-
-static int pm860x_onkey_remove(struct platform_device *pdev)
-{
- struct pm860x_onkey_info *info = platform_get_drvdata(pdev);
-
- free_irq(info->irq, info);
- input_unregister_device(info->idev);
- kfree(info);
- return 0;
}
#ifdef CONFIG_PM_SLEEP
@@ -161,7 +142,6 @@
.pm = &pm860x_onkey_pm_ops,
},
.probe = pm860x_onkey_probe,
- .remove = pm860x_onkey_remove,
};
module_platform_driver(pm860x_onkey_driver);
diff --git a/drivers/input/misc/Kconfig b/drivers/input/misc/Kconfig
index 5928ea7..2ff4425 100644
--- a/drivers/input/misc/Kconfig
+++ b/drivers/input/misc/Kconfig
@@ -224,7 +224,7 @@
config INPUT_GPIO_BEEPER
tristate "Generic GPIO Beeper support"
- depends on OF_GPIO
+ depends on GPIOLIB
help
Say Y here if you have a beeper connected to a GPIO pin.
diff --git a/drivers/input/misc/ab8500-ponkey.c b/drivers/input/misc/ab8500-ponkey.c
index f2fbdd8..95ef7dd 100644
--- a/drivers/input/misc/ab8500-ponkey.c
+++ b/drivers/input/misc/ab8500-ponkey.c
@@ -7,6 +7,7 @@
* AB8500 Power-On Key handler
*/
+#include <linux/device.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
@@ -65,12 +66,14 @@
return irq_dbr;
}
- ponkey = kzalloc(sizeof(struct ab8500_ponkey), GFP_KERNEL);
- input = input_allocate_device();
- if (!ponkey || !input) {
- error = -ENOMEM;
- goto err_free_mem;
- }
+ ponkey = devm_kzalloc(&pdev->dev, sizeof(struct ab8500_ponkey),
+ GFP_KERNEL);
+ if (!ponkey)
+ return -ENOMEM;
+
+ input = devm_input_allocate_device(&pdev->dev);
+ if (!input)
+ return -ENOMEM;
ponkey->idev = input;
ponkey->ab8500 = ab8500;
@@ -82,52 +85,32 @@
input_set_capability(input, EV_KEY, KEY_POWER);
- error = request_any_context_irq(ponkey->irq_dbf, ab8500_ponkey_handler,
- 0, "ab8500-ponkey-dbf", ponkey);
+ error = devm_request_any_context_irq(&pdev->dev, ponkey->irq_dbf,
+ ab8500_ponkey_handler, 0,
+ "ab8500-ponkey-dbf", ponkey);
if (error < 0) {
dev_err(ab8500->dev, "Failed to request dbf IRQ#%d: %d\n",
ponkey->irq_dbf, error);
- goto err_free_mem;
+ return error;
}
- error = request_any_context_irq(ponkey->irq_dbr, ab8500_ponkey_handler,
- 0, "ab8500-ponkey-dbr", ponkey);
+ error = devm_request_any_context_irq(&pdev->dev, ponkey->irq_dbr,
+ ab8500_ponkey_handler, 0,
+ "ab8500-ponkey-dbr", ponkey);
if (error < 0) {
dev_err(ab8500->dev, "Failed to request dbr IRQ#%d: %d\n",
ponkey->irq_dbr, error);
- goto err_free_dbf_irq;
+ return error;
}
error = input_register_device(ponkey->idev);
if (error) {
dev_err(ab8500->dev, "Can't register input device: %d\n", error);
- goto err_free_dbr_irq;
+ return error;
}
platform_set_drvdata(pdev, ponkey);
return 0;
-
-err_free_dbr_irq:
- free_irq(ponkey->irq_dbr, ponkey);
-err_free_dbf_irq:
- free_irq(ponkey->irq_dbf, ponkey);
-err_free_mem:
- input_free_device(input);
- kfree(ponkey);
-
- return error;
-}
-
-static int ab8500_ponkey_remove(struct platform_device *pdev)
-{
- struct ab8500_ponkey *ponkey = platform_get_drvdata(pdev);
-
- free_irq(ponkey->irq_dbf, ponkey);
- free_irq(ponkey->irq_dbr, ponkey);
- input_unregister_device(ponkey->idev);
- kfree(ponkey);
-
- return 0;
}
#ifdef CONFIG_OF
@@ -144,7 +127,6 @@
.of_match_table = of_match_ptr(ab8500_ponkey_match),
},
.probe = ab8500_ponkey_probe,
- .remove = ab8500_ponkey_remove,
};
module_platform_driver(ab8500_ponkey_driver);
diff --git a/drivers/input/misc/gpio-beeper.c b/drivers/input/misc/gpio-beeper.c
index b757435..8886af6 100644
--- a/drivers/input/misc/gpio-beeper.c
+++ b/drivers/input/misc/gpio-beeper.c
@@ -1,7 +1,7 @@
/*
* Generic GPIO beeper driver
*
- * Copyright (C) 2013 Alexander Shiyan <shc_work@mail.ru>
+ * Copyright (C) 2013-2014 Alexander Shiyan <shc_work@mail.ru>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -11,7 +11,8 @@
#include <linux/input.h>
#include <linux/module.h>
-#include <linux/of_gpio.h>
+#include <linux/gpio/consumer.h>
+#include <linux/of.h>
#include <linux/workqueue.h>
#include <linux/platform_device.h>
@@ -19,14 +20,13 @@
struct gpio_beeper {
struct work_struct work;
- int gpio;
- bool active_low;
+ struct gpio_desc *desc;
bool beeping;
};
static void gpio_beeper_toggle(struct gpio_beeper *beep, bool on)
{
- gpio_set_value_cansleep(beep->gpio, on ^ beep->active_low);
+ gpiod_set_value_cansleep(beep->desc, on);
}
static void gpio_beeper_work(struct work_struct *work)
@@ -65,18 +65,16 @@
static int gpio_beeper_probe(struct platform_device *pdev)
{
struct gpio_beeper *beep;
- enum of_gpio_flags flags;
struct input_dev *input;
- unsigned long gflags;
int err;
beep = devm_kzalloc(&pdev->dev, sizeof(*beep), GFP_KERNEL);
if (!beep)
return -ENOMEM;
- beep->gpio = of_get_gpio_flags(pdev->dev.of_node, 0, &flags);
- if (!gpio_is_valid(beep->gpio))
- return beep->gpio;
+ beep->desc = devm_gpiod_get(&pdev->dev, NULL);
+ if (IS_ERR(beep->desc))
+ return PTR_ERR(beep->desc);
input = devm_input_allocate_device(&pdev->dev);
if (!input)
@@ -94,10 +92,7 @@
input_set_capability(input, EV_SND, SND_BELL);
- beep->active_low = flags & OF_GPIO_ACTIVE_LOW;
- gflags = beep->active_low ? GPIOF_OUT_INIT_HIGH : GPIOF_OUT_INIT_LOW;
-
- err = devm_gpio_request_one(&pdev->dev, beep->gpio, gflags, pdev->name);
+ err = gpiod_direction_output(beep->desc, 0);
if (err)
return err;
@@ -106,17 +101,19 @@
return input_register_device(input);
}
-static struct of_device_id gpio_beeper_of_match[] = {
+#ifdef CONFIG_OF
+static const struct of_device_id gpio_beeper_of_match[] = {
{ .compatible = BEEPER_MODNAME, },
{ }
};
MODULE_DEVICE_TABLE(of, gpio_beeper_of_match);
+#endif
static struct platform_driver gpio_beeper_platform_driver = {
.driver = {
.name = BEEPER_MODNAME,
.owner = THIS_MODULE,
- .of_match_table = gpio_beeper_of_match,
+ .of_match_table = of_match_ptr(gpio_beeper_of_match),
},
.probe = gpio_beeper_probe,
};
diff --git a/drivers/input/misc/ims-pcu.c b/drivers/input/misc/ims-pcu.c
index 5a73639..719410f 100644
--- a/drivers/input/misc/ims-pcu.c
+++ b/drivers/input/misc/ims-pcu.c
@@ -1566,6 +1566,7 @@
if (!pcu->urb_ctrl_buf) {
dev_err(pcu->dev,
"Failed to allocate memory for read buffer\n");
+ error = -ENOMEM;
goto err_free_urb_out_buf;
}
diff --git a/drivers/input/misc/max8925_onkey.c b/drivers/input/misc/max8925_onkey.c
index eef41cf..3809618 100644
--- a/drivers/input/misc/max8925_onkey.c
+++ b/drivers/input/misc/max8925_onkey.c
@@ -26,6 +26,7 @@
#include <linux/interrupt.h>
#include <linux/mfd/max8925.h>
#include <linux/slab.h>
+#include <linux/device.h>
#define SW_INPUT (1 << 7) /* 0/1 -- up/down */
#define HARDRESET_EN (1 << 7)
@@ -81,12 +82,14 @@
return -EINVAL;
}
- info = kzalloc(sizeof(struct max8925_onkey_info), GFP_KERNEL);
- input = input_allocate_device();
- if (!info || !input) {
- error = -ENOMEM;
- goto err_free_mem;
- }
+ info = devm_kzalloc(&pdev->dev, sizeof(struct max8925_onkey_info),
+ GFP_KERNEL);
+ if (!info)
+ return -ENOMEM;
+
+ input = devm_input_allocate_device(&pdev->dev);
+ if (!input)
+ return -ENOMEM;
info->idev = input;
info->i2c = chip->i2c;
@@ -100,55 +103,34 @@
input->dev.parent = &pdev->dev;
input_set_capability(input, EV_KEY, KEY_POWER);
- error = request_threaded_irq(irq[0], NULL, max8925_onkey_handler,
- IRQF_ONESHOT, "onkey-down", info);
+ error = devm_request_threaded_irq(&pdev->dev, irq[0], NULL,
+ max8925_onkey_handler, IRQF_ONESHOT,
+ "onkey-down", info);
if (error < 0) {
dev_err(chip->dev, "Failed to request IRQ: #%d: %d\n",
irq[0], error);
- goto err_free_mem;
+ return error;
}
- error = request_threaded_irq(irq[1], NULL, max8925_onkey_handler,
- IRQF_ONESHOT, "onkey-up", info);
+ error = devm_request_threaded_irq(&pdev->dev, irq[1], NULL,
+ max8925_onkey_handler, IRQF_ONESHOT,
+ "onkey-up", info);
if (error < 0) {
dev_err(chip->dev, "Failed to request IRQ: #%d: %d\n",
irq[1], error);
- goto err_free_irq0;
+ return error;
}
error = input_register_device(info->idev);
if (error) {
dev_err(chip->dev, "Can't register input device: %d\n", error);
- goto err_free_irq1;
+ return error;
}
platform_set_drvdata(pdev, info);
device_init_wakeup(&pdev->dev, 1);
return 0;
-
-err_free_irq1:
- free_irq(irq[1], info);
-err_free_irq0:
- free_irq(irq[0], info);
-err_free_mem:
- input_free_device(input);
- kfree(info);
-
- return error;
-}
-
-static int max8925_onkey_remove(struct platform_device *pdev)
-{
- struct max8925_onkey_info *info = platform_get_drvdata(pdev);
- struct max8925_chip *chip = dev_get_drvdata(pdev->dev.parent);
-
- free_irq(info->irq[0] + chip->irq_base, info);
- free_irq(info->irq[1] + chip->irq_base, info);
- input_unregister_device(info->idev);
- kfree(info);
-
- return 0;
}
#ifdef CONFIG_PM_SLEEP
@@ -190,7 +172,6 @@
.pm = &max8925_onkey_pm_ops,
},
.probe = max8925_onkey_probe,
- .remove = max8925_onkey_remove,
};
module_platform_driver(max8925_onkey_driver);
diff --git a/drivers/input/misc/max8997_haptic.c b/drivers/input/misc/max8997_haptic.c
index 1fea548..a363ebbd 100644
--- a/drivers/input/misc/max8997_haptic.c
+++ b/drivers/input/misc/max8997_haptic.c
@@ -181,11 +181,21 @@
}
if (!chip->enabled) {
- chip->enabled = true;
- regulator_enable(chip->regulator);
+ error = regulator_enable(chip->regulator);
+ if (error) {
+ dev_err(chip->dev, "Failed to enable regulator\n");
+ goto out;
+ }
max8997_haptic_configure(chip);
- if (chip->mode == MAX8997_EXTERNAL_MODE)
- pwm_enable(chip->pwm);
+ if (chip->mode == MAX8997_EXTERNAL_MODE) {
+ error = pwm_enable(chip->pwm);
+ if (error) {
+ dev_err(chip->dev, "Failed to enable PWM\n");
+ regulator_disable(chip->regulator);
+ goto out;
+ }
+ }
+ chip->enabled = true;
}
out:
diff --git a/drivers/input/misc/pmic8xxx-pwrkey.c b/drivers/input/misc/pmic8xxx-pwrkey.c
index 1cb8fda..c91e3d3 100644
--- a/drivers/input/misc/pmic8xxx-pwrkey.c
+++ b/drivers/input/misc/pmic8xxx-pwrkey.c
@@ -92,15 +92,15 @@
bool pull_up;
if (of_property_read_u32(pdev->dev.of_node, "debounce", &kpd_delay))
- kpd_delay = 0;
+ kpd_delay = 15625;
- pull_up = of_property_read_bool(pdev->dev.of_node, "pull-up");
-
- if (kpd_delay > 62500) {
+ if (kpd_delay > 62500 || kpd_delay == 0) {
dev_err(&pdev->dev, "invalid power key trigger delay\n");
return -EINVAL;
}
+ pull_up = of_property_read_bool(pdev->dev.of_node, "pull-up");
+
regmap = dev_get_regmap(pdev->dev.parent, NULL);
if (!regmap) {
dev_err(&pdev->dev, "failed to locate regmap for the device\n");
diff --git a/drivers/input/misc/rotary_encoder.c b/drivers/input/misc/rotary_encoder.c
index 99b9e42..93558a1c 100644
--- a/drivers/input/misc/rotary_encoder.c
+++ b/drivers/input/misc/rotary_encoder.c
@@ -143,7 +143,7 @@
}
#ifdef CONFIG_OF
-static struct of_device_id rotary_encoder_of_match[] = {
+static const struct of_device_id rotary_encoder_of_match[] = {
{ .compatible = "rotary-encoder", },
{ },
};
diff --git a/drivers/input/misc/soc_button_array.c b/drivers/input/misc/soc_button_array.c
index 20c80f5..5a6334b 100644
--- a/drivers/input/misc/soc_button_array.c
+++ b/drivers/input/misc/soc_button_array.c
@@ -17,7 +17,6 @@
#include <linux/acpi.h>
#include <linux/gpio/consumer.h>
#include <linux/gpio_keys.h>
-#include <linux/input.h>
#include <linux/platform_device.h>
#include <linux/pnp.h>
diff --git a/drivers/input/misc/twl6040-vibra.c b/drivers/input/misc/twl6040-vibra.c
index 77dc23b..6d26eec 100644
--- a/drivers/input/misc/twl6040-vibra.c
+++ b/drivers/input/misc/twl6040-vibra.c
@@ -262,7 +262,7 @@
struct vibra_info *info;
int vddvibl_uV = 0;
int vddvibr_uV = 0;
- int ret;
+ int error;
twl6040_core_node = of_find_node_by_name(twl6040_core_dev->of_node,
"vibra");
@@ -309,12 +309,12 @@
mutex_init(&info->mutex);
- ret = devm_request_threaded_irq(&pdev->dev, info->irq, NULL,
- twl6040_vib_irq_handler, 0,
- "twl6040_irq_vib", info);
- if (ret) {
- dev_err(info->dev, "VIB IRQ request failed: %d\n", ret);
- return ret;
+ error = devm_request_threaded_irq(&pdev->dev, info->irq, NULL,
+ twl6040_vib_irq_handler, 0,
+ "twl6040_irq_vib", info);
+ if (error) {
+ dev_err(info->dev, "VIB IRQ request failed: %d\n", error);
+ return error;
}
info->supplies[0].supply = "vddvibl";
@@ -323,40 +323,40 @@
* When booted with Device tree the regulators are attached to the
* parent device (twl6040 MFD core)
*/
- ret = regulator_bulk_get(twl6040_core_dev, ARRAY_SIZE(info->supplies),
- info->supplies);
- if (ret) {
- dev_err(info->dev, "couldn't get regulators %d\n", ret);
- return ret;
+ error = devm_regulator_bulk_get(twl6040_core_dev,
+ ARRAY_SIZE(info->supplies),
+ info->supplies);
+ if (error) {
+ dev_err(info->dev, "couldn't get regulators %d\n", error);
+ return error;
}
if (vddvibl_uV) {
- ret = regulator_set_voltage(info->supplies[0].consumer,
- vddvibl_uV, vddvibl_uV);
- if (ret) {
+ error = regulator_set_voltage(info->supplies[0].consumer,
+ vddvibl_uV, vddvibl_uV);
+ if (error) {
dev_err(info->dev, "failed to set VDDVIBL volt %d\n",
- ret);
- goto err_regulator;
+ error);
+ return error;
}
}
if (vddvibr_uV) {
- ret = regulator_set_voltage(info->supplies[1].consumer,
- vddvibr_uV, vddvibr_uV);
- if (ret) {
+ error = regulator_set_voltage(info->supplies[1].consumer,
+ vddvibr_uV, vddvibr_uV);
+ if (error) {
dev_err(info->dev, "failed to set VDDVIBR volt %d\n",
- ret);
- goto err_regulator;
+ error);
+ return error;
}
}
INIT_WORK(&info->play_work, vibra_play_work);
- info->input_dev = input_allocate_device();
- if (info->input_dev == NULL) {
+ info->input_dev = devm_input_allocate_device(&pdev->dev);
+ if (!info->input_dev) {
dev_err(info->dev, "couldn't allocate input device\n");
- ret = -ENOMEM;
- goto err_regulator;
+ return -ENOMEM;
}
input_set_drvdata(info->input_dev, info);
@@ -367,44 +367,25 @@
info->input_dev->close = twl6040_vibra_close;
__set_bit(FF_RUMBLE, info->input_dev->ffbit);
- ret = input_ff_create_memless(info->input_dev, NULL, vibra_play);
- if (ret < 0) {
+ error = input_ff_create_memless(info->input_dev, NULL, vibra_play);
+ if (error) {
dev_err(info->dev, "couldn't register vibrator to FF\n");
- goto err_ialloc;
+ return error;
}
- ret = input_register_device(info->input_dev);
- if (ret < 0) {
+ error = input_register_device(info->input_dev);
+ if (error) {
dev_err(info->dev, "couldn't register input device\n");
- goto err_iff;
+ return error;
}
platform_set_drvdata(pdev, info);
return 0;
-
-err_iff:
- input_ff_destroy(info->input_dev);
-err_ialloc:
- input_free_device(info->input_dev);
-err_regulator:
- regulator_bulk_free(ARRAY_SIZE(info->supplies), info->supplies);
- return ret;
-}
-
-static int twl6040_vibra_remove(struct platform_device *pdev)
-{
- struct vibra_info *info = platform_get_drvdata(pdev);
-
- input_unregister_device(info->input_dev);
- regulator_bulk_free(ARRAY_SIZE(info->supplies), info->supplies);
-
- return 0;
}
static struct platform_driver twl6040_vibra_driver = {
.probe = twl6040_vibra_probe,
- .remove = twl6040_vibra_remove,
.driver = {
.name = "twl6040-vibra",
.owner = THIS_MODULE,
diff --git a/drivers/input/mouse/Kconfig b/drivers/input/mouse/Kconfig
index 6b8441f..366fc7a 100644
--- a/drivers/input/mouse/Kconfig
+++ b/drivers/input/mouse/Kconfig
@@ -53,7 +53,7 @@
default y
depends on MOUSE_PS2
help
- Say Y here if you have a Logictech PS/2++ mouse connected to
+ Say Y here if you have a Logitech PS/2++ mouse connected to
your system.
If unsure, say Y.
diff --git a/drivers/input/mouse/elantech.c b/drivers/input/mouse/elantech.c
index b96e978..ee2a04d 100644
--- a/drivers/input/mouse/elantech.c
+++ b/drivers/input/mouse/elantech.c
@@ -473,8 +473,15 @@
input_report_key(dev, BTN_TOOL_FINGER, fingers == 1);
input_report_key(dev, BTN_TOOL_DOUBLETAP, fingers == 2);
input_report_key(dev, BTN_TOOL_TRIPLETAP, fingers == 3);
- input_report_key(dev, BTN_LEFT, packet[0] & 0x01);
- input_report_key(dev, BTN_RIGHT, packet[0] & 0x02);
+
+ /* For clickpads map both buttons to BTN_LEFT */
+ if (etd->fw_version & 0x001000) {
+ input_report_key(dev, BTN_LEFT, packet[0] & 0x03);
+ } else {
+ input_report_key(dev, BTN_LEFT, packet[0] & 0x01);
+ input_report_key(dev, BTN_RIGHT, packet[0] & 0x02);
+ }
+
input_report_abs(dev, ABS_PRESSURE, pres);
input_report_abs(dev, ABS_TOOL_WIDTH, width);
@@ -484,10 +491,17 @@
static void elantech_input_sync_v4(struct psmouse *psmouse)
{
struct input_dev *dev = psmouse->dev;
+ struct elantech_data *etd = psmouse->private;
unsigned char *packet = psmouse->packet;
- input_report_key(dev, BTN_LEFT, packet[0] & 0x01);
- input_report_key(dev, BTN_RIGHT, packet[0] & 0x02);
+ /* For clickpads map both buttons to BTN_LEFT */
+ if (etd->fw_version & 0x001000) {
+ input_report_key(dev, BTN_LEFT, packet[0] & 0x03);
+ } else {
+ input_report_key(dev, BTN_LEFT, packet[0] & 0x01);
+ input_report_key(dev, BTN_RIGHT, packet[0] & 0x02);
+ }
+
input_mt_report_pointer_emulation(dev, true);
input_sync(dev);
}
@@ -835,7 +849,7 @@
if (etd->set_hw_resolution)
etd->reg_10 = 0x0b;
else
- etd->reg_10 = 0x03;
+ etd->reg_10 = 0x01;
if (elantech_write_reg(psmouse, 0x10, etd->reg_10))
rc = -1;
@@ -1336,7 +1350,8 @@
}
/*
- * Some hw_version 3 models go into error state when we try to set bit 3 of r10
+ * Some hw_version 3 models go into error state when we try to set
+ * bit 3 and/or bit 1 of r10.
*/
static const struct dmi_system_id no_hw_res_dmi_table[] = {
#if defined(CONFIG_DMI) && defined(CONFIG_X86)
diff --git a/drivers/input/mouse/synaptics.c b/drivers/input/mouse/synaptics.c
index c5ec703c..ec772d9 100644
--- a/drivers/input/mouse/synaptics.c
+++ b/drivers/input/mouse/synaptics.c
@@ -347,15 +347,6 @@
unsigned char resp[3];
int i;
- for (i = 0; min_max_pnpid_table[i].pnp_ids; i++)
- if (matches_pnp_id(psmouse, min_max_pnpid_table[i].pnp_ids)) {
- priv->x_min = min_max_pnpid_table[i].x_min;
- priv->x_max = min_max_pnpid_table[i].x_max;
- priv->y_min = min_max_pnpid_table[i].y_min;
- priv->y_max = min_max_pnpid_table[i].y_max;
- return 0;
- }
-
if (SYN_ID_MAJOR(priv->identity) < 4)
return 0;
@@ -366,6 +357,16 @@
}
}
+ for (i = 0; min_max_pnpid_table[i].pnp_ids; i++) {
+ if (matches_pnp_id(psmouse, min_max_pnpid_table[i].pnp_ids)) {
+ priv->x_min = min_max_pnpid_table[i].x_min;
+ priv->x_max = min_max_pnpid_table[i].x_max;
+ priv->y_min = min_max_pnpid_table[i].y_min;
+ priv->y_max = min_max_pnpid_table[i].y_max;
+ return 0;
+ }
+ }
+
if (SYN_EXT_CAP_REQUESTS(priv->capabilities) >= 5 &&
SYN_CAP_MAX_DIMENSIONS(priv->ext_cap_0c)) {
if (synaptics_send_cmd(psmouse, SYN_QUE_EXT_MAX_COORDS, resp)) {
diff --git a/drivers/input/serio/apbps2.c b/drivers/input/serio/apbps2.c
index 17e01a8..98be824 100644
--- a/drivers/input/serio/apbps2.c
+++ b/drivers/input/serio/apbps2.c
@@ -203,7 +203,7 @@
return 0;
}
-static struct of_device_id apbps2_of_match[] = {
+static const struct of_device_id apbps2_of_match[] = {
{ .name = "GAISLER_APBPS2", },
{ .name = "01_060", },
{}
diff --git a/drivers/input/serio/olpc_apsp.c b/drivers/input/serio/olpc_apsp.c
index 5d2fe7e..d906f3e 100644
--- a/drivers/input/serio/olpc_apsp.c
+++ b/drivers/input/serio/olpc_apsp.c
@@ -262,7 +262,7 @@
return 0;
}
-static struct of_device_id olpc_apsp_dt_ids[] = {
+static const struct of_device_id olpc_apsp_dt_ids[] = {
{ .compatible = "olpc,ap-sp", },
{}
};
diff --git a/drivers/input/tablet/wacom_sys.c b/drivers/input/tablet/wacom_sys.c
index 611fc39..2c613cd 100644
--- a/drivers/input/tablet/wacom_sys.c
+++ b/drivers/input/tablet/wacom_sys.c
@@ -349,6 +349,7 @@
break;
case MTTPC:
+ case MTTPC_B:
features->pktlen = WACOM_PKGLEN_MTTPC;
break;
@@ -380,6 +381,16 @@
i += 12;
break;
+ case MTTPC_B:
+ features->x_max =
+ get_unaligned_le16(&report[i + 3]);
+ features->x_phy =
+ get_unaligned_le16(&report[i + 6]);
+ features->unit = report[i - 5];
+ features->unitExpo = report[i - 3];
+ i += 9;
+ break;
+
default:
features->x_max =
get_unaligned_le16(&report[i + 3]);
@@ -430,6 +441,14 @@
i += 12;
break;
+ case MTTPC_B:
+ features->y_max =
+ get_unaligned_le16(&report[i + 3]);
+ features->y_phy =
+ get_unaligned_le16(&report[i + 6]);
+ i += 9;
+ break;
+
default:
features->y_max =
features->x_max;
diff --git a/drivers/input/tablet/wacom_wac.c b/drivers/input/tablet/wacom_wac.c
index 4822c57..977d05c 100644
--- a/drivers/input/tablet/wacom_wac.c
+++ b/drivers/input/tablet/wacom_wac.c
@@ -484,6 +484,8 @@
input_report_key(input, BTN_TOUCH, 0);
input_report_abs(input, ABS_PRESSURE, 0);
input_report_abs(input, ABS_DISTANCE, wacom->features.distance_max);
+ if (features->quirks & WACOM_QUIRK_MULTI_INPUT)
+ wacom->shared->stylus_in_proximity = true;
}
/* Exit report */
@@ -928,12 +930,12 @@
input_mt_report_slot_state(input, MT_TOOL_FINGER, touch);
if (touch) {
- int t_x = le16_to_cpup((__le16 *)&data[offset + 2]);
- int c_x = le16_to_cpup((__le16 *)&data[offset + 4]);
- int t_y = le16_to_cpup((__le16 *)&data[offset + 6]);
- int c_y = le16_to_cpup((__le16 *)&data[offset + 8]);
- int w = le16_to_cpup((__le16 *)&data[offset + 10]);
- int h = le16_to_cpup((__le16 *)&data[offset + 12]);
+ int t_x = get_unaligned_le16(&data[offset + 2]);
+ int c_x = get_unaligned_le16(&data[offset + 4]);
+ int t_y = get_unaligned_le16(&data[offset + 6]);
+ int c_y = get_unaligned_le16(&data[offset + 8]);
+ int w = get_unaligned_le16(&data[offset + 10]);
+ int h = get_unaligned_le16(&data[offset + 12]);
input_report_abs(input, ABS_MT_POSITION_X, t_x);
input_report_abs(input, ABS_MT_POSITION_Y, t_y);
@@ -962,7 +964,7 @@
int x_offset = 0;
/* MTTPC does not support Height and Width */
- if (wacom->features.type == MTTPC)
+ if (wacom->features.type == MTTPC || wacom->features.type == MTTPC_B)
x_offset = -4;
/*
@@ -978,7 +980,7 @@
for (i = 0; i < contacts_to_send; i++) {
int offset = (WACOM_BYTES_PER_MT_PACKET + x_offset) * i + 3;
bool touch = data[offset] & 0x1;
- int id = le16_to_cpup((__le16 *)&data[offset + 1]);
+ int id = get_unaligned_le16(&data[offset + 1]);
int slot = input_mt_get_slot_by_key(input, id);
if (slot < 0)
@@ -987,8 +989,8 @@
input_mt_slot(input, slot);
input_mt_report_slot_state(input, MT_TOOL_FINGER, touch);
if (touch) {
- int x = le16_to_cpup((__le16 *)&data[offset + x_offset + 7]);
- int y = le16_to_cpup((__le16 *)&data[offset + x_offset + 9]);
+ int x = get_unaligned_le16(&data[offset + x_offset + 7]);
+ int y = get_unaligned_le16(&data[offset + x_offset + 9]);
input_report_abs(input, ABS_MT_POSITION_X, x);
input_report_abs(input, ABS_MT_POSITION_Y, y);
}
@@ -1047,6 +1049,10 @@
prox = data[0] & 0x01;
x = get_unaligned_le16(&data[1]);
y = get_unaligned_le16(&data[3]);
+ } else if (len == WACOM_PKGLEN_TPC1FG_B) {
+ prox = data[2] & 0x01;
+ x = get_unaligned_le16(&data[3]);
+ y = get_unaligned_le16(&data[5]);
} else {
prox = data[1] & 0x01;
x = le16_to_cpup((__le16 *)&data[2]);
@@ -1110,6 +1116,9 @@
case WACOM_PKGLEN_TPC2FG:
return wacom_tpc_mt_touch(wacom);
+ case WACOM_PKGLEN_PENABLED:
+ return wacom_tpc_pen(wacom);
+
default:
switch (data[0]) {
case WACOM_REPORT_TPC1FG:
@@ -1119,6 +1128,7 @@
return wacom_tpc_single_touch(wacom, len);
case WACOM_REPORT_TPCMT:
+ case WACOM_REPORT_TPCMT2:
return wacom_mt_touch(wacom);
case WACOM_REPORT_PENABLED:
@@ -1461,6 +1471,7 @@
case TABLETPC2FG:
case MTSCREEN:
case MTTPC:
+ case MTTPC_B:
sync = wacom_tpc_irq(wacom_wac, len);
break;
@@ -1565,10 +1576,10 @@
struct wacom_features *features = &wacom_wac->features;
if (features->device_type == BTN_TOOL_PEN) {
- input_set_abs_params(input_dev, ABS_X, 0, features->x_max,
- features->x_fuzz, 0);
- input_set_abs_params(input_dev, ABS_Y, 0, features->y_max,
- features->y_fuzz, 0);
+ input_set_abs_params(input_dev, ABS_X, features->x_min,
+ features->x_max, features->x_fuzz, 0);
+ input_set_abs_params(input_dev, ABS_Y, features->y_min,
+ features->y_max, features->y_fuzz, 0);
input_set_abs_params(input_dev, ABS_PRESSURE, 0,
features->pressure_max, features->pressure_fuzz, 0);
@@ -1802,6 +1813,7 @@
case MTSCREEN:
case MTTPC:
+ case MTTPC_B:
case TABLETPC2FG:
if (features->device_type == BTN_TOOL_FINGER) {
unsigned int flags = INPUT_MT_DIRECT;
@@ -2123,11 +2135,11 @@
63, INTUOSPL, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES,
.touch_max = 16 };
static const struct wacom_features wacom_features_0xF4 =
- { "Wacom Cintiq 24HD", WACOM_PKGLEN_INTUOS, 104480, 65600, 2047,
- 63, WACOM_24HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES };
+ { "Wacom Cintiq 24HD", WACOM_PKGLEN_INTUOS, 104280, 65400, 2047,
+ 63, WACOM_24HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES, 200, 200 };
static const struct wacom_features wacom_features_0xF8 =
- { "Wacom Cintiq 24HD touch", WACOM_PKGLEN_INTUOS, 104480, 65600, 2047, /* Pen */
- 63, WACOM_24HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES,
+ { "Wacom Cintiq 24HD touch", WACOM_PKGLEN_INTUOS, 104280, 65400, 2047, /* Pen */
+ 63, WACOM_24HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES, 200, 200,
.oVid = USB_VENDOR_ID_WACOM, .oPid = 0xf6 };
static const struct wacom_features wacom_features_0xF6 =
{ "Wacom Cintiq 24HD touch", .type = WACOM_24HDT, /* Touch */
@@ -2142,8 +2154,8 @@
{ "Wacom Cintiq 12WX", WACOM_PKGLEN_INTUOS, 53020, 33440, 1023,
63, WACOM_BEE, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES };
static const struct wacom_features wacom_features_0x304 =
- { "Wacom Cintiq 13HD", WACOM_PKGLEN_INTUOS, 59552, 33848, 1023,
- 63, WACOM_13HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES };
+ { "Wacom Cintiq 13HD", WACOM_PKGLEN_INTUOS, 59352, 33648, 1023,
+ 63, WACOM_13HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES, 200, 200 };
static const struct wacom_features wacom_features_0xC7 =
{ "Wacom DTU1931", WACOM_PKGLEN_GRAPHIRE, 37832, 30305, 511,
0, PL, WACOM_INTUOS_RES, WACOM_INTUOS_RES };
@@ -2157,24 +2169,24 @@
{ "Wacom DTU1031", WACOM_PKGLEN_DTUS, 22096, 13960, 511,
0, DTUS, WACOM_INTUOS_RES, WACOM_INTUOS_RES };
static const struct wacom_features wacom_features_0x57 =
- { "Wacom DTK2241", WACOM_PKGLEN_INTUOS, 95840, 54260, 2047,
- 63, DTK, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES};
+ { "Wacom DTK2241", WACOM_PKGLEN_INTUOS, 95640, 54060, 2047,
+ 63, DTK, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES, 200, 200 };
static const struct wacom_features wacom_features_0x59 = /* Pen */
- { "Wacom DTH2242", WACOM_PKGLEN_INTUOS, 95840, 54260, 2047,
- 63, DTK, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES,
+ { "Wacom DTH2242", WACOM_PKGLEN_INTUOS, 95640, 54060, 2047,
+ 63, DTK, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES, 200, 200,
.oVid = USB_VENDOR_ID_WACOM, .oPid = 0x5D };
static const struct wacom_features wacom_features_0x5D = /* Touch */
{ "Wacom DTH2242", .type = WACOM_24HDT,
.oVid = USB_VENDOR_ID_WACOM, .oPid = 0x59, .touch_max = 10 };
static const struct wacom_features wacom_features_0xCC =
- { "Wacom Cintiq 21UX2", WACOM_PKGLEN_INTUOS, 87200, 65600, 2047,
- 63, WACOM_21UX2, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES };
+ { "Wacom Cintiq 21UX2", WACOM_PKGLEN_INTUOS, 87000, 65400, 2047,
+ 63, WACOM_21UX2, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES, 200, 200 };
static const struct wacom_features wacom_features_0xFA =
- { "Wacom Cintiq 22HD", WACOM_PKGLEN_INTUOS, 95840, 54260, 2047,
- 63, WACOM_22HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES };
+ { "Wacom Cintiq 22HD", WACOM_PKGLEN_INTUOS, 95640, 54060, 2047,
+ 63, WACOM_22HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES, 200, 200 };
static const struct wacom_features wacom_features_0x5B =
- { "Wacom Cintiq 22HDT", WACOM_PKGLEN_INTUOS, 95840, 54260, 2047,
- 63, WACOM_22HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES,
+ { "Wacom Cintiq 22HDT", WACOM_PKGLEN_INTUOS, 95640, 54060, 2047,
+ 63, WACOM_22HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES, 200, 200,
.oVid = USB_VENDOR_ID_WACOM, .oPid = 0x5e };
static const struct wacom_features wacom_features_0x5E =
{ "Wacom Cintiq 22HDT", .type = WACOM_24HDT,
@@ -2233,9 +2245,21 @@
static const struct wacom_features wacom_features_0x10F =
{ "Wacom ISDv4 10F", WACOM_PKGLEN_MTTPC, 27760, 15694, 255,
0, MTTPC, WACOM_INTUOS_RES, WACOM_INTUOS_RES };
+static const struct wacom_features wacom_features_0x116 =
+ { "Wacom ISDv4 116", WACOM_PKGLEN_GRAPHIRE, 26202, 16325, 255,
+ 0, TABLETPCE, WACOM_INTUOS_RES, WACOM_INTUOS_RES };
static const struct wacom_features wacom_features_0x4001 =
{ "Wacom ISDv4 4001", WACOM_PKGLEN_MTTPC, 26202, 16325, 255,
0, MTTPC, WACOM_INTUOS_RES, WACOM_INTUOS_RES };
+static const struct wacom_features wacom_features_0x4004 =
+ { "Wacom ISDv4 4004", WACOM_PKGLEN_MTTPC, 11060, 6220, 255,
+ 0, MTTPC_B, WACOM_INTUOS_RES, WACOM_INTUOS_RES };
+static const struct wacom_features wacom_features_0x5000 =
+ { "Wacom ISDv4 5000", WACOM_PKGLEN_MTTPC, 27848, 15752, 1023,
+ 0, MTTPC_B, WACOM_INTUOS_RES, WACOM_INTUOS_RES };
+static const struct wacom_features wacom_features_0x5002 =
+ { "Wacom ISDv4 5002", WACOM_PKGLEN_MTTPC, 29576, 16724, 1023,
+ 0, MTTPC_B, WACOM_INTUOS_RES, WACOM_INTUOS_RES };
static const struct wacom_features wacom_features_0x47 =
{ "Wacom Intuos2 6x8", WACOM_PKGLEN_INTUOS, 20320, 16240, 1023,
31, INTUOS, WACOM_INTUOS_RES, WACOM_INTUOS_RES };
@@ -2316,8 +2340,8 @@
{ "ISD-V4", WACOM_PKGLEN_GRAPHIRE, 12800, 8000, 255,
0, TABLETPC, WACOM_INTUOS_RES, WACOM_INTUOS_RES };
static const struct wacom_features wacom_features_0x0307 =
- { "Wacom ISDv5 307", WACOM_PKGLEN_INTUOS, 59552, 33848, 2047,
- 63, CINTIQ_HYBRID, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES,
+ { "Wacom ISDv5 307", WACOM_PKGLEN_INTUOS, 59352, 33648, 2047,
+ 63, CINTIQ_HYBRID, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES, 200, 200,
.oVid = USB_VENDOR_ID_WACOM, .oPid = 0x309 };
static const struct wacom_features wacom_features_0x0309 =
{ "Wacom ISDv5 309", .type = WACOM_24HDT, /* Touch */
@@ -2447,6 +2471,7 @@
{ USB_DEVICE_WACOM(0x10D) },
{ USB_DEVICE_WACOM(0x10E) },
{ USB_DEVICE_WACOM(0x10F) },
+ { USB_DEVICE_WACOM(0x116) },
{ USB_DEVICE_WACOM(0x300) },
{ USB_DEVICE_WACOM(0x301) },
{ USB_DEVICE_DETAILED(0x302, USB_CLASS_HID, 0, 0) },
@@ -2457,6 +2482,9 @@
{ USB_DEVICE_DETAILED(0x315, USB_CLASS_HID, 0, 0) },
{ USB_DEVICE_DETAILED(0x317, USB_CLASS_HID, 0, 0) },
{ USB_DEVICE_WACOM(0x4001) },
+ { USB_DEVICE_WACOM(0x4004) },
+ { USB_DEVICE_WACOM(0x5000) },
+ { USB_DEVICE_WACOM(0x5002) },
{ USB_DEVICE_WACOM(0x47) },
{ USB_DEVICE_WACOM(0xF4) },
{ USB_DEVICE_WACOM(0xF8) },
diff --git a/drivers/input/tablet/wacom_wac.h b/drivers/input/tablet/wacom_wac.h
index f69c0eb..b2c9a9c 100644
--- a/drivers/input/tablet/wacom_wac.h
+++ b/drivers/input/tablet/wacom_wac.h
@@ -22,6 +22,7 @@
#define WACOM_PKGLEN_BBFUN 9
#define WACOM_PKGLEN_INTUOS 10
#define WACOM_PKGLEN_TPC1FG 5
+#define WACOM_PKGLEN_TPC1FG_B 10
#define WACOM_PKGLEN_TPC2FG 14
#define WACOM_PKGLEN_BBTOUCH 20
#define WACOM_PKGLEN_BBTOUCH3 64
@@ -30,6 +31,7 @@
#define WACOM_PKGLEN_MTOUCH 62
#define WACOM_PKGLEN_MTTPC 40
#define WACOM_PKGLEN_DTUS 68
+#define WACOM_PKGLEN_PENABLED 8
/* wacom data size per MT contact */
#define WACOM_BYTES_PER_MT_PACKET 11
@@ -52,6 +54,7 @@
#define WACOM_REPORT_TPC1FG 6
#define WACOM_REPORT_TPC2FG 13
#define WACOM_REPORT_TPCMT 13
+#define WACOM_REPORT_TPCMT2 3
#define WACOM_REPORT_TPCHID 15
#define WACOM_REPORT_TPCST 16
#define WACOM_REPORT_DTUS 17
@@ -105,6 +108,7 @@
TABLETPC2FG,
MTSCREEN,
MTTPC,
+ MTTPC_B,
MAX_TYPE
};
@@ -118,6 +122,8 @@
int type;
int x_resolution;
int y_resolution;
+ int x_min;
+ int y_min;
int device_type;
int x_phy;
int y_phy;
diff --git a/drivers/input/touchscreen/88pm860x-ts.c b/drivers/input/touchscreen/88pm860x-ts.c
index 544e20c..0d4a9fa 100644
--- a/drivers/input/touchscreen/88pm860x-ts.c
+++ b/drivers/input/touchscreen/88pm860x-ts.c
@@ -16,6 +16,7 @@
#include <linux/input.h>
#include <linux/mfd/88pm860x.h>
#include <linux/slab.h>
+#include <linux/device.h>
#define MEAS_LEN (8)
#define ACCURATE_BIT (12)
@@ -234,16 +235,17 @@
if (ret)
return ret;
- touch = kzalloc(sizeof(struct pm860x_touch), GFP_KERNEL);
- if (touch == NULL)
+ touch = devm_kzalloc(&pdev->dev, sizeof(struct pm860x_touch),
+ GFP_KERNEL);
+ if (!touch)
return -ENOMEM;
+
platform_set_drvdata(pdev, touch);
- touch->idev = input_allocate_device();
- if (touch->idev == NULL) {
+ touch->idev = devm_input_allocate_device(&pdev->dev);
+ if (!touch->idev) {
dev_err(&pdev->dev, "Failed to allocate input device!\n");
- ret = -ENOMEM;
- goto out;
+ return -ENOMEM;
}
touch->idev->name = "88pm860x-touch";
@@ -258,10 +260,11 @@
touch->res_x = res_x;
input_set_drvdata(touch->idev, touch);
- ret = request_threaded_irq(touch->irq, NULL, pm860x_touch_handler,
- IRQF_ONESHOT, "touch", touch);
+ ret = devm_request_threaded_irq(&pdev->dev, touch->irq, NULL,
+ pm860x_touch_handler, IRQF_ONESHOT,
+ "touch", touch);
if (ret < 0)
- goto out_irq;
+ return ret;
__set_bit(EV_ABS, touch->idev->evbit);
__set_bit(ABS_X, touch->idev->absbit);
@@ -279,28 +282,11 @@
ret = input_register_device(touch->idev);
if (ret < 0) {
dev_err(chip->dev, "Failed to register touch!\n");
- goto out_rg;
+ return ret;
}
platform_set_drvdata(pdev, touch);
return 0;
-out_rg:
- free_irq(touch->irq, touch);
-out_irq:
- input_free_device(touch->idev);
-out:
- kfree(touch);
- return ret;
-}
-
-static int pm860x_touch_remove(struct platform_device *pdev)
-{
- struct pm860x_touch *touch = platform_get_drvdata(pdev);
-
- input_unregister_device(touch->idev);
- free_irq(touch->irq, touch);
- kfree(touch);
- return 0;
}
static struct platform_driver pm860x_touch_driver = {
@@ -309,7 +295,6 @@
.owner = THIS_MODULE,
},
.probe = pm860x_touch_probe,
- .remove = pm860x_touch_remove,
};
module_platform_driver(pm860x_touch_driver);
diff --git a/drivers/input/touchscreen/Kconfig b/drivers/input/touchscreen/Kconfig
index d4e5ab5..a23a94b 100644
--- a/drivers/input/touchscreen/Kconfig
+++ b/drivers/input/touchscreen/Kconfig
@@ -11,6 +11,10 @@
if INPUT_TOUCHSCREEN
+config OF_TOUCHSCREEN
+ def_tristate INPUT
+ depends on INPUT && OF
+
config TOUCHSCREEN_88PM860X
tristate "Marvell 88PM860x touchscreen"
depends on MFD_88PM860X
@@ -89,6 +93,7 @@
config TOUCHSCREEN_ATMEL_MXT
tristate "Atmel mXT I2C Touchscreen"
depends on I2C
+ select FW_LOADER
help
Say Y here if you have Atmel mXT series I2C touchscreen,
such as AT42QT602240/ATMXT224, connected to your system.
@@ -846,7 +851,7 @@
config TOUCHSCREEN_W90X900
tristate "W90P910 touchscreen driver"
- depends on HAVE_CLK
+ depends on ARCH_W90X900
help
Say Y here if you have a W90P910 based touchscreen.
@@ -885,6 +890,17 @@
To compile this driver as a module, choose M here: the
module will be called stmpe-ts.
+config TOUCHSCREEN_SUN4I
+ tristate "Allwinner sun4i resistive touchscreen controller support"
+ depends on ARCH_SUNXI || COMPILE_TEST
+ depends on HWMON
+ help
+ This selects support for the resistive touchscreen controller
+ found on Allwinner sunxi SoCs.
+
+ To compile this driver as a module, choose M here: the
+ module will be called sun4i-ts.
+
config TOUCHSCREEN_SUR40
tristate "Samsung SUR40 (Surface 2.0/PixelSense) touchscreen"
depends on USB
diff --git a/drivers/input/touchscreen/Makefile b/drivers/input/touchscreen/Makefile
index 03f12a1..126479d 100644
--- a/drivers/input/touchscreen/Makefile
+++ b/drivers/input/touchscreen/Makefile
@@ -6,6 +6,7 @@
wm97xx-ts-y := wm97xx-core.o
+obj-$(CONFIG_OF_TOUCHSCREEN) += of_touchscreen.o
obj-$(CONFIG_TOUCHSCREEN_88PM860X) += 88pm860x-ts.o
obj-$(CONFIG_TOUCHSCREEN_AD7877) += ad7877.o
obj-$(CONFIG_TOUCHSCREEN_AD7879) += ad7879.o
@@ -53,6 +54,7 @@
obj-$(CONFIG_TOUCHSCREEN_S3C2410) += s3c2410_ts.o
obj-$(CONFIG_TOUCHSCREEN_ST1232) += st1232.o
obj-$(CONFIG_TOUCHSCREEN_STMPE) += stmpe-ts.o
+obj-$(CONFIG_TOUCHSCREEN_SUN4I) += sun4i-ts.o
obj-$(CONFIG_TOUCHSCREEN_SUR40) += sur40.o
obj-$(CONFIG_TOUCHSCREEN_TI_AM335X_TSC) += ti_am335x_tsc.o
obj-$(CONFIG_TOUCHSCREEN_TOUCHIT213) += touchit213.o
diff --git a/drivers/input/touchscreen/ad7877.c b/drivers/input/touchscreen/ad7877.c
index 6793c85..523865d 100644
--- a/drivers/input/touchscreen/ad7877.c
+++ b/drivers/input/touchscreen/ad7877.c
@@ -210,11 +210,6 @@
module_param(gpio3, bool, 0);
MODULE_PARM_DESC(gpio3, "If gpio3 is set to 1 AUX3 acts as GPIO3");
-/*
- * ad7877_read/write are only used for initial setup and for sysfs controls.
- * The main traffic is done using spi_async() in the interrupt handler.
- */
-
static int ad7877_read(struct spi_device *spi, u16 reg)
{
struct ser_req *req;
diff --git a/drivers/input/touchscreen/ads7846.c b/drivers/input/touchscreen/ads7846.c
index 7f8aa98..da201b8 100644
--- a/drivers/input/touchscreen/ads7846.c
+++ b/drivers/input/touchscreen/ads7846.c
@@ -706,7 +706,7 @@
m = &ts->msg[msg_idx];
error = spi_sync(ts->spi, m);
if (error) {
- dev_err(&ts->spi->dev, "spi_async --> %d\n", error);
+ dev_err(&ts->spi->dev, "spi_sync --> %d\n", error);
packet->tc.ignore = true;
return;
}
diff --git a/drivers/input/touchscreen/atmel_mxt_ts.c b/drivers/input/touchscreen/atmel_mxt_ts.c
index a704007..6e0b4a2 100644
--- a/drivers/input/touchscreen/atmel_mxt_ts.c
+++ b/drivers/input/touchscreen/atmel_mxt_ts.c
@@ -2,6 +2,8 @@
* Atmel maXTouch Touchscreen driver
*
* Copyright (C) 2010 Samsung Electronics Co.Ltd
+ * Copyright (C) 2012 Google, Inc.
+ *
* Author: Joonyoung Shim <jy0922.shim@samsung.com>
*
* This program is free software; you can redistribute it and/or modify it
@@ -12,6 +14,8 @@
*/
#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/completion.h>
#include <linux/delay.h>
#include <linux/firmware.h>
#include <linux/i2c.h>
@@ -25,12 +29,6 @@
#define MXT_VER_21 21
#define MXT_VER_22 22
-/* Slave addresses */
-#define MXT_APP_LOW 0x4a
-#define MXT_APP_HIGH 0x4b
-#define MXT_BOOT_LOW 0x24
-#define MXT_BOOT_HIGH 0x25
-
/* Firmware */
#define MXT_FW_NAME "maxtouch.fw"
@@ -83,6 +81,9 @@
#define MXT_COMMAND_REPORTALL 3
#define MXT_COMMAND_DIAGNOSTIC 5
+/* Define for T6 status byte */
+#define MXT_T6_STATUS_RESET (1 << 7)
+
/* MXT_GEN_POWER_T7 field */
#define MXT_POWER_IDLEACQINT 0
#define MXT_POWER_ACTVACQINT 1
@@ -99,33 +100,26 @@
/* MXT_TOUCH_MULTI_T9 field */
#define MXT_TOUCH_CTRL 0
-#define MXT_TOUCH_XORIGIN 1
-#define MXT_TOUCH_YORIGIN 2
-#define MXT_TOUCH_XSIZE 3
-#define MXT_TOUCH_YSIZE 4
-#define MXT_TOUCH_BLEN 6
-#define MXT_TOUCH_TCHTHR 7
-#define MXT_TOUCH_TCHDI 8
-#define MXT_TOUCH_ORIENT 9
-#define MXT_TOUCH_MOVHYSTI 11
-#define MXT_TOUCH_MOVHYSTN 12
-#define MXT_TOUCH_NUMTOUCH 14
-#define MXT_TOUCH_MRGHYST 15
-#define MXT_TOUCH_MRGTHR 16
-#define MXT_TOUCH_AMPHYST 17
-#define MXT_TOUCH_XRANGE_LSB 18
-#define MXT_TOUCH_XRANGE_MSB 19
-#define MXT_TOUCH_YRANGE_LSB 20
-#define MXT_TOUCH_YRANGE_MSB 21
-#define MXT_TOUCH_XLOCLIP 22
-#define MXT_TOUCH_XHICLIP 23
-#define MXT_TOUCH_YLOCLIP 24
-#define MXT_TOUCH_YHICLIP 25
-#define MXT_TOUCH_XEDGECTRL 26
-#define MXT_TOUCH_XEDGEDIST 27
-#define MXT_TOUCH_YEDGECTRL 28
-#define MXT_TOUCH_YEDGEDIST 29
-#define MXT_TOUCH_JUMPLIMIT 30
+#define MXT_T9_ORIENT 9
+#define MXT_T9_RANGE 18
+
+/* MXT_TOUCH_MULTI_T9 status */
+#define MXT_T9_UNGRIP (1 << 0)
+#define MXT_T9_SUPPRESS (1 << 1)
+#define MXT_T9_AMP (1 << 2)
+#define MXT_T9_VECTOR (1 << 3)
+#define MXT_T9_MOVE (1 << 4)
+#define MXT_T9_RELEASE (1 << 5)
+#define MXT_T9_PRESS (1 << 6)
+#define MXT_T9_DETECT (1 << 7)
+
+struct t9_range {
+ u16 x;
+ u16 y;
+} __packed;
+
+/* MXT_TOUCH_MULTI_T9 orient */
+#define MXT_T9_ORIENT_SWITCH (1 << 0)
/* MXT_PROCI_GRIPFACE_T20 field */
#define MXT_GRIPFACE_CTRL 0
@@ -174,17 +168,16 @@
/* Define for MXT_GEN_COMMAND_T6 */
#define MXT_BOOT_VALUE 0xa5
+#define MXT_RESET_VALUE 0x01
#define MXT_BACKUP_VALUE 0x55
+
+/* Delay times */
#define MXT_BACKUP_TIME 50 /* msec */
#define MXT_RESET_TIME 200 /* msec */
-
-#define MXT_FWRESET_TIME 175 /* msec */
-
-/* MXT_SPT_GPIOPWM_T19 field */
-#define MXT_GPIO0_MASK 0x04
-#define MXT_GPIO1_MASK 0x08
-#define MXT_GPIO2_MASK 0x10
-#define MXT_GPIO3_MASK 0x20
+#define MXT_RESET_TIMEOUT 3000 /* msec */
+#define MXT_CRC_TIMEOUT 1000 /* msec */
+#define MXT_FW_RESET_TIME 3000 /* msec */
+#define MXT_FW_CHG_TIMEOUT 300 /* msec */
/* Command to unlock bootloader */
#define MXT_UNLOCK_CMD_MSB 0xaa
@@ -198,21 +191,8 @@
#define MXT_FRAME_CRC_PASS 0x04
#define MXT_APP_CRC_FAIL 0x40 /* valid 7 8 bit only */
#define MXT_BOOT_STATUS_MASK 0x3f
-
-/* Touch status */
-#define MXT_UNGRIP (1 << 0)
-#define MXT_SUPPRESS (1 << 1)
-#define MXT_AMP (1 << 2)
-#define MXT_VECTOR (1 << 3)
-#define MXT_MOVE (1 << 4)
-#define MXT_RELEASE (1 << 5)
-#define MXT_PRESS (1 << 6)
-#define MXT_DETECT (1 << 7)
-
-/* Touch orient bits */
-#define MXT_XY_SWITCH (1 << 0)
-#define MXT_X_INVERT (1 << 1)
-#define MXT_Y_INVERT (1 << 2)
+#define MXT_BOOT_EXTENDED_ID (1 << 5)
+#define MXT_BOOT_ID_MASK 0x1f
/* Touchscreen absolute values */
#define MXT_MAX_AREA 0xff
@@ -232,8 +212,8 @@
struct mxt_object {
u8 type;
u16 start_address;
- u8 size; /* Size of each instance - 1 */
- u8 instances; /* Number of instances - 1 */
+ u8 size_minus_one;
+ u8 instances_minus_one;
u8 num_report_ids;
} __packed;
@@ -250,19 +230,40 @@
const struct mxt_platform_data *pdata;
struct mxt_object *object_table;
struct mxt_info info;
- bool is_tp;
-
unsigned int irq;
unsigned int max_x;
unsigned int max_y;
+ bool in_bootloader;
+ u32 config_crc;
+ u8 bootloader_addr;
/* Cached parameters from object table */
u8 T6_reportid;
+ u16 T6_address;
u8 T9_reportid_min;
u8 T9_reportid_max;
u8 T19_reportid;
+
+ /* for fw update in bootloader */
+ struct completion bl_completion;
+
+ /* for reset handling */
+ struct completion reset_completion;
+
+ /* for config update handling */
+ struct completion crc_completion;
};
+static size_t mxt_obj_size(const struct mxt_object *obj)
+{
+ return obj->size_minus_one + 1;
+}
+
+static size_t mxt_obj_instances(const struct mxt_object *obj)
+{
+ return obj->instances_minus_one + 1;
+}
+
static bool mxt_object_readable(unsigned int type)
{
switch (type) {
@@ -334,60 +335,190 @@
message->reportid, 7, message->message);
}
-static int mxt_check_bootloader(struct i2c_client *client,
- unsigned int state)
+static int mxt_wait_for_completion(struct mxt_data *data,
+ struct completion *comp,
+ unsigned int timeout_ms)
{
+ struct device *dev = &data->client->dev;
+ unsigned long timeout = msecs_to_jiffies(timeout_ms);
+ long ret;
+
+ ret = wait_for_completion_interruptible_timeout(comp, timeout);
+ if (ret < 0) {
+ return ret;
+ } else if (ret == 0) {
+ dev_err(dev, "Wait for completion timed out.\n");
+ return -ETIMEDOUT;
+ }
+ return 0;
+}
+
+static int mxt_bootloader_read(struct mxt_data *data,
+ u8 *val, unsigned int count)
+{
+ int ret;
+ struct i2c_msg msg;
+
+ msg.addr = data->bootloader_addr;
+ msg.flags = data->client->flags & I2C_M_TEN;
+ msg.flags |= I2C_M_RD;
+ msg.len = count;
+ msg.buf = val;
+
+ ret = i2c_transfer(data->client->adapter, &msg, 1);
+
+ if (ret == 1) {
+ ret = 0;
+ } else {
+ ret = ret < 0 ? ret : -EIO;
+ dev_err(&data->client->dev, "%s: i2c recv failed (%d)\n",
+ __func__, ret);
+ }
+
+ return ret;
+}
+
+static int mxt_bootloader_write(struct mxt_data *data,
+ const u8 * const val, unsigned int count)
+{
+ int ret;
+ struct i2c_msg msg;
+
+ msg.addr = data->bootloader_addr;
+ msg.flags = data->client->flags & I2C_M_TEN;
+ msg.len = count;
+ msg.buf = (u8 *)val;
+
+ ret = i2c_transfer(data->client->adapter, &msg, 1);
+ if (ret == 1) {
+ ret = 0;
+ } else {
+ ret = ret < 0 ? ret : -EIO;
+ dev_err(&data->client->dev, "%s: i2c send failed (%d)\n",
+ __func__, ret);
+ }
+
+ return ret;
+}
+
+static int mxt_lookup_bootloader_address(struct mxt_data *data)
+{
+ u8 appmode = data->client->addr;
+ u8 bootloader;
+
+ switch (appmode) {
+ case 0x4a:
+ case 0x4b:
+ case 0x4c:
+ case 0x4d:
+ case 0x5a:
+ case 0x5b:
+ bootloader = appmode - 0x26;
+ break;
+ default:
+ dev_err(&data->client->dev,
+ "Appmode i2c address 0x%02x not found\n",
+ appmode);
+ return -EINVAL;
+ }
+
+ data->bootloader_addr = bootloader;
+ return 0;
+}
+
+static u8 mxt_get_bootloader_version(struct mxt_data *data, u8 val)
+{
+ struct device *dev = &data->client->dev;
+ u8 buf[3];
+
+ if (val & MXT_BOOT_EXTENDED_ID) {
+ if (mxt_bootloader_read(data, &buf[0], 3) != 0) {
+ dev_err(dev, "%s: i2c failure\n", __func__);
+ return val;
+ }
+
+ dev_dbg(dev, "Bootloader ID:%d Version:%d\n", buf[1], buf[2]);
+
+ return buf[0];
+ } else {
+ dev_dbg(dev, "Bootloader ID:%d\n", val & MXT_BOOT_ID_MASK);
+
+ return val;
+ }
+}
+
+static int mxt_check_bootloader(struct mxt_data *data, unsigned int state)
+{
+ struct device *dev = &data->client->dev;
u8 val;
+ int ret;
recheck:
- if (i2c_master_recv(client, &val, 1) != 1) {
- dev_err(&client->dev, "%s: i2c recv failed\n", __func__);
- return -EIO;
+ if (state != MXT_WAITING_BOOTLOAD_CMD) {
+ /*
+ * In application update mode, the interrupt
+ * line signals state transitions. We must wait for the
+ * CHG assertion before reading the status byte.
+ * Once the status byte has been read, the line is deasserted.
+ */
+ ret = mxt_wait_for_completion(data, &data->bl_completion,
+ MXT_FW_CHG_TIMEOUT);
+ if (ret) {
+ /*
+ * TODO: handle -ERESTARTSYS better by terminating
+ * fw update process before returning to userspace
+ * by writing length 0x000 to device (iff we are in
+ * WAITING_FRAME_DATA state).
+ */
+ dev_err(dev, "Update wait error %d\n", ret);
+ return ret;
+ }
}
+ ret = mxt_bootloader_read(data, &val, 1);
+ if (ret)
+ return ret;
+
+ if (state == MXT_WAITING_BOOTLOAD_CMD)
+ val = mxt_get_bootloader_version(data, val);
+
switch (state) {
case MXT_WAITING_BOOTLOAD_CMD:
case MXT_WAITING_FRAME_DATA:
val &= ~MXT_BOOT_STATUS_MASK;
break;
case MXT_FRAME_CRC_PASS:
- if (val == MXT_FRAME_CRC_CHECK)
+ if (val == MXT_FRAME_CRC_CHECK) {
goto recheck;
+ } else if (val == MXT_FRAME_CRC_FAIL) {
+ dev_err(dev, "Bootloader CRC fail\n");
+ return -EINVAL;
+ }
break;
default:
return -EINVAL;
}
if (val != state) {
- dev_err(&client->dev, "Unvalid bootloader mode state\n");
+ dev_err(dev, "Invalid bootloader state %02X != %02X\n",
+ val, state);
return -EINVAL;
}
return 0;
}
-static int mxt_unlock_bootloader(struct i2c_client *client)
+static int mxt_unlock_bootloader(struct mxt_data *data)
{
+ int ret;
u8 buf[2];
buf[0] = MXT_UNLOCK_CMD_LSB;
buf[1] = MXT_UNLOCK_CMD_MSB;
- if (i2c_master_send(client, buf, 2) != 2) {
- dev_err(&client->dev, "%s: i2c send failed\n", __func__);
- return -EIO;
- }
-
- return 0;
-}
-
-static int mxt_fw_write(struct i2c_client *client,
- const u8 *data, unsigned int frame_size)
-{
- if (i2c_master_send(client, data, frame_size) != frame_size) {
- dev_err(&client->dev, "%s: i2c send failed\n", __func__);
- return -EIO;
- }
+ ret = mxt_bootloader_write(data, buf, 2);
+ if (ret)
+ return ret;
return 0;
}
@@ -427,11 +558,6 @@
return ret;
}
-static int mxt_read_reg(struct i2c_client *client, u16 reg, u8 *val)
-{
- return __mxt_read_reg(client, reg, 1, val);
-}
-
static int __mxt_write_reg(struct i2c_client *client, u16 reg, u16 len,
const void *val)
{
@@ -479,7 +605,7 @@
return object;
}
- dev_err(&data->client->dev, "Invalid object type\n");
+ dev_err(&data->client->dev, "Invalid object type T%u\n", type);
return NULL;
}
@@ -505,7 +631,7 @@
u16 reg;
object = mxt_get_object(data, type);
- if (!object || offset >= object->size + 1)
+ if (!object || offset >= mxt_obj_size(object))
return -EINVAL;
reg = object->start_address;
@@ -515,18 +641,25 @@
static void mxt_input_button(struct mxt_data *data, struct mxt_message *message)
{
struct input_dev *input = data->input_dev;
+ const struct mxt_platform_data *pdata = data->pdata;
bool button;
int i;
/* Active-low switch */
- for (i = 0; i < MXT_NUM_GPIO; i++) {
- if (data->pdata->key_map[i] == KEY_RESERVED)
+ for (i = 0; i < pdata->t19_num_keys; i++) {
+ if (pdata->t19_keymap[i] == KEY_RESERVED)
continue;
- button = !(message->message[0] & MXT_GPIO0_MASK << i);
- input_report_key(input, data->pdata->key_map[i], button);
+ button = !(message->message[0] & (1 << i));
+ input_report_key(input, pdata->t19_keymap[i], button);
}
}
+static void mxt_input_sync(struct input_dev *input_dev)
+{
+ input_mt_report_pointer_emulation(input_dev, false);
+ input_sync(input_dev);
+}
+
static void mxt_input_touchevent(struct mxt_data *data,
struct mxt_message *message, int id)
{
@@ -536,44 +669,60 @@
int x;
int y;
int area;
- int pressure;
+ int amplitude;
x = (message->message[1] << 4) | ((message->message[3] >> 4) & 0xf);
y = (message->message[2] << 4) | ((message->message[3] & 0xf));
+
+ /* Handle 10/12 bit switching */
if (data->max_x < 1024)
- x = x >> 2;
+ x >>= 2;
if (data->max_y < 1024)
- y = y >> 2;
+ y >>= 2;
area = message->message[4];
- pressure = message->message[5];
+ amplitude = message->message[5];
dev_dbg(dev,
"[%u] %c%c%c%c%c%c%c%c x: %5u y: %5u area: %3u amp: %3u\n",
id,
- (status & MXT_DETECT) ? 'D' : '.',
- (status & MXT_PRESS) ? 'P' : '.',
- (status & MXT_RELEASE) ? 'R' : '.',
- (status & MXT_MOVE) ? 'M' : '.',
- (status & MXT_VECTOR) ? 'V' : '.',
- (status & MXT_AMP) ? 'A' : '.',
- (status & MXT_SUPPRESS) ? 'S' : '.',
- (status & MXT_UNGRIP) ? 'U' : '.',
- x, y, area, pressure);
+ (status & MXT_T9_DETECT) ? 'D' : '.',
+ (status & MXT_T9_PRESS) ? 'P' : '.',
+ (status & MXT_T9_RELEASE) ? 'R' : '.',
+ (status & MXT_T9_MOVE) ? 'M' : '.',
+ (status & MXT_T9_VECTOR) ? 'V' : '.',
+ (status & MXT_T9_AMP) ? 'A' : '.',
+ (status & MXT_T9_SUPPRESS) ? 'S' : '.',
+ (status & MXT_T9_UNGRIP) ? 'U' : '.',
+ x, y, area, amplitude);
input_mt_slot(input_dev, id);
- input_mt_report_slot_state(input_dev, MT_TOOL_FINGER,
- status & MXT_DETECT);
- if (status & MXT_DETECT) {
+ if (status & MXT_T9_DETECT) {
+ /*
+ * Multiple bits may be set if the host is slow to read
+ * the status messages, indicating all the events that
+ * have happened.
+ */
+ if (status & MXT_T9_RELEASE) {
+ input_mt_report_slot_state(input_dev,
+ MT_TOOL_FINGER, 0);
+ mxt_input_sync(input_dev);
+ }
+
+ /* Touch active */
+ input_mt_report_slot_state(input_dev, MT_TOOL_FINGER, 1);
input_report_abs(input_dev, ABS_MT_POSITION_X, x);
input_report_abs(input_dev, ABS_MT_POSITION_Y, y);
- input_report_abs(input_dev, ABS_MT_PRESSURE, pressure);
+ input_report_abs(input_dev, ABS_MT_PRESSURE, amplitude);
input_report_abs(input_dev, ABS_MT_TOUCH_MAJOR, area);
+ } else {
+ /* Touch no longer active, close out slot */
+ input_mt_report_slot_state(input_dev, MT_TOOL_FINGER, 0);
}
}
-static unsigned mxt_extract_T6_csum(const u8 *csum)
+static u16 mxt_extract_T6_csum(const u8 *csum)
{
return csum[0] | (csum[1] << 8) | (csum[2] << 16);
}
@@ -584,28 +733,37 @@
return (id >= data->T9_reportid_min && id <= data->T9_reportid_max);
}
-static irqreturn_t mxt_interrupt(int irq, void *dev_id)
+static irqreturn_t mxt_process_messages_until_invalid(struct mxt_data *data)
{
- struct mxt_data *data = dev_id;
struct mxt_message message;
const u8 *payload = &message.message[0];
struct device *dev = &data->client->dev;
u8 reportid;
bool update_input = false;
+ u32 crc;
do {
if (mxt_read_message(data, &message)) {
dev_err(dev, "Failed to read message\n");
- goto end;
+ return IRQ_NONE;
}
reportid = message.reportid;
if (reportid == data->T6_reportid) {
u8 status = payload[0];
- unsigned csum = mxt_extract_T6_csum(&payload[1]);
+
+ crc = mxt_extract_T6_csum(&payload[1]);
+ if (crc != data->config_crc) {
+ data->config_crc = crc;
+ complete(&data->crc_completion);
+ }
+
dev_dbg(dev, "Status: %02x Config Checksum: %06x\n",
- status, csum);
+ status, data->config_crc);
+
+ if (status & MXT_T6_STATUS_RESET)
+ complete(&data->reset_completion);
} else if (mxt_is_T9_message(data, &message)) {
int id = reportid - data->T9_reportid_min;
mxt_input_touchevent(data, &message, id);
@@ -618,13 +776,94 @@
}
} while (reportid != 0xff);
- if (update_input) {
- input_mt_report_pointer_emulation(data->input_dev, false);
- input_sync(data->input_dev);
+ if (update_input)
+ mxt_input_sync(data->input_dev);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t mxt_interrupt(int irq, void *dev_id)
+{
+ struct mxt_data *data = dev_id;
+
+ if (data->in_bootloader) {
+ /* bootloader state transition completion */
+ complete(&data->bl_completion);
+ return IRQ_HANDLED;
}
-end:
- return IRQ_HANDLED;
+ return mxt_process_messages_until_invalid(data);
+}
+
+static int mxt_t6_command(struct mxt_data *data, u16 cmd_offset,
+ u8 value, bool wait)
+{
+ u16 reg;
+ u8 command_register;
+ int timeout_counter = 0;
+ int ret;
+
+ reg = data->T6_address + cmd_offset;
+
+ ret = mxt_write_reg(data->client, reg, value);
+ if (ret)
+ return ret;
+
+ if (!wait)
+ return 0;
+
+ do {
+ msleep(20);
+ ret = __mxt_read_reg(data->client, reg, 1, &command_register);
+ if (ret)
+ return ret;
+ } while (command_register != 0 && timeout_counter++ <= 100);
+
+ if (timeout_counter > 100) {
+ dev_err(&data->client->dev, "Command failed!\n");
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int mxt_soft_reset(struct mxt_data *data)
+{
+ struct device *dev = &data->client->dev;
+ int ret = 0;
+
+ dev_info(dev, "Resetting chip\n");
+
+ reinit_completion(&data->reset_completion);
+
+ ret = mxt_t6_command(data, MXT_COMMAND_RESET, MXT_RESET_VALUE, false);
+ if (ret)
+ return ret;
+
+ ret = mxt_wait_for_completion(data, &data->reset_completion,
+ MXT_RESET_TIMEOUT);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static void mxt_update_crc(struct mxt_data *data, u8 cmd, u8 value)
+{
+ /*
+ * On failure, CRC is set to 0 and config will always be
+ * downloaded.
+ */
+ data->config_crc = 0;
+ reinit_completion(&data->crc_completion);
+
+ mxt_t6_command(data, cmd, value, true);
+
+ /*
+ * Wait for crc message. On failure, CRC is set to 0 and config will
+ * always be downloaded.
+ */
+ mxt_wait_for_completion(data, &data->crc_completion, MXT_CRC_TIMEOUT);
}
static int mxt_check_reg_init(struct mxt_data *data)
@@ -641,13 +880,23 @@
return 0;
}
+ mxt_update_crc(data, MXT_COMMAND_REPORTALL, 1);
+
+ if (data->config_crc == pdata->config_crc) {
+ dev_info(dev, "Config CRC 0x%06X: OK\n", data->config_crc);
+ return 0;
+ }
+
+ dev_info(dev, "Config CRC 0x%06X: does not match 0x%06X\n",
+ data->config_crc, pdata->config_crc);
+
for (i = 0; i < data->info.object_num; i++) {
object = data->object_table + i;
if (!mxt_object_writable(object->type))
continue;
- size = (object->size + 1) * (object->instances + 1);
+ size = mxt_obj_size(object) * mxt_obj_instances(object);
if (index + size > pdata->config_length) {
dev_err(dev, "Not enough config data!\n");
return -EINVAL;
@@ -660,6 +909,14 @@
index += size;
}
+ mxt_update_crc(data, MXT_COMMAND_BACKUPNV, MXT_BACKUP_VALUE);
+
+ ret = mxt_soft_reset(data);
+ if (ret)
+ return ret;
+
+ dev_info(dev, "Config successfully updated\n");
+
return 0;
}
@@ -685,54 +942,6 @@
return 0;
}
-static void mxt_handle_pdata(struct mxt_data *data)
-{
- const struct mxt_platform_data *pdata = data->pdata;
- u8 voltage;
-
- /* Set touchscreen lines */
- mxt_write_object(data, MXT_TOUCH_MULTI_T9, MXT_TOUCH_XSIZE,
- pdata->x_line);
- mxt_write_object(data, MXT_TOUCH_MULTI_T9, MXT_TOUCH_YSIZE,
- pdata->y_line);
-
- /* Set touchscreen orient */
- mxt_write_object(data, MXT_TOUCH_MULTI_T9, MXT_TOUCH_ORIENT,
- pdata->orient);
-
- /* Set touchscreen burst length */
- mxt_write_object(data, MXT_TOUCH_MULTI_T9,
- MXT_TOUCH_BLEN, pdata->blen);
-
- /* Set touchscreen threshold */
- mxt_write_object(data, MXT_TOUCH_MULTI_T9,
- MXT_TOUCH_TCHTHR, pdata->threshold);
-
- /* Set touchscreen resolution */
- mxt_write_object(data, MXT_TOUCH_MULTI_T9,
- MXT_TOUCH_XRANGE_LSB, (pdata->x_size - 1) & 0xff);
- mxt_write_object(data, MXT_TOUCH_MULTI_T9,
- MXT_TOUCH_XRANGE_MSB, (pdata->x_size - 1) >> 8);
- mxt_write_object(data, MXT_TOUCH_MULTI_T9,
- MXT_TOUCH_YRANGE_LSB, (pdata->y_size - 1) & 0xff);
- mxt_write_object(data, MXT_TOUCH_MULTI_T9,
- MXT_TOUCH_YRANGE_MSB, (pdata->y_size - 1) >> 8);
-
- /* Set touchscreen voltage */
- if (pdata->voltage) {
- if (pdata->voltage < MXT_VOLTAGE_DEFAULT) {
- voltage = (MXT_VOLTAGE_DEFAULT - pdata->voltage) /
- MXT_VOLTAGE_STEP;
- voltage = 0xff - voltage + 1;
- } else
- voltage = (pdata->voltage - MXT_VOLTAGE_DEFAULT) /
- MXT_VOLTAGE_STEP;
-
- mxt_write_object(data, MXT_SPT_CTECONFIG_T28,
- MXT_CTE_VOLTAGE, voltage);
- }
-}
-
static int mxt_get_info(struct mxt_data *data)
{
struct i2c_client *client = data->client;
@@ -772,7 +981,7 @@
if (object->num_report_ids) {
min_id = reportid;
reportid += object->num_report_ids *
- (object->instances + 1);
+ mxt_obj_instances(object);
max_id = reportid - 1;
} else {
min_id = 0;
@@ -780,13 +989,15 @@
}
dev_dbg(&data->client->dev,
- "Type %2d Start %3d Size %3d Instances %2d ReportIDs %3u : %3u\n",
- object->type, object->start_address, object->size + 1,
- object->instances + 1, min_id, max_id);
+ "T%u Start:%u Size:%zu Instances:%zu Report IDs:%u-%u\n",
+ object->type, object->start_address,
+ mxt_obj_size(object), mxt_obj_instances(object),
+ min_id, max_id);
switch (object->type) {
case MXT_GEN_COMMAND_T6:
data->T6_reportid = min_id;
+ data->T6_address = object->start_address;
break;
case MXT_TOUCH_MULTI_T9:
data->T9_reportid_min = min_id;
@@ -811,12 +1022,59 @@
data->T19_reportid = 0;
}
+static int mxt_read_t9_resolution(struct mxt_data *data)
+{
+ struct i2c_client *client = data->client;
+ int error;
+ struct t9_range range;
+ unsigned char orient;
+ struct mxt_object *object;
+
+ object = mxt_get_object(data, MXT_TOUCH_MULTI_T9);
+ if (!object)
+ return -EINVAL;
+
+ error = __mxt_read_reg(client,
+ object->start_address + MXT_T9_RANGE,
+ sizeof(range), &range);
+ if (error)
+ return error;
+
+ le16_to_cpus(&range.x);
+ le16_to_cpus(&range.y);
+
+ error = __mxt_read_reg(client,
+ object->start_address + MXT_T9_ORIENT,
+ 1, &orient);
+ if (error)
+ return error;
+
+ /* Handle default values */
+ if (range.x == 0)
+ range.x = 1023;
+
+ if (range.y == 0)
+ range.y = 1023;
+
+ if (orient & MXT_T9_ORIENT_SWITCH) {
+ data->max_x = range.y;
+ data->max_y = range.x;
+ } else {
+ data->max_x = range.x;
+ data->max_y = range.y;
+ }
+
+ dev_dbg(&client->dev,
+ "Touchscreen size X%uY%u\n", data->max_x, data->max_y);
+
+ return 0;
+}
+
static int mxt_initialize(struct mxt_data *data)
{
struct i2c_client *client = data->client;
struct mxt_info *info = &data->info;
int error;
- u8 val;
error = mxt_get_info(data);
if (error)
@@ -832,47 +1090,29 @@
/* Get object table information */
error = mxt_get_object_table(data);
- if (error)
+ if (error) {
+ dev_err(&client->dev, "Error %d reading object table\n", error);
goto err_free_object_table;
+ }
/* Check register init values */
error = mxt_check_reg_init(data);
- if (error)
+ if (error) {
+ dev_err(&client->dev, "Error %d initializing configuration\n",
+ error);
goto err_free_object_table;
+ }
- mxt_handle_pdata(data);
-
- /* Backup to memory */
- mxt_write_object(data, MXT_GEN_COMMAND_T6,
- MXT_COMMAND_BACKUPNV,
- MXT_BACKUP_VALUE);
- msleep(MXT_BACKUP_TIME);
-
- /* Soft reset */
- mxt_write_object(data, MXT_GEN_COMMAND_T6,
- MXT_COMMAND_RESET, 1);
- msleep(MXT_RESET_TIME);
-
- /* Update matrix size at info struct */
- error = mxt_read_reg(client, MXT_MATRIX_X_SIZE, &val);
- if (error)
+ error = mxt_read_t9_resolution(data);
+ if (error) {
+ dev_err(&client->dev, "Failed to initialize T9 resolution\n");
goto err_free_object_table;
- info->matrix_xsize = val;
-
- error = mxt_read_reg(client, MXT_MATRIX_Y_SIZE, &val);
- if (error)
- goto err_free_object_table;
- info->matrix_ysize = val;
+ }
dev_info(&client->dev,
- "Family ID: %u Variant ID: %u Major.Minor.Build: %u.%u.%02X\n",
- info->family_id, info->variant_id, info->version >> 4,
- info->version & 0xf, info->build);
-
- dev_info(&client->dev,
- "Matrix X Size: %u Matrix Y Size: %u Object Num: %u\n",
- info->matrix_xsize, info->matrix_ysize,
- info->object_num);
+ "Family: %u Variant: %u Firmware V%u.%u.%02X Objects: %u\n",
+ info->family_id, info->variant_id, info->version >> 4,
+ info->version & 0xf, info->build, info->object_num);
return 0;
@@ -881,20 +1121,6 @@
return error;
}
-static void mxt_calc_resolution(struct mxt_data *data)
-{
- unsigned int max_x = data->pdata->x_size - 1;
- unsigned int max_y = data->pdata->y_size - 1;
-
- if (data->pdata->orient & MXT_XY_SWITCH) {
- data->max_x = max_y;
- data->max_y = max_x;
- } else {
- data->max_x = max_x;
- data->max_y = max_y;
- }
-}
-
/* Firmware Version is returned as Major.Minor.Build */
static ssize_t mxt_fw_version_show(struct device *dev,
struct device_attribute *attr, char *buf)
@@ -921,11 +1147,11 @@
{
int i;
- if (object->instances > 0)
+ if (mxt_obj_instances(object) > 1)
count += scnprintf(buf + count, PAGE_SIZE - count,
"Instance %u\n", instance);
- for (i = 0; i < object->size + 1; i++)
+ for (i = 0; i < mxt_obj_size(object); i++)
count += scnprintf(buf + count, PAGE_SIZE - count,
"\t[%2u]: %02x (%d)\n", i, val[i], val[i]);
count += scnprintf(buf + count, PAGE_SIZE - count, "\n");
@@ -958,8 +1184,8 @@
count += scnprintf(buf + count, PAGE_SIZE - count,
"T%u:\n", object->type);
- for (j = 0; j < object->instances + 1; j++) {
- u16 size = object->size + 1;
+ for (j = 0; j < mxt_obj_instances(object); j++) {
+ u16 size = mxt_obj_size(object);
u16 addr = object->start_address + j * size;
error = __mxt_read_reg(data->client, addr, size, obuf);
@@ -975,13 +1201,38 @@
return error ?: count;
}
+static int mxt_check_firmware_format(struct device *dev,
+ const struct firmware *fw)
+{
+ unsigned int pos = 0;
+ char c;
+
+ while (pos < fw->size) {
+ c = *(fw->data + pos);
+
+ if (c < '0' || (c > '9' && c < 'A') || c > 'F')
+ return 0;
+
+ pos++;
+ }
+
+ /*
+ * To convert file try:
+ * xxd -r -p mXTXXX__APP_VX-X-XX.enc > maxtouch.fw
+ */
+ dev_err(dev, "Aborting: firmware file must be in binary format\n");
+
+ return -EINVAL;
+}
+
static int mxt_load_fw(struct device *dev, const char *fn)
{
struct mxt_data *data = dev_get_drvdata(dev);
- struct i2c_client *client = data->client;
const struct firmware *fw = NULL;
unsigned int frame_size;
unsigned int pos = 0;
+ unsigned int retry = 0;
+ unsigned int frame = 0;
int ret;
ret = request_firmware(&fw, fn, dev);
@@ -990,59 +1241,91 @@
return ret;
}
+ /* Check for incorrect enc file */
+ ret = mxt_check_firmware_format(dev, fw);
+ if (ret)
+ goto release_firmware;
+
+ ret = mxt_lookup_bootloader_address(data);
+ if (ret)
+ goto release_firmware;
+
/* Change to the bootloader mode */
- mxt_write_object(data, MXT_GEN_COMMAND_T6,
- MXT_COMMAND_RESET, MXT_BOOT_VALUE);
+ data->in_bootloader = true;
+
+ ret = mxt_t6_command(data, MXT_COMMAND_RESET, MXT_BOOT_VALUE, false);
+ if (ret)
+ goto release_firmware;
+
msleep(MXT_RESET_TIME);
- /* Change to slave address of bootloader */
- if (client->addr == MXT_APP_LOW)
- client->addr = MXT_BOOT_LOW;
- else
- client->addr = MXT_BOOT_HIGH;
+ reinit_completion(&data->bl_completion);
- ret = mxt_check_bootloader(client, MXT_WAITING_BOOTLOAD_CMD);
+ ret = mxt_check_bootloader(data, MXT_WAITING_BOOTLOAD_CMD);
if (ret)
- goto out;
+ goto disable_irq;
/* Unlock bootloader */
- mxt_unlock_bootloader(client);
+ mxt_unlock_bootloader(data);
while (pos < fw->size) {
- ret = mxt_check_bootloader(client,
- MXT_WAITING_FRAME_DATA);
+ ret = mxt_check_bootloader(data, MXT_WAITING_FRAME_DATA);
if (ret)
- goto out;
+ goto disable_irq;
frame_size = ((*(fw->data + pos) << 8) | *(fw->data + pos + 1));
- /* We should add 2 at frame size as the the firmware data is not
- * included the CRC bytes.
- */
+ /* Take account of CRC bytes */
frame_size += 2;
/* Write one frame to device */
- mxt_fw_write(client, fw->data + pos, frame_size);
-
- ret = mxt_check_bootloader(client,
- MXT_FRAME_CRC_PASS);
+ ret = mxt_bootloader_write(data, fw->data + pos, frame_size);
if (ret)
- goto out;
+ goto disable_irq;
- pos += frame_size;
+ ret = mxt_check_bootloader(data, MXT_FRAME_CRC_PASS);
+ if (ret) {
+ retry++;
- dev_dbg(dev, "Updated %d bytes / %zd bytes\n", pos, fw->size);
+ /* Back off by 20ms per retry */
+ msleep(retry * 20);
+
+ if (retry > 20) {
+ dev_err(dev, "Retry count exceeded\n");
+ goto disable_irq;
+ }
+ } else {
+ retry = 0;
+ pos += frame_size;
+ frame++;
+ }
+
+ if (frame % 50 == 0)
+ dev_dbg(dev, "Sent %d frames, %d/%zd bytes\n",
+ frame, pos, fw->size);
}
-out:
+ /* Wait for flash. */
+ ret = mxt_wait_for_completion(data, &data->bl_completion,
+ MXT_FW_RESET_TIME);
+ if (ret)
+ goto disable_irq;
+
+ dev_dbg(dev, "Sent %d frames, %d bytes\n", frame, pos);
+
+ /*
+ * Wait for device to reset. Some bootloader versions do not assert
+ * the CHG line after bootloading has finished, so ignore potential
+ * errors.
+ */
+ mxt_wait_for_completion(data, &data->bl_completion, MXT_FW_RESET_TIME);
+
+ data->in_bootloader = false;
+
+disable_irq:
+ disable_irq(data->irq);
+release_firmware:
release_firmware(fw);
-
- /* Change to slave address of application */
- if (client->addr == MXT_BOOT_LOW)
- client->addr = MXT_APP_LOW;
- else
- client->addr = MXT_APP_HIGH;
-
return ret;
}
@@ -1053,29 +1336,24 @@
struct mxt_data *data = dev_get_drvdata(dev);
int error;
- disable_irq(data->irq);
-
error = mxt_load_fw(dev, MXT_FW_NAME);
if (error) {
dev_err(dev, "The firmware update failed(%d)\n", error);
count = error;
} else {
- dev_dbg(dev, "The firmware update succeeded\n");
-
- /* Wait for reset */
- msleep(MXT_FWRESET_TIME);
+ dev_info(dev, "The firmware update succeeded\n");
mxt_free_object_table(data);
mxt_initialize(data);
+
+ enable_irq(data->irq);
+
+ error = mxt_make_highchg(data);
+ if (error)
+ return error;
}
- enable_irq(data->irq);
-
- error = mxt_make_highchg(data);
- if (error)
- return error;
-
return count;
}
@@ -1134,6 +1412,8 @@
struct input_dev *input_dev;
int error;
unsigned int num_mt_slots;
+ unsigned int mt_flags = 0;
+ int i;
if (!pdata)
return -EINVAL;
@@ -1146,10 +1426,7 @@
goto err_free_mem;
}
- data->is_tp = pdata && pdata->is_tp;
-
- input_dev->name = (data->is_tp) ? "Atmel maXTouch Touchpad" :
- "Atmel maXTouch Touchscreen";
+ input_dev->name = "Atmel maXTouch Touchscreen";
snprintf(data->phys, sizeof(data->phys), "i2c-%u-%04x/input0",
client->adapter->nr, client->addr);
@@ -1165,7 +1442,9 @@
data->pdata = pdata;
data->irq = client->irq;
- mxt_calc_resolution(data);
+ init_completion(&data->bl_completion);
+ init_completion(&data->reset_completion);
+ init_completion(&data->crc_completion);
error = mxt_initialize(data);
if (error)
@@ -1175,20 +1454,15 @@
__set_bit(EV_KEY, input_dev->evbit);
__set_bit(BTN_TOUCH, input_dev->keybit);
- if (data->is_tp) {
- int i;
- __set_bit(INPUT_PROP_POINTER, input_dev->propbit);
+ if (pdata->t19_num_keys) {
__set_bit(INPUT_PROP_BUTTONPAD, input_dev->propbit);
- for (i = 0; i < MXT_NUM_GPIO; i++)
- if (pdata->key_map[i] != KEY_RESERVED)
- __set_bit(pdata->key_map[i], input_dev->keybit);
+ for (i = 0; i < pdata->t19_num_keys; i++)
+ if (pdata->t19_keymap[i] != KEY_RESERVED)
+ input_set_capability(input_dev, EV_KEY,
+ pdata->t19_keymap[i]);
- __set_bit(BTN_TOOL_FINGER, input_dev->keybit);
- __set_bit(BTN_TOOL_DOUBLETAP, input_dev->keybit);
- __set_bit(BTN_TOOL_TRIPLETAP, input_dev->keybit);
- __set_bit(BTN_TOOL_QUADTAP, input_dev->keybit);
- __set_bit(BTN_TOOL_QUINTTAP, input_dev->keybit);
+ mt_flags |= INPUT_MT_POINTER;
input_abs_set_res(input_dev, ABS_X, MXT_PIXELS_PER_MM);
input_abs_set_res(input_dev, ABS_Y, MXT_PIXELS_PER_MM);
@@ -1196,6 +1470,8 @@
MXT_PIXELS_PER_MM);
input_abs_set_res(input_dev, ABS_MT_POSITION_Y,
MXT_PIXELS_PER_MM);
+
+ input_dev->name = "Atmel maXTouch Touchpad";
}
/* For single touch */
@@ -1208,7 +1484,7 @@
/* For multi touch */
num_mt_slots = data->T9_reportid_max - data->T9_reportid_min + 1;
- error = input_mt_init_slots(input_dev, num_mt_slots, 0);
+ error = input_mt_init_slots(input_dev, num_mt_slots, mt_flags);
if (error)
goto err_free_object;
input_set_abs_params(input_dev, ABS_MT_TOUCH_MAJOR,
@@ -1236,12 +1512,18 @@
goto err_free_irq;
error = input_register_device(input_dev);
- if (error)
+ if (error) {
+ dev_err(&client->dev, "Error %d registering input device\n",
+ error);
goto err_free_irq;
+ }
error = sysfs_create_group(&client->dev.kobj, &mxt_attr_group);
- if (error)
+ if (error) {
+ dev_err(&client->dev, "Failure %d creating sysfs group\n",
+ error);
goto err_unregister_device;
+ }
return 0;
@@ -1294,11 +1576,7 @@
struct mxt_data *data = i2c_get_clientdata(client);
struct input_dev *input_dev = data->input_dev;
- /* Soft reset */
- mxt_write_object(data, MXT_GEN_COMMAND_T6,
- MXT_COMMAND_RESET, 1);
-
- msleep(MXT_RESET_TIME);
+ mxt_soft_reset(data);
mutex_lock(&input_dev->mutex);
diff --git a/drivers/input/touchscreen/auo-pixcir-ts.c b/drivers/input/touchscreen/auo-pixcir-ts.c
index d3f9f6b..7f3c947 100644
--- a/drivers/input/touchscreen/auo-pixcir-ts.c
+++ b/drivers/input/touchscreen/auo-pixcir-ts.c
@@ -679,7 +679,7 @@
MODULE_DEVICE_TABLE(i2c, auo_pixcir_idtable);
#ifdef CONFIG_OF
-static struct of_device_id auo_pixcir_ts_dt_idtable[] = {
+static const struct of_device_id auo_pixcir_ts_dt_idtable[] = {
{ .compatible = "auo,auo_pixcir_ts" },
{},
};
diff --git a/drivers/input/touchscreen/da9034-ts.c b/drivers/input/touchscreen/da9034-ts.c
index 8ccf7bb..cf6f4b3 100644
--- a/drivers/input/touchscreen/da9034-ts.c
+++ b/drivers/input/touchscreen/da9034-ts.c
@@ -301,10 +301,11 @@
struct da9034_touch_pdata *pdata = dev_get_platdata(&pdev->dev);
struct da9034_touch *touch;
struct input_dev *input_dev;
- int ret;
+ int error;
- touch = kzalloc(sizeof(struct da9034_touch), GFP_KERNEL);
- if (touch == NULL) {
+ touch = devm_kzalloc(&pdev->dev, sizeof(struct da9034_touch),
+ GFP_KERNEL);
+ if (!touch) {
dev_err(&pdev->dev, "failed to allocate driver data\n");
return -ENOMEM;
}
@@ -315,18 +316,18 @@
touch->interval_ms = pdata->interval_ms;
touch->x_inverted = pdata->x_inverted;
touch->y_inverted = pdata->y_inverted;
- } else
+ } else {
/* fallback into default */
touch->interval_ms = 10;
+ }
INIT_DELAYED_WORK(&touch->tsi_work, da9034_tsi_work);
touch->notifier.notifier_call = da9034_touch_notifier;
- input_dev = input_allocate_device();
+ input_dev = devm_input_allocate_device(&pdev->dev);
if (!input_dev) {
dev_err(&pdev->dev, "failed to allocate input device\n");
- ret = -ENOMEM;
- goto err_free_touch;
+ return -ENOMEM;
}
input_dev->name = pdev->name;
@@ -346,26 +347,9 @@
touch->input_dev = input_dev;
input_set_drvdata(input_dev, touch);
- ret = input_register_device(input_dev);
- if (ret)
- goto err_free_input;
-
- platform_set_drvdata(pdev, touch);
- return 0;
-
-err_free_input:
- input_free_device(input_dev);
-err_free_touch:
- kfree(touch);
- return ret;
-}
-
-static int da9034_touch_remove(struct platform_device *pdev)
-{
- struct da9034_touch *touch = platform_get_drvdata(pdev);
-
- input_unregister_device(touch->input_dev);
- kfree(touch);
+ error = input_register_device(input_dev);
+ if (error)
+ return error;
return 0;
}
@@ -376,7 +360,6 @@
.owner = THIS_MODULE,
},
.probe = da9034_touch_probe,
- .remove = da9034_touch_remove,
};
module_platform_driver(da9034_touch_driver);
diff --git a/drivers/input/touchscreen/edt-ft5x06.c b/drivers/input/touchscreen/edt-ft5x06.c
index f8815be..d4f3399 100644
--- a/drivers/input/touchscreen/edt-ft5x06.c
+++ b/drivers/input/touchscreen/edt-ft5x06.c
@@ -271,7 +271,7 @@
wrbuf[0] = addr;
wrbuf[1] = value;
- return edt_ft5x06_ts_readwrite(tsdata->client, 3,
+ return edt_ft5x06_ts_readwrite(tsdata->client, 2,
wrbuf, 0, NULL);
default:
diff --git a/drivers/input/touchscreen/egalax_ts.c b/drivers/input/touchscreen/egalax_ts.c
index e6bcb13..c805784 100644
--- a/drivers/input/touchscreen/egalax_ts.c
+++ b/drivers/input/touchscreen/egalax_ts.c
@@ -262,7 +262,7 @@
static SIMPLE_DEV_PM_OPS(egalax_ts_pm_ops, egalax_ts_suspend, egalax_ts_resume);
-static struct of_device_id egalax_ts_dt_ids[] = {
+static const struct of_device_id egalax_ts_dt_ids[] = {
{ .compatible = "eeti,egalax_ts" },
{ /* sentinel */ }
};
diff --git a/drivers/input/touchscreen/intel-mid-touch.c b/drivers/input/touchscreen/intel-mid-touch.c
index 4f6b156..c38ca4a 100644
--- a/drivers/input/touchscreen/intel-mid-touch.c
+++ b/drivers/input/touchscreen/intel-mid-touch.c
@@ -36,6 +36,7 @@
#include <linux/irq.h>
#include <linux/delay.h>
#include <asm/intel_scu_ipc.h>
+#include <linux/device.h>
/* PMIC Interrupt registers */
#define PMIC_REG_ID1 0x00 /* PMIC ID1 register */
@@ -580,12 +581,17 @@
return -EINVAL;
}
- tsdev = kzalloc(sizeof(struct mrstouch_dev), GFP_KERNEL);
- input = input_allocate_device();
- if (!tsdev || !input) {
+ tsdev = devm_kzalloc(&pdev->dev, sizeof(struct mrstouch_dev),
+ GFP_KERNEL);
+ if (!tsdev) {
dev_err(&pdev->dev, "unable to allocate memory\n");
- err = -ENOMEM;
- goto err_free_mem;
+ return -ENOMEM;
+ }
+
+ input = devm_input_allocate_device(&pdev->dev);
+ if (!input) {
+ dev_err(&pdev->dev, "unable to allocate input device\n");
+ return -ENOMEM;
}
tsdev->dev = &pdev->dev;
@@ -598,7 +604,7 @@
err = mrstouch_adc_init(tsdev);
if (err) {
dev_err(&pdev->dev, "ADC initialization failed\n");
- goto err_free_mem;
+ return err;
}
input->name = "mrst_touchscreen";
@@ -618,38 +624,20 @@
input_set_abs_params(tsdev->input, ABS_PRESSURE,
MRST_PRESSURE_MIN, MRST_PRESSURE_MAX, 0, 0);
- err = request_threaded_irq(tsdev->irq, NULL, mrstouch_pendet_irq,
- IRQF_ONESHOT, "mrstouch", tsdev);
+ err = devm_request_threaded_irq(&pdev->dev, tsdev->irq, NULL,
+ mrstouch_pendet_irq, IRQF_ONESHOT,
+ "mrstouch", tsdev);
if (err) {
dev_err(tsdev->dev, "unable to allocate irq\n");
- goto err_free_mem;
+ return err;
}
err = input_register_device(tsdev->input);
if (err) {
dev_err(tsdev->dev, "unable to register input device\n");
- goto err_free_irq;
+ return err;
}
- platform_set_drvdata(pdev, tsdev);
- return 0;
-
-err_free_irq:
- free_irq(tsdev->irq, tsdev);
-err_free_mem:
- input_free_device(input);
- kfree(tsdev);
- return err;
-}
-
-static int mrstouch_remove(struct platform_device *pdev)
-{
- struct mrstouch_dev *tsdev = platform_get_drvdata(pdev);
-
- free_irq(tsdev->irq, tsdev);
- input_unregister_device(tsdev->input);
- kfree(tsdev);
-
return 0;
}
@@ -659,7 +647,6 @@
.owner = THIS_MODULE,
},
.probe = mrstouch_probe,
- .remove = mrstouch_remove,
};
module_platform_driver(mrstouch_driver);
diff --git a/drivers/input/touchscreen/lpc32xx_ts.c b/drivers/input/touchscreen/lpc32xx_ts.c
index 2058253..bb47d34 100644
--- a/drivers/input/touchscreen/lpc32xx_ts.c
+++ b/drivers/input/touchscreen/lpc32xx_ts.c
@@ -384,7 +384,7 @@
#endif
#ifdef CONFIG_OF
-static struct of_device_id lpc32xx_tsc_of_match[] = {
+static const struct of_device_id lpc32xx_tsc_of_match[] = {
{ .compatible = "nxp,lpc3220-tsc", },
{ },
};
diff --git a/drivers/input/touchscreen/mcs5000_ts.c b/drivers/input/touchscreen/mcs5000_ts.c
index 647e36f5..00510a9 100644
--- a/drivers/input/touchscreen/mcs5000_ts.c
+++ b/drivers/input/touchscreen/mcs5000_ts.c
@@ -161,10 +161,9 @@
return IRQ_HANDLED;
}
-static void mcs5000_ts_phys_init(struct mcs5000_ts_data *data)
+static void mcs5000_ts_phys_init(struct mcs5000_ts_data *data,
+ const struct mcs_platform_data *platform_data)
{
- const struct mcs_platform_data *platform_data =
- data->platform_data;
struct i2c_client *client = data->client;
/* Touch reset & sleep mode */
@@ -187,28 +186,32 @@
}
static int mcs5000_ts_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+ const struct i2c_device_id *id)
{
+ const struct mcs_platform_data *pdata;
struct mcs5000_ts_data *data;
struct input_dev *input_dev;
- int ret;
+ int error;
- if (!dev_get_platdata(&client->dev))
+ pdata = dev_get_platdata(&client->dev);
+ if (!pdata)
return -EINVAL;
- data = kzalloc(sizeof(struct mcs5000_ts_data), GFP_KERNEL);
- input_dev = input_allocate_device();
- if (!data || !input_dev) {
+ data = devm_kzalloc(&client->dev, sizeof(*data), GFP_KERNEL);
+ if (!data) {
dev_err(&client->dev, "Failed to allocate memory\n");
- ret = -ENOMEM;
- goto err_free_mem;
+ return -ENOMEM;
}
data->client = client;
- data->input_dev = input_dev;
- data->platform_data = dev_get_platdata(&client->dev);
- input_dev->name = "MELPAS MCS-5000 Touchscreen";
+ input_dev = devm_input_allocate_device(&client->dev);
+ if (!input_dev) {
+ dev_err(&client->dev, "Failed to allocate input device\n");
+ return -ENOMEM;
+ }
+
+ input_dev->name = "MELFAS MCS-5000 Touchscreen";
input_dev->id.bustype = BUS_I2C;
input_dev->dev.parent = &client->dev;
@@ -219,44 +222,30 @@
input_set_abs_params(input_dev, ABS_Y, 0, MCS5000_MAX_YC, 0, 0);
input_set_drvdata(input_dev, data);
+ data->input_dev = input_dev;
- if (data->platform_data->cfg_pin)
- data->platform_data->cfg_pin();
+ if (pdata->cfg_pin)
+ pdata->cfg_pin();
- ret = request_threaded_irq(client->irq, NULL, mcs5000_ts_interrupt,
- IRQF_TRIGGER_LOW | IRQF_ONESHOT, "mcs5000_ts", data);
-
- if (ret < 0) {
+ error = devm_request_threaded_irq(&client->dev, client->irq,
+ NULL, mcs5000_ts_interrupt,
+ IRQF_TRIGGER_LOW | IRQF_ONESHOT,
+ "mcs5000_ts", data);
+ if (error) {
dev_err(&client->dev, "Failed to register interrupt\n");
- goto err_free_mem;
+ return error;
}
- ret = input_register_device(data->input_dev);
- if (ret < 0)
- goto err_free_irq;
+ error = input_register_device(data->input_dev);
+ if (error) {
+ dev_err(&client->dev, "Failed to register input device\n");
+ return error;
+ }
- mcs5000_ts_phys_init(data);
+ mcs5000_ts_phys_init(data, pdata);
i2c_set_clientdata(client, data);
return 0;
-
-err_free_irq:
- free_irq(client->irq, data);
-err_free_mem:
- input_free_device(input_dev);
- kfree(data);
- return ret;
-}
-
-static int mcs5000_ts_remove(struct i2c_client *client)
-{
- struct mcs5000_ts_data *data = i2c_get_clientdata(client);
-
- free_irq(client->irq, data);
- input_unregister_device(data->input_dev);
- kfree(data);
-
- return 0;
}
#ifdef CONFIG_PM
@@ -274,14 +263,15 @@
{
struct i2c_client *client = to_i2c_client(dev);
struct mcs5000_ts_data *data = i2c_get_clientdata(client);
+ const struct mcs_platform_data *pdata = dev_get_platdata(dev);
- mcs5000_ts_phys_init(data);
+ mcs5000_ts_phys_init(data, pdata);
return 0;
}
+#endif
static SIMPLE_DEV_PM_OPS(mcs5000_ts_pm, mcs5000_ts_suspend, mcs5000_ts_resume);
-#endif
static const struct i2c_device_id mcs5000_ts_id[] = {
{ "mcs5000_ts", 0 },
@@ -291,12 +281,9 @@
static struct i2c_driver mcs5000_ts_driver = {
.probe = mcs5000_ts_probe,
- .remove = mcs5000_ts_remove,
.driver = {
.name = "mcs5000_ts",
-#ifdef CONFIG_PM
.pm = &mcs5000_ts_pm,
-#endif
},
.id_table = mcs5000_ts_id,
};
diff --git a/drivers/input/touchscreen/mms114.c b/drivers/input/touchscreen/mms114.c
index 8a598c0..372bbf7 100644
--- a/drivers/input/touchscreen/mms114.c
+++ b/drivers/input/touchscreen/mms114.c
@@ -456,7 +456,7 @@
data->input_dev = input_dev;
data->pdata = pdata;
- input_dev->name = "MELPAS MMS114 Touchscreen";
+ input_dev->name = "MELFAS MMS114 Touchscreen";
input_dev->id.bustype = BUS_I2C;
input_dev->dev.parent = &client->dev;
input_dev->open = mms114_input_open;
@@ -570,7 +570,7 @@
MODULE_DEVICE_TABLE(i2c, mms114_id);
#ifdef CONFIG_OF
-static struct of_device_id mms114_dt_match[] = {
+static const struct of_device_id mms114_dt_match[] = {
{ .compatible = "melfas,mms114" },
{ }
};
diff --git a/drivers/input/touchscreen/of_touchscreen.c b/drivers/input/touchscreen/of_touchscreen.c
new file mode 100644
index 0000000..f8f9b84
--- /dev/null
+++ b/drivers/input/touchscreen/of_touchscreen.c
@@ -0,0 +1,45 @@
+/*
+ * Generic DT helper functions for touchscreen devices
+ *
+ * Copyright (c) 2014 Sebastian Reichel <sre@kernel.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/of.h>
+#include <linux/input.h>
+#include <linux/input/touchscreen.h>
+
+/**
+ * touchscreen_parse_of_params - parse common touchscreen DT properties
+ * @dev: device that should be parsed
+ *
+ * This function parses common DT properties for touchscreens and setups the
+ * input device accordingly. The function keeps previously setuped default
+ * values if no value is specified via DT.
+ */
+void touchscreen_parse_of_params(struct input_dev *dev)
+{
+ struct device_node *np = dev->dev.parent->of_node;
+ struct input_absinfo *absinfo;
+
+ input_alloc_absinfo(dev);
+ if (!dev->absinfo)
+ return;
+
+ absinfo = &dev->absinfo[ABS_X];
+ of_property_read_u32(np, "touchscreen-size-x", &absinfo->maximum);
+ of_property_read_u32(np, "touchscreen-fuzz-x", &absinfo->fuzz);
+
+ absinfo = &dev->absinfo[ABS_Y];
+ of_property_read_u32(np, "touchscreen-size-y", &absinfo->maximum);
+ of_property_read_u32(np, "touchscreen-fuzz-y", &absinfo->fuzz);
+
+ absinfo = &dev->absinfo[ABS_PRESSURE];
+ of_property_read_u32(np, "touchscreen-max-pressure", &absinfo->maximum);
+ of_property_read_u32(np, "touchscreen-fuzz-pressure", &absinfo->fuzz);
+}
+EXPORT_SYMBOL(touchscreen_parse_of_params);
diff --git a/drivers/input/touchscreen/pixcir_i2c_ts.c b/drivers/input/touchscreen/pixcir_i2c_ts.c
index 02392d2..19c6c0f 100644
--- a/drivers/input/touchscreen/pixcir_i2c_ts.c
+++ b/drivers/input/touchscreen/pixcir_i2c_ts.c
@@ -24,12 +24,13 @@
#include <linux/i2c.h>
#include <linux/input.h>
#include <linux/input/pixcir_ts.h>
+#include <linux/gpio.h>
struct pixcir_i2c_ts_data {
struct i2c_client *client;
struct input_dev *input;
const struct pixcir_ts_platform_data *chip;
- bool exiting;
+ bool running;
};
static void pixcir_ts_poscheck(struct pixcir_i2c_ts_data *data)
@@ -87,11 +88,12 @@
static irqreturn_t pixcir_ts_isr(int irq, void *dev_id)
{
struct pixcir_i2c_ts_data *tsdata = dev_id;
+ const struct pixcir_ts_platform_data *pdata = tsdata->chip;
- while (!tsdata->exiting) {
+ while (tsdata->running) {
pixcir_ts_poscheck(tsdata);
- if (tsdata->chip->attb_read_val())
+ if (gpio_get_value(pdata->gpio_attb))
break;
msleep(20);
@@ -100,25 +102,221 @@
return IRQ_HANDLED;
}
+static int pixcir_set_power_mode(struct pixcir_i2c_ts_data *ts,
+ enum pixcir_power_mode mode)
+{
+ struct device *dev = &ts->client->dev;
+ int ret;
+
+ ret = i2c_smbus_read_byte_data(ts->client, PIXCIR_REG_POWER_MODE);
+ if (ret < 0) {
+ dev_err(dev, "%s: can't read reg 0x%x : %d\n",
+ __func__, PIXCIR_REG_POWER_MODE, ret);
+ return ret;
+ }
+
+ ret &= ~PIXCIR_POWER_MODE_MASK;
+ ret |= mode;
+
+ /* Always AUTO_IDLE */
+ ret |= PIXCIR_POWER_ALLOW_IDLE;
+
+ ret = i2c_smbus_write_byte_data(ts->client, PIXCIR_REG_POWER_MODE, ret);
+ if (ret < 0) {
+ dev_err(dev, "%s: can't write reg 0x%x : %d\n",
+ __func__, PIXCIR_REG_POWER_MODE, ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+/*
+ * Set the interrupt mode for the device i.e. ATTB line behaviour
+ *
+ * @polarity : 1 for active high, 0 for active low.
+ */
+static int pixcir_set_int_mode(struct pixcir_i2c_ts_data *ts,
+ enum pixcir_int_mode mode, bool polarity)
+{
+ struct device *dev = &ts->client->dev;
+ int ret;
+
+ ret = i2c_smbus_read_byte_data(ts->client, PIXCIR_REG_INT_MODE);
+ if (ret < 0) {
+ dev_err(dev, "%s: can't read reg 0x%x : %d\n",
+ __func__, PIXCIR_REG_INT_MODE, ret);
+ return ret;
+ }
+
+ ret &= ~PIXCIR_INT_MODE_MASK;
+ ret |= mode;
+
+ if (polarity)
+ ret |= PIXCIR_INT_POL_HIGH;
+ else
+ ret &= ~PIXCIR_INT_POL_HIGH;
+
+ ret = i2c_smbus_write_byte_data(ts->client, PIXCIR_REG_INT_MODE, ret);
+ if (ret < 0) {
+ dev_err(dev, "%s: can't write reg 0x%x : %d\n",
+ __func__, PIXCIR_REG_INT_MODE, ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+/*
+ * Enable/disable interrupt generation
+ */
+static int pixcir_int_enable(struct pixcir_i2c_ts_data *ts, bool enable)
+{
+ struct device *dev = &ts->client->dev;
+ int ret;
+
+ ret = i2c_smbus_read_byte_data(ts->client, PIXCIR_REG_INT_MODE);
+ if (ret < 0) {
+ dev_err(dev, "%s: can't read reg 0x%x : %d\n",
+ __func__, PIXCIR_REG_INT_MODE, ret);
+ return ret;
+ }
+
+ if (enable)
+ ret |= PIXCIR_INT_ENABLE;
+ else
+ ret &= ~PIXCIR_INT_ENABLE;
+
+ ret = i2c_smbus_write_byte_data(ts->client, PIXCIR_REG_INT_MODE, ret);
+ if (ret < 0) {
+ dev_err(dev, "%s: can't write reg 0x%x : %d\n",
+ __func__, PIXCIR_REG_INT_MODE, ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int pixcir_start(struct pixcir_i2c_ts_data *ts)
+{
+ struct device *dev = &ts->client->dev;
+ int error;
+
+ /* LEVEL_TOUCH interrupt with active low polarity */
+ error = pixcir_set_int_mode(ts, PIXCIR_INT_LEVEL_TOUCH, 0);
+ if (error) {
+ dev_err(dev, "Failed to set interrupt mode: %d\n", error);
+ return error;
+ }
+
+ ts->running = true;
+ mb(); /* Update status before IRQ can fire */
+
+ /* enable interrupt generation */
+ error = pixcir_int_enable(ts, true);
+ if (error) {
+ dev_err(dev, "Failed to enable interrupt generation: %d\n",
+ error);
+ return error;
+ }
+
+ return 0;
+}
+
+static int pixcir_stop(struct pixcir_i2c_ts_data *ts)
+{
+ int error;
+
+ /* Disable interrupt generation */
+ error = pixcir_int_enable(ts, false);
+ if (error) {
+ dev_err(&ts->client->dev,
+ "Failed to disable interrupt generation: %d\n",
+ error);
+ return error;
+ }
+
+ /* Exit ISR if running, no more report parsing */
+ ts->running = false;
+ mb(); /* update status before we synchronize irq */
+
+ /* Wait till running ISR is complete */
+ synchronize_irq(ts->client->irq);
+
+ return 0;
+}
+
+static int pixcir_input_open(struct input_dev *dev)
+{
+ struct pixcir_i2c_ts_data *ts = input_get_drvdata(dev);
+
+ return pixcir_start(ts);
+}
+
+static void pixcir_input_close(struct input_dev *dev)
+{
+ struct pixcir_i2c_ts_data *ts = input_get_drvdata(dev);
+
+ pixcir_stop(ts);
+}
+
#ifdef CONFIG_PM_SLEEP
static int pixcir_i2c_ts_suspend(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
+ struct pixcir_i2c_ts_data *ts = i2c_get_clientdata(client);
+ struct input_dev *input = ts->input;
+ int ret = 0;
- if (device_may_wakeup(&client->dev))
+ mutex_lock(&input->mutex);
+
+ if (device_may_wakeup(&client->dev)) {
+ if (!input->users) {
+ ret = pixcir_start(ts);
+ if (ret) {
+ dev_err(dev, "Failed to start\n");
+ goto unlock;
+ }
+ }
+
enable_irq_wake(client->irq);
+ } else if (input->users) {
+ ret = pixcir_stop(ts);
+ }
- return 0;
+unlock:
+ mutex_unlock(&input->mutex);
+
+ return ret;
}
static int pixcir_i2c_ts_resume(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
+ struct pixcir_i2c_ts_data *ts = i2c_get_clientdata(client);
+ struct input_dev *input = ts->input;
+ int ret = 0;
- if (device_may_wakeup(&client->dev))
+ mutex_lock(&input->mutex);
+
+ if (device_may_wakeup(&client->dev)) {
disable_irq_wake(client->irq);
- return 0;
+ if (!input->users) {
+ ret = pixcir_stop(ts);
+ if (ret) {
+ dev_err(dev, "Failed to stop\n");
+ goto unlock;
+ }
+ }
+ } else if (input->users) {
+ ret = pixcir_start(ts);
+ }
+
+unlock:
+ mutex_unlock(&input->mutex);
+
+ return ret;
}
#endif
@@ -130,6 +328,7 @@
{
const struct pixcir_ts_platform_data *pdata =
dev_get_platdata(&client->dev);
+ struct device *dev = &client->dev;
struct pixcir_i2c_ts_data *tsdata;
struct input_dev *input;
int error;
@@ -139,12 +338,19 @@
return -EINVAL;
}
- tsdata = kzalloc(sizeof(*tsdata), GFP_KERNEL);
- input = input_allocate_device();
- if (!tsdata || !input) {
- dev_err(&client->dev, "Failed to allocate driver data!\n");
- error = -ENOMEM;
- goto err_free_mem;
+ if (!gpio_is_valid(pdata->gpio_attb)) {
+ dev_err(dev, "Invalid gpio_attb in pdata\n");
+ return -EINVAL;
+ }
+
+ tsdata = devm_kzalloc(dev, sizeof(*tsdata), GFP_KERNEL);
+ if (!tsdata)
+ return -ENOMEM;
+
+ input = devm_input_allocate_device(dev);
+ if (!input) {
+ dev_err(dev, "Failed to allocate input device\n");
+ return -ENOMEM;
}
tsdata->client = client;
@@ -153,6 +359,8 @@
input->name = client->name;
input->id.bustype = BUS_I2C;
+ input->open = pixcir_input_open;
+ input->close = pixcir_input_close;
input->dev.parent = &client->dev;
__set_bit(EV_KEY, input->evbit);
@@ -165,44 +373,47 @@
input_set_drvdata(input, tsdata);
- error = request_threaded_irq(client->irq, NULL, pixcir_ts_isr,
- IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
- client->name, tsdata);
+ error = devm_gpio_request_one(dev, pdata->gpio_attb,
+ GPIOF_DIR_IN, "pixcir_i2c_attb");
if (error) {
- dev_err(&client->dev, "Unable to request touchscreen IRQ.\n");
- goto err_free_mem;
+ dev_err(dev, "Failed to request ATTB gpio\n");
+ return error;
}
+ error = devm_request_threaded_irq(dev, client->irq, NULL, pixcir_ts_isr,
+ IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
+ client->name, tsdata);
+ if (error) {
+ dev_err(dev, "failed to request irq %d\n", client->irq);
+ return error;
+ }
+
+ /* Always be in IDLE mode to save power, device supports auto wake */
+ error = pixcir_set_power_mode(tsdata, PIXCIR_POWER_IDLE);
+ if (error) {
+ dev_err(dev, "Failed to set IDLE mode\n");
+ return error;
+ }
+
+ /* Stop device till opened */
+ error = pixcir_stop(tsdata);
+ if (error)
+ return error;
+
error = input_register_device(input);
if (error)
- goto err_free_irq;
+ return error;
i2c_set_clientdata(client, tsdata);
device_init_wakeup(&client->dev, 1);
return 0;
-
-err_free_irq:
- free_irq(client->irq, tsdata);
-err_free_mem:
- input_free_device(input);
- kfree(tsdata);
- return error;
}
static int pixcir_i2c_ts_remove(struct i2c_client *client)
{
- struct pixcir_i2c_ts_data *tsdata = i2c_get_clientdata(client);
-
device_init_wakeup(&client->dev, 0);
- tsdata->exiting = true;
- mb();
- free_irq(client->irq, tsdata);
-
- input_unregister_device(tsdata->input);
- kfree(tsdata);
-
return 0;
}
diff --git a/drivers/input/touchscreen/sun4i-ts.c b/drivers/input/touchscreen/sun4i-ts.c
new file mode 100644
index 0000000..2ba8260
--- /dev/null
+++ b/drivers/input/touchscreen/sun4i-ts.c
@@ -0,0 +1,339 @@
+/*
+ * Allwinner sunxi resistive touchscreen controller driver
+ *
+ * Copyright (C) 2013 - 2014 Hans de Goede <hdegoede@redhat.com>
+ *
+ * The hwmon parts are based on work by Corentin LABBE which is:
+ * Copyright (C) 2013 Corentin LABBE <clabbe.montjoie@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+/*
+ * The sun4i-ts controller is capable of detecting a second touch, but when a
+ * second touch is present then the accuracy becomes so bad the reported touch
+ * location is not useable.
+ *
+ * The original android driver contains some complicated heuristics using the
+ * aprox. distance between the 2 touches to see if the user is making a pinch
+ * open / close movement, and then reports emulated multi-touch events around
+ * the last touch coordinate (as the dual-touch coordinates are worthless).
+ *
+ * These kinds of heuristics are just asking for trouble (and don't belong
+ * in the kernel). So this driver offers straight forward, reliable single
+ * touch functionality only.
+ */
+
+#include <linux/err.h>
+#include <linux/hwmon.h>
+#include <linux/init.h>
+#include <linux/input.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+
+#define TP_CTRL0 0x00
+#define TP_CTRL1 0x04
+#define TP_CTRL2 0x08
+#define TP_CTRL3 0x0c
+#define TP_INT_FIFOC 0x10
+#define TP_INT_FIFOS 0x14
+#define TP_TPR 0x18
+#define TP_CDAT 0x1c
+#define TEMP_DATA 0x20
+#define TP_DATA 0x24
+
+/* TP_CTRL0 bits */
+#define ADC_FIRST_DLY(x) ((x) << 24) /* 8 bits */
+#define ADC_FIRST_DLY_MODE(x) ((x) << 23)
+#define ADC_CLK_SEL(x) ((x) << 22)
+#define ADC_CLK_DIV(x) ((x) << 20) /* 3 bits */
+#define FS_DIV(x) ((x) << 16) /* 4 bits */
+#define T_ACQ(x) ((x) << 0) /* 16 bits */
+
+/* TP_CTRL1 bits */
+#define STYLUS_UP_DEBOUN(x) ((x) << 12) /* 8 bits */
+#define STYLUS_UP_DEBOUN_EN(x) ((x) << 9)
+#define TOUCH_PAN_CALI_EN(x) ((x) << 6)
+#define TP_DUAL_EN(x) ((x) << 5)
+#define TP_MODE_EN(x) ((x) << 4)
+#define TP_ADC_SELECT(x) ((x) << 3)
+#define ADC_CHAN_SELECT(x) ((x) << 0) /* 3 bits */
+
+/* TP_CTRL2 bits */
+#define TP_SENSITIVE_ADJUST(x) ((x) << 28) /* 4 bits */
+#define TP_MODE_SELECT(x) ((x) << 26) /* 2 bits */
+#define PRE_MEA_EN(x) ((x) << 24)
+#define PRE_MEA_THRE_CNT(x) ((x) << 0) /* 24 bits */
+
+/* TP_CTRL3 bits */
+#define FILTER_EN(x) ((x) << 2)
+#define FILTER_TYPE(x) ((x) << 0) /* 2 bits */
+
+/* TP_INT_FIFOC irq and fifo mask / control bits */
+#define TEMP_IRQ_EN(x) ((x) << 18)
+#define OVERRUN_IRQ_EN(x) ((x) << 17)
+#define DATA_IRQ_EN(x) ((x) << 16)
+#define TP_DATA_XY_CHANGE(x) ((x) << 13)
+#define FIFO_TRIG(x) ((x) << 8) /* 5 bits */
+#define DATA_DRQ_EN(x) ((x) << 7)
+#define FIFO_FLUSH(x) ((x) << 4)
+#define TP_UP_IRQ_EN(x) ((x) << 1)
+#define TP_DOWN_IRQ_EN(x) ((x) << 0)
+
+/* TP_INT_FIFOS irq and fifo status bits */
+#define TEMP_DATA_PENDING BIT(18)
+#define FIFO_OVERRUN_PENDING BIT(17)
+#define FIFO_DATA_PENDING BIT(16)
+#define TP_IDLE_FLG BIT(2)
+#define TP_UP_PENDING BIT(1)
+#define TP_DOWN_PENDING BIT(0)
+
+/* TP_TPR bits */
+#define TEMP_ENABLE(x) ((x) << 16)
+#define TEMP_PERIOD(x) ((x) << 0) /* t = x * 256 * 16 / clkin */
+
+struct sun4i_ts_data {
+ struct device *dev;
+ struct input_dev *input;
+ void __iomem *base;
+ unsigned int irq;
+ bool ignore_fifo_data;
+ int temp_data;
+};
+
+static void sun4i_ts_irq_handle_input(struct sun4i_ts_data *ts, u32 reg_val)
+{
+ u32 x, y;
+
+ if (reg_val & FIFO_DATA_PENDING) {
+ x = readl(ts->base + TP_DATA);
+ y = readl(ts->base + TP_DATA);
+ /* The 1st location reported after an up event is unreliable */
+ if (!ts->ignore_fifo_data) {
+ input_report_abs(ts->input, ABS_X, x);
+ input_report_abs(ts->input, ABS_Y, y);
+ /*
+ * The hardware has a separate down status bit, but
+ * that gets set before we get the first location,
+ * resulting in reporting a click on the old location.
+ */
+ input_report_key(ts->input, BTN_TOUCH, 1);
+ input_sync(ts->input);
+ } else {
+ ts->ignore_fifo_data = false;
+ }
+ }
+
+ if (reg_val & TP_UP_PENDING) {
+ ts->ignore_fifo_data = true;
+ input_report_key(ts->input, BTN_TOUCH, 0);
+ input_sync(ts->input);
+ }
+}
+
+static irqreturn_t sun4i_ts_irq(int irq, void *dev_id)
+{
+ struct sun4i_ts_data *ts = dev_id;
+ u32 reg_val;
+
+ reg_val = readl(ts->base + TP_INT_FIFOS);
+
+ if (reg_val & TEMP_DATA_PENDING)
+ ts->temp_data = readl(ts->base + TEMP_DATA);
+
+ if (ts->input)
+ sun4i_ts_irq_handle_input(ts, reg_val);
+
+ writel(reg_val, ts->base + TP_INT_FIFOS);
+
+ return IRQ_HANDLED;
+}
+
+static int sun4i_ts_open(struct input_dev *dev)
+{
+ struct sun4i_ts_data *ts = input_get_drvdata(dev);
+
+ /* Flush, set trig level to 1, enable temp, data and up irqs */
+ writel(TEMP_IRQ_EN(1) | DATA_IRQ_EN(1) | FIFO_TRIG(1) | FIFO_FLUSH(1) |
+ TP_UP_IRQ_EN(1), ts->base + TP_INT_FIFOC);
+
+ return 0;
+}
+
+static void sun4i_ts_close(struct input_dev *dev)
+{
+ struct sun4i_ts_data *ts = input_get_drvdata(dev);
+
+ /* Deactivate all input IRQs */
+ writel(TEMP_IRQ_EN(1), ts->base + TP_INT_FIFOC);
+}
+
+static ssize_t show_temp(struct device *dev, struct device_attribute *devattr,
+ char *buf)
+{
+ struct sun4i_ts_data *ts = dev_get_drvdata(dev);
+
+ /* No temp_data until the first irq */
+ if (ts->temp_data == -1)
+ return -EAGAIN;
+
+ return sprintf(buf, "%d\n", (ts->temp_data - 1447) * 100);
+}
+
+static ssize_t show_temp_label(struct device *dev,
+ struct device_attribute *devattr, char *buf)
+{
+ return sprintf(buf, "SoC temperature\n");
+}
+
+static DEVICE_ATTR(temp1_input, S_IRUGO, show_temp, NULL);
+static DEVICE_ATTR(temp1_label, S_IRUGO, show_temp_label, NULL);
+
+static struct attribute *sun4i_ts_attrs[] = {
+ &dev_attr_temp1_input.attr,
+ &dev_attr_temp1_label.attr,
+ NULL
+};
+ATTRIBUTE_GROUPS(sun4i_ts);
+
+static int sun4i_ts_probe(struct platform_device *pdev)
+{
+ struct sun4i_ts_data *ts;
+ struct device *dev = &pdev->dev;
+ struct device_node *np = dev->of_node;
+ struct device *hwmon;
+ int error;
+ bool ts_attached;
+
+ ts = devm_kzalloc(dev, sizeof(struct sun4i_ts_data), GFP_KERNEL);
+ if (!ts)
+ return -ENOMEM;
+
+ ts->dev = dev;
+ ts->ignore_fifo_data = true;
+ ts->temp_data = -1;
+
+ ts_attached = of_property_read_bool(np, "allwinner,ts-attached");
+ if (ts_attached) {
+ ts->input = devm_input_allocate_device(dev);
+ if (!ts->input)
+ return -ENOMEM;
+
+ ts->input->name = pdev->name;
+ ts->input->phys = "sun4i_ts/input0";
+ ts->input->open = sun4i_ts_open;
+ ts->input->close = sun4i_ts_close;
+ ts->input->id.bustype = BUS_HOST;
+ ts->input->id.vendor = 0x0001;
+ ts->input->id.product = 0x0001;
+ ts->input->id.version = 0x0100;
+ ts->input->evbit[0] = BIT(EV_SYN) | BIT(EV_KEY) | BIT(EV_ABS);
+ __set_bit(BTN_TOUCH, ts->input->keybit);
+ input_set_abs_params(ts->input, ABS_X, 0, 4095, 0, 0);
+ input_set_abs_params(ts->input, ABS_Y, 0, 4095, 0, 0);
+ input_set_drvdata(ts->input, ts);
+ }
+
+ ts->base = devm_ioremap_resource(dev,
+ platform_get_resource(pdev, IORESOURCE_MEM, 0));
+ if (IS_ERR(ts->base))
+ return PTR_ERR(ts->base);
+
+ ts->irq = platform_get_irq(pdev, 0);
+ error = devm_request_irq(dev, ts->irq, sun4i_ts_irq, 0, "sun4i-ts", ts);
+ if (error)
+ return error;
+
+ /*
+ * Select HOSC clk, clkin = clk / 6, adc samplefreq = clkin / 8192,
+ * t_acq = clkin / (16 * 64)
+ */
+ writel(ADC_CLK_SEL(0) | ADC_CLK_DIV(2) | FS_DIV(7) | T_ACQ(63),
+ ts->base + TP_CTRL0);
+
+ /*
+ * sensitive_adjust = 15 : max, which is not all that sensitive,
+ * tp_mode = 0 : only x and y coordinates, as we don't use dual touch
+ */
+ writel(TP_SENSITIVE_ADJUST(15) | TP_MODE_SELECT(0),
+ ts->base + TP_CTRL2);
+
+ /* Enable median filter, type 1 : 5/3 */
+ writel(FILTER_EN(1) | FILTER_TYPE(1), ts->base + TP_CTRL3);
+
+ /* Enable temperature measurement, period 1953 (2 seconds) */
+ writel(TEMP_ENABLE(1) | TEMP_PERIOD(1953), ts->base + TP_TPR);
+
+ /*
+ * Set stylus up debounce to aprox 10 ms, enable debounce, and
+ * finally enable tp mode.
+ */
+ writel(STYLUS_UP_DEBOUN(5) | STYLUS_UP_DEBOUN_EN(1) | TP_MODE_EN(1),
+ ts->base + TP_CTRL1);
+
+ hwmon = devm_hwmon_device_register_with_groups(ts->dev, "sun4i_ts",
+ ts, sun4i_ts_groups);
+ if (IS_ERR(hwmon))
+ return PTR_ERR(hwmon);
+
+ writel(TEMP_IRQ_EN(1), ts->base + TP_INT_FIFOC);
+
+ if (ts_attached) {
+ error = input_register_device(ts->input);
+ if (error) {
+ writel(0, ts->base + TP_INT_FIFOC);
+ return error;
+ }
+ }
+
+ platform_set_drvdata(pdev, ts);
+ return 0;
+}
+
+static int sun4i_ts_remove(struct platform_device *pdev)
+{
+ struct sun4i_ts_data *ts = platform_get_drvdata(pdev);
+
+ /* Explicit unregister to avoid open/close changing the imask later */
+ if (ts->input)
+ input_unregister_device(ts->input);
+
+ /* Deactivate all IRQs */
+ writel(0, ts->base + TP_INT_FIFOC);
+
+ return 0;
+}
+
+static const struct of_device_id sun4i_ts_of_match[] = {
+ { .compatible = "allwinner,sun4i-a10-ts", },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, sun4i_ts_of_match);
+
+static struct platform_driver sun4i_ts_driver = {
+ .driver = {
+ .owner = THIS_MODULE,
+ .name = "sun4i-ts",
+ .of_match_table = of_match_ptr(sun4i_ts_of_match),
+ },
+ .probe = sun4i_ts_probe,
+ .remove = sun4i_ts_remove,
+};
+
+module_platform_driver(sun4i_ts_driver);
+
+MODULE_DESCRIPTION("Allwinner sun4i resistive touchscreen controller driver");
+MODULE_AUTHOR("Hans de Goede <hdegoede@redhat.com>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/input/touchscreen/tsc2005.c b/drivers/input/touchscreen/tsc2005.c
index 550adcb..52380b6 100644
--- a/drivers/input/touchscreen/tsc2005.c
+++ b/drivers/input/touchscreen/tsc2005.c
@@ -25,11 +25,15 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/input.h>
+#include <linux/input/touchscreen.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/pm.h>
+#include <linux/of.h>
+#include <linux/of_gpio.h>
#include <linux/spi/spi.h>
#include <linux/spi/tsc2005.h>
+#include <linux/regulator/consumer.h>
/*
* The touchscreen interface operates as follows:
@@ -100,6 +104,11 @@
TSC2005_CFR2_AVG_7)
#define MAX_12BIT 0xfff
+#define TSC2005_DEF_X_FUZZ 4
+#define TSC2005_DEF_Y_FUZZ 8
+#define TSC2005_DEF_P_FUZZ 2
+#define TSC2005_DEF_RESISTOR 280
+
#define TSC2005_SPI_MAX_SPEED_HZ 10000000
#define TSC2005_PENUP_TIME_MS 40
@@ -143,6 +152,9 @@
bool pen_down;
+ struct regulator *vio;
+
+ int reset_gpio;
void (*set_reset)(bool enable);
};
@@ -337,6 +349,14 @@
tsc2005_cmd(ts, TSC2005_CMD_STOP);
}
+static void tsc2005_set_reset(struct tsc2005 *ts, bool enable)
+{
+ if (ts->reset_gpio >= 0)
+ gpio_set_value(ts->reset_gpio, enable);
+ else if (ts->set_reset)
+ ts->set_reset(enable);
+}
+
/* must be called with ts->mutex held */
static void __tsc2005_disable(struct tsc2005 *ts)
{
@@ -355,7 +375,7 @@
{
tsc2005_start_scan(ts);
- if (ts->esd_timeout && ts->set_reset) {
+ if (ts->esd_timeout && (ts->set_reset || ts->reset_gpio)) {
ts->last_valid_interrupt = jiffies;
schedule_delayed_work(&ts->esd_work,
round_jiffies_relative(
@@ -414,9 +434,9 @@
}
/* hardware reset */
- ts->set_reset(false);
+ tsc2005_set_reset(ts, false);
usleep_range(100, 500); /* only 10us required */
- ts->set_reset(true);
+ tsc2005_set_reset(ts, true);
if (!success)
goto out;
@@ -459,7 +479,7 @@
umode_t mode = attr->mode;
if (attr == &dev_attr_selftest.attr) {
- if (!ts->set_reset)
+ if (!ts->set_reset && !ts->reset_gpio)
mode = 0;
}
@@ -509,9 +529,9 @@
tsc2005_update_pen_state(ts, 0, 0, 0);
- ts->set_reset(false);
+ tsc2005_set_reset(ts, false);
usleep_range(100, 500); /* only 10us required */
- ts->set_reset(true);
+ tsc2005_set_reset(ts, true);
enable_irq(ts->spi->irq);
tsc2005_start_scan(ts);
@@ -572,29 +592,47 @@
static int tsc2005_probe(struct spi_device *spi)
{
const struct tsc2005_platform_data *pdata = dev_get_platdata(&spi->dev);
+ struct device_node *np = spi->dev.of_node;
+
struct tsc2005 *ts;
struct input_dev *input_dev;
- unsigned int max_x, max_y, max_p;
- unsigned int fudge_x, fudge_y, fudge_p;
+ unsigned int max_x = MAX_12BIT;
+ unsigned int max_y = MAX_12BIT;
+ unsigned int max_p = MAX_12BIT;
+ unsigned int fudge_x = TSC2005_DEF_X_FUZZ;
+ unsigned int fudge_y = TSC2005_DEF_Y_FUZZ;
+ unsigned int fudge_p = TSC2005_DEF_P_FUZZ;
+ unsigned int x_plate_ohm = TSC2005_DEF_RESISTOR;
+ unsigned int esd_timeout;
int error;
- if (!pdata) {
- dev_dbg(&spi->dev, "no platform data\n");
+ if (!np && !pdata) {
+ dev_err(&spi->dev, "no platform data\n");
return -ENODEV;
}
- fudge_x = pdata->ts_x_fudge ? : 4;
- fudge_y = pdata->ts_y_fudge ? : 8;
- fudge_p = pdata->ts_pressure_fudge ? : 2;
- max_x = pdata->ts_x_max ? : MAX_12BIT;
- max_y = pdata->ts_y_max ? : MAX_12BIT;
- max_p = pdata->ts_pressure_max ? : MAX_12BIT;
-
if (spi->irq <= 0) {
- dev_dbg(&spi->dev, "no irq\n");
+ dev_err(&spi->dev, "no irq\n");
return -ENODEV;
}
+ if (pdata) {
+ fudge_x = pdata->ts_x_fudge;
+ fudge_y = pdata->ts_y_fudge;
+ fudge_p = pdata->ts_pressure_fudge;
+ max_x = pdata->ts_x_max;
+ max_y = pdata->ts_y_max;
+ max_p = pdata->ts_pressure_max;
+ x_plate_ohm = pdata->ts_x_plate_ohm;
+ esd_timeout = pdata->esd_timeout_ms;
+ } else {
+ x_plate_ohm = TSC2005_DEF_RESISTOR;
+ of_property_read_u32(np, "ti,x-plate-ohms", &x_plate_ohm);
+ esd_timeout = 0;
+ of_property_read_u32(np, "ti,esd-recovery-timeout-ms",
+ &esd_timeout);
+ }
+
spi->mode = SPI_MODE_0;
spi->bits_per_word = 8;
if (!spi->max_speed_hz)
@@ -604,19 +642,48 @@
if (error)
return error;
- ts = kzalloc(sizeof(*ts), GFP_KERNEL);
- input_dev = input_allocate_device();
- if (!ts || !input_dev) {
- error = -ENOMEM;
- goto err_free_mem;
- }
+ ts = devm_kzalloc(&spi->dev, sizeof(*ts), GFP_KERNEL);
+ if (!ts)
+ return -ENOMEM;
+
+ input_dev = devm_input_allocate_device(&spi->dev);
+ if (!input_dev)
+ return -ENOMEM;
ts->spi = spi;
ts->idev = input_dev;
- ts->x_plate_ohm = pdata->ts_x_plate_ohm ? : 280;
- ts->esd_timeout = pdata->esd_timeout_ms;
- ts->set_reset = pdata->set_reset;
+ ts->x_plate_ohm = x_plate_ohm;
+ ts->esd_timeout = esd_timeout;
+
+ if (np) {
+ ts->reset_gpio = of_get_named_gpio(np, "reset-gpios", 0);
+ if (ts->reset_gpio == -EPROBE_DEFER)
+ return ts->reset_gpio;
+ if (ts->reset_gpio < 0) {
+ dev_err(&spi->dev, "error acquiring reset gpio: %d\n",
+ ts->reset_gpio);
+ return ts->reset_gpio;
+ }
+
+ error = devm_gpio_request_one(&spi->dev, ts->reset_gpio, 0,
+ "reset-gpios");
+ if (error) {
+ dev_err(&spi->dev, "error requesting reset gpio: %d\n",
+ error);
+ return error;
+ }
+
+ ts->vio = devm_regulator_get(&spi->dev, "vio");
+ if (IS_ERR(ts->vio)) {
+ error = PTR_ERR(ts->vio);
+ dev_err(&spi->dev, "vio regulator missing (%d)", error);
+ return error;
+ }
+ } else {
+ ts->reset_gpio = -1;
+ ts->set_reset = pdata->set_reset;
+ }
mutex_init(&ts->mutex);
@@ -641,6 +708,9 @@
input_set_abs_params(input_dev, ABS_Y, 0, max_y, fudge_y, 0);
input_set_abs_params(input_dev, ABS_PRESSURE, 0, max_p, fudge_p, 0);
+ if (np)
+ touchscreen_parse_of_params(input_dev);
+
input_dev->open = tsc2005_open;
input_dev->close = tsc2005_close;
@@ -649,12 +719,20 @@
/* Ensure the touchscreen is off */
tsc2005_stop_scan(ts);
- error = request_threaded_irq(spi->irq, NULL, tsc2005_irq_thread,
- IRQF_TRIGGER_RISING | IRQF_ONESHOT,
- "tsc2005", ts);
+ error = devm_request_threaded_irq(&spi->dev, spi->irq, NULL,
+ tsc2005_irq_thread,
+ IRQF_TRIGGER_RISING | IRQF_ONESHOT,
+ "tsc2005", ts);
if (error) {
dev_err(&spi->dev, "Failed to request irq, err: %d\n", error);
- goto err_free_mem;
+ return error;
+ }
+
+ /* enable regulator for DT */
+ if (ts->vio) {
+ error = regulator_enable(ts->vio);
+ if (error)
+ return error;
}
spi_set_drvdata(spi, ts);
@@ -662,7 +740,7 @@
if (error) {
dev_err(&spi->dev,
"Failed to create sysfs attributes, err: %d\n", error);
- goto err_clear_drvdata;
+ goto disable_regulator;
}
error = input_register_device(ts->idev);
@@ -677,11 +755,9 @@
err_remove_sysfs:
sysfs_remove_group(&spi->dev.kobj, &tsc2005_attr_group);
-err_clear_drvdata:
- free_irq(spi->irq, ts);
-err_free_mem:
- input_free_device(input_dev);
- kfree(ts);
+disable_regulator:
+ if (ts->vio)
+ regulator_disable(ts->vio);
return error;
}
@@ -689,11 +765,10 @@
{
struct tsc2005 *ts = spi_get_drvdata(spi);
- sysfs_remove_group(&ts->spi->dev.kobj, &tsc2005_attr_group);
+ sysfs_remove_group(&spi->dev.kobj, &tsc2005_attr_group);
- free_irq(ts->spi->irq, ts);
- input_unregister_device(ts->idev);
- kfree(ts);
+ if (ts->vio)
+ regulator_disable(ts->vio);
return 0;
}
diff --git a/drivers/input/touchscreen/zforce_ts.c b/drivers/input/touchscreen/zforce_ts.c
index 01d30ce..feea85b 100644
--- a/drivers/input/touchscreen/zforce_ts.c
+++ b/drivers/input/touchscreen/zforce_ts.c
@@ -880,7 +880,7 @@
MODULE_DEVICE_TABLE(i2c, zforce_idtable);
#ifdef CONFIG_OF
-static struct of_device_id zforce_dt_idtable[] = {
+static const struct of_device_id zforce_dt_idtable[] = {
{ .compatible = "neonode,zforce" },
{},
};
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index df56e4c..d260605 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -178,13 +178,13 @@
config EXYNOS_IOMMU
bool "Exynos IOMMU Support"
- depends on ARCH_EXYNOS && EXYNOS_DEV_SYSMMU
+ depends on ARCH_EXYNOS
select IOMMU_API
help
- Support for the IOMMU(System MMU) of Samsung Exynos application
- processor family. This enables H/W multimedia accellerators to see
- non-linear physical memory chunks as a linear memory in their
- address spaces
+ Support for the IOMMU (System MMU) of Samsung Exynos application
+ processor family. This enables H/W multimedia accelerators to see
+ non-linear physical memory chunks as linear memory in their
+ address space.
If unsure, say N here.
@@ -193,9 +193,9 @@
depends on EXYNOS_IOMMU
help
Select this to see the detailed log message that shows what
- happens in the IOMMU driver
+ happens in the IOMMU driver.
- Say N unless you need kernel log message for IOMMU debugging
+ Say N unless you need kernel log message for IOMMU debugging.
config SHMOBILE_IPMMU
bool
@@ -272,6 +272,18 @@
default 256 if SHMOBILE_IOMMU_ADDRSIZE_64MB
default 128 if SHMOBILE_IOMMU_ADDRSIZE_32MB
+config IPMMU_VMSA
+ bool "Renesas VMSA-compatible IPMMU"
+ depends on ARM_LPAE
+ depends on ARCH_SHMOBILE || COMPILE_TEST
+ select IOMMU_API
+ select ARM_DMA_USE_IOMMU
+ help
+ Support for the Renesas VMSA-compatible IPMMU Renesas found in the
+ R-Mobile APE6 and R-Car H2/M2 SoCs.
+
+ If unsure, say N.
+
config SPAPR_TCE_IOMMU
bool "sPAPR TCE IOMMU Support"
depends on PPC_POWERNV || PPC_PSERIES
diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile
index 5d58bf1..8893bad 100644
--- a/drivers/iommu/Makefile
+++ b/drivers/iommu/Makefile
@@ -7,6 +7,7 @@
obj-$(CONFIG_ARM_SMMU) += arm-smmu.o
obj-$(CONFIG_DMAR_TABLE) += dmar.o
obj-$(CONFIG_INTEL_IOMMU) += iova.o intel-iommu.o
+obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
obj-$(CONFIG_IRQ_REMAP) += intel_irq_remapping.o irq_remapping.o
obj-$(CONFIG_OMAP_IOMMU) += omap-iommu.o
obj-$(CONFIG_OMAP_IOMMU) += omap-iommu2.o
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 57068e8..4aec6a2 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -3499,8 +3499,6 @@
{
struct iommu_dev_data *dev_data;
struct pci_dev *dev = NULL;
- struct amd_iommu *iommu;
- u16 devid;
int ret;
ret = alloc_passthrough_domain();
@@ -3514,12 +3512,6 @@
dev_data = get_dev_data(&dev->dev);
dev_data->passthrough = true;
- devid = get_device_id(&dev->dev);
-
- iommu = amd_iommu_rlookup_table[devid];
- if (!iommu)
- continue;
-
attach_device(&dev->dev, pt_domain);
}
diff --git a/drivers/iommu/amd_iommu_v2.c b/drivers/iommu/amd_iommu_v2.c
index 203b2e6..d4daa05 100644
--- a/drivers/iommu/amd_iommu_v2.c
+++ b/drivers/iommu/amd_iommu_v2.c
@@ -45,6 +45,8 @@
struct pasid_state {
struct list_head list; /* For global state-list */
atomic_t count; /* Reference count */
+ atomic_t mmu_notifier_count; /* Counting nested mmu_notifier
+ calls */
struct task_struct *task; /* Task bound to this PASID */
struct mm_struct *mm; /* mm_struct for the faults */
struct mmu_notifier mn; /* mmu_otifier handle */
@@ -56,6 +58,8 @@
};
struct device_state {
+ struct list_head list;
+ u16 devid;
atomic_t count;
struct pci_dev *pdev;
struct pasid_state **states;
@@ -81,13 +85,9 @@
u16 flags;
};
-static struct device_state **state_table;
+static LIST_HEAD(state_list);
static spinlock_t state_lock;
-/* List and lock for all pasid_states */
-static LIST_HEAD(pasid_state_list);
-static DEFINE_SPINLOCK(ps_lock);
-
static struct workqueue_struct *iommu_wq;
/*
@@ -99,7 +99,6 @@
static void free_pasid_states(struct device_state *dev_state);
static void unbind_pasid(struct device_state *dev_state, int pasid);
-static int task_exit(struct notifier_block *nb, unsigned long e, void *data);
static u16 device_id(struct pci_dev *pdev)
{
@@ -111,13 +110,25 @@
return devid;
}
+static struct device_state *__get_device_state(u16 devid)
+{
+ struct device_state *dev_state;
+
+ list_for_each_entry(dev_state, &state_list, list) {
+ if (dev_state->devid == devid)
+ return dev_state;
+ }
+
+ return NULL;
+}
+
static struct device_state *get_device_state(u16 devid)
{
struct device_state *dev_state;
unsigned long flags;
spin_lock_irqsave(&state_lock, flags);
- dev_state = state_table[devid];
+ dev_state = __get_device_state(devid);
if (dev_state != NULL)
atomic_inc(&dev_state->count);
spin_unlock_irqrestore(&state_lock, flags);
@@ -158,29 +169,6 @@
free_device_state(dev_state);
}
-static struct notifier_block profile_nb = {
- .notifier_call = task_exit,
-};
-
-static void link_pasid_state(struct pasid_state *pasid_state)
-{
- spin_lock(&ps_lock);
- list_add_tail(&pasid_state->list, &pasid_state_list);
- spin_unlock(&ps_lock);
-}
-
-static void __unlink_pasid_state(struct pasid_state *pasid_state)
-{
- list_del(&pasid_state->list);
-}
-
-static void unlink_pasid_state(struct pasid_state *pasid_state)
-{
- spin_lock(&ps_lock);
- __unlink_pasid_state(pasid_state);
- spin_unlock(&ps_lock);
-}
-
/* Must be called under dev_state->lock */
static struct pasid_state **__get_pasid_state_ptr(struct device_state *dev_state,
int pasid, bool alloc)
@@ -337,7 +325,6 @@
if (pasid_state == NULL)
return;
- unlink_pasid_state(pasid_state);
__unbind_pasid(pasid_state);
put_pasid_state_wait(pasid_state); /* Reference taken in this function */
}
@@ -379,7 +366,12 @@
continue;
put_pasid_state(pasid_state);
- unbind_pasid(dev_state, i);
+
+ /*
+ * This will call the mn_release function and
+ * unbind the PASID
+ */
+ mmu_notifier_unregister(&pasid_state->mn, pasid_state->mm);
}
if (dev_state->pasid_levels == 2)
@@ -443,8 +435,11 @@
pasid_state = mn_to_state(mn);
dev_state = pasid_state->device_state;
- amd_iommu_domain_set_gcr3(dev_state->domain, pasid_state->pasid,
- __pa(empty_page_table));
+ if (atomic_add_return(1, &pasid_state->mmu_notifier_count) == 1) {
+ amd_iommu_domain_set_gcr3(dev_state->domain,
+ pasid_state->pasid,
+ __pa(empty_page_table));
+ }
}
static void mn_invalidate_range_end(struct mmu_notifier *mn,
@@ -457,11 +452,31 @@
pasid_state = mn_to_state(mn);
dev_state = pasid_state->device_state;
- amd_iommu_domain_set_gcr3(dev_state->domain, pasid_state->pasid,
- __pa(pasid_state->mm->pgd));
+ if (atomic_dec_and_test(&pasid_state->mmu_notifier_count)) {
+ amd_iommu_domain_set_gcr3(dev_state->domain,
+ pasid_state->pasid,
+ __pa(pasid_state->mm->pgd));
+ }
+}
+
+static void mn_release(struct mmu_notifier *mn, struct mm_struct *mm)
+{
+ struct pasid_state *pasid_state;
+ struct device_state *dev_state;
+
+ might_sleep();
+
+ pasid_state = mn_to_state(mn);
+ dev_state = pasid_state->device_state;
+
+ if (pasid_state->device_state->inv_ctx_cb)
+ dev_state->inv_ctx_cb(dev_state->pdev, pasid_state->pasid);
+
+ unbind_pasid(dev_state, pasid_state->pasid);
}
static struct mmu_notifier_ops iommu_mn = {
+ .release = mn_release,
.clear_flush_young = mn_clear_flush_young,
.change_pte = mn_change_pte,
.invalidate_page = mn_invalidate_page,
@@ -606,53 +621,6 @@
.notifier_call = ppr_notifier,
};
-static int task_exit(struct notifier_block *nb, unsigned long e, void *data)
-{
- struct pasid_state *pasid_state;
- struct task_struct *task;
-
- task = data;
-
- /*
- * Using this notifier is a hack - but there is no other choice
- * at the moment. What I really want is a sleeping notifier that
- * is called when an MM goes down. But such a notifier doesn't
- * exist yet. The notifier needs to sleep because it has to make
- * sure that the device does not use the PASID and the address
- * space anymore before it is destroyed. This includes waiting
- * for pending PRI requests to pass the workqueue. The
- * MMU-Notifiers would be a good fit, but they use RCU and so
- * they are not allowed to sleep. Lets see how we can solve this
- * in a more intelligent way in the future.
- */
-again:
- spin_lock(&ps_lock);
- list_for_each_entry(pasid_state, &pasid_state_list, list) {
- struct device_state *dev_state;
- int pasid;
-
- if (pasid_state->task != task)
- continue;
-
- /* Drop Lock and unbind */
- spin_unlock(&ps_lock);
-
- dev_state = pasid_state->device_state;
- pasid = pasid_state->pasid;
-
- if (pasid_state->device_state->inv_ctx_cb)
- dev_state->inv_ctx_cb(dev_state->pdev, pasid);
-
- unbind_pasid(dev_state, pasid);
-
- /* Task may be in the list multiple times */
- goto again;
- }
- spin_unlock(&ps_lock);
-
- return NOTIFY_OK;
-}
-
int amd_iommu_bind_pasid(struct pci_dev *pdev, int pasid,
struct task_struct *task)
{
@@ -682,6 +650,7 @@
goto out;
atomic_set(&pasid_state->count, 1);
+ atomic_set(&pasid_state->mmu_notifier_count, 0);
init_waitqueue_head(&pasid_state->wq);
spin_lock_init(&pasid_state->lock);
@@ -705,8 +674,6 @@
if (ret)
goto out_clear_state;
- link_pasid_state(pasid_state);
-
return 0;
out_clear_state:
@@ -727,6 +694,7 @@
void amd_iommu_unbind_pasid(struct pci_dev *pdev, int pasid)
{
+ struct pasid_state *pasid_state;
struct device_state *dev_state;
u16 devid;
@@ -743,7 +711,17 @@
if (pasid < 0 || pasid >= dev_state->max_pasids)
goto out;
- unbind_pasid(dev_state, pasid);
+ pasid_state = get_pasid_state(dev_state, pasid);
+ if (pasid_state == NULL)
+ goto out;
+ /*
+ * Drop reference taken here. We are safe because we still hold
+ * the reference taken in the amd_iommu_bind_pasid function.
+ */
+ put_pasid_state(pasid_state);
+
+ /* This will call the mn_release function and unbind the PASID */
+ mmu_notifier_unregister(&pasid_state->mn, pasid_state->mm);
out:
put_device_state(dev_state);
@@ -773,7 +751,8 @@
spin_lock_init(&dev_state->lock);
init_waitqueue_head(&dev_state->wq);
- dev_state->pdev = pdev;
+ dev_state->pdev = pdev;
+ dev_state->devid = devid;
tmp = pasids;
for (dev_state->pasid_levels = 0; (tmp - 1) & ~0x1ff; tmp >>= 9)
@@ -803,13 +782,13 @@
spin_lock_irqsave(&state_lock, flags);
- if (state_table[devid] != NULL) {
+ if (__get_device_state(devid) != NULL) {
spin_unlock_irqrestore(&state_lock, flags);
ret = -EBUSY;
goto out_free_domain;
}
- state_table[devid] = dev_state;
+ list_add_tail(&dev_state->list, &state_list);
spin_unlock_irqrestore(&state_lock, flags);
@@ -841,13 +820,13 @@
spin_lock_irqsave(&state_lock, flags);
- dev_state = state_table[devid];
+ dev_state = __get_device_state(devid);
if (dev_state == NULL) {
spin_unlock_irqrestore(&state_lock, flags);
return;
}
- state_table[devid] = NULL;
+ list_del(&dev_state->list);
spin_unlock_irqrestore(&state_lock, flags);
@@ -874,7 +853,7 @@
spin_lock_irqsave(&state_lock, flags);
ret = -EINVAL;
- dev_state = state_table[devid];
+ dev_state = __get_device_state(devid);
if (dev_state == NULL)
goto out_unlock;
@@ -905,7 +884,7 @@
spin_lock_irqsave(&state_lock, flags);
ret = -EINVAL;
- dev_state = state_table[devid];
+ dev_state = __get_device_state(devid);
if (dev_state == NULL)
goto out_unlock;
@@ -922,7 +901,6 @@
static int __init amd_iommu_v2_init(void)
{
- size_t state_table_size;
int ret;
pr_info("AMD IOMMUv2 driver by Joerg Roedel <joerg.roedel@amd.com>\n");
@@ -938,16 +916,10 @@
spin_lock_init(&state_lock);
- state_table_size = MAX_DEVICES * sizeof(struct device_state *);
- state_table = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
- get_order(state_table_size));
- if (state_table == NULL)
- return -ENOMEM;
-
ret = -ENOMEM;
iommu_wq = create_workqueue("amd_iommu_v2");
if (iommu_wq == NULL)
- goto out_free;
+ goto out;
ret = -ENOMEM;
empty_page_table = (u64 *)get_zeroed_page(GFP_KERNEL);
@@ -955,29 +927,24 @@
goto out_destroy_wq;
amd_iommu_register_ppr_notifier(&ppr_nb);
- profile_event_register(PROFILE_TASK_EXIT, &profile_nb);
return 0;
out_destroy_wq:
destroy_workqueue(iommu_wq);
-out_free:
- free_pages((unsigned long)state_table, get_order(state_table_size));
-
+out:
return ret;
}
static void __exit amd_iommu_v2_exit(void)
{
struct device_state *dev_state;
- size_t state_table_size;
int i;
if (!amd_iommu_v2_supported())
return;
- profile_event_unregister(PROFILE_TASK_EXIT, &profile_nb);
amd_iommu_unregister_ppr_notifier(&ppr_nb);
flush_workqueue(iommu_wq);
@@ -1000,9 +967,6 @@
destroy_workqueue(iommu_wq);
- state_table_size = MAX_DEVICES * sizeof(struct device_state *);
- free_pages((unsigned long)state_table, get_order(state_table_size));
-
free_page((unsigned long)empty_page_table);
}
diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c
index 647c3c7..1599354 100644
--- a/drivers/iommu/arm-smmu.c
+++ b/drivers/iommu/arm-smmu.c
@@ -1167,7 +1167,7 @@
for (i = 0; i < master->num_streamids; ++i) {
u32 idx, s2cr;
idx = master->smrs ? master->smrs[i].idx : master->streamids[i];
- s2cr = (S2CR_TYPE_TRANS << S2CR_TYPE_SHIFT) |
+ s2cr = S2CR_TYPE_TRANS |
(smmu_domain->root_cfg.cbndx << S2CR_CBNDX_SHIFT);
writel_relaxed(s2cr, gr0_base + ARM_SMMU_GR0_S2CR(idx));
}
@@ -1804,7 +1804,7 @@
* allocation (PTRS_PER_PGD).
*/
#ifdef CONFIG_64BIT
- smmu->s1_output_size = min(39UL, size);
+ smmu->s1_output_size = min((unsigned long)VA_BITS, size);
#else
smmu->s1_output_size = min(32UL, size);
#endif
diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 685f926..99054d2 100644
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -29,7 +29,8 @@
#include <asm/cacheflush.h>
#include <asm/pgtable.h>
-#include <mach/sysmmu.h>
+typedef u32 sysmmu_iova_t;
+typedef u32 sysmmu_pte_t;
/* We does not consider super section mapping (16MB) */
#define SECT_ORDER 20
@@ -44,28 +45,44 @@
#define LPAGE_MASK (~(LPAGE_SIZE - 1))
#define SPAGE_MASK (~(SPAGE_SIZE - 1))
-#define lv1ent_fault(sent) (((*(sent) & 3) == 0) || ((*(sent) & 3) == 3))
-#define lv1ent_page(sent) ((*(sent) & 3) == 1)
+#define lv1ent_fault(sent) ((*(sent) == ZERO_LV2LINK) || \
+ ((*(sent) & 3) == 0) || ((*(sent) & 3) == 3))
+#define lv1ent_zero(sent) (*(sent) == ZERO_LV2LINK)
+#define lv1ent_page_zero(sent) ((*(sent) & 3) == 1)
+#define lv1ent_page(sent) ((*(sent) != ZERO_LV2LINK) && \
+ ((*(sent) & 3) == 1))
#define lv1ent_section(sent) ((*(sent) & 3) == 2)
#define lv2ent_fault(pent) ((*(pent) & 3) == 0)
#define lv2ent_small(pent) ((*(pent) & 2) == 2)
#define lv2ent_large(pent) ((*(pent) & 3) == 1)
-#define section_phys(sent) (*(sent) & SECT_MASK)
-#define section_offs(iova) ((iova) & 0xFFFFF)
-#define lpage_phys(pent) (*(pent) & LPAGE_MASK)
-#define lpage_offs(iova) ((iova) & 0xFFFF)
-#define spage_phys(pent) (*(pent) & SPAGE_MASK)
-#define spage_offs(iova) ((iova) & 0xFFF)
+static u32 sysmmu_page_offset(sysmmu_iova_t iova, u32 size)
+{
+ return iova & (size - 1);
+}
-#define lv1ent_offset(iova) ((iova) >> SECT_ORDER)
-#define lv2ent_offset(iova) (((iova) & 0xFF000) >> SPAGE_ORDER)
+#define section_phys(sent) (*(sent) & SECT_MASK)
+#define section_offs(iova) sysmmu_page_offset((iova), SECT_SIZE)
+#define lpage_phys(pent) (*(pent) & LPAGE_MASK)
+#define lpage_offs(iova) sysmmu_page_offset((iova), LPAGE_SIZE)
+#define spage_phys(pent) (*(pent) & SPAGE_MASK)
+#define spage_offs(iova) sysmmu_page_offset((iova), SPAGE_SIZE)
#define NUM_LV1ENTRIES 4096
-#define NUM_LV2ENTRIES 256
+#define NUM_LV2ENTRIES (SECT_SIZE / SPAGE_SIZE)
-#define LV2TABLE_SIZE (NUM_LV2ENTRIES * sizeof(long))
+static u32 lv1ent_offset(sysmmu_iova_t iova)
+{
+ return iova >> SECT_ORDER;
+}
+
+static u32 lv2ent_offset(sysmmu_iova_t iova)
+{
+ return (iova >> SPAGE_ORDER) & (NUM_LV2ENTRIES - 1);
+}
+
+#define LV2TABLE_SIZE (NUM_LV2ENTRIES * sizeof(sysmmu_pte_t))
#define SPAGES_PER_LPAGE (LPAGE_SIZE / SPAGE_SIZE)
@@ -80,6 +97,13 @@
#define CTRL_BLOCK 0x7
#define CTRL_DISABLE 0x0
+#define CFG_LRU 0x1
+#define CFG_QOS(n) ((n & 0xF) << 7)
+#define CFG_MASK 0x0150FFFF /* Selecting bit 0-15, 20, 22 and 24 */
+#define CFG_ACGEN (1 << 24) /* System MMU 3.3 only */
+#define CFG_SYSSEL (1 << 22) /* System MMU 3.2 only */
+#define CFG_FLPDCACHE (1 << 20) /* System MMU 3.2+ only */
+
#define REG_MMU_CTRL 0x000
#define REG_MMU_CFG 0x004
#define REG_MMU_STATUS 0x008
@@ -96,19 +120,32 @@
#define REG_MMU_VERSION 0x034
+#define MMU_MAJ_VER(val) ((val) >> 7)
+#define MMU_MIN_VER(val) ((val) & 0x7F)
+#define MMU_RAW_VER(reg) (((reg) >> 21) & ((1 << 11) - 1)) /* 11 bits */
+
+#define MAKE_MMU_VER(maj, min) ((((maj) & 0xF) << 7) | ((min) & 0x7F))
+
#define REG_PB0_SADDR 0x04C
#define REG_PB0_EADDR 0x050
#define REG_PB1_SADDR 0x054
#define REG_PB1_EADDR 0x058
-static unsigned long *section_entry(unsigned long *pgtable, unsigned long iova)
+#define has_sysmmu(dev) (dev->archdata.iommu != NULL)
+
+static struct kmem_cache *lv2table_kmem_cache;
+static sysmmu_pte_t *zero_lv2_table;
+#define ZERO_LV2LINK mk_lv1ent_page(virt_to_phys(zero_lv2_table))
+
+static sysmmu_pte_t *section_entry(sysmmu_pte_t *pgtable, sysmmu_iova_t iova)
{
return pgtable + lv1ent_offset(iova);
}
-static unsigned long *page_entry(unsigned long *sent, unsigned long iova)
+static sysmmu_pte_t *page_entry(sysmmu_pte_t *sent, sysmmu_iova_t iova)
{
- return (unsigned long *)__va(lv2table_base(sent)) + lv2ent_offset(iova);
+ return (sysmmu_pte_t *)phys_to_virt(
+ lv2table_base(sent)) + lv2ent_offset(iova);
}
enum exynos_sysmmu_inttype {
@@ -124,16 +161,6 @@
SYSMMU_FAULTS_NUM
};
-/*
- * @itype: type of fault.
- * @pgtable_base: the physical address of page table base. This is 0 if @itype
- * is SYSMMU_BUSERROR.
- * @fault_addr: the device (virtual) address that the System MMU tried to
- * translated. This is 0 if @itype is SYSMMU_BUSERROR.
- */
-typedef int (*sysmmu_fault_handler_t)(enum exynos_sysmmu_inttype itype,
- unsigned long pgtable_base, unsigned long fault_addr);
-
static unsigned short fault_reg_offset[SYSMMU_FAULTS_NUM] = {
REG_PAGE_FAULT_ADDR,
REG_AR_FAULT_ADDR,
@@ -157,27 +184,34 @@
"UNKNOWN FAULT"
};
+/* attached to dev.archdata.iommu of the master device */
+struct exynos_iommu_owner {
+ struct list_head client; /* entry of exynos_iommu_domain.clients */
+ struct device *dev;
+ struct device *sysmmu;
+ struct iommu_domain *domain;
+ void *vmm_data; /* IO virtual memory manager's data */
+ spinlock_t lock; /* Lock to preserve consistency of System MMU */
+};
+
struct exynos_iommu_domain {
struct list_head clients; /* list of sysmmu_drvdata.node */
- unsigned long *pgtable; /* lv1 page table, 16KB */
+ sysmmu_pte_t *pgtable; /* lv1 page table, 16KB */
short *lv2entcnt; /* free lv2 entry counter for each section */
spinlock_t lock; /* lock for this structure */
spinlock_t pgtablelock; /* lock for modifying page table @ pgtable */
};
struct sysmmu_drvdata {
- struct list_head node; /* entry of exynos_iommu_domain.clients */
struct device *sysmmu; /* System MMU's device descriptor */
- struct device *dev; /* Owner of system MMU */
- char *dbgname;
- int nsfrs;
- void __iomem **sfrbases;
- struct clk *clk[2];
+ struct device *master; /* Owner of system MMU */
+ void __iomem *sfrbase;
+ struct clk *clk;
+ struct clk *clk_master;
int activations;
- rwlock_t lock;
+ spinlock_t lock;
struct iommu_domain *domain;
- sysmmu_fault_handler_t fault_handler;
- unsigned long pgtable;
+ phys_addr_t pgtable;
};
static bool set_sysmmu_active(struct sysmmu_drvdata *data)
@@ -204,6 +238,11 @@
__raw_writel(CTRL_ENABLE, sfrbase + REG_MMU_CTRL);
}
+static unsigned int __raw_sysmmu_version(struct sysmmu_drvdata *data)
+{
+ return MMU_RAW_VER(__raw_readl(data->sfrbase + REG_MMU_VERSION));
+}
+
static bool sysmmu_block(void __iomem *sfrbase)
{
int i = 120;
@@ -226,429 +265,428 @@
}
static void __sysmmu_tlb_invalidate_entry(void __iomem *sfrbase,
- unsigned long iova)
+ sysmmu_iova_t iova, unsigned int num_inv)
{
- __raw_writel((iova & SPAGE_MASK) | 1, sfrbase + REG_MMU_FLUSH_ENTRY);
+ unsigned int i;
+
+ for (i = 0; i < num_inv; i++) {
+ __raw_writel((iova & SPAGE_MASK) | 1,
+ sfrbase + REG_MMU_FLUSH_ENTRY);
+ iova += SPAGE_SIZE;
+ }
}
static void __sysmmu_set_ptbase(void __iomem *sfrbase,
- unsigned long pgd)
+ phys_addr_t pgd)
{
- __raw_writel(0x1, sfrbase + REG_MMU_CFG); /* 16KB LV1, LRU */
__raw_writel(pgd, sfrbase + REG_PT_BASE_ADDR);
__sysmmu_tlb_invalidate(sfrbase);
}
-static void __sysmmu_set_prefbuf(void __iomem *sfrbase, unsigned long base,
- unsigned long size, int idx)
+static void show_fault_information(const char *name,
+ enum exynos_sysmmu_inttype itype,
+ phys_addr_t pgtable_base, sysmmu_iova_t fault_addr)
{
- __raw_writel(base, sfrbase + REG_PB0_SADDR + idx * 8);
- __raw_writel(size - 1 + base, sfrbase + REG_PB0_EADDR + idx * 8);
-}
-
-static void __set_fault_handler(struct sysmmu_drvdata *data,
- sysmmu_fault_handler_t handler)
-{
- unsigned long flags;
-
- write_lock_irqsave(&data->lock, flags);
- data->fault_handler = handler;
- write_unlock_irqrestore(&data->lock, flags);
-}
-
-void exynos_sysmmu_set_fault_handler(struct device *dev,
- sysmmu_fault_handler_t handler)
-{
- struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
-
- __set_fault_handler(data, handler);
-}
-
-static int default_fault_handler(enum exynos_sysmmu_inttype itype,
- unsigned long pgtable_base, unsigned long fault_addr)
-{
- unsigned long *ent;
+ sysmmu_pte_t *ent;
if ((itype >= SYSMMU_FAULTS_NUM) || (itype < SYSMMU_PAGEFAULT))
itype = SYSMMU_FAULT_UNKNOWN;
- pr_err("%s occurred at 0x%lx(Page table base: 0x%lx)\n",
- sysmmu_fault_name[itype], fault_addr, pgtable_base);
+ pr_err("%s occurred at %#x by %s(Page table base: %pa)\n",
+ sysmmu_fault_name[itype], fault_addr, name, &pgtable_base);
- ent = section_entry(__va(pgtable_base), fault_addr);
- pr_err("\tLv1 entry: 0x%lx\n", *ent);
+ ent = section_entry(phys_to_virt(pgtable_base), fault_addr);
+ pr_err("\tLv1 entry: %#x\n", *ent);
if (lv1ent_page(ent)) {
ent = page_entry(ent, fault_addr);
- pr_err("\t Lv2 entry: 0x%lx\n", *ent);
+ pr_err("\t Lv2 entry: %#x\n", *ent);
}
-
- pr_err("Generating Kernel OOPS... because it is unrecoverable.\n");
-
- BUG();
-
- return 0;
}
static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
{
/* SYSMMU is in blocked when interrupt occurred. */
struct sysmmu_drvdata *data = dev_id;
- struct resource *irqres;
- struct platform_device *pdev;
enum exynos_sysmmu_inttype itype;
- unsigned long addr = -1;
-
- int i, ret = -ENOSYS;
-
- read_lock(&data->lock);
+ sysmmu_iova_t addr = -1;
+ int ret = -ENOSYS;
WARN_ON(!is_sysmmu_active(data));
- pdev = to_platform_device(data->sysmmu);
- for (i = 0; i < (pdev->num_resources / 2); i++) {
- irqres = platform_get_resource(pdev, IORESOURCE_IRQ, i);
- if (irqres && ((int)irqres->start == irq))
- break;
- }
+ spin_lock(&data->lock);
- if (i == pdev->num_resources) {
+ if (!IS_ERR(data->clk_master))
+ clk_enable(data->clk_master);
+
+ itype = (enum exynos_sysmmu_inttype)
+ __ffs(__raw_readl(data->sfrbase + REG_INT_STATUS));
+ if (WARN_ON(!((itype >= 0) && (itype < SYSMMU_FAULT_UNKNOWN))))
itype = SYSMMU_FAULT_UNKNOWN;
- } else {
- itype = (enum exynos_sysmmu_inttype)
- __ffs(__raw_readl(data->sfrbases[i] + REG_INT_STATUS));
- if (WARN_ON(!((itype >= 0) && (itype < SYSMMU_FAULT_UNKNOWN))))
- itype = SYSMMU_FAULT_UNKNOWN;
- else
- addr = __raw_readl(
- data->sfrbases[i] + fault_reg_offset[itype]);
- }
-
- if (data->domain)
- ret = report_iommu_fault(data->domain, data->dev,
- addr, itype);
-
- if ((ret == -ENOSYS) && data->fault_handler) {
- unsigned long base = data->pgtable;
- if (itype != SYSMMU_FAULT_UNKNOWN)
- base = __raw_readl(
- data->sfrbases[i] + REG_PT_BASE_ADDR);
- ret = data->fault_handler(itype, base, addr);
- }
-
- if (!ret && (itype != SYSMMU_FAULT_UNKNOWN))
- __raw_writel(1 << itype, data->sfrbases[i] + REG_INT_CLEAR);
else
- dev_dbg(data->sysmmu, "(%s) %s is not handled.\n",
- data->dbgname, sysmmu_fault_name[itype]);
+ addr = __raw_readl(data->sfrbase + fault_reg_offset[itype]);
- if (itype != SYSMMU_FAULT_UNKNOWN)
- sysmmu_unblock(data->sfrbases[i]);
+ if (itype == SYSMMU_FAULT_UNKNOWN) {
+ pr_err("%s: Fault is not occurred by System MMU '%s'!\n",
+ __func__, dev_name(data->sysmmu));
+ pr_err("%s: Please check if IRQ is correctly configured.\n",
+ __func__);
+ BUG();
+ } else {
+ unsigned int base =
+ __raw_readl(data->sfrbase + REG_PT_BASE_ADDR);
+ show_fault_information(dev_name(data->sysmmu),
+ itype, base, addr);
+ if (data->domain)
+ ret = report_iommu_fault(data->domain,
+ data->master, addr, itype);
+ }
- read_unlock(&data->lock);
+ /* fault is not recovered by fault handler */
+ BUG_ON(ret != 0);
+
+ __raw_writel(1 << itype, data->sfrbase + REG_INT_CLEAR);
+
+ sysmmu_unblock(data->sfrbase);
+
+ if (!IS_ERR(data->clk_master))
+ clk_disable(data->clk_master);
+
+ spin_unlock(&data->lock);
return IRQ_HANDLED;
}
-static bool __exynos_sysmmu_disable(struct sysmmu_drvdata *data)
+static void __sysmmu_disable_nocount(struct sysmmu_drvdata *data)
{
+ if (!IS_ERR(data->clk_master))
+ clk_enable(data->clk_master);
+
+ __raw_writel(CTRL_DISABLE, data->sfrbase + REG_MMU_CTRL);
+ __raw_writel(0, data->sfrbase + REG_MMU_CFG);
+
+ clk_disable(data->clk);
+ if (!IS_ERR(data->clk_master))
+ clk_disable(data->clk_master);
+}
+
+static bool __sysmmu_disable(struct sysmmu_drvdata *data)
+{
+ bool disabled;
unsigned long flags;
- bool disabled = false;
- int i;
- write_lock_irqsave(&data->lock, flags);
+ spin_lock_irqsave(&data->lock, flags);
- if (!set_sysmmu_inactive(data))
- goto finish;
+ disabled = set_sysmmu_inactive(data);
- for (i = 0; i < data->nsfrs; i++)
- __raw_writel(CTRL_DISABLE, data->sfrbases[i] + REG_MMU_CTRL);
+ if (disabled) {
+ data->pgtable = 0;
+ data->domain = NULL;
- if (data->clk[1])
- clk_disable(data->clk[1]);
- if (data->clk[0])
- clk_disable(data->clk[0]);
+ __sysmmu_disable_nocount(data);
- disabled = true;
- data->pgtable = 0;
- data->domain = NULL;
-finish:
- write_unlock_irqrestore(&data->lock, flags);
+ dev_dbg(data->sysmmu, "Disabled\n");
+ } else {
+ dev_dbg(data->sysmmu, "%d times left to disable\n",
+ data->activations);
+ }
- if (disabled)
- dev_dbg(data->sysmmu, "(%s) Disabled\n", data->dbgname);
- else
- dev_dbg(data->sysmmu, "(%s) %d times left to be disabled\n",
- data->dbgname, data->activations);
+ spin_unlock_irqrestore(&data->lock, flags);
return disabled;
}
+static void __sysmmu_init_config(struct sysmmu_drvdata *data)
+{
+ unsigned int cfg = CFG_LRU | CFG_QOS(15);
+ unsigned int ver;
+
+ ver = __raw_sysmmu_version(data);
+ if (MMU_MAJ_VER(ver) == 3) {
+ if (MMU_MIN_VER(ver) >= 2) {
+ cfg |= CFG_FLPDCACHE;
+ if (MMU_MIN_VER(ver) == 3) {
+ cfg |= CFG_ACGEN;
+ cfg &= ~CFG_LRU;
+ } else {
+ cfg |= CFG_SYSSEL;
+ }
+ }
+ }
+
+ __raw_writel(cfg, data->sfrbase + REG_MMU_CFG);
+}
+
+static void __sysmmu_enable_nocount(struct sysmmu_drvdata *data)
+{
+ if (!IS_ERR(data->clk_master))
+ clk_enable(data->clk_master);
+ clk_enable(data->clk);
+
+ __raw_writel(CTRL_BLOCK, data->sfrbase + REG_MMU_CTRL);
+
+ __sysmmu_init_config(data);
+
+ __sysmmu_set_ptbase(data->sfrbase, data->pgtable);
+
+ __raw_writel(CTRL_ENABLE, data->sfrbase + REG_MMU_CTRL);
+
+ if (!IS_ERR(data->clk_master))
+ clk_disable(data->clk_master);
+}
+
+static int __sysmmu_enable(struct sysmmu_drvdata *data,
+ phys_addr_t pgtable, struct iommu_domain *domain)
+{
+ int ret = 0;
+ unsigned long flags;
+
+ spin_lock_irqsave(&data->lock, flags);
+ if (set_sysmmu_active(data)) {
+ data->pgtable = pgtable;
+ data->domain = domain;
+
+ __sysmmu_enable_nocount(data);
+
+ dev_dbg(data->sysmmu, "Enabled\n");
+ } else {
+ ret = (pgtable == data->pgtable) ? 1 : -EBUSY;
+
+ dev_dbg(data->sysmmu, "already enabled\n");
+ }
+
+ if (WARN_ON(ret < 0))
+ set_sysmmu_inactive(data); /* decrement count */
+
+ spin_unlock_irqrestore(&data->lock, flags);
+
+ return ret;
+}
+
/* __exynos_sysmmu_enable: Enables System MMU
*
* returns -error if an error occurred and System MMU is not enabled,
* 0 if the System MMU has been just enabled and 1 if System MMU was already
* enabled before.
*/
-static int __exynos_sysmmu_enable(struct sysmmu_drvdata *data,
- unsigned long pgtable, struct iommu_domain *domain)
+static int __exynos_sysmmu_enable(struct device *dev, phys_addr_t pgtable,
+ struct iommu_domain *domain)
{
- int i, ret = 0;
+ int ret = 0;
unsigned long flags;
+ struct exynos_iommu_owner *owner = dev->archdata.iommu;
+ struct sysmmu_drvdata *data;
- write_lock_irqsave(&data->lock, flags);
+ BUG_ON(!has_sysmmu(dev));
- if (!set_sysmmu_active(data)) {
- if (WARN_ON(pgtable != data->pgtable)) {
- ret = -EBUSY;
- set_sysmmu_inactive(data);
- } else {
- ret = 1;
- }
+ spin_lock_irqsave(&owner->lock, flags);
- dev_dbg(data->sysmmu, "(%s) Already enabled\n", data->dbgname);
- goto finish;
- }
+ data = dev_get_drvdata(owner->sysmmu);
- if (data->clk[0])
- clk_enable(data->clk[0]);
- if (data->clk[1])
- clk_enable(data->clk[1]);
+ ret = __sysmmu_enable(data, pgtable, domain);
+ if (ret >= 0)
+ data->master = dev;
- data->pgtable = pgtable;
-
- for (i = 0; i < data->nsfrs; i++) {
- __sysmmu_set_ptbase(data->sfrbases[i], pgtable);
-
- if ((readl(data->sfrbases[i] + REG_MMU_VERSION) >> 28) == 3) {
- /* System MMU version is 3.x */
- __raw_writel((1 << 12) | (2 << 28),
- data->sfrbases[i] + REG_MMU_CFG);
- __sysmmu_set_prefbuf(data->sfrbases[i], 0, -1, 0);
- __sysmmu_set_prefbuf(data->sfrbases[i], 0, -1, 1);
- }
-
- __raw_writel(CTRL_ENABLE, data->sfrbases[i] + REG_MMU_CTRL);
- }
-
- data->domain = domain;
-
- dev_dbg(data->sysmmu, "(%s) Enabled\n", data->dbgname);
-finish:
- write_unlock_irqrestore(&data->lock, flags);
+ spin_unlock_irqrestore(&owner->lock, flags);
return ret;
}
-int exynos_sysmmu_enable(struct device *dev, unsigned long pgtable)
+int exynos_sysmmu_enable(struct device *dev, phys_addr_t pgtable)
{
- struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
- int ret;
-
BUG_ON(!memblock_is_memory(pgtable));
- ret = pm_runtime_get_sync(data->sysmmu);
- if (ret < 0) {
- dev_dbg(data->sysmmu, "(%s) Failed to enable\n", data->dbgname);
- return ret;
- }
-
- ret = __exynos_sysmmu_enable(data, pgtable, NULL);
- if (WARN_ON(ret < 0)) {
- pm_runtime_put(data->sysmmu);
- dev_err(data->sysmmu,
- "(%s) Already enabled with page table %#lx\n",
- data->dbgname, data->pgtable);
- } else {
- data->dev = dev;
- }
-
- return ret;
+ return __exynos_sysmmu_enable(dev, pgtable, NULL);
}
static bool exynos_sysmmu_disable(struct device *dev)
{
- struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
- bool disabled;
+ unsigned long flags;
+ bool disabled = true;
+ struct exynos_iommu_owner *owner = dev->archdata.iommu;
+ struct sysmmu_drvdata *data;
- disabled = __exynos_sysmmu_disable(data);
- pm_runtime_put(data->sysmmu);
+ BUG_ON(!has_sysmmu(dev));
+
+ spin_lock_irqsave(&owner->lock, flags);
+
+ data = dev_get_drvdata(owner->sysmmu);
+
+ disabled = __sysmmu_disable(data);
+ if (disabled)
+ data->master = NULL;
+
+ spin_unlock_irqrestore(&owner->lock, flags);
return disabled;
}
-static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova)
+static void __sysmmu_tlb_invalidate_flpdcache(struct sysmmu_drvdata *data,
+ sysmmu_iova_t iova)
+{
+ if (__raw_sysmmu_version(data) == MAKE_MMU_VER(3, 3))
+ __raw_writel(iova | 0x1, data->sfrbase + REG_MMU_FLUSH_ENTRY);
+}
+
+static void sysmmu_tlb_invalidate_flpdcache(struct device *dev,
+ sysmmu_iova_t iova)
{
unsigned long flags;
- struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
+ struct exynos_iommu_owner *owner = dev->archdata.iommu;
+ struct sysmmu_drvdata *data = dev_get_drvdata(owner->sysmmu);
- read_lock_irqsave(&data->lock, flags);
+ if (!IS_ERR(data->clk_master))
+ clk_enable(data->clk_master);
+ spin_lock_irqsave(&data->lock, flags);
+ if (is_sysmmu_active(data))
+ __sysmmu_tlb_invalidate_flpdcache(data, iova);
+ spin_unlock_irqrestore(&data->lock, flags);
+
+ if (!IS_ERR(data->clk_master))
+ clk_disable(data->clk_master);
+}
+
+static void sysmmu_tlb_invalidate_entry(struct device *dev, sysmmu_iova_t iova,
+ size_t size)
+{
+ struct exynos_iommu_owner *owner = dev->archdata.iommu;
+ unsigned long flags;
+ struct sysmmu_drvdata *data;
+
+ data = dev_get_drvdata(owner->sysmmu);
+
+ spin_lock_irqsave(&data->lock, flags);
if (is_sysmmu_active(data)) {
- int i;
- for (i = 0; i < data->nsfrs; i++) {
- if (sysmmu_block(data->sfrbases[i])) {
- __sysmmu_tlb_invalidate_entry(
- data->sfrbases[i], iova);
- sysmmu_unblock(data->sfrbases[i]);
- }
- }
- } else {
- dev_dbg(data->sysmmu,
- "(%s) Disabled. Skipping invalidating TLB.\n",
- data->dbgname);
- }
+ unsigned int num_inv = 1;
- read_unlock_irqrestore(&data->lock, flags);
+ if (!IS_ERR(data->clk_master))
+ clk_enable(data->clk_master);
+
+ /*
+ * L2TLB invalidation required
+ * 4KB page: 1 invalidation
+ * 64KB page: 16 invalidation
+ * 1MB page: 64 invalidation
+ * because it is set-associative TLB
+ * with 8-way and 64 sets.
+ * 1MB page can be cached in one of all sets.
+ * 64KB page can be one of 16 consecutive sets.
+ */
+ if (MMU_MAJ_VER(__raw_sysmmu_version(data)) == 2)
+ num_inv = min_t(unsigned int, size / PAGE_SIZE, 64);
+
+ if (sysmmu_block(data->sfrbase)) {
+ __sysmmu_tlb_invalidate_entry(
+ data->sfrbase, iova, num_inv);
+ sysmmu_unblock(data->sfrbase);
+ }
+ if (!IS_ERR(data->clk_master))
+ clk_disable(data->clk_master);
+ } else {
+ dev_dbg(dev, "disabled. Skipping TLB invalidation @ %#x\n",
+ iova);
+ }
+ spin_unlock_irqrestore(&data->lock, flags);
}
void exynos_sysmmu_tlb_invalidate(struct device *dev)
{
+ struct exynos_iommu_owner *owner = dev->archdata.iommu;
unsigned long flags;
- struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
-
- read_lock_irqsave(&data->lock, flags);
-
- if (is_sysmmu_active(data)) {
- int i;
- for (i = 0; i < data->nsfrs; i++) {
- if (sysmmu_block(data->sfrbases[i])) {
- __sysmmu_tlb_invalidate(data->sfrbases[i]);
- sysmmu_unblock(data->sfrbases[i]);
- }
- }
- } else {
- dev_dbg(data->sysmmu,
- "(%s) Disabled. Skipping invalidating TLB.\n",
- data->dbgname);
- }
-
- read_unlock_irqrestore(&data->lock, flags);
-}
-
-static int exynos_sysmmu_probe(struct platform_device *pdev)
-{
- int i, ret;
- struct device *dev;
struct sysmmu_drvdata *data;
- dev = &pdev->dev;
+ data = dev_get_drvdata(owner->sysmmu);
- data = kzalloc(sizeof(*data), GFP_KERNEL);
- if (!data) {
- dev_dbg(dev, "Not enough memory\n");
- ret = -ENOMEM;
- goto err_alloc;
+ spin_lock_irqsave(&data->lock, flags);
+ if (is_sysmmu_active(data)) {
+ if (!IS_ERR(data->clk_master))
+ clk_enable(data->clk_master);
+ if (sysmmu_block(data->sfrbase)) {
+ __sysmmu_tlb_invalidate(data->sfrbase);
+ sysmmu_unblock(data->sfrbase);
+ }
+ if (!IS_ERR(data->clk_master))
+ clk_disable(data->clk_master);
+ } else {
+ dev_dbg(dev, "disabled. Skipping TLB invalidation\n");
+ }
+ spin_unlock_irqrestore(&data->lock, flags);
+}
+
+static int __init exynos_sysmmu_probe(struct platform_device *pdev)
+{
+ int irq, ret;
+ struct device *dev = &pdev->dev;
+ struct sysmmu_drvdata *data;
+ struct resource *res;
+
+ data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ data->sfrbase = devm_ioremap_resource(dev, res);
+ if (IS_ERR(data->sfrbase))
+ return PTR_ERR(data->sfrbase);
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq <= 0) {
+ dev_err(dev, "Unable to find IRQ resource\n");
+ return irq;
}
- dev_set_drvdata(dev, data);
- data->nsfrs = pdev->num_resources / 2;
- data->sfrbases = kmalloc(sizeof(*data->sfrbases) * data->nsfrs,
- GFP_KERNEL);
- if (data->sfrbases == NULL) {
- dev_dbg(dev, "Not enough memory\n");
- ret = -ENOMEM;
- goto err_init;
+ ret = devm_request_irq(dev, irq, exynos_sysmmu_irq, 0,
+ dev_name(dev), data);
+ if (ret) {
+ dev_err(dev, "Unabled to register handler of irq %d\n", irq);
+ return ret;
}
- for (i = 0; i < data->nsfrs; i++) {
- struct resource *res;
- res = platform_get_resource(pdev, IORESOURCE_MEM, i);
- if (!res) {
- dev_dbg(dev, "Unable to find IOMEM region\n");
- ret = -ENOENT;
- goto err_res;
- }
-
- data->sfrbases[i] = ioremap(res->start, resource_size(res));
- if (!data->sfrbases[i]) {
- dev_dbg(dev, "Unable to map IOMEM @ PA:%#x\n",
- res->start);
- ret = -ENOENT;
- goto err_res;
- }
- }
-
- for (i = 0; i < data->nsfrs; i++) {
- ret = platform_get_irq(pdev, i);
- if (ret <= 0) {
- dev_dbg(dev, "Unable to find IRQ resource\n");
- goto err_irq;
- }
-
- ret = request_irq(ret, exynos_sysmmu_irq, 0,
- dev_name(dev), data);
+ data->clk = devm_clk_get(dev, "sysmmu");
+ if (IS_ERR(data->clk)) {
+ dev_err(dev, "Failed to get clock!\n");
+ return PTR_ERR(data->clk);
+ } else {
+ ret = clk_prepare(data->clk);
if (ret) {
- dev_dbg(dev, "Unabled to register interrupt handler\n");
- goto err_irq;
+ dev_err(dev, "Failed to prepare clk\n");
+ return ret;
}
}
- if (dev_get_platdata(dev)) {
- char *deli, *beg;
- struct sysmmu_platform_data *platdata = dev_get_platdata(dev);
-
- beg = platdata->clockname;
-
- for (deli = beg; (*deli != '\0') && (*deli != ','); deli++)
- /* NOTHING */;
-
- if (*deli == '\0')
- deli = NULL;
- else
- *deli = '\0';
-
- data->clk[0] = clk_get(dev, beg);
- if (IS_ERR(data->clk[0])) {
- data->clk[0] = NULL;
- dev_dbg(dev, "No clock descriptor registered\n");
+ data->clk_master = devm_clk_get(dev, "master");
+ if (!IS_ERR(data->clk_master)) {
+ ret = clk_prepare(data->clk_master);
+ if (ret) {
+ clk_unprepare(data->clk);
+ dev_err(dev, "Failed to prepare master's clk\n");
+ return ret;
}
-
- if (data->clk[0] && deli) {
- *deli = ',';
- data->clk[1] = clk_get(dev, deli + 1);
- if (IS_ERR(data->clk[1]))
- data->clk[1] = NULL;
- }
-
- data->dbgname = platdata->dbgname;
}
data->sysmmu = dev;
- rwlock_init(&data->lock);
- INIT_LIST_HEAD(&data->node);
+ spin_lock_init(&data->lock);
- __set_fault_handler(data, &default_fault_handler);
+ platform_set_drvdata(pdev, data);
- if (dev->parent)
- pm_runtime_enable(dev);
+ pm_runtime_enable(dev);
- dev_dbg(dev, "(%s) Initialized\n", data->dbgname);
return 0;
-err_irq:
- while (i-- > 0) {
- int irq;
-
- irq = platform_get_irq(pdev, i);
- free_irq(irq, data);
- }
-err_res:
- while (data->nsfrs-- > 0)
- iounmap(data->sfrbases[data->nsfrs]);
- kfree(data->sfrbases);
-err_init:
- kfree(data);
-err_alloc:
- dev_err(dev, "Failed to initialize\n");
- return ret;
}
-static struct platform_driver exynos_sysmmu_driver = {
- .probe = exynos_sysmmu_probe,
- .driver = {
+static const struct of_device_id sysmmu_of_match[] __initconst = {
+ { .compatible = "samsung,exynos-sysmmu", },
+ { },
+};
+
+static struct platform_driver exynos_sysmmu_driver __refdata = {
+ .probe = exynos_sysmmu_probe,
+ .driver = {
.owner = THIS_MODULE,
.name = "exynos-sysmmu",
+ .of_match_table = sysmmu_of_match,
}
};
@@ -662,21 +700,32 @@
static int exynos_iommu_domain_init(struct iommu_domain *domain)
{
struct exynos_iommu_domain *priv;
+ int i;
priv = kzalloc(sizeof(*priv), GFP_KERNEL);
if (!priv)
return -ENOMEM;
- priv->pgtable = (unsigned long *)__get_free_pages(
- GFP_KERNEL | __GFP_ZERO, 2);
+ priv->pgtable = (sysmmu_pte_t *)__get_free_pages(GFP_KERNEL, 2);
if (!priv->pgtable)
goto err_pgtable;
- priv->lv2entcnt = (short *)__get_free_pages(
- GFP_KERNEL | __GFP_ZERO, 1);
+ priv->lv2entcnt = (short *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, 1);
if (!priv->lv2entcnt)
goto err_counter;
+ /* w/a of System MMU v3.3 to prevent caching 1MiB mapping */
+ for (i = 0; i < NUM_LV1ENTRIES; i += 8) {
+ priv->pgtable[i + 0] = ZERO_LV2LINK;
+ priv->pgtable[i + 1] = ZERO_LV2LINK;
+ priv->pgtable[i + 2] = ZERO_LV2LINK;
+ priv->pgtable[i + 3] = ZERO_LV2LINK;
+ priv->pgtable[i + 4] = ZERO_LV2LINK;
+ priv->pgtable[i + 5] = ZERO_LV2LINK;
+ priv->pgtable[i + 6] = ZERO_LV2LINK;
+ priv->pgtable[i + 7] = ZERO_LV2LINK;
+ }
+
pgtable_flush(priv->pgtable, priv->pgtable + NUM_LV1ENTRIES);
spin_lock_init(&priv->lock);
@@ -700,7 +749,7 @@
static void exynos_iommu_domain_destroy(struct iommu_domain *domain)
{
struct exynos_iommu_domain *priv = domain->priv;
- struct sysmmu_drvdata *data;
+ struct exynos_iommu_owner *owner;
unsigned long flags;
int i;
@@ -708,16 +757,20 @@
spin_lock_irqsave(&priv->lock, flags);
- list_for_each_entry(data, &priv->clients, node) {
- while (!exynos_sysmmu_disable(data->dev))
+ list_for_each_entry(owner, &priv->clients, client) {
+ while (!exynos_sysmmu_disable(owner->dev))
; /* until System MMU is actually disabled */
}
+ while (!list_empty(&priv->clients))
+ list_del_init(priv->clients.next);
+
spin_unlock_irqrestore(&priv->lock, flags);
for (i = 0; i < NUM_LV1ENTRIES; i++)
if (lv1ent_page(priv->pgtable + i))
- kfree(__va(lv2table_base(priv->pgtable + i)));
+ kmem_cache_free(lv2table_kmem_cache,
+ phys_to_virt(lv2table_base(priv->pgtable + i)));
free_pages((unsigned long)priv->pgtable, 2);
free_pages((unsigned long)priv->lv2entcnt, 1);
@@ -728,114 +781,134 @@
static int exynos_iommu_attach_device(struct iommu_domain *domain,
struct device *dev)
{
- struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
+ struct exynos_iommu_owner *owner = dev->archdata.iommu;
struct exynos_iommu_domain *priv = domain->priv;
+ phys_addr_t pagetable = virt_to_phys(priv->pgtable);
unsigned long flags;
int ret;
- ret = pm_runtime_get_sync(data->sysmmu);
- if (ret < 0)
- return ret;
-
- ret = 0;
-
spin_lock_irqsave(&priv->lock, flags);
- ret = __exynos_sysmmu_enable(data, __pa(priv->pgtable), domain);
-
+ ret = __exynos_sysmmu_enable(dev, pagetable, domain);
if (ret == 0) {
- /* 'data->node' must not be appeared in priv->clients */
- BUG_ON(!list_empty(&data->node));
- data->dev = dev;
- list_add_tail(&data->node, &priv->clients);
+ list_add_tail(&owner->client, &priv->clients);
+ owner->domain = domain;
}
spin_unlock_irqrestore(&priv->lock, flags);
if (ret < 0) {
- dev_err(dev, "%s: Failed to attach IOMMU with pgtable %#lx\n",
- __func__, __pa(priv->pgtable));
- pm_runtime_put(data->sysmmu);
- } else if (ret > 0) {
- dev_dbg(dev, "%s: IOMMU with pgtable 0x%lx already attached\n",
- __func__, __pa(priv->pgtable));
- } else {
- dev_dbg(dev, "%s: Attached new IOMMU with pgtable 0x%lx\n",
- __func__, __pa(priv->pgtable));
+ dev_err(dev, "%s: Failed to attach IOMMU with pgtable %pa\n",
+ __func__, &pagetable);
+ return ret;
}
+ dev_dbg(dev, "%s: Attached IOMMU with pgtable %pa %s\n",
+ __func__, &pagetable, (ret == 0) ? "" : ", again");
+
return ret;
}
static void exynos_iommu_detach_device(struct iommu_domain *domain,
struct device *dev)
{
- struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
+ struct exynos_iommu_owner *owner;
struct exynos_iommu_domain *priv = domain->priv;
- struct list_head *pos;
+ phys_addr_t pagetable = virt_to_phys(priv->pgtable);
unsigned long flags;
- bool found = false;
spin_lock_irqsave(&priv->lock, flags);
- list_for_each(pos, &priv->clients) {
- if (list_entry(pos, struct sysmmu_drvdata, node) == data) {
- found = true;
+ list_for_each_entry(owner, &priv->clients, client) {
+ if (owner == dev->archdata.iommu) {
+ if (exynos_sysmmu_disable(dev)) {
+ list_del_init(&owner->client);
+ owner->domain = NULL;
+ }
break;
}
}
- if (!found)
- goto finish;
-
- if (__exynos_sysmmu_disable(data)) {
- dev_dbg(dev, "%s: Detached IOMMU with pgtable %#lx\n",
- __func__, __pa(priv->pgtable));
- list_del_init(&data->node);
-
- } else {
- dev_dbg(dev, "%s: Detaching IOMMU with pgtable %#lx delayed",
- __func__, __pa(priv->pgtable));
- }
-
-finish:
spin_unlock_irqrestore(&priv->lock, flags);
- if (found)
- pm_runtime_put(data->sysmmu);
+ if (owner == dev->archdata.iommu)
+ dev_dbg(dev, "%s: Detached IOMMU with pgtable %pa\n",
+ __func__, &pagetable);
+ else
+ dev_err(dev, "%s: No IOMMU is attached\n", __func__);
}
-static unsigned long *alloc_lv2entry(unsigned long *sent, unsigned long iova,
- short *pgcounter)
+static sysmmu_pte_t *alloc_lv2entry(struct exynos_iommu_domain *priv,
+ sysmmu_pte_t *sent, sysmmu_iova_t iova, short *pgcounter)
{
+ if (lv1ent_section(sent)) {
+ WARN(1, "Trying mapping on %#08x mapped with 1MiB page", iova);
+ return ERR_PTR(-EADDRINUSE);
+ }
+
if (lv1ent_fault(sent)) {
- unsigned long *pent;
+ sysmmu_pte_t *pent;
+ bool need_flush_flpd_cache = lv1ent_zero(sent);
- pent = kzalloc(LV2TABLE_SIZE, GFP_ATOMIC);
- BUG_ON((unsigned long)pent & (LV2TABLE_SIZE - 1));
+ pent = kmem_cache_zalloc(lv2table_kmem_cache, GFP_ATOMIC);
+ BUG_ON((unsigned int)pent & (LV2TABLE_SIZE - 1));
if (!pent)
- return NULL;
+ return ERR_PTR(-ENOMEM);
- *sent = mk_lv1ent_page(__pa(pent));
+ *sent = mk_lv1ent_page(virt_to_phys(pent));
*pgcounter = NUM_LV2ENTRIES;
pgtable_flush(pent, pent + NUM_LV2ENTRIES);
pgtable_flush(sent, sent + 1);
+
+ /*
+ * If pretched SLPD is a fault SLPD in zero_l2_table, FLPD cache
+ * may caches the address of zero_l2_table. This function
+ * replaces the zero_l2_table with new L2 page table to write
+ * valid mappings.
+ * Accessing the valid area may cause page fault since FLPD
+ * cache may still caches zero_l2_table for the valid area
+ * instead of new L2 page table that have the mapping
+ * information of the valid area
+ * Thus any replacement of zero_l2_table with other valid L2
+ * page table must involve FLPD cache invalidation for System
+ * MMU v3.3.
+ * FLPD cache invalidation is performed with TLB invalidation
+ * by VPN without blocking. It is safe to invalidate TLB without
+ * blocking because the target address of TLB invalidation is
+ * not currently mapped.
+ */
+ if (need_flush_flpd_cache) {
+ struct exynos_iommu_owner *owner;
+
+ spin_lock(&priv->lock);
+ list_for_each_entry(owner, &priv->clients, client)
+ sysmmu_tlb_invalidate_flpdcache(
+ owner->dev, iova);
+ spin_unlock(&priv->lock);
+ }
}
return page_entry(sent, iova);
}
-static int lv1set_section(unsigned long *sent, phys_addr_t paddr, short *pgcnt)
+static int lv1set_section(struct exynos_iommu_domain *priv,
+ sysmmu_pte_t *sent, sysmmu_iova_t iova,
+ phys_addr_t paddr, short *pgcnt)
{
- if (lv1ent_section(sent))
+ if (lv1ent_section(sent)) {
+ WARN(1, "Trying mapping on 1MiB@%#08x that is mapped",
+ iova);
return -EADDRINUSE;
+ }
if (lv1ent_page(sent)) {
- if (*pgcnt != NUM_LV2ENTRIES)
+ if (*pgcnt != NUM_LV2ENTRIES) {
+ WARN(1, "Trying mapping on 1MiB@%#08x that is mapped",
+ iova);
return -EADDRINUSE;
+ }
- kfree(page_entry(sent, 0));
-
+ kmem_cache_free(lv2table_kmem_cache, page_entry(sent, 0));
*pgcnt = 0;
}
@@ -843,14 +916,26 @@
pgtable_flush(sent, sent + 1);
+ spin_lock(&priv->lock);
+ if (lv1ent_page_zero(sent)) {
+ struct exynos_iommu_owner *owner;
+ /*
+ * Flushing FLPD cache in System MMU v3.3 that may cache a FLPD
+ * entry by speculative prefetch of SLPD which has no mapping.
+ */
+ list_for_each_entry(owner, &priv->clients, client)
+ sysmmu_tlb_invalidate_flpdcache(owner->dev, iova);
+ }
+ spin_unlock(&priv->lock);
+
return 0;
}
-static int lv2set_page(unsigned long *pent, phys_addr_t paddr, size_t size,
+static int lv2set_page(sysmmu_pte_t *pent, phys_addr_t paddr, size_t size,
short *pgcnt)
{
if (size == SPAGE_SIZE) {
- if (!lv2ent_fault(pent))
+ if (WARN_ON(!lv2ent_fault(pent)))
return -EADDRINUSE;
*pent = mk_lv2ent_spage(paddr);
@@ -858,9 +943,11 @@
*pgcnt -= 1;
} else { /* size == LPAGE_SIZE */
int i;
+
for (i = 0; i < SPAGES_PER_LPAGE; i++, pent++) {
- if (!lv2ent_fault(pent)) {
- memset(pent, 0, sizeof(*pent) * i);
+ if (WARN_ON(!lv2ent_fault(pent))) {
+ if (i > 0)
+ memset(pent - i, 0, sizeof(*pent) * i);
return -EADDRINUSE;
}
@@ -873,11 +960,38 @@
return 0;
}
-static int exynos_iommu_map(struct iommu_domain *domain, unsigned long iova,
+/*
+ * *CAUTION* to the I/O virtual memory managers that support exynos-iommu:
+ *
+ * System MMU v3.x have an advanced logic to improve address translation
+ * performance with caching more page table entries by a page table walk.
+ * However, the logic has a bug that caching fault page table entries and System
+ * MMU reports page fault if the cached fault entry is hit even though the fault
+ * entry is updated to a valid entry after the entry is cached.
+ * To prevent caching fault page table entries which may be updated to valid
+ * entries later, the virtual memory manager should care about the w/a about the
+ * problem. The followings describe w/a.
+ *
+ * Any two consecutive I/O virtual address regions must have a hole of 128KiB
+ * in maximum to prevent misbehavior of System MMU 3.x. (w/a of h/w bug)
+ *
+ * Precisely, any start address of I/O virtual region must be aligned by
+ * the following sizes for System MMU v3.1 and v3.2.
+ * System MMU v3.1: 128KiB
+ * System MMU v3.2: 256KiB
+ *
+ * Because System MMU v3.3 caches page table entries more aggressively, it needs
+ * more w/a.
+ * - Any two consecutive I/O virtual regions must be have a hole of larger size
+ * than or equal size to 128KiB.
+ * - Start address of an I/O virtual region must be aligned by 128KiB.
+ */
+static int exynos_iommu_map(struct iommu_domain *domain, unsigned long l_iova,
phys_addr_t paddr, size_t size, int prot)
{
struct exynos_iommu_domain *priv = domain->priv;
- unsigned long *entry;
+ sysmmu_pte_t *entry;
+ sysmmu_iova_t iova = (sysmmu_iova_t)l_iova;
unsigned long flags;
int ret = -ENOMEM;
@@ -888,38 +1002,52 @@
entry = section_entry(priv->pgtable, iova);
if (size == SECT_SIZE) {
- ret = lv1set_section(entry, paddr,
+ ret = lv1set_section(priv, entry, iova, paddr,
&priv->lv2entcnt[lv1ent_offset(iova)]);
} else {
- unsigned long *pent;
+ sysmmu_pte_t *pent;
- pent = alloc_lv2entry(entry, iova,
+ pent = alloc_lv2entry(priv, entry, iova,
&priv->lv2entcnt[lv1ent_offset(iova)]);
- if (!pent)
- ret = -ENOMEM;
+ if (IS_ERR(pent))
+ ret = PTR_ERR(pent);
else
ret = lv2set_page(pent, paddr, size,
&priv->lv2entcnt[lv1ent_offset(iova)]);
}
- if (ret) {
- pr_debug("%s: Failed to map iova 0x%lx/0x%x bytes\n",
- __func__, iova, size);
- }
+ if (ret)
+ pr_err("%s: Failed(%d) to map %#zx bytes @ %#x\n",
+ __func__, ret, size, iova);
spin_unlock_irqrestore(&priv->pgtablelock, flags);
return ret;
}
+static void exynos_iommu_tlb_invalidate_entry(struct exynos_iommu_domain *priv,
+ sysmmu_iova_t iova, size_t size)
+{
+ struct exynos_iommu_owner *owner;
+ unsigned long flags;
+
+ spin_lock_irqsave(&priv->lock, flags);
+
+ list_for_each_entry(owner, &priv->clients, client)
+ sysmmu_tlb_invalidate_entry(owner->dev, iova, size);
+
+ spin_unlock_irqrestore(&priv->lock, flags);
+}
+
static size_t exynos_iommu_unmap(struct iommu_domain *domain,
- unsigned long iova, size_t size)
+ unsigned long l_iova, size_t size)
{
struct exynos_iommu_domain *priv = domain->priv;
- struct sysmmu_drvdata *data;
+ sysmmu_iova_t iova = (sysmmu_iova_t)l_iova;
+ sysmmu_pte_t *ent;
+ size_t err_pgsize;
unsigned long flags;
- unsigned long *ent;
BUG_ON(priv->pgtable == NULL);
@@ -928,9 +1056,12 @@
ent = section_entry(priv->pgtable, iova);
if (lv1ent_section(ent)) {
- BUG_ON(size < SECT_SIZE);
+ if (WARN_ON(size < SECT_SIZE)) {
+ err_pgsize = SECT_SIZE;
+ goto err;
+ }
- *ent = 0;
+ *ent = ZERO_LV2LINK; /* w/a for h/w bug in Sysmem MMU v3.3 */
pgtable_flush(ent, ent + 1);
size = SECT_SIZE;
goto done;
@@ -954,34 +1085,42 @@
if (lv2ent_small(ent)) {
*ent = 0;
size = SPAGE_SIZE;
+ pgtable_flush(ent, ent + 1);
priv->lv2entcnt[lv1ent_offset(iova)] += 1;
goto done;
}
/* lv1ent_large(ent) == true here */
- BUG_ON(size < LPAGE_SIZE);
+ if (WARN_ON(size < LPAGE_SIZE)) {
+ err_pgsize = LPAGE_SIZE;
+ goto err;
+ }
memset(ent, 0, sizeof(*ent) * SPAGES_PER_LPAGE);
+ pgtable_flush(ent, ent + SPAGES_PER_LPAGE);
size = LPAGE_SIZE;
priv->lv2entcnt[lv1ent_offset(iova)] += SPAGES_PER_LPAGE;
done:
spin_unlock_irqrestore(&priv->pgtablelock, flags);
- spin_lock_irqsave(&priv->lock, flags);
- list_for_each_entry(data, &priv->clients, node)
- sysmmu_tlb_invalidate_entry(data->dev, iova);
- spin_unlock_irqrestore(&priv->lock, flags);
-
+ exynos_iommu_tlb_invalidate_entry(priv, iova, size);
return size;
+err:
+ spin_unlock_irqrestore(&priv->pgtablelock, flags);
+
+ pr_err("%s: Failed: size(%#zx) @ %#x is smaller than page size %#zx\n",
+ __func__, size, iova, err_pgsize);
+
+ return 0;
}
static phys_addr_t exynos_iommu_iova_to_phys(struct iommu_domain *domain,
dma_addr_t iova)
{
struct exynos_iommu_domain *priv = domain->priv;
- unsigned long *entry;
+ sysmmu_pte_t *entry;
unsigned long flags;
phys_addr_t phys = 0;
@@ -1005,6 +1144,32 @@
return phys;
}
+static int exynos_iommu_add_device(struct device *dev)
+{
+ struct iommu_group *group;
+ int ret;
+
+ group = iommu_group_get(dev);
+
+ if (!group) {
+ group = iommu_group_alloc();
+ if (IS_ERR(group)) {
+ dev_err(dev, "Failed to allocate IOMMU group\n");
+ return PTR_ERR(group);
+ }
+ }
+
+ ret = iommu_group_add_device(group, dev);
+ iommu_group_put(group);
+
+ return ret;
+}
+
+static void exynos_iommu_remove_device(struct device *dev)
+{
+ iommu_group_remove_device(dev);
+}
+
static struct iommu_ops exynos_iommu_ops = {
.domain_init = exynos_iommu_domain_init,
.domain_destroy = exynos_iommu_domain_destroy,
@@ -1013,6 +1178,8 @@
.map = exynos_iommu_map,
.unmap = exynos_iommu_unmap,
.iova_to_phys = exynos_iommu_iova_to_phys,
+ .add_device = exynos_iommu_add_device,
+ .remove_device = exynos_iommu_remove_device,
.pgsize_bitmap = SECT_SIZE | LPAGE_SIZE | SPAGE_SIZE,
};
@@ -1020,11 +1187,41 @@
{
int ret;
+ lv2table_kmem_cache = kmem_cache_create("exynos-iommu-lv2table",
+ LV2TABLE_SIZE, LV2TABLE_SIZE, 0, NULL);
+ if (!lv2table_kmem_cache) {
+ pr_err("%s: Failed to create kmem cache\n", __func__);
+ return -ENOMEM;
+ }
+
ret = platform_driver_register(&exynos_sysmmu_driver);
+ if (ret) {
+ pr_err("%s: Failed to register driver\n", __func__);
+ goto err_reg_driver;
+ }
- if (ret == 0)
- bus_set_iommu(&platform_bus_type, &exynos_iommu_ops);
+ zero_lv2_table = kmem_cache_zalloc(lv2table_kmem_cache, GFP_KERNEL);
+ if (zero_lv2_table == NULL) {
+ pr_err("%s: Failed to allocate zero level2 page table\n",
+ __func__);
+ ret = -ENOMEM;
+ goto err_zero_lv2;
+ }
+ ret = bus_set_iommu(&platform_bus_type, &exynos_iommu_ops);
+ if (ret) {
+ pr_err("%s: Failed to register exynos-iommu driver.\n",
+ __func__);
+ goto err_set_iommu;
+ }
+
+ return 0;
+err_set_iommu:
+ kmem_cache_free(lv2table_kmem_cache, zero_lv2_table);
+err_zero_lv2:
+ platform_driver_unregister(&exynos_sysmmu_driver);
+err_reg_driver:
+ kmem_cache_destroy(lv2table_kmem_cache);
return ret;
}
subsys_initcall(exynos_iommu_init);
diff --git a/drivers/iommu/fsl_pamu.c b/drivers/iommu/fsl_pamu.c
index cba0498..b99dd88 100644
--- a/drivers/iommu/fsl_pamu.c
+++ b/drivers/iommu/fsl_pamu.c
@@ -592,8 +592,7 @@
/* advance to next node in cache hierarchy */
node = of_find_node_by_phandle(*prop);
if (!node) {
- pr_debug("Invalid node for cache hierarchy %s\n",
- node->full_name);
+ pr_debug("Invalid node for cache hierarchy\n");
return ~(u32)0;
}
}
diff --git a/drivers/iommu/ipmmu-vmsa.c b/drivers/iommu/ipmmu-vmsa.c
new file mode 100644
index 0000000..53cde08
--- /dev/null
+++ b/drivers/iommu/ipmmu-vmsa.c
@@ -0,0 +1,1255 @@
+/*
+ * IPMMU VMSA
+ *
+ * Copyright (C) 2014 Renesas Electronics Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ */
+
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/export.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/iommu.h>
+#include <linux/module.h>
+#include <linux/platform_data/ipmmu-vmsa.h>
+#include <linux/platform_device.h>
+#include <linux/sizes.h>
+#include <linux/slab.h>
+
+#include <asm/dma-iommu.h>
+#include <asm/pgalloc.h>
+
+struct ipmmu_vmsa_device {
+ struct device *dev;
+ void __iomem *base;
+ struct list_head list;
+
+ const struct ipmmu_vmsa_platform_data *pdata;
+ unsigned int num_utlbs;
+
+ struct dma_iommu_mapping *mapping;
+};
+
+struct ipmmu_vmsa_domain {
+ struct ipmmu_vmsa_device *mmu;
+ struct iommu_domain *io_domain;
+
+ unsigned int context_id;
+ spinlock_t lock; /* Protects mappings */
+ pgd_t *pgd;
+};
+
+struct ipmmu_vmsa_archdata {
+ struct ipmmu_vmsa_device *mmu;
+ unsigned int utlb;
+};
+
+static DEFINE_SPINLOCK(ipmmu_devices_lock);
+static LIST_HEAD(ipmmu_devices);
+
+#define TLB_LOOP_TIMEOUT 100 /* 100us */
+
+/* -----------------------------------------------------------------------------
+ * Registers Definition
+ */
+
+#define IM_CTX_SIZE 0x40
+
+#define IMCTR 0x0000
+#define IMCTR_TRE (1 << 17)
+#define IMCTR_AFE (1 << 16)
+#define IMCTR_RTSEL_MASK (3 << 4)
+#define IMCTR_RTSEL_SHIFT 4
+#define IMCTR_TREN (1 << 3)
+#define IMCTR_INTEN (1 << 2)
+#define IMCTR_FLUSH (1 << 1)
+#define IMCTR_MMUEN (1 << 0)
+
+#define IMCAAR 0x0004
+
+#define IMTTBCR 0x0008
+#define IMTTBCR_EAE (1 << 31)
+#define IMTTBCR_PMB (1 << 30)
+#define IMTTBCR_SH1_NON_SHAREABLE (0 << 28)
+#define IMTTBCR_SH1_OUTER_SHAREABLE (2 << 28)
+#define IMTTBCR_SH1_INNER_SHAREABLE (3 << 28)
+#define IMTTBCR_SH1_MASK (3 << 28)
+#define IMTTBCR_ORGN1_NC (0 << 26)
+#define IMTTBCR_ORGN1_WB_WA (1 << 26)
+#define IMTTBCR_ORGN1_WT (2 << 26)
+#define IMTTBCR_ORGN1_WB (3 << 26)
+#define IMTTBCR_ORGN1_MASK (3 << 26)
+#define IMTTBCR_IRGN1_NC (0 << 24)
+#define IMTTBCR_IRGN1_WB_WA (1 << 24)
+#define IMTTBCR_IRGN1_WT (2 << 24)
+#define IMTTBCR_IRGN1_WB (3 << 24)
+#define IMTTBCR_IRGN1_MASK (3 << 24)
+#define IMTTBCR_TSZ1_MASK (7 << 16)
+#define IMTTBCR_TSZ1_SHIFT 16
+#define IMTTBCR_SH0_NON_SHAREABLE (0 << 12)
+#define IMTTBCR_SH0_OUTER_SHAREABLE (2 << 12)
+#define IMTTBCR_SH0_INNER_SHAREABLE (3 << 12)
+#define IMTTBCR_SH0_MASK (3 << 12)
+#define IMTTBCR_ORGN0_NC (0 << 10)
+#define IMTTBCR_ORGN0_WB_WA (1 << 10)
+#define IMTTBCR_ORGN0_WT (2 << 10)
+#define IMTTBCR_ORGN0_WB (3 << 10)
+#define IMTTBCR_ORGN0_MASK (3 << 10)
+#define IMTTBCR_IRGN0_NC (0 << 8)
+#define IMTTBCR_IRGN0_WB_WA (1 << 8)
+#define IMTTBCR_IRGN0_WT (2 << 8)
+#define IMTTBCR_IRGN0_WB (3 << 8)
+#define IMTTBCR_IRGN0_MASK (3 << 8)
+#define IMTTBCR_SL0_LVL_2 (0 << 4)
+#define IMTTBCR_SL0_LVL_1 (1 << 4)
+#define IMTTBCR_TSZ0_MASK (7 << 0)
+#define IMTTBCR_TSZ0_SHIFT O
+
+#define IMBUSCR 0x000c
+#define IMBUSCR_DVM (1 << 2)
+#define IMBUSCR_BUSSEL_SYS (0 << 0)
+#define IMBUSCR_BUSSEL_CCI (1 << 0)
+#define IMBUSCR_BUSSEL_IMCAAR (2 << 0)
+#define IMBUSCR_BUSSEL_CCI_IMCAAR (3 << 0)
+#define IMBUSCR_BUSSEL_MASK (3 << 0)
+
+#define IMTTLBR0 0x0010
+#define IMTTUBR0 0x0014
+#define IMTTLBR1 0x0018
+#define IMTTUBR1 0x001c
+
+#define IMSTR 0x0020
+#define IMSTR_ERRLVL_MASK (3 << 12)
+#define IMSTR_ERRLVL_SHIFT 12
+#define IMSTR_ERRCODE_TLB_FORMAT (1 << 8)
+#define IMSTR_ERRCODE_ACCESS_PERM (4 << 8)
+#define IMSTR_ERRCODE_SECURE_ACCESS (5 << 8)
+#define IMSTR_ERRCODE_MASK (7 << 8)
+#define IMSTR_MHIT (1 << 4)
+#define IMSTR_ABORT (1 << 2)
+#define IMSTR_PF (1 << 1)
+#define IMSTR_TF (1 << 0)
+
+#define IMMAIR0 0x0028
+#define IMMAIR1 0x002c
+#define IMMAIR_ATTR_MASK 0xff
+#define IMMAIR_ATTR_DEVICE 0x04
+#define IMMAIR_ATTR_NC 0x44
+#define IMMAIR_ATTR_WBRWA 0xff
+#define IMMAIR_ATTR_SHIFT(n) ((n) << 3)
+#define IMMAIR_ATTR_IDX_NC 0
+#define IMMAIR_ATTR_IDX_WBRWA 1
+#define IMMAIR_ATTR_IDX_DEV 2
+
+#define IMEAR 0x0030
+
+#define IMPCTR 0x0200
+#define IMPSTR 0x0208
+#define IMPEAR 0x020c
+#define IMPMBA(n) (0x0280 + ((n) * 4))
+#define IMPMBD(n) (0x02c0 + ((n) * 4))
+
+#define IMUCTR(n) (0x0300 + ((n) * 16))
+#define IMUCTR_FIXADDEN (1 << 31)
+#define IMUCTR_FIXADD_MASK (0xff << 16)
+#define IMUCTR_FIXADD_SHIFT 16
+#define IMUCTR_TTSEL_MMU(n) ((n) << 4)
+#define IMUCTR_TTSEL_PMB (8 << 4)
+#define IMUCTR_TTSEL_MASK (15 << 4)
+#define IMUCTR_FLUSH (1 << 1)
+#define IMUCTR_MMUEN (1 << 0)
+
+#define IMUASID(n) (0x0308 + ((n) * 16))
+#define IMUASID_ASID8_MASK (0xff << 8)
+#define IMUASID_ASID8_SHIFT 8
+#define IMUASID_ASID0_MASK (0xff << 0)
+#define IMUASID_ASID0_SHIFT 0
+
+/* -----------------------------------------------------------------------------
+ * Page Table Bits
+ */
+
+/*
+ * VMSA states in section B3.6.3 "Control of Secure or Non-secure memory access,
+ * Long-descriptor format" that the NStable bit being set in a table descriptor
+ * will result in the NStable and NS bits of all child entries being ignored and
+ * considered as being set. The IPMMU seems not to comply with this, as it
+ * generates a secure access page fault if any of the NStable and NS bits isn't
+ * set when running in non-secure mode.
+ */
+#ifndef PMD_NSTABLE
+#define PMD_NSTABLE (_AT(pmdval_t, 1) << 63)
+#endif
+
+#define ARM_VMSA_PTE_XN (((pteval_t)3) << 53)
+#define ARM_VMSA_PTE_CONT (((pteval_t)1) << 52)
+#define ARM_VMSA_PTE_AF (((pteval_t)1) << 10)
+#define ARM_VMSA_PTE_SH_NS (((pteval_t)0) << 8)
+#define ARM_VMSA_PTE_SH_OS (((pteval_t)2) << 8)
+#define ARM_VMSA_PTE_SH_IS (((pteval_t)3) << 8)
+#define ARM_VMSA_PTE_SH_MASK (((pteval_t)3) << 8)
+#define ARM_VMSA_PTE_NS (((pteval_t)1) << 5)
+#define ARM_VMSA_PTE_PAGE (((pteval_t)3) << 0)
+
+/* Stage-1 PTE */
+#define ARM_VMSA_PTE_nG (((pteval_t)1) << 11)
+#define ARM_VMSA_PTE_AP_UNPRIV (((pteval_t)1) << 6)
+#define ARM_VMSA_PTE_AP_RDONLY (((pteval_t)2) << 6)
+#define ARM_VMSA_PTE_AP_MASK (((pteval_t)3) << 6)
+#define ARM_VMSA_PTE_ATTRINDX_MASK (((pteval_t)3) << 2)
+#define ARM_VMSA_PTE_ATTRINDX_SHIFT 2
+
+#define ARM_VMSA_PTE_ATTRS_MASK \
+ (ARM_VMSA_PTE_XN | ARM_VMSA_PTE_CONT | ARM_VMSA_PTE_nG | \
+ ARM_VMSA_PTE_AF | ARM_VMSA_PTE_SH_MASK | ARM_VMSA_PTE_AP_MASK | \
+ ARM_VMSA_PTE_NS | ARM_VMSA_PTE_ATTRINDX_MASK)
+
+#define ARM_VMSA_PTE_CONT_ENTRIES 16
+#define ARM_VMSA_PTE_CONT_SIZE (PAGE_SIZE * ARM_VMSA_PTE_CONT_ENTRIES)
+
+#define IPMMU_PTRS_PER_PTE 512
+#define IPMMU_PTRS_PER_PMD 512
+#define IPMMU_PTRS_PER_PGD 4
+
+/* -----------------------------------------------------------------------------
+ * Read/Write Access
+ */
+
+static u32 ipmmu_read(struct ipmmu_vmsa_device *mmu, unsigned int offset)
+{
+ return ioread32(mmu->base + offset);
+}
+
+static void ipmmu_write(struct ipmmu_vmsa_device *mmu, unsigned int offset,
+ u32 data)
+{
+ iowrite32(data, mmu->base + offset);
+}
+
+static u32 ipmmu_ctx_read(struct ipmmu_vmsa_domain *domain, unsigned int reg)
+{
+ return ipmmu_read(domain->mmu, domain->context_id * IM_CTX_SIZE + reg);
+}
+
+static void ipmmu_ctx_write(struct ipmmu_vmsa_domain *domain, unsigned int reg,
+ u32 data)
+{
+ ipmmu_write(domain->mmu, domain->context_id * IM_CTX_SIZE + reg, data);
+}
+
+/* -----------------------------------------------------------------------------
+ * TLB and microTLB Management
+ */
+
+/* Wait for any pending TLB invalidations to complete */
+static void ipmmu_tlb_sync(struct ipmmu_vmsa_domain *domain)
+{
+ unsigned int count = 0;
+
+ while (ipmmu_ctx_read(domain, IMCTR) & IMCTR_FLUSH) {
+ cpu_relax();
+ if (++count == TLB_LOOP_TIMEOUT) {
+ dev_err_ratelimited(domain->mmu->dev,
+ "TLB sync timed out -- MMU may be deadlocked\n");
+ return;
+ }
+ udelay(1);
+ }
+}
+
+static void ipmmu_tlb_invalidate(struct ipmmu_vmsa_domain *domain)
+{
+ u32 reg;
+
+ reg = ipmmu_ctx_read(domain, IMCTR);
+ reg |= IMCTR_FLUSH;
+ ipmmu_ctx_write(domain, IMCTR, reg);
+
+ ipmmu_tlb_sync(domain);
+}
+
+/*
+ * Enable MMU translation for the microTLB.
+ */
+static void ipmmu_utlb_enable(struct ipmmu_vmsa_domain *domain,
+ unsigned int utlb)
+{
+ struct ipmmu_vmsa_device *mmu = domain->mmu;
+
+ /*
+ * TODO: Reference-count the microTLB as several bus masters can be
+ * connected to the same microTLB.
+ */
+
+ /* TODO: What should we set the ASID to ? */
+ ipmmu_write(mmu, IMUASID(utlb), 0);
+ /* TODO: Do we need to flush the microTLB ? */
+ ipmmu_write(mmu, IMUCTR(utlb),
+ IMUCTR_TTSEL_MMU(domain->context_id) | IMUCTR_FLUSH |
+ IMUCTR_MMUEN);
+}
+
+/*
+ * Disable MMU translation for the microTLB.
+ */
+static void ipmmu_utlb_disable(struct ipmmu_vmsa_domain *domain,
+ unsigned int utlb)
+{
+ struct ipmmu_vmsa_device *mmu = domain->mmu;
+
+ ipmmu_write(mmu, IMUCTR(utlb), 0);
+}
+
+static void ipmmu_flush_pgtable(struct ipmmu_vmsa_device *mmu, void *addr,
+ size_t size)
+{
+ unsigned long offset = (unsigned long)addr & ~PAGE_MASK;
+
+ /*
+ * TODO: Add support for coherent walk through CCI with DVM and remove
+ * cache handling.
+ */
+ dma_map_page(mmu->dev, virt_to_page(addr), offset, size, DMA_TO_DEVICE);
+}
+
+/* -----------------------------------------------------------------------------
+ * Domain/Context Management
+ */
+
+static int ipmmu_domain_init_context(struct ipmmu_vmsa_domain *domain)
+{
+ phys_addr_t ttbr;
+ u32 reg;
+
+ /*
+ * TODO: When adding support for multiple contexts, find an unused
+ * context.
+ */
+ domain->context_id = 0;
+
+ /* TTBR0 */
+ ipmmu_flush_pgtable(domain->mmu, domain->pgd,
+ IPMMU_PTRS_PER_PGD * sizeof(*domain->pgd));
+ ttbr = __pa(domain->pgd);
+ ipmmu_ctx_write(domain, IMTTLBR0, ttbr);
+ ipmmu_ctx_write(domain, IMTTUBR0, ttbr >> 32);
+
+ /*
+ * TTBCR
+ * We use long descriptors with inner-shareable WBWA tables and allocate
+ * the whole 32-bit VA space to TTBR0.
+ */
+ ipmmu_ctx_write(domain, IMTTBCR, IMTTBCR_EAE |
+ IMTTBCR_SH0_INNER_SHAREABLE | IMTTBCR_ORGN0_WB_WA |
+ IMTTBCR_IRGN0_WB_WA | IMTTBCR_SL0_LVL_1);
+
+ /*
+ * MAIR0
+ * We need three attributes only, non-cacheable, write-back read/write
+ * allocate and device memory.
+ */
+ reg = (IMMAIR_ATTR_NC << IMMAIR_ATTR_SHIFT(IMMAIR_ATTR_IDX_NC))
+ | (IMMAIR_ATTR_WBRWA << IMMAIR_ATTR_SHIFT(IMMAIR_ATTR_IDX_WBRWA))
+ | (IMMAIR_ATTR_DEVICE << IMMAIR_ATTR_SHIFT(IMMAIR_ATTR_IDX_DEV));
+ ipmmu_ctx_write(domain, IMMAIR0, reg);
+
+ /* IMBUSCR */
+ ipmmu_ctx_write(domain, IMBUSCR,
+ ipmmu_ctx_read(domain, IMBUSCR) &
+ ~(IMBUSCR_DVM | IMBUSCR_BUSSEL_MASK));
+
+ /*
+ * IMSTR
+ * Clear all interrupt flags.
+ */
+ ipmmu_ctx_write(domain, IMSTR, ipmmu_ctx_read(domain, IMSTR));
+
+ /*
+ * IMCTR
+ * Enable the MMU and interrupt generation. The long-descriptor
+ * translation table format doesn't use TEX remapping. Don't enable AF
+ * software management as we have no use for it. Flush the TLB as
+ * required when modifying the context registers.
+ */
+ ipmmu_ctx_write(domain, IMCTR, IMCTR_INTEN | IMCTR_FLUSH | IMCTR_MMUEN);
+
+ return 0;
+}
+
+static void ipmmu_domain_destroy_context(struct ipmmu_vmsa_domain *domain)
+{
+ /*
+ * Disable the context. Flush the TLB as required when modifying the
+ * context registers.
+ *
+ * TODO: Is TLB flush really needed ?
+ */
+ ipmmu_ctx_write(domain, IMCTR, IMCTR_FLUSH);
+ ipmmu_tlb_sync(domain);
+}
+
+/* -----------------------------------------------------------------------------
+ * Fault Handling
+ */
+
+static irqreturn_t ipmmu_domain_irq(struct ipmmu_vmsa_domain *domain)
+{
+ const u32 err_mask = IMSTR_MHIT | IMSTR_ABORT | IMSTR_PF | IMSTR_TF;
+ struct ipmmu_vmsa_device *mmu = domain->mmu;
+ u32 status;
+ u32 iova;
+
+ status = ipmmu_ctx_read(domain, IMSTR);
+ if (!(status & err_mask))
+ return IRQ_NONE;
+
+ iova = ipmmu_ctx_read(domain, IMEAR);
+
+ /*
+ * Clear the error status flags. Unlike traditional interrupt flag
+ * registers that must be cleared by writing 1, this status register
+ * seems to require 0. The error address register must be read before,
+ * otherwise its value will be 0.
+ */
+ ipmmu_ctx_write(domain, IMSTR, 0);
+
+ /* Log fatal errors. */
+ if (status & IMSTR_MHIT)
+ dev_err_ratelimited(mmu->dev, "Multiple TLB hits @0x%08x\n",
+ iova);
+ if (status & IMSTR_ABORT)
+ dev_err_ratelimited(mmu->dev, "Page Table Walk Abort @0x%08x\n",
+ iova);
+
+ if (!(status & (IMSTR_PF | IMSTR_TF)))
+ return IRQ_NONE;
+
+ /*
+ * Try to handle page faults and translation faults.
+ *
+ * TODO: We need to look up the faulty device based on the I/O VA. Use
+ * the IOMMU device for now.
+ */
+ if (!report_iommu_fault(domain->io_domain, mmu->dev, iova, 0))
+ return IRQ_HANDLED;
+
+ dev_err_ratelimited(mmu->dev,
+ "Unhandled fault: status 0x%08x iova 0x%08x\n",
+ status, iova);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t ipmmu_irq(int irq, void *dev)
+{
+ struct ipmmu_vmsa_device *mmu = dev;
+ struct iommu_domain *io_domain;
+ struct ipmmu_vmsa_domain *domain;
+
+ if (!mmu->mapping)
+ return IRQ_NONE;
+
+ io_domain = mmu->mapping->domain;
+ domain = io_domain->priv;
+
+ return ipmmu_domain_irq(domain);
+}
+
+/* -----------------------------------------------------------------------------
+ * Page Table Management
+ */
+
+#define pud_pgtable(pud) pfn_to_page(__phys_to_pfn(pud_val(pud) & PHYS_MASK))
+
+static void ipmmu_free_ptes(pmd_t *pmd)
+{
+ pgtable_t table = pmd_pgtable(*pmd);
+ __free_page(table);
+}
+
+static void ipmmu_free_pmds(pud_t *pud)
+{
+ pmd_t *pmd = pmd_offset(pud, 0);
+ pgtable_t table;
+ unsigned int i;
+
+ for (i = 0; i < IPMMU_PTRS_PER_PMD; ++i) {
+ if (!pmd_table(*pmd))
+ continue;
+
+ ipmmu_free_ptes(pmd);
+ pmd++;
+ }
+
+ table = pud_pgtable(*pud);
+ __free_page(table);
+}
+
+static void ipmmu_free_pgtables(struct ipmmu_vmsa_domain *domain)
+{
+ pgd_t *pgd, *pgd_base = domain->pgd;
+ unsigned int i;
+
+ /*
+ * Recursively free the page tables for this domain. We don't care about
+ * speculative TLB filling, because the TLB will be nuked next time this
+ * context bank is re-allocated and no devices currently map to these
+ * tables.
+ */
+ pgd = pgd_base;
+ for (i = 0; i < IPMMU_PTRS_PER_PGD; ++i) {
+ if (pgd_none(*pgd))
+ continue;
+ ipmmu_free_pmds((pud_t *)pgd);
+ pgd++;
+ }
+
+ kfree(pgd_base);
+}
+
+/*
+ * We can't use the (pgd|pud|pmd|pte)_populate or the set_(pgd|pud|pmd|pte)
+ * functions as they would flush the CPU TLB.
+ */
+
+static pte_t *ipmmu_alloc_pte(struct ipmmu_vmsa_device *mmu, pmd_t *pmd,
+ unsigned long iova)
+{
+ pte_t *pte;
+
+ if (!pmd_none(*pmd))
+ return pte_offset_kernel(pmd, iova);
+
+ pte = (pte_t *)get_zeroed_page(GFP_ATOMIC);
+ if (!pte)
+ return NULL;
+
+ ipmmu_flush_pgtable(mmu, pte, PAGE_SIZE);
+ *pmd = __pmd(__pa(pte) | PMD_NSTABLE | PMD_TYPE_TABLE);
+ ipmmu_flush_pgtable(mmu, pmd, sizeof(*pmd));
+
+ return pte + pte_index(iova);
+}
+
+static pmd_t *ipmmu_alloc_pmd(struct ipmmu_vmsa_device *mmu, pgd_t *pgd,
+ unsigned long iova)
+{
+ pud_t *pud = (pud_t *)pgd;
+ pmd_t *pmd;
+
+ if (!pud_none(*pud))
+ return pmd_offset(pud, iova);
+
+ pmd = (pmd_t *)get_zeroed_page(GFP_ATOMIC);
+ if (!pmd)
+ return NULL;
+
+ ipmmu_flush_pgtable(mmu, pmd, PAGE_SIZE);
+ *pud = __pud(__pa(pmd) | PMD_NSTABLE | PMD_TYPE_TABLE);
+ ipmmu_flush_pgtable(mmu, pud, sizeof(*pud));
+
+ return pmd + pmd_index(iova);
+}
+
+static u64 ipmmu_page_prot(unsigned int prot, u64 type)
+{
+ u64 pgprot = ARM_VMSA_PTE_XN | ARM_VMSA_PTE_nG | ARM_VMSA_PTE_AF
+ | ARM_VMSA_PTE_SH_IS | ARM_VMSA_PTE_AP_UNPRIV
+ | ARM_VMSA_PTE_NS | type;
+
+ if (!(prot & IOMMU_WRITE) && (prot & IOMMU_READ))
+ pgprot |= ARM_VMSA_PTE_AP_RDONLY;
+
+ if (prot & IOMMU_CACHE)
+ pgprot |= IMMAIR_ATTR_IDX_WBRWA << ARM_VMSA_PTE_ATTRINDX_SHIFT;
+
+ if (prot & IOMMU_EXEC)
+ pgprot &= ~ARM_VMSA_PTE_XN;
+ else if (!(prot & (IOMMU_READ | IOMMU_WRITE)))
+ /* If no access create a faulting entry to avoid TLB fills. */
+ pgprot &= ~ARM_VMSA_PTE_PAGE;
+
+ return pgprot;
+}
+
+static int ipmmu_alloc_init_pte(struct ipmmu_vmsa_device *mmu, pmd_t *pmd,
+ unsigned long iova, unsigned long pfn,
+ size_t size, int prot)
+{
+ pteval_t pteval = ipmmu_page_prot(prot, ARM_VMSA_PTE_PAGE);
+ unsigned int num_ptes = 1;
+ pte_t *pte, *start;
+ unsigned int i;
+
+ pte = ipmmu_alloc_pte(mmu, pmd, iova);
+ if (!pte)
+ return -ENOMEM;
+
+ start = pte;
+
+ /*
+ * Install the page table entries. We can be called both for a single
+ * page or for a block of 16 physically contiguous pages. In the latter
+ * case set the PTE contiguous hint.
+ */
+ if (size == SZ_64K) {
+ pteval |= ARM_VMSA_PTE_CONT;
+ num_ptes = ARM_VMSA_PTE_CONT_ENTRIES;
+ }
+
+ for (i = num_ptes; i; --i)
+ *pte++ = pfn_pte(pfn++, __pgprot(pteval));
+
+ ipmmu_flush_pgtable(mmu, start, sizeof(*pte) * num_ptes);
+
+ return 0;
+}
+
+static int ipmmu_alloc_init_pmd(struct ipmmu_vmsa_device *mmu, pmd_t *pmd,
+ unsigned long iova, unsigned long pfn,
+ int prot)
+{
+ pmdval_t pmdval = ipmmu_page_prot(prot, PMD_TYPE_SECT);
+
+ *pmd = pfn_pmd(pfn, __pgprot(pmdval));
+ ipmmu_flush_pgtable(mmu, pmd, sizeof(*pmd));
+
+ return 0;
+}
+
+static int ipmmu_create_mapping(struct ipmmu_vmsa_domain *domain,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int prot)
+{
+ struct ipmmu_vmsa_device *mmu = domain->mmu;
+ pgd_t *pgd = domain->pgd;
+ unsigned long flags;
+ unsigned long pfn;
+ pmd_t *pmd;
+ int ret;
+
+ if (!pgd)
+ return -EINVAL;
+
+ if (size & ~PAGE_MASK)
+ return -EINVAL;
+
+ if (paddr & ~((1ULL << 40) - 1))
+ return -ERANGE;
+
+ pfn = __phys_to_pfn(paddr);
+ pgd += pgd_index(iova);
+
+ /* Update the page tables. */
+ spin_lock_irqsave(&domain->lock, flags);
+
+ pmd = ipmmu_alloc_pmd(mmu, pgd, iova);
+ if (!pmd) {
+ ret = -ENOMEM;
+ goto done;
+ }
+
+ switch (size) {
+ case SZ_2M:
+ ret = ipmmu_alloc_init_pmd(mmu, pmd, iova, pfn, prot);
+ break;
+ case SZ_64K:
+ case SZ_4K:
+ ret = ipmmu_alloc_init_pte(mmu, pmd, iova, pfn, size, prot);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+done:
+ spin_unlock_irqrestore(&domain->lock, flags);
+
+ if (!ret)
+ ipmmu_tlb_invalidate(domain);
+
+ return ret;
+}
+
+static void ipmmu_clear_pud(struct ipmmu_vmsa_device *mmu, pud_t *pud)
+{
+ /* Free the page table. */
+ pgtable_t table = pud_pgtable(*pud);
+ __free_page(table);
+
+ /* Clear the PUD. */
+ *pud = __pud(0);
+ ipmmu_flush_pgtable(mmu, pud, sizeof(*pud));
+}
+
+static void ipmmu_clear_pmd(struct ipmmu_vmsa_device *mmu, pud_t *pud,
+ pmd_t *pmd)
+{
+ unsigned int i;
+
+ /* Free the page table. */
+ if (pmd_table(*pmd)) {
+ pgtable_t table = pmd_pgtable(*pmd);
+ __free_page(table);
+ }
+
+ /* Clear the PMD. */
+ *pmd = __pmd(0);
+ ipmmu_flush_pgtable(mmu, pmd, sizeof(*pmd));
+
+ /* Check whether the PUD is still needed. */
+ pmd = pmd_offset(pud, 0);
+ for (i = 0; i < IPMMU_PTRS_PER_PMD; ++i) {
+ if (!pmd_none(pmd[i]))
+ return;
+ }
+
+ /* Clear the parent PUD. */
+ ipmmu_clear_pud(mmu, pud);
+}
+
+static void ipmmu_clear_pte(struct ipmmu_vmsa_device *mmu, pud_t *pud,
+ pmd_t *pmd, pte_t *pte, unsigned int num_ptes)
+{
+ unsigned int i;
+
+ /* Clear the PTE. */
+ for (i = num_ptes; i; --i)
+ pte[i-1] = __pte(0);
+
+ ipmmu_flush_pgtable(mmu, pte, sizeof(*pte) * num_ptes);
+
+ /* Check whether the PMD is still needed. */
+ pte = pte_offset_kernel(pmd, 0);
+ for (i = 0; i < IPMMU_PTRS_PER_PTE; ++i) {
+ if (!pte_none(pte[i]))
+ return;
+ }
+
+ /* Clear the parent PMD. */
+ ipmmu_clear_pmd(mmu, pud, pmd);
+}
+
+static int ipmmu_split_pmd(struct ipmmu_vmsa_device *mmu, pmd_t *pmd)
+{
+ pte_t *pte, *start;
+ pteval_t pteval;
+ unsigned long pfn;
+ unsigned int i;
+
+ pte = (pte_t *)get_zeroed_page(GFP_ATOMIC);
+ if (!pte)
+ return -ENOMEM;
+
+ /* Copy the PMD attributes. */
+ pteval = (pmd_val(*pmd) & ARM_VMSA_PTE_ATTRS_MASK)
+ | ARM_VMSA_PTE_CONT | ARM_VMSA_PTE_PAGE;
+
+ pfn = pmd_pfn(*pmd);
+ start = pte;
+
+ for (i = IPMMU_PTRS_PER_PTE; i; --i)
+ *pte++ = pfn_pte(pfn++, __pgprot(pteval));
+
+ ipmmu_flush_pgtable(mmu, start, PAGE_SIZE);
+ *pmd = __pmd(__pa(start) | PMD_NSTABLE | PMD_TYPE_TABLE);
+ ipmmu_flush_pgtable(mmu, pmd, sizeof(*pmd));
+
+ return 0;
+}
+
+static void ipmmu_split_pte(struct ipmmu_vmsa_device *mmu, pte_t *pte)
+{
+ unsigned int i;
+
+ for (i = ARM_VMSA_PTE_CONT_ENTRIES; i; --i)
+ pte[i-1] = __pte(pte_val(*pte) & ~ARM_VMSA_PTE_CONT);
+
+ ipmmu_flush_pgtable(mmu, pte, sizeof(*pte) * ARM_VMSA_PTE_CONT_ENTRIES);
+}
+
+static int ipmmu_clear_mapping(struct ipmmu_vmsa_domain *domain,
+ unsigned long iova, size_t size)
+{
+ struct ipmmu_vmsa_device *mmu = domain->mmu;
+ unsigned long flags;
+ pgd_t *pgd = domain->pgd;
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+ int ret = 0;
+
+ if (!pgd)
+ return -EINVAL;
+
+ if (size & ~PAGE_MASK)
+ return -EINVAL;
+
+ pgd += pgd_index(iova);
+ pud = (pud_t *)pgd;
+
+ spin_lock_irqsave(&domain->lock, flags);
+
+ /* If there's no PUD or PMD we're done. */
+ if (pud_none(*pud))
+ goto done;
+
+ pmd = pmd_offset(pud, iova);
+ if (pmd_none(*pmd))
+ goto done;
+
+ /*
+ * When freeing a 2MB block just clear the PMD. In the unlikely case the
+ * block is mapped as individual pages this will free the corresponding
+ * PTE page table.
+ */
+ if (size == SZ_2M) {
+ ipmmu_clear_pmd(mmu, pud, pmd);
+ goto done;
+ }
+
+ /*
+ * If the PMD has been mapped as a section remap it as pages to allow
+ * freeing individual pages.
+ */
+ if (pmd_sect(*pmd))
+ ipmmu_split_pmd(mmu, pmd);
+
+ pte = pte_offset_kernel(pmd, iova);
+
+ /*
+ * When freeing a 64kB block just clear the PTE entries. We don't have
+ * to care about the contiguous hint of the surrounding entries.
+ */
+ if (size == SZ_64K) {
+ ipmmu_clear_pte(mmu, pud, pmd, pte, ARM_VMSA_PTE_CONT_ENTRIES);
+ goto done;
+ }
+
+ /*
+ * If the PTE has been mapped with the contiguous hint set remap it and
+ * its surrounding PTEs to allow unmapping a single page.
+ */
+ if (pte_val(*pte) & ARM_VMSA_PTE_CONT)
+ ipmmu_split_pte(mmu, pte);
+
+ /* Clear the PTE. */
+ ipmmu_clear_pte(mmu, pud, pmd, pte, 1);
+
+done:
+ spin_unlock_irqrestore(&domain->lock, flags);
+
+ if (ret)
+ ipmmu_tlb_invalidate(domain);
+
+ return 0;
+}
+
+/* -----------------------------------------------------------------------------
+ * IOMMU Operations
+ */
+
+static int ipmmu_domain_init(struct iommu_domain *io_domain)
+{
+ struct ipmmu_vmsa_domain *domain;
+
+ domain = kzalloc(sizeof(*domain), GFP_KERNEL);
+ if (!domain)
+ return -ENOMEM;
+
+ spin_lock_init(&domain->lock);
+
+ domain->pgd = kzalloc(IPMMU_PTRS_PER_PGD * sizeof(pgd_t), GFP_KERNEL);
+ if (!domain->pgd) {
+ kfree(domain);
+ return -ENOMEM;
+ }
+
+ io_domain->priv = domain;
+ domain->io_domain = io_domain;
+
+ return 0;
+}
+
+static void ipmmu_domain_destroy(struct iommu_domain *io_domain)
+{
+ struct ipmmu_vmsa_domain *domain = io_domain->priv;
+
+ /*
+ * Free the domain resources. We assume that all devices have already
+ * been detached.
+ */
+ ipmmu_domain_destroy_context(domain);
+ ipmmu_free_pgtables(domain);
+ kfree(domain);
+}
+
+static int ipmmu_attach_device(struct iommu_domain *io_domain,
+ struct device *dev)
+{
+ struct ipmmu_vmsa_archdata *archdata = dev->archdata.iommu;
+ struct ipmmu_vmsa_device *mmu = archdata->mmu;
+ struct ipmmu_vmsa_domain *domain = io_domain->priv;
+ unsigned long flags;
+ int ret = 0;
+
+ if (!mmu) {
+ dev_err(dev, "Cannot attach to IPMMU\n");
+ return -ENXIO;
+ }
+
+ spin_lock_irqsave(&domain->lock, flags);
+
+ if (!domain->mmu) {
+ /* The domain hasn't been used yet, initialize it. */
+ domain->mmu = mmu;
+ ret = ipmmu_domain_init_context(domain);
+ } else if (domain->mmu != mmu) {
+ /*
+ * Something is wrong, we can't attach two devices using
+ * different IOMMUs to the same domain.
+ */
+ dev_err(dev, "Can't attach IPMMU %s to domain on IPMMU %s\n",
+ dev_name(mmu->dev), dev_name(domain->mmu->dev));
+ ret = -EINVAL;
+ }
+
+ spin_unlock_irqrestore(&domain->lock, flags);
+
+ if (ret < 0)
+ return ret;
+
+ ipmmu_utlb_enable(domain, archdata->utlb);
+
+ return 0;
+}
+
+static void ipmmu_detach_device(struct iommu_domain *io_domain,
+ struct device *dev)
+{
+ struct ipmmu_vmsa_archdata *archdata = dev->archdata.iommu;
+ struct ipmmu_vmsa_domain *domain = io_domain->priv;
+
+ ipmmu_utlb_disable(domain, archdata->utlb);
+
+ /*
+ * TODO: Optimize by disabling the context when no device is attached.
+ */
+}
+
+static int ipmmu_map(struct iommu_domain *io_domain, unsigned long iova,
+ phys_addr_t paddr, size_t size, int prot)
+{
+ struct ipmmu_vmsa_domain *domain = io_domain->priv;
+
+ if (!domain)
+ return -ENODEV;
+
+ return ipmmu_create_mapping(domain, iova, paddr, size, prot);
+}
+
+static size_t ipmmu_unmap(struct iommu_domain *io_domain, unsigned long iova,
+ size_t size)
+{
+ struct ipmmu_vmsa_domain *domain = io_domain->priv;
+ int ret;
+
+ ret = ipmmu_clear_mapping(domain, iova, size);
+ return ret ? 0 : size;
+}
+
+static phys_addr_t ipmmu_iova_to_phys(struct iommu_domain *io_domain,
+ dma_addr_t iova)
+{
+ struct ipmmu_vmsa_domain *domain = io_domain->priv;
+ pgd_t pgd;
+ pud_t pud;
+ pmd_t pmd;
+ pte_t pte;
+
+ /* TODO: Is locking needed ? */
+
+ if (!domain->pgd)
+ return 0;
+
+ pgd = *(domain->pgd + pgd_index(iova));
+ if (pgd_none(pgd))
+ return 0;
+
+ pud = *pud_offset(&pgd, iova);
+ if (pud_none(pud))
+ return 0;
+
+ pmd = *pmd_offset(&pud, iova);
+ if (pmd_none(pmd))
+ return 0;
+
+ if (pmd_sect(pmd))
+ return __pfn_to_phys(pmd_pfn(pmd)) | (iova & ~PMD_MASK);
+
+ pte = *(pmd_page_vaddr(pmd) + pte_index(iova));
+ if (pte_none(pte))
+ return 0;
+
+ return __pfn_to_phys(pte_pfn(pte)) | (iova & ~PAGE_MASK);
+}
+
+static int ipmmu_find_utlb(struct ipmmu_vmsa_device *mmu, struct device *dev)
+{
+ const struct ipmmu_vmsa_master *master = mmu->pdata->masters;
+ const char *devname = dev_name(dev);
+ unsigned int i;
+
+ for (i = 0; i < mmu->pdata->num_masters; ++i, ++master) {
+ if (strcmp(master->name, devname) == 0)
+ return master->utlb;
+ }
+
+ return -1;
+}
+
+static int ipmmu_add_device(struct device *dev)
+{
+ struct ipmmu_vmsa_archdata *archdata;
+ struct ipmmu_vmsa_device *mmu;
+ struct iommu_group *group;
+ int utlb = -1;
+ int ret;
+
+ if (dev->archdata.iommu) {
+ dev_warn(dev, "IOMMU driver already assigned to device %s\n",
+ dev_name(dev));
+ return -EINVAL;
+ }
+
+ /* Find the master corresponding to the device. */
+ spin_lock(&ipmmu_devices_lock);
+
+ list_for_each_entry(mmu, &ipmmu_devices, list) {
+ utlb = ipmmu_find_utlb(mmu, dev);
+ if (utlb >= 0) {
+ /*
+ * TODO Take a reference to the MMU to protect
+ * against device removal.
+ */
+ break;
+ }
+ }
+
+ spin_unlock(&ipmmu_devices_lock);
+
+ if (utlb < 0)
+ return -ENODEV;
+
+ if (utlb >= mmu->num_utlbs)
+ return -EINVAL;
+
+ /* Create a device group and add the device to it. */
+ group = iommu_group_alloc();
+ if (IS_ERR(group)) {
+ dev_err(dev, "Failed to allocate IOMMU group\n");
+ return PTR_ERR(group);
+ }
+
+ ret = iommu_group_add_device(group, dev);
+ iommu_group_put(group);
+
+ if (ret < 0) {
+ dev_err(dev, "Failed to add device to IPMMU group\n");
+ return ret;
+ }
+
+ archdata = kzalloc(sizeof(*archdata), GFP_KERNEL);
+ if (!archdata) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ archdata->mmu = mmu;
+ archdata->utlb = utlb;
+ dev->archdata.iommu = archdata;
+
+ /*
+ * Create the ARM mapping, used by the ARM DMA mapping core to allocate
+ * VAs. This will allocate a corresponding IOMMU domain.
+ *
+ * TODO:
+ * - Create one mapping per context (TLB).
+ * - Make the mapping size configurable ? We currently use a 2GB mapping
+ * at a 1GB offset to ensure that NULL VAs will fault.
+ */
+ if (!mmu->mapping) {
+ struct dma_iommu_mapping *mapping;
+
+ mapping = arm_iommu_create_mapping(&platform_bus_type,
+ SZ_1G, SZ_2G);
+ if (IS_ERR(mapping)) {
+ dev_err(mmu->dev, "failed to create ARM IOMMU mapping\n");
+ return PTR_ERR(mapping);
+ }
+
+ mmu->mapping = mapping;
+ }
+
+ /* Attach the ARM VA mapping to the device. */
+ ret = arm_iommu_attach_device(dev, mmu->mapping);
+ if (ret < 0) {
+ dev_err(dev, "Failed to attach device to VA mapping\n");
+ goto error;
+ }
+
+ return 0;
+
+error:
+ kfree(dev->archdata.iommu);
+ dev->archdata.iommu = NULL;
+ iommu_group_remove_device(dev);
+ return ret;
+}
+
+static void ipmmu_remove_device(struct device *dev)
+{
+ arm_iommu_detach_device(dev);
+ iommu_group_remove_device(dev);
+ kfree(dev->archdata.iommu);
+ dev->archdata.iommu = NULL;
+}
+
+static struct iommu_ops ipmmu_ops = {
+ .domain_init = ipmmu_domain_init,
+ .domain_destroy = ipmmu_domain_destroy,
+ .attach_dev = ipmmu_attach_device,
+ .detach_dev = ipmmu_detach_device,
+ .map = ipmmu_map,
+ .unmap = ipmmu_unmap,
+ .iova_to_phys = ipmmu_iova_to_phys,
+ .add_device = ipmmu_add_device,
+ .remove_device = ipmmu_remove_device,
+ .pgsize_bitmap = SZ_2M | SZ_64K | SZ_4K,
+};
+
+/* -----------------------------------------------------------------------------
+ * Probe/remove and init
+ */
+
+static void ipmmu_device_reset(struct ipmmu_vmsa_device *mmu)
+{
+ unsigned int i;
+
+ /* Disable all contexts. */
+ for (i = 0; i < 4; ++i)
+ ipmmu_write(mmu, i * IM_CTX_SIZE + IMCTR, 0);
+}
+
+static int ipmmu_probe(struct platform_device *pdev)
+{
+ struct ipmmu_vmsa_device *mmu;
+ struct resource *res;
+ int irq;
+ int ret;
+
+ if (!pdev->dev.platform_data) {
+ dev_err(&pdev->dev, "missing platform data\n");
+ return -EINVAL;
+ }
+
+ mmu = devm_kzalloc(&pdev->dev, sizeof(*mmu), GFP_KERNEL);
+ if (!mmu) {
+ dev_err(&pdev->dev, "cannot allocate device data\n");
+ return -ENOMEM;
+ }
+
+ mmu->dev = &pdev->dev;
+ mmu->pdata = pdev->dev.platform_data;
+ mmu->num_utlbs = 32;
+
+ /* Map I/O memory and request IRQ. */
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ mmu->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(mmu->base))
+ return PTR_ERR(mmu->base);
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0) {
+ dev_err(&pdev->dev, "no IRQ found\n");
+ return irq;
+ }
+
+ ret = devm_request_irq(&pdev->dev, irq, ipmmu_irq, 0,
+ dev_name(&pdev->dev), mmu);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "failed to request IRQ %d\n", irq);
+ return irq;
+ }
+
+ ipmmu_device_reset(mmu);
+
+ /*
+ * We can't create the ARM mapping here as it requires the bus to have
+ * an IOMMU, which only happens when bus_set_iommu() is called in
+ * ipmmu_init() after the probe function returns.
+ */
+
+ spin_lock(&ipmmu_devices_lock);
+ list_add(&mmu->list, &ipmmu_devices);
+ spin_unlock(&ipmmu_devices_lock);
+
+ platform_set_drvdata(pdev, mmu);
+
+ return 0;
+}
+
+static int ipmmu_remove(struct platform_device *pdev)
+{
+ struct ipmmu_vmsa_device *mmu = platform_get_drvdata(pdev);
+
+ spin_lock(&ipmmu_devices_lock);
+ list_del(&mmu->list);
+ spin_unlock(&ipmmu_devices_lock);
+
+ arm_iommu_release_mapping(mmu->mapping);
+
+ ipmmu_device_reset(mmu);
+
+ return 0;
+}
+
+static struct platform_driver ipmmu_driver = {
+ .driver = {
+ .owner = THIS_MODULE,
+ .name = "ipmmu-vmsa",
+ },
+ .probe = ipmmu_probe,
+ .remove = ipmmu_remove,
+};
+
+static int __init ipmmu_init(void)
+{
+ int ret;
+
+ ret = platform_driver_register(&ipmmu_driver);
+ if (ret < 0)
+ return ret;
+
+ if (!iommu_present(&platform_bus_type))
+ bus_set_iommu(&platform_bus_type, &ipmmu_ops);
+
+ return 0;
+}
+
+static void __exit ipmmu_exit(void)
+{
+ return platform_driver_unregister(&ipmmu_driver);
+}
+
+subsys_initcall(ipmmu_init);
+module_exit(ipmmu_exit);
+
+MODULE_DESCRIPTION("IOMMU API for Renesas VMSA-compatible IPMMU");
+MODULE_AUTHOR("Laurent Pinchart <laurent.pinchart@ideasonboard.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iommu/msm_iommu_dev.c b/drivers/iommu/msm_iommu_dev.c
index 08ba4972..61def7cb 100644
--- a/drivers/iommu/msm_iommu_dev.c
+++ b/drivers/iommu/msm_iommu_dev.c
@@ -127,13 +127,12 @@
static int msm_iommu_probe(struct platform_device *pdev)
{
- struct resource *r, *r2;
+ struct resource *r;
struct clk *iommu_clk;
struct clk *iommu_pclk;
struct msm_iommu_drvdata *drvdata;
struct msm_iommu_dev *iommu_dev = pdev->dev.platform_data;
void __iomem *regs_base;
- resource_size_t len;
int ret, irq, par;
if (pdev->id == -1) {
@@ -178,35 +177,16 @@
iommu_clk = NULL;
r = platform_get_resource_byname(pdev, IORESOURCE_MEM, "physbase");
-
- if (!r) {
- ret = -ENODEV;
+ regs_base = devm_ioremap_resource(&pdev->dev, r);
+ if (IS_ERR(regs_base)) {
+ ret = PTR_ERR(regs_base);
goto fail_clk;
}
- len = resource_size(r);
-
- r2 = request_mem_region(r->start, len, r->name);
- if (!r2) {
- pr_err("Could not request memory region: start=%p, len=%d\n",
- (void *) r->start, len);
- ret = -EBUSY;
- goto fail_clk;
- }
-
- regs_base = ioremap(r2->start, len);
-
- if (!regs_base) {
- pr_err("Could not ioremap: start=%p, len=%d\n",
- (void *) r2->start, len);
- ret = -EBUSY;
- goto fail_mem;
- }
-
irq = platform_get_irq_byname(pdev, "secure_irq");
if (irq < 0) {
ret = -ENODEV;
- goto fail_io;
+ goto fail_clk;
}
msm_iommu_reset(regs_base, iommu_dev->ncb);
@@ -222,14 +202,14 @@
if (!par) {
pr_err("%s: Invalid PAR value detected\n", iommu_dev->name);
ret = -ENODEV;
- goto fail_io;
+ goto fail_clk;
}
ret = request_irq(irq, msm_iommu_fault_handler, 0,
"msm_iommu_secure_irpt_handler", drvdata);
if (ret) {
pr_err("Request IRQ %d failed with ret=%d\n", irq, ret);
- goto fail_io;
+ goto fail_clk;
}
@@ -250,10 +230,6 @@
clk_disable(iommu_pclk);
return 0;
-fail_io:
- iounmap(regs_base);
-fail_mem:
- release_mem_region(r->start, len);
fail_clk:
if (iommu_clk) {
clk_disable(iommu_clk);
diff --git a/drivers/iommu/omap-iommu.c b/drivers/iommu/omap-iommu.c
index 7fcbfc4..895af06 100644
--- a/drivers/iommu/omap-iommu.c
+++ b/drivers/iommu/omap-iommu.c
@@ -34,6 +34,9 @@
#include "omap-iopgtable.h"
#include "omap-iommu.h"
+#define to_iommu(dev) \
+ ((struct omap_iommu *)platform_get_drvdata(to_platform_device(dev)))
+
#define for_each_iotlb_cr(obj, n, __i, cr) \
for (__i = 0; \
(__i < (n)) && (cr = __iotlb_read_cr((obj), __i), true); \
@@ -391,6 +394,7 @@
__func__, start, da, bytes);
iotlb_load_cr(obj, &cr);
iommu_write_reg(obj, 1, MMU_FLUSH_ENTRY);
+ break;
}
}
pm_runtime_put_sync(obj->dev);
@@ -1037,19 +1041,18 @@
clean_dcache_area(iopte, IOPTE_TABLE_SIZE);
}
-static u32 iotlb_init_entry(struct iotlb_entry *e, u32 da, u32 pa,
- u32 flags)
+static u32 iotlb_init_entry(struct iotlb_entry *e, u32 da, u32 pa, int pgsz)
{
memset(e, 0, sizeof(*e));
e->da = da;
e->pa = pa;
- e->valid = 1;
+ e->valid = MMU_CAM_V;
/* FIXME: add OMAP1 support */
- e->pgsz = flags & MMU_CAM_PGSZ_MASK;
- e->endian = flags & MMU_RAM_ENDIAN_MASK;
- e->elsz = flags & MMU_RAM_ELSZ_MASK;
- e->mixed = flags & MMU_RAM_MIXED_MASK;
+ e->pgsz = pgsz;
+ e->endian = MMU_RAM_ENDIAN_LITTLE;
+ e->elsz = MMU_RAM_ELSZ_8;
+ e->mixed = 0;
return iopgsz_to_bytes(e->pgsz);
}
@@ -1062,9 +1065,8 @@
struct device *dev = oiommu->dev;
struct iotlb_entry e;
int omap_pgsz;
- u32 ret, flags;
+ u32 ret;
- /* we only support mapping a single iommu page for now */
omap_pgsz = bytes_to_iopgsz(bytes);
if (omap_pgsz < 0) {
dev_err(dev, "invalid size to map: %d\n", bytes);
@@ -1073,9 +1075,7 @@
dev_dbg(dev, "mapping da 0x%lx to pa 0x%x size 0x%x\n", da, pa, bytes);
- flags = omap_pgsz | prot;
-
- iotlb_init_entry(&e, da, pa, flags);
+ iotlb_init_entry(&e, da, pa, omap_pgsz);
ret = omap_iopgtable_store_entry(oiommu, &e);
if (ret)
@@ -1248,12 +1248,6 @@
return ret;
}
-static int omap_iommu_domain_has_cap(struct iommu_domain *domain,
- unsigned long cap)
-{
- return 0;
-}
-
static int omap_iommu_add_device(struct device *dev)
{
struct omap_iommu_arch_data *arch_data;
@@ -1305,7 +1299,6 @@
.map = omap_iommu_map,
.unmap = omap_iommu_unmap,
.iova_to_phys = omap_iommu_iova_to_phys,
- .domain_has_cap = omap_iommu_domain_has_cap,
.add_device = omap_iommu_add_device,
.remove_device = omap_iommu_remove_device,
.pgsize_bitmap = OMAP_IOMMU_PGSIZES,
diff --git a/drivers/iommu/omap-iopgtable.h b/drivers/iommu/omap-iopgtable.h
index b6f9a51..f891683 100644
--- a/drivers/iommu/omap-iopgtable.h
+++ b/drivers/iommu/omap-iopgtable.h
@@ -93,6 +93,3 @@
/* to find an entry in the second-level page table. */
#define iopte_index(da) (((da) >> IOPTE_SHIFT) & (PTRS_PER_IOPTE - 1))
#define iopte_offset(iopgd, da) (iopgd_page_vaddr(iopgd) + iopte_index(da))
-
-#define to_iommu(dev) \
- (platform_get_drvdata(to_platform_device(dev)))
diff --git a/drivers/iommu/shmobile-ipmmu.c b/drivers/iommu/shmobile-ipmmu.c
index e3bc2e1..bd97ade 100644
--- a/drivers/iommu/shmobile-ipmmu.c
+++ b/drivers/iommu/shmobile-ipmmu.c
@@ -94,11 +94,6 @@
struct resource *res;
struct shmobile_ipmmu_platform_data *pdata = pdev->dev.platform_data;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (!res) {
- dev_err(&pdev->dev, "cannot get platform resources\n");
- return -ENOENT;
- }
ipmmu = devm_kzalloc(&pdev->dev, sizeof(*ipmmu), GFP_KERNEL);
if (!ipmmu) {
dev_err(&pdev->dev, "cannot allocate device data\n");
@@ -106,19 +101,18 @@
}
spin_lock_init(&ipmmu->flush_lock);
ipmmu->dev = &pdev->dev;
- ipmmu->ipmmu_base = devm_ioremap_nocache(&pdev->dev, res->start,
- resource_size(res));
- if (!ipmmu->ipmmu_base) {
- dev_err(&pdev->dev, "ioremap_nocache failed\n");
- return -ENOMEM;
- }
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ ipmmu->ipmmu_base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(ipmmu->ipmmu_base))
+ return PTR_ERR(ipmmu->ipmmu_base);
+
ipmmu->dev_names = pdata->dev_names;
ipmmu->num_dev_names = pdata->num_dev_names;
platform_set_drvdata(pdev, ipmmu);
ipmmu_reg_write(ipmmu, IMCTR1, 0x0); /* disable TLB */
ipmmu_reg_write(ipmmu, IMCTR2, 0x0); /* disable PMB */
- ipmmu_iommu_init(ipmmu);
- return 0;
+ return ipmmu_iommu_init(ipmmu);
}
static struct platform_driver ipmmu_driver = {
diff --git a/drivers/macintosh/windfarm_pm121.c b/drivers/macintosh/windfarm_pm121.c
index 7fe58b0..b350fb8 100644
--- a/drivers/macintosh/windfarm_pm121.c
+++ b/drivers/macintosh/windfarm_pm121.c
@@ -555,8 +555,18 @@
pid_param.interval = PM121_SYS_INTERVAL;
pid_param.history_len = PM121_SYS_HISTORY_SIZE;
pid_param.itarget = param->itarget;
- pid_param.min = control->ops->get_min(control);
- pid_param.max = control->ops->get_max(control);
+ if(control)
+ {
+ pid_param.min = control->ops->get_min(control);
+ pid_param.max = control->ops->get_max(control);
+ } else {
+ /*
+ * This is probably not the right!?
+ * Perhaps goto fail if control == NULL above?
+ */
+ pid_param.min = 0;
+ pid_param.max = 0;
+ }
wf_pid_init(&pm121_sys_state[loop_id]->pid, &pid_param);
@@ -571,7 +581,7 @@
control the same control */
printk(KERN_WARNING "pm121: failed to set up %s loop "
"setting \"%s\" to max speed.\n",
- loop_names[loop_id], control->name);
+ loop_names[loop_id], control ? control->name : "uninitialized value");
if (control)
wf_control_set_max(control);
diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
index 9a8e66a..67f8b31 100644
--- a/drivers/md/bitmap.c
+++ b/drivers/md/bitmap.c
@@ -669,17 +669,13 @@
/*
* return a pointer to the page in the filemap that contains the given bit
*
- * this lookup is complicated by the fact that the bitmap sb might be exactly
- * 1 page (e.g., x86) or less than 1 page -- so the bitmap might start on page
- * 0 or page 1
*/
static inline struct page *filemap_get_page(struct bitmap_storage *store,
unsigned long chunk)
{
if (file_page_index(store, chunk) >= store->file_pages)
return NULL;
- return store->filemap[file_page_index(store, chunk)
- - file_page_index(store, 0)];
+ return store->filemap[file_page_index(store, chunk)];
}
static int bitmap_storage_alloc(struct bitmap_storage *store,
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 237b7e0..3484685 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -3448,6 +3448,8 @@
mddev->level = LEVEL_NONE;
return rv;
}
+ if (mddev->ro)
+ return -EROFS;
/* request to change the personality. Need to ensure:
* - array is not engaged in resync/recovery/reshape
@@ -3634,6 +3636,8 @@
int err;
if (mddev->pers->check_reshape == NULL)
return -EBUSY;
+ if (mddev->ro)
+ return -EROFS;
mddev->new_layout = n;
err = mddev->pers->check_reshape(mddev);
if (err) {
@@ -3723,6 +3727,8 @@
int err;
if (mddev->pers->check_reshape == NULL)
return -EBUSY;
+ if (mddev->ro)
+ return -EROFS;
mddev->new_chunk_sectors = n >> 9;
err = mddev->pers->check_reshape(mddev);
if (err) {
@@ -6135,6 +6141,8 @@
*/
if (mddev->sync_thread)
return -EBUSY;
+ if (mddev->ro)
+ return -EROFS;
rdev_for_each(rdev, mddev) {
sector_t avail = rdev->sectors;
@@ -6157,6 +6165,8 @@
/* change the number of raid disks */
if (mddev->pers->check_reshape == NULL)
return -EINVAL;
+ if (mddev->ro)
+ return -EROFS;
if (raid_disks <= 0 ||
(mddev->max_disks && raid_disks >= mddev->max_disks))
return -EINVAL;
@@ -7381,8 +7391,10 @@
/* just incase thread restarts... */
if (test_bit(MD_RECOVERY_DONE, &mddev->recovery))
return;
- if (mddev->ro) /* never try to sync a read-only array */
+ if (mddev->ro) {/* never try to sync a read-only array */
+ set_bit(MD_RECOVERY_INTR, &mddev->recovery);
return;
+ }
if (test_bit(MD_RECOVERY_SYNC, &mddev->recovery)) {
if (test_bit(MD_RECOVERY_CHECK, &mddev->recovery)) {
@@ -7824,6 +7836,7 @@
/* There is no thread, but we need to call
* ->spare_active and clear saved_raid_disk
*/
+ set_bit(MD_RECOVERY_INTR, &mddev->recovery);
md_reap_sync_thread(mddev);
clear_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
goto unlock;
@@ -8330,7 +8343,7 @@
if (a < s) {
/* we need to split this range */
if (bb->count >= MD_MAX_BADBLOCKS) {
- rv = 0;
+ rv = -ENOSPC;
goto out;
}
memmove(p+lo+1, p+lo, (bb->count - lo) * 8);
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 2afef4e..6234b2e 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -292,9 +292,12 @@
BUG_ON(atomic_read(&conf->active_stripes)==0);
if (test_bit(STRIPE_HANDLE, &sh->state)) {
if (test_bit(STRIPE_DELAYED, &sh->state) &&
- !test_bit(STRIPE_PREREAD_ACTIVE, &sh->state))
+ !test_bit(STRIPE_PREREAD_ACTIVE, &sh->state)) {
list_add_tail(&sh->lru, &conf->delayed_list);
- else if (test_bit(STRIPE_BIT_DELAY, &sh->state) &&
+ if (atomic_read(&conf->preread_active_stripes)
+ < IO_THRESHOLD)
+ md_wakeup_thread(conf->mddev->thread);
+ } else if (test_bit(STRIPE_BIT_DELAY, &sh->state) &&
sh->bm_seq - conf->seq_write > 0)
list_add_tail(&sh->lru, &conf->bitmap_list);
else {
@@ -413,6 +416,11 @@
int hash;
bool wakeup;
+ /* Avoid release_list until the last reference.
+ */
+ if (atomic_add_unless(&sh->count, -1, 1))
+ return;
+
if (unlikely(!conf->mddev->thread) ||
test_and_set_bit(STRIPE_ON_RELEASE_LIST, &sh->state))
goto slow_path;
@@ -479,6 +487,7 @@
int num = sh->raid_conf->pool_size;
for (i = 0; i < num ; i++) {
+ WARN_ON(sh->dev[i].page != sh->dev[i].orig_page);
p = sh->dev[i].page;
if (!p)
continue;
@@ -499,6 +508,7 @@
return 1;
}
sh->dev[i].page = page;
+ sh->dev[i].orig_page = page;
}
return 0;
}
@@ -855,6 +865,9 @@
if (test_bit(R5_ReadNoMerge, &sh->dev[i].flags))
bi->bi_rw |= REQ_NOMERGE;
+ if (test_bit(R5_SkipCopy, &sh->dev[i].flags))
+ WARN_ON(test_bit(R5_UPTODATE, &sh->dev[i].flags));
+ sh->dev[i].vec.bv_page = sh->dev[i].page;
bi->bi_vcnt = 1;
bi->bi_io_vec[0].bv_len = STRIPE_SIZE;
bi->bi_io_vec[0].bv_offset = 0;
@@ -899,6 +912,9 @@
else
rbi->bi_iter.bi_sector = (sh->sector
+ rrdev->data_offset);
+ if (test_bit(R5_SkipCopy, &sh->dev[i].flags))
+ WARN_ON(test_bit(R5_UPTODATE, &sh->dev[i].flags));
+ sh->dev[i].rvec.bv_page = sh->dev[i].page;
rbi->bi_vcnt = 1;
rbi->bi_io_vec[0].bv_len = STRIPE_SIZE;
rbi->bi_io_vec[0].bv_offset = 0;
@@ -927,8 +943,9 @@
}
static struct dma_async_tx_descriptor *
-async_copy_data(int frombio, struct bio *bio, struct page *page,
- sector_t sector, struct dma_async_tx_descriptor *tx)
+async_copy_data(int frombio, struct bio *bio, struct page **page,
+ sector_t sector, struct dma_async_tx_descriptor *tx,
+ struct stripe_head *sh)
{
struct bio_vec bvl;
struct bvec_iter iter;
@@ -965,11 +982,16 @@
if (clen > 0) {
b_offset += bvl.bv_offset;
bio_page = bvl.bv_page;
- if (frombio)
- tx = async_memcpy(page, bio_page, page_offset,
+ if (frombio) {
+ if (sh->raid_conf->skip_copy &&
+ b_offset == 0 && page_offset == 0 &&
+ clen == STRIPE_SIZE)
+ *page = bio_page;
+ else
+ tx = async_memcpy(*page, bio_page, page_offset,
b_offset, clen, &submit);
- else
- tx = async_memcpy(bio_page, page, b_offset,
+ } else
+ tx = async_memcpy(bio_page, *page, b_offset,
page_offset, clen, &submit);
}
/* chain the operations */
@@ -1045,8 +1067,8 @@
spin_unlock_irq(&sh->stripe_lock);
while (rbi && rbi->bi_iter.bi_sector <
dev->sector + STRIPE_SECTORS) {
- tx = async_copy_data(0, rbi, dev->page,
- dev->sector, tx);
+ tx = async_copy_data(0, rbi, &dev->page,
+ dev->sector, tx, sh);
rbi = r5_next_bio(rbi, dev->sector);
}
}
@@ -1384,6 +1406,7 @@
BUG_ON(dev->written);
wbi = dev->written = chosen;
spin_unlock_irq(&sh->stripe_lock);
+ WARN_ON(dev->page != dev->orig_page);
while (wbi && wbi->bi_iter.bi_sector <
dev->sector + STRIPE_SECTORS) {
@@ -1393,9 +1416,15 @@
set_bit(R5_SyncIO, &dev->flags);
if (wbi->bi_rw & REQ_DISCARD)
set_bit(R5_Discard, &dev->flags);
- else
- tx = async_copy_data(1, wbi, dev->page,
- dev->sector, tx);
+ else {
+ tx = async_copy_data(1, wbi, &dev->page,
+ dev->sector, tx, sh);
+ if (dev->page != dev->orig_page) {
+ set_bit(R5_SkipCopy, &dev->flags);
+ clear_bit(R5_UPTODATE, &dev->flags);
+ clear_bit(R5_OVERWRITE, &dev->flags);
+ }
+ }
wbi = r5_next_bio(wbi, dev->sector);
}
}
@@ -1426,7 +1455,7 @@
struct r5dev *dev = &sh->dev[i];
if (dev->written || i == pd_idx || i == qd_idx) {
- if (!discard)
+ if (!discard && !test_bit(R5_SkipCopy, &dev->flags))
set_bit(R5_UPTODATE, &dev->flags);
if (fua)
set_bit(R5_WantFUA, &dev->flags);
@@ -1839,8 +1868,10 @@
osh = get_free_stripe(conf, hash);
unlock_device_hash_lock(conf, hash);
atomic_set(&nsh->count, 1);
- for(i=0; i<conf->pool_size; i++)
+ for(i=0; i<conf->pool_size; i++) {
nsh->dev[i].page = osh->dev[i].page;
+ nsh->dev[i].orig_page = osh->dev[i].page;
+ }
for( ; i<newsize; i++)
nsh->dev[i].page = NULL;
nsh->hash_lock_index = hash;
@@ -1896,6 +1927,7 @@
if (nsh->dev[i].page == NULL) {
struct page *p = alloc_page(GFP_NOIO);
nsh->dev[i].page = p;
+ nsh->dev[i].orig_page = p;
if (!p)
err = -ENOMEM;
}
@@ -2133,24 +2165,20 @@
}
static sector_t compute_blocknr(struct stripe_head *sh, int i, int previous);
-
+
static void raid5_build_block(struct stripe_head *sh, int i, int previous)
{
struct r5dev *dev = &sh->dev[i];
bio_init(&dev->req);
dev->req.bi_io_vec = &dev->vec;
- dev->req.bi_vcnt++;
- dev->req.bi_max_vecs++;
+ dev->req.bi_max_vecs = 1;
dev->req.bi_private = sh;
- dev->vec.bv_page = dev->page;
bio_init(&dev->rreq);
dev->rreq.bi_io_vec = &dev->rvec;
- dev->rreq.bi_vcnt++;
- dev->rreq.bi_max_vecs++;
+ dev->rreq.bi_max_vecs = 1;
dev->rreq.bi_private = sh;
- dev->rvec.bv_page = dev->page;
dev->flags = 0;
dev->sector = compute_blocknr(sh, i, previous);
@@ -2750,6 +2778,11 @@
/* and fail all 'written' */
bi = sh->dev[i].written;
sh->dev[i].written = NULL;
+ if (test_and_clear_bit(R5_SkipCopy, &sh->dev[i].flags)) {
+ WARN_ON(test_bit(R5_UPTODATE, &sh->dev[i].flags));
+ sh->dev[i].page = sh->dev[i].orig_page;
+ }
+
if (bi) bitmap_end = 1;
while (bi && bi->bi_iter.bi_sector <
sh->dev[i].sector + STRIPE_SECTORS) {
@@ -2886,8 +2919,11 @@
(s->failed >= 1 && fdev[0]->toread) ||
(s->failed >= 2 && fdev[1]->toread) ||
(sh->raid_conf->level <= 5 && s->failed && fdev[0]->towrite &&
+ (!test_bit(R5_Insync, &dev->flags) || test_bit(STRIPE_PREREAD_ACTIVE, &sh->state)) &&
!test_bit(R5_OVERWRITE, &fdev[0]->flags)) ||
- (sh->raid_conf->level == 6 && s->failed && s->to_write))) {
+ (sh->raid_conf->level == 6 && s->failed && s->to_write &&
+ s->to_write < sh->raid_conf->raid_disks - 2 &&
+ (!test_bit(R5_Insync, &dev->flags) || test_bit(STRIPE_PREREAD_ACTIVE, &sh->state))))) {
/* we would like to get this block, possibly by computing it,
* otherwise read it if the backing disk is insync
*/
@@ -2991,12 +3027,17 @@
dev = &sh->dev[i];
if (!test_bit(R5_LOCKED, &dev->flags) &&
(test_bit(R5_UPTODATE, &dev->flags) ||
- test_bit(R5_Discard, &dev->flags))) {
+ test_bit(R5_Discard, &dev->flags) ||
+ test_bit(R5_SkipCopy, &dev->flags))) {
/* We can return any write requests */
struct bio *wbi, *wbi2;
pr_debug("Return write for disc %d\n", i);
if (test_and_clear_bit(R5_Discard, &dev->flags))
clear_bit(R5_UPTODATE, &dev->flags);
+ if (test_and_clear_bit(R5_SkipCopy, &dev->flags)) {
+ WARN_ON(test_bit(R5_UPTODATE, &dev->flags));
+ dev->page = dev->orig_page;
+ }
wbi = dev->written;
dev->written = NULL;
while (wbi && wbi->bi_iter.bi_sector <
@@ -3015,6 +3056,8 @@
0);
} else if (test_bit(R5_Discard, &dev->flags))
discard_pending = 1;
+ WARN_ON(test_bit(R5_SkipCopy, &dev->flags));
+ WARN_ON(dev->page != dev->orig_page);
}
if (!discard_pending &&
test_bit(R5_Discard, &sh->dev[sh->pd_idx].flags)) {
@@ -3086,7 +3129,8 @@
!test_bit(R5_LOCKED, &dev->flags) &&
!(test_bit(R5_UPTODATE, &dev->flags) ||
test_bit(R5_Wantcompute, &dev->flags))) {
- if (test_bit(R5_Insync, &dev->flags)) rcw++;
+ if (test_bit(R5_Insync, &dev->flags))
+ rcw++;
else
rcw += 2*disks;
}
@@ -3107,10 +3151,10 @@
!(test_bit(R5_UPTODATE, &dev->flags) ||
test_bit(R5_Wantcompute, &dev->flags)) &&
test_bit(R5_Insync, &dev->flags)) {
- if (
- test_bit(STRIPE_PREREAD_ACTIVE, &sh->state)) {
- pr_debug("Read_old block "
- "%d for r-m-w\n", i);
+ if (test_bit(STRIPE_PREREAD_ACTIVE,
+ &sh->state)) {
+ pr_debug("Read_old block %d for r-m-w\n",
+ i);
set_bit(R5_LOCKED, &dev->flags);
set_bit(R5_Wantread, &dev->flags);
s->locked++;
@@ -3133,10 +3177,9 @@
!(test_bit(R5_UPTODATE, &dev->flags) ||
test_bit(R5_Wantcompute, &dev->flags))) {
rcw++;
- if (!test_bit(R5_Insync, &dev->flags))
- continue; /* it's a failed drive */
- if (
- test_bit(STRIPE_PREREAD_ACTIVE, &sh->state)) {
+ if (test_bit(R5_Insync, &dev->flags) &&
+ test_bit(STRIPE_PREREAD_ACTIVE,
+ &sh->state)) {
pr_debug("Read_old block "
"%d for Reconstruct\n", i);
set_bit(R5_LOCKED, &dev->flags);
@@ -5031,8 +5074,8 @@
bitmap_start_sync(mddev->bitmap, sector_nr, &sync_blocks, still_degraded);
set_bit(STRIPE_SYNC_REQUESTED, &sh->state);
+ set_bit(STRIPE_HANDLE, &sh->state);
- handle_stripe(sh);
release_stripe(sh);
return STRIPE_SECTORS;
@@ -5072,7 +5115,7 @@
/* already done this stripe */
continue;
- sh = get_active_stripe(conf, sector, 0, 1, 0);
+ sh = get_active_stripe(conf, sector, 0, 1, 1);
if (!sh) {
/* failed to get a stripe - must wait */
@@ -5355,6 +5398,50 @@
raid5_store_preread_threshold);
static ssize_t
+raid5_show_skip_copy(struct mddev *mddev, char *page)
+{
+ struct r5conf *conf = mddev->private;
+ if (conf)
+ return sprintf(page, "%d\n", conf->skip_copy);
+ else
+ return 0;
+}
+
+static ssize_t
+raid5_store_skip_copy(struct mddev *mddev, const char *page, size_t len)
+{
+ struct r5conf *conf = mddev->private;
+ unsigned long new;
+ if (len >= PAGE_SIZE)
+ return -EINVAL;
+ if (!conf)
+ return -ENODEV;
+
+ if (kstrtoul(page, 10, &new))
+ return -EINVAL;
+ new = !!new;
+ if (new == conf->skip_copy)
+ return len;
+
+ mddev_suspend(mddev);
+ conf->skip_copy = new;
+ if (new)
+ mddev->queue->backing_dev_info.capabilities |=
+ BDI_CAP_STABLE_WRITES;
+ else
+ mddev->queue->backing_dev_info.capabilities &=
+ ~BDI_CAP_STABLE_WRITES;
+ mddev_resume(mddev);
+ return len;
+}
+
+static struct md_sysfs_entry
+raid5_skip_copy = __ATTR(skip_copy, S_IRUGO | S_IWUSR,
+ raid5_show_skip_copy,
+ raid5_store_skip_copy);
+
+
+static ssize_t
stripe_cache_active_show(struct mddev *mddev, char *page)
{
struct r5conf *conf = mddev->private;
@@ -5439,6 +5526,7 @@
&raid5_stripecache_active.attr,
&raid5_preread_bypass_threshold.attr,
&raid5_group_thread_cnt.attr,
+ &raid5_skip_copy.attr,
NULL,
};
static struct attribute_group raid5_attrs_group = {
diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h
index 01ad8ae..bc72cd4 100644
--- a/drivers/md/raid5.h
+++ b/drivers/md/raid5.h
@@ -232,7 +232,7 @@
*/
struct bio req, rreq;
struct bio_vec vec, rvec;
- struct page *page;
+ struct page *page, *orig_page;
struct bio *toread, *read, *towrite, *written;
sector_t sector; /* sector of this page */
unsigned long flags;
@@ -299,6 +299,7 @@
* data in, and now is a good time to write it out.
*/
R5_Discard, /* Discard the stripe */
+ R5_SkipCopy, /* Don't copy data from bio to stripe cache */
};
/*
@@ -436,6 +437,7 @@
atomic_t pending_full_writes; /* full write backlog */
int bypass_count; /* bypassed prereads */
int bypass_threshold; /* preread nice */
+ int skip_copy; /* Don't copy data from bio to stripe cache */
struct list_head *last_hold; /* detect hold_list promotions */
atomic_t reshape_stripes; /* stripes with pending writes for reshape */
diff --git a/drivers/media/i2c/adv7604.c b/drivers/media/i2c/adv7604.c
index 338baa4..1778d32 100644
--- a/drivers/media/i2c/adv7604.c
+++ b/drivers/media/i2c/adv7604.c
@@ -27,19 +27,21 @@
* REF_03 - Analog devices, ADV7604, Hardware Manual, Rev. F, August 2010
*/
-
+#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
+#include <linux/i2c.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/slab.h>
-#include <linux/i2c.h>
-#include <linux/delay.h>
+#include <linux/v4l2-dv-timings.h>
#include <linux/videodev2.h>
#include <linux/workqueue.h>
-#include <linux/v4l2-dv-timings.h>
-#include <media/v4l2-device.h>
-#include <media/v4l2-ctrls.h>
-#include <media/v4l2-dv-timings.h>
+
#include <media/adv7604.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-dv-timings.h>
+#include <media/v4l2-of.h>
static int debug;
module_param(debug, int, 0644);
@@ -53,6 +55,76 @@
/* ADV7604 system clock frequency */
#define ADV7604_fsc (28636360)
+#define ADV7604_RGB_OUT (1 << 1)
+
+#define ADV7604_OP_FORMAT_SEL_8BIT (0 << 0)
+#define ADV7604_OP_FORMAT_SEL_10BIT (1 << 0)
+#define ADV7604_OP_FORMAT_SEL_12BIT (2 << 0)
+
+#define ADV7604_OP_MODE_SEL_SDR_422 (0 << 5)
+#define ADV7604_OP_MODE_SEL_DDR_422 (1 << 5)
+#define ADV7604_OP_MODE_SEL_SDR_444 (2 << 5)
+#define ADV7604_OP_MODE_SEL_DDR_444 (3 << 5)
+#define ADV7604_OP_MODE_SEL_SDR_422_2X (4 << 5)
+#define ADV7604_OP_MODE_SEL_ADI_CM (5 << 5)
+
+#define ADV7604_OP_CH_SEL_GBR (0 << 5)
+#define ADV7604_OP_CH_SEL_GRB (1 << 5)
+#define ADV7604_OP_CH_SEL_BGR (2 << 5)
+#define ADV7604_OP_CH_SEL_RGB (3 << 5)
+#define ADV7604_OP_CH_SEL_BRG (4 << 5)
+#define ADV7604_OP_CH_SEL_RBG (5 << 5)
+
+#define ADV7604_OP_SWAP_CB_CR (1 << 0)
+
+enum adv7604_type {
+ ADV7604,
+ ADV7611,
+};
+
+struct adv7604_reg_seq {
+ unsigned int reg;
+ u8 val;
+};
+
+struct adv7604_format_info {
+ enum v4l2_mbus_pixelcode code;
+ u8 op_ch_sel;
+ bool rgb_out;
+ bool swap_cb_cr;
+ u8 op_format_sel;
+};
+
+struct adv7604_chip_info {
+ enum adv7604_type type;
+
+ bool has_afe;
+ unsigned int max_port;
+ unsigned int num_dv_ports;
+
+ unsigned int edid_enable_reg;
+ unsigned int edid_status_reg;
+ unsigned int lcf_reg;
+
+ unsigned int cable_det_mask;
+ unsigned int tdms_lock_mask;
+ unsigned int fmt_change_digital_mask;
+
+ const struct adv7604_format_info *formats;
+ unsigned int nformats;
+
+ void (*set_termination)(struct v4l2_subdev *sd, bool enable);
+ void (*setup_irqs)(struct v4l2_subdev *sd);
+ unsigned int (*read_hdmi_pixelclock)(struct v4l2_subdev *sd);
+ unsigned int (*read_cable_det)(struct v4l2_subdev *sd);
+
+ /* 0 = AFE, 1 = HDMI */
+ const struct adv7604_reg_seq *recommended_settings[2];
+ unsigned int num_recommended_settings[2];
+
+ unsigned long page_mask;
+};
+
/*
**********************************************************************
*
@@ -60,13 +132,24 @@
*
**********************************************************************
*/
+
struct adv7604_state {
+ const struct adv7604_chip_info *info;
struct adv7604_platform_data pdata;
+
+ struct gpio_desc *hpd_gpio[4];
+
struct v4l2_subdev sd;
- struct media_pad pad;
+ struct media_pad pads[ADV7604_PAD_MAX];
+ unsigned int source_pad;
+
struct v4l2_ctrl_handler hdl;
- enum adv7604_input_port selected_input;
+
+ enum adv7604_pad selected_input;
+
struct v4l2_dv_timings timings;
+ const struct adv7604_format_info *format;
+
struct {
u8 edid[256];
u32 present;
@@ -80,18 +163,7 @@
bool restart_stdi_once;
/* i2c clients */
- struct i2c_client *i2c_avlink;
- struct i2c_client *i2c_cec;
- struct i2c_client *i2c_infoframe;
- struct i2c_client *i2c_esdp;
- struct i2c_client *i2c_dpp;
- struct i2c_client *i2c_afe;
- struct i2c_client *i2c_repeater;
- struct i2c_client *i2c_edid;
- struct i2c_client *i2c_hdmi;
- struct i2c_client *i2c_test;
- struct i2c_client *i2c_cp;
- struct i2c_client *i2c_vdp;
+ struct i2c_client *i2c_clients[ADV7604_PAGE_MAX];
/* controls */
struct v4l2_ctrl *detect_tx_5v_ctrl;
@@ -101,6 +173,11 @@
struct v4l2_ctrl *rgb_quantization_range_ctrl;
};
+static bool adv7604_has_afe(struct adv7604_state *state)
+{
+ return state->info->has_afe;
+}
+
/* Supported CEA and DMT timings */
static const struct v4l2_dv_timings adv7604_timings[] = {
V4L2_DV_BT_CEA_720X480P59_94,
@@ -256,11 +333,6 @@
return container_of(sd, struct adv7604_state, sd);
}
-static inline struct v4l2_subdev *to_sd(struct v4l2_ctrl *ctrl)
-{
- return &container_of(ctrl->handler, struct adv7604_state, hdl)->sd;
-}
-
static inline unsigned hblanking(const struct v4l2_bt_timings *t)
{
return V4L2_DV_BT_BLANKING_WIDTH(t);
@@ -298,14 +370,18 @@
return -EIO;
}
-static s32 adv_smbus_read_byte_data(struct i2c_client *client, u8 command)
+static s32 adv_smbus_read_byte_data(struct adv7604_state *state,
+ enum adv7604_page page, u8 command)
{
- return adv_smbus_read_byte_data_check(client, command, true);
+ return adv_smbus_read_byte_data_check(state->i2c_clients[page],
+ command, true);
}
-static s32 adv_smbus_write_byte_data(struct i2c_client *client,
- u8 command, u8 value)
+static s32 adv_smbus_write_byte_data(struct adv7604_state *state,
+ enum adv7604_page page, u8 command,
+ u8 value)
{
+ struct i2c_client *client = state->i2c_clients[page];
union i2c_smbus_data data;
int err;
int i;
@@ -325,9 +401,11 @@
return err;
}
-static s32 adv_smbus_write_i2c_block_data(struct i2c_client *client,
- u8 command, unsigned length, const u8 *values)
+static s32 adv_smbus_write_i2c_block_data(struct adv7604_state *state,
+ enum adv7604_page page, u8 command,
+ unsigned length, const u8 *values)
{
+ struct i2c_client *client = state->i2c_clients[page];
union i2c_smbus_data data;
if (length > I2C_SMBUS_BLOCK_MAX)
@@ -343,149 +421,150 @@
static inline int io_read(struct v4l2_subdev *sd, u8 reg)
{
- struct i2c_client *client = v4l2_get_subdevdata(sd);
+ struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(client, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_IO, reg);
}
static inline int io_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
- struct i2c_client *client = v4l2_get_subdevdata(sd);
+ struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(client, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_IO, reg, val);
}
-static inline int io_write_and_or(struct v4l2_subdev *sd, u8 reg, u8 mask, u8 val)
+static inline int io_write_clr_set(struct v4l2_subdev *sd, u8 reg, u8 mask, u8 val)
{
- return io_write(sd, reg, (io_read(sd, reg) & mask) | val);
+ return io_write(sd, reg, (io_read(sd, reg) & ~mask) | val);
}
static inline int avlink_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_avlink, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_AVLINK, reg);
}
static inline int avlink_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_avlink, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_AVLINK, reg, val);
}
static inline int cec_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_cec, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_CEC, reg);
}
static inline int cec_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_cec, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_CEC, reg, val);
}
-static inline int cec_write_and_or(struct v4l2_subdev *sd, u8 reg, u8 mask, u8 val)
+static inline int cec_write_clr_set(struct v4l2_subdev *sd, u8 reg, u8 mask, u8 val)
{
- return cec_write(sd, reg, (cec_read(sd, reg) & mask) | val);
+ return cec_write(sd, reg, (cec_read(sd, reg) & ~mask) | val);
}
static inline int infoframe_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_infoframe, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_INFOFRAME, reg);
}
static inline int infoframe_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_infoframe, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_INFOFRAME,
+ reg, val);
}
static inline int esdp_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_esdp, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_ESDP, reg);
}
static inline int esdp_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_esdp, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_ESDP, reg, val);
}
static inline int dpp_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_dpp, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_DPP, reg);
}
static inline int dpp_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_dpp, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_DPP, reg, val);
}
static inline int afe_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_afe, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_AFE, reg);
}
static inline int afe_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_afe, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_AFE, reg, val);
}
static inline int rep_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_repeater, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_REP, reg);
}
static inline int rep_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_repeater, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_REP, reg, val);
}
-static inline int rep_write_and_or(struct v4l2_subdev *sd, u8 reg, u8 mask, u8 val)
+static inline int rep_write_clr_set(struct v4l2_subdev *sd, u8 reg, u8 mask, u8 val)
{
- return rep_write(sd, reg, (rep_read(sd, reg) & mask) | val);
+ return rep_write(sd, reg, (rep_read(sd, reg) & ~mask) | val);
}
static inline int edid_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_edid, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_EDID, reg);
}
static inline int edid_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_edid, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_EDID, reg, val);
}
static inline int edid_read_block(struct v4l2_subdev *sd, unsigned len, u8 *val)
{
struct adv7604_state *state = to_state(sd);
- struct i2c_client *client = state->i2c_edid;
+ struct i2c_client *client = state->i2c_clients[ADV7604_PAGE_EDID];
u8 msgbuf0[1] = { 0 };
u8 msgbuf1[256];
struct i2c_msg msg[2] = {
@@ -518,11 +597,25 @@
v4l2_dbg(2, debug, sd, "%s: write EDID block (%d byte)\n", __func__, len);
for (i = 0; !err && i < len; i += I2C_SMBUS_BLOCK_MAX)
- err = adv_smbus_write_i2c_block_data(state->i2c_edid, i,
- I2C_SMBUS_BLOCK_MAX, val + i);
+ err = adv_smbus_write_i2c_block_data(state, ADV7604_PAGE_EDID,
+ i, I2C_SMBUS_BLOCK_MAX, val + i);
return err;
}
+static void adv7604_set_hpd(struct adv7604_state *state, unsigned int hpd)
+{
+ unsigned int i;
+
+ for (i = 0; i < state->info->num_dv_ports; ++i) {
+ if (IS_ERR(state->hpd_gpio[i]))
+ continue;
+
+ gpiod_set_value_cansleep(state->hpd_gpio[i], hpd & BIT(i));
+ }
+
+ v4l2_subdev_notify(&state->sd, ADV7604_HOTPLUG, &hpd);
+}
+
static void adv7604_delayed_work_enable_hotplug(struct work_struct *work)
{
struct delayed_work *dwork = to_delayed_work(work);
@@ -532,73 +625,210 @@
v4l2_dbg(2, debug, sd, "%s: enable hotplug\n", __func__);
- v4l2_subdev_notify(sd, ADV7604_HOTPLUG, (void *)&state->edid.present);
+ adv7604_set_hpd(state, state->edid.present);
}
static inline int hdmi_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_hdmi, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_HDMI, reg);
+}
+
+static u16 hdmi_read16(struct v4l2_subdev *sd, u8 reg, u16 mask)
+{
+ return ((hdmi_read(sd, reg) << 8) | hdmi_read(sd, reg + 1)) & mask;
}
static inline int hdmi_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_hdmi, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_HDMI, reg, val);
}
-static inline int hdmi_write_and_or(struct v4l2_subdev *sd, u8 reg, u8 mask, u8 val)
+static inline int hdmi_write_clr_set(struct v4l2_subdev *sd, u8 reg, u8 mask, u8 val)
{
- return hdmi_write(sd, reg, (hdmi_read(sd, reg) & mask) | val);
+ return hdmi_write(sd, reg, (hdmi_read(sd, reg) & ~mask) | val);
}
static inline int test_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_test, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_TEST, reg);
}
static inline int test_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_test, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_TEST, reg, val);
}
static inline int cp_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_cp, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_CP, reg);
+}
+
+static u16 cp_read16(struct v4l2_subdev *sd, u8 reg, u16 mask)
+{
+ return ((cp_read(sd, reg) << 8) | cp_read(sd, reg + 1)) & mask;
}
static inline int cp_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_cp, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_CP, reg, val);
}
-static inline int cp_write_and_or(struct v4l2_subdev *sd, u8 reg, u8 mask, u8 val)
+static inline int cp_write_clr_set(struct v4l2_subdev *sd, u8 reg, u8 mask, u8 val)
{
- return cp_write(sd, reg, (cp_read(sd, reg) & mask) | val);
+ return cp_write(sd, reg, (cp_read(sd, reg) & ~mask) | val);
}
static inline int vdp_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_read_byte_data(state->i2c_vdp, reg);
+ return adv_smbus_read_byte_data(state, ADV7604_PAGE_VDP, reg);
}
static inline int vdp_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv7604_state *state = to_state(sd);
- return adv_smbus_write_byte_data(state->i2c_vdp, reg, val);
+ return adv_smbus_write_byte_data(state, ADV7604_PAGE_VDP, reg, val);
+}
+
+#define ADV7604_REG(page, offset) (((page) << 8) | (offset))
+#define ADV7604_REG_SEQ_TERM 0xffff
+
+#ifdef CONFIG_VIDEO_ADV_DEBUG
+static int adv7604_read_reg(struct v4l2_subdev *sd, unsigned int reg)
+{
+ struct adv7604_state *state = to_state(sd);
+ unsigned int page = reg >> 8;
+
+ if (!(BIT(page) & state->info->page_mask))
+ return -EINVAL;
+
+ reg &= 0xff;
+
+ return adv_smbus_read_byte_data(state, page, reg);
+}
+#endif
+
+static int adv7604_write_reg(struct v4l2_subdev *sd, unsigned int reg, u8 val)
+{
+ struct adv7604_state *state = to_state(sd);
+ unsigned int page = reg >> 8;
+
+ if (!(BIT(page) & state->info->page_mask))
+ return -EINVAL;
+
+ reg &= 0xff;
+
+ return adv_smbus_write_byte_data(state, page, reg, val);
+}
+
+static void adv7604_write_reg_seq(struct v4l2_subdev *sd,
+ const struct adv7604_reg_seq *reg_seq)
+{
+ unsigned int i;
+
+ for (i = 0; reg_seq[i].reg != ADV7604_REG_SEQ_TERM; i++)
+ adv7604_write_reg(sd, reg_seq[i].reg, reg_seq[i].val);
+}
+
+/* -----------------------------------------------------------------------------
+ * Format helpers
+ */
+
+static const struct adv7604_format_info adv7604_formats[] = {
+ { V4L2_MBUS_FMT_RGB888_1X24, ADV7604_OP_CH_SEL_RGB, true, false,
+ ADV7604_OP_MODE_SEL_SDR_444 | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_YUYV8_2X8, ADV7604_OP_CH_SEL_RGB, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422 | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_YVYU8_2X8, ADV7604_OP_CH_SEL_RGB, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422 | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_YUYV10_2X10, ADV7604_OP_CH_SEL_RGB, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422 | ADV7604_OP_FORMAT_SEL_10BIT },
+ { V4L2_MBUS_FMT_YVYU10_2X10, ADV7604_OP_CH_SEL_RGB, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422 | ADV7604_OP_FORMAT_SEL_10BIT },
+ { V4L2_MBUS_FMT_YUYV12_2X12, ADV7604_OP_CH_SEL_RGB, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422 | ADV7604_OP_FORMAT_SEL_12BIT },
+ { V4L2_MBUS_FMT_YVYU12_2X12, ADV7604_OP_CH_SEL_RGB, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422 | ADV7604_OP_FORMAT_SEL_12BIT },
+ { V4L2_MBUS_FMT_UYVY8_1X16, ADV7604_OP_CH_SEL_RBG, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_VYUY8_1X16, ADV7604_OP_CH_SEL_RBG, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_YUYV8_1X16, ADV7604_OP_CH_SEL_RGB, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_YVYU8_1X16, ADV7604_OP_CH_SEL_RGB, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_UYVY10_1X20, ADV7604_OP_CH_SEL_RBG, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_10BIT },
+ { V4L2_MBUS_FMT_VYUY10_1X20, ADV7604_OP_CH_SEL_RBG, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_10BIT },
+ { V4L2_MBUS_FMT_YUYV10_1X20, ADV7604_OP_CH_SEL_RGB, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_10BIT },
+ { V4L2_MBUS_FMT_YVYU10_1X20, ADV7604_OP_CH_SEL_RGB, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_10BIT },
+ { V4L2_MBUS_FMT_UYVY12_1X24, ADV7604_OP_CH_SEL_RBG, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_12BIT },
+ { V4L2_MBUS_FMT_VYUY12_1X24, ADV7604_OP_CH_SEL_RBG, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_12BIT },
+ { V4L2_MBUS_FMT_YUYV12_1X24, ADV7604_OP_CH_SEL_RGB, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_12BIT },
+ { V4L2_MBUS_FMT_YVYU12_1X24, ADV7604_OP_CH_SEL_RGB, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_12BIT },
+};
+
+static const struct adv7604_format_info adv7611_formats[] = {
+ { V4L2_MBUS_FMT_RGB888_1X24, ADV7604_OP_CH_SEL_RGB, true, false,
+ ADV7604_OP_MODE_SEL_SDR_444 | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_YUYV8_2X8, ADV7604_OP_CH_SEL_RGB, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422 | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_YVYU8_2X8, ADV7604_OP_CH_SEL_RGB, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422 | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_YUYV12_2X12, ADV7604_OP_CH_SEL_RGB, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422 | ADV7604_OP_FORMAT_SEL_12BIT },
+ { V4L2_MBUS_FMT_YVYU12_2X12, ADV7604_OP_CH_SEL_RGB, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422 | ADV7604_OP_FORMAT_SEL_12BIT },
+ { V4L2_MBUS_FMT_UYVY8_1X16, ADV7604_OP_CH_SEL_RBG, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_VYUY8_1X16, ADV7604_OP_CH_SEL_RBG, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_YUYV8_1X16, ADV7604_OP_CH_SEL_RGB, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_YVYU8_1X16, ADV7604_OP_CH_SEL_RGB, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_8BIT },
+ { V4L2_MBUS_FMT_UYVY12_1X24, ADV7604_OP_CH_SEL_RBG, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_12BIT },
+ { V4L2_MBUS_FMT_VYUY12_1X24, ADV7604_OP_CH_SEL_RBG, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_12BIT },
+ { V4L2_MBUS_FMT_YUYV12_1X24, ADV7604_OP_CH_SEL_RGB, false, false,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_12BIT },
+ { V4L2_MBUS_FMT_YVYU12_1X24, ADV7604_OP_CH_SEL_RGB, false, true,
+ ADV7604_OP_MODE_SEL_SDR_422_2X | ADV7604_OP_FORMAT_SEL_12BIT },
+};
+
+static const struct adv7604_format_info *
+adv7604_format_info(struct adv7604_state *state, enum v4l2_mbus_pixelcode code)
+{
+ unsigned int i;
+
+ for (i = 0; i < state->info->nformats; ++i) {
+ if (state->info->formats[i].code == code)
+ return &state->info->formats[i];
+ }
+
+ return NULL;
}
/* ----------------------------------------------------------------------- */
@@ -607,18 +837,18 @@
{
struct adv7604_state *state = to_state(sd);
- return state->selected_input == ADV7604_INPUT_VGA_RGB ||
- state->selected_input == ADV7604_INPUT_VGA_COMP;
+ return state->selected_input == ADV7604_PAD_VGA_RGB ||
+ state->selected_input == ADV7604_PAD_VGA_COMP;
}
static inline bool is_digital_input(struct v4l2_subdev *sd)
{
struct adv7604_state *state = to_state(sd);
- return state->selected_input == ADV7604_INPUT_HDMI_PORT_A ||
- state->selected_input == ADV7604_INPUT_HDMI_PORT_B ||
- state->selected_input == ADV7604_INPUT_HDMI_PORT_C ||
- state->selected_input == ADV7604_INPUT_HDMI_PORT_D;
+ return state->selected_input == ADV7604_PAD_HDMI_PORT_A ||
+ state->selected_input == ADV7604_PAD_HDMI_PORT_B ||
+ state->selected_input == ADV7604_PAD_HDMI_PORT_C ||
+ state->selected_input == ADV7604_PAD_HDMI_PORT_D;
}
/* ----------------------------------------------------------------------- */
@@ -644,119 +874,61 @@
static int adv7604_g_register(struct v4l2_subdev *sd,
struct v4l2_dbg_register *reg)
{
- reg->size = 1;
- switch (reg->reg >> 8) {
- case 0:
- reg->val = io_read(sd, reg->reg & 0xff);
- break;
- case 1:
- reg->val = avlink_read(sd, reg->reg & 0xff);
- break;
- case 2:
- reg->val = cec_read(sd, reg->reg & 0xff);
- break;
- case 3:
- reg->val = infoframe_read(sd, reg->reg & 0xff);
- break;
- case 4:
- reg->val = esdp_read(sd, reg->reg & 0xff);
- break;
- case 5:
- reg->val = dpp_read(sd, reg->reg & 0xff);
- break;
- case 6:
- reg->val = afe_read(sd, reg->reg & 0xff);
- break;
- case 7:
- reg->val = rep_read(sd, reg->reg & 0xff);
- break;
- case 8:
- reg->val = edid_read(sd, reg->reg & 0xff);
- break;
- case 9:
- reg->val = hdmi_read(sd, reg->reg & 0xff);
- break;
- case 0xa:
- reg->val = test_read(sd, reg->reg & 0xff);
- break;
- case 0xb:
- reg->val = cp_read(sd, reg->reg & 0xff);
- break;
- case 0xc:
- reg->val = vdp_read(sd, reg->reg & 0xff);
- break;
- default:
+ int ret;
+
+ ret = adv7604_read_reg(sd, reg->reg);
+ if (ret < 0) {
v4l2_info(sd, "Register %03llx not supported\n", reg->reg);
adv7604_inv_register(sd);
- break;
+ return ret;
}
+
+ reg->size = 1;
+ reg->val = ret;
+
return 0;
}
static int adv7604_s_register(struct v4l2_subdev *sd,
const struct v4l2_dbg_register *reg)
{
- u8 val = reg->val & 0xff;
+ int ret;
- switch (reg->reg >> 8) {
- case 0:
- io_write(sd, reg->reg & 0xff, val);
- break;
- case 1:
- avlink_write(sd, reg->reg & 0xff, val);
- break;
- case 2:
- cec_write(sd, reg->reg & 0xff, val);
- break;
- case 3:
- infoframe_write(sd, reg->reg & 0xff, val);
- break;
- case 4:
- esdp_write(sd, reg->reg & 0xff, val);
- break;
- case 5:
- dpp_write(sd, reg->reg & 0xff, val);
- break;
- case 6:
- afe_write(sd, reg->reg & 0xff, val);
- break;
- case 7:
- rep_write(sd, reg->reg & 0xff, val);
- break;
- case 8:
- edid_write(sd, reg->reg & 0xff, val);
- break;
- case 9:
- hdmi_write(sd, reg->reg & 0xff, val);
- break;
- case 0xa:
- test_write(sd, reg->reg & 0xff, val);
- break;
- case 0xb:
- cp_write(sd, reg->reg & 0xff, val);
- break;
- case 0xc:
- vdp_write(sd, reg->reg & 0xff, val);
- break;
- default:
+ ret = adv7604_write_reg(sd, reg->reg, reg->val);
+ if (ret < 0) {
v4l2_info(sd, "Register %03llx not supported\n", reg->reg);
adv7604_inv_register(sd);
- break;
+ return ret;
}
+
return 0;
}
#endif
+static unsigned int adv7604_read_cable_det(struct v4l2_subdev *sd)
+{
+ u8 value = io_read(sd, 0x6f);
+
+ return ((value & 0x10) >> 4)
+ | ((value & 0x08) >> 2)
+ | ((value & 0x04) << 0)
+ | ((value & 0x02) << 2);
+}
+
+static unsigned int adv7611_read_cable_det(struct v4l2_subdev *sd)
+{
+ u8 value = io_read(sd, 0x6f);
+
+ return value & 1;
+}
+
static int adv7604_s_detect_tx_5v_ctrl(struct v4l2_subdev *sd)
{
struct adv7604_state *state = to_state(sd);
- u8 reg_io_6f = io_read(sd, 0x6f);
+ const struct adv7604_chip_info *info = state->info;
return v4l2_ctrl_s_ctrl(state->detect_tx_5v_ctrl,
- ((reg_io_6f & 0x10) >> 4) |
- ((reg_io_6f & 0x08) >> 2) |
- (reg_io_6f & 0x04) |
- ((reg_io_6f & 0x02) << 2));
+ info->read_cable_det(sd));
}
static int find_and_set_predefined_video_timings(struct v4l2_subdev *sd,
@@ -787,11 +959,13 @@
v4l2_dbg(1, debug, sd, "%s", __func__);
- /* reset to default values */
- io_write(sd, 0x16, 0x43);
- io_write(sd, 0x17, 0x5a);
+ if (adv7604_has_afe(state)) {
+ /* reset to default values */
+ io_write(sd, 0x16, 0x43);
+ io_write(sd, 0x17, 0x5a);
+ }
/* disable embedded syncs for auto graphics mode */
- cp_write_and_or(sd, 0x81, 0xef, 0x00);
+ cp_write_clr_set(sd, 0x81, 0x10, 0x00);
cp_write(sd, 0x8f, 0x00);
cp_write(sd, 0x90, 0x00);
cp_write(sd, 0xa2, 0x00);
@@ -829,7 +1003,6 @@
const struct v4l2_bt_timings *bt)
{
struct adv7604_state *state = to_state(sd);
- struct i2c_client *client = v4l2_get_subdevdata(sd);
u32 width = htotal(bt);
u32 height = vtotal(bt);
u16 cp_start_sav = bt->hsync + bt->hbackporch - 4;
@@ -850,12 +1023,13 @@
io_write(sd, 0x00, 0x07); /* video std */
io_write(sd, 0x01, 0x02); /* prim mode */
/* enable embedded syncs for auto graphics mode */
- cp_write_and_or(sd, 0x81, 0xef, 0x10);
+ cp_write_clr_set(sd, 0x81, 0x10, 0x10);
/* Should only be set in auto-graphics mode [REF_02, p. 91-92] */
/* setup PLL_DIV_MAN_EN and PLL_DIV_RATIO */
/* IO-map reg. 0x16 and 0x17 should be written in sequence */
- if (adv_smbus_write_i2c_block_data(client, 0x16, 2, pll))
+ if (adv_smbus_write_i2c_block_data(state, ADV7604_PAGE_IO,
+ 0x16, 2, pll))
v4l2_err(sd, "writing to reg 0x16 and 0x17 failed\n");
/* active video - horizontal timing */
@@ -906,7 +1080,8 @@
offset_buf[3] = offset_c & 0x0ff;
/* Registers must be written in this order with no i2c access in between */
- if (adv_smbus_write_i2c_block_data(state->i2c_cp, 0x77, 4, offset_buf))
+ if (adv_smbus_write_i2c_block_data(state, ADV7604_PAGE_CP,
+ 0x77, 4, offset_buf))
v4l2_err(sd, "%s: i2c error writing to CP reg 0x77, 0x78, 0x79, 0x7a\n", __func__);
}
@@ -935,7 +1110,8 @@
gain_buf[3] = ((gain_c & 0x0ff));
/* Registers must be written in this order with no i2c access in between */
- if (adv_smbus_write_i2c_block_data(state->i2c_cp, 0x73, 4, gain_buf))
+ if (adv_smbus_write_i2c_block_data(state, ADV7604_PAGE_CP,
+ 0x73, 4, gain_buf))
v4l2_err(sd, "%s: i2c error writing to CP reg 0x73, 0x74, 0x75, 0x76\n", __func__);
}
@@ -954,24 +1130,24 @@
switch (state->rgb_quantization_range) {
case V4L2_DV_RGB_RANGE_AUTO:
- if (state->selected_input == ADV7604_INPUT_VGA_RGB) {
+ if (state->selected_input == ADV7604_PAD_VGA_RGB) {
/* Receiving analog RGB signal
* Set RGB full range (0-255) */
- io_write_and_or(sd, 0x02, 0x0f, 0x10);
+ io_write_clr_set(sd, 0x02, 0xf0, 0x10);
break;
}
- if (state->selected_input == ADV7604_INPUT_VGA_COMP) {
+ if (state->selected_input == ADV7604_PAD_VGA_COMP) {
/* Receiving analog YPbPr signal
* Set automode */
- io_write_and_or(sd, 0x02, 0x0f, 0xf0);
+ io_write_clr_set(sd, 0x02, 0xf0, 0xf0);
break;
}
if (hdmi_signal) {
/* Receiving HDMI signal
* Set automode */
- io_write_and_or(sd, 0x02, 0x0f, 0xf0);
+ io_write_clr_set(sd, 0x02, 0xf0, 0xf0);
break;
}
@@ -980,10 +1156,10 @@
* input format (CE/IT) in automatic mode */
if (state->timings.bt.standards & V4L2_DV_BT_STD_CEA861) {
/* RGB limited range (16-235) */
- io_write_and_or(sd, 0x02, 0x0f, 0x00);
+ io_write_clr_set(sd, 0x02, 0xf0, 0x00);
} else {
/* RGB full range (0-255) */
- io_write_and_or(sd, 0x02, 0x0f, 0x10);
+ io_write_clr_set(sd, 0x02, 0xf0, 0x10);
if (is_digital_input(sd) && rgb_output) {
adv7604_set_offset(sd, false, 0x40, 0x40, 0x40);
@@ -994,25 +1170,25 @@
}
break;
case V4L2_DV_RGB_RANGE_LIMITED:
- if (state->selected_input == ADV7604_INPUT_VGA_COMP) {
+ if (state->selected_input == ADV7604_PAD_VGA_COMP) {
/* YCrCb limited range (16-235) */
- io_write_and_or(sd, 0x02, 0x0f, 0x20);
+ io_write_clr_set(sd, 0x02, 0xf0, 0x20);
break;
}
/* RGB limited range (16-235) */
- io_write_and_or(sd, 0x02, 0x0f, 0x00);
+ io_write_clr_set(sd, 0x02, 0xf0, 0x00);
break;
case V4L2_DV_RGB_RANGE_FULL:
- if (state->selected_input == ADV7604_INPUT_VGA_COMP) {
+ if (state->selected_input == ADV7604_PAD_VGA_COMP) {
/* YCrCb full range (0-255) */
- io_write_and_or(sd, 0x02, 0x0f, 0x60);
+ io_write_clr_set(sd, 0x02, 0xf0, 0x60);
break;
}
/* RGB full range (0-255) */
- io_write_and_or(sd, 0x02, 0x0f, 0x10);
+ io_write_clr_set(sd, 0x02, 0xf0, 0x10);
if (is_analog_input(sd) || hdmi_signal)
break;
@@ -1030,7 +1206,9 @@
static int adv7604_s_ctrl(struct v4l2_ctrl *ctrl)
{
- struct v4l2_subdev *sd = to_sd(ctrl);
+ struct v4l2_subdev *sd =
+ &container_of(ctrl->handler, struct adv7604_state, hdl)->sd;
+
struct adv7604_state *state = to_state(sd);
switch (ctrl->id) {
@@ -1051,6 +1229,8 @@
set_rgb_quantization_range(sd);
return 0;
case V4L2_CID_ADV_RX_ANALOG_SAMPLING_PHASE:
+ if (!adv7604_has_afe(state))
+ return -EINVAL;
/* Set the analog sampling phase. This is needed to find the
best sampling phase for analog video: an application or
driver has to try a number of phases and analyze the picture
@@ -1060,7 +1240,7 @@
case V4L2_CID_ADV_RX_FREE_RUN_COLOR_MANUAL:
/* Use the default blue color for free running mode,
or supply your own. */
- cp_write_and_or(sd, 0xbf, ~0x04, (ctrl->val << 2));
+ cp_write_clr_set(sd, 0xbf, 0x04, ctrl->val << 2);
return 0;
case V4L2_CID_ADV_RX_FREE_RUN_COLOR:
cp_write(sd, 0xc0, (ctrl->val & 0xff0000) >> 16);
@@ -1088,7 +1268,10 @@
static inline bool no_lock_tmds(struct v4l2_subdev *sd)
{
- return (io_read(sd, 0x6a) & 0xe0) != 0xe0;
+ struct adv7604_state *state = to_state(sd);
+ const struct adv7604_chip_info *info = state->info;
+
+ return (io_read(sd, 0x6a) & info->tdms_lock_mask) != info->tdms_lock_mask;
}
static inline bool is_hdmi(struct v4l2_subdev *sd)
@@ -1098,6 +1281,15 @@
static inline bool no_lock_sspd(struct v4l2_subdev *sd)
{
+ struct adv7604_state *state = to_state(sd);
+
+ /*
+ * Chips without a AFE don't expose registers for the SSPD, so just assume
+ * that we have a lock.
+ */
+ if (adv7604_has_afe(state))
+ return false;
+
/* TODO channel 2 */
return ((cp_read(sd, 0xb5) & 0xd0) != 0xd0);
}
@@ -1127,6 +1319,11 @@
static inline bool no_lock_cp(struct v4l2_subdev *sd)
{
+ struct adv7604_state *state = to_state(sd);
+
+ if (!adv7604_has_afe(state))
+ return false;
+
/* CP has detected a non standard number of lines on the incoming
video compared to what it is configured to receive by s_dv_timings */
return io_read(sd, 0x12) & 0x01;
@@ -1195,28 +1392,40 @@
return -1;
}
+
static int read_stdi(struct v4l2_subdev *sd, struct stdi_readback *stdi)
{
+ struct adv7604_state *state = to_state(sd);
+ const struct adv7604_chip_info *info = state->info;
+ u8 polarity;
+
if (no_lock_stdi(sd) || no_lock_sspd(sd)) {
v4l2_dbg(2, debug, sd, "%s: STDI and/or SSPD not locked\n", __func__);
return -1;
}
/* read STDI */
- stdi->bl = ((cp_read(sd, 0xb1) & 0x3f) << 8) | cp_read(sd, 0xb2);
- stdi->lcf = ((cp_read(sd, 0xb3) & 0x7) << 8) | cp_read(sd, 0xb4);
+ stdi->bl = cp_read16(sd, 0xb1, 0x3fff);
+ stdi->lcf = cp_read16(sd, info->lcf_reg, 0x7ff);
stdi->lcvs = cp_read(sd, 0xb3) >> 3;
stdi->interlaced = io_read(sd, 0x12) & 0x10;
- /* read SSPD */
- if ((cp_read(sd, 0xb5) & 0x03) == 0x01) {
- stdi->hs_pol = ((cp_read(sd, 0xb5) & 0x10) ?
- ((cp_read(sd, 0xb5) & 0x08) ? '+' : '-') : 'x');
- stdi->vs_pol = ((cp_read(sd, 0xb5) & 0x40) ?
- ((cp_read(sd, 0xb5) & 0x20) ? '+' : '-') : 'x');
+ if (adv7604_has_afe(state)) {
+ /* read SSPD */
+ polarity = cp_read(sd, 0xb5);
+ if ((polarity & 0x03) == 0x01) {
+ stdi->hs_pol = polarity & 0x10
+ ? (polarity & 0x08 ? '+' : '-') : 'x';
+ stdi->vs_pol = polarity & 0x40
+ ? (polarity & 0x20 ? '+' : '-') : 'x';
+ } else {
+ stdi->hs_pol = 'x';
+ stdi->vs_pol = 'x';
+ }
} else {
- stdi->hs_pol = 'x';
- stdi->vs_pol = 'x';
+ polarity = hdmi_read(sd, 0x05);
+ stdi->hs_pol = polarity & 0x20 ? '+' : '-';
+ stdi->vs_pol = polarity & 0x10 ? '+' : '-';
}
if (no_lock_stdi(sd) || no_lock_sspd(sd)) {
@@ -1243,8 +1452,14 @@
static int adv7604_enum_dv_timings(struct v4l2_subdev *sd,
struct v4l2_enum_dv_timings *timings)
{
+ struct adv7604_state *state = to_state(sd);
+
if (timings->index >= ARRAY_SIZE(adv7604_timings) - 1)
return -EINVAL;
+
+ if (timings->pad >= state->source_pad)
+ return -EINVAL;
+
memset(timings->reserved, 0, sizeof(timings->reserved));
timings->timings = adv7604_timings[timings->index];
return 0;
@@ -1253,14 +1468,30 @@
static int adv7604_dv_timings_cap(struct v4l2_subdev *sd,
struct v4l2_dv_timings_cap *cap)
{
+ struct adv7604_state *state = to_state(sd);
+
+ if (cap->pad >= state->source_pad)
+ return -EINVAL;
+
cap->type = V4L2_DV_BT_656_1120;
cap->bt.max_width = 1920;
cap->bt.max_height = 1200;
cap->bt.min_pixelclock = 25000000;
- if (is_digital_input(sd))
+
+ switch (cap->pad) {
+ case ADV7604_PAD_HDMI_PORT_A:
+ case ADV7604_PAD_HDMI_PORT_B:
+ case ADV7604_PAD_HDMI_PORT_C:
+ case ADV7604_PAD_HDMI_PORT_D:
cap->bt.max_pixelclock = 225000000;
- else
+ break;
+ case ADV7604_PAD_VGA_RGB:
+ case ADV7604_PAD_VGA_COMP:
+ default:
cap->bt.max_pixelclock = 170000000;
+ break;
+ }
+
cap->bt.standards = V4L2_DV_BT_STD_CEA861 | V4L2_DV_BT_STD_DMT |
V4L2_DV_BT_STD_GTF | V4L2_DV_BT_STD_CVT;
cap->bt.capabilities = V4L2_DV_BT_CAP_PROGRESSIVE |
@@ -1284,10 +1515,43 @@
}
}
+static unsigned int adv7604_read_hdmi_pixelclock(struct v4l2_subdev *sd)
+{
+ unsigned int freq;
+ int a, b;
+
+ a = hdmi_read(sd, 0x06);
+ b = hdmi_read(sd, 0x3b);
+ if (a < 0 || b < 0)
+ return 0;
+ freq = a * 1000000 + ((b & 0x30) >> 4) * 250000;
+
+ if (is_hdmi(sd)) {
+ /* adjust for deep color mode */
+ unsigned bits_per_channel = ((hdmi_read(sd, 0x0b) & 0x60) >> 4) + 8;
+
+ freq = freq * 8 / bits_per_channel;
+ }
+
+ return freq;
+}
+
+static unsigned int adv7611_read_hdmi_pixelclock(struct v4l2_subdev *sd)
+{
+ int a, b;
+
+ a = hdmi_read(sd, 0x51);
+ b = hdmi_read(sd, 0x52);
+ if (a < 0 || b < 0)
+ return 0;
+ return ((a << 1) | (b >> 7)) * 1000000 + (b & 0x7f) * 1000000 / 128;
+}
+
static int adv7604_query_dv_timings(struct v4l2_subdev *sd,
struct v4l2_dv_timings *timings)
{
struct adv7604_state *state = to_state(sd);
+ const struct adv7604_chip_info *info = state->info;
struct v4l2_bt_timings *bt = &timings->bt;
struct stdi_readback stdi;
@@ -1311,44 +1575,25 @@
V4L2_DV_INTERLACED : V4L2_DV_PROGRESSIVE;
if (is_digital_input(sd)) {
- uint32_t freq;
-
timings->type = V4L2_DV_BT_656_1120;
- bt->width = (hdmi_read(sd, 0x07) & 0x0f) * 256 + hdmi_read(sd, 0x08);
- bt->height = (hdmi_read(sd, 0x09) & 0x0f) * 256 + hdmi_read(sd, 0x0a);
- freq = (hdmi_read(sd, 0x06) * 1000000) +
- ((hdmi_read(sd, 0x3b) & 0x30) >> 4) * 250000;
- if (is_hdmi(sd)) {
- /* adjust for deep color mode */
- unsigned bits_per_channel = ((hdmi_read(sd, 0x0b) & 0x60) >> 4) + 8;
-
- freq = freq * 8 / bits_per_channel;
- }
- bt->pixelclock = freq;
- bt->hfrontporch = (hdmi_read(sd, 0x20) & 0x03) * 256 +
- hdmi_read(sd, 0x21);
- bt->hsync = (hdmi_read(sd, 0x22) & 0x03) * 256 +
- hdmi_read(sd, 0x23);
- bt->hbackporch = (hdmi_read(sd, 0x24) & 0x03) * 256 +
- hdmi_read(sd, 0x25);
- bt->vfrontporch = ((hdmi_read(sd, 0x2a) & 0x1f) * 256 +
- hdmi_read(sd, 0x2b)) / 2;
- bt->vsync = ((hdmi_read(sd, 0x2e) & 0x1f) * 256 +
- hdmi_read(sd, 0x2f)) / 2;
- bt->vbackporch = ((hdmi_read(sd, 0x32) & 0x1f) * 256 +
- hdmi_read(sd, 0x33)) / 2;
+ /* FIXME: All masks are incorrect for ADV7611 */
+ bt->width = hdmi_read16(sd, 0x07, 0xfff);
+ bt->height = hdmi_read16(sd, 0x09, 0xfff);
+ bt->pixelclock = info->read_hdmi_pixelclock(sd);
+ bt->hfrontporch = hdmi_read16(sd, 0x20, 0x3ff);
+ bt->hsync = hdmi_read16(sd, 0x22, 0x3ff);
+ bt->hbackporch = hdmi_read16(sd, 0x24, 0x3ff);
+ bt->vfrontporch = hdmi_read16(sd, 0x2a, 0x1fff) / 2;
+ bt->vsync = hdmi_read16(sd, 0x2e, 0x1fff) / 2;
+ bt->vbackporch = hdmi_read16(sd, 0x32, 0x1fff) / 2;
bt->polarities = ((hdmi_read(sd, 0x05) & 0x10) ? V4L2_DV_VSYNC_POS_POL : 0) |
((hdmi_read(sd, 0x05) & 0x20) ? V4L2_DV_HSYNC_POS_POL : 0);
if (bt->interlaced == V4L2_DV_INTERLACED) {
- bt->height += (hdmi_read(sd, 0x0b) & 0x0f) * 256 +
- hdmi_read(sd, 0x0c);
- bt->il_vfrontporch = ((hdmi_read(sd, 0x2c) & 0x1f) * 256 +
- hdmi_read(sd, 0x2d)) / 2;
- bt->il_vsync = ((hdmi_read(sd, 0x30) & 0x1f) * 256 +
- hdmi_read(sd, 0x31)) / 2;
- bt->vbackporch = ((hdmi_read(sd, 0x34) & 0x1f) * 256 +
- hdmi_read(sd, 0x35)) / 2;
+ bt->height += hdmi_read16(sd, 0x0b, 0xfff);
+ bt->il_vfrontporch = hdmi_read16(sd, 0x2c, 0x1fff) / 2;
+ bt->il_vsync = hdmi_read16(sd, 0x30, 0x1fff) / 2;
+ bt->vbackporch = hdmi_read16(sd, 0x34, 0x1fff) / 2;
}
adv7604_fill_optional_dv_timings_fields(sd, timings);
} else {
@@ -1378,11 +1623,11 @@
v4l2_dbg(1, debug, sd, "%s: restart STDI\n", __func__);
/* TODO restart STDI for Sync Channel 2 */
/* enter one-shot mode */
- cp_write_and_or(sd, 0x86, 0xf9, 0x00);
+ cp_write_clr_set(sd, 0x86, 0x06, 0x00);
/* trigger STDI restart */
- cp_write_and_or(sd, 0x86, 0xf9, 0x04);
+ cp_write_clr_set(sd, 0x86, 0x06, 0x04);
/* reset to continuous mode */
- cp_write_and_or(sd, 0x86, 0xf9, 0x02);
+ cp_write_clr_set(sd, 0x86, 0x06, 0x02);
state->restart_stdi_once = false;
return -ENOLINK;
}
@@ -1441,7 +1686,7 @@
state->timings = *timings;
- cp_write(sd, 0x91, bt->interlaced ? 0x50 : 0x10);
+ cp_write_clr_set(sd, 0x91, 0x40, bt->interlaced ? 0x40 : 0x00);
/* Use prim_mode and vid_std when available */
err = configure_predefined_video_timings(sd, timings);
@@ -1468,6 +1713,16 @@
return 0;
}
+static void adv7604_set_termination(struct v4l2_subdev *sd, bool enable)
+{
+ hdmi_write(sd, 0x01, enable ? 0x00 : 0x78);
+}
+
+static void adv7611_set_termination(struct v4l2_subdev *sd, bool enable)
+{
+ hdmi_write(sd, 0x83, enable ? 0xfe : 0xff);
+}
+
static void enable_input(struct v4l2_subdev *sd)
{
struct adv7604_state *state = to_state(sd);
@@ -1475,10 +1730,10 @@
if (is_analog_input(sd)) {
io_write(sd, 0x15, 0xb0); /* Disable Tristate of Pins (no audio) */
} else if (is_digital_input(sd)) {
- hdmi_write_and_or(sd, 0x00, 0xfc, state->selected_input);
- hdmi_write(sd, 0x01, 0x00); /* Enable HDMI clock terminators */
+ hdmi_write_clr_set(sd, 0x00, 0x03, state->selected_input);
+ state->info->set_termination(sd, true);
io_write(sd, 0x15, 0xa0); /* Disable Tristate of Pins */
- hdmi_write_and_or(sd, 0x1a, 0xef, 0x00); /* Unmute audio */
+ hdmi_write_clr_set(sd, 0x1a, 0x10, 0x00); /* Unmute audio */
} else {
v4l2_dbg(2, debug, sd, "%s: Unknown port %d selected\n",
__func__, state->selected_input);
@@ -1487,67 +1742,36 @@
static void disable_input(struct v4l2_subdev *sd)
{
- hdmi_write_and_or(sd, 0x1a, 0xef, 0x10); /* Mute audio */
+ struct adv7604_state *state = to_state(sd);
+
+ hdmi_write_clr_set(sd, 0x1a, 0x10, 0x10); /* Mute audio */
msleep(16); /* 512 samples with >= 32 kHz sample rate [REF_03, c. 7.16.10] */
io_write(sd, 0x15, 0xbe); /* Tristate all outputs from video core */
- hdmi_write(sd, 0x01, 0x78); /* Disable HDMI clock terminators */
+ state->info->set_termination(sd, false);
}
static void select_input(struct v4l2_subdev *sd)
{
struct adv7604_state *state = to_state(sd);
+ const struct adv7604_chip_info *info = state->info;
if (is_analog_input(sd)) {
- /* reset ADI recommended settings for HDMI: */
- /* "ADV7604 Register Settings Recommendations (rev. 2.5, June 2010)" p. 4. */
- hdmi_write(sd, 0x0d, 0x04); /* HDMI filter optimization */
- hdmi_write(sd, 0x3d, 0x00); /* DDC bus active pull-up control */
- hdmi_write(sd, 0x3e, 0x74); /* TMDS PLL optimization */
- hdmi_write(sd, 0x4e, 0x3b); /* TMDS PLL optimization */
- hdmi_write(sd, 0x57, 0x74); /* TMDS PLL optimization */
- hdmi_write(sd, 0x58, 0x63); /* TMDS PLL optimization */
- hdmi_write(sd, 0x8d, 0x18); /* equaliser */
- hdmi_write(sd, 0x8e, 0x34); /* equaliser */
- hdmi_write(sd, 0x93, 0x88); /* equaliser */
- hdmi_write(sd, 0x94, 0x2e); /* equaliser */
- hdmi_write(sd, 0x96, 0x00); /* enable automatic EQ changing */
+ adv7604_write_reg_seq(sd, info->recommended_settings[0]);
afe_write(sd, 0x00, 0x08); /* power up ADC */
afe_write(sd, 0x01, 0x06); /* power up Analog Front End */
afe_write(sd, 0xc8, 0x00); /* phase control */
-
- /* set ADI recommended settings for digitizer */
- /* "ADV7604 Register Settings Recommendations (rev. 2.5, June 2010)" p. 17. */
- afe_write(sd, 0x12, 0x7b); /* ADC noise shaping filter controls */
- afe_write(sd, 0x0c, 0x1f); /* CP core gain controls */
- cp_write(sd, 0x3e, 0x04); /* CP core pre-gain control */
- cp_write(sd, 0xc3, 0x39); /* CP coast control. Graphics mode */
- cp_write(sd, 0x40, 0x5c); /* CP core pre-gain control. Graphics mode */
} else if (is_digital_input(sd)) {
hdmi_write(sd, 0x00, state->selected_input & 0x03);
- /* set ADI recommended settings for HDMI: */
- /* "ADV7604 Register Settings Recommendations (rev. 2.5, June 2010)" p. 4. */
- hdmi_write(sd, 0x0d, 0x84); /* HDMI filter optimization */
- hdmi_write(sd, 0x3d, 0x10); /* DDC bus active pull-up control */
- hdmi_write(sd, 0x3e, 0x39); /* TMDS PLL optimization */
- hdmi_write(sd, 0x4e, 0x3b); /* TMDS PLL optimization */
- hdmi_write(sd, 0x57, 0xb6); /* TMDS PLL optimization */
- hdmi_write(sd, 0x58, 0x03); /* TMDS PLL optimization */
- hdmi_write(sd, 0x8d, 0x18); /* equaliser */
- hdmi_write(sd, 0x8e, 0x34); /* equaliser */
- hdmi_write(sd, 0x93, 0x8b); /* equaliser */
- hdmi_write(sd, 0x94, 0x2d); /* equaliser */
- hdmi_write(sd, 0x96, 0x01); /* enable automatic EQ changing */
+ adv7604_write_reg_seq(sd, info->recommended_settings[1]);
- afe_write(sd, 0x00, 0xff); /* power down ADC */
- afe_write(sd, 0x01, 0xfe); /* power down Analog Front End */
- afe_write(sd, 0xc8, 0x40); /* phase control */
+ if (adv7604_has_afe(state)) {
+ afe_write(sd, 0x00, 0xff); /* power down ADC */
+ afe_write(sd, 0x01, 0xfe); /* power down Analog Front End */
+ afe_write(sd, 0xc8, 0x40); /* phase control */
+ }
- /* reset ADI recommended settings for digitizer */
- /* "ADV7604 Register Settings Recommendations (rev. 2.5, June 2010)" p. 17. */
- afe_write(sd, 0x12, 0xfb); /* ADC noise shaping filter controls */
- afe_write(sd, 0x0c, 0x0d); /* CP core gain controls */
cp_write(sd, 0x3e, 0x00); /* CP core pre-gain control */
cp_write(sd, 0xc3, 0x39); /* CP coast control. Graphics mode */
cp_write(sd, 0x40, 0x80); /* CP core pre-gain control. Graphics mode */
@@ -1568,6 +1792,9 @@
if (input == state->selected_input)
return 0;
+ if (input > state->info->max_port)
+ return -EINVAL;
+
state->selected_input = input;
disable_input(sd);
@@ -1579,34 +1806,139 @@
return 0;
}
-static int adv7604_enum_mbus_fmt(struct v4l2_subdev *sd, unsigned int index,
- enum v4l2_mbus_pixelcode *code)
-{
- if (index)
- return -EINVAL;
- /* Good enough for now */
- *code = V4L2_MBUS_FMT_FIXED;
- return 0;
-}
-
-static int adv7604_g_mbus_fmt(struct v4l2_subdev *sd,
- struct v4l2_mbus_framefmt *fmt)
+static int adv7604_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_fh *fh,
+ struct v4l2_subdev_mbus_code_enum *code)
{
struct adv7604_state *state = to_state(sd);
- fmt->width = state->timings.bt.width;
- fmt->height = state->timings.bt.height;
- fmt->code = V4L2_MBUS_FMT_FIXED;
- fmt->field = V4L2_FIELD_NONE;
- if (state->timings.bt.standards & V4L2_DV_BT_STD_CEA861) {
- fmt->colorspace = (state->timings.bt.height <= 576) ?
+ if (code->index >= state->info->nformats)
+ return -EINVAL;
+
+ code->code = state->info->formats[code->index].code;
+
+ return 0;
+}
+
+static void adv7604_fill_format(struct adv7604_state *state,
+ struct v4l2_mbus_framefmt *format)
+{
+ memset(format, 0, sizeof(*format));
+
+ format->width = state->timings.bt.width;
+ format->height = state->timings.bt.height;
+ format->field = V4L2_FIELD_NONE;
+
+ if (state->timings.bt.standards & V4L2_DV_BT_STD_CEA861)
+ format->colorspace = (state->timings.bt.height <= 576) ?
V4L2_COLORSPACE_SMPTE170M : V4L2_COLORSPACE_REC709;
+}
+
+/*
+ * Compute the op_ch_sel value required to obtain on the bus the component order
+ * corresponding to the selected format taking into account bus reordering
+ * applied by the board at the output of the device.
+ *
+ * The following table gives the op_ch_value from the format component order
+ * (expressed as op_ch_sel value in column) and the bus reordering (expressed as
+ * adv7604_bus_order value in row).
+ *
+ * | GBR(0) GRB(1) BGR(2) RGB(3) BRG(4) RBG(5)
+ * ----------+-------------------------------------------------
+ * RGB (NOP) | GBR GRB BGR RGB BRG RBG
+ * GRB (1-2) | BGR RGB GBR GRB RBG BRG
+ * RBG (2-3) | GRB GBR BRG RBG BGR RGB
+ * BGR (1-3) | RBG BRG RGB BGR GRB GBR
+ * BRG (ROR) | BRG RBG GRB GBR RGB BGR
+ * GBR (ROL) | RGB BGR RBG BRG GBR GRB
+ */
+static unsigned int adv7604_op_ch_sel(struct adv7604_state *state)
+{
+#define _SEL(a,b,c,d,e,f) { \
+ ADV7604_OP_CH_SEL_##a, ADV7604_OP_CH_SEL_##b, ADV7604_OP_CH_SEL_##c, \
+ ADV7604_OP_CH_SEL_##d, ADV7604_OP_CH_SEL_##e, ADV7604_OP_CH_SEL_##f }
+#define _BUS(x) [ADV7604_BUS_ORDER_##x]
+
+ static const unsigned int op_ch_sel[6][6] = {
+ _BUS(RGB) /* NOP */ = _SEL(GBR, GRB, BGR, RGB, BRG, RBG),
+ _BUS(GRB) /* 1-2 */ = _SEL(BGR, RGB, GBR, GRB, RBG, BRG),
+ _BUS(RBG) /* 2-3 */ = _SEL(GRB, GBR, BRG, RBG, BGR, RGB),
+ _BUS(BGR) /* 1-3 */ = _SEL(RBG, BRG, RGB, BGR, GRB, GBR),
+ _BUS(BRG) /* ROR */ = _SEL(BRG, RBG, GRB, GBR, RGB, BGR),
+ _BUS(GBR) /* ROL */ = _SEL(RGB, BGR, RBG, BRG, GBR, GRB),
+ };
+
+ return op_ch_sel[state->pdata.bus_order][state->format->op_ch_sel >> 5];
+}
+
+static void adv7604_setup_format(struct adv7604_state *state)
+{
+ struct v4l2_subdev *sd = &state->sd;
+
+ io_write_clr_set(sd, 0x02, 0x02,
+ state->format->rgb_out ? ADV7604_RGB_OUT : 0);
+ io_write(sd, 0x03, state->format->op_format_sel |
+ state->pdata.op_format_mode_sel);
+ io_write_clr_set(sd, 0x04, 0xe0, adv7604_op_ch_sel(state));
+ io_write_clr_set(sd, 0x05, 0x01,
+ state->format->swap_cb_cr ? ADV7604_OP_SWAP_CB_CR : 0);
+}
+
+static int adv7604_get_format(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh,
+ struct v4l2_subdev_format *format)
+{
+ struct adv7604_state *state = to_state(sd);
+
+ if (format->pad != state->source_pad)
+ return -EINVAL;
+
+ adv7604_fill_format(state, &format->format);
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
+ struct v4l2_mbus_framefmt *fmt;
+
+ fmt = v4l2_subdev_get_try_format(fh, format->pad);
+ format->format.code = fmt->code;
+ } else {
+ format->format.code = state->format->code;
}
+
+ return 0;
+}
+
+static int adv7604_set_format(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh,
+ struct v4l2_subdev_format *format)
+{
+ struct adv7604_state *state = to_state(sd);
+ const struct adv7604_format_info *info;
+
+ if (format->pad != state->source_pad)
+ return -EINVAL;
+
+ info = adv7604_format_info(state, format->format.code);
+ if (info == NULL)
+ info = adv7604_format_info(state, V4L2_MBUS_FMT_YUYV8_2X8);
+
+ adv7604_fill_format(state, &format->format);
+ format->format.code = info->code;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
+ struct v4l2_mbus_framefmt *fmt;
+
+ fmt = v4l2_subdev_get_try_format(fh, format->pad);
+ fmt->code = format->format.code;
+ } else {
+ state->format = info;
+ adv7604_setup_format(state);
+ }
+
return 0;
}
static int adv7604_isr(struct v4l2_subdev *sd, u32 status, bool *handled)
{
+ struct adv7604_state *state = to_state(sd);
+ const struct adv7604_chip_info *info = state->info;
const u8 irq_reg_0x43 = io_read(sd, 0x43);
const u8 irq_reg_0x6b = io_read(sd, 0x6b);
const u8 irq_reg_0x70 = io_read(sd, 0x70);
@@ -1625,7 +1957,9 @@
/* format change */
fmt_change = irq_reg_0x43 & 0x98;
- fmt_change_digital = is_digital_input(sd) ? (irq_reg_0x6b & 0xc0) : 0;
+ fmt_change_digital = is_digital_input(sd)
+ ? irq_reg_0x6b & info->fmt_change_digital_mask
+ : 0;
if (fmt_change || fmt_change_digital) {
v4l2_dbg(1, debug, sd,
@@ -1647,7 +1981,7 @@
}
/* tx 5v detect */
- tx_5v = io_read(sd, 0x70) & 0x1e;
+ tx_5v = io_read(sd, 0x70) & info->cable_det_mask;
if (tx_5v) {
v4l2_dbg(1, debug, sd, "%s: tx_5v: 0x%x\n", __func__, tx_5v);
io_write(sd, 0x71, tx_5v);
@@ -1663,7 +1997,7 @@
struct adv7604_state *state = to_state(sd);
u8 *data = NULL;
- if (edid->pad > ADV7604_EDID_PORT_D)
+ if (edid->pad > ADV7604_PAD_HDMI_PORT_D)
return -EINVAL;
if (edid->blocks == 0)
return -EINVAL;
@@ -1678,10 +2012,10 @@
edid->blocks = state->edid.blocks;
switch (edid->pad) {
- case ADV7604_EDID_PORT_A:
- case ADV7604_EDID_PORT_B:
- case ADV7604_EDID_PORT_C:
- case ADV7604_EDID_PORT_D:
+ case ADV7604_PAD_HDMI_PORT_A:
+ case ADV7604_PAD_HDMI_PORT_B:
+ case ADV7604_PAD_HDMI_PORT_C:
+ case ADV7604_PAD_HDMI_PORT_D:
if (state->edid.present & (1 << edid->pad))
data = state->edid.edid;
break;
@@ -1729,20 +2063,20 @@
static int adv7604_set_edid(struct v4l2_subdev *sd, struct v4l2_edid *edid)
{
struct adv7604_state *state = to_state(sd);
+ const struct adv7604_chip_info *info = state->info;
int spa_loc;
- int tmp = 0;
int err;
int i;
- if (edid->pad > ADV7604_EDID_PORT_D)
+ if (edid->pad > ADV7604_PAD_HDMI_PORT_D)
return -EINVAL;
if (edid->start_block != 0)
return -EINVAL;
if (edid->blocks == 0) {
/* Disable hotplug and I2C access to EDID RAM from DDC port */
state->edid.present &= ~(1 << edid->pad);
- v4l2_subdev_notify(sd, ADV7604_HOTPLUG, (void *)&state->edid.present);
- rep_write_and_or(sd, 0x77, 0xf0, state->edid.present);
+ adv7604_set_hpd(state, state->edid.present);
+ rep_write_clr_set(sd, info->edid_enable_reg, 0x0f, state->edid.present);
/* Fall back to a 16:9 aspect ratio */
state->aspect_ratio.numerator = 16;
@@ -1765,35 +2099,41 @@
/* Disable hotplug and I2C access to EDID RAM from DDC port */
cancel_delayed_work_sync(&state->delayed_work_enable_hotplug);
- v4l2_subdev_notify(sd, ADV7604_HOTPLUG, (void *)&tmp);
- rep_write_and_or(sd, 0x77, 0xf0, 0x00);
+ adv7604_set_hpd(state, 0);
+ rep_write_clr_set(sd, info->edid_enable_reg, 0x0f, 0x00);
spa_loc = get_edid_spa_location(edid->edid);
if (spa_loc < 0)
spa_loc = 0xc0; /* Default value [REF_02, p. 116] */
switch (edid->pad) {
- case ADV7604_EDID_PORT_A:
+ case ADV7604_PAD_HDMI_PORT_A:
state->spa_port_a[0] = edid->edid[spa_loc];
state->spa_port_a[1] = edid->edid[spa_loc + 1];
break;
- case ADV7604_EDID_PORT_B:
+ case ADV7604_PAD_HDMI_PORT_B:
rep_write(sd, 0x70, edid->edid[spa_loc]);
rep_write(sd, 0x71, edid->edid[spa_loc + 1]);
break;
- case ADV7604_EDID_PORT_C:
+ case ADV7604_PAD_HDMI_PORT_C:
rep_write(sd, 0x72, edid->edid[spa_loc]);
rep_write(sd, 0x73, edid->edid[spa_loc + 1]);
break;
- case ADV7604_EDID_PORT_D:
+ case ADV7604_PAD_HDMI_PORT_D:
rep_write(sd, 0x74, edid->edid[spa_loc]);
rep_write(sd, 0x75, edid->edid[spa_loc + 1]);
break;
default:
return -EINVAL;
}
- rep_write(sd, 0x76, spa_loc & 0xff);
- rep_write_and_or(sd, 0x77, 0xbf, (spa_loc >> 2) & 0x40);
+
+ if (info->type == ADV7604) {
+ rep_write(sd, 0x76, spa_loc & 0xff);
+ rep_write_clr_set(sd, 0x77, 0x40, (spa_loc & 0x100) >> 2);
+ } else {
+ /* FIXME: Where is the SPA location LSB register ? */
+ rep_write_clr_set(sd, 0x71, 0x01, (spa_loc & 0x100) >> 8);
+ }
edid->edid[spa_loc] = state->spa_port_a[0];
edid->edid[spa_loc + 1] = state->spa_port_a[1];
@@ -1812,10 +2152,10 @@
/* adv7604 calculates the checksums and enables I2C access to internal
EDID RAM from DDC port. */
- rep_write_and_or(sd, 0x77, 0xf0, state->edid.present);
+ rep_write_clr_set(sd, info->edid_enable_reg, 0x0f, state->edid.present);
for (i = 0; i < 1000; i++) {
- if (rep_read(sd, 0x7d) & state->edid.present)
+ if (rep_read(sd, info->edid_status_reg) & state->edid.present)
break;
mdelay(1);
}
@@ -1878,17 +2218,20 @@
static int adv7604_log_status(struct v4l2_subdev *sd)
{
struct adv7604_state *state = to_state(sd);
+ const struct adv7604_chip_info *info = state->info;
struct v4l2_dv_timings timings;
struct stdi_readback stdi;
u8 reg_io_0x02 = io_read(sd, 0x02);
+ u8 edid_enabled;
+ u8 cable_det;
- char *csc_coeff_sel_rb[16] = {
+ static const char * const csc_coeff_sel_rb[16] = {
"bypassed", "YPbPr601 -> RGB", "reserved", "YPbPr709 -> RGB",
"reserved", "RGB -> YPbPr601", "reserved", "RGB -> YPbPr709",
"reserved", "YPbPr709 -> YPbPr601", "YPbPr601 -> YPbPr709",
"reserved", "reserved", "reserved", "reserved", "manual"
};
- char *input_color_space_txt[16] = {
+ static const char * const input_color_space_txt[16] = {
"RGB limited range (16-235)", "RGB full range (0-255)",
"YCbCr Bt.601 (16-235)", "YCbCr Bt.709 (16-235)",
"xvYCC Bt.601", "xvYCC Bt.709",
@@ -1896,12 +2239,12 @@
"invalid", "invalid", "invalid", "invalid", "invalid",
"invalid", "invalid", "automatic"
};
- char *rgb_quantization_range_txt[] = {
+ static const char * const rgb_quantization_range_txt[] = {
"Automatic",
"RGB limited range (16-235)",
"RGB full range (0-255)",
};
- char *deep_color_mode_txt[4] = {
+ static const char * const deep_color_mode_txt[4] = {
"8-bits per channel",
"10-bits per channel",
"12-bits per channel",
@@ -1910,20 +2253,22 @@
v4l2_info(sd, "-----Chip status-----\n");
v4l2_info(sd, "Chip power: %s\n", no_power(sd) ? "off" : "on");
+ edid_enabled = rep_read(sd, info->edid_status_reg);
v4l2_info(sd, "EDID enabled port A: %s, B: %s, C: %s, D: %s\n",
- ((rep_read(sd, 0x7d) & 0x01) ? "Yes" : "No"),
- ((rep_read(sd, 0x7d) & 0x02) ? "Yes" : "No"),
- ((rep_read(sd, 0x7d) & 0x04) ? "Yes" : "No"),
- ((rep_read(sd, 0x7d) & 0x08) ? "Yes" : "No"));
+ ((edid_enabled & 0x01) ? "Yes" : "No"),
+ ((edid_enabled & 0x02) ? "Yes" : "No"),
+ ((edid_enabled & 0x04) ? "Yes" : "No"),
+ ((edid_enabled & 0x08) ? "Yes" : "No"));
v4l2_info(sd, "CEC: %s\n", !!(cec_read(sd, 0x2a) & 0x01) ?
"enabled" : "disabled");
v4l2_info(sd, "-----Signal status-----\n");
+ cable_det = info->read_cable_det(sd);
v4l2_info(sd, "Cable detected (+5V power) port A: %s, B: %s, C: %s, D: %s\n",
- ((io_read(sd, 0x6f) & 0x10) ? "Yes" : "No"),
- ((io_read(sd, 0x6f) & 0x08) ? "Yes" : "No"),
- ((io_read(sd, 0x6f) & 0x04) ? "Yes" : "No"),
- ((io_read(sd, 0x6f) & 0x02) ? "Yes" : "No"));
+ ((cable_det & 0x01) ? "Yes" : "No"),
+ ((cable_det & 0x02) ? "Yes" : "No"),
+ ((cable_det & 0x04) ? "Yes" : "No"),
+ ((cable_det & 0x08) ? "Yes" : "No"));
v4l2_info(sd, "TMDS signal detected: %s\n",
no_signal_tmds(sd) ? "false" : "true");
v4l2_info(sd, "TMDS signal locked: %s\n",
@@ -2017,13 +2362,6 @@
static const struct v4l2_subdev_core_ops adv7604_core_ops = {
.log_status = adv7604_log_status,
- .g_ext_ctrls = v4l2_subdev_g_ext_ctrls,
- .try_ext_ctrls = v4l2_subdev_try_ext_ctrls,
- .s_ext_ctrls = v4l2_subdev_s_ext_ctrls,
- .g_ctrl = v4l2_subdev_g_ctrl,
- .s_ctrl = v4l2_subdev_s_ctrl,
- .queryctrl = v4l2_subdev_queryctrl,
- .querymenu = v4l2_subdev_querymenu,
.interrupt_service_routine = adv7604_isr,
#ifdef CONFIG_VIDEO_ADV_DEBUG
.g_register = adv7604_g_register,
@@ -2037,17 +2375,16 @@
.s_dv_timings = adv7604_s_dv_timings,
.g_dv_timings = adv7604_g_dv_timings,
.query_dv_timings = adv7604_query_dv_timings,
- .enum_dv_timings = adv7604_enum_dv_timings,
- .dv_timings_cap = adv7604_dv_timings_cap,
- .enum_mbus_fmt = adv7604_enum_mbus_fmt,
- .g_mbus_fmt = adv7604_g_mbus_fmt,
- .try_mbus_fmt = adv7604_g_mbus_fmt,
- .s_mbus_fmt = adv7604_g_mbus_fmt,
};
static const struct v4l2_subdev_pad_ops adv7604_pad_ops = {
+ .enum_mbus_code = adv7604_enum_mbus_code,
+ .get_fmt = adv7604_get_format,
+ .set_fmt = adv7604_set_format,
.get_edid = adv7604_get_edid,
.set_edid = adv7604_set_edid,
+ .dv_timings_cap = adv7604_dv_timings_cap,
+ .enum_dv_timings = adv7604_enum_dv_timings,
};
static const struct v4l2_subdev_ops adv7604_ops = {
@@ -2096,6 +2433,7 @@
static int adv7604_core_init(struct v4l2_subdev *sd)
{
struct adv7604_state *state = to_state(sd);
+ const struct adv7604_chip_info *info = state->info;
struct adv7604_platform_data *pdata = &state->pdata;
hdmi_write(sd, 0x48,
@@ -2104,28 +2442,33 @@
disable_input(sd);
+ if (pdata->default_input >= 0 &&
+ pdata->default_input < state->source_pad) {
+ state->selected_input = pdata->default_input;
+ select_input(sd);
+ enable_input(sd);
+ }
+
/* power */
io_write(sd, 0x0c, 0x42); /* Power up part and power down VDP */
io_write(sd, 0x0b, 0x44); /* Power down ESDP block */
cp_write(sd, 0xcf, 0x01); /* Power down macrovision */
/* video format */
- io_write_and_or(sd, 0x02, 0xf0,
+ io_write_clr_set(sd, 0x02, 0x0f,
pdata->alt_gamma << 3 |
pdata->op_656_range << 2 |
- pdata->rgb_out << 1 |
pdata->alt_data_sat << 0);
- io_write(sd, 0x03, pdata->op_format_sel);
- io_write_and_or(sd, 0x04, 0x1f, pdata->op_ch_sel << 5);
- io_write_and_or(sd, 0x05, 0xf0, pdata->blank_data << 3 |
- pdata->insert_av_codes << 2 |
- pdata->replicate_av_codes << 1 |
- pdata->invert_cbcr << 0);
+ io_write_clr_set(sd, 0x05, 0x0e, pdata->blank_data << 3 |
+ pdata->insert_av_codes << 2 |
+ pdata->replicate_av_codes << 1);
+ adv7604_setup_format(state);
cp_write(sd, 0x69, 0x30); /* Enable CP CSC */
/* VS, HS polarities */
- io_write(sd, 0x06, 0xa0 | pdata->inv_vs_pol << 2 | pdata->inv_hs_pol << 1);
+ io_write(sd, 0x06, 0xa0 | pdata->inv_vs_pol << 2 |
+ pdata->inv_hs_pol << 1 | pdata->inv_llc_pol);
/* Adjust drive strength */
io_write(sd, 0x14, 0x40 | pdata->dr_str_data << 4 |
@@ -2142,52 +2485,46 @@
for digital formats */
/* HDMI audio */
- hdmi_write_and_or(sd, 0x15, 0xfc, 0x03); /* Mute on FIFO over-/underflow [REF_01, c. 1.2.18] */
- hdmi_write_and_or(sd, 0x1a, 0xf1, 0x08); /* Wait 1 s before unmute */
- hdmi_write_and_or(sd, 0x68, 0xf9, 0x06); /* FIFO reset on over-/underflow [REF_01, c. 1.2.19] */
+ hdmi_write_clr_set(sd, 0x15, 0x03, 0x03); /* Mute on FIFO over-/underflow [REF_01, c. 1.2.18] */
+ hdmi_write_clr_set(sd, 0x1a, 0x0e, 0x08); /* Wait 1 s before unmute */
+ hdmi_write_clr_set(sd, 0x68, 0x06, 0x06); /* FIFO reset on over-/underflow [REF_01, c. 1.2.19] */
/* TODO from platform data */
afe_write(sd, 0xb5, 0x01); /* Setting MCLK to 256Fs */
- afe_write(sd, 0x02, pdata->ain_sel); /* Select analog input muxing mode */
- io_write_and_or(sd, 0x30, ~(1 << 4), pdata->output_bus_lsb_to_msb << 4);
+ if (adv7604_has_afe(state)) {
+ afe_write(sd, 0x02, pdata->ain_sel); /* Select analog input muxing mode */
+ io_write_clr_set(sd, 0x30, 1 << 4, pdata->output_bus_lsb_to_msb << 4);
+ }
/* interrupts */
- io_write(sd, 0x40, 0xc2); /* Configure INT1 */
- io_write(sd, 0x41, 0xd7); /* STDI irq for any change, disable INT2 */
+ io_write(sd, 0x40, 0xc0 | pdata->int1_config); /* Configure INT1 */
io_write(sd, 0x46, 0x98); /* Enable SSPD, STDI and CP unlocked interrupts */
- io_write(sd, 0x6e, 0xc1); /* Enable V_LOCKED, DE_REGEN_LCK, HDMI_MODE interrupts */
- io_write(sd, 0x73, 0x1e); /* Enable CABLE_DET_A_ST (+5v) interrupts */
+ io_write(sd, 0x6e, info->fmt_change_digital_mask); /* Enable V_LOCKED and DE_REGEN_LCK interrupts */
+ io_write(sd, 0x73, info->cable_det_mask); /* Enable cable detection (+5v) interrupts */
+ info->setup_irqs(sd);
return v4l2_ctrl_handler_setup(sd->ctrl_handler);
}
+static void adv7604_setup_irqs(struct v4l2_subdev *sd)
+{
+ io_write(sd, 0x41, 0xd7); /* STDI irq for any change, disable INT2 */
+}
+
+static void adv7611_setup_irqs(struct v4l2_subdev *sd)
+{
+ io_write(sd, 0x41, 0xd0); /* STDI irq for any change, disable INT2 */
+}
+
static void adv7604_unregister_clients(struct adv7604_state *state)
{
- if (state->i2c_avlink)
- i2c_unregister_device(state->i2c_avlink);
- if (state->i2c_cec)
- i2c_unregister_device(state->i2c_cec);
- if (state->i2c_infoframe)
- i2c_unregister_device(state->i2c_infoframe);
- if (state->i2c_esdp)
- i2c_unregister_device(state->i2c_esdp);
- if (state->i2c_dpp)
- i2c_unregister_device(state->i2c_dpp);
- if (state->i2c_afe)
- i2c_unregister_device(state->i2c_afe);
- if (state->i2c_repeater)
- i2c_unregister_device(state->i2c_repeater);
- if (state->i2c_edid)
- i2c_unregister_device(state->i2c_edid);
- if (state->i2c_hdmi)
- i2c_unregister_device(state->i2c_hdmi);
- if (state->i2c_test)
- i2c_unregister_device(state->i2c_test);
- if (state->i2c_cp)
- i2c_unregister_device(state->i2c_cp);
- if (state->i2c_vdp)
- i2c_unregister_device(state->i2c_vdp);
+ unsigned int i;
+
+ for (i = 1; i < ARRAY_SIZE(state->i2c_clients); ++i) {
+ if (state->i2c_clients[i])
+ i2c_unregister_device(state->i2c_clients[i]);
+ }
}
static struct i2c_client *adv7604_dummy_client(struct v4l2_subdev *sd,
@@ -2200,15 +2537,219 @@
return i2c_new_dummy(client->adapter, io_read(sd, io_reg) >> 1);
}
+static const struct adv7604_reg_seq adv7604_recommended_settings_afe[] = {
+ /* reset ADI recommended settings for HDMI: */
+ /* "ADV7604 Register Settings Recommendations (rev. 2.5, June 2010)" p. 4. */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x0d), 0x04 }, /* HDMI filter optimization */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x0d), 0x04 }, /* HDMI filter optimization */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x3d), 0x00 }, /* DDC bus active pull-up control */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x3e), 0x74 }, /* TMDS PLL optimization */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x4e), 0x3b }, /* TMDS PLL optimization */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x57), 0x74 }, /* TMDS PLL optimization */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x58), 0x63 }, /* TMDS PLL optimization */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x8d), 0x18 }, /* equaliser */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x8e), 0x34 }, /* equaliser */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x93), 0x88 }, /* equaliser */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x94), 0x2e }, /* equaliser */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x96), 0x00 }, /* enable automatic EQ changing */
+
+ /* set ADI recommended settings for digitizer */
+ /* "ADV7604 Register Settings Recommendations (rev. 2.5, June 2010)" p. 17. */
+ { ADV7604_REG(ADV7604_PAGE_AFE, 0x12), 0x7b }, /* ADC noise shaping filter controls */
+ { ADV7604_REG(ADV7604_PAGE_AFE, 0x0c), 0x1f }, /* CP core gain controls */
+ { ADV7604_REG(ADV7604_PAGE_CP, 0x3e), 0x04 }, /* CP core pre-gain control */
+ { ADV7604_REG(ADV7604_PAGE_CP, 0xc3), 0x39 }, /* CP coast control. Graphics mode */
+ { ADV7604_REG(ADV7604_PAGE_CP, 0x40), 0x5c }, /* CP core pre-gain control. Graphics mode */
+
+ { ADV7604_REG_SEQ_TERM, 0 },
+};
+
+static const struct adv7604_reg_seq adv7604_recommended_settings_hdmi[] = {
+ /* set ADI recommended settings for HDMI: */
+ /* "ADV7604 Register Settings Recommendations (rev. 2.5, June 2010)" p. 4. */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x0d), 0x84 }, /* HDMI filter optimization */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x3d), 0x10 }, /* DDC bus active pull-up control */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x3e), 0x39 }, /* TMDS PLL optimization */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x4e), 0x3b }, /* TMDS PLL optimization */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x57), 0xb6 }, /* TMDS PLL optimization */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x58), 0x03 }, /* TMDS PLL optimization */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x8d), 0x18 }, /* equaliser */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x8e), 0x34 }, /* equaliser */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x93), 0x8b }, /* equaliser */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x94), 0x2d }, /* equaliser */
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x96), 0x01 }, /* enable automatic EQ changing */
+
+ /* reset ADI recommended settings for digitizer */
+ /* "ADV7604 Register Settings Recommendations (rev. 2.5, June 2010)" p. 17. */
+ { ADV7604_REG(ADV7604_PAGE_AFE, 0x12), 0xfb }, /* ADC noise shaping filter controls */
+ { ADV7604_REG(ADV7604_PAGE_AFE, 0x0c), 0x0d }, /* CP core gain controls */
+
+ { ADV7604_REG_SEQ_TERM, 0 },
+};
+
+static const struct adv7604_reg_seq adv7611_recommended_settings_hdmi[] = {
+ { ADV7604_REG(ADV7604_PAGE_CP, 0x6c), 0x00 },
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x6f), 0x0c },
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x87), 0x70 },
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x57), 0xda },
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x58), 0x01 },
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x03), 0x98 },
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x4c), 0x44 },
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x8d), 0x04 },
+ { ADV7604_REG(ADV7604_PAGE_HDMI, 0x8e), 0x1e },
+
+ { ADV7604_REG_SEQ_TERM, 0 },
+};
+
+static const struct adv7604_chip_info adv7604_chip_info[] = {
+ [ADV7604] = {
+ .type = ADV7604,
+ .has_afe = true,
+ .max_port = ADV7604_PAD_VGA_COMP,
+ .num_dv_ports = 4,
+ .edid_enable_reg = 0x77,
+ .edid_status_reg = 0x7d,
+ .lcf_reg = 0xb3,
+ .tdms_lock_mask = 0xe0,
+ .cable_det_mask = 0x1e,
+ .fmt_change_digital_mask = 0xc1,
+ .formats = adv7604_formats,
+ .nformats = ARRAY_SIZE(adv7604_formats),
+ .set_termination = adv7604_set_termination,
+ .setup_irqs = adv7604_setup_irqs,
+ .read_hdmi_pixelclock = adv7604_read_hdmi_pixelclock,
+ .read_cable_det = adv7604_read_cable_det,
+ .recommended_settings = {
+ [0] = adv7604_recommended_settings_afe,
+ [1] = adv7604_recommended_settings_hdmi,
+ },
+ .num_recommended_settings = {
+ [0] = ARRAY_SIZE(adv7604_recommended_settings_afe),
+ [1] = ARRAY_SIZE(adv7604_recommended_settings_hdmi),
+ },
+ .page_mask = BIT(ADV7604_PAGE_IO) | BIT(ADV7604_PAGE_AVLINK) |
+ BIT(ADV7604_PAGE_CEC) | BIT(ADV7604_PAGE_INFOFRAME) |
+ BIT(ADV7604_PAGE_ESDP) | BIT(ADV7604_PAGE_DPP) |
+ BIT(ADV7604_PAGE_AFE) | BIT(ADV7604_PAGE_REP) |
+ BIT(ADV7604_PAGE_EDID) | BIT(ADV7604_PAGE_HDMI) |
+ BIT(ADV7604_PAGE_TEST) | BIT(ADV7604_PAGE_CP) |
+ BIT(ADV7604_PAGE_VDP),
+ },
+ [ADV7611] = {
+ .type = ADV7611,
+ .has_afe = false,
+ .max_port = ADV7604_PAD_HDMI_PORT_A,
+ .num_dv_ports = 1,
+ .edid_enable_reg = 0x74,
+ .edid_status_reg = 0x76,
+ .lcf_reg = 0xa3,
+ .tdms_lock_mask = 0x43,
+ .cable_det_mask = 0x01,
+ .fmt_change_digital_mask = 0x03,
+ .formats = adv7611_formats,
+ .nformats = ARRAY_SIZE(adv7611_formats),
+ .set_termination = adv7611_set_termination,
+ .setup_irqs = adv7611_setup_irqs,
+ .read_hdmi_pixelclock = adv7611_read_hdmi_pixelclock,
+ .read_cable_det = adv7611_read_cable_det,
+ .recommended_settings = {
+ [1] = adv7611_recommended_settings_hdmi,
+ },
+ .num_recommended_settings = {
+ [1] = ARRAY_SIZE(adv7611_recommended_settings_hdmi),
+ },
+ .page_mask = BIT(ADV7604_PAGE_IO) | BIT(ADV7604_PAGE_CEC) |
+ BIT(ADV7604_PAGE_INFOFRAME) | BIT(ADV7604_PAGE_AFE) |
+ BIT(ADV7604_PAGE_REP) | BIT(ADV7604_PAGE_EDID) |
+ BIT(ADV7604_PAGE_HDMI) | BIT(ADV7604_PAGE_CP),
+ },
+};
+
+static struct i2c_device_id adv7604_i2c_id[] = {
+ { "adv7604", (kernel_ulong_t)&adv7604_chip_info[ADV7604] },
+ { "adv7611", (kernel_ulong_t)&adv7604_chip_info[ADV7611] },
+ { }
+};
+MODULE_DEVICE_TABLE(i2c, adv7604_i2c_id);
+
+static struct of_device_id adv7604_of_id[] __maybe_unused = {
+ { .compatible = "adi,adv7611", .data = &adv7604_chip_info[ADV7611] },
+ { }
+};
+MODULE_DEVICE_TABLE(of, adv7604_of_id);
+
+static int adv7604_parse_dt(struct adv7604_state *state)
+{
+ struct v4l2_of_endpoint bus_cfg;
+ struct device_node *endpoint;
+ struct device_node *np;
+ unsigned int flags;
+
+ np = state->i2c_clients[ADV7604_PAGE_IO]->dev.of_node;
+
+ /* Parse the endpoint. */
+ endpoint = of_graph_get_next_endpoint(np, NULL);
+ if (!endpoint)
+ return -EINVAL;
+
+ v4l2_of_parse_endpoint(endpoint, &bus_cfg);
+ of_node_put(endpoint);
+
+ flags = bus_cfg.bus.parallel.flags;
+
+ if (flags & V4L2_MBUS_HSYNC_ACTIVE_HIGH)
+ state->pdata.inv_hs_pol = 1;
+
+ if (flags & V4L2_MBUS_VSYNC_ACTIVE_HIGH)
+ state->pdata.inv_vs_pol = 1;
+
+ if (flags & V4L2_MBUS_PCLK_SAMPLE_RISING)
+ state->pdata.inv_llc_pol = 1;
+
+ if (bus_cfg.bus_type == V4L2_MBUS_BT656) {
+ state->pdata.insert_av_codes = 1;
+ state->pdata.op_656_range = 1;
+ }
+
+ /* Disable the interrupt for now as no DT-based board uses it. */
+ state->pdata.int1_config = ADV7604_INT1_CONFIG_DISABLED;
+
+ /* Use the default I2C addresses. */
+ state->pdata.i2c_addresses[ADV7604_PAGE_AVLINK] = 0x42;
+ state->pdata.i2c_addresses[ADV7604_PAGE_CEC] = 0x40;
+ state->pdata.i2c_addresses[ADV7604_PAGE_INFOFRAME] = 0x3e;
+ state->pdata.i2c_addresses[ADV7604_PAGE_ESDP] = 0x38;
+ state->pdata.i2c_addresses[ADV7604_PAGE_DPP] = 0x3c;
+ state->pdata.i2c_addresses[ADV7604_PAGE_AFE] = 0x26;
+ state->pdata.i2c_addresses[ADV7604_PAGE_REP] = 0x32;
+ state->pdata.i2c_addresses[ADV7604_PAGE_EDID] = 0x36;
+ state->pdata.i2c_addresses[ADV7604_PAGE_HDMI] = 0x34;
+ state->pdata.i2c_addresses[ADV7604_PAGE_TEST] = 0x30;
+ state->pdata.i2c_addresses[ADV7604_PAGE_CP] = 0x22;
+ state->pdata.i2c_addresses[ADV7604_PAGE_VDP] = 0x24;
+
+ /* Hardcode the remaining platform data fields. */
+ state->pdata.disable_pwrdnb = 0;
+ state->pdata.disable_cable_det_rst = 0;
+ state->pdata.default_input = -1;
+ state->pdata.blank_data = 1;
+ state->pdata.alt_data_sat = 1;
+ state->pdata.op_format_mode_sel = ADV7604_OP_FORMAT_MODE0;
+ state->pdata.bus_order = ADV7604_BUS_ORDER_RGB;
+
+ return 0;
+}
+
static int adv7604_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
static const struct v4l2_dv_timings cea640x480 =
V4L2_DV_BT_CEA_640X480P59_94;
struct adv7604_state *state;
- struct adv7604_platform_data *pdata = client->dev.platform_data;
struct v4l2_ctrl_handler *hdl;
struct v4l2_subdev *sd;
+ unsigned int i;
+ u16 val;
int err;
/* Check if the adapter supports the needed features */
@@ -2223,32 +2764,80 @@
return -ENOMEM;
}
+ state->i2c_clients[ADV7604_PAGE_IO] = client;
+
/* initialize variables */
state->restart_stdi_once = true;
state->selected_input = ~0;
- /* platform data */
- if (!pdata) {
+ if (IS_ENABLED(CONFIG_OF) && client->dev.of_node) {
+ const struct of_device_id *oid;
+
+ oid = of_match_node(adv7604_of_id, client->dev.of_node);
+ state->info = oid->data;
+
+ err = adv7604_parse_dt(state);
+ if (err < 0) {
+ v4l_err(client, "DT parsing error\n");
+ return err;
+ }
+ } else if (client->dev.platform_data) {
+ struct adv7604_platform_data *pdata = client->dev.platform_data;
+
+ state->info = (const struct adv7604_chip_info *)id->driver_data;
+ state->pdata = *pdata;
+ } else {
v4l_err(client, "No platform data!\n");
return -ENODEV;
}
- state->pdata = *pdata;
+
+ /* Request GPIOs. */
+ for (i = 0; i < state->info->num_dv_ports; ++i) {
+ state->hpd_gpio[i] =
+ devm_gpiod_get_index(&client->dev, "hpd", i);
+ if (IS_ERR(state->hpd_gpio[i]))
+ continue;
+
+ gpiod_direction_output(state->hpd_gpio[i], 0);
+
+ v4l_info(client, "Handling HPD %u GPIO\n", i);
+ }
+
state->timings = cea640x480;
+ state->format = adv7604_format_info(state, V4L2_MBUS_FMT_YUYV8_2X8);
sd = &state->sd;
v4l2_i2c_subdev_init(sd, client, &adv7604_ops);
+ snprintf(sd->name, sizeof(sd->name), "%s %d-%04x",
+ id->name, i2c_adapter_id(client->adapter),
+ client->addr);
sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
- /* i2c access to adv7604? */
- if (adv_smbus_read_byte_data_check(client, 0xfb, false) != 0x68) {
- v4l2_info(sd, "not an adv7604 on address 0x%x\n",
- client->addr << 1);
- return -ENODEV;
+ /*
+ * Verify that the chip is present. On ADV7604 the RD_INFO register only
+ * identifies the revision, while on ADV7611 it identifies the model as
+ * well. Use the HDMI slave address on ADV7604 and RD_INFO on ADV7611.
+ */
+ if (state->info->type == ADV7604) {
+ val = adv_smbus_read_byte_data_check(client, 0xfb, false);
+ if (val != 0x68) {
+ v4l2_info(sd, "not an adv7604 on address 0x%x\n",
+ client->addr << 1);
+ return -ENODEV;
+ }
+ } else {
+ val = (adv_smbus_read_byte_data_check(client, 0xea, false) << 8)
+ | (adv_smbus_read_byte_data_check(client, 0xeb, false) << 0);
+ if (val != 0x2051) {
+ v4l2_info(sd, "not an adv7611 on address 0x%x\n",
+ client->addr << 1);
+ return -ENODEV;
+ }
}
/* control handlers */
hdl = &state->hdl;
- v4l2_ctrl_handler_init(hdl, 9);
+ v4l2_ctrl_handler_init(hdl, adv7604_has_afe(state) ? 9 : 8);
v4l2_ctrl_new_std(hdl, &adv7604_ctrl_ops,
V4L2_CID_BRIGHTNESS, -128, 127, 1, 0);
@@ -2261,15 +2850,17 @@
/* private controls */
state->detect_tx_5v_ctrl = v4l2_ctrl_new_std(hdl, NULL,
- V4L2_CID_DV_RX_POWER_PRESENT, 0, 0x0f, 0, 0);
+ V4L2_CID_DV_RX_POWER_PRESENT, 0,
+ (1 << state->info->num_dv_ports) - 1, 0, 0);
state->rgb_quantization_range_ctrl =
v4l2_ctrl_new_std_menu(hdl, &adv7604_ctrl_ops,
V4L2_CID_DV_RX_RGB_RANGE, V4L2_DV_RGB_RANGE_FULL,
0, V4L2_DV_RGB_RANGE_AUTO);
/* custom controls */
- state->analog_sampling_phase_ctrl =
- v4l2_ctrl_new_custom(hdl, &adv7604_ctrl_analog_sampling_phase, NULL);
+ if (adv7604_has_afe(state))
+ state->analog_sampling_phase_ctrl =
+ v4l2_ctrl_new_custom(hdl, &adv7604_ctrl_analog_sampling_phase, NULL);
state->free_run_color_manual_ctrl =
v4l2_ctrl_new_custom(hdl, &adv7604_ctrl_free_run_color_manual, NULL);
state->free_run_color_ctrl =
@@ -2282,7 +2873,8 @@
}
state->detect_tx_5v_ctrl->is_private = true;
state->rgb_quantization_range_ctrl->is_private = true;
- state->analog_sampling_phase_ctrl->is_private = true;
+ if (adv7604_has_afe(state))
+ state->analog_sampling_phase_ctrl->is_private = true;
state->free_run_color_manual_ctrl->is_private = true;
state->free_run_color_ctrl->is_private = true;
@@ -2291,25 +2883,18 @@
goto err_hdl;
}
- state->i2c_avlink = adv7604_dummy_client(sd, pdata->i2c_avlink, 0xf3);
- state->i2c_cec = adv7604_dummy_client(sd, pdata->i2c_cec, 0xf4);
- state->i2c_infoframe = adv7604_dummy_client(sd, pdata->i2c_infoframe, 0xf5);
- state->i2c_esdp = adv7604_dummy_client(sd, pdata->i2c_esdp, 0xf6);
- state->i2c_dpp = adv7604_dummy_client(sd, pdata->i2c_dpp, 0xf7);
- state->i2c_afe = adv7604_dummy_client(sd, pdata->i2c_afe, 0xf8);
- state->i2c_repeater = adv7604_dummy_client(sd, pdata->i2c_repeater, 0xf9);
- state->i2c_edid = adv7604_dummy_client(sd, pdata->i2c_edid, 0xfa);
- state->i2c_hdmi = adv7604_dummy_client(sd, pdata->i2c_hdmi, 0xfb);
- state->i2c_test = adv7604_dummy_client(sd, pdata->i2c_test, 0xfc);
- state->i2c_cp = adv7604_dummy_client(sd, pdata->i2c_cp, 0xfd);
- state->i2c_vdp = adv7604_dummy_client(sd, pdata->i2c_vdp, 0xfe);
- if (!state->i2c_avlink || !state->i2c_cec || !state->i2c_infoframe ||
- !state->i2c_esdp || !state->i2c_dpp || !state->i2c_afe ||
- !state->i2c_repeater || !state->i2c_edid || !state->i2c_hdmi ||
- !state->i2c_test || !state->i2c_cp || !state->i2c_vdp) {
- err = -ENOMEM;
- v4l2_err(sd, "failed to create all i2c clients\n");
- goto err_i2c;
+ for (i = 1; i < ADV7604_PAGE_MAX; ++i) {
+ if (!(BIT(i) & state->info->page_mask))
+ continue;
+
+ state->i2c_clients[i] =
+ adv7604_dummy_client(sd, state->pdata.i2c_addresses[i],
+ 0xf2 + i);
+ if (state->i2c_clients[i] == NULL) {
+ err = -ENOMEM;
+ v4l2_err(sd, "failed to create i2c client %u\n", i);
+ goto err_i2c;
+ }
}
/* work queues */
@@ -2323,8 +2908,14 @@
INIT_DELAYED_WORK(&state->delayed_work_enable_hotplug,
adv7604_delayed_work_enable_hotplug);
- state->pad.flags = MEDIA_PAD_FL_SOURCE;
- err = media_entity_init(&sd->entity, 1, &state->pad, 0);
+ state->source_pad = state->info->num_dv_ports
+ + (state->info->has_afe ? 2 : 0);
+ for (i = 0; i < state->source_pad; ++i)
+ state->pads[i].flags = MEDIA_PAD_FL_SINK;
+ state->pads[state->source_pad].flags = MEDIA_PAD_FL_SOURCE;
+
+ err = media_entity_init(&sd->entity, state->source_pad + 1,
+ state->pads, 0);
if (err)
goto err_work_queues;
@@ -2333,6 +2924,11 @@
goto err_entity;
v4l2_info(sd, "%s found @ 0x%x (%s)\n", client->name,
client->addr << 1, client->adapter->name);
+
+ err = v4l2_async_register_subdev(sd);
+ if (err)
+ goto err_entity;
+
return 0;
err_entity:
@@ -2356,6 +2952,7 @@
cancel_delayed_work(&state->delayed_work_enable_hotplug);
destroy_workqueue(state->work_queues);
+ v4l2_async_unregister_subdev(sd);
v4l2_device_unregister_subdev(sd);
media_entity_cleanup(&sd->entity);
adv7604_unregister_clients(to_state(sd));
@@ -2365,20 +2962,15 @@
/* ----------------------------------------------------------------------- */
-static struct i2c_device_id adv7604_id[] = {
- { "adv7604", 0 },
- { }
-};
-MODULE_DEVICE_TABLE(i2c, adv7604_id);
-
static struct i2c_driver adv7604_driver = {
.driver = {
.owner = THIS_MODULE,
.name = "adv7604",
+ .of_match_table = of_match_ptr(adv7604_of_id),
},
.probe = adv7604_probe,
.remove = adv7604_remove,
- .id_table = adv7604_id,
+ .id_table = adv7604_i2c_id,
};
module_i2c_driver(adv7604_driver);
diff --git a/drivers/media/platform/vsp1/Makefile b/drivers/media/platform/vsp1/Makefile
index 151cecd..6a93f92 100644
--- a/drivers/media/platform/vsp1/Makefile
+++ b/drivers/media/platform/vsp1/Makefile
@@ -1,6 +1,6 @@
vsp1-y := vsp1_drv.o vsp1_entity.o vsp1_video.o
vsp1-y += vsp1_rpf.o vsp1_rwpf.o vsp1_wpf.o
vsp1-y += vsp1_hsit.o vsp1_lif.o vsp1_lut.o
-vsp1-y += vsp1_sru.o vsp1_uds.o
+vsp1-y += vsp1_bru.o vsp1_sru.o vsp1_uds.o
obj-$(CONFIG_VIDEO_RENESAS_VSP1) += vsp1.o
diff --git a/drivers/media/platform/vsp1/vsp1.h b/drivers/media/platform/vsp1/vsp1.h
index 0313210..6ca2cf2 100644
--- a/drivers/media/platform/vsp1/vsp1.h
+++ b/drivers/media/platform/vsp1/vsp1.h
@@ -28,6 +28,7 @@
struct device;
struct vsp1_platform_data;
+struct vsp1_bru;
struct vsp1_hsit;
struct vsp1_lif;
struct vsp1_lut;
@@ -45,11 +46,11 @@
void __iomem *mmio;
struct clk *clock;
- struct clk *rt_clock;
struct mutex lock;
int ref_count;
+ struct vsp1_bru *bru;
struct vsp1_hsit *hsi;
struct vsp1_hsit *hst;
struct vsp1_lif *lif;
diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
new file mode 100644
index 0000000..f806954
--- /dev/null
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -0,0 +1,395 @@
+/*
+ * vsp1_bru.c -- R-Car VSP1 Blend ROP Unit
+ *
+ * Copyright (C) 2013 Renesas Corporation
+ *
+ * Contact: Laurent Pinchart (laurent.pinchart@ideasonboard.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include <linux/device.h>
+#include <linux/gfp.h>
+
+#include <media/v4l2-subdev.h>
+
+#include "vsp1.h"
+#include "vsp1_bru.h"
+
+#define BRU_MIN_SIZE 4U
+#define BRU_MAX_SIZE 8190U
+
+/* -----------------------------------------------------------------------------
+ * Device Access
+ */
+
+static inline u32 vsp1_bru_read(struct vsp1_bru *bru, u32 reg)
+{
+ return vsp1_read(bru->entity.vsp1, reg);
+}
+
+static inline void vsp1_bru_write(struct vsp1_bru *bru, u32 reg, u32 data)
+{
+ vsp1_write(bru->entity.vsp1, reg, data);
+}
+
+/* -----------------------------------------------------------------------------
+ * V4L2 Subdevice Core Operations
+ */
+
+static bool bru_is_input_enabled(struct vsp1_bru *bru, unsigned int input)
+{
+ return media_entity_remote_pad(&bru->entity.pads[input]) != NULL;
+}
+
+static int bru_s_stream(struct v4l2_subdev *subdev, int enable)
+{
+ struct vsp1_bru *bru = to_bru(subdev);
+ struct v4l2_mbus_framefmt *format;
+ unsigned int i;
+
+ if (!enable)
+ return 0;
+
+ format = &bru->entity.formats[BRU_PAD_SOURCE];
+
+ /* The hardware is extremely flexible but we have no userspace API to
+ * expose all the parameters, nor is it clear whether we would have use
+ * cases for all the supported modes. Let's just harcode the parameters
+ * to sane default values for now.
+ */
+
+ /* Disable both color data normalization and dithering. */
+ vsp1_bru_write(bru, VI6_BRU_INCTRL, 0);
+
+ /* Set the background position to cover the whole output image and
+ * set its color to opaque black.
+ */
+ vsp1_bru_write(bru, VI6_BRU_VIRRPF_SIZE,
+ (format->width << VI6_BRU_VIRRPF_SIZE_HSIZE_SHIFT) |
+ (format->height << VI6_BRU_VIRRPF_SIZE_VSIZE_SHIFT));
+ vsp1_bru_write(bru, VI6_BRU_VIRRPF_LOC, 0);
+ vsp1_bru_write(bru, VI6_BRU_VIRRPF_COL,
+ 0xff << VI6_BRU_VIRRPF_COL_A_SHIFT);
+
+ /* Route BRU input 1 as SRC input to the ROP unit and configure the ROP
+ * unit with a NOP operation to make BRU input 1 available as the
+ * Blend/ROP unit B SRC input.
+ */
+ vsp1_bru_write(bru, VI6_BRU_ROP, VI6_BRU_ROP_DSTSEL_BRUIN(1) |
+ VI6_BRU_ROP_CROP(VI6_ROP_NOP) |
+ VI6_BRU_ROP_AROP(VI6_ROP_NOP));
+
+ for (i = 0; i < 4; ++i) {
+ u32 ctrl = 0;
+
+ /* Configure all Blend/ROP units corresponding to an enabled BRU
+ * input for alpha blending. Blend/ROP units corresponding to
+ * disabled BRU inputs are used in ROP NOP mode to ignore the
+ * SRC input.
+ */
+ if (bru_is_input_enabled(bru, i))
+ ctrl |= VI6_BRU_CTRL_RBC;
+ else
+ ctrl |= VI6_BRU_CTRL_CROP(VI6_ROP_NOP)
+ | VI6_BRU_CTRL_AROP(VI6_ROP_NOP);
+
+ /* Select the virtual RPF as the Blend/ROP unit A DST input to
+ * serve as a background color.
+ */
+ if (i == 0)
+ ctrl |= VI6_BRU_CTRL_DSTSEL_VRPF;
+
+ /* Route BRU inputs 0 to 3 as SRC inputs to Blend/ROP units A to
+ * D in that order. The Blend/ROP unit B SRC is hardwired to the
+ * ROP unit output, the corresponding register bits must be set
+ * to 0.
+ */
+ if (i != 1)
+ ctrl |= VI6_BRU_CTRL_SRCSEL_BRUIN(i);
+
+ vsp1_bru_write(bru, VI6_BRU_CTRL(i), ctrl);
+
+ /* Harcode the blending formula to
+ *
+ * DSTc = DSTc * (1 - SRCa) + SRCc * SRCa
+ * DSTa = DSTa * (1 - SRCa) + SRCa
+ */
+ vsp1_bru_write(bru, VI6_BRU_BLD(i),
+ VI6_BRU_BLD_CCMDX_255_SRC_A |
+ VI6_BRU_BLD_CCMDY_SRC_A |
+ VI6_BRU_BLD_ACMDX_255_SRC_A |
+ VI6_BRU_BLD_ACMDY_COEFY |
+ (0xff << VI6_BRU_BLD_COEFY_SHIFT));
+ }
+
+ return 0;
+}
+
+/* -----------------------------------------------------------------------------
+ * V4L2 Subdevice Pad Operations
+ */
+
+/*
+ * The BRU can't perform format conversion, all sink and source formats must be
+ * identical. We pick the format on the first sink pad (pad 0) and propagate it
+ * to all other pads.
+ */
+
+static int bru_enum_mbus_code(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_fh *fh,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ static const unsigned int codes[] = {
+ V4L2_MBUS_FMT_ARGB8888_1X32,
+ V4L2_MBUS_FMT_AYUV8_1X32,
+ };
+ struct v4l2_mbus_framefmt *format;
+
+ if (code->pad == BRU_PAD_SINK(0)) {
+ if (code->index >= ARRAY_SIZE(codes))
+ return -EINVAL;
+
+ code->code = codes[code->index];
+ } else {
+ if (code->index)
+ return -EINVAL;
+
+ format = v4l2_subdev_get_try_format(fh, BRU_PAD_SINK(0));
+ code->code = format->code;
+ }
+
+ return 0;
+}
+
+static int bru_enum_frame_size(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_fh *fh,
+ struct v4l2_subdev_frame_size_enum *fse)
+{
+ if (fse->index)
+ return -EINVAL;
+
+ if (fse->code != V4L2_MBUS_FMT_ARGB8888_1X32 &&
+ fse->code != V4L2_MBUS_FMT_AYUV8_1X32)
+ return -EINVAL;
+
+ fse->min_width = BRU_MIN_SIZE;
+ fse->max_width = BRU_MAX_SIZE;
+ fse->min_height = BRU_MIN_SIZE;
+ fse->max_height = BRU_MAX_SIZE;
+
+ return 0;
+}
+
+static struct v4l2_rect *bru_get_compose(struct vsp1_bru *bru,
+ struct v4l2_subdev_fh *fh,
+ unsigned int pad, u32 which)
+{
+ switch (which) {
+ case V4L2_SUBDEV_FORMAT_TRY:
+ return v4l2_subdev_get_try_crop(fh, pad);
+ case V4L2_SUBDEV_FORMAT_ACTIVE:
+ return &bru->compose[pad];
+ default:
+ return NULL;
+ }
+}
+
+static int bru_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_fh *fh,
+ struct v4l2_subdev_format *fmt)
+{
+ struct vsp1_bru *bru = to_bru(subdev);
+
+ fmt->format = *vsp1_entity_get_pad_format(&bru->entity, fh, fmt->pad,
+ fmt->which);
+
+ return 0;
+}
+
+static void bru_try_format(struct vsp1_bru *bru, struct v4l2_subdev_fh *fh,
+ unsigned int pad, struct v4l2_mbus_framefmt *fmt,
+ enum v4l2_subdev_format_whence which)
+{
+ struct v4l2_mbus_framefmt *format;
+
+ switch (pad) {
+ case BRU_PAD_SINK(0):
+ /* Default to YUV if the requested format is not supported. */
+ if (fmt->code != V4L2_MBUS_FMT_ARGB8888_1X32 &&
+ fmt->code != V4L2_MBUS_FMT_AYUV8_1X32)
+ fmt->code = V4L2_MBUS_FMT_AYUV8_1X32;
+ break;
+
+ default:
+ /* The BRU can't perform format conversion. */
+ format = vsp1_entity_get_pad_format(&bru->entity, fh,
+ BRU_PAD_SINK(0), which);
+ fmt->code = format->code;
+ break;
+ }
+
+ fmt->width = clamp(fmt->width, BRU_MIN_SIZE, BRU_MAX_SIZE);
+ fmt->height = clamp(fmt->height, BRU_MIN_SIZE, BRU_MAX_SIZE);
+ fmt->field = V4L2_FIELD_NONE;
+ fmt->colorspace = V4L2_COLORSPACE_SRGB;
+}
+
+static int bru_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_fh *fh,
+ struct v4l2_subdev_format *fmt)
+{
+ struct vsp1_bru *bru = to_bru(subdev);
+ struct v4l2_mbus_framefmt *format;
+
+ bru_try_format(bru, fh, fmt->pad, &fmt->format, fmt->which);
+
+ format = vsp1_entity_get_pad_format(&bru->entity, fh, fmt->pad,
+ fmt->which);
+ *format = fmt->format;
+
+ /* Reset the compose rectangle */
+ if (fmt->pad != BRU_PAD_SOURCE) {
+ struct v4l2_rect *compose;
+
+ compose = bru_get_compose(bru, fh, fmt->pad, fmt->which);
+ compose->left = 0;
+ compose->top = 0;
+ compose->width = format->width;
+ compose->height = format->height;
+ }
+
+ /* Propagate the format code to all pads */
+ if (fmt->pad == BRU_PAD_SINK(0)) {
+ unsigned int i;
+
+ for (i = 0; i <= BRU_PAD_SOURCE; ++i) {
+ format = vsp1_entity_get_pad_format(&bru->entity, fh,
+ i, fmt->which);
+ format->code = fmt->format.code;
+ }
+ }
+
+ return 0;
+}
+
+static int bru_get_selection(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_fh *fh,
+ struct v4l2_subdev_selection *sel)
+{
+ struct vsp1_bru *bru = to_bru(subdev);
+
+ if (sel->pad == BRU_PAD_SOURCE)
+ return -EINVAL;
+
+ switch (sel->target) {
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ sel->r.left = 0;
+ sel->r.top = 0;
+ sel->r.width = BRU_MAX_SIZE;
+ sel->r.height = BRU_MAX_SIZE;
+ return 0;
+
+ case V4L2_SEL_TGT_COMPOSE:
+ sel->r = *bru_get_compose(bru, fh, sel->pad, sel->which);
+ return 0;
+
+ default:
+ return -EINVAL;
+ }
+}
+
+static int bru_set_selection(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_fh *fh,
+ struct v4l2_subdev_selection *sel)
+{
+ struct vsp1_bru *bru = to_bru(subdev);
+ struct v4l2_mbus_framefmt *format;
+ struct v4l2_rect *compose;
+
+ if (sel->pad == BRU_PAD_SOURCE)
+ return -EINVAL;
+
+ if (sel->target != V4L2_SEL_TGT_COMPOSE)
+ return -EINVAL;
+
+ /* The compose rectangle top left corner must be inside the output
+ * frame.
+ */
+ format = vsp1_entity_get_pad_format(&bru->entity, fh, BRU_PAD_SOURCE,
+ sel->which);
+ sel->r.left = clamp_t(unsigned int, sel->r.left, 0, format->width - 1);
+ sel->r.top = clamp_t(unsigned int, sel->r.top, 0, format->height - 1);
+
+ /* Scaling isn't supported, the compose rectangle size must be identical
+ * to the sink format size.
+ */
+ format = vsp1_entity_get_pad_format(&bru->entity, fh, sel->pad,
+ sel->which);
+ sel->r.width = format->width;
+ sel->r.height = format->height;
+
+ compose = bru_get_compose(bru, fh, sel->pad, sel->which);
+ *compose = sel->r;
+
+ return 0;
+}
+
+/* -----------------------------------------------------------------------------
+ * V4L2 Subdevice Operations
+ */
+
+static struct v4l2_subdev_video_ops bru_video_ops = {
+ .s_stream = bru_s_stream,
+};
+
+static struct v4l2_subdev_pad_ops bru_pad_ops = {
+ .enum_mbus_code = bru_enum_mbus_code,
+ .enum_frame_size = bru_enum_frame_size,
+ .get_fmt = bru_get_format,
+ .set_fmt = bru_set_format,
+ .get_selection = bru_get_selection,
+ .set_selection = bru_set_selection,
+};
+
+static struct v4l2_subdev_ops bru_ops = {
+ .video = &bru_video_ops,
+ .pad = &bru_pad_ops,
+};
+
+/* -----------------------------------------------------------------------------
+ * Initialization and Cleanup
+ */
+
+struct vsp1_bru *vsp1_bru_create(struct vsp1_device *vsp1)
+{
+ struct v4l2_subdev *subdev;
+ struct vsp1_bru *bru;
+ int ret;
+
+ bru = devm_kzalloc(vsp1->dev, sizeof(*bru), GFP_KERNEL);
+ if (bru == NULL)
+ return ERR_PTR(-ENOMEM);
+
+ bru->entity.type = VSP1_ENTITY_BRU;
+
+ ret = vsp1_entity_init(vsp1, &bru->entity, 5);
+ if (ret < 0)
+ return ERR_PTR(ret);
+
+ /* Initialize the V4L2 subdev. */
+ subdev = &bru->entity.subdev;
+ v4l2_subdev_init(subdev, &bru_ops);
+
+ subdev->entity.ops = &vsp1_media_ops;
+ subdev->internal_ops = &vsp1_subdev_internal_ops;
+ snprintf(subdev->name, sizeof(subdev->name), "%s bru",
+ dev_name(vsp1->dev));
+ v4l2_set_subdevdata(subdev, bru);
+ subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+
+ vsp1_entity_init_formats(subdev, NULL);
+
+ return bru;
+}
diff --git a/drivers/media/platform/vsp1/vsp1_bru.h b/drivers/media/platform/vsp1/vsp1_bru.h
new file mode 100644
index 0000000..3706270
--- /dev/null
+++ b/drivers/media/platform/vsp1/vsp1_bru.h
@@ -0,0 +1,39 @@
+/*
+ * vsp1_bru.h -- R-Car VSP1 Blend ROP Unit
+ *
+ * Copyright (C) 2013 Renesas Corporation
+ *
+ * Contact: Laurent Pinchart (laurent.pinchart@ideasonboard.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#ifndef __VSP1_BRU_H__
+#define __VSP1_BRU_H__
+
+#include <media/media-entity.h>
+#include <media/v4l2-subdev.h>
+
+#include "vsp1_entity.h"
+
+struct vsp1_device;
+
+#define BRU_PAD_SINK(n) (n)
+#define BRU_PAD_SOURCE 4
+
+struct vsp1_bru {
+ struct vsp1_entity entity;
+
+ struct v4l2_rect compose[4];
+};
+
+static inline struct vsp1_bru *to_bru(struct v4l2_subdev *subdev)
+{
+ return container_of(subdev, struct vsp1_bru, entity.subdev);
+}
+
+struct vsp1_bru *vsp1_bru_create(struct vsp1_device *vsp1);
+
+#endif /* __VSP1_BRU_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_drv.c b/drivers/media/platform/vsp1/vsp1_drv.c
index 2f74f0e..c69ee06 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -16,10 +16,12 @@
#include <linux/device.h>
#include <linux/interrupt.h>
#include <linux/module.h>
+#include <linux/of.h>
#include <linux/platform_device.h>
#include <linux/videodev2.h>
#include "vsp1.h"
+#include "vsp1_bru.h"
#include "vsp1_hsit.h"
#include "vsp1_lif.h"
#include "vsp1_lut.h"
@@ -155,6 +157,14 @@
}
/* Instantiate all the entities. */
+ vsp1->bru = vsp1_bru_create(vsp1);
+ if (IS_ERR(vsp1->bru)) {
+ ret = PTR_ERR(vsp1->bru);
+ goto done;
+ }
+
+ list_add_tail(&vsp1->bru->entity.list_dev, &vsp1->entities);
+
vsp1->hsi = vsp1_hsit_create(vsp1, true);
if (IS_ERR(vsp1->hsi)) {
ret = PTR_ERR(vsp1->hsi);
@@ -329,33 +339,6 @@
return 0;
}
-static int vsp1_clocks_enable(struct vsp1_device *vsp1)
-{
- int ret;
-
- ret = clk_prepare_enable(vsp1->clock);
- if (ret < 0)
- return ret;
-
- if (IS_ERR(vsp1->rt_clock))
- return 0;
-
- ret = clk_prepare_enable(vsp1->rt_clock);
- if (ret < 0) {
- clk_disable_unprepare(vsp1->clock);
- return ret;
- }
-
- return 0;
-}
-
-static void vsp1_clocks_disable(struct vsp1_device *vsp1)
-{
- if (!IS_ERR(vsp1->rt_clock))
- clk_disable_unprepare(vsp1->rt_clock);
- clk_disable_unprepare(vsp1->clock);
-}
-
/*
* vsp1_device_get - Acquire the VSP1 device
*
@@ -373,7 +356,7 @@
if (vsp1->ref_count > 0)
goto done;
- ret = vsp1_clocks_enable(vsp1);
+ ret = clk_prepare_enable(vsp1->clock);
if (ret < 0) {
__vsp1 = NULL;
goto done;
@@ -381,7 +364,7 @@
ret = vsp1_device_init(vsp1);
if (ret < 0) {
- vsp1_clocks_disable(vsp1);
+ clk_disable_unprepare(vsp1->clock);
__vsp1 = NULL;
goto done;
}
@@ -405,7 +388,7 @@
mutex_lock(&vsp1->lock);
if (--vsp1->ref_count == 0)
- vsp1_clocks_disable(vsp1);
+ clk_disable_unprepare(vsp1->clock);
mutex_unlock(&vsp1->lock);
}
@@ -424,7 +407,7 @@
if (vsp1->ref_count == 0)
return 0;
- vsp1_clocks_disable(vsp1);
+ clk_disable_unprepare(vsp1->clock);
return 0;
}
@@ -437,7 +420,7 @@
if (vsp1->ref_count)
return 0;
- return vsp1_clocks_enable(vsp1);
+ return clk_prepare_enable(vsp1->clock);
}
#endif
@@ -449,34 +432,59 @@
* Platform Driver
*/
-static struct vsp1_platform_data *
-vsp1_get_platform_data(struct platform_device *pdev)
+static int vsp1_validate_platform_data(struct platform_device *pdev,
+ struct vsp1_platform_data *pdata)
{
- struct vsp1_platform_data *pdata = pdev->dev.platform_data;
-
if (pdata == NULL) {
dev_err(&pdev->dev, "missing platform data\n");
- return NULL;
+ return -EINVAL;
}
if (pdata->rpf_count <= 0 || pdata->rpf_count > VPS1_MAX_RPF) {
dev_err(&pdev->dev, "invalid number of RPF (%u)\n",
pdata->rpf_count);
- return NULL;
+ return -EINVAL;
}
if (pdata->uds_count <= 0 || pdata->uds_count > VPS1_MAX_UDS) {
dev_err(&pdev->dev, "invalid number of UDS (%u)\n",
pdata->uds_count);
- return NULL;
+ return -EINVAL;
}
if (pdata->wpf_count <= 0 || pdata->wpf_count > VPS1_MAX_WPF) {
dev_err(&pdev->dev, "invalid number of WPF (%u)\n",
pdata->wpf_count);
- return NULL;
+ return -EINVAL;
}
+ return 0;
+}
+
+static struct vsp1_platform_data *
+vsp1_get_platform_data(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct vsp1_platform_data *pdata;
+
+ if (!IS_ENABLED(CONFIG_OF) || np == NULL)
+ return pdev->dev.platform_data;
+
+ pdata = devm_kzalloc(&pdev->dev, sizeof(*pdata), GFP_KERNEL);
+ if (pdata == NULL)
+ return NULL;
+
+ if (of_property_read_bool(np, "renesas,has-lif"))
+ pdata->features |= VSP1_HAS_LIF;
+ if (of_property_read_bool(np, "renesas,has-lut"))
+ pdata->features |= VSP1_HAS_LUT;
+ if (of_property_read_bool(np, "renesas,has-sru"))
+ pdata->features |= VSP1_HAS_SRU;
+
+ of_property_read_u32(np, "renesas,#rpf", &pdata->rpf_count);
+ of_property_read_u32(np, "renesas,#uds", &pdata->uds_count);
+ of_property_read_u32(np, "renesas,#wpf", &pdata->wpf_count);
+
return pdata;
}
@@ -499,6 +507,10 @@
if (vsp1->pdata == NULL)
return -ENODEV;
+ ret = vsp1_validate_platform_data(pdev, vsp1->pdata);
+ if (ret < 0)
+ return ret;
+
/* I/O, IRQ and clock resources */
io = platform_get_resource(pdev, IORESOURCE_MEM, 0);
vsp1->mmio = devm_ioremap_resource(&pdev->dev, io);
@@ -511,9 +523,6 @@
return PTR_ERR(vsp1->clock);
}
- /* The RT clock is optional */
- vsp1->rt_clock = devm_clk_get(&pdev->dev, "rt");
-
irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
if (!irq) {
dev_err(&pdev->dev, "missing IRQ\n");
@@ -548,6 +557,11 @@
return 0;
}
+static const struct of_device_id vsp1_of_match[] = {
+ { .compatible = "renesas,vsp1" },
+ { },
+};
+
static struct platform_driver vsp1_platform_driver = {
.probe = vsp1_probe,
.remove = vsp1_remove,
@@ -555,6 +569,7 @@
.owner = THIS_MODULE,
.name = "vsp1",
.pm = &vsp1_pm_ops,
+ .of_match_table = vsp1_of_match,
},
};
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 3fc9e42..4416783 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -100,8 +100,10 @@
if (source->sink)
return -EBUSY;
source->sink = remote->entity;
+ source->sink_pad = remote->index;
} else {
source->sink = NULL;
+ source->sink_pad = 0;
}
return 0;
@@ -116,42 +118,43 @@
* Initialization
*/
+static const struct vsp1_route vsp1_routes[] = {
+ { VSP1_ENTITY_BRU, 0, VI6_DPR_BRU_ROUTE,
+ { VI6_DPR_NODE_BRU_IN(0), VI6_DPR_NODE_BRU_IN(1),
+ VI6_DPR_NODE_BRU_IN(2), VI6_DPR_NODE_BRU_IN(3), } },
+ { VSP1_ENTITY_HSI, 0, VI6_DPR_HSI_ROUTE, { VI6_DPR_NODE_HSI, } },
+ { VSP1_ENTITY_HST, 0, VI6_DPR_HST_ROUTE, { VI6_DPR_NODE_HST, } },
+ { VSP1_ENTITY_LIF, 0, 0, { VI6_DPR_NODE_LIF, } },
+ { VSP1_ENTITY_LUT, 0, VI6_DPR_LUT_ROUTE, { VI6_DPR_NODE_LUT, } },
+ { VSP1_ENTITY_RPF, 0, VI6_DPR_RPF_ROUTE(0), { VI6_DPR_NODE_RPF(0), } },
+ { VSP1_ENTITY_RPF, 1, VI6_DPR_RPF_ROUTE(1), { VI6_DPR_NODE_RPF(1), } },
+ { VSP1_ENTITY_RPF, 2, VI6_DPR_RPF_ROUTE(2), { VI6_DPR_NODE_RPF(2), } },
+ { VSP1_ENTITY_RPF, 3, VI6_DPR_RPF_ROUTE(3), { VI6_DPR_NODE_RPF(3), } },
+ { VSP1_ENTITY_RPF, 4, VI6_DPR_RPF_ROUTE(4), { VI6_DPR_NODE_RPF(4), } },
+ { VSP1_ENTITY_SRU, 0, VI6_DPR_SRU_ROUTE, { VI6_DPR_NODE_SRU, } },
+ { VSP1_ENTITY_UDS, 0, VI6_DPR_UDS_ROUTE(0), { VI6_DPR_NODE_UDS(0), } },
+ { VSP1_ENTITY_UDS, 1, VI6_DPR_UDS_ROUTE(1), { VI6_DPR_NODE_UDS(1), } },
+ { VSP1_ENTITY_UDS, 2, VI6_DPR_UDS_ROUTE(2), { VI6_DPR_NODE_UDS(2), } },
+ { VSP1_ENTITY_WPF, 0, 0, { VI6_DPR_NODE_WPF(0), } },
+ { VSP1_ENTITY_WPF, 1, 0, { VI6_DPR_NODE_WPF(1), } },
+ { VSP1_ENTITY_WPF, 2, 0, { VI6_DPR_NODE_WPF(2), } },
+ { VSP1_ENTITY_WPF, 3, 0, { VI6_DPR_NODE_WPF(3), } },
+};
+
int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
unsigned int num_pads)
{
- static const struct {
- unsigned int id;
- unsigned int reg;
- } routes[] = {
- { VI6_DPR_NODE_HSI, VI6_DPR_HSI_ROUTE },
- { VI6_DPR_NODE_HST, VI6_DPR_HST_ROUTE },
- { VI6_DPR_NODE_LIF, 0 },
- { VI6_DPR_NODE_LUT, VI6_DPR_LUT_ROUTE },
- { VI6_DPR_NODE_RPF(0), VI6_DPR_RPF_ROUTE(0) },
- { VI6_DPR_NODE_RPF(1), VI6_DPR_RPF_ROUTE(1) },
- { VI6_DPR_NODE_RPF(2), VI6_DPR_RPF_ROUTE(2) },
- { VI6_DPR_NODE_RPF(3), VI6_DPR_RPF_ROUTE(3) },
- { VI6_DPR_NODE_RPF(4), VI6_DPR_RPF_ROUTE(4) },
- { VI6_DPR_NODE_SRU, VI6_DPR_SRU_ROUTE },
- { VI6_DPR_NODE_UDS(0), VI6_DPR_UDS_ROUTE(0) },
- { VI6_DPR_NODE_UDS(1), VI6_DPR_UDS_ROUTE(1) },
- { VI6_DPR_NODE_UDS(2), VI6_DPR_UDS_ROUTE(2) },
- { VI6_DPR_NODE_WPF(0), 0 },
- { VI6_DPR_NODE_WPF(1), 0 },
- { VI6_DPR_NODE_WPF(2), 0 },
- { VI6_DPR_NODE_WPF(3), 0 },
- };
-
unsigned int i;
- for (i = 0; i < ARRAY_SIZE(routes); ++i) {
- if (routes[i].id == entity->id) {
- entity->route = routes[i].reg;
+ for (i = 0; i < ARRAY_SIZE(vsp1_routes); ++i) {
+ if (vsp1_routes[i].type == entity->type &&
+ vsp1_routes[i].index == entity->index) {
+ entity->route = &vsp1_routes[i];
break;
}
}
- if (i == ARRAY_SIZE(routes))
+ if (i == ARRAY_SIZE(vsp1_routes))
return -EINVAL;
entity->vsp1 = vsp1;
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index f6fd698..7afbd8a 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -20,6 +20,7 @@
struct vsp1_device;
enum vsp1_entity_type {
+ VSP1_ENTITY_BRU,
VSP1_ENTITY_HSI,
VSP1_ENTITY_HST,
VSP1_ENTITY_LIF,
@@ -30,13 +31,31 @@
VSP1_ENTITY_WPF,
};
+/*
+ * struct vsp1_route - Entity routing configuration
+ * @type: Entity type this routing entry is associated with
+ * @index: Entity index this routing entry is associated with
+ * @reg: Output routing configuration register
+ * @inputs: Target node value for each input
+ *
+ * Each $vsp1_route entry describes routing configuration for the entity
+ * specified by the entry's @type and @index. @reg indicates the register that
+ * holds output routing configuration for the entity, and the @inputs array
+ * store the target node value for each input of the entity.
+ */
+struct vsp1_route {
+ enum vsp1_entity_type type;
+ unsigned int index;
+ unsigned int reg;
+ unsigned int inputs[4];
+};
+
struct vsp1_entity {
struct vsp1_device *vsp1;
enum vsp1_entity_type type;
unsigned int index;
- unsigned int id;
- unsigned int route;
+ const struct vsp1_route *route;
struct list_head list_dev;
struct list_head list_pipe;
@@ -45,6 +64,7 @@
unsigned int source_pad;
struct media_entity *sink;
+ unsigned int sink_pad;
struct v4l2_subdev subdev;
struct v4l2_mbus_framefmt *formats;
diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index 2854853..db2950a 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -193,13 +193,10 @@
hsit->inverse = inverse;
- if (inverse) {
+ if (inverse)
hsit->entity.type = VSP1_ENTITY_HSI;
- hsit->entity.id = VI6_DPR_NODE_HSI;
- } else {
+ else
hsit->entity.type = VSP1_ENTITY_HST;
- hsit->entity.id = VI6_DPR_NODE_HST;
- }
ret = vsp1_entity_init(vsp1, &hsit->entity, 2);
if (ret < 0)
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index 135a789..d4fb23e 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -215,7 +215,6 @@
return ERR_PTR(-ENOMEM);
lif->entity.type = VSP1_ENTITY_LIF;
- lif->entity.id = VI6_DPR_NODE_LIF;
ret = vsp1_entity_init(vsp1, &lif->entity, 2);
if (ret < 0)
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 4e9dc7c..fea36eb 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -229,7 +229,6 @@
return ERR_PTR(-ENOMEM);
lut->entity.type = VSP1_ENTITY_LUT;
- lut->entity.id = VI6_DPR_NODE_LUT;
ret = vsp1_entity_init(vsp1, &lut->entity, 2);
if (ret < 0)
diff --git a/drivers/media/platform/vsp1/vsp1_regs.h b/drivers/media/platform/vsp1/vsp1_regs.h
index 2865080..3e74b44 100644
--- a/drivers/media/platform/vsp1/vsp1_regs.h
+++ b/drivers/media/platform/vsp1/vsp1_regs.h
@@ -451,13 +451,111 @@
* BRU Control Registers
*/
+#define VI6_ROP_NOP 0
+#define VI6_ROP_AND 1
+#define VI6_ROP_AND_REV 2
+#define VI6_ROP_COPY 3
+#define VI6_ROP_AND_INV 4
+#define VI6_ROP_CLEAR 5
+#define VI6_ROP_XOR 6
+#define VI6_ROP_OR 7
+#define VI6_ROP_NOR 8
+#define VI6_ROP_EQUIV 9
+#define VI6_ROP_INVERT 10
+#define VI6_ROP_OR_REV 11
+#define VI6_ROP_COPY_INV 12
+#define VI6_ROP_OR_INV 13
+#define VI6_ROP_NAND 14
+#define VI6_ROP_SET 15
+
#define VI6_BRU_INCTRL 0x2c00
+#define VI6_BRU_INCTRL_NRM (1 << 28)
+#define VI6_BRU_INCTRL_DnON (1 << (16 + (n)))
+#define VI6_BRU_INCTRL_DITHn_OFF (0 << ((n) * 4))
+#define VI6_BRU_INCTRL_DITHn_18BPP (1 << ((n) * 4))
+#define VI6_BRU_INCTRL_DITHn_16BPP (2 << ((n) * 4))
+#define VI6_BRU_INCTRL_DITHn_15BPP (3 << ((n) * 4))
+#define VI6_BRU_INCTRL_DITHn_12BPP (4 << ((n) * 4))
+#define VI6_BRU_INCTRL_DITHn_8BPP (5 << ((n) * 4))
+#define VI6_BRU_INCTRL_DITHn_MASK (7 << ((n) * 4))
+#define VI6_BRU_INCTRL_DITHn_SHIFT ((n) * 4)
+
#define VI6_BRU_VIRRPF_SIZE 0x2c04
+#define VI6_BRU_VIRRPF_SIZE_HSIZE_MASK (0x1fff << 16)
+#define VI6_BRU_VIRRPF_SIZE_HSIZE_SHIFT 16
+#define VI6_BRU_VIRRPF_SIZE_VSIZE_MASK (0x1fff << 0)
+#define VI6_BRU_VIRRPF_SIZE_VSIZE_SHIFT 0
+
#define VI6_BRU_VIRRPF_LOC 0x2c08
+#define VI6_BRU_VIRRPF_LOC_HCOORD_MASK (0x1fff << 16)
+#define VI6_BRU_VIRRPF_LOC_HCOORD_SHIFT 16
+#define VI6_BRU_VIRRPF_LOC_VCOORD_MASK (0x1fff << 0)
+#define VI6_BRU_VIRRPF_LOC_VCOORD_SHIFT 0
+
#define VI6_BRU_VIRRPF_COL 0x2c0c
+#define VI6_BRU_VIRRPF_COL_A_MASK (0xff << 24)
+#define VI6_BRU_VIRRPF_COL_A_SHIFT 24
+#define VI6_BRU_VIRRPF_COL_RCR_MASK (0xff << 16)
+#define VI6_BRU_VIRRPF_COL_RCR_SHIFT 16
+#define VI6_BRU_VIRRPF_COL_GY_MASK (0xff << 8)
+#define VI6_BRU_VIRRPF_COL_GY_SHIFT 8
+#define VI6_BRU_VIRRPF_COL_BCB_MASK (0xff << 0)
+#define VI6_BRU_VIRRPF_COL_BCB_SHIFT 0
+
#define VI6_BRU_CTRL(n) (0x2c10 + (n) * 8)
+#define VI6_BRU_CTRL_RBC (1 << 31)
+#define VI6_BRU_CTRL_DSTSEL_BRUIN(n) ((n) << 20)
+#define VI6_BRU_CTRL_DSTSEL_VRPF (4 << 20)
+#define VI6_BRU_CTRL_DSTSEL_MASK (7 << 20)
+#define VI6_BRU_CTRL_SRCSEL_BRUIN(n) ((n) << 16)
+#define VI6_BRU_CTRL_SRCSEL_VRPF (4 << 16)
+#define VI6_BRU_CTRL_SRCSEL_MASK (7 << 16)
+#define VI6_BRU_CTRL_CROP(rop) ((rop) << 4)
+#define VI6_BRU_CTRL_CROP_MASK (0xf << 4)
+#define VI6_BRU_CTRL_AROP(rop) ((rop) << 0)
+#define VI6_BRU_CTRL_AROP_MASK (0xf << 0)
+
#define VI6_BRU_BLD(n) (0x2c14 + (n) * 8)
+#define VI6_BRU_BLD_CBES (1 << 31)
+#define VI6_BRU_BLD_CCMDX_DST_A (0 << 28)
+#define VI6_BRU_BLD_CCMDX_255_DST_A (1 << 28)
+#define VI6_BRU_BLD_CCMDX_SRC_A (2 << 28)
+#define VI6_BRU_BLD_CCMDX_255_SRC_A (3 << 28)
+#define VI6_BRU_BLD_CCMDX_COEFX (4 << 28)
+#define VI6_BRU_BLD_CCMDX_MASK (7 << 28)
+#define VI6_BRU_BLD_CCMDY_DST_A (0 << 24)
+#define VI6_BRU_BLD_CCMDY_255_DST_A (1 << 24)
+#define VI6_BRU_BLD_CCMDY_SRC_A (2 << 24)
+#define VI6_BRU_BLD_CCMDY_255_SRC_A (3 << 24)
+#define VI6_BRU_BLD_CCMDY_COEFY (4 << 24)
+#define VI6_BRU_BLD_CCMDY_MASK (7 << 24)
+#define VI6_BRU_BLD_CCMDY_SHIFT 24
+#define VI6_BRU_BLD_ABES (1 << 23)
+#define VI6_BRU_BLD_ACMDX_DST_A (0 << 20)
+#define VI6_BRU_BLD_ACMDX_255_DST_A (1 << 20)
+#define VI6_BRU_BLD_ACMDX_SRC_A (2 << 20)
+#define VI6_BRU_BLD_ACMDX_255_SRC_A (3 << 20)
+#define VI6_BRU_BLD_ACMDX_COEFX (4 << 20)
+#define VI6_BRU_BLD_ACMDX_MASK (7 << 20)
+#define VI6_BRU_BLD_ACMDY_DST_A (0 << 16)
+#define VI6_BRU_BLD_ACMDY_255_DST_A (1 << 16)
+#define VI6_BRU_BLD_ACMDY_SRC_A (2 << 16)
+#define VI6_BRU_BLD_ACMDY_255_SRC_A (3 << 16)
+#define VI6_BRU_BLD_ACMDY_COEFY (4 << 16)
+#define VI6_BRU_BLD_ACMDY_MASK (7 << 16)
+#define VI6_BRU_BLD_COEFX_MASK (0xff << 8)
+#define VI6_BRU_BLD_COEFX_SHIFT 8
+#define VI6_BRU_BLD_COEFY_MASK (0xff << 0)
+#define VI6_BRU_BLD_COEFY_SHIFT 0
+
#define VI6_BRU_ROP 0x2c30
+#define VI6_BRU_ROP_DSTSEL_BRUIN(n) ((n) << 20)
+#define VI6_BRU_ROP_DSTSEL_VRPF (4 << 20)
+#define VI6_BRU_ROP_DSTSEL_MASK (7 << 20)
+#define VI6_BRU_ROP_CROP(rop) ((rop) << 4)
+#define VI6_BRU_ROP_CROP_MASK (0xf << 4)
+#define VI6_BRU_ROP_AROP(rop) ((rop) << 0)
+#define VI6_BRU_ROP_AROP_MASK (0xf << 0)
/* -----------------------------------------------------------------------------
* HGO Control Registers
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 2b04d0f..c3d98642 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -96,8 +96,10 @@
vsp1_rpf_write(rpf, VI6_RPF_INFMT, infmt);
vsp1_rpf_write(rpf, VI6_RPF_DSWAP, fmtinfo->swap);
- /* Output location. Composing isn't supported yet. */
- vsp1_rpf_write(rpf, VI6_RPF_LOC, 0);
+ /* Output location */
+ vsp1_rpf_write(rpf, VI6_RPF_LOC,
+ (rpf->location.left << VI6_RPF_LOC_HCOORD_SHIFT) |
+ (rpf->location.top << VI6_RPF_LOC_VCOORD_SHIFT));
/* Disable alpha, mask and color key. Set the alpha channel to a fixed
* value of 255.
@@ -176,7 +178,6 @@
rpf->entity.type = VSP1_ENTITY_RPF;
rpf->entity.index = index;
- rpf->entity.id = VI6_DPR_NODE_RPF(index);
ret = vsp1_entity_init(vsp1, &rpf->entity, 2);
if (ret < 0)
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index 5c5ee81..b4fb65e 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -30,6 +30,10 @@
unsigned int max_width;
unsigned int max_height;
+ struct {
+ unsigned int left;
+ unsigned int top;
+ } location;
struct v4l2_rect crop;
unsigned int offsets[2];
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index 7ab1a0b..aa0e04c 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -327,7 +327,6 @@
return ERR_PTR(-ENOMEM);
sru->entity.type = VSP1_ENTITY_SRU;
- sru->entity.id = VI6_DPR_NODE_SRU;
ret = vsp1_entity_init(vsp1, &sru->entity, 2);
if (ret < 0)
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index 622342a..0293bdb 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -131,7 +131,7 @@
return 0;
/* Enable multi-tap scaling. */
- vsp1_uds_write(uds, VI6_UDS_CTRL, VI6_UDS_CTRL_BC);
+ vsp1_uds_write(uds, VI6_UDS_CTRL, VI6_UDS_CTRL_AON | VI6_UDS_CTRL_BC);
vsp1_uds_write(uds, VI6_UDS_PASS_BWIDTH,
(uds_passband_width(uds->hscale)
@@ -139,7 +139,6 @@
(uds_passband_width(uds->vscale)
<< VI6_UDS_PASS_BWIDTH_V_SHIFT));
-
/* Set the scaling ratios and the output size. */
format = &uds->entity.formats[UDS_PAD_SOURCE];
@@ -323,7 +322,6 @@
uds->entity.type = VSP1_ENTITY_UDS;
uds->entity.index = index;
- uds->entity.id = VI6_DPR_NODE_UDS(index);
ret = vsp1_entity_init(vsp1, &uds->entity, 2);
if (ret < 0)
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index a0595c1..8a1253e 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -28,6 +28,7 @@
#include <media/videobuf2-dma-contig.h>
#include "vsp1.h"
+#include "vsp1_bru.h"
#include "vsp1_entity.h"
#include "vsp1_rwpf.h"
#include "vsp1_video.h"
@@ -280,6 +281,9 @@
struct media_pad *pad;
bool uds_found = false;
+ input->location.left = 0;
+ input->location.top = 0;
+
pad = media_entity_remote_pad(&input->entity.pads[RWPF_PAD_SOURCE]);
while (1) {
@@ -292,6 +296,17 @@
entity = to_vsp1_entity(media_entity_to_v4l2_subdev(pad->entity));
+ /* A BRU is present in the pipeline, store the compose rectangle
+ * location in the input RPF for use when configuring the RPF.
+ */
+ if (entity->type == VSP1_ENTITY_BRU) {
+ struct vsp1_bru *bru = to_bru(&entity->subdev);
+ struct v4l2_rect *rect = &bru->compose[pad->index];
+
+ input->location.left = rect->left;
+ input->location.top = rect->top;
+ }
+
/* We've reached the WPF, we're done. */
if (entity->type == VSP1_ENTITY_WPF)
break;
@@ -363,6 +378,8 @@
rwpf->video.pipe_index = 0;
} else if (e->type == VSP1_ENTITY_LIF) {
pipe->lif = e;
+ } else if (e->type == VSP1_ENTITY_BRU) {
+ pipe->bru = e;
}
}
@@ -392,6 +409,7 @@
pipe->num_video = 0;
pipe->num_inputs = 0;
pipe->output = NULL;
+ pipe->bru = NULL;
pipe->lif = NULL;
return ret;
}
@@ -430,6 +448,7 @@
pipe->num_video = 0;
pipe->num_inputs = 0;
pipe->output = NULL;
+ pipe->bru = NULL;
pipe->lif = NULL;
}
@@ -461,7 +480,7 @@
list_for_each_entry(entity, &pipe->entities, list_pipe) {
if (entity->route)
- vsp1_write(entity->vsp1, entity->route,
+ vsp1_write(entity->vsp1, entity->route->reg,
VI6_DPR_NODE_UNUSED);
v4l2_subdev_call(&entity->subdev, video, s_stream, 0);
@@ -680,11 +699,12 @@
{
struct vsp1_entity *sink;
- if (source->route == 0)
+ if (source->route->reg == 0)
return;
sink = container_of(source->sink, struct vsp1_entity, subdev.entity);
- vsp1_write(source->vsp1, source->route, sink->id);
+ vsp1_write(source->vsp1, source->route->reg,
+ sink->route->inputs[source->sink_pad]);
}
static int vsp1_video_start_streaming(struct vb2_queue *vq, unsigned int count)
diff --git a/drivers/media/platform/vsp1/vsp1_video.h b/drivers/media/platform/vsp1/vsp1_video.h
index 53e4b374..c04d48f 100644
--- a/drivers/media/platform/vsp1/vsp1_video.h
+++ b/drivers/media/platform/vsp1/vsp1_video.h
@@ -75,6 +75,7 @@
unsigned int num_inputs;
struct vsp1_rwpf *inputs[VPS1_MAX_RPF];
struct vsp1_rwpf *output;
+ struct vsp1_entity *bru;
struct vsp1_entity *lif;
struct list_head entities;
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 11a61c6..1294340d 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -58,13 +58,21 @@
return 0;
}
- /* Sources */
+ /* Sources. If the pipeline has a single input configure it as the
+ * master layer. Otherwise configure all inputs as sub-layers and
+ * select the virtual RPF as the master layer.
+ */
for (i = 0; i < pipe->num_inputs; ++i) {
struct vsp1_rwpf *input = pipe->inputs[i];
- srcrpf |= VI6_WPF_SRCRPF_RPF_ACT_MST(input->entity.index);
+ srcrpf |= pipe->num_inputs == 1
+ ? VI6_WPF_SRCRPF_RPF_ACT_MST(input->entity.index)
+ : VI6_WPF_SRCRPF_RPF_ACT_SUB(input->entity.index);
}
+ if (pipe->num_inputs > 1)
+ srcrpf |= VI6_WPF_SRCRPF_VIRACT_MST;
+
vsp1_wpf_write(wpf, VI6_WPF_SRCRPF, srcrpf);
/* Destination stride. */
@@ -181,7 +189,6 @@
wpf->entity.type = VSP1_ENTITY_WPF;
wpf->entity.index = index;
- wpf->entity.id = VI6_DPR_NODE_WPF(index);
ret = vsp1_entity_init(vsp1, &wpf->entity, 2);
if (ret < 0)
diff --git a/drivers/memstick/host/Kconfig b/drivers/memstick/host/Kconfig
index 1b37cf8..7310e32 100644
--- a/drivers/memstick/host/Kconfig
+++ b/drivers/memstick/host/Kconfig
@@ -52,3 +52,13 @@
To compile this driver as a module, choose M here: the module will
be called rtsx_pci_ms.
+
+config MEMSTICK_REALTEK_USB
+ tristate "Realtek USB Memstick Card Interface Driver"
+ depends on MFD_RTSX_USB
+ help
+ Say Y here to include driver code to support Memstick card interface
+ of Realtek RTS5129/39 series USB card reader
+
+ To compile this driver as a module, choose M here: the module will
+ be called rts5139_ms.
diff --git a/drivers/memstick/host/Makefile b/drivers/memstick/host/Makefile
index af3459d..491c955 100644
--- a/drivers/memstick/host/Makefile
+++ b/drivers/memstick/host/Makefile
@@ -6,3 +6,4 @@
obj-$(CONFIG_MEMSTICK_JMICRON_38X) += jmb38x_ms.o
obj-$(CONFIG_MEMSTICK_R592) += r592.o
obj-$(CONFIG_MEMSTICK_REALTEK_PCI) += rtsx_pci_ms.o
+obj-$(CONFIG_MEMSTICK_REALTEK_USB) += rtsx_usb_ms.o
diff --git a/drivers/memstick/host/rtsx_usb_ms.c b/drivers/memstick/host/rtsx_usb_ms.c
new file mode 100644
index 0000000..a7282b7
--- /dev/null
+++ b/drivers/memstick/host/rtsx_usb_ms.c
@@ -0,0 +1,839 @@
+/* Realtek USB Memstick Card Interface driver
+ *
+ * Copyright(c) 2009-2013 Realtek Semiconductor Corp. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author:
+ * Roger Tseng <rogerable@realtek.com>
+ */
+
+#include <linux/module.h>
+#include <linux/highmem.h>
+#include <linux/delay.h>
+#include <linux/platform_device.h>
+#include <linux/workqueue.h>
+#include <linux/memstick.h>
+#include <linux/kthread.h>
+#include <linux/mfd/rtsx_usb.h>
+#include <linux/pm_runtime.h>
+#include <linux/mutex.h>
+#include <linux/sched.h>
+#include <linux/completion.h>
+#include <asm/unaligned.h>
+
+struct rtsx_usb_ms {
+ struct platform_device *pdev;
+ struct rtsx_ucr *ucr;
+ struct memstick_host *msh;
+ struct memstick_request *req;
+
+ struct mutex host_mutex;
+ struct work_struct handle_req;
+
+ struct task_struct *detect_ms;
+ struct completion detect_ms_exit;
+
+ u8 ssc_depth;
+ unsigned int clock;
+ int power_mode;
+ unsigned char ifmode;
+ bool eject;
+};
+
+static inline struct device *ms_dev(struct rtsx_usb_ms *host)
+{
+ return &(host->pdev->dev);
+}
+
+static inline void ms_clear_error(struct rtsx_usb_ms *host)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ rtsx_usb_ep0_write_register(ucr, CARD_STOP,
+ MS_STOP | MS_CLR_ERR,
+ MS_STOP | MS_CLR_ERR);
+
+ rtsx_usb_clear_dma_err(ucr);
+ rtsx_usb_clear_fsm_err(ucr);
+}
+
+#ifdef DEBUG
+
+static void ms_print_debug_regs(struct rtsx_usb_ms *host)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ u16 i;
+ u8 *ptr;
+
+ /* Print MS host internal registers */
+ rtsx_usb_init_cmd(ucr);
+
+ /* MS_CFG to MS_INT_REG */
+ for (i = 0xFD40; i <= 0xFD44; i++)
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, i, 0, 0);
+
+ /* CARD_SHARE_MODE to CARD_GPIO */
+ for (i = 0xFD51; i <= 0xFD56; i++)
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, i, 0, 0);
+
+ /* CARD_PULL_CTLx */
+ for (i = 0xFD60; i <= 0xFD65; i++)
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, i, 0, 0);
+
+ /* CARD_DATA_SOURCE, CARD_SELECT, CARD_CLK_EN, CARD_PWR_CTL */
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, CARD_DATA_SOURCE, 0, 0);
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, CARD_SELECT, 0, 0);
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, CARD_CLK_EN, 0, 0);
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, CARD_PWR_CTL, 0, 0);
+
+ rtsx_usb_send_cmd(ucr, MODE_CR, 100);
+ rtsx_usb_get_rsp(ucr, 21, 100);
+
+ ptr = ucr->rsp_buf;
+ for (i = 0xFD40; i <= 0xFD44; i++)
+ dev_dbg(ms_dev(host), "0x%04X: 0x%02x\n", i, *(ptr++));
+ for (i = 0xFD51; i <= 0xFD56; i++)
+ dev_dbg(ms_dev(host), "0x%04X: 0x%02x\n", i, *(ptr++));
+ for (i = 0xFD60; i <= 0xFD65; i++)
+ dev_dbg(ms_dev(host), "0x%04X: 0x%02x\n", i, *(ptr++));
+
+ dev_dbg(ms_dev(host), "0x%04X: 0x%02x\n", CARD_DATA_SOURCE, *(ptr++));
+ dev_dbg(ms_dev(host), "0x%04X: 0x%02x\n", CARD_SELECT, *(ptr++));
+ dev_dbg(ms_dev(host), "0x%04X: 0x%02x\n", CARD_CLK_EN, *(ptr++));
+ dev_dbg(ms_dev(host), "0x%04X: 0x%02x\n", CARD_PWR_CTL, *(ptr++));
+}
+
+#else
+
+static void ms_print_debug_regs(struct rtsx_usb_ms *host)
+{
+}
+
+#endif
+
+static int ms_pull_ctl_disable_lqfp48(struct rtsx_ucr *ucr)
+{
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL1, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL2, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL3, 0xFF, 0x95);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL4, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL5, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL6, 0xFF, 0xA5);
+
+ return rtsx_usb_send_cmd(ucr, MODE_C, 100);
+}
+
+static int ms_pull_ctl_disable_qfn24(struct rtsx_ucr *ucr)
+{
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL1, 0xFF, 0x65);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL2, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL3, 0xFF, 0x95);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL4, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL5, 0xFF, 0x56);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL6, 0xFF, 0x59);
+
+ return rtsx_usb_send_cmd(ucr, MODE_C, 100);
+}
+
+static int ms_pull_ctl_enable_lqfp48(struct rtsx_ucr *ucr)
+{
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL1, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL2, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL3, 0xFF, 0x95);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL4, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL5, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL6, 0xFF, 0xA5);
+
+ return rtsx_usb_send_cmd(ucr, MODE_C, 100);
+}
+
+static int ms_pull_ctl_enable_qfn24(struct rtsx_ucr *ucr)
+{
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL1, 0xFF, 0x65);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL2, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL3, 0xFF, 0x95);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL4, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL5, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL6, 0xFF, 0x59);
+
+ return rtsx_usb_send_cmd(ucr, MODE_C, 100);
+}
+
+static int ms_power_on(struct rtsx_usb_ms *host)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+
+ dev_dbg(ms_dev(host), "%s\n", __func__);
+
+ rtsx_usb_init_cmd(ucr);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_SELECT, 0x07, MS_MOD_SEL);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_SHARE_MODE,
+ CARD_SHARE_MASK, CARD_SHARE_MS);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_CLK_EN,
+ MS_CLK_EN, MS_CLK_EN);
+ err = rtsx_usb_send_cmd(ucr, MODE_C, 100);
+ if (err < 0)
+ return err;
+
+ if (CHECK_PKG(ucr, LQFP48))
+ err = ms_pull_ctl_enable_lqfp48(ucr);
+ else
+ err = ms_pull_ctl_enable_qfn24(ucr);
+ if (err < 0)
+ return err;
+
+ err = rtsx_usb_write_register(ucr, CARD_PWR_CTL,
+ POWER_MASK, PARTIAL_POWER_ON);
+ if (err)
+ return err;
+
+ usleep_range(800, 1000);
+
+ rtsx_usb_init_cmd(ucr);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PWR_CTL,
+ POWER_MASK, POWER_ON);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_OE,
+ MS_OUTPUT_EN, MS_OUTPUT_EN);
+
+ return rtsx_usb_send_cmd(ucr, MODE_C, 100);
+}
+
+static int ms_power_off(struct rtsx_usb_ms *host)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+
+ dev_dbg(ms_dev(host), "%s\n", __func__);
+
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_CLK_EN, MS_CLK_EN, 0);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_OE, MS_OUTPUT_EN, 0);
+
+ err = rtsx_usb_send_cmd(ucr, MODE_C, 100);
+ if (err < 0)
+ return err;
+
+ if (CHECK_PKG(ucr, LQFP48))
+ return ms_pull_ctl_disable_lqfp48(ucr);
+
+ return ms_pull_ctl_disable_qfn24(ucr);
+}
+
+static int ms_transfer_data(struct rtsx_usb_ms *host, unsigned char data_dir,
+ u8 tpc, u8 cfg, struct scatterlist *sg)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+ unsigned int length = sg->length;
+ u16 sec_cnt = (u16)(length / 512);
+ u8 trans_mode, dma_dir, flag;
+ unsigned int pipe;
+ struct memstick_dev *card = host->msh->card;
+
+ dev_dbg(ms_dev(host), "%s: tpc = 0x%02x, data_dir = %s, length = %d\n",
+ __func__, tpc, (data_dir == READ) ? "READ" : "WRITE",
+ length);
+
+ if (data_dir == READ) {
+ flag = MODE_CDIR;
+ dma_dir = DMA_DIR_FROM_CARD;
+ if (card->id.type != MEMSTICK_TYPE_PRO)
+ trans_mode = MS_TM_NORMAL_READ;
+ else
+ trans_mode = MS_TM_AUTO_READ;
+ pipe = usb_rcvbulkpipe(ucr->pusb_dev, EP_BULK_IN);
+ } else {
+ flag = MODE_CDOR;
+ dma_dir = DMA_DIR_TO_CARD;
+ if (card->id.type != MEMSTICK_TYPE_PRO)
+ trans_mode = MS_TM_NORMAL_WRITE;
+ else
+ trans_mode = MS_TM_AUTO_WRITE;
+ pipe = usb_sndbulkpipe(ucr->pusb_dev, EP_BULK_OUT);
+ }
+
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_TPC, 0xFF, tpc);
+ if (card->id.type == MEMSTICK_TYPE_PRO) {
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_SECTOR_CNT_H,
+ 0xFF, (u8)(sec_cnt >> 8));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_SECTOR_CNT_L,
+ 0xFF, (u8)sec_cnt);
+ }
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_TRANS_CFG, 0xFF, cfg);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MC_DMA_TC3,
+ 0xFF, (u8)(length >> 24));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MC_DMA_TC2,
+ 0xFF, (u8)(length >> 16));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MC_DMA_TC1,
+ 0xFF, (u8)(length >> 8));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MC_DMA_TC0, 0xFF,
+ (u8)length);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MC_DMA_CTL,
+ 0x03 | DMA_PACK_SIZE_MASK, dma_dir | DMA_EN | DMA_512);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_DATA_SOURCE,
+ 0x01, RING_BUFFER);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_TRANSFER,
+ 0xFF, MS_TRANSFER_START | trans_mode);
+ rtsx_usb_add_cmd(ucr, CHECK_REG_CMD, MS_TRANSFER,
+ MS_TRANSFER_END, MS_TRANSFER_END);
+
+ err = rtsx_usb_send_cmd(ucr, flag | STAGE_MS_STATUS, 100);
+ if (err)
+ return err;
+
+ err = rtsx_usb_transfer_data(ucr, pipe, sg, length,
+ 1, NULL, 10000);
+ if (err)
+ goto err_out;
+
+ err = rtsx_usb_get_rsp(ucr, 3, 15000);
+ if (err)
+ goto err_out;
+
+ if (ucr->rsp_buf[0] & MS_TRANSFER_ERR ||
+ ucr->rsp_buf[1] & (MS_CRC16_ERR | MS_RDY_TIMEOUT)) {
+ err = -EIO;
+ goto err_out;
+ }
+ return 0;
+err_out:
+ ms_clear_error(host);
+ return err;
+}
+
+static int ms_write_bytes(struct rtsx_usb_ms *host, u8 tpc,
+ u8 cfg, u8 cnt, u8 *data, u8 *int_reg)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ int err, i;
+
+ dev_dbg(ms_dev(host), "%s: tpc = 0x%02x\n", __func__, tpc);
+
+ rtsx_usb_init_cmd(ucr);
+
+ for (i = 0; i < cnt; i++)
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ PPBUF_BASE2 + i, 0xFF, data[i]);
+
+ if (cnt % 2)
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ PPBUF_BASE2 + i, 0xFF, 0xFF);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_TPC, 0xFF, tpc);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_BYTE_CNT, 0xFF, cnt);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_TRANS_CFG, 0xFF, cfg);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_DATA_SOURCE,
+ 0x01, PINGPONG_BUFFER);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_TRANSFER,
+ 0xFF, MS_TRANSFER_START | MS_TM_WRITE_BYTES);
+ rtsx_usb_add_cmd(ucr, CHECK_REG_CMD, MS_TRANSFER,
+ MS_TRANSFER_END, MS_TRANSFER_END);
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, MS_TRANS_CFG, 0, 0);
+
+ err = rtsx_usb_send_cmd(ucr, MODE_CR, 100);
+ if (err)
+ return err;
+
+ err = rtsx_usb_get_rsp(ucr, 2, 5000);
+ if (err || (ucr->rsp_buf[0] & MS_TRANSFER_ERR)) {
+ u8 val;
+
+ rtsx_usb_ep0_read_register(ucr, MS_TRANS_CFG, &val);
+ dev_dbg(ms_dev(host), "MS_TRANS_CFG: 0x%02x\n", val);
+
+ if (int_reg)
+ *int_reg = val & 0x0F;
+
+ ms_print_debug_regs(host);
+
+ ms_clear_error(host);
+
+ if (!(tpc & 0x08)) {
+ if (val & MS_CRC16_ERR)
+ return -EIO;
+ } else {
+ if (!(val & 0x80)) {
+ if (val & (MS_INT_ERR | MS_INT_CMDNK))
+ return -EIO;
+ }
+ }
+
+ return -ETIMEDOUT;
+ }
+
+ if (int_reg)
+ *int_reg = ucr->rsp_buf[1] & 0x0F;
+
+ return 0;
+}
+
+static int ms_read_bytes(struct rtsx_usb_ms *host, u8 tpc,
+ u8 cfg, u8 cnt, u8 *data, u8 *int_reg)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ int err, i;
+ u8 *ptr;
+
+ dev_dbg(ms_dev(host), "%s: tpc = 0x%02x\n", __func__, tpc);
+
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_TPC, 0xFF, tpc);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_BYTE_CNT, 0xFF, cnt);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_TRANS_CFG, 0xFF, cfg);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_DATA_SOURCE,
+ 0x01, PINGPONG_BUFFER);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MS_TRANSFER,
+ 0xFF, MS_TRANSFER_START | MS_TM_READ_BYTES);
+ rtsx_usb_add_cmd(ucr, CHECK_REG_CMD, MS_TRANSFER,
+ MS_TRANSFER_END, MS_TRANSFER_END);
+ for (i = 0; i < cnt - 1; i++)
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, PPBUF_BASE2 + i, 0, 0);
+ if (cnt % 2)
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, PPBUF_BASE2 + cnt, 0, 0);
+ else
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD,
+ PPBUF_BASE2 + cnt - 1, 0, 0);
+
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, MS_TRANS_CFG, 0, 0);
+
+ err = rtsx_usb_send_cmd(ucr, MODE_CR, 100);
+ if (err)
+ return err;
+
+ err = rtsx_usb_get_rsp(ucr, cnt + 2, 5000);
+ if (err || (ucr->rsp_buf[0] & MS_TRANSFER_ERR)) {
+ u8 val;
+
+ rtsx_usb_ep0_read_register(ucr, MS_TRANS_CFG, &val);
+ dev_dbg(ms_dev(host), "MS_TRANS_CFG: 0x%02x\n", val);
+
+ if (int_reg && (host->ifmode != MEMSTICK_SERIAL))
+ *int_reg = val & 0x0F;
+
+ ms_print_debug_regs(host);
+
+ ms_clear_error(host);
+
+ if (!(tpc & 0x08)) {
+ if (val & MS_CRC16_ERR)
+ return -EIO;
+ } else {
+ if (!(val & 0x80)) {
+ if (val & (MS_INT_ERR | MS_INT_CMDNK))
+ return -EIO;
+ }
+ }
+
+ return -ETIMEDOUT;
+ }
+
+ ptr = ucr->rsp_buf + 1;
+ for (i = 0; i < cnt; i++)
+ data[i] = *ptr++;
+
+
+ if (int_reg && (host->ifmode != MEMSTICK_SERIAL))
+ *int_reg = *ptr & 0x0F;
+
+ return 0;
+}
+
+static int rtsx_usb_ms_issue_cmd(struct rtsx_usb_ms *host)
+{
+ struct memstick_request *req = host->req;
+ int err = 0;
+ u8 cfg = 0, int_reg;
+
+ dev_dbg(ms_dev(host), "%s\n", __func__);
+
+ if (req->need_card_int) {
+ if (host->ifmode != MEMSTICK_SERIAL)
+ cfg = WAIT_INT;
+ }
+
+ if (req->long_data) {
+ err = ms_transfer_data(host, req->data_dir,
+ req->tpc, cfg, &(req->sg));
+ } else {
+ if (req->data_dir == READ)
+ err = ms_read_bytes(host, req->tpc, cfg,
+ req->data_len, req->data, &int_reg);
+ else
+ err = ms_write_bytes(host, req->tpc, cfg,
+ req->data_len, req->data, &int_reg);
+ }
+ if (err < 0)
+ return err;
+
+ if (req->need_card_int) {
+ if (host->ifmode == MEMSTICK_SERIAL) {
+ err = ms_read_bytes(host, MS_TPC_GET_INT,
+ NO_WAIT_INT, 1, &req->int_reg, NULL);
+ if (err < 0)
+ return err;
+ } else {
+
+ if (int_reg & MS_INT_CMDNK)
+ req->int_reg |= MEMSTICK_INT_CMDNAK;
+ if (int_reg & MS_INT_BREQ)
+ req->int_reg |= MEMSTICK_INT_BREQ;
+ if (int_reg & MS_INT_ERR)
+ req->int_reg |= MEMSTICK_INT_ERR;
+ if (int_reg & MS_INT_CED)
+ req->int_reg |= MEMSTICK_INT_CED;
+ }
+ dev_dbg(ms_dev(host), "int_reg: 0x%02x\n", req->int_reg);
+ }
+
+ return 0;
+}
+
+static void rtsx_usb_ms_handle_req(struct work_struct *work)
+{
+ struct rtsx_usb_ms *host = container_of(work,
+ struct rtsx_usb_ms, handle_req);
+ struct rtsx_ucr *ucr = host->ucr;
+ struct memstick_host *msh = host->msh;
+ int rc;
+
+ if (!host->req) {
+ do {
+ rc = memstick_next_req(msh, &host->req);
+ dev_dbg(ms_dev(host), "next req %d\n", rc);
+
+ if (!rc) {
+ mutex_lock(&ucr->dev_mutex);
+
+ if (rtsx_usb_card_exclusive_check(ucr,
+ RTSX_USB_MS_CARD))
+ host->req->error = -EIO;
+ else
+ host->req->error =
+ rtsx_usb_ms_issue_cmd(host);
+
+ mutex_unlock(&ucr->dev_mutex);
+
+ dev_dbg(ms_dev(host), "req result %d\n",
+ host->req->error);
+ }
+ } while (!rc);
+ }
+
+}
+
+static void rtsx_usb_ms_request(struct memstick_host *msh)
+{
+ struct rtsx_usb_ms *host = memstick_priv(msh);
+
+ dev_dbg(ms_dev(host), "--> %s\n", __func__);
+
+ if (!host->eject)
+ schedule_work(&host->handle_req);
+}
+
+static int rtsx_usb_ms_set_param(struct memstick_host *msh,
+ enum memstick_param param, int value)
+{
+ struct rtsx_usb_ms *host = memstick_priv(msh);
+ struct rtsx_ucr *ucr = host->ucr;
+ unsigned int clock = 0;
+ u8 ssc_depth = 0;
+ int err;
+
+ dev_dbg(ms_dev(host), "%s: param = %d, value = %d\n",
+ __func__, param, value);
+
+ mutex_lock(&ucr->dev_mutex);
+
+ err = rtsx_usb_card_exclusive_check(ucr, RTSX_USB_MS_CARD);
+ if (err)
+ goto out;
+
+ switch (param) {
+ case MEMSTICK_POWER:
+ if (value == host->power_mode)
+ break;
+
+ if (value == MEMSTICK_POWER_ON) {
+ pm_runtime_get_sync(ms_dev(host));
+ err = ms_power_on(host);
+ } else if (value == MEMSTICK_POWER_OFF) {
+ err = ms_power_off(host);
+ if (host->msh->card)
+ pm_runtime_put_noidle(ms_dev(host));
+ else
+ pm_runtime_put(ms_dev(host));
+ } else
+ err = -EINVAL;
+ if (!err)
+ host->power_mode = value;
+ break;
+
+ case MEMSTICK_INTERFACE:
+ if (value == MEMSTICK_SERIAL) {
+ clock = 19000000;
+ ssc_depth = SSC_DEPTH_512K;
+ err = rtsx_usb_write_register(ucr, MS_CFG, 0x5A,
+ MS_BUS_WIDTH_1 | PUSH_TIME_DEFAULT);
+ if (err < 0)
+ break;
+ } else if (value == MEMSTICK_PAR4) {
+ clock = 39000000;
+ ssc_depth = SSC_DEPTH_1M;
+
+ err = rtsx_usb_write_register(ucr, MS_CFG, 0x5A,
+ MS_BUS_WIDTH_4 | PUSH_TIME_ODD |
+ MS_NO_CHECK_INT);
+ if (err < 0)
+ break;
+ } else {
+ err = -EINVAL;
+ break;
+ }
+
+ err = rtsx_usb_switch_clock(ucr, clock,
+ ssc_depth, false, true, false);
+ if (err < 0) {
+ dev_dbg(ms_dev(host), "switch clock failed\n");
+ break;
+ }
+
+ host->ssc_depth = ssc_depth;
+ host->clock = clock;
+ host->ifmode = value;
+ break;
+ default:
+ err = -EINVAL;
+ break;
+ }
+out:
+ mutex_unlock(&ucr->dev_mutex);
+
+ /* power-on delay */
+ if (param == MEMSTICK_POWER && value == MEMSTICK_POWER_ON)
+ usleep_range(10000, 12000);
+
+ dev_dbg(ms_dev(host), "%s: return = %d\n", __func__, err);
+ return err;
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int rtsx_usb_ms_suspend(struct device *dev)
+{
+ struct rtsx_usb_ms *host = dev_get_drvdata(dev);
+ struct memstick_host *msh = host->msh;
+
+ dev_dbg(ms_dev(host), "--> %s\n", __func__);
+
+ memstick_suspend_host(msh);
+ return 0;
+}
+
+static int rtsx_usb_ms_resume(struct device *dev)
+{
+ struct rtsx_usb_ms *host = dev_get_drvdata(dev);
+ struct memstick_host *msh = host->msh;
+
+ dev_dbg(ms_dev(host), "--> %s\n", __func__);
+
+ memstick_resume_host(msh);
+ return 0;
+}
+#endif /* CONFIG_PM_SLEEP */
+
+/*
+ * Thread function of ms card slot detection. The thread starts right after
+ * successful host addition. It stops while the driver removal function sets
+ * host->eject true.
+ */
+static int rtsx_usb_detect_ms_card(void *__host)
+{
+ struct rtsx_usb_ms *host = (struct rtsx_usb_ms *)__host;
+ struct rtsx_ucr *ucr = host->ucr;
+ u8 val = 0;
+ int err;
+
+ for (;;) {
+ mutex_lock(&ucr->dev_mutex);
+
+ /* Check pending MS card changes */
+ err = rtsx_usb_read_register(ucr, CARD_INT_PEND, &val);
+ if (err) {
+ mutex_unlock(&ucr->dev_mutex);
+ goto poll_again;
+ }
+
+ /* Clear the pending */
+ rtsx_usb_write_register(ucr, CARD_INT_PEND,
+ XD_INT | MS_INT | SD_INT,
+ XD_INT | MS_INT | SD_INT);
+
+ mutex_unlock(&ucr->dev_mutex);
+
+ if (val & MS_INT) {
+ dev_dbg(ms_dev(host), "MS slot change detected\n");
+ memstick_detect_change(host->msh);
+ }
+
+poll_again:
+ if (host->eject)
+ break;
+
+ msleep(1000);
+ }
+
+ complete(&host->detect_ms_exit);
+ return 0;
+}
+
+static int rtsx_usb_ms_drv_probe(struct platform_device *pdev)
+{
+ struct memstick_host *msh;
+ struct rtsx_usb_ms *host;
+ struct rtsx_ucr *ucr;
+ int err;
+
+ ucr = usb_get_intfdata(to_usb_interface(pdev->dev.parent));
+ if (!ucr)
+ return -ENXIO;
+
+ dev_dbg(&(pdev->dev),
+ "Realtek USB Memstick controller found\n");
+
+ msh = memstick_alloc_host(sizeof(*host), &pdev->dev);
+ if (!msh)
+ return -ENOMEM;
+
+ host = memstick_priv(msh);
+ host->ucr = ucr;
+ host->msh = msh;
+ host->pdev = pdev;
+ host->power_mode = MEMSTICK_POWER_OFF;
+ platform_set_drvdata(pdev, host);
+
+ mutex_init(&host->host_mutex);
+ INIT_WORK(&host->handle_req, rtsx_usb_ms_handle_req);
+
+ init_completion(&host->detect_ms_exit);
+ host->detect_ms = kthread_create(rtsx_usb_detect_ms_card, host,
+ "rtsx_usb_ms_%d", pdev->id);
+ if (IS_ERR(host->detect_ms)) {
+ dev_dbg(&(pdev->dev),
+ "Unable to create polling thread.\n");
+ err = PTR_ERR(host->detect_ms);
+ goto err_out;
+ }
+
+ msh->request = rtsx_usb_ms_request;
+ msh->set_param = rtsx_usb_ms_set_param;
+ msh->caps = MEMSTICK_CAP_PAR4;
+
+ pm_runtime_enable(&pdev->dev);
+ err = memstick_add_host(msh);
+ if (err)
+ goto err_out;
+
+ wake_up_process(host->detect_ms);
+ return 0;
+err_out:
+ memstick_free_host(msh);
+ return err;
+}
+
+static int rtsx_usb_ms_drv_remove(struct platform_device *pdev)
+{
+ struct rtsx_usb_ms *host = platform_get_drvdata(pdev);
+ struct memstick_host *msh;
+ int err;
+
+ msh = host->msh;
+ host->eject = true;
+ cancel_work_sync(&host->handle_req);
+
+ mutex_lock(&host->host_mutex);
+ if (host->req) {
+ dev_dbg(&(pdev->dev),
+ "%s: Controller removed during transfer\n",
+ dev_name(&msh->dev));
+ host->req->error = -ENOMEDIUM;
+ do {
+ err = memstick_next_req(msh, &host->req);
+ if (!err)
+ host->req->error = -ENOMEDIUM;
+ } while (!err);
+ }
+ mutex_unlock(&host->host_mutex);
+
+ wait_for_completion(&host->detect_ms_exit);
+ memstick_remove_host(msh);
+ memstick_free_host(msh);
+
+ /* Balance possible unbalanced usage count
+ * e.g. unconditional module removal
+ */
+ if (pm_runtime_active(ms_dev(host)))
+ pm_runtime_put(ms_dev(host));
+
+ pm_runtime_disable(&pdev->dev);
+ platform_set_drvdata(pdev, NULL);
+
+ dev_dbg(&(pdev->dev),
+ ": Realtek USB Memstick controller has been removed\n");
+
+ return 0;
+}
+
+static SIMPLE_DEV_PM_OPS(rtsx_usb_ms_pm_ops,
+ rtsx_usb_ms_suspend, rtsx_usb_ms_resume);
+
+static struct platform_device_id rtsx_usb_ms_ids[] = {
+ {
+ .name = "rtsx_usb_ms",
+ }, {
+ /* sentinel */
+ }
+};
+MODULE_DEVICE_TABLE(platform, rtsx_usb_ms_ids);
+
+static struct platform_driver rtsx_usb_ms_driver = {
+ .probe = rtsx_usb_ms_drv_probe,
+ .remove = rtsx_usb_ms_drv_remove,
+ .id_table = rtsx_usb_ms_ids,
+ .driver = {
+ .owner = THIS_MODULE,
+ .name = "rtsx_usb_ms",
+ .pm = &rtsx_usb_ms_pm_ops,
+ },
+};
+module_platform_driver(rtsx_usb_ms_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Roger Tseng <rogerable@realtek.com>");
+MODULE_DESCRIPTION("Realtek USB Memstick Card Host Driver");
diff --git a/drivers/message/fusion/mptbase.c b/drivers/message/fusion/mptbase.c
index 570b18a..ebc0af7 100644
--- a/drivers/message/fusion/mptbase.c
+++ b/drivers/message/fusion/mptbase.c
@@ -1037,7 +1037,7 @@
goto out;
/* signature to know if this mf is freed */
mf->u.frame.linkage.arg1 = cpu_to_le32(0xdeadbeaf);
- list_add_tail(&mf->u.frame.linkage.list, &ioc->FreeQ);
+ list_add(&mf->u.frame.linkage.list, &ioc->FreeQ);
#ifdef MFCNT
ioc->mfcnt--;
#endif
diff --git a/drivers/message/fusion/mptctl.c b/drivers/message/fusion/mptctl.c
index dcc8385..8a050e8 100644
--- a/drivers/message/fusion/mptctl.c
+++ b/drivers/message/fusion/mptctl.c
@@ -2432,9 +2432,9 @@
int rc, cim_rev;
ToolboxIstwiReadWriteRequest_t *IstwiRWRequest;
MPT_FRAME_HDR *mf = NULL;
- MPIHeader_t *mpi_hdr;
unsigned long timeleft;
int retval;
+ u32 msgcontext;
/* Reset long to int. Should affect IA64 and SPARC only
*/
@@ -2581,11 +2581,11 @@
}
IstwiRWRequest = (ToolboxIstwiReadWriteRequest_t *)mf;
- mpi_hdr = (MPIHeader_t *) mf;
+ msgcontext = IstwiRWRequest->MsgContext;
memset(IstwiRWRequest,0,sizeof(ToolboxIstwiReadWriteRequest_t));
+ IstwiRWRequest->MsgContext = msgcontext;
IstwiRWRequest->Function = MPI_FUNCTION_TOOLBOX;
IstwiRWRequest->Tool = MPI_TOOLBOX_ISTWI_READ_WRITE_TOOL;
- IstwiRWRequest->MsgContext = mpi_hdr->MsgContext;
IstwiRWRequest->Flags = MPI_TB_ISTWI_FLAGS_READ;
IstwiRWRequest->NumAddressBytes = 0x01;
IstwiRWRequest->DataLength = cpu_to_le16(0x04);
diff --git a/drivers/message/fusion/mptfc.c b/drivers/message/fusion/mptfc.c
index fd75108..02a3eef 100644
--- a/drivers/message/fusion/mptfc.c
+++ b/drivers/message/fusion/mptfc.c
@@ -649,7 +649,7 @@
}
static int
-mptfc_qcmd_lck(struct scsi_cmnd *SCpnt, void (*done)(struct scsi_cmnd *))
+mptfc_qcmd(struct Scsi_Host *shost, struct scsi_cmnd *SCpnt)
{
struct mptfc_rport_info *ri;
struct fc_rport *rport = starget_to_rport(scsi_target(SCpnt->device));
@@ -658,14 +658,14 @@
if (!vdevice || !vdevice->vtarget) {
SCpnt->result = DID_NO_CONNECT << 16;
- done(SCpnt);
+ SCpnt->scsi_done(SCpnt);
return 0;
}
err = fc_remote_port_chkready(rport);
if (unlikely(err)) {
SCpnt->result = err;
- done(SCpnt);
+ SCpnt->scsi_done(SCpnt);
return 0;
}
@@ -673,15 +673,13 @@
ri = *((struct mptfc_rport_info **)rport->dd_data);
if (unlikely(!ri)) {
SCpnt->result = DID_IMM_RETRY << 16;
- done(SCpnt);
+ SCpnt->scsi_done(SCpnt);
return 0;
}
- return mptscsih_qcmd(SCpnt,done);
+ return mptscsih_qcmd(SCpnt);
}
-static DEF_SCSI_QCMD(mptfc_qcmd)
-
/*
* mptfc_display_port_link_speed - displaying link speed
* @ioc: Pointer to MPT_ADAPTER structure
diff --git a/drivers/message/fusion/mptsas.c b/drivers/message/fusion/mptsas.c
index 00d339c..711fcb5 100644
--- a/drivers/message/fusion/mptsas.c
+++ b/drivers/message/fusion/mptsas.c
@@ -1896,7 +1896,7 @@
}
static int
-mptsas_qcmd_lck(struct scsi_cmnd *SCpnt, void (*done)(struct scsi_cmnd *))
+mptsas_qcmd(struct Scsi_Host *shost, struct scsi_cmnd *SCpnt)
{
MPT_SCSI_HOST *hd;
MPT_ADAPTER *ioc;
@@ -1904,11 +1904,11 @@
if (!vdevice || !vdevice->vtarget || vdevice->vtarget->deleted) {
SCpnt->result = DID_NO_CONNECT << 16;
- done(SCpnt);
+ SCpnt->scsi_done(SCpnt);
return 0;
}
- hd = shost_priv(SCpnt->device->host);
+ hd = shost_priv(shost);
ioc = hd->ioc;
if (ioc->sas_discovery_quiesce_io)
@@ -1917,11 +1917,9 @@
if (ioc->debug_level & MPT_DEBUG_SCSI)
scsi_print_command(SCpnt);
- return mptscsih_qcmd(SCpnt,done);
+ return mptscsih_qcmd(SCpnt);
}
-static DEF_SCSI_QCMD(mptsas_qcmd)
-
/**
* mptsas_mptsas_eh_timed_out - resets the scsi_cmnd timeout
* if the device under question is currently in the
diff --git a/drivers/message/fusion/mptscsih.c b/drivers/message/fusion/mptscsih.c
index 727819c..2a1c6f2 100644
--- a/drivers/message/fusion/mptscsih.c
+++ b/drivers/message/fusion/mptscsih.c
@@ -1304,7 +1304,6 @@
/**
* mptscsih_qcmd - Primary Fusion MPT SCSI initiator IO start routine.
* @SCpnt: Pointer to scsi_cmnd structure
- * @done: Pointer SCSI mid-layer IO completion function
*
* (linux scsi_host_template.queuecommand routine)
* This is the primary SCSI IO start routine. Create a MPI SCSIIORequest
@@ -1313,7 +1312,7 @@
* Returns 0. (rtn value discarded by linux scsi mid-layer)
*/
int
-mptscsih_qcmd(struct scsi_cmnd *SCpnt, void (*done)(struct scsi_cmnd *))
+mptscsih_qcmd(struct scsi_cmnd *SCpnt)
{
MPT_SCSI_HOST *hd;
MPT_FRAME_HDR *mf;
@@ -1329,10 +1328,9 @@
hd = shost_priv(SCpnt->device->host);
ioc = hd->ioc;
- SCpnt->scsi_done = done;
- dmfprintk(ioc, printk(MYIOC_s_DEBUG_FMT "qcmd: SCpnt=%p, done()=%p\n",
- ioc->name, SCpnt, done));
+ dmfprintk(ioc, printk(MYIOC_s_DEBUG_FMT "qcmd: SCpnt=%p\n",
+ ioc->name, SCpnt));
if (ioc->taskmgmt_quiesce_io)
return SCSI_MLQUEUE_HOST_BUSY;
diff --git a/drivers/message/fusion/mptscsih.h b/drivers/message/fusion/mptscsih.h
index 83f5031..99e3390 100644
--- a/drivers/message/fusion/mptscsih.h
+++ b/drivers/message/fusion/mptscsih.h
@@ -113,7 +113,7 @@
#endif
extern int mptscsih_show_info(struct seq_file *, struct Scsi_Host *);
extern const char * mptscsih_info(struct Scsi_Host *SChost);
-extern int mptscsih_qcmd(struct scsi_cmnd *SCpnt, void (*done)(struct scsi_cmnd *));
+extern int mptscsih_qcmd(struct scsi_cmnd *SCpnt);
extern int mptscsih_IssueTaskMgmt(MPT_SCSI_HOST *hd, u8 type, u8 channel,
u8 id, int lun, int ctx2abort, ulong timeout);
extern void mptscsih_slave_destroy(struct scsi_device *device);
diff --git a/drivers/message/fusion/mptspi.c b/drivers/message/fusion/mptspi.c
index 5653e50..49d1133 100644
--- a/drivers/message/fusion/mptspi.c
+++ b/drivers/message/fusion/mptspi.c
@@ -780,33 +780,31 @@
}
static int
-mptspi_qcmd_lck(struct scsi_cmnd *SCpnt, void (*done)(struct scsi_cmnd *))
+mptspi_qcmd(struct Scsi_Host *shost, struct scsi_cmnd *SCpnt)
{
- struct _MPT_SCSI_HOST *hd = shost_priv(SCpnt->device->host);
+ struct _MPT_SCSI_HOST *hd = shost_priv(shost);
VirtDevice *vdevice = SCpnt->device->hostdata;
MPT_ADAPTER *ioc = hd->ioc;
if (!vdevice || !vdevice->vtarget) {
SCpnt->result = DID_NO_CONNECT << 16;
- done(SCpnt);
+ SCpnt->scsi_done(SCpnt);
return 0;
}
if (SCpnt->device->channel == 1 &&
mptscsih_is_phys_disk(ioc, 0, SCpnt->device->id) == 0) {
SCpnt->result = DID_NO_CONNECT << 16;
- done(SCpnt);
+ SCpnt->scsi_done(SCpnt);
return 0;
}
if (spi_dv_pending(scsi_target(SCpnt->device)))
ddvprintk(ioc, scsi_print_command(SCpnt));
- return mptscsih_qcmd(SCpnt,done);
+ return mptscsih_qcmd(SCpnt);
}
-static DEF_SCSI_QCMD(mptspi_qcmd)
-
static void mptspi_slave_destroy(struct scsi_device *sdev)
{
struct scsi_target *starget = scsi_target(sdev);
diff --git a/drivers/mfd/Kconfig b/drivers/mfd/Kconfig
index 6deb8a1..ee8204c 100644
--- a/drivers/mfd/Kconfig
+++ b/drivers/mfd/Kconfig
@@ -67,6 +67,18 @@
help
Support for the BCM590xx PMUs from Broadcom
+config MFD_AXP20X
+ bool "X-Powers AXP20X"
+ select MFD_CORE
+ select REGMAP_I2C
+ select REGMAP_IRQ
+ depends on I2C=y
+ help
+ If you say Y here you get support for the X-Powers AXP202 and AXP209.
+ This driver include only the core APIs. You have to select individual
+ components like regulators or the PEK (Power Enable Key) under the
+ corresponding menus.
+
config MFD_CROS_EC
tristate "ChromeOS Embedded Controller"
select MFD_CORE
@@ -250,6 +262,16 @@
Passage) chip. This chip embeds audio, battery, GPIO, etc.
devices used in Intel Medfield platforms.
+config MFD_IPAQ_MICRO
+ bool "Atmel Micro ASIC (iPAQ h3100/h3600/h3700) Support"
+ depends on SA1100_H3100 || SA1100_H3600
+ select MFD_CORE
+ help
+ Select this to get support for the Microcontroller found in
+ the Compaq iPAQ handheld computers. This is an Atmel
+ AT90LS8535 microcontroller flashed with a special iPAQ
+ firmware using the custom protocol implemented in this driver.
+
config MFD_JANZ_CMODIO
tristate "Janz CMOD-IO PCI MODULbus Carrier Board"
select MFD_CORE
@@ -675,6 +697,7 @@
config MFD_STMPE
bool "STMicroelectronics STMPE"
depends on (I2C=y || SPI_MASTER=y)
+ depends on OF
select MFD_CORE
help
Support for the STMPE family of I/O Expanders from
@@ -719,6 +742,14 @@
select MFD_CORE
select REGMAP_MMIO
+config MFD_SUN6I_PRCM
+ bool "Allwinner A31 PRCM controller"
+ depends on ARCH_SUNXI
+ select MFD_CORE
+ help
+ Support for the PRCM (Power/Reset/Clock Management) unit available
+ in A31 SoC.
+
config MFD_SYSCON
bool "System Controller Register R/W Based on Regmap"
select REGMAP_MMIO
diff --git a/drivers/mfd/Makefile b/drivers/mfd/Makefile
index cec3487..8afedba 100644
--- a/drivers/mfd/Makefile
+++ b/drivers/mfd/Makefile
@@ -29,6 +29,7 @@
obj-$(CONFIG_MFD_STMPE) += stmpe.o
obj-$(CONFIG_STMPE_I2C) += stmpe-i2c.o
obj-$(CONFIG_STMPE_SPI) += stmpe-spi.o
+obj-$(CONFIG_MFD_SUN6I_PRCM) += sun6i-prcm.o
obj-$(CONFIG_MFD_TC3589X) += tc3589x.o
obj-$(CONFIG_MFD_T7L66XB) += t7l66xb.o tmio_core.o
obj-$(CONFIG_MFD_TC6387XB) += tc6387xb.o tmio_core.o
@@ -102,6 +103,7 @@
obj-$(CONFIG_PMIC_DA9052) += da9052-core.o
obj-$(CONFIG_MFD_DA9052_SPI) += da9052-spi.o
obj-$(CONFIG_MFD_DA9052_I2C) += da9052-i2c.o
+obj-$(CONFIG_MFD_AXP20X) += axp20x.o
obj-$(CONFIG_MFD_LP3943) += lp3943.o
obj-$(CONFIG_MFD_LP8788) += lp8788.o lp8788-irq.o
@@ -166,3 +168,4 @@
obj-$(CONFIG_MFD_AS3711) += as3711.o
obj-$(CONFIG_MFD_AS3722) += as3722.o
obj-$(CONFIG_MFD_STW481X) += stw481x.o
+obj-$(CONFIG_MFD_IPAQ_MICRO) += ipaq-micro.o
diff --git a/drivers/mfd/abx500-core.c b/drivers/mfd/abx500-core.c
index f3a15aa..fe41899 100644
--- a/drivers/mfd/abx500-core.c
+++ b/drivers/mfd/abx500-core.c
@@ -151,22 +151,6 @@
}
EXPORT_SYMBOL(abx500_startup_irq_enabled);
-void abx500_dump_all_banks(void)
-{
- struct abx500_ops *ops;
- struct device dummy_child = {NULL};
- struct abx500_device_entry *dev_entry;
-
- list_for_each_entry(dev_entry, &abx500_list, list) {
- dummy_child.parent = dev_entry->dev;
- ops = &dev_entry->ops;
-
- if ((ops != NULL) && (ops->dump_all_banks != NULL))
- ops->dump_all_banks(&dummy_child);
- }
-}
-EXPORT_SYMBOL(abx500_dump_all_banks);
-
MODULE_AUTHOR("Mattias Wallin <mattias.wallin@stericsson.com>");
MODULE_DESCRIPTION("ABX500 core driver");
MODULE_LICENSE("GPL");
diff --git a/drivers/mfd/arizona-core.c b/drivers/mfd/arizona-core.c
index 07e6e27..cfc191a 100644
--- a/drivers/mfd/arizona-core.c
+++ b/drivers/mfd/arizona-core.c
@@ -583,6 +583,7 @@
"CPVDD",
"SPKVDDL",
"SPKVDDR",
+ "MICVDD",
};
static const struct mfd_cell wm5102_devs[] = {
diff --git a/drivers/mfd/arizona-irq.c b/drivers/mfd/arizona-irq.c
index 88758ab..17102f5 100644
--- a/drivers/mfd/arizona-irq.c
+++ b/drivers/mfd/arizona-irq.c
@@ -285,7 +285,7 @@
IRQF_ONESHOT, -1, irq,
&arizona->irq_chip);
if (ret != 0) {
- dev_err(arizona->dev, "Failed to add AOD IRQs: %d\n", ret);
+ dev_err(arizona->dev, "Failed to add main IRQs: %d\n", ret);
goto err_aod;
}
diff --git a/drivers/mfd/as3711.c b/drivers/mfd/as3711.c
index ec684fc..d9706ed 100644
--- a/drivers/mfd/as3711.c
+++ b/drivers/mfd/as3711.c
@@ -114,7 +114,7 @@
};
#ifdef CONFIG_OF
-static struct of_device_id as3711_of_match[] = {
+static const struct of_device_id as3711_of_match[] = {
{.compatible = "ams,as3711",},
{}
};
diff --git a/drivers/mfd/axp20x.c b/drivers/mfd/axp20x.c
new file mode 100644
index 0000000..dee6539
--- /dev/null
+++ b/drivers/mfd/axp20x.c
@@ -0,0 +1,258 @@
+/*
+ * axp20x.c - MFD core driver for the X-Powers AXP202 and AXP209
+ *
+ * AXP20x comprises an adaptive USB-Compatible PWM charger, 2 BUCK DC-DC
+ * converters, 5 LDOs, multiple 12-bit ADCs of voltage, current and temperature
+ * as well as 4 configurable GPIOs.
+ *
+ * Author: Carlo Caione <carlo@caione.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/err.h>
+#include <linux/i2c.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/pm_runtime.h>
+#include <linux/regmap.h>
+#include <linux/slab.h>
+#include <linux/regulator/consumer.h>
+#include <linux/mfd/axp20x.h>
+#include <linux/mfd/core.h>
+#include <linux/of_device.h>
+#include <linux/of_irq.h>
+
+#define AXP20X_OFF 0x80
+
+static const struct regmap_range axp20x_writeable_ranges[] = {
+ regmap_reg_range(AXP20X_DATACACHE(0), AXP20X_IRQ5_STATE),
+ regmap_reg_range(AXP20X_DCDC_MODE, AXP20X_FG_RES),
+};
+
+static const struct regmap_range axp20x_volatile_ranges[] = {
+ regmap_reg_range(AXP20X_IRQ1_EN, AXP20X_IRQ5_STATE),
+};
+
+static const struct regmap_access_table axp20x_writeable_table = {
+ .yes_ranges = axp20x_writeable_ranges,
+ .n_yes_ranges = ARRAY_SIZE(axp20x_writeable_ranges),
+};
+
+static const struct regmap_access_table axp20x_volatile_table = {
+ .yes_ranges = axp20x_volatile_ranges,
+ .n_yes_ranges = ARRAY_SIZE(axp20x_volatile_ranges),
+};
+
+static struct resource axp20x_pek_resources[] = {
+ {
+ .name = "PEK_DBR",
+ .start = AXP20X_IRQ_PEK_RIS_EDGE,
+ .end = AXP20X_IRQ_PEK_RIS_EDGE,
+ .flags = IORESOURCE_IRQ,
+ }, {
+ .name = "PEK_DBF",
+ .start = AXP20X_IRQ_PEK_FAL_EDGE,
+ .end = AXP20X_IRQ_PEK_FAL_EDGE,
+ .flags = IORESOURCE_IRQ,
+ },
+};
+
+static const struct regmap_config axp20x_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+ .wr_table = &axp20x_writeable_table,
+ .volatile_table = &axp20x_volatile_table,
+ .max_register = AXP20X_FG_RES,
+ .cache_type = REGCACHE_RBTREE,
+};
+
+#define AXP20X_IRQ(_irq, _off, _mask) \
+ [AXP20X_IRQ_##_irq] = { .reg_offset = (_off), .mask = BIT(_mask) }
+
+static const struct regmap_irq axp20x_regmap_irqs[] = {
+ AXP20X_IRQ(ACIN_OVER_V, 0, 7),
+ AXP20X_IRQ(ACIN_PLUGIN, 0, 6),
+ AXP20X_IRQ(ACIN_REMOVAL, 0, 5),
+ AXP20X_IRQ(VBUS_OVER_V, 0, 4),
+ AXP20X_IRQ(VBUS_PLUGIN, 0, 3),
+ AXP20X_IRQ(VBUS_REMOVAL, 0, 2),
+ AXP20X_IRQ(VBUS_V_LOW, 0, 1),
+ AXP20X_IRQ(BATT_PLUGIN, 1, 7),
+ AXP20X_IRQ(BATT_REMOVAL, 1, 6),
+ AXP20X_IRQ(BATT_ENT_ACT_MODE, 1, 5),
+ AXP20X_IRQ(BATT_EXIT_ACT_MODE, 1, 4),
+ AXP20X_IRQ(CHARG, 1, 3),
+ AXP20X_IRQ(CHARG_DONE, 1, 2),
+ AXP20X_IRQ(BATT_TEMP_HIGH, 1, 1),
+ AXP20X_IRQ(BATT_TEMP_LOW, 1, 0),
+ AXP20X_IRQ(DIE_TEMP_HIGH, 2, 7),
+ AXP20X_IRQ(CHARG_I_LOW, 2, 6),
+ AXP20X_IRQ(DCDC1_V_LONG, 2, 5),
+ AXP20X_IRQ(DCDC2_V_LONG, 2, 4),
+ AXP20X_IRQ(DCDC3_V_LONG, 2, 3),
+ AXP20X_IRQ(PEK_SHORT, 2, 1),
+ AXP20X_IRQ(PEK_LONG, 2, 0),
+ AXP20X_IRQ(N_OE_PWR_ON, 3, 7),
+ AXP20X_IRQ(N_OE_PWR_OFF, 3, 6),
+ AXP20X_IRQ(VBUS_VALID, 3, 5),
+ AXP20X_IRQ(VBUS_NOT_VALID, 3, 4),
+ AXP20X_IRQ(VBUS_SESS_VALID, 3, 3),
+ AXP20X_IRQ(VBUS_SESS_END, 3, 2),
+ AXP20X_IRQ(LOW_PWR_LVL1, 3, 1),
+ AXP20X_IRQ(LOW_PWR_LVL2, 3, 0),
+ AXP20X_IRQ(TIMER, 4, 7),
+ AXP20X_IRQ(PEK_RIS_EDGE, 4, 6),
+ AXP20X_IRQ(PEK_FAL_EDGE, 4, 5),
+ AXP20X_IRQ(GPIO3_INPUT, 4, 3),
+ AXP20X_IRQ(GPIO2_INPUT, 4, 2),
+ AXP20X_IRQ(GPIO1_INPUT, 4, 1),
+ AXP20X_IRQ(GPIO0_INPUT, 4, 0),
+};
+
+static const struct of_device_id axp20x_of_match[] = {
+ { .compatible = "x-powers,axp202", .data = (void *) AXP202_ID },
+ { .compatible = "x-powers,axp209", .data = (void *) AXP209_ID },
+ { },
+};
+MODULE_DEVICE_TABLE(of, axp20x_of_match);
+
+/*
+ * This is useless for OF-enabled devices, but it is needed by I2C subsystem
+ */
+static const struct i2c_device_id axp20x_i2c_id[] = {
+ { },
+};
+MODULE_DEVICE_TABLE(i2c, axp20x_i2c_id);
+
+static const struct regmap_irq_chip axp20x_regmap_irq_chip = {
+ .name = "axp20x_irq_chip",
+ .status_base = AXP20X_IRQ1_STATE,
+ .ack_base = AXP20X_IRQ1_STATE,
+ .mask_base = AXP20X_IRQ1_EN,
+ .num_regs = 5,
+ .irqs = axp20x_regmap_irqs,
+ .num_irqs = ARRAY_SIZE(axp20x_regmap_irqs),
+ .mask_invert = true,
+ .init_ack_masked = true,
+};
+
+static const char * const axp20x_supplies[] = {
+ "acin",
+ "vin2",
+ "vin3",
+ "ldo24in",
+ "ldo3in",
+ "ldo5in",
+};
+
+static struct mfd_cell axp20x_cells[] = {
+ {
+ .name = "axp20x-pek",
+ .num_resources = ARRAY_SIZE(axp20x_pek_resources),
+ .resources = axp20x_pek_resources,
+ }, {
+ .name = "axp20x-regulator",
+ .parent_supplies = axp20x_supplies,
+ .num_parent_supplies = ARRAY_SIZE(axp20x_supplies),
+ },
+};
+
+static struct axp20x_dev *axp20x_pm_power_off;
+static void axp20x_power_off(void)
+{
+ regmap_write(axp20x_pm_power_off->regmap, AXP20X_OFF_CTRL,
+ AXP20X_OFF);
+}
+
+static int axp20x_i2c_probe(struct i2c_client *i2c,
+ const struct i2c_device_id *id)
+{
+ struct axp20x_dev *axp20x;
+ const struct of_device_id *of_id;
+ int ret;
+
+ axp20x = devm_kzalloc(&i2c->dev, sizeof(*axp20x), GFP_KERNEL);
+ if (!axp20x)
+ return -ENOMEM;
+
+ of_id = of_match_device(axp20x_of_match, &i2c->dev);
+ if (!of_id) {
+ dev_err(&i2c->dev, "Unable to setup AXP20X data\n");
+ return -ENODEV;
+ }
+ axp20x->variant = (long) of_id->data;
+
+ axp20x->i2c_client = i2c;
+ axp20x->dev = &i2c->dev;
+ dev_set_drvdata(axp20x->dev, axp20x);
+
+ axp20x->regmap = devm_regmap_init_i2c(i2c, &axp20x_regmap_config);
+ if (IS_ERR(axp20x->regmap)) {
+ ret = PTR_ERR(axp20x->regmap);
+ dev_err(&i2c->dev, "regmap init failed: %d\n", ret);
+ return ret;
+ }
+
+ ret = regmap_add_irq_chip(axp20x->regmap, i2c->irq,
+ IRQF_ONESHOT | IRQF_SHARED, -1,
+ &axp20x_regmap_irq_chip,
+ &axp20x->regmap_irqc);
+ if (ret) {
+ dev_err(&i2c->dev, "failed to add irq chip: %d\n", ret);
+ return ret;
+ }
+
+ ret = mfd_add_devices(axp20x->dev, -1, axp20x_cells,
+ ARRAY_SIZE(axp20x_cells), NULL, 0, NULL);
+
+ if (ret) {
+ dev_err(&i2c->dev, "failed to add MFD devices: %d\n", ret);
+ regmap_del_irq_chip(i2c->irq, axp20x->regmap_irqc);
+ return ret;
+ }
+
+ if (!pm_power_off) {
+ axp20x_pm_power_off = axp20x;
+ pm_power_off = axp20x_power_off;
+ }
+
+ dev_info(&i2c->dev, "AXP20X driver loaded\n");
+
+ return 0;
+}
+
+static int axp20x_i2c_remove(struct i2c_client *i2c)
+{
+ struct axp20x_dev *axp20x = i2c_get_clientdata(i2c);
+
+ if (axp20x == axp20x_pm_power_off) {
+ axp20x_pm_power_off = NULL;
+ pm_power_off = NULL;
+ }
+
+ mfd_remove_devices(axp20x->dev);
+ regmap_del_irq_chip(axp20x->i2c_client->irq, axp20x->regmap_irqc);
+
+ return 0;
+}
+
+static struct i2c_driver axp20x_i2c_driver = {
+ .driver = {
+ .name = "axp20x",
+ .owner = THIS_MODULE,
+ .of_match_table = of_match_ptr(axp20x_of_match),
+ },
+ .probe = axp20x_i2c_probe,
+ .remove = axp20x_i2c_remove,
+ .id_table = axp20x_i2c_id,
+};
+
+module_i2c_driver(axp20x_i2c_driver);
+
+MODULE_DESCRIPTION("PMIC MFD core driver for AXP20X");
+MODULE_AUTHOR("Carlo Caione <carlo@caione.org>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/mfd/bcm590xx.c b/drivers/mfd/bcm590xx.c
index 43cba1a..e334de0 100644
--- a/drivers/mfd/bcm590xx.c
+++ b/drivers/mfd/bcm590xx.c
@@ -96,6 +96,12 @@
return ret;
}
+static int bcm590xx_i2c_remove(struct i2c_client *i2c)
+{
+ mfd_remove_devices(&i2c->dev);
+ return 0;
+}
+
static const struct of_device_id bcm590xx_of_match[] = {
{ .compatible = "brcm,bcm59056" },
{ }
@@ -115,6 +121,7 @@
.of_match_table = of_match_ptr(bcm590xx_of_match),
},
.probe = bcm590xx_i2c_probe,
+ .remove = bcm590xx_i2c_remove,
.id_table = bcm590xx_i2c_id,
};
module_i2c_driver(bcm590xx_i2c_driver);
@@ -122,4 +129,4 @@
MODULE_AUTHOR("Matt Porter <mporter@linaro.org>");
MODULE_DESCRIPTION("BCM590xx multi-function driver");
MODULE_LICENSE("GPL v2");
-MODULE_ALIAS("platform:bcm590xx");
+MODULE_ALIAS("i2c:bcm590xx");
diff --git a/drivers/mfd/cros_ec.c b/drivers/mfd/cros_ec.c
index 783fe2e..38fe9bf 100644
--- a/drivers/mfd/cros_ec.c
+++ b/drivers/mfd/cros_ec.c
@@ -30,7 +30,7 @@
uint8_t *out;
int csum, i;
- BUG_ON(msg->out_len > EC_HOST_PARAM_SIZE);
+ BUG_ON(msg->out_len > EC_PROTO2_MAX_PARAM_SIZE);
out = ec_dev->dout;
out[0] = EC_CMD_VERSION0 + msg->version;
out[1] = msg->cmd;
@@ -90,6 +90,11 @@
.id = 1,
.of_compatible = "google,cros-ec-keyb",
},
+ {
+ .name = "cros-ec-i2c-tunnel",
+ .id = 2,
+ .of_compatible = "google,cros-ec-i2c-tunnel",
+ },
};
int cros_ec_register(struct cros_ec_device *ec_dev)
@@ -184,3 +189,6 @@
EXPORT_SYMBOL(cros_ec_resume);
#endif
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("ChromeOS EC core driver");
diff --git a/drivers/mfd/cros_ec_spi.c b/drivers/mfd/cros_ec_spi.c
index 84af8d7..0b8d328 100644
--- a/drivers/mfd/cros_ec_spi.c
+++ b/drivers/mfd/cros_ec_spi.c
@@ -39,14 +39,22 @@
#define EC_MSG_PREAMBLE_COUNT 32
/*
- * We must get a response from the EC in 5ms. This is a very long
- * time, but the flash write command can take 2-3ms. The EC command
- * processing is currently not very fast (about 500us). We could
- * look at speeding this up and making the flash write command a
- * 'slow' command, requiring a GET_STATUS wait loop, like flash
- * erase.
- */
-#define EC_MSG_DEADLINE_MS 5
+ * Allow for a long time for the EC to respond. We support i2c
+ * tunneling and support fairly long messages for the tunnel (249
+ * bytes long at the moment). If we're talking to a 100 kHz device
+ * on the other end and need to transfer ~256 bytes, then we need:
+ * 10 us/bit * ~10 bits/byte * ~256 bytes = ~25ms
+ *
+ * We'll wait 4 times that to handle clock stretching and other
+ * paranoia.
+ *
+ * It's pretty unlikely that we'll really see a 249 byte tunnel in
+ * anything other than testing. If this was more common we might
+ * consider having slow commands like this require a GET_STATUS
+ * wait loop. The 'flash write' command would be another candidate
+ * for this, clocking in at 2-3ms.
+ */
+#define EC_MSG_DEADLINE_MS 100
/*
* Time between raising the SPI chip select (for the end of a
@@ -65,11 +73,13 @@
* if no record
* @end_of_msg_delay: used to set the delay_usecs on the spi_transfer that
* is sent when we want to turn off CS at the end of a transaction.
+ * @lock: mutex to ensure only one user of cros_ec_command_spi_xfer at a time
*/
struct cros_ec_spi {
struct spi_device *spi;
s64 last_transfer_ns;
unsigned int end_of_msg_delay;
+ struct mutex lock;
};
static void debug_packet(struct device *dev, const char *name, u8 *ptr,
@@ -111,7 +121,9 @@
/* Receive data until we see the header byte */
deadline = jiffies + msecs_to_jiffies(EC_MSG_DEADLINE_MS);
- do {
+ while (true) {
+ unsigned long start_jiffies = jiffies;
+
memset(&trans, 0, sizeof(trans));
trans.cs_change = 1;
trans.rx_buf = ptr = ec_dev->din;
@@ -132,12 +144,19 @@
break;
}
}
+ if (ptr != end)
+ break;
- if (time_after(jiffies, deadline)) {
+ /*
+ * Use the time at the start of the loop as a timeout. This
+ * gives us one last shot at getting the transfer and is useful
+ * in case we got context switched out for a while.
+ */
+ if (time_after(start_jiffies, deadline)) {
dev_warn(ec_dev->dev, "EC failed to respond in time\n");
return -ETIMEDOUT;
}
- } while (ptr == end);
+ }
/*
* ptr now points to the header byte. Copy any valid data to the
@@ -208,6 +227,13 @@
int ret = 0, final_ret;
struct timespec ts;
+ /*
+ * We have the shared ec_dev buffer plus we do lots of separate spi_sync
+ * calls, so we need to make sure only one person is using this at a
+ * time.
+ */
+ mutex_lock(&ec_spi->lock);
+
len = cros_ec_prepare_tx(ec_dev, ec_msg);
dev_dbg(ec_dev->dev, "prepared, len=%d\n", len);
@@ -219,7 +245,7 @@
ktime_get_ts(&ts);
delay = timespec_to_ns(&ts) - ec_spi->last_transfer_ns;
if (delay < EC_SPI_RECOVERY_TIME_NS)
- ndelay(delay);
+ ndelay(EC_SPI_RECOVERY_TIME_NS - delay);
}
/* Transmit phase - send our message */
@@ -260,7 +286,7 @@
ret = final_ret;
if (ret < 0) {
dev_err(ec_dev->dev, "spi transfer failed: %d\n", ret);
- return ret;
+ goto exit;
}
/* check response error code */
@@ -269,14 +295,16 @@
dev_warn(ec_dev->dev, "command 0x%02x returned an error %d\n",
ec_msg->cmd, ptr[0]);
debug_packet(ec_dev->dev, "in_err", ptr, len);
- return -EINVAL;
+ ret = -EINVAL;
+ goto exit;
}
len = ptr[1];
sum = ptr[0] + ptr[1];
if (len > ec_msg->in_len) {
dev_err(ec_dev->dev, "packet too long (%d bytes, expected %d)",
len, ec_msg->in_len);
- return -ENOSPC;
+ ret = -ENOSPC;
+ goto exit;
}
/* copy response packet payload and compute checksum */
@@ -293,10 +321,14 @@
dev_err(ec_dev->dev,
"bad packet checksum, expected %02x, got %02x\n",
sum, ptr[len + 2]);
- return -EBADMSG;
+ ret = -EBADMSG;
+ goto exit;
}
- return 0;
+ ret = 0;
+exit:
+ mutex_unlock(&ec_spi->lock);
+ return ret;
}
static void cros_ec_spi_dt_probe(struct cros_ec_spi *ec_spi, struct device *dev)
@@ -327,6 +359,7 @@
if (ec_spi == NULL)
return -ENOMEM;
ec_spi->spi = spi;
+ mutex_init(&ec_spi->lock);
ec_dev = devm_kzalloc(dev, sizeof(*ec_dev), GFP_KERNEL);
if (!ec_dev)
return -ENOMEM;
diff --git a/drivers/mfd/db8500-prcmu.c b/drivers/mfd/db8500-prcmu.c
index b11fdd6..193cf16 100644
--- a/drivers/mfd/db8500-prcmu.c
+++ b/drivers/mfd/db8500-prcmu.c
@@ -2300,9 +2300,6 @@
if (!wait_for_completion_timeout(&mb0_transfer.ac_wake_work,
msecs_to_jiffies(5000))) {
-#if defined(CONFIG_DBX500_PRCMU_DEBUG)
- db8500_prcmu_debug_dump(__func__, true, true);
-#endif
pr_crit("prcmu: %s timed out (5 s) waiting for a reply.\n",
__func__);
ret = -EFAULT;
@@ -3112,7 +3109,7 @@
{
struct device_node *np;
struct resource ab8500_resource;
- struct mfd_cell ab8500_cell = {
+ const struct mfd_cell ab8500_cell = {
.name = "ab8500-core",
.of_compatible = "stericsson,ab8500",
.id = AB8500_VERSION_AB8500,
diff --git a/drivers/mfd/ipaq-micro.c b/drivers/mfd/ipaq-micro.c
new file mode 100644
index 0000000..7e50fe0
--- /dev/null
+++ b/drivers/mfd/ipaq-micro.c
@@ -0,0 +1,482 @@
+/*
+ * Compaq iPAQ h3xxx Atmel microcontroller companion support
+ *
+ * This is an Atmel AT90LS8535 with a special flashed-in firmware that
+ * implements the special protocol used by this driver.
+ *
+ * based on previous kernel 2.4 version by Andrew Christian
+ * Author : Alessandro Gardich <gremlin@gremlin.it>
+ * Author : Dmitry Artamonow <mad_soft@inbox.ru>
+ * Author : Linus Walleij <linus.walleij@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/pm.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/io.h>
+#include <linux/mfd/core.h>
+#include <linux/mfd/ipaq-micro.h>
+#include <linux/string.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+
+#include <mach/hardware.h>
+
+static void ipaq_micro_trigger_tx(struct ipaq_micro *micro)
+{
+ struct ipaq_micro_txdev *tx = µ->tx;
+ struct ipaq_micro_msg *msg = micro->msg;
+ int i, bp;
+ u8 checksum;
+ u32 val;
+
+ bp = 0;
+ tx->buf[bp++] = CHAR_SOF;
+
+ checksum = ((msg->id & 0x0f) << 4) | (msg->tx_len & 0x0f);
+ tx->buf[bp++] = checksum;
+
+ for (i = 0; i < msg->tx_len; i++) {
+ tx->buf[bp++] = msg->tx_data[i];
+ checksum += msg->tx_data[i];
+ }
+
+ tx->buf[bp++] = checksum;
+ tx->len = bp;
+ tx->index = 0;
+ print_hex_dump(KERN_DEBUG, "data: ", DUMP_PREFIX_OFFSET, 16, 1,
+ tx->buf, tx->len, true);
+
+ /* Enable interrupt */
+ val = readl(micro->base + UTCR3);
+ val |= UTCR3_TIE;
+ writel(val, micro->base + UTCR3);
+}
+
+int ipaq_micro_tx_msg(struct ipaq_micro *micro, struct ipaq_micro_msg *msg)
+{
+ unsigned long flags;
+
+ dev_dbg(micro->dev, "TX msg: %02x, %d bytes\n", msg->id, msg->tx_len);
+
+ spin_lock_irqsave(µ->lock, flags);
+ if (micro->msg) {
+ list_add_tail(&msg->node, µ->queue);
+ spin_unlock_irqrestore(µ->lock, flags);
+ return 0;
+ }
+ micro->msg = msg;
+ ipaq_micro_trigger_tx(micro);
+ spin_unlock_irqrestore(µ->lock, flags);
+ return 0;
+}
+EXPORT_SYMBOL(ipaq_micro_tx_msg);
+
+static void micro_rx_msg(struct ipaq_micro *micro, u8 id, int len, u8 *data)
+{
+ int i;
+
+ dev_dbg(micro->dev, "RX msg: %02x, %d bytes\n", id, len);
+
+ spin_lock(µ->lock);
+ switch (id) {
+ case MSG_VERSION:
+ case MSG_EEPROM_READ:
+ case MSG_EEPROM_WRITE:
+ case MSG_BACKLIGHT:
+ case MSG_NOTIFY_LED:
+ case MSG_THERMAL_SENSOR:
+ case MSG_BATTERY:
+ /* Handle synchronous messages */
+ if (micro->msg && micro->msg->id == id) {
+ struct ipaq_micro_msg *msg = micro->msg;
+
+ memcpy(msg->rx_data, data, len);
+ msg->rx_len = len;
+ complete(µ->msg->ack);
+ if (!list_empty(µ->queue)) {
+ micro->msg = list_entry(micro->queue.next,
+ struct ipaq_micro_msg,
+ node);
+ list_del_init(µ->msg->node);
+ ipaq_micro_trigger_tx(micro);
+ } else
+ micro->msg = NULL;
+ dev_dbg(micro->dev, "OK RX message 0x%02x\n", id);
+ } else {
+ dev_err(micro->dev,
+ "out of band RX message 0x%02x\n", id);
+ if(!micro->msg)
+ dev_info(micro->dev, "no message queued\n");
+ else
+ dev_info(micro->dev, "expected message %02x\n",
+ micro->msg->id);
+ }
+ break;
+ case MSG_KEYBOARD:
+ if (micro->key)
+ micro->key(micro->key_data, len, data);
+ else
+ dev_dbg(micro->dev, "key message ignored, no handle \n");
+ break;
+ case MSG_TOUCHSCREEN:
+ if (micro->ts)
+ micro->ts(micro->ts_data, len, data);
+ else
+ dev_dbg(micro->dev, "touchscreen message ignored, no handle \n");
+ break;
+ default:
+ dev_err(micro->dev,
+ "unknown msg %d [%d] ", id, len);
+ for (i = 0; i < len; ++i)
+ pr_cont("0x%02x ", data[i]);
+ pr_cont("\n");
+ }
+ spin_unlock(µ->lock);
+}
+
+static void micro_process_char(struct ipaq_micro *micro, u8 ch)
+{
+ struct ipaq_micro_rxdev *rx = µ->rx;
+
+ switch (rx->state) {
+ case STATE_SOF: /* Looking for SOF */
+ if (ch == CHAR_SOF)
+ rx->state = STATE_ID; /* Next byte is the id and len */
+ break;
+ case STATE_ID: /* Looking for id and len byte */
+ rx->id = (ch & 0xf0) >> 4 ;
+ rx->len = (ch & 0x0f);
+ rx->index = 0;
+ rx->chksum = ch;
+ rx->state = (rx->len > 0) ? STATE_DATA : STATE_CHKSUM;
+ break;
+ case STATE_DATA: /* Looking for 'len' data bytes */
+ rx->chksum += ch;
+ rx->buf[rx->index] = ch;
+ if (++rx->index == rx->len)
+ rx->state = STATE_CHKSUM;
+ break;
+ case STATE_CHKSUM: /* Looking for the checksum */
+ if (ch == rx->chksum)
+ micro_rx_msg(micro, rx->id, rx->len, rx->buf);
+ rx->state = STATE_SOF;
+ break;
+ }
+}
+
+static void micro_rx_chars(struct ipaq_micro *micro)
+{
+ u32 status, ch;
+
+ while ((status = readl(micro->base + UTSR1)) & UTSR1_RNE) {
+ ch = readl(micro->base + UTDR);
+ if (status & UTSR1_PRE)
+ dev_err(micro->dev, "rx: parity error\n");
+ else if (status & UTSR1_FRE)
+ dev_err(micro->dev, "rx: framing error\n");
+ else if (status & UTSR1_ROR)
+ dev_err(micro->dev, "rx: overrun error\n");
+ micro_process_char(micro, ch);
+ }
+}
+
+static void ipaq_micro_get_version(struct ipaq_micro *micro)
+{
+ struct ipaq_micro_msg msg = {
+ .id = MSG_VERSION,
+ };
+
+ ipaq_micro_tx_msg_sync(micro, &msg);
+ if (msg.rx_len == 4) {
+ memcpy(micro->version, msg.rx_data, 4);
+ micro->version[4] = '\0';
+ } else if (msg.rx_len == 9) {
+ memcpy(micro->version, msg.rx_data, 4);
+ micro->version[4] = '\0';
+ /* Bytes 4-7 are "pack", byte 8 is "boot type" */
+ } else {
+ dev_err(micro->dev,
+ "illegal version message %d bytes\n", msg.rx_len);
+ }
+}
+
+static void ipaq_micro_eeprom_read(struct ipaq_micro *micro,
+ u8 address, u8 len, u8 *data)
+{
+ struct ipaq_micro_msg msg = {
+ .id = MSG_EEPROM_READ,
+ };
+ u8 i;
+
+ for (i = 0; i < len; i++) {
+ msg.tx_data[0] = address + i;
+ msg.tx_data[1] = 1;
+ msg.tx_len = 2;
+ ipaq_micro_tx_msg_sync(micro, &msg);
+ memcpy(data + (i * 2), msg.rx_data, 2);
+ }
+}
+
+static char *ipaq_micro_str(u8 *wchar, u8 len)
+{
+ char retstr[256];
+ u8 i;
+
+ for (i = 0; i < len / 2; i++)
+ retstr[i] = wchar[i * 2];
+ return kstrdup(retstr, GFP_KERNEL);
+}
+
+static u16 ipaq_micro_to_u16(u8 *data)
+{
+ return data[1] << 8 | data[0];
+}
+
+static void ipaq_micro_eeprom_dump(struct ipaq_micro *micro)
+{
+ u8 dump[256];
+ char *str;
+
+ ipaq_micro_eeprom_read(micro, 0, 128, dump);
+ str = ipaq_micro_str(dump, 10);
+ if (str) {
+ dev_info(micro->dev, "HM version %s\n", str);
+ kfree(str);
+ }
+ str = ipaq_micro_str(dump+10, 40);
+ if (str) {
+ dev_info(micro->dev, "serial number: %s\n", str);
+ /* Feed the random pool with this */
+ add_device_randomness(str, strlen(str));
+ kfree(str);
+ }
+ str = ipaq_micro_str(dump+50, 20);
+ if (str) {
+ dev_info(micro->dev, "module ID: %s\n", str);
+ kfree(str);
+ }
+ str = ipaq_micro_str(dump+70, 10);
+ if (str) {
+ dev_info(micro->dev, "product revision: %s\n", str);
+ kfree(str);
+ }
+ dev_info(micro->dev, "product ID: %u\n", ipaq_micro_to_u16(dump+80));
+ dev_info(micro->dev, "frame rate: %u fps\n",
+ ipaq_micro_to_u16(dump+82));
+ dev_info(micro->dev, "page mode: %u\n", ipaq_micro_to_u16(dump+84));
+ dev_info(micro->dev, "country ID: %u\n", ipaq_micro_to_u16(dump+86));
+ dev_info(micro->dev, "color display: %s\n",
+ ipaq_micro_to_u16(dump+88) ? "yes" : "no");
+ dev_info(micro->dev, "ROM size: %u MiB\n", ipaq_micro_to_u16(dump+90));
+ dev_info(micro->dev, "RAM size: %u KiB\n", ipaq_micro_to_u16(dump+92));
+ dev_info(micro->dev, "screen: %u x %u\n",
+ ipaq_micro_to_u16(dump+94), ipaq_micro_to_u16(dump+96));
+ print_hex_dump(KERN_DEBUG, "eeprom: ", DUMP_PREFIX_OFFSET, 16, 1,
+ dump, 256, true);
+
+}
+
+static void micro_tx_chars(struct ipaq_micro *micro)
+{
+ struct ipaq_micro_txdev *tx = µ->tx;
+ u32 val;
+
+ while ((tx->index < tx->len) &&
+ (readl(micro->base + UTSR1) & UTSR1_TNF)) {
+ writel(tx->buf[tx->index], micro->base + UTDR);
+ tx->index++;
+ }
+
+ /* Stop interrupts */
+ val = readl(micro->base + UTCR3);
+ val &= ~UTCR3_TIE;
+ writel(val, micro->base + UTCR3);
+}
+
+static void micro_reset_comm(struct ipaq_micro *micro)
+{
+ struct ipaq_micro_rxdev *rx = µ->rx;
+ u32 val;
+
+ if (micro->msg)
+ complete(µ->msg->ack);
+
+ /* Initialize Serial channel protocol frame */
+ rx->state = STATE_SOF; /* Reset the state machine */
+
+ /* Set up interrupts */
+ writel(0x01, micro->sdlc + 0x0); /* Select UART mode */
+
+ /* Clean up CR3 */
+ writel(0x0, micro->base + UTCR3);
+
+ /* Format: 8N1 */
+ writel(UTCR0_8BitData | UTCR0_1StpBit, micro->base + UTCR0);
+
+ /* Baud rate: 115200 */
+ writel(0x0, micro->base + UTCR1);
+ writel(0x1, micro->base + UTCR2);
+
+ /* Clear SR0 */
+ writel(0xff, micro->base + UTSR0);
+
+ /* Enable RX int, disable TX int */
+ writel(UTCR3_TXE | UTCR3_RXE | UTCR3_RIE, micro->base + UTCR3);
+ val = readl(micro->base + UTCR3);
+ val &= ~UTCR3_TIE;
+ writel(val, micro->base + UTCR3);
+}
+
+static irqreturn_t micro_serial_isr(int irq, void *dev_id)
+{
+ struct ipaq_micro *micro = dev_id;
+ struct ipaq_micro_txdev *tx = µ->tx;
+ u32 status;
+
+ status = readl(micro->base + UTSR0);
+ do {
+ if (status & (UTSR0_RID | UTSR0_RFS)) {
+ if (status & UTSR0_RID)
+ /* Clear the Receiver IDLE bit */
+ writel(UTSR0_RID, micro->base + UTSR0);
+ micro_rx_chars(micro);
+ }
+
+ /* Clear break bits */
+ if (status & (UTSR0_RBB | UTSR0_REB))
+ writel(status & (UTSR0_RBB | UTSR0_REB),
+ micro->base + UTSR0);
+
+ if (status & UTSR0_TFS)
+ micro_tx_chars(micro);
+
+ status = readl(micro->base + UTSR0);
+
+ } while (((tx->index < tx->len) && (status & UTSR0_TFS)) ||
+ (status & (UTSR0_RFS | UTSR0_RID)));
+
+ return IRQ_HANDLED;
+}
+
+static const struct mfd_cell micro_cells[] = {
+ { .name = "ipaq-micro-backlight", },
+ { .name = "ipaq-micro-battery", },
+ { .name = "ipaq-micro-keys", },
+ { .name = "ipaq-micro-ts", },
+ { .name = "ipaq-micro-leds", },
+};
+
+static int micro_resume(struct device *dev)
+{
+ struct ipaq_micro *micro = dev_get_drvdata(dev);
+
+ micro_reset_comm(micro);
+ mdelay(10);
+
+ return 0;
+}
+
+static int micro_probe(struct platform_device *pdev)
+{
+ struct ipaq_micro *micro;
+ struct resource *res;
+ int ret;
+ int irq;
+
+ micro = devm_kzalloc(&pdev->dev, sizeof(*micro), GFP_KERNEL);
+ if (!micro)
+ return -ENOMEM;
+
+ micro->dev = &pdev->dev;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res)
+ return -EINVAL;
+
+ micro->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(micro->base))
+ return PTR_ERR(micro->base);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+ if (!res)
+ return -EINVAL;
+
+ micro->sdlc = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(micro->sdlc))
+ return PTR_ERR(micro->sdlc);
+
+ micro_reset_comm(micro);
+
+ irq = platform_get_irq(pdev, 0);
+ if (!irq)
+ return -EINVAL;
+ ret = devm_request_irq(&pdev->dev, irq, micro_serial_isr,
+ IRQF_SHARED, "ipaq-micro",
+ micro);
+ if (ret) {
+ dev_err(&pdev->dev, "unable to grab serial port IRQ\n");
+ return ret;
+ } else
+ dev_info(&pdev->dev, "grabbed serial port IRQ\n");
+
+ spin_lock_init(µ->lock);
+ INIT_LIST_HEAD(µ->queue);
+ platform_set_drvdata(pdev, micro);
+
+ ret = mfd_add_devices(&pdev->dev, pdev->id, micro_cells,
+ ARRAY_SIZE(micro_cells), NULL, 0, NULL);
+ if (ret) {
+ dev_err(&pdev->dev, "error adding MFD cells");
+ return ret;
+ }
+
+ /* Check version */
+ ipaq_micro_get_version(micro);
+ dev_info(&pdev->dev, "Atmel micro ASIC version %s\n", micro->version);
+ ipaq_micro_eeprom_dump(micro);
+
+ return 0;
+}
+
+static int micro_remove(struct platform_device *pdev)
+{
+ struct ipaq_micro *micro = platform_get_drvdata(pdev);
+ u32 val;
+
+ mfd_remove_devices(&pdev->dev);
+
+ val = readl(micro->base + UTCR3);
+ val &= ~(UTCR3_RXE | UTCR3_RIE); /* disable receive interrupt */
+ val &= ~(UTCR3_TXE | UTCR3_TIE); /* disable transmit interrupt */
+ writel(val, micro->base + UTCR3);
+
+ return 0;
+}
+
+static const struct dev_pm_ops micro_dev_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(NULL, micro_resume)
+};
+
+static struct platform_driver micro_device_driver = {
+ .driver = {
+ .name = "ipaq-h3xxx-micro",
+ .pm = µ_dev_pm_ops,
+ },
+ .probe = micro_probe,
+ .remove = micro_remove,
+ /* .shutdown = micro_suspend, // FIXME */
+};
+module_platform_driver(micro_device_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("driver for iPAQ Atmel micro core and backlight");
diff --git a/drivers/mfd/kempld-core.c b/drivers/mfd/kempld-core.c
index 0769260..f7ff018 100644
--- a/drivers/mfd/kempld-core.c
+++ b/drivers/mfd/kempld-core.c
@@ -86,7 +86,7 @@
KEMPLD_UART,
};
-static struct mfd_cell kempld_devs[] = {
+static const struct mfd_cell kempld_devs[] = {
[KEMPLD_I2C] = {
.name = "kempld-i2c",
},
@@ -288,9 +288,38 @@
*/
static int kempld_get_info(struct kempld_device_data *pld)
{
+ int ret;
struct kempld_platform_data *pdata = dev_get_platdata(pld->dev);
+ char major, minor;
- return pdata->get_info(pld);
+ ret = pdata->get_info(pld);
+ if (ret)
+ return ret;
+
+ /* The Kontron PLD firmware version string has the following format:
+ * Pwxy.zzzz
+ * P: Fixed
+ * w: PLD number - 1 hex digit
+ * x: Major version - 1 alphanumerical digit (0-9A-V)
+ * y: Minor version - 1 alphanumerical digit (0-9A-V)
+ * zzzz: Build number - 4 zero padded hex digits */
+
+ if (pld->info.major < 10)
+ major = pld->info.major + '0';
+ else
+ major = (pld->info.major - 10) + 'A';
+ if (pld->info.minor < 10)
+ minor = pld->info.minor + '0';
+ else
+ minor = (pld->info.minor - 10) + 'A';
+
+ ret = scnprintf(pld->info.version, sizeof(pld->info.version),
+ "P%X%c%c.%04X", pld->info.number, major, minor,
+ pld->info.buildnr);
+ if (ret < 0)
+ return ret;
+
+ return 0;
}
/*
@@ -307,9 +336,71 @@
return pdata->register_cells(pld);
}
+static const char *kempld_get_type_string(struct kempld_device_data *pld)
+{
+ const char *version_type;
+
+ switch (pld->info.type) {
+ case 0:
+ version_type = "release";
+ break;
+ case 1:
+ version_type = "debug";
+ break;
+ case 2:
+ version_type = "custom";
+ break;
+ default:
+ version_type = "unspecified";
+ break;
+ }
+
+ return version_type;
+}
+
+static ssize_t kempld_version_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct kempld_device_data *pld = dev_get_drvdata(dev);
+
+ return scnprintf(buf, PAGE_SIZE, "%s\n", pld->info.version);
+}
+
+static ssize_t kempld_specification_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct kempld_device_data *pld = dev_get_drvdata(dev);
+
+ return scnprintf(buf, PAGE_SIZE, "%d.%d\n", pld->info.spec_major,
+ pld->info.spec_minor);
+}
+
+static ssize_t kempld_type_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct kempld_device_data *pld = dev_get_drvdata(dev);
+
+ return scnprintf(buf, PAGE_SIZE, "%s\n", kempld_get_type_string(pld));
+}
+
+static DEVICE_ATTR(pld_version, S_IRUGO, kempld_version_show, NULL);
+static DEVICE_ATTR(pld_specification, S_IRUGO, kempld_specification_show,
+ NULL);
+static DEVICE_ATTR(pld_type, S_IRUGO, kempld_type_show, NULL);
+
+static struct attribute *pld_attributes[] = {
+ &dev_attr_pld_version.attr,
+ &dev_attr_pld_specification.attr,
+ &dev_attr_pld_type.attr,
+ NULL
+};
+
+static const struct attribute_group pld_attr_group = {
+ .attrs = pld_attributes,
+};
+
static int kempld_detect_device(struct kempld_device_data *pld)
{
- char *version_type;
u8 index_reg;
int ret;
@@ -335,27 +426,19 @@
if (ret)
return ret;
- switch (pld->info.type) {
- case 0:
- version_type = "release";
- break;
- case 1:
- version_type = "debug";
- break;
- case 2:
- version_type = "custom";
- break;
- default:
- version_type = "unspecified";
- }
+ dev_info(pld->dev, "Found Kontron PLD - %s (%s), spec %d.%d\n",
+ pld->info.version, kempld_get_type_string(pld),
+ pld->info.spec_major, pld->info.spec_minor);
- dev_info(pld->dev, "Found Kontron PLD %d\n", pld->info.number);
- dev_info(pld->dev, "%s version %d.%d build %d, specification %d.%d\n",
- version_type, pld->info.major, pld->info.minor,
- pld->info.buildnr, pld->info.spec_major,
- pld->info.spec_minor);
+ ret = sysfs_create_group(&pld->dev->kobj, &pld_attr_group);
+ if (ret)
+ return ret;
- return kempld_register_cells(pld);
+ ret = kempld_register_cells(pld);
+ if (ret)
+ sysfs_remove_group(&pld->dev->kobj, &pld_attr_group);
+
+ return ret;
}
static int kempld_probe(struct platform_device *pdev)
@@ -399,6 +482,8 @@
struct kempld_device_data *pld = platform_get_drvdata(pdev);
struct kempld_platform_data *pdata = dev_get_platdata(pld->dev);
+ sysfs_remove_group(&pld->dev->kobj, &pld_attr_group);
+
mfd_remove_devices(&pdev->dev);
pdata->release_hardware_mutex(pld);
diff --git a/drivers/mfd/lp3943.c b/drivers/mfd/lp3943.c
index e322268..335b930 100644
--- a/drivers/mfd/lp3943.c
+++ b/drivers/mfd/lp3943.c
@@ -62,7 +62,7 @@
{ LP3943_REG_MUX3, 0xC0, 6 },
};
-static struct mfd_cell lp3943_devs[] = {
+static const struct mfd_cell lp3943_devs[] = {
{
.name = "lp3943-pwm",
.of_compatible = "ti,lp3943-pwm",
diff --git a/drivers/mfd/lpc_ich.c b/drivers/mfd/lpc_ich.c
index 3f10ea3..7d8482f 100644
--- a/drivers/mfd/lpc_ich.c
+++ b/drivers/mfd/lpc_ich.c
@@ -488,6 +488,7 @@
[LPC_PPT] = {
.name = "Panther Point",
.iTCO_version = 2,
+ .gpio_version = ICH_V5_GPIO,
},
[LPC_LPT] = {
.name = "Lynx Point",
diff --git a/drivers/mfd/max14577.c b/drivers/mfd/max14577.c
index 484d372..4a5e885 100644
--- a/drivers/mfd/max14577.c
+++ b/drivers/mfd/max14577.c
@@ -26,7 +26,7 @@
#include <linux/mfd/max14577.h>
#include <linux/mfd/max14577-private.h>
-static struct mfd_cell max14577_devs[] = {
+static const struct mfd_cell max14577_devs[] = {
{
.name = "max14577-muic",
.of_compatible = "maxim,max14577-muic",
@@ -38,7 +38,7 @@
{ .name = "max14577-charger", },
};
-static struct mfd_cell max77836_devs[] = {
+static const struct mfd_cell max77836_devs[] = {
{
.name = "max77836-muic",
.of_compatible = "maxim,max77836-muic",
@@ -57,7 +57,7 @@
},
};
-static struct of_device_id max14577_dt_match[] = {
+static const struct of_device_id max14577_dt_match[] = {
{
.compatible = "maxim,max14577",
.data = (void *)MAXIM_DEVICE_TYPE_MAX14577,
@@ -292,7 +292,7 @@
struct device_node *np = i2c->dev.of_node;
int ret = 0;
const struct regmap_irq_chip *irq_chip;
- struct mfd_cell *mfd_devs;
+ const struct mfd_cell *mfd_devs;
unsigned int mfd_devs_size;
int irq_flags;
@@ -331,7 +331,8 @@
of_id = of_match_device(max14577_dt_match, &i2c->dev);
if (of_id)
- max14577->dev_type = (unsigned int)of_id->data;
+ max14577->dev_type =
+ (enum maxim_device_type)of_id->data;
} else {
max14577->dev_type = id->driver_data;
}
@@ -414,20 +415,18 @@
struct i2c_client *i2c = container_of(dev, struct i2c_client, dev);
struct max14577 *max14577 = i2c_get_clientdata(i2c);
- if (device_may_wakeup(dev)) {
+ if (device_may_wakeup(dev))
enable_irq_wake(max14577->irq);
- /*
- * MUIC IRQ must be disabled during suspend if this is
- * a wake up source because it will be handled before
- * resuming I2C.
- *
- * When device is woken up from suspend (e.g. by ADC change),
- * an interrupt occurs before resuming I2C bus controller.
- * Interrupt handler tries to read registers but this read
- * will fail because I2C is still suspended.
- */
- disable_irq(max14577->irq);
- }
+ /*
+ * MUIC IRQ must be disabled during suspend because if it happens
+ * while suspended it will be handled before resuming I2C.
+ *
+ * When device is woken up from suspend (e.g. by ADC change),
+ * an interrupt occurs before resuming I2C bus controller.
+ * Interrupt handler tries to read registers but this read
+ * will fail because I2C is still suspended.
+ */
+ disable_irq(max14577->irq);
return 0;
}
@@ -437,10 +436,9 @@
struct i2c_client *i2c = container_of(dev, struct i2c_client, dev);
struct max14577 *max14577 = i2c_get_clientdata(i2c);
- if (device_may_wakeup(dev)) {
+ if (device_may_wakeup(dev))
disable_irq_wake(max14577->irq);
- enable_irq(max14577->irq);
- }
+ enable_irq(max14577->irq);
return 0;
}
diff --git a/drivers/mfd/max77686.c b/drivers/mfd/max77686.c
index e5fce76..ce869ac 100644
--- a/drivers/mfd/max77686.c
+++ b/drivers/mfd/max77686.c
@@ -47,7 +47,7 @@
};
#ifdef CONFIG_OF
-static struct of_device_id max77686_pmic_dt_match[] = {
+static const struct of_device_id max77686_pmic_dt_match[] = {
{.compatible = "maxim,max77686", .data = NULL},
{},
};
diff --git a/drivers/mfd/max77693.c b/drivers/mfd/max77693.c
index c5535f0..7e05428 100644
--- a/drivers/mfd/max77693.c
+++ b/drivers/mfd/max77693.c
@@ -243,7 +243,7 @@
};
#ifdef CONFIG_OF
-static struct of_device_id max77693_dt_match[] = {
+static const struct of_device_id max77693_dt_match[] = {
{ .compatible = "maxim,max77693" },
{},
};
diff --git a/drivers/mfd/max8907.c b/drivers/mfd/max8907.c
index 0774031..232749c 100644
--- a/drivers/mfd/max8907.c
+++ b/drivers/mfd/max8907.c
@@ -305,7 +305,7 @@
}
#ifdef CONFIG_OF
-static struct of_device_id max8907_of_match[] = {
+static const struct of_device_id max8907_of_match[] = {
{ .compatible = "maxim,max8907" },
{ },
};
diff --git a/drivers/mfd/max8997.c b/drivers/mfd/max8997.c
index 8cf7a01..595364e 100644
--- a/drivers/mfd/max8997.c
+++ b/drivers/mfd/max8997.c
@@ -51,7 +51,7 @@
};
#ifdef CONFIG_OF
-static struct of_device_id max8997_pmic_dt_match[] = {
+static const struct of_device_id max8997_pmic_dt_match[] = {
{ .compatible = "maxim,max8997-pmic", .data = (void *)TYPE_MAX8997 },
{},
};
diff --git a/drivers/mfd/max8998.c b/drivers/mfd/max8998.c
index 592db06..a37cb74 100644
--- a/drivers/mfd/max8998.c
+++ b/drivers/mfd/max8998.c
@@ -132,7 +132,7 @@
EXPORT_SYMBOL(max8998_update_reg);
#ifdef CONFIG_OF
-static struct of_device_id max8998_dt_match[] = {
+static const struct of_device_id max8998_dt_match[] = {
{ .compatible = "maxim,max8998", .data = (void *)TYPE_MAX8998 },
{ .compatible = "national,lp3974", .data = (void *)TYPE_LP3974 },
{ .compatible = "ti,lp3974", .data = (void *)TYPE_LP3974 },
diff --git a/drivers/mfd/mc13xxx-core.c b/drivers/mfd/mc13xxx-core.c
index 0c6c21c5..acf5dd7 100644
--- a/drivers/mfd/mc13xxx-core.c
+++ b/drivers/mfd/mc13xxx-core.c
@@ -660,34 +660,22 @@
if (ret)
return ret;
+ mutex_init(&mc13xxx->lock);
+
ret = request_threaded_irq(mc13xxx->irq, NULL, mc13xxx_irq_thread,
IRQF_ONESHOT | IRQF_TRIGGER_HIGH, "mc13xxx", mc13xxx);
if (ret)
return ret;
- mutex_init(&mc13xxx->lock);
-
if (mc13xxx_probe_flags_dt(mc13xxx) < 0 && pdata)
mc13xxx->flags = pdata->flags;
if (mc13xxx->flags & MC13XXX_USE_ADC)
mc13xxx_add_subdevice(mc13xxx, "%s-adc");
- if (mc13xxx->flags & MC13XXX_USE_CODEC) {
- if (pdata)
- mc13xxx_add_subdevice_pdata(mc13xxx, "%s-codec",
- pdata->codec, sizeof(*pdata->codec));
- else
- mc13xxx_add_subdevice(mc13xxx, "%s-codec");
- }
-
if (mc13xxx->flags & MC13XXX_USE_RTC)
mc13xxx_add_subdevice(mc13xxx, "%s-rtc");
- if (mc13xxx->flags & MC13XXX_USE_TOUCHSCREEN)
- mc13xxx_add_subdevice_pdata(mc13xxx, "%s-ts",
- &pdata->touch, sizeof(pdata->touch));
-
if (pdata) {
mc13xxx_add_subdevice_pdata(mc13xxx, "%s-regulator",
&pdata->regulators, sizeof(pdata->regulators));
@@ -695,10 +683,20 @@
pdata->leds, sizeof(*pdata->leds));
mc13xxx_add_subdevice_pdata(mc13xxx, "%s-pwrbutton",
pdata->buttons, sizeof(*pdata->buttons));
+ if (mc13xxx->flags & MC13XXX_USE_CODEC)
+ mc13xxx_add_subdevice_pdata(mc13xxx, "%s-codec",
+ pdata->codec, sizeof(*pdata->codec));
+ if (mc13xxx->flags & MC13XXX_USE_TOUCHSCREEN)
+ mc13xxx_add_subdevice_pdata(mc13xxx, "%s-ts",
+ &pdata->touch, sizeof(pdata->touch));
} else {
mc13xxx_add_subdevice(mc13xxx, "%s-regulator");
mc13xxx_add_subdevice(mc13xxx, "%s-led");
mc13xxx_add_subdevice(mc13xxx, "%s-pwrbutton");
+ if (mc13xxx->flags & MC13XXX_USE_CODEC)
+ mc13xxx_add_subdevice(mc13xxx, "%s-codec");
+ if (mc13xxx->flags & MC13XXX_USE_TOUCHSCREEN)
+ mc13xxx_add_subdevice(mc13xxx, "%s-ts");
}
return 0;
diff --git a/drivers/mfd/menelaus.c b/drivers/mfd/menelaus.c
index ad25bfa..5e2667a 100644
--- a/drivers/mfd/menelaus.c
+++ b/drivers/mfd/menelaus.c
@@ -1287,29 +1287,8 @@
.id_table = menelaus_id,
};
-static int __init menelaus_init(void)
-{
- int res;
-
- res = i2c_add_driver(&menelaus_i2c_driver);
- if (res < 0) {
- pr_err(DRIVER_NAME ": driver registration failed\n");
- return res;
- }
-
- return 0;
-}
-
-static void __exit menelaus_exit(void)
-{
- i2c_del_driver(&menelaus_i2c_driver);
-
- /* FIXME: Shutdown menelaus parts that can be shut down */
-}
+module_i2c_driver(menelaus_i2c_driver);
MODULE_AUTHOR("Texas Instruments, Inc. (and others)");
MODULE_DESCRIPTION("I2C interface for Menelaus.");
MODULE_LICENSE("GPL");
-
-module_init(menelaus_init);
-module_exit(menelaus_exit);
diff --git a/drivers/mfd/mfd-core.c b/drivers/mfd/mfd-core.c
index 2676492..892d343 100644
--- a/drivers/mfd/mfd-core.c
+++ b/drivers/mfd/mfd-core.c
@@ -102,7 +102,7 @@
pdev->dev.dma_mask = parent->dma_mask;
pdev->dev.dma_parms = parent->dma_parms;
- ret = devm_regulator_bulk_register_supply_alias(
+ ret = regulator_bulk_register_supply_alias(
&pdev->dev, cell->parent_supplies,
parent, cell->parent_supplies,
cell->num_parent_supplies);
@@ -182,9 +182,9 @@
return 0;
fail_alias:
- devm_regulator_bulk_unregister_supply_alias(&pdev->dev,
- cell->parent_supplies,
- cell->num_parent_supplies);
+ regulator_bulk_unregister_supply_alias(&pdev->dev,
+ cell->parent_supplies,
+ cell->num_parent_supplies);
fail_res:
kfree(res);
fail_device:
@@ -238,6 +238,9 @@
pdev = to_platform_device(dev);
cell = mfd_get_cell(pdev);
+ regulator_bulk_unregister_supply_alias(dev, cell->parent_supplies,
+ cell->num_parent_supplies);
+
/* find the base address of usage_count pointers (for freeing) */
if (!*usage_count || (cell->usage_count < *usage_count))
*usage_count = cell->usage_count;
diff --git a/drivers/mfd/omap-usb-host.c b/drivers/mfd/omap-usb-host.c
index 651e249..b48d80c 100644
--- a/drivers/mfd/omap-usb-host.c
+++ b/drivers/mfd/omap-usb-host.c
@@ -557,7 +557,7 @@
return 0;
}
-static struct of_device_id usbhs_child_match_table[] = {
+static const struct of_device_id usbhs_child_match_table[] = {
{ .compatible = "ti,omap-ehci", },
{ .compatible = "ti,omap-ohci", },
{ }
diff --git a/drivers/mfd/pm8921-core.c b/drivers/mfd/pm8921-core.c
index b97a9718..9595138 100644
--- a/drivers/mfd/pm8921-core.c
+++ b/drivers/mfd/pm8921-core.c
@@ -26,7 +26,6 @@
#include <linux/regmap.h>
#include <linux/of_platform.h>
#include <linux/mfd/core.h>
-#include <linux/mfd/pm8xxx/core.h>
#define SSBI_REG_ADDR_IRQ_BASE 0x1BB
@@ -57,7 +56,6 @@
#define PM8921_NR_IRQS 256
struct pm_irq_chip {
- struct device *dev;
struct regmap *regmap;
spinlock_t pm_irq_lock;
struct irq_domain *irqdomain;
@@ -67,11 +65,6 @@
u8 config[0];
};
-struct pm8921 {
- struct device *dev;
- struct pm_irq_chip *irq_chip;
-};
-
static int pm8xxx_read_block_irq(struct pm_irq_chip *chip, unsigned int bp,
unsigned int *ip)
{
@@ -255,55 +248,6 @@
.flags = IRQCHIP_MASK_ON_SUSPEND | IRQCHIP_SKIP_SET_WAKE,
};
-/**
- * pm8xxx_get_irq_stat - get the status of the irq line
- * @chip: pointer to identify a pmic irq controller
- * @irq: the irq number
- *
- * The pm8xxx gpio and mpp rely on the interrupt block to read
- * the values on their pins. This function is to facilitate reading
- * the status of a gpio or an mpp line. The caller has to convert the
- * gpio number to irq number.
- *
- * RETURNS:
- * an int indicating the value read on that line
- */
-static int pm8xxx_get_irq_stat(struct pm_irq_chip *chip, int irq)
-{
- int pmirq, rc;
- unsigned int block, bits, bit;
- unsigned long flags;
- struct irq_data *irq_data = irq_get_irq_data(irq);
-
- pmirq = irq_data->hwirq;
-
- block = pmirq / 8;
- bit = pmirq % 8;
-
- spin_lock_irqsave(&chip->pm_irq_lock, flags);
-
- rc = regmap_write(chip->regmap, SSBI_REG_ADDR_IRQ_BLK_SEL, block);
- if (rc) {
- pr_err("Failed Selecting block irq=%d pmirq=%d blk=%d rc=%d\n",
- irq, pmirq, block, rc);
- goto bail_out;
- }
-
- rc = regmap_read(chip->regmap, SSBI_REG_ADDR_IRQ_RT_STATUS, &bits);
- if (rc) {
- pr_err("Failed Configuring irq=%d pmirq=%d blk=%d rc=%d\n",
- irq, pmirq, block, rc);
- goto bail_out;
- }
-
- rc = (bits & (1 << bit)) ? 1 : 0;
-
-bail_out:
- spin_unlock_irqrestore(&chip->pm_irq_lock, flags);
-
- return rc;
-}
-
static int pm8xxx_irq_domain_map(struct irq_domain *d, unsigned int irq,
irq_hw_number_t hwirq)
{
@@ -324,56 +268,6 @@
.map = pm8xxx_irq_domain_map,
};
-static int pm8921_readb(const struct device *dev, u16 addr, u8 *val)
-{
- const struct pm8xxx_drvdata *pm8921_drvdata = dev_get_drvdata(dev);
- const struct pm8921 *pmic = pm8921_drvdata->pm_chip_data;
-
- return ssbi_read(pmic->dev->parent, addr, val, 1);
-}
-
-static int pm8921_writeb(const struct device *dev, u16 addr, u8 val)
-{
- const struct pm8xxx_drvdata *pm8921_drvdata = dev_get_drvdata(dev);
- const struct pm8921 *pmic = pm8921_drvdata->pm_chip_data;
-
- return ssbi_write(pmic->dev->parent, addr, &val, 1);
-}
-
-static int pm8921_read_buf(const struct device *dev, u16 addr, u8 *buf,
- int cnt)
-{
- const struct pm8xxx_drvdata *pm8921_drvdata = dev_get_drvdata(dev);
- const struct pm8921 *pmic = pm8921_drvdata->pm_chip_data;
-
- return ssbi_read(pmic->dev->parent, addr, buf, cnt);
-}
-
-static int pm8921_write_buf(const struct device *dev, u16 addr, u8 *buf,
- int cnt)
-{
- const struct pm8xxx_drvdata *pm8921_drvdata = dev_get_drvdata(dev);
- const struct pm8921 *pmic = pm8921_drvdata->pm_chip_data;
-
- return ssbi_write(pmic->dev->parent, addr, buf, cnt);
-}
-
-static int pm8921_read_irq_stat(const struct device *dev, int irq)
-{
- const struct pm8xxx_drvdata *pm8921_drvdata = dev_get_drvdata(dev);
- const struct pm8921 *pmic = pm8921_drvdata->pm_chip_data;
-
- return pm8xxx_get_irq_stat(pmic->irq_chip, irq);
-}
-
-static struct pm8xxx_drvdata pm8921_drvdata = {
- .pmic_readb = pm8921_readb,
- .pmic_writeb = pm8921_writeb,
- .pmic_read_buf = pm8921_read_buf,
- .pmic_write_buf = pm8921_write_buf,
- .pmic_read_irq_stat = pm8921_read_irq_stat,
-};
-
static const struct regmap_config ssbi_regmap_config = {
.reg_bits = 16,
.val_bits = 8,
@@ -392,7 +286,6 @@
static int pm8921_probe(struct platform_device *pdev)
{
- struct pm8921 *pmic;
struct regmap *regmap;
int irq, rc;
unsigned int val;
@@ -404,12 +297,6 @@
if (irq < 0)
return irq;
- pmic = devm_kzalloc(&pdev->dev, sizeof(struct pm8921), GFP_KERNEL);
- if (!pmic) {
- pr_err("Cannot alloc pm8921 struct\n");
- return -ENOMEM;
- }
-
regmap = devm_regmap_init(&pdev->dev, NULL, pdev->dev.parent,
&ssbi_regmap_config);
if (IS_ERR(regmap))
@@ -434,18 +321,13 @@
pr_info("PMIC revision 2: %02X\n", val);
rev |= val << BITS_PER_BYTE;
- pmic->dev = &pdev->dev;
- pm8921_drvdata.pm_chip_data = pmic;
- platform_set_drvdata(pdev, &pm8921_drvdata);
-
chip = devm_kzalloc(&pdev->dev, sizeof(*chip) +
sizeof(chip->config[0]) * nirqs,
GFP_KERNEL);
if (!chip)
return -ENOMEM;
- pmic->irq_chip = chip;
- chip->dev = &pdev->dev;
+ platform_set_drvdata(pdev, chip);
chip->regmap = regmap;
chip->num_irqs = nirqs;
chip->num_blocks = DIV_ROUND_UP(chip->num_irqs, 8);
@@ -481,8 +363,7 @@
static int pm8921_remove(struct platform_device *pdev)
{
int irq = platform_get_irq(pdev, 0);
- struct pm8921 *pmic = pm8921_drvdata.pm_chip_data;
- struct pm_irq_chip *chip = pmic->irq_chip;
+ struct pm_irq_chip *chip = platform_get_drvdata(pdev);
device_for_each_child(&pdev->dev, NULL, pm8921_remove_child);
irq_set_chained_handler(irq, NULL);
diff --git a/drivers/mfd/rdc321x-southbridge.c b/drivers/mfd/rdc321x-southbridge.c
index c795697..6575585 100644
--- a/drivers/mfd/rdc321x-southbridge.c
+++ b/drivers/mfd/rdc321x-southbridge.c
@@ -38,7 +38,7 @@
};
static struct rdc321x_gpio_pdata rdc321x_gpio_pdata = {
- .max_gpios = RDC321X_MAX_GPIO,
+ .max_gpios = RDC321X_NUM_GPIO,
};
static struct resource rdc321x_gpio_resources[] = {
diff --git a/drivers/mfd/rtsx_usb.c b/drivers/mfd/rtsx_usb.c
index b53b9d4..6352bec 100644
--- a/drivers/mfd/rtsx_usb.c
+++ b/drivers/mfd/rtsx_usb.c
@@ -29,7 +29,7 @@
module_param(polling_pipe, int, S_IRUGO | S_IWUSR);
MODULE_PARM_DESC(polling_pipe, "polling pipe (0: ctl, 1: bulk)");
-static struct mfd_cell rtsx_usb_cells[] = {
+static const struct mfd_cell rtsx_usb_cells[] = {
[RTSX_USB_SD_CARD] = {
.name = "rtsx_usb_sdmmc",
.pdata_size = 0,
@@ -67,7 +67,7 @@
ucr->sg_timer.expires = jiffies + msecs_to_jiffies(timeout);
add_timer(&ucr->sg_timer);
usb_sg_wait(&ucr->current_sg);
- del_timer(&ucr->sg_timer);
+ del_timer_sync(&ucr->sg_timer);
if (act_len)
*act_len = ucr->current_sg.bytes;
@@ -644,14 +644,14 @@
if (ret)
goto out_init_fail;
+ /* initialize USB SG transfer timer */
+ setup_timer(&ucr->sg_timer, rtsx_usb_sg_timed_out, (unsigned long) ucr);
+
ret = mfd_add_devices(&intf->dev, usb_dev->devnum, rtsx_usb_cells,
ARRAY_SIZE(rtsx_usb_cells), NULL, 0, NULL);
if (ret)
goto out_init_fail;
- /* initialize USB SG transfer timer */
- init_timer(&ucr->sg_timer);
- setup_timer(&ucr->sg_timer, rtsx_usb_sg_timed_out, (unsigned long) ucr);
#ifdef CONFIG_PM
intf->needs_remote_wakeup = 1;
usb_enable_autosuspend(usb_dev);
@@ -687,9 +687,15 @@
dev_dbg(&intf->dev, "%s called with pm message 0x%04u\n",
__func__, message.event);
+ /*
+ * Call to make sure LED is off during suspend to save more power.
+ * It is NOT a permanent state and could be turned on anytime later.
+ * Thus no need to call turn_on when resunming.
+ */
mutex_lock(&ucr->dev_mutex);
rtsx_usb_turn_off_led(ucr);
mutex_unlock(&ucr->dev_mutex);
+
return 0;
}
diff --git a/drivers/mfd/sec-core.c b/drivers/mfd/sec-core.c
index 1cf2752..be06d0a 100644
--- a/drivers/mfd/sec-core.c
+++ b/drivers/mfd/sec-core.c
@@ -25,7 +25,6 @@
#include <linux/mfd/core.h>
#include <linux/mfd/samsung/core.h>
#include <linux/mfd/samsung/irq.h>
-#include <linux/mfd/samsung/rtc.h>
#include <linux/mfd/samsung/s2mpa01.h>
#include <linux/mfd/samsung/s2mps11.h>
#include <linux/mfd/samsung/s2mps14.h>
@@ -91,7 +90,7 @@
};
#ifdef CONFIG_OF
-static struct of_device_id sec_dt_match[] = {
+static const struct of_device_id sec_dt_match[] = {
{ .compatible = "samsung,s5m8767-pmic",
.data = (void *)S5M8767X,
}, {
@@ -196,20 +195,6 @@
.cache_type = REGCACHE_FLAT,
};
-static const struct regmap_config s5m_rtc_regmap_config = {
- .reg_bits = 8,
- .val_bits = 8,
-
- .max_register = SEC_RTC_REG_MAX,
-};
-
-static const struct regmap_config s2mps14_rtc_regmap_config = {
- .reg_bits = 8,
- .val_bits = 8,
-
- .max_register = S2MPS_RTC_REG_MAX,
-};
-
#ifdef CONFIG_OF
/*
* Only the common platform data elements for s5m8767 are parsed here from the
@@ -264,8 +249,9 @@
const struct i2c_device_id *id)
{
struct sec_platform_data *pdata = dev_get_platdata(&i2c->dev);
- const struct regmap_config *regmap, *regmap_rtc;
+ const struct regmap_config *regmap;
struct sec_pmic_dev *sec_pmic;
+ unsigned long device_type;
int ret;
sec_pmic = devm_kzalloc(&i2c->dev, sizeof(struct sec_pmic_dev),
@@ -277,7 +263,7 @@
sec_pmic->dev = &i2c->dev;
sec_pmic->i2c = i2c;
sec_pmic->irq = i2c->irq;
- sec_pmic->type = sec_i2c_get_driver_data(i2c, id);
+ device_type = sec_i2c_get_driver_data(i2c, id);
if (sec_pmic->dev->of_node) {
pdata = sec_pmic_i2c_parse_dt_pdata(sec_pmic->dev);
@@ -285,7 +271,7 @@
ret = PTR_ERR(pdata);
return ret;
}
- pdata->device_type = sec_pmic->type;
+ pdata->device_type = device_type;
}
if (pdata) {
sec_pmic->device_type = pdata->device_type;
@@ -298,39 +284,21 @@
switch (sec_pmic->device_type) {
case S2MPA01:
regmap = &s2mpa01_regmap_config;
- /*
- * The rtc-s5m driver does not support S2MPA01 and there
- * is no mfd_cell for S2MPA01 RTC device.
- * However we must pass something to devm_regmap_init_i2c()
- * so use S5M-like regmap config even though it wouldn't work.
- */
- regmap_rtc = &s5m_rtc_regmap_config;
break;
case S2MPS11X:
regmap = &s2mps11_regmap_config;
- /*
- * The rtc-s5m driver does not support S2MPS11 and there
- * is no mfd_cell for S2MPS11 RTC device.
- * However we must pass something to devm_regmap_init_i2c()
- * so use S5M-like regmap config even though it wouldn't work.
- */
- regmap_rtc = &s5m_rtc_regmap_config;
break;
case S2MPS14X:
regmap = &s2mps14_regmap_config;
- regmap_rtc = &s2mps14_rtc_regmap_config;
break;
case S5M8763X:
regmap = &s5m8763_regmap_config;
- regmap_rtc = &s5m_rtc_regmap_config;
break;
case S5M8767X:
regmap = &s5m8767_regmap_config;
- regmap_rtc = &s5m_rtc_regmap_config;
break;
default:
regmap = &sec_regmap_config;
- regmap_rtc = &s5m_rtc_regmap_config;
break;
}
@@ -342,21 +310,6 @@
return ret;
}
- sec_pmic->rtc = i2c_new_dummy(i2c->adapter, RTC_I2C_ADDR);
- if (!sec_pmic->rtc) {
- dev_err(&i2c->dev, "Failed to allocate I2C for RTC\n");
- return -ENODEV;
- }
- i2c_set_clientdata(sec_pmic->rtc, sec_pmic);
-
- sec_pmic->regmap_rtc = devm_regmap_init_i2c(sec_pmic->rtc, regmap_rtc);
- if (IS_ERR(sec_pmic->regmap_rtc)) {
- ret = PTR_ERR(sec_pmic->regmap_rtc);
- dev_err(&i2c->dev, "Failed to allocate RTC register map: %d\n",
- ret);
- goto err_regmap_rtc;
- }
-
if (pdata && pdata->cfg_pmic_irq)
pdata->cfg_pmic_irq();
@@ -403,8 +356,6 @@
err_mfd:
sec_irq_exit(sec_pmic);
-err_regmap_rtc:
- i2c_unregister_device(sec_pmic->rtc);
return ret;
}
@@ -414,7 +365,6 @@
mfd_remove_devices(sec_pmic->dev);
sec_irq_exit(sec_pmic);
- i2c_unregister_device(sec_pmic->rtc);
return 0;
}
@@ -424,19 +374,18 @@
struct i2c_client *i2c = container_of(dev, struct i2c_client, dev);
struct sec_pmic_dev *sec_pmic = i2c_get_clientdata(i2c);
- if (device_may_wakeup(dev)) {
+ if (device_may_wakeup(dev))
enable_irq_wake(sec_pmic->irq);
- /*
- * PMIC IRQ must be disabled during suspend for RTC alarm
- * to work properly.
- * When device is woken up from suspend by RTC Alarm, an
- * interrupt occurs before resuming I2C bus controller.
- * The interrupt is handled by regmap_irq_thread which tries
- * to read RTC registers. This read fails (I2C is still
- * suspended) and RTC Alarm interrupt is disabled.
- */
- disable_irq(sec_pmic->irq);
- }
+ /*
+ * PMIC IRQ must be disabled during suspend for RTC alarm
+ * to work properly.
+ * When device is woken up from suspend, an
+ * interrupt occurs before resuming I2C bus controller.
+ * The interrupt is handled by regmap_irq_thread which tries
+ * to read RTC registers. This read fails (I2C is still
+ * suspended) and RTC Alarm interrupt is disabled.
+ */
+ disable_irq(sec_pmic->irq);
return 0;
}
@@ -446,10 +395,9 @@
struct i2c_client *i2c = container_of(dev, struct i2c_client, dev);
struct sec_pmic_dev *sec_pmic = i2c_get_clientdata(i2c);
- if (device_may_wakeup(dev)) {
+ if (device_may_wakeup(dev))
disable_irq_wake(sec_pmic->irq);
- enable_irq(sec_pmic->irq);
- }
+ enable_irq(sec_pmic->irq);
return 0;
}
diff --git a/drivers/mfd/sec-irq.c b/drivers/mfd/sec-irq.c
index 64e7913..654e2c1 100644
--- a/drivers/mfd/sec-irq.c
+++ b/drivers/mfd/sec-irq.c
@@ -385,7 +385,7 @@
&sec_pmic->irq_data);
break;
default:
- dev_err(sec_pmic->dev, "Unknown device type %d\n",
+ dev_err(sec_pmic->dev, "Unknown device type %lu\n",
sec_pmic->device_type);
return -EINVAL;
}
diff --git a/drivers/mfd/sm501.c b/drivers/mfd/sm501.c
index e7dc441..81e6d09 100644
--- a/drivers/mfd/sm501.c
+++ b/drivers/mfd/sm501.c
@@ -1726,7 +1726,7 @@
MODULE_ALIAS("platform:sm501");
-static struct of_device_id of_sm501_match_tbl[] = {
+static const struct of_device_id of_sm501_match_tbl[] = {
{ .compatible = "smi,sm501", },
{ /* end */ }
};
diff --git a/drivers/mfd/stmpe-i2c.c b/drivers/mfd/stmpe-i2c.c
index 0da02e1..a45f9c0 100644
--- a/drivers/mfd/stmpe-i2c.c
+++ b/drivers/mfd/stmpe-i2c.c
@@ -14,6 +14,7 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/of_device.h>
#include "stmpe.h"
static int i2c_reg_read(struct stmpe *stmpe, u8 reg)
@@ -52,15 +53,41 @@
.write_block = i2c_block_write,
};
+static const struct of_device_id stmpe_of_match[] = {
+ { .compatible = "st,stmpe610", .data = (void *)STMPE610, },
+ { .compatible = "st,stmpe801", .data = (void *)STMPE801, },
+ { .compatible = "st,stmpe811", .data = (void *)STMPE811, },
+ { .compatible = "st,stmpe1601", .data = (void *)STMPE1601, },
+ { .compatible = "st,stmpe1801", .data = (void *)STMPE1801, },
+ { .compatible = "st,stmpe2401", .data = (void *)STMPE2401, },
+ { .compatible = "st,stmpe2403", .data = (void *)STMPE2403, },
+ {},
+};
+MODULE_DEVICE_TABLE(of, stmpe_of_match);
+
static int
stmpe_i2c_probe(struct i2c_client *i2c, const struct i2c_device_id *id)
{
+ int partnum;
+ const struct of_device_id *of_id;
+
i2c_ci.data = (void *)id;
i2c_ci.irq = i2c->irq;
i2c_ci.client = i2c;
i2c_ci.dev = &i2c->dev;
- return stmpe_probe(&i2c_ci, id->driver_data);
+ of_id = of_match_device(stmpe_of_match, &i2c->dev);
+ if (!of_id) {
+ /*
+ * This happens when the I2C ID matches the node name
+ * but no real compatible string has been given.
+ */
+ dev_info(&i2c->dev, "matching on node name, compatible is preferred\n");
+ partnum = id->driver_data;
+ } else
+ partnum = (int)of_id->data;
+
+ return stmpe_probe(&i2c_ci, partnum);
}
static int stmpe_i2c_remove(struct i2c_client *i2c)
@@ -89,6 +116,7 @@
#ifdef CONFIG_PM
.pm = &stmpe_dev_pm_ops,
#endif
+ .of_match_table = stmpe_of_match,
},
.probe = stmpe_i2c_probe,
.remove = stmpe_i2c_remove,
diff --git a/drivers/mfd/stmpe.c b/drivers/mfd/stmpe.c
index 4a91f67..3b6bfa7 100644
--- a/drivers/mfd/stmpe.c
+++ b/drivers/mfd/stmpe.c
@@ -20,6 +20,7 @@
#include <linux/slab.h>
#include <linux/mfd/core.h>
#include <linux/delay.h>
+#include <linux/regulator/consumer.h>
#include "stmpe.h"
static int __stmpe_enable(struct stmpe *stmpe, unsigned int blocks)
@@ -605,9 +606,18 @@
if (blocks & STMPE_BLOCK_GPIO)
mask |= STMPE1601_SYS_CTRL_ENABLE_GPIO;
+ else
+ mask &= ~STMPE1601_SYS_CTRL_ENABLE_GPIO;
if (blocks & STMPE_BLOCK_KEYPAD)
mask |= STMPE1601_SYS_CTRL_ENABLE_KPC;
+ else
+ mask &= ~STMPE1601_SYS_CTRL_ENABLE_KPC;
+
+ if (blocks & STMPE_BLOCK_PWM)
+ mask |= STMPE1601_SYS_CTRL_ENABLE_SPWM;
+ else
+ mask &= ~STMPE1601_SYS_CTRL_ENABLE_SPWM;
return __stmpe_set_bits(stmpe, STMPE1601_REG_SYS_CTRL, mask,
enable ? mask : 0);
@@ -986,9 +996,6 @@
int base = 0;
int num_irqs = stmpe->variant->num_irqs;
- if (!np)
- base = stmpe->irq_base;
-
stmpe->domain = irq_domain_add_simple(np, num_irqs, base,
&stmpe_irq_ops, stmpe);
if (!stmpe->domain) {
@@ -1067,7 +1074,7 @@
static int stmpe_add_device(struct stmpe *stmpe, const struct mfd_cell *cell)
{
return mfd_add_devices(stmpe->dev, stmpe->pdata->id, cell, 1,
- NULL, stmpe->irq_base, stmpe->domain);
+ NULL, 0, stmpe->domain);
}
static int stmpe_devices_init(struct stmpe *stmpe)
@@ -1171,12 +1178,23 @@
stmpe->dev = ci->dev;
stmpe->client = ci->client;
stmpe->pdata = pdata;
- stmpe->irq_base = pdata->irq_base;
stmpe->ci = ci;
stmpe->partnum = partnum;
stmpe->variant = stmpe_variant_info[partnum];
stmpe->regs = stmpe->variant->regs;
stmpe->num_gpios = stmpe->variant->num_gpios;
+ stmpe->vcc = devm_regulator_get_optional(ci->dev, "vcc");
+ if (!IS_ERR(stmpe->vcc)) {
+ ret = regulator_enable(stmpe->vcc);
+ if (ret)
+ dev_warn(ci->dev, "failed to enable VCC supply\n");
+ }
+ stmpe->vio = devm_regulator_get_optional(ci->dev, "vio");
+ if (!IS_ERR(stmpe->vio)) {
+ ret = regulator_enable(stmpe->vio);
+ if (ret)
+ dev_warn(ci->dev, "failed to enable VIO supply\n");
+ }
dev_set_drvdata(stmpe->dev, stmpe);
if (ci->init)
@@ -1243,6 +1261,11 @@
int stmpe_remove(struct stmpe *stmpe)
{
+ if (!IS_ERR(stmpe->vio))
+ regulator_disable(stmpe->vio);
+ if (!IS_ERR(stmpe->vcc))
+ regulator_disable(stmpe->vcc);
+
mfd_remove_devices(stmpe->dev);
return 0;
diff --git a/drivers/mfd/stmpe.h b/drivers/mfd/stmpe.h
index 6639f1b..9e4d21d 100644
--- a/drivers/mfd/stmpe.h
+++ b/drivers/mfd/stmpe.h
@@ -192,7 +192,7 @@
#define STMPE1601_SYS_CTRL_ENABLE_GPIO (1 << 3)
#define STMPE1601_SYS_CTRL_ENABLE_KPC (1 << 1)
-#define STMPE1601_SYSCON_ENABLE_SPWM (1 << 0)
+#define STMPE1601_SYS_CTRL_ENABLE_SPWM (1 << 0)
/* The 1601/2403 share the same masks */
#define STMPE1601_AUTOSLEEP_TIMEOUT_MASK (0x7)
diff --git a/drivers/mfd/sun6i-prcm.c b/drivers/mfd/sun6i-prcm.c
new file mode 100644
index 0000000..718fc4d
--- /dev/null
+++ b/drivers/mfd/sun6i-prcm.c
@@ -0,0 +1,134 @@
+/*
+ * Copyright (C) 2014 Free Electrons
+ *
+ * License Terms: GNU General Public License v2
+ * Author: Boris BREZILLON <boris.brezillon@free-electrons.com>
+ *
+ * Allwinner PRCM (Power/Reset/Clock Management) driver
+ *
+ */
+
+#include <linux/mfd/core.h>
+#include <linux/module.h>
+#include <linux/of.h>
+
+struct prcm_data {
+ int nsubdevs;
+ const struct mfd_cell *subdevs;
+};
+
+static const struct resource sun6i_a31_ar100_clk_res[] = {
+ {
+ .start = 0x0,
+ .end = 0x3,
+ .flags = IORESOURCE_MEM,
+ },
+};
+
+static const struct resource sun6i_a31_apb0_clk_res[] = {
+ {
+ .start = 0xc,
+ .end = 0xf,
+ .flags = IORESOURCE_MEM,
+ },
+};
+
+static const struct resource sun6i_a31_apb0_gates_clk_res[] = {
+ {
+ .start = 0x28,
+ .end = 0x2b,
+ .flags = IORESOURCE_MEM,
+ },
+};
+
+static const struct resource sun6i_a31_apb0_rstc_res[] = {
+ {
+ .start = 0xb0,
+ .end = 0xb3,
+ .flags = IORESOURCE_MEM,
+ },
+};
+
+static const struct mfd_cell sun6i_a31_prcm_subdevs[] = {
+ {
+ .name = "sun6i-a31-ar100-clk",
+ .of_compatible = "allwinner,sun6i-a31-ar100-clk",
+ .num_resources = ARRAY_SIZE(sun6i_a31_ar100_clk_res),
+ .resources = sun6i_a31_ar100_clk_res,
+ },
+ {
+ .name = "sun6i-a31-apb0-clk",
+ .of_compatible = "allwinner,sun6i-a31-apb0-clk",
+ .num_resources = ARRAY_SIZE(sun6i_a31_apb0_clk_res),
+ .resources = sun6i_a31_apb0_clk_res,
+ },
+ {
+ .name = "sun6i-a31-apb0-gates-clk",
+ .of_compatible = "allwinner,sun6i-a31-apb0-gates-clk",
+ .num_resources = ARRAY_SIZE(sun6i_a31_apb0_gates_clk_res),
+ .resources = sun6i_a31_apb0_gates_clk_res,
+ },
+ {
+ .name = "sun6i-a31-apb0-clock-reset",
+ .of_compatible = "allwinner,sun6i-a31-clock-reset",
+ .num_resources = ARRAY_SIZE(sun6i_a31_apb0_rstc_res),
+ .resources = sun6i_a31_apb0_rstc_res,
+ },
+};
+
+static const struct prcm_data sun6i_a31_prcm_data = {
+ .nsubdevs = ARRAY_SIZE(sun6i_a31_prcm_subdevs),
+ .subdevs = sun6i_a31_prcm_subdevs,
+};
+
+static const struct of_device_id sun6i_prcm_dt_ids[] = {
+ {
+ .compatible = "allwinner,sun6i-a31-prcm",
+ .data = &sun6i_a31_prcm_data,
+ },
+ { /* sentinel */ },
+};
+
+static int sun6i_prcm_probe(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ const struct of_device_id *match;
+ const struct prcm_data *data;
+ struct resource *res;
+ int ret;
+
+ match = of_match_node(sun6i_prcm_dt_ids, np);
+ if (!match)
+ return -EINVAL;
+
+ data = match->data;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res) {
+ dev_err(&pdev->dev, "no prcm memory region provided\n");
+ return -ENOENT;
+ }
+
+ ret = mfd_add_devices(&pdev->dev, 0, data->subdevs, data->nsubdevs,
+ res, -1, NULL);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to add subdevices\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static struct platform_driver sun6i_prcm_driver = {
+ .driver = {
+ .name = "sun6i-prcm",
+ .owner = THIS_MODULE,
+ .of_match_table = sun6i_prcm_dt_ids,
+ },
+ .probe = sun6i_prcm_probe,
+};
+module_platform_driver(sun6i_prcm_driver);
+
+MODULE_AUTHOR("Boris BREZILLON <boris.brezillon@free-electrons.com>");
+MODULE_DESCRIPTION("Allwinner sun6i PRCM driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/mfd/syscon.c b/drivers/mfd/syscon.c
index e2a04bb..ca15878 100644
--- a/drivers/mfd/syscon.c
+++ b/drivers/mfd/syscon.c
@@ -95,7 +95,11 @@
struct device_node *syscon_np;
struct regmap *regmap;
- syscon_np = of_parse_phandle(np, property, 0);
+ if (property)
+ syscon_np = of_parse_phandle(np, property, 0);
+ else
+ syscon_np = np;
+
if (!syscon_np)
return ERR_PTR(-ENODEV);
diff --git a/drivers/mfd/tps6507x.c b/drivers/mfd/tps6507x.c
index 3b27482..a2e1990 100644
--- a/drivers/mfd/tps6507x.c
+++ b/drivers/mfd/tps6507x.c
@@ -119,7 +119,7 @@
MODULE_DEVICE_TABLE(i2c, tps6507x_i2c_id);
#ifdef CONFIG_OF
-static struct of_device_id tps6507x_of_match[] = {
+static const struct of_device_id tps6507x_of_match[] = {
{.compatible = "ti,tps6507x", },
{},
};
diff --git a/drivers/mfd/tps65218.c b/drivers/mfd/tps65218.c
index a74bfb5..0d256cb 100644
--- a/drivers/mfd/tps65218.c
+++ b/drivers/mfd/tps65218.c
@@ -197,6 +197,7 @@
static const struct of_device_id of_tps65218_match_table[] = {
{ .compatible = "ti,tps65218", },
+ {}
};
static int tps65218_probe(struct i2c_client *client,
diff --git a/drivers/mfd/tps6586x.c b/drivers/mfd/tps6586x.c
index 835e554..8e1dbc4 100644
--- a/drivers/mfd/tps6586x.c
+++ b/drivers/mfd/tps6586x.c
@@ -444,7 +444,7 @@
return pdata;
}
-static struct of_device_id tps6586x_of_match[] = {
+static const struct of_device_id tps6586x_of_match[] = {
{ .compatible = "ti,tps6586x", },
{ },
};
diff --git a/drivers/mfd/tps65910.c b/drivers/mfd/tps65910.c
index 460a014..f9e42ea 100644
--- a/drivers/mfd/tps65910.c
+++ b/drivers/mfd/tps65910.c
@@ -379,7 +379,7 @@
}
#ifdef CONFIG_OF
-static struct of_device_id tps65910_of_match[] = {
+static const struct of_device_id tps65910_of_match[] = {
{ .compatible = "ti,tps65910", .data = (void *)TPS65910},
{ .compatible = "ti,tps65911", .data = (void *)TPS65911},
{ },
diff --git a/drivers/mfd/twl6040.c b/drivers/mfd/twl6040.c
index 6e88f25..ae26d84 100644
--- a/drivers/mfd/twl6040.c
+++ b/drivers/mfd/twl6040.c
@@ -87,8 +87,13 @@
};
static struct reg_default twl6040_patch[] = {
- /* Select I2C bus access to dual access registers */
- { TWL6040_REG_ACCCTL, 0x09 },
+ /*
+ * Select I2C bus access to dual access registers
+ * Interrupt register is cleared on read
+ * Select fast mode for i2c (400KHz)
+ */
+ { TWL6040_REG_ACCCTL,
+ TWL6040_I2CSEL | TWL6040_INTCLRMODE | TWL6040_I2CMODE(1) },
};
@@ -286,6 +291,8 @@
if (twl6040->power_count++)
goto out;
+ clk_prepare_enable(twl6040->clk32k);
+
/* Allow writes to the chip */
regcache_cache_only(twl6040->regmap, false);
@@ -341,6 +348,8 @@
twl6040->sysclk = 0;
twl6040->mclk = 0;
+
+ clk_disable_unprepare(twl6040->clk32k);
}
out:
@@ -432,12 +441,9 @@
TWL6040_HPLLENA;
break;
case 19200000:
- /*
- * PLL disabled
- * (enable PLL if MCLK jitter quality
- * doesn't meet specification)
- */
- hppllctl |= TWL6040_MCLK_19200KHZ;
+ /* PLL enabled, bypass mode */
+ hppllctl |= TWL6040_MCLK_19200KHZ |
+ TWL6040_HPLLBP | TWL6040_HPLLENA;
break;
case 26000000:
/* PLL enabled, active mode */
@@ -445,9 +451,9 @@
TWL6040_HPLLENA;
break;
case 38400000:
- /* PLL enabled, active mode */
+ /* PLL enabled, bypass mode */
hppllctl |= TWL6040_MCLK_38400KHZ |
- TWL6040_HPLLENA;
+ TWL6040_HPLLBP | TWL6040_HPLLENA;
break;
default:
dev_err(twl6040->dev,
@@ -639,6 +645,12 @@
i2c_set_clientdata(client, twl6040);
+ twl6040->clk32k = devm_clk_get(&client->dev, "clk32k");
+ if (IS_ERR(twl6040->clk32k)) {
+ dev_info(&client->dev, "clk32k is not handled\n");
+ twl6040->clk32k = NULL;
+ }
+
twl6040->supplies[0].supply = "vio";
twl6040->supplies[1].supply = "v2v1";
ret = devm_regulator_bulk_get(&client->dev, TWL6040_NUM_SUPPLIES,
@@ -660,6 +672,9 @@
mutex_init(&twl6040->mutex);
init_completion(&twl6040->ready);
+ regmap_register_patch(twl6040->regmap, twl6040_patch,
+ ARRAY_SIZE(twl6040_patch));
+
twl6040->rev = twl6040_reg_read(twl6040, TWL6040_REG_ASICREV);
if (twl6040->rev < 0) {
dev_err(&client->dev, "Failed to read revision register: %d\n",
@@ -679,6 +694,9 @@
GPIOF_OUT_INIT_LOW, "audpwron");
if (ret)
goto gpio_err;
+
+ /* Clear any pending interrupt */
+ twl6040_reg_read(twl6040, TWL6040_REG_INTID);
}
ret = regmap_add_irq_chip(twl6040->regmap, twl6040->irq, IRQF_ONESHOT,
@@ -707,10 +725,6 @@
goto readyirq_err;
}
- /* dual-access registers controlled by I2C only */
- regmap_register_patch(twl6040->regmap, twl6040_patch,
- ARRAY_SIZE(twl6040_patch));
-
/*
* The main functionality of twl6040 to provide audio on OMAP4+ systems.
* We can add the ASoC codec child whenever this driver has been loaded.
diff --git a/drivers/mfd/wm5102-tables.c b/drivers/mfd/wm5102-tables.c
index 070f8cf..c8a993b 100644
--- a/drivers/mfd/wm5102-tables.c
+++ b/drivers/mfd/wm5102-tables.c
@@ -333,7 +333,7 @@
{ 0x000001AA, 0x0004 }, /* R426 - FLL2 GPIO Clock */
{ 0x00000200, 0x0006 }, /* R512 - Mic Charge Pump 1 */
{ 0x00000210, 0x00D4 }, /* R528 - LDO1 Control 1 */
- { 0x00000212, 0x0001 }, /* R530 - LDO1 Control 2 */
+ { 0x00000212, 0x0000 }, /* R530 - LDO1 Control 2 */
{ 0x00000213, 0x0344 }, /* R531 - LDO2 Control 1 */
{ 0x00000218, 0x01A6 }, /* R536 - Mic Bias Ctrl 1 */
{ 0x00000219, 0x01A6 }, /* R537 - Mic Bias Ctrl 2 */
@@ -1037,6 +1037,8 @@
case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_4:
case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_5:
case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_6:
+ case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_7:
+ case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_8:
case ARIZONA_COMFORT_NOISE_GENERATOR:
case ARIZONA_HAPTICS_CONTROL_1:
case ARIZONA_HAPTICS_CONTROL_2:
diff --git a/drivers/mfd/wm5110-tables.c b/drivers/mfd/wm5110-tables.c
index 1942b6f..41a7f6f 100644
--- a/drivers/mfd/wm5110-tables.c
+++ b/drivers/mfd/wm5110-tables.c
@@ -468,10 +468,12 @@
{ 0x00000062, 0x01FF }, /* R98 - Sample Rate Sequence Select 2 */
{ 0x00000063, 0x01FF }, /* R99 - Sample Rate Sequence Select 3 */
{ 0x00000064, 0x01FF }, /* R100 - Sample Rate Sequence Select 4 */
- { 0x00000068, 0x01FF }, /* R104 - Always On Triggers Sequence Select 1 */
- { 0x00000069, 0x01FF }, /* R105 - Always On Triggers Sequence Select 2 */
- { 0x0000006A, 0x01FF }, /* R106 - Always On Triggers Sequence Select 3 */
- { 0x0000006B, 0x01FF }, /* R107 - Always On Triggers Sequence Select 4 */
+ { 0x00000066, 0x01FF }, /* R102 - Always On Triggers Sequence Select 1 */
+ { 0x00000067, 0x01FF }, /* R103 - Always On Triggers Sequence Select 2 */
+ { 0x00000068, 0x01FF }, /* R104 - Always On Triggers Sequence Select 3 */
+ { 0x00000069, 0x01FF }, /* R105 - Always On Triggers Sequence Select 4 */
+ { 0x0000006A, 0x01FF }, /* R106 - Always On Triggers Sequence Select 5 */
+ { 0x0000006B, 0x01FF }, /* R107 - Always On Triggers Sequence Select 6 */
{ 0x00000070, 0x0000 }, /* R112 - Comfort Noise Generator */
{ 0x00000090, 0x0000 }, /* R144 - Haptics Control 1 */
{ 0x00000091, 0x7FFF }, /* R145 - Haptics Control 2 */
@@ -549,6 +551,7 @@
{ 0x000002A8, 0x1422 }, /* R680 - Mic Detect Level 3 */
{ 0x000002A9, 0x300A }, /* R681 - Mic Detect Level 4 */
{ 0x000002C3, 0x0000 }, /* R707 - Mic noise mix control 1 */
+ { 0x000002CB, 0x0000 }, /* R715 - Isolation control */
{ 0x000002D3, 0x0000 }, /* R723 - Jack detect analogue */
{ 0x00000300, 0x0000 }, /* R768 - Input Enables */
{ 0x00000308, 0x0000 }, /* R776 - Input Rate */
@@ -1498,6 +1501,8 @@
case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_2:
case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_3:
case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_4:
+ case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_5:
+ case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_6:
case ARIZONA_COMFORT_NOISE_GENERATOR:
case ARIZONA_HAPTICS_CONTROL_1:
case ARIZONA_HAPTICS_CONTROL_2:
@@ -1580,6 +1585,7 @@
case ARIZONA_MIC_DETECT_LEVEL_3:
case ARIZONA_MIC_DETECT_LEVEL_4:
case ARIZONA_MIC_NOISE_MIX_CONTROL_1:
+ case ARIZONA_ISOLATION_CONTROL:
case ARIZONA_JACK_DETECT_ANALOGUE:
case ARIZONA_INPUT_ENABLES:
case ARIZONA_INPUT_ENABLES_STATUS:
diff --git a/drivers/mfd/wm8400-core.c b/drivers/mfd/wm8400-core.c
index e5eae75..c6fb5d1 100644
--- a/drivers/mfd/wm8400-core.c
+++ b/drivers/mfd/wm8400-core.c
@@ -64,7 +64,7 @@
static int wm8400_register_codec(struct wm8400 *wm8400)
{
- struct mfd_cell cell = {
+ const struct mfd_cell cell = {
.name = "wm8400-codec",
.platform_data = wm8400,
.pdata_size = sizeof(*wm8400),
diff --git a/drivers/mfd/wm8997-tables.c b/drivers/mfd/wm8997-tables.c
index 5aa8076..c7a81da 100644
--- a/drivers/mfd/wm8997-tables.c
+++ b/drivers/mfd/wm8997-tables.c
@@ -174,10 +174,10 @@
{ 0x00000062, 0x01FF }, /* R98 - Sample Rate Sequence Select 2 */
{ 0x00000063, 0x01FF }, /* R99 - Sample Rate Sequence Select 3 */
{ 0x00000064, 0x01FF }, /* R100 - Sample Rate Sequence Select 4 */
- { 0x00000068, 0x01FF }, /* R104 - Always On Triggers Sequence Select 1 */
- { 0x00000069, 0x01FF }, /* R105 - Always On Triggers Sequence Select 2 */
- { 0x0000006A, 0x01FF }, /* R106 - Always On Triggers Sequence Select 3 */
- { 0x0000006B, 0x01FF }, /* R107 - Always On Triggers Sequence Select 4 */
+ { 0x00000068, 0x01FF }, /* R104 - Always On Triggers Sequence Select 3 */
+ { 0x00000069, 0x01FF }, /* R105 - Always On Triggers Sequence Select 4 */
+ { 0x0000006A, 0x01FF }, /* R106 - Always On Triggers Sequence Select 5 */
+ { 0x0000006B, 0x01FF }, /* R107 - Always On Triggers Sequence Select 6 */
{ 0x00000070, 0x0000 }, /* R112 - Comfort Noise Generator */
{ 0x00000090, 0x0000 }, /* R144 - Haptics Control 1 */
{ 0x00000091, 0x7FFF }, /* R145 - Haptics Control 2 */
@@ -814,10 +814,10 @@
case ARIZONA_SAMPLE_RATE_SEQUENCE_SELECT_2:
case ARIZONA_SAMPLE_RATE_SEQUENCE_SELECT_3:
case ARIZONA_SAMPLE_RATE_SEQUENCE_SELECT_4:
- case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_1:
- case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_2:
case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_3:
case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_4:
+ case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_5:
+ case ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_6:
case ARIZONA_COMFORT_NOISE_GENERATOR:
case ARIZONA_HAPTICS_CONTROL_1:
case ARIZONA_HAPTICS_CONTROL_2:
@@ -846,6 +846,7 @@
case ARIZONA_RATE_ESTIMATOR_3:
case ARIZONA_RATE_ESTIMATOR_4:
case ARIZONA_RATE_ESTIMATOR_5:
+ case ARIZONA_DYNAMIC_FREQUENCY_SCALING_1:
case ARIZONA_FLL1_CONTROL_1:
case ARIZONA_FLL1_CONTROL_2:
case ARIZONA_FLL1_CONTROL_3:
@@ -880,6 +881,7 @@
case ARIZONA_FLL2_GPIO_CLOCK:
case ARIZONA_MIC_CHARGE_PUMP_1:
case ARIZONA_LDO1_CONTROL_1:
+ case ARIZONA_LDO1_CONTROL_2:
case ARIZONA_LDO2_CONTROL_1:
case ARIZONA_MIC_BIAS_CTRL_1:
case ARIZONA_MIC_BIAS_CTRL_2:
diff --git a/drivers/mmc/core/bus.c b/drivers/mmc/core/bus.c
index 8246448..d2dbf02 100644
--- a/drivers/mmc/core/bus.c
+++ b/drivers/mmc/core/bus.c
@@ -341,16 +341,17 @@
if (mmc_host_is_spi(card->host)) {
pr_info("%s: new %s%s%s card on SPI\n",
mmc_hostname(card->host),
- mmc_card_highspeed(card) ? "high speed " : "",
- mmc_card_ddr_mode(card) ? "DDR " : "",
+ mmc_card_hs(card) ? "high speed " : "",
+ mmc_card_ddr52(card) ? "DDR " : "",
type);
} else {
pr_info("%s: new %s%s%s%s%s card at address %04x\n",
mmc_hostname(card->host),
mmc_card_uhs(card) ? "ultra high speed " :
- (mmc_card_highspeed(card) ? "high speed " : ""),
+ (mmc_card_hs(card) ? "high speed " : ""),
+ mmc_card_hs400(card) ? "HS400 " :
(mmc_card_hs200(card) ? "HS200 " : ""),
- mmc_card_ddr_mode(card) ? "DDR " : "",
+ mmc_card_ddr52(card) ? "DDR " : "",
uhs_bus_speed_mode, type, card->rca);
}
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index acbc3f2..7dc0c85 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -800,6 +800,10 @@
data->timeout_ns = limit_us * 1000;
data->timeout_clks = 0;
}
+
+ /* assign limit value if invalid */
+ if (timeout_us == 0)
+ data->timeout_ns = limit_us * 1000;
}
/*
@@ -1310,31 +1314,38 @@
}
EXPORT_SYMBOL_GPL(mmc_regulator_set_ocr);
+#endif /* CONFIG_REGULATOR */
+
int mmc_regulator_get_supply(struct mmc_host *mmc)
{
struct device *dev = mmc_dev(mmc);
- struct regulator *supply;
int ret;
- supply = devm_regulator_get(dev, "vmmc");
- mmc->supply.vmmc = supply;
+ mmc->supply.vmmc = devm_regulator_get_optional(dev, "vmmc");
mmc->supply.vqmmc = devm_regulator_get_optional(dev, "vqmmc");
- if (IS_ERR(supply))
- return PTR_ERR(supply);
+ if (IS_ERR(mmc->supply.vmmc)) {
+ if (PTR_ERR(mmc->supply.vmmc) == -EPROBE_DEFER)
+ return -EPROBE_DEFER;
+ dev_info(dev, "No vmmc regulator found\n");
+ } else {
+ ret = mmc_regulator_get_ocrmask(mmc->supply.vmmc);
+ if (ret > 0)
+ mmc->ocr_avail = ret;
+ else
+ dev_warn(dev, "Failed getting OCR mask: %d\n", ret);
+ }
- ret = mmc_regulator_get_ocrmask(supply);
- if (ret > 0)
- mmc->ocr_avail = ret;
- else
- dev_warn(mmc_dev(mmc), "Failed getting OCR mask: %d\n", ret);
+ if (IS_ERR(mmc->supply.vqmmc)) {
+ if (PTR_ERR(mmc->supply.vqmmc) == -EPROBE_DEFER)
+ return -EPROBE_DEFER;
+ dev_info(dev, "No vqmmc regulator found\n");
+ }
return 0;
}
EXPORT_SYMBOL_GPL(mmc_regulator_get_supply);
-#endif /* CONFIG_REGULATOR */
-
/*
* Mask off any voltages we don't support and select
* the lowest voltage
@@ -1533,8 +1544,13 @@
host->ios.timing = MMC_TIMING_LEGACY;
mmc_set_ios(host);
- /* Set signal voltage to 3.3V */
- __mmc_set_signal_voltage(host, MMC_SIGNAL_VOLTAGE_330);
+ /* Try to set signal voltage to 3.3V but fall back to 1.8v or 1.2v */
+ if (__mmc_set_signal_voltage(host, MMC_SIGNAL_VOLTAGE_330) == 0)
+ dev_dbg(mmc_dev(host), "Initial signal voltage of 3.3v\n");
+ else if (__mmc_set_signal_voltage(host, MMC_SIGNAL_VOLTAGE_180) == 0)
+ dev_dbg(mmc_dev(host), "Initial signal voltage of 1.8v\n");
+ else if (__mmc_set_signal_voltage(host, MMC_SIGNAL_VOLTAGE_120) == 0)
+ dev_dbg(mmc_dev(host), "Initial signal voltage of 1.2v\n");
/*
* This delay should be sufficient to allow the power supply
@@ -2183,7 +2199,7 @@
{
struct mmc_command cmd = {0};
- if (mmc_card_blockaddr(card) || mmc_card_ddr_mode(card))
+ if (mmc_card_blockaddr(card) || mmc_card_ddr52(card))
return 0;
cmd.opcode = MMC_SET_BLOCKLEN;
@@ -2263,7 +2279,6 @@
}
}
- host->card->state &= ~(MMC_STATE_HIGHSPEED | MMC_STATE_HIGHSPEED_DDR);
if (mmc_host_is_spi(host)) {
host->ios.chip_select = MMC_CS_HIGH;
host->ios.bus_mode = MMC_BUSMODE_PUSHPULL;
@@ -2403,6 +2418,11 @@
container_of(work, struct mmc_host, detect.work);
int i;
+ if (host->trigger_card_event && host->ops->card_event) {
+ host->ops->card_event(host);
+ host->trigger_card_event = false;
+ }
+
if (host->rescan_disable)
return;
diff --git a/drivers/mmc/core/debugfs.c b/drivers/mmc/core/debugfs.c
index 54829c0..91eb162 100644
--- a/drivers/mmc/core/debugfs.c
+++ b/drivers/mmc/core/debugfs.c
@@ -135,8 +135,14 @@
case MMC_TIMING_UHS_DDR50:
str = "sd uhs DDR50";
break;
+ case MMC_TIMING_MMC_DDR52:
+ str = "mmc DDR52";
+ break;
case MMC_TIMING_MMC_HS200:
- str = "mmc high-speed SDR200";
+ str = "mmc HS200";
+ break;
+ case MMC_TIMING_MMC_HS400:
+ str = "mmc HS400";
break;
default:
str = "invalid";
diff --git a/drivers/mmc/core/host.c b/drivers/mmc/core/host.c
index fdea825..95cceae 100644
--- a/drivers/mmc/core/host.c
+++ b/drivers/mmc/core/host.c
@@ -447,6 +447,10 @@
host->caps2 |= MMC_CAP2_HS200_1_8V_SDR;
if (of_find_property(np, "mmc-hs200-1_2v", &len))
host->caps2 |= MMC_CAP2_HS200_1_2V_SDR;
+ if (of_find_property(np, "mmc-hs400-1_8v", &len))
+ host->caps2 |= MMC_CAP2_HS400_1_8V | MMC_CAP2_HS200_1_8V_SDR;
+ if (of_find_property(np, "mmc-hs400-1_2v", &len))
+ host->caps2 |= MMC_CAP2_HS400_1_2V | MMC_CAP2_HS200_1_2V_SDR;
return 0;
diff --git a/drivers/mmc/core/mmc.c b/drivers/mmc/core/mmc.c
index 1ab5f3a..793c6f7 100644
--- a/drivers/mmc/core/mmc.c
+++ b/drivers/mmc/core/mmc.c
@@ -240,31 +240,62 @@
static void mmc_select_card_type(struct mmc_card *card)
{
struct mmc_host *host = card->host;
- u8 card_type = card->ext_csd.raw_card_type & EXT_CSD_CARD_TYPE_MASK;
+ u8 card_type = card->ext_csd.raw_card_type;
u32 caps = host->caps, caps2 = host->caps2;
- unsigned int hs_max_dtr = 0;
-
- if (card_type & EXT_CSD_CARD_TYPE_26)
- hs_max_dtr = MMC_HIGH_26_MAX_DTR;
+ unsigned int hs_max_dtr = 0, hs200_max_dtr = 0;
+ unsigned int avail_type = 0;
if (caps & MMC_CAP_MMC_HIGHSPEED &&
- card_type & EXT_CSD_CARD_TYPE_52)
+ card_type & EXT_CSD_CARD_TYPE_HS_26) {
+ hs_max_dtr = MMC_HIGH_26_MAX_DTR;
+ avail_type |= EXT_CSD_CARD_TYPE_HS_26;
+ }
+
+ if (caps & MMC_CAP_MMC_HIGHSPEED &&
+ card_type & EXT_CSD_CARD_TYPE_HS_52) {
hs_max_dtr = MMC_HIGH_52_MAX_DTR;
+ avail_type |= EXT_CSD_CARD_TYPE_HS_52;
+ }
- if ((caps & MMC_CAP_1_8V_DDR &&
- card_type & EXT_CSD_CARD_TYPE_DDR_1_8V) ||
- (caps & MMC_CAP_1_2V_DDR &&
- card_type & EXT_CSD_CARD_TYPE_DDR_1_2V))
+ if (caps & MMC_CAP_1_8V_DDR &&
+ card_type & EXT_CSD_CARD_TYPE_DDR_1_8V) {
hs_max_dtr = MMC_HIGH_DDR_MAX_DTR;
+ avail_type |= EXT_CSD_CARD_TYPE_DDR_1_8V;
+ }
- if ((caps2 & MMC_CAP2_HS200_1_8V_SDR &&
- card_type & EXT_CSD_CARD_TYPE_SDR_1_8V) ||
- (caps2 & MMC_CAP2_HS200_1_2V_SDR &&
- card_type & EXT_CSD_CARD_TYPE_SDR_1_2V))
- hs_max_dtr = MMC_HS200_MAX_DTR;
+ if (caps & MMC_CAP_1_2V_DDR &&
+ card_type & EXT_CSD_CARD_TYPE_DDR_1_2V) {
+ hs_max_dtr = MMC_HIGH_DDR_MAX_DTR;
+ avail_type |= EXT_CSD_CARD_TYPE_DDR_1_2V;
+ }
+
+ if (caps2 & MMC_CAP2_HS200_1_8V_SDR &&
+ card_type & EXT_CSD_CARD_TYPE_HS200_1_8V) {
+ hs200_max_dtr = MMC_HS200_MAX_DTR;
+ avail_type |= EXT_CSD_CARD_TYPE_HS200_1_8V;
+ }
+
+ if (caps2 & MMC_CAP2_HS200_1_2V_SDR &&
+ card_type & EXT_CSD_CARD_TYPE_HS200_1_2V) {
+ hs200_max_dtr = MMC_HS200_MAX_DTR;
+ avail_type |= EXT_CSD_CARD_TYPE_HS200_1_2V;
+ }
+
+ if (caps2 & MMC_CAP2_HS400_1_8V &&
+ card_type & EXT_CSD_CARD_TYPE_HS400_1_8V) {
+ hs200_max_dtr = MMC_HS200_MAX_DTR;
+ avail_type |= EXT_CSD_CARD_TYPE_HS400_1_8V;
+ }
+
+ if (caps2 & MMC_CAP2_HS400_1_2V &&
+ card_type & EXT_CSD_CARD_TYPE_HS400_1_2V) {
+ hs200_max_dtr = MMC_HS200_MAX_DTR;
+ avail_type |= EXT_CSD_CARD_TYPE_HS400_1_2V;
+ }
card->ext_csd.hs_max_dtr = hs_max_dtr;
- card->ext_csd.card_type = card_type;
+ card->ext_csd.hs200_max_dtr = hs200_max_dtr;
+ card->mmc_avail_type = avail_type;
}
/*
@@ -480,6 +511,8 @@
ext_csd[EXT_CSD_PWR_CL_DDR_52_195];
card->ext_csd.raw_pwr_cl_ddr_52_360 =
ext_csd[EXT_CSD_PWR_CL_DDR_52_360];
+ card->ext_csd.raw_pwr_cl_ddr_200_360 =
+ ext_csd[EXT_CSD_PWR_CL_DDR_200_360];
}
if (card->ext_csd.rev >= 5) {
@@ -646,7 +679,10 @@
(card->ext_csd.raw_pwr_cl_ddr_52_195 ==
bw_ext_csd[EXT_CSD_PWR_CL_DDR_52_195]) &&
(card->ext_csd.raw_pwr_cl_ddr_52_360 ==
- bw_ext_csd[EXT_CSD_PWR_CL_DDR_52_360]));
+ bw_ext_csd[EXT_CSD_PWR_CL_DDR_52_360]) &&
+ (card->ext_csd.raw_pwr_cl_ddr_200_360 ==
+ bw_ext_csd[EXT_CSD_PWR_CL_DDR_200_360]));
+
if (err)
err = -EINVAL;
@@ -694,18 +730,10 @@
&dev_attr_rel_sectors.attr,
NULL,
};
-
-static struct attribute_group mmc_std_attr_group = {
- .attrs = mmc_std_attrs,
-};
-
-static const struct attribute_group *mmc_attr_groups[] = {
- &mmc_std_attr_group,
- NULL,
-};
+ATTRIBUTE_GROUPS(mmc_std);
static struct device_type mmc_type = {
- .groups = mmc_attr_groups,
+ .groups = mmc_std_groups,
};
/*
@@ -714,17 +742,13 @@
* extended CSD register, select it by executing the
* mmc_switch command.
*/
-static int mmc_select_powerclass(struct mmc_card *card,
- unsigned int bus_width)
+static int __mmc_select_powerclass(struct mmc_card *card,
+ unsigned int bus_width)
{
- int err = 0;
+ struct mmc_host *host = card->host;
+ struct mmc_ext_csd *ext_csd = &card->ext_csd;
unsigned int pwrclass_val = 0;
- struct mmc_host *host;
-
- BUG_ON(!card);
-
- host = card->host;
- BUG_ON(!host);
+ int err = 0;
/* Power class selection is supported for versions >= 4.0 */
if (card->csd.mmca_vsn < CSD_SPEC_VER_4)
@@ -736,14 +760,14 @@
switch (1 << host->ios.vdd) {
case MMC_VDD_165_195:
- if (host->ios.clock <= 26000000)
- pwrclass_val = card->ext_csd.raw_pwr_cl_26_195;
- else if (host->ios.clock <= 52000000)
+ if (host->ios.clock <= MMC_HIGH_26_MAX_DTR)
+ pwrclass_val = ext_csd->raw_pwr_cl_26_195;
+ else if (host->ios.clock <= MMC_HIGH_52_MAX_DTR)
pwrclass_val = (bus_width <= EXT_CSD_BUS_WIDTH_8) ?
- card->ext_csd.raw_pwr_cl_52_195 :
- card->ext_csd.raw_pwr_cl_ddr_52_195;
- else if (host->ios.clock <= 200000000)
- pwrclass_val = card->ext_csd.raw_pwr_cl_200_195;
+ ext_csd->raw_pwr_cl_52_195 :
+ ext_csd->raw_pwr_cl_ddr_52_195;
+ else if (host->ios.clock <= MMC_HS200_MAX_DTR)
+ pwrclass_val = ext_csd->raw_pwr_cl_200_195;
break;
case MMC_VDD_27_28:
case MMC_VDD_28_29:
@@ -754,14 +778,16 @@
case MMC_VDD_33_34:
case MMC_VDD_34_35:
case MMC_VDD_35_36:
- if (host->ios.clock <= 26000000)
- pwrclass_val = card->ext_csd.raw_pwr_cl_26_360;
- else if (host->ios.clock <= 52000000)
+ if (host->ios.clock <= MMC_HIGH_26_MAX_DTR)
+ pwrclass_val = ext_csd->raw_pwr_cl_26_360;
+ else if (host->ios.clock <= MMC_HIGH_52_MAX_DTR)
pwrclass_val = (bus_width <= EXT_CSD_BUS_WIDTH_8) ?
- card->ext_csd.raw_pwr_cl_52_360 :
- card->ext_csd.raw_pwr_cl_ddr_52_360;
- else if (host->ios.clock <= 200000000)
- pwrclass_val = card->ext_csd.raw_pwr_cl_200_360;
+ ext_csd->raw_pwr_cl_52_360 :
+ ext_csd->raw_pwr_cl_ddr_52_360;
+ else if (host->ios.clock <= MMC_HS200_MAX_DTR)
+ pwrclass_val = (bus_width == EXT_CSD_DDR_BUS_WIDTH_8) ?
+ ext_csd->raw_pwr_cl_ddr_200_360 :
+ ext_csd->raw_pwr_cl_200_360;
break;
default:
pr_warning("%s: Voltage range not supported "
@@ -787,40 +813,79 @@
return err;
}
-/*
- * Selects the desired buswidth and switch to the HS200 mode
- * if bus width set without error
- */
-static int mmc_select_hs200(struct mmc_card *card)
+static int mmc_select_powerclass(struct mmc_card *card)
{
- int idx, err = -EINVAL;
- struct mmc_host *host;
+ struct mmc_host *host = card->host;
+ u32 bus_width, ext_csd_bits;
+ int err, ddr;
+
+ /* Power class selection is supported for versions >= 4.0 */
+ if (card->csd.mmca_vsn < CSD_SPEC_VER_4)
+ return 0;
+
+ bus_width = host->ios.bus_width;
+ /* Power class values are defined only for 4/8 bit bus */
+ if (bus_width == MMC_BUS_WIDTH_1)
+ return 0;
+
+ ddr = card->mmc_avail_type & EXT_CSD_CARD_TYPE_DDR_52;
+ if (ddr)
+ ext_csd_bits = (bus_width == MMC_BUS_WIDTH_8) ?
+ EXT_CSD_DDR_BUS_WIDTH_8 : EXT_CSD_DDR_BUS_WIDTH_4;
+ else
+ ext_csd_bits = (bus_width == MMC_BUS_WIDTH_8) ?
+ EXT_CSD_BUS_WIDTH_8 : EXT_CSD_BUS_WIDTH_4;
+
+ err = __mmc_select_powerclass(card, ext_csd_bits);
+ if (err)
+ pr_warn("%s: power class selection to bus width %d ddr %d failed\n",
+ mmc_hostname(host), 1 << bus_width, ddr);
+
+ return err;
+}
+
+/*
+ * Set the bus speed for the selected speed mode.
+ */
+static void mmc_set_bus_speed(struct mmc_card *card)
+{
+ unsigned int max_dtr = (unsigned int)-1;
+
+ if ((mmc_card_hs200(card) || mmc_card_hs400(card)) &&
+ max_dtr > card->ext_csd.hs200_max_dtr)
+ max_dtr = card->ext_csd.hs200_max_dtr;
+ else if (mmc_card_hs(card) && max_dtr > card->ext_csd.hs_max_dtr)
+ max_dtr = card->ext_csd.hs_max_dtr;
+ else if (max_dtr > card->csd.max_dtr)
+ max_dtr = card->csd.max_dtr;
+
+ mmc_set_clock(card->host, max_dtr);
+}
+
+/*
+ * Select the bus width amoung 4-bit and 8-bit(SDR).
+ * If the bus width is changed successfully, return the selected width value.
+ * Zero is returned instead of error value if the wide width is not supported.
+ */
+static int mmc_select_bus_width(struct mmc_card *card)
+{
static unsigned ext_csd_bits[] = {
- EXT_CSD_BUS_WIDTH_4,
EXT_CSD_BUS_WIDTH_8,
+ EXT_CSD_BUS_WIDTH_4,
};
static unsigned bus_widths[] = {
- MMC_BUS_WIDTH_4,
MMC_BUS_WIDTH_8,
+ MMC_BUS_WIDTH_4,
};
+ struct mmc_host *host = card->host;
+ unsigned idx, bus_width = 0;
+ int err = 0;
- BUG_ON(!card);
+ if ((card->csd.mmca_vsn < CSD_SPEC_VER_4) &&
+ !(host->caps & (MMC_CAP_4_BIT_DATA | MMC_CAP_8_BIT_DATA)))
+ return 0;
- host = card->host;
-
- if (card->ext_csd.card_type & EXT_CSD_CARD_TYPE_SDR_1_2V &&
- host->caps2 & MMC_CAP2_HS200_1_2V_SDR)
- err = __mmc_set_signal_voltage(host, MMC_SIGNAL_VOLTAGE_120);
-
- if (err && card->ext_csd.card_type & EXT_CSD_CARD_TYPE_SDR_1_8V &&
- host->caps2 & MMC_CAP2_HS200_1_8V_SDR)
- err = __mmc_set_signal_voltage(host, MMC_SIGNAL_VOLTAGE_180);
-
- /* If fails try again during next card power cycle */
- if (err)
- goto err;
-
- idx = (host->caps & MMC_CAP_8_BIT_DATA) ? 1 : 0;
+ idx = (host->caps & MMC_CAP_8_BIT_DATA) ? 0 : 1;
/*
* Unlike SD, MMC cards dont have a configuration register to notify
@@ -828,8 +893,7 @@
* the supported bus width or compare the ext csd values of current
* bus width and ext csd values of 1 bit mode read earlier.
*/
- for (; idx >= 0; idx--) {
-
+ for (; idx < ARRAY_SIZE(bus_widths); idx++) {
/*
* Host is capable of 8bit transfer, then switch
* the device to work in 8bit transfer mode. If the
@@ -844,27 +908,266 @@
if (err)
continue;
- mmc_set_bus_width(card->host, bus_widths[idx]);
+ bus_width = bus_widths[idx];
+ mmc_set_bus_width(host, bus_width);
+ /*
+ * If controller can't handle bus width test,
+ * compare ext_csd previously read in 1 bit mode
+ * against ext_csd at new bus width
+ */
if (!(host->caps & MMC_CAP_BUS_WIDTH_TEST))
- err = mmc_compare_ext_csds(card, bus_widths[idx]);
+ err = mmc_compare_ext_csds(card, bus_width);
else
- err = mmc_bus_test(card, bus_widths[idx]);
- if (!err)
+ err = mmc_bus_test(card, bus_width);
+
+ if (!err) {
+ err = bus_width;
break;
+ } else {
+ pr_warn("%s: switch to bus width %d failed\n",
+ mmc_hostname(host), ext_csd_bits[idx]);
+ }
}
- /* switch to HS200 mode if bus width set successfully */
+ return err;
+}
+
+/*
+ * Switch to the high-speed mode
+ */
+static int mmc_select_hs(struct mmc_card *card)
+{
+ int err;
+
+ err = __mmc_switch(card, EXT_CSD_CMD_SET_NORMAL,
+ EXT_CSD_HS_TIMING, EXT_CSD_TIMING_HS,
+ card->ext_csd.generic_cmd6_time,
+ true, true, true);
if (!err)
+ mmc_set_timing(card->host, MMC_TIMING_MMC_HS);
+
+ return err;
+}
+
+/*
+ * Activate wide bus and DDR if supported.
+ */
+static int mmc_select_hs_ddr(struct mmc_card *card)
+{
+ struct mmc_host *host = card->host;
+ u32 bus_width, ext_csd_bits;
+ int err = 0;
+
+ if (!(card->mmc_avail_type & EXT_CSD_CARD_TYPE_DDR_52))
+ return 0;
+
+ bus_width = host->ios.bus_width;
+ if (bus_width == MMC_BUS_WIDTH_1)
+ return 0;
+
+ ext_csd_bits = (bus_width == MMC_BUS_WIDTH_8) ?
+ EXT_CSD_DDR_BUS_WIDTH_8 : EXT_CSD_DDR_BUS_WIDTH_4;
+
+ err = mmc_switch(card, EXT_CSD_CMD_SET_NORMAL,
+ EXT_CSD_BUS_WIDTH,
+ ext_csd_bits,
+ card->ext_csd.generic_cmd6_time);
+ if (err) {
+ pr_warn("%s: switch to bus width %d ddr failed\n",
+ mmc_hostname(host), 1 << bus_width);
+ return err;
+ }
+
+ /*
+ * eMMC cards can support 3.3V to 1.2V i/o (vccq)
+ * signaling.
+ *
+ * EXT_CSD_CARD_TYPE_DDR_1_8V means 3.3V or 1.8V vccq.
+ *
+ * 1.8V vccq at 3.3V core voltage (vcc) is not required
+ * in the JEDEC spec for DDR.
+ *
+ * Do not force change in vccq since we are obviously
+ * working and no change to vccq is needed.
+ *
+ * WARNING: eMMC rules are NOT the same as SD DDR
+ */
+ if (card->mmc_avail_type & EXT_CSD_CARD_TYPE_DDR_1_2V) {
+ err = __mmc_set_signal_voltage(host,
+ MMC_SIGNAL_VOLTAGE_120);
+ if (err)
+ return err;
+ }
+
+ mmc_set_timing(host, MMC_TIMING_MMC_DDR52);
+
+ return err;
+}
+
+static int mmc_select_hs400(struct mmc_card *card)
+{
+ struct mmc_host *host = card->host;
+ int err = 0;
+
+ /*
+ * HS400 mode requires 8-bit bus width
+ */
+ if (!(card->mmc_avail_type & EXT_CSD_CARD_TYPE_HS400 &&
+ host->ios.bus_width == MMC_BUS_WIDTH_8))
+ return 0;
+
+ /*
+ * Before switching to dual data rate operation for HS400,
+ * it is required to convert from HS200 mode to HS mode.
+ */
+ mmc_set_timing(card->host, MMC_TIMING_MMC_HS);
+ mmc_set_bus_speed(card);
+
+ err = __mmc_switch(card, EXT_CSD_CMD_SET_NORMAL,
+ EXT_CSD_HS_TIMING, EXT_CSD_TIMING_HS,
+ card->ext_csd.generic_cmd6_time,
+ true, true, true);
+ if (err) {
+ pr_warn("%s: switch to high-speed from hs200 failed, err:%d\n",
+ mmc_hostname(host), err);
+ return err;
+ }
+
+ err = mmc_switch(card, EXT_CSD_CMD_SET_NORMAL,
+ EXT_CSD_BUS_WIDTH,
+ EXT_CSD_DDR_BUS_WIDTH_8,
+ card->ext_csd.generic_cmd6_time);
+ if (err) {
+ pr_warn("%s: switch to bus width for hs400 failed, err:%d\n",
+ mmc_hostname(host), err);
+ return err;
+ }
+
+ err = __mmc_switch(card, EXT_CSD_CMD_SET_NORMAL,
+ EXT_CSD_HS_TIMING, EXT_CSD_TIMING_HS400,
+ card->ext_csd.generic_cmd6_time,
+ true, true, true);
+ if (err) {
+ pr_warn("%s: switch to hs400 failed, err:%d\n",
+ mmc_hostname(host), err);
+ return err;
+ }
+
+ mmc_set_timing(host, MMC_TIMING_MMC_HS400);
+ mmc_set_bus_speed(card);
+
+ return 0;
+}
+
+/*
+ * For device supporting HS200 mode, the following sequence
+ * should be done before executing the tuning process.
+ * 1. set the desired bus width(4-bit or 8-bit, 1-bit is not supported)
+ * 2. switch to HS200 mode
+ * 3. set the clock to > 52Mhz and <=200MHz
+ */
+static int mmc_select_hs200(struct mmc_card *card)
+{
+ struct mmc_host *host = card->host;
+ int err = -EINVAL;
+
+ if (card->mmc_avail_type & EXT_CSD_CARD_TYPE_HS200_1_2V)
+ err = __mmc_set_signal_voltage(host, MMC_SIGNAL_VOLTAGE_120);
+
+ if (err && card->mmc_avail_type & EXT_CSD_CARD_TYPE_HS200_1_8V)
+ err = __mmc_set_signal_voltage(host, MMC_SIGNAL_VOLTAGE_180);
+
+ /* If fails try again during next card power cycle */
+ if (err)
+ goto err;
+
+ /*
+ * Set the bus width(4 or 8) with host's support and
+ * switch to HS200 mode if bus width is set successfully.
+ */
+ err = mmc_select_bus_width(card);
+ if (!IS_ERR_VALUE(err)) {
err = __mmc_switch(card, EXT_CSD_CMD_SET_NORMAL,
- EXT_CSD_HS_TIMING, 2,
- card->ext_csd.generic_cmd6_time,
- true, true, true);
+ EXT_CSD_HS_TIMING, EXT_CSD_TIMING_HS200,
+ card->ext_csd.generic_cmd6_time,
+ true, true, true);
+ if (!err)
+ mmc_set_timing(host, MMC_TIMING_MMC_HS200);
+ }
err:
return err;
}
/*
+ * Activate High Speed or HS200 mode if supported.
+ */
+static int mmc_select_timing(struct mmc_card *card)
+{
+ int err = 0;
+
+ if ((card->csd.mmca_vsn < CSD_SPEC_VER_4 &&
+ card->ext_csd.hs_max_dtr == 0))
+ goto bus_speed;
+
+ if (card->mmc_avail_type & EXT_CSD_CARD_TYPE_HS200)
+ err = mmc_select_hs200(card);
+ else if (card->mmc_avail_type & EXT_CSD_CARD_TYPE_HS)
+ err = mmc_select_hs(card);
+
+ if (err && err != -EBADMSG)
+ return err;
+
+ if (err) {
+ pr_warn("%s: switch to %s failed\n",
+ mmc_card_hs(card) ? "high-speed" :
+ (mmc_card_hs200(card) ? "hs200" : ""),
+ mmc_hostname(card->host));
+ err = 0;
+ }
+
+bus_speed:
+ /*
+ * Set the bus speed to the selected bus timing.
+ * If timing is not selected, backward compatible is the default.
+ */
+ mmc_set_bus_speed(card);
+ return err;
+}
+
+/*
+ * Execute tuning sequence to seek the proper bus operating
+ * conditions for HS200 and HS400, which sends CMD21 to the device.
+ */
+static int mmc_hs200_tuning(struct mmc_card *card)
+{
+ struct mmc_host *host = card->host;
+ int err = 0;
+
+ /*
+ * Timing should be adjusted to the HS400 target
+ * operation frequency for tuning process
+ */
+ if (card->mmc_avail_type & EXT_CSD_CARD_TYPE_HS400 &&
+ host->ios.bus_width == MMC_BUS_WIDTH_8)
+ if (host->ops->prepare_hs400_tuning)
+ host->ops->prepare_hs400_tuning(host, &host->ios);
+
+ if (host->ops->execute_tuning) {
+ mmc_host_clk_hold(host);
+ err = host->ops->execute_tuning(host,
+ MMC_SEND_TUNING_BLOCK_HS200);
+ mmc_host_clk_release(host);
+
+ if (err)
+ pr_warn("%s: tuning execution failed\n",
+ mmc_hostname(host));
+ }
+
+ return err;
+}
+
+/*
* Handle the detection and initialisation of a card.
*
* In the case of a resume, "oldcard" will contain the card
@@ -874,9 +1177,8 @@
struct mmc_card *oldcard)
{
struct mmc_card *card;
- int err, ddr = 0;
+ int err;
u32 cid[4];
- unsigned int max_dtr;
u32 rocr;
u8 *ext_csd = NULL;
@@ -1068,206 +1370,34 @@
}
/*
- * Activate high speed (if supported)
+ * Select timing interface
*/
- if (card->ext_csd.hs_max_dtr != 0) {
- err = 0;
- if (card->ext_csd.hs_max_dtr > 52000000 &&
- host->caps2 & MMC_CAP2_HS200)
- err = mmc_select_hs200(card);
- else if (host->caps & MMC_CAP_MMC_HIGHSPEED)
- err = __mmc_switch(card, EXT_CSD_CMD_SET_NORMAL,
- EXT_CSD_HS_TIMING, 1,
- card->ext_csd.generic_cmd6_time,
- true, true, true);
+ err = mmc_select_timing(card);
+ if (err)
+ goto free_card;
- if (err && err != -EBADMSG)
- goto free_card;
-
- if (err) {
- pr_warning("%s: switch to highspeed failed\n",
- mmc_hostname(card->host));
- err = 0;
- } else {
- if (card->ext_csd.hs_max_dtr > 52000000 &&
- host->caps2 & MMC_CAP2_HS200) {
- mmc_card_set_hs200(card);
- mmc_set_timing(card->host,
- MMC_TIMING_MMC_HS200);
- } else {
- mmc_card_set_highspeed(card);
- mmc_set_timing(card->host, MMC_TIMING_MMC_HS);
- }
- }
- }
-
- /*
- * Compute bus speed.
- */
- max_dtr = (unsigned int)-1;
-
- if (mmc_card_highspeed(card) || mmc_card_hs200(card)) {
- if (max_dtr > card->ext_csd.hs_max_dtr)
- max_dtr = card->ext_csd.hs_max_dtr;
- if (mmc_card_highspeed(card) && (max_dtr > 52000000))
- max_dtr = 52000000;
- } else if (max_dtr > card->csd.max_dtr) {
- max_dtr = card->csd.max_dtr;
- }
-
- mmc_set_clock(host, max_dtr);
-
- /*
- * Indicate DDR mode (if supported).
- */
- if (mmc_card_highspeed(card)) {
- if ((card->ext_csd.card_type & EXT_CSD_CARD_TYPE_DDR_1_8V)
- && (host->caps & MMC_CAP_1_8V_DDR))
- ddr = MMC_1_8V_DDR_MODE;
- else if ((card->ext_csd.card_type & EXT_CSD_CARD_TYPE_DDR_1_2V)
- && (host->caps & MMC_CAP_1_2V_DDR))
- ddr = MMC_1_2V_DDR_MODE;
- }
-
- /*
- * Indicate HS200 SDR mode (if supported).
- */
if (mmc_card_hs200(card)) {
- u32 ext_csd_bits;
- u32 bus_width = card->host->ios.bus_width;
-
- /*
- * For devices supporting HS200 mode, the bus width has
- * to be set before executing the tuning function. If
- * set before tuning, then device will respond with CRC
- * errors for responses on CMD line. So for HS200 the
- * sequence will be
- * 1. set bus width 4bit / 8 bit (1 bit not supported)
- * 2. switch to HS200 mode
- * 3. set the clock to > 52Mhz <=200MHz and
- * 4. execute tuning for HS200
- */
- if ((host->caps2 & MMC_CAP2_HS200) &&
- card->host->ops->execute_tuning) {
- mmc_host_clk_hold(card->host);
- err = card->host->ops->execute_tuning(card->host,
- MMC_SEND_TUNING_BLOCK_HS200);
- mmc_host_clk_release(card->host);
- }
- if (err) {
- pr_warning("%s: tuning execution failed\n",
- mmc_hostname(card->host));
- goto err;
- }
-
- ext_csd_bits = (bus_width == MMC_BUS_WIDTH_8) ?
- EXT_CSD_BUS_WIDTH_8 : EXT_CSD_BUS_WIDTH_4;
- err = mmc_select_powerclass(card, ext_csd_bits);
+ err = mmc_hs200_tuning(card);
if (err)
- pr_warning("%s: power class selection to bus width %d"
- " failed\n", mmc_hostname(card->host),
- 1 << bus_width);
+ goto err;
+
+ err = mmc_select_hs400(card);
+ if (err)
+ goto err;
+ } else if (mmc_card_hs(card)) {
+ /* Select the desired bus width optionally */
+ err = mmc_select_bus_width(card);
+ if (!IS_ERR_VALUE(err)) {
+ err = mmc_select_hs_ddr(card);
+ if (err)
+ goto err;
+ }
}
/*
- * Activate wide bus and DDR (if supported).
+ * Choose the power class with selected bus interface
*/
- if (!mmc_card_hs200(card) &&
- (card->csd.mmca_vsn >= CSD_SPEC_VER_4) &&
- (host->caps & (MMC_CAP_4_BIT_DATA | MMC_CAP_8_BIT_DATA))) {
- static unsigned ext_csd_bits[][2] = {
- { EXT_CSD_BUS_WIDTH_8, EXT_CSD_DDR_BUS_WIDTH_8 },
- { EXT_CSD_BUS_WIDTH_4, EXT_CSD_DDR_BUS_WIDTH_4 },
- { EXT_CSD_BUS_WIDTH_1, EXT_CSD_BUS_WIDTH_1 },
- };
- static unsigned bus_widths[] = {
- MMC_BUS_WIDTH_8,
- MMC_BUS_WIDTH_4,
- MMC_BUS_WIDTH_1
- };
- unsigned idx, bus_width = 0;
-
- if (host->caps & MMC_CAP_8_BIT_DATA)
- idx = 0;
- else
- idx = 1;
- for (; idx < ARRAY_SIZE(bus_widths); idx++) {
- bus_width = bus_widths[idx];
- if (bus_width == MMC_BUS_WIDTH_1)
- ddr = 0; /* no DDR for 1-bit width */
- err = mmc_select_powerclass(card, ext_csd_bits[idx][0]);
- if (err)
- pr_warning("%s: power class selection to "
- "bus width %d failed\n",
- mmc_hostname(card->host),
- 1 << bus_width);
-
- err = mmc_switch(card, EXT_CSD_CMD_SET_NORMAL,
- EXT_CSD_BUS_WIDTH,
- ext_csd_bits[idx][0],
- card->ext_csd.generic_cmd6_time);
- if (!err) {
- mmc_set_bus_width(card->host, bus_width);
-
- /*
- * If controller can't handle bus width test,
- * compare ext_csd previously read in 1 bit mode
- * against ext_csd at new bus width
- */
- if (!(host->caps & MMC_CAP_BUS_WIDTH_TEST))
- err = mmc_compare_ext_csds(card,
- bus_width);
- else
- err = mmc_bus_test(card, bus_width);
- if (!err)
- break;
- }
- }
-
- if (!err && ddr) {
- err = mmc_select_powerclass(card, ext_csd_bits[idx][1]);
- if (err)
- pr_warning("%s: power class selection to "
- "bus width %d ddr %d failed\n",
- mmc_hostname(card->host),
- 1 << bus_width, ddr);
-
- err = mmc_switch(card, EXT_CSD_CMD_SET_NORMAL,
- EXT_CSD_BUS_WIDTH,
- ext_csd_bits[idx][1],
- card->ext_csd.generic_cmd6_time);
- }
- if (err) {
- pr_warning("%s: switch to bus width %d ddr %d "
- "failed\n", mmc_hostname(card->host),
- 1 << bus_width, ddr);
- goto free_card;
- } else if (ddr) {
- /*
- * eMMC cards can support 3.3V to 1.2V i/o (vccq)
- * signaling.
- *
- * EXT_CSD_CARD_TYPE_DDR_1_8V means 3.3V or 1.8V vccq.
- *
- * 1.8V vccq at 3.3V core voltage (vcc) is not required
- * in the JEDEC spec for DDR.
- *
- * Do not force change in vccq since we are obviously
- * working and no change to vccq is needed.
- *
- * WARNING: eMMC rules are NOT the same as SD DDR
- */
- if (ddr == MMC_1_2V_DDR_MODE) {
- err = __mmc_set_signal_voltage(host,
- MMC_SIGNAL_VOLTAGE_120);
- if (err)
- goto err;
- }
- mmc_card_set_ddr_mode(card);
- mmc_set_timing(card->host, MMC_TIMING_UHS_DDR50);
- mmc_set_bus_width(card->host, bus_width);
- }
- }
+ mmc_select_powerclass(card);
/*
* Enable HPI feature (if supported)
@@ -1507,7 +1637,6 @@
err = mmc_sleep(host);
else if (!mmc_host_is_spi(host))
err = mmc_deselect_cards(host);
- host->card->state &= ~(MMC_STATE_HIGHSPEED | MMC_STATE_HIGHSPEED_200);
if (!err) {
mmc_power_off(host);
@@ -1637,7 +1766,6 @@
{
int ret;
- host->card->state &= ~(MMC_STATE_HIGHSPEED | MMC_STATE_HIGHSPEED_200);
mmc_claim_host(host);
ret = mmc_init_card(host, host->card->ocr, host->card);
mmc_release_host(host);
diff --git a/drivers/mmc/core/sd.c b/drivers/mmc/core/sd.c
index 2dd359d..0c44510 100644
--- a/drivers/mmc/core/sd.c
+++ b/drivers/mmc/core/sd.c
@@ -707,18 +707,10 @@
&dev_attr_serial.attr,
NULL,
};
-
-static struct attribute_group sd_std_attr_group = {
- .attrs = sd_std_attrs,
-};
-
-static const struct attribute_group *sd_attr_groups[] = {
- &sd_std_attr_group,
- NULL,
-};
+ATTRIBUTE_GROUPS(sd_std);
struct device_type sd_type = {
- .groups = sd_attr_groups,
+ .groups = sd_std_groups,
};
/*
@@ -895,7 +887,7 @@
{
unsigned max_dtr = (unsigned int)-1;
- if (mmc_card_highspeed(card)) {
+ if (mmc_card_hs(card)) {
if (max_dtr > card->sw_caps.hs_max_dtr)
max_dtr = card->sw_caps.hs_max_dtr;
} else if (max_dtr > card->csd.max_dtr) {
@@ -905,12 +897,6 @@
return max_dtr;
}
-void mmc_sd_go_highspeed(struct mmc_card *card)
-{
- mmc_card_set_highspeed(card);
- mmc_set_timing(card->host, MMC_TIMING_SD_HS);
-}
-
/*
* Handle the detection and initialisation of a card.
*
@@ -985,16 +971,13 @@
err = mmc_sd_init_uhs_card(card);
if (err)
goto free_card;
-
- /* Card is an ultra-high-speed card */
- mmc_card_set_uhs(card);
} else {
/*
* Attempt to change to high-speed (if supported)
*/
err = mmc_sd_switch_hs(card);
if (err > 0)
- mmc_sd_go_highspeed(card);
+ mmc_set_timing(card->host, MMC_TIMING_SD_HS);
else if (err)
goto free_card;
@@ -1089,7 +1072,7 @@
if (!mmc_host_is_spi(host))
err = mmc_deselect_cards(host);
- host->card->state &= ~MMC_STATE_HIGHSPEED;
+
if (!err) {
mmc_power_off(host);
mmc_card_set_suspended(host->card);
@@ -1198,7 +1181,6 @@
{
int ret;
- host->card->state &= ~MMC_STATE_HIGHSPEED;
mmc_claim_host(host);
ret = mmc_sd_init_card(host, host->card->ocr, host->card);
mmc_release_host(host);
diff --git a/drivers/mmc/core/sd.h b/drivers/mmc/core/sd.h
index 4b34b24..aab824a 100644
--- a/drivers/mmc/core/sd.h
+++ b/drivers/mmc/core/sd.h
@@ -12,6 +12,5 @@
bool reinit);
unsigned mmc_sd_get_max_clock(struct mmc_card *card);
int mmc_sd_switch_hs(struct mmc_card *card);
-void mmc_sd_go_highspeed(struct mmc_card *card);
#endif
diff --git a/drivers/mmc/core/sdio.c b/drivers/mmc/core/sdio.c
index 4d721c6..e636d9e 100644
--- a/drivers/mmc/core/sdio.c
+++ b/drivers/mmc/core/sdio.c
@@ -363,7 +363,7 @@
{
unsigned max_dtr;
- if (mmc_card_highspeed(card)) {
+ if (mmc_card_hs(card)) {
/*
* The SDIO specification doesn't mention how
* the CIS transfer speed register relates to
@@ -733,7 +733,6 @@
mmc_set_clock(host, card->cis.max_dtr);
if (card->cccr.high_speed) {
- mmc_card_set_highspeed(card);
mmc_set_timing(card->host, MMC_TIMING_SD_HS);
}
@@ -792,16 +791,13 @@
err = mmc_sdio_init_uhs_card(card);
if (err)
goto remove;
-
- /* Card is an ultra-high-speed card */
- mmc_card_set_uhs(card);
} else {
/*
* Switch to high-speed (if supported).
*/
err = sdio_enable_hs(card);
if (err > 0)
- mmc_sd_go_highspeed(card);
+ mmc_set_timing(card->host, MMC_TIMING_SD_HS);
else if (err)
goto remove;
@@ -943,40 +939,21 @@
*/
static int mmc_sdio_suspend(struct mmc_host *host)
{
- int i, err = 0;
-
- for (i = 0; i < host->card->sdio_funcs; i++) {
- struct sdio_func *func = host->card->sdio_func[i];
- if (func && sdio_func_present(func) && func->dev.driver) {
- const struct dev_pm_ops *pmops = func->dev.driver->pm;
- err = pmops->suspend(&func->dev);
- if (err)
- break;
- }
- }
- while (err && --i >= 0) {
- struct sdio_func *func = host->card->sdio_func[i];
- if (func && sdio_func_present(func) && func->dev.driver) {
- const struct dev_pm_ops *pmops = func->dev.driver->pm;
- pmops->resume(&func->dev);
- }
- }
-
- if (!err && mmc_card_keep_power(host) && mmc_card_wake_sdio_irq(host)) {
+ if (mmc_card_keep_power(host) && mmc_card_wake_sdio_irq(host)) {
mmc_claim_host(host);
sdio_disable_wide(host->card);
mmc_release_host(host);
}
- if (!err && !mmc_card_keep_power(host))
+ if (!mmc_card_keep_power(host))
mmc_power_off(host);
- return err;
+ return 0;
}
static int mmc_sdio_resume(struct mmc_host *host)
{
- int i, err = 0;
+ int err = 0;
BUG_ON(!host);
BUG_ON(!host->card);
@@ -1019,24 +996,6 @@
wake_up_process(host->sdio_irq_thread);
mmc_release_host(host);
- /*
- * If the card looked to be the same as before suspending, then
- * we proceed to resume all card functions. If one of them returns
- * an error then we simply return that error to the core and the
- * card will be redetected as new. It is the responsibility of
- * the function driver to perform further tests with the extra
- * knowledge it has of the card to confirm the card is indeed the
- * same as before suspending (same MAC address for network cards,
- * etc.) and return an error otherwise.
- */
- for (i = 0; !err && i < host->card->sdio_funcs; i++) {
- struct sdio_func *func = host->card->sdio_func[i];
- if (func && sdio_func_present(func) && func->dev.driver) {
- const struct dev_pm_ops *pmops = func->dev.driver->pm;
- err = pmops->resume(&func->dev);
- }
- }
-
host->pm_flags &= ~MMC_PM_KEEP_POWER;
return err;
}
diff --git a/drivers/mmc/core/sdio_bus.c b/drivers/mmc/core/sdio_bus.c
index 92d1ba8..4fa8fef9 100644
--- a/drivers/mmc/core/sdio_bus.c
+++ b/drivers/mmc/core/sdio_bus.c
@@ -197,20 +197,8 @@
#ifdef CONFIG_PM
-#ifdef CONFIG_PM_SLEEP
-static int pm_no_operation(struct device *dev)
-{
- /*
- * Prevent the PM core from calling SDIO device drivers' suspend
- * callback routines, which it is not supposed to do, by using this
- * empty function as the bus type suspend callaback for SDIO.
- */
- return 0;
-}
-#endif
-
static const struct dev_pm_ops sdio_bus_pm_ops = {
- SET_SYSTEM_SLEEP_PM_OPS(pm_no_operation, pm_no_operation)
+ SET_SYSTEM_SLEEP_PM_OPS(pm_generic_suspend, pm_generic_resume)
SET_RUNTIME_PM_OPS(
pm_generic_runtime_suspend,
pm_generic_runtime_resume,
diff --git a/drivers/mmc/core/sdio_irq.c b/drivers/mmc/core/sdio_irq.c
index aaa9046..5cc13c8 100644
--- a/drivers/mmc/core/sdio_irq.c
+++ b/drivers/mmc/core/sdio_irq.c
@@ -90,6 +90,15 @@
return ret;
}
+void sdio_run_irqs(struct mmc_host *host)
+{
+ mmc_claim_host(host);
+ host->sdio_irq_pending = true;
+ process_sdio_pending_irqs(host);
+ mmc_release_host(host);
+}
+EXPORT_SYMBOL_GPL(sdio_run_irqs);
+
static int sdio_irq_thread(void *_host)
{
struct mmc_host *host = _host;
@@ -189,14 +198,20 @@
WARN_ON(!host->claimed);
if (!host->sdio_irqs++) {
- atomic_set(&host->sdio_irq_thread_abort, 0);
- host->sdio_irq_thread =
- kthread_run(sdio_irq_thread, host, "ksdioirqd/%s",
- mmc_hostname(host));
- if (IS_ERR(host->sdio_irq_thread)) {
- int err = PTR_ERR(host->sdio_irq_thread);
- host->sdio_irqs--;
- return err;
+ if (!(host->caps2 & MMC_CAP2_SDIO_IRQ_NOTHREAD)) {
+ atomic_set(&host->sdio_irq_thread_abort, 0);
+ host->sdio_irq_thread =
+ kthread_run(sdio_irq_thread, host,
+ "ksdioirqd/%s", mmc_hostname(host));
+ if (IS_ERR(host->sdio_irq_thread)) {
+ int err = PTR_ERR(host->sdio_irq_thread);
+ host->sdio_irqs--;
+ return err;
+ }
+ } else {
+ mmc_host_clk_hold(host);
+ host->ops->enable_sdio_irq(host, 1);
+ mmc_host_clk_release(host);
}
}
@@ -211,8 +226,14 @@
BUG_ON(host->sdio_irqs < 1);
if (!--host->sdio_irqs) {
- atomic_set(&host->sdio_irq_thread_abort, 1);
- kthread_stop(host->sdio_irq_thread);
+ if (!(host->caps2 & MMC_CAP2_SDIO_IRQ_NOTHREAD)) {
+ atomic_set(&host->sdio_irq_thread_abort, 1);
+ kthread_stop(host->sdio_irq_thread);
+ } else {
+ mmc_host_clk_hold(host);
+ host->ops->enable_sdio_irq(host, 0);
+ mmc_host_clk_release(host);
+ }
}
return 0;
diff --git a/drivers/mmc/core/slot-gpio.c b/drivers/mmc/core/slot-gpio.c
index f7650b8..5f89cb8 100644
--- a/drivers/mmc/core/slot-gpio.c
+++ b/drivers/mmc/core/slot-gpio.c
@@ -32,9 +32,7 @@
/* Schedule a card detection after a debounce timeout */
struct mmc_host *host = dev_id;
- if (host->ops->card_event)
- host->ops->card_event(host);
-
+ host->trigger_card_event = true;
mmc_detect_change(host, msecs_to_jiffies(200));
return IRQ_HANDLED;
diff --git a/drivers/mmc/host/Kconfig b/drivers/mmc/host/Kconfig
index 8aaf8c1..7fee224 100644
--- a/drivers/mmc/host/Kconfig
+++ b/drivers/mmc/host/Kconfig
@@ -168,7 +168,7 @@
config MMC_SDHCI_DOVE
tristate "SDHCI support on Marvell's Dove SoC"
- depends on ARCH_DOVE
+ depends on ARCH_DOVE || MACH_DOVE
depends on MMC_SDHCI_PLTFM
select MMC_SDHCI_IO_ACCESSORS
help
@@ -283,6 +283,15 @@
If unsure, say N.
+config MMC_MOXART
+ tristate "MOXART SD/MMC Host Controller support"
+ depends on ARCH_MOXART && MMC
+ help
+ This selects support for the MOXART SD/MMC Host Controller.
+ MOXA provides one multi-functional card reader which can
+ be found on some embedded hardware such as UC-7112-LX.
+ If you have a controller with this interface, say Y here.
+
config MMC_OMAP
tristate "TI OMAP Multimedia Card Interface support"
depends on ARCH_OMAP
@@ -688,9 +697,29 @@
To compile this driver as a module, choose M here: the
module will be called wmt-sdmmc.
+config MMC_USDHI6ROL0
+ tristate "Renesas USDHI6ROL0 SD/SDIO Host Controller support"
+ help
+ This selects support for the Renesas USDHI6ROL0 SD/SDIO
+ Host Controller
+
config MMC_REALTEK_PCI
tristate "Realtek PCI-E SD/MMC Card Interface Driver"
depends on MFD_RTSX_PCI
help
Say Y here to include driver code to support SD/MMC card interface
of Realtek PCI-E card reader
+
+config MMC_REALTEK_USB
+ tristate "Realtek USB SD/MMC Card Interface Driver"
+ depends on MFD_RTSX_USB
+ help
+ Say Y here to include driver code to support SD/MMC card interface
+ of Realtek RTS5129/39 series card reader
+
+config MMC_SUNXI
+ tristate "Allwinner sunxi SD/MMC Host Controller support"
+ depends on ARCH_SUNXI
+ help
+ This selects support for the SD/MMC Host Controller on
+ Allwinner sunxi SoCs.
diff --git a/drivers/mmc/host/Makefile b/drivers/mmc/host/Makefile
index 0c8aa5e..7f81ddf 100644
--- a/drivers/mmc/host/Makefile
+++ b/drivers/mmc/host/Makefile
@@ -50,8 +50,12 @@
obj-$(CONFIG_MMC_VUB300) += vub300.o
obj-$(CONFIG_MMC_USHC) += ushc.o
obj-$(CONFIG_MMC_WMT) += wmt-sdmmc.o
+obj-$(CONFIG_MMC_MOXART) += moxart-mmc.o
+obj-$(CONFIG_MMC_SUNXI) += sunxi-mmc.o
+obj-$(CONFIG_MMC_USDHI6ROL0) += usdhi6rol0.o
obj-$(CONFIG_MMC_REALTEK_PCI) += rtsx_pci_sdmmc.o
+obj-$(CONFIG_MMC_REALTEK_USB) += rtsx_usb_sdmmc.o
obj-$(CONFIG_MMC_SDHCI_PLTFM) += sdhci-pltfm.o
obj-$(CONFIG_MMC_SDHCI_CNS3XXX) += sdhci-cns3xxx.o
diff --git a/drivers/mmc/host/atmel-mci.c b/drivers/mmc/host/atmel-mci.c
index 42706ea..aece7ca 100644
--- a/drivers/mmc/host/atmel-mci.c
+++ b/drivers/mmc/host/atmel-mci.c
@@ -820,16 +820,9 @@
atmci_pdc_cleanup(host);
- /*
- * If the card was removed, data will be NULL. No point trying
- * to send the stop command or waiting for NBUSY in this case.
- */
- if (host->data) {
- dev_dbg(&host->pdev->dev,
- "(%s) set pending xfer complete\n", __func__);
- atmci_set_pending(host, EVENT_XFER_COMPLETE);
- tasklet_schedule(&host->tasklet);
- }
+ dev_dbg(&host->pdev->dev, "(%s) set pending xfer complete\n", __func__);
+ atmci_set_pending(host, EVENT_XFER_COMPLETE);
+ tasklet_schedule(&host->tasklet);
}
static void atmci_dma_cleanup(struct atmel_mci *host)
diff --git a/drivers/mmc/host/dw_mmc-exynos.c b/drivers/mmc/host/dw_mmc-exynos.c
index 3423c5e..0fbc53a 100644
--- a/drivers/mmc/host/dw_mmc-exynos.c
+++ b/drivers/mmc/host/dw_mmc-exynos.c
@@ -187,7 +187,7 @@
unsigned long actual;
u8 div = priv->ciu_div + 1;
- if (ios->timing == MMC_TIMING_UHS_DDR50) {
+ if (ios->timing == MMC_TIMING_MMC_DDR52) {
mci_writel(host, CLKSEL, priv->ddr_timing);
/* Should be double rate for DDR mode */
if (ios->bus_width == MMC_BUS_WIDTH_8)
@@ -386,8 +386,7 @@
/* Common capabilities of Exynos4/Exynos5 SoC */
static unsigned long exynos_dwmmc_caps[4] = {
- MMC_CAP_UHS_DDR50 | MMC_CAP_1_8V_DDR |
- MMC_CAP_8_BIT_DATA | MMC_CAP_CMD23,
+ MMC_CAP_1_8V_DDR | MMC_CAP_8_BIT_DATA | MMC_CAP_CMD23,
MMC_CAP_CMD23,
MMC_CAP_CMD23,
MMC_CAP_CMD23,
@@ -426,7 +425,7 @@
return dw_mci_pltfm_register(pdev, drv_data);
}
-const struct dev_pm_ops dw_mci_exynos_pmops = {
+static const struct dev_pm_ops dw_mci_exynos_pmops = {
SET_SYSTEM_SLEEP_PM_OPS(dw_mci_exynos_suspend, dw_mci_exynos_resume)
.resume_noirq = dw_mci_exynos_resume_noirq,
.thaw_noirq = dw_mci_exynos_resume_noirq,
diff --git a/drivers/mmc/host/dw_mmc.c b/drivers/mmc/host/dw_mmc.c
index cced599..1ac227c 100644
--- a/drivers/mmc/host/dw_mmc.c
+++ b/drivers/mmc/host/dw_mmc.c
@@ -235,12 +235,6 @@
}
#endif /* defined(CONFIG_DEBUG_FS) */
-static void dw_mci_set_timeout(struct dw_mci *host)
-{
- /* timeout (maximum) */
- mci_writel(host, TMOUT, 0xffffffff);
-}
-
static u32 dw_mci_prepare_command(struct mmc_host *mmc, struct mmc_command *cmd)
{
struct mmc_data *data;
@@ -257,9 +251,8 @@
(cmd->opcode == SD_IO_RW_DIRECT &&
((cmd->arg >> 9) & 0x1FFFF) == SDIO_CCCR_ABORT))
cmdr |= SDMMC_CMD_STOP;
- else
- if (cmd->opcode != MMC_SEND_STATUS && cmd->data)
- cmdr |= SDMMC_CMD_PRV_DAT_WAIT;
+ else if (cmd->opcode != MMC_SEND_STATUS && cmd->data)
+ cmdr |= SDMMC_CMD_PRV_DAT_WAIT;
if (cmd->flags & MMC_RSP_PRESENT) {
/* We expect a response, so set this bit */
@@ -850,8 +843,6 @@
u32 cmdflags;
mrq = slot->mrq;
- if (host->pdata->select_slot)
- host->pdata->select_slot(slot->id);
host->cur_slot = slot;
host->mrq = mrq;
@@ -864,7 +855,7 @@
data = cmd->data;
if (data) {
- dw_mci_set_timeout(host);
+ mci_writel(host, TMOUT, 0xFFFFFFFF);
mci_writel(host, BYTCNT, data->blksz*data->blocks);
mci_writel(host, BLKSIZ, data->blksz);
}
@@ -962,7 +953,7 @@
regs = mci_readl(slot->host, UHS_REG);
/* DDR mode set */
- if (ios->timing == MMC_TIMING_UHS_DDR50)
+ if (ios->timing == MMC_TIMING_MMC_DDR52)
regs |= ((0x1 << slot->id) << 16);
else
regs &= ~((0x1 << slot->id) << 16);
@@ -985,17 +976,11 @@
switch (ios->power_mode) {
case MMC_POWER_UP:
set_bit(DW_MMC_CARD_NEED_INIT, &slot->flags);
- /* Power up slot */
- if (slot->host->pdata->setpower)
- slot->host->pdata->setpower(slot->id, mmc->ocr_avail);
regs = mci_readl(slot->host, PWREN);
regs |= (1 << slot->id);
mci_writel(slot->host, PWREN, regs);
break;
case MMC_POWER_OFF:
- /* Power down slot */
- if (slot->host->pdata->setpower)
- slot->host->pdata->setpower(slot->id, 0);
regs = mci_readl(slot->host, PWREN);
regs &= ~(1 << slot->id);
mci_writel(slot->host, PWREN, regs);
@@ -1009,15 +994,13 @@
{
int read_only;
struct dw_mci_slot *slot = mmc_priv(mmc);
- struct dw_mci_board *brd = slot->host->pdata;
+ int gpio_ro = mmc_gpio_get_ro(mmc);
/* Use platform get_ro function, else try on board write protect */
if (slot->quirks & DW_MCI_SLOT_QUIRK_NO_WRITE_PROTECT)
read_only = 0;
- else if (brd->get_ro)
- read_only = brd->get_ro(slot->id);
- else if (gpio_is_valid(slot->wp_gpio))
- read_only = gpio_get_value(slot->wp_gpio);
+ else if (!IS_ERR_VALUE(gpio_ro))
+ read_only = gpio_ro;
else
read_only =
mci_readl(slot->host, WRTPRT) & (1 << slot->id) ? 1 : 0;
@@ -1039,8 +1022,6 @@
/* Use platform get_cd function, else try onboard card detect */
if (brd->quirks & DW_MCI_QUIRK_BROKEN_CARD_DETECTION)
present = 1;
- else if (brd->get_cd)
- present = !brd->get_cd(slot->id);
else if (!IS_ERR_VALUE(gpio_cd))
present = gpio_cd;
else
@@ -1248,7 +1229,7 @@
data->error = -EIO;
}
- dev_err(host->dev, "data error, status 0x%08x\n", status);
+ dev_dbg(host->dev, "data error, status 0x%08x\n", status);
/*
* After an error, there may be data lingering
@@ -2045,86 +2026,15 @@
return quirks;
}
-
-/* find out bus-width for a given slot */
-static u32 dw_mci_of_get_bus_wd(struct device *dev, u8 slot)
-{
- struct device_node *np = dw_mci_of_find_slot_node(dev, slot);
- u32 bus_wd = 1;
-
- if (!np)
- return 1;
-
- if (of_property_read_u32(np, "bus-width", &bus_wd))
- dev_err(dev, "bus-width property not found, assuming width"
- " as 1\n");
- return bus_wd;
-}
-
-/* find the write protect gpio for a given slot; or -1 if none specified */
-static int dw_mci_of_get_wp_gpio(struct device *dev, u8 slot)
-{
- struct device_node *np = dw_mci_of_find_slot_node(dev, slot);
- int gpio;
-
- if (!np)
- return -EINVAL;
-
- gpio = of_get_named_gpio(np, "wp-gpios", 0);
-
- /* Having a missing entry is valid; return silently */
- if (!gpio_is_valid(gpio))
- return -EINVAL;
-
- if (devm_gpio_request(dev, gpio, "dw-mci-wp")) {
- dev_warn(dev, "gpio [%d] request failed\n", gpio);
- return -EINVAL;
- }
-
- return gpio;
-}
-
-/* find the cd gpio for a given slot */
-static void dw_mci_of_get_cd_gpio(struct device *dev, u8 slot,
- struct mmc_host *mmc)
-{
- struct device_node *np = dw_mci_of_find_slot_node(dev, slot);
- int gpio;
-
- if (!np)
- return;
-
- gpio = of_get_named_gpio(np, "cd-gpios", 0);
-
- /* Having a missing entry is valid; return silently */
- if (!gpio_is_valid(gpio))
- return;
-
- if (mmc_gpio_request_cd(mmc, gpio, 0))
- dev_warn(dev, "gpio [%d] request failed\n", gpio);
-}
#else /* CONFIG_OF */
static int dw_mci_of_get_slot_quirks(struct device *dev, u8 slot)
{
return 0;
}
-static u32 dw_mci_of_get_bus_wd(struct device *dev, u8 slot)
-{
- return 1;
-}
static struct device_node *dw_mci_of_find_slot_node(struct device *dev, u8 slot)
{
return NULL;
}
-static int dw_mci_of_get_wp_gpio(struct device *dev, u8 slot)
-{
- return -EINVAL;
-}
-static void dw_mci_of_get_cd_gpio(struct device *dev, u8 slot,
- struct mmc_host *mmc)
-{
- return;
-}
#endif /* CONFIG_OF */
static int dw_mci_init_slot(struct dw_mci *host, unsigned int id)
@@ -2134,7 +2044,6 @@
const struct dw_mci_drv_data *drv_data = host->drv_data;
int ctrl_id, ret;
u32 freq[2];
- u8 bus_width;
mmc = mmc_alloc_host(sizeof(struct dw_mci_slot), host->dev);
if (!mmc)
@@ -2158,17 +2067,7 @@
mmc->f_max = freq[1];
}
- if (host->pdata->get_ocr)
- mmc->ocr_avail = host->pdata->get_ocr(id);
- else
- mmc->ocr_avail = MMC_VDD_32_33 | MMC_VDD_33_34;
-
- /*
- * Start with slot power disabled, it will be enabled when a card
- * is detected.
- */
- if (host->pdata->setpower)
- host->pdata->setpower(id, 0);
+ mmc->ocr_avail = MMC_VDD_32_33 | MMC_VDD_33_34;
if (host->pdata->caps)
mmc->caps = host->pdata->caps;
@@ -2189,19 +2088,7 @@
if (host->pdata->caps2)
mmc->caps2 = host->pdata->caps2;
- if (host->pdata->get_bus_wd)
- bus_width = host->pdata->get_bus_wd(slot->id);
- else if (host->dev->of_node)
- bus_width = dw_mci_of_get_bus_wd(host->dev, slot->id);
- else
- bus_width = 1;
-
- switch (bus_width) {
- case 8:
- mmc->caps |= MMC_CAP_8_BIT_DATA;
- case 4:
- mmc->caps |= MMC_CAP_4_BIT_DATA;
- }
+ mmc_of_parse(mmc);
if (host->pdata->blk_settings) {
mmc->max_segs = host->pdata->blk_settings->max_segs;
@@ -2226,8 +2113,10 @@
#endif /* CONFIG_MMC_DW_IDMAC */
}
- slot->wp_gpio = dw_mci_of_get_wp_gpio(host->dev, slot->id);
- dw_mci_of_get_cd_gpio(host->dev, slot->id, mmc);
+ if (dw_mci_get_cd(mmc))
+ set_bit(DW_MMC_CARD_PRESENT, &slot->flags);
+ else
+ clear_bit(DW_MMC_CARD_PRESENT, &slot->flags);
ret = mmc_add_host(mmc);
if (ret)
@@ -2249,10 +2138,6 @@
static void dw_mci_cleanup_slot(struct dw_mci_slot *slot, unsigned int id)
{
- /* Shutdown detect IRQ */
- if (slot->host->pdata->exit)
- slot->host->pdata->exit(id);
-
/* Debugfs stuff is cleaned up by mmc core */
mmc_remove_host(slot->mmc);
slot->host->slot[id] = NULL;
@@ -2399,24 +2284,9 @@
return ERR_PTR(ret);
}
- if (of_find_property(np, "keep-power-in-suspend", NULL))
- pdata->pm_caps |= MMC_PM_KEEP_POWER;
-
- if (of_find_property(np, "enable-sdio-wakeup", NULL))
- pdata->pm_caps |= MMC_PM_WAKE_SDIO_IRQ;
-
if (of_find_property(np, "supports-highspeed", NULL))
pdata->caps |= MMC_CAP_SD_HIGHSPEED | MMC_CAP_MMC_HIGHSPEED;
- if (of_find_property(np, "caps2-mmc-hs200-1_8v", NULL))
- pdata->caps2 |= MMC_CAP2_HS200_1_8V_SDR;
-
- if (of_find_property(np, "caps2-mmc-hs200-1_2v", NULL))
- pdata->caps2 |= MMC_CAP2_HS200_1_2V_SDR;
-
- if (of_get_property(np, "cd-inverted", NULL))
- pdata->caps2 |= MMC_CAP2_CD_ACTIVE_HIGH;
-
return pdata;
}
@@ -2442,9 +2312,9 @@
}
}
- if (!host->pdata->select_slot && host->pdata->num_slots > 1) {
+ if (host->pdata->num_slots > 1) {
dev_err(host->dev,
- "Platform data must supply select_slot function\n");
+ "Platform data must supply num_slots.\n");
return -ENODEV;
}
@@ -2474,12 +2344,19 @@
ret = clk_set_rate(host->ciu_clk, host->pdata->bus_hz);
if (ret)
dev_warn(host->dev,
- "Unable to set bus rate to %ul\n",
+ "Unable to set bus rate to %uHz\n",
host->pdata->bus_hz);
}
host->bus_hz = clk_get_rate(host->ciu_clk);
}
+ if (!host->bus_hz) {
+ dev_err(host->dev,
+ "Platform data must supply bus speed\n");
+ ret = -ENODEV;
+ goto err_clk_ciu;
+ }
+
if (drv_data && drv_data->init) {
ret = drv_data->init(host);
if (ret) {
@@ -2516,13 +2393,6 @@
}
}
- if (!host->bus_hz) {
- dev_err(host->dev,
- "Platform data must supply bus speed\n");
- ret = -ENODEV;
- goto err_regulator;
- }
-
host->quirks = host->pdata->quirks;
spin_lock_init(&host->lock);
@@ -2666,8 +2536,6 @@
err_dmaunmap:
if (host->use_dma && host->dma_ops->exit)
host->dma_ops->exit(host);
-
-err_regulator:
if (host->vmmc)
regulator_disable(host->vmmc);
diff --git a/drivers/mmc/host/dw_mmc.h b/drivers/mmc/host/dw_mmc.h
index 6834977..738fa24 100644
--- a/drivers/mmc/host/dw_mmc.h
+++ b/drivers/mmc/host/dw_mmc.h
@@ -195,7 +195,6 @@
* @mmc: The mmc_host representing this slot.
* @host: The MMC controller this slot is using.
* @quirks: Slot-level quirks (DW_MCI_SLOT_QUIRK_XXX)
- * @wp_gpio: If gpio_is_valid() we'll use this to read write protect.
* @ctype: Card type for this slot.
* @mrq: mmc_request currently being processed or waiting to be
* processed, or NULL when the slot is idle.
@@ -214,7 +213,6 @@
struct dw_mci *host;
int quirks;
- int wp_gpio;
u32 ctype;
diff --git a/drivers/mmc/host/jz4740_mmc.c b/drivers/mmc/host/jz4740_mmc.c
index de2139c..537d6c7 100644
--- a/drivers/mmc/host/jz4740_mmc.c
+++ b/drivers/mmc/host/jz4740_mmc.c
@@ -515,10 +515,13 @@
jz4740_mmc_send_command(host, req->stop);
- timeout = jz4740_mmc_poll_irq(host, JZ_MMC_IRQ_PRG_DONE);
- if (timeout) {
- host->state = JZ4740_MMC_STATE_DONE;
- break;
+ if (mmc_resp_type(req->stop) & MMC_RSP_BUSY) {
+ timeout = jz4740_mmc_poll_irq(host,
+ JZ_MMC_IRQ_PRG_DONE);
+ if (timeout) {
+ host->state = JZ4740_MMC_STATE_DONE;
+ break;
+ }
}
case JZ4740_MMC_STATE_DONE:
break;
diff --git a/drivers/mmc/host/mmci.c b/drivers/mmc/host/mmci.c
index a084edd..7ad463e 100644
--- a/drivers/mmc/host/mmci.c
+++ b/drivers/mmc/host/mmci.c
@@ -301,7 +301,8 @@
if (host->mmc->ios.bus_width == MMC_BUS_WIDTH_8)
clk |= MCI_ST_8BIT_BUS;
- if (host->mmc->ios.timing == MMC_TIMING_UHS_DDR50)
+ if (host->mmc->ios.timing == MMC_TIMING_UHS_DDR50 ||
+ host->mmc->ios.timing == MMC_TIMING_MMC_DDR52)
clk |= MCI_ST_UX500_NEG_EDGE;
mmci_write_clkreg(host, clk);
@@ -764,7 +765,8 @@
mmci_write_clkreg(host, clk);
}
- if (host->mmc->ios.timing == MMC_TIMING_UHS_DDR50)
+ if (host->mmc->ios.timing == MMC_TIMING_UHS_DDR50 ||
+ host->mmc->ios.timing == MMC_TIMING_MMC_DDR52)
datactrl |= MCI_ST_DPSM_DDRMODE;
/*
diff --git a/drivers/mmc/host/moxart-mmc.c b/drivers/mmc/host/moxart-mmc.c
new file mode 100644
index 0000000..74924a0
--- /dev/null
+++ b/drivers/mmc/host/moxart-mmc.c
@@ -0,0 +1,730 @@
+/*
+ * MOXA ART MMC host driver.
+ *
+ * Copyright (C) 2014 Jonas Jensen
+ *
+ * Jonas Jensen <jonas.jensen@gmail.com>
+ *
+ * Based on code from
+ * Moxa Technologies Co., Ltd. <www.moxa.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/version.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/platform_device.h>
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/blkdev.h>
+#include <linux/dma-mapping.h>
+#include <linux/dmaengine.h>
+#include <linux/mmc/host.h>
+#include <linux/mmc/sd.h>
+#include <linux/sched.h>
+#include <linux/io.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/clk.h>
+#include <linux/bitops.h>
+#include <linux/of_dma.h>
+#include <linux/spinlock.h>
+
+#define REG_COMMAND 0
+#define REG_ARGUMENT 4
+#define REG_RESPONSE0 8
+#define REG_RESPONSE1 12
+#define REG_RESPONSE2 16
+#define REG_RESPONSE3 20
+#define REG_RESPONSE_COMMAND 24
+#define REG_DATA_CONTROL 28
+#define REG_DATA_TIMER 32
+#define REG_DATA_LENGTH 36
+#define REG_STATUS 40
+#define REG_CLEAR 44
+#define REG_INTERRUPT_MASK 48
+#define REG_POWER_CONTROL 52
+#define REG_CLOCK_CONTROL 56
+#define REG_BUS_WIDTH 60
+#define REG_DATA_WINDOW 64
+#define REG_FEATURE 68
+#define REG_REVISION 72
+
+/* REG_COMMAND */
+#define CMD_SDC_RESET BIT(10)
+#define CMD_EN BIT(9)
+#define CMD_APP_CMD BIT(8)
+#define CMD_LONG_RSP BIT(7)
+#define CMD_NEED_RSP BIT(6)
+#define CMD_IDX_MASK 0x3f
+
+/* REG_RESPONSE_COMMAND */
+#define RSP_CMD_APP BIT(6)
+#define RSP_CMD_IDX_MASK 0x3f
+
+/* REG_DATA_CONTROL */
+#define DCR_DATA_FIFO_RESET BIT(8)
+#define DCR_DATA_THRES BIT(7)
+#define DCR_DATA_EN BIT(6)
+#define DCR_DMA_EN BIT(5)
+#define DCR_DATA_WRITE BIT(4)
+#define DCR_BLK_SIZE 0x0f
+
+/* REG_DATA_LENGTH */
+#define DATA_LEN_MASK 0xffffff
+
+/* REG_STATUS */
+#define WRITE_PROT BIT(12)
+#define CARD_DETECT BIT(11)
+/* 1-10 below can be sent to either registers, interrupt or clear. */
+#define CARD_CHANGE BIT(10)
+#define FIFO_ORUN BIT(9)
+#define FIFO_URUN BIT(8)
+#define DATA_END BIT(7)
+#define CMD_SENT BIT(6)
+#define DATA_CRC_OK BIT(5)
+#define RSP_CRC_OK BIT(4)
+#define DATA_TIMEOUT BIT(3)
+#define RSP_TIMEOUT BIT(2)
+#define DATA_CRC_FAIL BIT(1)
+#define RSP_CRC_FAIL BIT(0)
+
+#define MASK_RSP (RSP_TIMEOUT | RSP_CRC_FAIL | \
+ RSP_CRC_OK | CARD_DETECT | CMD_SENT)
+
+#define MASK_DATA (DATA_CRC_OK | DATA_END | \
+ DATA_CRC_FAIL | DATA_TIMEOUT)
+
+#define MASK_INTR_PIO (FIFO_URUN | FIFO_ORUN | CARD_CHANGE)
+
+/* REG_POWER_CONTROL */
+#define SD_POWER_ON BIT(4)
+#define SD_POWER_MASK 0x0f
+
+/* REG_CLOCK_CONTROL */
+#define CLK_HISPD BIT(9)
+#define CLK_OFF BIT(8)
+#define CLK_SD BIT(7)
+#define CLK_DIV_MASK 0x7f
+
+/* REG_BUS_WIDTH */
+#define BUS_WIDTH_8 BIT(2)
+#define BUS_WIDTH_4 BIT(1)
+#define BUS_WIDTH_1 BIT(0)
+
+#define MMC_VDD_360 23
+#define MIN_POWER (MMC_VDD_360 - SD_POWER_MASK)
+#define MAX_RETRIES 500000
+
+struct moxart_host {
+ spinlock_t lock;
+
+ void __iomem *base;
+
+ phys_addr_t reg_phys;
+
+ struct dma_chan *dma_chan_tx;
+ struct dma_chan *dma_chan_rx;
+ struct dma_async_tx_descriptor *tx_desc;
+ struct mmc_host *mmc;
+ struct mmc_request *mrq;
+ struct scatterlist *cur_sg;
+ struct completion dma_complete;
+ struct completion pio_complete;
+
+ u32 num_sg;
+ u32 data_remain;
+ u32 data_len;
+ u32 fifo_width;
+ u32 timeout;
+ u32 rate;
+
+ long sysclk;
+
+ bool have_dma;
+ bool is_removed;
+};
+
+static inline void moxart_init_sg(struct moxart_host *host,
+ struct mmc_data *data)
+{
+ host->cur_sg = data->sg;
+ host->num_sg = data->sg_len;
+ host->data_remain = host->cur_sg->length;
+
+ if (host->data_remain > host->data_len)
+ host->data_remain = host->data_len;
+}
+
+static inline int moxart_next_sg(struct moxart_host *host)
+{
+ int remain;
+ struct mmc_data *data = host->mrq->cmd->data;
+
+ host->cur_sg++;
+ host->num_sg--;
+
+ if (host->num_sg > 0) {
+ host->data_remain = host->cur_sg->length;
+ remain = host->data_len - data->bytes_xfered;
+ if (remain > 0 && remain < host->data_remain)
+ host->data_remain = remain;
+ }
+
+ return host->num_sg;
+}
+
+static int moxart_wait_for_status(struct moxart_host *host,
+ u32 mask, u32 *status)
+{
+ int ret = -ETIMEDOUT;
+ u32 i;
+
+ for (i = 0; i < MAX_RETRIES; i++) {
+ *status = readl(host->base + REG_STATUS);
+ if (!(*status & mask)) {
+ udelay(5);
+ continue;
+ }
+ writel(*status & mask, host->base + REG_CLEAR);
+ ret = 0;
+ break;
+ }
+
+ if (ret)
+ dev_err(mmc_dev(host->mmc), "timed out waiting for status\n");
+
+ return ret;
+}
+
+
+static void moxart_send_command(struct moxart_host *host,
+ struct mmc_command *cmd)
+{
+ u32 status, cmdctrl;
+
+ writel(RSP_TIMEOUT | RSP_CRC_OK |
+ RSP_CRC_FAIL | CMD_SENT, host->base + REG_CLEAR);
+ writel(cmd->arg, host->base + REG_ARGUMENT);
+
+ cmdctrl = cmd->opcode & CMD_IDX_MASK;
+ if (cmdctrl == SD_APP_SET_BUS_WIDTH || cmdctrl == SD_APP_OP_COND ||
+ cmdctrl == SD_APP_SEND_SCR || cmdctrl == SD_APP_SD_STATUS ||
+ cmdctrl == SD_APP_SEND_NUM_WR_BLKS)
+ cmdctrl |= CMD_APP_CMD;
+
+ if (cmd->flags & MMC_RSP_PRESENT)
+ cmdctrl |= CMD_NEED_RSP;
+
+ if (cmd->flags & MMC_RSP_136)
+ cmdctrl |= CMD_LONG_RSP;
+
+ writel(cmdctrl | CMD_EN, host->base + REG_COMMAND);
+
+ if (moxart_wait_for_status(host, MASK_RSP, &status) == -ETIMEDOUT)
+ cmd->error = -ETIMEDOUT;
+
+ if (status & RSP_TIMEOUT) {
+ cmd->error = -ETIMEDOUT;
+ return;
+ }
+ if (status & RSP_CRC_FAIL) {
+ cmd->error = -EIO;
+ return;
+ }
+ if (status & RSP_CRC_OK) {
+ if (cmd->flags & MMC_RSP_136) {
+ cmd->resp[3] = readl(host->base + REG_RESPONSE0);
+ cmd->resp[2] = readl(host->base + REG_RESPONSE1);
+ cmd->resp[1] = readl(host->base + REG_RESPONSE2);
+ cmd->resp[0] = readl(host->base + REG_RESPONSE3);
+ } else {
+ cmd->resp[0] = readl(host->base + REG_RESPONSE0);
+ }
+ }
+}
+
+static void moxart_dma_complete(void *param)
+{
+ struct moxart_host *host = param;
+
+ complete(&host->dma_complete);
+}
+
+static void moxart_transfer_dma(struct mmc_data *data, struct moxart_host *host)
+{
+ u32 len, dir_data, dir_slave;
+ unsigned long dma_time;
+ struct dma_async_tx_descriptor *desc = NULL;
+ struct dma_chan *dma_chan;
+
+ if (host->data_len == data->bytes_xfered)
+ return;
+
+ if (data->flags & MMC_DATA_WRITE) {
+ dma_chan = host->dma_chan_tx;
+ dir_data = DMA_TO_DEVICE;
+ dir_slave = DMA_MEM_TO_DEV;
+ } else {
+ dma_chan = host->dma_chan_rx;
+ dir_data = DMA_FROM_DEVICE;
+ dir_slave = DMA_DEV_TO_MEM;
+ }
+
+ len = dma_map_sg(dma_chan->device->dev, data->sg,
+ data->sg_len, dir_data);
+
+ if (len > 0) {
+ desc = dmaengine_prep_slave_sg(dma_chan, data->sg,
+ len, dir_slave,
+ DMA_PREP_INTERRUPT |
+ DMA_CTRL_ACK);
+ } else {
+ dev_err(mmc_dev(host->mmc), "dma_map_sg returned zero length\n");
+ }
+
+ if (desc) {
+ host->tx_desc = desc;
+ desc->callback = moxart_dma_complete;
+ desc->callback_param = host;
+ dmaengine_submit(desc);
+ dma_async_issue_pending(dma_chan);
+ }
+
+ data->bytes_xfered += host->data_remain;
+
+ dma_time = wait_for_completion_interruptible_timeout(
+ &host->dma_complete, host->timeout);
+
+ dma_unmap_sg(dma_chan->device->dev,
+ data->sg, data->sg_len,
+ dir_data);
+}
+
+
+static void moxart_transfer_pio(struct moxart_host *host)
+{
+ struct mmc_data *data = host->mrq->cmd->data;
+ u32 *sgp, len = 0, remain, status;
+
+ if (host->data_len == data->bytes_xfered)
+ return;
+
+ sgp = sg_virt(host->cur_sg);
+ remain = host->data_remain;
+
+ if (data->flags & MMC_DATA_WRITE) {
+ while (remain > 0) {
+ if (moxart_wait_for_status(host, FIFO_URUN, &status)
+ == -ETIMEDOUT) {
+ data->error = -ETIMEDOUT;
+ complete(&host->pio_complete);
+ return;
+ }
+ for (len = 0; len < remain && len < host->fifo_width;) {
+ iowrite32(*sgp, host->base + REG_DATA_WINDOW);
+ sgp++;
+ len += 4;
+ }
+ remain -= len;
+ }
+
+ } else {
+ while (remain > 0) {
+ if (moxart_wait_for_status(host, FIFO_ORUN, &status)
+ == -ETIMEDOUT) {
+ data->error = -ETIMEDOUT;
+ complete(&host->pio_complete);
+ return;
+ }
+ for (len = 0; len < remain && len < host->fifo_width;) {
+ /* SCR data must be read in big endian. */
+ if (data->mrq->cmd->opcode == SD_APP_SEND_SCR)
+ *sgp = ioread32be(host->base +
+ REG_DATA_WINDOW);
+ else
+ *sgp = ioread32(host->base +
+ REG_DATA_WINDOW);
+ sgp++;
+ len += 4;
+ }
+ remain -= len;
+ }
+ }
+
+ data->bytes_xfered += host->data_remain - remain;
+ host->data_remain = remain;
+
+ if (host->data_len != data->bytes_xfered)
+ moxart_next_sg(host);
+ else
+ complete(&host->pio_complete);
+}
+
+static void moxart_prepare_data(struct moxart_host *host)
+{
+ struct mmc_data *data = host->mrq->cmd->data;
+ u32 datactrl;
+ int blksz_bits;
+
+ if (!data)
+ return;
+
+ host->data_len = data->blocks * data->blksz;
+ blksz_bits = ffs(data->blksz) - 1;
+ BUG_ON(1 << blksz_bits != data->blksz);
+
+ moxart_init_sg(host, data);
+
+ datactrl = DCR_DATA_EN | (blksz_bits & DCR_BLK_SIZE);
+
+ if (data->flags & MMC_DATA_WRITE)
+ datactrl |= DCR_DATA_WRITE;
+
+ if ((host->data_len > host->fifo_width) && host->have_dma)
+ datactrl |= DCR_DMA_EN;
+
+ writel(DCR_DATA_FIFO_RESET, host->base + REG_DATA_CONTROL);
+ writel(MASK_DATA | FIFO_URUN | FIFO_ORUN, host->base + REG_CLEAR);
+ writel(host->rate, host->base + REG_DATA_TIMER);
+ writel(host->data_len, host->base + REG_DATA_LENGTH);
+ writel(datactrl, host->base + REG_DATA_CONTROL);
+}
+
+static void moxart_request(struct mmc_host *mmc, struct mmc_request *mrq)
+{
+ struct moxart_host *host = mmc_priv(mmc);
+ unsigned long pio_time, flags;
+ u32 status;
+
+ spin_lock_irqsave(&host->lock, flags);
+
+ init_completion(&host->dma_complete);
+ init_completion(&host->pio_complete);
+
+ host->mrq = mrq;
+
+ if (readl(host->base + REG_STATUS) & CARD_DETECT) {
+ mrq->cmd->error = -ETIMEDOUT;
+ goto request_done;
+ }
+
+ moxart_prepare_data(host);
+ moxart_send_command(host, host->mrq->cmd);
+
+ if (mrq->cmd->data) {
+ if ((host->data_len > host->fifo_width) && host->have_dma) {
+
+ writel(CARD_CHANGE, host->base + REG_INTERRUPT_MASK);
+
+ spin_unlock_irqrestore(&host->lock, flags);
+
+ moxart_transfer_dma(mrq->cmd->data, host);
+
+ spin_lock_irqsave(&host->lock, flags);
+ } else {
+
+ writel(MASK_INTR_PIO, host->base + REG_INTERRUPT_MASK);
+
+ spin_unlock_irqrestore(&host->lock, flags);
+
+ /* PIO transfers start from interrupt. */
+ pio_time = wait_for_completion_interruptible_timeout(
+ &host->pio_complete, host->timeout);
+
+ spin_lock_irqsave(&host->lock, flags);
+ }
+
+ if (host->is_removed) {
+ dev_err(mmc_dev(host->mmc), "card removed\n");
+ mrq->cmd->error = -ETIMEDOUT;
+ goto request_done;
+ }
+
+ if (moxart_wait_for_status(host, MASK_DATA, &status)
+ == -ETIMEDOUT) {
+ mrq->cmd->data->error = -ETIMEDOUT;
+ goto request_done;
+ }
+
+ if (status & DATA_CRC_FAIL)
+ mrq->cmd->data->error = -ETIMEDOUT;
+
+ if (mrq->cmd->data->stop)
+ moxart_send_command(host, mrq->cmd->data->stop);
+ }
+
+request_done:
+ spin_unlock_irqrestore(&host->lock, flags);
+ mmc_request_done(host->mmc, mrq);
+}
+
+static irqreturn_t moxart_irq(int irq, void *devid)
+{
+ struct moxart_host *host = (struct moxart_host *)devid;
+ u32 status;
+ unsigned long flags;
+
+ spin_lock_irqsave(&host->lock, flags);
+
+ status = readl(host->base + REG_STATUS);
+ if (status & CARD_CHANGE) {
+ host->is_removed = status & CARD_DETECT;
+ if (host->is_removed && host->have_dma) {
+ dmaengine_terminate_all(host->dma_chan_tx);
+ dmaengine_terminate_all(host->dma_chan_rx);
+ }
+ host->mrq = NULL;
+ writel(MASK_INTR_PIO, host->base + REG_CLEAR);
+ writel(CARD_CHANGE, host->base + REG_INTERRUPT_MASK);
+ mmc_detect_change(host->mmc, 0);
+ }
+ if (status & (FIFO_ORUN | FIFO_URUN) && host->mrq)
+ moxart_transfer_pio(host);
+
+ spin_unlock_irqrestore(&host->lock, flags);
+
+ return IRQ_HANDLED;
+}
+
+static void moxart_set_ios(struct mmc_host *mmc, struct mmc_ios *ios)
+{
+ struct moxart_host *host = mmc_priv(mmc);
+ unsigned long flags;
+ u8 power, div;
+ u32 ctrl;
+
+ spin_lock_irqsave(&host->lock, flags);
+
+ if (ios->clock) {
+ for (div = 0; div < CLK_DIV_MASK; ++div) {
+ if (ios->clock >= host->sysclk / (2 * (div + 1)))
+ break;
+ }
+ ctrl = CLK_SD | div;
+ host->rate = host->sysclk / (2 * (div + 1));
+ if (host->rate > host->sysclk)
+ ctrl |= CLK_HISPD;
+ writel(ctrl, host->base + REG_CLOCK_CONTROL);
+ }
+
+ if (ios->power_mode == MMC_POWER_OFF) {
+ writel(readl(host->base + REG_POWER_CONTROL) & ~SD_POWER_ON,
+ host->base + REG_POWER_CONTROL);
+ } else {
+ if (ios->vdd < MIN_POWER)
+ power = 0;
+ else
+ power = ios->vdd - MIN_POWER;
+
+ writel(SD_POWER_ON | (u32) power,
+ host->base + REG_POWER_CONTROL);
+ }
+
+ switch (ios->bus_width) {
+ case MMC_BUS_WIDTH_4:
+ writel(BUS_WIDTH_4, host->base + REG_BUS_WIDTH);
+ break;
+ case MMC_BUS_WIDTH_8:
+ writel(BUS_WIDTH_8, host->base + REG_BUS_WIDTH);
+ break;
+ default:
+ writel(BUS_WIDTH_1, host->base + REG_BUS_WIDTH);
+ break;
+ }
+
+ spin_unlock_irqrestore(&host->lock, flags);
+}
+
+
+static int moxart_get_ro(struct mmc_host *mmc)
+{
+ struct moxart_host *host = mmc_priv(mmc);
+
+ return !!(readl(host->base + REG_STATUS) & WRITE_PROT);
+}
+
+static struct mmc_host_ops moxart_ops = {
+ .request = moxart_request,
+ .set_ios = moxart_set_ios,
+ .get_ro = moxart_get_ro,
+};
+
+static int moxart_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *node = dev->of_node;
+ struct resource res_mmc;
+ struct mmc_host *mmc;
+ struct moxart_host *host = NULL;
+ struct dma_slave_config cfg;
+ struct clk *clk;
+ void __iomem *reg_mmc;
+ dma_cap_mask_t mask;
+ int irq, ret;
+ u32 i;
+
+ mmc = mmc_alloc_host(sizeof(struct moxart_host), dev);
+ if (!mmc) {
+ dev_err(dev, "mmc_alloc_host failed\n");
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ret = of_address_to_resource(node, 0, &res_mmc);
+ if (ret) {
+ dev_err(dev, "of_address_to_resource failed\n");
+ goto out;
+ }
+
+ irq = irq_of_parse_and_map(node, 0);
+ if (irq <= 0) {
+ dev_err(dev, "irq_of_parse_and_map failed\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ clk = of_clk_get(node, 0);
+ if (IS_ERR(clk)) {
+ dev_err(dev, "of_clk_get failed\n");
+ ret = PTR_ERR(clk);
+ goto out;
+ }
+
+ reg_mmc = devm_ioremap_resource(dev, &res_mmc);
+ if (IS_ERR(reg_mmc)) {
+ ret = PTR_ERR(reg_mmc);
+ goto out;
+ }
+
+ mmc_of_parse(mmc);
+
+ dma_cap_zero(mask);
+ dma_cap_set(DMA_SLAVE, mask);
+
+ host = mmc_priv(mmc);
+ host->mmc = mmc;
+ host->base = reg_mmc;
+ host->reg_phys = res_mmc.start;
+ host->timeout = msecs_to_jiffies(1000);
+ host->sysclk = clk_get_rate(clk);
+ host->fifo_width = readl(host->base + REG_FEATURE) << 2;
+ host->dma_chan_tx = of_dma_request_slave_channel(node, "tx");
+ host->dma_chan_rx = of_dma_request_slave_channel(node, "rx");
+
+ spin_lock_init(&host->lock);
+
+ mmc->ops = &moxart_ops;
+ mmc->f_max = DIV_ROUND_CLOSEST(host->sysclk, 2);
+ mmc->f_min = DIV_ROUND_CLOSEST(host->sysclk, CLK_DIV_MASK * 2);
+ mmc->ocr_avail = 0xffff00; /* Support 2.0v - 3.6v power. */
+
+ if (IS_ERR(host->dma_chan_tx) || IS_ERR(host->dma_chan_rx)) {
+ dev_dbg(dev, "PIO mode transfer enabled\n");
+ host->have_dma = false;
+ } else {
+ dev_dbg(dev, "DMA channels found (%p,%p)\n",
+ host->dma_chan_tx, host->dma_chan_rx);
+ host->have_dma = true;
+
+ cfg.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
+ cfg.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
+
+ cfg.direction = DMA_MEM_TO_DEV;
+ cfg.src_addr = 0;
+ cfg.dst_addr = host->reg_phys + REG_DATA_WINDOW;
+ dmaengine_slave_config(host->dma_chan_tx, &cfg);
+
+ cfg.direction = DMA_DEV_TO_MEM;
+ cfg.src_addr = host->reg_phys + REG_DATA_WINDOW;
+ cfg.dst_addr = 0;
+ dmaengine_slave_config(host->dma_chan_rx, &cfg);
+ }
+
+ switch ((readl(host->base + REG_BUS_WIDTH) >> 3) & 3) {
+ case 1:
+ mmc->caps |= MMC_CAP_4_BIT_DATA;
+ break;
+ case 2:
+ mmc->caps |= MMC_CAP_4_BIT_DATA | MMC_CAP_8_BIT_DATA;
+ break;
+ default:
+ break;
+ }
+
+ writel(0, host->base + REG_INTERRUPT_MASK);
+
+ writel(CMD_SDC_RESET, host->base + REG_COMMAND);
+ for (i = 0; i < MAX_RETRIES; i++) {
+ if (!(readl(host->base + REG_COMMAND) & CMD_SDC_RESET))
+ break;
+ udelay(5);
+ }
+
+ ret = devm_request_irq(dev, irq, moxart_irq, 0, "moxart-mmc", host);
+ if (ret)
+ goto out;
+
+ dev_set_drvdata(dev, mmc);
+ mmc_add_host(mmc);
+
+ dev_dbg(dev, "IRQ=%d, FIFO is %d bytes\n", irq, host->fifo_width);
+
+ return 0;
+
+out:
+ if (mmc)
+ mmc_free_host(mmc);
+ return ret;
+}
+
+static int moxart_remove(struct platform_device *pdev)
+{
+ struct mmc_host *mmc = dev_get_drvdata(&pdev->dev);
+ struct moxart_host *host = mmc_priv(mmc);
+
+ dev_set_drvdata(&pdev->dev, NULL);
+
+ if (mmc) {
+ if (!IS_ERR(host->dma_chan_tx))
+ dma_release_channel(host->dma_chan_tx);
+ if (!IS_ERR(host->dma_chan_rx))
+ dma_release_channel(host->dma_chan_rx);
+ mmc_remove_host(mmc);
+ mmc_free_host(mmc);
+
+ writel(0, host->base + REG_INTERRUPT_MASK);
+ writel(0, host->base + REG_POWER_CONTROL);
+ writel(readl(host->base + REG_CLOCK_CONTROL) | CLK_OFF,
+ host->base + REG_CLOCK_CONTROL);
+ }
+
+ kfree(host);
+
+ return 0;
+}
+
+static const struct of_device_id moxart_mmc_match[] = {
+ { .compatible = "moxa,moxart-mmc" },
+ { .compatible = "faraday,ftsdc010" },
+ { }
+};
+
+static struct platform_driver moxart_mmc_driver = {
+ .probe = moxart_probe,
+ .remove = moxart_remove,
+ .driver = {
+ .name = "mmc-moxart",
+ .owner = THIS_MODULE,
+ .of_match_table = moxart_mmc_match,
+ },
+};
+module_platform_driver(moxart_mmc_driver);
+
+MODULE_ALIAS("platform:mmc-moxart");
+MODULE_DESCRIPTION("MOXA ART MMC driver");
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Jonas Jensen <jonas.jensen@gmail.com>");
diff --git a/drivers/mmc/host/mvsdio.c b/drivers/mmc/host/mvsdio.c
index 45aa220..9377284 100644
--- a/drivers/mmc/host/mvsdio.c
+++ b/drivers/mmc/host/mvsdio.c
@@ -354,6 +354,20 @@
intr_status, mvsd_read(MVSD_NOR_INTR_EN),
mvsd_read(MVSD_HW_STATE));
+ /*
+ * It looks like, SDIO IP can issue one late, spurious irq
+ * although all irqs should be disabled. To work around this,
+ * bail out early, if we didn't expect any irqs to occur.
+ */
+ if (!mvsd_read(MVSD_NOR_INTR_EN) && !mvsd_read(MVSD_ERR_INTR_EN)) {
+ dev_dbg(host->dev, "spurious irq detected intr 0x%04x intr_en 0x%04x erri 0x%04x erri_en 0x%04x\n",
+ mvsd_read(MVSD_NOR_INTR_STATUS),
+ mvsd_read(MVSD_NOR_INTR_EN),
+ mvsd_read(MVSD_ERR_INTR_STATUS),
+ mvsd_read(MVSD_ERR_INTR_EN));
+ return IRQ_HANDLED;
+ }
+
spin_lock(&host->lock);
/* PIO handling, if needed. Messy business... */
@@ -801,10 +815,10 @@
goto out;
if (!(mmc->caps & MMC_CAP_NEEDS_POLL))
- dev_notice(&pdev->dev, "using GPIO for card detection\n");
+ dev_dbg(&pdev->dev, "using GPIO for card detection\n");
else
- dev_notice(&pdev->dev,
- "lacking card detect (fall back to polling)\n");
+ dev_dbg(&pdev->dev, "lacking card detect (fall back to polling)\n");
+
return 0;
out:
diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
index f7199c8..ed1cb93 100644
--- a/drivers/mmc/host/mxcmmc.c
+++ b/drivers/mmc/host/mxcmmc.c
@@ -124,9 +124,8 @@
struct mxcmci_host {
struct mmc_host *mmc;
- struct resource *res;
void __iomem *base;
- int irq;
+ dma_addr_t phys_base;
int detect_irq;
struct dma_chan *dma;
struct dma_async_tx_descriptor *desc;
@@ -154,8 +153,6 @@
struct work_struct datawork;
spinlock_t lock;
- struct regulator *vcc;
-
int burstlen;
int dmareq;
struct dma_slave_config dma_slave_config;
@@ -241,37 +238,15 @@
static void mxcmci_set_clk_rate(struct mxcmci_host *host, unsigned int clk_ios);
-static inline void mxcmci_init_ocr(struct mxcmci_host *host)
+static void mxcmci_set_power(struct mxcmci_host *host, unsigned int vdd)
{
- host->vcc = regulator_get(mmc_dev(host->mmc), "vmmc");
-
- if (IS_ERR(host->vcc)) {
- host->vcc = NULL;
- } else {
- host->mmc->ocr_avail = mmc_regulator_get_ocrmask(host->vcc);
- if (host->pdata && host->pdata->ocr_avail)
- dev_warn(mmc_dev(host->mmc),
- "pdata->ocr_avail will not be used\n");
- }
-
- if (host->vcc == NULL) {
- /* fall-back to platform data */
- if (host->pdata && host->pdata->ocr_avail)
- host->mmc->ocr_avail = host->pdata->ocr_avail;
- else
- host->mmc->ocr_avail = MMC_VDD_32_33 | MMC_VDD_33_34;
- }
-}
-
-static inline void mxcmci_set_power(struct mxcmci_host *host,
- unsigned char power_mode,
- unsigned int vdd)
-{
- if (host->vcc) {
- if (power_mode == MMC_POWER_UP)
- mmc_regulator_set_ocr(host->mmc, host->vcc, vdd);
- else if (power_mode == MMC_POWER_OFF)
- mmc_regulator_set_ocr(host->mmc, host->vcc, 0);
+ if (!IS_ERR(host->mmc->supply.vmmc)) {
+ if (host->power_mode == MMC_POWER_UP)
+ mmc_regulator_set_ocr(host->mmc,
+ host->mmc->supply.vmmc, vdd);
+ else if (host->power_mode == MMC_POWER_OFF)
+ mmc_regulator_set_ocr(host->mmc,
+ host->mmc->supply.vmmc, 0);
}
if (host->pdata && host->pdata->setpower)
@@ -299,7 +274,6 @@
mxcmci_writew(host, 0xff, MMC_REG_RES_TO);
}
-static int mxcmci_setup_dma(struct mmc_host *mmc);
#if IS_ENABLED(CONFIG_PPC_MPC512x)
static inline void buffer_swap32(u32 *buf, int len)
@@ -868,8 +842,8 @@
struct mxcmci_host *host = mmc_priv(mmc);
struct dma_slave_config *config = &host->dma_slave_config;
- config->dst_addr = host->res->start + MMC_REG_BUFFER_ACCESS;
- config->src_addr = host->res->start + MMC_REG_BUFFER_ACCESS;
+ config->dst_addr = host->phys_base + MMC_REG_BUFFER_ACCESS;
+ config->src_addr = host->phys_base + MMC_REG_BUFFER_ACCESS;
config->dst_addr_width = 4;
config->src_addr_width = 4;
config->dst_maxburst = host->burstlen;
@@ -911,8 +885,8 @@
host->cmdat &= ~CMD_DAT_CONT_BUS_WIDTH_4;
if (host->power_mode != ios->power_mode) {
- mxcmci_set_power(host, ios->power_mode, ios->vdd);
host->power_mode = ios->power_mode;
+ mxcmci_set_power(host, ios->vdd);
if (ios->power_mode == MMC_POWER_ON)
host->cmdat |= CMD_DAT_CONT_INIT;
@@ -1040,8 +1014,8 @@
static int mxcmci_probe(struct platform_device *pdev)
{
struct mmc_host *mmc;
- struct mxcmci_host *host = NULL;
- struct resource *iores, *r;
+ struct mxcmci_host *host;
+ struct resource *res;
int ret = 0, irq;
bool dat3_card_detect = false;
dma_cap_mask_t mask;
@@ -1052,21 +1026,25 @@
of_id = of_match_device(mxcmci_of_match, &pdev->dev);
- iores = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
irq = platform_get_irq(pdev, 0);
- if (!iores || irq < 0)
+ if (irq < 0)
return -EINVAL;
- r = request_mem_region(iores->start, resource_size(iores), pdev->name);
- if (!r)
- return -EBUSY;
+ mmc = mmc_alloc_host(sizeof(*host), &pdev->dev);
+ if (!mmc)
+ return -ENOMEM;
- mmc = mmc_alloc_host(sizeof(struct mxcmci_host), &pdev->dev);
- if (!mmc) {
- ret = -ENOMEM;
- goto out_release_mem;
+ host = mmc_priv(mmc);
+
+ host->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(host->base)) {
+ ret = PTR_ERR(host->base);
+ goto out_free;
}
+ host->phys_base = res->start;
+
ret = mmc_of_parse(mmc);
if (ret)
goto out_free;
@@ -1084,13 +1062,6 @@
mmc->max_req_size = mmc->max_blk_size * mmc->max_blk_count;
mmc->max_seg_size = mmc->max_req_size;
- host = mmc_priv(mmc);
- host->base = ioremap(r->start, resource_size(r));
- if (!host->base) {
- ret = -ENOMEM;
- goto out_free;
- }
-
if (of_id) {
const struct platform_device_id *id_entry = of_id->data;
host->devtype = id_entry->driver_data;
@@ -1112,7 +1083,14 @@
&& !of_property_read_bool(pdev->dev.of_node, "cd-gpios"))
dat3_card_detect = true;
- mxcmci_init_ocr(host);
+ ret = mmc_regulator_get_supply(mmc);
+ if (ret) {
+ if (pdata && ret != -EPROBE_DEFER)
+ mmc->ocr_avail = pdata->ocr_avail ? :
+ MMC_VDD_32_33 | MMC_VDD_33_34;
+ else
+ goto out_free;
+ }
if (dat3_card_detect)
host->default_irq_mask =
@@ -1120,19 +1098,16 @@
else
host->default_irq_mask = 0;
- host->res = r;
- host->irq = irq;
-
host->clk_ipg = devm_clk_get(&pdev->dev, "ipg");
if (IS_ERR(host->clk_ipg)) {
ret = PTR_ERR(host->clk_ipg);
- goto out_iounmap;
+ goto out_free;
}
host->clk_per = devm_clk_get(&pdev->dev, "per");
if (IS_ERR(host->clk_per)) {
ret = PTR_ERR(host->clk_per);
- goto out_iounmap;
+ goto out_free;
}
clk_prepare_enable(host->clk_per);
@@ -1159,9 +1134,9 @@
if (!host->pdata) {
host->dma = dma_request_slave_channel(&pdev->dev, "rx-tx");
} else {
- r = platform_get_resource(pdev, IORESOURCE_DMA, 0);
- if (r) {
- host->dmareq = r->start;
+ res = platform_get_resource(pdev, IORESOURCE_DMA, 0);
+ if (res) {
+ host->dmareq = res->start;
host->dma_data.peripheral_type = IMX_DMATYPE_SDHC;
host->dma_data.priority = DMA_PRIO_LOW;
host->dma_data.dma_request = host->dmareq;
@@ -1178,7 +1153,8 @@
INIT_WORK(&host->datawork, mxcmci_datawork);
- ret = request_irq(host->irq, mxcmci_irq, 0, DRIVER_NAME, host);
+ ret = devm_request_irq(&pdev->dev, irq, mxcmci_irq, 0,
+ dev_name(&pdev->dev), host);
if (ret)
goto out_free_dma;
@@ -1188,7 +1164,7 @@
ret = host->pdata->init(&pdev->dev, mxcmci_detect_irq,
host->mmc);
if (ret)
- goto out_free_irq;
+ goto out_free_dma;
}
init_timer(&host->watchdog);
@@ -1199,20 +1175,17 @@
return 0;
-out_free_irq:
- free_irq(host->irq, host);
out_free_dma:
if (host->dma)
dma_release_channel(host->dma);
+
out_clk_put:
clk_disable_unprepare(host->clk_per);
clk_disable_unprepare(host->clk_ipg);
-out_iounmap:
- iounmap(host->base);
+
out_free:
mmc_free_host(mmc);
-out_release_mem:
- release_mem_region(iores->start, resource_size(iores));
+
return ret;
}
@@ -1223,30 +1196,21 @@
mmc_remove_host(mmc);
- if (host->vcc)
- regulator_put(host->vcc);
-
if (host->pdata && host->pdata->exit)
host->pdata->exit(&pdev->dev, mmc);
- free_irq(host->irq, host);
- iounmap(host->base);
-
if (host->dma)
dma_release_channel(host->dma);
clk_disable_unprepare(host->clk_per);
clk_disable_unprepare(host->clk_ipg);
- release_mem_region(host->res->start, resource_size(host->res));
-
mmc_free_host(mmc);
return 0;
}
-#ifdef CONFIG_PM
-static int mxcmci_suspend(struct device *dev)
+static int __maybe_unused mxcmci_suspend(struct device *dev)
{
struct mmc_host *mmc = dev_get_drvdata(dev);
struct mxcmci_host *host = mmc_priv(mmc);
@@ -1256,7 +1220,7 @@
return 0;
}
-static int mxcmci_resume(struct device *dev)
+static int __maybe_unused mxcmci_resume(struct device *dev)
{
struct mmc_host *mmc = dev_get_drvdata(dev);
struct mxcmci_host *host = mmc_priv(mmc);
@@ -1266,11 +1230,7 @@
return 0;
}
-static const struct dev_pm_ops mxcmci_pm_ops = {
- .suspend = mxcmci_suspend,
- .resume = mxcmci_resume,
-};
-#endif
+static SIMPLE_DEV_PM_OPS(mxcmci_pm_ops, mxcmci_suspend, mxcmci_resume);
static struct platform_driver mxcmci_driver = {
.probe = mxcmci_probe,
@@ -1279,9 +1239,7 @@
.driver = {
.name = DRIVER_NAME,
.owner = THIS_MODULE,
-#ifdef CONFIG_PM
.pm = &mxcmci_pm_ops,
-#endif
.of_match_table = mxcmci_of_match,
}
};
diff --git a/drivers/mmc/host/mxs-mmc.c b/drivers/mmc/host/mxs-mmc.c
index 073e871..babfea0 100644
--- a/drivers/mmc/host/mxs-mmc.c
+++ b/drivers/mmc/host/mxs-mmc.c
@@ -70,6 +70,7 @@
unsigned char bus_width;
spinlock_t lock;
int sdio_irq_en;
+ bool broken_cd;
};
static int mxs_mmc_get_cd(struct mmc_host *mmc)
@@ -78,6 +79,9 @@
struct mxs_ssp *ssp = &host->ssp;
int present, ret;
+ if (host->broken_cd)
+ return -ENOSYS;
+
ret = mmc_gpio_get_cd(mmc);
if (ret >= 0)
return ret;
@@ -568,6 +572,7 @@
{
const struct of_device_id *of_id =
of_match_device(mxs_mmc_dt_ids, &pdev->dev);
+ struct device_node *np = pdev->dev.of_node;
struct mxs_mmc_host *host;
struct mmc_host *mmc;
struct resource *iores;
@@ -634,6 +639,8 @@
mmc->caps = MMC_CAP_SD_HIGHSPEED | MMC_CAP_MMC_HIGHSPEED |
MMC_CAP_SDIO_IRQ | MMC_CAP_NEEDS_POLL;
+ host->broken_cd = of_property_read_bool(np, "broken-cd");
+
mmc->f_min = 400000;
mmc->f_max = 288000000;
diff --git a/drivers/mmc/host/omap.c b/drivers/mmc/host/omap.c
index 5c2e58b..81974ec 100644
--- a/drivers/mmc/host/omap.c
+++ b/drivers/mmc/host/omap.c
@@ -177,7 +177,7 @@
unsigned long tick_ns;
if (slot != NULL && slot->host->fclk_enabled && slot->fclk_freq > 0) {
- tick_ns = (1000000000 + slot->fclk_freq - 1) / slot->fclk_freq;
+ tick_ns = DIV_ROUND_UP(NSEC_PER_SEC, slot->fclk_freq);
ndelay(8 * tick_ns);
}
}
@@ -435,7 +435,7 @@
struct mmc_data *data = host->stop_data;
unsigned long tick_ns;
- tick_ns = (1000000000 + slot->fclk_freq - 1)/slot->fclk_freq;
+ tick_ns = DIV_ROUND_UP(NSEC_PER_SEC, slot->fclk_freq);
ndelay(8*tick_ns);
mmc_omap_start_command(host, data->stop);
@@ -477,7 +477,7 @@
u16 stat = 0;
/* Sending abort takes 80 clocks. Have some extra and round up */
- timeout = (120*1000000 + slot->fclk_freq - 1)/slot->fclk_freq;
+ timeout = DIV_ROUND_UP(120 * USEC_PER_SEC, slot->fclk_freq);
restarts = 0;
while (restarts < maxloops) {
OMAP_MMC_WRITE(host, STAT, 0xFFFF);
@@ -677,8 +677,8 @@
if (n > host->buffer_bytes_left)
n = host->buffer_bytes_left;
- nwords = n / 2;
- nwords += n & 1; /* handle odd number of bytes to transfer */
+ /* Round up to handle odd number of bytes to transfer */
+ nwords = DIV_ROUND_UP(n, 2);
host->buffer_bytes_left -= n;
host->total_bytes_left -= n;
diff --git a/drivers/mmc/host/omap_hsmmc.c b/drivers/mmc/host/omap_hsmmc.c
index e91ee21..6b7b755 100644
--- a/drivers/mmc/host/omap_hsmmc.c
+++ b/drivers/mmc/host/omap_hsmmc.c
@@ -31,7 +31,7 @@
#include <linux/of.h>
#include <linux/of_gpio.h>
#include <linux/of_device.h>
-#include <linux/omap-dma.h>
+#include <linux/omap-dmaengine.h>
#include <linux/mmc/host.h>
#include <linux/mmc/core.h>
#include <linux/mmc/mmc.h>
@@ -582,7 +582,7 @@
* - MMC/SD clock coming out of controller > 25MHz
*/
if ((mmc_slot(host).features & HSMMC_HAS_HSPE_SUPPORT) &&
- (ios->timing != MMC_TIMING_UHS_DDR50) &&
+ (ios->timing != MMC_TIMING_MMC_DDR52) &&
((OMAP_HSMMC_READ(host->base, CAPA) & HSS) == HSS)) {
regval = OMAP_HSMMC_READ(host->base, HCTL);
if (clkdiv && (clk_get_rate(host->fclk)/clkdiv) > 25000000)
@@ -602,7 +602,7 @@
u32 con;
con = OMAP_HSMMC_READ(host->base, CON);
- if (ios->timing == MMC_TIMING_UHS_DDR50)
+ if (ios->timing == MMC_TIMING_MMC_DDR52)
con |= DDR; /* configure in DDR mode */
else
con &= ~DDR;
@@ -920,16 +920,17 @@
static void
omap_hsmmc_cmd_done(struct omap_hsmmc_host *host, struct mmc_command *cmd)
{
- host->cmd = NULL;
-
if (host->mrq->sbc && (host->cmd == host->mrq->sbc) &&
!host->mrq->sbc->error && !(host->flags & AUTO_CMD23)) {
+ host->cmd = NULL;
omap_hsmmc_start_dma_transfer(host);
omap_hsmmc_start_command(host, host->mrq->cmd,
host->mrq->data);
return;
}
+ host->cmd = NULL;
+
if (cmd->flags & MMC_RSP_PRESENT) {
if (cmd->flags & MMC_RSP_136) {
/* response type 2 */
@@ -1851,6 +1852,7 @@
unsigned tx_req, rx_req;
struct pinctrl *pinctrl;
const struct omap_mmc_of_data *data;
+ void __iomem *base;
match = of_match_device(of_match_ptr(omap_mmc_of_match), &pdev->dev);
if (match) {
@@ -1881,9 +1883,9 @@
if (res == NULL || irq < 0)
return -ENXIO;
- res = request_mem_region(res->start, resource_size(res), pdev->name);
- if (res == NULL)
- return -EBUSY;
+ base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
ret = omap_hsmmc_gpio_init(pdata);
if (ret)
@@ -1904,7 +1906,7 @@
host->irq = irq;
host->slot_id = 0;
host->mapbase = res->start + pdata->reg_offset;
- host->base = ioremap(host->mapbase, SZ_4K);
+ host->base = base + pdata->reg_offset;
host->power_mode = MMC_POWER_OFF;
host->next_data.cookie = 1;
host->pbias_enabled = 0;
@@ -1922,7 +1924,7 @@
spin_lock_init(&host->irq_lock);
- host->fclk = clk_get(&pdev->dev, "fck");
+ host->fclk = devm_clk_get(&pdev->dev, "fck");
if (IS_ERR(host->fclk)) {
ret = PTR_ERR(host->fclk);
host->fclk = NULL;
@@ -1941,7 +1943,7 @@
omap_hsmmc_context_save(host);
- host->dbclk = clk_get(&pdev->dev, "mmchsdb_fck");
+ host->dbclk = devm_clk_get(&pdev->dev, "mmchsdb_fck");
/*
* MMC can still work without debounce clock.
*/
@@ -1949,7 +1951,6 @@
host->dbclk = NULL;
} else if (clk_prepare_enable(host->dbclk) != 0) {
dev_warn(mmc_dev(host->mmc), "Failed to enable debounce clk\n");
- clk_put(host->dbclk);
host->dbclk = NULL;
}
@@ -2018,7 +2019,7 @@
}
/* Request IRQ for MMC operations */
- ret = request_irq(host->irq, omap_hsmmc_irq, 0,
+ ret = devm_request_irq(&pdev->dev, host->irq, omap_hsmmc_irq, 0,
mmc_hostname(mmc), host);
if (ret) {
dev_err(mmc_dev(host->mmc), "Unable to grab HSMMC IRQ\n");
@@ -2029,7 +2030,7 @@
if (pdata->init(&pdev->dev) != 0) {
dev_err(mmc_dev(host->mmc),
"Unable to configure MMC IRQs\n");
- goto err_irq_cd_init;
+ goto err_irq;
}
}
@@ -2044,9 +2045,9 @@
/* Request IRQ for card detect */
if ((mmc_slot(host).card_detect_irq)) {
- ret = request_threaded_irq(mmc_slot(host).card_detect_irq,
- NULL,
- omap_hsmmc_detect,
+ ret = devm_request_threaded_irq(&pdev->dev,
+ mmc_slot(host).card_detect_irq,
+ NULL, omap_hsmmc_detect,
IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
mmc_hostname(mmc), host);
if (ret) {
@@ -2089,15 +2090,12 @@
err_slot_name:
mmc_remove_host(mmc);
- free_irq(mmc_slot(host).card_detect_irq, host);
err_irq_cd:
if (host->use_reg)
omap_hsmmc_reg_put(host);
err_reg:
if (host->pdata->cleanup)
host->pdata->cleanup(&pdev->dev);
-err_irq_cd_init:
- free_irq(host->irq, host);
err_irq:
if (host->tx_chan)
dma_release_channel(host->tx_chan);
@@ -2105,27 +2103,19 @@
dma_release_channel(host->rx_chan);
pm_runtime_put_sync(host->dev);
pm_runtime_disable(host->dev);
- clk_put(host->fclk);
- if (host->dbclk) {
+ if (host->dbclk)
clk_disable_unprepare(host->dbclk);
- clk_put(host->dbclk);
- }
err1:
- iounmap(host->base);
mmc_free_host(mmc);
err_alloc:
omap_hsmmc_gpio_free(pdata);
err:
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (res)
- release_mem_region(res->start, resource_size(res));
return ret;
}
static int omap_hsmmc_remove(struct platform_device *pdev)
{
struct omap_hsmmc_host *host = platform_get_drvdata(pdev);
- struct resource *res;
pm_runtime_get_sync(host->dev);
mmc_remove_host(host->mmc);
@@ -2133,9 +2123,6 @@
omap_hsmmc_reg_put(host);
if (host->pdata->cleanup)
host->pdata->cleanup(&pdev->dev);
- free_irq(host->irq, host);
- if (mmc_slot(host).card_detect_irq)
- free_irq(mmc_slot(host).card_detect_irq, host);
if (host->tx_chan)
dma_release_channel(host->tx_chan);
@@ -2144,20 +2131,12 @@
pm_runtime_put_sync(host->dev);
pm_runtime_disable(host->dev);
- clk_put(host->fclk);
- if (host->dbclk) {
+ if (host->dbclk)
clk_disable_unprepare(host->dbclk);
- clk_put(host->dbclk);
- }
omap_hsmmc_gpio_free(host->pdata);
- iounmap(host->base);
mmc_free_host(host->mmc);
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (res)
- release_mem_region(res->start, resource_size(res));
-
return 0;
}
diff --git a/drivers/mmc/host/rtsx_pci_sdmmc.c b/drivers/mmc/host/rtsx_pci_sdmmc.c
index 0b9ded1..0d51964 100644
--- a/drivers/mmc/host/rtsx_pci_sdmmc.c
+++ b/drivers/mmc/host/rtsx_pci_sdmmc.c
@@ -236,6 +236,9 @@
case MMC_RSP_R1:
rsp_type = SD_RSP_TYPE_R1;
break;
+ case MMC_RSP_R1 & ~MMC_RSP_CRC:
+ rsp_type = SD_RSP_TYPE_R1 | SD_NO_CHECK_CRC7;
+ break;
case MMC_RSP_R1B:
rsp_type = SD_RSP_TYPE_R1b;
break;
@@ -816,6 +819,7 @@
rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, CLK_CTL, CLK_LOW_FREQ, 0);
break;
+ case MMC_TIMING_MMC_DDR52:
case MMC_TIMING_UHS_DDR50:
rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, SD_CFG1,
0x0C | SD_ASYNC_FIFO_NOT_RST,
@@ -896,6 +900,7 @@
host->vpclk = true;
host->double_clk = false;
break;
+ case MMC_TIMING_MMC_DDR52:
case MMC_TIMING_UHS_DDR50:
case MMC_TIMING_UHS_SDR25:
host->ssc_depth = RTSX_SSC_DEPTH_1M;
diff --git a/drivers/mmc/host/rtsx_usb_sdmmc.c b/drivers/mmc/host/rtsx_usb_sdmmc.c
new file mode 100644
index 0000000..5d3766e
--- /dev/null
+++ b/drivers/mmc/host/rtsx_usb_sdmmc.c
@@ -0,0 +1,1456 @@
+/* Realtek USB SD/MMC Card Interface driver
+ *
+ * Copyright(c) 2009-2013 Realtek Semiconductor Corp. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author:
+ * Roger Tseng <rogerable@realtek.com>
+ */
+
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/delay.h>
+#include <linux/platform_device.h>
+#include <linux/usb.h>
+#include <linux/mmc/host.h>
+#include <linux/mmc/mmc.h>
+#include <linux/mmc/sd.h>
+#include <linux/mmc/sdio.h>
+#include <linux/mmc/card.h>
+#include <linux/scatterlist.h>
+#include <linux/pm_runtime.h>
+
+#include <linux/mfd/rtsx_usb.h>
+#include <asm/unaligned.h>
+
+#if defined(CONFIG_LEDS_CLASS) || (defined(CONFIG_LEDS_CLASS_MODULE) && \
+ defined(CONFIG_MMC_REALTEK_USB_MODULE))
+#include <linux/leds.h>
+#include <linux/workqueue.h>
+#define RTSX_USB_USE_LEDS_CLASS
+#endif
+
+struct rtsx_usb_sdmmc {
+ struct platform_device *pdev;
+ struct rtsx_ucr *ucr;
+ struct mmc_host *mmc;
+ struct mmc_request *mrq;
+
+ struct mutex host_mutex;
+
+ u8 ssc_depth;
+ unsigned int clock;
+ bool vpclk;
+ bool double_clk;
+ bool host_removal;
+ bool card_exist;
+ bool initial_mode;
+ bool ddr_mode;
+
+ unsigned char power_mode;
+
+#ifdef RTSX_USB_USE_LEDS_CLASS
+ struct led_classdev led;
+ char led_name[32];
+ struct work_struct led_work;
+#endif
+};
+
+static inline struct device *sdmmc_dev(struct rtsx_usb_sdmmc *host)
+{
+ return &(host->pdev->dev);
+}
+
+static inline void sd_clear_error(struct rtsx_usb_sdmmc *host)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ rtsx_usb_ep0_write_register(ucr, CARD_STOP,
+ SD_STOP | SD_CLR_ERR,
+ SD_STOP | SD_CLR_ERR);
+
+ rtsx_usb_clear_dma_err(ucr);
+ rtsx_usb_clear_fsm_err(ucr);
+}
+
+#ifdef DEBUG
+static void sd_print_debug_regs(struct rtsx_usb_sdmmc *host)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ u8 val = 0;
+
+ rtsx_usb_ep0_read_register(ucr, SD_STAT1, &val);
+ dev_dbg(sdmmc_dev(host), "SD_STAT1: 0x%x\n", val);
+ rtsx_usb_ep0_read_register(ucr, SD_STAT2, &val);
+ dev_dbg(sdmmc_dev(host), "SD_STAT2: 0x%x\n", val);
+ rtsx_usb_ep0_read_register(ucr, SD_BUS_STAT, &val);
+ dev_dbg(sdmmc_dev(host), "SD_BUS_STAT: 0x%x\n", val);
+}
+#else
+#define sd_print_debug_regs(host)
+#endif /* DEBUG */
+
+static int sd_read_data(struct rtsx_usb_sdmmc *host, struct mmc_command *cmd,
+ u16 byte_cnt, u8 *buf, int buf_len, int timeout)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+ u8 trans_mode;
+
+ if (!buf)
+ buf_len = 0;
+
+ rtsx_usb_init_cmd(ucr);
+ if (cmd != NULL) {
+ dev_dbg(sdmmc_dev(host), "%s: SD/MMC CMD%d\n", __func__
+ , cmd->opcode);
+ if (cmd->opcode == MMC_SEND_TUNING_BLOCK)
+ trans_mode = SD_TM_AUTO_TUNING;
+ else
+ trans_mode = SD_TM_NORMAL_READ;
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_CMD0, 0xFF, (u8)(cmd->opcode) | 0x40);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_CMD1, 0xFF, (u8)(cmd->arg >> 24));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_CMD2, 0xFF, (u8)(cmd->arg >> 16));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_CMD3, 0xFF, (u8)(cmd->arg >> 8));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_CMD4, 0xFF, (u8)cmd->arg);
+ } else {
+ trans_mode = SD_TM_AUTO_READ_3;
+ }
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BYTE_CNT_L, 0xFF, (u8)byte_cnt);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BYTE_CNT_H,
+ 0xFF, (u8)(byte_cnt >> 8));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BLOCK_CNT_L, 0xFF, 1);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BLOCK_CNT_H, 0xFF, 0);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CFG2, 0xFF,
+ SD_CALCULATE_CRC7 | SD_CHECK_CRC16 |
+ SD_NO_WAIT_BUSY_END | SD_CHECK_CRC7 | SD_RSP_LEN_6);
+ if (trans_mode != SD_TM_AUTO_TUNING)
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ CARD_DATA_SOURCE, 0x01, PINGPONG_BUFFER);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_TRANSFER,
+ 0xFF, trans_mode | SD_TRANSFER_START);
+ rtsx_usb_add_cmd(ucr, CHECK_REG_CMD, SD_TRANSFER,
+ SD_TRANSFER_END, SD_TRANSFER_END);
+
+ if (cmd != NULL) {
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, SD_CMD1, 0, 0);
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, SD_CMD2, 0, 0);
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, SD_CMD3, 0, 0);
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, SD_CMD4, 0, 0);
+ }
+
+ err = rtsx_usb_send_cmd(ucr, MODE_CR, timeout);
+ if (err) {
+ dev_dbg(sdmmc_dev(host),
+ "rtsx_usb_send_cmd failed (err = %d)\n", err);
+ return err;
+ }
+
+ err = rtsx_usb_get_rsp(ucr, !cmd ? 1 : 5, timeout);
+ if (err || (ucr->rsp_buf[0] & SD_TRANSFER_ERR)) {
+ sd_print_debug_regs(host);
+
+ if (!err) {
+ dev_dbg(sdmmc_dev(host),
+ "Transfer failed (SD_TRANSFER = %02x)\n",
+ ucr->rsp_buf[0]);
+ err = -EIO;
+ } else {
+ dev_dbg(sdmmc_dev(host),
+ "rtsx_usb_get_rsp failed (err = %d)\n", err);
+ }
+
+ return err;
+ }
+
+ if (cmd != NULL) {
+ cmd->resp[0] = get_unaligned_be32(ucr->rsp_buf + 1);
+ dev_dbg(sdmmc_dev(host), "cmd->resp[0] = 0x%08x\n",
+ cmd->resp[0]);
+ }
+
+ if (buf && buf_len) {
+ /* 2-byte aligned part */
+ err = rtsx_usb_read_ppbuf(ucr, buf, byte_cnt - (byte_cnt % 2));
+ if (err) {
+ dev_dbg(sdmmc_dev(host),
+ "rtsx_usb_read_ppbuf failed (err = %d)\n", err);
+ return err;
+ }
+
+ /* unaligned byte */
+ if (byte_cnt % 2)
+ return rtsx_usb_read_register(ucr,
+ PPBUF_BASE2 + byte_cnt,
+ buf + byte_cnt - 1);
+ }
+
+ return 0;
+}
+
+static int sd_write_data(struct rtsx_usb_sdmmc *host, struct mmc_command *cmd,
+ u16 byte_cnt, u8 *buf, int buf_len, int timeout)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+ u8 trans_mode;
+
+ if (!buf)
+ buf_len = 0;
+
+ if (buf && buf_len) {
+ err = rtsx_usb_write_ppbuf(ucr, buf, buf_len);
+ if (err) {
+ dev_dbg(sdmmc_dev(host),
+ "rtsx_usb_write_ppbuf failed (err = %d)\n",
+ err);
+ return err;
+ }
+ }
+
+ trans_mode = (cmd != NULL) ? SD_TM_AUTO_WRITE_2 : SD_TM_AUTO_WRITE_3;
+ rtsx_usb_init_cmd(ucr);
+
+ if (cmd != NULL) {
+ dev_dbg(sdmmc_dev(host), "%s: SD/MMC CMD%d\n", __func__,
+ cmd->opcode);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_CMD0, 0xFF, (u8)(cmd->opcode) | 0x40);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_CMD1, 0xFF, (u8)(cmd->arg >> 24));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_CMD2, 0xFF, (u8)(cmd->arg >> 16));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_CMD3, 0xFF, (u8)(cmd->arg >> 8));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_CMD4, 0xFF, (u8)cmd->arg);
+ }
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BYTE_CNT_L, 0xFF, (u8)byte_cnt);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BYTE_CNT_H,
+ 0xFF, (u8)(byte_cnt >> 8));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BLOCK_CNT_L, 0xFF, 1);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BLOCK_CNT_H, 0xFF, 0);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CFG2, 0xFF,
+ SD_CALCULATE_CRC7 | SD_CHECK_CRC16 |
+ SD_NO_WAIT_BUSY_END | SD_CHECK_CRC7 | SD_RSP_LEN_6);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ CARD_DATA_SOURCE, 0x01, PINGPONG_BUFFER);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_TRANSFER, 0xFF,
+ trans_mode | SD_TRANSFER_START);
+ rtsx_usb_add_cmd(ucr, CHECK_REG_CMD, SD_TRANSFER,
+ SD_TRANSFER_END, SD_TRANSFER_END);
+
+ if (cmd != NULL) {
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, SD_CMD1, 0, 0);
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, SD_CMD2, 0, 0);
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, SD_CMD3, 0, 0);
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, SD_CMD4, 0, 0);
+ }
+
+ err = rtsx_usb_send_cmd(ucr, MODE_CR, timeout);
+ if (err) {
+ dev_dbg(sdmmc_dev(host),
+ "rtsx_usb_send_cmd failed (err = %d)\n", err);
+ return err;
+ }
+
+ err = rtsx_usb_get_rsp(ucr, !cmd ? 1 : 5, timeout);
+ if (err) {
+ sd_print_debug_regs(host);
+ dev_dbg(sdmmc_dev(host),
+ "rtsx_usb_get_rsp failed (err = %d)\n", err);
+ return err;
+ }
+
+ if (cmd != NULL) {
+ cmd->resp[0] = get_unaligned_be32(ucr->rsp_buf + 1);
+ dev_dbg(sdmmc_dev(host), "cmd->resp[0] = 0x%08x\n",
+ cmd->resp[0]);
+ }
+
+ return 0;
+}
+
+static void sd_send_cmd_get_rsp(struct rtsx_usb_sdmmc *host,
+ struct mmc_command *cmd)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ u8 cmd_idx = (u8)cmd->opcode;
+ u32 arg = cmd->arg;
+ int err = 0;
+ int timeout = 100;
+ int i;
+ u8 *ptr;
+ int stat_idx = 0;
+ int len = 2;
+ u8 rsp_type;
+
+ dev_dbg(sdmmc_dev(host), "%s: SD/MMC CMD %d, arg = 0x%08x\n",
+ __func__, cmd_idx, arg);
+
+ /* Response type:
+ * R0
+ * R1, R5, R6, R7
+ * R1b
+ * R2
+ * R3, R4
+ */
+ switch (mmc_resp_type(cmd)) {
+ case MMC_RSP_NONE:
+ rsp_type = SD_RSP_TYPE_R0;
+ break;
+ case MMC_RSP_R1:
+ rsp_type = SD_RSP_TYPE_R1;
+ break;
+ case MMC_RSP_R1 & ~MMC_RSP_CRC:
+ rsp_type = SD_RSP_TYPE_R1 | SD_NO_CHECK_CRC7;
+ break;
+ case MMC_RSP_R1B:
+ rsp_type = SD_RSP_TYPE_R1b;
+ break;
+ case MMC_RSP_R2:
+ rsp_type = SD_RSP_TYPE_R2;
+ break;
+ case MMC_RSP_R3:
+ rsp_type = SD_RSP_TYPE_R3;
+ break;
+ default:
+ dev_dbg(sdmmc_dev(host), "cmd->flag is not valid\n");
+ err = -EINVAL;
+ goto out;
+ }
+
+ if (rsp_type == SD_RSP_TYPE_R1b)
+ timeout = 3000;
+
+ if (cmd->opcode == SD_SWITCH_VOLTAGE) {
+ err = rtsx_usb_write_register(ucr, SD_BUS_STAT,
+ SD_CLK_TOGGLE_EN | SD_CLK_FORCE_STOP,
+ SD_CLK_TOGGLE_EN);
+ if (err)
+ goto out;
+ }
+
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CMD0, 0xFF, 0x40 | cmd_idx);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CMD1, 0xFF, (u8)(arg >> 24));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CMD2, 0xFF, (u8)(arg >> 16));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CMD3, 0xFF, (u8)(arg >> 8));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CMD4, 0xFF, (u8)arg);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CFG2, 0xFF, rsp_type);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_DATA_SOURCE,
+ 0x01, PINGPONG_BUFFER);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_TRANSFER,
+ 0xFF, SD_TM_CMD_RSP | SD_TRANSFER_START);
+ rtsx_usb_add_cmd(ucr, CHECK_REG_CMD, SD_TRANSFER,
+ SD_TRANSFER_END | SD_STAT_IDLE,
+ SD_TRANSFER_END | SD_STAT_IDLE);
+
+ if (rsp_type == SD_RSP_TYPE_R2) {
+ /* Read data from ping-pong buffer */
+ for (i = PPBUF_BASE2; i < PPBUF_BASE2 + 16; i++)
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, (u16)i, 0, 0);
+ stat_idx = 16;
+ } else if (rsp_type != SD_RSP_TYPE_R0) {
+ /* Read data from SD_CMDx registers */
+ for (i = SD_CMD0; i <= SD_CMD4; i++)
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, (u16)i, 0, 0);
+ stat_idx = 5;
+ }
+ len += stat_idx;
+
+ rtsx_usb_add_cmd(ucr, READ_REG_CMD, SD_STAT1, 0, 0);
+
+ err = rtsx_usb_send_cmd(ucr, MODE_CR, 100);
+ if (err) {
+ dev_dbg(sdmmc_dev(host),
+ "rtsx_usb_send_cmd error (err = %d)\n", err);
+ goto out;
+ }
+
+ err = rtsx_usb_get_rsp(ucr, len, timeout);
+ if (err || (ucr->rsp_buf[0] & SD_TRANSFER_ERR)) {
+ sd_print_debug_regs(host);
+ sd_clear_error(host);
+
+ if (!err) {
+ dev_dbg(sdmmc_dev(host),
+ "Transfer failed (SD_TRANSFER = %02x)\n",
+ ucr->rsp_buf[0]);
+ err = -EIO;
+ } else {
+ dev_dbg(sdmmc_dev(host),
+ "rtsx_usb_get_rsp failed (err = %d)\n", err);
+ }
+
+ goto out;
+ }
+
+ if (rsp_type == SD_RSP_TYPE_R0) {
+ err = 0;
+ goto out;
+ }
+
+ /* Skip result of CHECK_REG_CMD */
+ ptr = ucr->rsp_buf + 1;
+
+ /* Check (Start,Transmission) bit of Response */
+ if ((ptr[0] & 0xC0) != 0) {
+ err = -EILSEQ;
+ dev_dbg(sdmmc_dev(host), "Invalid response bit\n");
+ goto out;
+ }
+
+ /* Check CRC7 */
+ if (!(rsp_type & SD_NO_CHECK_CRC7)) {
+ if (ptr[stat_idx] & SD_CRC7_ERR) {
+ err = -EILSEQ;
+ dev_dbg(sdmmc_dev(host), "CRC7 error\n");
+ goto out;
+ }
+ }
+
+ if (rsp_type == SD_RSP_TYPE_R2) {
+ for (i = 0; i < 4; i++) {
+ cmd->resp[i] = get_unaligned_be32(ptr + 1 + i * 4);
+ dev_dbg(sdmmc_dev(host), "cmd->resp[%d] = 0x%08x\n",
+ i, cmd->resp[i]);
+ }
+ } else {
+ cmd->resp[0] = get_unaligned_be32(ptr + 1);
+ dev_dbg(sdmmc_dev(host), "cmd->resp[0] = 0x%08x\n",
+ cmd->resp[0]);
+ }
+
+out:
+ cmd->error = err;
+}
+
+static int sd_rw_multi(struct rtsx_usb_sdmmc *host, struct mmc_request *mrq)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ struct mmc_data *data = mrq->data;
+ int read = (data->flags & MMC_DATA_READ) ? 1 : 0;
+ u8 cfg2, trans_mode;
+ int err;
+ u8 flag;
+ size_t data_len = data->blksz * data->blocks;
+ unsigned int pipe;
+
+ if (read) {
+ dev_dbg(sdmmc_dev(host), "%s: read %zu bytes\n",
+ __func__, data_len);
+ cfg2 = SD_CALCULATE_CRC7 | SD_CHECK_CRC16 |
+ SD_NO_WAIT_BUSY_END | SD_CHECK_CRC7 | SD_RSP_LEN_0;
+ trans_mode = SD_TM_AUTO_READ_3;
+ } else {
+ dev_dbg(sdmmc_dev(host), "%s: write %zu bytes\n",
+ __func__, data_len);
+ cfg2 = SD_NO_CALCULATE_CRC7 | SD_CHECK_CRC16 |
+ SD_NO_WAIT_BUSY_END | SD_NO_CHECK_CRC7 | SD_RSP_LEN_0;
+ trans_mode = SD_TM_AUTO_WRITE_3;
+ }
+
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BYTE_CNT_L, 0xFF, 0x00);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BYTE_CNT_H, 0xFF, 0x02);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BLOCK_CNT_L,
+ 0xFF, (u8)data->blocks);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BLOCK_CNT_H,
+ 0xFF, (u8)(data->blocks >> 8));
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_DATA_SOURCE,
+ 0x01, RING_BUFFER);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MC_DMA_TC3,
+ 0xFF, (u8)(data_len >> 24));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MC_DMA_TC2,
+ 0xFF, (u8)(data_len >> 16));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MC_DMA_TC1,
+ 0xFF, (u8)(data_len >> 8));
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MC_DMA_TC0,
+ 0xFF, (u8)data_len);
+ if (read) {
+ flag = MODE_CDIR;
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MC_DMA_CTL,
+ 0x03 | DMA_PACK_SIZE_MASK,
+ DMA_DIR_FROM_CARD | DMA_EN | DMA_512);
+ } else {
+ flag = MODE_CDOR;
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, MC_DMA_CTL,
+ 0x03 | DMA_PACK_SIZE_MASK,
+ DMA_DIR_TO_CARD | DMA_EN | DMA_512);
+ }
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CFG2, 0xFF, cfg2);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_TRANSFER, 0xFF,
+ trans_mode | SD_TRANSFER_START);
+ rtsx_usb_add_cmd(ucr, CHECK_REG_CMD, SD_TRANSFER,
+ SD_TRANSFER_END, SD_TRANSFER_END);
+
+ err = rtsx_usb_send_cmd(ucr, flag, 100);
+ if (err)
+ return err;
+
+ if (read)
+ pipe = usb_rcvbulkpipe(ucr->pusb_dev, EP_BULK_IN);
+ else
+ pipe = usb_sndbulkpipe(ucr->pusb_dev, EP_BULK_OUT);
+
+ err = rtsx_usb_transfer_data(ucr, pipe, data->sg, data_len,
+ data->sg_len, NULL, 10000);
+ if (err) {
+ dev_dbg(sdmmc_dev(host), "rtsx_usb_transfer_data error %d\n"
+ , err);
+ sd_clear_error(host);
+ return err;
+ }
+
+ return rtsx_usb_get_rsp(ucr, 1, 2000);
+}
+
+static inline void sd_enable_initial_mode(struct rtsx_usb_sdmmc *host)
+{
+ rtsx_usb_write_register(host->ucr, SD_CFG1,
+ SD_CLK_DIVIDE_MASK, SD_CLK_DIVIDE_128);
+}
+
+static inline void sd_disable_initial_mode(struct rtsx_usb_sdmmc *host)
+{
+ rtsx_usb_write_register(host->ucr, SD_CFG1,
+ SD_CLK_DIVIDE_MASK, SD_CLK_DIVIDE_0);
+}
+
+static void sd_normal_rw(struct rtsx_usb_sdmmc *host,
+ struct mmc_request *mrq)
+{
+ struct mmc_command *cmd = mrq->cmd;
+ struct mmc_data *data = mrq->data;
+ u8 *buf;
+
+ buf = kzalloc(data->blksz, GFP_NOIO);
+ if (!buf) {
+ cmd->error = -ENOMEM;
+ return;
+ }
+
+ if (data->flags & MMC_DATA_READ) {
+ if (host->initial_mode)
+ sd_disable_initial_mode(host);
+
+ cmd->error = sd_read_data(host, cmd, (u16)data->blksz, buf,
+ data->blksz, 200);
+
+ if (host->initial_mode)
+ sd_enable_initial_mode(host);
+
+ sg_copy_from_buffer(data->sg, data->sg_len, buf, data->blksz);
+ } else {
+ sg_copy_to_buffer(data->sg, data->sg_len, buf, data->blksz);
+
+ cmd->error = sd_write_data(host, cmd, (u16)data->blksz, buf,
+ data->blksz, 200);
+ }
+
+ kfree(buf);
+}
+
+static int sd_change_phase(struct rtsx_usb_sdmmc *host, u8 sample_point, int tx)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+
+ dev_dbg(sdmmc_dev(host), "%s: %s sample_point = %d\n",
+ __func__, tx ? "TX" : "RX", sample_point);
+
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CLK_DIV, CLK_CHANGE, CLK_CHANGE);
+
+ if (tx)
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_VPCLK0_CTL,
+ 0x0F, sample_point);
+ else
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_VPCLK1_CTL,
+ 0x0F, sample_point);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_VPCLK0_CTL, PHASE_NOT_RESET, 0);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_VPCLK0_CTL,
+ PHASE_NOT_RESET, PHASE_NOT_RESET);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CLK_DIV, CLK_CHANGE, 0);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CFG1, SD_ASYNC_FIFO_RST, 0);
+
+ err = rtsx_usb_send_cmd(ucr, MODE_C, 100);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static inline u32 get_phase_point(u32 phase_map, unsigned int idx)
+{
+ idx &= MAX_PHASE;
+ return phase_map & (1 << idx);
+}
+
+static int get_phase_len(u32 phase_map, unsigned int idx)
+{
+ int i;
+
+ for (i = 0; i < MAX_PHASE + 1; i++) {
+ if (get_phase_point(phase_map, idx + i) == 0)
+ return i;
+ }
+ return MAX_PHASE + 1;
+}
+
+static u8 sd_search_final_phase(struct rtsx_usb_sdmmc *host, u32 phase_map)
+{
+ int start = 0, len = 0;
+ int start_final = 0, len_final = 0;
+ u8 final_phase = 0xFF;
+
+ if (phase_map == 0) {
+ dev_dbg(sdmmc_dev(host), "Phase: [map:%x]\n", phase_map);
+ return final_phase;
+ }
+
+ while (start < MAX_PHASE + 1) {
+ len = get_phase_len(phase_map, start);
+ if (len_final < len) {
+ start_final = start;
+ len_final = len;
+ }
+ start += len ? len : 1;
+ }
+
+ final_phase = (start_final + len_final / 2) & MAX_PHASE;
+ dev_dbg(sdmmc_dev(host), "Phase: [map:%x] [maxlen:%d] [final:%d]\n",
+ phase_map, len_final, final_phase);
+
+ return final_phase;
+}
+
+static void sd_wait_data_idle(struct rtsx_usb_sdmmc *host)
+{
+ int err, i;
+ u8 val = 0;
+
+ for (i = 0; i < 100; i++) {
+ err = rtsx_usb_ep0_read_register(host->ucr,
+ SD_DATA_STATE, &val);
+ if (val & SD_DATA_IDLE)
+ return;
+
+ usleep_range(100, 1000);
+ }
+}
+
+static int sd_tuning_rx_cmd(struct rtsx_usb_sdmmc *host,
+ u8 opcode, u8 sample_point)
+{
+ int err;
+ struct mmc_command cmd = {0};
+
+ err = sd_change_phase(host, sample_point, 0);
+ if (err)
+ return err;
+
+ cmd.opcode = MMC_SEND_TUNING_BLOCK;
+ err = sd_read_data(host, &cmd, 0x40, NULL, 0, 100);
+ if (err) {
+ /* Wait till SD DATA IDLE */
+ sd_wait_data_idle(host);
+ sd_clear_error(host);
+ return err;
+ }
+
+ return 0;
+}
+
+static void sd_tuning_phase(struct rtsx_usb_sdmmc *host,
+ u8 opcode, u16 *phase_map)
+{
+ int err, i;
+ u16 raw_phase_map = 0;
+
+ for (i = MAX_PHASE; i >= 0; i--) {
+ err = sd_tuning_rx_cmd(host, opcode, (u8)i);
+ if (!err)
+ raw_phase_map |= 1 << i;
+ }
+
+ if (phase_map)
+ *phase_map = raw_phase_map;
+}
+
+static int sd_tuning_rx(struct rtsx_usb_sdmmc *host, u8 opcode)
+{
+ int err, i;
+ u16 raw_phase_map[RX_TUNING_CNT] = {0}, phase_map;
+ u8 final_phase;
+
+ /* setting fixed default TX phase */
+ err = sd_change_phase(host, 0x01, 1);
+ if (err) {
+ dev_dbg(sdmmc_dev(host), "TX phase setting failed\n");
+ return err;
+ }
+
+ /* tuning RX phase */
+ for (i = 0; i < RX_TUNING_CNT; i++) {
+ sd_tuning_phase(host, opcode, &(raw_phase_map[i]));
+
+ if (raw_phase_map[i] == 0)
+ break;
+ }
+
+ phase_map = 0xFFFF;
+ for (i = 0; i < RX_TUNING_CNT; i++) {
+ dev_dbg(sdmmc_dev(host), "RX raw_phase_map[%d] = 0x%04x\n",
+ i, raw_phase_map[i]);
+ phase_map &= raw_phase_map[i];
+ }
+ dev_dbg(sdmmc_dev(host), "RX phase_map = 0x%04x\n", phase_map);
+
+ if (phase_map) {
+ final_phase = sd_search_final_phase(host, phase_map);
+ if (final_phase == 0xFF)
+ return -EINVAL;
+
+ err = sd_change_phase(host, final_phase, 0);
+ if (err)
+ return err;
+ } else {
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int sdmmc_get_ro(struct mmc_host *mmc)
+{
+ struct rtsx_usb_sdmmc *host = mmc_priv(mmc);
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+ u16 val;
+
+ if (host->host_removal)
+ return -ENOMEDIUM;
+
+ mutex_lock(&ucr->dev_mutex);
+
+ /* Check SD card detect */
+ err = rtsx_usb_get_card_status(ucr, &val);
+
+ mutex_unlock(&ucr->dev_mutex);
+
+
+ /* Treat failed detection as non-ro */
+ if (err)
+ return 0;
+
+ if (val & SD_WP)
+ return 1;
+
+ return 0;
+}
+
+static int sdmmc_get_cd(struct mmc_host *mmc)
+{
+ struct rtsx_usb_sdmmc *host = mmc_priv(mmc);
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+ u16 val;
+
+ if (host->host_removal)
+ return -ENOMEDIUM;
+
+ mutex_lock(&ucr->dev_mutex);
+
+ /* Check SD card detect */
+ err = rtsx_usb_get_card_status(ucr, &val);
+
+ mutex_unlock(&ucr->dev_mutex);
+
+ /* Treat failed detection as non-exist */
+ if (err)
+ goto no_card;
+
+ if (val & SD_CD) {
+ host->card_exist = true;
+ return 1;
+ }
+
+no_card:
+ host->card_exist = false;
+ return 0;
+}
+
+static void sdmmc_request(struct mmc_host *mmc, struct mmc_request *mrq)
+{
+ struct rtsx_usb_sdmmc *host = mmc_priv(mmc);
+ struct rtsx_ucr *ucr = host->ucr;
+ struct mmc_command *cmd = mrq->cmd;
+ struct mmc_data *data = mrq->data;
+ unsigned int data_size = 0;
+
+ dev_dbg(sdmmc_dev(host), "%s\n", __func__);
+
+ if (host->host_removal) {
+ cmd->error = -ENOMEDIUM;
+ goto finish;
+ }
+
+ if ((!host->card_exist)) {
+ cmd->error = -ENOMEDIUM;
+ goto finish_detect_card;
+ }
+
+ /*
+ * Reject SDIO CMDs to speed up card identification
+ * since unsupported
+ */
+ if (cmd->opcode == SD_IO_SEND_OP_COND ||
+ cmd->opcode == SD_IO_RW_DIRECT ||
+ cmd->opcode == SD_IO_RW_EXTENDED) {
+ cmd->error = -EINVAL;
+ goto finish;
+ }
+
+ mutex_lock(&ucr->dev_mutex);
+
+ mutex_lock(&host->host_mutex);
+ host->mrq = mrq;
+ mutex_unlock(&host->host_mutex);
+
+ if (mrq->data)
+ data_size = data->blocks * data->blksz;
+
+ if (!data_size) {
+ sd_send_cmd_get_rsp(host, cmd);
+ } else if ((!(data_size % 512) && cmd->opcode != MMC_SEND_EXT_CSD) ||
+ mmc_op_multi(cmd->opcode)) {
+ sd_send_cmd_get_rsp(host, cmd);
+
+ if (!cmd->error) {
+ sd_rw_multi(host, mrq);
+
+ if (mmc_op_multi(cmd->opcode) && mrq->stop) {
+ sd_send_cmd_get_rsp(host, mrq->stop);
+ rtsx_usb_write_register(ucr, MC_FIFO_CTL,
+ FIFO_FLUSH, FIFO_FLUSH);
+ }
+ }
+ } else {
+ sd_normal_rw(host, mrq);
+ }
+
+ if (mrq->data) {
+ if (cmd->error || data->error)
+ data->bytes_xfered = 0;
+ else
+ data->bytes_xfered = data->blocks * data->blksz;
+ }
+
+ mutex_unlock(&ucr->dev_mutex);
+
+finish_detect_card:
+ if (cmd->error) {
+ /*
+ * detect card when fail to update card existence state and
+ * speed up card removal when retry
+ */
+ sdmmc_get_cd(mmc);
+ dev_dbg(sdmmc_dev(host), "cmd->error = %d\n", cmd->error);
+ }
+
+finish:
+ mutex_lock(&host->host_mutex);
+ host->mrq = NULL;
+ mutex_unlock(&host->host_mutex);
+
+ mmc_request_done(mmc, mrq);
+}
+
+static int sd_set_bus_width(struct rtsx_usb_sdmmc *host,
+ unsigned char bus_width)
+{
+ int err = 0;
+ u8 width[] = {
+ [MMC_BUS_WIDTH_1] = SD_BUS_WIDTH_1BIT,
+ [MMC_BUS_WIDTH_4] = SD_BUS_WIDTH_4BIT,
+ [MMC_BUS_WIDTH_8] = SD_BUS_WIDTH_8BIT,
+ };
+
+ if (bus_width <= MMC_BUS_WIDTH_8)
+ err = rtsx_usb_write_register(host->ucr, SD_CFG1,
+ 0x03, width[bus_width]);
+
+ return err;
+}
+
+static int sd_pull_ctl_disable_lqfp48(struct rtsx_ucr *ucr)
+{
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL1, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL2, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL3, 0xFF, 0x95);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL4, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL5, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL6, 0xFF, 0xA5);
+
+ return rtsx_usb_send_cmd(ucr, MODE_C, 100);
+}
+
+static int sd_pull_ctl_disable_qfn24(struct rtsx_ucr *ucr)
+{
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL1, 0xFF, 0x65);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL2, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL3, 0xFF, 0x95);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL4, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL5, 0xFF, 0x56);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL6, 0xFF, 0x59);
+
+ return rtsx_usb_send_cmd(ucr, MODE_C, 100);
+}
+
+static int sd_pull_ctl_enable_lqfp48(struct rtsx_ucr *ucr)
+{
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL1, 0xFF, 0xAA);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL2, 0xFF, 0xAA);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL3, 0xFF, 0xA9);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL4, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL5, 0xFF, 0x55);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL6, 0xFF, 0xA5);
+
+ return rtsx_usb_send_cmd(ucr, MODE_C, 100);
+}
+
+static int sd_pull_ctl_enable_qfn24(struct rtsx_ucr *ucr)
+{
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL1, 0xFF, 0xA5);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL2, 0xFF, 0x9A);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL3, 0xFF, 0xA5);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL4, 0xFF, 0x9A);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL5, 0xFF, 0x65);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PULL_CTL6, 0xFF, 0x5A);
+
+ return rtsx_usb_send_cmd(ucr, MODE_C, 100);
+}
+
+static int sd_power_on(struct rtsx_usb_sdmmc *host)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+
+ dev_dbg(sdmmc_dev(host), "%s\n", __func__);
+ rtsx_usb_init_cmd(ucr);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_SELECT, 0x07, SD_MOD_SEL);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_SHARE_MODE,
+ CARD_SHARE_MASK, CARD_SHARE_SD);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_CLK_EN,
+ SD_CLK_EN, SD_CLK_EN);
+ err = rtsx_usb_send_cmd(ucr, MODE_C, 100);
+ if (err)
+ return err;
+
+ if (CHECK_PKG(ucr, LQFP48))
+ err = sd_pull_ctl_enable_lqfp48(ucr);
+ else
+ err = sd_pull_ctl_enable_qfn24(ucr);
+ if (err)
+ return err;
+
+ err = rtsx_usb_write_register(ucr, CARD_PWR_CTL,
+ POWER_MASK, PARTIAL_POWER_ON);
+ if (err)
+ return err;
+
+ usleep_range(800, 1000);
+
+ rtsx_usb_init_cmd(ucr);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PWR_CTL,
+ POWER_MASK|LDO3318_PWR_MASK, POWER_ON|LDO_ON);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_OE,
+ SD_OUTPUT_EN, SD_OUTPUT_EN);
+
+ return rtsx_usb_send_cmd(ucr, MODE_C, 100);
+}
+
+static int sd_power_off(struct rtsx_usb_sdmmc *host)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+
+ dev_dbg(sdmmc_dev(host), "%s\n", __func__);
+ rtsx_usb_init_cmd(ucr);
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_CLK_EN, SD_CLK_EN, 0);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_OE, SD_OUTPUT_EN, 0);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PWR_CTL,
+ POWER_MASK, POWER_OFF);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_PWR_CTL,
+ POWER_MASK|LDO3318_PWR_MASK, POWER_OFF|LDO_SUSPEND);
+
+ err = rtsx_usb_send_cmd(ucr, MODE_C, 100);
+ if (err)
+ return err;
+
+ if (CHECK_PKG(ucr, LQFP48))
+ return sd_pull_ctl_disable_lqfp48(ucr);
+ return sd_pull_ctl_disable_qfn24(ucr);
+}
+
+static int sd_set_power_mode(struct rtsx_usb_sdmmc *host,
+ unsigned char power_mode)
+{
+ int err;
+
+ if (power_mode != MMC_POWER_OFF)
+ power_mode = MMC_POWER_ON;
+
+ if (power_mode == host->power_mode)
+ return 0;
+
+ if (power_mode == MMC_POWER_OFF) {
+ err = sd_power_off(host);
+ pm_runtime_put(sdmmc_dev(host));
+ } else {
+ pm_runtime_get_sync(sdmmc_dev(host));
+ err = sd_power_on(host);
+ }
+
+ if (!err)
+ host->power_mode = power_mode;
+
+ return err;
+}
+
+static int sd_set_timing(struct rtsx_usb_sdmmc *host,
+ unsigned char timing, bool *ddr_mode)
+{
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+
+ *ddr_mode = false;
+
+ rtsx_usb_init_cmd(ucr);
+
+ switch (timing) {
+ case MMC_TIMING_UHS_SDR104:
+ case MMC_TIMING_UHS_SDR50:
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CFG1,
+ 0x0C | SD_ASYNC_FIFO_RST,
+ SD_30_MODE | SD_ASYNC_FIFO_RST);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_CLK_SOURCE, 0xFF,
+ CRC_VAR_CLK0 | SD30_FIX_CLK | SAMPLE_VAR_CLK1);
+ break;
+
+ case MMC_TIMING_UHS_DDR50:
+ *ddr_mode = true;
+
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CFG1,
+ 0x0C | SD_ASYNC_FIFO_RST,
+ SD_DDR_MODE | SD_ASYNC_FIFO_RST);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_CLK_SOURCE, 0xFF,
+ CRC_VAR_CLK0 | SD30_FIX_CLK | SAMPLE_VAR_CLK1);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_PUSH_POINT_CTL,
+ DDR_VAR_TX_CMD_DAT, DDR_VAR_TX_CMD_DAT);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_SAMPLE_POINT_CTL,
+ DDR_VAR_RX_DAT | DDR_VAR_RX_CMD,
+ DDR_VAR_RX_DAT | DDR_VAR_RX_CMD);
+ break;
+
+ case MMC_TIMING_MMC_HS:
+ case MMC_TIMING_SD_HS:
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_CFG1,
+ 0x0C, SD_20_MODE);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_CLK_SOURCE, 0xFF,
+ CRC_FIX_CLK | SD30_VAR_CLK0 | SAMPLE_VAR_CLK1);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_PUSH_POINT_CTL,
+ SD20_TX_SEL_MASK, SD20_TX_14_AHEAD);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_SAMPLE_POINT_CTL,
+ SD20_RX_SEL_MASK, SD20_RX_14_DELAY);
+ break;
+
+ default:
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_CFG1, 0x0C, SD_20_MODE);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, CARD_CLK_SOURCE, 0xFF,
+ CRC_FIX_CLK | SD30_VAR_CLK0 | SAMPLE_VAR_CLK1);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD,
+ SD_PUSH_POINT_CTL, 0xFF, 0);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_SAMPLE_POINT_CTL,
+ SD20_RX_SEL_MASK, SD20_RX_POS_EDGE);
+ break;
+ }
+
+ err = rtsx_usb_send_cmd(ucr, MODE_C, 100);
+
+ return err;
+}
+
+static void sdmmc_set_ios(struct mmc_host *mmc, struct mmc_ios *ios)
+{
+ struct rtsx_usb_sdmmc *host = mmc_priv(mmc);
+ struct rtsx_ucr *ucr = host->ucr;
+
+ dev_dbg(sdmmc_dev(host), "%s\n", __func__);
+ mutex_lock(&ucr->dev_mutex);
+
+ if (rtsx_usb_card_exclusive_check(ucr, RTSX_USB_SD_CARD)) {
+ mutex_unlock(&ucr->dev_mutex);
+ return;
+ }
+
+ sd_set_power_mode(host, ios->power_mode);
+ sd_set_bus_width(host, ios->bus_width);
+ sd_set_timing(host, ios->timing, &host->ddr_mode);
+
+ host->vpclk = false;
+ host->double_clk = true;
+
+ switch (ios->timing) {
+ case MMC_TIMING_UHS_SDR104:
+ case MMC_TIMING_UHS_SDR50:
+ host->ssc_depth = SSC_DEPTH_2M;
+ host->vpclk = true;
+ host->double_clk = false;
+ break;
+ case MMC_TIMING_UHS_DDR50:
+ case MMC_TIMING_UHS_SDR25:
+ host->ssc_depth = SSC_DEPTH_1M;
+ break;
+ default:
+ host->ssc_depth = SSC_DEPTH_512K;
+ break;
+ }
+
+ host->initial_mode = (ios->clock <= 1000000) ? true : false;
+ host->clock = ios->clock;
+
+ rtsx_usb_switch_clock(host->ucr, host->clock, host->ssc_depth,
+ host->initial_mode, host->double_clk, host->vpclk);
+
+ mutex_unlock(&ucr->dev_mutex);
+ dev_dbg(sdmmc_dev(host), "%s end\n", __func__);
+}
+
+static int sdmmc_switch_voltage(struct mmc_host *mmc, struct mmc_ios *ios)
+{
+ struct rtsx_usb_sdmmc *host = mmc_priv(mmc);
+ struct rtsx_ucr *ucr = host->ucr;
+ int err = 0;
+
+ dev_dbg(sdmmc_dev(host), "%s: signal_voltage = %d\n",
+ __func__, ios->signal_voltage);
+
+ if (host->host_removal)
+ return -ENOMEDIUM;
+
+ if (ios->signal_voltage == MMC_SIGNAL_VOLTAGE_120)
+ return -EPERM;
+
+ mutex_lock(&ucr->dev_mutex);
+
+ err = rtsx_usb_card_exclusive_check(ucr, RTSX_USB_SD_CARD);
+ if (err) {
+ mutex_unlock(&ucr->dev_mutex);
+ return err;
+ }
+
+ /* Let mmc core do the busy checking, simply stop the forced-toggle
+ * clock(while issuing CMD11) and switch voltage.
+ */
+ rtsx_usb_init_cmd(ucr);
+
+ if (ios->signal_voltage == MMC_SIGNAL_VOLTAGE_330) {
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_PAD_CTL,
+ SD_IO_USING_1V8, SD_IO_USING_3V3);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, LDO_POWER_CFG,
+ TUNE_SD18_MASK, TUNE_SD18_3V3);
+ } else {
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_BUS_STAT,
+ SD_CLK_TOGGLE_EN | SD_CLK_FORCE_STOP,
+ SD_CLK_FORCE_STOP);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, SD_PAD_CTL,
+ SD_IO_USING_1V8, SD_IO_USING_1V8);
+ rtsx_usb_add_cmd(ucr, WRITE_REG_CMD, LDO_POWER_CFG,
+ TUNE_SD18_MASK, TUNE_SD18_1V8);
+ }
+
+ err = rtsx_usb_send_cmd(ucr, MODE_C, 100);
+ mutex_unlock(&ucr->dev_mutex);
+
+ return err;
+}
+
+static int sdmmc_card_busy(struct mmc_host *mmc)
+{
+ struct rtsx_usb_sdmmc *host = mmc_priv(mmc);
+ struct rtsx_ucr *ucr = host->ucr;
+ int err;
+ u8 stat;
+ u8 mask = SD_DAT3_STATUS | SD_DAT2_STATUS | SD_DAT1_STATUS
+ | SD_DAT0_STATUS;
+
+ dev_dbg(sdmmc_dev(host), "%s\n", __func__);
+
+ mutex_lock(&ucr->dev_mutex);
+
+ err = rtsx_usb_write_register(ucr, SD_BUS_STAT,
+ SD_CLK_TOGGLE_EN | SD_CLK_FORCE_STOP,
+ SD_CLK_TOGGLE_EN);
+ if (err)
+ goto out;
+
+ mdelay(1);
+
+ err = rtsx_usb_read_register(ucr, SD_BUS_STAT, &stat);
+ if (err)
+ goto out;
+
+ err = rtsx_usb_write_register(ucr, SD_BUS_STAT,
+ SD_CLK_TOGGLE_EN | SD_CLK_FORCE_STOP, 0);
+out:
+ mutex_unlock(&ucr->dev_mutex);
+
+ if (err)
+ return err;
+
+ /* check if any pin between dat[0:3] is low */
+ if ((stat & mask) != mask)
+ return 1;
+ else
+ return 0;
+}
+
+static int sdmmc_execute_tuning(struct mmc_host *mmc, u32 opcode)
+{
+ struct rtsx_usb_sdmmc *host = mmc_priv(mmc);
+ struct rtsx_ucr *ucr = host->ucr;
+ int err = 0;
+
+ if (host->host_removal)
+ return -ENOMEDIUM;
+
+ mutex_lock(&ucr->dev_mutex);
+
+ if (!host->ddr_mode)
+ err = sd_tuning_rx(host, MMC_SEND_TUNING_BLOCK);
+
+ mutex_unlock(&ucr->dev_mutex);
+
+ return err;
+}
+
+static const struct mmc_host_ops rtsx_usb_sdmmc_ops = {
+ .request = sdmmc_request,
+ .set_ios = sdmmc_set_ios,
+ .get_ro = sdmmc_get_ro,
+ .get_cd = sdmmc_get_cd,
+ .start_signal_voltage_switch = sdmmc_switch_voltage,
+ .card_busy = sdmmc_card_busy,
+ .execute_tuning = sdmmc_execute_tuning,
+};
+
+#ifdef RTSX_USB_USE_LEDS_CLASS
+static void rtsx_usb_led_control(struct led_classdev *led,
+ enum led_brightness brightness)
+{
+ struct rtsx_usb_sdmmc *host = container_of(led,
+ struct rtsx_usb_sdmmc, led);
+
+ if (host->host_removal)
+ return;
+
+ host->led.brightness = brightness;
+ schedule_work(&host->led_work);
+}
+
+static void rtsx_usb_update_led(struct work_struct *work)
+{
+ struct rtsx_usb_sdmmc *host =
+ container_of(work, struct rtsx_usb_sdmmc, led_work);
+ struct rtsx_ucr *ucr = host->ucr;
+
+ mutex_lock(&ucr->dev_mutex);
+
+ if (host->led.brightness == LED_OFF)
+ rtsx_usb_turn_off_led(ucr);
+ else
+ rtsx_usb_turn_on_led(ucr);
+
+ mutex_unlock(&ucr->dev_mutex);
+}
+#endif
+
+static void rtsx_usb_init_host(struct rtsx_usb_sdmmc *host)
+{
+ struct mmc_host *mmc = host->mmc;
+
+ mmc->f_min = 250000;
+ mmc->f_max = 208000000;
+ mmc->ocr_avail = MMC_VDD_32_33 | MMC_VDD_33_34 | MMC_VDD_165_195;
+ mmc->caps = MMC_CAP_4_BIT_DATA | MMC_CAP_SD_HIGHSPEED |
+ MMC_CAP_MMC_HIGHSPEED | MMC_CAP_BUS_WIDTH_TEST |
+ MMC_CAP_UHS_SDR12 | MMC_CAP_UHS_SDR25 | MMC_CAP_UHS_SDR50 |
+ MMC_CAP_NEEDS_POLL;
+
+ mmc->max_current_330 = 400;
+ mmc->max_current_180 = 800;
+ mmc->ops = &rtsx_usb_sdmmc_ops;
+ mmc->max_segs = 256;
+ mmc->max_seg_size = 65536;
+ mmc->max_blk_size = 512;
+ mmc->max_blk_count = 65535;
+ mmc->max_req_size = 524288;
+
+ host->power_mode = MMC_POWER_OFF;
+}
+
+static int rtsx_usb_sdmmc_drv_probe(struct platform_device *pdev)
+{
+ struct mmc_host *mmc;
+ struct rtsx_usb_sdmmc *host;
+ struct rtsx_ucr *ucr;
+#ifdef RTSX_USB_USE_LEDS_CLASS
+ int err;
+#endif
+
+ ucr = usb_get_intfdata(to_usb_interface(pdev->dev.parent));
+ if (!ucr)
+ return -ENXIO;
+
+ dev_dbg(&(pdev->dev), ": Realtek USB SD/MMC controller found\n");
+
+ mmc = mmc_alloc_host(sizeof(*host), &pdev->dev);
+ if (!mmc)
+ return -ENOMEM;
+
+ host = mmc_priv(mmc);
+ host->ucr = ucr;
+ host->mmc = mmc;
+ host->pdev = pdev;
+ platform_set_drvdata(pdev, host);
+
+ mutex_init(&host->host_mutex);
+ rtsx_usb_init_host(host);
+ pm_runtime_enable(&pdev->dev);
+
+#ifdef RTSX_USB_USE_LEDS_CLASS
+ snprintf(host->led_name, sizeof(host->led_name),
+ "%s::", mmc_hostname(mmc));
+ host->led.name = host->led_name;
+ host->led.brightness = LED_OFF;
+ host->led.default_trigger = mmc_hostname(mmc);
+ host->led.brightness_set = rtsx_usb_led_control;
+
+ err = led_classdev_register(mmc_dev(mmc), &host->led);
+ if (err)
+ dev_err(&(pdev->dev),
+ "Failed to register LED device: %d\n", err);
+ INIT_WORK(&host->led_work, rtsx_usb_update_led);
+
+#endif
+ mmc_add_host(mmc);
+
+ return 0;
+}
+
+static int rtsx_usb_sdmmc_drv_remove(struct platform_device *pdev)
+{
+ struct rtsx_usb_sdmmc *host = platform_get_drvdata(pdev);
+ struct mmc_host *mmc;
+
+ if (!host)
+ return 0;
+
+ mmc = host->mmc;
+ host->host_removal = true;
+
+ mutex_lock(&host->host_mutex);
+ if (host->mrq) {
+ dev_dbg(&(pdev->dev),
+ "%s: Controller removed during transfer\n",
+ mmc_hostname(mmc));
+ host->mrq->cmd->error = -ENOMEDIUM;
+ if (host->mrq->stop)
+ host->mrq->stop->error = -ENOMEDIUM;
+ mmc_request_done(mmc, host->mrq);
+ }
+ mutex_unlock(&host->host_mutex);
+
+ mmc_remove_host(mmc);
+
+#ifdef RTSX_USB_USE_LEDS_CLASS
+ cancel_work_sync(&host->led_work);
+ led_classdev_unregister(&host->led);
+#endif
+
+ mmc_free_host(mmc);
+ pm_runtime_disable(&pdev->dev);
+ platform_set_drvdata(pdev, NULL);
+
+ dev_dbg(&(pdev->dev),
+ ": Realtek USB SD/MMC module has been removed\n");
+
+ return 0;
+}
+
+static struct platform_device_id rtsx_usb_sdmmc_ids[] = {
+ {
+ .name = "rtsx_usb_sdmmc",
+ }, {
+ /* sentinel */
+ }
+};
+MODULE_DEVICE_TABLE(platform, rtsx_usb_sdmmc_ids);
+
+static struct platform_driver rtsx_usb_sdmmc_driver = {
+ .probe = rtsx_usb_sdmmc_drv_probe,
+ .remove = rtsx_usb_sdmmc_drv_remove,
+ .id_table = rtsx_usb_sdmmc_ids,
+ .driver = {
+ .owner = THIS_MODULE,
+ .name = "rtsx_usb_sdmmc",
+ },
+};
+module_platform_driver(rtsx_usb_sdmmc_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Roger Tseng <rogerable@realtek.com>");
+MODULE_DESCRIPTION("Realtek USB SD/MMC Card Host Driver");
diff --git a/drivers/mmc/host/sdhci-acpi.c b/drivers/mmc/host/sdhci-acpi.c
index ebb3f39..8ce3c28 100644
--- a/drivers/mmc/host/sdhci-acpi.c
+++ b/drivers/mmc/host/sdhci-acpi.c
@@ -102,11 +102,19 @@
}
static const struct sdhci_ops sdhci_acpi_ops_dflt = {
+ .set_clock = sdhci_set_clock,
.enable_dma = sdhci_acpi_enable_dma,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
static const struct sdhci_ops sdhci_acpi_ops_int = {
+ .set_clock = sdhci_set_clock,
.enable_dma = sdhci_acpi_enable_dma,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
.hw_reset = sdhci_acpi_int_hw_reset,
};
diff --git a/drivers/mmc/host/sdhci-bcm-kona.c b/drivers/mmc/host/sdhci-bcm-kona.c
index 6f166e6..dd780c3 100644
--- a/drivers/mmc/host/sdhci-bcm-kona.c
+++ b/drivers/mmc/host/sdhci-bcm-kona.c
@@ -206,9 +206,13 @@
}
static struct sdhci_ops sdhci_bcm_kona_ops = {
+ .set_clock = sdhci_set_clock,
.get_max_clock = sdhci_bcm_kona_get_max_clk,
.get_timeout_clock = sdhci_bcm_kona_get_timeout_clock,
.platform_send_init_74_clocks = sdhci_bcm_kona_init_74_clocks,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
.card_event = sdhci_bcm_kona_card_event,
};
diff --git a/drivers/mmc/host/sdhci-bcm2835.c b/drivers/mmc/host/sdhci-bcm2835.c
index f6d8d67..46af9a4 100644
--- a/drivers/mmc/host/sdhci-bcm2835.c
+++ b/drivers/mmc/host/sdhci-bcm2835.c
@@ -131,8 +131,12 @@
.read_l = bcm2835_sdhci_readl,
.read_w = bcm2835_sdhci_readw,
.read_b = bcm2835_sdhci_readb,
+ .set_clock = sdhci_set_clock,
.get_max_clock = sdhci_pltfm_clk_get_max_clock,
.get_min_clock = bcm2835_sdhci_get_min_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
static const struct sdhci_pltfm_data bcm2835_sdhci_pdata = {
diff --git a/drivers/mmc/host/sdhci-cns3xxx.c b/drivers/mmc/host/sdhci-cns3xxx.c
index f2cc266..14b7407 100644
--- a/drivers/mmc/host/sdhci-cns3xxx.c
+++ b/drivers/mmc/host/sdhci-cns3xxx.c
@@ -30,13 +30,12 @@
u16 clk;
unsigned long timeout;
- if (clock == host->clock)
- return;
+ host->mmc->actual_clock = 0;
sdhci_writew(host, 0, SDHCI_CLOCK_CONTROL);
if (clock == 0)
- goto out;
+ return;
while (host->max_clk / div > clock) {
/*
@@ -75,13 +74,14 @@
clk |= SDHCI_CLOCK_CARD_EN;
sdhci_writew(host, clk, SDHCI_CLOCK_CONTROL);
-out:
- host->clock = clock;
}
static const struct sdhci_ops sdhci_cns3xxx_ops = {
.get_max_clock = sdhci_cns3xxx_get_max_clk,
.set_clock = sdhci_cns3xxx_set_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
static const struct sdhci_pltfm_data sdhci_cns3xxx_pdata = {
@@ -90,8 +90,7 @@
SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK |
SDHCI_QUIRK_INVERTED_WRITE_PROTECT |
SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN |
- SDHCI_QUIRK_BROKEN_TIMEOUT_VAL |
- SDHCI_QUIRK_NONSTANDARD_CLOCK,
+ SDHCI_QUIRK_BROKEN_TIMEOUT_VAL,
};
static int sdhci_cns3xxx_probe(struct platform_device *pdev)
diff --git a/drivers/mmc/host/sdhci-dove.c b/drivers/mmc/host/sdhci-dove.c
index 736d7a2..e6278ec 100644
--- a/drivers/mmc/host/sdhci-dove.c
+++ b/drivers/mmc/host/sdhci-dove.c
@@ -21,28 +21,17 @@
#include <linux/clk.h>
#include <linux/err.h>
-#include <linux/gpio.h>
#include <linux/io.h>
#include <linux/mmc/host.h>
#include <linux/module.h>
#include <linux/of.h>
-#include <linux/of_gpio.h>
#include "sdhci-pltfm.h"
struct sdhci_dove_priv {
struct clk *clk;
- int gpio_cd;
};
-static irqreturn_t sdhci_dove_carddetect_irq(int irq, void *data)
-{
- struct sdhci_host *host = data;
-
- tasklet_schedule(&host->card_tasklet);
- return IRQ_HANDLED;
-}
-
static u16 sdhci_dove_readw(struct sdhci_host *host, int reg)
{
u16 ret;
@@ -60,8 +49,6 @@
static u32 sdhci_dove_readl(struct sdhci_host *host, int reg)
{
- struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
- struct sdhci_dove_priv *priv = pltfm_host->priv;
u32 ret;
ret = readl(host->ioaddr + reg);
@@ -71,14 +58,6 @@
/* Mask the support for 3.0V */
ret &= ~SDHCI_CAN_VDD_300;
break;
- case SDHCI_PRESENT_STATE:
- if (gpio_is_valid(priv->gpio_cd)) {
- if (gpio_get_value(priv->gpio_cd) == 0)
- ret |= SDHCI_CARD_PRESENT;
- else
- ret &= ~SDHCI_CARD_PRESENT;
- }
- break;
}
return ret;
}
@@ -86,6 +65,10 @@
static const struct sdhci_ops sdhci_dove_ops = {
.read_w = sdhci_dove_readw,
.read_l = sdhci_dove_readl,
+ .set_clock = sdhci_set_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
static const struct sdhci_pltfm_data sdhci_dove_pdata = {
@@ -113,28 +96,9 @@
priv->clk = devm_clk_get(&pdev->dev, NULL);
- if (pdev->dev.of_node) {
- priv->gpio_cd = of_get_named_gpio(pdev->dev.of_node,
- "cd-gpios", 0);
- } else {
- priv->gpio_cd = -EINVAL;
- }
-
- if (gpio_is_valid(priv->gpio_cd)) {
- ret = gpio_request(priv->gpio_cd, "sdhci-cd");
- if (ret) {
- dev_err(&pdev->dev, "card detect gpio request failed: %d\n",
- ret);
- return ret;
- }
- gpio_direction_input(priv->gpio_cd);
- }
-
host = sdhci_pltfm_init(pdev, &sdhci_dove_pdata, 0);
- if (IS_ERR(host)) {
- ret = PTR_ERR(host);
- goto err_sdhci_pltfm_init;
- }
+ if (IS_ERR(host))
+ return PTR_ERR(host);
pltfm_host = sdhci_priv(host);
pltfm_host->priv = priv;
@@ -142,39 +106,20 @@
if (!IS_ERR(priv->clk))
clk_prepare_enable(priv->clk);
- sdhci_get_of_property(pdev);
+ ret = mmc_of_parse(host->mmc);
+ if (ret)
+ goto err_sdhci_add;
ret = sdhci_add_host(host);
if (ret)
goto err_sdhci_add;
- /*
- * We must request the IRQ after sdhci_add_host(), as the tasklet only
- * gets setup in sdhci_add_host() and we oops.
- */
- if (gpio_is_valid(priv->gpio_cd)) {
- ret = request_irq(gpio_to_irq(priv->gpio_cd),
- sdhci_dove_carddetect_irq,
- IRQF_TRIGGER_FALLING | IRQF_TRIGGER_RISING,
- mmc_hostname(host->mmc), host);
- if (ret) {
- dev_err(&pdev->dev, "card detect irq request failed: %d\n",
- ret);
- goto err_request_irq;
- }
- }
-
return 0;
-err_request_irq:
- sdhci_remove_host(host, 0);
err_sdhci_add:
if (!IS_ERR(priv->clk))
clk_disable_unprepare(priv->clk);
sdhci_pltfm_free(pdev);
-err_sdhci_pltfm_init:
- if (gpio_is_valid(priv->gpio_cd))
- gpio_free(priv->gpio_cd);
return ret;
}
@@ -186,11 +131,6 @@
sdhci_pltfm_unregister(pdev);
- if (gpio_is_valid(priv->gpio_cd)) {
- free_irq(gpio_to_irq(priv->gpio_cd), host);
- gpio_free(priv->gpio_cd);
- }
-
if (!IS_ERR(priv->clk))
clk_disable_unprepare(priv->clk);
diff --git a/drivers/mmc/host/sdhci-esdhc-imx.c b/drivers/mmc/host/sdhci-esdhc-imx.c
index b841bb7..ccec0e3 100644
--- a/drivers/mmc/host/sdhci-esdhc-imx.c
+++ b/drivers/mmc/host/sdhci-esdhc-imx.c
@@ -160,7 +160,6 @@
MULTIBLK_IN_PROCESS, /* exact multiblock cmd in process */
WAIT_FOR_INT, /* sent CMD12, waiting for response INT */
} multiblock_status;
- u32 uhs_mode;
u32 is_ddr;
};
@@ -382,7 +381,6 @@
if (val & ESDHC_MIX_CTRL_SMPCLK_SEL)
ret |= SDHCI_CTRL_TUNED_CLK;
- ret |= (imx_data->uhs_mode & SDHCI_CTRL_UHS_MASK);
ret &= ~SDHCI_CTRL_PRESET_VAL_ENABLE;
return ret;
@@ -429,7 +427,6 @@
else
new_val &= ~ESDHC_VENDOR_SPEC_VSELECT;
writel(new_val, host->ioaddr + ESDHC_VENDOR_SPEC);
- imx_data->uhs_mode = val & SDHCI_CTRL_UHS_MASK;
if (imx_data->socdata->flags & ESDHC_FLAG_MAN_TUNING) {
new_val = readl(host->ioaddr + ESDHC_MIX_CTRL);
if (val & SDHCI_CTRL_TUNED_CLK)
@@ -600,12 +597,14 @@
u32 temp, val;
if (clock == 0) {
+ host->mmc->actual_clock = 0;
+
if (esdhc_is_usdhc(imx_data)) {
val = readl(host->ioaddr + ESDHC_VENDOR_SPEC);
writel(val & ~ESDHC_VENDOR_SPEC_FRC_SDCLK_ON,
host->ioaddr + ESDHC_VENDOR_SPEC);
}
- goto out;
+ return;
}
if (esdhc_is_usdhc(imx_data) && !imx_data->is_ddr)
@@ -645,8 +644,6 @@
}
mdelay(1);
-out:
- host->clock = clock;
}
static unsigned int esdhc_pltfm_get_ro(struct sdhci_host *host)
@@ -668,7 +665,7 @@
return -ENOSYS;
}
-static int esdhc_pltfm_bus_width(struct sdhci_host *host, int width)
+static void esdhc_pltfm_set_bus_width(struct sdhci_host *host, int width)
{
u32 ctrl;
@@ -686,8 +683,6 @@
esdhc_clrset_le(host, ESDHC_CTRL_BUSWIDTH_MASK, ctrl,
SDHCI_HOST_CONTROL);
-
- return 0;
}
static void esdhc_prepare_tuning(struct sdhci_host *host, u32 val)
@@ -697,6 +692,7 @@
/* FIXME: delay a bit for card to be ready for next tuning due to errors */
mdelay(1);
+ /* This is balanced by the runtime put in sdhci_tasklet_finish */
pm_runtime_get_sync(host->mmc->parent);
reg = readl(host->ioaddr + ESDHC_MIX_CTRL);
reg |= ESDHC_MIX_CTRL_EXE_TUNE | ESDHC_MIX_CTRL_SMPCLK_SEL |
@@ -713,13 +709,12 @@
complete(&mrq->completion);
}
-static int esdhc_send_tuning_cmd(struct sdhci_host *host, u32 opcode)
+static int esdhc_send_tuning_cmd(struct sdhci_host *host, u32 opcode,
+ struct scatterlist *sg)
{
struct mmc_command cmd = {0};
struct mmc_request mrq = {NULL};
struct mmc_data data = {0};
- struct scatterlist sg;
- char tuning_pattern[ESDHC_TUNING_BLOCK_PATTERN_LEN];
cmd.opcode = opcode;
cmd.arg = 0;
@@ -728,11 +723,9 @@
data.blksz = ESDHC_TUNING_BLOCK_PATTERN_LEN;
data.blocks = 1;
data.flags = MMC_DATA_READ;
- data.sg = &sg;
+ data.sg = sg;
data.sg_len = 1;
- sg_init_one(&sg, tuning_pattern, sizeof(tuning_pattern));
-
mrq.cmd = &cmd;
mrq.cmd->mrq = &mrq;
mrq.data = &data;
@@ -742,14 +735,12 @@
mrq.done = esdhc_request_done;
init_completion(&(mrq.completion));
- disable_irq(host->irq);
- spin_lock(&host->lock);
+ spin_lock_irq(&host->lock);
host->mrq = &mrq;
sdhci_send_command(host, mrq.cmd);
- spin_unlock(&host->lock);
- enable_irq(host->irq);
+ spin_unlock_irq(&host->lock);
wait_for_completion(&mrq.completion);
@@ -772,13 +763,21 @@
static int esdhc_executing_tuning(struct sdhci_host *host, u32 opcode)
{
+ struct scatterlist sg;
+ char *tuning_pattern;
int min, max, avg, ret;
+ tuning_pattern = kmalloc(ESDHC_TUNING_BLOCK_PATTERN_LEN, GFP_KERNEL);
+ if (!tuning_pattern)
+ return -ENOMEM;
+
+ sg_init_one(&sg, tuning_pattern, ESDHC_TUNING_BLOCK_PATTERN_LEN);
+
/* find the mininum delay first which can pass tuning */
min = ESDHC_TUNE_CTRL_MIN;
while (min < ESDHC_TUNE_CTRL_MAX) {
esdhc_prepare_tuning(host, min);
- if (!esdhc_send_tuning_cmd(host, opcode))
+ if (!esdhc_send_tuning_cmd(host, opcode, &sg))
break;
min += ESDHC_TUNE_CTRL_STEP;
}
@@ -787,7 +786,7 @@
max = min + ESDHC_TUNE_CTRL_STEP;
while (max < ESDHC_TUNE_CTRL_MAX) {
esdhc_prepare_tuning(host, max);
- if (esdhc_send_tuning_cmd(host, opcode)) {
+ if (esdhc_send_tuning_cmd(host, opcode, &sg)) {
max -= ESDHC_TUNE_CTRL_STEP;
break;
}
@@ -797,9 +796,11 @@
/* use average delay to get the best timing */
avg = (min + max) / 2;
esdhc_prepare_tuning(host, avg);
- ret = esdhc_send_tuning_cmd(host, opcode);
+ ret = esdhc_send_tuning_cmd(host, opcode, &sg);
esdhc_post_tuning(host);
+ kfree(tuning_pattern);
+
dev_dbg(mmc_dev(host->mmc), "tunning %s at 0x%x ret %d\n",
ret ? "failed" : "passed", avg, ret);
@@ -837,28 +838,21 @@
return pinctrl_select_state(imx_data->pinctrl, pinctrl);
}
-static int esdhc_set_uhs_signaling(struct sdhci_host *host, unsigned int uhs)
+static void esdhc_set_uhs_signaling(struct sdhci_host *host, unsigned timing)
{
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
struct pltfm_imx_data *imx_data = pltfm_host->priv;
struct esdhc_platform_data *boarddata = &imx_data->boarddata;
- switch (uhs) {
+ switch (timing) {
case MMC_TIMING_UHS_SDR12:
- imx_data->uhs_mode = SDHCI_CTRL_UHS_SDR12;
- break;
case MMC_TIMING_UHS_SDR25:
- imx_data->uhs_mode = SDHCI_CTRL_UHS_SDR25;
- break;
case MMC_TIMING_UHS_SDR50:
- imx_data->uhs_mode = SDHCI_CTRL_UHS_SDR50;
- break;
case MMC_TIMING_UHS_SDR104:
case MMC_TIMING_MMC_HS200:
- imx_data->uhs_mode = SDHCI_CTRL_UHS_SDR104;
break;
case MMC_TIMING_UHS_DDR50:
- imx_data->uhs_mode = SDHCI_CTRL_UHS_DDR50;
+ case MMC_TIMING_MMC_DDR52:
writel(readl(host->ioaddr + ESDHC_MIX_CTRL) |
ESDHC_MIX_CTRL_DDREN,
host->ioaddr + ESDHC_MIX_CTRL);
@@ -875,7 +869,15 @@
break;
}
- return esdhc_change_pinstate(host, uhs);
+ esdhc_change_pinstate(host, timing);
+}
+
+static void esdhc_reset(struct sdhci_host *host, u8 mask)
+{
+ sdhci_reset(host, mask);
+
+ sdhci_writel(host, host->ier, SDHCI_INT_ENABLE);
+ sdhci_writel(host, host->ier, SDHCI_SIGNAL_ENABLE);
}
static struct sdhci_ops sdhci_esdhc_ops = {
@@ -888,8 +890,9 @@
.get_max_clock = esdhc_pltfm_get_max_clock,
.get_min_clock = esdhc_pltfm_get_min_clock,
.get_ro = esdhc_pltfm_get_ro,
- .platform_bus_width = esdhc_pltfm_bus_width,
+ .set_bus_width = esdhc_pltfm_set_bus_width,
.set_uhs_signaling = esdhc_set_uhs_signaling,
+ .reset = esdhc_reset,
};
static const struct sdhci_pltfm_data sdhci_esdhc_imx_pdata = {
@@ -1170,8 +1173,10 @@
ret = sdhci_runtime_suspend_host(host);
- clk_disable_unprepare(imx_data->clk_per);
- clk_disable_unprepare(imx_data->clk_ipg);
+ if (!sdhci_sdio_irq_enabled(host)) {
+ clk_disable_unprepare(imx_data->clk_per);
+ clk_disable_unprepare(imx_data->clk_ipg);
+ }
clk_disable_unprepare(imx_data->clk_ahb);
return ret;
@@ -1183,8 +1188,10 @@
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
struct pltfm_imx_data *imx_data = pltfm_host->priv;
- clk_prepare_enable(imx_data->clk_per);
- clk_prepare_enable(imx_data->clk_ipg);
+ if (!sdhci_sdio_irq_enabled(host)) {
+ clk_prepare_enable(imx_data->clk_per);
+ clk_prepare_enable(imx_data->clk_ipg);
+ }
clk_prepare_enable(imx_data->clk_ahb);
return sdhci_runtime_resume_host(host);
diff --git a/drivers/mmc/host/sdhci-esdhc.h b/drivers/mmc/host/sdhci-esdhc.h
index a7d9f95..3497cfa 100644
--- a/drivers/mmc/host/sdhci-esdhc.h
+++ b/drivers/mmc/host/sdhci-esdhc.h
@@ -20,10 +20,8 @@
#define ESDHC_DEFAULT_QUIRKS (SDHCI_QUIRK_FORCE_BLK_SZ_2048 | \
SDHCI_QUIRK_NO_BUSY_IRQ | \
- SDHCI_QUIRK_NONSTANDARD_CLOCK | \
SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK | \
- SDHCI_QUIRK_PIO_NEEDS_DELAY | \
- SDHCI_QUIRK_RESTORE_IRQS_AFTER_RESET)
+ SDHCI_QUIRK_PIO_NEEDS_DELAY)
#define ESDHC_SYSTEM_CONTROL 0x2c
#define ESDHC_CLOCK_MASK 0x0000fff0
diff --git a/drivers/mmc/host/sdhci-of-arasan.c b/drivers/mmc/host/sdhci-of-arasan.c
index f7c7cf6..5bd1092 100644
--- a/drivers/mmc/host/sdhci-of-arasan.c
+++ b/drivers/mmc/host/sdhci-of-arasan.c
@@ -52,8 +52,12 @@
}
static struct sdhci_ops sdhci_arasan_ops = {
+ .set_clock = sdhci_set_clock,
.get_max_clock = sdhci_pltfm_clk_get_max_clock,
.get_timeout_clock = sdhci_arasan_get_timeout_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
static struct sdhci_pltfm_data sdhci_arasan_pdata = {
diff --git a/drivers/mmc/host/sdhci-of-esdhc.c b/drivers/mmc/host/sdhci-of-esdhc.c
index 0b24997..8be4dcf 100644
--- a/drivers/mmc/host/sdhci-of-esdhc.c
+++ b/drivers/mmc/host/sdhci-of-esdhc.c
@@ -199,13 +199,14 @@
static void esdhc_of_set_clock(struct sdhci_host *host, unsigned int clock)
{
-
int pre_div = 2;
int div = 1;
u32 temp;
+ host->mmc->actual_clock = 0;
+
if (clock == 0)
- goto out;
+ return;
/* Workaround to reduce the clock frequency for p1010 esdhc */
if (of_find_compatible_node(NULL, NULL, "fsl,p1010-esdhc")) {
@@ -238,24 +239,8 @@
| (pre_div << ESDHC_PREDIV_SHIFT));
sdhci_writel(host, temp, ESDHC_SYSTEM_CONTROL);
mdelay(1);
-out:
- host->clock = clock;
}
-#ifdef CONFIG_PM
-static u32 esdhc_proctl;
-static void esdhc_of_suspend(struct sdhci_host *host)
-{
- esdhc_proctl = sdhci_be32bs_readl(host, SDHCI_HOST_CONTROL);
-}
-
-static void esdhc_of_resume(struct sdhci_host *host)
-{
- esdhc_of_enable_dma(host);
- sdhci_be32bs_writel(host, esdhc_proctl, SDHCI_HOST_CONTROL);
-}
-#endif
-
static void esdhc_of_platform_init(struct sdhci_host *host)
{
u32 vvn;
@@ -269,7 +254,7 @@
host->quirks &= ~SDHCI_QUIRK_NO_BUSY_IRQ;
}
-static int esdhc_pltfm_bus_width(struct sdhci_host *host, int width)
+static void esdhc_pltfm_set_bus_width(struct sdhci_host *host, int width)
{
u32 ctrl;
@@ -289,8 +274,6 @@
clrsetbits_be32(host->ioaddr + SDHCI_HOST_CONTROL,
ESDHC_CTRL_BUSWIDTH_MASK, ctrl);
-
- return 0;
}
static const struct sdhci_ops sdhci_esdhc_ops = {
@@ -305,14 +288,47 @@
.get_max_clock = esdhc_of_get_max_clock,
.get_min_clock = esdhc_of_get_min_clock,
.platform_init = esdhc_of_platform_init,
-#ifdef CONFIG_PM
- .platform_suspend = esdhc_of_suspend,
- .platform_resume = esdhc_of_resume,
-#endif
.adma_workaround = esdhci_of_adma_workaround,
- .platform_bus_width = esdhc_pltfm_bus_width,
+ .set_bus_width = esdhc_pltfm_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
+#ifdef CONFIG_PM
+
+static u32 esdhc_proctl;
+static int esdhc_of_suspend(struct device *dev)
+{
+ struct sdhci_host *host = dev_get_drvdata(dev);
+
+ esdhc_proctl = sdhci_be32bs_readl(host, SDHCI_HOST_CONTROL);
+
+ return sdhci_suspend_host(host);
+}
+
+static int esdhc_of_resume(struct device *dev)
+{
+ struct sdhci_host *host = dev_get_drvdata(dev);
+ int ret = sdhci_resume_host(host);
+
+ if (ret == 0) {
+ /* Isn't this already done by sdhci_resume_host() ? --rmk */
+ esdhc_of_enable_dma(host);
+ sdhci_be32bs_writel(host, esdhc_proctl, SDHCI_HOST_CONTROL);
+ }
+
+ return ret;
+}
+
+static const struct dev_pm_ops esdhc_pmops = {
+ .suspend = esdhc_of_suspend,
+ .resume = esdhc_of_resume,
+};
+#define ESDHC_PMOPS (&esdhc_pmops)
+#else
+#define ESDHC_PMOPS NULL
+#endif
+
static const struct sdhci_pltfm_data sdhci_esdhc_pdata = {
/*
* card detection could be handled via GPIO
@@ -374,7 +390,7 @@
.name = "sdhci-esdhc",
.owner = THIS_MODULE,
.of_match_table = sdhci_esdhc_of_match,
- .pm = SDHCI_PLTFM_PMOPS,
+ .pm = ESDHC_PMOPS,
},
.probe = sdhci_esdhc_probe,
.remove = sdhci_esdhc_remove,
diff --git a/drivers/mmc/host/sdhci-of-hlwd.c b/drivers/mmc/host/sdhci-of-hlwd.c
index 57c514a..b341661 100644
--- a/drivers/mmc/host/sdhci-of-hlwd.c
+++ b/drivers/mmc/host/sdhci-of-hlwd.c
@@ -58,6 +58,10 @@
.write_l = sdhci_hlwd_writel,
.write_w = sdhci_hlwd_writew,
.write_b = sdhci_hlwd_writeb,
+ .set_clock = sdhci_set_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
static const struct sdhci_pltfm_data sdhci_hlwd_pdata = {
diff --git a/drivers/mmc/host/sdhci-pci-o2micro.c b/drivers/mmc/host/sdhci-pci-o2micro.c
index f49666b..5670e38 100644
--- a/drivers/mmc/host/sdhci-pci-o2micro.c
+++ b/drivers/mmc/host/sdhci-pci-o2micro.c
@@ -21,6 +21,45 @@
#include "sdhci-pci.h"
#include "sdhci-pci-o2micro.h"
+static void o2_pci_set_baseclk(struct sdhci_pci_chip *chip, u32 value)
+{
+ u32 scratch_32;
+ pci_read_config_dword(chip->pdev,
+ O2_SD_PLL_SETTING, &scratch_32);
+
+ scratch_32 &= 0x0000FFFF;
+ scratch_32 |= value;
+
+ pci_write_config_dword(chip->pdev,
+ O2_SD_PLL_SETTING, scratch_32);
+}
+
+static void o2_pci_led_enable(struct sdhci_pci_chip *chip)
+{
+ int ret;
+ u32 scratch_32;
+
+ /* Set led of SD host function enable */
+ ret = pci_read_config_dword(chip->pdev,
+ O2_SD_FUNC_REG0, &scratch_32);
+ if (ret)
+ return;
+
+ scratch_32 &= ~O2_SD_FREG0_LEDOFF;
+ pci_write_config_dword(chip->pdev,
+ O2_SD_FUNC_REG0, scratch_32);
+
+ ret = pci_read_config_dword(chip->pdev,
+ O2_SD_TEST_REG, &scratch_32);
+ if (ret)
+ return;
+
+ scratch_32 |= O2_SD_LED_ENABLE;
+ pci_write_config_dword(chip->pdev,
+ O2_SD_TEST_REG, scratch_32);
+
+}
+
void sdhci_pci_o2_fujin2_pci_init(struct sdhci_pci_chip *chip)
{
u32 scratch_32;
@@ -216,6 +255,40 @@
scratch &= 0x7f;
pci_write_config_byte(chip->pdev, O2_SD_LOCK_WP, scratch);
+ /* DevId=8520 subId= 0x11 or 0x12 Type Chip support */
+ if (chip->pdev->device == PCI_DEVICE_ID_O2_FUJIN2) {
+ ret = pci_read_config_dword(chip->pdev,
+ O2_SD_FUNC_REG0,
+ &scratch_32);
+ scratch_32 = ((scratch_32 & 0xFF000000) >> 24);
+
+ /* Check Whether subId is 0x11 or 0x12 */
+ if ((scratch_32 == 0x11) || (scratch_32 == 0x12)) {
+ scratch_32 = 0x2c280000;
+
+ /* Set Base Clock to 208MZ */
+ o2_pci_set_baseclk(chip, scratch_32);
+ ret = pci_read_config_dword(chip->pdev,
+ O2_SD_FUNC_REG4,
+ &scratch_32);
+
+ /* Enable Base Clk setting change */
+ scratch_32 |= O2_SD_FREG4_ENABLE_CLK_SET;
+ pci_write_config_dword(chip->pdev,
+ O2_SD_FUNC_REG4,
+ scratch_32);
+
+ /* Set Tuning Window to 4 */
+ pci_write_config_byte(chip->pdev,
+ O2_SD_TUNING_CTRL, 0x44);
+
+ break;
+ }
+ }
+
+ /* Enable 8520 led function */
+ o2_pci_led_enable(chip);
+
/* Set timeout CLK */
ret = pci_read_config_dword(chip->pdev,
O2_SD_CLK_SETTING, &scratch_32);
@@ -276,7 +349,7 @@
pci_write_config_byte(chip->pdev, O2_SD_LOCK_WP, scratch);
ret = pci_read_config_dword(chip->pdev,
- O2_SD_FUNC_REG0, &scratch_32);
+ O2_SD_PLL_SETTING, &scratch_32);
if ((scratch_32 & 0xff000000) == 0x01000000) {
scratch_32 &= 0x0000FFFF;
@@ -299,6 +372,9 @@
O2_SD_FUNC_REG4, scratch_32);
}
+ /* Set Tuning Windows to 5 */
+ pci_write_config_byte(chip->pdev,
+ O2_SD_TUNING_CTRL, 0x55);
/* Lock WP */
ret = pci_read_config_byte(chip->pdev,
O2_SD_LOCK_WP, &scratch);
diff --git a/drivers/mmc/host/sdhci-pci-o2micro.h b/drivers/mmc/host/sdhci-pci-o2micro.h
index dbec4c93..f7ffc90 100644
--- a/drivers/mmc/host/sdhci-pci-o2micro.h
+++ b/drivers/mmc/host/sdhci-pci-o2micro.h
@@ -57,6 +57,9 @@
#define O2_SD_UHS2_L1_CTRL 0x35C
#define O2_SD_FUNC_REG3 0x3E0
#define O2_SD_FUNC_REG4 0x3E4
+#define O2_SD_LED_ENABLE BIT(6)
+#define O2_SD_FREG0_LEDOFF BIT(13)
+#define O2_SD_FREG4_ENABLE_CLK_SET BIT(22)
#define O2_SD_VENDOR_SETTING 0x110
#define O2_SD_VENDOR_SETTING2 0x1C8
diff --git a/drivers/mmc/host/sdhci-pci.c b/drivers/mmc/host/sdhci-pci.c
index fdc6121..52c42fc 100644
--- a/drivers/mmc/host/sdhci-pci.c
+++ b/drivers/mmc/host/sdhci-pci.c
@@ -1031,7 +1031,7 @@
return 0;
}
-static int sdhci_pci_bus_width(struct sdhci_host *host, int width)
+static void sdhci_pci_set_bus_width(struct sdhci_host *host, int width)
{
u8 ctrl;
@@ -1052,8 +1052,6 @@
}
sdhci_writeb(host, ctrl, SDHCI_HOST_CONTROL);
-
- return 0;
}
static void sdhci_pci_gpio_hw_reset(struct sdhci_host *host)
@@ -1080,8 +1078,11 @@
}
static const struct sdhci_ops sdhci_pci_ops = {
+ .set_clock = sdhci_set_clock,
.enable_dma = sdhci_pci_enable_dma,
- .platform_bus_width = sdhci_pci_bus_width,
+ .set_bus_width = sdhci_pci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
.hw_reset = sdhci_pci_hw_reset,
};
diff --git a/drivers/mmc/host/sdhci-pltfm.c b/drivers/mmc/host/sdhci-pltfm.c
index bef250e..7e834fb 100644
--- a/drivers/mmc/host/sdhci-pltfm.c
+++ b/drivers/mmc/host/sdhci-pltfm.c
@@ -45,6 +45,10 @@
EXPORT_SYMBOL_GPL(sdhci_pltfm_clk_get_max_clock);
static const struct sdhci_ops sdhci_pltfm_ops = {
+ .set_clock = sdhci_set_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
#ifdef CONFIG_OF
diff --git a/drivers/mmc/host/sdhci-pxav2.c b/drivers/mmc/host/sdhci-pxav2.c
index d51e061..3c0f3c0 100644
--- a/drivers/mmc/host/sdhci-pxav2.c
+++ b/drivers/mmc/host/sdhci-pxav2.c
@@ -51,11 +51,13 @@
#define MMC_CARD 0x1000
#define MMC_WIDTH 0x0100
-static void pxav2_set_private_registers(struct sdhci_host *host, u8 mask)
+static void pxav2_reset(struct sdhci_host *host, u8 mask)
{
struct platform_device *pdev = to_platform_device(mmc_dev(host->mmc));
struct sdhci_pxa_platdata *pdata = pdev->dev.platform_data;
+ sdhci_reset(host, mask);
+
if (mask == SDHCI_RESET_ALL) {
u16 tmp = 0;
@@ -88,7 +90,7 @@
}
}
-static int pxav2_mmc_set_width(struct sdhci_host *host, int width)
+static void pxav2_mmc_set_bus_width(struct sdhci_host *host, int width)
{
u8 ctrl;
u16 tmp;
@@ -107,14 +109,14 @@
}
writew(tmp, host->ioaddr + SD_CE_ATA_2);
writeb(ctrl, host->ioaddr + SDHCI_HOST_CONTROL);
-
- return 0;
}
static const struct sdhci_ops pxav2_sdhci_ops = {
+ .set_clock = sdhci_set_clock,
.get_max_clock = sdhci_pltfm_clk_get_max_clock,
- .platform_reset_exit = pxav2_set_private_registers,
- .platform_bus_width = pxav2_mmc_set_width,
+ .set_bus_width = pxav2_mmc_set_bus_width,
+ .reset = pxav2_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
#ifdef CONFIG_OF
diff --git a/drivers/mmc/host/sdhci-pxav3.c b/drivers/mmc/host/sdhci-pxav3.c
index 2fd73b3..f4f1289 100644
--- a/drivers/mmc/host/sdhci-pxav3.c
+++ b/drivers/mmc/host/sdhci-pxav3.c
@@ -112,11 +112,13 @@
return 0;
}
-static void pxav3_set_private_registers(struct sdhci_host *host, u8 mask)
+static void pxav3_reset(struct sdhci_host *host, u8 mask)
{
struct platform_device *pdev = to_platform_device(mmc_dev(host->mmc));
struct sdhci_pxa_platdata *pdata = pdev->dev.platform_data;
+ sdhci_reset(host, mask);
+
if (mask == SDHCI_RESET_ALL) {
/*
* tune timing of read data/command when crc error happen
@@ -184,7 +186,7 @@
pxa->power_mode = power_mode;
}
-static int pxav3_set_uhs_signaling(struct sdhci_host *host, unsigned int uhs)
+static void pxav3_set_uhs_signaling(struct sdhci_host *host, unsigned int uhs)
{
u16 ctrl_2;
@@ -218,15 +220,16 @@
dev_dbg(mmc_dev(host->mmc),
"%s uhs = %d, ctrl_2 = %04X\n",
__func__, uhs, ctrl_2);
-
- return 0;
}
static const struct sdhci_ops pxav3_sdhci_ops = {
- .platform_reset_exit = pxav3_set_private_registers,
+ .set_clock = sdhci_set_clock,
.set_uhs_signaling = pxav3_set_uhs_signaling,
.platform_send_init_74_clocks = pxav3_gen_init_74_clocks,
.get_max_clock = sdhci_pltfm_clk_get_max_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = pxav3_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
static struct sdhci_pltfm_data sdhci_pxav3_pdata = {
diff --git a/drivers/mmc/host/sdhci-s3c.c b/drivers/mmc/host/sdhci-s3c.c
index d61eb5a..fa5954a 100644
--- a/drivers/mmc/host/sdhci-s3c.c
+++ b/drivers/mmc/host/sdhci-s3c.c
@@ -33,9 +33,6 @@
#define MAX_BUS_CLK (4)
-/* Number of gpio's used is max data bus width + command and clock lines */
-#define NUM_GPIOS(x) (x + 2)
-
/**
* struct sdhci_s3c - S3C SDHCI instance
* @host: The SDHCI host created
@@ -58,6 +55,8 @@
struct clk *clk_io;
struct clk *clk_bus[MAX_BUS_CLK];
unsigned long clk_rates[MAX_BUS_CLK];
+
+ bool no_divider;
};
/**
@@ -70,6 +69,7 @@
*/
struct sdhci_s3c_drv_data {
unsigned int sdhci_quirks;
+ bool no_divider;
};
static inline struct sdhci_s3c *to_s3c(struct sdhci_host *host)
@@ -119,7 +119,7 @@
* If controller uses a non-standard clock division, find the best clock
* speed possible with selected clock source and skip the division.
*/
- if (ourhost->host->quirks & SDHCI_QUIRK_NONSTANDARD_CLOCK) {
+ if (ourhost->no_divider) {
rate = clk_round_rate(clksrc, wanted);
return wanted - rate;
}
@@ -161,9 +161,13 @@
int src;
u32 ctrl;
+ host->mmc->actual_clock = 0;
+
/* don't bother if the clock is going off. */
- if (clock == 0)
+ if (clock == 0) {
+ sdhci_set_clock(host, clock);
return;
+ }
for (src = 0; src < MAX_BUS_CLK; src++) {
delta = sdhci_s3c_consider_clock(ourhost, src, clock);
@@ -215,6 +219,8 @@
if (clock < 25 * 1000000)
ctrl |= (S3C_SDHCI_CTRL3_FCSEL3 | S3C_SDHCI_CTRL3_FCSEL2);
writel(ctrl, host->ioaddr + S3C_SDHCI_CONTROL3);
+
+ sdhci_set_clock(host, clock);
}
/**
@@ -295,10 +301,11 @@
unsigned long timeout;
u16 clk = 0;
+ host->mmc->actual_clock = 0;
+
/* If the clock is going off, set to 0 at clock control register */
if (clock == 0) {
sdhci_writew(host, 0, SDHCI_CLOCK_CONTROL);
- host->clock = clock;
return;
}
@@ -306,8 +313,6 @@
clk_set_rate(ourhost->clk_bus[ourhost->cur_clk], clock);
- host->clock = clock;
-
clk = SDHCI_CLOCK_INT_EN;
sdhci_writew(host, clk, SDHCI_CLOCK_CONTROL);
@@ -329,14 +334,14 @@
}
/**
- * sdhci_s3c_platform_bus_width - support 8bit buswidth
+ * sdhci_s3c_set_bus_width - support 8bit buswidth
* @host: The SDHCI host being queried
* @width: MMC_BUS_WIDTH_ macro for the bus width being requested
*
* We have 8-bit width support but is not a v3 controller.
* So we add platform_bus_width() and support 8bit width.
*/
-static int sdhci_s3c_platform_bus_width(struct sdhci_host *host, int width)
+static void sdhci_s3c_set_bus_width(struct sdhci_host *host, int width)
{
u8 ctrl;
@@ -358,93 +363,23 @@
}
sdhci_writeb(host, ctrl, SDHCI_HOST_CONTROL);
-
- return 0;
}
static struct sdhci_ops sdhci_s3c_ops = {
.get_max_clock = sdhci_s3c_get_max_clk,
.set_clock = sdhci_s3c_set_clock,
.get_min_clock = sdhci_s3c_get_min_clock,
- .platform_bus_width = sdhci_s3c_platform_bus_width,
+ .set_bus_width = sdhci_s3c_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
-static void sdhci_s3c_notify_change(struct platform_device *dev, int state)
-{
- struct sdhci_host *host = platform_get_drvdata(dev);
-#ifdef CONFIG_PM_RUNTIME
- struct sdhci_s3c *sc = sdhci_priv(host);
-#endif
- unsigned long flags;
-
- if (host) {
- spin_lock_irqsave(&host->lock, flags);
- if (state) {
- dev_dbg(&dev->dev, "card inserted.\n");
-#ifdef CONFIG_PM_RUNTIME
- clk_prepare_enable(sc->clk_io);
-#endif
- host->flags &= ~SDHCI_DEVICE_DEAD;
- host->quirks |= SDHCI_QUIRK_BROKEN_CARD_DETECTION;
- } else {
- dev_dbg(&dev->dev, "card removed.\n");
- host->flags |= SDHCI_DEVICE_DEAD;
- host->quirks &= ~SDHCI_QUIRK_BROKEN_CARD_DETECTION;
-#ifdef CONFIG_PM_RUNTIME
- clk_disable_unprepare(sc->clk_io);
-#endif
- }
- tasklet_schedule(&host->card_tasklet);
- spin_unlock_irqrestore(&host->lock, flags);
- }
-}
-
-static irqreturn_t sdhci_s3c_gpio_card_detect_thread(int irq, void *dev_id)
-{
- struct sdhci_s3c *sc = dev_id;
- int status = gpio_get_value(sc->ext_cd_gpio);
- if (sc->pdata->ext_cd_gpio_invert)
- status = !status;
- sdhci_s3c_notify_change(sc->pdev, status);
- return IRQ_HANDLED;
-}
-
-static void sdhci_s3c_setup_card_detect_gpio(struct sdhci_s3c *sc)
-{
- struct s3c_sdhci_platdata *pdata = sc->pdata;
- struct device *dev = &sc->pdev->dev;
-
- if (devm_gpio_request(dev, pdata->ext_cd_gpio, "SDHCI EXT CD") == 0) {
- sc->ext_cd_gpio = pdata->ext_cd_gpio;
- sc->ext_cd_irq = gpio_to_irq(pdata->ext_cd_gpio);
- if (sc->ext_cd_irq &&
- request_threaded_irq(sc->ext_cd_irq, NULL,
- sdhci_s3c_gpio_card_detect_thread,
- IRQF_TRIGGER_RISING |
- IRQF_TRIGGER_FALLING |
- IRQF_ONESHOT,
- dev_name(dev), sc) == 0) {
- int status = gpio_get_value(sc->ext_cd_gpio);
- if (pdata->ext_cd_gpio_invert)
- status = !status;
- sdhci_s3c_notify_change(sc->pdev, status);
- } else {
- dev_warn(dev, "cannot request irq for card detect\n");
- sc->ext_cd_irq = 0;
- }
- } else {
- dev_err(dev, "cannot request gpio for card detect\n");
- }
-}
-
#ifdef CONFIG_OF
static int sdhci_s3c_parse_dt(struct device *dev,
struct sdhci_host *host, struct s3c_sdhci_platdata *pdata)
{
struct device_node *node = dev->of_node;
- struct sdhci_s3c *ourhost = to_s3c(host);
u32 max_width;
- int gpio;
/* if the bus-width property is not specified, assume width as 1 */
if (of_property_read_u32(node, "bus-width", &max_width))
@@ -462,18 +397,8 @@
return 0;
}
- gpio = of_get_named_gpio(node, "cd-gpios", 0);
- if (gpio_is_valid(gpio)) {
- pdata->cd_type = S3C_SDHCI_CD_GPIO;
- pdata->ext_cd_gpio = gpio;
- ourhost->ext_cd_gpio = -1;
- if (of_get_property(node, "cd-inverted", NULL))
- pdata->ext_cd_gpio_invert = 1;
+ if (of_get_named_gpio(node, "cd-gpios", 0))
return 0;
- } else if (gpio != -ENOENT) {
- dev_err(dev, "invalid card detect gpio specified\n");
- return -EINVAL;
- }
/* assuming internal card detect that will be configured by pinctrl */
pdata->cd_type = S3C_SDHCI_CD_INTERNAL;
@@ -606,8 +531,10 @@
/* Setup quirks for the controller */
host->quirks |= SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC;
host->quirks |= SDHCI_QUIRK_NO_HISPD_BIT;
- if (drv_data)
+ if (drv_data) {
host->quirks |= drv_data->sdhci_quirks;
+ sc->no_divider = drv_data->no_divider;
+ }
#ifndef CONFIG_MMC_SDHCI_S3C_DMA
@@ -656,7 +583,7 @@
* If controller does not have internal clock divider,
* we can use overriding functions instead of default.
*/
- if (host->quirks & SDHCI_QUIRK_NONSTANDARD_CLOCK) {
+ if (sc->no_divider) {
sdhci_s3c_ops.set_clock = sdhci_cmu_set_clock;
sdhci_s3c_ops.get_min_clock = sdhci_cmu_get_min_clock;
sdhci_s3c_ops.get_max_clock = sdhci_cmu_get_max_clock;
@@ -674,6 +601,8 @@
pm_runtime_use_autosuspend(&pdev->dev);
pm_suspend_ignore_children(&pdev->dev, 1);
+ mmc_of_parse(host->mmc);
+
ret = sdhci_add_host(host);
if (ret) {
dev_err(dev, "sdhci_add_host() failed\n");
@@ -682,15 +611,6 @@
goto err_req_regs;
}
- /* The following two methods of card detection might call
- sdhci_s3c_notify_change() immediately, so they can be called
- only after sdhci_add_host(). Setup errors are ignored. */
- if (pdata->cd_type == S3C_SDHCI_CD_EXTERNAL && pdata->ext_cd_init)
- pdata->ext_cd_init(&sdhci_s3c_notify_change);
- if (pdata->cd_type == S3C_SDHCI_CD_GPIO &&
- gpio_is_valid(pdata->ext_cd_gpio))
- sdhci_s3c_setup_card_detect_gpio(sc);
-
#ifdef CONFIG_PM_RUNTIME
if (pdata->cd_type != S3C_SDHCI_CD_INTERNAL)
clk_disable_unprepare(sc->clk_io);
@@ -711,16 +631,12 @@
{
struct sdhci_host *host = platform_get_drvdata(pdev);
struct sdhci_s3c *sc = sdhci_priv(host);
- struct s3c_sdhci_platdata *pdata = sc->pdata;
-
- if (pdata->cd_type == S3C_SDHCI_CD_EXTERNAL && pdata->ext_cd_cleanup)
- pdata->ext_cd_cleanup(&sdhci_s3c_notify_change);
if (sc->ext_cd_irq)
free_irq(sc->ext_cd_irq, sc);
#ifdef CONFIG_PM_RUNTIME
- if (pdata->cd_type != S3C_SDHCI_CD_INTERNAL)
+ if (sc->pdata->cd_type != S3C_SDHCI_CD_INTERNAL)
clk_prepare_enable(sc->clk_io);
#endif
sdhci_remove_host(host, 1);
@@ -797,7 +713,7 @@
#if defined(CONFIG_CPU_EXYNOS4210) || defined(CONFIG_SOC_EXYNOS4212)
static struct sdhci_s3c_drv_data exynos4_sdhci_drv_data = {
- .sdhci_quirks = SDHCI_QUIRK_NONSTANDARD_CLOCK,
+ .no_divider = true,
};
#define EXYNOS4_SDHCI_DRV_DATA ((kernel_ulong_t)&exynos4_sdhci_drv_data)
#else
diff --git a/drivers/mmc/host/sdhci-sirf.c b/drivers/mmc/host/sdhci-sirf.c
index 696122c..1700453 100644
--- a/drivers/mmc/host/sdhci-sirf.c
+++ b/drivers/mmc/host/sdhci-sirf.c
@@ -28,7 +28,11 @@
}
static struct sdhci_ops sdhci_sirf_ops = {
+ .set_clock = sdhci_set_clock,
.get_max_clock = sdhci_sirf_get_max_clk,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
static struct sdhci_pltfm_data sdhci_sirf_pdata = {
diff --git a/drivers/mmc/host/sdhci-spear.c b/drivers/mmc/host/sdhci-spear.c
index 0316dec..9d535c7 100644
--- a/drivers/mmc/host/sdhci-spear.c
+++ b/drivers/mmc/host/sdhci-spear.c
@@ -38,7 +38,10 @@
/* sdhci ops */
static const struct sdhci_ops sdhci_pltfm_ops = {
- /* Nothing to do for now. */
+ .set_clock = sdhci_set_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
};
#ifdef CONFIG_OF
diff --git a/drivers/mmc/host/sdhci-tegra.c b/drivers/mmc/host/sdhci-tegra.c
index a835898..d93a063 100644
--- a/drivers/mmc/host/sdhci-tegra.c
+++ b/drivers/mmc/host/sdhci-tegra.c
@@ -32,11 +32,17 @@
/* Tegra SDHOST controller vendor register definitions */
#define SDHCI_TEGRA_VENDOR_MISC_CTRL 0x120
+#define SDHCI_MISC_CTRL_ENABLE_SDR104 0x8
+#define SDHCI_MISC_CTRL_ENABLE_SDR50 0x10
#define SDHCI_MISC_CTRL_ENABLE_SDHCI_SPEC_300 0x20
+#define SDHCI_MISC_CTRL_ENABLE_DDR50 0x200
#define NVQUIRK_FORCE_SDHCI_SPEC_200 BIT(0)
#define NVQUIRK_ENABLE_BLOCK_GAP_DET BIT(1)
#define NVQUIRK_ENABLE_SDHCI_SPEC_300 BIT(2)
+#define NVQUIRK_DISABLE_SDR50 BIT(3)
+#define NVQUIRK_DISABLE_SDR104 BIT(4)
+#define NVQUIRK_DISABLE_DDR50 BIT(5)
struct sdhci_tegra_soc_data {
const struct sdhci_pltfm_data *pdata;
@@ -48,19 +54,6 @@
int power_gpio;
};
-static u32 tegra_sdhci_readl(struct sdhci_host *host, int reg)
-{
- u32 val;
-
- if (unlikely(reg == SDHCI_PRESENT_STATE)) {
- /* Use wp_gpio here instead? */
- val = readl(host->ioaddr + reg);
- return val | SDHCI_WRITE_PROTECT;
- }
-
- return readl(host->ioaddr + reg);
-}
-
static u16 tegra_sdhci_readw(struct sdhci_host *host, int reg)
{
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
@@ -108,26 +101,33 @@
return mmc_gpio_get_ro(host->mmc);
}
-static void tegra_sdhci_reset_exit(struct sdhci_host *host, u8 mask)
+static void tegra_sdhci_reset(struct sdhci_host *host, u8 mask)
{
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
struct sdhci_tegra *tegra_host = pltfm_host->priv;
const struct sdhci_tegra_soc_data *soc_data = tegra_host->soc_data;
+ u32 misc_ctrl;
+
+ sdhci_reset(host, mask);
if (!(mask & SDHCI_RESET_ALL))
return;
+ misc_ctrl = sdhci_readw(host, SDHCI_TEGRA_VENDOR_MISC_CTRL);
/* Erratum: Enable SDHCI spec v3.00 support */
- if (soc_data->nvquirks & NVQUIRK_ENABLE_SDHCI_SPEC_300) {
- u32 misc_ctrl;
-
- misc_ctrl = sdhci_readb(host, SDHCI_TEGRA_VENDOR_MISC_CTRL);
+ if (soc_data->nvquirks & NVQUIRK_ENABLE_SDHCI_SPEC_300)
misc_ctrl |= SDHCI_MISC_CTRL_ENABLE_SDHCI_SPEC_300;
- sdhci_writeb(host, misc_ctrl, SDHCI_TEGRA_VENDOR_MISC_CTRL);
- }
+ /* Don't advertise UHS modes which aren't supported yet */
+ if (soc_data->nvquirks & NVQUIRK_DISABLE_SDR50)
+ misc_ctrl &= ~SDHCI_MISC_CTRL_ENABLE_SDR50;
+ if (soc_data->nvquirks & NVQUIRK_DISABLE_DDR50)
+ misc_ctrl &= ~SDHCI_MISC_CTRL_ENABLE_DDR50;
+ if (soc_data->nvquirks & NVQUIRK_DISABLE_SDR104)
+ misc_ctrl &= ~SDHCI_MISC_CTRL_ENABLE_SDR104;
+ sdhci_writew(host, misc_ctrl, SDHCI_TEGRA_VENDOR_MISC_CTRL);
}
-static int tegra_sdhci_buswidth(struct sdhci_host *host, int bus_width)
+static void tegra_sdhci_set_bus_width(struct sdhci_host *host, int bus_width)
{
u32 ctrl;
@@ -144,23 +144,25 @@
ctrl &= ~SDHCI_CTRL_4BITBUS;
}
sdhci_writeb(host, ctrl, SDHCI_HOST_CONTROL);
- return 0;
}
static const struct sdhci_ops tegra_sdhci_ops = {
.get_ro = tegra_sdhci_get_ro,
- .read_l = tegra_sdhci_readl,
.read_w = tegra_sdhci_readw,
.write_l = tegra_sdhci_writel,
- .platform_bus_width = tegra_sdhci_buswidth,
- .platform_reset_exit = tegra_sdhci_reset_exit,
+ .set_clock = sdhci_set_clock,
+ .set_bus_width = tegra_sdhci_set_bus_width,
+ .reset = tegra_sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
+ .get_max_clock = sdhci_pltfm_clk_get_max_clock,
};
static const struct sdhci_pltfm_data sdhci_tegra20_pdata = {
.quirks = SDHCI_QUIRK_BROKEN_TIMEOUT_VAL |
SDHCI_QUIRK_SINGLE_POWER_WRITE |
SDHCI_QUIRK_NO_HISPD_BIT |
- SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC,
+ SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC |
+ SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN,
.ops = &tegra_sdhci_ops,
};
@@ -175,13 +177,16 @@
SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK |
SDHCI_QUIRK_SINGLE_POWER_WRITE |
SDHCI_QUIRK_NO_HISPD_BIT |
- SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC,
+ SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC |
+ SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN,
.ops = &tegra_sdhci_ops,
};
static struct sdhci_tegra_soc_data soc_data_tegra30 = {
.pdata = &sdhci_tegra30_pdata,
- .nvquirks = NVQUIRK_ENABLE_SDHCI_SPEC_300,
+ .nvquirks = NVQUIRK_ENABLE_SDHCI_SPEC_300 |
+ NVQUIRK_DISABLE_SDR50 |
+ NVQUIRK_DISABLE_SDR104,
};
static const struct sdhci_pltfm_data sdhci_tegra114_pdata = {
@@ -189,12 +194,16 @@
SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK |
SDHCI_QUIRK_SINGLE_POWER_WRITE |
SDHCI_QUIRK_NO_HISPD_BIT |
- SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC,
+ SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC |
+ SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN,
.ops = &tegra_sdhci_ops,
};
static struct sdhci_tegra_soc_data soc_data_tegra114 = {
.pdata = &sdhci_tegra114_pdata,
+ .nvquirks = NVQUIRK_DISABLE_SDR50 |
+ NVQUIRK_DISABLE_DDR50 |
+ NVQUIRK_DISABLE_SDR104,
};
static const struct of_device_id sdhci_tegra_dt_match[] = {
diff --git a/drivers/mmc/host/sdhci.c b/drivers/mmc/host/sdhci.c
index 9a79fc4..47055f3 100644
--- a/drivers/mmc/host/sdhci.c
+++ b/drivers/mmc/host/sdhci.c
@@ -44,6 +44,8 @@
#define MAX_TUNING_LOOP 40
+#define ADMA_SIZE ((128 * 2 + 1) * 4)
+
static unsigned int debug_quirks = 0;
static unsigned int debug_quirks2;
@@ -131,43 +133,26 @@
* *
\*****************************************************************************/
-static void sdhci_clear_set_irqs(struct sdhci_host *host, u32 clear, u32 set)
-{
- u32 ier;
-
- ier = sdhci_readl(host, SDHCI_INT_ENABLE);
- ier &= ~clear;
- ier |= set;
- sdhci_writel(host, ier, SDHCI_INT_ENABLE);
- sdhci_writel(host, ier, SDHCI_SIGNAL_ENABLE);
-}
-
-static void sdhci_unmask_irqs(struct sdhci_host *host, u32 irqs)
-{
- sdhci_clear_set_irqs(host, 0, irqs);
-}
-
-static void sdhci_mask_irqs(struct sdhci_host *host, u32 irqs)
-{
- sdhci_clear_set_irqs(host, irqs, 0);
-}
-
static void sdhci_set_card_detection(struct sdhci_host *host, bool enable)
{
- u32 present, irqs;
+ u32 present;
if ((host->quirks & SDHCI_QUIRK_BROKEN_CARD_DETECTION) ||
(host->mmc->caps & MMC_CAP_NONREMOVABLE))
return;
- present = sdhci_readl(host, SDHCI_PRESENT_STATE) &
- SDHCI_CARD_PRESENT;
- irqs = present ? SDHCI_INT_CARD_REMOVE : SDHCI_INT_CARD_INSERT;
+ if (enable) {
+ present = sdhci_readl(host, SDHCI_PRESENT_STATE) &
+ SDHCI_CARD_PRESENT;
- if (enable)
- sdhci_unmask_irqs(host, irqs);
- else
- sdhci_mask_irqs(host, irqs);
+ host->ier |= present ? SDHCI_INT_CARD_REMOVE :
+ SDHCI_INT_CARD_INSERT;
+ } else {
+ host->ier &= ~(SDHCI_INT_CARD_REMOVE | SDHCI_INT_CARD_INSERT);
+ }
+
+ sdhci_writel(host, host->ier, SDHCI_INT_ENABLE);
+ sdhci_writel(host, host->ier, SDHCI_SIGNAL_ENABLE);
}
static void sdhci_enable_card_detection(struct sdhci_host *host)
@@ -180,22 +165,9 @@
sdhci_set_card_detection(host, false);
}
-static void sdhci_reset(struct sdhci_host *host, u8 mask)
+void sdhci_reset(struct sdhci_host *host, u8 mask)
{
unsigned long timeout;
- u32 uninitialized_var(ier);
-
- if (host->quirks & SDHCI_QUIRK_NO_CARD_NO_RESET) {
- if (!(sdhci_readl(host, SDHCI_PRESENT_STATE) &
- SDHCI_CARD_PRESENT))
- return;
- }
-
- if (host->quirks & SDHCI_QUIRK_RESTORE_IRQS_AFTER_RESET)
- ier = sdhci_readl(host, SDHCI_INT_ENABLE);
-
- if (host->ops->platform_reset_enter)
- host->ops->platform_reset_enter(host, mask);
sdhci_writeb(host, mask, SDHCI_SOFTWARE_RESET);
@@ -220,16 +192,27 @@
timeout--;
mdelay(1);
}
+}
+EXPORT_SYMBOL_GPL(sdhci_reset);
- if (host->ops->platform_reset_exit)
- host->ops->platform_reset_exit(host, mask);
+static void sdhci_do_reset(struct sdhci_host *host, u8 mask)
+{
+ if (host->quirks & SDHCI_QUIRK_NO_CARD_NO_RESET) {
+ if (!(sdhci_readl(host, SDHCI_PRESENT_STATE) &
+ SDHCI_CARD_PRESENT))
+ return;
+ }
- if (host->quirks & SDHCI_QUIRK_RESTORE_IRQS_AFTER_RESET)
- sdhci_clear_set_irqs(host, SDHCI_INT_ALL_MASK, ier);
+ host->ops->reset(host, mask);
- if (host->flags & (SDHCI_USE_SDMA | SDHCI_USE_ADMA)) {
- if ((host->ops->enable_dma) && (mask & SDHCI_RESET_ALL))
- host->ops->enable_dma(host);
+ if (mask & SDHCI_RESET_ALL) {
+ if (host->flags & (SDHCI_USE_SDMA | SDHCI_USE_ADMA)) {
+ if (host->ops->enable_dma)
+ host->ops->enable_dma(host);
+ }
+
+ /* Resetting the controller clears many */
+ host->preset_enabled = false;
}
}
@@ -238,15 +221,18 @@
static void sdhci_init(struct sdhci_host *host, int soft)
{
if (soft)
- sdhci_reset(host, SDHCI_RESET_CMD|SDHCI_RESET_DATA);
+ sdhci_do_reset(host, SDHCI_RESET_CMD|SDHCI_RESET_DATA);
else
- sdhci_reset(host, SDHCI_RESET_ALL);
+ sdhci_do_reset(host, SDHCI_RESET_ALL);
- sdhci_clear_set_irqs(host, SDHCI_INT_ALL_MASK,
- SDHCI_INT_BUS_POWER | SDHCI_INT_DATA_END_BIT |
- SDHCI_INT_DATA_CRC | SDHCI_INT_DATA_TIMEOUT | SDHCI_INT_INDEX |
- SDHCI_INT_END_BIT | SDHCI_INT_CRC | SDHCI_INT_TIMEOUT |
- SDHCI_INT_DATA_END | SDHCI_INT_RESPONSE);
+ host->ier = SDHCI_INT_BUS_POWER | SDHCI_INT_DATA_END_BIT |
+ SDHCI_INT_DATA_CRC | SDHCI_INT_DATA_TIMEOUT |
+ SDHCI_INT_INDEX | SDHCI_INT_END_BIT | SDHCI_INT_CRC |
+ SDHCI_INT_TIMEOUT | SDHCI_INT_DATA_END |
+ SDHCI_INT_RESPONSE;
+
+ sdhci_writel(host, host->ier, SDHCI_INT_ENABLE);
+ sdhci_writel(host, host->ier, SDHCI_SIGNAL_ENABLE);
if (soft) {
/* force clock reconfiguration */
@@ -502,11 +488,6 @@
else
direction = DMA_TO_DEVICE;
- /*
- * The ADMA descriptor table is mapped further down as we
- * need to fill it with data first.
- */
-
host->align_addr = dma_map_single(mmc_dev(host->mmc),
host->align_buffer, 128 * 4, direction);
if (dma_mapping_error(mmc_dev(host->mmc), host->align_addr))
@@ -567,7 +548,7 @@
* If this triggers then we have a calculation bug
* somewhere. :/
*/
- WARN_ON((desc - host->adma_desc) > (128 * 2 + 1) * 4);
+ WARN_ON((desc - host->adma_desc) > ADMA_SIZE);
}
if (host->quirks & SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC) {
@@ -595,17 +576,8 @@
host->align_addr, 128 * 4, direction);
}
- host->adma_addr = dma_map_single(mmc_dev(host->mmc),
- host->adma_desc, (128 * 2 + 1) * 4, DMA_TO_DEVICE);
- if (dma_mapping_error(mmc_dev(host->mmc), host->adma_addr))
- goto unmap_entries;
- BUG_ON(host->adma_addr & 0x3);
-
return 0;
-unmap_entries:
- dma_unmap_sg(mmc_dev(host->mmc), data->sg,
- data->sg_len, direction);
unmap_align:
dma_unmap_single(mmc_dev(host->mmc), host->align_addr,
128 * 4, direction);
@@ -623,19 +595,25 @@
u8 *align;
char *buffer;
unsigned long flags;
+ bool has_unaligned;
if (data->flags & MMC_DATA_READ)
direction = DMA_FROM_DEVICE;
else
direction = DMA_TO_DEVICE;
- dma_unmap_single(mmc_dev(host->mmc), host->adma_addr,
- (128 * 2 + 1) * 4, DMA_TO_DEVICE);
-
dma_unmap_single(mmc_dev(host->mmc), host->align_addr,
128 * 4, direction);
- if (data->flags & MMC_DATA_READ) {
+ /* Do a quick scan of the SG list for any unaligned mappings */
+ has_unaligned = false;
+ for_each_sg(data->sg, sg, host->sg_count, i)
+ if (sg_dma_address(sg) & 3) {
+ has_unaligned = true;
+ break;
+ }
+
+ if (has_unaligned && data->flags & MMC_DATA_READ) {
dma_sync_sg_for_cpu(mmc_dev(host->mmc), data->sg,
data->sg_len, direction);
@@ -721,9 +699,12 @@
u32 dma_irqs = SDHCI_INT_DMA_END | SDHCI_INT_ADMA_ERROR;
if (host->flags & SDHCI_REQ_USE_DMA)
- sdhci_clear_set_irqs(host, pio_irqs, dma_irqs);
+ host->ier = (host->ier & ~pio_irqs) | dma_irqs;
else
- sdhci_clear_set_irqs(host, dma_irqs, pio_irqs);
+ host->ier = (host->ier & ~dma_irqs) | pio_irqs;
+
+ sdhci_writel(host, host->ier, SDHCI_INT_ENABLE);
+ sdhci_writel(host, host->ier, SDHCI_SIGNAL_ENABLE);
}
static void sdhci_prepare_data(struct sdhci_host *host, struct mmc_command *cmd)
@@ -976,8 +957,8 @@
* upon error conditions.
*/
if (data->error) {
- sdhci_reset(host, SDHCI_RESET_CMD);
- sdhci_reset(host, SDHCI_RESET_DATA);
+ sdhci_do_reset(host, SDHCI_RESET_CMD);
+ sdhci_do_reset(host, SDHCI_RESET_DATA);
}
sdhci_send_command(host, data->stop);
@@ -1107,24 +1088,23 @@
static u16 sdhci_get_preset_value(struct sdhci_host *host)
{
- u16 ctrl, preset = 0;
+ u16 preset = 0;
- ctrl = sdhci_readw(host, SDHCI_HOST_CONTROL2);
-
- switch (ctrl & SDHCI_CTRL_UHS_MASK) {
- case SDHCI_CTRL_UHS_SDR12:
+ switch (host->timing) {
+ case MMC_TIMING_UHS_SDR12:
preset = sdhci_readw(host, SDHCI_PRESET_FOR_SDR12);
break;
- case SDHCI_CTRL_UHS_SDR25:
+ case MMC_TIMING_UHS_SDR25:
preset = sdhci_readw(host, SDHCI_PRESET_FOR_SDR25);
break;
- case SDHCI_CTRL_UHS_SDR50:
+ case MMC_TIMING_UHS_SDR50:
preset = sdhci_readw(host, SDHCI_PRESET_FOR_SDR50);
break;
- case SDHCI_CTRL_UHS_SDR104:
+ case MMC_TIMING_UHS_SDR104:
+ case MMC_TIMING_MMC_HS200:
preset = sdhci_readw(host, SDHCI_PRESET_FOR_SDR104);
break;
- case SDHCI_CTRL_UHS_DDR50:
+ case MMC_TIMING_UHS_DDR50:
preset = sdhci_readw(host, SDHCI_PRESET_FOR_DDR50);
break;
default:
@@ -1136,32 +1116,22 @@
return preset;
}
-static void sdhci_set_clock(struct sdhci_host *host, unsigned int clock)
+void sdhci_set_clock(struct sdhci_host *host, unsigned int clock)
{
int div = 0; /* Initialized for compiler warning */
int real_div = div, clk_mul = 1;
u16 clk = 0;
unsigned long timeout;
- if (clock && clock == host->clock)
- return;
-
host->mmc->actual_clock = 0;
- if (host->ops->set_clock) {
- host->ops->set_clock(host, clock);
- if (host->quirks & SDHCI_QUIRK_NONSTANDARD_CLOCK)
- return;
- }
-
sdhci_writew(host, 0, SDHCI_CLOCK_CONTROL);
if (clock == 0)
- goto out;
+ return;
if (host->version >= SDHCI_SPEC_300) {
- if (sdhci_readw(host, SDHCI_HOST_CONTROL2) &
- SDHCI_CTRL_PRESET_VAL_ENABLE) {
+ if (host->preset_enabled) {
u16 pre_val;
clk = sdhci_readw(host, SDHCI_CLOCK_CONTROL);
@@ -1247,26 +1217,16 @@
clk |= SDHCI_CLOCK_CARD_EN;
sdhci_writew(host, clk, SDHCI_CLOCK_CONTROL);
-
-out:
- host->clock = clock;
}
+EXPORT_SYMBOL_GPL(sdhci_set_clock);
-static inline void sdhci_update_clock(struct sdhci_host *host)
-{
- unsigned int clock;
-
- clock = host->clock;
- host->clock = 0;
- sdhci_set_clock(host, clock);
-}
-
-static int sdhci_set_power(struct sdhci_host *host, unsigned short power)
+static void sdhci_set_power(struct sdhci_host *host, unsigned char mode,
+ unsigned short vdd)
{
u8 pwr = 0;
- if (power != (unsigned short)-1) {
- switch (1 << power) {
+ if (mode != MMC_POWER_OFF) {
+ switch (1 << vdd) {
case MMC_VDD_165_195:
pwr = SDHCI_POWER_180;
break;
@@ -1284,7 +1244,7 @@
}
if (host->pwr == pwr)
- return -1;
+ return;
host->pwr = pwr;
@@ -1292,38 +1252,43 @@
sdhci_writeb(host, 0, SDHCI_POWER_CONTROL);
if (host->quirks2 & SDHCI_QUIRK2_CARD_ON_NEEDS_BUS_ON)
sdhci_runtime_pm_bus_off(host);
- return 0;
- }
+ vdd = 0;
+ } else {
+ /*
+ * Spec says that we should clear the power reg before setting
+ * a new value. Some controllers don't seem to like this though.
+ */
+ if (!(host->quirks & SDHCI_QUIRK_SINGLE_POWER_WRITE))
+ sdhci_writeb(host, 0, SDHCI_POWER_CONTROL);
- /*
- * Spec says that we should clear the power reg before setting
- * a new value. Some controllers don't seem to like this though.
- */
- if (!(host->quirks & SDHCI_QUIRK_SINGLE_POWER_WRITE))
- sdhci_writeb(host, 0, SDHCI_POWER_CONTROL);
+ /*
+ * At least the Marvell CaFe chip gets confused if we set the
+ * voltage and set turn on power at the same time, so set the
+ * voltage first.
+ */
+ if (host->quirks & SDHCI_QUIRK_NO_SIMULT_VDD_AND_POWER)
+ sdhci_writeb(host, pwr, SDHCI_POWER_CONTROL);
- /*
- * At least the Marvell CaFe chip gets confused if we set the voltage
- * and set turn on power at the same time, so set the voltage first.
- */
- if (host->quirks & SDHCI_QUIRK_NO_SIMULT_VDD_AND_POWER)
+ pwr |= SDHCI_POWER_ON;
+
sdhci_writeb(host, pwr, SDHCI_POWER_CONTROL);
- pwr |= SDHCI_POWER_ON;
+ if (host->quirks2 & SDHCI_QUIRK2_CARD_ON_NEEDS_BUS_ON)
+ sdhci_runtime_pm_bus_on(host);
- sdhci_writeb(host, pwr, SDHCI_POWER_CONTROL);
+ /*
+ * Some controllers need an extra 10ms delay of 10ms before
+ * they can apply clock after applying power
+ */
+ if (host->quirks & SDHCI_QUIRK_DELAY_AFTER_POWER)
+ mdelay(10);
+ }
- if (host->quirks2 & SDHCI_QUIRK2_CARD_ON_NEEDS_BUS_ON)
- sdhci_runtime_pm_bus_on(host);
-
- /*
- * Some controllers need an extra 10ms delay of 10ms before they
- * can apply clock after applying power
- */
- if (host->quirks & SDHCI_QUIRK_DELAY_AFTER_POWER)
- mdelay(10);
-
- return power;
+ if (host->vmmc) {
+ spin_unlock_irq(&host->lock);
+ mmc_regulator_set_ocr(host->mmc, host->vmmc, vdd);
+ spin_lock_irq(&host->lock);
+ }
}
/*****************************************************************************\
@@ -1427,10 +1392,53 @@
spin_unlock_irqrestore(&host->lock, flags);
}
+void sdhci_set_bus_width(struct sdhci_host *host, int width)
+{
+ u8 ctrl;
+
+ ctrl = sdhci_readb(host, SDHCI_HOST_CONTROL);
+ if (width == MMC_BUS_WIDTH_8) {
+ ctrl &= ~SDHCI_CTRL_4BITBUS;
+ if (host->version >= SDHCI_SPEC_300)
+ ctrl |= SDHCI_CTRL_8BITBUS;
+ } else {
+ if (host->version >= SDHCI_SPEC_300)
+ ctrl &= ~SDHCI_CTRL_8BITBUS;
+ if (width == MMC_BUS_WIDTH_4)
+ ctrl |= SDHCI_CTRL_4BITBUS;
+ else
+ ctrl &= ~SDHCI_CTRL_4BITBUS;
+ }
+ sdhci_writeb(host, ctrl, SDHCI_HOST_CONTROL);
+}
+EXPORT_SYMBOL_GPL(sdhci_set_bus_width);
+
+void sdhci_set_uhs_signaling(struct sdhci_host *host, unsigned timing)
+{
+ u16 ctrl_2;
+
+ ctrl_2 = sdhci_readw(host, SDHCI_HOST_CONTROL2);
+ /* Select Bus Speed Mode for host */
+ ctrl_2 &= ~SDHCI_CTRL_UHS_MASK;
+ if ((timing == MMC_TIMING_MMC_HS200) ||
+ (timing == MMC_TIMING_UHS_SDR104))
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR104;
+ else if (timing == MMC_TIMING_UHS_SDR12)
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR12;
+ else if (timing == MMC_TIMING_UHS_SDR25)
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR25;
+ else if (timing == MMC_TIMING_UHS_SDR50)
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR50;
+ else if ((timing == MMC_TIMING_UHS_DDR50) ||
+ (timing == MMC_TIMING_MMC_DDR52))
+ ctrl_2 |= SDHCI_CTRL_UHS_DDR50;
+ sdhci_writew(host, ctrl_2, SDHCI_HOST_CONTROL2);
+}
+EXPORT_SYMBOL_GPL(sdhci_set_uhs_signaling);
+
static void sdhci_do_set_ios(struct sdhci_host *host, struct mmc_ios *ios)
{
unsigned long flags;
- int vdd_bit = -1;
u8 ctrl;
spin_lock_irqsave(&host->lock, flags);
@@ -1456,45 +1464,17 @@
!(host->quirks2 & SDHCI_QUIRK2_PRESET_VALUE_BROKEN))
sdhci_enable_preset_value(host, false);
- sdhci_set_clock(host, ios->clock);
-
- if (ios->power_mode == MMC_POWER_OFF)
- vdd_bit = sdhci_set_power(host, -1);
- else
- vdd_bit = sdhci_set_power(host, ios->vdd);
-
- if (host->vmmc && vdd_bit != -1) {
- spin_unlock_irqrestore(&host->lock, flags);
- mmc_regulator_set_ocr(host->mmc, host->vmmc, vdd_bit);
- spin_lock_irqsave(&host->lock, flags);
+ if (!ios->clock || ios->clock != host->clock) {
+ host->ops->set_clock(host, ios->clock);
+ host->clock = ios->clock;
}
+ sdhci_set_power(host, ios->power_mode, ios->vdd);
+
if (host->ops->platform_send_init_74_clocks)
host->ops->platform_send_init_74_clocks(host, ios->power_mode);
- /*
- * If your platform has 8-bit width support but is not a v3 controller,
- * or if it requires special setup code, you should implement that in
- * platform_bus_width().
- */
- if (host->ops->platform_bus_width) {
- host->ops->platform_bus_width(host, ios->bus_width);
- } else {
- ctrl = sdhci_readb(host, SDHCI_HOST_CONTROL);
- if (ios->bus_width == MMC_BUS_WIDTH_8) {
- ctrl &= ~SDHCI_CTRL_4BITBUS;
- if (host->version >= SDHCI_SPEC_300)
- ctrl |= SDHCI_CTRL_8BITBUS;
- } else {
- if (host->version >= SDHCI_SPEC_300)
- ctrl &= ~SDHCI_CTRL_8BITBUS;
- if (ios->bus_width == MMC_BUS_WIDTH_4)
- ctrl |= SDHCI_CTRL_4BITBUS;
- else
- ctrl &= ~SDHCI_CTRL_4BITBUS;
- }
- sdhci_writeb(host, ctrl, SDHCI_HOST_CONTROL);
- }
+ host->ops->set_bus_width(host, ios->bus_width);
ctrl = sdhci_readb(host, SDHCI_HOST_CONTROL);
@@ -1510,19 +1490,20 @@
/* In case of UHS-I modes, set High Speed Enable */
if ((ios->timing == MMC_TIMING_MMC_HS200) ||
+ (ios->timing == MMC_TIMING_MMC_DDR52) ||
(ios->timing == MMC_TIMING_UHS_SDR50) ||
(ios->timing == MMC_TIMING_UHS_SDR104) ||
(ios->timing == MMC_TIMING_UHS_DDR50) ||
(ios->timing == MMC_TIMING_UHS_SDR25))
ctrl |= SDHCI_CTRL_HISPD;
- ctrl_2 = sdhci_readw(host, SDHCI_HOST_CONTROL2);
- if (!(ctrl_2 & SDHCI_CTRL_PRESET_VAL_ENABLE)) {
+ if (!host->preset_enabled) {
sdhci_writeb(host, ctrl, SDHCI_HOST_CONTROL);
/*
* We only need to set Driver Strength if the
* preset value enable is not set.
*/
+ ctrl_2 = sdhci_readw(host, SDHCI_HOST_CONTROL2);
ctrl_2 &= ~SDHCI_CTRL_DRV_TYPE_MASK;
if (ios->drv_type == MMC_SET_DRIVER_TYPE_A)
ctrl_2 |= SDHCI_CTRL_DRV_TYPE_A;
@@ -1546,7 +1527,7 @@
sdhci_writeb(host, ctrl, SDHCI_HOST_CONTROL);
/* Re-enable SD Clock */
- sdhci_update_clock(host);
+ host->ops->set_clock(host, host->clock);
}
@@ -1555,25 +1536,8 @@
clk &= ~SDHCI_CLOCK_CARD_EN;
sdhci_writew(host, clk, SDHCI_CLOCK_CONTROL);
- if (host->ops->set_uhs_signaling)
- host->ops->set_uhs_signaling(host, ios->timing);
- else {
- ctrl_2 = sdhci_readw(host, SDHCI_HOST_CONTROL2);
- /* Select Bus Speed Mode for host */
- ctrl_2 &= ~SDHCI_CTRL_UHS_MASK;
- if ((ios->timing == MMC_TIMING_MMC_HS200) ||
- (ios->timing == MMC_TIMING_UHS_SDR104))
- ctrl_2 |= SDHCI_CTRL_UHS_SDR104;
- else if (ios->timing == MMC_TIMING_UHS_SDR12)
- ctrl_2 |= SDHCI_CTRL_UHS_SDR12;
- else if (ios->timing == MMC_TIMING_UHS_SDR25)
- ctrl_2 |= SDHCI_CTRL_UHS_SDR25;
- else if (ios->timing == MMC_TIMING_UHS_SDR50)
- ctrl_2 |= SDHCI_CTRL_UHS_SDR50;
- else if (ios->timing == MMC_TIMING_UHS_DDR50)
- ctrl_2 |= SDHCI_CTRL_UHS_DDR50;
- sdhci_writew(host, ctrl_2, SDHCI_HOST_CONTROL2);
- }
+ host->ops->set_uhs_signaling(host, ios->timing);
+ host->timing = ios->timing;
if (!(host->quirks2 & SDHCI_QUIRK2_PRESET_VALUE_BROKEN) &&
((ios->timing == MMC_TIMING_UHS_SDR12) ||
@@ -1590,7 +1554,7 @@
}
/* Re-enable SD Clock */
- sdhci_update_clock(host);
+ host->ops->set_clock(host, host->clock);
} else
sdhci_writeb(host, ctrl, SDHCI_HOST_CONTROL);
@@ -1600,7 +1564,7 @@
* it on each ios seems to solve the problem.
*/
if(host->quirks & SDHCI_QUIRK_RESET_CMD_DATA_ON_IOS)
- sdhci_reset(host, SDHCI_RESET_CMD | SDHCI_RESET_DATA);
+ sdhci_do_reset(host, SDHCI_RESET_CMD | SDHCI_RESET_DATA);
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
@@ -1709,24 +1673,16 @@
static void sdhci_enable_sdio_irq_nolock(struct sdhci_host *host, int enable)
{
- if (host->flags & SDHCI_DEVICE_DEAD)
- goto out;
+ if (!(host->flags & SDHCI_DEVICE_DEAD)) {
+ if (enable)
+ host->ier |= SDHCI_INT_CARD_INT;
+ else
+ host->ier &= ~SDHCI_INT_CARD_INT;
- if (enable)
- host->flags |= SDHCI_SDIO_IRQ_ENABLED;
- else
- host->flags &= ~SDHCI_SDIO_IRQ_ENABLED;
-
- /* SDIO IRQ will be enabled as appropriate in runtime resume */
- if (host->runtime_suspended)
- goto out;
-
- if (enable)
- sdhci_unmask_irqs(host, SDHCI_INT_CARD_INT);
- else
- sdhci_mask_irqs(host, SDHCI_INT_CARD_INT);
-out:
- mmiowb();
+ sdhci_writel(host, host->ier, SDHCI_INT_ENABLE);
+ sdhci_writel(host, host->ier, SDHCI_SIGNAL_ENABLE);
+ mmiowb();
+ }
}
static void sdhci_enable_sdio_irq(struct mmc_host *mmc, int enable)
@@ -1734,9 +1690,18 @@
struct sdhci_host *host = mmc_priv(mmc);
unsigned long flags;
+ sdhci_runtime_pm_get(host);
+
spin_lock_irqsave(&host->lock, flags);
+ if (enable)
+ host->flags |= SDHCI_SDIO_IRQ_ENABLED;
+ else
+ host->flags &= ~SDHCI_SDIO_IRQ_ENABLED;
+
sdhci_enable_sdio_irq_nolock(host, enable);
spin_unlock_irqrestore(&host->lock, flags);
+
+ sdhci_runtime_pm_put(host);
}
static int sdhci_do_start_signal_voltage_switch(struct sdhci_host *host,
@@ -1855,22 +1820,15 @@
static int sdhci_execute_tuning(struct mmc_host *mmc, u32 opcode)
{
- struct sdhci_host *host;
+ struct sdhci_host *host = mmc_priv(mmc);
u16 ctrl;
- u32 ier;
int tuning_loop_counter = MAX_TUNING_LOOP;
- unsigned long timeout;
int err = 0;
- bool requires_tuning_nonuhs = false;
unsigned long flags;
- host = mmc_priv(mmc);
-
sdhci_runtime_pm_get(host);
spin_lock_irqsave(&host->lock, flags);
- ctrl = sdhci_readw(host, SDHCI_HOST_CONTROL2);
-
/*
* The Host Controller needs tuning only in case of SDR104 mode
* and for SDR50 mode when Use Tuning for SDR50 is set in the
@@ -1878,15 +1836,18 @@
* If the Host Controller supports the HS200 mode then the
* tuning function has to be executed.
*/
- if (((ctrl & SDHCI_CTRL_UHS_MASK) == SDHCI_CTRL_UHS_SDR50) &&
- (host->flags & SDHCI_SDR50_NEEDS_TUNING ||
- host->flags & SDHCI_SDR104_NEEDS_TUNING))
- requires_tuning_nonuhs = true;
+ switch (host->timing) {
+ case MMC_TIMING_MMC_HS200:
+ case MMC_TIMING_UHS_SDR104:
+ break;
- if (((ctrl & SDHCI_CTRL_UHS_MASK) == SDHCI_CTRL_UHS_SDR104) ||
- requires_tuning_nonuhs)
- ctrl |= SDHCI_CTRL_EXEC_TUNING;
- else {
+ case MMC_TIMING_UHS_SDR50:
+ if (host->flags & SDHCI_SDR50_NEEDS_TUNING ||
+ host->flags & SDHCI_SDR104_NEEDS_TUNING)
+ break;
+ /* FALLTHROUGH */
+
+ default:
spin_unlock_irqrestore(&host->lock, flags);
sdhci_runtime_pm_put(host);
return 0;
@@ -1899,6 +1860,8 @@
return err;
}
+ ctrl = sdhci_readw(host, SDHCI_HOST_CONTROL2);
+ ctrl |= SDHCI_CTRL_EXEC_TUNING;
sdhci_writew(host, ctrl, SDHCI_HOST_CONTROL2);
/*
@@ -1911,21 +1874,17 @@
* to make sure we don't hit a controller bug, we _only_
* enable Buffer Read Ready interrupt here.
*/
- ier = sdhci_readl(host, SDHCI_INT_ENABLE);
- sdhci_clear_set_irqs(host, ier, SDHCI_INT_DATA_AVAIL);
+ sdhci_writel(host, SDHCI_INT_DATA_AVAIL, SDHCI_INT_ENABLE);
+ sdhci_writel(host, SDHCI_INT_DATA_AVAIL, SDHCI_SIGNAL_ENABLE);
/*
* Issue CMD19 repeatedly till Execute Tuning is set to 0 or the number
* of loops reaches 40 times or a timeout of 150ms occurs.
*/
- timeout = 150;
do {
struct mmc_command cmd = {0};
struct mmc_request mrq = {NULL};
- if (!tuning_loop_counter && !timeout)
- break;
-
cmd.opcode = opcode;
cmd.arg = 0;
cmd.flags = MMC_RSP_R1 | MMC_CMD_ADTC;
@@ -1933,6 +1892,9 @@
cmd.data = NULL;
cmd.error = 0;
+ if (tuning_loop_counter-- == 0)
+ break;
+
mrq.cmd = &cmd;
host->mrq = &mrq;
@@ -1990,26 +1952,25 @@
host->tuning_done = 0;
ctrl = sdhci_readw(host, SDHCI_HOST_CONTROL2);
- tuning_loop_counter--;
- timeout--;
- mdelay(1);
+
+ /* eMMC spec does not require a delay between tuning cycles */
+ if (opcode == MMC_SEND_TUNING_BLOCK)
+ mdelay(1);
} while (ctrl & SDHCI_CTRL_EXEC_TUNING);
/*
* The Host Driver has exhausted the maximum number of loops allowed,
* so use fixed sampling frequency.
*/
- if (!tuning_loop_counter || !timeout) {
+ if (tuning_loop_counter < 0) {
ctrl &= ~SDHCI_CTRL_TUNED_CLK;
sdhci_writew(host, ctrl, SDHCI_HOST_CONTROL2);
+ }
+ if (!(ctrl & SDHCI_CTRL_TUNED_CLK)) {
+ pr_info(DRIVER_NAME ": Tuning procedure"
+ " failed, falling back to fixed sampling"
+ " clock\n");
err = -EIO;
- } else {
- if (!(ctrl & SDHCI_CTRL_TUNED_CLK)) {
- pr_info(DRIVER_NAME ": Tuning procedure"
- " failed, falling back to fixed sampling"
- " clock\n");
- err = -EIO;
- }
}
out:
@@ -2044,7 +2005,8 @@
if (err && (host->flags & SDHCI_USING_RETUNING_TIMER))
err = 0;
- sdhci_clear_set_irqs(host, SDHCI_INT_DATA_AVAIL, ier);
+ sdhci_writel(host, host->ier, SDHCI_INT_ENABLE);
+ sdhci_writel(host, host->ier, SDHCI_SIGNAL_ENABLE);
spin_unlock_irqrestore(&host->lock, flags);
sdhci_runtime_pm_put(host);
@@ -2054,26 +2016,30 @@
static void sdhci_enable_preset_value(struct sdhci_host *host, bool enable)
{
- u16 ctrl;
-
/* Host Controller v3.00 defines preset value registers */
if (host->version < SDHCI_SPEC_300)
return;
- ctrl = sdhci_readw(host, SDHCI_HOST_CONTROL2);
-
/*
* We only enable or disable Preset Value if they are not already
* enabled or disabled respectively. Otherwise, we bail out.
*/
- if (enable && !(ctrl & SDHCI_CTRL_PRESET_VAL_ENABLE)) {
- ctrl |= SDHCI_CTRL_PRESET_VAL_ENABLE;
+ if (host->preset_enabled != enable) {
+ u16 ctrl = sdhci_readw(host, SDHCI_HOST_CONTROL2);
+
+ if (enable)
+ ctrl |= SDHCI_CTRL_PRESET_VAL_ENABLE;
+ else
+ ctrl &= ~SDHCI_CTRL_PRESET_VAL_ENABLE;
+
sdhci_writew(host, ctrl, SDHCI_HOST_CONTROL2);
- host->flags |= SDHCI_PV_ENABLED;
- } else if (!enable && (ctrl & SDHCI_CTRL_PRESET_VAL_ENABLE)) {
- ctrl &= ~SDHCI_CTRL_PRESET_VAL_ENABLE;
- sdhci_writew(host, ctrl, SDHCI_HOST_CONTROL2);
- host->flags &= ~SDHCI_PV_ENABLED;
+
+ if (enable)
+ host->flags |= SDHCI_PV_ENABLED;
+ else
+ host->flags &= ~SDHCI_PV_ENABLED;
+
+ host->preset_enabled = enable;
}
}
@@ -2095,8 +2061,8 @@
pr_err("%s: Resetting controller.\n",
mmc_hostname(host->mmc));
- sdhci_reset(host, SDHCI_RESET_CMD);
- sdhci_reset(host, SDHCI_RESET_DATA);
+ sdhci_do_reset(host, SDHCI_RESET_CMD);
+ sdhci_do_reset(host, SDHCI_RESET_DATA);
host->mrq->cmd->error = -ENOMEDIUM;
tasklet_schedule(&host->finish_tasklet);
@@ -2124,15 +2090,6 @@
* *
\*****************************************************************************/
-static void sdhci_tasklet_card(unsigned long param)
-{
- struct sdhci_host *host = (struct sdhci_host*)param;
-
- sdhci_card_event(host->mmc);
-
- mmc_detect_change(host->mmc, msecs_to_jiffies(200));
-}
-
static void sdhci_tasklet_finish(unsigned long param)
{
struct sdhci_host *host;
@@ -2169,12 +2126,12 @@
/* Some controllers need this kick or reset won't work here */
if (host->quirks & SDHCI_QUIRK_CLOCK_BEFORE_RESET)
/* This is to force an update */
- sdhci_update_clock(host);
+ host->ops->set_clock(host, host->clock);
/* Spec says we should do both at the same time, but Ricoh
controllers do not like that. */
- sdhci_reset(host, SDHCI_RESET_CMD);
- sdhci_reset(host, SDHCI_RESET_DATA);
+ sdhci_do_reset(host, SDHCI_RESET_CMD);
+ sdhci_do_reset(host, SDHCI_RESET_DATA);
}
host->mrq = NULL;
@@ -2424,101 +2381,94 @@
static irqreturn_t sdhci_irq(int irq, void *dev_id)
{
- irqreturn_t result;
+ irqreturn_t result = IRQ_NONE;
struct sdhci_host *host = dev_id;
- u32 intmask, unexpected = 0;
- int cardint = 0, max_loops = 16;
+ u32 intmask, mask, unexpected = 0;
+ int max_loops = 16;
spin_lock(&host->lock);
- if (host->runtime_suspended) {
+ if (host->runtime_suspended && !sdhci_sdio_irq_enabled(host)) {
spin_unlock(&host->lock);
return IRQ_NONE;
}
intmask = sdhci_readl(host, SDHCI_INT_STATUS);
-
if (!intmask || intmask == 0xffffffff) {
result = IRQ_NONE;
goto out;
}
-again:
- DBG("*** %s got interrupt: 0x%08x\n",
- mmc_hostname(host->mmc), intmask);
+ do {
+ /* Clear selected interrupts. */
+ mask = intmask & (SDHCI_INT_CMD_MASK | SDHCI_INT_DATA_MASK |
+ SDHCI_INT_BUS_POWER);
+ sdhci_writel(host, mask, SDHCI_INT_STATUS);
- if (intmask & (SDHCI_INT_CARD_INSERT | SDHCI_INT_CARD_REMOVE)) {
- u32 present = sdhci_readl(host, SDHCI_PRESENT_STATE) &
- SDHCI_CARD_PRESENT;
+ DBG("*** %s got interrupt: 0x%08x\n",
+ mmc_hostname(host->mmc), intmask);
- /*
- * There is a observation on i.mx esdhc. INSERT bit will be
- * immediately set again when it gets cleared, if a card is
- * inserted. We have to mask the irq to prevent interrupt
- * storm which will freeze the system. And the REMOVE gets
- * the same situation.
- *
- * More testing are needed here to ensure it works for other
- * platforms though.
- */
- sdhci_mask_irqs(host, present ? SDHCI_INT_CARD_INSERT :
- SDHCI_INT_CARD_REMOVE);
- sdhci_unmask_irqs(host, present ? SDHCI_INT_CARD_REMOVE :
- SDHCI_INT_CARD_INSERT);
+ if (intmask & (SDHCI_INT_CARD_INSERT | SDHCI_INT_CARD_REMOVE)) {
+ u32 present = sdhci_readl(host, SDHCI_PRESENT_STATE) &
+ SDHCI_CARD_PRESENT;
- sdhci_writel(host, intmask & (SDHCI_INT_CARD_INSERT |
- SDHCI_INT_CARD_REMOVE), SDHCI_INT_STATUS);
- intmask &= ~(SDHCI_INT_CARD_INSERT | SDHCI_INT_CARD_REMOVE);
- tasklet_schedule(&host->card_tasklet);
- }
+ /*
+ * There is a observation on i.mx esdhc. INSERT
+ * bit will be immediately set again when it gets
+ * cleared, if a card is inserted. We have to mask
+ * the irq to prevent interrupt storm which will
+ * freeze the system. And the REMOVE gets the
+ * same situation.
+ *
+ * More testing are needed here to ensure it works
+ * for other platforms though.
+ */
+ host->ier &= ~(SDHCI_INT_CARD_INSERT |
+ SDHCI_INT_CARD_REMOVE);
+ host->ier |= present ? SDHCI_INT_CARD_REMOVE :
+ SDHCI_INT_CARD_INSERT;
+ sdhci_writel(host, host->ier, SDHCI_INT_ENABLE);
+ sdhci_writel(host, host->ier, SDHCI_SIGNAL_ENABLE);
- if (intmask & SDHCI_INT_CMD_MASK) {
- sdhci_writel(host, intmask & SDHCI_INT_CMD_MASK,
- SDHCI_INT_STATUS);
- sdhci_cmd_irq(host, intmask & SDHCI_INT_CMD_MASK);
- }
+ sdhci_writel(host, intmask & (SDHCI_INT_CARD_INSERT |
+ SDHCI_INT_CARD_REMOVE), SDHCI_INT_STATUS);
- if (intmask & SDHCI_INT_DATA_MASK) {
- sdhci_writel(host, intmask & SDHCI_INT_DATA_MASK,
- SDHCI_INT_STATUS);
- sdhci_data_irq(host, intmask & SDHCI_INT_DATA_MASK);
- }
+ host->thread_isr |= intmask & (SDHCI_INT_CARD_INSERT |
+ SDHCI_INT_CARD_REMOVE);
+ result = IRQ_WAKE_THREAD;
+ }
- intmask &= ~(SDHCI_INT_CMD_MASK | SDHCI_INT_DATA_MASK);
+ if (intmask & SDHCI_INT_CMD_MASK)
+ sdhci_cmd_irq(host, intmask & SDHCI_INT_CMD_MASK);
- intmask &= ~SDHCI_INT_ERROR;
+ if (intmask & SDHCI_INT_DATA_MASK)
+ sdhci_data_irq(host, intmask & SDHCI_INT_DATA_MASK);
- if (intmask & SDHCI_INT_BUS_POWER) {
- pr_err("%s: Card is consuming too much power!\n",
- mmc_hostname(host->mmc));
- sdhci_writel(host, SDHCI_INT_BUS_POWER, SDHCI_INT_STATUS);
- }
+ if (intmask & SDHCI_INT_BUS_POWER)
+ pr_err("%s: Card is consuming too much power!\n",
+ mmc_hostname(host->mmc));
- intmask &= ~SDHCI_INT_BUS_POWER;
+ if (intmask & SDHCI_INT_CARD_INT) {
+ sdhci_enable_sdio_irq_nolock(host, false);
+ host->thread_isr |= SDHCI_INT_CARD_INT;
+ result = IRQ_WAKE_THREAD;
+ }
- if (intmask & SDHCI_INT_CARD_INT)
- cardint = 1;
+ intmask &= ~(SDHCI_INT_CARD_INSERT | SDHCI_INT_CARD_REMOVE |
+ SDHCI_INT_CMD_MASK | SDHCI_INT_DATA_MASK |
+ SDHCI_INT_ERROR | SDHCI_INT_BUS_POWER |
+ SDHCI_INT_CARD_INT);
- intmask &= ~SDHCI_INT_CARD_INT;
+ if (intmask) {
+ unexpected |= intmask;
+ sdhci_writel(host, intmask, SDHCI_INT_STATUS);
+ }
- if (intmask) {
- unexpected |= intmask;
- sdhci_writel(host, intmask, SDHCI_INT_STATUS);
- }
+ if (result == IRQ_NONE)
+ result = IRQ_HANDLED;
- result = IRQ_HANDLED;
-
- intmask = sdhci_readl(host, SDHCI_INT_STATUS);
-
- /*
- * If we know we'll call the driver to signal SDIO IRQ, disregard
- * further indications of Card Interrupt in the status to avoid a
- * needless loop.
- */
- if (cardint)
- intmask &= ~SDHCI_INT_CARD_INT;
- if (intmask && --max_loops)
- goto again;
+ intmask = sdhci_readl(host, SDHCI_INT_STATUS);
+ } while (intmask && --max_loops);
out:
spin_unlock(&host->lock);
@@ -2527,15 +2477,38 @@
mmc_hostname(host->mmc), unexpected);
sdhci_dumpregs(host);
}
- /*
- * We have to delay this as it calls back into the driver.
- */
- if (cardint)
- mmc_signal_sdio_irq(host->mmc);
return result;
}
+static irqreturn_t sdhci_thread_irq(int irq, void *dev_id)
+{
+ struct sdhci_host *host = dev_id;
+ unsigned long flags;
+ u32 isr;
+
+ spin_lock_irqsave(&host->lock, flags);
+ isr = host->thread_isr;
+ host->thread_isr = 0;
+ spin_unlock_irqrestore(&host->lock, flags);
+
+ if (isr & (SDHCI_INT_CARD_INSERT | SDHCI_INT_CARD_REMOVE)) {
+ sdhci_card_event(host->mmc);
+ mmc_detect_change(host->mmc, msecs_to_jiffies(200));
+ }
+
+ if (isr & SDHCI_INT_CARD_INT) {
+ sdio_run_irqs(host->mmc);
+
+ spin_lock_irqsave(&host->lock, flags);
+ if (host->flags & SDHCI_SDIO_IRQ_ENABLED)
+ sdhci_enable_sdio_irq_nolock(host, true);
+ spin_unlock_irqrestore(&host->lock, flags);
+ }
+
+ return isr ? IRQ_HANDLED : IRQ_NONE;
+}
+
/*****************************************************************************\
* *
* Suspend/resume *
@@ -2572,9 +2545,6 @@
int sdhci_suspend_host(struct sdhci_host *host)
{
- if (host->ops->platform_suspend)
- host->ops->platform_suspend(host);
-
sdhci_disable_card_detection(host);
/* Disable tuning since we are suspending */
@@ -2584,7 +2554,9 @@
}
if (!device_may_wakeup(mmc_dev(host->mmc))) {
- sdhci_mask_irqs(host, SDHCI_INT_ALL_MASK);
+ host->ier = 0;
+ sdhci_writel(host, 0, SDHCI_INT_ENABLE);
+ sdhci_writel(host, 0, SDHCI_SIGNAL_ENABLE);
free_irq(host->irq, host);
} else {
sdhci_enable_irq_wakeups(host);
@@ -2605,8 +2577,9 @@
}
if (!device_may_wakeup(mmc_dev(host->mmc))) {
- ret = request_irq(host->irq, sdhci_irq, IRQF_SHARED,
- mmc_hostname(host->mmc), host);
+ ret = request_threaded_irq(host->irq, sdhci_irq,
+ sdhci_thread_irq, IRQF_SHARED,
+ mmc_hostname(host->mmc), host);
if (ret)
return ret;
} else {
@@ -2628,9 +2601,6 @@
sdhci_enable_card_detection(host);
- if (host->ops->platform_resume)
- host->ops->platform_resume(host);
-
/* Set the re-tuning expiration flag */
if (host->flags & SDHCI_USING_RETUNING_TIMER)
host->flags |= SDHCI_NEEDS_RETUNING;
@@ -2682,10 +2652,12 @@
}
spin_lock_irqsave(&host->lock, flags);
- sdhci_mask_irqs(host, SDHCI_INT_ALL_MASK);
+ host->ier &= SDHCI_INT_CARD_INT;
+ sdhci_writel(host, host->ier, SDHCI_INT_ENABLE);
+ sdhci_writel(host, host->ier, SDHCI_SIGNAL_ENABLE);
spin_unlock_irqrestore(&host->lock, flags);
- synchronize_irq(host->irq);
+ synchronize_hardirq(host->irq);
spin_lock_irqsave(&host->lock, flags);
host->runtime_suspended = true;
@@ -2729,7 +2701,7 @@
host->runtime_suspended = false;
/* Enable SDIO IRQ */
- if ((host->flags & SDHCI_SDIO_IRQ_ENABLED))
+ if (host->flags & SDHCI_SDIO_IRQ_ENABLED)
sdhci_enable_sdio_irq_nolock(host, true);
/* Enable Card Detection */
@@ -2788,7 +2760,7 @@
if (debug_quirks2)
host->quirks2 = debug_quirks2;
- sdhci_reset(host, SDHCI_RESET_ALL);
+ sdhci_do_reset(host, SDHCI_RESET_ALL);
host->version = sdhci_readw(host, SDHCI_HOST_VERSION);
host->version = (host->version & SDHCI_SPEC_VER_MASK)
@@ -2848,15 +2820,29 @@
* (128) and potentially one alignment transfer for
* each of those entries.
*/
- host->adma_desc = kmalloc((128 * 2 + 1) * 4, GFP_KERNEL);
+ host->adma_desc = dma_alloc_coherent(mmc_dev(host->mmc),
+ ADMA_SIZE, &host->adma_addr,
+ GFP_KERNEL);
host->align_buffer = kmalloc(128 * 4, GFP_KERNEL);
if (!host->adma_desc || !host->align_buffer) {
- kfree(host->adma_desc);
+ dma_free_coherent(mmc_dev(host->mmc), ADMA_SIZE,
+ host->adma_desc, host->adma_addr);
kfree(host->align_buffer);
pr_warning("%s: Unable to allocate ADMA "
"buffers. Falling back to standard DMA.\n",
mmc_hostname(mmc));
host->flags &= ~SDHCI_USE_ADMA;
+ host->adma_desc = NULL;
+ host->align_buffer = NULL;
+ } else if (host->adma_addr & 3) {
+ pr_warning("%s: unable to allocate aligned ADMA descriptor\n",
+ mmc_hostname(mmc));
+ host->flags &= ~SDHCI_USE_ADMA;
+ dma_free_coherent(mmc_dev(host->mmc), ADMA_SIZE,
+ host->adma_desc, host->adma_addr);
+ kfree(host->align_buffer);
+ host->adma_desc = NULL;
+ host->align_buffer = NULL;
}
}
@@ -2941,6 +2927,7 @@
mmc->max_busy_timeout = (1 << 27) / host->timeout_clk;
mmc->caps |= MMC_CAP_SDIO_IRQ | MMC_CAP_ERASE | MMC_CAP_CMD23;
+ mmc->caps2 |= MMC_CAP2_SDIO_IRQ_NOTHREAD;
if (host->quirks & SDHCI_QUIRK_MULTIBLOCK_READ_ACMD12)
host->flags |= SDHCI_AUTO_CMD12;
@@ -3212,8 +3199,6 @@
/*
* Init tasklets.
*/
- tasklet_init(&host->card_tasklet,
- sdhci_tasklet_card, (unsigned long)host);
tasklet_init(&host->finish_tasklet,
sdhci_tasklet_finish, (unsigned long)host);
@@ -3230,8 +3215,8 @@
sdhci_init(host, 0);
- ret = request_irq(host->irq, sdhci_irq, IRQF_SHARED,
- mmc_hostname(mmc), host);
+ ret = request_threaded_irq(host->irq, sdhci_irq, sdhci_thread_irq,
+ IRQF_SHARED, mmc_hostname(mmc), host);
if (ret) {
pr_err("%s: Failed to request IRQ %d: %d\n",
mmc_hostname(mmc), host->irq, ret);
@@ -3273,12 +3258,12 @@
#ifdef SDHCI_USE_LEDS_CLASS
reset:
- sdhci_reset(host, SDHCI_RESET_ALL);
- sdhci_mask_irqs(host, SDHCI_INT_ALL_MASK);
+ sdhci_do_reset(host, SDHCI_RESET_ALL);
+ sdhci_writel(host, 0, SDHCI_INT_ENABLE);
+ sdhci_writel(host, 0, SDHCI_SIGNAL_ENABLE);
free_irq(host->irq, host);
#endif
untasklet:
- tasklet_kill(&host->card_tasklet);
tasklet_kill(&host->finish_tasklet);
return ret;
@@ -3315,14 +3300,14 @@
#endif
if (!dead)
- sdhci_reset(host, SDHCI_RESET_ALL);
+ sdhci_do_reset(host, SDHCI_RESET_ALL);
- sdhci_mask_irqs(host, SDHCI_INT_ALL_MASK);
+ sdhci_writel(host, 0, SDHCI_INT_ENABLE);
+ sdhci_writel(host, 0, SDHCI_SIGNAL_ENABLE);
free_irq(host->irq, host);
del_timer_sync(&host->timer);
- tasklet_kill(&host->card_tasklet);
tasklet_kill(&host->finish_tasklet);
if (host->vmmc) {
@@ -3335,7 +3320,9 @@
regulator_put(host->vqmmc);
}
- kfree(host->adma_desc);
+ if (host->adma_desc)
+ dma_free_coherent(mmc_dev(host->mmc), ADMA_SIZE,
+ host->adma_desc, host->adma_addr);
kfree(host->align_buffer);
host->adma_desc = NULL;
diff --git a/drivers/mmc/host/sdhci.h b/drivers/mmc/host/sdhci.h
index 0a3ed01..4a5cd5e 100644
--- a/drivers/mmc/host/sdhci.h
+++ b/drivers/mmc/host/sdhci.h
@@ -281,18 +281,14 @@
unsigned int (*get_max_clock)(struct sdhci_host *host);
unsigned int (*get_min_clock)(struct sdhci_host *host);
unsigned int (*get_timeout_clock)(struct sdhci_host *host);
- int (*platform_bus_width)(struct sdhci_host *host,
- int width);
+ void (*set_bus_width)(struct sdhci_host *host, int width);
void (*platform_send_init_74_clocks)(struct sdhci_host *host,
u8 power_mode);
unsigned int (*get_ro)(struct sdhci_host *host);
- void (*platform_reset_enter)(struct sdhci_host *host, u8 mask);
- void (*platform_reset_exit)(struct sdhci_host *host, u8 mask);
+ void (*reset)(struct sdhci_host *host, u8 mask);
int (*platform_execute_tuning)(struct sdhci_host *host, u32 opcode);
- int (*set_uhs_signaling)(struct sdhci_host *host, unsigned int uhs);
+ void (*set_uhs_signaling)(struct sdhci_host *host, unsigned int uhs);
void (*hw_reset)(struct sdhci_host *host);
- void (*platform_suspend)(struct sdhci_host *host);
- void (*platform_resume)(struct sdhci_host *host);
void (*adma_workaround)(struct sdhci_host *host, u32 intmask);
void (*platform_init)(struct sdhci_host *host);
void (*card_event)(struct sdhci_host *host);
@@ -397,6 +393,16 @@
extern void sdhci_send_command(struct sdhci_host *host,
struct mmc_command *cmd);
+static inline bool sdhci_sdio_irq_enabled(struct sdhci_host *host)
+{
+ return !!(host->flags & SDHCI_SDIO_IRQ_ENABLED);
+}
+
+void sdhci_set_clock(struct sdhci_host *host, unsigned int clock);
+void sdhci_set_bus_width(struct sdhci_host *host, int width);
+void sdhci_reset(struct sdhci_host *host, u8 mask);
+void sdhci_set_uhs_signaling(struct sdhci_host *host, unsigned timing);
+
#ifdef CONFIG_PM
extern int sdhci_suspend_host(struct sdhci_host *host);
extern int sdhci_resume_host(struct sdhci_host *host);
diff --git a/drivers/mmc/host/sh_mmcif.c b/drivers/mmc/host/sh_mmcif.c
index 54730f4..656fbba 100644
--- a/drivers/mmc/host/sh_mmcif.c
+++ b/drivers/mmc/host/sh_mmcif.c
@@ -803,12 +803,13 @@
break;
}
switch (host->timing) {
- case MMC_TIMING_UHS_DDR50:
+ case MMC_TIMING_MMC_DDR52:
/*
* MMC core will only set this timing, if the host
- * advertises the MMC_CAP_UHS_DDR50 capability. MMCIF
- * implementations with this capability, e.g. sh73a0,
- * will have to set it in their platform data.
+ * advertises the MMC_CAP_1_8V_DDR/MMC_CAP_1_2V_DDR
+ * capability. MMCIF implementations with this
+ * capability, e.g. sh73a0, will have to set it
+ * in their platform data.
*/
tmp |= CMD_SET_DARS;
break;
diff --git a/drivers/mmc/host/sunxi-mmc.c b/drivers/mmc/host/sunxi-mmc.c
new file mode 100644
index 0000000..024f67c
--- /dev/null
+++ b/drivers/mmc/host/sunxi-mmc.c
@@ -0,0 +1,1049 @@
+/*
+ * Driver for sunxi SD/MMC host controllers
+ * (C) Copyright 2007-2011 Reuuimlla Technology Co., Ltd.
+ * (C) Copyright 2007-2011 Aaron Maoye <leafy.myeh@reuuimllatech.com>
+ * (C) Copyright 2013-2014 O2S GmbH <www.o2s.ch>
+ * (C) Copyright 2013-2014 David Lanzend�rfer <david.lanzendoerfer@o2s.ch>
+ * (C) Copyright 2013-2014 Hans de Goede <hdegoede@redhat.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/io.h>
+#include <linux/device.h>
+#include <linux/interrupt.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+
+#include <linux/clk.h>
+#include <linux/clk-private.h>
+#include <linux/clk/sunxi.h>
+
+#include <linux/gpio.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/scatterlist.h>
+#include <linux/dma-mapping.h>
+#include <linux/slab.h>
+#include <linux/reset.h>
+
+#include <linux/of_address.h>
+#include <linux/of_gpio.h>
+#include <linux/of_platform.h>
+
+#include <linux/mmc/host.h>
+#include <linux/mmc/sd.h>
+#include <linux/mmc/sdio.h>
+#include <linux/mmc/mmc.h>
+#include <linux/mmc/core.h>
+#include <linux/mmc/card.h>
+#include <linux/mmc/slot-gpio.h>
+
+/* register offset definitions */
+#define SDXC_REG_GCTRL (0x00) /* SMC Global Control Register */
+#define SDXC_REG_CLKCR (0x04) /* SMC Clock Control Register */
+#define SDXC_REG_TMOUT (0x08) /* SMC Time Out Register */
+#define SDXC_REG_WIDTH (0x0C) /* SMC Bus Width Register */
+#define SDXC_REG_BLKSZ (0x10) /* SMC Block Size Register */
+#define SDXC_REG_BCNTR (0x14) /* SMC Byte Count Register */
+#define SDXC_REG_CMDR (0x18) /* SMC Command Register */
+#define SDXC_REG_CARG (0x1C) /* SMC Argument Register */
+#define SDXC_REG_RESP0 (0x20) /* SMC Response Register 0 */
+#define SDXC_REG_RESP1 (0x24) /* SMC Response Register 1 */
+#define SDXC_REG_RESP2 (0x28) /* SMC Response Register 2 */
+#define SDXC_REG_RESP3 (0x2C) /* SMC Response Register 3 */
+#define SDXC_REG_IMASK (0x30) /* SMC Interrupt Mask Register */
+#define SDXC_REG_MISTA (0x34) /* SMC Masked Interrupt Status Register */
+#define SDXC_REG_RINTR (0x38) /* SMC Raw Interrupt Status Register */
+#define SDXC_REG_STAS (0x3C) /* SMC Status Register */
+#define SDXC_REG_FTRGL (0x40) /* SMC FIFO Threshold Watermark Registe */
+#define SDXC_REG_FUNS (0x44) /* SMC Function Select Register */
+#define SDXC_REG_CBCR (0x48) /* SMC CIU Byte Count Register */
+#define SDXC_REG_BBCR (0x4C) /* SMC BIU Byte Count Register */
+#define SDXC_REG_DBGC (0x50) /* SMC Debug Enable Register */
+#define SDXC_REG_HWRST (0x78) /* SMC Card Hardware Reset for Register */
+#define SDXC_REG_DMAC (0x80) /* SMC IDMAC Control Register */
+#define SDXC_REG_DLBA (0x84) /* SMC IDMAC Descriptor List Base Addre */
+#define SDXC_REG_IDST (0x88) /* SMC IDMAC Status Register */
+#define SDXC_REG_IDIE (0x8C) /* SMC IDMAC Interrupt Enable Register */
+#define SDXC_REG_CHDA (0x90)
+#define SDXC_REG_CBDA (0x94)
+
+#define mmc_readl(host, reg) \
+ readl((host)->reg_base + SDXC_##reg)
+#define mmc_writel(host, reg, value) \
+ writel((value), (host)->reg_base + SDXC_##reg)
+
+/* global control register bits */
+#define SDXC_SOFT_RESET BIT(0)
+#define SDXC_FIFO_RESET BIT(1)
+#define SDXC_DMA_RESET BIT(2)
+#define SDXC_INTERRUPT_ENABLE_BIT BIT(4)
+#define SDXC_DMA_ENABLE_BIT BIT(5)
+#define SDXC_DEBOUNCE_ENABLE_BIT BIT(8)
+#define SDXC_POSEDGE_LATCH_DATA BIT(9)
+#define SDXC_DDR_MODE BIT(10)
+#define SDXC_MEMORY_ACCESS_DONE BIT(29)
+#define SDXC_ACCESS_DONE_DIRECT BIT(30)
+#define SDXC_ACCESS_BY_AHB BIT(31)
+#define SDXC_ACCESS_BY_DMA (0 << 31)
+#define SDXC_HARDWARE_RESET \
+ (SDXC_SOFT_RESET | SDXC_FIFO_RESET | SDXC_DMA_RESET)
+
+/* clock control bits */
+#define SDXC_CARD_CLOCK_ON BIT(16)
+#define SDXC_LOW_POWER_ON BIT(17)
+
+/* bus width */
+#define SDXC_WIDTH1 0
+#define SDXC_WIDTH4 1
+#define SDXC_WIDTH8 2
+
+/* smc command bits */
+#define SDXC_RESP_EXPIRE BIT(6)
+#define SDXC_LONG_RESPONSE BIT(7)
+#define SDXC_CHECK_RESPONSE_CRC BIT(8)
+#define SDXC_DATA_EXPIRE BIT(9)
+#define SDXC_WRITE BIT(10)
+#define SDXC_SEQUENCE_MODE BIT(11)
+#define SDXC_SEND_AUTO_STOP BIT(12)
+#define SDXC_WAIT_PRE_OVER BIT(13)
+#define SDXC_STOP_ABORT_CMD BIT(14)
+#define SDXC_SEND_INIT_SEQUENCE BIT(15)
+#define SDXC_UPCLK_ONLY BIT(21)
+#define SDXC_READ_CEATA_DEV BIT(22)
+#define SDXC_CCS_EXPIRE BIT(23)
+#define SDXC_ENABLE_BIT_BOOT BIT(24)
+#define SDXC_ALT_BOOT_OPTIONS BIT(25)
+#define SDXC_BOOT_ACK_EXPIRE BIT(26)
+#define SDXC_BOOT_ABORT BIT(27)
+#define SDXC_VOLTAGE_SWITCH BIT(28)
+#define SDXC_USE_HOLD_REGISTER BIT(29)
+#define SDXC_START BIT(31)
+
+/* interrupt bits */
+#define SDXC_RESP_ERROR BIT(1)
+#define SDXC_COMMAND_DONE BIT(2)
+#define SDXC_DATA_OVER BIT(3)
+#define SDXC_TX_DATA_REQUEST BIT(4)
+#define SDXC_RX_DATA_REQUEST BIT(5)
+#define SDXC_RESP_CRC_ERROR BIT(6)
+#define SDXC_DATA_CRC_ERROR BIT(7)
+#define SDXC_RESP_TIMEOUT BIT(8)
+#define SDXC_DATA_TIMEOUT BIT(9)
+#define SDXC_VOLTAGE_CHANGE_DONE BIT(10)
+#define SDXC_FIFO_RUN_ERROR BIT(11)
+#define SDXC_HARD_WARE_LOCKED BIT(12)
+#define SDXC_START_BIT_ERROR BIT(13)
+#define SDXC_AUTO_COMMAND_DONE BIT(14)
+#define SDXC_END_BIT_ERROR BIT(15)
+#define SDXC_SDIO_INTERRUPT BIT(16)
+#define SDXC_CARD_INSERT BIT(30)
+#define SDXC_CARD_REMOVE BIT(31)
+#define SDXC_INTERRUPT_ERROR_BIT \
+ (SDXC_RESP_ERROR | SDXC_RESP_CRC_ERROR | SDXC_DATA_CRC_ERROR | \
+ SDXC_RESP_TIMEOUT | SDXC_DATA_TIMEOUT | SDXC_FIFO_RUN_ERROR | \
+ SDXC_HARD_WARE_LOCKED | SDXC_START_BIT_ERROR | SDXC_END_BIT_ERROR)
+#define SDXC_INTERRUPT_DONE_BIT \
+ (SDXC_AUTO_COMMAND_DONE | SDXC_DATA_OVER | \
+ SDXC_COMMAND_DONE | SDXC_VOLTAGE_CHANGE_DONE)
+
+/* status */
+#define SDXC_RXWL_FLAG BIT(0)
+#define SDXC_TXWL_FLAG BIT(1)
+#define SDXC_FIFO_EMPTY BIT(2)
+#define SDXC_FIFO_FULL BIT(3)
+#define SDXC_CARD_PRESENT BIT(8)
+#define SDXC_CARD_DATA_BUSY BIT(9)
+#define SDXC_DATA_FSM_BUSY BIT(10)
+#define SDXC_DMA_REQUEST BIT(31)
+#define SDXC_FIFO_SIZE 16
+
+/* Function select */
+#define SDXC_CEATA_ON (0xceaa << 16)
+#define SDXC_SEND_IRQ_RESPONSE BIT(0)
+#define SDXC_SDIO_READ_WAIT BIT(1)
+#define SDXC_ABORT_READ_DATA BIT(2)
+#define SDXC_SEND_CCSD BIT(8)
+#define SDXC_SEND_AUTO_STOPCCSD BIT(9)
+#define SDXC_CEATA_DEV_IRQ_ENABLE BIT(10)
+
+/* IDMA controller bus mod bit field */
+#define SDXC_IDMAC_SOFT_RESET BIT(0)
+#define SDXC_IDMAC_FIX_BURST BIT(1)
+#define SDXC_IDMAC_IDMA_ON BIT(7)
+#define SDXC_IDMAC_REFETCH_DES BIT(31)
+
+/* IDMA status bit field */
+#define SDXC_IDMAC_TRANSMIT_INTERRUPT BIT(0)
+#define SDXC_IDMAC_RECEIVE_INTERRUPT BIT(1)
+#define SDXC_IDMAC_FATAL_BUS_ERROR BIT(2)
+#define SDXC_IDMAC_DESTINATION_INVALID BIT(4)
+#define SDXC_IDMAC_CARD_ERROR_SUM BIT(5)
+#define SDXC_IDMAC_NORMAL_INTERRUPT_SUM BIT(8)
+#define SDXC_IDMAC_ABNORMAL_INTERRUPT_SUM BIT(9)
+#define SDXC_IDMAC_HOST_ABORT_INTERRUPT BIT(10)
+#define SDXC_IDMAC_IDLE (0 << 13)
+#define SDXC_IDMAC_SUSPEND (1 << 13)
+#define SDXC_IDMAC_DESC_READ (2 << 13)
+#define SDXC_IDMAC_DESC_CHECK (3 << 13)
+#define SDXC_IDMAC_READ_REQUEST_WAIT (4 << 13)
+#define SDXC_IDMAC_WRITE_REQUEST_WAIT (5 << 13)
+#define SDXC_IDMAC_READ (6 << 13)
+#define SDXC_IDMAC_WRITE (7 << 13)
+#define SDXC_IDMAC_DESC_CLOSE (8 << 13)
+
+/*
+* If the idma-des-size-bits of property is ie 13, bufsize bits are:
+* Bits 0-12: buf1 size
+* Bits 13-25: buf2 size
+* Bits 26-31: not used
+* Since we only ever set buf1 size, we can simply store it directly.
+*/
+#define SDXC_IDMAC_DES0_DIC BIT(1) /* disable interrupt on completion */
+#define SDXC_IDMAC_DES0_LD BIT(2) /* last descriptor */
+#define SDXC_IDMAC_DES0_FD BIT(3) /* first descriptor */
+#define SDXC_IDMAC_DES0_CH BIT(4) /* chain mode */
+#define SDXC_IDMAC_DES0_ER BIT(5) /* end of ring */
+#define SDXC_IDMAC_DES0_CES BIT(30) /* card error summary */
+#define SDXC_IDMAC_DES0_OWN BIT(31) /* 1-idma owns it, 0-host owns it */
+
+struct sunxi_idma_des {
+ u32 config;
+ u32 buf_size;
+ u32 buf_addr_ptr1;
+ u32 buf_addr_ptr2;
+};
+
+struct sunxi_mmc_host {
+ struct mmc_host *mmc;
+ struct reset_control *reset;
+
+ /* IO mapping base */
+ void __iomem *reg_base;
+
+ /* clock management */
+ struct clk *clk_ahb;
+ struct clk *clk_mmc;
+
+ /* irq */
+ spinlock_t lock;
+ int irq;
+ u32 int_sum;
+ u32 sdio_imask;
+
+ /* dma */
+ u32 idma_des_size_bits;
+ dma_addr_t sg_dma;
+ void *sg_cpu;
+ bool wait_dma;
+
+ struct mmc_request *mrq;
+ struct mmc_request *manual_stop_mrq;
+ int ferror;
+};
+
+static int sunxi_mmc_reset_host(struct sunxi_mmc_host *host)
+{
+ unsigned long expire = jiffies + msecs_to_jiffies(250);
+ u32 rval;
+
+ mmc_writel(host, REG_CMDR, SDXC_HARDWARE_RESET);
+ do {
+ rval = mmc_readl(host, REG_GCTRL);
+ } while (time_before(jiffies, expire) && (rval & SDXC_HARDWARE_RESET));
+
+ if (rval & SDXC_HARDWARE_RESET) {
+ dev_err(mmc_dev(host->mmc), "fatal err reset timeout\n");
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int sunxi_mmc_init_host(struct mmc_host *mmc)
+{
+ u32 rval;
+ struct sunxi_mmc_host *host = mmc_priv(mmc);
+
+ if (sunxi_mmc_reset_host(host))
+ return -EIO;
+
+ mmc_writel(host, REG_FTRGL, 0x20070008);
+ mmc_writel(host, REG_TMOUT, 0xffffffff);
+ mmc_writel(host, REG_IMASK, host->sdio_imask);
+ mmc_writel(host, REG_RINTR, 0xffffffff);
+ mmc_writel(host, REG_DBGC, 0xdeb);
+ mmc_writel(host, REG_FUNS, SDXC_CEATA_ON);
+ mmc_writel(host, REG_DLBA, host->sg_dma);
+
+ rval = mmc_readl(host, REG_GCTRL);
+ rval |= SDXC_INTERRUPT_ENABLE_BIT;
+ rval &= ~SDXC_ACCESS_DONE_DIRECT;
+ mmc_writel(host, REG_GCTRL, rval);
+
+ return 0;
+}
+
+static void sunxi_mmc_init_idma_des(struct sunxi_mmc_host *host,
+ struct mmc_data *data)
+{
+ struct sunxi_idma_des *pdes = (struct sunxi_idma_des *)host->sg_cpu;
+ struct sunxi_idma_des *pdes_pa = (struct sunxi_idma_des *)host->sg_dma;
+ int i, max_len = (1 << host->idma_des_size_bits);
+
+ for (i = 0; i < data->sg_len; i++) {
+ pdes[i].config = SDXC_IDMAC_DES0_CH | SDXC_IDMAC_DES0_OWN |
+ SDXC_IDMAC_DES0_DIC;
+
+ if (data->sg[i].length == max_len)
+ pdes[i].buf_size = 0; /* 0 == max_len */
+ else
+ pdes[i].buf_size = data->sg[i].length;
+
+ pdes[i].buf_addr_ptr1 = sg_dma_address(&data->sg[i]);
+ pdes[i].buf_addr_ptr2 = (u32)&pdes_pa[i + 1];
+ }
+
+ pdes[0].config |= SDXC_IDMAC_DES0_FD;
+ pdes[i - 1].config = SDXC_IDMAC_DES0_OWN | SDXC_IDMAC_DES0_LD;
+
+ /*
+ * Avoid the io-store starting the idmac hitting io-mem before the
+ * descriptors hit the main-mem.
+ */
+ wmb();
+}
+
+static enum dma_data_direction sunxi_mmc_get_dma_dir(struct mmc_data *data)
+{
+ if (data->flags & MMC_DATA_WRITE)
+ return DMA_TO_DEVICE;
+ else
+ return DMA_FROM_DEVICE;
+}
+
+static int sunxi_mmc_map_dma(struct sunxi_mmc_host *host,
+ struct mmc_data *data)
+{
+ u32 i, dma_len;
+ struct scatterlist *sg;
+
+ dma_len = dma_map_sg(mmc_dev(host->mmc), data->sg, data->sg_len,
+ sunxi_mmc_get_dma_dir(data));
+ if (dma_len == 0) {
+ dev_err(mmc_dev(host->mmc), "dma_map_sg failed\n");
+ return -ENOMEM;
+ }
+
+ for_each_sg(data->sg, sg, data->sg_len, i) {
+ if (sg->offset & 3 || sg->length & 3) {
+ dev_err(mmc_dev(host->mmc),
+ "unaligned scatterlist: os %x length %d\n",
+ sg->offset, sg->length);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static void sunxi_mmc_start_dma(struct sunxi_mmc_host *host,
+ struct mmc_data *data)
+{
+ u32 rval;
+
+ sunxi_mmc_init_idma_des(host, data);
+
+ rval = mmc_readl(host, REG_GCTRL);
+ rval |= SDXC_DMA_ENABLE_BIT;
+ mmc_writel(host, REG_GCTRL, rval);
+ rval |= SDXC_DMA_RESET;
+ mmc_writel(host, REG_GCTRL, rval);
+
+ mmc_writel(host, REG_DMAC, SDXC_IDMAC_SOFT_RESET);
+
+ if (!(data->flags & MMC_DATA_WRITE))
+ mmc_writel(host, REG_IDIE, SDXC_IDMAC_RECEIVE_INTERRUPT);
+
+ mmc_writel(host, REG_DMAC,
+ SDXC_IDMAC_FIX_BURST | SDXC_IDMAC_IDMA_ON);
+}
+
+static void sunxi_mmc_send_manual_stop(struct sunxi_mmc_host *host,
+ struct mmc_request *req)
+{
+ u32 arg, cmd_val, ri;
+ unsigned long expire = jiffies + msecs_to_jiffies(1000);
+
+ cmd_val = SDXC_START | SDXC_RESP_EXPIRE |
+ SDXC_STOP_ABORT_CMD | SDXC_CHECK_RESPONSE_CRC;
+
+ if (req->cmd->opcode == SD_IO_RW_EXTENDED) {
+ cmd_val |= SD_IO_RW_DIRECT;
+ arg = (1 << 31) | (0 << 28) | (SDIO_CCCR_ABORT << 9) |
+ ((req->cmd->arg >> 28) & 0x7);
+ } else {
+ cmd_val |= MMC_STOP_TRANSMISSION;
+ arg = 0;
+ }
+
+ mmc_writel(host, REG_CARG, arg);
+ mmc_writel(host, REG_CMDR, cmd_val);
+
+ do {
+ ri = mmc_readl(host, REG_RINTR);
+ } while (!(ri & (SDXC_COMMAND_DONE | SDXC_INTERRUPT_ERROR_BIT)) &&
+ time_before(jiffies, expire));
+
+ if (!(ri & SDXC_COMMAND_DONE) || (ri & SDXC_INTERRUPT_ERROR_BIT)) {
+ dev_err(mmc_dev(host->mmc), "send stop command failed\n");
+ if (req->stop)
+ req->stop->resp[0] = -ETIMEDOUT;
+ } else {
+ if (req->stop)
+ req->stop->resp[0] = mmc_readl(host, REG_RESP0);
+ }
+
+ mmc_writel(host, REG_RINTR, 0xffff);
+}
+
+static void sunxi_mmc_dump_errinfo(struct sunxi_mmc_host *host)
+{
+ struct mmc_command *cmd = host->mrq->cmd;
+ struct mmc_data *data = host->mrq->data;
+
+ /* For some cmds timeout is normal with sd/mmc cards */
+ if ((host->int_sum & SDXC_INTERRUPT_ERROR_BIT) ==
+ SDXC_RESP_TIMEOUT && (cmd->opcode == SD_IO_SEND_OP_COND ||
+ cmd->opcode == SD_IO_RW_DIRECT))
+ return;
+
+ dev_err(mmc_dev(host->mmc),
+ "smc %d err, cmd %d,%s%s%s%s%s%s%s%s%s%s !!\n",
+ host->mmc->index, cmd->opcode,
+ data ? (data->flags & MMC_DATA_WRITE ? " WR" : " RD") : "",
+ host->int_sum & SDXC_RESP_ERROR ? " RE" : "",
+ host->int_sum & SDXC_RESP_CRC_ERROR ? " RCE" : "",
+ host->int_sum & SDXC_DATA_CRC_ERROR ? " DCE" : "",
+ host->int_sum & SDXC_RESP_TIMEOUT ? " RTO" : "",
+ host->int_sum & SDXC_DATA_TIMEOUT ? " DTO" : "",
+ host->int_sum & SDXC_FIFO_RUN_ERROR ? " FE" : "",
+ host->int_sum & SDXC_HARD_WARE_LOCKED ? " HL" : "",
+ host->int_sum & SDXC_START_BIT_ERROR ? " SBE" : "",
+ host->int_sum & SDXC_END_BIT_ERROR ? " EBE" : ""
+ );
+}
+
+/* Called in interrupt context! */
+static irqreturn_t sunxi_mmc_finalize_request(struct sunxi_mmc_host *host)
+{
+ struct mmc_request *mrq = host->mrq;
+ struct mmc_data *data = mrq->data;
+ u32 rval;
+
+ mmc_writel(host, REG_IMASK, host->sdio_imask);
+ mmc_writel(host, REG_IDIE, 0);
+
+ if (host->int_sum & SDXC_INTERRUPT_ERROR_BIT) {
+ sunxi_mmc_dump_errinfo(host);
+ mrq->cmd->error = -ETIMEDOUT;
+
+ if (data) {
+ data->error = -ETIMEDOUT;
+ host->manual_stop_mrq = mrq;
+ }
+
+ if (mrq->stop)
+ mrq->stop->error = -ETIMEDOUT;
+ } else {
+ if (mrq->cmd->flags & MMC_RSP_136) {
+ mrq->cmd->resp[0] = mmc_readl(host, REG_RESP3);
+ mrq->cmd->resp[1] = mmc_readl(host, REG_RESP2);
+ mrq->cmd->resp[2] = mmc_readl(host, REG_RESP1);
+ mrq->cmd->resp[3] = mmc_readl(host, REG_RESP0);
+ } else {
+ mrq->cmd->resp[0] = mmc_readl(host, REG_RESP0);
+ }
+
+ if (data)
+ data->bytes_xfered = data->blocks * data->blksz;
+ }
+
+ if (data) {
+ mmc_writel(host, REG_IDST, 0x337);
+ mmc_writel(host, REG_DMAC, 0);
+ rval = mmc_readl(host, REG_GCTRL);
+ rval |= SDXC_DMA_RESET;
+ mmc_writel(host, REG_GCTRL, rval);
+ rval &= ~SDXC_DMA_ENABLE_BIT;
+ mmc_writel(host, REG_GCTRL, rval);
+ rval |= SDXC_FIFO_RESET;
+ mmc_writel(host, REG_GCTRL, rval);
+ dma_unmap_sg(mmc_dev(host->mmc), data->sg, data->sg_len,
+ sunxi_mmc_get_dma_dir(data));
+ }
+
+ mmc_writel(host, REG_RINTR, 0xffff);
+
+ host->mrq = NULL;
+ host->int_sum = 0;
+ host->wait_dma = false;
+
+ return host->manual_stop_mrq ? IRQ_WAKE_THREAD : IRQ_HANDLED;
+}
+
+static irqreturn_t sunxi_mmc_irq(int irq, void *dev_id)
+{
+ struct sunxi_mmc_host *host = dev_id;
+ struct mmc_request *mrq;
+ u32 msk_int, idma_int;
+ bool finalize = false;
+ bool sdio_int = false;
+ irqreturn_t ret = IRQ_HANDLED;
+
+ spin_lock(&host->lock);
+
+ idma_int = mmc_readl(host, REG_IDST);
+ msk_int = mmc_readl(host, REG_MISTA);
+
+ dev_dbg(mmc_dev(host->mmc), "irq: rq %p mi %08x idi %08x\n",
+ host->mrq, msk_int, idma_int);
+
+ mrq = host->mrq;
+ if (mrq) {
+ if (idma_int & SDXC_IDMAC_RECEIVE_INTERRUPT)
+ host->wait_dma = false;
+
+ host->int_sum |= msk_int;
+
+ /* Wait for COMMAND_DONE on RESPONSE_TIMEOUT before finalize */
+ if ((host->int_sum & SDXC_RESP_TIMEOUT) &&
+ !(host->int_sum & SDXC_COMMAND_DONE))
+ mmc_writel(host, REG_IMASK,
+ host->sdio_imask | SDXC_COMMAND_DONE);
+ /* Don't wait for dma on error */
+ else if (host->int_sum & SDXC_INTERRUPT_ERROR_BIT)
+ finalize = true;
+ else if ((host->int_sum & SDXC_INTERRUPT_DONE_BIT) &&
+ !host->wait_dma)
+ finalize = true;
+ }
+
+ if (msk_int & SDXC_SDIO_INTERRUPT)
+ sdio_int = true;
+
+ mmc_writel(host, REG_RINTR, msk_int);
+ mmc_writel(host, REG_IDST, idma_int);
+
+ if (finalize)
+ ret = sunxi_mmc_finalize_request(host);
+
+ spin_unlock(&host->lock);
+
+ if (finalize && ret == IRQ_HANDLED)
+ mmc_request_done(host->mmc, mrq);
+
+ if (sdio_int)
+ mmc_signal_sdio_irq(host->mmc);
+
+ return ret;
+}
+
+static irqreturn_t sunxi_mmc_handle_manual_stop(int irq, void *dev_id)
+{
+ struct sunxi_mmc_host *host = dev_id;
+ struct mmc_request *mrq;
+ unsigned long iflags;
+
+ spin_lock_irqsave(&host->lock, iflags);
+ mrq = host->manual_stop_mrq;
+ spin_unlock_irqrestore(&host->lock, iflags);
+
+ if (!mrq) {
+ dev_err(mmc_dev(host->mmc), "no request for manual stop\n");
+ return IRQ_HANDLED;
+ }
+
+ dev_err(mmc_dev(host->mmc), "data error, sending stop command\n");
+ sunxi_mmc_send_manual_stop(host, mrq);
+
+ spin_lock_irqsave(&host->lock, iflags);
+ host->manual_stop_mrq = NULL;
+ spin_unlock_irqrestore(&host->lock, iflags);
+
+ mmc_request_done(host->mmc, mrq);
+
+ return IRQ_HANDLED;
+}
+
+static int sunxi_mmc_oclk_onoff(struct sunxi_mmc_host *host, u32 oclk_en)
+{
+ unsigned long expire = jiffies + msecs_to_jiffies(250);
+ u32 rval;
+
+ rval = mmc_readl(host, REG_CLKCR);
+ rval &= ~(SDXC_CARD_CLOCK_ON | SDXC_LOW_POWER_ON);
+
+ if (oclk_en)
+ rval |= SDXC_CARD_CLOCK_ON;
+
+ mmc_writel(host, REG_CLKCR, rval);
+
+ rval = SDXC_START | SDXC_UPCLK_ONLY | SDXC_WAIT_PRE_OVER;
+ mmc_writel(host, REG_CMDR, rval);
+
+ do {
+ rval = mmc_readl(host, REG_CMDR);
+ } while (time_before(jiffies, expire) && (rval & SDXC_START));
+
+ /* clear irq status bits set by the command */
+ mmc_writel(host, REG_RINTR,
+ mmc_readl(host, REG_RINTR) & ~SDXC_SDIO_INTERRUPT);
+
+ if (rval & SDXC_START) {
+ dev_err(mmc_dev(host->mmc), "fatal err update clk timeout\n");
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int sunxi_mmc_clk_set_rate(struct sunxi_mmc_host *host,
+ struct mmc_ios *ios)
+{
+ u32 rate, oclk_dly, rval, sclk_dly, src_clk;
+ int ret;
+
+ rate = clk_round_rate(host->clk_mmc, ios->clock);
+ dev_dbg(mmc_dev(host->mmc), "setting clk to %d, rounded %d\n",
+ ios->clock, rate);
+
+ /* setting clock rate */
+ ret = clk_set_rate(host->clk_mmc, rate);
+ if (ret) {
+ dev_err(mmc_dev(host->mmc), "error setting clk to %d: %d\n",
+ rate, ret);
+ return ret;
+ }
+
+ ret = sunxi_mmc_oclk_onoff(host, 0);
+ if (ret)
+ return ret;
+
+ /* clear internal divider */
+ rval = mmc_readl(host, REG_CLKCR);
+ rval &= ~0xff;
+ mmc_writel(host, REG_CLKCR, rval);
+
+ /* determine delays */
+ if (rate <= 400000) {
+ oclk_dly = 0;
+ sclk_dly = 7;
+ } else if (rate <= 25000000) {
+ oclk_dly = 0;
+ sclk_dly = 5;
+ } else if (rate <= 50000000) {
+ if (ios->timing == MMC_TIMING_UHS_DDR50) {
+ oclk_dly = 2;
+ sclk_dly = 4;
+ } else {
+ oclk_dly = 3;
+ sclk_dly = 5;
+ }
+ } else {
+ /* rate > 50000000 */
+ oclk_dly = 2;
+ sclk_dly = 4;
+ }
+
+ src_clk = clk_get_rate(clk_get_parent(host->clk_mmc));
+ if (src_clk >= 300000000 && src_clk <= 400000000) {
+ if (oclk_dly)
+ oclk_dly--;
+ if (sclk_dly)
+ sclk_dly--;
+ }
+
+ clk_sunxi_mmc_phase_control(host->clk_mmc, sclk_dly, oclk_dly);
+
+ return sunxi_mmc_oclk_onoff(host, 1);
+}
+
+static void sunxi_mmc_set_ios(struct mmc_host *mmc, struct mmc_ios *ios)
+{
+ struct sunxi_mmc_host *host = mmc_priv(mmc);
+ u32 rval;
+
+ /* Set the power state */
+ switch (ios->power_mode) {
+ case MMC_POWER_ON:
+ break;
+
+ case MMC_POWER_UP:
+ mmc_regulator_set_ocr(mmc, mmc->supply.vmmc, ios->vdd);
+
+ host->ferror = sunxi_mmc_init_host(mmc);
+ if (host->ferror)
+ return;
+
+ dev_dbg(mmc_dev(mmc), "power on!\n");
+ break;
+
+ case MMC_POWER_OFF:
+ dev_dbg(mmc_dev(mmc), "power off!\n");
+ sunxi_mmc_reset_host(host);
+ mmc_regulator_set_ocr(mmc, mmc->supply.vmmc, 0);
+ break;
+ }
+
+ /* set bus width */
+ switch (ios->bus_width) {
+ case MMC_BUS_WIDTH_1:
+ mmc_writel(host, REG_WIDTH, SDXC_WIDTH1);
+ break;
+ case MMC_BUS_WIDTH_4:
+ mmc_writel(host, REG_WIDTH, SDXC_WIDTH4);
+ break;
+ case MMC_BUS_WIDTH_8:
+ mmc_writel(host, REG_WIDTH, SDXC_WIDTH8);
+ break;
+ }
+
+ /* set ddr mode */
+ rval = mmc_readl(host, REG_GCTRL);
+ if (ios->timing == MMC_TIMING_UHS_DDR50)
+ rval |= SDXC_DDR_MODE;
+ else
+ rval &= ~SDXC_DDR_MODE;
+ mmc_writel(host, REG_GCTRL, rval);
+
+ /* set up clock */
+ if (ios->clock && ios->power_mode) {
+ host->ferror = sunxi_mmc_clk_set_rate(host, ios);
+ /* Android code had a usleep_range(50000, 55000); here */
+ }
+}
+
+static void sunxi_mmc_enable_sdio_irq(struct mmc_host *mmc, int enable)
+{
+ struct sunxi_mmc_host *host = mmc_priv(mmc);
+ unsigned long flags;
+ u32 imask;
+
+ spin_lock_irqsave(&host->lock, flags);
+
+ imask = mmc_readl(host, REG_IMASK);
+ if (enable) {
+ host->sdio_imask = SDXC_SDIO_INTERRUPT;
+ imask |= SDXC_SDIO_INTERRUPT;
+ } else {
+ host->sdio_imask = 0;
+ imask &= ~SDXC_SDIO_INTERRUPT;
+ }
+ mmc_writel(host, REG_IMASK, imask);
+ spin_unlock_irqrestore(&host->lock, flags);
+}
+
+static void sunxi_mmc_hw_reset(struct mmc_host *mmc)
+{
+ struct sunxi_mmc_host *host = mmc_priv(mmc);
+ mmc_writel(host, REG_HWRST, 0);
+ udelay(10);
+ mmc_writel(host, REG_HWRST, 1);
+ udelay(300);
+}
+
+static void sunxi_mmc_request(struct mmc_host *mmc, struct mmc_request *mrq)
+{
+ struct sunxi_mmc_host *host = mmc_priv(mmc);
+ struct mmc_command *cmd = mrq->cmd;
+ struct mmc_data *data = mrq->data;
+ unsigned long iflags;
+ u32 imask = SDXC_INTERRUPT_ERROR_BIT;
+ u32 cmd_val = SDXC_START | (cmd->opcode & 0x3f);
+ int ret;
+
+ /* Check for set_ios errors (should never happen) */
+ if (host->ferror) {
+ mrq->cmd->error = host->ferror;
+ mmc_request_done(mmc, mrq);
+ return;
+ }
+
+ if (data) {
+ ret = sunxi_mmc_map_dma(host, data);
+ if (ret < 0) {
+ dev_err(mmc_dev(mmc), "map DMA failed\n");
+ cmd->error = ret;
+ data->error = ret;
+ mmc_request_done(mmc, mrq);
+ return;
+ }
+ }
+
+ if (cmd->opcode == MMC_GO_IDLE_STATE) {
+ cmd_val |= SDXC_SEND_INIT_SEQUENCE;
+ imask |= SDXC_COMMAND_DONE;
+ }
+
+ if (cmd->flags & MMC_RSP_PRESENT) {
+ cmd_val |= SDXC_RESP_EXPIRE;
+ if (cmd->flags & MMC_RSP_136)
+ cmd_val |= SDXC_LONG_RESPONSE;
+ if (cmd->flags & MMC_RSP_CRC)
+ cmd_val |= SDXC_CHECK_RESPONSE_CRC;
+
+ if ((cmd->flags & MMC_CMD_MASK) == MMC_CMD_ADTC) {
+ cmd_val |= SDXC_DATA_EXPIRE | SDXC_WAIT_PRE_OVER;
+ if (cmd->data->flags & MMC_DATA_STREAM) {
+ imask |= SDXC_AUTO_COMMAND_DONE;
+ cmd_val |= SDXC_SEQUENCE_MODE |
+ SDXC_SEND_AUTO_STOP;
+ }
+
+ if (cmd->data->stop) {
+ imask |= SDXC_AUTO_COMMAND_DONE;
+ cmd_val |= SDXC_SEND_AUTO_STOP;
+ } else {
+ imask |= SDXC_DATA_OVER;
+ }
+
+ if (cmd->data->flags & MMC_DATA_WRITE)
+ cmd_val |= SDXC_WRITE;
+ else
+ host->wait_dma = true;
+ } else {
+ imask |= SDXC_COMMAND_DONE;
+ }
+ } else {
+ imask |= SDXC_COMMAND_DONE;
+ }
+
+ dev_dbg(mmc_dev(mmc), "cmd %d(%08x) arg %x ie 0x%08x len %d\n",
+ cmd_val & 0x3f, cmd_val, cmd->arg, imask,
+ mrq->data ? mrq->data->blksz * mrq->data->blocks : 0);
+
+ spin_lock_irqsave(&host->lock, iflags);
+
+ if (host->mrq || host->manual_stop_mrq) {
+ spin_unlock_irqrestore(&host->lock, iflags);
+
+ if (data)
+ dma_unmap_sg(mmc_dev(mmc), data->sg, data->sg_len,
+ sunxi_mmc_get_dma_dir(data));
+
+ dev_err(mmc_dev(mmc), "request already pending\n");
+ mrq->cmd->error = -EBUSY;
+ mmc_request_done(mmc, mrq);
+ return;
+ }
+
+ if (data) {
+ mmc_writel(host, REG_BLKSZ, data->blksz);
+ mmc_writel(host, REG_BCNTR, data->blksz * data->blocks);
+ sunxi_mmc_start_dma(host, data);
+ }
+
+ host->mrq = mrq;
+ mmc_writel(host, REG_IMASK, host->sdio_imask | imask);
+ mmc_writel(host, REG_CARG, cmd->arg);
+ mmc_writel(host, REG_CMDR, cmd_val);
+
+ spin_unlock_irqrestore(&host->lock, iflags);
+}
+
+static const struct of_device_id sunxi_mmc_of_match[] = {
+ { .compatible = "allwinner,sun4i-a10-mmc", },
+ { .compatible = "allwinner,sun5i-a13-mmc", },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, sunxi_mmc_of_match);
+
+static struct mmc_host_ops sunxi_mmc_ops = {
+ .request = sunxi_mmc_request,
+ .set_ios = sunxi_mmc_set_ios,
+ .get_ro = mmc_gpio_get_ro,
+ .get_cd = mmc_gpio_get_cd,
+ .enable_sdio_irq = sunxi_mmc_enable_sdio_irq,
+ .hw_reset = sunxi_mmc_hw_reset,
+};
+
+static int sunxi_mmc_resource_request(struct sunxi_mmc_host *host,
+ struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ int ret;
+
+ if (of_device_is_compatible(np, "allwinner,sun4i-a10-mmc"))
+ host->idma_des_size_bits = 13;
+ else
+ host->idma_des_size_bits = 16;
+
+ ret = mmc_regulator_get_supply(host->mmc);
+ if (ret) {
+ if (ret != -EPROBE_DEFER)
+ dev_err(&pdev->dev, "Could not get vmmc supply\n");
+ return ret;
+ }
+
+ host->reg_base = devm_ioremap_resource(&pdev->dev,
+ platform_get_resource(pdev, IORESOURCE_MEM, 0));
+ if (IS_ERR(host->reg_base))
+ return PTR_ERR(host->reg_base);
+
+ host->clk_ahb = devm_clk_get(&pdev->dev, "ahb");
+ if (IS_ERR(host->clk_ahb)) {
+ dev_err(&pdev->dev, "Could not get ahb clock\n");
+ return PTR_ERR(host->clk_ahb);
+ }
+
+ host->clk_mmc = devm_clk_get(&pdev->dev, "mmc");
+ if (IS_ERR(host->clk_mmc)) {
+ dev_err(&pdev->dev, "Could not get mmc clock\n");
+ return PTR_ERR(host->clk_mmc);
+ }
+
+ host->reset = devm_reset_control_get(&pdev->dev, "ahb");
+
+ ret = clk_prepare_enable(host->clk_ahb);
+ if (ret) {
+ dev_err(&pdev->dev, "Enable ahb clk err %d\n", ret);
+ return ret;
+ }
+
+ ret = clk_prepare_enable(host->clk_mmc);
+ if (ret) {
+ dev_err(&pdev->dev, "Enable mmc clk err %d\n", ret);
+ goto error_disable_clk_ahb;
+ }
+
+ if (!IS_ERR(host->reset)) {
+ ret = reset_control_deassert(host->reset);
+ if (ret) {
+ dev_err(&pdev->dev, "reset err %d\n", ret);
+ goto error_disable_clk_mmc;
+ }
+ }
+
+ /*
+ * Sometimes the controller asserts the irq on boot for some reason,
+ * make sure the controller is in a sane state before enabling irqs.
+ */
+ ret = sunxi_mmc_reset_host(host);
+ if (ret)
+ goto error_assert_reset;
+
+ host->irq = platform_get_irq(pdev, 0);
+ return devm_request_threaded_irq(&pdev->dev, host->irq, sunxi_mmc_irq,
+ sunxi_mmc_handle_manual_stop, 0, "sunxi-mmc", host);
+
+error_assert_reset:
+ if (!IS_ERR(host->reset))
+ reset_control_assert(host->reset);
+error_disable_clk_mmc:
+ clk_disable_unprepare(host->clk_mmc);
+error_disable_clk_ahb:
+ clk_disable_unprepare(host->clk_ahb);
+ return ret;
+}
+
+static int sunxi_mmc_probe(struct platform_device *pdev)
+{
+ struct sunxi_mmc_host *host;
+ struct mmc_host *mmc;
+ int ret;
+
+ mmc = mmc_alloc_host(sizeof(struct sunxi_mmc_host), &pdev->dev);
+ if (!mmc) {
+ dev_err(&pdev->dev, "mmc alloc host failed\n");
+ return -ENOMEM;
+ }
+
+ host = mmc_priv(mmc);
+ host->mmc = mmc;
+ spin_lock_init(&host->lock);
+
+ ret = sunxi_mmc_resource_request(host, pdev);
+ if (ret)
+ goto error_free_host;
+
+ host->sg_cpu = dma_alloc_coherent(&pdev->dev, PAGE_SIZE,
+ &host->sg_dma, GFP_KERNEL);
+ if (!host->sg_cpu) {
+ dev_err(&pdev->dev, "Failed to allocate DMA descriptor mem\n");
+ ret = -ENOMEM;
+ goto error_free_host;
+ }
+
+ mmc->ops = &sunxi_mmc_ops;
+ mmc->max_blk_count = 8192;
+ mmc->max_blk_size = 4096;
+ mmc->max_segs = PAGE_SIZE / sizeof(struct sunxi_idma_des);
+ mmc->max_seg_size = (1 << host->idma_des_size_bits);
+ mmc->max_req_size = mmc->max_seg_size * mmc->max_segs;
+ /* 400kHz ~ 50MHz */
+ mmc->f_min = 400000;
+ mmc->f_max = 50000000;
+ mmc->caps |= MMC_CAP_MMC_HIGHSPEED | MMC_CAP_SD_HIGHSPEED;
+
+ ret = mmc_of_parse(mmc);
+ if (ret)
+ goto error_free_dma;
+
+ ret = mmc_add_host(mmc);
+ if (ret)
+ goto error_free_dma;
+
+ dev_info(&pdev->dev, "base:0x%p irq:%u\n", host->reg_base, host->irq);
+ platform_set_drvdata(pdev, mmc);
+ return 0;
+
+error_free_dma:
+ dma_free_coherent(&pdev->dev, PAGE_SIZE, host->sg_cpu, host->sg_dma);
+error_free_host:
+ mmc_free_host(mmc);
+ return ret;
+}
+
+static int sunxi_mmc_remove(struct platform_device *pdev)
+{
+ struct mmc_host *mmc = platform_get_drvdata(pdev);
+ struct sunxi_mmc_host *host = mmc_priv(mmc);
+
+ mmc_remove_host(mmc);
+ disable_irq(host->irq);
+ sunxi_mmc_reset_host(host);
+
+ if (!IS_ERR(host->reset))
+ reset_control_assert(host->reset);
+
+ clk_disable_unprepare(host->clk_mmc);
+ clk_disable_unprepare(host->clk_ahb);
+
+ dma_free_coherent(&pdev->dev, PAGE_SIZE, host->sg_cpu, host->sg_dma);
+ mmc_free_host(mmc);
+
+ return 0;
+}
+
+static struct platform_driver sunxi_mmc_driver = {
+ .driver = {
+ .name = "sunxi-mmc",
+ .owner = THIS_MODULE,
+ .of_match_table = of_match_ptr(sunxi_mmc_of_match),
+ },
+ .probe = sunxi_mmc_probe,
+ .remove = sunxi_mmc_remove,
+};
+module_platform_driver(sunxi_mmc_driver);
+
+MODULE_DESCRIPTION("Allwinner's SD/MMC Card Controller Driver");
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("David Lanzend�rfer <david.lanzendoerfer@o2s.ch>");
+MODULE_ALIAS("platform:sunxi-mmc");
diff --git a/drivers/mmc/host/usdhi6rol0.c b/drivers/mmc/host/usdhi6rol0.c
new file mode 100644
index 0000000..eb2bbbe
--- /dev/null
+++ b/drivers/mmc/host/usdhi6rol0.c
@@ -0,0 +1,1847 @@
+/*
+ * Copyright (C) 2013-2014 Renesas Electronics Europe Ltd.
+ * Author: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/dmaengine.h>
+#include <linux/highmem.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/log2.h>
+#include <linux/mmc/host.h>
+#include <linux/mmc/mmc.h>
+#include <linux/mmc/sd.h>
+#include <linux/mmc/sdio.h>
+#include <linux/module.h>
+#include <linux/pagemap.h>
+#include <linux/platform_device.h>
+#include <linux/scatterlist.h>
+#include <linux/string.h>
+#include <linux/time.h>
+#include <linux/virtio.h>
+#include <linux/workqueue.h>
+
+#define USDHI6_SD_CMD 0x0000
+#define USDHI6_SD_PORT_SEL 0x0004
+#define USDHI6_SD_ARG 0x0008
+#define USDHI6_SD_STOP 0x0010
+#define USDHI6_SD_SECCNT 0x0014
+#define USDHI6_SD_RSP10 0x0018
+#define USDHI6_SD_RSP32 0x0020
+#define USDHI6_SD_RSP54 0x0028
+#define USDHI6_SD_RSP76 0x0030
+#define USDHI6_SD_INFO1 0x0038
+#define USDHI6_SD_INFO2 0x003c
+#define USDHI6_SD_INFO1_MASK 0x0040
+#define USDHI6_SD_INFO2_MASK 0x0044
+#define USDHI6_SD_CLK_CTRL 0x0048
+#define USDHI6_SD_SIZE 0x004c
+#define USDHI6_SD_OPTION 0x0050
+#define USDHI6_SD_ERR_STS1 0x0058
+#define USDHI6_SD_ERR_STS2 0x005c
+#define USDHI6_SD_BUF0 0x0060
+#define USDHI6_SDIO_MODE 0x0068
+#define USDHI6_SDIO_INFO1 0x006c
+#define USDHI6_SDIO_INFO1_MASK 0x0070
+#define USDHI6_CC_EXT_MODE 0x01b0
+#define USDHI6_SOFT_RST 0x01c0
+#define USDHI6_VERSION 0x01c4
+#define USDHI6_HOST_MODE 0x01c8
+#define USDHI6_SDIF_MODE 0x01cc
+
+#define USDHI6_SD_CMD_APP 0x0040
+#define USDHI6_SD_CMD_MODE_RSP_AUTO 0x0000
+#define USDHI6_SD_CMD_MODE_RSP_NONE 0x0300
+#define USDHI6_SD_CMD_MODE_RSP_R1 0x0400 /* Also R5, R6, R7 */
+#define USDHI6_SD_CMD_MODE_RSP_R1B 0x0500 /* R1b */
+#define USDHI6_SD_CMD_MODE_RSP_R2 0x0600
+#define USDHI6_SD_CMD_MODE_RSP_R3 0x0700 /* Also R4 */
+#define USDHI6_SD_CMD_DATA 0x0800
+#define USDHI6_SD_CMD_READ 0x1000
+#define USDHI6_SD_CMD_MULTI 0x2000
+#define USDHI6_SD_CMD_CMD12_AUTO_OFF 0x4000
+
+#define USDHI6_CC_EXT_MODE_SDRW BIT(1)
+
+#define USDHI6_SD_INFO1_RSP_END BIT(0)
+#define USDHI6_SD_INFO1_ACCESS_END BIT(2)
+#define USDHI6_SD_INFO1_CARD_OUT BIT(3)
+#define USDHI6_SD_INFO1_CARD_IN BIT(4)
+#define USDHI6_SD_INFO1_CD BIT(5)
+#define USDHI6_SD_INFO1_WP BIT(7)
+#define USDHI6_SD_INFO1_D3_CARD_OUT BIT(8)
+#define USDHI6_SD_INFO1_D3_CARD_IN BIT(9)
+
+#define USDHI6_SD_INFO2_CMD_ERR BIT(0)
+#define USDHI6_SD_INFO2_CRC_ERR BIT(1)
+#define USDHI6_SD_INFO2_END_ERR BIT(2)
+#define USDHI6_SD_INFO2_TOUT BIT(3)
+#define USDHI6_SD_INFO2_IWA_ERR BIT(4)
+#define USDHI6_SD_INFO2_IRA_ERR BIT(5)
+#define USDHI6_SD_INFO2_RSP_TOUT BIT(6)
+#define USDHI6_SD_INFO2_SDDAT0 BIT(7)
+#define USDHI6_SD_INFO2_BRE BIT(8)
+#define USDHI6_SD_INFO2_BWE BIT(9)
+#define USDHI6_SD_INFO2_SCLKDIVEN BIT(13)
+#define USDHI6_SD_INFO2_CBSY BIT(14)
+#define USDHI6_SD_INFO2_ILA BIT(15)
+
+#define USDHI6_SD_INFO1_CARD_INSERT (USDHI6_SD_INFO1_CARD_IN | USDHI6_SD_INFO1_D3_CARD_IN)
+#define USDHI6_SD_INFO1_CARD_EJECT (USDHI6_SD_INFO1_CARD_OUT | USDHI6_SD_INFO1_D3_CARD_OUT)
+#define USDHI6_SD_INFO1_CARD (USDHI6_SD_INFO1_CARD_INSERT | USDHI6_SD_INFO1_CARD_EJECT)
+#define USDHI6_SD_INFO1_CARD_CD (USDHI6_SD_INFO1_CARD_IN | USDHI6_SD_INFO1_CARD_OUT)
+
+#define USDHI6_SD_INFO2_ERR (USDHI6_SD_INFO2_CMD_ERR | \
+ USDHI6_SD_INFO2_CRC_ERR | USDHI6_SD_INFO2_END_ERR | \
+ USDHI6_SD_INFO2_TOUT | USDHI6_SD_INFO2_IWA_ERR | \
+ USDHI6_SD_INFO2_IRA_ERR | USDHI6_SD_INFO2_RSP_TOUT | \
+ USDHI6_SD_INFO2_ILA)
+
+#define USDHI6_SD_INFO1_IRQ (USDHI6_SD_INFO1_RSP_END | USDHI6_SD_INFO1_ACCESS_END | \
+ USDHI6_SD_INFO1_CARD)
+
+#define USDHI6_SD_INFO2_IRQ (USDHI6_SD_INFO2_ERR | USDHI6_SD_INFO2_BRE | \
+ USDHI6_SD_INFO2_BWE | 0x0800 | USDHI6_SD_INFO2_ILA)
+
+#define USDHI6_SD_CLK_CTRL_SCLKEN BIT(8)
+
+#define USDHI6_SD_STOP_STP BIT(0)
+#define USDHI6_SD_STOP_SEC BIT(8)
+
+#define USDHI6_SDIO_INFO1_IOIRQ BIT(0)
+#define USDHI6_SDIO_INFO1_EXPUB52 BIT(14)
+#define USDHI6_SDIO_INFO1_EXWT BIT(15)
+
+#define USDHI6_SD_ERR_STS1_CRC_NO_ERROR BIT(13)
+
+#define USDHI6_SOFT_RST_RESERVED (BIT(1) | BIT(2))
+#define USDHI6_SOFT_RST_RESET BIT(0)
+
+#define USDHI6_SD_OPTION_TIMEOUT_SHIFT 4
+#define USDHI6_SD_OPTION_TIMEOUT_MASK (0xf << USDHI6_SD_OPTION_TIMEOUT_SHIFT)
+#define USDHI6_SD_OPTION_WIDTH_1 BIT(15)
+
+#define USDHI6_SD_PORT_SEL_PORTS_SHIFT 8
+
+#define USDHI6_SD_CLK_CTRL_DIV_MASK 0xff
+
+#define USDHI6_SDIO_INFO1_IRQ (USDHI6_SDIO_INFO1_IOIRQ | 3 | \
+ USDHI6_SDIO_INFO1_EXPUB52 | USDHI6_SDIO_INFO1_EXWT)
+
+#define USDHI6_MIN_DMA 64
+
+enum usdhi6_wait_for {
+ USDHI6_WAIT_FOR_REQUEST,
+ USDHI6_WAIT_FOR_CMD,
+ USDHI6_WAIT_FOR_MREAD,
+ USDHI6_WAIT_FOR_MWRITE,
+ USDHI6_WAIT_FOR_READ,
+ USDHI6_WAIT_FOR_WRITE,
+ USDHI6_WAIT_FOR_DATA_END,
+ USDHI6_WAIT_FOR_STOP,
+ USDHI6_WAIT_FOR_DMA,
+};
+
+struct usdhi6_page {
+ struct page *page;
+ void *mapped; /* mapped page */
+};
+
+struct usdhi6_host {
+ struct mmc_host *mmc;
+ struct mmc_request *mrq;
+ void __iomem *base;
+ struct clk *clk;
+
+ /* SG memory handling */
+
+ /* Common for multiple and single block requests */
+ struct usdhi6_page pg; /* current page from an SG */
+ void *blk_page; /* either a mapped page, or the bounce buffer */
+ size_t offset; /* offset within a page, including sg->offset */
+
+ /* Blocks, crossing a page boundary */
+ size_t head_len;
+ struct usdhi6_page head_pg;
+
+ /* A bounce buffer for unaligned blocks or blocks, crossing a page boundary */
+ struct scatterlist bounce_sg;
+ u8 bounce_buf[512];
+
+ /* Multiple block requests only */
+ struct scatterlist *sg; /* current SG segment */
+ int page_idx; /* page index within an SG segment */
+
+ enum usdhi6_wait_for wait;
+ u32 status_mask;
+ u32 status2_mask;
+ u32 sdio_mask;
+ u32 io_error;
+ u32 irq_status;
+ unsigned long imclk;
+ unsigned long rate;
+ bool app_cmd;
+
+ /* Timeout handling */
+ struct delayed_work timeout_work;
+ unsigned long timeout;
+
+ /* DMA support */
+ struct dma_chan *chan_rx;
+ struct dma_chan *chan_tx;
+ bool dma_active;
+};
+
+/* I/O primitives */
+
+static void usdhi6_write(struct usdhi6_host *host, u32 reg, u32 data)
+{
+ iowrite32(data, host->base + reg);
+ dev_vdbg(mmc_dev(host->mmc), "%s(0x%p + 0x%x) = 0x%x\n", __func__,
+ host->base, reg, data);
+}
+
+static void usdhi6_write16(struct usdhi6_host *host, u32 reg, u16 data)
+{
+ iowrite16(data, host->base + reg);
+ dev_vdbg(mmc_dev(host->mmc), "%s(0x%p + 0x%x) = 0x%x\n", __func__,
+ host->base, reg, data);
+}
+
+static u32 usdhi6_read(struct usdhi6_host *host, u32 reg)
+{
+ u32 data = ioread32(host->base + reg);
+ dev_vdbg(mmc_dev(host->mmc), "%s(0x%p + 0x%x) = 0x%x\n", __func__,
+ host->base, reg, data);
+ return data;
+}
+
+static u16 usdhi6_read16(struct usdhi6_host *host, u32 reg)
+{
+ u16 data = ioread16(host->base + reg);
+ dev_vdbg(mmc_dev(host->mmc), "%s(0x%p + 0x%x) = 0x%x\n", __func__,
+ host->base, reg, data);
+ return data;
+}
+
+static void usdhi6_irq_enable(struct usdhi6_host *host, u32 info1, u32 info2)
+{
+ host->status_mask = USDHI6_SD_INFO1_IRQ & ~info1;
+ host->status2_mask = USDHI6_SD_INFO2_IRQ & ~info2;
+ usdhi6_write(host, USDHI6_SD_INFO1_MASK, host->status_mask);
+ usdhi6_write(host, USDHI6_SD_INFO2_MASK, host->status2_mask);
+}
+
+static void usdhi6_wait_for_resp(struct usdhi6_host *host)
+{
+ usdhi6_irq_enable(host, USDHI6_SD_INFO1_RSP_END |
+ USDHI6_SD_INFO1_ACCESS_END | USDHI6_SD_INFO1_CARD_CD,
+ USDHI6_SD_INFO2_ERR);
+}
+
+static void usdhi6_wait_for_brwe(struct usdhi6_host *host, bool read)
+{
+ usdhi6_irq_enable(host, USDHI6_SD_INFO1_ACCESS_END |
+ USDHI6_SD_INFO1_CARD_CD, USDHI6_SD_INFO2_ERR |
+ (read ? USDHI6_SD_INFO2_BRE : USDHI6_SD_INFO2_BWE));
+}
+
+static void usdhi6_only_cd(struct usdhi6_host *host)
+{
+ /* Mask all except card hotplug */
+ usdhi6_irq_enable(host, USDHI6_SD_INFO1_CARD_CD, 0);
+}
+
+static void usdhi6_mask_all(struct usdhi6_host *host)
+{
+ usdhi6_irq_enable(host, 0, 0);
+}
+
+static int usdhi6_error_code(struct usdhi6_host *host)
+{
+ u32 err;
+
+ usdhi6_write(host, USDHI6_SD_STOP, USDHI6_SD_STOP_STP);
+
+ if (host->io_error &
+ (USDHI6_SD_INFO2_RSP_TOUT | USDHI6_SD_INFO2_TOUT)) {
+ u32 rsp54 = usdhi6_read(host, USDHI6_SD_RSP54);
+ int opc = host->mrq ? host->mrq->cmd->opcode : -1;
+
+ err = usdhi6_read(host, USDHI6_SD_ERR_STS2);
+ /* Response timeout is often normal, don't spam the log */
+ if (host->wait == USDHI6_WAIT_FOR_CMD)
+ dev_dbg(mmc_dev(host->mmc),
+ "T-out sts 0x%x, resp 0x%x, state %u, CMD%d\n",
+ err, rsp54, host->wait, opc);
+ else
+ dev_warn(mmc_dev(host->mmc),
+ "T-out sts 0x%x, resp 0x%x, state %u, CMD%d\n",
+ err, rsp54, host->wait, opc);
+ return -ETIMEDOUT;
+ }
+
+ err = usdhi6_read(host, USDHI6_SD_ERR_STS1);
+ if (err != USDHI6_SD_ERR_STS1_CRC_NO_ERROR)
+ dev_warn(mmc_dev(host->mmc), "Err sts 0x%x, state %u, CMD%d\n",
+ err, host->wait, host->mrq ? host->mrq->cmd->opcode : -1);
+ if (host->io_error & USDHI6_SD_INFO2_ILA)
+ return -EILSEQ;
+
+ return -EIO;
+}
+
+/* Scatter-Gather management */
+
+/*
+ * In PIO mode we have to map each page separately, using kmap(). That way
+ * adjacent pages are mapped to non-adjacent virtual addresses. That's why we
+ * have to use a bounce buffer for blocks, crossing page boundaries. Such blocks
+ * have been observed with an SDIO WiFi card (b43 driver).
+ */
+static void usdhi6_blk_bounce(struct usdhi6_host *host,
+ struct scatterlist *sg)
+{
+ struct mmc_data *data = host->mrq->data;
+ size_t blk_head = host->head_len;
+
+ dev_dbg(mmc_dev(host->mmc), "%s(): CMD%u of %u SG: %ux%u @ 0x%x\n",
+ __func__, host->mrq->cmd->opcode, data->sg_len,
+ data->blksz, data->blocks, sg->offset);
+
+ host->head_pg.page = host->pg.page;
+ host->head_pg.mapped = host->pg.mapped;
+ host->pg.page = nth_page(host->pg.page, 1);
+ host->pg.mapped = kmap(host->pg.page);
+
+ host->blk_page = host->bounce_buf;
+ host->offset = 0;
+
+ if (data->flags & MMC_DATA_READ)
+ return;
+
+ memcpy(host->bounce_buf, host->head_pg.mapped + PAGE_SIZE - blk_head,
+ blk_head);
+ memcpy(host->bounce_buf + blk_head, host->pg.mapped,
+ data->blksz - blk_head);
+}
+
+/* Only called for multiple block IO */
+static void usdhi6_sg_prep(struct usdhi6_host *host)
+{
+ struct mmc_request *mrq = host->mrq;
+ struct mmc_data *data = mrq->data;
+
+ usdhi6_write(host, USDHI6_SD_SECCNT, data->blocks);
+
+ host->sg = data->sg;
+ /* TODO: if we always map, this is redundant */
+ host->offset = host->sg->offset;
+}
+
+/* Map the first page in an SG segment: common for multiple and single block IO */
+static void *usdhi6_sg_map(struct usdhi6_host *host)
+{
+ struct mmc_data *data = host->mrq->data;
+ struct scatterlist *sg = data->sg_len > 1 ? host->sg : data->sg;
+ size_t head = PAGE_SIZE - sg->offset;
+ size_t blk_head = head % data->blksz;
+
+ WARN(host->pg.page, "%p not properly unmapped!\n", host->pg.page);
+ if (WARN(sg_dma_len(sg) % data->blksz,
+ "SG size %zd isn't a multiple of block size %zd\n",
+ sg_dma_len(sg), data->blksz))
+ return NULL;
+
+ host->pg.page = sg_page(sg);
+ host->pg.mapped = kmap(host->pg.page);
+ host->offset = sg->offset;
+
+ /*
+ * Block size must be a power of 2 for multi-block transfers,
+ * therefore blk_head is equal for all pages in this SG
+ */
+ host->head_len = blk_head;
+
+ if (head < data->blksz)
+ /*
+ * The first block in the SG crosses a page boundary.
+ * Max blksz = 512, so blocks can only span 2 pages
+ */
+ usdhi6_blk_bounce(host, sg);
+ else
+ host->blk_page = host->pg.mapped;
+
+ dev_dbg(mmc_dev(host->mmc), "Mapped %p (%lx) at %p + %u for CMD%u @ 0x%p\n",
+ host->pg.page, page_to_pfn(host->pg.page), host->pg.mapped,
+ sg->offset, host->mrq->cmd->opcode, host->mrq);
+
+ return host->blk_page + host->offset;
+}
+
+/* Unmap the current page: common for multiple and single block IO */
+static void usdhi6_sg_unmap(struct usdhi6_host *host, bool force)
+{
+ struct mmc_data *data = host->mrq->data;
+ struct page *page = host->head_pg.page;
+
+ if (page) {
+ /* Previous block was cross-page boundary */
+ struct scatterlist *sg = data->sg_len > 1 ?
+ host->sg : data->sg;
+ size_t blk_head = host->head_len;
+
+ if (!data->error && data->flags & MMC_DATA_READ) {
+ memcpy(host->head_pg.mapped + PAGE_SIZE - blk_head,
+ host->bounce_buf, blk_head);
+ memcpy(host->pg.mapped, host->bounce_buf + blk_head,
+ data->blksz - blk_head);
+ }
+
+ flush_dcache_page(page);
+ kunmap(page);
+
+ host->head_pg.page = NULL;
+
+ if (!force && sg_dma_len(sg) + sg->offset >
+ (host->page_idx << PAGE_SHIFT) + data->blksz - blk_head)
+ /* More blocks in this SG, don't unmap the next page */
+ return;
+ }
+
+ page = host->pg.page;
+ if (!page)
+ return;
+
+ flush_dcache_page(page);
+ kunmap(page);
+
+ host->pg.page = NULL;
+}
+
+/* Called from MMC_WRITE_MULTIPLE_BLOCK or MMC_READ_MULTIPLE_BLOCK */
+static void usdhi6_sg_advance(struct usdhi6_host *host)
+{
+ struct mmc_data *data = host->mrq->data;
+ size_t done, total;
+
+ /* New offset: set at the end of the previous block */
+ if (host->head_pg.page) {
+ /* Finished a cross-page block, jump to the new page */
+ host->page_idx++;
+ host->offset = data->blksz - host->head_len;
+ host->blk_page = host->pg.mapped;
+ usdhi6_sg_unmap(host, false);
+ } else {
+ host->offset += data->blksz;
+ /* The completed block didn't cross a page boundary */
+ if (host->offset == PAGE_SIZE) {
+ /* If required, we'll map the page below */
+ host->offset = 0;
+ host->page_idx++;
+ }
+ }
+
+ /*
+ * Now host->blk_page + host->offset point at the end of our last block
+ * and host->page_idx is the index of the page, in which our new block
+ * is located, if any
+ */
+
+ done = (host->page_idx << PAGE_SHIFT) + host->offset;
+ total = host->sg->offset + sg_dma_len(host->sg);
+
+ dev_dbg(mmc_dev(host->mmc), "%s(): %zu of %zu @ %u\n", __func__,
+ done, total, host->offset);
+
+ if (done < total && host->offset) {
+ /* More blocks in this page */
+ if (host->offset + data->blksz > PAGE_SIZE)
+ /* We approached at a block, that spans 2 pages */
+ usdhi6_blk_bounce(host, host->sg);
+
+ return;
+ }
+
+ /* Finished current page or an SG segment */
+ usdhi6_sg_unmap(host, false);
+
+ if (done == total) {
+ /*
+ * End of an SG segment or the complete SG: jump to the next
+ * segment, we'll map it later in usdhi6_blk_read() or
+ * usdhi6_blk_write()
+ */
+ struct scatterlist *next = sg_next(host->sg);
+
+ host->page_idx = 0;
+
+ if (!next)
+ host->wait = USDHI6_WAIT_FOR_DATA_END;
+ host->sg = next;
+
+ if (WARN(next && sg_dma_len(next) % data->blksz,
+ "SG size %zd isn't a multiple of block size %zd\n",
+ sg_dma_len(next), data->blksz))
+ data->error = -EINVAL;
+
+ return;
+ }
+
+ /* We cannot get here after crossing a page border */
+
+ /* Next page in the same SG */
+ host->pg.page = nth_page(sg_page(host->sg), host->page_idx);
+ host->pg.mapped = kmap(host->pg.page);
+ host->blk_page = host->pg.mapped;
+
+ dev_dbg(mmc_dev(host->mmc), "Mapped %p (%lx) at %p for CMD%u @ 0x%p\n",
+ host->pg.page, page_to_pfn(host->pg.page), host->pg.mapped,
+ host->mrq->cmd->opcode, host->mrq);
+}
+
+/* DMA handling */
+
+static void usdhi6_dma_release(struct usdhi6_host *host)
+{
+ host->dma_active = false;
+ if (host->chan_tx) {
+ struct dma_chan *chan = host->chan_tx;
+ host->chan_tx = NULL;
+ dma_release_channel(chan);
+ }
+ if (host->chan_rx) {
+ struct dma_chan *chan = host->chan_rx;
+ host->chan_rx = NULL;
+ dma_release_channel(chan);
+ }
+}
+
+static void usdhi6_dma_stop_unmap(struct usdhi6_host *host)
+{
+ struct mmc_data *data = host->mrq->data;
+
+ if (!host->dma_active)
+ return;
+
+ usdhi6_write(host, USDHI6_CC_EXT_MODE, 0);
+ host->dma_active = false;
+
+ if (data->flags & MMC_DATA_READ)
+ dma_unmap_sg(host->chan_rx->device->dev, data->sg,
+ data->sg_len, DMA_FROM_DEVICE);
+ else
+ dma_unmap_sg(host->chan_tx->device->dev, data->sg,
+ data->sg_len, DMA_TO_DEVICE);
+}
+
+static void usdhi6_dma_complete(void *arg)
+{
+ struct usdhi6_host *host = arg;
+ struct mmc_request *mrq = host->mrq;
+
+ if (WARN(!mrq || !mrq->data, "%s: NULL data in DMA completion for %p!\n",
+ dev_name(mmc_dev(host->mmc)), mrq))
+ return;
+
+ dev_dbg(mmc_dev(host->mmc), "%s(): CMD%u DMA completed\n", __func__,
+ mrq->cmd->opcode);
+
+ usdhi6_dma_stop_unmap(host);
+ usdhi6_wait_for_brwe(host, mrq->data->flags & MMC_DATA_READ);
+}
+
+static int usdhi6_dma_setup(struct usdhi6_host *host, struct dma_chan *chan,
+ enum dma_transfer_direction dir)
+{
+ struct mmc_data *data = host->mrq->data;
+ struct scatterlist *sg = data->sg;
+ struct dma_async_tx_descriptor *desc = NULL;
+ dma_cookie_t cookie = -EINVAL;
+ enum dma_data_direction data_dir;
+ int ret;
+
+ switch (dir) {
+ case DMA_MEM_TO_DEV:
+ data_dir = DMA_TO_DEVICE;
+ break;
+ case DMA_DEV_TO_MEM:
+ data_dir = DMA_FROM_DEVICE;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ ret = dma_map_sg(chan->device->dev, sg, data->sg_len, data_dir);
+ if (ret > 0) {
+ host->dma_active = true;
+ desc = dmaengine_prep_slave_sg(chan, sg, ret, dir,
+ DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
+ }
+
+ if (desc) {
+ desc->callback = usdhi6_dma_complete;
+ desc->callback_param = host;
+ cookie = dmaengine_submit(desc);
+ }
+
+ dev_dbg(mmc_dev(host->mmc), "%s(): mapped %d -> %d, cookie %d @ %p\n",
+ __func__, data->sg_len, ret, cookie, desc);
+
+ if (cookie < 0) {
+ /* DMA failed, fall back to PIO */
+ if (ret >= 0)
+ ret = cookie;
+ usdhi6_dma_release(host);
+ dev_warn(mmc_dev(host->mmc),
+ "DMA failed: %d, falling back to PIO\n", ret);
+ }
+
+ return cookie;
+}
+
+static int usdhi6_dma_start(struct usdhi6_host *host)
+{
+ if (!host->chan_rx || !host->chan_tx)
+ return -ENODEV;
+
+ if (host->mrq->data->flags & MMC_DATA_READ)
+ return usdhi6_dma_setup(host, host->chan_rx, DMA_DEV_TO_MEM);
+
+ return usdhi6_dma_setup(host, host->chan_tx, DMA_MEM_TO_DEV);
+}
+
+static void usdhi6_dma_kill(struct usdhi6_host *host)
+{
+ struct mmc_data *data = host->mrq->data;
+
+ dev_dbg(mmc_dev(host->mmc), "%s(): SG of %u: %ux%u\n",
+ __func__, data->sg_len, data->blocks, data->blksz);
+ /* Abort DMA */
+ if (data->flags & MMC_DATA_READ)
+ dmaengine_terminate_all(host->chan_rx);
+ else
+ dmaengine_terminate_all(host->chan_tx);
+}
+
+static void usdhi6_dma_check_error(struct usdhi6_host *host)
+{
+ struct mmc_data *data = host->mrq->data;
+
+ dev_dbg(mmc_dev(host->mmc), "%s(): IO error %d, status 0x%x\n",
+ __func__, host->io_error, usdhi6_read(host, USDHI6_SD_INFO1));
+
+ if (host->io_error) {
+ data->error = usdhi6_error_code(host);
+ data->bytes_xfered = 0;
+ usdhi6_dma_kill(host);
+ usdhi6_dma_release(host);
+ dev_warn(mmc_dev(host->mmc),
+ "DMA failed: %d, falling back to PIO\n", data->error);
+ return;
+ }
+
+ /*
+ * The datasheet tells us to check a response from the card, whereas
+ * responses only come after the command phase, not after the data
+ * phase. Let's check anyway.
+ */
+ if (host->irq_status & USDHI6_SD_INFO1_RSP_END)
+ dev_warn(mmc_dev(host->mmc), "Unexpected response received!\n");
+}
+
+static void usdhi6_dma_kick(struct usdhi6_host *host)
+{
+ if (host->mrq->data->flags & MMC_DATA_READ)
+ dma_async_issue_pending(host->chan_rx);
+ else
+ dma_async_issue_pending(host->chan_tx);
+}
+
+static void usdhi6_dma_request(struct usdhi6_host *host, phys_addr_t start)
+{
+ struct dma_slave_config cfg = {
+ .src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES,
+ .dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES,
+ };
+ int ret;
+
+ host->chan_tx = dma_request_slave_channel(mmc_dev(host->mmc), "tx");
+ dev_dbg(mmc_dev(host->mmc), "%s: TX: got channel %p\n", __func__,
+ host->chan_tx);
+
+ if (!host->chan_tx)
+ return;
+
+ cfg.direction = DMA_MEM_TO_DEV;
+ cfg.dst_addr = start + USDHI6_SD_BUF0;
+ cfg.dst_maxburst = 128; /* 128 words * 4 bytes = 512 bytes */
+ cfg.src_addr = 0;
+ ret = dmaengine_slave_config(host->chan_tx, &cfg);
+ if (ret < 0)
+ goto e_release_tx;
+
+ host->chan_rx = dma_request_slave_channel(mmc_dev(host->mmc), "rx");
+ dev_dbg(mmc_dev(host->mmc), "%s: RX: got channel %p\n", __func__,
+ host->chan_rx);
+
+ if (!host->chan_rx)
+ goto e_release_tx;
+
+ cfg.direction = DMA_DEV_TO_MEM;
+ cfg.src_addr = cfg.dst_addr;
+ cfg.src_maxburst = 128; /* 128 words * 4 bytes = 512 bytes */
+ cfg.dst_addr = 0;
+ ret = dmaengine_slave_config(host->chan_rx, &cfg);
+ if (ret < 0)
+ goto e_release_rx;
+
+ return;
+
+e_release_rx:
+ dma_release_channel(host->chan_rx);
+ host->chan_rx = NULL;
+e_release_tx:
+ dma_release_channel(host->chan_tx);
+ host->chan_tx = NULL;
+}
+
+/* API helpers */
+
+static void usdhi6_clk_set(struct usdhi6_host *host, struct mmc_ios *ios)
+{
+ unsigned long rate = ios->clock;
+ u32 val;
+ unsigned int i;
+
+ for (i = 1000; i; i--) {
+ if (usdhi6_read(host, USDHI6_SD_INFO2) & USDHI6_SD_INFO2_SCLKDIVEN)
+ break;
+ usleep_range(10, 100);
+ }
+
+ if (!i) {
+ dev_err(mmc_dev(host->mmc), "SD bus busy, clock set aborted\n");
+ return;
+ }
+
+ val = usdhi6_read(host, USDHI6_SD_CLK_CTRL) & ~USDHI6_SD_CLK_CTRL_DIV_MASK;
+
+ if (rate) {
+ unsigned long new_rate;
+
+ if (host->imclk <= rate) {
+ if (ios->timing != MMC_TIMING_UHS_DDR50) {
+ /* Cannot have 1-to-1 clock in DDR mode */
+ new_rate = host->imclk;
+ val |= 0xff;
+ } else {
+ new_rate = host->imclk / 2;
+ }
+ } else {
+ unsigned long div =
+ roundup_pow_of_two(DIV_ROUND_UP(host->imclk, rate));
+ val |= div >> 2;
+ new_rate = host->imclk / div;
+ }
+
+ if (host->rate == new_rate)
+ return;
+
+ host->rate = new_rate;
+
+ dev_dbg(mmc_dev(host->mmc), "target %lu, div %u, set %lu\n",
+ rate, (val & 0xff) << 2, new_rate);
+ }
+
+ /*
+ * if old or new rate is equal to input rate, have to switch the clock
+ * off before changing and on after
+ */
+ if (host->imclk == rate || host->imclk == host->rate || !rate)
+ usdhi6_write(host, USDHI6_SD_CLK_CTRL,
+ val & ~USDHI6_SD_CLK_CTRL_SCLKEN);
+
+ if (!rate) {
+ host->rate = 0;
+ return;
+ }
+
+ usdhi6_write(host, USDHI6_SD_CLK_CTRL, val);
+
+ if (host->imclk == rate || host->imclk == host->rate ||
+ !(val & USDHI6_SD_CLK_CTRL_SCLKEN))
+ usdhi6_write(host, USDHI6_SD_CLK_CTRL,
+ val | USDHI6_SD_CLK_CTRL_SCLKEN);
+}
+
+static void usdhi6_set_power(struct usdhi6_host *host, struct mmc_ios *ios)
+{
+ struct mmc_host *mmc = host->mmc;
+
+ if (!IS_ERR(mmc->supply.vmmc))
+ /* Errors ignored... */
+ mmc_regulator_set_ocr(mmc, mmc->supply.vmmc,
+ ios->power_mode ? ios->vdd : 0);
+}
+
+static int usdhi6_reset(struct usdhi6_host *host)
+{
+ int i;
+
+ usdhi6_write(host, USDHI6_SOFT_RST, USDHI6_SOFT_RST_RESERVED);
+ cpu_relax();
+ usdhi6_write(host, USDHI6_SOFT_RST, USDHI6_SOFT_RST_RESERVED | USDHI6_SOFT_RST_RESET);
+ for (i = 1000; i; i--)
+ if (usdhi6_read(host, USDHI6_SOFT_RST) & USDHI6_SOFT_RST_RESET)
+ break;
+
+ return i ? 0 : -ETIMEDOUT;
+}
+
+static void usdhi6_set_ios(struct mmc_host *mmc, struct mmc_ios *ios)
+{
+ struct usdhi6_host *host = mmc_priv(mmc);
+ u32 option, mode;
+ int ret;
+
+ dev_dbg(mmc_dev(mmc), "%uHz, OCR: %u, power %u, bus-width %u, timing %u\n",
+ ios->clock, ios->vdd, ios->power_mode, ios->bus_width, ios->timing);
+
+ switch (ios->power_mode) {
+ case MMC_POWER_OFF:
+ usdhi6_set_power(host, ios);
+ usdhi6_only_cd(host);
+ break;
+ case MMC_POWER_UP:
+ /*
+ * We only also touch USDHI6_SD_OPTION from .request(), which
+ * cannot race with MMC_POWER_UP
+ */
+ ret = usdhi6_reset(host);
+ if (ret < 0) {
+ dev_err(mmc_dev(mmc), "Cannot reset the interface!\n");
+ } else {
+ usdhi6_set_power(host, ios);
+ usdhi6_only_cd(host);
+ }
+ break;
+ case MMC_POWER_ON:
+ option = usdhi6_read(host, USDHI6_SD_OPTION);
+ /*
+ * The eMMC standard only allows 4 or 8 bits in the DDR mode,
+ * the same probably holds for SD cards. We check here anyway,
+ * since the datasheet explicitly requires 4 bits for DDR.
+ */
+ if (ios->bus_width == MMC_BUS_WIDTH_1) {
+ if (ios->timing == MMC_TIMING_UHS_DDR50)
+ dev_err(mmc_dev(mmc),
+ "4 bits are required for DDR\n");
+ option |= USDHI6_SD_OPTION_WIDTH_1;
+ mode = 0;
+ } else {
+ option &= ~USDHI6_SD_OPTION_WIDTH_1;
+ mode = ios->timing == MMC_TIMING_UHS_DDR50;
+ }
+ usdhi6_write(host, USDHI6_SD_OPTION, option);
+ usdhi6_write(host, USDHI6_SDIF_MODE, mode);
+ break;
+ }
+
+ if (host->rate != ios->clock)
+ usdhi6_clk_set(host, ios);
+}
+
+/* This is data timeout. Response timeout is fixed to 640 clock cycles */
+static void usdhi6_timeout_set(struct usdhi6_host *host)
+{
+ struct mmc_request *mrq = host->mrq;
+ u32 val;
+ unsigned long ticks;
+
+ if (!mrq->data)
+ ticks = host->rate / 1000 * mrq->cmd->busy_timeout;
+ else
+ ticks = host->rate / 1000000 * (mrq->data->timeout_ns / 1000) +
+ mrq->data->timeout_clks;
+
+ if (!ticks || ticks > 1 << 27)
+ /* Max timeout */
+ val = 14;
+ else if (ticks < 1 << 13)
+ /* Min timeout */
+ val = 0;
+ else
+ val = order_base_2(ticks) - 13;
+
+ dev_dbg(mmc_dev(host->mmc), "Set %s timeout %lu ticks @ %lu Hz\n",
+ mrq->data ? "data" : "cmd", ticks, host->rate);
+
+ /* Timeout Counter mask: 0xf0 */
+ usdhi6_write(host, USDHI6_SD_OPTION, (val << USDHI6_SD_OPTION_TIMEOUT_SHIFT) |
+ (usdhi6_read(host, USDHI6_SD_OPTION) & ~USDHI6_SD_OPTION_TIMEOUT_MASK));
+}
+
+static void usdhi6_request_done(struct usdhi6_host *host)
+{
+ struct mmc_request *mrq = host->mrq;
+ struct mmc_data *data = mrq->data;
+
+ if (WARN(host->pg.page || host->head_pg.page,
+ "Page %p or %p not unmapped: wait %u, CMD%d(%c) @ +0x%x %ux%u in SG%u!\n",
+ host->pg.page, host->head_pg.page, host->wait, mrq->cmd->opcode,
+ data ? (data->flags & MMC_DATA_READ ? 'R' : 'W') : '-',
+ data ? host->offset : 0, data ? data->blocks : 0,
+ data ? data->blksz : 0, data ? data->sg_len : 0))
+ usdhi6_sg_unmap(host, true);
+
+ if (mrq->cmd->error ||
+ (data && data->error) ||
+ (mrq->stop && mrq->stop->error))
+ dev_dbg(mmc_dev(host->mmc), "%s(CMD%d: %ux%u): err %d %d %d\n",
+ __func__, mrq->cmd->opcode, data ? data->blocks : 0,
+ data ? data->blksz : 0,
+ mrq->cmd->error,
+ data ? data->error : 1,
+ mrq->stop ? mrq->stop->error : 1);
+
+ /* Disable DMA */
+ usdhi6_write(host, USDHI6_CC_EXT_MODE, 0);
+ host->wait = USDHI6_WAIT_FOR_REQUEST;
+ host->mrq = NULL;
+
+ mmc_request_done(host->mmc, mrq);
+}
+
+static int usdhi6_cmd_flags(struct usdhi6_host *host)
+{
+ struct mmc_request *mrq = host->mrq;
+ struct mmc_command *cmd = mrq->cmd;
+ u16 opc = cmd->opcode;
+
+ if (host->app_cmd) {
+ host->app_cmd = false;
+ opc |= USDHI6_SD_CMD_APP;
+ }
+
+ if (mrq->data) {
+ opc |= USDHI6_SD_CMD_DATA;
+
+ if (mrq->data->flags & MMC_DATA_READ)
+ opc |= USDHI6_SD_CMD_READ;
+
+ if (cmd->opcode == MMC_READ_MULTIPLE_BLOCK ||
+ cmd->opcode == MMC_WRITE_MULTIPLE_BLOCK ||
+ (cmd->opcode == SD_IO_RW_EXTENDED &&
+ mrq->data->blocks > 1)) {
+ opc |= USDHI6_SD_CMD_MULTI;
+ if (!mrq->stop)
+ opc |= USDHI6_SD_CMD_CMD12_AUTO_OFF;
+ }
+
+ switch (mmc_resp_type(cmd)) {
+ case MMC_RSP_NONE:
+ opc |= USDHI6_SD_CMD_MODE_RSP_NONE;
+ break;
+ case MMC_RSP_R1:
+ opc |= USDHI6_SD_CMD_MODE_RSP_R1;
+ break;
+ case MMC_RSP_R1B:
+ opc |= USDHI6_SD_CMD_MODE_RSP_R1B;
+ break;
+ case MMC_RSP_R2:
+ opc |= USDHI6_SD_CMD_MODE_RSP_R2;
+ break;
+ case MMC_RSP_R3:
+ opc |= USDHI6_SD_CMD_MODE_RSP_R3;
+ break;
+ default:
+ dev_warn(mmc_dev(host->mmc),
+ "Unknown response type %d\n",
+ mmc_resp_type(cmd));
+ return -EINVAL;
+ }
+ }
+
+ return opc;
+}
+
+static int usdhi6_rq_start(struct usdhi6_host *host)
+{
+ struct mmc_request *mrq = host->mrq;
+ struct mmc_command *cmd = mrq->cmd;
+ struct mmc_data *data = mrq->data;
+ int opc = usdhi6_cmd_flags(host);
+ int i;
+
+ if (opc < 0)
+ return opc;
+
+ for (i = 1000; i; i--) {
+ if (!(usdhi6_read(host, USDHI6_SD_INFO2) & USDHI6_SD_INFO2_CBSY))
+ break;
+ usleep_range(10, 100);
+ }
+
+ if (!i) {
+ dev_dbg(mmc_dev(host->mmc), "Command active, request aborted\n");
+ return -EAGAIN;
+ }
+
+ if (data) {
+ bool use_dma;
+ int ret = 0;
+
+ host->page_idx = 0;
+
+ if (cmd->opcode == SD_IO_RW_EXTENDED && data->blocks > 1) {
+ switch (data->blksz) {
+ case 512:
+ break;
+ case 32:
+ case 64:
+ case 128:
+ case 256:
+ if (mrq->stop)
+ ret = -EINVAL;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ } else if ((cmd->opcode == MMC_READ_MULTIPLE_BLOCK ||
+ cmd->opcode == MMC_WRITE_MULTIPLE_BLOCK) &&
+ data->blksz != 512) {
+ ret = -EINVAL;
+ }
+
+ if (ret < 0) {
+ dev_warn(mmc_dev(host->mmc), "%s(): %u blocks of %u bytes\n",
+ __func__, data->blocks, data->blksz);
+ return -EINVAL;
+ }
+
+ if (cmd->opcode == MMC_READ_MULTIPLE_BLOCK ||
+ cmd->opcode == MMC_WRITE_MULTIPLE_BLOCK ||
+ (cmd->opcode == SD_IO_RW_EXTENDED &&
+ data->blocks > 1))
+ usdhi6_sg_prep(host);
+
+ usdhi6_write(host, USDHI6_SD_SIZE, data->blksz);
+
+ if ((data->blksz >= USDHI6_MIN_DMA ||
+ data->blocks > 1) &&
+ (data->blksz % 4 ||
+ data->sg->offset % 4))
+ dev_dbg(mmc_dev(host->mmc),
+ "Bad SG of %u: %ux%u @ %u\n", data->sg_len,
+ data->blksz, data->blocks, data->sg->offset);
+
+ /* Enable DMA for USDHI6_MIN_DMA bytes or more */
+ use_dma = data->blksz >= USDHI6_MIN_DMA &&
+ !(data->blksz % 4) &&
+ usdhi6_dma_start(host) >= DMA_MIN_COOKIE;
+
+ if (use_dma)
+ usdhi6_write(host, USDHI6_CC_EXT_MODE, USDHI6_CC_EXT_MODE_SDRW);
+
+ dev_dbg(mmc_dev(host->mmc),
+ "%s(): request opcode %u, %u blocks of %u bytes in %u segments, %s %s @+0x%x%s\n",
+ __func__, cmd->opcode, data->blocks, data->blksz,
+ data->sg_len, use_dma ? "DMA" : "PIO",
+ data->flags & MMC_DATA_READ ? "read" : "write",
+ data->sg->offset, mrq->stop ? " + stop" : "");
+ } else {
+ dev_dbg(mmc_dev(host->mmc), "%s(): request opcode %u\n",
+ __func__, cmd->opcode);
+ }
+
+ /* We have to get a command completion interrupt with DMA too */
+ usdhi6_wait_for_resp(host);
+
+ host->wait = USDHI6_WAIT_FOR_CMD;
+ schedule_delayed_work(&host->timeout_work, host->timeout);
+
+ /* SEC bit is required to enable block counting by the core */
+ usdhi6_write(host, USDHI6_SD_STOP,
+ data && data->blocks > 1 ? USDHI6_SD_STOP_SEC : 0);
+ usdhi6_write(host, USDHI6_SD_ARG, cmd->arg);
+
+ /* Kick command execution */
+ usdhi6_write(host, USDHI6_SD_CMD, opc);
+
+ return 0;
+}
+
+static void usdhi6_request(struct mmc_host *mmc, struct mmc_request *mrq)
+{
+ struct usdhi6_host *host = mmc_priv(mmc);
+ int ret;
+
+ cancel_delayed_work_sync(&host->timeout_work);
+
+ host->mrq = mrq;
+ host->sg = NULL;
+
+ usdhi6_timeout_set(host);
+ ret = usdhi6_rq_start(host);
+ if (ret < 0) {
+ mrq->cmd->error = ret;
+ usdhi6_request_done(host);
+ }
+}
+
+static int usdhi6_get_cd(struct mmc_host *mmc)
+{
+ struct usdhi6_host *host = mmc_priv(mmc);
+ /* Read is atomic, no need to lock */
+ u32 status = usdhi6_read(host, USDHI6_SD_INFO1) & USDHI6_SD_INFO1_CD;
+
+/*
+ * level status.CD CD_ACTIVE_HIGH card present
+ * 1 0 0 0
+ * 1 0 1 1
+ * 0 1 0 1
+ * 0 1 1 0
+ */
+ return !status ^ !(mmc->caps2 & MMC_CAP2_CD_ACTIVE_HIGH);
+}
+
+static int usdhi6_get_ro(struct mmc_host *mmc)
+{
+ struct usdhi6_host *host = mmc_priv(mmc);
+ /* No locking as above */
+ u32 status = usdhi6_read(host, USDHI6_SD_INFO1) & USDHI6_SD_INFO1_WP;
+
+/*
+ * level status.WP RO_ACTIVE_HIGH card read-only
+ * 1 0 0 0
+ * 1 0 1 1
+ * 0 1 0 1
+ * 0 1 1 0
+ */
+ return !status ^ !(mmc->caps2 & MMC_CAP2_RO_ACTIVE_HIGH);
+}
+
+static void usdhi6_enable_sdio_irq(struct mmc_host *mmc, int enable)
+{
+ struct usdhi6_host *host = mmc_priv(mmc);
+
+ dev_dbg(mmc_dev(mmc), "%s(): %sable\n", __func__, enable ? "en" : "dis");
+
+ if (enable) {
+ host->sdio_mask = USDHI6_SDIO_INFO1_IRQ & ~USDHI6_SDIO_INFO1_IOIRQ;
+ usdhi6_write(host, USDHI6_SDIO_INFO1_MASK, host->sdio_mask);
+ usdhi6_write(host, USDHI6_SDIO_MODE, 1);
+ } else {
+ usdhi6_write(host, USDHI6_SDIO_MODE, 0);
+ usdhi6_write(host, USDHI6_SDIO_INFO1_MASK, USDHI6_SDIO_INFO1_IRQ);
+ host->sdio_mask = USDHI6_SDIO_INFO1_IRQ;
+ }
+}
+
+static struct mmc_host_ops usdhi6_ops = {
+ .request = usdhi6_request,
+ .set_ios = usdhi6_set_ios,
+ .get_cd = usdhi6_get_cd,
+ .get_ro = usdhi6_get_ro,
+ .enable_sdio_irq = usdhi6_enable_sdio_irq,
+};
+
+/* State machine handlers */
+
+static void usdhi6_resp_cmd12(struct usdhi6_host *host)
+{
+ struct mmc_command *cmd = host->mrq->stop;
+ cmd->resp[0] = usdhi6_read(host, USDHI6_SD_RSP10);
+}
+
+static void usdhi6_resp_read(struct usdhi6_host *host)
+{
+ struct mmc_command *cmd = host->mrq->cmd;
+ u32 *rsp = cmd->resp, tmp = 0;
+ int i;
+
+/*
+ * RSP10 39-8
+ * RSP32 71-40
+ * RSP54 103-72
+ * RSP76 127-104
+ * R2-type response:
+ * resp[0] = r[127..96]
+ * resp[1] = r[95..64]
+ * resp[2] = r[63..32]
+ * resp[3] = r[31..0]
+ * Other responses:
+ * resp[0] = r[39..8]
+ */
+
+ if (mmc_resp_type(cmd) == MMC_RSP_NONE)
+ return;
+
+ if (!(host->irq_status & USDHI6_SD_INFO1_RSP_END)) {
+ dev_err(mmc_dev(host->mmc),
+ "CMD%d: response expected but is missing!\n", cmd->opcode);
+ return;
+ }
+
+ if (mmc_resp_type(cmd) & MMC_RSP_136)
+ for (i = 0; i < 4; i++) {
+ if (i)
+ rsp[3 - i] = tmp >> 24;
+ tmp = usdhi6_read(host, USDHI6_SD_RSP10 + i * 8);
+ rsp[3 - i] |= tmp << 8;
+ }
+ else if (cmd->opcode == MMC_READ_MULTIPLE_BLOCK ||
+ cmd->opcode == MMC_WRITE_MULTIPLE_BLOCK)
+ /* Read RSP54 to avoid conflict with auto CMD12 */
+ rsp[0] = usdhi6_read(host, USDHI6_SD_RSP54);
+ else
+ rsp[0] = usdhi6_read(host, USDHI6_SD_RSP10);
+
+ dev_dbg(mmc_dev(host->mmc), "Response 0x%x\n", rsp[0]);
+}
+
+static int usdhi6_blk_read(struct usdhi6_host *host)
+{
+ struct mmc_data *data = host->mrq->data;
+ u32 *p;
+ int i, rest;
+
+ if (host->io_error) {
+ data->error = usdhi6_error_code(host);
+ goto error;
+ }
+
+ if (host->pg.page) {
+ p = host->blk_page + host->offset;
+ } else {
+ p = usdhi6_sg_map(host);
+ if (!p) {
+ data->error = -ENOMEM;
+ goto error;
+ }
+ }
+
+ for (i = 0; i < data->blksz / 4; i++, p++)
+ *p = usdhi6_read(host, USDHI6_SD_BUF0);
+
+ rest = data->blksz % 4;
+ for (i = 0; i < (rest + 1) / 2; i++) {
+ u16 d = usdhi6_read16(host, USDHI6_SD_BUF0);
+ ((u8 *)p)[2 * i] = ((u8 *)&d)[0];
+ if (rest > 1 && !i)
+ ((u8 *)p)[2 * i + 1] = ((u8 *)&d)[1];
+ }
+
+ return 0;
+
+error:
+ dev_dbg(mmc_dev(host->mmc), "%s(): %d\n", __func__, data->error);
+ host->wait = USDHI6_WAIT_FOR_REQUEST;
+ return data->error;
+}
+
+static int usdhi6_blk_write(struct usdhi6_host *host)
+{
+ struct mmc_data *data = host->mrq->data;
+ u32 *p;
+ int i, rest;
+
+ if (host->io_error) {
+ data->error = usdhi6_error_code(host);
+ goto error;
+ }
+
+ if (host->pg.page) {
+ p = host->blk_page + host->offset;
+ } else {
+ p = usdhi6_sg_map(host);
+ if (!p) {
+ data->error = -ENOMEM;
+ goto error;
+ }
+ }
+
+ for (i = 0; i < data->blksz / 4; i++, p++)
+ usdhi6_write(host, USDHI6_SD_BUF0, *p);
+
+ rest = data->blksz % 4;
+ for (i = 0; i < (rest + 1) / 2; i++) {
+ u16 d;
+ ((u8 *)&d)[0] = ((u8 *)p)[2 * i];
+ if (rest > 1 && !i)
+ ((u8 *)&d)[1] = ((u8 *)p)[2 * i + 1];
+ else
+ ((u8 *)&d)[1] = 0;
+ usdhi6_write16(host, USDHI6_SD_BUF0, d);
+ }
+
+ return 0;
+
+error:
+ dev_dbg(mmc_dev(host->mmc), "%s(): %d\n", __func__, data->error);
+ host->wait = USDHI6_WAIT_FOR_REQUEST;
+ return data->error;
+}
+
+static int usdhi6_stop_cmd(struct usdhi6_host *host)
+{
+ struct mmc_request *mrq = host->mrq;
+
+ switch (mrq->cmd->opcode) {
+ case MMC_READ_MULTIPLE_BLOCK:
+ case MMC_WRITE_MULTIPLE_BLOCK:
+ if (mrq->stop->opcode == MMC_STOP_TRANSMISSION) {
+ host->wait = USDHI6_WAIT_FOR_STOP;
+ return 0;
+ }
+ /* Unsupported STOP command */
+ default:
+ dev_err(mmc_dev(host->mmc),
+ "unsupported stop CMD%d for CMD%d\n",
+ mrq->stop->opcode, mrq->cmd->opcode);
+ mrq->stop->error = -EOPNOTSUPP;
+ }
+
+ return -EOPNOTSUPP;
+}
+
+static bool usdhi6_end_cmd(struct usdhi6_host *host)
+{
+ struct mmc_request *mrq = host->mrq;
+ struct mmc_command *cmd = mrq->cmd;
+
+ if (host->io_error) {
+ cmd->error = usdhi6_error_code(host);
+ return false;
+ }
+
+ usdhi6_resp_read(host);
+
+ if (!mrq->data)
+ return false;
+
+ if (host->dma_active) {
+ usdhi6_dma_kick(host);
+ if (!mrq->stop)
+ host->wait = USDHI6_WAIT_FOR_DMA;
+ else if (usdhi6_stop_cmd(host) < 0)
+ return false;
+ } else if (mrq->data->flags & MMC_DATA_READ) {
+ if (cmd->opcode == MMC_READ_MULTIPLE_BLOCK ||
+ (cmd->opcode == SD_IO_RW_EXTENDED &&
+ mrq->data->blocks > 1))
+ host->wait = USDHI6_WAIT_FOR_MREAD;
+ else
+ host->wait = USDHI6_WAIT_FOR_READ;
+ } else {
+ if (cmd->opcode == MMC_WRITE_MULTIPLE_BLOCK ||
+ (cmd->opcode == SD_IO_RW_EXTENDED &&
+ mrq->data->blocks > 1))
+ host->wait = USDHI6_WAIT_FOR_MWRITE;
+ else
+ host->wait = USDHI6_WAIT_FOR_WRITE;
+ }
+
+ return true;
+}
+
+static bool usdhi6_read_block(struct usdhi6_host *host)
+{
+ /* ACCESS_END IRQ is already unmasked */
+ int ret = usdhi6_blk_read(host);
+
+ /*
+ * Have to force unmapping both pages: the single block could have been
+ * cross-page, in which case for single-block IO host->page_idx == 0.
+ * So, if we don't force, the second page won't be unmapped.
+ */
+ usdhi6_sg_unmap(host, true);
+
+ if (ret < 0)
+ return false;
+
+ host->wait = USDHI6_WAIT_FOR_DATA_END;
+ return true;
+}
+
+static bool usdhi6_mread_block(struct usdhi6_host *host)
+{
+ int ret = usdhi6_blk_read(host);
+
+ if (ret < 0)
+ return false;
+
+ usdhi6_sg_advance(host);
+
+ return !host->mrq->data->error &&
+ (host->wait != USDHI6_WAIT_FOR_DATA_END || !host->mrq->stop);
+}
+
+static bool usdhi6_write_block(struct usdhi6_host *host)
+{
+ int ret = usdhi6_blk_write(host);
+
+ /* See comment in usdhi6_read_block() */
+ usdhi6_sg_unmap(host, true);
+
+ if (ret < 0)
+ return false;
+
+ host->wait = USDHI6_WAIT_FOR_DATA_END;
+ return true;
+}
+
+static bool usdhi6_mwrite_block(struct usdhi6_host *host)
+{
+ int ret = usdhi6_blk_write(host);
+
+ if (ret < 0)
+ return false;
+
+ usdhi6_sg_advance(host);
+
+ return !host->mrq->data->error &&
+ (host->wait != USDHI6_WAIT_FOR_DATA_END || !host->mrq->stop);
+}
+
+/* Interrupt & timeout handlers */
+
+static irqreturn_t usdhi6_sd_bh(int irq, void *dev_id)
+{
+ struct usdhi6_host *host = dev_id;
+ struct mmc_request *mrq;
+ struct mmc_command *cmd;
+ struct mmc_data *data;
+ bool io_wait = false;
+
+ cancel_delayed_work_sync(&host->timeout_work);
+
+ mrq = host->mrq;
+ if (!mrq)
+ return IRQ_HANDLED;
+
+ cmd = mrq->cmd;
+ data = mrq->data;
+
+ switch (host->wait) {
+ case USDHI6_WAIT_FOR_REQUEST:
+ /* We're too late, the timeout has already kicked in */
+ return IRQ_HANDLED;
+ case USDHI6_WAIT_FOR_CMD:
+ /* Wait for data? */
+ io_wait = usdhi6_end_cmd(host);
+ break;
+ case USDHI6_WAIT_FOR_MREAD:
+ /* Wait for more data? */
+ io_wait = usdhi6_mread_block(host);
+ break;
+ case USDHI6_WAIT_FOR_READ:
+ /* Wait for data end? */
+ io_wait = usdhi6_read_block(host);
+ break;
+ case USDHI6_WAIT_FOR_MWRITE:
+ /* Wait data to write? */
+ io_wait = usdhi6_mwrite_block(host);
+ break;
+ case USDHI6_WAIT_FOR_WRITE:
+ /* Wait for data end? */
+ io_wait = usdhi6_write_block(host);
+ break;
+ case USDHI6_WAIT_FOR_DMA:
+ usdhi6_dma_check_error(host);
+ break;
+ case USDHI6_WAIT_FOR_STOP:
+ usdhi6_write(host, USDHI6_SD_STOP, 0);
+ if (host->io_error) {
+ int ret = usdhi6_error_code(host);
+ if (mrq->stop)
+ mrq->stop->error = ret;
+ else
+ mrq->data->error = ret;
+ dev_warn(mmc_dev(host->mmc), "%s(): %d\n", __func__, ret);
+ break;
+ }
+ usdhi6_resp_cmd12(host);
+ mrq->stop->error = 0;
+ break;
+ case USDHI6_WAIT_FOR_DATA_END:
+ if (host->io_error) {
+ mrq->data->error = usdhi6_error_code(host);
+ dev_warn(mmc_dev(host->mmc), "%s(): %d\n", __func__,
+ mrq->data->error);
+ }
+ break;
+ default:
+ cmd->error = -EFAULT;
+ dev_err(mmc_dev(host->mmc), "Invalid state %u\n", host->wait);
+ usdhi6_request_done(host);
+ return IRQ_HANDLED;
+ }
+
+ if (io_wait) {
+ schedule_delayed_work(&host->timeout_work, host->timeout);
+ /* Wait for more data or ACCESS_END */
+ if (!host->dma_active)
+ usdhi6_wait_for_brwe(host, mrq->data->flags & MMC_DATA_READ);
+ return IRQ_HANDLED;
+ }
+
+ if (!cmd->error) {
+ if (data) {
+ if (!data->error) {
+ if (host->wait != USDHI6_WAIT_FOR_STOP &&
+ host->mrq->stop &&
+ !host->mrq->stop->error &&
+ !usdhi6_stop_cmd(host)) {
+ /* Sending STOP */
+ usdhi6_wait_for_resp(host);
+
+ schedule_delayed_work(&host->timeout_work,
+ host->timeout);
+
+ return IRQ_HANDLED;
+ }
+
+ data->bytes_xfered = data->blocks * data->blksz;
+ } else {
+ /* Data error: might need to unmap the last page */
+ dev_warn(mmc_dev(host->mmc), "%s(): data error %d\n",
+ __func__, data->error);
+ usdhi6_sg_unmap(host, true);
+ }
+ } else if (cmd->opcode == MMC_APP_CMD) {
+ host->app_cmd = true;
+ }
+ }
+
+ usdhi6_request_done(host);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t usdhi6_sd(int irq, void *dev_id)
+{
+ struct usdhi6_host *host = dev_id;
+ u16 status, status2, error;
+
+ status = usdhi6_read(host, USDHI6_SD_INFO1) & ~host->status_mask &
+ ~USDHI6_SD_INFO1_CARD;
+ status2 = usdhi6_read(host, USDHI6_SD_INFO2) & ~host->status2_mask;
+
+ usdhi6_only_cd(host);
+
+ dev_dbg(mmc_dev(host->mmc),
+ "IRQ status = 0x%08x, status2 = 0x%08x\n", status, status2);
+
+ if (!status && !status2)
+ return IRQ_NONE;
+
+ error = status2 & USDHI6_SD_INFO2_ERR;
+
+ /* Ack / clear interrupts */
+ if (USDHI6_SD_INFO1_IRQ & status)
+ usdhi6_write(host, USDHI6_SD_INFO1,
+ 0xffff & ~(USDHI6_SD_INFO1_IRQ & status));
+
+ if (USDHI6_SD_INFO2_IRQ & status2) {
+ if (error)
+ /* In error cases BWE and BRE aren't cleared automatically */
+ status2 |= USDHI6_SD_INFO2_BWE | USDHI6_SD_INFO2_BRE;
+
+ usdhi6_write(host, USDHI6_SD_INFO2,
+ 0xffff & ~(USDHI6_SD_INFO2_IRQ & status2));
+ }
+
+ host->io_error = error;
+ host->irq_status = status;
+
+ if (error) {
+ /* Don't pollute the log with unsupported command timeouts */
+ if (host->wait != USDHI6_WAIT_FOR_CMD ||
+ error != USDHI6_SD_INFO2_RSP_TOUT)
+ dev_warn(mmc_dev(host->mmc),
+ "%s(): INFO2 error bits 0x%08x\n",
+ __func__, error);
+ else
+ dev_dbg(mmc_dev(host->mmc),
+ "%s(): INFO2 error bits 0x%08x\n",
+ __func__, error);
+ }
+
+ return IRQ_WAKE_THREAD;
+}
+
+static irqreturn_t usdhi6_sdio(int irq, void *dev_id)
+{
+ struct usdhi6_host *host = dev_id;
+ u32 status = usdhi6_read(host, USDHI6_SDIO_INFO1) & ~host->sdio_mask;
+
+ dev_dbg(mmc_dev(host->mmc), "%s(): status 0x%x\n", __func__, status);
+
+ if (!status)
+ return IRQ_NONE;
+
+ usdhi6_write(host, USDHI6_SDIO_INFO1, ~status);
+
+ mmc_signal_sdio_irq(host->mmc);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t usdhi6_cd(int irq, void *dev_id)
+{
+ struct usdhi6_host *host = dev_id;
+ struct mmc_host *mmc = host->mmc;
+ u16 status;
+
+ /* We're only interested in hotplug events here */
+ status = usdhi6_read(host, USDHI6_SD_INFO1) & ~host->status_mask &
+ USDHI6_SD_INFO1_CARD;
+
+ if (!status)
+ return IRQ_NONE;
+
+ /* Ack */
+ usdhi6_write(host, USDHI6_SD_INFO1, !status);
+
+ if (!work_pending(&mmc->detect.work) &&
+ (((status & USDHI6_SD_INFO1_CARD_INSERT) &&
+ !mmc->card) ||
+ ((status & USDHI6_SD_INFO1_CARD_EJECT) &&
+ mmc->card)))
+ mmc_detect_change(mmc, msecs_to_jiffies(100));
+
+ return IRQ_HANDLED;
+}
+
+/*
+ * Actually this should not be needed, if the built-in timeout works reliably in
+ * the both PIO cases and DMA never fails. But if DMA does fail, a timeout
+ * handler might be the only way to catch the error.
+ */
+static void usdhi6_timeout_work(struct work_struct *work)
+{
+ struct delayed_work *d = container_of(work, struct delayed_work, work);
+ struct usdhi6_host *host = container_of(d, struct usdhi6_host, timeout_work);
+ struct mmc_request *mrq = host->mrq;
+ struct mmc_data *data = mrq ? mrq->data : NULL;
+
+ dev_warn(mmc_dev(host->mmc),
+ "%s timeout wait %u CMD%d: IRQ 0x%08x:0x%08x, last IRQ 0x%08x\n",
+ host->dma_active ? "DMA" : "PIO",
+ host->wait, mrq ? mrq->cmd->opcode : -1,
+ usdhi6_read(host, USDHI6_SD_INFO1),
+ usdhi6_read(host, USDHI6_SD_INFO2), host->irq_status);
+
+ if (host->dma_active) {
+ usdhi6_dma_kill(host);
+ usdhi6_dma_stop_unmap(host);
+ }
+
+ switch (host->wait) {
+ default:
+ dev_err(mmc_dev(host->mmc), "Invalid state %u\n", host->wait);
+ /* mrq can be NULL in this actually impossible case */
+ case USDHI6_WAIT_FOR_CMD:
+ usdhi6_error_code(host);
+ if (mrq)
+ mrq->cmd->error = -ETIMEDOUT;
+ break;
+ case USDHI6_WAIT_FOR_STOP:
+ usdhi6_error_code(host);
+ mrq->stop->error = -ETIMEDOUT;
+ break;
+ case USDHI6_WAIT_FOR_DMA:
+ case USDHI6_WAIT_FOR_MREAD:
+ case USDHI6_WAIT_FOR_MWRITE:
+ case USDHI6_WAIT_FOR_READ:
+ case USDHI6_WAIT_FOR_WRITE:
+ dev_dbg(mmc_dev(host->mmc),
+ "%c: page #%u @ +0x%x %ux%u in SG%u. Current SG %u bytes @ %u\n",
+ data->flags & MMC_DATA_READ ? 'R' : 'W', host->page_idx,
+ host->offset, data->blocks, data->blksz, data->sg_len,
+ sg_dma_len(host->sg), host->sg->offset);
+ usdhi6_sg_unmap(host, true);
+ /*
+ * If USDHI6_WAIT_FOR_DATA_END times out, we have already unmapped
+ * the page
+ */
+ case USDHI6_WAIT_FOR_DATA_END:
+ usdhi6_error_code(host);
+ data->error = -ETIMEDOUT;
+ }
+
+ if (mrq)
+ usdhi6_request_done(host);
+}
+
+/* Probe / release */
+
+static const struct of_device_id usdhi6_of_match[] = {
+ {.compatible = "renesas,usdhi6rol0"},
+ {}
+};
+MODULE_DEVICE_TABLE(of, usdhi6_of_match);
+
+static int usdhi6_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct mmc_host *mmc;
+ struct usdhi6_host *host;
+ struct resource *res;
+ int irq_cd, irq_sd, irq_sdio;
+ u32 version;
+ int ret;
+
+ if (!dev->of_node)
+ return -ENODEV;
+
+ irq_cd = platform_get_irq_byname(pdev, "card detect");
+ irq_sd = platform_get_irq_byname(pdev, "data");
+ irq_sdio = platform_get_irq_byname(pdev, "SDIO");
+ if (irq_sd < 0 || irq_sdio < 0)
+ return -ENODEV;
+
+ mmc = mmc_alloc_host(sizeof(struct usdhi6_host), dev);
+ if (!mmc)
+ return -ENOMEM;
+
+ ret = mmc_of_parse(mmc);
+ if (ret < 0)
+ goto e_free_mmc;
+
+ mmc_regulator_get_supply(mmc);
+
+ host = mmc_priv(mmc);
+ host->mmc = mmc;
+ host->wait = USDHI6_WAIT_FOR_REQUEST;
+ host->timeout = msecs_to_jiffies(4000);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ host->base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(host->base)) {
+ ret = PTR_ERR(host->base);
+ goto e_free_mmc;
+ }
+
+ host->clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(host->clk))
+ goto e_free_mmc;
+
+ host->imclk = clk_get_rate(host->clk);
+
+ ret = clk_prepare_enable(host->clk);
+ if (ret < 0)
+ goto e_free_mmc;
+
+ version = usdhi6_read(host, USDHI6_VERSION);
+ if ((version & 0xfff) != 0xa0d) {
+ dev_err(dev, "Version not recognized %x\n", version);
+ goto e_clk_off;
+ }
+
+ dev_info(dev, "A USDHI6ROL0 SD host detected with %d ports\n",
+ usdhi6_read(host, USDHI6_SD_PORT_SEL) >> USDHI6_SD_PORT_SEL_PORTS_SHIFT);
+
+ usdhi6_mask_all(host);
+
+ if (irq_cd >= 0) {
+ ret = devm_request_irq(dev, irq_cd, usdhi6_cd, 0,
+ dev_name(dev), host);
+ if (ret < 0)
+ goto e_clk_off;
+ } else {
+ mmc->caps |= MMC_CAP_NEEDS_POLL;
+ }
+
+ ret = devm_request_threaded_irq(dev, irq_sd, usdhi6_sd, usdhi6_sd_bh, 0,
+ dev_name(dev), host);
+ if (ret < 0)
+ goto e_clk_off;
+
+ ret = devm_request_irq(dev, irq_sdio, usdhi6_sdio, 0,
+ dev_name(dev), host);
+ if (ret < 0)
+ goto e_clk_off;
+
+ INIT_DELAYED_WORK(&host->timeout_work, usdhi6_timeout_work);
+
+ usdhi6_dma_request(host, res->start);
+
+ mmc->ops = &usdhi6_ops;
+ mmc->caps |= MMC_CAP_SD_HIGHSPEED | MMC_CAP_MMC_HIGHSPEED |
+ MMC_CAP_UHS_SDR50 | MMC_CAP_UHS_DDR50 | MMC_CAP_SDIO_IRQ;
+ /* Set .max_segs to some random number. Feel free to adjust. */
+ mmc->max_segs = 32;
+ mmc->max_blk_size = 512;
+ mmc->max_req_size = PAGE_CACHE_SIZE * mmc->max_segs;
+ mmc->max_blk_count = mmc->max_req_size / mmc->max_blk_size;
+ /*
+ * Setting .max_seg_size to 1 page would simplify our page-mapping code,
+ * But OTOH, having large segments makes DMA more efficient. We could
+ * check, whether we managed to get DMA and fall back to 1 page
+ * segments, but if we do manage to obtain DMA and then it fails at
+ * run-time and we fall back to PIO, we will continue getting large
+ * segments. So, we wouldn't be able to get rid of the code anyway.
+ */
+ mmc->max_seg_size = mmc->max_req_size;
+ if (!mmc->f_max)
+ mmc->f_max = host->imclk;
+ mmc->f_min = host->imclk / 512;
+
+ platform_set_drvdata(pdev, host);
+
+ ret = mmc_add_host(mmc);
+ if (ret < 0)
+ goto e_clk_off;
+
+ return 0;
+
+e_clk_off:
+ clk_disable_unprepare(host->clk);
+e_free_mmc:
+ mmc_free_host(mmc);
+
+ return ret;
+}
+
+static int usdhi6_remove(struct platform_device *pdev)
+{
+ struct usdhi6_host *host = platform_get_drvdata(pdev);
+
+ mmc_remove_host(host->mmc);
+
+ usdhi6_mask_all(host);
+ cancel_delayed_work_sync(&host->timeout_work);
+ usdhi6_dma_release(host);
+ clk_disable_unprepare(host->clk);
+ mmc_free_host(host->mmc);
+
+ return 0;
+}
+
+static struct platform_driver usdhi6_driver = {
+ .probe = usdhi6_probe,
+ .remove = usdhi6_remove,
+ .driver = {
+ .name = "usdhi6rol0",
+ .owner = THIS_MODULE,
+ .of_match_table = usdhi6_of_match,
+ },
+};
+
+module_platform_driver(usdhi6_driver);
+
+MODULE_DESCRIPTION("Renesas usdhi6rol0 SD/SDIO host driver");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:usdhi6rol0");
+MODULE_AUTHOR("Guennadi Liakhovetski <g.liakhovetski@gmx.de>");
diff --git a/drivers/mmc/host/wmt-sdmmc.c b/drivers/mmc/host/wmt-sdmmc.c
index 498d1f7..282891a 100644
--- a/drivers/mmc/host/wmt-sdmmc.c
+++ b/drivers/mmc/host/wmt-sdmmc.c
@@ -840,7 +840,7 @@
priv->dma_desc_buffer = dma_alloc_coherent(&pdev->dev,
mmc->max_blk_count * 16,
&priv->dma_desc_device_addr,
- 208);
+ GFP_KERNEL);
if (!priv->dma_desc_buffer) {
dev_err(&pdev->dev, "DMA alloc fail\n");
ret = -EPERM;
diff --git a/drivers/mtd/Kconfig b/drivers/mtd/Kconfig
index 5d49a21..94b8210 100644
--- a/drivers/mtd/Kconfig
+++ b/drivers/mtd/Kconfig
@@ -321,6 +321,8 @@
source "drivers/mtd/lpddr/Kconfig"
+source "drivers/mtd/spi-nor/Kconfig"
+
source "drivers/mtd/ubi/Kconfig"
endif # MTD
diff --git a/drivers/mtd/Makefile b/drivers/mtd/Makefile
index 4cfb31e..99bb9a1 100644
--- a/drivers/mtd/Makefile
+++ b/drivers/mtd/Makefile
@@ -32,4 +32,5 @@
obj-y += chips/ lpddr/ maps/ devices/ nand/ onenand/ tests/
+obj-$(CONFIG_MTD_SPI_NOR) += spi-nor/
obj-$(CONFIG_MTD_UBI) += ubi/
diff --git a/drivers/mtd/chips/Kconfig b/drivers/mtd/chips/Kconfig
index e4696b3..9f02c28 100644
--- a/drivers/mtd/chips/Kconfig
+++ b/drivers/mtd/chips/Kconfig
@@ -169,33 +169,33 @@
in the programming of OTP bits will waste them.
config MTD_CFI_INTELEXT
- tristate "Support for Intel/Sharp flash chips"
+ tristate "Support for CFI command set 0001 (Intel/Sharp chips)"
depends on MTD_GEN_PROBE
select MTD_CFI_UTIL
help
The Common Flash Interface defines a number of different command
sets which a CFI-compliant chip may claim to implement. This code
- provides support for one of those command sets, used on Intel
- StrataFlash and other parts.
+ provides support for command set 0001, used on Intel StrataFlash
+ and other parts.
config MTD_CFI_AMDSTD
- tristate "Support for AMD/Fujitsu/Spansion flash chips"
+ tristate "Support for CFI command set 0002 (AMD/Fujitsu/Spansion chips)"
depends on MTD_GEN_PROBE
select MTD_CFI_UTIL
help
The Common Flash Interface defines a number of different command
sets which a CFI-compliant chip may claim to implement. This code
- provides support for one of those command sets, used on chips
- including the AMD Am29LV320.
+ provides support for command set 0002, used on chips including
+ the AMD Am29LV320.
config MTD_CFI_STAA
- tristate "Support for ST (Advanced Architecture) flash chips"
+ tristate "Support for CFI command set 0020 (ST (Advanced Architecture) chips)"
depends on MTD_GEN_PROBE
select MTD_CFI_UTIL
help
The Common Flash Interface defines a number of different command
sets which a CFI-compliant chip may claim to implement. This code
- provides support for one of those command sets.
+ provides support for command set 0020.
config MTD_CFI_UTIL
tristate
diff --git a/drivers/mtd/chips/cfi_cmdset_0020.c b/drivers/mtd/chips/cfi_cmdset_0020.c
index 6293855..423666b 100644
--- a/drivers/mtd/chips/cfi_cmdset_0020.c
+++ b/drivers/mtd/chips/cfi_cmdset_0020.c
@@ -961,7 +961,7 @@
chipnum++;
if (chipnum >= cfi->numchips)
- break;
+ break;
}
}
@@ -1170,7 +1170,7 @@
chipnum++;
if (chipnum >= cfi->numchips)
- break;
+ break;
}
}
return 0;
diff --git a/drivers/mtd/chips/cfi_util.c b/drivers/mtd/chips/cfi_util.c
index 08049f6..09c79bd 100644
--- a/drivers/mtd/chips/cfi_util.c
+++ b/drivers/mtd/chips/cfi_util.c
@@ -239,7 +239,7 @@
chipnum++;
if (chipnum >= cfi->numchips)
- break;
+ break;
}
}
diff --git a/drivers/mtd/devices/Kconfig b/drivers/mtd/devices/Kconfig
index 1210bc2..c49d0b1 100644
--- a/drivers/mtd/devices/Kconfig
+++ b/drivers/mtd/devices/Kconfig
@@ -80,7 +80,7 @@
config MTD_M25P80
tristate "Support most SPI Flash chips (AT26DF, M25P, W25X, ...)"
- depends on SPI_MASTER
+ depends on SPI_MASTER && MTD_SPI_NOR
help
This enables access to most modern SPI flash chips, used for
program and data storage. Series supported include Atmel AT26DF,
@@ -212,7 +212,7 @@
config MTD_ST_SPI_FSM
tristate "ST Microelectronics SPI FSM Serial Flash Controller"
- depends on ARM || SH
+ depends on ARCH_STI
help
This provides an MTD device driver for the ST Microelectronics
SPI Fast Sequence Mode (FSM) Serial Flash Controller and support
diff --git a/drivers/mtd/devices/elm.c b/drivers/mtd/devices/elm.c
index 1fd4a0f..7df8694 100644
--- a/drivers/mtd/devices/elm.c
+++ b/drivers/mtd/devices/elm.c
@@ -213,6 +213,28 @@
val = cpu_to_be32(*(u32 *) &ecc[0]) >> 12;
elm_write_reg(info, offset, val);
break;
+ case BCH16_ECC:
+ val = cpu_to_be32(*(u32 *) &ecc[22]);
+ elm_write_reg(info, offset, val);
+ offset += 4;
+ val = cpu_to_be32(*(u32 *) &ecc[18]);
+ elm_write_reg(info, offset, val);
+ offset += 4;
+ val = cpu_to_be32(*(u32 *) &ecc[14]);
+ elm_write_reg(info, offset, val);
+ offset += 4;
+ val = cpu_to_be32(*(u32 *) &ecc[10]);
+ elm_write_reg(info, offset, val);
+ offset += 4;
+ val = cpu_to_be32(*(u32 *) &ecc[6]);
+ elm_write_reg(info, offset, val);
+ offset += 4;
+ val = cpu_to_be32(*(u32 *) &ecc[2]);
+ elm_write_reg(info, offset, val);
+ offset += 4;
+ val = cpu_to_be32(*(u32 *) &ecc[0]) >> 16;
+ elm_write_reg(info, offset, val);
+ break;
default:
pr_err("invalid config bch_type\n");
}
@@ -418,6 +440,7 @@
return 0;
}
+#ifdef CONFIG_PM_SLEEP
/**
* elm_context_save
* saves ELM configurations to preserve them across Hardware powered-down
@@ -435,6 +458,13 @@
for (i = 0; i < ERROR_VECTOR_MAX; i++) {
offset = i * SYNDROME_FRAGMENT_REG_SIZE;
switch (bch_type) {
+ case BCH16_ECC:
+ regs->elm_syndrome_fragment_6[i] = elm_read_reg(info,
+ ELM_SYNDROME_FRAGMENT_6 + offset);
+ regs->elm_syndrome_fragment_5[i] = elm_read_reg(info,
+ ELM_SYNDROME_FRAGMENT_5 + offset);
+ regs->elm_syndrome_fragment_4[i] = elm_read_reg(info,
+ ELM_SYNDROME_FRAGMENT_4 + offset);
case BCH8_ECC:
regs->elm_syndrome_fragment_3[i] = elm_read_reg(info,
ELM_SYNDROME_FRAGMENT_3 + offset);
@@ -473,6 +503,13 @@
for (i = 0; i < ERROR_VECTOR_MAX; i++) {
offset = i * SYNDROME_FRAGMENT_REG_SIZE;
switch (bch_type) {
+ case BCH16_ECC:
+ elm_write_reg(info, ELM_SYNDROME_FRAGMENT_6 + offset,
+ regs->elm_syndrome_fragment_6[i]);
+ elm_write_reg(info, ELM_SYNDROME_FRAGMENT_5 + offset,
+ regs->elm_syndrome_fragment_5[i]);
+ elm_write_reg(info, ELM_SYNDROME_FRAGMENT_4 + offset,
+ regs->elm_syndrome_fragment_4[i]);
case BCH8_ECC:
elm_write_reg(info, ELM_SYNDROME_FRAGMENT_3 + offset,
regs->elm_syndrome_fragment_3[i]);
@@ -509,6 +546,7 @@
elm_context_restore(info);
return 0;
}
+#endif
static SIMPLE_DEV_PM_OPS(elm_pm_ops, elm_suspend, elm_resume);
diff --git a/drivers/mtd/devices/m25p80.c b/drivers/mtd/devices/m25p80.c
index 524dab3..ed7e0a1b 100644
--- a/drivers/mtd/devices/m25p80.c
+++ b/drivers/mtd/devices/m25p80.c
@@ -19,485 +19,98 @@
#include <linux/errno.h>
#include <linux/module.h>
#include <linux/device.h>
-#include <linux/interrupt.h>
-#include <linux/mutex.h>
-#include <linux/math64.h>
-#include <linux/slab.h>
-#include <linux/sched.h>
-#include <linux/mod_devicetable.h>
-#include <linux/mtd/cfi.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/partitions.h>
-#include <linux/of_platform.h>
#include <linux/spi/spi.h>
#include <linux/spi/flash.h>
+#include <linux/mtd/spi-nor.h>
-/* Flash opcodes. */
-#define OPCODE_WREN 0x06 /* Write enable */
-#define OPCODE_RDSR 0x05 /* Read status register */
-#define OPCODE_WRSR 0x01 /* Write status register 1 byte */
-#define OPCODE_NORM_READ 0x03 /* Read data bytes (low frequency) */
-#define OPCODE_FAST_READ 0x0b /* Read data bytes (high frequency) */
-#define OPCODE_DUAL_READ 0x3b /* Read data bytes (Dual SPI) */
-#define OPCODE_QUAD_READ 0x6b /* Read data bytes (Quad SPI) */
-#define OPCODE_PP 0x02 /* Page program (up to 256 bytes) */
-#define OPCODE_BE_4K 0x20 /* Erase 4KiB block */
-#define OPCODE_BE_4K_PMC 0xd7 /* Erase 4KiB block on PMC chips */
-#define OPCODE_BE_32K 0x52 /* Erase 32KiB block */
-#define OPCODE_CHIP_ERASE 0xc7 /* Erase whole flash chip */
-#define OPCODE_SE 0xd8 /* Sector erase (usually 64KiB) */
-#define OPCODE_RDID 0x9f /* Read JEDEC ID */
-#define OPCODE_RDCR 0x35 /* Read configuration register */
-
-/* 4-byte address opcodes - used on Spansion and some Macronix flashes. */
-#define OPCODE_NORM_READ_4B 0x13 /* Read data bytes (low frequency) */
-#define OPCODE_FAST_READ_4B 0x0c /* Read data bytes (high frequency) */
-#define OPCODE_DUAL_READ_4B 0x3c /* Read data bytes (Dual SPI) */
-#define OPCODE_QUAD_READ_4B 0x6c /* Read data bytes (Quad SPI) */
-#define OPCODE_PP_4B 0x12 /* Page program (up to 256 bytes) */
-#define OPCODE_SE_4B 0xdc /* Sector erase (usually 64KiB) */
-
-/* Used for SST flashes only. */
-#define OPCODE_BP 0x02 /* Byte program */
-#define OPCODE_WRDI 0x04 /* Write disable */
-#define OPCODE_AAI_WP 0xad /* Auto address increment word program */
-
-/* Used for Macronix and Winbond flashes. */
-#define OPCODE_EN4B 0xb7 /* Enter 4-byte mode */
-#define OPCODE_EX4B 0xe9 /* Exit 4-byte mode */
-
-/* Used for Spansion flashes only. */
-#define OPCODE_BRWR 0x17 /* Bank register write */
-
-/* Status Register bits. */
-#define SR_WIP 1 /* Write in progress */
-#define SR_WEL 2 /* Write enable latch */
-/* meaning of other SR_* bits may differ between vendors */
-#define SR_BP0 4 /* Block protect 0 */
-#define SR_BP1 8 /* Block protect 1 */
-#define SR_BP2 0x10 /* Block protect 2 */
-#define SR_SRWD 0x80 /* SR write protect */
-
-#define SR_QUAD_EN_MX 0x40 /* Macronix Quad I/O */
-
-/* Configuration Register bits. */
-#define CR_QUAD_EN_SPAN 0x2 /* Spansion Quad I/O */
-
-/* Define max times to check status register before we give up. */
-#define MAX_READY_WAIT_JIFFIES (40 * HZ) /* M25P16 specs 40s max chip erase */
#define MAX_CMD_SIZE 6
-
-#define JEDEC_MFR(_jedec_id) ((_jedec_id) >> 16)
-
-/****************************************************************************/
-
-enum read_type {
- M25P80_NORMAL = 0,
- M25P80_FAST,
- M25P80_DUAL,
- M25P80_QUAD,
-};
-
struct m25p {
struct spi_device *spi;
- struct mutex lock;
+ struct spi_nor spi_nor;
struct mtd_info mtd;
- u16 page_size;
- u16 addr_width;
- u8 erase_opcode;
- u8 read_opcode;
- u8 program_opcode;
- u8 *command;
- enum read_type flash_read;
+ u8 command[MAX_CMD_SIZE];
};
-static inline struct m25p *mtd_to_m25p(struct mtd_info *mtd)
+static int m25p80_read_reg(struct spi_nor *nor, u8 code, u8 *val, int len)
{
- return container_of(mtd, struct m25p, mtd);
-}
-
-/****************************************************************************/
-
-/*
- * Internal helper functions
- */
-
-/*
- * Read the status register, returning its value in the location
- * Return the status register value.
- * Returns negative if error occurred.
- */
-static int read_sr(struct m25p *flash)
-{
- ssize_t retval;
- u8 code = OPCODE_RDSR;
- u8 val;
-
- retval = spi_write_then_read(flash->spi, &code, 1, &val, 1);
-
- if (retval < 0) {
- dev_err(&flash->spi->dev, "error %d reading SR\n",
- (int) retval);
- return retval;
- }
-
- return val;
-}
-
-/*
- * Read configuration register, returning its value in the
- * location. Return the configuration register value.
- * Returns negative if error occured.
- */
-static int read_cr(struct m25p *flash)
-{
- u8 code = OPCODE_RDCR;
+ struct m25p *flash = nor->priv;
+ struct spi_device *spi = flash->spi;
int ret;
- u8 val;
- ret = spi_write_then_read(flash->spi, &code, 1, &val, 1);
- if (ret < 0) {
- dev_err(&flash->spi->dev, "error %d reading CR\n", ret);
- return ret;
- }
+ ret = spi_write_then_read(spi, &code, 1, val, len);
+ if (ret < 0)
+ dev_err(&spi->dev, "error %d reading %x\n", ret, code);
- return val;
+ return ret;
}
-/*
- * Write status register 1 byte
- * Returns negative if error occurred.
- */
-static int write_sr(struct m25p *flash, u8 val)
-{
- flash->command[0] = OPCODE_WRSR;
- flash->command[1] = val;
-
- return spi_write(flash->spi, flash->command, 2);
-}
-
-/*
- * Set write enable latch with Write Enable command.
- * Returns negative if error occurred.
- */
-static inline int write_enable(struct m25p *flash)
-{
- u8 code = OPCODE_WREN;
-
- return spi_write_then_read(flash->spi, &code, 1, NULL, 0);
-}
-
-/*
- * Send write disble instruction to the chip.
- */
-static inline int write_disable(struct m25p *flash)
-{
- u8 code = OPCODE_WRDI;
-
- return spi_write_then_read(flash->spi, &code, 1, NULL, 0);
-}
-
-/*
- * Enable/disable 4-byte addressing mode.
- */
-static inline int set_4byte(struct m25p *flash, u32 jedec_id, int enable)
-{
- int status;
- bool need_wren = false;
-
- switch (JEDEC_MFR(jedec_id)) {
- case CFI_MFR_ST: /* Micron, actually */
- /* Some Micron need WREN command; all will accept it */
- need_wren = true;
- case CFI_MFR_MACRONIX:
- case 0xEF /* winbond */:
- if (need_wren)
- write_enable(flash);
-
- flash->command[0] = enable ? OPCODE_EN4B : OPCODE_EX4B;
- status = spi_write(flash->spi, flash->command, 1);
-
- if (need_wren)
- write_disable(flash);
-
- return status;
- default:
- /* Spansion style */
- flash->command[0] = OPCODE_BRWR;
- flash->command[1] = enable << 7;
- return spi_write(flash->spi, flash->command, 2);
- }
-}
-
-/*
- * Service routine to read status register until ready, or timeout occurs.
- * Returns non-zero if error.
- */
-static int wait_till_ready(struct m25p *flash)
-{
- unsigned long deadline;
- int sr;
-
- deadline = jiffies + MAX_READY_WAIT_JIFFIES;
-
- do {
- if ((sr = read_sr(flash)) < 0)
- break;
- else if (!(sr & SR_WIP))
- return 0;
-
- cond_resched();
-
- } while (!time_after_eq(jiffies, deadline));
-
- return 1;
-}
-
-/*
- * Write status Register and configuration register with 2 bytes
- * The first byte will be written to the status register, while the
- * second byte will be written to the configuration register.
- * Return negative if error occured.
- */
-static int write_sr_cr(struct m25p *flash, u16 val)
-{
- flash->command[0] = OPCODE_WRSR;
- flash->command[1] = val & 0xff;
- flash->command[2] = (val >> 8);
-
- return spi_write(flash->spi, flash->command, 3);
-}
-
-static int macronix_quad_enable(struct m25p *flash)
-{
- int ret, val;
- u8 cmd[2];
- cmd[0] = OPCODE_WRSR;
-
- val = read_sr(flash);
- cmd[1] = val | SR_QUAD_EN_MX;
- write_enable(flash);
-
- spi_write(flash->spi, &cmd, 2);
-
- if (wait_till_ready(flash))
- return 1;
-
- ret = read_sr(flash);
- if (!(ret > 0 && (ret & SR_QUAD_EN_MX))) {
- dev_err(&flash->spi->dev, "Macronix Quad bit not set\n");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static int spansion_quad_enable(struct m25p *flash)
-{
- int ret;
- int quad_en = CR_QUAD_EN_SPAN << 8;
-
- write_enable(flash);
-
- ret = write_sr_cr(flash, quad_en);
- if (ret < 0) {
- dev_err(&flash->spi->dev,
- "error while writing configuration register\n");
- return -EINVAL;
- }
-
- /* read back and check it */
- ret = read_cr(flash);
- if (!(ret > 0 && (ret & CR_QUAD_EN_SPAN))) {
- dev_err(&flash->spi->dev, "Spansion Quad bit not set\n");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static int set_quad_mode(struct m25p *flash, u32 jedec_id)
-{
- int status;
-
- switch (JEDEC_MFR(jedec_id)) {
- case CFI_MFR_MACRONIX:
- status = macronix_quad_enable(flash);
- if (status) {
- dev_err(&flash->spi->dev,
- "Macronix quad-read not enabled\n");
- return -EINVAL;
- }
- return status;
- default:
- status = spansion_quad_enable(flash);
- if (status) {
- dev_err(&flash->spi->dev,
- "Spansion quad-read not enabled\n");
- return -EINVAL;
- }
- return status;
- }
-}
-
-/*
- * Erase the whole flash memory
- *
- * Returns 0 if successful, non-zero otherwise.
- */
-static int erase_chip(struct m25p *flash)
-{
- pr_debug("%s: %s %lldKiB\n", dev_name(&flash->spi->dev), __func__,
- (long long)(flash->mtd.size >> 10));
-
- /* Wait until finished previous write command. */
- if (wait_till_ready(flash))
- return 1;
-
- /* Send write enable, then erase commands. */
- write_enable(flash);
-
- /* Set up command buffer. */
- flash->command[0] = OPCODE_CHIP_ERASE;
-
- spi_write(flash->spi, flash->command, 1);
-
- return 0;
-}
-
-static void m25p_addr2cmd(struct m25p *flash, unsigned int addr, u8 *cmd)
+static void m25p_addr2cmd(struct spi_nor *nor, unsigned int addr, u8 *cmd)
{
/* opcode is in cmd[0] */
- cmd[1] = addr >> (flash->addr_width * 8 - 8);
- cmd[2] = addr >> (flash->addr_width * 8 - 16);
- cmd[3] = addr >> (flash->addr_width * 8 - 24);
- cmd[4] = addr >> (flash->addr_width * 8 - 32);
+ cmd[1] = addr >> (nor->addr_width * 8 - 8);
+ cmd[2] = addr >> (nor->addr_width * 8 - 16);
+ cmd[3] = addr >> (nor->addr_width * 8 - 24);
+ cmd[4] = addr >> (nor->addr_width * 8 - 32);
}
-static int m25p_cmdsz(struct m25p *flash)
+static int m25p_cmdsz(struct spi_nor *nor)
{
- return 1 + flash->addr_width;
+ return 1 + nor->addr_width;
}
-/*
- * Erase one sector of flash memory at offset ``offset'' which is any
- * address within the sector which should be erased.
- *
- * Returns 0 if successful, non-zero otherwise.
- */
-static int erase_sector(struct m25p *flash, u32 offset)
+static int m25p80_write_reg(struct spi_nor *nor, u8 opcode, u8 *buf, int len,
+ int wr_en)
{
- pr_debug("%s: %s %dKiB at 0x%08x\n", dev_name(&flash->spi->dev),
- __func__, flash->mtd.erasesize / 1024, offset);
+ struct m25p *flash = nor->priv;
+ struct spi_device *spi = flash->spi;
- /* Wait until finished previous write command. */
- if (wait_till_ready(flash))
- return 1;
+ flash->command[0] = opcode;
+ if (buf)
+ memcpy(&flash->command[1], buf, len);
- /* Send write enable, then erase commands. */
- write_enable(flash);
-
- /* Set up command buffer. */
- flash->command[0] = flash->erase_opcode;
- m25p_addr2cmd(flash, offset, flash->command);
-
- spi_write(flash->spi, flash->command, m25p_cmdsz(flash));
-
- return 0;
+ return spi_write(spi, flash->command, len + 1);
}
-/****************************************************************************/
-
-/*
- * MTD implementation
- */
-
-/*
- * Erase an address range on the flash chip. The address range may extend
- * one or more erase sectors. Return an error is there is a problem erasing.
- */
-static int m25p80_erase(struct mtd_info *mtd, struct erase_info *instr)
+static void m25p80_write(struct spi_nor *nor, loff_t to, size_t len,
+ size_t *retlen, const u_char *buf)
{
- struct m25p *flash = mtd_to_m25p(mtd);
- u32 addr,len;
- uint32_t rem;
+ struct m25p *flash = nor->priv;
+ struct spi_device *spi = flash->spi;
+ struct spi_transfer t[2] = {};
+ struct spi_message m;
+ int cmd_sz = m25p_cmdsz(nor);
- pr_debug("%s: %s at 0x%llx, len %lld\n", dev_name(&flash->spi->dev),
- __func__, (long long)instr->addr,
- (long long)instr->len);
+ spi_message_init(&m);
- div_u64_rem(instr->len, mtd->erasesize, &rem);
- if (rem)
- return -EINVAL;
+ if (nor->program_opcode == SPINOR_OP_AAI_WP && nor->sst_write_second)
+ cmd_sz = 1;
- addr = instr->addr;
- len = instr->len;
+ flash->command[0] = nor->program_opcode;
+ m25p_addr2cmd(nor, to, flash->command);
- mutex_lock(&flash->lock);
+ t[0].tx_buf = flash->command;
+ t[0].len = cmd_sz;
+ spi_message_add_tail(&t[0], &m);
- /* whole-chip erase? */
- if (len == flash->mtd.size) {
- if (erase_chip(flash)) {
- instr->state = MTD_ERASE_FAILED;
- mutex_unlock(&flash->lock);
- return -EIO;
- }
+ t[1].tx_buf = buf;
+ t[1].len = len;
+ spi_message_add_tail(&t[1], &m);
- /* REVISIT in some cases we could speed up erasing large regions
- * by using OPCODE_SE instead of OPCODE_BE_4K. We may have set up
- * to use "small sector erase", but that's not always optimal.
- */
+ spi_sync(spi, &m);
- /* "sector"-at-a-time erase */
- } else {
- while (len) {
- if (erase_sector(flash, addr)) {
- instr->state = MTD_ERASE_FAILED;
- mutex_unlock(&flash->lock);
- return -EIO;
- }
-
- addr += mtd->erasesize;
- len -= mtd->erasesize;
- }
- }
-
- mutex_unlock(&flash->lock);
-
- instr->state = MTD_ERASE_DONE;
- mtd_erase_callback(instr);
-
- return 0;
+ *retlen += m.actual_length - cmd_sz;
}
-/*
- * Dummy Cycle calculation for different type of read.
- * It can be used to support more commands with
- * different dummy cycle requirements.
- */
-static inline int m25p80_dummy_cycles_read(struct m25p *flash)
+static inline unsigned int m25p80_rx_nbits(struct spi_nor *nor)
{
- switch (flash->flash_read) {
- case M25P80_FAST:
- case M25P80_DUAL:
- case M25P80_QUAD:
- return 1;
- case M25P80_NORMAL:
- return 0;
- default:
- dev_err(&flash->spi->dev, "No valid read type supported\n");
- return -1;
- }
-}
-
-static inline unsigned int m25p80_rx_nbits(const struct m25p *flash)
-{
- switch (flash->flash_read) {
- case M25P80_DUAL:
+ switch (nor->flash_read) {
+ case SPI_NOR_DUAL:
return 2;
- case M25P80_QUAD:
+ case SPI_NOR_QUAD:
return 4;
default:
return 0;
@@ -505,590 +118,72 @@
}
/*
- * Read an address range from the flash chip. The address range
+ * Read an address range from the nor chip. The address range
* may be any size provided it is within the physical boundaries.
*/
-static int m25p80_read(struct mtd_info *mtd, loff_t from, size_t len,
- size_t *retlen, u_char *buf)
+static int m25p80_read(struct spi_nor *nor, loff_t from, size_t len,
+ size_t *retlen, u_char *buf)
{
- struct m25p *flash = mtd_to_m25p(mtd);
+ struct m25p *flash = nor->priv;
+ struct spi_device *spi = flash->spi;
struct spi_transfer t[2];
struct spi_message m;
- uint8_t opcode;
- int dummy;
+ int dummy = nor->read_dummy;
+ int ret;
- pr_debug("%s: %s from 0x%08x, len %zd\n", dev_name(&flash->spi->dev),
- __func__, (u32)from, len);
+ /* Wait till previous write/erase is done. */
+ ret = nor->wait_till_ready(nor);
+ if (ret)
+ return ret;
spi_message_init(&m);
memset(t, 0, (sizeof t));
- dummy = m25p80_dummy_cycles_read(flash);
- if (dummy < 0) {
- dev_err(&flash->spi->dev, "No valid read command supported\n");
- return -EINVAL;
- }
+ flash->command[0] = nor->read_opcode;
+ m25p_addr2cmd(nor, from, flash->command);
t[0].tx_buf = flash->command;
- t[0].len = m25p_cmdsz(flash) + dummy;
+ t[0].len = m25p_cmdsz(nor) + dummy;
spi_message_add_tail(&t[0], &m);
t[1].rx_buf = buf;
- t[1].rx_nbits = m25p80_rx_nbits(flash);
+ t[1].rx_nbits = m25p80_rx_nbits(nor);
t[1].len = len;
spi_message_add_tail(&t[1], &m);
- mutex_lock(&flash->lock);
+ spi_sync(spi, &m);
- /* Wait till previous write/erase is done. */
- if (wait_till_ready(flash)) {
- /* REVISIT status return?? */
- mutex_unlock(&flash->lock);
- return 1;
- }
-
- /* Set up the write data buffer. */
- opcode = flash->read_opcode;
- flash->command[0] = opcode;
- m25p_addr2cmd(flash, from, flash->command);
-
- spi_sync(flash->spi, &m);
-
- *retlen = m.actual_length - m25p_cmdsz(flash) - dummy;
-
- mutex_unlock(&flash->lock);
-
+ *retlen = m.actual_length - m25p_cmdsz(nor) - dummy;
return 0;
}
-/*
- * Write an address range to the flash chip. Data must be written in
- * FLASH_PAGESIZE chunks. The address range may be any size provided
- * it is within the physical boundaries.
- */
-static int m25p80_write(struct mtd_info *mtd, loff_t to, size_t len,
- size_t *retlen, const u_char *buf)
+static int m25p80_erase(struct spi_nor *nor, loff_t offset)
{
- struct m25p *flash = mtd_to_m25p(mtd);
- u32 page_offset, page_size;
- struct spi_transfer t[2];
- struct spi_message m;
+ struct m25p *flash = nor->priv;
+ int ret;
- pr_debug("%s: %s to 0x%08x, len %zd\n", dev_name(&flash->spi->dev),
- __func__, (u32)to, len);
-
- spi_message_init(&m);
- memset(t, 0, (sizeof t));
-
- t[0].tx_buf = flash->command;
- t[0].len = m25p_cmdsz(flash);
- spi_message_add_tail(&t[0], &m);
-
- t[1].tx_buf = buf;
- spi_message_add_tail(&t[1], &m);
-
- mutex_lock(&flash->lock);
+ dev_dbg(nor->dev, "%dKiB at 0x%08x\n",
+ flash->mtd.erasesize / 1024, (u32)offset);
/* Wait until finished previous write command. */
- if (wait_till_ready(flash)) {
- mutex_unlock(&flash->lock);
- return 1;
- }
+ ret = nor->wait_till_ready(nor);
+ if (ret)
+ return ret;
- write_enable(flash);
+ /* Send write enable, then erase commands. */
+ ret = nor->write_reg(nor, SPINOR_OP_WREN, NULL, 0, 0);
+ if (ret)
+ return ret;
- /* Set up the opcode in the write buffer. */
- flash->command[0] = flash->program_opcode;
- m25p_addr2cmd(flash, to, flash->command);
+ /* Set up command buffer. */
+ flash->command[0] = nor->erase_opcode;
+ m25p_addr2cmd(nor, offset, flash->command);
- page_offset = to & (flash->page_size - 1);
-
- /* do all the bytes fit onto one page? */
- if (page_offset + len <= flash->page_size) {
- t[1].len = len;
-
- spi_sync(flash->spi, &m);
-
- *retlen = m.actual_length - m25p_cmdsz(flash);
- } else {
- u32 i;
-
- /* the size of data remaining on the first page */
- page_size = flash->page_size - page_offset;
-
- t[1].len = page_size;
- spi_sync(flash->spi, &m);
-
- *retlen = m.actual_length - m25p_cmdsz(flash);
-
- /* write everything in flash->page_size chunks */
- for (i = page_size; i < len; i += page_size) {
- page_size = len - i;
- if (page_size > flash->page_size)
- page_size = flash->page_size;
-
- /* write the next page to flash */
- m25p_addr2cmd(flash, to + i, flash->command);
-
- t[1].tx_buf = buf + i;
- t[1].len = page_size;
-
- wait_till_ready(flash);
-
- write_enable(flash);
-
- spi_sync(flash->spi, &m);
-
- *retlen += m.actual_length - m25p_cmdsz(flash);
- }
- }
-
- mutex_unlock(&flash->lock);
+ spi_write(flash->spi, flash->command, m25p_cmdsz(nor));
return 0;
}
-static int sst_write(struct mtd_info *mtd, loff_t to, size_t len,
- size_t *retlen, const u_char *buf)
-{
- struct m25p *flash = mtd_to_m25p(mtd);
- struct spi_transfer t[2];
- struct spi_message m;
- size_t actual;
- int cmd_sz, ret;
-
- pr_debug("%s: %s to 0x%08x, len %zd\n", dev_name(&flash->spi->dev),
- __func__, (u32)to, len);
-
- spi_message_init(&m);
- memset(t, 0, (sizeof t));
-
- t[0].tx_buf = flash->command;
- t[0].len = m25p_cmdsz(flash);
- spi_message_add_tail(&t[0], &m);
-
- t[1].tx_buf = buf;
- spi_message_add_tail(&t[1], &m);
-
- mutex_lock(&flash->lock);
-
- /* Wait until finished previous write command. */
- ret = wait_till_ready(flash);
- if (ret)
- goto time_out;
-
- write_enable(flash);
-
- actual = to % 2;
- /* Start write from odd address. */
- if (actual) {
- flash->command[0] = OPCODE_BP;
- m25p_addr2cmd(flash, to, flash->command);
-
- /* write one byte. */
- t[1].len = 1;
- spi_sync(flash->spi, &m);
- ret = wait_till_ready(flash);
- if (ret)
- goto time_out;
- *retlen += m.actual_length - m25p_cmdsz(flash);
- }
- to += actual;
-
- flash->command[0] = OPCODE_AAI_WP;
- m25p_addr2cmd(flash, to, flash->command);
-
- /* Write out most of the data here. */
- cmd_sz = m25p_cmdsz(flash);
- for (; actual < len - 1; actual += 2) {
- t[0].len = cmd_sz;
- /* write two bytes. */
- t[1].len = 2;
- t[1].tx_buf = buf + actual;
-
- spi_sync(flash->spi, &m);
- ret = wait_till_ready(flash);
- if (ret)
- goto time_out;
- *retlen += m.actual_length - cmd_sz;
- cmd_sz = 1;
- to += 2;
- }
- write_disable(flash);
- ret = wait_till_ready(flash);
- if (ret)
- goto time_out;
-
- /* Write out trailing byte if it exists. */
- if (actual != len) {
- write_enable(flash);
- flash->command[0] = OPCODE_BP;
- m25p_addr2cmd(flash, to, flash->command);
- t[0].len = m25p_cmdsz(flash);
- t[1].len = 1;
- t[1].tx_buf = buf + actual;
-
- spi_sync(flash->spi, &m);
- ret = wait_till_ready(flash);
- if (ret)
- goto time_out;
- *retlen += m.actual_length - m25p_cmdsz(flash);
- write_disable(flash);
- }
-
-time_out:
- mutex_unlock(&flash->lock);
- return ret;
-}
-
-static int m25p80_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
-{
- struct m25p *flash = mtd_to_m25p(mtd);
- uint32_t offset = ofs;
- uint8_t status_old, status_new;
- int res = 0;
-
- mutex_lock(&flash->lock);
- /* Wait until finished previous command */
- if (wait_till_ready(flash)) {
- res = 1;
- goto err;
- }
-
- status_old = read_sr(flash);
-
- if (offset < flash->mtd.size-(flash->mtd.size/2))
- status_new = status_old | SR_BP2 | SR_BP1 | SR_BP0;
- else if (offset < flash->mtd.size-(flash->mtd.size/4))
- status_new = (status_old & ~SR_BP0) | SR_BP2 | SR_BP1;
- else if (offset < flash->mtd.size-(flash->mtd.size/8))
- status_new = (status_old & ~SR_BP1) | SR_BP2 | SR_BP0;
- else if (offset < flash->mtd.size-(flash->mtd.size/16))
- status_new = (status_old & ~(SR_BP0|SR_BP1)) | SR_BP2;
- else if (offset < flash->mtd.size-(flash->mtd.size/32))
- status_new = (status_old & ~SR_BP2) | SR_BP1 | SR_BP0;
- else if (offset < flash->mtd.size-(flash->mtd.size/64))
- status_new = (status_old & ~(SR_BP2|SR_BP0)) | SR_BP1;
- else
- status_new = (status_old & ~(SR_BP2|SR_BP1)) | SR_BP0;
-
- /* Only modify protection if it will not unlock other areas */
- if ((status_new&(SR_BP2|SR_BP1|SR_BP0)) >
- (status_old&(SR_BP2|SR_BP1|SR_BP0))) {
- write_enable(flash);
- if (write_sr(flash, status_new) < 0) {
- res = 1;
- goto err;
- }
- }
-
-err: mutex_unlock(&flash->lock);
- return res;
-}
-
-static int m25p80_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
-{
- struct m25p *flash = mtd_to_m25p(mtd);
- uint32_t offset = ofs;
- uint8_t status_old, status_new;
- int res = 0;
-
- mutex_lock(&flash->lock);
- /* Wait until finished previous command */
- if (wait_till_ready(flash)) {
- res = 1;
- goto err;
- }
-
- status_old = read_sr(flash);
-
- if (offset+len > flash->mtd.size-(flash->mtd.size/64))
- status_new = status_old & ~(SR_BP2|SR_BP1|SR_BP0);
- else if (offset+len > flash->mtd.size-(flash->mtd.size/32))
- status_new = (status_old & ~(SR_BP2|SR_BP1)) | SR_BP0;
- else if (offset+len > flash->mtd.size-(flash->mtd.size/16))
- status_new = (status_old & ~(SR_BP2|SR_BP0)) | SR_BP1;
- else if (offset+len > flash->mtd.size-(flash->mtd.size/8))
- status_new = (status_old & ~SR_BP2) | SR_BP1 | SR_BP0;
- else if (offset+len > flash->mtd.size-(flash->mtd.size/4))
- status_new = (status_old & ~(SR_BP0|SR_BP1)) | SR_BP2;
- else if (offset+len > flash->mtd.size-(flash->mtd.size/2))
- status_new = (status_old & ~SR_BP1) | SR_BP2 | SR_BP0;
- else
- status_new = (status_old & ~SR_BP0) | SR_BP2 | SR_BP1;
-
- /* Only modify protection if it will not lock other areas */
- if ((status_new&(SR_BP2|SR_BP1|SR_BP0)) <
- (status_old&(SR_BP2|SR_BP1|SR_BP0))) {
- write_enable(flash);
- if (write_sr(flash, status_new) < 0) {
- res = 1;
- goto err;
- }
- }
-
-err: mutex_unlock(&flash->lock);
- return res;
-}
-
-/****************************************************************************/
-
-/*
- * SPI device driver setup and teardown
- */
-
-struct flash_info {
- /* JEDEC id zero means "no ID" (most older chips); otherwise it has
- * a high byte of zero plus three data bytes: the manufacturer id,
- * then a two byte device id.
- */
- u32 jedec_id;
- u16 ext_id;
-
- /* The size listed here is what works with OPCODE_SE, which isn't
- * necessarily called a "sector" by the vendor.
- */
- unsigned sector_size;
- u16 n_sectors;
-
- u16 page_size;
- u16 addr_width;
-
- u16 flags;
-#define SECT_4K 0x01 /* OPCODE_BE_4K works uniformly */
-#define M25P_NO_ERASE 0x02 /* No erase command needed */
-#define SST_WRITE 0x04 /* use SST byte programming */
-#define M25P_NO_FR 0x08 /* Can't do fastread */
-#define SECT_4K_PMC 0x10 /* OPCODE_BE_4K_PMC works uniformly */
-#define M25P80_DUAL_READ 0x20 /* Flash supports Dual Read */
-#define M25P80_QUAD_READ 0x40 /* Flash supports Quad Read */
-};
-
-#define INFO(_jedec_id, _ext_id, _sector_size, _n_sectors, _flags) \
- ((kernel_ulong_t)&(struct flash_info) { \
- .jedec_id = (_jedec_id), \
- .ext_id = (_ext_id), \
- .sector_size = (_sector_size), \
- .n_sectors = (_n_sectors), \
- .page_size = 256, \
- .flags = (_flags), \
- })
-
-#define CAT25_INFO(_sector_size, _n_sectors, _page_size, _addr_width, _flags) \
- ((kernel_ulong_t)&(struct flash_info) { \
- .sector_size = (_sector_size), \
- .n_sectors = (_n_sectors), \
- .page_size = (_page_size), \
- .addr_width = (_addr_width), \
- .flags = (_flags), \
- })
-
-/* NOTE: double check command sets and memory organization when you add
- * more flash chips. This current list focusses on newer chips, which
- * have been converging on command sets which including JEDEC ID.
- */
-static const struct spi_device_id m25p_ids[] = {
- /* Atmel -- some are (confusingly) marketed as "DataFlash" */
- { "at25fs010", INFO(0x1f6601, 0, 32 * 1024, 4, SECT_4K) },
- { "at25fs040", INFO(0x1f6604, 0, 64 * 1024, 8, SECT_4K) },
-
- { "at25df041a", INFO(0x1f4401, 0, 64 * 1024, 8, SECT_4K) },
- { "at25df321a", INFO(0x1f4701, 0, 64 * 1024, 64, SECT_4K) },
- { "at25df641", INFO(0x1f4800, 0, 64 * 1024, 128, SECT_4K) },
-
- { "at26f004", INFO(0x1f0400, 0, 64 * 1024, 8, SECT_4K) },
- { "at26df081a", INFO(0x1f4501, 0, 64 * 1024, 16, SECT_4K) },
- { "at26df161a", INFO(0x1f4601, 0, 64 * 1024, 32, SECT_4K) },
- { "at26df321", INFO(0x1f4700, 0, 64 * 1024, 64, SECT_4K) },
-
- { "at45db081d", INFO(0x1f2500, 0, 64 * 1024, 16, SECT_4K) },
-
- /* EON -- en25xxx */
- { "en25f32", INFO(0x1c3116, 0, 64 * 1024, 64, SECT_4K) },
- { "en25p32", INFO(0x1c2016, 0, 64 * 1024, 64, 0) },
- { "en25q32b", INFO(0x1c3016, 0, 64 * 1024, 64, 0) },
- { "en25p64", INFO(0x1c2017, 0, 64 * 1024, 128, 0) },
- { "en25q64", INFO(0x1c3017, 0, 64 * 1024, 128, SECT_4K) },
- { "en25qh256", INFO(0x1c7019, 0, 64 * 1024, 512, 0) },
-
- /* ESMT */
- { "f25l32pa", INFO(0x8c2016, 0, 64 * 1024, 64, SECT_4K) },
-
- /* Everspin */
- { "mr25h256", CAT25_INFO( 32 * 1024, 1, 256, 2, M25P_NO_ERASE | M25P_NO_FR) },
- { "mr25h10", CAT25_INFO(128 * 1024, 1, 256, 3, M25P_NO_ERASE | M25P_NO_FR) },
-
- /* GigaDevice */
- { "gd25q32", INFO(0xc84016, 0, 64 * 1024, 64, SECT_4K) },
- { "gd25q64", INFO(0xc84017, 0, 64 * 1024, 128, SECT_4K) },
-
- /* Intel/Numonyx -- xxxs33b */
- { "160s33b", INFO(0x898911, 0, 64 * 1024, 32, 0) },
- { "320s33b", INFO(0x898912, 0, 64 * 1024, 64, 0) },
- { "640s33b", INFO(0x898913, 0, 64 * 1024, 128, 0) },
-
- /* Macronix */
- { "mx25l2005a", INFO(0xc22012, 0, 64 * 1024, 4, SECT_4K) },
- { "mx25l4005a", INFO(0xc22013, 0, 64 * 1024, 8, SECT_4K) },
- { "mx25l8005", INFO(0xc22014, 0, 64 * 1024, 16, 0) },
- { "mx25l1606e", INFO(0xc22015, 0, 64 * 1024, 32, SECT_4K) },
- { "mx25l3205d", INFO(0xc22016, 0, 64 * 1024, 64, 0) },
- { "mx25l3255e", INFO(0xc29e16, 0, 64 * 1024, 64, SECT_4K) },
- { "mx25l6405d", INFO(0xc22017, 0, 64 * 1024, 128, 0) },
- { "mx25l12805d", INFO(0xc22018, 0, 64 * 1024, 256, 0) },
- { "mx25l12855e", INFO(0xc22618, 0, 64 * 1024, 256, 0) },
- { "mx25l25635e", INFO(0xc22019, 0, 64 * 1024, 512, 0) },
- { "mx25l25655e", INFO(0xc22619, 0, 64 * 1024, 512, 0) },
- { "mx66l51235l", INFO(0xc2201a, 0, 64 * 1024, 1024, M25P80_QUAD_READ) },
- { "mx66l1g55g", INFO(0xc2261b, 0, 64 * 1024, 2048, M25P80_QUAD_READ) },
-
- /* Micron */
- { "n25q064", INFO(0x20ba17, 0, 64 * 1024, 128, 0) },
- { "n25q128a11", INFO(0x20bb18, 0, 64 * 1024, 256, 0) },
- { "n25q128a13", INFO(0x20ba18, 0, 64 * 1024, 256, 0) },
- { "n25q256a", INFO(0x20ba19, 0, 64 * 1024, 512, SECT_4K) },
- { "n25q512a", INFO(0x20bb20, 0, 64 * 1024, 1024, SECT_4K) },
-
- /* PMC */
- { "pm25lv512", INFO(0, 0, 32 * 1024, 2, SECT_4K_PMC) },
- { "pm25lv010", INFO(0, 0, 32 * 1024, 4, SECT_4K_PMC) },
- { "pm25lq032", INFO(0x7f9d46, 0, 64 * 1024, 64, SECT_4K) },
-
- /* Spansion -- single (large) sector size only, at least
- * for the chips listed here (without boot sectors).
- */
- { "s25sl032p", INFO(0x010215, 0x4d00, 64 * 1024, 64, 0) },
- { "s25sl064p", INFO(0x010216, 0x4d00, 64 * 1024, 128, 0) },
- { "s25fl256s0", INFO(0x010219, 0x4d00, 256 * 1024, 128, 0) },
- { "s25fl256s1", INFO(0x010219, 0x4d01, 64 * 1024, 512, M25P80_DUAL_READ | M25P80_QUAD_READ) },
- { "s25fl512s", INFO(0x010220, 0x4d00, 256 * 1024, 256, M25P80_DUAL_READ | M25P80_QUAD_READ) },
- { "s70fl01gs", INFO(0x010221, 0x4d00, 256 * 1024, 256, 0) },
- { "s25sl12800", INFO(0x012018, 0x0300, 256 * 1024, 64, 0) },
- { "s25sl12801", INFO(0x012018, 0x0301, 64 * 1024, 256, 0) },
- { "s25fl129p0", INFO(0x012018, 0x4d00, 256 * 1024, 64, 0) },
- { "s25fl129p1", INFO(0x012018, 0x4d01, 64 * 1024, 256, 0) },
- { "s25sl004a", INFO(0x010212, 0, 64 * 1024, 8, 0) },
- { "s25sl008a", INFO(0x010213, 0, 64 * 1024, 16, 0) },
- { "s25sl016a", INFO(0x010214, 0, 64 * 1024, 32, 0) },
- { "s25sl032a", INFO(0x010215, 0, 64 * 1024, 64, 0) },
- { "s25sl064a", INFO(0x010216, 0, 64 * 1024, 128, 0) },
- { "s25fl008k", INFO(0xef4014, 0, 64 * 1024, 16, SECT_4K) },
- { "s25fl016k", INFO(0xef4015, 0, 64 * 1024, 32, SECT_4K) },
- { "s25fl064k", INFO(0xef4017, 0, 64 * 1024, 128, SECT_4K) },
-
- /* SST -- large erase sizes are "overlays", "sectors" are 4K */
- { "sst25vf040b", INFO(0xbf258d, 0, 64 * 1024, 8, SECT_4K | SST_WRITE) },
- { "sst25vf080b", INFO(0xbf258e, 0, 64 * 1024, 16, SECT_4K | SST_WRITE) },
- { "sst25vf016b", INFO(0xbf2541, 0, 64 * 1024, 32, SECT_4K | SST_WRITE) },
- { "sst25vf032b", INFO(0xbf254a, 0, 64 * 1024, 64, SECT_4K | SST_WRITE) },
- { "sst25vf064c", INFO(0xbf254b, 0, 64 * 1024, 128, SECT_4K) },
- { "sst25wf512", INFO(0xbf2501, 0, 64 * 1024, 1, SECT_4K | SST_WRITE) },
- { "sst25wf010", INFO(0xbf2502, 0, 64 * 1024, 2, SECT_4K | SST_WRITE) },
- { "sst25wf020", INFO(0xbf2503, 0, 64 * 1024, 4, SECT_4K | SST_WRITE) },
- { "sst25wf040", INFO(0xbf2504, 0, 64 * 1024, 8, SECT_4K | SST_WRITE) },
-
- /* ST Microelectronics -- newer production may have feature updates */
- { "m25p05", INFO(0x202010, 0, 32 * 1024, 2, 0) },
- { "m25p10", INFO(0x202011, 0, 32 * 1024, 4, 0) },
- { "m25p20", INFO(0x202012, 0, 64 * 1024, 4, 0) },
- { "m25p40", INFO(0x202013, 0, 64 * 1024, 8, 0) },
- { "m25p80", INFO(0x202014, 0, 64 * 1024, 16, 0) },
- { "m25p16", INFO(0x202015, 0, 64 * 1024, 32, 0) },
- { "m25p32", INFO(0x202016, 0, 64 * 1024, 64, 0) },
- { "m25p64", INFO(0x202017, 0, 64 * 1024, 128, 0) },
- { "m25p128", INFO(0x202018, 0, 256 * 1024, 64, 0) },
- { "n25q032", INFO(0x20ba16, 0, 64 * 1024, 64, 0) },
-
- { "m25p05-nonjedec", INFO(0, 0, 32 * 1024, 2, 0) },
- { "m25p10-nonjedec", INFO(0, 0, 32 * 1024, 4, 0) },
- { "m25p20-nonjedec", INFO(0, 0, 64 * 1024, 4, 0) },
- { "m25p40-nonjedec", INFO(0, 0, 64 * 1024, 8, 0) },
- { "m25p80-nonjedec", INFO(0, 0, 64 * 1024, 16, 0) },
- { "m25p16-nonjedec", INFO(0, 0, 64 * 1024, 32, 0) },
- { "m25p32-nonjedec", INFO(0, 0, 64 * 1024, 64, 0) },
- { "m25p64-nonjedec", INFO(0, 0, 64 * 1024, 128, 0) },
- { "m25p128-nonjedec", INFO(0, 0, 256 * 1024, 64, 0) },
-
- { "m45pe10", INFO(0x204011, 0, 64 * 1024, 2, 0) },
- { "m45pe80", INFO(0x204014, 0, 64 * 1024, 16, 0) },
- { "m45pe16", INFO(0x204015, 0, 64 * 1024, 32, 0) },
-
- { "m25pe20", INFO(0x208012, 0, 64 * 1024, 4, 0) },
- { "m25pe80", INFO(0x208014, 0, 64 * 1024, 16, 0) },
- { "m25pe16", INFO(0x208015, 0, 64 * 1024, 32, SECT_4K) },
-
- { "m25px16", INFO(0x207115, 0, 64 * 1024, 32, SECT_4K) },
- { "m25px32", INFO(0x207116, 0, 64 * 1024, 64, SECT_4K) },
- { "m25px32-s0", INFO(0x207316, 0, 64 * 1024, 64, SECT_4K) },
- { "m25px32-s1", INFO(0x206316, 0, 64 * 1024, 64, SECT_4K) },
- { "m25px64", INFO(0x207117, 0, 64 * 1024, 128, 0) },
-
- /* Winbond -- w25x "blocks" are 64K, "sectors" are 4KiB */
- { "w25x10", INFO(0xef3011, 0, 64 * 1024, 2, SECT_4K) },
- { "w25x20", INFO(0xef3012, 0, 64 * 1024, 4, SECT_4K) },
- { "w25x40", INFO(0xef3013, 0, 64 * 1024, 8, SECT_4K) },
- { "w25x80", INFO(0xef3014, 0, 64 * 1024, 16, SECT_4K) },
- { "w25x16", INFO(0xef3015, 0, 64 * 1024, 32, SECT_4K) },
- { "w25x32", INFO(0xef3016, 0, 64 * 1024, 64, SECT_4K) },
- { "w25q32", INFO(0xef4016, 0, 64 * 1024, 64, SECT_4K) },
- { "w25q32dw", INFO(0xef6016, 0, 64 * 1024, 64, SECT_4K) },
- { "w25x64", INFO(0xef3017, 0, 64 * 1024, 128, SECT_4K) },
- { "w25q64", INFO(0xef4017, 0, 64 * 1024, 128, SECT_4K) },
- { "w25q128", INFO(0xef4018, 0, 64 * 1024, 256, SECT_4K) },
- { "w25q80", INFO(0xef5014, 0, 64 * 1024, 16, SECT_4K) },
- { "w25q80bl", INFO(0xef4014, 0, 64 * 1024, 16, SECT_4K) },
- { "w25q128", INFO(0xef4018, 0, 64 * 1024, 256, SECT_4K) },
- { "w25q256", INFO(0xef4019, 0, 64 * 1024, 512, SECT_4K) },
-
- /* Catalyst / On Semiconductor -- non-JEDEC */
- { "cat25c11", CAT25_INFO( 16, 8, 16, 1, M25P_NO_ERASE | M25P_NO_FR) },
- { "cat25c03", CAT25_INFO( 32, 8, 16, 2, M25P_NO_ERASE | M25P_NO_FR) },
- { "cat25c09", CAT25_INFO( 128, 8, 32, 2, M25P_NO_ERASE | M25P_NO_FR) },
- { "cat25c17", CAT25_INFO( 256, 8, 32, 2, M25P_NO_ERASE | M25P_NO_FR) },
- { "cat25128", CAT25_INFO(2048, 8, 64, 2, M25P_NO_ERASE | M25P_NO_FR) },
- { },
-};
-MODULE_DEVICE_TABLE(spi, m25p_ids);
-
-static const struct spi_device_id *jedec_probe(struct spi_device *spi)
-{
- int tmp;
- u8 code = OPCODE_RDID;
- u8 id[5];
- u32 jedec;
- u16 ext_jedec;
- struct flash_info *info;
-
- /* JEDEC also defines an optional "extended device information"
- * string for after vendor-specific data, after the three bytes
- * we use here. Supporting some chips might require using it.
- */
- tmp = spi_write_then_read(spi, &code, 1, id, 5);
- if (tmp < 0) {
- pr_debug("%s: error %d reading JEDEC ID\n",
- dev_name(&spi->dev), tmp);
- return ERR_PTR(tmp);
- }
- jedec = id[0];
- jedec = jedec << 8;
- jedec |= id[1];
- jedec = jedec << 8;
- jedec |= id[2];
-
- ext_jedec = id[3] << 8 | id[4];
-
- for (tmp = 0; tmp < ARRAY_SIZE(m25p_ids) - 1; tmp++) {
- info = (void *)m25p_ids[tmp].driver_data;
- if (info->jedec_id == jedec) {
- if (info->ext_id == 0 || info->ext_id == ext_jedec)
- return &m25p_ids[tmp];
- }
- }
- dev_err(&spi->dev, "unrecognized JEDEC id %06x\n", jedec);
- return ERR_PTR(-ENODEV);
-}
-
-
/*
* board specific setup should have ensured the SPI clock used here
* matches what the READ command supports, at least until this driver
@@ -1096,231 +191,45 @@
*/
static int m25p_probe(struct spi_device *spi)
{
- const struct spi_device_id *id = spi_get_device_id(spi);
- struct flash_platform_data *data;
- struct m25p *flash;
- struct flash_info *info;
- unsigned i;
struct mtd_part_parser_data ppdata;
- struct device_node *np = spi->dev.of_node;
+ struct flash_platform_data *data;
+ struct m25p *flash;
+ struct spi_nor *nor;
+ enum read_mode mode = SPI_NOR_NORMAL;
int ret;
- /* Platform data helps sort out which chip type we have, as
- * well as how this board partitions it. If we don't have
- * a chip ID, try the JEDEC id commands; they'll work for most
- * newer chips, even if we don't recognize the particular chip.
- */
- data = dev_get_platdata(&spi->dev);
- if (data && data->type) {
- const struct spi_device_id *plat_id;
-
- for (i = 0; i < ARRAY_SIZE(m25p_ids) - 1; i++) {
- plat_id = &m25p_ids[i];
- if (strcmp(data->type, plat_id->name))
- continue;
- break;
- }
-
- if (i < ARRAY_SIZE(m25p_ids) - 1)
- id = plat_id;
- else
- dev_warn(&spi->dev, "unrecognized id %s\n", data->type);
- }
-
- info = (void *)id->driver_data;
-
- if (info->jedec_id) {
- const struct spi_device_id *jid;
-
- jid = jedec_probe(spi);
- if (IS_ERR(jid)) {
- return PTR_ERR(jid);
- } else if (jid != id) {
- /*
- * JEDEC knows better, so overwrite platform ID. We
- * can't trust partitions any longer, but we'll let
- * mtd apply them anyway, since some partitions may be
- * marked read-only, and we don't want to lose that
- * information, even if it's not 100% accurate.
- */
- dev_warn(&spi->dev, "found %s, expected %s\n",
- jid->name, id->name);
- id = jid;
- info = (void *)jid->driver_data;
- }
- }
-
flash = devm_kzalloc(&spi->dev, sizeof(*flash), GFP_KERNEL);
if (!flash)
return -ENOMEM;
- flash->command = devm_kzalloc(&spi->dev, MAX_CMD_SIZE, GFP_KERNEL);
- if (!flash->command)
- return -ENOMEM;
+ nor = &flash->spi_nor;
- flash->spi = spi;
- mutex_init(&flash->lock);
+ /* install the hooks */
+ nor->read = m25p80_read;
+ nor->write = m25p80_write;
+ nor->erase = m25p80_erase;
+ nor->write_reg = m25p80_write_reg;
+ nor->read_reg = m25p80_read_reg;
+
+ nor->dev = &spi->dev;
+ nor->mtd = &flash->mtd;
+ nor->priv = flash;
+
spi_set_drvdata(spi, flash);
+ flash->mtd.priv = nor;
+ flash->spi = spi;
- /*
- * Atmel, SST and Intel/Numonyx serial flash tend to power
- * up with the software protection bits set
- */
+ if (spi->mode & SPI_RX_QUAD)
+ mode = SPI_NOR_QUAD;
+ else if (spi->mode & SPI_RX_DUAL)
+ mode = SPI_NOR_DUAL;
+ ret = spi_nor_scan(nor, spi_get_device_id(spi), mode);
+ if (ret)
+ return ret;
- if (JEDEC_MFR(info->jedec_id) == CFI_MFR_ATMEL ||
- JEDEC_MFR(info->jedec_id) == CFI_MFR_INTEL ||
- JEDEC_MFR(info->jedec_id) == CFI_MFR_SST) {
- write_enable(flash);
- write_sr(flash, 0);
- }
-
- if (data && data->name)
- flash->mtd.name = data->name;
- else
- flash->mtd.name = dev_name(&spi->dev);
-
- flash->mtd.type = MTD_NORFLASH;
- flash->mtd.writesize = 1;
- flash->mtd.flags = MTD_CAP_NORFLASH;
- flash->mtd.size = info->sector_size * info->n_sectors;
- flash->mtd._erase = m25p80_erase;
- flash->mtd._read = m25p80_read;
-
- /* flash protection support for STmicro chips */
- if (JEDEC_MFR(info->jedec_id) == CFI_MFR_ST) {
- flash->mtd._lock = m25p80_lock;
- flash->mtd._unlock = m25p80_unlock;
- }
-
- /* sst flash chips use AAI word program */
- if (info->flags & SST_WRITE)
- flash->mtd._write = sst_write;
- else
- flash->mtd._write = m25p80_write;
-
- /* prefer "small sector" erase if possible */
- if (info->flags & SECT_4K) {
- flash->erase_opcode = OPCODE_BE_4K;
- flash->mtd.erasesize = 4096;
- } else if (info->flags & SECT_4K_PMC) {
- flash->erase_opcode = OPCODE_BE_4K_PMC;
- flash->mtd.erasesize = 4096;
- } else {
- flash->erase_opcode = OPCODE_SE;
- flash->mtd.erasesize = info->sector_size;
- }
-
- if (info->flags & M25P_NO_ERASE)
- flash->mtd.flags |= MTD_NO_ERASE;
-
+ data = dev_get_platdata(&spi->dev);
ppdata.of_node = spi->dev.of_node;
- flash->mtd.dev.parent = &spi->dev;
- flash->page_size = info->page_size;
- flash->mtd.writebufsize = flash->page_size;
- if (np) {
- /* If we were instantiated by DT, use it */
- if (of_property_read_bool(np, "m25p,fast-read"))
- flash->flash_read = M25P80_FAST;
- else
- flash->flash_read = M25P80_NORMAL;
- } else {
- /* If we weren't instantiated by DT, default to fast-read */
- flash->flash_read = M25P80_FAST;
- }
-
- /* Some devices cannot do fast-read, no matter what DT tells us */
- if (info->flags & M25P_NO_FR)
- flash->flash_read = M25P80_NORMAL;
-
- /* Quad/Dual-read mode takes precedence over fast/normal */
- if (spi->mode & SPI_RX_QUAD && info->flags & M25P80_QUAD_READ) {
- ret = set_quad_mode(flash, info->jedec_id);
- if (ret) {
- dev_err(&flash->spi->dev, "quad mode not supported\n");
- return ret;
- }
- flash->flash_read = M25P80_QUAD;
- } else if (spi->mode & SPI_RX_DUAL && info->flags & M25P80_DUAL_READ) {
- flash->flash_read = M25P80_DUAL;
- }
-
- /* Default commands */
- switch (flash->flash_read) {
- case M25P80_QUAD:
- flash->read_opcode = OPCODE_QUAD_READ;
- break;
- case M25P80_DUAL:
- flash->read_opcode = OPCODE_DUAL_READ;
- break;
- case M25P80_FAST:
- flash->read_opcode = OPCODE_FAST_READ;
- break;
- case M25P80_NORMAL:
- flash->read_opcode = OPCODE_NORM_READ;
- break;
- default:
- dev_err(&flash->spi->dev, "No Read opcode defined\n");
- return -EINVAL;
- }
-
- flash->program_opcode = OPCODE_PP;
-
- if (info->addr_width)
- flash->addr_width = info->addr_width;
- else if (flash->mtd.size > 0x1000000) {
- /* enable 4-byte addressing if the device exceeds 16MiB */
- flash->addr_width = 4;
- if (JEDEC_MFR(info->jedec_id) == CFI_MFR_AMD) {
- /* Dedicated 4-byte command set */
- switch (flash->flash_read) {
- case M25P80_QUAD:
- flash->read_opcode = OPCODE_QUAD_READ_4B;
- break;
- case M25P80_DUAL:
- flash->read_opcode = OPCODE_DUAL_READ_4B;
- break;
- case M25P80_FAST:
- flash->read_opcode = OPCODE_FAST_READ_4B;
- break;
- case M25P80_NORMAL:
- flash->read_opcode = OPCODE_NORM_READ_4B;
- break;
- }
- flash->program_opcode = OPCODE_PP_4B;
- /* No small sector erase for 4-byte command set */
- flash->erase_opcode = OPCODE_SE_4B;
- flash->mtd.erasesize = info->sector_size;
- } else
- set_4byte(flash, info->jedec_id, 1);
- } else {
- flash->addr_width = 3;
- }
-
- dev_info(&spi->dev, "%s (%lld Kbytes)\n", id->name,
- (long long)flash->mtd.size >> 10);
-
- pr_debug("mtd .name = %s, .size = 0x%llx (%lldMiB) "
- ".erasesize = 0x%.8x (%uKiB) .numeraseregions = %d\n",
- flash->mtd.name,
- (long long)flash->mtd.size, (long long)(flash->mtd.size >> 20),
- flash->mtd.erasesize, flash->mtd.erasesize / 1024,
- flash->mtd.numeraseregions);
-
- if (flash->mtd.numeraseregions)
- for (i = 0; i < flash->mtd.numeraseregions; i++)
- pr_debug("mtd.eraseregions[%d] = { .offset = 0x%llx, "
- ".erasesize = 0x%.8x (%uKiB), "
- ".numblocks = %d }\n",
- i, (long long)flash->mtd.eraseregions[i].offset,
- flash->mtd.eraseregions[i].erasesize,
- flash->mtd.eraseregions[i].erasesize / 1024,
- flash->mtd.eraseregions[i].numblocks);
-
-
- /* partitions should match sector boundaries; and it may be good to
- * use readonly partitions for writeprotected sectors (BP2..BP0).
- */
return mtd_device_parse_register(&flash->mtd, NULL, &ppdata,
data ? data->parts : NULL,
data ? data->nr_parts : 0);
@@ -1341,7 +250,7 @@
.name = "m25p80",
.owner = THIS_MODULE,
},
- .id_table = m25p_ids,
+ .id_table = spi_nor_ids,
.probe = m25p_probe,
.remove = m25p_remove,
diff --git a/drivers/mtd/devices/serial_flash_cmds.h b/drivers/mtd/devices/serial_flash_cmds.h
index 4f0c2c7c..f59a125 100644
--- a/drivers/mtd/devices/serial_flash_cmds.h
+++ b/drivers/mtd/devices/serial_flash_cmds.h
@@ -13,43 +13,23 @@
#define _MTD_SERIAL_FLASH_CMDS_H
/* Generic Flash Commands/OPCODEs */
-#define FLASH_CMD_WREN 0x06
-#define FLASH_CMD_WRDI 0x04
-#define FLASH_CMD_RDID 0x9f
-#define FLASH_CMD_RDSR 0x05
-#define FLASH_CMD_RDSR2 0x35
-#define FLASH_CMD_WRSR 0x01
-#define FLASH_CMD_SE_4K 0x20
-#define FLASH_CMD_SE_32K 0x52
-#define FLASH_CMD_SE 0xd8
-#define FLASH_CMD_CHIPERASE 0xc7
-#define FLASH_CMD_WRVCR 0x81
-#define FLASH_CMD_RDVCR 0x85
+#define SPINOR_OP_RDSR2 0x35
+#define SPINOR_OP_WRVCR 0x81
+#define SPINOR_OP_RDVCR 0x85
/* JEDEC Standard - Serial Flash Discoverable Parmeters (SFDP) Commands */
-#define FLASH_CMD_READ 0x03 /* READ */
-#define FLASH_CMD_READ_FAST 0x0b /* FAST READ */
-#define FLASH_CMD_READ_1_1_2 0x3b /* DUAL OUTPUT READ */
-#define FLASH_CMD_READ_1_2_2 0xbb /* DUAL I/O READ */
-#define FLASH_CMD_READ_1_1_4 0x6b /* QUAD OUTPUT READ */
-#define FLASH_CMD_READ_1_4_4 0xeb /* QUAD I/O READ */
+#define SPINOR_OP_READ_1_2_2 0xbb /* DUAL I/O READ */
+#define SPINOR_OP_READ_1_4_4 0xeb /* QUAD I/O READ */
-#define FLASH_CMD_WRITE 0x02 /* PAGE PROGRAM */
-#define FLASH_CMD_WRITE_1_1_2 0xa2 /* DUAL INPUT PROGRAM */
-#define FLASH_CMD_WRITE_1_2_2 0xd2 /* DUAL INPUT EXT PROGRAM */
-#define FLASH_CMD_WRITE_1_1_4 0x32 /* QUAD INPUT PROGRAM */
-#define FLASH_CMD_WRITE_1_4_4 0x12 /* QUAD INPUT EXT PROGRAM */
-
-#define FLASH_CMD_EN4B_ADDR 0xb7 /* Enter 4-byte address mode */
-#define FLASH_CMD_EX4B_ADDR 0xe9 /* Exit 4-byte address mode */
+#define SPINOR_OP_WRITE 0x02 /* PAGE PROGRAM */
+#define SPINOR_OP_WRITE_1_1_2 0xa2 /* DUAL INPUT PROGRAM */
+#define SPINOR_OP_WRITE_1_2_2 0xd2 /* DUAL INPUT EXT PROGRAM */
+#define SPINOR_OP_WRITE_1_1_4 0x32 /* QUAD INPUT PROGRAM */
+#define SPINOR_OP_WRITE_1_4_4 0x12 /* QUAD INPUT EXT PROGRAM */
/* READ commands with 32-bit addressing */
-#define FLASH_CMD_READ4 0x13
-#define FLASH_CMD_READ4_FAST 0x0c
-#define FLASH_CMD_READ4_1_1_2 0x3c
-#define FLASH_CMD_READ4_1_2_2 0xbc
-#define FLASH_CMD_READ4_1_1_4 0x6c
-#define FLASH_CMD_READ4_1_4_4 0xec
+#define SPINOR_OP_READ4_1_2_2 0xbc
+#define SPINOR_OP_READ4_1_4_4 0xec
/* Configuration flags */
#define FLASH_FLAG_SINGLE 0x000000ff
diff --git a/drivers/mtd/devices/slram.c b/drivers/mtd/devices/slram.c
index 5a5cd2a..2fc4957c 100644
--- a/drivers/mtd/devices/slram.c
+++ b/drivers/mtd/devices/slram.c
@@ -280,14 +280,11 @@
static int __init init_slram(void)
{
char *devname;
- int i;
#ifndef MODULE
char *devstart;
char *devlength;
- i = 0;
-
if (!map) {
E("slram: not enough parameters.\n");
return(-EINVAL);
@@ -314,6 +311,7 @@
}
#else
int count;
+ int i;
for (count = 0; count < SLRAM_MAX_DEVICES_PARAMS && map[count];
count++) {
diff --git a/drivers/mtd/devices/st_spi_fsm.c b/drivers/mtd/devices/st_spi_fsm.c
index 1957d7c..d252514 100644
--- a/drivers/mtd/devices/st_spi_fsm.c
+++ b/drivers/mtd/devices/st_spi_fsm.c
@@ -19,6 +19,7 @@
#include <linux/mfd/syscon.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/partitions.h>
+#include <linux/mtd/spi-nor.h>
#include <linux/sched.h>
#include <linux/delay.h>
#include <linux/io.h>
@@ -201,44 +202,6 @@
#define STFSM_MAX_WAIT_SEQ_MS 1000 /* FSM execution time */
-/* Flash Commands */
-#define FLASH_CMD_WREN 0x06
-#define FLASH_CMD_WRDI 0x04
-#define FLASH_CMD_RDID 0x9f
-#define FLASH_CMD_RDSR 0x05
-#define FLASH_CMD_RDSR2 0x35
-#define FLASH_CMD_WRSR 0x01
-#define FLASH_CMD_SE_4K 0x20
-#define FLASH_CMD_SE_32K 0x52
-#define FLASH_CMD_SE 0xd8
-#define FLASH_CMD_CHIPERASE 0xc7
-#define FLASH_CMD_WRVCR 0x81
-#define FLASH_CMD_RDVCR 0x85
-
-#define FLASH_CMD_READ 0x03 /* READ */
-#define FLASH_CMD_READ_FAST 0x0b /* FAST READ */
-#define FLASH_CMD_READ_1_1_2 0x3b /* DUAL OUTPUT READ */
-#define FLASH_CMD_READ_1_2_2 0xbb /* DUAL I/O READ */
-#define FLASH_CMD_READ_1_1_4 0x6b /* QUAD OUTPUT READ */
-#define FLASH_CMD_READ_1_4_4 0xeb /* QUAD I/O READ */
-
-#define FLASH_CMD_WRITE 0x02 /* PAGE PROGRAM */
-#define FLASH_CMD_WRITE_1_1_2 0xa2 /* DUAL INPUT PROGRAM */
-#define FLASH_CMD_WRITE_1_2_2 0xd2 /* DUAL INPUT EXT PROGRAM */
-#define FLASH_CMD_WRITE_1_1_4 0x32 /* QUAD INPUT PROGRAM */
-#define FLASH_CMD_WRITE_1_4_4 0x12 /* QUAD INPUT EXT PROGRAM */
-
-#define FLASH_CMD_EN4B_ADDR 0xb7 /* Enter 4-byte address mode */
-#define FLASH_CMD_EX4B_ADDR 0xe9 /* Exit 4-byte address mode */
-
-/* READ commands with 32-bit addressing (N25Q256 and S25FLxxxS) */
-#define FLASH_CMD_READ4 0x13
-#define FLASH_CMD_READ4_FAST 0x0c
-#define FLASH_CMD_READ4_1_1_2 0x3c
-#define FLASH_CMD_READ4_1_2_2 0xbc
-#define FLASH_CMD_READ4_1_1_4 0x6c
-#define FLASH_CMD_READ4_1_4_4 0xec
-
/* S25FLxxxS commands */
#define S25FL_CMD_WRITE4_1_1_4 0x34
#define S25FL_CMD_SE4 0xdc
@@ -246,7 +209,7 @@
#define S25FL_CMD_DYBWR 0xe1
#define S25FL_CMD_DYBRD 0xe0
#define S25FL_CMD_WRITE4 0x12 /* Note, opcode clashes with
- * 'FLASH_CMD_WRITE_1_4_4'
+ * 'SPINOR_OP_WRITE_1_4_4'
* as found on N25Qxxx devices! */
/* Status register */
@@ -261,6 +224,12 @@
#define S25FL_STATUS_E_ERR 0x20
#define S25FL_STATUS_P_ERR 0x40
+#define N25Q_CMD_WRVCR 0x81
+#define N25Q_CMD_RDVCR 0x85
+#define N25Q_CMD_RDVECR 0x65
+#define N25Q_CMD_RDNVCR 0xb5
+#define N25Q_CMD_WRNVCR 0xb1
+
#define FLASH_PAGESIZE 256 /* In Bytes */
#define FLASH_PAGESIZE_32 (FLASH_PAGESIZE / 4) /* In uint32_t */
#define FLASH_MAX_BUSY_WAIT (300 * HZ) /* Maximum 'CHIPERASE' time */
@@ -270,7 +239,6 @@
*/
#define CFG_READ_TOGGLE_32BIT_ADDR 0x00000001
#define CFG_WRITE_TOGGLE_32BIT_ADDR 0x00000002
-#define CFG_WRITE_EX_32BIT_ADDR_DELAY 0x00000004
#define CFG_ERASESEC_TOGGLE_32BIT_ADDR 0x00000008
#define CFG_S25FL_CHECK_ERROR_FLAGS 0x00000010
@@ -329,7 +297,7 @@
u32 jedec_id;
u16 ext_id;
/*
- * The size listed here is what works with FLASH_CMD_SE, which isn't
+ * The size listed here is what works with SPINOR_OP_SE, which isn't
* necessarily called a "sector" by the vendor.
*/
unsigned sector_size;
@@ -369,17 +337,26 @@
{ "m25px32", 0x207116, 0, 64 * 1024, 64, M25PX_FLAG, 75, NULL },
{ "m25px64", 0x207117, 0, 64 * 1024, 128, M25PX_FLAG, 75, NULL },
+ /* Macronix MX25xxx
+ * - Support for 'FLASH_FLAG_WRITE_1_4_4' is omitted for devices
+ * where operating frequency must be reduced.
+ */
#define MX25_FLAG (FLASH_FLAG_READ_WRITE | \
FLASH_FLAG_READ_FAST | \
FLASH_FLAG_READ_1_1_2 | \
FLASH_FLAG_READ_1_2_2 | \
FLASH_FLAG_READ_1_1_4 | \
- FLASH_FLAG_READ_1_4_4 | \
FLASH_FLAG_SE_4K | \
FLASH_FLAG_SE_32K)
+ { "mx25l3255e", 0xc29e16, 0, 64 * 1024, 64,
+ (MX25_FLAG | FLASH_FLAG_WRITE_1_4_4), 86,
+ stfsm_mx25_config},
{ "mx25l25635e", 0xc22019, 0, 64*1024, 512,
(MX25_FLAG | FLASH_FLAG_32BIT_ADDR | FLASH_FLAG_RESET), 70,
stfsm_mx25_config },
+ { "mx25l25655e", 0xc22619, 0, 64*1024, 512,
+ (MX25_FLAG | FLASH_FLAG_32BIT_ADDR | FLASH_FLAG_RESET), 70,
+ stfsm_mx25_config},
#define N25Q_FLAG (FLASH_FLAG_READ_WRITE | \
FLASH_FLAG_READ_FAST | \
@@ -407,6 +384,8 @@
FLASH_FLAG_READ_1_4_4 | \
FLASH_FLAG_WRITE_1_1_4 | \
FLASH_FLAG_READ_FAST)
+ { "s25fl032p", 0x010215, 0x4d00, 64 * 1024, 64, S25FLXXXP_FLAG, 80,
+ stfsm_s25fl_config},
{ "s25fl129p0", 0x012018, 0x4d00, 256 * 1024, 64, S25FLXXXP_FLAG, 80,
stfsm_s25fl_config },
{ "s25fl129p1", 0x012018, 0x4d01, 64 * 1024, 256, S25FLXXXP_FLAG, 80,
@@ -473,22 +452,22 @@
/* Default READ configurations, in order of preference */
static struct seq_rw_config default_read_configs[] = {
- {FLASH_FLAG_READ_1_4_4, FLASH_CMD_READ_1_4_4, 0, 4, 4, 0x00, 2, 4},
- {FLASH_FLAG_READ_1_1_4, FLASH_CMD_READ_1_1_4, 0, 1, 4, 0x00, 4, 0},
- {FLASH_FLAG_READ_1_2_2, FLASH_CMD_READ_1_2_2, 0, 2, 2, 0x00, 4, 0},
- {FLASH_FLAG_READ_1_1_2, FLASH_CMD_READ_1_1_2, 0, 1, 2, 0x00, 0, 8},
- {FLASH_FLAG_READ_FAST, FLASH_CMD_READ_FAST, 0, 1, 1, 0x00, 0, 8},
- {FLASH_FLAG_READ_WRITE, FLASH_CMD_READ, 0, 1, 1, 0x00, 0, 0},
+ {FLASH_FLAG_READ_1_4_4, SPINOR_OP_READ_1_4_4, 0, 4, 4, 0x00, 2, 4},
+ {FLASH_FLAG_READ_1_1_4, SPINOR_OP_READ_1_1_4, 0, 1, 4, 0x00, 4, 0},
+ {FLASH_FLAG_READ_1_2_2, SPINOR_OP_READ_1_2_2, 0, 2, 2, 0x00, 4, 0},
+ {FLASH_FLAG_READ_1_1_2, SPINOR_OP_READ_1_1_2, 0, 1, 2, 0x00, 0, 8},
+ {FLASH_FLAG_READ_FAST, SPINOR_OP_READ_FAST, 0, 1, 1, 0x00, 0, 8},
+ {FLASH_FLAG_READ_WRITE, SPINOR_OP_READ, 0, 1, 1, 0x00, 0, 0},
{0x00, 0, 0, 0, 0, 0x00, 0, 0},
};
/* Default WRITE configurations */
static struct seq_rw_config default_write_configs[] = {
- {FLASH_FLAG_WRITE_1_4_4, FLASH_CMD_WRITE_1_4_4, 1, 4, 4, 0x00, 0, 0},
- {FLASH_FLAG_WRITE_1_1_4, FLASH_CMD_WRITE_1_1_4, 1, 1, 4, 0x00, 0, 0},
- {FLASH_FLAG_WRITE_1_2_2, FLASH_CMD_WRITE_1_2_2, 1, 2, 2, 0x00, 0, 0},
- {FLASH_FLAG_WRITE_1_1_2, FLASH_CMD_WRITE_1_1_2, 1, 1, 2, 0x00, 0, 0},
- {FLASH_FLAG_READ_WRITE, FLASH_CMD_WRITE, 1, 1, 1, 0x00, 0, 0},
+ {FLASH_FLAG_WRITE_1_4_4, SPINOR_OP_WRITE_1_4_4, 1, 4, 4, 0x00, 0, 0},
+ {FLASH_FLAG_WRITE_1_1_4, SPINOR_OP_WRITE_1_1_4, 1, 1, 4, 0x00, 0, 0},
+ {FLASH_FLAG_WRITE_1_2_2, SPINOR_OP_WRITE_1_2_2, 1, 2, 2, 0x00, 0, 0},
+ {FLASH_FLAG_WRITE_1_1_2, SPINOR_OP_WRITE_1_1_2, 1, 1, 2, 0x00, 0, 0},
+ {FLASH_FLAG_READ_WRITE, SPINOR_OP_WRITE, 1, 1, 1, 0x00, 0, 0},
{0x00, 0, 0, 0, 0, 0x00, 0, 0},
};
@@ -511,12 +490,12 @@
* cycles.
*/
static struct seq_rw_config n25q_read3_configs[] = {
- {FLASH_FLAG_READ_1_4_4, FLASH_CMD_READ_1_4_4, 0, 4, 4, 0x00, 0, 8},
- {FLASH_FLAG_READ_1_1_4, FLASH_CMD_READ_1_1_4, 0, 1, 4, 0x00, 0, 8},
- {FLASH_FLAG_READ_1_2_2, FLASH_CMD_READ_1_2_2, 0, 2, 2, 0x00, 0, 8},
- {FLASH_FLAG_READ_1_1_2, FLASH_CMD_READ_1_1_2, 0, 1, 2, 0x00, 0, 8},
- {FLASH_FLAG_READ_FAST, FLASH_CMD_READ_FAST, 0, 1, 1, 0x00, 0, 8},
- {FLASH_FLAG_READ_WRITE, FLASH_CMD_READ, 0, 1, 1, 0x00, 0, 0},
+ {FLASH_FLAG_READ_1_4_4, SPINOR_OP_READ_1_4_4, 0, 4, 4, 0x00, 0, 8},
+ {FLASH_FLAG_READ_1_1_4, SPINOR_OP_READ_1_1_4, 0, 1, 4, 0x00, 0, 8},
+ {FLASH_FLAG_READ_1_2_2, SPINOR_OP_READ_1_2_2, 0, 2, 2, 0x00, 0, 8},
+ {FLASH_FLAG_READ_1_1_2, SPINOR_OP_READ_1_1_2, 0, 1, 2, 0x00, 0, 8},
+ {FLASH_FLAG_READ_FAST, SPINOR_OP_READ_FAST, 0, 1, 1, 0x00, 0, 8},
+ {FLASH_FLAG_READ_WRITE, SPINOR_OP_READ, 0, 1, 1, 0x00, 0, 0},
{0x00, 0, 0, 0, 0, 0x00, 0, 0},
};
@@ -526,12 +505,12 @@
* - 'FAST' variants configured for 8 dummy cycles (see note above.)
*/
static struct seq_rw_config n25q_read4_configs[] = {
- {FLASH_FLAG_READ_1_4_4, FLASH_CMD_READ4_1_4_4, 0, 4, 4, 0x00, 0, 8},
- {FLASH_FLAG_READ_1_1_4, FLASH_CMD_READ4_1_1_4, 0, 1, 4, 0x00, 0, 8},
- {FLASH_FLAG_READ_1_2_2, FLASH_CMD_READ4_1_2_2, 0, 2, 2, 0x00, 0, 8},
- {FLASH_FLAG_READ_1_1_2, FLASH_CMD_READ4_1_1_2, 0, 1, 2, 0x00, 0, 8},
- {FLASH_FLAG_READ_FAST, FLASH_CMD_READ4_FAST, 0, 1, 1, 0x00, 0, 8},
- {FLASH_FLAG_READ_WRITE, FLASH_CMD_READ4, 0, 1, 1, 0x00, 0, 0},
+ {FLASH_FLAG_READ_1_4_4, SPINOR_OP_READ4_1_4_4, 0, 4, 4, 0x00, 0, 8},
+ {FLASH_FLAG_READ_1_1_4, SPINOR_OP_READ4_1_1_4, 0, 1, 4, 0x00, 0, 8},
+ {FLASH_FLAG_READ_1_2_2, SPINOR_OP_READ4_1_2_2, 0, 2, 2, 0x00, 0, 8},
+ {FLASH_FLAG_READ_1_1_2, SPINOR_OP_READ4_1_1_2, 0, 1, 2, 0x00, 0, 8},
+ {FLASH_FLAG_READ_FAST, SPINOR_OP_READ4_FAST, 0, 1, 1, 0x00, 0, 8},
+ {FLASH_FLAG_READ_WRITE, SPINOR_OP_READ4, 0, 1, 1, 0x00, 0, 0},
{0x00, 0, 0, 0, 0, 0x00, 0, 0},
};
@@ -544,7 +523,7 @@
{
seq->seq_opc[0] = (SEQ_OPC_PADS_1 |
SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_EN4B_ADDR) |
+ SEQ_OPC_OPCODE(SPINOR_OP_EN4B) |
SEQ_OPC_CSDEASSERT);
seq->seq[0] = STFSM_INST_CMD1;
@@ -572,12 +551,12 @@
* entering a state that is incompatible with the SPIBoot Controller.
*/
static struct seq_rw_config stfsm_s25fl_read4_configs[] = {
- {FLASH_FLAG_READ_1_4_4, FLASH_CMD_READ4_1_4_4, 0, 4, 4, 0x00, 2, 4},
- {FLASH_FLAG_READ_1_1_4, FLASH_CMD_READ4_1_1_4, 0, 1, 4, 0x00, 0, 8},
- {FLASH_FLAG_READ_1_2_2, FLASH_CMD_READ4_1_2_2, 0, 2, 2, 0x00, 4, 0},
- {FLASH_FLAG_READ_1_1_2, FLASH_CMD_READ4_1_1_2, 0, 1, 2, 0x00, 0, 8},
- {FLASH_FLAG_READ_FAST, FLASH_CMD_READ4_FAST, 0, 1, 1, 0x00, 0, 8},
- {FLASH_FLAG_READ_WRITE, FLASH_CMD_READ4, 0, 1, 1, 0x00, 0, 0},
+ {FLASH_FLAG_READ_1_4_4, SPINOR_OP_READ4_1_4_4, 0, 4, 4, 0x00, 2, 4},
+ {FLASH_FLAG_READ_1_1_4, SPINOR_OP_READ4_1_1_4, 0, 1, 4, 0x00, 0, 8},
+ {FLASH_FLAG_READ_1_2_2, SPINOR_OP_READ4_1_2_2, 0, 2, 2, 0x00, 4, 0},
+ {FLASH_FLAG_READ_1_1_2, SPINOR_OP_READ4_1_1_2, 0, 1, 2, 0x00, 0, 8},
+ {FLASH_FLAG_READ_FAST, SPINOR_OP_READ4_FAST, 0, 1, 1, 0x00, 0, 8},
+ {FLASH_FLAG_READ_WRITE, SPINOR_OP_READ4, 0, 1, 1, 0x00, 0, 0},
{0x00, 0, 0, 0, 0, 0x00, 0, 0},
};
@@ -590,13 +569,13 @@
/*
* [W25Qxxx] Configuration
*/
-#define W25Q_STATUS_QE (0x1 << 9)
+#define W25Q_STATUS_QE (0x1 << 1)
static struct stfsm_seq stfsm_seq_read_jedec = {
.data_size = TRANSFER_SIZE(8),
.seq_opc[0] = (SEQ_OPC_PADS_1 |
SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_RDID)),
+ SEQ_OPC_OPCODE(SPINOR_OP_RDID)),
.seq = {
STFSM_INST_CMD1,
STFSM_INST_DATA_READ,
@@ -612,7 +591,7 @@
.data_size = TRANSFER_SIZE(4),
.seq_opc[0] = (SEQ_OPC_PADS_1 |
SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_RDSR)),
+ SEQ_OPC_OPCODE(SPINOR_OP_RDSR)),
.seq = {
STFSM_INST_CMD1,
STFSM_INST_DATA_READ,
@@ -628,10 +607,10 @@
/* 'addr_cfg' configured during initialisation */
.seq_opc = {
(SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_WREN) | SEQ_OPC_CSDEASSERT),
+ SEQ_OPC_OPCODE(SPINOR_OP_WREN) | SEQ_OPC_CSDEASSERT),
(SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_SE)),
+ SEQ_OPC_OPCODE(SPINOR_OP_SE)),
},
.seq = {
STFSM_INST_CMD1,
@@ -649,10 +628,10 @@
static struct stfsm_seq stfsm_seq_erase_chip = {
.seq_opc = {
(SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_WREN) | SEQ_OPC_CSDEASSERT),
+ SEQ_OPC_OPCODE(SPINOR_OP_WREN) | SEQ_OPC_CSDEASSERT),
(SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_CHIPERASE) | SEQ_OPC_CSDEASSERT),
+ SEQ_OPC_OPCODE(SPINOR_OP_CHIP_ERASE) | SEQ_OPC_CSDEASSERT),
},
.seq = {
STFSM_INST_CMD1,
@@ -669,26 +648,9 @@
static struct stfsm_seq stfsm_seq_write_status = {
.seq_opc[0] = (SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_WREN) | SEQ_OPC_CSDEASSERT),
+ SEQ_OPC_OPCODE(SPINOR_OP_WREN) | SEQ_OPC_CSDEASSERT),
.seq_opc[1] = (SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_WRSR)),
- .seq = {
- STFSM_INST_CMD1,
- STFSM_INST_CMD2,
- STFSM_INST_STA_WR1,
- STFSM_INST_STOP,
- },
- .seq_cfg = (SEQ_CFG_PADS_1 |
- SEQ_CFG_READNOTWRITE |
- SEQ_CFG_CSDEASSERT |
- SEQ_CFG_STARTSEQ),
-};
-
-static struct stfsm_seq stfsm_seq_wrvcr = {
- .seq_opc[0] = (SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_WREN) | SEQ_OPC_CSDEASSERT),
- .seq_opc[1] = (SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_WRVCR)),
+ SEQ_OPC_OPCODE(SPINOR_OP_WRSR)),
.seq = {
STFSM_INST_CMD1,
STFSM_INST_CMD2,
@@ -704,9 +666,9 @@
static int stfsm_n25q_en_32bit_addr_seq(struct stfsm_seq *seq)
{
seq->seq_opc[0] = (SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_EN4B_ADDR));
+ SEQ_OPC_OPCODE(SPINOR_OP_EN4B));
seq->seq_opc[1] = (SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_WREN) |
+ SEQ_OPC_OPCODE(SPINOR_OP_WREN) |
SEQ_OPC_CSDEASSERT);
seq->seq[0] = STFSM_INST_CMD2;
@@ -793,7 +755,7 @@
dev_dbg(fsm->dev, "Reading %d bytes from FIFO\n", size);
- BUG_ON((((uint32_t)buf) & 0x3) || (size & 0x3));
+ BUG_ON((((uintptr_t)buf) & 0x3) || (size & 0x3));
while (remaining) {
for (;;) {
@@ -817,7 +779,7 @@
dev_dbg(fsm->dev, "writing %d bytes to FIFO\n", size);
- BUG_ON((((uint32_t)buf) & 0x3) || (size & 0x3));
+ BUG_ON((((uintptr_t)buf) & 0x3) || (size & 0x3));
writesl(fsm->base + SPI_FAST_SEQ_DATA_REG, buf, words);
@@ -827,7 +789,7 @@
static int stfsm_enter_32bit_addr(struct stfsm *fsm, int enter)
{
struct stfsm_seq *seq = &fsm->stfsm_seq_en_32bit_addr;
- uint32_t cmd = enter ? FLASH_CMD_EN4B_ADDR : FLASH_CMD_EX4B_ADDR;
+ uint32_t cmd = enter ? SPINOR_OP_EN4B : SPINOR_OP_EX4B;
seq->seq_opc[0] = (SEQ_OPC_PADS_1 |
SEQ_OPC_CYCLES(8) |
@@ -851,7 +813,7 @@
/* Use RDRS1 */
seq->seq_opc[0] = (SEQ_OPC_PADS_1 |
SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_RDSR));
+ SEQ_OPC_OPCODE(SPINOR_OP_RDSR));
/* Load read_status sequence */
stfsm_load_seq(fsm, seq);
@@ -889,59 +851,56 @@
}
static int stfsm_read_status(struct stfsm *fsm, uint8_t cmd,
- uint8_t *status)
+ uint8_t *data, int bytes)
{
struct stfsm_seq *seq = &stfsm_seq_read_status_fifo;
uint32_t tmp;
+ uint8_t *t = (uint8_t *)&tmp;
+ int i;
- dev_dbg(fsm->dev, "reading STA[%s]\n",
- (cmd == FLASH_CMD_RDSR) ? "1" : "2");
+ dev_dbg(fsm->dev, "read 'status' register [0x%02x], %d byte(s)\n",
+ cmd, bytes);
- seq->seq_opc[0] = (SEQ_OPC_PADS_1 |
- SEQ_OPC_CYCLES(8) |
+ BUG_ON(bytes != 1 && bytes != 2);
+
+ seq->seq_opc[0] = (SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
SEQ_OPC_OPCODE(cmd)),
stfsm_load_seq(fsm, seq);
stfsm_read_fifo(fsm, &tmp, 4);
- *status = (uint8_t)(tmp >> 24);
+ for (i = 0; i < bytes; i++)
+ data[i] = t[i];
stfsm_wait_seq(fsm);
return 0;
}
-static int stfsm_write_status(struct stfsm *fsm, uint16_t status,
- int sta_bytes)
+static int stfsm_write_status(struct stfsm *fsm, uint8_t cmd,
+ uint16_t data, int bytes, int wait_busy)
{
struct stfsm_seq *seq = &stfsm_seq_write_status;
- dev_dbg(fsm->dev, "writing STA[%s] 0x%04x\n",
- (sta_bytes == 1) ? "1" : "1+2", status);
+ dev_dbg(fsm->dev,
+ "write 'status' register [0x%02x], %d byte(s), 0x%04x\n"
+ " %s wait-busy\n", cmd, bytes, data, wait_busy ? "with" : "no");
- seq->status = (uint32_t)status | STA_PADS_1 | STA_CSDEASSERT;
- seq->seq[2] = (sta_bytes == 1) ?
- STFSM_INST_STA_WR1 : STFSM_INST_STA_WR1_2;
+ BUG_ON(bytes != 1 && bytes != 2);
+
+ seq->seq_opc[1] = (SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
+ SEQ_OPC_OPCODE(cmd));
+
+ seq->status = (uint32_t)data | STA_PADS_1 | STA_CSDEASSERT;
+ seq->seq[2] = (bytes == 1) ? STFSM_INST_STA_WR1 : STFSM_INST_STA_WR1_2;
stfsm_load_seq(fsm, seq);
stfsm_wait_seq(fsm);
- return 0;
-};
-
-static int stfsm_wrvcr(struct stfsm *fsm, uint8_t data)
-{
- struct stfsm_seq *seq = &stfsm_seq_wrvcr;
-
- dev_dbg(fsm->dev, "writing VCR 0x%02x\n", data);
-
- seq->status = (STA_DATA_BYTE1(data) | STA_PADS_1 | STA_CSDEASSERT);
-
- stfsm_load_seq(fsm, seq);
-
- stfsm_wait_seq(fsm);
+ if (wait_busy)
+ stfsm_wait_busy(fsm);
return 0;
}
@@ -1027,7 +986,7 @@
if (cfg->write)
seq->seq_opc[i++] = (SEQ_OPC_PADS_1 |
SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_WREN) |
+ SEQ_OPC_OPCODE(SPINOR_OP_WREN) |
SEQ_OPC_CSDEASSERT);
/* Address configuration (24 or 32-bit addresses) */
@@ -1149,31 +1108,36 @@
stfsm_mx25_en_32bit_addr_seq(&fsm->stfsm_seq_en_32bit_addr);
soc_reset = stfsm_can_handle_soc_reset(fsm);
- if (soc_reset || !fsm->booted_from_spi) {
+ if (soc_reset || !fsm->booted_from_spi)
/* If we can handle SoC resets, we enable 32-bit address
* mode pervasively */
stfsm_enter_32bit_addr(fsm, 1);
- } else {
+ else
/* Else, enable/disable 32-bit addressing before/after
* each operation */
fsm->configuration = (CFG_READ_TOGGLE_32BIT_ADDR |
CFG_WRITE_TOGGLE_32BIT_ADDR |
CFG_ERASESEC_TOGGLE_32BIT_ADDR);
- /* It seems a small delay is required after exiting
- * 32-bit mode following a write operation. The issue
- * is under investigation.
- */
- fsm->configuration |= CFG_WRITE_EX_32BIT_ADDR_DELAY;
- }
}
- /* For QUAD mode, set 'QE' STATUS bit */
+ /* Check status of 'QE' bit, update if required. */
+ stfsm_read_status(fsm, SPINOR_OP_RDSR, &sta, 1);
data_pads = ((fsm->stfsm_seq_read.seq_cfg >> 16) & 0x3) + 1;
if (data_pads == 4) {
- stfsm_read_status(fsm, FLASH_CMD_RDSR, &sta);
- sta |= MX25_STATUS_QE;
- stfsm_write_status(fsm, sta, 1);
+ if (!(sta & MX25_STATUS_QE)) {
+ /* Set 'QE' */
+ sta |= MX25_STATUS_QE;
+
+ stfsm_write_status(fsm, SPINOR_OP_WRSR, sta, 1, 1);
+ }
+ } else {
+ if (sta & MX25_STATUS_QE) {
+ /* Clear 'QE' */
+ sta &= ~MX25_STATUS_QE;
+
+ stfsm_write_status(fsm, SPINOR_OP_WRSR, sta, 1, 1);
+ }
}
return 0;
@@ -1239,7 +1203,7 @@
*/
vcr = (N25Q_VCR_DUMMY_CYCLES(8) | N25Q_VCR_XIP_DISABLED |
N25Q_VCR_WRAP_CONT);
- stfsm_wrvcr(fsm, vcr);
+ stfsm_write_status(fsm, N25Q_CMD_WRVCR, vcr, 1, 0);
return 0;
}
@@ -1297,7 +1261,7 @@
{
struct stfsm_seq seq = {
.seq_opc[0] = (SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_WREN) |
+ SEQ_OPC_OPCODE(SPINOR_OP_WREN) |
SEQ_OPC_CSDEASSERT),
.seq_opc[1] = (SEQ_OPC_PADS_1 | SEQ_OPC_CYCLES(8) |
SEQ_OPC_OPCODE(S25FL_CMD_DYBWR)),
@@ -1337,7 +1301,7 @@
SEQ_OPC_CSDEASSERT),
.seq_opc[1] = (SEQ_OPC_PADS_1 |
SEQ_OPC_CYCLES(8) |
- SEQ_OPC_OPCODE(FLASH_CMD_WRDI) |
+ SEQ_OPC_OPCODE(SPINOR_OP_WRDI) |
SEQ_OPC_CSDEASSERT),
.seq = {
STFSM_INST_CMD1,
@@ -1367,6 +1331,7 @@
uint32_t offs;
uint16_t sta_wr;
uint8_t sr1, cr1, dyb;
+ int update_sr = 0;
int ret;
if (flags & FLASH_FLAG_32BIT_ADDR) {
@@ -1414,34 +1379,28 @@
}
}
- /* Check status of 'QE' bit */
+ /* Check status of 'QE' bit, update if required. */
+ stfsm_read_status(fsm, SPINOR_OP_RDSR2, &cr1, 1);
data_pads = ((fsm->stfsm_seq_read.seq_cfg >> 16) & 0x3) + 1;
- stfsm_read_status(fsm, FLASH_CMD_RDSR2, &cr1);
if (data_pads == 4) {
if (!(cr1 & STFSM_S25FL_CONFIG_QE)) {
/* Set 'QE' */
cr1 |= STFSM_S25FL_CONFIG_QE;
- stfsm_read_status(fsm, FLASH_CMD_RDSR, &sr1);
- sta_wr = ((uint16_t)cr1 << 8) | sr1;
-
- stfsm_write_status(fsm, sta_wr, 2);
-
- stfsm_wait_busy(fsm);
+ update_sr = 1;
}
} else {
- if ((cr1 & STFSM_S25FL_CONFIG_QE)) {
+ if (cr1 & STFSM_S25FL_CONFIG_QE) {
/* Clear 'QE' */
cr1 &= ~STFSM_S25FL_CONFIG_QE;
- stfsm_read_status(fsm, FLASH_CMD_RDSR, &sr1);
- sta_wr = ((uint16_t)cr1 << 8) | sr1;
-
- stfsm_write_status(fsm, sta_wr, 2);
-
- stfsm_wait_busy(fsm);
+ update_sr = 1;
}
-
+ }
+ if (update_sr) {
+ stfsm_read_status(fsm, SPINOR_OP_RDSR, &sr1, 1);
+ sta_wr = ((uint16_t)cr1 << 8) | sr1;
+ stfsm_write_status(fsm, SPINOR_OP_WRSR, sta_wr, 2, 1);
}
/*
@@ -1456,27 +1415,36 @@
static int stfsm_w25q_config(struct stfsm *fsm)
{
uint32_t data_pads;
- uint16_t sta_wr;
- uint8_t sta1, sta2;
+ uint8_t sr1, sr2;
+ uint16_t sr_wr;
+ int update_sr = 0;
int ret;
ret = stfsm_prepare_rwe_seqs_default(fsm);
if (ret)
return ret;
- /* If using QUAD mode, set QE STATUS bit */
+ /* Check status of 'QE' bit, update if required. */
+ stfsm_read_status(fsm, SPINOR_OP_RDSR2, &sr2, 1);
data_pads = ((fsm->stfsm_seq_read.seq_cfg >> 16) & 0x3) + 1;
if (data_pads == 4) {
- stfsm_read_status(fsm, FLASH_CMD_RDSR, &sta1);
- stfsm_read_status(fsm, FLASH_CMD_RDSR2, &sta2);
-
- sta_wr = ((uint16_t)sta2 << 8) | sta1;
-
- sta_wr |= W25Q_STATUS_QE;
-
- stfsm_write_status(fsm, sta_wr, 2);
-
- stfsm_wait_busy(fsm);
+ if (!(sr2 & W25Q_STATUS_QE)) {
+ /* Set 'QE' */
+ sr2 |= W25Q_STATUS_QE;
+ update_sr = 1;
+ }
+ } else {
+ if (sr2 & W25Q_STATUS_QE) {
+ /* Clear 'QE' */
+ sr2 &= ~W25Q_STATUS_QE;
+ update_sr = 1;
+ }
+ }
+ if (update_sr) {
+ /* Write status register */
+ stfsm_read_status(fsm, SPINOR_OP_RDSR, &sr1, 1);
+ sr_wr = ((uint16_t)sr2 << 8) | sr1;
+ stfsm_write_status(fsm, SPINOR_OP_WRSR, sr_wr, 2, 1);
}
return 0;
@@ -1506,7 +1474,7 @@
read_mask = (data_pads << 2) - 1;
/* Handle non-aligned buf */
- p = ((uint32_t)buf & 0x3) ? (uint8_t *)page_buf : buf;
+ p = ((uintptr_t)buf & 0x3) ? (uint8_t *)page_buf : buf;
/* Handle non-aligned size */
size_ub = (size + read_mask) & ~read_mask;
@@ -1528,7 +1496,7 @@
}
/* Handle non-aligned buf */
- if ((uint32_t)buf & 0x3)
+ if ((uintptr_t)buf & 0x3)
memcpy(buf, page_buf, size);
/* Wait for sequence to finish */
@@ -1570,7 +1538,7 @@
write_mask = (data_pads << 2) - 1;
/* Handle non-aligned buf */
- if ((uint32_t)buf & 0x3) {
+ if ((uintptr_t)buf & 0x3) {
memcpy(page_buf, buf, size);
p = (uint8_t *)page_buf;
} else {
@@ -1628,11 +1596,8 @@
stfsm_s25fl_clear_status_reg(fsm);
/* Exit 32-bit address mode, if required */
- if (fsm->configuration & CFG_WRITE_TOGGLE_32BIT_ADDR) {
+ if (fsm->configuration & CFG_WRITE_TOGGLE_32BIT_ADDR)
stfsm_enter_32bit_addr(fsm, 0);
- if (fsm->configuration & CFG_WRITE_EX_32BIT_ADDR_DELAY)
- udelay(1);
- }
return 0;
}
@@ -1736,7 +1701,7 @@
while (len) {
/* Write up to page boundary */
- bytes = min(FLASH_PAGESIZE - page_offs, len);
+ bytes = min_t(size_t, FLASH_PAGESIZE - page_offs, len);
ret = stfsm_write(fsm, b, bytes, to);
if (ret)
@@ -1935,6 +1900,13 @@
fsm->base + SPI_CONFIGDATA);
writel(STFSM_DEFAULT_WR_TIME, fsm->base + SPI_STATUS_WR_TIME_REG);
+ /*
+ * Set the FSM 'WAIT' delay to the minimum workable value. Note, for
+ * our purposes, the WAIT instruction is used purely to achieve
+ * "sequence validity" rather than actually implement a delay.
+ */
+ writel(0x00000001, fsm->base + SPI_PROGRAM_ERASE_TIME);
+
/* Clear FIFO, just in case */
stfsm_clear_fifo(fsm);
@@ -2086,7 +2058,7 @@
return mtd_device_unregister(&fsm->mtd);
}
-static struct of_device_id stfsm_match[] = {
+static const struct of_device_id stfsm_match[] = {
{ .compatible = "st,spi-fsm", },
{},
};
diff --git a/drivers/mtd/lpddr/Kconfig b/drivers/mtd/lpddr/Kconfig
index 265f969..3a19cbe 100644
--- a/drivers/mtd/lpddr/Kconfig
+++ b/drivers/mtd/lpddr/Kconfig
@@ -1,5 +1,5 @@
-menu "LPDDR flash memory drivers"
- depends on MTD!=n
+menu "LPDDR & LPDDR2 PCM memory drivers"
+ depends on MTD
config MTD_LPDDR
tristate "Support for LPDDR flash chips"
@@ -17,4 +17,13 @@
Window QINFO interface, permits software to be used for entire
families of devices. This serves similar purpose of CFI on legacy
Flash products
+
+config MTD_LPDDR2_NVM
+ # ARM dependency is only for writel_relaxed()
+ depends on MTD && ARM
+ tristate "Support for LPDDR2-NVM flash chips"
+ help
+ This option enables support of PCM memories with a LPDDR2-NVM
+ (Low power double data rate 2) interface.
+
endmenu
diff --git a/drivers/mtd/lpddr/Makefile b/drivers/mtd/lpddr/Makefile
index da48e46..881d440 100644
--- a/drivers/mtd/lpddr/Makefile
+++ b/drivers/mtd/lpddr/Makefile
@@ -4,3 +4,4 @@
obj-$(CONFIG_MTD_QINFO_PROBE) += qinfo_probe.o
obj-$(CONFIG_MTD_LPDDR) += lpddr_cmds.o
+obj-$(CONFIG_MTD_LPDDR2_NVM) += lpddr2_nvm.o
diff --git a/drivers/mtd/lpddr/lpddr2_nvm.c b/drivers/mtd/lpddr/lpddr2_nvm.c
new file mode 100644
index 0000000..063cec4
--- /dev/null
+++ b/drivers/mtd/lpddr/lpddr2_nvm.c
@@ -0,0 +1,507 @@
+/*
+ * LPDDR2-NVM MTD driver. This module provides read, write, erase, lock/unlock
+ * support for LPDDR2-NVM PCM memories
+ *
+ * Copyright © 2012 Micron Technology, Inc.
+ *
+ * Vincenzo Aliberti <vincenzo.aliberti@gmail.com>
+ * Domenico Manna <domenico.manna@gmail.com>
+ * Many thanks to Andrea Vigilante for initial enabling
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": %s: " fmt, __func__
+
+#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/mtd/map.h>
+#include <linux/mtd/mtd.h>
+#include <linux/mtd/partitions.h>
+#include <linux/slab.h>
+#include <linux/platform_device.h>
+#include <linux/ioport.h>
+#include <linux/err.h>
+
+/* Parameters */
+#define ERASE_BLOCKSIZE (0x00020000/2) /* in Word */
+#define WRITE_BUFFSIZE (0x00000400/2) /* in Word */
+#define OW_BASE_ADDRESS 0x00000000 /* OW offset */
+#define BUS_WIDTH 0x00000020 /* x32 devices */
+
+/* PFOW symbols address offset */
+#define PFOW_QUERY_STRING_P (0x0000/2) /* in Word */
+#define PFOW_QUERY_STRING_F (0x0002/2) /* in Word */
+#define PFOW_QUERY_STRING_O (0x0004/2) /* in Word */
+#define PFOW_QUERY_STRING_W (0x0006/2) /* in Word */
+
+/* OW registers address */
+#define CMD_CODE_OFS (0x0080/2) /* in Word */
+#define CMD_DATA_OFS (0x0084/2) /* in Word */
+#define CMD_ADD_L_OFS (0x0088/2) /* in Word */
+#define CMD_ADD_H_OFS (0x008A/2) /* in Word */
+#define MPR_L_OFS (0x0090/2) /* in Word */
+#define MPR_H_OFS (0x0092/2) /* in Word */
+#define CMD_EXEC_OFS (0x00C0/2) /* in Word */
+#define STATUS_REG_OFS (0x00CC/2) /* in Word */
+#define PRG_BUFFER_OFS (0x0010/2) /* in Word */
+
+/* Datamask */
+#define MR_CFGMASK 0x8000
+#define SR_OK_DATAMASK 0x0080
+
+/* LPDDR2-NVM Commands */
+#define LPDDR2_NVM_LOCK 0x0061
+#define LPDDR2_NVM_UNLOCK 0x0062
+#define LPDDR2_NVM_SW_PROGRAM 0x0041
+#define LPDDR2_NVM_SW_OVERWRITE 0x0042
+#define LPDDR2_NVM_BUF_PROGRAM 0x00E9
+#define LPDDR2_NVM_BUF_OVERWRITE 0x00EA
+#define LPDDR2_NVM_ERASE 0x0020
+
+/* LPDDR2-NVM Registers offset */
+#define LPDDR2_MODE_REG_DATA 0x0040
+#define LPDDR2_MODE_REG_CFG 0x0050
+
+/*
+ * Internal Type Definitions
+ * pcm_int_data contains memory controller details:
+ * @reg_data : LPDDR2_MODE_REG_DATA register address after remapping
+ * @reg_cfg : LPDDR2_MODE_REG_CFG register address after remapping
+ * &bus_width: memory bus-width (eg: x16 2 Bytes, x32 4 Bytes)
+ */
+struct pcm_int_data {
+ void __iomem *ctl_regs;
+ int bus_width;
+};
+
+static DEFINE_MUTEX(lpdd2_nvm_mutex);
+
+/*
+ * Build a map_word starting from an u_long
+ */
+static inline map_word build_map_word(u_long myword)
+{
+ map_word val = { {0} };
+ val.x[0] = myword;
+ return val;
+}
+
+/*
+ * Build Mode Register Configuration DataMask based on device bus-width
+ */
+static inline u_int build_mr_cfgmask(u_int bus_width)
+{
+ u_int val = MR_CFGMASK;
+
+ if (bus_width == 0x0004) /* x32 device */
+ val = val << 16;
+
+ return val;
+}
+
+/*
+ * Build Status Register OK DataMask based on device bus-width
+ */
+static inline u_int build_sr_ok_datamask(u_int bus_width)
+{
+ u_int val = SR_OK_DATAMASK;
+
+ if (bus_width == 0x0004) /* x32 device */
+ val = (val << 16)+val;
+
+ return val;
+}
+
+/*
+ * Evaluates Overlay Window Control Registers address
+ */
+static inline u_long ow_reg_add(struct map_info *map, u_long offset)
+{
+ u_long val = 0;
+ struct pcm_int_data *pcm_data = map->fldrv_priv;
+
+ val = map->pfow_base + offset*pcm_data->bus_width;
+
+ return val;
+}
+
+/*
+ * Enable lpddr2-nvm Overlay Window
+ * Overlay Window is a memory mapped area containing all LPDDR2-NVM registers
+ * used by device commands as well as uservisible resources like Device Status
+ * Register, Device ID, etc
+ */
+static inline void ow_enable(struct map_info *map)
+{
+ struct pcm_int_data *pcm_data = map->fldrv_priv;
+
+ writel_relaxed(build_mr_cfgmask(pcm_data->bus_width) | 0x18,
+ pcm_data->ctl_regs + LPDDR2_MODE_REG_CFG);
+ writel_relaxed(0x01, pcm_data->ctl_regs + LPDDR2_MODE_REG_DATA);
+}
+
+/*
+ * Disable lpddr2-nvm Overlay Window
+ * Overlay Window is a memory mapped area containing all LPDDR2-NVM registers
+ * used by device commands as well as uservisible resources like Device Status
+ * Register, Device ID, etc
+ */
+static inline void ow_disable(struct map_info *map)
+{
+ struct pcm_int_data *pcm_data = map->fldrv_priv;
+
+ writel_relaxed(build_mr_cfgmask(pcm_data->bus_width) | 0x18,
+ pcm_data->ctl_regs + LPDDR2_MODE_REG_CFG);
+ writel_relaxed(0x02, pcm_data->ctl_regs + LPDDR2_MODE_REG_DATA);
+}
+
+/*
+ * Execute lpddr2-nvm operations
+ */
+static int lpddr2_nvm_do_op(struct map_info *map, u_long cmd_code,
+ u_long cmd_data, u_long cmd_add, u_long cmd_mpr, u_char *buf)
+{
+ map_word add_l = { {0} }, add_h = { {0} }, mpr_l = { {0} },
+ mpr_h = { {0} }, data_l = { {0} }, cmd = { {0} },
+ exec_cmd = { {0} }, sr;
+ map_word data_h = { {0} }; /* only for 2x x16 devices stacked */
+ u_long i, status_reg, prg_buff_ofs;
+ struct pcm_int_data *pcm_data = map->fldrv_priv;
+ u_int sr_ok_datamask = build_sr_ok_datamask(pcm_data->bus_width);
+
+ /* Builds low and high words for OW Control Registers */
+ add_l.x[0] = cmd_add & 0x0000FFFF;
+ add_h.x[0] = (cmd_add >> 16) & 0x0000FFFF;
+ mpr_l.x[0] = cmd_mpr & 0x0000FFFF;
+ mpr_h.x[0] = (cmd_mpr >> 16) & 0x0000FFFF;
+ cmd.x[0] = cmd_code & 0x0000FFFF;
+ exec_cmd.x[0] = 0x0001;
+ data_l.x[0] = cmd_data & 0x0000FFFF;
+ data_h.x[0] = (cmd_data >> 16) & 0x0000FFFF; /* only for 2x x16 */
+
+ /* Set Overlay Window Control Registers */
+ map_write(map, cmd, ow_reg_add(map, CMD_CODE_OFS));
+ map_write(map, data_l, ow_reg_add(map, CMD_DATA_OFS));
+ map_write(map, add_l, ow_reg_add(map, CMD_ADD_L_OFS));
+ map_write(map, add_h, ow_reg_add(map, CMD_ADD_H_OFS));
+ map_write(map, mpr_l, ow_reg_add(map, MPR_L_OFS));
+ map_write(map, mpr_h, ow_reg_add(map, MPR_H_OFS));
+ if (pcm_data->bus_width == 0x0004) { /* 2x16 devices stacked */
+ map_write(map, cmd, ow_reg_add(map, CMD_CODE_OFS) + 2);
+ map_write(map, data_h, ow_reg_add(map, CMD_DATA_OFS) + 2);
+ map_write(map, add_l, ow_reg_add(map, CMD_ADD_L_OFS) + 2);
+ map_write(map, add_h, ow_reg_add(map, CMD_ADD_H_OFS) + 2);
+ map_write(map, mpr_l, ow_reg_add(map, MPR_L_OFS) + 2);
+ map_write(map, mpr_h, ow_reg_add(map, MPR_H_OFS) + 2);
+ }
+
+ /* Fill Program Buffer */
+ if ((cmd_code == LPDDR2_NVM_BUF_PROGRAM) ||
+ (cmd_code == LPDDR2_NVM_BUF_OVERWRITE)) {
+ prg_buff_ofs = (map_read(map,
+ ow_reg_add(map, PRG_BUFFER_OFS))).x[0];
+ for (i = 0; i < cmd_mpr; i++) {
+ map_write(map, build_map_word(buf[i]), map->pfow_base +
+ prg_buff_ofs + i);
+ }
+ }
+
+ /* Command Execute */
+ map_write(map, exec_cmd, ow_reg_add(map, CMD_EXEC_OFS));
+ if (pcm_data->bus_width == 0x0004) /* 2x16 devices stacked */
+ map_write(map, exec_cmd, ow_reg_add(map, CMD_EXEC_OFS) + 2);
+
+ /* Status Register Check */
+ do {
+ sr = map_read(map, ow_reg_add(map, STATUS_REG_OFS));
+ status_reg = sr.x[0];
+ if (pcm_data->bus_width == 0x0004) {/* 2x16 devices stacked */
+ sr = map_read(map, ow_reg_add(map,
+ STATUS_REG_OFS) + 2);
+ status_reg += sr.x[0] << 16;
+ }
+ } while ((status_reg & sr_ok_datamask) != sr_ok_datamask);
+
+ return (((status_reg & sr_ok_datamask) == sr_ok_datamask) ? 0 : -EIO);
+}
+
+/*
+ * Execute lpddr2-nvm operations @ block level
+ */
+static int lpddr2_nvm_do_block_op(struct mtd_info *mtd, loff_t start_add,
+ uint64_t len, u_char block_op)
+{
+ struct map_info *map = mtd->priv;
+ u_long add, end_add;
+ int ret = 0;
+
+ mutex_lock(&lpdd2_nvm_mutex);
+
+ ow_enable(map);
+
+ add = start_add;
+ end_add = add + len;
+
+ do {
+ ret = lpddr2_nvm_do_op(map, block_op, 0x00, add, add, NULL);
+ if (ret)
+ goto out;
+ add += mtd->erasesize;
+ } while (add < end_add);
+
+out:
+ ow_disable(map);
+ mutex_unlock(&lpdd2_nvm_mutex);
+ return ret;
+}
+
+/*
+ * verify presence of PFOW string
+ */
+static int lpddr2_nvm_pfow_present(struct map_info *map)
+{
+ map_word pfow_val[4];
+ unsigned int found = 1;
+
+ mutex_lock(&lpdd2_nvm_mutex);
+
+ ow_enable(map);
+
+ /* Load string from array */
+ pfow_val[0] = map_read(map, ow_reg_add(map, PFOW_QUERY_STRING_P));
+ pfow_val[1] = map_read(map, ow_reg_add(map, PFOW_QUERY_STRING_F));
+ pfow_val[2] = map_read(map, ow_reg_add(map, PFOW_QUERY_STRING_O));
+ pfow_val[3] = map_read(map, ow_reg_add(map, PFOW_QUERY_STRING_W));
+
+ /* Verify the string loaded vs expected */
+ if (!map_word_equal(map, build_map_word('P'), pfow_val[0]))
+ found = 0;
+ if (!map_word_equal(map, build_map_word('F'), pfow_val[1]))
+ found = 0;
+ if (!map_word_equal(map, build_map_word('O'), pfow_val[2]))
+ found = 0;
+ if (!map_word_equal(map, build_map_word('W'), pfow_val[3]))
+ found = 0;
+
+ ow_disable(map);
+
+ mutex_unlock(&lpdd2_nvm_mutex);
+
+ return found;
+}
+
+/*
+ * lpddr2_nvm driver read method
+ */
+static int lpddr2_nvm_read(struct mtd_info *mtd, loff_t start_add,
+ size_t len, size_t *retlen, u_char *buf)
+{
+ struct map_info *map = mtd->priv;
+
+ mutex_lock(&lpdd2_nvm_mutex);
+
+ *retlen = len;
+
+ map_copy_from(map, buf, start_add, *retlen);
+
+ mutex_unlock(&lpdd2_nvm_mutex);
+ return 0;
+}
+
+/*
+ * lpddr2_nvm driver write method
+ */
+static int lpddr2_nvm_write(struct mtd_info *mtd, loff_t start_add,
+ size_t len, size_t *retlen, const u_char *buf)
+{
+ struct map_info *map = mtd->priv;
+ struct pcm_int_data *pcm_data = map->fldrv_priv;
+ u_long add, current_len, tot_len, target_len, my_data;
+ u_char *write_buf = (u_char *)buf;
+ int ret = 0;
+
+ mutex_lock(&lpdd2_nvm_mutex);
+
+ ow_enable(map);
+
+ /* Set start value for the variables */
+ add = start_add;
+ target_len = len;
+ tot_len = 0;
+
+ while (tot_len < target_len) {
+ if (!(IS_ALIGNED(add, mtd->writesize))) { /* do sw program */
+ my_data = write_buf[tot_len];
+ my_data += (write_buf[tot_len+1]) << 8;
+ if (pcm_data->bus_width == 0x0004) {/* 2x16 devices */
+ my_data += (write_buf[tot_len+2]) << 16;
+ my_data += (write_buf[tot_len+3]) << 24;
+ }
+ ret = lpddr2_nvm_do_op(map, LPDDR2_NVM_SW_OVERWRITE,
+ my_data, add, 0x00, NULL);
+ if (ret)
+ goto out;
+
+ add += pcm_data->bus_width;
+ tot_len += pcm_data->bus_width;
+ } else { /* do buffer program */
+ current_len = min(target_len - tot_len,
+ (u_long) mtd->writesize);
+ ret = lpddr2_nvm_do_op(map, LPDDR2_NVM_BUF_OVERWRITE,
+ 0x00, add, current_len, write_buf + tot_len);
+ if (ret)
+ goto out;
+
+ add += current_len;
+ tot_len += current_len;
+ }
+ }
+
+out:
+ *retlen = tot_len;
+ ow_disable(map);
+ mutex_unlock(&lpdd2_nvm_mutex);
+ return ret;
+}
+
+/*
+ * lpddr2_nvm driver erase method
+ */
+static int lpddr2_nvm_erase(struct mtd_info *mtd, struct erase_info *instr)
+{
+ int ret = lpddr2_nvm_do_block_op(mtd, instr->addr, instr->len,
+ LPDDR2_NVM_ERASE);
+ if (!ret) {
+ instr->state = MTD_ERASE_DONE;
+ mtd_erase_callback(instr);
+ }
+
+ return ret;
+}
+
+/*
+ * lpddr2_nvm driver unlock method
+ */
+static int lpddr2_nvm_unlock(struct mtd_info *mtd, loff_t start_add,
+ uint64_t len)
+{
+ return lpddr2_nvm_do_block_op(mtd, start_add, len, LPDDR2_NVM_UNLOCK);
+}
+
+/*
+ * lpddr2_nvm driver lock method
+ */
+static int lpddr2_nvm_lock(struct mtd_info *mtd, loff_t start_add,
+ uint64_t len)
+{
+ return lpddr2_nvm_do_block_op(mtd, start_add, len, LPDDR2_NVM_LOCK);
+}
+
+/*
+ * lpddr2_nvm driver probe method
+ */
+static int lpddr2_nvm_probe(struct platform_device *pdev)
+{
+ struct map_info *map;
+ struct mtd_info *mtd;
+ struct resource *add_range;
+ struct resource *control_regs;
+ struct pcm_int_data *pcm_data;
+
+ /* Allocate memory control_regs data structures */
+ pcm_data = devm_kzalloc(&pdev->dev, sizeof(*pcm_data), GFP_KERNEL);
+ if (!pcm_data)
+ return -ENOMEM;
+
+ pcm_data->bus_width = BUS_WIDTH;
+
+ /* Allocate memory for map_info & mtd_info data structures */
+ map = devm_kzalloc(&pdev->dev, sizeof(*map), GFP_KERNEL);
+ if (!map)
+ return -ENOMEM;
+
+ mtd = devm_kzalloc(&pdev->dev, sizeof(*mtd), GFP_KERNEL);
+ if (!mtd)
+ return -ENOMEM;
+
+ /* lpddr2_nvm address range */
+ add_range = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+
+ /* Populate map_info data structure */
+ *map = (struct map_info) {
+ .virt = devm_ioremap_resource(&pdev->dev, add_range),
+ .name = pdev->dev.init_name,
+ .phys = add_range->start,
+ .size = resource_size(add_range),
+ .bankwidth = pcm_data->bus_width / 2,
+ .pfow_base = OW_BASE_ADDRESS,
+ .fldrv_priv = pcm_data,
+ };
+ if (IS_ERR(map->virt))
+ return PTR_ERR(map->virt);
+
+ simple_map_init(map); /* fill with default methods */
+
+ control_regs = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+ pcm_data->ctl_regs = devm_ioremap_resource(&pdev->dev, control_regs);
+ if (IS_ERR(pcm_data->ctl_regs))
+ return PTR_ERR(pcm_data->ctl_regs);
+
+ /* Populate mtd_info data structure */
+ *mtd = (struct mtd_info) {
+ .name = pdev->dev.init_name,
+ .type = MTD_RAM,
+ .priv = map,
+ .size = resource_size(add_range),
+ .erasesize = ERASE_BLOCKSIZE * pcm_data->bus_width,
+ .writesize = 1,
+ .writebufsize = WRITE_BUFFSIZE * pcm_data->bus_width,
+ .flags = (MTD_CAP_NVRAM | MTD_POWERUP_LOCK),
+ ._read = lpddr2_nvm_read,
+ ._write = lpddr2_nvm_write,
+ ._erase = lpddr2_nvm_erase,
+ ._unlock = lpddr2_nvm_unlock,
+ ._lock = lpddr2_nvm_lock,
+ };
+
+ /* Verify the presence of the device looking for PFOW string */
+ if (!lpddr2_nvm_pfow_present(map)) {
+ pr_err("device not recognized\n");
+ return -EINVAL;
+ }
+ /* Parse partitions and register the MTD device */
+ return mtd_device_parse_register(mtd, NULL, NULL, NULL, 0);
+}
+
+/*
+ * lpddr2_nvm driver remove method
+ */
+static int lpddr2_nvm_remove(struct platform_device *pdev)
+{
+ return mtd_device_unregister(dev_get_drvdata(&pdev->dev));
+}
+
+/* Initialize platform_driver data structure for lpddr2_nvm */
+static struct platform_driver lpddr2_nvm_drv = {
+ .driver = {
+ .name = "lpddr2_nvm",
+ },
+ .probe = lpddr2_nvm_probe,
+ .remove = lpddr2_nvm_remove,
+};
+
+module_platform_driver(lpddr2_nvm_drv);
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Vincenzo Aliberti <vincenzo.aliberti@gmail.com>");
+MODULE_DESCRIPTION("MTD driver for LPDDR2-NVM PCM memories");
diff --git a/drivers/mtd/maps/Kconfig b/drivers/mtd/maps/Kconfig
index fce23fe..21b2874 100644
--- a/drivers/mtd/maps/Kconfig
+++ b/drivers/mtd/maps/Kconfig
@@ -108,7 +108,7 @@
config MTD_SC520CDP
tristate "CFI Flash device mapped on AMD SC520 CDP"
- depends on X86 && MTD_CFI
+ depends on (MELAN || COMPILE_TEST) && MTD_CFI
help
The SC520 CDP board has two banks of CFI-compliant chips and one
Dual-in-line JEDEC chip. This 'mapping' driver supports that
@@ -116,7 +116,7 @@
config MTD_NETSC520
tristate "CFI Flash device mapped on AMD NetSc520"
- depends on X86 && MTD_CFI
+ depends on (MELAN || COMPILE_TEST) && MTD_CFI
help
This enables access routines for the flash chips on the AMD NetSc520
demonstration board. If you have one of these boards and would like
diff --git a/drivers/mtd/maps/sc520cdp.c b/drivers/mtd/maps/sc520cdp.c
index 8fead8e..093edd5 100644
--- a/drivers/mtd/maps/sc520cdp.c
+++ b/drivers/mtd/maps/sc520cdp.c
@@ -183,7 +183,7 @@
static void sc520cdp_setup_par(void)
{
- volatile unsigned long __iomem *mmcr;
+ unsigned long __iomem *mmcr;
unsigned long mmcr_val;
int i, j;
@@ -203,11 +203,11 @@
*/
for(i = 0; i < NUM_FLASH_BANKS; i++) { /* for each par_table entry */
for(j = 0; j < NUM_SC520_PAR; j++) { /* for each PAR register */
- mmcr_val = mmcr[SC520_PAR(j)];
+ mmcr_val = readl(&mmcr[SC520_PAR(j)]);
/* if target device field matches, reprogram the PAR */
if((mmcr_val & SC520_PAR_TRGDEV) == par_table[i].trgdev)
{
- mmcr[SC520_PAR(j)] = par_table[i].new_par;
+ writel(par_table[i].new_par, &mmcr[SC520_PAR(j)]);
break;
}
}
diff --git a/drivers/mtd/maps/solutionengine.c b/drivers/mtd/maps/solutionengine.c
index 83a7a70..bb580bc 100644
--- a/drivers/mtd/maps/solutionengine.c
+++ b/drivers/mtd/maps/solutionengine.c
@@ -33,28 +33,6 @@
static const char * const probes[] = { "RedBoot", "cmdlinepart", NULL };
-#ifdef CONFIG_MTD_SUPERH_RESERVE
-static struct mtd_partition superh_se_partitions[] = {
- /* Reserved for boot code, read-only */
- {
- .name = "flash_boot",
- .offset = 0x00000000,
- .size = CONFIG_MTD_SUPERH_RESERVE,
- .mask_flags = MTD_WRITEABLE,
- },
- /* All else is writable (e.g. JFFS) */
- {
- .name = "Flash FS",
- .offset = MTDPART_OFS_NXTBLK,
- .size = MTDPART_SIZ_FULL,
- }
-};
-#define NUM_PARTITIONS ARRAY_SIZE(superh_se_partitions)
-#else
-#define superh_se_partitions NULL
-#define NUM_PARTITIONS 0
-#endif /* CONFIG_MTD_SUPERH_RESERVE */
-
static int __init init_soleng_maps(void)
{
/* First probe at offset 0 */
@@ -92,8 +70,7 @@
mtd_device_register(eprom_mtd, NULL, 0);
}
- mtd_device_parse_register(flash_mtd, probes, NULL,
- superh_se_partitions, NUM_PARTITIONS);
+ mtd_device_parse_register(flash_mtd, probes, NULL, NULL, 0);
return 0;
}
diff --git a/drivers/mtd/mtd_blkdevs.c b/drivers/mtd/mtd_blkdevs.c
index 4dbfaee..43e3099 100644
--- a/drivers/mtd/mtd_blkdevs.c
+++ b/drivers/mtd/mtd_blkdevs.c
@@ -87,6 +87,9 @@
if (req->cmd_type != REQ_TYPE_FS)
return -EIO;
+ if (req->cmd_flags & REQ_FLUSH)
+ return tr->flush(dev);
+
if (blk_rq_pos(req) + blk_rq_cur_sectors(req) >
get_capacity(req->rq_disk))
return -EIO;
@@ -407,6 +410,9 @@
if (!new->rq)
goto error3;
+ if (tr->flush)
+ blk_queue_flush(new->rq, REQ_FLUSH);
+
new->rq->queuedata = new;
blk_queue_logical_block_size(new->rq, tr->blksize);
diff --git a/drivers/mtd/mtdchar.c b/drivers/mtd/mtdchar.c
index 7d4e7b9..a0f54e8 100644
--- a/drivers/mtd/mtdchar.c
+++ b/drivers/mtd/mtdchar.c
@@ -568,13 +568,18 @@
{
struct mtd_write_req req;
struct mtd_oob_ops ops;
- void __user *usr_data, *usr_oob;
+ const void __user *usr_data, *usr_oob;
int ret;
- if (copy_from_user(&req, argp, sizeof(req)) ||
- !access_ok(VERIFY_READ, req.usr_data, req.len) ||
- !access_ok(VERIFY_READ, req.usr_oob, req.ooblen))
+ if (copy_from_user(&req, argp, sizeof(req)))
return -EFAULT;
+
+ usr_data = (const void __user *)(uintptr_t)req.usr_data;
+ usr_oob = (const void __user *)(uintptr_t)req.usr_oob;
+ if (!access_ok(VERIFY_READ, usr_data, req.len) ||
+ !access_ok(VERIFY_READ, usr_oob, req.ooblen))
+ return -EFAULT;
+
if (!mtd->_write_oob)
return -EOPNOTSUPP;
@@ -583,10 +588,7 @@
ops.ooblen = (size_t)req.ooblen;
ops.ooboffs = 0;
- usr_data = (void __user *)(uintptr_t)req.usr_data;
- usr_oob = (void __user *)(uintptr_t)req.usr_oob;
-
- if (req.usr_data) {
+ if (usr_data) {
ops.datbuf = memdup_user(usr_data, ops.len);
if (IS_ERR(ops.datbuf))
return PTR_ERR(ops.datbuf);
@@ -594,7 +596,7 @@
ops.datbuf = NULL;
}
- if (req.usr_oob) {
+ if (usr_oob) {
ops.oobbuf = memdup_user(usr_oob, ops.ooblen);
if (IS_ERR(ops.oobbuf)) {
kfree(ops.datbuf);
diff --git a/drivers/mtd/nand/bf5xx_nand.c b/drivers/mtd/nand/bf5xx_nand.c
index b7a2494..722898a 100644
--- a/drivers/mtd/nand/bf5xx_nand.c
+++ b/drivers/mtd/nand/bf5xx_nand.c
@@ -679,9 +679,6 @@
peripheral_free_list(bfin_nfc_pin_req);
bf5xx_nand_dma_remove(info);
- /* free the common resources */
- kfree(info);
-
return 0;
}
@@ -742,10 +739,10 @@
return -EFAULT;
}
- info = kzalloc(sizeof(*info), GFP_KERNEL);
+ info = devm_kzalloc(&pdev->dev, sizeof(*info), GFP_KERNEL);
if (info == NULL) {
err = -ENOMEM;
- goto out_err_kzalloc;
+ goto out_err;
}
platform_set_drvdata(pdev, info);
@@ -790,7 +787,7 @@
/* initialise the hardware */
err = bf5xx_nand_hw_init(info);
if (err)
- goto out_err_hw_init;
+ goto out_err;
/* setup hardware ECC data struct */
if (hardware_ecc) {
@@ -827,9 +824,7 @@
out_err_nand_scan:
bf5xx_nand_dma_remove(info);
-out_err_hw_init:
- kfree(info);
-out_err_kzalloc:
+out_err:
peripheral_free_list(bfin_nfc_pin_req);
return err;
diff --git a/drivers/mtd/nand/denali.c b/drivers/mtd/nand/denali.c
index c07cd57..9f2012a 100644
--- a/drivers/mtd/nand/denali.c
+++ b/drivers/mtd/nand/denali.c
@@ -1233,7 +1233,7 @@
return status;
}
-static void denali_erase(struct mtd_info *mtd, int page)
+static int denali_erase(struct mtd_info *mtd, int page)
{
struct denali_nand_info *denali = mtd_to_denali(mtd);
@@ -1250,8 +1250,7 @@
irq_status = wait_for_irq(denali, INTR_STATUS__ERASE_COMP |
INTR_STATUS__ERASE_FAIL);
- denali->status = (irq_status & INTR_STATUS__ERASE_FAIL) ?
- NAND_STATUS_FAIL : PASS;
+ return (irq_status & INTR_STATUS__ERASE_FAIL) ? NAND_STATUS_FAIL : PASS;
}
static void denali_cmdfunc(struct mtd_info *mtd, unsigned int cmd, int col,
@@ -1584,7 +1583,7 @@
denali->nand.ecc.write_page_raw = denali_write_page_raw;
denali->nand.ecc.read_oob = denali_read_oob;
denali->nand.ecc.write_oob = denali_write_oob;
- denali->nand.erase_cmd = denali_erase;
+ denali->nand.erase = denali_erase;
if (nand_scan_tail(&denali->mtd)) {
ret = -ENXIO;
diff --git a/drivers/mtd/nand/docg4.c b/drivers/mtd/nand/docg4.c
index 1b0265e..ce24637 100644
--- a/drivers/mtd/nand/docg4.c
+++ b/drivers/mtd/nand/docg4.c
@@ -872,7 +872,7 @@
return 0;
}
-static void docg4_erase_block(struct mtd_info *mtd, int page)
+static int docg4_erase_block(struct mtd_info *mtd, int page)
{
struct nand_chip *nand = mtd->priv;
struct docg4_priv *doc = nand->priv;
@@ -916,6 +916,8 @@
write_nop(docptr);
poll_status(doc);
write_nop(docptr);
+
+ return nand->waitfunc(mtd, nand);
}
static int write_page(struct mtd_info *mtd, struct nand_chip *nand,
@@ -1236,7 +1238,7 @@
nand->block_markbad = docg4_block_markbad;
nand->read_buf = docg4_read_buf;
nand->write_buf = docg4_write_buf16;
- nand->erase_cmd = docg4_erase_block;
+ nand->erase = docg4_erase_block;
nand->ecc.read_page = docg4_read_page;
nand->ecc.write_page = docg4_write_page;
nand->ecc.read_page_raw = docg4_read_page_raw;
diff --git a/drivers/mtd/nand/fsl_elbc_nand.c b/drivers/mtd/nand/fsl_elbc_nand.c
index ec549cd..545a5c0 100644
--- a/drivers/mtd/nand/fsl_elbc_nand.c
+++ b/drivers/mtd/nand/fsl_elbc_nand.c
@@ -723,6 +723,19 @@
return 0;
}
+/* ECC will be calculated automatically, and errors will be detected in
+ * waitfunc.
+ */
+static int fsl_elbc_write_subpage(struct mtd_info *mtd, struct nand_chip *chip,
+ uint32_t offset, uint32_t data_len,
+ const uint8_t *buf, int oob_required)
+{
+ fsl_elbc_write_buf(mtd, buf, mtd->writesize);
+ fsl_elbc_write_buf(mtd, chip->oob_poi, mtd->oobsize);
+
+ return 0;
+}
+
static int fsl_elbc_chip_init(struct fsl_elbc_mtd *priv)
{
struct fsl_lbc_ctrl *ctrl = priv->ctrl;
@@ -761,6 +774,7 @@
chip->ecc.read_page = fsl_elbc_read_page;
chip->ecc.write_page = fsl_elbc_write_page;
+ chip->ecc.write_subpage = fsl_elbc_write_subpage;
/* If CS Base Register selects full hardware ECC then use it */
if ((in_be32(&lbc->bank[priv->bank].br) & BR_DECC) ==
diff --git a/drivers/mtd/nand/fsl_ifc_nand.c b/drivers/mtd/nand/fsl_ifc_nand.c
index cb45d2f..2338124 100644
--- a/drivers/mtd/nand/fsl_ifc_nand.c
+++ b/drivers/mtd/nand/fsl_ifc_nand.c
@@ -56,7 +56,7 @@
struct nand_hw_control controller;
struct fsl_ifc_mtd *chips[FSL_IFC_BANK_COUNT];
- u8 __iomem *addr; /* Address of assigned IFC buffer */
+ void __iomem *addr; /* Address of assigned IFC buffer */
unsigned int page; /* Last page written to / read from */
unsigned int read_bytes;/* Number of bytes read during command */
unsigned int column; /* Saved column from SEQIN */
@@ -591,7 +591,10 @@
* The chip always seems to report that it is
* write-protected, even when it is not.
*/
- setbits8(ifc_nand_ctrl->addr, NAND_STATUS_WP);
+ if (chip->options & NAND_BUSWIDTH_16)
+ setbits16(ifc_nand_ctrl->addr, NAND_STATUS_WP);
+ else
+ setbits8(ifc_nand_ctrl->addr, NAND_STATUS_WP);
return;
case NAND_CMD_RESET:
@@ -636,7 +639,7 @@
len = bufsize - ifc_nand_ctrl->index;
}
- memcpy_toio(&ifc_nand_ctrl->addr[ifc_nand_ctrl->index], buf, len);
+ memcpy_toio(ifc_nand_ctrl->addr + ifc_nand_ctrl->index, buf, len);
ifc_nand_ctrl->index += len;
}
@@ -648,13 +651,16 @@
{
struct nand_chip *chip = mtd->priv;
struct fsl_ifc_mtd *priv = chip->priv;
+ unsigned int offset;
/*
* If there are still bytes in the IFC buffer, then use the
* next byte.
*/
- if (ifc_nand_ctrl->index < ifc_nand_ctrl->read_bytes)
- return in_8(&ifc_nand_ctrl->addr[ifc_nand_ctrl->index++]);
+ if (ifc_nand_ctrl->index < ifc_nand_ctrl->read_bytes) {
+ offset = ifc_nand_ctrl->index++;
+ return in_8(ifc_nand_ctrl->addr + offset);
+ }
dev_err(priv->dev, "%s: beyond end of buffer\n", __func__);
return ERR_BYTE;
@@ -675,8 +681,7 @@
* next byte.
*/
if (ifc_nand_ctrl->index < ifc_nand_ctrl->read_bytes) {
- data = in_be16((uint16_t __iomem *)&ifc_nand_ctrl->
- addr[ifc_nand_ctrl->index]);
+ data = in_be16(ifc_nand_ctrl->addr + ifc_nand_ctrl->index);
ifc_nand_ctrl->index += 2;
return (uint8_t) data;
}
@@ -701,7 +706,7 @@
avail = min((unsigned int)len,
ifc_nand_ctrl->read_bytes - ifc_nand_ctrl->index);
- memcpy_fromio(buf, &ifc_nand_ctrl->addr[ifc_nand_ctrl->index], avail);
+ memcpy_fromio(buf, ifc_nand_ctrl->addr + ifc_nand_ctrl->index, avail);
ifc_nand_ctrl->index += avail;
if (len > avail)
diff --git a/drivers/mtd/nand/gpmi-nand/bch-regs.h b/drivers/mtd/nand/gpmi-nand/bch-regs.h
index 588f537..05bb91f 100644
--- a/drivers/mtd/nand/gpmi-nand/bch-regs.h
+++ b/drivers/mtd/nand/gpmi-nand/bch-regs.h
@@ -54,7 +54,7 @@
#define MX6Q_BP_BCH_FLASH0LAYOUT0_ECC0 11
#define MX6Q_BM_BCH_FLASH0LAYOUT0_ECC0 (0x1f << MX6Q_BP_BCH_FLASH0LAYOUT0_ECC0)
#define BF_BCH_FLASH0LAYOUT0_ECC0(v, x) \
- (GPMI_IS_MX6Q(x) \
+ (GPMI_IS_MX6(x) \
? (((v) << MX6Q_BP_BCH_FLASH0LAYOUT0_ECC0) \
& MX6Q_BM_BCH_FLASH0LAYOUT0_ECC0) \
: (((v) << BP_BCH_FLASH0LAYOUT0_ECC0) \
@@ -65,7 +65,7 @@
#define MX6Q_BM_BCH_FLASH0LAYOUT0_GF_13_14 \
(0x1 << MX6Q_BP_BCH_FLASH0LAYOUT0_GF_13_14)
#define BF_BCH_FLASH0LAYOUT0_GF(v, x) \
- ((GPMI_IS_MX6Q(x) && ((v) == 14)) \
+ ((GPMI_IS_MX6(x) && ((v) == 14)) \
? (((1) << MX6Q_BP_BCH_FLASH0LAYOUT0_GF_13_14) \
& MX6Q_BM_BCH_FLASH0LAYOUT0_GF_13_14) \
: 0 \
@@ -77,7 +77,7 @@
#define MX6Q_BM_BCH_FLASH0LAYOUT0_DATA0_SIZE \
(0x3ff << BP_BCH_FLASH0LAYOUT0_DATA0_SIZE)
#define BF_BCH_FLASH0LAYOUT0_DATA0_SIZE(v, x) \
- (GPMI_IS_MX6Q(x) \
+ (GPMI_IS_MX6(x) \
? (((v) >> 2) & MX6Q_BM_BCH_FLASH0LAYOUT0_DATA0_SIZE) \
: ((v) & BM_BCH_FLASH0LAYOUT0_DATA0_SIZE) \
)
@@ -96,7 +96,7 @@
#define MX6Q_BP_BCH_FLASH0LAYOUT1_ECCN 11
#define MX6Q_BM_BCH_FLASH0LAYOUT1_ECCN (0x1f << MX6Q_BP_BCH_FLASH0LAYOUT1_ECCN)
#define BF_BCH_FLASH0LAYOUT1_ECCN(v, x) \
- (GPMI_IS_MX6Q(x) \
+ (GPMI_IS_MX6(x) \
? (((v) << MX6Q_BP_BCH_FLASH0LAYOUT1_ECCN) \
& MX6Q_BM_BCH_FLASH0LAYOUT1_ECCN) \
: (((v) << BP_BCH_FLASH0LAYOUT1_ECCN) \
@@ -107,7 +107,7 @@
#define MX6Q_BM_BCH_FLASH0LAYOUT1_GF_13_14 \
(0x1 << MX6Q_BP_BCH_FLASH0LAYOUT1_GF_13_14)
#define BF_BCH_FLASH0LAYOUT1_GF(v, x) \
- ((GPMI_IS_MX6Q(x) && ((v) == 14)) \
+ ((GPMI_IS_MX6(x) && ((v) == 14)) \
? (((1) << MX6Q_BP_BCH_FLASH0LAYOUT1_GF_13_14) \
& MX6Q_BM_BCH_FLASH0LAYOUT1_GF_13_14) \
: 0 \
@@ -119,7 +119,7 @@
#define MX6Q_BM_BCH_FLASH0LAYOUT1_DATAN_SIZE \
(0x3ff << BP_BCH_FLASH0LAYOUT1_DATAN_SIZE)
#define BF_BCH_FLASH0LAYOUT1_DATAN_SIZE(v, x) \
- (GPMI_IS_MX6Q(x) \
+ (GPMI_IS_MX6(x) \
? (((v) >> 2) & MX6Q_BM_BCH_FLASH0LAYOUT1_DATAN_SIZE) \
: ((v) & BM_BCH_FLASH0LAYOUT1_DATAN_SIZE) \
)
diff --git a/drivers/mtd/nand/gpmi-nand/gpmi-lib.c b/drivers/mtd/nand/gpmi-nand/gpmi-lib.c
index dd1df605..87e658c 100644
--- a/drivers/mtd/nand/gpmi-nand/gpmi-lib.c
+++ b/drivers/mtd/nand/gpmi-nand/gpmi-lib.c
@@ -861,7 +861,7 @@
struct resources *r = &this->resources;
unsigned long rate = clk_get_rate(r->clock[0]);
int mode = this->timing_mode;
- int dll_threshold = 16; /* in ns */
+ int dll_threshold = this->devdata->max_chain_delay;
unsigned long delay;
unsigned long clk_period;
int t_rea;
@@ -886,9 +886,6 @@
/* [3] for GPMI_HW_GPMI_CTRL1 */
hw->wrn_dly_sel = BV_GPMI_CTRL1_WRN_DLY_SEL_NO_DELAY;
- if (GPMI_IS_MX6Q(this))
- dll_threshold = 12;
-
/*
* Enlarge 10 times for the numerator and denominator in {3}.
* This make us to get more accurate result.
@@ -974,7 +971,7 @@
struct nand_chip *chip = &this->nand;
/* Enable the asynchronous EDO feature. */
- if (GPMI_IS_MX6Q(this) && chip->onfi_version) {
+ if (GPMI_IS_MX6(this) && chip->onfi_version) {
int mode = onfi_get_async_timing_mode(chip);
/* We only support the timing mode 4 and mode 5. */
@@ -1096,12 +1093,12 @@
if (GPMI_IS_MX23(this)) {
mask = MX23_BM_GPMI_DEBUG_READY0 << chip;
reg = readl(r->gpmi_regs + HW_GPMI_DEBUG);
- } else if (GPMI_IS_MX28(this) || GPMI_IS_MX6Q(this)) {
+ } else if (GPMI_IS_MX28(this) || GPMI_IS_MX6(this)) {
/*
* In the imx6, all the ready/busy pins are bound
* together. So we only need to check chip 0.
*/
- if (GPMI_IS_MX6Q(this))
+ if (GPMI_IS_MX6(this))
chip = 0;
/* MX28 shares the same R/B register as MX6Q. */
diff --git a/drivers/mtd/nand/gpmi-nand/gpmi-nand.c b/drivers/mtd/nand/gpmi-nand/gpmi-nand.c
index bb77f75..f638cd8 100644
--- a/drivers/mtd/nand/gpmi-nand/gpmi-nand.c
+++ b/drivers/mtd/nand/gpmi-nand/gpmi-nand.c
@@ -53,6 +53,30 @@
.oobfree = { {.offset = 0, .length = 0} }
};
+static const struct gpmi_devdata gpmi_devdata_imx23 = {
+ .type = IS_MX23,
+ .bch_max_ecc_strength = 20,
+ .max_chain_delay = 16,
+};
+
+static const struct gpmi_devdata gpmi_devdata_imx28 = {
+ .type = IS_MX28,
+ .bch_max_ecc_strength = 20,
+ .max_chain_delay = 16,
+};
+
+static const struct gpmi_devdata gpmi_devdata_imx6q = {
+ .type = IS_MX6Q,
+ .bch_max_ecc_strength = 40,
+ .max_chain_delay = 12,
+};
+
+static const struct gpmi_devdata gpmi_devdata_imx6sx = {
+ .type = IS_MX6SX,
+ .bch_max_ecc_strength = 62,
+ .max_chain_delay = 12,
+};
+
static irqreturn_t bch_irq(int irq, void *cookie)
{
struct gpmi_nand_data *this = cookie;
@@ -102,14 +126,8 @@
/* The mx23/mx28 only support the GF13. */
if (geo->gf_len == 14)
return false;
-
- if (geo->ecc_strength > MXS_ECC_STRENGTH_MAX)
- return false;
- } else if (GPMI_IS_MX6Q(this)) {
- if (geo->ecc_strength > MX6_ECC_STRENGTH_MAX)
- return false;
}
- return true;
+ return geo->ecc_strength <= this->devdata->bch_max_ecc_strength;
}
/*
@@ -270,8 +288,7 @@
"We can not support this nand chip."
" Its required ecc strength(%d) is beyond our"
" capability(%d).\n", geo->ecc_strength,
- (GPMI_IS_MX6Q(this) ? MX6_ECC_STRENGTH_MAX
- : MXS_ECC_STRENGTH_MAX));
+ this->devdata->bch_max_ecc_strength);
return -EINVAL;
}
@@ -572,7 +589,7 @@
}
/* Get extra clocks */
- if (GPMI_IS_MX6Q(this))
+ if (GPMI_IS_MX6(this))
extra_clks = extra_clks_for_mx6q;
if (!extra_clks)
return 0;
@@ -590,9 +607,9 @@
r->clock[i] = clk;
}
- if (GPMI_IS_MX6Q(this))
+ if (GPMI_IS_MX6(this))
/*
- * Set the default value for the gpmi clock in mx6q:
+ * Set the default value for the gpmi clock.
*
* If you want to use the ONFI nand which is in the
* Synchronous Mode, you should change the clock as you need.
@@ -1655,7 +1672,7 @@
* (1) the chip is imx6, and
* (2) the size of the ECC parity is byte aligned.
*/
- if (GPMI_IS_MX6Q(this) &&
+ if (GPMI_IS_MX6(this) &&
((bch_geo->gf_len * bch_geo->ecc_strength) % 8) == 0) {
ecc->read_subpage = gpmi_ecc_read_subpage;
chip->options |= NAND_SUBPAGE_READ;
@@ -1711,7 +1728,7 @@
if (ret)
goto err_out;
- ret = nand_scan_ident(mtd, GPMI_IS_MX6Q(this) ? 2 : 1, NULL);
+ ret = nand_scan_ident(mtd, GPMI_IS_MX6(this) ? 2 : 1, NULL);
if (ret)
goto err_out;
@@ -1740,23 +1757,19 @@
return ret;
}
-static const struct platform_device_id gpmi_ids[] = {
- { .name = "imx23-gpmi-nand", .driver_data = IS_MX23, },
- { .name = "imx28-gpmi-nand", .driver_data = IS_MX28, },
- { .name = "imx6q-gpmi-nand", .driver_data = IS_MX6Q, },
- {}
-};
-
static const struct of_device_id gpmi_nand_id_table[] = {
{
.compatible = "fsl,imx23-gpmi-nand",
- .data = (void *)&gpmi_ids[IS_MX23],
+ .data = (void *)&gpmi_devdata_imx23,
}, {
.compatible = "fsl,imx28-gpmi-nand",
- .data = (void *)&gpmi_ids[IS_MX28],
+ .data = (void *)&gpmi_devdata_imx28,
}, {
.compatible = "fsl,imx6q-gpmi-nand",
- .data = (void *)&gpmi_ids[IS_MX6Q],
+ .data = (void *)&gpmi_devdata_imx6q,
+ }, {
+ .compatible = "fsl,imx6sx-gpmi-nand",
+ .data = (void *)&gpmi_devdata_imx6sx,
}, {}
};
MODULE_DEVICE_TABLE(of, gpmi_nand_id_table);
@@ -1767,18 +1780,18 @@
const struct of_device_id *of_id;
int ret;
+ this = devm_kzalloc(&pdev->dev, sizeof(*this), GFP_KERNEL);
+ if (!this)
+ return -ENOMEM;
+
of_id = of_match_device(gpmi_nand_id_table, &pdev->dev);
if (of_id) {
- pdev->id_entry = of_id->data;
+ this->devdata = of_id->data;
} else {
dev_err(&pdev->dev, "Failed to find the right device id.\n");
return -ENODEV;
}
- this = devm_kzalloc(&pdev->dev, sizeof(*this), GFP_KERNEL);
- if (!this)
- return -ENOMEM;
-
platform_set_drvdata(pdev, this);
this->pdev = pdev;
this->dev = &pdev->dev;
@@ -1823,7 +1836,6 @@
},
.probe = gpmi_nand_probe,
.remove = gpmi_nand_remove,
- .id_table = gpmi_ids,
};
module_platform_driver(gpmi_nand_driver);
diff --git a/drivers/mtd/nand/gpmi-nand/gpmi-nand.h b/drivers/mtd/nand/gpmi-nand/gpmi-nand.h
index 4c801fa..32c6ba4 100644
--- a/drivers/mtd/nand/gpmi-nand/gpmi-nand.h
+++ b/drivers/mtd/nand/gpmi-nand/gpmi-nand.h
@@ -119,11 +119,25 @@
int8_t tRHOH_in_ns;
};
+enum gpmi_type {
+ IS_MX23,
+ IS_MX28,
+ IS_MX6Q,
+ IS_MX6SX
+};
+
+struct gpmi_devdata {
+ enum gpmi_type type;
+ int bch_max_ecc_strength;
+ int max_chain_delay; /* See the async EDO mode */
+};
+
struct gpmi_nand_data {
/* flags */
#define GPMI_ASYNC_EDO_ENABLED (1 << 0)
#define GPMI_TIMING_INIT_OK (1 << 1)
int flags;
+ const struct gpmi_devdata *devdata;
/* System Interface */
struct device *dev;
@@ -281,15 +295,11 @@
#define STATUS_ERASED 0xff
#define STATUS_UNCORRECTABLE 0xfe
-/* BCH's bit correction capability. */
-#define MXS_ECC_STRENGTH_MAX 20 /* mx23 and mx28 */
-#define MX6_ECC_STRENGTH_MAX 40
+/* Use the devdata to distinguish different Archs. */
+#define GPMI_IS_MX23(x) ((x)->devdata->type == IS_MX23)
+#define GPMI_IS_MX28(x) ((x)->devdata->type == IS_MX28)
+#define GPMI_IS_MX6Q(x) ((x)->devdata->type == IS_MX6Q)
+#define GPMI_IS_MX6SX(x) ((x)->devdata->type == IS_MX6SX)
-/* Use the platform_id to distinguish different Archs. */
-#define IS_MX23 0x0
-#define IS_MX28 0x1
-#define IS_MX6Q 0x2
-#define GPMI_IS_MX23(x) ((x)->pdev->id_entry->driver_data == IS_MX23)
-#define GPMI_IS_MX28(x) ((x)->pdev->id_entry->driver_data == IS_MX28)
-#define GPMI_IS_MX6Q(x) ((x)->pdev->id_entry->driver_data == IS_MX6Q)
+#define GPMI_IS_MX6(x) (GPMI_IS_MX6Q(x) || GPMI_IS_MX6SX(x))
#endif
diff --git a/drivers/mtd/nand/nand_base.c b/drivers/mtd/nand/nand_base.c
index 9d01c4d..41167e9 100644
--- a/drivers/mtd/nand/nand_base.c
+++ b/drivers/mtd/nand/nand_base.c
@@ -37,6 +37,7 @@
#include <linux/err.h>
#include <linux/sched.h>
#include <linux/slab.h>
+#include <linux/mm.h>
#include <linux/types.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/nand.h>
@@ -1204,8 +1205,7 @@
* ecc.pos. Let's make sure that there are no gaps in ECC positions.
*/
for (i = 0; i < eccfrag_len - 1; i++) {
- if (eccpos[i + start_step * chip->ecc.bytes] + 1 !=
- eccpos[i + start_step * chip->ecc.bytes + 1]) {
+ if (eccpos[i + index] + 1 != eccpos[i + index + 1]) {
gaps = 1;
break;
}
@@ -1501,6 +1501,7 @@
mtd->oobavail : mtd->oobsize;
uint8_t *bufpoi, *oob, *buf;
+ int use_bufpoi;
unsigned int max_bitflips = 0;
int retry_mode = 0;
bool ecc_fail = false;
@@ -1523,9 +1524,20 @@
bytes = min(mtd->writesize - col, readlen);
aligned = (bytes == mtd->writesize);
+ if (!aligned)
+ use_bufpoi = 1;
+ else if (chip->options & NAND_USE_BOUNCE_BUFFER)
+ use_bufpoi = !virt_addr_valid(buf);
+ else
+ use_bufpoi = 0;
+
/* Is the current page in the buffer? */
if (realpage != chip->pagebuf || oob) {
- bufpoi = aligned ? buf : chip->buffers->databuf;
+ bufpoi = use_bufpoi ? chip->buffers->databuf : buf;
+
+ if (use_bufpoi && aligned)
+ pr_debug("%s: using read bounce buffer for buf@%p\n",
+ __func__, buf);
read_retry:
chip->cmdfunc(mtd, NAND_CMD_READ0, 0x00, page);
@@ -1547,7 +1559,7 @@
ret = chip->ecc.read_page(mtd, chip, bufpoi,
oob_required, page);
if (ret < 0) {
- if (!aligned)
+ if (use_bufpoi)
/* Invalidate page cache */
chip->pagebuf = -1;
break;
@@ -1556,7 +1568,7 @@
max_bitflips = max_t(unsigned int, max_bitflips, ret);
/* Transfer not aligned data */
- if (!aligned) {
+ if (use_bufpoi) {
if (!NAND_HAS_SUBPAGE_READ(chip) && !oob &&
!(mtd->ecc_stats.failed - ecc_failures) &&
(ops->mode != MTD_OPS_RAW)) {
@@ -2376,11 +2388,23 @@
int bytes = mtd->writesize;
int cached = writelen > bytes && page != blockmask;
uint8_t *wbuf = buf;
+ int use_bufpoi;
+ int part_pagewr = (column || writelen < (mtd->writesize - 1));
- /* Partial page write? */
- if (unlikely(column || writelen < (mtd->writesize - 1))) {
+ if (part_pagewr)
+ use_bufpoi = 1;
+ else if (chip->options & NAND_USE_BOUNCE_BUFFER)
+ use_bufpoi = !virt_addr_valid(buf);
+ else
+ use_bufpoi = 0;
+
+ /* Partial page write?, or need to use bounce buffer */
+ if (use_bufpoi) {
+ pr_debug("%s: using write bounce buffer for buf@%p\n",
+ __func__, buf);
cached = 0;
- bytes = min_t(int, bytes - column, (int) writelen);
+ if (part_pagewr)
+ bytes = min_t(int, bytes - column, writelen);
chip->pagebuf = -1;
memset(chip->buffers->databuf, 0xff, mtd->writesize);
memcpy(&chip->buffers->databuf[column], buf, bytes);
@@ -2618,18 +2642,20 @@
}
/**
- * single_erase_cmd - [GENERIC] NAND standard block erase command function
+ * single_erase - [GENERIC] NAND standard block erase command function
* @mtd: MTD device structure
* @page: the page address of the block which will be erased
*
- * Standard erase command for NAND chips.
+ * Standard erase command for NAND chips. Returns NAND status.
*/
-static void single_erase_cmd(struct mtd_info *mtd, int page)
+static int single_erase(struct mtd_info *mtd, int page)
{
struct nand_chip *chip = mtd->priv;
/* Send commands to erase a block */
chip->cmdfunc(mtd, NAND_CMD_ERASE1, -1, page);
chip->cmdfunc(mtd, NAND_CMD_ERASE2, -1, -1);
+
+ return chip->waitfunc(mtd, chip);
}
/**
@@ -2710,9 +2736,7 @@
(page + pages_per_block))
chip->pagebuf = -1;
- chip->erase_cmd(mtd, page & chip->pagemask);
-
- status = chip->waitfunc(mtd, chip);
+ status = chip->erase(mtd, page & chip->pagemask);
/*
* See if operation failed and additional status checks are
@@ -3607,7 +3631,7 @@
chip->onfi_version = 0;
if (!type->name || !type->pagesize) {
- /* Check is chip is ONFI compliant */
+ /* Check if the chip is ONFI compliant */
if (nand_flash_detect_onfi(mtd, chip, &busw))
goto ident_done;
@@ -3685,7 +3709,7 @@
}
chip->badblockbits = 8;
- chip->erase_cmd = single_erase_cmd;
+ chip->erase = single_erase;
/* Do not replace user supplied command function! */
if (mtd->writesize > 512 && chip->cmdfunc == nand_command)
@@ -3770,6 +3794,39 @@
}
EXPORT_SYMBOL(nand_scan_ident);
+/*
+ * Check if the chip configuration meet the datasheet requirements.
+
+ * If our configuration corrects A bits per B bytes and the minimum
+ * required correction level is X bits per Y bytes, then we must ensure
+ * both of the following are true:
+ *
+ * (1) A / B >= X / Y
+ * (2) A >= X
+ *
+ * Requirement (1) ensures we can correct for the required bitflip density.
+ * Requirement (2) ensures we can correct even when all bitflips are clumped
+ * in the same sector.
+ */
+static bool nand_ecc_strength_good(struct mtd_info *mtd)
+{
+ struct nand_chip *chip = mtd->priv;
+ struct nand_ecc_ctrl *ecc = &chip->ecc;
+ int corr, ds_corr;
+
+ if (ecc->size == 0 || chip->ecc_step_ds == 0)
+ /* Not enough information */
+ return true;
+
+ /*
+ * We get the number of corrected bits per page to compare
+ * the correction density.
+ */
+ corr = (mtd->writesize * ecc->strength) / ecc->size;
+ ds_corr = (mtd->writesize * chip->ecc_strength_ds) / chip->ecc_step_ds;
+
+ return corr >= ds_corr && ecc->strength >= chip->ecc_strength_ds;
+}
/**
* nand_scan_tail - [NAND Interface] Scan for the NAND device
@@ -3990,6 +4047,9 @@
ecc->layout->oobavail += ecc->layout->oobfree[i].length;
mtd->oobavail = ecc->layout->oobavail;
+ /* ECC sanity check: warn noisily if it's too weak */
+ WARN_ON(!nand_ecc_strength_good(mtd));
+
/*
* Set the number of read / write steps for one page depending on ECC
* mode.
@@ -4023,8 +4083,16 @@
chip->pagebuf = -1;
/* Large page NAND with SOFT_ECC should support subpage reads */
- if ((ecc->mode == NAND_ECC_SOFT) && (chip->page_shift > 9))
- chip->options |= NAND_SUBPAGE_READ;
+ switch (ecc->mode) {
+ case NAND_ECC_SOFT:
+ case NAND_ECC_SOFT_BCH:
+ if (chip->page_shift > 9)
+ chip->options |= NAND_SUBPAGE_READ;
+ break;
+
+ default:
+ break;
+ }
/* Fill in remaining MTD driver data */
mtd->type = nand_is_slc(chip) ? MTD_NANDFLASH : MTD_MLCNANDFLASH;
diff --git a/drivers/mtd/nand/nand_bbt.c b/drivers/mtd/nand/nand_bbt.c
index c0615d1..7f0c3b4 100644
--- a/drivers/mtd/nand/nand_bbt.c
+++ b/drivers/mtd/nand/nand_bbt.c
@@ -528,7 +528,7 @@
{
struct nand_chip *this = mtd->priv;
int i, chips;
- int bits, startblock, block, dir;
+ int startblock, block, dir;
int scanlen = mtd->writesize + mtd->oobsize;
int bbtblocks;
int blocktopage = this->bbt_erase_shift - this->page_shift;
@@ -552,9 +552,6 @@
bbtblocks = mtd->size >> this->bbt_erase_shift;
}
- /* Number of bits for each erase block in the bbt */
- bits = td->options & NAND_BBT_NRBITS_MSK;
-
for (i = 0; i < chips; i++) {
/* Reset version information */
td->version[i] = 0;
@@ -1285,6 +1282,7 @@
int nand_default_bbt(struct mtd_info *mtd)
{
struct nand_chip *this = mtd->priv;
+ int ret;
/* Is a flash based bad block table requested? */
if (this->bbt_options & NAND_BBT_USE_FLASH) {
@@ -1303,8 +1301,11 @@
this->bbt_md = NULL;
}
- if (!this->badblock_pattern)
- nand_create_badblock_pattern(this);
+ if (!this->badblock_pattern) {
+ ret = nand_create_badblock_pattern(this);
+ if (ret)
+ return ret;
+ }
return nand_scan_bbt(mtd, this->badblock_pattern);
}
diff --git a/drivers/mtd/nand/nand_ecc.c b/drivers/mtd/nand/nand_ecc.c
index 053c9a2..97c4c02 100644
--- a/drivers/mtd/nand/nand_ecc.c
+++ b/drivers/mtd/nand/nand_ecc.c
@@ -506,7 +506,7 @@
if ((bitsperbyte[b0] + bitsperbyte[b1] + bitsperbyte[b2]) == 1)
return 1; /* error in ECC data; no action needed */
- pr_err("%s: uncorrectable ECC error", __func__);
+ pr_err("%s: uncorrectable ECC error\n", __func__);
return -1;
}
EXPORT_SYMBOL(__nand_correct_data);
diff --git a/drivers/mtd/nand/omap2.c b/drivers/mtd/nand/omap2.c
index 1ff49b8..f0ed92e 100644
--- a/drivers/mtd/nand/omap2.c
+++ b/drivers/mtd/nand/omap2.c
@@ -137,6 +137,10 @@
#define BADBLOCK_MARKER_LENGTH 2
#ifdef CONFIG_MTD_NAND_OMAP_BCH
+static u_char bch16_vector[] = {0xf5, 0x24, 0x1c, 0xd0, 0x61, 0xb3, 0xf1, 0x55,
+ 0x2e, 0x2c, 0x86, 0xa3, 0xed, 0x36, 0x1b, 0x78,
+ 0x48, 0x76, 0xa9, 0x3b, 0x97, 0xd1, 0x7a, 0x93,
+ 0x07, 0x0e};
static u_char bch8_vector[] = {0xf3, 0xdb, 0x14, 0x16, 0x8b, 0xd2, 0xbe, 0xcc,
0xac, 0x6b, 0xff, 0x99, 0x7b};
static u_char bch4_vector[] = {0x00, 0x6b, 0x31, 0xdd, 0x41, 0xbc, 0x10};
@@ -1114,6 +1118,19 @@
ecc_size1 = BCH_ECC_SIZE1;
}
break;
+ case OMAP_ECC_BCH16_CODE_HW:
+ bch_type = 0x2;
+ nsectors = chip->ecc.steps;
+ if (mode == NAND_ECC_READ) {
+ wr_mode = 0x01;
+ ecc_size0 = 52; /* ECC bits in nibbles per sector */
+ ecc_size1 = 0; /* non-ECC bits in nibbles per sector */
+ } else {
+ wr_mode = 0x01;
+ ecc_size0 = 0; /* extra bits in nibbles per sector */
+ ecc_size1 = 52; /* OOB bits in nibbles per sector */
+ }
+ break;
default:
return;
}
@@ -1162,7 +1179,8 @@
struct gpmc_nand_regs *gpmc_regs = &info->reg;
u8 *ecc_code;
unsigned long nsectors, bch_val1, bch_val2, bch_val3, bch_val4;
- int i;
+ u32 val;
+ int i, j;
nsectors = ((readl(info->reg.gpmc_ecc_config) >> 4) & 0x7) + 1;
for (i = 0; i < nsectors; i++) {
@@ -1201,6 +1219,41 @@
*ecc_code++ = ((bch_val1 >> 4) & 0xFF);
*ecc_code++ = ((bch_val1 & 0xF) << 4);
break;
+ case OMAP_ECC_BCH16_CODE_HW:
+ val = readl(gpmc_regs->gpmc_bch_result6[i]);
+ ecc_code[0] = ((val >> 8) & 0xFF);
+ ecc_code[1] = ((val >> 0) & 0xFF);
+ val = readl(gpmc_regs->gpmc_bch_result5[i]);
+ ecc_code[2] = ((val >> 24) & 0xFF);
+ ecc_code[3] = ((val >> 16) & 0xFF);
+ ecc_code[4] = ((val >> 8) & 0xFF);
+ ecc_code[5] = ((val >> 0) & 0xFF);
+ val = readl(gpmc_regs->gpmc_bch_result4[i]);
+ ecc_code[6] = ((val >> 24) & 0xFF);
+ ecc_code[7] = ((val >> 16) & 0xFF);
+ ecc_code[8] = ((val >> 8) & 0xFF);
+ ecc_code[9] = ((val >> 0) & 0xFF);
+ val = readl(gpmc_regs->gpmc_bch_result3[i]);
+ ecc_code[10] = ((val >> 24) & 0xFF);
+ ecc_code[11] = ((val >> 16) & 0xFF);
+ ecc_code[12] = ((val >> 8) & 0xFF);
+ ecc_code[13] = ((val >> 0) & 0xFF);
+ val = readl(gpmc_regs->gpmc_bch_result2[i]);
+ ecc_code[14] = ((val >> 24) & 0xFF);
+ ecc_code[15] = ((val >> 16) & 0xFF);
+ ecc_code[16] = ((val >> 8) & 0xFF);
+ ecc_code[17] = ((val >> 0) & 0xFF);
+ val = readl(gpmc_regs->gpmc_bch_result1[i]);
+ ecc_code[18] = ((val >> 24) & 0xFF);
+ ecc_code[19] = ((val >> 16) & 0xFF);
+ ecc_code[20] = ((val >> 8) & 0xFF);
+ ecc_code[21] = ((val >> 0) & 0xFF);
+ val = readl(gpmc_regs->gpmc_bch_result0[i]);
+ ecc_code[22] = ((val >> 24) & 0xFF);
+ ecc_code[23] = ((val >> 16) & 0xFF);
+ ecc_code[24] = ((val >> 8) & 0xFF);
+ ecc_code[25] = ((val >> 0) & 0xFF);
+ break;
default:
return -EINVAL;
}
@@ -1210,8 +1263,8 @@
case OMAP_ECC_BCH4_CODE_HW_DETECTION_SW:
/* Add constant polynomial to remainder, so that
* ECC of blank pages results in 0x0 on reading back */
- for (i = 0; i < eccbytes; i++)
- ecc_calc[i] ^= bch4_polynomial[i];
+ for (j = 0; j < eccbytes; j++)
+ ecc_calc[j] ^= bch4_polynomial[j];
break;
case OMAP_ECC_BCH4_CODE_HW:
/* Set 8th ECC byte as 0x0 for ROM compatibility */
@@ -1220,13 +1273,15 @@
case OMAP_ECC_BCH8_CODE_HW_DETECTION_SW:
/* Add constant polynomial to remainder, so that
* ECC of blank pages results in 0x0 on reading back */
- for (i = 0; i < eccbytes; i++)
- ecc_calc[i] ^= bch8_polynomial[i];
+ for (j = 0; j < eccbytes; j++)
+ ecc_calc[j] ^= bch8_polynomial[j];
break;
case OMAP_ECC_BCH8_CODE_HW:
/* Set 14th ECC byte as 0x0 for ROM compatibility */
ecc_calc[eccbytes - 1] = 0x0;
break;
+ case OMAP_ECC_BCH16_CODE_HW:
+ break;
default:
return -EINVAL;
}
@@ -1237,6 +1292,7 @@
return 0;
}
+#ifdef CONFIG_MTD_NAND_OMAP_BCH
/**
* erased_sector_bitflips - count bit flips
* @data: data sector buffer
@@ -1276,7 +1332,6 @@
return flip_bits;
}
-#ifdef CONFIG_MTD_NAND_OMAP_BCH
/**
* omap_elm_correct_data - corrects page data area in case error reported
* @mtd: MTD device structure
@@ -1318,6 +1373,10 @@
actual_eccbytes = ecc->bytes - 1;
erased_ecc_vec = bch8_vector;
break;
+ case OMAP_ECC_BCH16_CODE_HW:
+ actual_eccbytes = ecc->bytes;
+ erased_ecc_vec = bch16_vector;
+ break;
default:
pr_err("invalid driver configuration\n");
return -EINVAL;
@@ -1382,7 +1441,7 @@
/* Check if any error reported */
if (!is_error_reported)
- return 0;
+ return stat;
/* Decode BCH error using ELM module */
elm_decode_bch_error_page(info->elm_dev, ecc_vec, err_vec);
@@ -1401,6 +1460,7 @@
BCH4_BIT_PAD;
break;
case OMAP_ECC_BCH8_CODE_HW:
+ case OMAP_ECC_BCH16_CODE_HW:
pos = err_vec[i].error_loc[j];
break;
default:
@@ -1912,6 +1972,40 @@
goto return_error;
#endif
+ case OMAP_ECC_BCH16_CODE_HW:
+#ifdef CONFIG_MTD_NAND_OMAP_BCH
+ pr_info("using OMAP_ECC_BCH16_CODE_HW ECC scheme\n");
+ nand_chip->ecc.mode = NAND_ECC_HW;
+ nand_chip->ecc.size = 512;
+ nand_chip->ecc.bytes = 26;
+ nand_chip->ecc.strength = 16;
+ nand_chip->ecc.hwctl = omap_enable_hwecc_bch;
+ nand_chip->ecc.correct = omap_elm_correct_data;
+ nand_chip->ecc.calculate = omap_calculate_ecc_bch;
+ nand_chip->ecc.read_page = omap_read_page_bch;
+ nand_chip->ecc.write_page = omap_write_page_bch;
+ /* This ECC scheme requires ELM H/W block */
+ err = is_elm_present(info, pdata->elm_of_node, BCH16_ECC);
+ if (err < 0) {
+ pr_err("ELM is required for this ECC scheme\n");
+ goto return_error;
+ }
+ /* define ECC layout */
+ ecclayout->eccbytes = nand_chip->ecc.bytes *
+ (mtd->writesize /
+ nand_chip->ecc.size);
+ oob_index = BADBLOCK_MARKER_LENGTH;
+ for (i = 0; i < ecclayout->eccbytes; i++, oob_index++)
+ ecclayout->eccpos[i] = oob_index;
+ /* reserved marker already included in ecclayout->eccbytes */
+ ecclayout->oobfree->offset =
+ ecclayout->eccpos[ecclayout->eccbytes - 1] + 1;
+ break;
+#else
+ pr_err("nand: error: CONFIG_MTD_NAND_OMAP_BCH not enabled\n");
+ err = -EINVAL;
+ goto return_error;
+#endif
default:
pr_err("nand: error: invalid or unsupported ECC scheme\n");
err = -EINVAL;
diff --git a/drivers/mtd/nand/orion_nand.c b/drivers/mtd/nand/orion_nand.c
index dd7fe81..471b4df 100644
--- a/drivers/mtd/nand/orion_nand.c
+++ b/drivers/mtd/nand/orion_nand.c
@@ -214,7 +214,7 @@
}
#ifdef CONFIG_OF
-static struct of_device_id orion_nand_of_match_table[] = {
+static const struct of_device_id orion_nand_of_match_table[] = {
{ .compatible = "marvell,orion-nand", },
{},
};
diff --git a/drivers/mtd/nand/pxa3xx_nand.c b/drivers/mtd/nand/pxa3xx_nand.c
index 7588fe2..96b0b1d 100644
--- a/drivers/mtd/nand/pxa3xx_nand.c
+++ b/drivers/mtd/nand/pxa3xx_nand.c
@@ -127,10 +127,10 @@
/* macros for registers read/write */
#define nand_writel(info, off, val) \
- __raw_writel((val), (info)->mmio_base + (off))
+ writel_relaxed((val), (info)->mmio_base + (off))
#define nand_readl(info, off) \
- __raw_readl((info)->mmio_base + (off))
+ readl_relaxed((info)->mmio_base + (off))
/* error code and state */
enum {
@@ -337,7 +337,7 @@
/* convert nano-seconds to nand flash controller clock cycles */
#define ns2cycle(ns, clk) (int)((ns) * (clk / 1000000) / 1000)
-static struct of_device_id pxa3xx_nand_dt_ids[] = {
+static const struct of_device_id pxa3xx_nand_dt_ids[] = {
{
.compatible = "marvell,pxa3xx-nand",
.data = (void *)PXA3XX_NAND_VARIANT_PXA,
@@ -1354,7 +1354,6 @@
ecc->mode = NAND_ECC_HW;
ecc->size = 512;
ecc->strength = 1;
- return 1;
} else if (strength == 1 && ecc_stepsize == 512 && page_size == 512) {
info->chunk_size = 512;
@@ -1363,7 +1362,6 @@
ecc->mode = NAND_ECC_HW;
ecc->size = 512;
ecc->strength = 1;
- return 1;
/*
* Required ECC: 4-bit correction per 512 bytes
@@ -1378,7 +1376,6 @@
ecc->size = info->chunk_size;
ecc->layout = &ecc_layout_2KB_bch4bit;
ecc->strength = 16;
- return 1;
} else if (strength == 4 && ecc_stepsize == 512 && page_size == 4096) {
info->ecc_bch = 1;
@@ -1389,7 +1386,6 @@
ecc->size = info->chunk_size;
ecc->layout = &ecc_layout_4KB_bch4bit;
ecc->strength = 16;
- return 1;
/*
* Required ECC: 8-bit correction per 512 bytes
@@ -1404,8 +1400,15 @@
ecc->size = info->chunk_size;
ecc->layout = &ecc_layout_4KB_bch8bit;
ecc->strength = 16;
- return 1;
+ } else {
+ dev_err(&info->pdev->dev,
+ "ECC strength %d at page size %d is not supported\n",
+ strength, page_size);
+ return -ENODEV;
}
+
+ dev_info(&info->pdev->dev, "ECC strength %d, ECC step size %d\n",
+ ecc->strength, ecc->size);
return 0;
}
@@ -1516,8 +1519,13 @@
}
}
- ecc_strength = chip->ecc_strength_ds;
- ecc_step = chip->ecc_step_ds;
+ if (pdata->ecc_strength && pdata->ecc_step_size) {
+ ecc_strength = pdata->ecc_strength;
+ ecc_step = pdata->ecc_step_size;
+ } else {
+ ecc_strength = chip->ecc_strength_ds;
+ ecc_step = chip->ecc_step_ds;
+ }
/* Set default ECC strength requirements on non-ONFI devices */
if (ecc_strength < 1 && ecc_step < 1) {
@@ -1527,12 +1535,8 @@
ret = pxa_ecc_init(info, &chip->ecc, ecc_strength,
ecc_step, mtd->writesize);
- if (!ret) {
- dev_err(&info->pdev->dev,
- "ECC strength %d at page size %d is not supported\n",
- ecc_strength, mtd->writesize);
- return -ENODEV;
- }
+ if (ret)
+ return ret;
/* calculate addressing information */
if (mtd->writesize >= 2048)
@@ -1730,6 +1734,14 @@
of_property_read_u32(np, "num-cs", &pdata->num_cs);
pdata->flash_bbt = of_get_nand_on_flash_bbt(np);
+ pdata->ecc_strength = of_get_nand_ecc_strength(np);
+ if (pdata->ecc_strength < 0)
+ pdata->ecc_strength = 0;
+
+ pdata->ecc_step_size = of_get_nand_ecc_step_size(np);
+ if (pdata->ecc_step_size < 0)
+ pdata->ecc_step_size = 0;
+
pdev->dev.platform_data = pdata;
return 0;
diff --git a/drivers/mtd/nand/r852.c b/drivers/mtd/nand/r852.c
index 325930d..baea83f 100644
--- a/drivers/mtd/nand/r852.c
+++ b/drivers/mtd/nand/r852.c
@@ -245,7 +245,7 @@
}
/* write DWORD chinks - faster */
- while (len) {
+ while (len >= 4) {
reg = buf[0] | buf[1] << 8 | buf[2] << 16 | buf[3] << 24;
r852_write_reg_dword(dev, R852_DATALINE, reg);
buf += 4;
@@ -254,8 +254,10 @@
}
/* write rest */
- while (len)
+ while (len > 0) {
r852_write_reg(dev, R852_DATALINE, *buf++);
+ len--;
+ }
}
/*
diff --git a/drivers/mtd/onenand/samsung.c b/drivers/mtd/onenand/samsung.c
index b1a792f..efb819c 100644
--- a/drivers/mtd/onenand/samsung.c
+++ b/drivers/mtd/onenand/samsung.c
@@ -537,9 +537,9 @@
return 0;
}
-static int (*s5pc110_dma_ops)(void *dst, void *src, size_t count, int direction);
+static int (*s5pc110_dma_ops)(dma_addr_t dst, dma_addr_t src, size_t count, int direction);
-static int s5pc110_dma_poll(void *dst, void *src, size_t count, int direction)
+static int s5pc110_dma_poll(dma_addr_t dst, dma_addr_t src, size_t count, int direction)
{
void __iomem *base = onenand->dma_addr;
int status;
@@ -605,7 +605,7 @@
return IRQ_HANDLED;
}
-static int s5pc110_dma_irq(void *dst, void *src, size_t count, int direction)
+static int s5pc110_dma_irq(dma_addr_t dst, dma_addr_t src, size_t count, int direction)
{
void __iomem *base = onenand->dma_addr;
int status;
@@ -686,7 +686,7 @@
dev_err(dev, "Couldn't map a %d byte buffer for DMA\n", count);
goto normal;
}
- err = s5pc110_dma_ops((void *) dma_dst, (void *) dma_src,
+ err = s5pc110_dma_ops(dma_dst, dma_src,
count, S5PC110_DMA_DIR_READ);
if (page_dma)
diff --git a/drivers/mtd/spi-nor/Kconfig b/drivers/mtd/spi-nor/Kconfig
new file mode 100644
index 0000000..f8acfa4
--- /dev/null
+++ b/drivers/mtd/spi-nor/Kconfig
@@ -0,0 +1,17 @@
+menuconfig MTD_SPI_NOR
+ tristate "SPI-NOR device support"
+ depends on MTD
+ help
+ This is the framework for the SPI NOR which can be used by the SPI
+ device drivers and the SPI-NOR device driver.
+
+if MTD_SPI_NOR
+
+config SPI_FSL_QUADSPI
+ tristate "Freescale Quad SPI controller"
+ depends on ARCH_MXC
+ help
+ This enables support for the Quad SPI controller in master mode.
+ We only connect the NOR to this controller now.
+
+endif # MTD_SPI_NOR
diff --git a/drivers/mtd/spi-nor/Makefile b/drivers/mtd/spi-nor/Makefile
new file mode 100644
index 0000000..6a7ce14
--- /dev/null
+++ b/drivers/mtd/spi-nor/Makefile
@@ -0,0 +1,2 @@
+obj-$(CONFIG_MTD_SPI_NOR) += spi-nor.o
+obj-$(CONFIG_SPI_FSL_QUADSPI) += fsl-quadspi.o
diff --git a/drivers/mtd/spi-nor/fsl-quadspi.c b/drivers/mtd/spi-nor/fsl-quadspi.c
new file mode 100644
index 0000000..8d659a28
--- /dev/null
+++ b/drivers/mtd/spi-nor/fsl-quadspi.c
@@ -0,0 +1,1009 @@
+/*
+ * Freescale QuadSPI driver.
+ *
+ * Copyright (C) 2013 Freescale Semiconductor, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/interrupt.h>
+#include <linux/errno.h>
+#include <linux/platform_device.h>
+#include <linux/sched.h>
+#include <linux/delay.h>
+#include <linux/io.h>
+#include <linux/clk.h>
+#include <linux/err.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/timer.h>
+#include <linux/jiffies.h>
+#include <linux/completion.h>
+#include <linux/mtd/mtd.h>
+#include <linux/mtd/partitions.h>
+#include <linux/mtd/spi-nor.h>
+
+/* The registers */
+#define QUADSPI_MCR 0x00
+#define QUADSPI_MCR_RESERVED_SHIFT 16
+#define QUADSPI_MCR_RESERVED_MASK (0xF << QUADSPI_MCR_RESERVED_SHIFT)
+#define QUADSPI_MCR_MDIS_SHIFT 14
+#define QUADSPI_MCR_MDIS_MASK (1 << QUADSPI_MCR_MDIS_SHIFT)
+#define QUADSPI_MCR_CLR_TXF_SHIFT 11
+#define QUADSPI_MCR_CLR_TXF_MASK (1 << QUADSPI_MCR_CLR_TXF_SHIFT)
+#define QUADSPI_MCR_CLR_RXF_SHIFT 10
+#define QUADSPI_MCR_CLR_RXF_MASK (1 << QUADSPI_MCR_CLR_RXF_SHIFT)
+#define QUADSPI_MCR_DDR_EN_SHIFT 7
+#define QUADSPI_MCR_DDR_EN_MASK (1 << QUADSPI_MCR_DDR_EN_SHIFT)
+#define QUADSPI_MCR_END_CFG_SHIFT 2
+#define QUADSPI_MCR_END_CFG_MASK (3 << QUADSPI_MCR_END_CFG_SHIFT)
+#define QUADSPI_MCR_SWRSTHD_SHIFT 1
+#define QUADSPI_MCR_SWRSTHD_MASK (1 << QUADSPI_MCR_SWRSTHD_SHIFT)
+#define QUADSPI_MCR_SWRSTSD_SHIFT 0
+#define QUADSPI_MCR_SWRSTSD_MASK (1 << QUADSPI_MCR_SWRSTSD_SHIFT)
+
+#define QUADSPI_IPCR 0x08
+#define QUADSPI_IPCR_SEQID_SHIFT 24
+#define QUADSPI_IPCR_SEQID_MASK (0xF << QUADSPI_IPCR_SEQID_SHIFT)
+
+#define QUADSPI_BUF0CR 0x10
+#define QUADSPI_BUF1CR 0x14
+#define QUADSPI_BUF2CR 0x18
+#define QUADSPI_BUFXCR_INVALID_MSTRID 0xe
+
+#define QUADSPI_BUF3CR 0x1c
+#define QUADSPI_BUF3CR_ALLMST_SHIFT 31
+#define QUADSPI_BUF3CR_ALLMST (1 << QUADSPI_BUF3CR_ALLMST_SHIFT)
+
+#define QUADSPI_BFGENCR 0x20
+#define QUADSPI_BFGENCR_PAR_EN_SHIFT 16
+#define QUADSPI_BFGENCR_PAR_EN_MASK (1 << (QUADSPI_BFGENCR_PAR_EN_SHIFT))
+#define QUADSPI_BFGENCR_SEQID_SHIFT 12
+#define QUADSPI_BFGENCR_SEQID_MASK (0xF << QUADSPI_BFGENCR_SEQID_SHIFT)
+
+#define QUADSPI_BUF0IND 0x30
+#define QUADSPI_BUF1IND 0x34
+#define QUADSPI_BUF2IND 0x38
+#define QUADSPI_SFAR 0x100
+
+#define QUADSPI_SMPR 0x108
+#define QUADSPI_SMPR_DDRSMP_SHIFT 16
+#define QUADSPI_SMPR_DDRSMP_MASK (7 << QUADSPI_SMPR_DDRSMP_SHIFT)
+#define QUADSPI_SMPR_FSDLY_SHIFT 6
+#define QUADSPI_SMPR_FSDLY_MASK (1 << QUADSPI_SMPR_FSDLY_SHIFT)
+#define QUADSPI_SMPR_FSPHS_SHIFT 5
+#define QUADSPI_SMPR_FSPHS_MASK (1 << QUADSPI_SMPR_FSPHS_SHIFT)
+#define QUADSPI_SMPR_HSENA_SHIFT 0
+#define QUADSPI_SMPR_HSENA_MASK (1 << QUADSPI_SMPR_HSENA_SHIFT)
+
+#define QUADSPI_RBSR 0x10c
+#define QUADSPI_RBSR_RDBFL_SHIFT 8
+#define QUADSPI_RBSR_RDBFL_MASK (0x3F << QUADSPI_RBSR_RDBFL_SHIFT)
+
+#define QUADSPI_RBCT 0x110
+#define QUADSPI_RBCT_WMRK_MASK 0x1F
+#define QUADSPI_RBCT_RXBRD_SHIFT 8
+#define QUADSPI_RBCT_RXBRD_USEIPS (0x1 << QUADSPI_RBCT_RXBRD_SHIFT)
+
+#define QUADSPI_TBSR 0x150
+#define QUADSPI_TBDR 0x154
+#define QUADSPI_SR 0x15c
+#define QUADSPI_SR_IP_ACC_SHIFT 1
+#define QUADSPI_SR_IP_ACC_MASK (0x1 << QUADSPI_SR_IP_ACC_SHIFT)
+#define QUADSPI_SR_AHB_ACC_SHIFT 2
+#define QUADSPI_SR_AHB_ACC_MASK (0x1 << QUADSPI_SR_AHB_ACC_SHIFT)
+
+#define QUADSPI_FR 0x160
+#define QUADSPI_FR_TFF_MASK 0x1
+
+#define QUADSPI_SFA1AD 0x180
+#define QUADSPI_SFA2AD 0x184
+#define QUADSPI_SFB1AD 0x188
+#define QUADSPI_SFB2AD 0x18c
+#define QUADSPI_RBDR 0x200
+
+#define QUADSPI_LUTKEY 0x300
+#define QUADSPI_LUTKEY_VALUE 0x5AF05AF0
+
+#define QUADSPI_LCKCR 0x304
+#define QUADSPI_LCKER_LOCK 0x1
+#define QUADSPI_LCKER_UNLOCK 0x2
+
+#define QUADSPI_RSER 0x164
+#define QUADSPI_RSER_TFIE (0x1 << 0)
+
+#define QUADSPI_LUT_BASE 0x310
+
+/*
+ * The definition of the LUT register shows below:
+ *
+ * ---------------------------------------------------
+ * | INSTR1 | PAD1 | OPRND1 | INSTR0 | PAD0 | OPRND0 |
+ * ---------------------------------------------------
+ */
+#define OPRND0_SHIFT 0
+#define PAD0_SHIFT 8
+#define INSTR0_SHIFT 10
+#define OPRND1_SHIFT 16
+
+/* Instruction set for the LUT register. */
+#define LUT_STOP 0
+#define LUT_CMD 1
+#define LUT_ADDR 2
+#define LUT_DUMMY 3
+#define LUT_MODE 4
+#define LUT_MODE2 5
+#define LUT_MODE4 6
+#define LUT_READ 7
+#define LUT_WRITE 8
+#define LUT_JMP_ON_CS 9
+#define LUT_ADDR_DDR 10
+#define LUT_MODE_DDR 11
+#define LUT_MODE2_DDR 12
+#define LUT_MODE4_DDR 13
+#define LUT_READ_DDR 14
+#define LUT_WRITE_DDR 15
+#define LUT_DATA_LEARN 16
+
+/*
+ * The PAD definitions for LUT register.
+ *
+ * The pad stands for the lines number of IO[0:3].
+ * For example, the Quad read need four IO lines, so you should
+ * set LUT_PAD4 which means we use four IO lines.
+ */
+#define LUT_PAD1 0
+#define LUT_PAD2 1
+#define LUT_PAD4 2
+
+/* Oprands for the LUT register. */
+#define ADDR24BIT 0x18
+#define ADDR32BIT 0x20
+
+/* Macros for constructing the LUT register. */
+#define LUT0(ins, pad, opr) \
+ (((opr) << OPRND0_SHIFT) | ((LUT_##pad) << PAD0_SHIFT) | \
+ ((LUT_##ins) << INSTR0_SHIFT))
+
+#define LUT1(ins, pad, opr) (LUT0(ins, pad, opr) << OPRND1_SHIFT)
+
+/* other macros for LUT register. */
+#define QUADSPI_LUT(x) (QUADSPI_LUT_BASE + (x) * 4)
+#define QUADSPI_LUT_NUM 64
+
+/* SEQID -- we can have 16 seqids at most. */
+#define SEQID_QUAD_READ 0
+#define SEQID_WREN 1
+#define SEQID_WRDI 2
+#define SEQID_RDSR 3
+#define SEQID_SE 4
+#define SEQID_CHIP_ERASE 5
+#define SEQID_PP 6
+#define SEQID_RDID 7
+#define SEQID_WRSR 8
+#define SEQID_RDCR 9
+#define SEQID_EN4B 10
+#define SEQID_BRWR 11
+
+enum fsl_qspi_devtype {
+ FSL_QUADSPI_VYBRID,
+ FSL_QUADSPI_IMX6SX,
+};
+
+struct fsl_qspi_devtype_data {
+ enum fsl_qspi_devtype devtype;
+ int rxfifo;
+ int txfifo;
+};
+
+static struct fsl_qspi_devtype_data vybrid_data = {
+ .devtype = FSL_QUADSPI_VYBRID,
+ .rxfifo = 128,
+ .txfifo = 64
+};
+
+static struct fsl_qspi_devtype_data imx6sx_data = {
+ .devtype = FSL_QUADSPI_IMX6SX,
+ .rxfifo = 128,
+ .txfifo = 512
+};
+
+#define FSL_QSPI_MAX_CHIP 4
+struct fsl_qspi {
+ struct mtd_info mtd[FSL_QSPI_MAX_CHIP];
+ struct spi_nor nor[FSL_QSPI_MAX_CHIP];
+ void __iomem *iobase;
+ void __iomem *ahb_base; /* Used when read from AHB bus */
+ u32 memmap_phy;
+ struct clk *clk, *clk_en;
+ struct device *dev;
+ struct completion c;
+ struct fsl_qspi_devtype_data *devtype_data;
+ u32 nor_size;
+ u32 nor_num;
+ u32 clk_rate;
+ unsigned int chip_base_addr; /* We may support two chips. */
+};
+
+static inline int is_vybrid_qspi(struct fsl_qspi *q)
+{
+ return q->devtype_data->devtype == FSL_QUADSPI_VYBRID;
+}
+
+static inline int is_imx6sx_qspi(struct fsl_qspi *q)
+{
+ return q->devtype_data->devtype == FSL_QUADSPI_IMX6SX;
+}
+
+/*
+ * An IC bug makes us to re-arrange the 32-bit data.
+ * The following chips, such as IMX6SLX, have fixed this bug.
+ */
+static inline u32 fsl_qspi_endian_xchg(struct fsl_qspi *q, u32 a)
+{
+ return is_vybrid_qspi(q) ? __swab32(a) : a;
+}
+
+static inline void fsl_qspi_unlock_lut(struct fsl_qspi *q)
+{
+ writel(QUADSPI_LUTKEY_VALUE, q->iobase + QUADSPI_LUTKEY);
+ writel(QUADSPI_LCKER_UNLOCK, q->iobase + QUADSPI_LCKCR);
+}
+
+static inline void fsl_qspi_lock_lut(struct fsl_qspi *q)
+{
+ writel(QUADSPI_LUTKEY_VALUE, q->iobase + QUADSPI_LUTKEY);
+ writel(QUADSPI_LCKER_LOCK, q->iobase + QUADSPI_LCKCR);
+}
+
+static irqreturn_t fsl_qspi_irq_handler(int irq, void *dev_id)
+{
+ struct fsl_qspi *q = dev_id;
+ u32 reg;
+
+ /* clear interrupt */
+ reg = readl(q->iobase + QUADSPI_FR);
+ writel(reg, q->iobase + QUADSPI_FR);
+
+ if (reg & QUADSPI_FR_TFF_MASK)
+ complete(&q->c);
+
+ dev_dbg(q->dev, "QUADSPI_FR : 0x%.8x:0x%.8x\n", q->chip_base_addr, reg);
+ return IRQ_HANDLED;
+}
+
+static void fsl_qspi_init_lut(struct fsl_qspi *q)
+{
+ void __iomem *base = q->iobase;
+ int rxfifo = q->devtype_data->rxfifo;
+ u32 lut_base;
+ u8 cmd, addrlen, dummy;
+ int i;
+
+ fsl_qspi_unlock_lut(q);
+
+ /* Clear all the LUT table */
+ for (i = 0; i < QUADSPI_LUT_NUM; i++)
+ writel(0, base + QUADSPI_LUT_BASE + i * 4);
+
+ /* Quad Read */
+ lut_base = SEQID_QUAD_READ * 4;
+
+ if (q->nor_size <= SZ_16M) {
+ cmd = SPINOR_OP_READ_1_1_4;
+ addrlen = ADDR24BIT;
+ dummy = 8;
+ } else {
+ /* use the 4-byte address */
+ cmd = SPINOR_OP_READ_1_1_4;
+ addrlen = ADDR32BIT;
+ dummy = 8;
+ }
+
+ writel(LUT0(CMD, PAD1, cmd) | LUT1(ADDR, PAD1, addrlen),
+ base + QUADSPI_LUT(lut_base));
+ writel(LUT0(DUMMY, PAD1, dummy) | LUT1(READ, PAD4, rxfifo),
+ base + QUADSPI_LUT(lut_base + 1));
+
+ /* Write enable */
+ lut_base = SEQID_WREN * 4;
+ writel(LUT0(CMD, PAD1, SPINOR_OP_WREN), base + QUADSPI_LUT(lut_base));
+
+ /* Page Program */
+ lut_base = SEQID_PP * 4;
+
+ if (q->nor_size <= SZ_16M) {
+ cmd = SPINOR_OP_PP;
+ addrlen = ADDR24BIT;
+ } else {
+ /* use the 4-byte address */
+ cmd = SPINOR_OP_PP;
+ addrlen = ADDR32BIT;
+ }
+
+ writel(LUT0(CMD, PAD1, cmd) | LUT1(ADDR, PAD1, addrlen),
+ base + QUADSPI_LUT(lut_base));
+ writel(LUT0(WRITE, PAD1, 0), base + QUADSPI_LUT(lut_base + 1));
+
+ /* Read Status */
+ lut_base = SEQID_RDSR * 4;
+ writel(LUT0(CMD, PAD1, SPINOR_OP_RDSR) | LUT1(READ, PAD1, 0x1),
+ base + QUADSPI_LUT(lut_base));
+
+ /* Erase a sector */
+ lut_base = SEQID_SE * 4;
+
+ if (q->nor_size <= SZ_16M) {
+ cmd = SPINOR_OP_SE;
+ addrlen = ADDR24BIT;
+ } else {
+ /* use the 4-byte address */
+ cmd = SPINOR_OP_SE;
+ addrlen = ADDR32BIT;
+ }
+
+ writel(LUT0(CMD, PAD1, cmd) | LUT1(ADDR, PAD1, addrlen),
+ base + QUADSPI_LUT(lut_base));
+
+ /* Erase the whole chip */
+ lut_base = SEQID_CHIP_ERASE * 4;
+ writel(LUT0(CMD, PAD1, SPINOR_OP_CHIP_ERASE),
+ base + QUADSPI_LUT(lut_base));
+
+ /* READ ID */
+ lut_base = SEQID_RDID * 4;
+ writel(LUT0(CMD, PAD1, SPINOR_OP_RDID) | LUT1(READ, PAD1, 0x8),
+ base + QUADSPI_LUT(lut_base));
+
+ /* Write Register */
+ lut_base = SEQID_WRSR * 4;
+ writel(LUT0(CMD, PAD1, SPINOR_OP_WRSR) | LUT1(WRITE, PAD1, 0x2),
+ base + QUADSPI_LUT(lut_base));
+
+ /* Read Configuration Register */
+ lut_base = SEQID_RDCR * 4;
+ writel(LUT0(CMD, PAD1, SPINOR_OP_RDCR) | LUT1(READ, PAD1, 0x1),
+ base + QUADSPI_LUT(lut_base));
+
+ /* Write disable */
+ lut_base = SEQID_WRDI * 4;
+ writel(LUT0(CMD, PAD1, SPINOR_OP_WRDI), base + QUADSPI_LUT(lut_base));
+
+ /* Enter 4 Byte Mode (Micron) */
+ lut_base = SEQID_EN4B * 4;
+ writel(LUT0(CMD, PAD1, SPINOR_OP_EN4B), base + QUADSPI_LUT(lut_base));
+
+ /* Enter 4 Byte Mode (Spansion) */
+ lut_base = SEQID_BRWR * 4;
+ writel(LUT0(CMD, PAD1, SPINOR_OP_BRWR), base + QUADSPI_LUT(lut_base));
+
+ fsl_qspi_lock_lut(q);
+}
+
+/* Get the SEQID for the command */
+static int fsl_qspi_get_seqid(struct fsl_qspi *q, u8 cmd)
+{
+ switch (cmd) {
+ case SPINOR_OP_READ_1_1_4:
+ return SEQID_QUAD_READ;
+ case SPINOR_OP_WREN:
+ return SEQID_WREN;
+ case SPINOR_OP_WRDI:
+ return SEQID_WRDI;
+ case SPINOR_OP_RDSR:
+ return SEQID_RDSR;
+ case SPINOR_OP_SE:
+ return SEQID_SE;
+ case SPINOR_OP_CHIP_ERASE:
+ return SEQID_CHIP_ERASE;
+ case SPINOR_OP_PP:
+ return SEQID_PP;
+ case SPINOR_OP_RDID:
+ return SEQID_RDID;
+ case SPINOR_OP_WRSR:
+ return SEQID_WRSR;
+ case SPINOR_OP_RDCR:
+ return SEQID_RDCR;
+ case SPINOR_OP_EN4B:
+ return SEQID_EN4B;
+ case SPINOR_OP_BRWR:
+ return SEQID_BRWR;
+ default:
+ dev_err(q->dev, "Unsupported cmd 0x%.2x\n", cmd);
+ break;
+ }
+ return -EINVAL;
+}
+
+static int
+fsl_qspi_runcmd(struct fsl_qspi *q, u8 cmd, unsigned int addr, int len)
+{
+ void __iomem *base = q->iobase;
+ int seqid;
+ u32 reg, reg2;
+ int err;
+
+ init_completion(&q->c);
+ dev_dbg(q->dev, "to 0x%.8x:0x%.8x, len:%d, cmd:%.2x\n",
+ q->chip_base_addr, addr, len, cmd);
+
+ /* save the reg */
+ reg = readl(base + QUADSPI_MCR);
+
+ writel(q->memmap_phy + q->chip_base_addr + addr, base + QUADSPI_SFAR);
+ writel(QUADSPI_RBCT_WMRK_MASK | QUADSPI_RBCT_RXBRD_USEIPS,
+ base + QUADSPI_RBCT);
+ writel(reg | QUADSPI_MCR_CLR_RXF_MASK, base + QUADSPI_MCR);
+
+ do {
+ reg2 = readl(base + QUADSPI_SR);
+ if (reg2 & (QUADSPI_SR_IP_ACC_MASK | QUADSPI_SR_AHB_ACC_MASK)) {
+ udelay(1);
+ dev_dbg(q->dev, "The controller is busy, 0x%x\n", reg2);
+ continue;
+ }
+ break;
+ } while (1);
+
+ /* trigger the LUT now */
+ seqid = fsl_qspi_get_seqid(q, cmd);
+ writel((seqid << QUADSPI_IPCR_SEQID_SHIFT) | len, base + QUADSPI_IPCR);
+
+ /* Wait for the interrupt. */
+ err = wait_for_completion_timeout(&q->c, msecs_to_jiffies(1000));
+ if (!err) {
+ dev_err(q->dev,
+ "cmd 0x%.2x timeout, addr@%.8x, FR:0x%.8x, SR:0x%.8x\n",
+ cmd, addr, readl(base + QUADSPI_FR),
+ readl(base + QUADSPI_SR));
+ err = -ETIMEDOUT;
+ } else {
+ err = 0;
+ }
+
+ /* restore the MCR */
+ writel(reg, base + QUADSPI_MCR);
+
+ return err;
+}
+
+/* Read out the data from the QUADSPI_RBDR buffer registers. */
+static void fsl_qspi_read_data(struct fsl_qspi *q, int len, u8 *rxbuf)
+{
+ u32 tmp;
+ int i = 0;
+
+ while (len > 0) {
+ tmp = readl(q->iobase + QUADSPI_RBDR + i * 4);
+ tmp = fsl_qspi_endian_xchg(q, tmp);
+ dev_dbg(q->dev, "chip addr:0x%.8x, rcv:0x%.8x\n",
+ q->chip_base_addr, tmp);
+
+ if (len >= 4) {
+ *((u32 *)rxbuf) = tmp;
+ rxbuf += 4;
+ } else {
+ memcpy(rxbuf, &tmp, len);
+ break;
+ }
+
+ len -= 4;
+ i++;
+ }
+}
+
+/*
+ * If we have changed the content of the flash by writing or erasing,
+ * we need to invalidate the AHB buffer. If we do not do so, we may read out
+ * the wrong data. The spec tells us reset the AHB domain and Serial Flash
+ * domain at the same time.
+ */
+static inline void fsl_qspi_invalid(struct fsl_qspi *q)
+{
+ u32 reg;
+
+ reg = readl(q->iobase + QUADSPI_MCR);
+ reg |= QUADSPI_MCR_SWRSTHD_MASK | QUADSPI_MCR_SWRSTSD_MASK;
+ writel(reg, q->iobase + QUADSPI_MCR);
+
+ /*
+ * The minimum delay : 1 AHB + 2 SFCK clocks.
+ * Delay 1 us is enough.
+ */
+ udelay(1);
+
+ reg &= ~(QUADSPI_MCR_SWRSTHD_MASK | QUADSPI_MCR_SWRSTSD_MASK);
+ writel(reg, q->iobase + QUADSPI_MCR);
+}
+
+static int fsl_qspi_nor_write(struct fsl_qspi *q, struct spi_nor *nor,
+ u8 opcode, unsigned int to, u32 *txbuf,
+ unsigned count, size_t *retlen)
+{
+ int ret, i, j;
+ u32 tmp;
+
+ dev_dbg(q->dev, "to 0x%.8x:0x%.8x, len : %d\n",
+ q->chip_base_addr, to, count);
+
+ /* clear the TX FIFO. */
+ tmp = readl(q->iobase + QUADSPI_MCR);
+ writel(tmp | QUADSPI_MCR_CLR_RXF_MASK, q->iobase + QUADSPI_MCR);
+
+ /* fill the TX data to the FIFO */
+ for (j = 0, i = ((count + 3) / 4); j < i; j++) {
+ tmp = fsl_qspi_endian_xchg(q, *txbuf);
+ writel(tmp, q->iobase + QUADSPI_TBDR);
+ txbuf++;
+ }
+
+ /* Trigger it */
+ ret = fsl_qspi_runcmd(q, opcode, to, count);
+
+ if (ret == 0 && retlen)
+ *retlen += count;
+
+ return ret;
+}
+
+static void fsl_qspi_set_map_addr(struct fsl_qspi *q)
+{
+ int nor_size = q->nor_size;
+ void __iomem *base = q->iobase;
+
+ writel(nor_size + q->memmap_phy, base + QUADSPI_SFA1AD);
+ writel(nor_size * 2 + q->memmap_phy, base + QUADSPI_SFA2AD);
+ writel(nor_size * 3 + q->memmap_phy, base + QUADSPI_SFB1AD);
+ writel(nor_size * 4 + q->memmap_phy, base + QUADSPI_SFB2AD);
+}
+
+/*
+ * There are two different ways to read out the data from the flash:
+ * the "IP Command Read" and the "AHB Command Read".
+ *
+ * The IC guy suggests we use the "AHB Command Read" which is faster
+ * then the "IP Command Read". (What's more is that there is a bug in
+ * the "IP Command Read" in the Vybrid.)
+ *
+ * After we set up the registers for the "AHB Command Read", we can use
+ * the memcpy to read the data directly. A "missed" access to the buffer
+ * causes the controller to clear the buffer, and use the sequence pointed
+ * by the QUADSPI_BFGENCR[SEQID] to initiate a read from the flash.
+ */
+static void fsl_qspi_init_abh_read(struct fsl_qspi *q)
+{
+ void __iomem *base = q->iobase;
+ int seqid;
+
+ /* AHB configuration for access buffer 0/1/2 .*/
+ writel(QUADSPI_BUFXCR_INVALID_MSTRID, base + QUADSPI_BUF0CR);
+ writel(QUADSPI_BUFXCR_INVALID_MSTRID, base + QUADSPI_BUF1CR);
+ writel(QUADSPI_BUFXCR_INVALID_MSTRID, base + QUADSPI_BUF2CR);
+ writel(QUADSPI_BUF3CR_ALLMST, base + QUADSPI_BUF3CR);
+
+ /* We only use the buffer3 */
+ writel(0, base + QUADSPI_BUF0IND);
+ writel(0, base + QUADSPI_BUF1IND);
+ writel(0, base + QUADSPI_BUF2IND);
+
+ /* Set the default lut sequence for AHB Read. */
+ seqid = fsl_qspi_get_seqid(q, q->nor[0].read_opcode);
+ writel(seqid << QUADSPI_BFGENCR_SEQID_SHIFT,
+ q->iobase + QUADSPI_BFGENCR);
+}
+
+/* We use this function to do some basic init for spi_nor_scan(). */
+static int fsl_qspi_nor_setup(struct fsl_qspi *q)
+{
+ void __iomem *base = q->iobase;
+ u32 reg;
+ int ret;
+
+ /* the default frequency, we will change it in the future.*/
+ ret = clk_set_rate(q->clk, 66000000);
+ if (ret)
+ return ret;
+
+ /* Init the LUT table. */
+ fsl_qspi_init_lut(q);
+
+ /* Disable the module */
+ writel(QUADSPI_MCR_MDIS_MASK | QUADSPI_MCR_RESERVED_MASK,
+ base + QUADSPI_MCR);
+
+ reg = readl(base + QUADSPI_SMPR);
+ writel(reg & ~(QUADSPI_SMPR_FSDLY_MASK
+ | QUADSPI_SMPR_FSPHS_MASK
+ | QUADSPI_SMPR_HSENA_MASK
+ | QUADSPI_SMPR_DDRSMP_MASK), base + QUADSPI_SMPR);
+
+ /* Enable the module */
+ writel(QUADSPI_MCR_RESERVED_MASK | QUADSPI_MCR_END_CFG_MASK,
+ base + QUADSPI_MCR);
+
+ /* enable the interrupt */
+ writel(QUADSPI_RSER_TFIE, q->iobase + QUADSPI_RSER);
+
+ return 0;
+}
+
+static int fsl_qspi_nor_setup_last(struct fsl_qspi *q)
+{
+ unsigned long rate = q->clk_rate;
+ int ret;
+
+ if (is_imx6sx_qspi(q))
+ rate *= 4;
+
+ ret = clk_set_rate(q->clk, rate);
+ if (ret)
+ return ret;
+
+ /* Init the LUT table again. */
+ fsl_qspi_init_lut(q);
+
+ /* Init for AHB read */
+ fsl_qspi_init_abh_read(q);
+
+ return 0;
+}
+
+static struct of_device_id fsl_qspi_dt_ids[] = {
+ { .compatible = "fsl,vf610-qspi", .data = (void *)&vybrid_data, },
+ { .compatible = "fsl,imx6sx-qspi", .data = (void *)&imx6sx_data, },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, fsl_qspi_dt_ids);
+
+static void fsl_qspi_set_base_addr(struct fsl_qspi *q, struct spi_nor *nor)
+{
+ q->chip_base_addr = q->nor_size * (nor - q->nor);
+}
+
+static int fsl_qspi_read_reg(struct spi_nor *nor, u8 opcode, u8 *buf, int len)
+{
+ int ret;
+ struct fsl_qspi *q = nor->priv;
+
+ ret = fsl_qspi_runcmd(q, opcode, 0, len);
+ if (ret)
+ return ret;
+
+ fsl_qspi_read_data(q, len, buf);
+ return 0;
+}
+
+static int fsl_qspi_write_reg(struct spi_nor *nor, u8 opcode, u8 *buf, int len,
+ int write_enable)
+{
+ struct fsl_qspi *q = nor->priv;
+ int ret;
+
+ if (!buf) {
+ ret = fsl_qspi_runcmd(q, opcode, 0, 1);
+ if (ret)
+ return ret;
+
+ if (opcode == SPINOR_OP_CHIP_ERASE)
+ fsl_qspi_invalid(q);
+
+ } else if (len > 0) {
+ ret = fsl_qspi_nor_write(q, nor, opcode, 0,
+ (u32 *)buf, len, NULL);
+ } else {
+ dev_err(q->dev, "invalid cmd %d\n", opcode);
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static void fsl_qspi_write(struct spi_nor *nor, loff_t to,
+ size_t len, size_t *retlen, const u_char *buf)
+{
+ struct fsl_qspi *q = nor->priv;
+
+ fsl_qspi_nor_write(q, nor, nor->program_opcode, to,
+ (u32 *)buf, len, retlen);
+
+ /* invalid the data in the AHB buffer. */
+ fsl_qspi_invalid(q);
+}
+
+static int fsl_qspi_read(struct spi_nor *nor, loff_t from,
+ size_t len, size_t *retlen, u_char *buf)
+{
+ struct fsl_qspi *q = nor->priv;
+ u8 cmd = nor->read_opcode;
+ int ret;
+
+ dev_dbg(q->dev, "cmd [%x],read from (0x%p, 0x%.8x, 0x%.8x),len:%d\n",
+ cmd, q->ahb_base, q->chip_base_addr, (unsigned int)from, len);
+
+ /* Wait until the previous command is finished. */
+ ret = nor->wait_till_ready(nor);
+ if (ret)
+ return ret;
+
+ /* Read out the data directly from the AHB buffer.*/
+ memcpy(buf, q->ahb_base + q->chip_base_addr + from, len);
+
+ *retlen += len;
+ return 0;
+}
+
+static int fsl_qspi_erase(struct spi_nor *nor, loff_t offs)
+{
+ struct fsl_qspi *q = nor->priv;
+ int ret;
+
+ dev_dbg(nor->dev, "%dKiB at 0x%08x:0x%08x\n",
+ nor->mtd->erasesize / 1024, q->chip_base_addr, (u32)offs);
+
+ /* Wait until finished previous write command. */
+ ret = nor->wait_till_ready(nor);
+ if (ret)
+ return ret;
+
+ /* Send write enable, then erase commands. */
+ ret = nor->write_reg(nor, SPINOR_OP_WREN, NULL, 0, 0);
+ if (ret)
+ return ret;
+
+ ret = fsl_qspi_runcmd(q, nor->erase_opcode, offs, 0);
+ if (ret)
+ return ret;
+
+ fsl_qspi_invalid(q);
+ return 0;
+}
+
+static int fsl_qspi_prep(struct spi_nor *nor, enum spi_nor_ops ops)
+{
+ struct fsl_qspi *q = nor->priv;
+ int ret;
+
+ ret = clk_enable(q->clk_en);
+ if (ret)
+ return ret;
+
+ ret = clk_enable(q->clk);
+ if (ret) {
+ clk_disable(q->clk_en);
+ return ret;
+ }
+
+ fsl_qspi_set_base_addr(q, nor);
+ return 0;
+}
+
+static void fsl_qspi_unprep(struct spi_nor *nor, enum spi_nor_ops ops)
+{
+ struct fsl_qspi *q = nor->priv;
+
+ clk_disable(q->clk);
+ clk_disable(q->clk_en);
+}
+
+static int fsl_qspi_probe(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct mtd_part_parser_data ppdata;
+ struct device *dev = &pdev->dev;
+ struct fsl_qspi *q;
+ struct resource *res;
+ struct spi_nor *nor;
+ struct mtd_info *mtd;
+ int ret, i = 0;
+ bool has_second_chip = false;
+ const struct of_device_id *of_id =
+ of_match_device(fsl_qspi_dt_ids, &pdev->dev);
+
+ q = devm_kzalloc(dev, sizeof(*q), GFP_KERNEL);
+ if (!q)
+ return -ENOMEM;
+
+ q->nor_num = of_get_child_count(dev->of_node);
+ if (!q->nor_num || q->nor_num > FSL_QSPI_MAX_CHIP)
+ return -ENODEV;
+
+ /* find the resources */
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "QuadSPI");
+ q->iobase = devm_ioremap_resource(dev, res);
+ if (IS_ERR(q->iobase)) {
+ ret = PTR_ERR(q->iobase);
+ goto map_failed;
+ }
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
+ "QuadSPI-memory");
+ q->ahb_base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(q->ahb_base)) {
+ ret = PTR_ERR(q->ahb_base);
+ goto map_failed;
+ }
+ q->memmap_phy = res->start;
+
+ /* find the clocks */
+ q->clk_en = devm_clk_get(dev, "qspi_en");
+ if (IS_ERR(q->clk_en)) {
+ ret = PTR_ERR(q->clk_en);
+ goto map_failed;
+ }
+
+ q->clk = devm_clk_get(dev, "qspi");
+ if (IS_ERR(q->clk)) {
+ ret = PTR_ERR(q->clk);
+ goto map_failed;
+ }
+
+ ret = clk_prepare_enable(q->clk_en);
+ if (ret) {
+ dev_err(dev, "can not enable the qspi_en clock\n");
+ goto map_failed;
+ }
+
+ ret = clk_prepare_enable(q->clk);
+ if (ret) {
+ clk_disable_unprepare(q->clk_en);
+ dev_err(dev, "can not enable the qspi clock\n");
+ goto map_failed;
+ }
+
+ /* find the irq */
+ ret = platform_get_irq(pdev, 0);
+ if (ret < 0) {
+ dev_err(dev, "failed to get the irq\n");
+ goto irq_failed;
+ }
+
+ ret = devm_request_irq(dev, ret,
+ fsl_qspi_irq_handler, 0, pdev->name, q);
+ if (ret) {
+ dev_err(dev, "failed to request irq.\n");
+ goto irq_failed;
+ }
+
+ q->dev = dev;
+ q->devtype_data = (struct fsl_qspi_devtype_data *)of_id->data;
+ platform_set_drvdata(pdev, q);
+
+ ret = fsl_qspi_nor_setup(q);
+ if (ret)
+ goto irq_failed;
+
+ if (of_get_property(np, "fsl,qspi-has-second-chip", NULL))
+ has_second_chip = true;
+
+ /* iterate the subnodes. */
+ for_each_available_child_of_node(dev->of_node, np) {
+ const struct spi_device_id *id;
+ char modalias[40];
+
+ /* skip the holes */
+ if (!has_second_chip)
+ i *= 2;
+
+ nor = &q->nor[i];
+ mtd = &q->mtd[i];
+
+ nor->mtd = mtd;
+ nor->dev = dev;
+ nor->priv = q;
+ mtd->priv = nor;
+
+ /* fill the hooks */
+ nor->read_reg = fsl_qspi_read_reg;
+ nor->write_reg = fsl_qspi_write_reg;
+ nor->read = fsl_qspi_read;
+ nor->write = fsl_qspi_write;
+ nor->erase = fsl_qspi_erase;
+
+ nor->prepare = fsl_qspi_prep;
+ nor->unprepare = fsl_qspi_unprep;
+
+ if (of_modalias_node(np, modalias, sizeof(modalias)) < 0)
+ goto map_failed;
+
+ id = spi_nor_match_id(modalias);
+ if (!id)
+ goto map_failed;
+
+ ret = of_property_read_u32(np, "spi-max-frequency",
+ &q->clk_rate);
+ if (ret < 0)
+ goto map_failed;
+
+ /* set the chip address for READID */
+ fsl_qspi_set_base_addr(q, nor);
+
+ ret = spi_nor_scan(nor, id, SPI_NOR_QUAD);
+ if (ret)
+ goto map_failed;
+
+ ppdata.of_node = np;
+ ret = mtd_device_parse_register(mtd, NULL, &ppdata, NULL, 0);
+ if (ret)
+ goto map_failed;
+
+ /* Set the correct NOR size now. */
+ if (q->nor_size == 0) {
+ q->nor_size = mtd->size;
+
+ /* Map the SPI NOR to accessiable address */
+ fsl_qspi_set_map_addr(q);
+ }
+
+ /*
+ * The TX FIFO is 64 bytes in the Vybrid, but the Page Program
+ * may writes 265 bytes per time. The write is working in the
+ * unit of the TX FIFO, not in the unit of the SPI NOR's page
+ * size.
+ *
+ * So shrink the spi_nor->page_size if it is larger then the
+ * TX FIFO.
+ */
+ if (nor->page_size > q->devtype_data->txfifo)
+ nor->page_size = q->devtype_data->txfifo;
+
+ i++;
+ }
+
+ /* finish the rest init. */
+ ret = fsl_qspi_nor_setup_last(q);
+ if (ret)
+ goto last_init_failed;
+
+ clk_disable(q->clk);
+ clk_disable(q->clk_en);
+ dev_info(dev, "QuadSPI SPI NOR flash driver\n");
+ return 0;
+
+last_init_failed:
+ for (i = 0; i < q->nor_num; i++)
+ mtd_device_unregister(&q->mtd[i]);
+
+irq_failed:
+ clk_disable_unprepare(q->clk);
+ clk_disable_unprepare(q->clk_en);
+map_failed:
+ dev_err(dev, "Freescale QuadSPI probe failed\n");
+ return ret;
+}
+
+static int fsl_qspi_remove(struct platform_device *pdev)
+{
+ struct fsl_qspi *q = platform_get_drvdata(pdev);
+ int i;
+
+ for (i = 0; i < q->nor_num; i++)
+ mtd_device_unregister(&q->mtd[i]);
+
+ /* disable the hardware */
+ writel(QUADSPI_MCR_MDIS_MASK, q->iobase + QUADSPI_MCR);
+ writel(0x0, q->iobase + QUADSPI_RSER);
+
+ clk_unprepare(q->clk);
+ clk_unprepare(q->clk_en);
+ return 0;
+}
+
+static struct platform_driver fsl_qspi_driver = {
+ .driver = {
+ .name = "fsl-quadspi",
+ .bus = &platform_bus_type,
+ .owner = THIS_MODULE,
+ .of_match_table = fsl_qspi_dt_ids,
+ },
+ .probe = fsl_qspi_probe,
+ .remove = fsl_qspi_remove,
+};
+module_platform_driver(fsl_qspi_driver);
+
+MODULE_DESCRIPTION("Freescale QuadSPI Controller Driver");
+MODULE_AUTHOR("Freescale Semiconductor Inc.");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/mtd/spi-nor/spi-nor.c b/drivers/mtd/spi-nor/spi-nor.c
new file mode 100644
index 0000000..c713c86
--- /dev/null
+++ b/drivers/mtd/spi-nor/spi-nor.c
@@ -0,0 +1,1107 @@
+/*
+ * Based on m25p80.c, by Mike Lavender (mike@steroidmicros.com), with
+ * influence from lart.c (Abraham Van Der Merwe) and mtd_dataflash.c
+ *
+ * Copyright (C) 2005, Intec Automation Inc.
+ * Copyright (C) 2014, Freescale Semiconductor, Inc.
+ *
+ * This code is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/err.h>
+#include <linux/errno.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/mutex.h>
+#include <linux/math64.h>
+
+#include <linux/mtd/cfi.h>
+#include <linux/mtd/mtd.h>
+#include <linux/of_platform.h>
+#include <linux/spi/flash.h>
+#include <linux/mtd/spi-nor.h>
+
+/* Define max times to check status register before we give up. */
+#define MAX_READY_WAIT_JIFFIES (40 * HZ) /* M25P16 specs 40s max chip erase */
+
+#define JEDEC_MFR(_jedec_id) ((_jedec_id) >> 16)
+
+/*
+ * Read the status register, returning its value in the location
+ * Return the status register value.
+ * Returns negative if error occurred.
+ */
+static int read_sr(struct spi_nor *nor)
+{
+ int ret;
+ u8 val;
+
+ ret = nor->read_reg(nor, SPINOR_OP_RDSR, &val, 1);
+ if (ret < 0) {
+ pr_err("error %d reading SR\n", (int) ret);
+ return ret;
+ }
+
+ return val;
+}
+
+/*
+ * Read configuration register, returning its value in the
+ * location. Return the configuration register value.
+ * Returns negative if error occured.
+ */
+static int read_cr(struct spi_nor *nor)
+{
+ int ret;
+ u8 val;
+
+ ret = nor->read_reg(nor, SPINOR_OP_RDCR, &val, 1);
+ if (ret < 0) {
+ dev_err(nor->dev, "error %d reading CR\n", ret);
+ return ret;
+ }
+
+ return val;
+}
+
+/*
+ * Dummy Cycle calculation for different type of read.
+ * It can be used to support more commands with
+ * different dummy cycle requirements.
+ */
+static inline int spi_nor_read_dummy_cycles(struct spi_nor *nor)
+{
+ switch (nor->flash_read) {
+ case SPI_NOR_FAST:
+ case SPI_NOR_DUAL:
+ case SPI_NOR_QUAD:
+ return 1;
+ case SPI_NOR_NORMAL:
+ return 0;
+ }
+ return 0;
+}
+
+/*
+ * Write status register 1 byte
+ * Returns negative if error occurred.
+ */
+static inline int write_sr(struct spi_nor *nor, u8 val)
+{
+ nor->cmd_buf[0] = val;
+ return nor->write_reg(nor, SPINOR_OP_WRSR, nor->cmd_buf, 1, 0);
+}
+
+/*
+ * Set write enable latch with Write Enable command.
+ * Returns negative if error occurred.
+ */
+static inline int write_enable(struct spi_nor *nor)
+{
+ return nor->write_reg(nor, SPINOR_OP_WREN, NULL, 0, 0);
+}
+
+/*
+ * Send write disble instruction to the chip.
+ */
+static inline int write_disable(struct spi_nor *nor)
+{
+ return nor->write_reg(nor, SPINOR_OP_WRDI, NULL, 0, 0);
+}
+
+static inline struct spi_nor *mtd_to_spi_nor(struct mtd_info *mtd)
+{
+ return mtd->priv;
+}
+
+/* Enable/disable 4-byte addressing mode. */
+static inline int set_4byte(struct spi_nor *nor, u32 jedec_id, int enable)
+{
+ int status;
+ bool need_wren = false;
+ u8 cmd;
+
+ switch (JEDEC_MFR(jedec_id)) {
+ case CFI_MFR_ST: /* Micron, actually */
+ /* Some Micron need WREN command; all will accept it */
+ need_wren = true;
+ case CFI_MFR_MACRONIX:
+ case 0xEF /* winbond */:
+ if (need_wren)
+ write_enable(nor);
+
+ cmd = enable ? SPINOR_OP_EN4B : SPINOR_OP_EX4B;
+ status = nor->write_reg(nor, cmd, NULL, 0, 0);
+ if (need_wren)
+ write_disable(nor);
+
+ return status;
+ default:
+ /* Spansion style */
+ nor->cmd_buf[0] = enable << 7;
+ return nor->write_reg(nor, SPINOR_OP_BRWR, nor->cmd_buf, 1, 0);
+ }
+}
+
+static int spi_nor_wait_till_ready(struct spi_nor *nor)
+{
+ unsigned long deadline;
+ int sr;
+
+ deadline = jiffies + MAX_READY_WAIT_JIFFIES;
+
+ do {
+ cond_resched();
+
+ sr = read_sr(nor);
+ if (sr < 0)
+ break;
+ else if (!(sr & SR_WIP))
+ return 0;
+ } while (!time_after_eq(jiffies, deadline));
+
+ return -ETIMEDOUT;
+}
+
+/*
+ * Service routine to read status register until ready, or timeout occurs.
+ * Returns non-zero if error.
+ */
+static int wait_till_ready(struct spi_nor *nor)
+{
+ return nor->wait_till_ready(nor);
+}
+
+/*
+ * Erase the whole flash memory
+ *
+ * Returns 0 if successful, non-zero otherwise.
+ */
+static int erase_chip(struct spi_nor *nor)
+{
+ int ret;
+
+ dev_dbg(nor->dev, " %lldKiB\n", (long long)(nor->mtd->size >> 10));
+
+ /* Wait until finished previous write command. */
+ ret = wait_till_ready(nor);
+ if (ret)
+ return ret;
+
+ /* Send write enable, then erase commands. */
+ write_enable(nor);
+
+ return nor->write_reg(nor, SPINOR_OP_CHIP_ERASE, NULL, 0, 0);
+}
+
+static int spi_nor_lock_and_prep(struct spi_nor *nor, enum spi_nor_ops ops)
+{
+ int ret = 0;
+
+ mutex_lock(&nor->lock);
+
+ if (nor->prepare) {
+ ret = nor->prepare(nor, ops);
+ if (ret) {
+ dev_err(nor->dev, "failed in the preparation.\n");
+ mutex_unlock(&nor->lock);
+ return ret;
+ }
+ }
+ return ret;
+}
+
+static void spi_nor_unlock_and_unprep(struct spi_nor *nor, enum spi_nor_ops ops)
+{
+ if (nor->unprepare)
+ nor->unprepare(nor, ops);
+ mutex_unlock(&nor->lock);
+}
+
+/*
+ * Erase an address range on the nor chip. The address range may extend
+ * one or more erase sectors. Return an error is there is a problem erasing.
+ */
+static int spi_nor_erase(struct mtd_info *mtd, struct erase_info *instr)
+{
+ struct spi_nor *nor = mtd_to_spi_nor(mtd);
+ u32 addr, len;
+ uint32_t rem;
+ int ret;
+
+ dev_dbg(nor->dev, "at 0x%llx, len %lld\n", (long long)instr->addr,
+ (long long)instr->len);
+
+ div_u64_rem(instr->len, mtd->erasesize, &rem);
+ if (rem)
+ return -EINVAL;
+
+ addr = instr->addr;
+ len = instr->len;
+
+ ret = spi_nor_lock_and_prep(nor, SPI_NOR_OPS_ERASE);
+ if (ret)
+ return ret;
+
+ /* whole-chip erase? */
+ if (len == mtd->size) {
+ if (erase_chip(nor)) {
+ ret = -EIO;
+ goto erase_err;
+ }
+
+ /* REVISIT in some cases we could speed up erasing large regions
+ * by using SPINOR_OP_SE instead of SPINOR_OP_BE_4K. We may have set up
+ * to use "small sector erase", but that's not always optimal.
+ */
+
+ /* "sector"-at-a-time erase */
+ } else {
+ while (len) {
+ if (nor->erase(nor, addr)) {
+ ret = -EIO;
+ goto erase_err;
+ }
+
+ addr += mtd->erasesize;
+ len -= mtd->erasesize;
+ }
+ }
+
+ spi_nor_unlock_and_unprep(nor, SPI_NOR_OPS_ERASE);
+
+ instr->state = MTD_ERASE_DONE;
+ mtd_erase_callback(instr);
+
+ return ret;
+
+erase_err:
+ spi_nor_unlock_and_unprep(nor, SPI_NOR_OPS_ERASE);
+ instr->state = MTD_ERASE_FAILED;
+ return ret;
+}
+
+static int spi_nor_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
+{
+ struct spi_nor *nor = mtd_to_spi_nor(mtd);
+ uint32_t offset = ofs;
+ uint8_t status_old, status_new;
+ int ret = 0;
+
+ ret = spi_nor_lock_and_prep(nor, SPI_NOR_OPS_LOCK);
+ if (ret)
+ return ret;
+
+ /* Wait until finished previous command */
+ ret = wait_till_ready(nor);
+ if (ret)
+ goto err;
+
+ status_old = read_sr(nor);
+
+ if (offset < mtd->size - (mtd->size / 2))
+ status_new = status_old | SR_BP2 | SR_BP1 | SR_BP0;
+ else if (offset < mtd->size - (mtd->size / 4))
+ status_new = (status_old & ~SR_BP0) | SR_BP2 | SR_BP1;
+ else if (offset < mtd->size - (mtd->size / 8))
+ status_new = (status_old & ~SR_BP1) | SR_BP2 | SR_BP0;
+ else if (offset < mtd->size - (mtd->size / 16))
+ status_new = (status_old & ~(SR_BP0 | SR_BP1)) | SR_BP2;
+ else if (offset < mtd->size - (mtd->size / 32))
+ status_new = (status_old & ~SR_BP2) | SR_BP1 | SR_BP0;
+ else if (offset < mtd->size - (mtd->size / 64))
+ status_new = (status_old & ~(SR_BP2 | SR_BP0)) | SR_BP1;
+ else
+ status_new = (status_old & ~(SR_BP2 | SR_BP1)) | SR_BP0;
+
+ /* Only modify protection if it will not unlock other areas */
+ if ((status_new & (SR_BP2 | SR_BP1 | SR_BP0)) >
+ (status_old & (SR_BP2 | SR_BP1 | SR_BP0))) {
+ write_enable(nor);
+ ret = write_sr(nor, status_new);
+ if (ret)
+ goto err;
+ }
+
+err:
+ spi_nor_unlock_and_unprep(nor, SPI_NOR_OPS_LOCK);
+ return ret;
+}
+
+static int spi_nor_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
+{
+ struct spi_nor *nor = mtd_to_spi_nor(mtd);
+ uint32_t offset = ofs;
+ uint8_t status_old, status_new;
+ int ret = 0;
+
+ ret = spi_nor_lock_and_prep(nor, SPI_NOR_OPS_UNLOCK);
+ if (ret)
+ return ret;
+
+ /* Wait until finished previous command */
+ ret = wait_till_ready(nor);
+ if (ret)
+ goto err;
+
+ status_old = read_sr(nor);
+
+ if (offset+len > mtd->size - (mtd->size / 64))
+ status_new = status_old & ~(SR_BP2 | SR_BP1 | SR_BP0);
+ else if (offset+len > mtd->size - (mtd->size / 32))
+ status_new = (status_old & ~(SR_BP2 | SR_BP1)) | SR_BP0;
+ else if (offset+len > mtd->size - (mtd->size / 16))
+ status_new = (status_old & ~(SR_BP2 | SR_BP0)) | SR_BP1;
+ else if (offset+len > mtd->size - (mtd->size / 8))
+ status_new = (status_old & ~SR_BP2) | SR_BP1 | SR_BP0;
+ else if (offset+len > mtd->size - (mtd->size / 4))
+ status_new = (status_old & ~(SR_BP0 | SR_BP1)) | SR_BP2;
+ else if (offset+len > mtd->size - (mtd->size / 2))
+ status_new = (status_old & ~SR_BP1) | SR_BP2 | SR_BP0;
+ else
+ status_new = (status_old & ~SR_BP0) | SR_BP2 | SR_BP1;
+
+ /* Only modify protection if it will not lock other areas */
+ if ((status_new & (SR_BP2 | SR_BP1 | SR_BP0)) <
+ (status_old & (SR_BP2 | SR_BP1 | SR_BP0))) {
+ write_enable(nor);
+ ret = write_sr(nor, status_new);
+ if (ret)
+ goto err;
+ }
+
+err:
+ spi_nor_unlock_and_unprep(nor, SPI_NOR_OPS_UNLOCK);
+ return ret;
+}
+
+struct flash_info {
+ /* JEDEC id zero means "no ID" (most older chips); otherwise it has
+ * a high byte of zero plus three data bytes: the manufacturer id,
+ * then a two byte device id.
+ */
+ u32 jedec_id;
+ u16 ext_id;
+
+ /* The size listed here is what works with SPINOR_OP_SE, which isn't
+ * necessarily called a "sector" by the vendor.
+ */
+ unsigned sector_size;
+ u16 n_sectors;
+
+ u16 page_size;
+ u16 addr_width;
+
+ u16 flags;
+#define SECT_4K 0x01 /* SPINOR_OP_BE_4K works uniformly */
+#define SPI_NOR_NO_ERASE 0x02 /* No erase command needed */
+#define SST_WRITE 0x04 /* use SST byte programming */
+#define SPI_NOR_NO_FR 0x08 /* Can't do fastread */
+#define SECT_4K_PMC 0x10 /* SPINOR_OP_BE_4K_PMC works uniformly */
+#define SPI_NOR_DUAL_READ 0x20 /* Flash supports Dual Read */
+#define SPI_NOR_QUAD_READ 0x40 /* Flash supports Quad Read */
+};
+
+#define INFO(_jedec_id, _ext_id, _sector_size, _n_sectors, _flags) \
+ ((kernel_ulong_t)&(struct flash_info) { \
+ .jedec_id = (_jedec_id), \
+ .ext_id = (_ext_id), \
+ .sector_size = (_sector_size), \
+ .n_sectors = (_n_sectors), \
+ .page_size = 256, \
+ .flags = (_flags), \
+ })
+
+#define CAT25_INFO(_sector_size, _n_sectors, _page_size, _addr_width, _flags) \
+ ((kernel_ulong_t)&(struct flash_info) { \
+ .sector_size = (_sector_size), \
+ .n_sectors = (_n_sectors), \
+ .page_size = (_page_size), \
+ .addr_width = (_addr_width), \
+ .flags = (_flags), \
+ })
+
+/* NOTE: double check command sets and memory organization when you add
+ * more nor chips. This current list focusses on newer chips, which
+ * have been converging on command sets which including JEDEC ID.
+ */
+const struct spi_device_id spi_nor_ids[] = {
+ /* Atmel -- some are (confusingly) marketed as "DataFlash" */
+ { "at25fs010", INFO(0x1f6601, 0, 32 * 1024, 4, SECT_4K) },
+ { "at25fs040", INFO(0x1f6604, 0, 64 * 1024, 8, SECT_4K) },
+
+ { "at25df041a", INFO(0x1f4401, 0, 64 * 1024, 8, SECT_4K) },
+ { "at25df321a", INFO(0x1f4701, 0, 64 * 1024, 64, SECT_4K) },
+ { "at25df641", INFO(0x1f4800, 0, 64 * 1024, 128, SECT_4K) },
+
+ { "at26f004", INFO(0x1f0400, 0, 64 * 1024, 8, SECT_4K) },
+ { "at26df081a", INFO(0x1f4501, 0, 64 * 1024, 16, SECT_4K) },
+ { "at26df161a", INFO(0x1f4601, 0, 64 * 1024, 32, SECT_4K) },
+ { "at26df321", INFO(0x1f4700, 0, 64 * 1024, 64, SECT_4K) },
+
+ { "at45db081d", INFO(0x1f2500, 0, 64 * 1024, 16, SECT_4K) },
+
+ /* EON -- en25xxx */
+ { "en25f32", INFO(0x1c3116, 0, 64 * 1024, 64, SECT_4K) },
+ { "en25p32", INFO(0x1c2016, 0, 64 * 1024, 64, 0) },
+ { "en25q32b", INFO(0x1c3016, 0, 64 * 1024, 64, 0) },
+ { "en25p64", INFO(0x1c2017, 0, 64 * 1024, 128, 0) },
+ { "en25q64", INFO(0x1c3017, 0, 64 * 1024, 128, SECT_4K) },
+ { "en25qh256", INFO(0x1c7019, 0, 64 * 1024, 512, 0) },
+
+ /* ESMT */
+ { "f25l32pa", INFO(0x8c2016, 0, 64 * 1024, 64, SECT_4K) },
+
+ /* Everspin */
+ { "mr25h256", CAT25_INFO( 32 * 1024, 1, 256, 2, SPI_NOR_NO_ERASE | SPI_NOR_NO_FR) },
+ { "mr25h10", CAT25_INFO(128 * 1024, 1, 256, 3, SPI_NOR_NO_ERASE | SPI_NOR_NO_FR) },
+
+ /* GigaDevice */
+ { "gd25q32", INFO(0xc84016, 0, 64 * 1024, 64, SECT_4K) },
+ { "gd25q64", INFO(0xc84017, 0, 64 * 1024, 128, SECT_4K) },
+
+ /* Intel/Numonyx -- xxxs33b */
+ { "160s33b", INFO(0x898911, 0, 64 * 1024, 32, 0) },
+ { "320s33b", INFO(0x898912, 0, 64 * 1024, 64, 0) },
+ { "640s33b", INFO(0x898913, 0, 64 * 1024, 128, 0) },
+
+ /* Macronix */
+ { "mx25l2005a", INFO(0xc22012, 0, 64 * 1024, 4, SECT_4K) },
+ { "mx25l4005a", INFO(0xc22013, 0, 64 * 1024, 8, SECT_4K) },
+ { "mx25l8005", INFO(0xc22014, 0, 64 * 1024, 16, 0) },
+ { "mx25l1606e", INFO(0xc22015, 0, 64 * 1024, 32, SECT_4K) },
+ { "mx25l3205d", INFO(0xc22016, 0, 64 * 1024, 64, 0) },
+ { "mx25l3255e", INFO(0xc29e16, 0, 64 * 1024, 64, SECT_4K) },
+ { "mx25l6405d", INFO(0xc22017, 0, 64 * 1024, 128, 0) },
+ { "mx25l12805d", INFO(0xc22018, 0, 64 * 1024, 256, 0) },
+ { "mx25l12855e", INFO(0xc22618, 0, 64 * 1024, 256, 0) },
+ { "mx25l25635e", INFO(0xc22019, 0, 64 * 1024, 512, 0) },
+ { "mx25l25655e", INFO(0xc22619, 0, 64 * 1024, 512, 0) },
+ { "mx66l51235l", INFO(0xc2201a, 0, 64 * 1024, 1024, SPI_NOR_QUAD_READ) },
+ { "mx66l1g55g", INFO(0xc2261b, 0, 64 * 1024, 2048, SPI_NOR_QUAD_READ) },
+
+ /* Micron */
+ { "n25q064", INFO(0x20ba17, 0, 64 * 1024, 128, 0) },
+ { "n25q128a11", INFO(0x20bb18, 0, 64 * 1024, 256, 0) },
+ { "n25q128a13", INFO(0x20ba18, 0, 64 * 1024, 256, 0) },
+ { "n25q256a", INFO(0x20ba19, 0, 64 * 1024, 512, SECT_4K) },
+ { "n25q512a", INFO(0x20bb20, 0, 64 * 1024, 1024, SECT_4K) },
+
+ /* PMC */
+ { "pm25lv512", INFO(0, 0, 32 * 1024, 2, SECT_4K_PMC) },
+ { "pm25lv010", INFO(0, 0, 32 * 1024, 4, SECT_4K_PMC) },
+ { "pm25lq032", INFO(0x7f9d46, 0, 64 * 1024, 64, SECT_4K) },
+
+ /* Spansion -- single (large) sector size only, at least
+ * for the chips listed here (without boot sectors).
+ */
+ { "s25sl032p", INFO(0x010215, 0x4d00, 64 * 1024, 64, SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ) },
+ { "s25sl064p", INFO(0x010216, 0x4d00, 64 * 1024, 128, 0) },
+ { "s25fl256s0", INFO(0x010219, 0x4d00, 256 * 1024, 128, 0) },
+ { "s25fl256s1", INFO(0x010219, 0x4d01, 64 * 1024, 512, SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ) },
+ { "s25fl512s", INFO(0x010220, 0x4d00, 256 * 1024, 256, SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ) },
+ { "s70fl01gs", INFO(0x010221, 0x4d00, 256 * 1024, 256, 0) },
+ { "s25sl12800", INFO(0x012018, 0x0300, 256 * 1024, 64, 0) },
+ { "s25sl12801", INFO(0x012018, 0x0301, 64 * 1024, 256, 0) },
+ { "s25fl129p0", INFO(0x012018, 0x4d00, 256 * 1024, 64, 0) },
+ { "s25fl129p1", INFO(0x012018, 0x4d01, 64 * 1024, 256, 0) },
+ { "s25sl004a", INFO(0x010212, 0, 64 * 1024, 8, 0) },
+ { "s25sl008a", INFO(0x010213, 0, 64 * 1024, 16, 0) },
+ { "s25sl016a", INFO(0x010214, 0, 64 * 1024, 32, 0) },
+ { "s25sl032a", INFO(0x010215, 0, 64 * 1024, 64, 0) },
+ { "s25sl064a", INFO(0x010216, 0, 64 * 1024, 128, 0) },
+ { "s25fl008k", INFO(0xef4014, 0, 64 * 1024, 16, SECT_4K) },
+ { "s25fl016k", INFO(0xef4015, 0, 64 * 1024, 32, SECT_4K) },
+ { "s25fl064k", INFO(0xef4017, 0, 64 * 1024, 128, SECT_4K) },
+
+ /* SST -- large erase sizes are "overlays", "sectors" are 4K */
+ { "sst25vf040b", INFO(0xbf258d, 0, 64 * 1024, 8, SECT_4K | SST_WRITE) },
+ { "sst25vf080b", INFO(0xbf258e, 0, 64 * 1024, 16, SECT_4K | SST_WRITE) },
+ { "sst25vf016b", INFO(0xbf2541, 0, 64 * 1024, 32, SECT_4K | SST_WRITE) },
+ { "sst25vf032b", INFO(0xbf254a, 0, 64 * 1024, 64, SECT_4K | SST_WRITE) },
+ { "sst25vf064c", INFO(0xbf254b, 0, 64 * 1024, 128, SECT_4K) },
+ { "sst25wf512", INFO(0xbf2501, 0, 64 * 1024, 1, SECT_4K | SST_WRITE) },
+ { "sst25wf010", INFO(0xbf2502, 0, 64 * 1024, 2, SECT_4K | SST_WRITE) },
+ { "sst25wf020", INFO(0xbf2503, 0, 64 * 1024, 4, SECT_4K | SST_WRITE) },
+ { "sst25wf040", INFO(0xbf2504, 0, 64 * 1024, 8, SECT_4K | SST_WRITE) },
+
+ /* ST Microelectronics -- newer production may have feature updates */
+ { "m25p05", INFO(0x202010, 0, 32 * 1024, 2, 0) },
+ { "m25p10", INFO(0x202011, 0, 32 * 1024, 4, 0) },
+ { "m25p20", INFO(0x202012, 0, 64 * 1024, 4, 0) },
+ { "m25p40", INFO(0x202013, 0, 64 * 1024, 8, 0) },
+ { "m25p80", INFO(0x202014, 0, 64 * 1024, 16, 0) },
+ { "m25p16", INFO(0x202015, 0, 64 * 1024, 32, 0) },
+ { "m25p32", INFO(0x202016, 0, 64 * 1024, 64, 0) },
+ { "m25p64", INFO(0x202017, 0, 64 * 1024, 128, 0) },
+ { "m25p128", INFO(0x202018, 0, 256 * 1024, 64, 0) },
+ { "n25q032", INFO(0x20ba16, 0, 64 * 1024, 64, 0) },
+
+ { "m25p05-nonjedec", INFO(0, 0, 32 * 1024, 2, 0) },
+ { "m25p10-nonjedec", INFO(0, 0, 32 * 1024, 4, 0) },
+ { "m25p20-nonjedec", INFO(0, 0, 64 * 1024, 4, 0) },
+ { "m25p40-nonjedec", INFO(0, 0, 64 * 1024, 8, 0) },
+ { "m25p80-nonjedec", INFO(0, 0, 64 * 1024, 16, 0) },
+ { "m25p16-nonjedec", INFO(0, 0, 64 * 1024, 32, 0) },
+ { "m25p32-nonjedec", INFO(0, 0, 64 * 1024, 64, 0) },
+ { "m25p64-nonjedec", INFO(0, 0, 64 * 1024, 128, 0) },
+ { "m25p128-nonjedec", INFO(0, 0, 256 * 1024, 64, 0) },
+
+ { "m45pe10", INFO(0x204011, 0, 64 * 1024, 2, 0) },
+ { "m45pe80", INFO(0x204014, 0, 64 * 1024, 16, 0) },
+ { "m45pe16", INFO(0x204015, 0, 64 * 1024, 32, 0) },
+
+ { "m25pe20", INFO(0x208012, 0, 64 * 1024, 4, 0) },
+ { "m25pe80", INFO(0x208014, 0, 64 * 1024, 16, 0) },
+ { "m25pe16", INFO(0x208015, 0, 64 * 1024, 32, SECT_4K) },
+
+ { "m25px16", INFO(0x207115, 0, 64 * 1024, 32, SECT_4K) },
+ { "m25px32", INFO(0x207116, 0, 64 * 1024, 64, SECT_4K) },
+ { "m25px32-s0", INFO(0x207316, 0, 64 * 1024, 64, SECT_4K) },
+ { "m25px32-s1", INFO(0x206316, 0, 64 * 1024, 64, SECT_4K) },
+ { "m25px64", INFO(0x207117, 0, 64 * 1024, 128, 0) },
+
+ /* Winbond -- w25x "blocks" are 64K, "sectors" are 4KiB */
+ { "w25x10", INFO(0xef3011, 0, 64 * 1024, 2, SECT_4K) },
+ { "w25x20", INFO(0xef3012, 0, 64 * 1024, 4, SECT_4K) },
+ { "w25x40", INFO(0xef3013, 0, 64 * 1024, 8, SECT_4K) },
+ { "w25x80", INFO(0xef3014, 0, 64 * 1024, 16, SECT_4K) },
+ { "w25x16", INFO(0xef3015, 0, 64 * 1024, 32, SECT_4K) },
+ { "w25x32", INFO(0xef3016, 0, 64 * 1024, 64, SECT_4K) },
+ { "w25q32", INFO(0xef4016, 0, 64 * 1024, 64, SECT_4K) },
+ { "w25q32dw", INFO(0xef6016, 0, 64 * 1024, 64, SECT_4K) },
+ { "w25x64", INFO(0xef3017, 0, 64 * 1024, 128, SECT_4K) },
+ { "w25q64", INFO(0xef4017, 0, 64 * 1024, 128, SECT_4K) },
+ { "w25q128", INFO(0xef4018, 0, 64 * 1024, 256, SECT_4K) },
+ { "w25q80", INFO(0xef5014, 0, 64 * 1024, 16, SECT_4K) },
+ { "w25q80bl", INFO(0xef4014, 0, 64 * 1024, 16, SECT_4K) },
+ { "w25q128", INFO(0xef4018, 0, 64 * 1024, 256, SECT_4K) },
+ { "w25q256", INFO(0xef4019, 0, 64 * 1024, 512, SECT_4K) },
+
+ /* Catalyst / On Semiconductor -- non-JEDEC */
+ { "cat25c11", CAT25_INFO( 16, 8, 16, 1, SPI_NOR_NO_ERASE | SPI_NOR_NO_FR) },
+ { "cat25c03", CAT25_INFO( 32, 8, 16, 2, SPI_NOR_NO_ERASE | SPI_NOR_NO_FR) },
+ { "cat25c09", CAT25_INFO( 128, 8, 32, 2, SPI_NOR_NO_ERASE | SPI_NOR_NO_FR) },
+ { "cat25c17", CAT25_INFO( 256, 8, 32, 2, SPI_NOR_NO_ERASE | SPI_NOR_NO_FR) },
+ { "cat25128", CAT25_INFO(2048, 8, 64, 2, SPI_NOR_NO_ERASE | SPI_NOR_NO_FR) },
+ { },
+};
+EXPORT_SYMBOL_GPL(spi_nor_ids);
+
+static const struct spi_device_id *spi_nor_read_id(struct spi_nor *nor)
+{
+ int tmp;
+ u8 id[5];
+ u32 jedec;
+ u16 ext_jedec;
+ struct flash_info *info;
+
+ tmp = nor->read_reg(nor, SPINOR_OP_RDID, id, 5);
+ if (tmp < 0) {
+ dev_dbg(nor->dev, " error %d reading JEDEC ID\n", tmp);
+ return ERR_PTR(tmp);
+ }
+ jedec = id[0];
+ jedec = jedec << 8;
+ jedec |= id[1];
+ jedec = jedec << 8;
+ jedec |= id[2];
+
+ ext_jedec = id[3] << 8 | id[4];
+
+ for (tmp = 0; tmp < ARRAY_SIZE(spi_nor_ids) - 1; tmp++) {
+ info = (void *)spi_nor_ids[tmp].driver_data;
+ if (info->jedec_id == jedec) {
+ if (info->ext_id == 0 || info->ext_id == ext_jedec)
+ return &spi_nor_ids[tmp];
+ }
+ }
+ dev_err(nor->dev, "unrecognized JEDEC id %06x\n", jedec);
+ return ERR_PTR(-ENODEV);
+}
+
+static const struct spi_device_id *jedec_probe(struct spi_nor *nor)
+{
+ return nor->read_id(nor);
+}
+
+static int spi_nor_read(struct mtd_info *mtd, loff_t from, size_t len,
+ size_t *retlen, u_char *buf)
+{
+ struct spi_nor *nor = mtd_to_spi_nor(mtd);
+ int ret;
+
+ dev_dbg(nor->dev, "from 0x%08x, len %zd\n", (u32)from, len);
+
+ ret = spi_nor_lock_and_prep(nor, SPI_NOR_OPS_READ);
+ if (ret)
+ return ret;
+
+ ret = nor->read(nor, from, len, retlen, buf);
+
+ spi_nor_unlock_and_unprep(nor, SPI_NOR_OPS_READ);
+ return ret;
+}
+
+static int sst_write(struct mtd_info *mtd, loff_t to, size_t len,
+ size_t *retlen, const u_char *buf)
+{
+ struct spi_nor *nor = mtd_to_spi_nor(mtd);
+ size_t actual;
+ int ret;
+
+ dev_dbg(nor->dev, "to 0x%08x, len %zd\n", (u32)to, len);
+
+ ret = spi_nor_lock_and_prep(nor, SPI_NOR_OPS_WRITE);
+ if (ret)
+ return ret;
+
+ /* Wait until finished previous write command. */
+ ret = wait_till_ready(nor);
+ if (ret)
+ goto time_out;
+
+ write_enable(nor);
+
+ nor->sst_write_second = false;
+
+ actual = to % 2;
+ /* Start write from odd address. */
+ if (actual) {
+ nor->program_opcode = SPINOR_OP_BP;
+
+ /* write one byte. */
+ nor->write(nor, to, 1, retlen, buf);
+ ret = wait_till_ready(nor);
+ if (ret)
+ goto time_out;
+ }
+ to += actual;
+
+ /* Write out most of the data here. */
+ for (; actual < len - 1; actual += 2) {
+ nor->program_opcode = SPINOR_OP_AAI_WP;
+
+ /* write two bytes. */
+ nor->write(nor, to, 2, retlen, buf + actual);
+ ret = wait_till_ready(nor);
+ if (ret)
+ goto time_out;
+ to += 2;
+ nor->sst_write_second = true;
+ }
+ nor->sst_write_second = false;
+
+ write_disable(nor);
+ ret = wait_till_ready(nor);
+ if (ret)
+ goto time_out;
+
+ /* Write out trailing byte if it exists. */
+ if (actual != len) {
+ write_enable(nor);
+
+ nor->program_opcode = SPINOR_OP_BP;
+ nor->write(nor, to, 1, retlen, buf + actual);
+
+ ret = wait_till_ready(nor);
+ if (ret)
+ goto time_out;
+ write_disable(nor);
+ }
+time_out:
+ spi_nor_unlock_and_unprep(nor, SPI_NOR_OPS_WRITE);
+ return ret;
+}
+
+/*
+ * Write an address range to the nor chip. Data must be written in
+ * FLASH_PAGESIZE chunks. The address range may be any size provided
+ * it is within the physical boundaries.
+ */
+static int spi_nor_write(struct mtd_info *mtd, loff_t to, size_t len,
+ size_t *retlen, const u_char *buf)
+{
+ struct spi_nor *nor = mtd_to_spi_nor(mtd);
+ u32 page_offset, page_size, i;
+ int ret;
+
+ dev_dbg(nor->dev, "to 0x%08x, len %zd\n", (u32)to, len);
+
+ ret = spi_nor_lock_and_prep(nor, SPI_NOR_OPS_WRITE);
+ if (ret)
+ return ret;
+
+ /* Wait until finished previous write command. */
+ ret = wait_till_ready(nor);
+ if (ret)
+ goto write_err;
+
+ write_enable(nor);
+
+ page_offset = to & (nor->page_size - 1);
+
+ /* do all the bytes fit onto one page? */
+ if (page_offset + len <= nor->page_size) {
+ nor->write(nor, to, len, retlen, buf);
+ } else {
+ /* the size of data remaining on the first page */
+ page_size = nor->page_size - page_offset;
+ nor->write(nor, to, page_size, retlen, buf);
+
+ /* write everything in nor->page_size chunks */
+ for (i = page_size; i < len; i += page_size) {
+ page_size = len - i;
+ if (page_size > nor->page_size)
+ page_size = nor->page_size;
+
+ wait_till_ready(nor);
+ write_enable(nor);
+
+ nor->write(nor, to + i, page_size, retlen, buf + i);
+ }
+ }
+
+write_err:
+ spi_nor_unlock_and_unprep(nor, SPI_NOR_OPS_WRITE);
+ return 0;
+}
+
+static int macronix_quad_enable(struct spi_nor *nor)
+{
+ int ret, val;
+
+ val = read_sr(nor);
+ write_enable(nor);
+
+ nor->cmd_buf[0] = val | SR_QUAD_EN_MX;
+ nor->write_reg(nor, SPINOR_OP_WRSR, nor->cmd_buf, 1, 0);
+
+ if (wait_till_ready(nor))
+ return 1;
+
+ ret = read_sr(nor);
+ if (!(ret > 0 && (ret & SR_QUAD_EN_MX))) {
+ dev_err(nor->dev, "Macronix Quad bit not set\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/*
+ * Write status Register and configuration register with 2 bytes
+ * The first byte will be written to the status register, while the
+ * second byte will be written to the configuration register.
+ * Return negative if error occured.
+ */
+static int write_sr_cr(struct spi_nor *nor, u16 val)
+{
+ nor->cmd_buf[0] = val & 0xff;
+ nor->cmd_buf[1] = (val >> 8);
+
+ return nor->write_reg(nor, SPINOR_OP_WRSR, nor->cmd_buf, 2, 0);
+}
+
+static int spansion_quad_enable(struct spi_nor *nor)
+{
+ int ret;
+ int quad_en = CR_QUAD_EN_SPAN << 8;
+
+ write_enable(nor);
+
+ ret = write_sr_cr(nor, quad_en);
+ if (ret < 0) {
+ dev_err(nor->dev,
+ "error while writing configuration register\n");
+ return -EINVAL;
+ }
+
+ /* read back and check it */
+ ret = read_cr(nor);
+ if (!(ret > 0 && (ret & CR_QUAD_EN_SPAN))) {
+ dev_err(nor->dev, "Spansion Quad bit not set\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int set_quad_mode(struct spi_nor *nor, u32 jedec_id)
+{
+ int status;
+
+ switch (JEDEC_MFR(jedec_id)) {
+ case CFI_MFR_MACRONIX:
+ status = macronix_quad_enable(nor);
+ if (status) {
+ dev_err(nor->dev, "Macronix quad-read not enabled\n");
+ return -EINVAL;
+ }
+ return status;
+ default:
+ status = spansion_quad_enable(nor);
+ if (status) {
+ dev_err(nor->dev, "Spansion quad-read not enabled\n");
+ return -EINVAL;
+ }
+ return status;
+ }
+}
+
+static int spi_nor_check(struct spi_nor *nor)
+{
+ if (!nor->dev || !nor->read || !nor->write ||
+ !nor->read_reg || !nor->write_reg || !nor->erase) {
+ pr_err("spi-nor: please fill all the necessary fields!\n");
+ return -EINVAL;
+ }
+
+ if (!nor->read_id)
+ nor->read_id = spi_nor_read_id;
+ if (!nor->wait_till_ready)
+ nor->wait_till_ready = spi_nor_wait_till_ready;
+
+ return 0;
+}
+
+int spi_nor_scan(struct spi_nor *nor, const struct spi_device_id *id,
+ enum read_mode mode)
+{
+ struct flash_info *info;
+ struct flash_platform_data *data;
+ struct device *dev = nor->dev;
+ struct mtd_info *mtd = nor->mtd;
+ struct device_node *np = dev->of_node;
+ int ret;
+ int i;
+
+ ret = spi_nor_check(nor);
+ if (ret)
+ return ret;
+
+ /* Platform data helps sort out which chip type we have, as
+ * well as how this board partitions it. If we don't have
+ * a chip ID, try the JEDEC id commands; they'll work for most
+ * newer chips, even if we don't recognize the particular chip.
+ */
+ data = dev_get_platdata(dev);
+ if (data && data->type) {
+ const struct spi_device_id *plat_id;
+
+ for (i = 0; i < ARRAY_SIZE(spi_nor_ids) - 1; i++) {
+ plat_id = &spi_nor_ids[i];
+ if (strcmp(data->type, plat_id->name))
+ continue;
+ break;
+ }
+
+ if (i < ARRAY_SIZE(spi_nor_ids) - 1)
+ id = plat_id;
+ else
+ dev_warn(dev, "unrecognized id %s\n", data->type);
+ }
+
+ info = (void *)id->driver_data;
+
+ if (info->jedec_id) {
+ const struct spi_device_id *jid;
+
+ jid = jedec_probe(nor);
+ if (IS_ERR(jid)) {
+ return PTR_ERR(jid);
+ } else if (jid != id) {
+ /*
+ * JEDEC knows better, so overwrite platform ID. We
+ * can't trust partitions any longer, but we'll let
+ * mtd apply them anyway, since some partitions may be
+ * marked read-only, and we don't want to lose that
+ * information, even if it's not 100% accurate.
+ */
+ dev_warn(dev, "found %s, expected %s\n",
+ jid->name, id->name);
+ id = jid;
+ info = (void *)jid->driver_data;
+ }
+ }
+
+ mutex_init(&nor->lock);
+
+ /*
+ * Atmel, SST and Intel/Numonyx serial nor tend to power
+ * up with the software protection bits set
+ */
+
+ if (JEDEC_MFR(info->jedec_id) == CFI_MFR_ATMEL ||
+ JEDEC_MFR(info->jedec_id) == CFI_MFR_INTEL ||
+ JEDEC_MFR(info->jedec_id) == CFI_MFR_SST) {
+ write_enable(nor);
+ write_sr(nor, 0);
+ }
+
+ if (data && data->name)
+ mtd->name = data->name;
+ else
+ mtd->name = dev_name(dev);
+
+ mtd->type = MTD_NORFLASH;
+ mtd->writesize = 1;
+ mtd->flags = MTD_CAP_NORFLASH;
+ mtd->size = info->sector_size * info->n_sectors;
+ mtd->_erase = spi_nor_erase;
+ mtd->_read = spi_nor_read;
+
+ /* nor protection support for STmicro chips */
+ if (JEDEC_MFR(info->jedec_id) == CFI_MFR_ST) {
+ mtd->_lock = spi_nor_lock;
+ mtd->_unlock = spi_nor_unlock;
+ }
+
+ /* sst nor chips use AAI word program */
+ if (info->flags & SST_WRITE)
+ mtd->_write = sst_write;
+ else
+ mtd->_write = spi_nor_write;
+
+ /* prefer "small sector" erase if possible */
+ if (info->flags & SECT_4K) {
+ nor->erase_opcode = SPINOR_OP_BE_4K;
+ mtd->erasesize = 4096;
+ } else if (info->flags & SECT_4K_PMC) {
+ nor->erase_opcode = SPINOR_OP_BE_4K_PMC;
+ mtd->erasesize = 4096;
+ } else {
+ nor->erase_opcode = SPINOR_OP_SE;
+ mtd->erasesize = info->sector_size;
+ }
+
+ if (info->flags & SPI_NOR_NO_ERASE)
+ mtd->flags |= MTD_NO_ERASE;
+
+ mtd->dev.parent = dev;
+ nor->page_size = info->page_size;
+ mtd->writebufsize = nor->page_size;
+
+ if (np) {
+ /* If we were instantiated by DT, use it */
+ if (of_property_read_bool(np, "m25p,fast-read"))
+ nor->flash_read = SPI_NOR_FAST;
+ else
+ nor->flash_read = SPI_NOR_NORMAL;
+ } else {
+ /* If we weren't instantiated by DT, default to fast-read */
+ nor->flash_read = SPI_NOR_FAST;
+ }
+
+ /* Some devices cannot do fast-read, no matter what DT tells us */
+ if (info->flags & SPI_NOR_NO_FR)
+ nor->flash_read = SPI_NOR_NORMAL;
+
+ /* Quad/Dual-read mode takes precedence over fast/normal */
+ if (mode == SPI_NOR_QUAD && info->flags & SPI_NOR_QUAD_READ) {
+ ret = set_quad_mode(nor, info->jedec_id);
+ if (ret) {
+ dev_err(dev, "quad mode not supported\n");
+ return ret;
+ }
+ nor->flash_read = SPI_NOR_QUAD;
+ } else if (mode == SPI_NOR_DUAL && info->flags & SPI_NOR_DUAL_READ) {
+ nor->flash_read = SPI_NOR_DUAL;
+ }
+
+ /* Default commands */
+ switch (nor->flash_read) {
+ case SPI_NOR_QUAD:
+ nor->read_opcode = SPINOR_OP_READ_1_1_4;
+ break;
+ case SPI_NOR_DUAL:
+ nor->read_opcode = SPINOR_OP_READ_1_1_2;
+ break;
+ case SPI_NOR_FAST:
+ nor->read_opcode = SPINOR_OP_READ_FAST;
+ break;
+ case SPI_NOR_NORMAL:
+ nor->read_opcode = SPINOR_OP_READ;
+ break;
+ default:
+ dev_err(dev, "No Read opcode defined\n");
+ return -EINVAL;
+ }
+
+ nor->program_opcode = SPINOR_OP_PP;
+
+ if (info->addr_width)
+ nor->addr_width = info->addr_width;
+ else if (mtd->size > 0x1000000) {
+ /* enable 4-byte addressing if the device exceeds 16MiB */
+ nor->addr_width = 4;
+ if (JEDEC_MFR(info->jedec_id) == CFI_MFR_AMD) {
+ /* Dedicated 4-byte command set */
+ switch (nor->flash_read) {
+ case SPI_NOR_QUAD:
+ nor->read_opcode = SPINOR_OP_READ4_1_1_4;
+ break;
+ case SPI_NOR_DUAL:
+ nor->read_opcode = SPINOR_OP_READ4_1_1_2;
+ break;
+ case SPI_NOR_FAST:
+ nor->read_opcode = SPINOR_OP_READ4_FAST;
+ break;
+ case SPI_NOR_NORMAL:
+ nor->read_opcode = SPINOR_OP_READ4;
+ break;
+ }
+ nor->program_opcode = SPINOR_OP_PP_4B;
+ /* No small sector erase for 4-byte command set */
+ nor->erase_opcode = SPINOR_OP_SE_4B;
+ mtd->erasesize = info->sector_size;
+ } else
+ set_4byte(nor, info->jedec_id, 1);
+ } else {
+ nor->addr_width = 3;
+ }
+
+ nor->read_dummy = spi_nor_read_dummy_cycles(nor);
+
+ dev_info(dev, "%s (%lld Kbytes)\n", id->name,
+ (long long)mtd->size >> 10);
+
+ dev_dbg(dev,
+ "mtd .name = %s, .size = 0x%llx (%lldMiB), "
+ ".erasesize = 0x%.8x (%uKiB) .numeraseregions = %d\n",
+ mtd->name, (long long)mtd->size, (long long)(mtd->size >> 20),
+ mtd->erasesize, mtd->erasesize / 1024, mtd->numeraseregions);
+
+ if (mtd->numeraseregions)
+ for (i = 0; i < mtd->numeraseregions; i++)
+ dev_dbg(dev,
+ "mtd.eraseregions[%d] = { .offset = 0x%llx, "
+ ".erasesize = 0x%.8x (%uKiB), "
+ ".numblocks = %d }\n",
+ i, (long long)mtd->eraseregions[i].offset,
+ mtd->eraseregions[i].erasesize,
+ mtd->eraseregions[i].erasesize / 1024,
+ mtd->eraseregions[i].numblocks);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(spi_nor_scan);
+
+const struct spi_device_id *spi_nor_match_id(char *name)
+{
+ const struct spi_device_id *id = spi_nor_ids;
+
+ while (id->name[0]) {
+ if (!strcmp(name, id->name))
+ return id;
+ id++;
+ }
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(spi_nor_match_id);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Huang Shijie <shijie8@gmail.com>");
+MODULE_AUTHOR("Mike Lavender");
+MODULE_DESCRIPTION("framework for SPI NOR");
diff --git a/drivers/mtd/tests/oobtest.c b/drivers/mtd/tests/oobtest.c
index 2e9e2d1..f19ab1a 100644
--- a/drivers/mtd/tests/oobtest.c
+++ b/drivers/mtd/tests/oobtest.c
@@ -69,8 +69,8 @@
int err = 0;
loff_t addr = ebnum * mtd->erasesize;
+ prandom_bytes_state(&rnd_state, writebuf, use_len_max * pgcnt);
for (i = 0; i < pgcnt; ++i, addr += mtd->writesize) {
- prandom_bytes_state(&rnd_state, writebuf, use_len);
ops.mode = MTD_OPS_AUTO_OOB;
ops.len = 0;
ops.retlen = 0;
@@ -78,7 +78,7 @@
ops.oobretlen = 0;
ops.ooboffs = use_offset;
ops.datbuf = NULL;
- ops.oobbuf = writebuf;
+ ops.oobbuf = writebuf + (use_len_max * i) + use_offset;
err = mtd_write_oob(mtd, addr, &ops);
if (err || ops.oobretlen != use_len) {
pr_err("error: writeoob failed at %#llx\n",
@@ -122,8 +122,8 @@
int err = 0;
loff_t addr = ebnum * mtd->erasesize;
+ prandom_bytes_state(&rnd_state, writebuf, use_len_max * pgcnt);
for (i = 0; i < pgcnt; ++i, addr += mtd->writesize) {
- prandom_bytes_state(&rnd_state, writebuf, use_len);
ops.mode = MTD_OPS_AUTO_OOB;
ops.len = 0;
ops.retlen = 0;
@@ -139,7 +139,8 @@
errcnt += 1;
return err ? err : -1;
}
- if (memcmp(readbuf, writebuf, use_len)) {
+ if (memcmp(readbuf, writebuf + (use_len_max * i) + use_offset,
+ use_len)) {
pr_err("error: verify failed at %#llx\n",
(long long)addr);
errcnt += 1;
@@ -166,7 +167,9 @@
errcnt += 1;
return err ? err : -1;
}
- if (memcmp(readbuf + use_offset, writebuf, use_len)) {
+ if (memcmp(readbuf + use_offset,
+ writebuf + (use_len_max * i) + use_offset,
+ use_len)) {
pr_err("error: verify failed at %#llx\n",
(long long)addr);
errcnt += 1;
@@ -566,8 +569,8 @@
if (bbt[i] || bbt[i + 1])
continue;
addr = (i + 1) * mtd->erasesize - mtd->writesize;
+ prandom_bytes_state(&rnd_state, writebuf, sz * cnt);
for (pg = 0; pg < cnt; ++pg) {
- prandom_bytes_state(&rnd_state, writebuf, sz);
ops.mode = MTD_OPS_AUTO_OOB;
ops.len = 0;
ops.retlen = 0;
@@ -575,7 +578,7 @@
ops.oobretlen = 0;
ops.ooboffs = 0;
ops.datbuf = NULL;
- ops.oobbuf = writebuf;
+ ops.oobbuf = writebuf + pg * sz;
err = mtd_write_oob(mtd, addr, &ops);
if (err)
goto out;
diff --git a/drivers/mtd/ubi/block.c b/drivers/mtd/ubi/block.c
index 20a667c..8457df7 100644
--- a/drivers/mtd/ubi/block.c
+++ b/drivers/mtd/ubi/block.c
@@ -432,8 +432,10 @@
* Rembember workqueues are cheap, they're not threads.
*/
dev->wq = alloc_workqueue("%s", 0, 0, gd->disk_name);
- if (!dev->wq)
+ if (!dev->wq) {
+ ret = -ENOMEM;
goto out_free_queue;
+ }
INIT_WORK(&dev->work, ubiblock_do_work);
mutex_lock(&devices_mutex);
diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
index f54562a..7646220 100644
--- a/drivers/mtd/ubi/cdev.c
+++ b/drivers/mtd/ubi/cdev.c
@@ -731,7 +731,7 @@
goto out_free;
}
- re->desc = ubi_open_volume(ubi->ubi_num, vol_id, UBI_EXCLUSIVE);
+ re->desc = ubi_open_volume(ubi->ubi_num, vol_id, UBI_READWRITE);
if (IS_ERR(re->desc)) {
err = PTR_ERR(re->desc);
ubi_err("cannot open volume %d, error %d", vol_id, err);
diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index c5dad65..b04e7d0 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -125,9 +125,9 @@
parent = *p;
av = rb_entry(parent, struct ubi_ainf_volume, rb);
- if (vol_id > av->vol_id)
+ if (vol_id < av->vol_id)
p = &(*p)->rb_left;
- else if (vol_id > av->vol_id)
+ else
p = &(*p)->rb_right;
}
diff --git a/drivers/net/can/led.c b/drivers/net/can/led.c
index a3d99a8..ab7f1b0 100644
--- a/drivers/net/can/led.c
+++ b/drivers/net/can/led.c
@@ -97,6 +97,9 @@
if (!priv)
return NOTIFY_DONE;
+ if (!priv->tx_led_trig || !priv->rx_led_trig)
+ return NOTIFY_DONE;
+
if (msg == NETDEV_CHANGENAME) {
snprintf(name, sizeof(name), "%s-tx", netdev->name);
led_trigger_rename_static(name, priv->tx_led_trig);
diff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig
index d740101..0513494 100644
--- a/drivers/net/ethernet/Kconfig
+++ b/drivers/net/ethernet/Kconfig
@@ -39,6 +39,7 @@
config CX_ECAT
tristate "Beckhoff CX5020 EtherCAT master support"
depends on PCI
+ depends on X86 || COMPILE_TEST
---help---
Driver for EtherCAT master module located on CCAT FPGA
that can be found on Beckhoff CX5020, and possibly other of CX
diff --git a/drivers/net/ethernet/ibm/emac/mal.c b/drivers/net/ethernet/ibm/emac/mal.c
index 9d75fef..63eb959 100644
--- a/drivers/net/ethernet/ibm/emac/mal.c
+++ b/drivers/net/ethernet/ibm/emac/mal.c
@@ -682,10 +682,7 @@
goto fail6;
/* Enable all MAL SERR interrupt sources */
- if (mal->version == 2)
- set_mal_dcrn(mal, MAL_IER, MAL2_IER_EVENTS);
- else
- set_mal_dcrn(mal, MAL_IER, MAL1_IER_EVENTS);
+ set_mal_dcrn(mal, MAL_IER, MAL_IER_EVENTS);
/* Enable EOB interrupt */
mal_enable_eob_irq(mal);
diff --git a/drivers/net/ethernet/ibm/emac/mal.h b/drivers/net/ethernet/ibm/emac/mal.h
index e431a32..eeade2e 100644
--- a/drivers/net/ethernet/ibm/emac/mal.h
+++ b/drivers/net/ethernet/ibm/emac/mal.h
@@ -95,24 +95,20 @@
#define MAL_IER 0x02
+/* MAL IER bits */
#define MAL_IER_DE 0x00000010
#define MAL_IER_OTE 0x00000004
#define MAL_IER_OE 0x00000002
#define MAL_IER_PE 0x00000001
-/* MAL V1 IER bits */
-#define MAL1_IER_NWE 0x00000008
-#define MAL1_IER_SOC_EVENTS MAL1_IER_NWE
-#define MAL1_IER_EVENTS (MAL1_IER_SOC_EVENTS | MAL_IER_DE | \
- MAL_IER_OTE | MAL_IER_OE | MAL_IER_PE)
-/* MAL V2 IER bits */
-#define MAL2_IER_PT 0x00000080
-#define MAL2_IER_PRE 0x00000040
-#define MAL2_IER_PWE 0x00000020
-#define MAL2_IER_SOC_EVENTS (MAL2_IER_PT | MAL2_IER_PRE | MAL2_IER_PWE)
-#define MAL2_IER_EVENTS (MAL2_IER_SOC_EVENTS | MAL_IER_DE | \
- MAL_IER_OTE | MAL_IER_OE | MAL_IER_PE)
+/* PLB read/write/timeout errors */
+#define MAL_IER_PTE 0x00000080
+#define MAL_IER_PRE 0x00000040
+#define MAL_IER_PWE 0x00000020
+#define MAL_IER_SOC_EVENTS (MAL_IER_PTE | MAL_IER_PRE | MAL_IER_PWE)
+#define MAL_IER_EVENTS (MAL_IER_SOC_EVENTS | MAL_IER_DE | \
+ MAL_IER_OTE | MAL_IER_OE | MAL_IER_PE)
#define MAL_TXCASR 0x04
#define MAL_TXCARR 0x05
diff --git a/drivers/net/ethernet/ibm/emac/rgmii.c b/drivers/net/ethernet/ibm/emac/rgmii.c
index 4fb2f96..a01182c 100644
--- a/drivers/net/ethernet/ibm/emac/rgmii.c
+++ b/drivers/net/ethernet/ibm/emac/rgmii.c
@@ -45,6 +45,7 @@
/* RGMIIx_SSR */
#define RGMII_SSR_MASK(idx) (0x7 << ((idx) * 8))
+#define RGMII_SSR_10(idx) (0x1 << ((idx) * 8))
#define RGMII_SSR_100(idx) (0x2 << ((idx) * 8))
#define RGMII_SSR_1000(idx) (0x4 << ((idx) * 8))
@@ -139,6 +140,8 @@
ssr |= RGMII_SSR_1000(input);
else if (speed == SPEED_100)
ssr |= RGMII_SSR_100(input);
+ else if (speed == SPEED_10)
+ ssr |= RGMII_SSR_10(input);
out_be32(&p->ssr, ssr);
diff --git a/drivers/net/ethernet/mellanox/mlx4/alloc.c b/drivers/net/ethernet/mellanox/mlx4/alloc.c
index c3ad464..b0297da 100644
--- a/drivers/net/ethernet/mellanox/mlx4/alloc.c
+++ b/drivers/net/ethernet/mellanox/mlx4/alloc.c
@@ -171,7 +171,7 @@
*/
int mlx4_buf_alloc(struct mlx4_dev *dev, int size, int max_direct,
- struct mlx4_buf *buf)
+ struct mlx4_buf *buf, gfp_t gfp)
{
dma_addr_t t;
@@ -180,7 +180,7 @@
buf->npages = 1;
buf->page_shift = get_order(size) + PAGE_SHIFT;
buf->direct.buf = dma_alloc_coherent(&dev->pdev->dev,
- size, &t, GFP_KERNEL);
+ size, &t, gfp);
if (!buf->direct.buf)
return -ENOMEM;
@@ -200,14 +200,14 @@
buf->npages = buf->nbufs;
buf->page_shift = PAGE_SHIFT;
buf->page_list = kcalloc(buf->nbufs, sizeof(*buf->page_list),
- GFP_KERNEL);
+ gfp);
if (!buf->page_list)
return -ENOMEM;
for (i = 0; i < buf->nbufs; ++i) {
buf->page_list[i].buf =
dma_alloc_coherent(&dev->pdev->dev, PAGE_SIZE,
- &t, GFP_KERNEL);
+ &t, gfp);
if (!buf->page_list[i].buf)
goto err_free;
@@ -218,7 +218,7 @@
if (BITS_PER_LONG == 64) {
struct page **pages;
- pages = kmalloc(sizeof *pages * buf->nbufs, GFP_KERNEL);
+ pages = kmalloc(sizeof *pages * buf->nbufs, gfp);
if (!pages)
goto err_free;
for (i = 0; i < buf->nbufs; ++i)
@@ -260,11 +260,12 @@
}
EXPORT_SYMBOL_GPL(mlx4_buf_free);
-static struct mlx4_db_pgdir *mlx4_alloc_db_pgdir(struct device *dma_device)
+static struct mlx4_db_pgdir *mlx4_alloc_db_pgdir(struct device *dma_device,
+ gfp_t gfp)
{
struct mlx4_db_pgdir *pgdir;
- pgdir = kzalloc(sizeof *pgdir, GFP_KERNEL);
+ pgdir = kzalloc(sizeof *pgdir, gfp);
if (!pgdir)
return NULL;
@@ -272,7 +273,7 @@
pgdir->bits[0] = pgdir->order0;
pgdir->bits[1] = pgdir->order1;
pgdir->db_page = dma_alloc_coherent(dma_device, PAGE_SIZE,
- &pgdir->db_dma, GFP_KERNEL);
+ &pgdir->db_dma, gfp);
if (!pgdir->db_page) {
kfree(pgdir);
return NULL;
@@ -312,7 +313,7 @@
return 0;
}
-int mlx4_db_alloc(struct mlx4_dev *dev, struct mlx4_db *db, int order)
+int mlx4_db_alloc(struct mlx4_dev *dev, struct mlx4_db *db, int order, gfp_t gfp)
{
struct mlx4_priv *priv = mlx4_priv(dev);
struct mlx4_db_pgdir *pgdir;
@@ -324,7 +325,7 @@
if (!mlx4_alloc_db_from_pgdir(pgdir, db, order))
goto out;
- pgdir = mlx4_alloc_db_pgdir(&(dev->pdev->dev));
+ pgdir = mlx4_alloc_db_pgdir(&(dev->pdev->dev), gfp);
if (!pgdir) {
ret = -ENOMEM;
goto out;
@@ -376,13 +377,13 @@
{
int err;
- err = mlx4_db_alloc(dev, &wqres->db, 1);
+ err = mlx4_db_alloc(dev, &wqres->db, 1, GFP_KERNEL);
if (err)
return err;
*wqres->db.db = 0;
- err = mlx4_buf_alloc(dev, size, max_direct, &wqres->buf);
+ err = mlx4_buf_alloc(dev, size, max_direct, &wqres->buf, GFP_KERNEL);
if (err)
goto err_db;
@@ -391,7 +392,7 @@
if (err)
goto err_buf;
- err = mlx4_buf_write_mtt(dev, &wqres->mtt, &wqres->buf);
+ err = mlx4_buf_write_mtt(dev, &wqres->mtt, &wqres->buf, GFP_KERNEL);
if (err)
goto err_mtt;
diff --git a/drivers/net/ethernet/mellanox/mlx4/cmd.c b/drivers/net/ethernet/mellanox/mlx4/cmd.c
index 92d3249..29b6169 100644
--- a/drivers/net/ethernet/mellanox/mlx4/cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx4/cmd.c
@@ -705,20 +705,28 @@
struct ib_smp *smp = inbox->buf;
u32 index;
u8 port;
+ u8 opcode_modifier;
u16 *table;
int err;
int vidx, pidx;
+ int network_view;
struct mlx4_priv *priv = mlx4_priv(dev);
struct ib_smp *outsmp = outbox->buf;
__be16 *outtab = (__be16 *)(outsmp->data);
__be32 slave_cap_mask;
__be64 slave_node_guid;
+
port = vhcr->in_modifier;
+ /* network-view bit is for driver use only, and should not be passed to FW */
+ opcode_modifier = vhcr->op_modifier & ~0x8; /* clear netw view bit */
+ network_view = !!(vhcr->op_modifier & 0x8);
+
if (smp->base_version == 1 &&
smp->mgmt_class == IB_MGMT_CLASS_SUBN_LID_ROUTED &&
smp->class_version == 1) {
- if (smp->method == IB_MGMT_METHOD_GET) {
+ /* host view is paravirtualized */
+ if (!network_view && smp->method == IB_MGMT_METHOD_GET) {
if (smp->attr_id == IB_SMP_ATTR_PKEY_TABLE) {
index = be32_to_cpu(smp->attr_mod);
if (port < 1 || port > dev->caps.num_ports)
@@ -743,7 +751,7 @@
/*get the slave specific caps:*/
/*do the command */
err = mlx4_cmd_box(dev, inbox->dma, outbox->dma,
- vhcr->in_modifier, vhcr->op_modifier,
+ vhcr->in_modifier, opcode_modifier,
vhcr->op, MLX4_CMD_TIME_CLASS_C, MLX4_CMD_NATIVE);
/* modify the response for slaves */
if (!err && slave != mlx4_master_func_num(dev)) {
@@ -760,7 +768,7 @@
smp->attr_mod = cpu_to_be32(slave / 8);
/* execute cmd */
err = mlx4_cmd_box(dev, inbox->dma, outbox->dma,
- vhcr->in_modifier, vhcr->op_modifier,
+ vhcr->in_modifier, opcode_modifier,
vhcr->op, MLX4_CMD_TIME_CLASS_C, MLX4_CMD_NATIVE);
if (!err) {
/* if needed, move slave gid to index 0 */
@@ -774,7 +782,7 @@
}
if (smp->attr_id == IB_SMP_ATTR_NODE_INFO) {
err = mlx4_cmd_box(dev, inbox->dma, outbox->dma,
- vhcr->in_modifier, vhcr->op_modifier,
+ vhcr->in_modifier, opcode_modifier,
vhcr->op, MLX4_CMD_TIME_CLASS_C, MLX4_CMD_NATIVE);
if (!err) {
slave_node_guid = mlx4_get_slave_node_guid(dev, slave);
@@ -784,19 +792,24 @@
}
}
}
+
+ /* Non-privileged VFs are only allowed "host" view LID-routed 'Get' MADs.
+ * These are the MADs used by ib verbs (such as ib_query_gids).
+ */
if (slave != mlx4_master_func_num(dev) &&
- ((smp->mgmt_class == IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE) ||
- (smp->mgmt_class == IB_MGMT_CLASS_SUBN_LID_ROUTED &&
- smp->method == IB_MGMT_METHOD_SET))) {
- mlx4_err(dev, "slave %d is trying to execute a Subnet MGMT MAD, "
- "class 0x%x, method 0x%x for attr 0x%x. Rejecting\n",
- slave, smp->method, smp->mgmt_class,
- be16_to_cpu(smp->attr_id));
- return -EPERM;
+ !mlx4_vf_smi_enabled(dev, slave, port)) {
+ if (!(smp->mgmt_class == IB_MGMT_CLASS_SUBN_LID_ROUTED &&
+ smp->method == IB_MGMT_METHOD_GET) || network_view) {
+ mlx4_err(dev, "Unprivileged slave %d is trying to execute a Subnet MGMT MAD, class 0x%x, method 0x%x, view=%s for attr 0x%x. Rejecting\n",
+ slave, smp->method, smp->mgmt_class,
+ network_view ? "Network" : "Host",
+ be16_to_cpu(smp->attr_id));
+ return -EPERM;
+ }
}
- /*default:*/
+
return mlx4_cmd_box(dev, inbox->dma, outbox->dma,
- vhcr->in_modifier, vhcr->op_modifier,
+ vhcr->in_modifier, opcode_modifier,
vhcr->op, MLX4_CMD_TIME_CLASS_C, MLX4_CMD_NATIVE);
}
@@ -1653,6 +1666,8 @@
for (port = min_port; port <= max_port; port++) {
if (!test_bit(port - 1, actv_ports.ports))
continue;
+ priv->mfunc.master.vf_oper[slave].smi_enabled[port] =
+ priv->mfunc.master.vf_admin[slave].enable_smi[port];
vp_oper = &priv->mfunc.master.vf_oper[slave].vport[port];
vp_admin = &priv->mfunc.master.vf_admin[slave].vport[port];
vp_oper->state = *vp_admin;
@@ -1704,6 +1719,8 @@
for (port = min_port; port <= max_port; port++) {
if (!test_bit(port - 1, actv_ports.ports))
continue;
+ priv->mfunc.master.vf_oper[slave].smi_enabled[port] =
+ MLX4_VF_SMI_DISABLED;
vp_oper = &priv->mfunc.master.vf_oper[slave].vport[port];
if (NO_INDX != vp_oper->vlan_idx) {
__mlx4_unregister_vlan(&priv->dev,
@@ -2537,3 +2554,50 @@
return 0;
}
EXPORT_SYMBOL_GPL(mlx4_set_vf_link_state);
+
+int mlx4_vf_smi_enabled(struct mlx4_dev *dev, int slave, int port)
+{
+ struct mlx4_priv *priv = mlx4_priv(dev);
+
+ if (slave < 1 || slave >= dev->num_slaves ||
+ port < 1 || port > MLX4_MAX_PORTS)
+ return 0;
+
+ return priv->mfunc.master.vf_oper[slave].smi_enabled[port] ==
+ MLX4_VF_SMI_ENABLED;
+}
+EXPORT_SYMBOL_GPL(mlx4_vf_smi_enabled);
+
+int mlx4_vf_get_enable_smi_admin(struct mlx4_dev *dev, int slave, int port)
+{
+ struct mlx4_priv *priv = mlx4_priv(dev);
+
+ if (slave == mlx4_master_func_num(dev))
+ return 1;
+
+ if (slave < 1 || slave >= dev->num_slaves ||
+ port < 1 || port > MLX4_MAX_PORTS)
+ return 0;
+
+ return priv->mfunc.master.vf_admin[slave].enable_smi[port] ==
+ MLX4_VF_SMI_ENABLED;
+}
+EXPORT_SYMBOL_GPL(mlx4_vf_get_enable_smi_admin);
+
+int mlx4_vf_set_enable_smi_admin(struct mlx4_dev *dev, int slave, int port,
+ int enabled)
+{
+ struct mlx4_priv *priv = mlx4_priv(dev);
+
+ if (slave == mlx4_master_func_num(dev))
+ return 0;
+
+ if (slave < 1 || slave >= dev->num_slaves ||
+ port < 1 || port > MLX4_MAX_PORTS ||
+ enabled < 0 || enabled > 1)
+ return -EINVAL;
+
+ priv->mfunc.master.vf_admin[slave].enable_smi[port] = enabled;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(mlx4_vf_set_enable_smi_admin);
diff --git a/drivers/net/ethernet/mellanox/mlx4/cq.c b/drivers/net/ethernet/mellanox/mlx4/cq.c
index 0487121..c90cde5 100644
--- a/drivers/net/ethernet/mellanox/mlx4/cq.c
+++ b/drivers/net/ethernet/mellanox/mlx4/cq.c
@@ -173,11 +173,11 @@
if (*cqn == -1)
return -ENOMEM;
- err = mlx4_table_get(dev, &cq_table->table, *cqn);
+ err = mlx4_table_get(dev, &cq_table->table, *cqn, GFP_KERNEL);
if (err)
goto err_out;
- err = mlx4_table_get(dev, &cq_table->cmpt_table, *cqn);
+ err = mlx4_table_get(dev, &cq_table->cmpt_table, *cqn, GFP_KERNEL);
if (err)
goto err_put;
return 0;
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
index ba049ae..87857a6 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
@@ -972,7 +972,7 @@
if (!context)
return -ENOMEM;
- err = mlx4_qp_alloc(mdev->dev, qpn, qp);
+ err = mlx4_qp_alloc(mdev->dev, qpn, qp, GFP_KERNEL);
if (err) {
en_err(priv, "Failed to allocate qp #%x\n", qpn);
goto out;
@@ -1012,7 +1012,7 @@
en_err(priv, "Failed reserving drop qpn\n");
return err;
}
- err = mlx4_qp_alloc(priv->mdev->dev, qpn, &priv->drop_qp);
+ err = mlx4_qp_alloc(priv->mdev->dev, qpn, &priv->drop_qp, GFP_KERNEL);
if (err) {
en_err(priv, "Failed allocating drop qp\n");
mlx4_qp_release_range(priv->mdev->dev, qpn, 1);
@@ -1071,7 +1071,7 @@
}
/* Configure RSS indirection qp */
- err = mlx4_qp_alloc(mdev->dev, priv->base_qpn, &rss_map->indir_qp);
+ err = mlx4_qp_alloc(mdev->dev, priv->base_qpn, &rss_map->indir_qp, GFP_KERNEL);
if (err) {
en_err(priv, "Failed to allocate RSS indirection QP\n");
goto rss_err;
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_tx.c b/drivers/net/ethernet/mellanox/mlx4/en_tx.c
index dd1f6d3..bc0cc1e 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_tx.c
@@ -113,7 +113,7 @@
ring->buf_size, (unsigned long long) ring->wqres.buf.direct.map);
ring->qpn = qpn;
- err = mlx4_qp_alloc(mdev->dev, ring->qpn, &ring->qp);
+ err = mlx4_qp_alloc(mdev->dev, ring->qpn, &ring->qp, GFP_KERNEL);
if (err) {
en_err(priv, "Failed allocating qp %d\n", ring->qpn);
goto err_map;
diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.c b/drivers/net/ethernet/mellanox/mlx4/fw.c
index d16a4d1..01e6dd6 100644
--- a/drivers/net/ethernet/mellanox/mlx4/fw.c
+++ b/drivers/net/ethernet/mellanox/mlx4/fw.c
@@ -178,8 +178,8 @@
struct mlx4_cmd_info *cmd)
{
struct mlx4_priv *priv = mlx4_priv(dev);
- u8 field;
- u32 size;
+ u8 field, port;
+ u32 size, proxy_qp, qkey;
int err = 0;
#define QUERY_FUNC_CAP_FLAGS_OFFSET 0x0
@@ -209,6 +209,7 @@
/* when opcode modifier = 1 */
#define QUERY_FUNC_CAP_PHYS_PORT_OFFSET 0x3
+#define QUERY_FUNC_CAP_PRIV_VF_QKEY_OFFSET 0x4
#define QUERY_FUNC_CAP_FLAGS0_OFFSET 0x8
#define QUERY_FUNC_CAP_FLAGS1_OFFSET 0xc
@@ -221,6 +222,7 @@
#define QUERY_FUNC_CAP_FLAGS1_FORCE_MAC 0x40
#define QUERY_FUNC_CAP_FLAGS1_FORCE_VLAN 0x80
#define QUERY_FUNC_CAP_FLAGS1_NIC_INFO 0x10
+#define QUERY_FUNC_CAP_VF_ENABLE_QP0 0x08
#define QUERY_FUNC_CAP_FLAGS0_FORCE_PHY_WQE_GID 0x80
@@ -234,28 +236,35 @@
return -EINVAL;
vhcr->in_modifier = converted_port;
- /* Set nic_info bit to mark new fields support */
- field = QUERY_FUNC_CAP_FLAGS1_NIC_INFO;
- MLX4_PUT(outbox->buf, field, QUERY_FUNC_CAP_FLAGS1_OFFSET);
-
/* phys-port = logical-port */
field = vhcr->in_modifier -
find_first_bit(actv_ports.ports, dev->caps.num_ports);
MLX4_PUT(outbox->buf, field, QUERY_FUNC_CAP_PHYS_PORT_OFFSET);
- field = vhcr->in_modifier;
+ port = vhcr->in_modifier;
+ proxy_qp = dev->phys_caps.base_proxy_sqpn + 8 * slave + port - 1;
+
+ /* Set nic_info bit to mark new fields support */
+ field = QUERY_FUNC_CAP_FLAGS1_NIC_INFO;
+
+ if (mlx4_vf_smi_enabled(dev, slave, port) &&
+ !mlx4_get_parav_qkey(dev, proxy_qp, &qkey)) {
+ field |= QUERY_FUNC_CAP_VF_ENABLE_QP0;
+ MLX4_PUT(outbox->buf, qkey,
+ QUERY_FUNC_CAP_PRIV_VF_QKEY_OFFSET);
+ }
+ MLX4_PUT(outbox->buf, field, QUERY_FUNC_CAP_FLAGS1_OFFSET);
+
/* size is now the QP number */
- size = dev->phys_caps.base_tunnel_sqpn + 8 * slave + field - 1;
+ size = dev->phys_caps.base_tunnel_sqpn + 8 * slave + port - 1;
MLX4_PUT(outbox->buf, size, QUERY_FUNC_CAP_QP0_TUNNEL);
size += 2;
MLX4_PUT(outbox->buf, size, QUERY_FUNC_CAP_QP1_TUNNEL);
- size = dev->phys_caps.base_proxy_sqpn + 8 * slave + field - 1;
- MLX4_PUT(outbox->buf, size, QUERY_FUNC_CAP_QP0_PROXY);
-
- size += 2;
- MLX4_PUT(outbox->buf, size, QUERY_FUNC_CAP_QP1_PROXY);
+ MLX4_PUT(outbox->buf, proxy_qp, QUERY_FUNC_CAP_QP0_PROXY);
+ proxy_qp += 2;
+ MLX4_PUT(outbox->buf, proxy_qp, QUERY_FUNC_CAP_QP1_PROXY);
MLX4_PUT(outbox->buf, dev->caps.phys_port_id[vhcr->in_modifier],
QUERY_FUNC_CAP_PHYS_PORT_ID);
@@ -326,7 +335,7 @@
struct mlx4_cmd_mailbox *mailbox;
u32 *outbox;
u8 field, op_modifier;
- u32 size;
+ u32 size, qkey;
int err = 0, quotas = 0;
op_modifier = !!gen_or_port; /* 0 = general, 1 = logical port */
@@ -414,7 +423,7 @@
MLX4_GET(func_cap->flags1, outbox, QUERY_FUNC_CAP_FLAGS1_OFFSET);
if (dev->caps.port_type[gen_or_port] == MLX4_PORT_TYPE_ETH) {
- if (func_cap->flags1 & QUERY_FUNC_CAP_FLAGS1_OFFSET) {
+ if (func_cap->flags1 & QUERY_FUNC_CAP_FLAGS1_FORCE_VLAN) {
mlx4_err(dev, "VLAN is enforced on this port\n");
err = -EPROTONOSUPPORT;
goto out;
@@ -442,6 +451,13 @@
goto out;
}
+ if (func_cap->flags1 & QUERY_FUNC_CAP_VF_ENABLE_QP0) {
+ MLX4_GET(qkey, outbox, QUERY_FUNC_CAP_PRIV_VF_QKEY_OFFSET);
+ func_cap->qp0_qkey = qkey;
+ } else {
+ func_cap->qp0_qkey = 0;
+ }
+
MLX4_GET(size, outbox, QUERY_FUNC_CAP_QP0_TUNNEL);
func_cap->qp0_tunnel_qpn = size & 0xFFFFFF;
diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.h b/drivers/net/ethernet/mellanox/mlx4/fw.h
index 6811ee0..1fce03e 100644
--- a/drivers/net/ethernet/mellanox/mlx4/fw.h
+++ b/drivers/net/ethernet/mellanox/mlx4/fw.h
@@ -134,6 +134,7 @@
int max_eq;
int reserved_eq;
int mcg_quota;
+ u32 qp0_qkey;
u32 qp0_tunnel_qpn;
u32 qp0_proxy_qpn;
u32 qp1_tunnel_qpn;
diff --git a/drivers/net/ethernet/mellanox/mlx4/icm.c b/drivers/net/ethernet/mellanox/mlx4/icm.c
index 5fbf492..97c9b1d 100644
--- a/drivers/net/ethernet/mellanox/mlx4/icm.c
+++ b/drivers/net/ethernet/mellanox/mlx4/icm.c
@@ -245,7 +245,8 @@
MLX4_CMD_TIME_CLASS_B, MLX4_CMD_NATIVE);
}
-int mlx4_table_get(struct mlx4_dev *dev, struct mlx4_icm_table *table, u32 obj)
+int mlx4_table_get(struct mlx4_dev *dev, struct mlx4_icm_table *table, u32 obj,
+ gfp_t gfp)
{
u32 i = (obj & (table->num_obj - 1)) /
(MLX4_TABLE_CHUNK_SIZE / table->obj_size);
@@ -259,7 +260,7 @@
}
table->icm[i] = mlx4_alloc_icm(dev, MLX4_TABLE_CHUNK_SIZE >> PAGE_SHIFT,
- (table->lowmem ? GFP_KERNEL : GFP_HIGHUSER) |
+ (table->lowmem ? gfp : GFP_HIGHUSER) |
__GFP_NOWARN, table->coherent);
if (!table->icm[i]) {
ret = -ENOMEM;
@@ -356,7 +357,7 @@
u32 i;
for (i = start; i <= end; i += inc) {
- err = mlx4_table_get(dev, table, i);
+ err = mlx4_table_get(dev, table, i, GFP_KERNEL);
if (err)
goto fail;
}
diff --git a/drivers/net/ethernet/mellanox/mlx4/icm.h b/drivers/net/ethernet/mellanox/mlx4/icm.h
index dee67fa..0c73645 100644
--- a/drivers/net/ethernet/mellanox/mlx4/icm.h
+++ b/drivers/net/ethernet/mellanox/mlx4/icm.h
@@ -71,7 +71,8 @@
gfp_t gfp_mask, int coherent);
void mlx4_free_icm(struct mlx4_dev *dev, struct mlx4_icm *icm, int coherent);
-int mlx4_table_get(struct mlx4_dev *dev, struct mlx4_icm_table *table, u32 obj);
+int mlx4_table_get(struct mlx4_dev *dev, struct mlx4_icm_table *table, u32 obj,
+ gfp_t gfp);
void mlx4_table_put(struct mlx4_dev *dev, struct mlx4_icm_table *table, u32 obj);
int mlx4_table_get_range(struct mlx4_dev *dev, struct mlx4_icm_table *table,
u32 start, u32 end);
diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index 7cf9dad..26169b3 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -666,13 +666,15 @@
return -ENODEV;
}
+ dev->caps.qp0_qkey = kcalloc(dev->caps.num_ports, sizeof(u32), GFP_KERNEL);
dev->caps.qp0_tunnel = kcalloc(dev->caps.num_ports, sizeof (u32), GFP_KERNEL);
dev->caps.qp0_proxy = kcalloc(dev->caps.num_ports, sizeof (u32), GFP_KERNEL);
dev->caps.qp1_tunnel = kcalloc(dev->caps.num_ports, sizeof (u32), GFP_KERNEL);
dev->caps.qp1_proxy = kcalloc(dev->caps.num_ports, sizeof (u32), GFP_KERNEL);
if (!dev->caps.qp0_tunnel || !dev->caps.qp0_proxy ||
- !dev->caps.qp1_tunnel || !dev->caps.qp1_proxy) {
+ !dev->caps.qp1_tunnel || !dev->caps.qp1_proxy ||
+ !dev->caps.qp0_qkey) {
err = -ENOMEM;
goto err_mem;
}
@@ -684,6 +686,7 @@
" port %d, aborting (%d).\n", i, err);
goto err_mem;
}
+ dev->caps.qp0_qkey[i - 1] = func_cap.qp0_qkey;
dev->caps.qp0_tunnel[i - 1] = func_cap.qp0_tunnel_qpn;
dev->caps.qp0_proxy[i - 1] = func_cap.qp0_proxy_qpn;
dev->caps.qp1_tunnel[i - 1] = func_cap.qp1_tunnel_qpn;
@@ -729,12 +732,16 @@
return 0;
err_mem:
+ kfree(dev->caps.qp0_qkey);
kfree(dev->caps.qp0_tunnel);
kfree(dev->caps.qp0_proxy);
kfree(dev->caps.qp1_tunnel);
kfree(dev->caps.qp1_proxy);
- dev->caps.qp0_tunnel = dev->caps.qp0_proxy =
- dev->caps.qp1_tunnel = dev->caps.qp1_proxy = NULL;
+ dev->caps.qp0_qkey = NULL;
+ dev->caps.qp0_tunnel = NULL;
+ dev->caps.qp0_proxy = NULL;
+ dev->caps.qp1_tunnel = NULL;
+ dev->caps.qp1_proxy = NULL;
return err;
}
@@ -1696,6 +1703,14 @@
unmap_internal_clock(dev);
unmap_bf_area(dev);
+ if (mlx4_is_slave(dev)) {
+ kfree(dev->caps.qp0_qkey);
+ kfree(dev->caps.qp0_tunnel);
+ kfree(dev->caps.qp0_proxy);
+ kfree(dev->caps.qp1_tunnel);
+ kfree(dev->caps.qp1_proxy);
+ }
+
err_close:
if (mlx4_is_slave(dev))
mlx4_slave_exit(dev);
@@ -2044,6 +2059,7 @@
if (!mlx4_is_slave(dev)) {
mlx4_init_mac_table(dev, &info->mac_table);
mlx4_init_vlan_table(dev, &info->vlan_table);
+ mlx4_init_roce_gid_table(dev, &info->gid_table);
info->base_qpn = mlx4_get_base_qpn(dev, port);
}
@@ -2565,6 +2581,14 @@
if (mlx4_is_master(dev))
mlx4_multi_func_cleanup(dev);
+ if (mlx4_is_slave(dev)) {
+ kfree(dev->caps.qp0_qkey);
+ kfree(dev->caps.qp0_tunnel);
+ kfree(dev->caps.qp0_proxy);
+ kfree(dev->caps.qp1_tunnel);
+ kfree(dev->caps.qp1_proxy);
+ }
+
err_close:
if (dev->flags & MLX4_FLAG_MSI_X)
pci_disable_msix(pdev);
@@ -2688,6 +2712,7 @@
if (!mlx4_is_slave(dev))
mlx4_free_ownership(dev);
+ kfree(dev->caps.qp0_qkey);
kfree(dev->caps.qp0_tunnel);
kfree(dev->caps.qp0_proxy);
kfree(dev->caps.qp1_tunnel);
diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4.h b/drivers/net/ethernet/mellanox/mlx4/mlx4.h
index 212cea4..7a0665b 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4.h
@@ -133,6 +133,11 @@
MLX4_COMM_CMD_FLR = 254
};
+enum {
+ MLX4_VF_SMI_DISABLED,
+ MLX4_VF_SMI_ENABLED
+};
+
/*The flag indicates that the slave should delay the RESET cmd*/
#define MLX4_DELAY_RESET_SLAVE 0xbbbbbbb
/*indicates how many retries will be done if we are in the middle of FLR*/
@@ -488,6 +493,7 @@
struct mlx4_vf_admin_state {
struct mlx4_vport_state vport[MLX4_MAX_PORTS + 1];
+ u8 enable_smi[MLX4_MAX_PORTS + 1];
};
struct mlx4_vport_oper_state {
@@ -495,8 +501,10 @@
int mac_idx;
int vlan_idx;
};
+
struct mlx4_vf_oper_state {
struct mlx4_vport_oper_state vport[MLX4_MAX_PORTS + 1];
+ u8 smi_enabled[MLX4_MAX_PORTS + 1];
};
struct slave_list {
@@ -695,6 +703,17 @@
int max;
};
+#define MLX4_ROCE_GID_ENTRY_SIZE 16
+
+struct mlx4_roce_gid_entry {
+ u8 raw[MLX4_ROCE_GID_ENTRY_SIZE];
+};
+
+struct mlx4_roce_gid_table {
+ struct mlx4_roce_gid_entry roce_gids[MLX4_ROCE_MAX_GIDS];
+ struct mutex mutex;
+};
+
#define MLX4_MAX_VLAN_NUM 128
#define MLX4_VLAN_TABLE_SIZE (MLX4_MAX_VLAN_NUM << 2)
@@ -758,6 +777,7 @@
struct device_attribute port_mtu_attr;
struct mlx4_mac_table mac_table;
struct mlx4_vlan_table vlan_table;
+ struct mlx4_roce_gid_table gid_table;
int base_qpn;
};
@@ -788,10 +808,6 @@
MLX4_USE_RR = 1,
};
-struct mlx4_roce_gid_entry {
- u8 raw[16];
-};
-
struct mlx4_priv {
struct mlx4_dev dev;
@@ -839,7 +855,6 @@
int fs_hash_mode;
u8 virt2phys_pkey[MLX4_MFUNC_MAX][MLX4_MAX_PORTS][MLX4_MAX_PORT_PKEYS];
__be64 slave_node_guids[MLX4_MFUNC_MAX];
- struct mlx4_roce_gid_entry roce_gids[MLX4_MAX_PORTS][MLX4_ROCE_MAX_GIDS];
atomic_t opreq_count;
struct work_struct opreq_task;
@@ -888,7 +903,7 @@
void mlx4_cleanup_qp_table(struct mlx4_dev *dev);
void mlx4_cleanup_srq_table(struct mlx4_dev *dev);
void mlx4_cleanup_mcg_table(struct mlx4_dev *dev);
-int __mlx4_qp_alloc_icm(struct mlx4_dev *dev, int qpn);
+int __mlx4_qp_alloc_icm(struct mlx4_dev *dev, int qpn, gfp_t gfp);
void __mlx4_qp_free_icm(struct mlx4_dev *dev, int qpn);
int __mlx4_cq_alloc_icm(struct mlx4_dev *dev, int *cqn);
void __mlx4_cq_free_icm(struct mlx4_dev *dev, int cqn);
@@ -896,7 +911,7 @@
void __mlx4_srq_free_icm(struct mlx4_dev *dev, int srqn);
int __mlx4_mpt_reserve(struct mlx4_dev *dev);
void __mlx4_mpt_release(struct mlx4_dev *dev, u32 index);
-int __mlx4_mpt_alloc_icm(struct mlx4_dev *dev, u32 index);
+int __mlx4_mpt_alloc_icm(struct mlx4_dev *dev, u32 index, gfp_t gfp);
void __mlx4_mpt_free_icm(struct mlx4_dev *dev, u32 index);
u32 __mlx4_alloc_mtt_range(struct mlx4_dev *dev, int order);
void __mlx4_free_mtt_range(struct mlx4_dev *dev, u32 first_seg, int order);
@@ -1140,6 +1155,8 @@
void mlx4_init_mac_table(struct mlx4_dev *dev, struct mlx4_mac_table *table);
void mlx4_init_vlan_table(struct mlx4_dev *dev, struct mlx4_vlan_table *table);
+void mlx4_init_roce_gid_table(struct mlx4_dev *dev,
+ struct mlx4_roce_gid_table *table);
void __mlx4_unregister_vlan(struct mlx4_dev *dev, u8 port, u16 vlan);
int __mlx4_register_vlan(struct mlx4_dev *dev, u8 port, u16 vlan, int *index);
@@ -1149,6 +1166,7 @@
enum mlx4_resource resource_type,
u64 resource_id, int *slave);
void mlx4_delete_all_resources_for_slave(struct mlx4_dev *dev, int slave_id);
+void mlx4_reset_roce_gids(struct mlx4_dev *dev, int slave);
int mlx4_init_resource_tracker(struct mlx4_dev *dev);
void mlx4_free_resource_tracker(struct mlx4_dev *dev,
diff --git a/drivers/net/ethernet/mellanox/mlx4/mr.c b/drivers/net/ethernet/mellanox/mlx4/mr.c
index 2483585..4c71daf 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mr.c
+++ b/drivers/net/ethernet/mellanox/mlx4/mr.c
@@ -364,14 +364,14 @@
__mlx4_mpt_release(dev, index);
}
-int __mlx4_mpt_alloc_icm(struct mlx4_dev *dev, u32 index)
+int __mlx4_mpt_alloc_icm(struct mlx4_dev *dev, u32 index, gfp_t gfp)
{
struct mlx4_mr_table *mr_table = &mlx4_priv(dev)->mr_table;
- return mlx4_table_get(dev, &mr_table->dmpt_table, index);
+ return mlx4_table_get(dev, &mr_table->dmpt_table, index, gfp);
}
-static int mlx4_mpt_alloc_icm(struct mlx4_dev *dev, u32 index)
+static int mlx4_mpt_alloc_icm(struct mlx4_dev *dev, u32 index, gfp_t gfp)
{
u64 param = 0;
@@ -382,7 +382,7 @@
MLX4_CMD_TIME_CLASS_A,
MLX4_CMD_WRAPPED);
}
- return __mlx4_mpt_alloc_icm(dev, index);
+ return __mlx4_mpt_alloc_icm(dev, index, gfp);
}
void __mlx4_mpt_free_icm(struct mlx4_dev *dev, u32 index)
@@ -469,7 +469,7 @@
struct mlx4_mpt_entry *mpt_entry;
int err;
- err = mlx4_mpt_alloc_icm(dev, key_to_hw_index(mr->key));
+ err = mlx4_mpt_alloc_icm(dev, key_to_hw_index(mr->key), GFP_KERNEL);
if (err)
return err;
@@ -627,13 +627,14 @@
EXPORT_SYMBOL_GPL(mlx4_write_mtt);
int mlx4_buf_write_mtt(struct mlx4_dev *dev, struct mlx4_mtt *mtt,
- struct mlx4_buf *buf)
+ struct mlx4_buf *buf, gfp_t gfp)
{
u64 *page_list;
int err;
int i;
- page_list = kmalloc(buf->npages * sizeof *page_list, GFP_KERNEL);
+ page_list = kmalloc(buf->npages * sizeof *page_list,
+ gfp);
if (!page_list)
return -ENOMEM;
@@ -680,7 +681,7 @@
struct mlx4_mpt_entry *mpt_entry;
int err;
- err = mlx4_mpt_alloc_icm(dev, key_to_hw_index(mw->key));
+ err = mlx4_mpt_alloc_icm(dev, key_to_hw_index(mw->key), GFP_KERNEL);
if (err)
return err;
diff --git a/drivers/net/ethernet/mellanox/mlx4/port.c b/drivers/net/ethernet/mellanox/mlx4/port.c
index b5b3549..5ec6f20 100644
--- a/drivers/net/ethernet/mellanox/mlx4/port.c
+++ b/drivers/net/ethernet/mellanox/mlx4/port.c
@@ -75,6 +75,16 @@
table->total = 0;
}
+void mlx4_init_roce_gid_table(struct mlx4_dev *dev,
+ struct mlx4_roce_gid_table *table)
+{
+ int i;
+
+ mutex_init(&table->mutex);
+ for (i = 0; i < MLX4_ROCE_MAX_GIDS; i++)
+ memset(table->roce_gids[i].raw, 0, MLX4_ROCE_GID_ENTRY_SIZE);
+}
+
static int validate_index(struct mlx4_dev *dev,
struct mlx4_mac_table *table, int index)
{
@@ -584,6 +594,84 @@
}
EXPORT_SYMBOL_GPL(mlx4_get_base_gid_ix);
+static int mlx4_reset_roce_port_gids(struct mlx4_dev *dev, int slave,
+ int port, struct mlx4_cmd_mailbox *mailbox)
+{
+ struct mlx4_roce_gid_entry *gid_entry_mbox;
+ struct mlx4_priv *priv = mlx4_priv(dev);
+ int num_gids, base, offset;
+ int i, err;
+
+ num_gids = mlx4_get_slave_num_gids(dev, slave, port);
+ base = mlx4_get_base_gid_ix(dev, slave, port);
+
+ memset(mailbox->buf, 0, MLX4_MAILBOX_SIZE);
+
+ mutex_lock(&(priv->port[port].gid_table.mutex));
+ /* Zero-out gids belonging to that slave in the port GID table */
+ for (i = 0, offset = base; i < num_gids; offset++, i++)
+ memcpy(priv->port[port].gid_table.roce_gids[offset].raw,
+ zgid_entry.raw, MLX4_ROCE_GID_ENTRY_SIZE);
+
+ /* Now, copy roce port gids table to mailbox for passing to FW */
+ gid_entry_mbox = (struct mlx4_roce_gid_entry *)mailbox->buf;
+ for (i = 0; i < MLX4_ROCE_MAX_GIDS; gid_entry_mbox++, i++)
+ memcpy(gid_entry_mbox->raw,
+ priv->port[port].gid_table.roce_gids[i].raw,
+ MLX4_ROCE_GID_ENTRY_SIZE);
+
+ err = mlx4_cmd(dev, mailbox->dma,
+ ((u32)port) | (MLX4_SET_PORT_GID_TABLE << 8), 1,
+ MLX4_CMD_SET_PORT, MLX4_CMD_TIME_CLASS_B,
+ MLX4_CMD_NATIVE);
+ mutex_unlock(&(priv->port[port].gid_table.mutex));
+ return err;
+}
+
+
+void mlx4_reset_roce_gids(struct mlx4_dev *dev, int slave)
+{
+ struct mlx4_active_ports actv_ports;
+ struct mlx4_cmd_mailbox *mailbox;
+ int num_eth_ports, err;
+ int i;
+
+ if (slave < 0 || slave > dev->num_vfs)
+ return;
+
+ actv_ports = mlx4_get_active_ports(dev, slave);
+
+ for (i = 0, num_eth_ports = 0; i < dev->caps.num_ports; i++) {
+ if (test_bit(i, actv_ports.ports)) {
+ if (dev->caps.port_type[i + 1] != MLX4_PORT_TYPE_ETH)
+ continue;
+ num_eth_ports++;
+ }
+ }
+
+ if (!num_eth_ports)
+ return;
+
+ /* have ETH ports. Alloc mailbox for SET_PORT command */
+ mailbox = mlx4_alloc_cmd_mailbox(dev);
+ if (IS_ERR(mailbox))
+ return;
+
+ for (i = 0; i < dev->caps.num_ports; i++) {
+ if (test_bit(i, actv_ports.ports)) {
+ if (dev->caps.port_type[i + 1] != MLX4_PORT_TYPE_ETH)
+ continue;
+ err = mlx4_reset_roce_port_gids(dev, slave, i + 1, mailbox);
+ if (err)
+ mlx4_warn(dev, "Could not reset ETH port GID table for slave %d, port %d (%d)\n",
+ slave, i + 1, err);
+ }
+ }
+
+ mlx4_free_cmd_mailbox(dev, mailbox);
+ return;
+}
+
static int mlx4_common_set_port(struct mlx4_dev *dev, int slave, u32 in_mod,
u8 op_mod, struct mlx4_cmd_mailbox *inbox)
{
@@ -692,10 +780,12 @@
/* 2. Check that do not have duplicates in OTHER
* entries in the port GID table
*/
+
+ mutex_lock(&(priv->port[port].gid_table.mutex));
for (i = 0; i < MLX4_ROCE_MAX_GIDS; i++) {
if (i >= base && i < base + num_gids)
continue; /* don't compare to slave's current gids */
- gid_entry_tbl = &priv->roce_gids[port - 1][i];
+ gid_entry_tbl = &priv->port[port].gid_table.roce_gids[i];
if (!memcmp(gid_entry_tbl->raw, zgid_entry.raw, sizeof(zgid_entry)))
continue;
gid_entry_mbox = (struct mlx4_roce_gid_entry *)(inbox->buf);
@@ -709,6 +799,7 @@
mlx4_warn(dev, "requested gid entry for slave:%d "
"is a duplicate of gid at index %d\n",
slave, i);
+ mutex_unlock(&(priv->port[port].gid_table.mutex));
return -EINVAL;
}
}
@@ -717,16 +808,24 @@
/* insert slave GIDs with memcpy, starting at slave's base index */
gid_entry_mbox = (struct mlx4_roce_gid_entry *)(inbox->buf);
for (i = 0, offset = base; i < num_gids; gid_entry_mbox++, offset++, i++)
- memcpy(priv->roce_gids[port - 1][offset].raw, gid_entry_mbox->raw, 16);
+ memcpy(priv->port[port].gid_table.roce_gids[offset].raw,
+ gid_entry_mbox->raw, MLX4_ROCE_GID_ENTRY_SIZE);
/* Now, copy roce port gids table to current mailbox for passing to FW */
gid_entry_mbox = (struct mlx4_roce_gid_entry *)(inbox->buf);
for (i = 0; i < MLX4_ROCE_MAX_GIDS; gid_entry_mbox++, i++)
- memcpy(gid_entry_mbox->raw, priv->roce_gids[port - 1][i].raw, 16);
+ memcpy(gid_entry_mbox->raw,
+ priv->port[port].gid_table.roce_gids[i].raw,
+ MLX4_ROCE_GID_ENTRY_SIZE);
- break;
+ err = mlx4_cmd(dev, inbox->dma, in_mod & 0xffff, op_mod,
+ MLX4_CMD_SET_PORT, MLX4_CMD_TIME_CLASS_B,
+ MLX4_CMD_NATIVE);
+ mutex_unlock(&(priv->port[port].gid_table.mutex));
+ return err;
}
- return mlx4_cmd(dev, inbox->dma, in_mod, op_mod,
+
+ return mlx4_cmd(dev, inbox->dma, in_mod & 0xffff, op_mod,
MLX4_CMD_SET_PORT, MLX4_CMD_TIME_CLASS_B,
MLX4_CMD_NATIVE);
}
@@ -1099,7 +1198,8 @@
num_vfs = bitmap_weight(slaves_pport.slaves, dev->num_vfs + 1) - 1;
for (i = 0; i < MLX4_ROCE_MAX_GIDS; i++) {
- if (!memcmp(priv->roce_gids[port - 1][i].raw, gid, 16)) {
+ if (!memcmp(priv->port[port].gid_table.roce_gids[i].raw, gid,
+ MLX4_ROCE_GID_ENTRY_SIZE)) {
found_ix = i;
break;
}
@@ -1187,7 +1287,8 @@
if (!mlx4_is_master(dev))
return -EINVAL;
- memcpy(gid, priv->roce_gids[port - 1][slave_id].raw, 16);
+ memcpy(gid, priv->port[port].gid_table.roce_gids[slave_id].raw,
+ MLX4_ROCE_GID_ENTRY_SIZE);
return 0;
}
EXPORT_SYMBOL(mlx4_get_roce_gid_from_slave);
diff --git a/drivers/net/ethernet/mellanox/mlx4/qp.c b/drivers/net/ethernet/mellanox/mlx4/qp.c
index fbd32af..40af619 100644
--- a/drivers/net/ethernet/mellanox/mlx4/qp.c
+++ b/drivers/net/ethernet/mellanox/mlx4/qp.c
@@ -272,29 +272,29 @@
}
EXPORT_SYMBOL_GPL(mlx4_qp_release_range);
-int __mlx4_qp_alloc_icm(struct mlx4_dev *dev, int qpn)
+int __mlx4_qp_alloc_icm(struct mlx4_dev *dev, int qpn, gfp_t gfp)
{
struct mlx4_priv *priv = mlx4_priv(dev);
struct mlx4_qp_table *qp_table = &priv->qp_table;
int err;
- err = mlx4_table_get(dev, &qp_table->qp_table, qpn);
+ err = mlx4_table_get(dev, &qp_table->qp_table, qpn, gfp);
if (err)
goto err_out;
- err = mlx4_table_get(dev, &qp_table->auxc_table, qpn);
+ err = mlx4_table_get(dev, &qp_table->auxc_table, qpn, gfp);
if (err)
goto err_put_qp;
- err = mlx4_table_get(dev, &qp_table->altc_table, qpn);
+ err = mlx4_table_get(dev, &qp_table->altc_table, qpn, gfp);
if (err)
goto err_put_auxc;
- err = mlx4_table_get(dev, &qp_table->rdmarc_table, qpn);
+ err = mlx4_table_get(dev, &qp_table->rdmarc_table, qpn, gfp);
if (err)
goto err_put_altc;
- err = mlx4_table_get(dev, &qp_table->cmpt_table, qpn);
+ err = mlx4_table_get(dev, &qp_table->cmpt_table, qpn, gfp);
if (err)
goto err_put_rdmarc;
@@ -316,7 +316,7 @@
return err;
}
-static int mlx4_qp_alloc_icm(struct mlx4_dev *dev, int qpn)
+static int mlx4_qp_alloc_icm(struct mlx4_dev *dev, int qpn, gfp_t gfp)
{
u64 param = 0;
@@ -326,7 +326,7 @@
MLX4_CMD_ALLOC_RES, MLX4_CMD_TIME_CLASS_A,
MLX4_CMD_WRAPPED);
}
- return __mlx4_qp_alloc_icm(dev, qpn);
+ return __mlx4_qp_alloc_icm(dev, qpn, gfp);
}
void __mlx4_qp_free_icm(struct mlx4_dev *dev, int qpn)
@@ -355,7 +355,7 @@
__mlx4_qp_free_icm(dev, qpn);
}
-int mlx4_qp_alloc(struct mlx4_dev *dev, int qpn, struct mlx4_qp *qp)
+int mlx4_qp_alloc(struct mlx4_dev *dev, int qpn, struct mlx4_qp *qp, gfp_t gfp)
{
struct mlx4_priv *priv = mlx4_priv(dev);
struct mlx4_qp_table *qp_table = &priv->qp_table;
@@ -366,7 +366,7 @@
qp->qpn = qpn;
- err = mlx4_qp_alloc_icm(dev, qpn);
+ err = mlx4_qp_alloc_icm(dev, qpn, gfp);
if (err)
return err;
diff --git a/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c b/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
index 8f1254a..2ba3b76 100644
--- a/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
+++ b/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
@@ -586,6 +586,7 @@
}
/* free master's vlans */
i = dev->caps.function;
+ mlx4_reset_roce_gids(dev, i);
mutex_lock(&priv->mfunc.master.res_tracker.slave_list[i].mutex);
rem_slave_vlans(dev, i);
mutex_unlock(&priv->mfunc.master.res_tracker.slave_list[i].mutex);
@@ -1532,7 +1533,7 @@
return err;
if (!fw_reserved(dev, qpn)) {
- err = __mlx4_qp_alloc_icm(dev, qpn);
+ err = __mlx4_qp_alloc_icm(dev, qpn, GFP_KERNEL);
if (err) {
res_abort_move(dev, slave, RES_QP, qpn);
return err;
@@ -1619,7 +1620,7 @@
if (err)
return err;
- err = __mlx4_mpt_alloc_icm(dev, mpt->key);
+ err = __mlx4_mpt_alloc_icm(dev, mpt->key, GFP_KERNEL);
if (err) {
res_abort_move(dev, slave, RES_MPT, id);
return err;
@@ -2827,10 +2828,12 @@
}
static int verify_qp_parameters(struct mlx4_dev *dev,
+ struct mlx4_vhcr *vhcr,
struct mlx4_cmd_mailbox *inbox,
enum qp_transition transition, u8 slave)
{
u32 qp_type;
+ u32 qpn;
struct mlx4_qp_context *qp_ctx;
enum mlx4_qp_optpar optpar;
int port;
@@ -2873,8 +2876,22 @@
default:
break;
}
-
break;
+
+ case MLX4_QP_ST_MLX:
+ qpn = vhcr->in_modifier & 0x7fffff;
+ port = (qp_ctx->pri_path.sched_queue >> 6 & 1) + 1;
+ if (transition == QP_TRANS_INIT2RTR &&
+ slave != mlx4_master_func_num(dev) &&
+ mlx4_is_qp_reserved(dev, qpn) &&
+ !mlx4_vf_smi_enabled(dev, slave, port)) {
+ /* only enabled VFs may create MLX proxy QPs */
+ mlx4_err(dev, "%s: unprivileged slave %d attempting to create an MLX proxy special QP on port %d\n",
+ __func__, slave, port);
+ return -EPERM;
+ }
+ break;
+
default:
break;
}
@@ -3454,7 +3471,7 @@
err = adjust_qp_sched_queue(dev, slave, qpc, inbox);
if (err)
return err;
- err = verify_qp_parameters(dev, inbox, QP_TRANS_INIT2RTR, slave);
+ err = verify_qp_parameters(dev, vhcr, inbox, QP_TRANS_INIT2RTR, slave);
if (err)
return err;
@@ -3508,7 +3525,7 @@
err = adjust_qp_sched_queue(dev, slave, context, inbox);
if (err)
return err;
- err = verify_qp_parameters(dev, inbox, QP_TRANS_RTR2RTS, slave);
+ err = verify_qp_parameters(dev, vhcr, inbox, QP_TRANS_RTR2RTS, slave);
if (err)
return err;
@@ -3530,7 +3547,7 @@
err = adjust_qp_sched_queue(dev, slave, context, inbox);
if (err)
return err;
- err = verify_qp_parameters(dev, inbox, QP_TRANS_RTS2RTS, slave);
+ err = verify_qp_parameters(dev, vhcr, inbox, QP_TRANS_RTS2RTS, slave);
if (err)
return err;
@@ -3567,7 +3584,7 @@
err = adjust_qp_sched_queue(dev, slave, context, inbox);
if (err)
return err;
- err = verify_qp_parameters(dev, inbox, QP_TRANS_SQD2SQD, slave);
+ err = verify_qp_parameters(dev, vhcr, inbox, QP_TRANS_SQD2SQD, slave);
if (err)
return err;
@@ -3589,7 +3606,7 @@
err = adjust_qp_sched_queue(dev, slave, context, inbox);
if (err)
return err;
- err = verify_qp_parameters(dev, inbox, QP_TRANS_SQD2RTS, slave);
+ err = verify_qp_parameters(dev, vhcr, inbox, QP_TRANS_SQD2RTS, slave);
if (err)
return err;
@@ -4681,7 +4698,7 @@
void mlx4_delete_all_resources_for_slave(struct mlx4_dev *dev, int slave)
{
struct mlx4_priv *priv = mlx4_priv(dev);
-
+ mlx4_reset_roce_gids(dev, slave);
mutex_lock(&priv->mfunc.master.res_tracker.slave_list[slave].mutex);
rem_slave_vlans(dev, slave);
rem_slave_macs(dev, slave);
diff --git a/drivers/net/ethernet/mellanox/mlx4/srq.c b/drivers/net/ethernet/mellanox/mlx4/srq.c
index 98faf87..6714662 100644
--- a/drivers/net/ethernet/mellanox/mlx4/srq.c
+++ b/drivers/net/ethernet/mellanox/mlx4/srq.c
@@ -103,11 +103,11 @@
if (*srqn == -1)
return -ENOMEM;
- err = mlx4_table_get(dev, &srq_table->table, *srqn);
+ err = mlx4_table_get(dev, &srq_table->table, *srqn, GFP_KERNEL);
if (err)
goto err_out;
- err = mlx4_table_get(dev, &srq_table->cmpt_table, *srqn);
+ err = mlx4_table_get(dev, &srq_table->cmpt_table, *srqn, GFP_KERNEL);
if (err)
goto err_put;
return 0;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mr.c b/drivers/net/ethernet/mellanox/mlx5/core/mr.c
index 4cc9276..ac52a0f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/mr.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/mr.c
@@ -82,7 +82,11 @@
return mlx5_cmd_status_to_err(&lout.hdr);
}
+ mr->iova = be64_to_cpu(in->seg.start_addr);
+ mr->size = be64_to_cpu(in->seg.len);
mr->key = mlx5_idx_to_mkey(be32_to_cpu(lout.mkey) & 0xffffff) | key;
+ mr->pd = be32_to_cpu(in->seg.flags_pd) & 0xffffff;
+
mlx5_core_dbg(dev, "out 0x%x, key 0x%x, mkey 0x%x\n",
be32_to_cpu(lout.mkey), key, mr->key);
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_dcb.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_dcb.c
index a51fe18..561cb11 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_dcb.c
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_dcb.c
@@ -1020,6 +1020,7 @@
struct qlcnic_dcb_cee *peer;
int i;
+ memset(info, 0, sizeof(*info));
*app_count = 0;
if (!test_bit(QLCNIC_DCB_STATE, &adapter->dcb->state))
diff --git a/drivers/net/team/team.c b/drivers/net/team/team.c
index 767fe61..ce4989b 100644
--- a/drivers/net/team/team.c
+++ b/drivers/net/team/team.c
@@ -1724,6 +1724,7 @@
* to traverse list in reverse under rcu_read_lock
*/
mutex_lock(&team->lock);
+ team->port_mtu_change_allowed = true;
list_for_each_entry(port, &team->port_list, list) {
err = dev_set_mtu(port->dev, new_mtu);
if (err) {
@@ -1732,6 +1733,7 @@
goto unwind;
}
}
+ team->port_mtu_change_allowed = false;
mutex_unlock(&team->lock);
dev->mtu = new_mtu;
@@ -1741,6 +1743,7 @@
unwind:
list_for_each_entry_continue_reverse(port, &team->port_list, list)
dev_set_mtu(port->dev, dev->mtu);
+ team->port_mtu_change_allowed = false;
mutex_unlock(&team->lock);
return err;
@@ -2851,7 +2854,9 @@
break;
case NETDEV_PRECHANGEMTU:
/* Forbid to change mtu of underlaying device */
- return NOTIFY_BAD;
+ if (!port->team->port_mtu_change_allowed)
+ return NOTIFY_BAD;
+ break;
case NETDEV_PRE_TYPE_CHANGE:
/* Forbid to change type of underlaying device */
return NOTIFY_BAD;
diff --git a/drivers/net/usb/ipheth.c b/drivers/net/usb/ipheth.c
index 421934c..973275f 100644
--- a/drivers/net/usb/ipheth.c
+++ b/drivers/net/usb/ipheth.c
@@ -59,6 +59,8 @@
#define USB_PRODUCT_IPHONE_3GS 0x1294
#define USB_PRODUCT_IPHONE_4 0x1297
#define USB_PRODUCT_IPAD 0x129a
+#define USB_PRODUCT_IPAD_2 0x12a2
+#define USB_PRODUCT_IPAD_3 0x12a6
#define USB_PRODUCT_IPAD_MINI 0x12ab
#define USB_PRODUCT_IPHONE_4_VZW 0x129c
#define USB_PRODUCT_IPHONE_4S 0x12a0
@@ -107,6 +109,14 @@
IPHETH_USBINTF_CLASS, IPHETH_USBINTF_SUBCLASS,
IPHETH_USBINTF_PROTO) },
{ USB_DEVICE_AND_INTERFACE_INFO(
+ USB_VENDOR_APPLE, USB_PRODUCT_IPAD_2,
+ IPHETH_USBINTF_CLASS, IPHETH_USBINTF_SUBCLASS,
+ IPHETH_USBINTF_PROTO) },
+ { USB_DEVICE_AND_INTERFACE_INFO(
+ USB_VENDOR_APPLE, USB_PRODUCT_IPAD_3,
+ IPHETH_USBINTF_CLASS, IPHETH_USBINTF_SUBCLASS,
+ IPHETH_USBINTF_PROTO) },
+ { USB_DEVICE_AND_INTERFACE_INFO(
USB_VENDOR_APPLE, USB_PRODUCT_IPAD_MINI,
IPHETH_USBINTF_CLASS, IPHETH_USBINTF_SUBCLASS,
IPHETH_USBINTF_PROTO) },
diff --git a/drivers/net/usb/qmi_wwan.c b/drivers/net/usb/qmi_wwan.c
index 83208d4..dc4bf06 100644
--- a/drivers/net/usb/qmi_wwan.c
+++ b/drivers/net/usb/qmi_wwan.c
@@ -748,11 +748,15 @@
{QMI_FIXED_INTF(0x1199, 0x68a2, 19)}, /* Sierra Wireless MC7710 in QMI mode */
{QMI_FIXED_INTF(0x1199, 0x68c0, 8)}, /* Sierra Wireless MC73xx */
{QMI_FIXED_INTF(0x1199, 0x68c0, 10)}, /* Sierra Wireless MC73xx */
- {QMI_FIXED_INTF(0x1199, 0x68c0, 11)}, /* Sierra Wireless MC73xx */
{QMI_FIXED_INTF(0x1199, 0x901c, 8)}, /* Sierra Wireless EM7700 */
{QMI_FIXED_INTF(0x1199, 0x901f, 8)}, /* Sierra Wireless EM7355 */
{QMI_FIXED_INTF(0x1199, 0x9041, 8)}, /* Sierra Wireless MC7305/MC7355 */
{QMI_FIXED_INTF(0x1199, 0x9051, 8)}, /* Netgear AirCard 340U */
+ {QMI_FIXED_INTF(0x1199, 0x9053, 8)}, /* Sierra Wireless Modem */
+ {QMI_FIXED_INTF(0x1199, 0x9054, 8)}, /* Sierra Wireless Modem */
+ {QMI_FIXED_INTF(0x1199, 0x9055, 8)}, /* Netgear AirCard 341U */
+ {QMI_FIXED_INTF(0x1199, 0x9056, 8)}, /* Sierra Wireless Modem */
+ {QMI_FIXED_INTF(0x1199, 0x9061, 8)}, /* Sierra Wireless Modem */
{QMI_FIXED_INTF(0x1bbb, 0x011e, 4)}, /* Telekom Speedstick LTE II (Alcatel One Touch L100V LTE) */
{QMI_FIXED_INTF(0x1bbb, 0x0203, 2)}, /* Alcatel L800MA */
{QMI_FIXED_INTF(0x2357, 0x0201, 4)}, /* TP-LINK HSUPA Modem MA180 */
diff --git a/drivers/net/wireless/iwlwifi/mvm/mac80211.c b/drivers/net/wireless/iwlwifi/mvm/mac80211.c
index b41dc84..8735ef1 100644
--- a/drivers/net/wireless/iwlwifi/mvm/mac80211.c
+++ b/drivers/net/wireless/iwlwifi/mvm/mac80211.c
@@ -826,7 +826,7 @@
if (ret)
goto out_remove_mac;
- if (!mvm->bf_allowed_vif &&
+ if (!mvm->bf_allowed_vif && false &&
vif->type == NL80211_IFTYPE_STATION && !vif->p2p &&
mvm->fw->ucode_capa.flags & IWL_UCODE_TLV_FLAGS_BF_UPDATED){
mvm->bf_allowed_vif = mvmvif;
diff --git a/drivers/net/wireless/rsi/rsi_91x_sdio.c b/drivers/net/wireless/rsi/rsi_91x_sdio.c
index 2e39d38..46e7af4 100644
--- a/drivers/net/wireless/rsi/rsi_91x_sdio.c
+++ b/drivers/net/wireless/rsi/rsi_91x_sdio.c
@@ -285,7 +285,6 @@
if (err) {
rsi_dbg(ERR_ZONE, "%s: CCCR speed reg read failed: %d\n",
__func__, err);
- card->state &= ~MMC_STATE_HIGHSPEED;
} else {
err = rsi_cmd52writebyte(card,
SDIO_CCCR_SPEED,
@@ -296,14 +295,13 @@
__func__, err);
return;
}
- mmc_card_set_highspeed(card);
host->ios.timing = MMC_TIMING_SD_HS;
host->ops->set_ios(host, &host->ios);
}
}
/* Set clock */
- if (mmc_card_highspeed(card))
+ if (mmc_card_hs(card))
clock = 50000000;
else
clock = card->cis.max_dtr;
diff --git a/drivers/of/address.c b/drivers/of/address.c
index 95351b2..5edfcb0 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -701,3 +701,113 @@
return ioremap(res.start, resource_size(&res));
}
EXPORT_SYMBOL(of_iomap);
+
+/**
+ * of_dma_get_range - Get DMA range info
+ * @np: device node to get DMA range info
+ * @dma_addr: pointer to store initial DMA address of DMA range
+ * @paddr: pointer to store initial CPU address of DMA range
+ * @size: pointer to store size of DMA range
+ *
+ * Look in bottom up direction for the first "dma-ranges" property
+ * and parse it.
+ * dma-ranges format:
+ * DMA addr (dma_addr) : naddr cells
+ * CPU addr (phys_addr_t) : pna cells
+ * size : nsize cells
+ *
+ * It returns -ENODEV if "dma-ranges" property was not found
+ * for this device in DT.
+ */
+int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size)
+{
+ struct device_node *node = of_node_get(np);
+ const __be32 *ranges = NULL;
+ int len, naddr, nsize, pna;
+ int ret = 0;
+ u64 dmaaddr;
+
+ if (!node)
+ return -EINVAL;
+
+ while (1) {
+ naddr = of_n_addr_cells(node);
+ nsize = of_n_size_cells(node);
+ node = of_get_next_parent(node);
+ if (!node)
+ break;
+
+ ranges = of_get_property(node, "dma-ranges", &len);
+
+ /* Ignore empty ranges, they imply no translation required */
+ if (ranges && len > 0)
+ break;
+
+ /*
+ * At least empty ranges has to be defined for parent node if
+ * DMA is supported
+ */
+ if (!ranges)
+ break;
+ }
+
+ if (!ranges) {
+ pr_debug("%s: no dma-ranges found for node(%s)\n",
+ __func__, np->full_name);
+ ret = -ENODEV;
+ goto out;
+ }
+
+ len /= sizeof(u32);
+
+ pna = of_n_addr_cells(node);
+
+ /* dma-ranges format:
+ * DMA addr : naddr cells
+ * CPU addr : pna cells
+ * size : nsize cells
+ */
+ dmaaddr = of_read_number(ranges, naddr);
+ *paddr = of_translate_dma_address(np, ranges);
+ if (*paddr == OF_BAD_ADDR) {
+ pr_err("%s: translation of DMA address(%pad) to CPU address failed node(%s)\n",
+ __func__, dma_addr, np->full_name);
+ ret = -EINVAL;
+ goto out;
+ }
+ *dma_addr = dmaaddr;
+
+ *size = of_read_number(ranges + naddr + pna, nsize);
+
+ pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
+ *dma_addr, *paddr, *size);
+
+out:
+ of_node_put(node);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(of_dma_get_range);
+
+/**
+ * of_dma_is_coherent - Check if device is coherent
+ * @np: device node
+ *
+ * It returns true if "dma-coherent" property was found
+ * for this device in DT.
+ */
+bool of_dma_is_coherent(struct device_node *np)
+{
+ struct device_node *node = of_node_get(np);
+
+ while (node) {
+ if (of_property_read_bool(node, "dma-coherent")) {
+ of_node_put(node);
+ return true;
+ }
+ node = of_get_next_parent(node);
+ }
+ of_node_put(node);
+ return false;
+}
+EXPORT_SYMBOL_GPL(of_dma_is_coherent);
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 92c060e..6c48d73 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -138,9 +138,6 @@
}
dev->dev.of_node = of_node_get(np);
-#if defined(CONFIG_MICROBLAZE)
- dev->dev.dma_mask = &dev->archdata.dma_mask;
-#endif
dev->dev.parent = parent;
if (bus_id)
@@ -153,6 +150,64 @@
EXPORT_SYMBOL(of_device_alloc);
/**
+ * of_dma_configure - Setup DMA configuration
+ * @dev: Device to apply DMA configuration
+ *
+ * Try to get devices's DMA configuration from DT and update it
+ * accordingly.
+ *
+ * In case if platform code need to use own special DMA configuration,it
+ * can use Platform bus notifier and handle BUS_NOTIFY_ADD_DEVICE event
+ * to fix up DMA configuration.
+ */
+static void of_dma_configure(struct platform_device *pdev)
+{
+ u64 dma_addr, paddr, size;
+ int ret;
+ struct device *dev = &pdev->dev;
+
+#if defined(CONFIG_MICROBLAZE)
+ pdev->archdata.dma_mask = 0xffffffffUL;
+#endif
+
+ /*
+ * Set default dma-mask to 32 bit. Drivers are expected to setup
+ * the correct supported dma_mask.
+ */
+ dev->coherent_dma_mask = DMA_BIT_MASK(32);
+
+ /*
+ * Set it to coherent_dma_mask by default if the architecture
+ * code has not set it.
+ */
+ if (!dev->dma_mask)
+ dev->dma_mask = &dev->coherent_dma_mask;
+
+ /*
+ * if dma-coherent property exist, call arch hook to setup
+ * dma coherent operations.
+ */
+ if (of_dma_is_coherent(dev->of_node)) {
+ set_arch_dma_coherent_ops(dev);
+ dev_dbg(dev, "device is dma coherent\n");
+ }
+
+ /*
+ * if dma-ranges property doesn't exist - just return else
+ * setup the dma offset
+ */
+ ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size);
+ if (ret < 0) {
+ dev_dbg(dev, "no dma range information to setup\n");
+ return;
+ }
+
+ /* DMA ranges found. Calculate and set dma_pfn_offset */
+ dev->dma_pfn_offset = PFN_DOWN(paddr - dma_addr);
+ dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", dev->dma_pfn_offset);
+}
+
+/**
* of_platform_device_create_pdata - Alloc, initialize and register an of_device
* @np: pointer to node to create device for
* @bus_id: name to assign device
@@ -178,12 +233,7 @@
if (!dev)
goto err_clear_flag;
-#if defined(CONFIG_MICROBLAZE)
- dev->archdata.dma_mask = 0xffffffffUL;
-#endif
- dev->dev.coherent_dma_mask = DMA_BIT_MASK(32);
- if (!dev->dev.dma_mask)
- dev->dev.dma_mask = &dev->dev.coherent_dma_mask;
+ of_dma_configure(dev);
dev->dev.bus = &platform_bus_type;
dev->dev.platform_data = platform_data;
diff --git a/drivers/parport/procfs.c b/drivers/parport/procfs.c
index 92ed045..3b47080 100644
--- a/drivers/parport/procfs.c
+++ b/drivers/parport/procfs.c
@@ -31,7 +31,7 @@
#define PARPORT_MIN_SPINTIME_VALUE 1
#define PARPORT_MAX_SPINTIME_VALUE 1000
-static int do_active_device(ctl_table *table, int write,
+static int do_active_device(struct ctl_table *table, int write,
void __user *result, size_t *lenp, loff_t *ppos)
{
struct parport *port = (struct parport *)table->extra1;
@@ -68,7 +68,7 @@
}
#ifdef CONFIG_PARPORT_1284
-static int do_autoprobe(ctl_table *table, int write,
+static int do_autoprobe(struct ctl_table *table, int write,
void __user *result, size_t *lenp, loff_t *ppos)
{
struct parport_device_info *info = table->extra2;
@@ -110,9 +110,9 @@
}
#endif /* IEEE1284.3 support. */
-static int do_hardware_base_addr (ctl_table *table, int write,
- void __user *result,
- size_t *lenp, loff_t *ppos)
+static int do_hardware_base_addr(struct ctl_table *table, int write,
+ void __user *result,
+ size_t *lenp, loff_t *ppos)
{
struct parport *port = (struct parport *)table->extra1;
char buffer[20];
@@ -138,9 +138,9 @@
return copy_to_user(result, buffer, len) ? -EFAULT : 0;
}
-static int do_hardware_irq (ctl_table *table, int write,
- void __user *result,
- size_t *lenp, loff_t *ppos)
+static int do_hardware_irq(struct ctl_table *table, int write,
+ void __user *result,
+ size_t *lenp, loff_t *ppos)
{
struct parport *port = (struct parport *)table->extra1;
char buffer[20];
@@ -166,9 +166,9 @@
return copy_to_user(result, buffer, len) ? -EFAULT : 0;
}
-static int do_hardware_dma (ctl_table *table, int write,
- void __user *result,
- size_t *lenp, loff_t *ppos)
+static int do_hardware_dma(struct ctl_table *table, int write,
+ void __user *result,
+ size_t *lenp, loff_t *ppos)
{
struct parport *port = (struct parport *)table->extra1;
char buffer[20];
@@ -194,9 +194,9 @@
return copy_to_user(result, buffer, len) ? -EFAULT : 0;
}
-static int do_hardware_modes (ctl_table *table, int write,
- void __user *result,
- size_t *lenp, loff_t *ppos)
+static int do_hardware_modes(struct ctl_table *table, int write,
+ void __user *result,
+ size_t *lenp, loff_t *ppos)
{
struct parport *port = (struct parport *)table->extra1;
char buffer[40];
@@ -255,11 +255,11 @@
struct parport_sysctl_table {
struct ctl_table_header *sysctl_header;
- ctl_table vars[12];
- ctl_table device_dir[2];
- ctl_table port_dir[2];
- ctl_table parport_dir[2];
- ctl_table dev_dir[2];
+ struct ctl_table vars[12];
+ struct ctl_table device_dir[2];
+ struct ctl_table port_dir[2];
+ struct ctl_table parport_dir[2];
+ struct ctl_table dev_dir[2];
};
static const struct parport_sysctl_table parport_sysctl_template = {
@@ -369,12 +369,12 @@
struct parport_device_sysctl_table
{
struct ctl_table_header *sysctl_header;
- ctl_table vars[2];
- ctl_table device_dir[2];
- ctl_table devices_root_dir[2];
- ctl_table port_dir[2];
- ctl_table parport_dir[2];
- ctl_table dev_dir[2];
+ struct ctl_table vars[2];
+ struct ctl_table device_dir[2];
+ struct ctl_table devices_root_dir[2];
+ struct ctl_table port_dir[2];
+ struct ctl_table parport_dir[2];
+ struct ctl_table dev_dir[2];
};
static const struct parport_device_sysctl_table
@@ -422,10 +422,10 @@
struct parport_default_sysctl_table
{
struct ctl_table_header *sysctl_header;
- ctl_table vars[3];
- ctl_table default_dir[2];
- ctl_table parport_dir[2];
- ctl_table dev_dir[2];
+ struct ctl_table vars[3];
+ struct ctl_table default_dir[2];
+ struct ctl_table parport_dir[2];
+ struct ctl_table dev_dir[2];
};
static struct parport_default_sysctl_table
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 436a76a..212c63d 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3179,14 +3179,7 @@
return 0;
}
-/**
- * pci_reset_bridge_secondary_bus - Reset the secondary bus on a PCI bridge.
- * @dev: Bridge device
- *
- * Use the bridge control register to assert reset on the secondary bus.
- * Devices on the secondary bus are left in power-on state.
- */
-void pci_reset_bridge_secondary_bus(struct pci_dev *dev)
+void __weak pcibios_reset_secondary_bus(struct pci_dev *dev)
{
u16 ctrl;
@@ -3211,6 +3204,18 @@
*/
ssleep(1);
}
+
+/**
+ * pci_reset_bridge_secondary_bus - Reset the secondary bus on a PCI bridge.
+ * @dev: Bridge device
+ *
+ * Use the bridge control register to assert reset on the secondary bus.
+ * Devices on the secondary bus are left in power-on state.
+ */
+void pci_reset_bridge_secondary_bus(struct pci_dev *dev)
+{
+ pcibios_reset_secondary_bus(dev);
+}
EXPORT_SYMBOL_GPL(pci_reset_bridge_secondary_bus);
static int pci_parent_bus_reset(struct pci_dev *dev, int probe)
diff --git a/drivers/platform/chrome/chromeos_laptop.c b/drivers/platform/chrome/chromeos_laptop.c
index 7f3aad0..7f1a2e2 100644
--- a/drivers/platform/chrome/chromeos_laptop.c
+++ b/drivers/platform/chrome/chromeos_laptop.c
@@ -84,21 +84,19 @@
I2C_BOARD_INFO("tsl2563", TAOS_ALS_I2C_ADDR),
};
+static int mxt_t19_keys[] = {
+ KEY_RESERVED,
+ KEY_RESERVED,
+ KEY_RESERVED,
+ KEY_RESERVED,
+ KEY_RESERVED,
+ BTN_LEFT
+};
+
static struct mxt_platform_data atmel_224s_tp_platform_data = {
- .x_line = 18,
- .y_line = 12,
- .x_size = 102*20,
- .y_size = 68*20,
- .blen = 0x80, /* Gain setting is in upper 4 bits */
- .threshold = 0x32,
- .voltage = 0, /* 3.3V */
- .orient = MXT_VERTICAL_FLIP,
.irqflags = IRQF_TRIGGER_FALLING,
- .is_tp = true,
- .key_map = { KEY_RESERVED,
- KEY_RESERVED,
- KEY_RESERVED,
- BTN_LEFT },
+ .t19_num_keys = ARRAY_SIZE(mxt_t19_keys),
+ .t19_keymap = mxt_t19_keys,
.config = NULL,
.config_length = 0,
};
@@ -110,16 +108,7 @@
};
static struct mxt_platform_data atmel_1664s_platform_data = {
- .x_line = 32,
- .y_line = 50,
- .x_size = 1700,
- .y_size = 2560,
- .blen = 0x89, /* Gain setting is in upper 4 bits */
- .threshold = 0x28,
- .voltage = 0, /* 3.3V */
- .orient = MXT_ROTATED_90_COUNTER,
.irqflags = IRQF_TRIGGER_FALLING,
- .is_tp = false,
.config = NULL,
.config_length = 0,
};
diff --git a/drivers/platform/x86/Kconfig b/drivers/platform/x86/Kconfig
index 27df2c5..172f26c 100644
--- a/drivers/platform/x86/Kconfig
+++ b/drivers/platform/x86/Kconfig
@@ -102,7 +102,7 @@
default n
---help---
This driver adds support for rfkill and backlight control to Dell
- laptops.
+ laptops (except for some models covered by the Compal driver).
config DELL_WMI
tristate "Dell WMI extras"
@@ -127,6 +127,16 @@
To compile this driver as a module, choose M here: the module will
be called dell-wmi-aio.
+config DELL_SMO8800
+ tristate "Dell Latitude freefall driver (ACPI SMO8800/SMO8810)"
+ depends on ACPI
+ ---help---
+ Say Y here if you want to support SMO8800/SMO8810 freefall device
+ on Dell Latitude laptops.
+
+ To compile this driver as a module, choose M here: the module will
+ be called dell-smo8800.
+
config FUJITSU_LAPTOP
tristate "Fujitsu Laptop Extras"
@@ -265,23 +275,21 @@
R2, R3, R5, T2, W2 and Y2 series), say Y.
config COMPAL_LAPTOP
- tristate "Compal Laptop Extras"
+ tristate "Compal (and others) Laptop Extras"
depends on ACPI
depends on BACKLIGHT_CLASS_DEVICE
depends on RFKILL
depends on HWMON
depends on POWER_SUPPLY
---help---
- This is a driver for laptops built by Compal:
+ This is a driver for laptops built by Compal, and some models by
+ other brands (e.g. Dell, Toshiba).
- Compal FL90/IFL90
- Compal FL91/IFL91
- Compal FL92/JFL92
- Compal FT00/IFT00
+ It adds support for rfkill, Bluetooth, WLAN and LCD brightness
+ control.
- It adds support for Bluetooth, WLAN and LCD brightness control.
-
- If you have an Compal FL9x/IFL9x/FT00 laptop, say Y or M here.
+ For a (possibly incomplete) list of supported laptops, please refer
+ to: Documentation/platform/x86-laptop-drivers.txt
config SONY_LAPTOP
tristate "Sony Laptop Extras"
@@ -724,7 +732,7 @@
config XO1_RFKILL
tristate "OLPC XO-1 software RF kill switch"
- depends on OLPC
+ depends on OLPC || COMPILE_TEST
depends on RFKILL
---help---
Support for enabling/disabling the WLAN interface on the OLPC XO-1
@@ -732,6 +740,7 @@
config XO15_EBOOK
tristate "OLPC XO-1.5 ebook switch"
+ depends on OLPC || COMPILE_TEST
depends on ACPI && INPUT
---help---
Support for the ebook switch on the OLPC XO-1.5 laptop.
diff --git a/drivers/platform/x86/Makefile b/drivers/platform/x86/Makefile
index 1a2eafc9..c4ca428 100644
--- a/drivers/platform/x86/Makefile
+++ b/drivers/platform/x86/Makefile
@@ -13,6 +13,7 @@
obj-$(CONFIG_DELL_LAPTOP) += dell-laptop.o
obj-$(CONFIG_DELL_WMI) += dell-wmi.o
obj-$(CONFIG_DELL_WMI_AIO) += dell-wmi-aio.o
+obj-$(CONFIG_DELL_SMO8800) += dell-smo8800.o
obj-$(CONFIG_ACER_WMI) += acer-wmi.o
obj-$(CONFIG_ACERHDF) += acerhdf.o
obj-$(CONFIG_HP_ACCEL) += hp_accel.o
diff --git a/drivers/platform/x86/alienware-wmi.c b/drivers/platform/x86/alienware-wmi.c
index 541f9514..297b664 100644
--- a/drivers/platform/x86/alienware-wmi.c
+++ b/drivers/platform/x86/alienware-wmi.c
@@ -32,6 +32,7 @@
#define WMAX_METHOD_HDMI_STATUS 0x2
#define WMAX_METHOD_BRIGHTNESS 0x3
#define WMAX_METHOD_ZONE_CONTROL 0x4
+#define WMAX_METHOD_HDMI_CABLE 0x5
MODULE_AUTHOR("Mario Limonciello <mario_limonciello@dell.com>");
MODULE_DESCRIPTION("Alienware special feature control");
@@ -350,12 +351,11 @@
char *name;
if (interface == WMAX) {
- global_led.max_brightness = 100;
lighting_control_state = WMAX_RUNNING;
} else if (interface == LEGACY) {
- global_led.max_brightness = 0x0F;
lighting_control_state = LEGACY_RUNNING;
}
+ global_led.max_brightness = 0x0F;
global_brightness = global_led.max_brightness;
/*
@@ -423,41 +423,85 @@
The HDMI mux sysfs node indicates the status of the HDMI input mux.
It can toggle between standard system GPU output and HDMI input.
*/
-static ssize_t show_hdmi(struct device *dev, struct device_attribute *attr,
- char *buf)
+static acpi_status alienware_hdmi_command(struct hdmi_args *in_args,
+ u32 command, int *out_data)
{
acpi_status status;
- struct acpi_buffer input;
union acpi_object *obj;
- u32 tmp = 0;
- struct acpi_buffer output = { ACPI_ALLOCATE_BUFFER, NULL };
+ struct acpi_buffer input;
+ struct acpi_buffer output;
+
+ input.length = (acpi_size) sizeof(*in_args);
+ input.pointer = in_args;
+ if (out_data != NULL) {
+ output.length = ACPI_ALLOCATE_BUFFER;
+ output.pointer = NULL;
+ status = wmi_evaluate_method(WMAX_CONTROL_GUID, 1,
+ command, &input, &output);
+ } else
+ status = wmi_evaluate_method(WMAX_CONTROL_GUID, 1,
+ command, &input, NULL);
+
+ if (ACPI_SUCCESS(status) && out_data != NULL) {
+ obj = (union acpi_object *)output.pointer;
+ if (obj && obj->type == ACPI_TYPE_INTEGER)
+ *out_data = (u32) obj->integer.value;
+ }
+ return status;
+
+}
+
+static ssize_t show_hdmi_cable(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ acpi_status status;
+ u32 out_data;
struct hdmi_args in_args = {
.arg = 0,
};
- input.length = (acpi_size) sizeof(in_args);
- input.pointer = &in_args;
- status = wmi_evaluate_method(WMAX_CONTROL_GUID, 1,
- WMAX_METHOD_HDMI_STATUS, &input, &output);
+ status =
+ alienware_hdmi_command(&in_args, WMAX_METHOD_HDMI_CABLE,
+ (u32 *) &out_data);
+ if (ACPI_SUCCESS(status)) {
+ if (out_data == 0)
+ return scnprintf(buf, PAGE_SIZE,
+ "[unconnected] connected unknown\n");
+ else if (out_data == 1)
+ return scnprintf(buf, PAGE_SIZE,
+ "unconnected [connected] unknown\n");
+ }
+ pr_err("alienware-wmi: unknown HDMI cable status: %d\n", status);
+ return scnprintf(buf, PAGE_SIZE, "unconnected connected [unknown]\n");
+}
+
+static ssize_t show_hdmi_source(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ acpi_status status;
+ u32 out_data;
+ struct hdmi_args in_args = {
+ .arg = 0,
+ };
+ status =
+ alienware_hdmi_command(&in_args, WMAX_METHOD_HDMI_STATUS,
+ (u32 *) &out_data);
if (ACPI_SUCCESS(status)) {
- obj = (union acpi_object *)output.pointer;
- if (obj && obj->type == ACPI_TYPE_INTEGER)
- tmp = (u32) obj->integer.value;
- if (tmp == 1)
+ if (out_data == 1)
return scnprintf(buf, PAGE_SIZE,
"[input] gpu unknown\n");
- else if (tmp == 2)
+ else if (out_data == 2)
return scnprintf(buf, PAGE_SIZE,
"input [gpu] unknown\n");
}
- pr_err("alienware-wmi: unknown HDMI status: %d\n", status);
+ pr_err("alienware-wmi: unknown HDMI source status: %d\n", out_data);
return scnprintf(buf, PAGE_SIZE, "input gpu [unknown]\n");
}
-static ssize_t toggle_hdmi(struct device *dev, struct device_attribute *attr,
- const char *buf, size_t count)
+static ssize_t toggle_hdmi_source(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
{
- struct acpi_buffer input;
acpi_status status;
struct hdmi_args args;
if (strcmp(buf, "gpu\n") == 0)
@@ -467,33 +511,46 @@
else
args.arg = 3;
pr_debug("alienware-wmi: setting hdmi to %d : %s", args.arg, buf);
- input.length = (acpi_size) sizeof(args);
- input.pointer = &args;
- status = wmi_evaluate_method(WMAX_CONTROL_GUID, 1,
- WMAX_METHOD_HDMI_SOURCE, &input, NULL);
+
+ status = alienware_hdmi_command(&args, WMAX_METHOD_HDMI_SOURCE, NULL);
+
if (ACPI_FAILURE(status))
pr_err("alienware-wmi: HDMI toggle failed: results: %u\n",
status);
return count;
}
-static DEVICE_ATTR(hdmi, S_IRUGO | S_IWUSR, show_hdmi, toggle_hdmi);
+static DEVICE_ATTR(cable, S_IRUGO, show_hdmi_cable, NULL);
+static DEVICE_ATTR(source, S_IRUGO | S_IWUSR, show_hdmi_source,
+ toggle_hdmi_source);
-static void remove_hdmi(struct platform_device *device)
+static struct attribute *hdmi_attrs[] = {
+ &dev_attr_cable.attr,
+ &dev_attr_source.attr,
+ NULL,
+};
+
+static struct attribute_group hdmi_attribute_group = {
+ .name = "hdmi",
+ .attrs = hdmi_attrs,
+};
+
+static void remove_hdmi(struct platform_device *dev)
{
- device_remove_file(&device->dev, &dev_attr_hdmi);
+ sysfs_remove_group(&dev->dev.kobj, &hdmi_attribute_group);
}
-static int create_hdmi(void)
+static int create_hdmi(struct platform_device *dev)
{
- int ret = -ENOMEM;
- ret = device_create_file(&platform_device->dev, &dev_attr_hdmi);
+ int ret;
+
+ ret = sysfs_create_group(&dev->dev.kobj, &hdmi_attribute_group);
if (ret)
goto error_create_hdmi;
return 0;
error_create_hdmi:
- remove_hdmi(platform_device);
+ remove_hdmi(dev);
return ret;
}
@@ -527,7 +584,7 @@
goto fail_platform_device2;
if (interface == WMAX) {
- ret = create_hdmi();
+ ret = create_hdmi(platform_device);
if (ret)
goto fail_prep_hdmi;
}
diff --git a/drivers/platform/x86/asus-nb-wmi.c b/drivers/platform/x86/asus-nb-wmi.c
index 563f59e..ddf0eef 100644
--- a/drivers/platform/x86/asus-nb-wmi.c
+++ b/drivers/platform/x86/asus-nb-wmi.c
@@ -137,6 +137,15 @@
},
{
.callback = dmi_matched,
+ .ident = "ASUSTeK COMPUTER INC. X550CA",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "X550CA"),
+ },
+ .driver_data = &quirk_asus_x401u,
+ },
+ {
+ .callback = dmi_matched,
.ident = "ASUSTeK COMPUTER INC. X55A",
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."),
diff --git a/drivers/platform/x86/asus-wmi.c b/drivers/platform/x86/asus-wmi.c
index 91ef69a..3c6cced 100644
--- a/drivers/platform/x86/asus-wmi.c
+++ b/drivers/platform/x86/asus-wmi.c
@@ -266,7 +266,7 @@
struct acpi_buffer output = { ACPI_ALLOCATE_BUFFER, NULL };
acpi_status status;
union acpi_object *obj;
- u32 tmp;
+ u32 tmp = 0;
status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 1, method_id,
&input, &output);
@@ -277,8 +277,6 @@
obj = (union acpi_object *)output.pointer;
if (obj && obj->type == ACPI_TYPE_INTEGER)
tmp = (u32) obj->integer.value;
- else
- tmp = 0;
if (retval)
*retval = tmp;
diff --git a/drivers/platform/x86/dell-smo8800.c b/drivers/platform/x86/dell-smo8800.c
new file mode 100644
index 0000000..a653716
--- /dev/null
+++ b/drivers/platform/x86/dell-smo8800.c
@@ -0,0 +1,233 @@
+/*
+ * dell-smo8800.c - Dell Latitude ACPI SMO8800/SMO8810 freefall sensor driver
+ *
+ * Copyright (C) 2012 Sonal Santan <sonal.santan@gmail.com>
+ * Copyright (C) 2014 Pali Rohár <pali.rohar@gmail.com>
+ *
+ * This is loosely based on lis3lv02d driver.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#define DRIVER_NAME "smo8800"
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/acpi.h>
+#include <linux/interrupt.h>
+#include <linux/miscdevice.h>
+
+struct smo8800_device {
+ u32 irq; /* acpi device irq */
+ atomic_t counter; /* count after last read */
+ struct miscdevice miscdev; /* for /dev/freefall */
+ unsigned long misc_opened; /* whether the device is open */
+ wait_queue_head_t misc_wait; /* Wait queue for the misc dev */
+ struct device *dev; /* acpi device */
+};
+
+static irqreturn_t smo8800_interrupt_quick(int irq, void *data)
+{
+ struct smo8800_device *smo8800 = data;
+
+ atomic_inc(&smo8800->counter);
+ wake_up_interruptible(&smo8800->misc_wait);
+ return IRQ_WAKE_THREAD;
+}
+
+static irqreturn_t smo8800_interrupt_thread(int irq, void *data)
+{
+ struct smo8800_device *smo8800 = data;
+
+ dev_info(smo8800->dev, "detected free fall\n");
+ return IRQ_HANDLED;
+}
+
+static acpi_status smo8800_get_resource(struct acpi_resource *resource,
+ void *context)
+{
+ struct acpi_resource_extended_irq *irq;
+
+ if (resource->type != ACPI_RESOURCE_TYPE_EXTENDED_IRQ)
+ return AE_OK;
+
+ irq = &resource->data.extended_irq;
+ if (!irq || !irq->interrupt_count)
+ return AE_OK;
+
+ *((u32 *)context) = irq->interrupts[0];
+ return AE_CTRL_TERMINATE;
+}
+
+static u32 smo8800_get_irq(struct acpi_device *device)
+{
+ u32 irq = 0;
+ acpi_status status;
+
+ status = acpi_walk_resources(device->handle, METHOD_NAME__CRS,
+ smo8800_get_resource, &irq);
+ if (ACPI_FAILURE(status)) {
+ dev_err(&device->dev, "acpi_walk_resources failed\n");
+ return 0;
+ }
+
+ return irq;
+}
+
+static ssize_t smo8800_misc_read(struct file *file, char __user *buf,
+ size_t count, loff_t *pos)
+{
+ struct smo8800_device *smo8800 = container_of(file->private_data,
+ struct smo8800_device, miscdev);
+
+ u32 data = 0;
+ unsigned char byte_data = 0;
+ ssize_t retval = 1;
+
+ if (count < 1)
+ return -EINVAL;
+
+ atomic_set(&smo8800->counter, 0);
+ retval = wait_event_interruptible(smo8800->misc_wait,
+ (data = atomic_xchg(&smo8800->counter, 0)));
+
+ if (retval)
+ return retval;
+
+ byte_data = 1;
+ retval = 1;
+
+ if (data < 255)
+ byte_data = data;
+ else
+ byte_data = 255;
+
+ if (put_user(byte_data, buf))
+ retval = -EFAULT;
+
+ return retval;
+}
+
+static int smo8800_misc_open(struct inode *inode, struct file *file)
+{
+ struct smo8800_device *smo8800 = container_of(file->private_data,
+ struct smo8800_device, miscdev);
+
+ if (test_and_set_bit(0, &smo8800->misc_opened))
+ return -EBUSY; /* already open */
+
+ atomic_set(&smo8800->counter, 0);
+ return 0;
+}
+
+static int smo8800_misc_release(struct inode *inode, struct file *file)
+{
+ struct smo8800_device *smo8800 = container_of(file->private_data,
+ struct smo8800_device, miscdev);
+
+ clear_bit(0, &smo8800->misc_opened); /* release the device */
+ return 0;
+}
+
+static const struct file_operations smo8800_misc_fops = {
+ .owner = THIS_MODULE,
+ .read = smo8800_misc_read,
+ .open = smo8800_misc_open,
+ .release = smo8800_misc_release,
+};
+
+static int smo8800_add(struct acpi_device *device)
+{
+ int err;
+ struct smo8800_device *smo8800;
+
+ smo8800 = devm_kzalloc(&device->dev, sizeof(*smo8800), GFP_KERNEL);
+ if (!smo8800) {
+ dev_err(&device->dev, "failed to allocate device data\n");
+ return -ENOMEM;
+ }
+
+ smo8800->dev = &device->dev;
+ smo8800->miscdev.minor = MISC_DYNAMIC_MINOR;
+ smo8800->miscdev.name = "freefall";
+ smo8800->miscdev.fops = &smo8800_misc_fops;
+
+ init_waitqueue_head(&smo8800->misc_wait);
+
+ err = misc_register(&smo8800->miscdev);
+ if (err) {
+ dev_err(&device->dev, "failed to register misc dev: %d\n", err);
+ return err;
+ }
+
+ device->driver_data = smo8800;
+
+ smo8800->irq = smo8800_get_irq(device);
+ if (!smo8800->irq) {
+ dev_err(&device->dev, "failed to obtain IRQ\n");
+ err = -EINVAL;
+ goto error;
+ }
+
+ err = request_threaded_irq(smo8800->irq, smo8800_interrupt_quick,
+ smo8800_interrupt_thread,
+ IRQF_TRIGGER_RISING | IRQF_ONESHOT,
+ DRIVER_NAME, smo8800);
+ if (err) {
+ dev_err(&device->dev,
+ "failed to request thread for IRQ %d: %d\n",
+ smo8800->irq, err);
+ goto error;
+ }
+
+ dev_dbg(&device->dev, "device /dev/freefall registered with IRQ %d\n",
+ smo8800->irq);
+ return 0;
+
+error:
+ misc_deregister(&smo8800->miscdev);
+ return err;
+}
+
+static int smo8800_remove(struct acpi_device *device)
+{
+ struct smo8800_device *smo8800 = device->driver_data;
+
+ free_irq(smo8800->irq, smo8800);
+ misc_deregister(&smo8800->miscdev);
+ dev_dbg(&device->dev, "device /dev/freefall unregistered\n");
+ return 0;
+}
+
+static const struct acpi_device_id smo8800_ids[] = {
+ { "SMO8800", 0 },
+ { "SMO8810", 0 },
+ { "", 0 },
+};
+
+MODULE_DEVICE_TABLE(acpi, smo8800_ids);
+
+static struct acpi_driver smo8800_driver = {
+ .name = DRIVER_NAME,
+ .class = "Latitude",
+ .ids = smo8800_ids,
+ .ops = {
+ .add = smo8800_add,
+ .remove = smo8800_remove,
+ },
+ .owner = THIS_MODULE,
+};
+
+module_acpi_driver(smo8800_driver);
+
+MODULE_DESCRIPTION("Dell Latitude freefall driver (ACPI SMO8800/SMO8810)");
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Sonal Santan, Pali Rohár");
diff --git a/drivers/platform/x86/hp-wmi.c b/drivers/platform/x86/hp-wmi.c
index 8ba8956..484a867 100644
--- a/drivers/platform/x86/hp-wmi.c
+++ b/drivers/platform/x86/hp-wmi.c
@@ -53,6 +53,7 @@
#define HPWMI_ALS_QUERY 0x3
#define HPWMI_HARDWARE_QUERY 0x4
#define HPWMI_WIRELESS_QUERY 0x5
+#define HPWMI_BIOS_QUERY 0x9
#define HPWMI_HOTKEY_QUERY 0xc
#define HPWMI_FEATURE_QUERY 0xd
#define HPWMI_WIRELESS2_QUERY 0x1b
@@ -144,6 +145,7 @@
{ KE_KEY, 0x2142, { KEY_MEDIA } },
{ KE_KEY, 0x213b, { KEY_INFO } },
{ KE_KEY, 0x2169, { KEY_DIRECTION } },
+ { KE_KEY, 0x216a, { KEY_SETUP } },
{ KE_KEY, 0x231b, { KEY_HELP } },
{ KE_END, 0 }
};
@@ -304,6 +306,19 @@
return (state & 0x10) ? 1 : 0;
}
+static int hp_wmi_enable_hotkeys(void)
+{
+ int ret;
+ int query = 0x6e;
+
+ ret = hp_wmi_perform_query(HPWMI_BIOS_QUERY, 1, &query, sizeof(query),
+ 0);
+
+ if (ret)
+ return -EINVAL;
+ return 0;
+}
+
static int hp_wmi_set_block(void *data, bool blocked)
{
enum hp_wmi_radio r = (enum hp_wmi_radio) data;
@@ -648,6 +663,9 @@
hp_wmi_tablet_state());
input_sync(hp_wmi_input_dev);
+ if (hp_wmi_bios_2009_later() == 4)
+ hp_wmi_enable_hotkeys();
+
status = wmi_install_notify_handler(HPWMI_EVENT_GUID, hp_wmi_notify, NULL);
if (ACPI_FAILURE(status)) {
err = -EIO;
diff --git a/drivers/platform/x86/ideapad-laptop.c b/drivers/platform/x86/ideapad-laptop.c
index 6dd060a..b4c495a 100644
--- a/drivers/platform/x86/ideapad-laptop.c
+++ b/drivers/platform/x86/ideapad-laptop.c
@@ -36,6 +36,8 @@
#include <linux/debugfs.h>
#include <linux/seq_file.h>
#include <linux/i8042.h>
+#include <linux/dmi.h>
+#include <linux/device.h>
#define IDEAPAD_RFKILL_DEV_NUM (3)
@@ -819,6 +821,19 @@
}
}
+/* Blacklist for devices where the ideapad rfkill interface does not work */
+static struct dmi_system_id rfkill_blacklist[] = {
+ /* The Lenovo Yoga 2 11 always reports everything as blocked */
+ {
+ .ident = "Lenovo Yoga 2 11",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "LENOVO"),
+ DMI_MATCH(DMI_PRODUCT_VERSION, "Lenovo Yoga 2 11"),
+ },
+ },
+ {}
+};
+
static int ideapad_acpi_add(struct platform_device *pdev)
{
int ret, i;
@@ -833,7 +848,7 @@
if (read_method_int(adev->handle, "_CFG", &cfg))
return -ENODEV;
- priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
if (!priv)
return -ENOMEM;
@@ -844,7 +859,7 @@
ret = ideapad_sysfs_init(priv);
if (ret)
- goto sysfs_failed;
+ return ret;
ret = ideapad_debugfs_init(priv);
if (ret)
@@ -854,11 +869,10 @@
if (ret)
goto input_failed;
- for (i = 0; i < IDEAPAD_RFKILL_DEV_NUM; i++) {
- if (test_bit(ideapad_rfk_data[i].cfgbit, &priv->cfg))
- ideapad_register_rfkill(priv, i);
- else
- priv->rfk[i] = NULL;
+ if (!dmi_check_system(rfkill_blacklist)) {
+ for (i = 0; i < IDEAPAD_RFKILL_DEV_NUM; i++)
+ if (test_bit(ideapad_rfk_data[i].cfgbit, &priv->cfg))
+ ideapad_register_rfkill(priv, i);
}
ideapad_sync_rfk_state(priv);
ideapad_sync_touchpad_state(priv);
@@ -884,8 +898,6 @@
ideapad_debugfs_exit(priv);
debugfs_failed:
ideapad_sysfs_exit(priv);
-sysfs_failed:
- kfree(priv);
return ret;
}
@@ -903,7 +915,6 @@
ideapad_debugfs_exit(priv);
ideapad_sysfs_exit(priv);
dev_set_drvdata(&pdev->dev, NULL);
- kfree(priv);
return 0;
}
diff --git a/drivers/platform/x86/intel_mid_thermal.c b/drivers/platform/x86/intel_mid_thermal.c
index 93fab8b..ab7860a 100644
--- a/drivers/platform/x86/intel_mid_thermal.c
+++ b/drivers/platform/x86/intel_mid_thermal.c
@@ -481,7 +481,8 @@
int i;
struct platform_info *pinfo;
- pinfo = kzalloc(sizeof(struct platform_info), GFP_KERNEL);
+ pinfo = devm_kzalloc(&pdev->dev, sizeof(struct platform_info),
+ GFP_KERNEL);
if (!pinfo)
return -ENOMEM;
@@ -489,7 +490,6 @@
ret = mid_initialize_adc(&pdev->dev);
if (ret) {
dev_err(&pdev->dev, "ADC init failed");
- kfree(pinfo);
return ret;
}
@@ -520,7 +520,6 @@
thermal_zone_device_unregister(pinfo->tzd[i]);
}
configure_adc(0);
- kfree(pinfo);
return ret;
}
@@ -541,8 +540,6 @@
thermal_zone_device_unregister(pinfo->tzd[i]);
}
- kfree(pinfo);
-
/* Stop the ADC */
return configure_adc(0);
}
diff --git a/drivers/platform/x86/intel_pmic_gpio.c b/drivers/platform/x86/intel_pmic_gpio.c
index 2805988..40929e4 100644
--- a/drivers/platform/x86/intel_pmic_gpio.c
+++ b/drivers/platform/x86/intel_pmic_gpio.c
@@ -91,7 +91,7 @@
static int pmic_gpio_direction_input(struct gpio_chip *chip, unsigned offset)
{
- if (offset > 8) {
+ if (offset >= 8) {
pr_err("only pin 0-7 support input\n");
return -1;/* we only have 8 GPIO can use as input */
}
@@ -130,7 +130,7 @@
int ret;
/* we only have 8 GPIO pins we can use as input */
- if (offset > 8)
+ if (offset >= 8)
return -EOPNOTSUPP;
ret = intel_scu_ipc_ioread8(GPIO0 + offset, &r);
if (ret < 0)
diff --git a/drivers/platform/x86/pvpanic.c b/drivers/platform/x86/pvpanic.c
index c9f6e51..073a90a 100644
--- a/drivers/platform/x86/pvpanic.c
+++ b/drivers/platform/x86/pvpanic.c
@@ -70,6 +70,7 @@
static struct notifier_block pvpanic_panic_nb = {
.notifier_call = pvpanic_panic_notify,
+ .priority = 1, /* let this called before broken drm_fb_helper */
};
diff --git a/drivers/platform/x86/samsung-laptop.c b/drivers/platform/x86/samsung-laptop.c
index d1f03005..5a59665 100644
--- a/drivers/platform/x86/samsung-laptop.c
+++ b/drivers/platform/x86/samsung-laptop.c
@@ -27,6 +27,7 @@
#include <linux/debugfs.h>
#include <linux/ctype.h>
#include <linux/efi.h>
+#include <linux/suspend.h>
#include <acpi/video.h>
/*
@@ -340,6 +341,8 @@
struct samsung_laptop_debug debug;
struct samsung_quirks *quirks;
+ struct notifier_block pm_nb;
+
bool handle_backlight;
bool has_stepping_quirk;
@@ -348,6 +351,8 @@
struct samsung_quirks {
bool broken_acpi_video;
+ bool four_kbd_backlight_levels;
+ bool enable_kbd_backlight;
};
static struct samsung_quirks samsung_unknown = {};
@@ -356,6 +361,11 @@
.broken_acpi_video = true,
};
+static struct samsung_quirks samsung_np740u3e = {
+ .four_kbd_backlight_levels = true,
+ .enable_kbd_backlight = true,
+};
+
static bool force;
module_param(force, bool, 0);
MODULE_PARM_DESC(force,
@@ -1051,6 +1061,8 @@
samsung->kbd_led.brightness_set = kbd_led_set;
samsung->kbd_led.brightness_get = kbd_led_get;
samsung->kbd_led.max_brightness = 8;
+ if (samsung->quirks->four_kbd_backlight_levels)
+ samsung->kbd_led.max_brightness = 4;
ret = led_classdev_register(&samsung->platform_device->dev,
&samsung->kbd_led);
@@ -1414,6 +1426,19 @@
}
}
+static int samsung_pm_notification(struct notifier_block *nb,
+ unsigned long val, void *ptr)
+{
+ struct samsung_laptop *samsung;
+
+ samsung = container_of(nb, struct samsung_laptop, pm_nb);
+ if (val == PM_POST_HIBERNATION &&
+ samsung->quirks->enable_kbd_backlight)
+ kbd_backlight_enable(samsung);
+
+ return 0;
+}
+
static int __init samsung_platform_init(struct samsung_laptop *samsung)
{
struct platform_device *pdev;
@@ -1534,6 +1559,15 @@
},
.driver_data = &samsung_broken_acpi_video,
},
+ {
+ .callback = samsung_dmi_matched,
+ .ident = "730U3E/740U3E",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "SAMSUNG ELECTRONICS CO., LTD."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "730U3E/740U3E"),
+ },
+ .driver_data = &samsung_np740u3e,
+ },
{ },
};
MODULE_DEVICE_TABLE(dmi, samsung_dmi_table);
@@ -1608,6 +1642,9 @@
if (ret)
goto error_debugfs;
+ samsung->pm_nb.notifier_call = samsung_pm_notification;
+ register_pm_notifier(&samsung->pm_nb);
+
samsung_platform_device = samsung->platform_device;
return ret;
@@ -1633,6 +1670,7 @@
struct samsung_laptop *samsung;
samsung = platform_get_drvdata(samsung_platform_device);
+ unregister_pm_notifier(&samsung->pm_nb);
samsung_debugfs_exit(samsung);
samsung_leds_exit(samsung);
diff --git a/drivers/platform/x86/thinkpad_acpi.c b/drivers/platform/x86/thinkpad_acpi.c
index 15e61c1..d82f196e 100644
--- a/drivers/platform/x86/thinkpad_acpi.c
+++ b/drivers/platform/x86/thinkpad_acpi.c
@@ -3171,8 +3171,10 @@
KEY_MICMUTE, /* 0x1a: Mic mute (since ?400 or so) */
/* (assignments unknown, please report if found) */
- KEY_UNKNOWN, KEY_UNKNOWN, KEY_UNKNOWN, KEY_UNKNOWN,
KEY_UNKNOWN,
+
+ /* Extra keys in use since the X240 / T440 / T540 */
+ KEY_CONFIG, KEY_SEARCH, KEY_SCALE, KEY_COMPUTER,
},
};
diff --git a/drivers/platform/x86/toshiba_acpi.c b/drivers/platform/x86/toshiba_acpi.c
index 46473ca..76441dc 100644
--- a/drivers/platform/x86/toshiba_acpi.c
+++ b/drivers/platform/x86/toshiba_acpi.c
@@ -56,6 +56,7 @@
#include <linux/workqueue.h>
#include <linux/i8042.h>
#include <linux/acpi.h>
+#include <linux/dmi.h>
#include <asm/uaccess.h>
MODULE_AUTHOR("John Belmonte");
@@ -213,6 +214,30 @@
{ KE_END, 0 },
};
+/* alternative keymap */
+static const struct dmi_system_id toshiba_alt_keymap_dmi[] = {
+ {
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "TOSHIBA"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Satellite M840"),
+ },
+ },
+ {}
+};
+
+static const struct key_entry toshiba_acpi_alt_keymap[] = {
+ { KE_KEY, 0x157, { KEY_MUTE } },
+ { KE_KEY, 0x102, { KEY_ZOOMOUT } },
+ { KE_KEY, 0x103, { KEY_ZOOMIN } },
+ { KE_KEY, 0x139, { KEY_ZOOMRESET } },
+ { KE_KEY, 0x13e, { KEY_SWITCHVIDEOMODE } },
+ { KE_KEY, 0x13c, { KEY_BRIGHTNESSDOWN } },
+ { KE_KEY, 0x13d, { KEY_BRIGHTNESSUP } },
+ { KE_KEY, 0x158, { KEY_WLAN } },
+ { KE_KEY, 0x13f, { KEY_TOUCHPAD_TOGGLE } },
+ { KE_END, 0 },
+};
+
/* utility
*/
@@ -1440,6 +1465,7 @@
acpi_handle ec_handle;
int error;
u32 hci_result;
+ const struct key_entry *keymap = toshiba_acpi_keymap;
dev->hotkey_dev = input_allocate_device();
if (!dev->hotkey_dev)
@@ -1449,7 +1475,9 @@
dev->hotkey_dev->phys = "toshiba_acpi/input0";
dev->hotkey_dev->id.bustype = BUS_HOST;
- error = sparse_keymap_setup(dev->hotkey_dev, toshiba_acpi_keymap, NULL);
+ if (dmi_check_system(toshiba_alt_keymap_dmi))
+ keymap = toshiba_acpi_alt_keymap;
+ error = sparse_keymap_setup(dev->hotkey_dev, keymap, NULL);
if (error)
goto err_free_dev;
diff --git a/drivers/rapidio/devices/tsi721.c b/drivers/rapidio/devices/tsi721.c
index 1753dc6..2ca1a0b 100644
--- a/drivers/rapidio/devices/tsi721.c
+++ b/drivers/rapidio/devices/tsi721.c
@@ -768,15 +768,10 @@
}
#endif /* CONFIG_RAPIDIO_DMA_ENGINE */
- err = pci_enable_msix(priv->pdev, entries, ARRAY_SIZE(entries));
+ err = pci_enable_msix_exact(priv->pdev, entries, ARRAY_SIZE(entries));
if (err) {
- if (err > 0)
- dev_info(&priv->pdev->dev,
- "Only %d MSI-X vectors available, "
- "not using MSI-X\n", err);
- else
- dev_err(&priv->pdev->dev,
- "Failed to enable MSI-X (err=%d)\n", err);
+ dev_err(&priv->pdev->dev,
+ "Failed to enable MSI-X (err=%d)\n", err);
return err;
}
diff --git a/drivers/rtc/Kconfig b/drivers/rtc/Kconfig
index 2e565f8..0754f5c 100644
--- a/drivers/rtc/Kconfig
+++ b/drivers/rtc/Kconfig
@@ -386,12 +386,12 @@
will be called rtc-pcf8583.
config RTC_DRV_M41T80
- tristate "ST M41T62/65/M41T80/81/82/83/84/85/87"
+ tristate "ST M41T62/65/M41T80/81/82/83/84/85/87 and compatible"
help
If you say Y here you will get support for the ST M41T60
and M41T80 RTC chips series. Currently, the following chips are
supported: M41T62, M41T65, M41T80, M41T81, M41T82, M41T83, M41ST84,
- M41ST85, and M41ST87.
+ M41ST85, M41ST87, and MicroCrystal RV4162.
This driver can also be built as a module. If so, the module
will be called rtc-m41t80.
@@ -530,11 +530,11 @@
will be called rtc-rv3029c2.
config RTC_DRV_S5M
- tristate "Samsung S5M series"
+ tristate "Samsung S2M/S5M series"
depends on MFD_SEC_CORE
help
If you say yes here you will get support for the
- RTC of Samsung S5M PMIC series.
+ RTC of Samsung S2MPS14 and S5M PMIC series.
This driver can also be built as a module. If so, the module
will be called rtc-s5m.
@@ -573,6 +573,17 @@
This driver can also be built as a module. If so, the module
will be called rtc-ds1305.
+config RTC_DRV_DS1343
+ select REGMAP_SPI
+ tristate "Dallas/Maxim DS1343/DS1344"
+ help
+ If you say yes here you get support for the
+ Dallas/Maxim DS1343 and DS1344 real time clock chips.
+ Support for trickle charger, alarm is provided.
+
+ This driver can also be built as a module. If so, the module
+ will be called rtc-ds1343.
+
config RTC_DRV_DS1347
tristate "Dallas/Maxim DS1347"
help
@@ -650,6 +661,14 @@
This driver can also be built as a module. If so the module
will be called rtc-rx4581.
+config RTC_DRV_MCP795
+ tristate "Microchip MCP795"
+ help
+ If you say yes here you will get support for the Microchip MCP795.
+
+ This driver can also be built as a module. If so the module
+ will be called rtc-mcp795.
+
endif # SPI_MASTER
comment "Platform RTC drivers"
@@ -758,6 +777,16 @@
This driver can also be built as a module. If so, the module
will be called rtc-da9055
+config RTC_DRV_DA9063
+ tristate "Dialog Semiconductor DA9063 RTC"
+ depends on MFD_DA9063
+ help
+ If you say yes here you will get support for the RTC subsystem
+ of the Dialog Semiconductor DA9063.
+
+ This driver can also be built as a module. If so, the module
+ will be called "rtc-da9063".
+
config RTC_DRV_EFI
tristate "EFI RTC"
depends on IA64
@@ -1327,6 +1356,15 @@
This driver can also be built as a module. If so, the module
will be called rtc-moxart
+config RTC_DRV_XGENE
+ tristate "APM X-Gene RTC"
+ help
+ If you say yes here you get support for the APM X-Gene SoC real time
+ clock.
+
+ This driver can also be built as a module, if so, the module
+ will be called "rtc-xgene".
+
comment "HID Sensor RTC drivers"
config RTC_DRV_HID_SENSOR_TIME
diff --git a/drivers/rtc/Makefile b/drivers/rtc/Makefile
index 40a0991..70347d0 100644
--- a/drivers/rtc/Makefile
+++ b/drivers/rtc/Makefile
@@ -32,6 +32,7 @@
obj-$(CONFIG_RTC_DRV_COH901331) += rtc-coh901331.o
obj-$(CONFIG_RTC_DRV_DA9052) += rtc-da9052.o
obj-$(CONFIG_RTC_DRV_DA9055) += rtc-da9055.o
+obj-$(CONFIG_RTC_DRV_DA9063) += rtc-da9063.o
obj-$(CONFIG_RTC_DRV_DAVINCI) += rtc-davinci.o
obj-$(CONFIG_RTC_DRV_DM355EVM) += rtc-dm355evm.o
obj-$(CONFIG_RTC_DRV_VRTC) += rtc-mrst.o
@@ -40,6 +41,7 @@
obj-$(CONFIG_RTC_DRV_DS1302) += rtc-ds1302.o
obj-$(CONFIG_RTC_DRV_DS1305) += rtc-ds1305.o
obj-$(CONFIG_RTC_DRV_DS1307) += rtc-ds1307.o
+obj-$(CONFIG_RTC_DRV_DS1343) += rtc-ds1343.o
obj-$(CONFIG_RTC_DRV_DS1347) += rtc-ds1347.o
obj-$(CONFIG_RTC_DRV_DS1374) += rtc-ds1374.o
obj-$(CONFIG_RTC_DRV_DS1390) += rtc-ds1390.o
@@ -80,6 +82,7 @@
obj-$(CONFIG_RTC_DRV_MAX6902) += rtc-max6902.o
obj-$(CONFIG_RTC_DRV_MAX77686) += rtc-max77686.o
obj-$(CONFIG_RTC_DRV_MC13XXX) += rtc-mc13xxx.o
+obj-$(CONFIG_RTC_DRV_MCP795) += rtc-mcp795.o
obj-$(CONFIG_RTC_DRV_MSM6242) += rtc-msm6242.o
obj-$(CONFIG_RTC_DRV_MPC5121) += rtc-mpc5121.o
obj-$(CONFIG_RTC_DRV_MV) += rtc-mv.o
@@ -135,5 +138,6 @@
obj-$(CONFIG_RTC_DRV_WM831X) += rtc-wm831x.o
obj-$(CONFIG_RTC_DRV_WM8350) += rtc-wm8350.o
obj-$(CONFIG_RTC_DRV_X1205) += rtc-x1205.o
+obj-$(CONFIG_RTC_DRV_XGENE) += rtc-xgene.o
obj-$(CONFIG_RTC_DRV_SIRFSOC) += rtc-sirfsoc.o
obj-$(CONFIG_RTC_DRV_MOXART) += rtc-moxart.o
diff --git a/drivers/rtc/interface.c b/drivers/rtc/interface.c
index c2eff60..5813fa5 100644
--- a/drivers/rtc/interface.c
+++ b/drivers/rtc/interface.c
@@ -292,7 +292,8 @@
dev_dbg(&rtc->dev, "alarm rollover: %s\n", "year");
do {
alarm->time.tm_year++;
- } while (rtc_valid_tm(&alarm->time) != 0);
+ } while (!is_leap_year(alarm->time.tm_year + 1900)
+ && rtc_valid_tm(&alarm->time) != 0);
break;
default:
@@ -300,7 +301,16 @@
}
done:
- return 0;
+ err = rtc_valid_tm(&alarm->time);
+
+ if (err) {
+ dev_warn(&rtc->dev, "invalid alarm value: %d-%d-%d %d:%d:%d\n",
+ alarm->time.tm_year + 1900, alarm->time.tm_mon + 1,
+ alarm->time.tm_mday, alarm->time.tm_hour, alarm->time.tm_min,
+ alarm->time.tm_sec);
+ }
+
+ return err;
}
int rtc_read_alarm(struct rtc_device *rtc, struct rtc_wkalrm *alarm)
diff --git a/drivers/rtc/rtc-88pm860x.c b/drivers/rtc/rtc-88pm860x.c
index 81650484..0c6add1 100644
--- a/drivers/rtc/rtc-88pm860x.c
+++ b/drivers/rtc/rtc-88pm860x.c
@@ -293,7 +293,7 @@
int ret;
if (!np)
return -ENODEV;
- np = of_find_node_by_name(np, "rtc");
+ np = of_get_child_by_name(np, "rtc");
if (!np) {
dev_err(&pdev->dev, "failed to find rtc node\n");
return -ENODEV;
@@ -301,6 +301,7 @@
ret = of_property_read_u32(np, "marvell,88pm860x-vrtc", &info->vrtc);
if (ret)
info->vrtc = 0;
+ of_node_put(np);
return 0;
}
#else
diff --git a/drivers/rtc/rtc-at91rm9200.c b/drivers/rtc/rtc-at91rm9200.c
index 3281c90..44fe83e 100644
--- a/drivers/rtc/rtc-at91rm9200.c
+++ b/drivers/rtc/rtc-at91rm9200.c
@@ -48,6 +48,7 @@
static const struct at91_rtc_config *at91_rtc_config;
static DECLARE_COMPLETION(at91_rtc_updated);
+static DECLARE_COMPLETION(at91_rtc_upd_rdy);
static unsigned int at91_alarm_year = AT91_RTC_EPOCH;
static void __iomem *at91_rtc_regs;
static int irq;
@@ -161,6 +162,8 @@
1900 + tm->tm_year, tm->tm_mon, tm->tm_mday,
tm->tm_hour, tm->tm_min, tm->tm_sec);
+ wait_for_completion(&at91_rtc_upd_rdy);
+
/* Stop Time/Calendar from counting */
cr = at91_rtc_read(AT91_RTC_CR);
at91_rtc_write(AT91_RTC_CR, cr | AT91_RTC_UPDCAL | AT91_RTC_UPDTIM);
@@ -183,7 +186,9 @@
/* Restart Time/Calendar */
cr = at91_rtc_read(AT91_RTC_CR);
+ at91_rtc_write(AT91_RTC_SCCR, AT91_RTC_SECEV);
at91_rtc_write(AT91_RTC_CR, cr & ~(AT91_RTC_UPDCAL | AT91_RTC_UPDTIM));
+ at91_rtc_write_ier(AT91_RTC_SECEV);
return 0;
}
@@ -290,8 +295,10 @@
if (rtsr) { /* this interrupt is shared! Is it ours? */
if (rtsr & AT91_RTC_ALARM)
events |= (RTC_AF | RTC_IRQF);
- if (rtsr & AT91_RTC_SECEV)
- events |= (RTC_UF | RTC_IRQF);
+ if (rtsr & AT91_RTC_SECEV) {
+ complete(&at91_rtc_upd_rdy);
+ at91_rtc_write_idr(AT91_RTC_SECEV);
+ }
if (rtsr & AT91_RTC_ACKUPD)
complete(&at91_rtc_updated);
@@ -413,6 +420,11 @@
return PTR_ERR(rtc);
platform_set_drvdata(pdev, rtc);
+ /* enable SECEV interrupt in order to initialize at91_rtc_upd_rdy
+ * completion.
+ */
+ at91_rtc_write_ier(AT91_RTC_SECEV);
+
dev_info(&pdev->dev, "AT91 Real Time Clock driver.\n");
return 0;
}
diff --git a/drivers/rtc/rtc-bfin.c b/drivers/rtc/rtc-bfin.c
index 0c53f45..fe4bdb0 100644
--- a/drivers/rtc/rtc-bfin.c
+++ b/drivers/rtc/rtc-bfin.c
@@ -346,7 +346,7 @@
{
struct bfin_rtc *rtc;
struct device *dev = &pdev->dev;
- int ret = 0;
+ int ret;
unsigned long timeout = jiffies + HZ;
dev_dbg_stamp(dev);
@@ -361,16 +361,17 @@
/* Register our RTC with the RTC framework */
rtc->rtc_dev = devm_rtc_device_register(dev, pdev->name, &bfin_rtc_ops,
THIS_MODULE);
- if (unlikely(IS_ERR(rtc->rtc_dev))) {
- ret = PTR_ERR(rtc->rtc_dev);
- goto err;
- }
+ if (unlikely(IS_ERR(rtc->rtc_dev)))
+ return PTR_ERR(rtc->rtc_dev);
/* Grab the IRQ and init the hardware */
ret = devm_request_irq(dev, IRQ_RTC, bfin_rtc_interrupt, 0,
pdev->name, dev);
if (unlikely(ret))
- goto err;
+ dev_err(&pdev->dev,
+ "unable to request IRQ; alarm won't work, "
+ "and writes will be delayed\n");
+
/* sometimes the bootloader touched things, but the write complete was not
* enabled, so let's just do a quick timeout here since the IRQ will not fire ...
*/
@@ -381,9 +382,6 @@
bfin_write_RTC_SWCNT(0);
return 0;
-
-err:
- return ret;
}
static int bfin_rtc_remove(struct platform_device *pdev)
diff --git a/drivers/rtc/rtc-cmos.c b/drivers/rtc/rtc-cmos.c
index 0963c93..b0e4a3e 100644
--- a/drivers/rtc/rtc-cmos.c
+++ b/drivers/rtc/rtc-cmos.c
@@ -647,6 +647,7 @@
int retval = 0;
unsigned char rtc_control;
unsigned address_space;
+ u32 flags = 0;
/* there can be only one ... */
if (cmos_rtc.dev)
@@ -660,9 +661,12 @@
* REVISIT non-x86 systems may instead use memory space resources
* (needing ioremap etc), not i/o space resources like this ...
*/
- ports = request_region(ports->start,
- resource_size(ports),
- driver_name);
+ if (RTC_IOMAPPED)
+ ports = request_region(ports->start, resource_size(ports),
+ driver_name);
+ else
+ ports = request_mem_region(ports->start, resource_size(ports),
+ driver_name);
if (!ports) {
dev_dbg(dev, "i/o registers already in use\n");
return -EBUSY;
@@ -699,6 +703,11 @@
* expect CMOS_READ and friends to handle.
*/
if (info) {
+ if (info->flags)
+ flags = info->flags;
+ if (info->address_space)
+ address_space = info->address_space;
+
if (info->rtc_day_alarm && info->rtc_day_alarm < 128)
cmos_rtc.day_alrm = info->rtc_day_alarm;
if (info->rtc_mon_alarm && info->rtc_mon_alarm < 128)
@@ -726,18 +735,21 @@
spin_lock_irq(&rtc_lock);
- /* force periodic irq to CMOS reset default of 1024Hz;
- *
- * REVISIT it's been reported that at least one x86_64 ALI mobo
- * doesn't use 32KHz here ... for portability we might need to
- * do something about other clock frequencies.
- */
- cmos_rtc.rtc->irq_freq = 1024;
- hpet_set_periodic_freq(cmos_rtc.rtc->irq_freq);
- CMOS_WRITE(RTC_REF_CLCK_32KHZ | 0x06, RTC_FREQ_SELECT);
+ if (!(flags & CMOS_RTC_FLAGS_NOFREQ)) {
+ /* force periodic irq to CMOS reset default of 1024Hz;
+ *
+ * REVISIT it's been reported that at least one x86_64 ALI
+ * mobo doesn't use 32KHz here ... for portability we might
+ * need to do something about other clock frequencies.
+ */
+ cmos_rtc.rtc->irq_freq = 1024;
+ hpet_set_periodic_freq(cmos_rtc.rtc->irq_freq);
+ CMOS_WRITE(RTC_REF_CLCK_32KHZ | 0x06, RTC_FREQ_SELECT);
+ }
/* disable irqs */
- cmos_irq_disable(&cmos_rtc, RTC_PIE | RTC_AIE | RTC_UIE);
+ if (is_valid_irq(rtc_irq))
+ cmos_irq_disable(&cmos_rtc, RTC_PIE | RTC_AIE | RTC_UIE);
rtc_control = CMOS_READ(RTC_CONTROL);
@@ -802,14 +814,18 @@
cmos_rtc.dev = NULL;
rtc_device_unregister(cmos_rtc.rtc);
cleanup0:
- release_region(ports->start, resource_size(ports));
+ if (RTC_IOMAPPED)
+ release_region(ports->start, resource_size(ports));
+ else
+ release_mem_region(ports->start, resource_size(ports));
return retval;
}
-static void cmos_do_shutdown(void)
+static void cmos_do_shutdown(int rtc_irq)
{
spin_lock_irq(&rtc_lock);
- cmos_irq_disable(&cmos_rtc, RTC_IRQMASK);
+ if (is_valid_irq(rtc_irq))
+ cmos_irq_disable(&cmos_rtc, RTC_IRQMASK);
spin_unlock_irq(&rtc_lock);
}
@@ -818,7 +834,7 @@
struct cmos_rtc *cmos = dev_get_drvdata(dev);
struct resource *ports;
- cmos_do_shutdown();
+ cmos_do_shutdown(cmos->irq);
sysfs_remove_bin_file(&dev->kobj, &nvram);
@@ -831,7 +847,10 @@
cmos->rtc = NULL;
ports = cmos->iomem;
- release_region(ports->start, resource_size(ports));
+ if (RTC_IOMAPPED)
+ release_region(ports->start, resource_size(ports));
+ else
+ release_mem_region(ports->start, resource_size(ports));
cmos->iomem = NULL;
cmos->dev = NULL;
@@ -1065,10 +1084,13 @@
static void cmos_pnp_shutdown(struct pnp_dev *pnp)
{
- if (system_state == SYSTEM_POWER_OFF && !cmos_poweroff(&pnp->dev))
+ struct device *dev = &pnp->dev;
+ struct cmos_rtc *cmos = dev_get_drvdata(dev);
+
+ if (system_state == SYSTEM_POWER_OFF && !cmos_poweroff(dev))
return;
- cmos_do_shutdown();
+ cmos_do_shutdown(cmos->irq);
}
static const struct pnp_device_id rtc_ids[] = {
@@ -1143,11 +1165,21 @@
static int __init cmos_platform_probe(struct platform_device *pdev)
{
+ struct resource *resource;
+ int irq;
+
cmos_of_init(pdev);
cmos_wake_setup(&pdev->dev);
- return cmos_do_probe(&pdev->dev,
- platform_get_resource(pdev, IORESOURCE_IO, 0),
- platform_get_irq(pdev, 0));
+
+ if (RTC_IOMAPPED)
+ resource = platform_get_resource(pdev, IORESOURCE_IO, 0);
+ else
+ resource = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ irq = -1;
+
+ return cmos_do_probe(&pdev->dev, resource, irq);
}
static int __exit cmos_platform_remove(struct platform_device *pdev)
@@ -1158,10 +1190,13 @@
static void cmos_platform_shutdown(struct platform_device *pdev)
{
- if (system_state == SYSTEM_POWER_OFF && !cmos_poweroff(&pdev->dev))
+ struct device *dev = &pdev->dev;
+ struct cmos_rtc *cmos = dev_get_drvdata(dev);
+
+ if (system_state == SYSTEM_POWER_OFF && !cmos_poweroff(dev))
return;
- cmos_do_shutdown();
+ cmos_do_shutdown(cmos->irq);
}
/* work with hotplug and coldplug */
diff --git a/drivers/rtc/rtc-da9052.c b/drivers/rtc/rtc-da9052.c
index a1cbf64..e5c9486 100644
--- a/drivers/rtc/rtc-da9052.c
+++ b/drivers/rtc/rtc-da9052.c
@@ -20,28 +20,28 @@
#include <linux/mfd/da9052/da9052.h>
#include <linux/mfd/da9052/reg.h>
-#define rtc_err(da9052, fmt, ...) \
- dev_err(da9052->dev, "%s: " fmt, __func__, ##__VA_ARGS__)
+#define rtc_err(rtc, fmt, ...) \
+ dev_err(rtc->da9052->dev, "%s: " fmt, __func__, ##__VA_ARGS__)
struct da9052_rtc {
struct rtc_device *rtc;
struct da9052 *da9052;
};
-static int da9052_rtc_enable_alarm(struct da9052 *da9052, bool enable)
+static int da9052_rtc_enable_alarm(struct da9052_rtc *rtc, bool enable)
{
int ret;
if (enable) {
- ret = da9052_reg_update(da9052, DA9052_ALARM_Y_REG,
- DA9052_ALARM_Y_ALARM_ON,
- DA9052_ALARM_Y_ALARM_ON);
+ ret = da9052_reg_update(rtc->da9052, DA9052_ALARM_Y_REG,
+ DA9052_ALARM_Y_ALARM_ON|DA9052_ALARM_Y_TICK_ON,
+ DA9052_ALARM_Y_ALARM_ON);
if (ret != 0)
- rtc_err(da9052, "Failed to enable ALM: %d\n", ret);
+ rtc_err(rtc, "Failed to enable ALM: %d\n", ret);
} else {
- ret = da9052_reg_update(da9052, DA9052_ALARM_Y_REG,
- DA9052_ALARM_Y_ALARM_ON, 0);
+ ret = da9052_reg_update(rtc->da9052, DA9052_ALARM_Y_REG,
+ DA9052_ALARM_Y_ALARM_ON|DA9052_ALARM_Y_TICK_ON, 0);
if (ret != 0)
- rtc_err(da9052, "Write error: %d\n", ret);
+ rtc_err(rtc, "Write error: %d\n", ret);
}
return ret;
}
@@ -49,31 +49,20 @@
static irqreturn_t da9052_rtc_irq(int irq, void *data)
{
struct da9052_rtc *rtc = data;
- int ret;
- ret = da9052_reg_read(rtc->da9052, DA9052_ALARM_MI_REG);
- if (ret < 0) {
- rtc_err(rtc->da9052, "Read error: %d\n", ret);
- return IRQ_NONE;
- }
-
- if (ret & DA9052_ALARMMI_ALARMTYPE) {
- da9052_rtc_enable_alarm(rtc->da9052, 0);
- rtc_update_irq(rtc->rtc, 1, RTC_IRQF | RTC_AF);
- } else
- rtc_update_irq(rtc->rtc, 1, RTC_IRQF | RTC_PF);
+ rtc_update_irq(rtc->rtc, 1, RTC_IRQF | RTC_AF);
return IRQ_HANDLED;
}
-static int da9052_read_alarm(struct da9052 *da9052, struct rtc_time *rtc_tm)
+static int da9052_read_alarm(struct da9052_rtc *rtc, struct rtc_time *rtc_tm)
{
int ret;
uint8_t v[5];
- ret = da9052_group_read(da9052, DA9052_ALARM_MI_REG, 5, v);
+ ret = da9052_group_read(rtc->da9052, DA9052_ALARM_MI_REG, 5, v);
if (ret != 0) {
- rtc_err(da9052, "Failed to group read ALM: %d\n", ret);
+ rtc_err(rtc, "Failed to group read ALM: %d\n", ret);
return ret;
}
@@ -84,23 +73,33 @@
rtc_tm->tm_min = v[0] & DA9052_RTC_MIN;
ret = rtc_valid_tm(rtc_tm);
- if (ret != 0)
- return ret;
return ret;
}
-static int da9052_set_alarm(struct da9052 *da9052, struct rtc_time *rtc_tm)
+static int da9052_set_alarm(struct da9052_rtc *rtc, struct rtc_time *rtc_tm)
{
+ struct da9052 *da9052 = rtc->da9052;
+ unsigned long alm_time;
int ret;
uint8_t v[3];
+ ret = rtc_tm_to_time(rtc_tm, &alm_time);
+ if (ret != 0)
+ return ret;
+
+ if (rtc_tm->tm_sec > 0) {
+ alm_time += 60 - rtc_tm->tm_sec;
+ rtc_time_to_tm(alm_time, rtc_tm);
+ }
+ BUG_ON(rtc_tm->tm_sec); /* it will cause repeated irqs if not zero */
+
rtc_tm->tm_year -= 100;
rtc_tm->tm_mon += 1;
ret = da9052_reg_update(da9052, DA9052_ALARM_MI_REG,
DA9052_RTC_MIN, rtc_tm->tm_min);
if (ret != 0) {
- rtc_err(da9052, "Failed to write ALRM MIN: %d\n", ret);
+ rtc_err(rtc, "Failed to write ALRM MIN: %d\n", ret);
return ret;
}
@@ -115,22 +114,22 @@
ret = da9052_reg_update(da9052, DA9052_ALARM_Y_REG,
DA9052_RTC_YEAR, rtc_tm->tm_year);
if (ret != 0)
- rtc_err(da9052, "Failed to write ALRM YEAR: %d\n", ret);
+ rtc_err(rtc, "Failed to write ALRM YEAR: %d\n", ret);
return ret;
}
-static int da9052_rtc_get_alarm_status(struct da9052 *da9052)
+static int da9052_rtc_get_alarm_status(struct da9052_rtc *rtc)
{
int ret;
- ret = da9052_reg_read(da9052, DA9052_ALARM_Y_REG);
+ ret = da9052_reg_read(rtc->da9052, DA9052_ALARM_Y_REG);
if (ret < 0) {
- rtc_err(da9052, "Failed to read ALM: %d\n", ret);
+ rtc_err(rtc, "Failed to read ALM: %d\n", ret);
return ret;
}
- ret &= DA9052_ALARM_Y_ALARM_ON;
- return (ret > 0) ? 1 : 0;
+
+ return !!(ret&DA9052_ALARM_Y_ALARM_ON);
}
static int da9052_rtc_read_time(struct device *dev, struct rtc_time *rtc_tm)
@@ -141,7 +140,7 @@
ret = da9052_group_read(rtc->da9052, DA9052_COUNT_S_REG, 6, v);
if (ret < 0) {
- rtc_err(rtc->da9052, "Failed to read RTC time : %d\n", ret);
+ rtc_err(rtc, "Failed to read RTC time : %d\n", ret);
return ret;
}
@@ -153,18 +152,14 @@
rtc_tm->tm_sec = v[0] & DA9052_RTC_SEC;
ret = rtc_valid_tm(rtc_tm);
- if (ret != 0) {
- rtc_err(rtc->da9052, "rtc_valid_tm failed: %d\n", ret);
- return ret;
- }
-
- return 0;
+ return ret;
}
static int da9052_rtc_set_time(struct device *dev, struct rtc_time *tm)
{
struct da9052_rtc *rtc;
uint8_t v[6];
+ int ret;
rtc = dev_get_drvdata(dev);
@@ -175,7 +170,10 @@
v[4] = tm->tm_mon + 1;
v[5] = tm->tm_year - 100;
- return da9052_group_write(rtc->da9052, DA9052_COUNT_S_REG, 6, v);
+ ret = da9052_group_write(rtc->da9052, DA9052_COUNT_S_REG, 6, v);
+ if (ret < 0)
+ rtc_err(rtc, "failed to set RTC time: %d\n", ret);
+ return ret;
}
static int da9052_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *alrm)
@@ -184,13 +182,13 @@
struct rtc_time *tm = &alrm->time;
struct da9052_rtc *rtc = dev_get_drvdata(dev);
- ret = da9052_read_alarm(rtc->da9052, tm);
-
- if (ret)
+ ret = da9052_read_alarm(rtc, tm);
+ if (ret < 0) {
+ rtc_err(rtc, "failed to read RTC alarm: %d\n", ret);
return ret;
+ }
- alrm->enabled = da9052_rtc_get_alarm_status(rtc->da9052);
-
+ alrm->enabled = da9052_rtc_get_alarm_status(rtc);
return 0;
}
@@ -200,16 +198,15 @@
struct rtc_time *tm = &alrm->time;
struct da9052_rtc *rtc = dev_get_drvdata(dev);
- ret = da9052_rtc_enable_alarm(rtc->da9052, 0);
+ ret = da9052_rtc_enable_alarm(rtc, 0);
if (ret < 0)
return ret;
- ret = da9052_set_alarm(rtc->da9052, tm);
- if (ret)
+ ret = da9052_set_alarm(rtc, tm);
+ if (ret < 0)
return ret;
- ret = da9052_rtc_enable_alarm(rtc->da9052, 1);
-
+ ret = da9052_rtc_enable_alarm(rtc, 1);
return ret;
}
@@ -217,7 +214,7 @@
{
struct da9052_rtc *rtc = dev_get_drvdata(dev);
- return da9052_rtc_enable_alarm(rtc->da9052, enabled);
+ return da9052_rtc_enable_alarm(rtc, enabled);
}
static const struct rtc_class_ops da9052_rtc_ops = {
@@ -239,10 +236,23 @@
rtc->da9052 = dev_get_drvdata(pdev->dev.parent);
platform_set_drvdata(pdev, rtc);
+
+ ret = da9052_reg_write(rtc->da9052, DA9052_BBAT_CONT_REG, 0xFE);
+ if (ret < 0) {
+ rtc_err(rtc,
+ "Failed to setup RTC battery charging: %d\n", ret);
+ return ret;
+ }
+
+ ret = da9052_reg_update(rtc->da9052, DA9052_ALARM_Y_REG,
+ DA9052_ALARM_Y_TICK_ON, 0);
+ if (ret != 0)
+ rtc_err(rtc, "Failed to disable TICKS: %d\n", ret);
+
ret = da9052_request_irq(rtc->da9052, DA9052_IRQ_ALARM, "ALM",
da9052_rtc_irq, rtc);
if (ret != 0) {
- rtc_err(rtc->da9052, "irq registration failed: %d\n", ret);
+ rtc_err(rtc, "irq registration failed: %d\n", ret);
return ret;
}
@@ -261,7 +271,7 @@
module_platform_driver(da9052_rtc_driver);
-MODULE_AUTHOR("David Dajun Chen <dchen@diasemi.com>");
+MODULE_AUTHOR("Anthony Olech <Anthony.Olech@diasemi.com>");
MODULE_DESCRIPTION("RTC driver for Dialog DA9052 PMIC");
MODULE_LICENSE("GPL");
MODULE_ALIAS("platform:da9052-rtc");
diff --git a/drivers/rtc/rtc-da9063.c b/drivers/rtc/rtc-da9063.c
new file mode 100644
index 0000000..5953930
--- /dev/null
+++ b/drivers/rtc/rtc-da9063.c
@@ -0,0 +1,333 @@
+/* rtc-da9063.c - Real time clock device driver for DA9063
+ * Copyright (C) 2013-14 Dialog Semiconductor Ltd.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Library General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Library General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/platform_device.h>
+#include <linux/interrupt.h>
+#include <linux/rtc.h>
+#include <linux/slab.h>
+#include <linux/delay.h>
+#include <linux/regmap.h>
+#include <linux/mfd/da9063/registers.h>
+#include <linux/mfd/da9063/core.h>
+
+#define YEARS_TO_DA9063(year) ((year) - 100)
+#define MONTHS_TO_DA9063(month) ((month) + 1)
+#define YEARS_FROM_DA9063(year) ((year) + 100)
+#define MONTHS_FROM_DA9063(month) ((month) - 1)
+
+#define RTC_DATA_LEN (DA9063_REG_COUNT_Y - DA9063_REG_COUNT_S + 1)
+#define RTC_SEC 0
+#define RTC_MIN 1
+#define RTC_HOUR 2
+#define RTC_DAY 3
+#define RTC_MONTH 4
+#define RTC_YEAR 5
+
+struct da9063_rtc {
+ struct rtc_device *rtc_dev;
+ struct da9063 *hw;
+ struct rtc_time alarm_time;
+ bool rtc_sync;
+};
+
+static void da9063_data_to_tm(u8 *data, struct rtc_time *tm)
+{
+ tm->tm_sec = data[RTC_SEC] & DA9063_COUNT_SEC_MASK;
+ tm->tm_min = data[RTC_MIN] & DA9063_COUNT_MIN_MASK;
+ tm->tm_hour = data[RTC_HOUR] & DA9063_COUNT_HOUR_MASK;
+ tm->tm_mday = data[RTC_DAY] & DA9063_COUNT_DAY_MASK;
+ tm->tm_mon = MONTHS_FROM_DA9063(data[RTC_MONTH] &
+ DA9063_COUNT_MONTH_MASK);
+ tm->tm_year = YEARS_FROM_DA9063(data[RTC_YEAR] &
+ DA9063_COUNT_YEAR_MASK);
+}
+
+static void da9063_tm_to_data(struct rtc_time *tm, u8 *data)
+{
+ data[RTC_SEC] &= ~DA9063_COUNT_SEC_MASK;
+ data[RTC_SEC] |= tm->tm_sec & DA9063_COUNT_SEC_MASK;
+
+ data[RTC_MIN] &= ~DA9063_COUNT_MIN_MASK;
+ data[RTC_MIN] |= tm->tm_min & DA9063_COUNT_MIN_MASK;
+
+ data[RTC_HOUR] &= ~DA9063_COUNT_HOUR_MASK;
+ data[RTC_HOUR] |= tm->tm_hour & DA9063_COUNT_HOUR_MASK;
+
+ data[RTC_DAY] &= ~DA9063_COUNT_DAY_MASK;
+ data[RTC_DAY] |= tm->tm_mday & DA9063_COUNT_DAY_MASK;
+
+ data[RTC_MONTH] &= ~DA9063_COUNT_MONTH_MASK;
+ data[RTC_MONTH] |= MONTHS_TO_DA9063(tm->tm_mon) &
+ DA9063_COUNT_MONTH_MASK;
+
+ data[RTC_YEAR] &= ~DA9063_COUNT_YEAR_MASK;
+ data[RTC_YEAR] |= YEARS_TO_DA9063(tm->tm_year) &
+ DA9063_COUNT_YEAR_MASK;
+}
+
+static int da9063_rtc_stop_alarm(struct device *dev)
+{
+ struct da9063_rtc *rtc = dev_get_drvdata(dev);
+
+ return regmap_update_bits(rtc->hw->regmap, DA9063_REG_ALARM_Y,
+ DA9063_ALARM_ON, 0);
+}
+
+static int da9063_rtc_start_alarm(struct device *dev)
+{
+ struct da9063_rtc *rtc = dev_get_drvdata(dev);
+
+ return regmap_update_bits(rtc->hw->regmap, DA9063_REG_ALARM_Y,
+ DA9063_ALARM_ON, DA9063_ALARM_ON);
+}
+
+static int da9063_rtc_read_time(struct device *dev, struct rtc_time *tm)
+{
+ struct da9063_rtc *rtc = dev_get_drvdata(dev);
+ unsigned long tm_secs;
+ unsigned long al_secs;
+ u8 data[RTC_DATA_LEN];
+ int ret;
+
+ ret = regmap_bulk_read(rtc->hw->regmap, DA9063_REG_COUNT_S,
+ data, RTC_DATA_LEN);
+ if (ret < 0) {
+ dev_err(dev, "Failed to read RTC time data: %d\n", ret);
+ return ret;
+ }
+
+ if (!(data[RTC_SEC] & DA9063_RTC_READ)) {
+ dev_dbg(dev, "RTC not yet ready to be read by the host\n");
+ return -EINVAL;
+ }
+
+ da9063_data_to_tm(data, tm);
+
+ rtc_tm_to_time(tm, &tm_secs);
+ rtc_tm_to_time(&rtc->alarm_time, &al_secs);
+
+ /* handle the rtc synchronisation delay */
+ if (rtc->rtc_sync == true && al_secs - tm_secs == 1)
+ memcpy(tm, &rtc->alarm_time, sizeof(struct rtc_time));
+ else
+ rtc->rtc_sync = false;
+
+ return rtc_valid_tm(tm);
+}
+
+static int da9063_rtc_set_time(struct device *dev, struct rtc_time *tm)
+{
+ struct da9063_rtc *rtc = dev_get_drvdata(dev);
+ u8 data[RTC_DATA_LEN];
+ int ret;
+
+ da9063_tm_to_data(tm, data);
+ ret = regmap_bulk_write(rtc->hw->regmap, DA9063_REG_COUNT_S,
+ data, RTC_DATA_LEN);
+ if (ret < 0)
+ dev_err(dev, "Failed to set RTC time data: %d\n", ret);
+
+ return ret;
+}
+
+static int da9063_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *alrm)
+{
+ struct da9063_rtc *rtc = dev_get_drvdata(dev);
+ u8 data[RTC_DATA_LEN];
+ int ret;
+ unsigned int val;
+
+ ret = regmap_bulk_read(rtc->hw->regmap, DA9063_REG_ALARM_S,
+ &data[RTC_SEC], RTC_DATA_LEN);
+ if (ret < 0)
+ return ret;
+
+ da9063_data_to_tm(data, &alrm->time);
+
+ alrm->enabled = !!(data[RTC_YEAR] & DA9063_ALARM_ON);
+
+ ret = regmap_read(rtc->hw->regmap, DA9063_REG_EVENT_A, &val);
+ if (ret < 0)
+ return ret;
+
+ if (val & (DA9063_E_ALARM))
+ alrm->pending = 1;
+ else
+ alrm->pending = 0;
+
+ return 0;
+}
+
+static int da9063_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alrm)
+{
+ struct da9063_rtc *rtc = dev_get_drvdata(dev);
+ u8 data[RTC_DATA_LEN];
+ int ret;
+
+ da9063_tm_to_data(&alrm->time, data);
+
+ ret = da9063_rtc_stop_alarm(dev);
+ if (ret < 0) {
+ dev_err(dev, "Failed to stop alarm: %d\n", ret);
+ return ret;
+ }
+
+ ret = regmap_bulk_write(rtc->hw->regmap, DA9063_REG_ALARM_S,
+ data, RTC_DATA_LEN);
+ if (ret < 0) {
+ dev_err(dev, "Failed to write alarm: %d\n", ret);
+ return ret;
+ }
+
+ rtc->alarm_time = alrm->time;
+
+ if (alrm->enabled) {
+ ret = da9063_rtc_start_alarm(dev);
+ if (ret < 0) {
+ dev_err(dev, "Failed to start alarm: %d\n", ret);
+ return ret;
+ }
+ }
+
+ return ret;
+}
+
+static int da9063_rtc_alarm_irq_enable(struct device *dev, unsigned int enabled)
+{
+ if (enabled)
+ return da9063_rtc_start_alarm(dev);
+ else
+ return da9063_rtc_stop_alarm(dev);
+}
+
+static irqreturn_t da9063_alarm_event(int irq, void *data)
+{
+ struct da9063_rtc *rtc = data;
+
+ regmap_update_bits(rtc->hw->regmap, DA9063_REG_ALARM_Y,
+ DA9063_ALARM_ON, 0);
+
+ rtc->rtc_sync = true;
+ rtc_update_irq(rtc->rtc_dev, 1, RTC_IRQF | RTC_AF);
+
+ return IRQ_HANDLED;
+}
+
+static const struct rtc_class_ops da9063_rtc_ops = {
+ .read_time = da9063_rtc_read_time,
+ .set_time = da9063_rtc_set_time,
+ .read_alarm = da9063_rtc_read_alarm,
+ .set_alarm = da9063_rtc_set_alarm,
+ .alarm_irq_enable = da9063_rtc_alarm_irq_enable,
+};
+
+static int da9063_rtc_probe(struct platform_device *pdev)
+{
+ struct da9063 *da9063 = dev_get_drvdata(pdev->dev.parent);
+ struct da9063_rtc *rtc;
+ int irq_alarm;
+ u8 data[RTC_DATA_LEN];
+ int ret;
+
+ ret = regmap_update_bits(da9063->regmap, DA9063_REG_CONTROL_E,
+ DA9063_RTC_EN, DA9063_RTC_EN);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to enable RTC\n");
+ goto err;
+ }
+
+ ret = regmap_update_bits(da9063->regmap, DA9063_REG_EN_32K,
+ DA9063_CRYSTAL, DA9063_CRYSTAL);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to run 32kHz oscillator\n");
+ goto err;
+ }
+
+ ret = regmap_update_bits(da9063->regmap, DA9063_REG_ALARM_S,
+ DA9063_ALARM_STATUS_TICK | DA9063_ALARM_STATUS_ALARM,
+ 0);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to access RTC alarm register\n");
+ goto err;
+ }
+
+ ret = regmap_update_bits(da9063->regmap, DA9063_REG_ALARM_S,
+ DA9063_ALARM_STATUS_ALARM,
+ DA9063_ALARM_STATUS_ALARM);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to access RTC alarm register\n");
+ goto err;
+ }
+
+ ret = regmap_update_bits(da9063->regmap, DA9063_REG_ALARM_Y,
+ DA9063_TICK_ON, 0);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to disable TICKs\n");
+ goto err;
+ }
+
+ ret = regmap_bulk_read(da9063->regmap, DA9063_REG_ALARM_S,
+ data, RTC_DATA_LEN);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to read initial alarm data: %d\n",
+ ret);
+ goto err;
+ }
+
+ rtc = devm_kzalloc(&pdev->dev, sizeof(*rtc), GFP_KERNEL);
+ if (!rtc)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, rtc);
+
+ irq_alarm = platform_get_irq_byname(pdev, "ALARM");
+ ret = devm_request_threaded_irq(&pdev->dev, irq_alarm, NULL,
+ da9063_alarm_event,
+ IRQF_TRIGGER_LOW | IRQF_ONESHOT,
+ "ALARM", rtc);
+ if (ret) {
+ dev_err(&pdev->dev, "Failed to request ALARM IRQ %d: %d\n",
+ irq_alarm, ret);
+ goto err;
+ }
+
+ rtc->hw = da9063;
+ rtc->rtc_dev = devm_rtc_device_register(&pdev->dev, DA9063_DRVNAME_RTC,
+ &da9063_rtc_ops, THIS_MODULE);
+ if (IS_ERR(rtc->rtc_dev))
+ return PTR_ERR(rtc->rtc_dev);
+
+ da9063_data_to_tm(data, &rtc->alarm_time);
+ rtc->rtc_sync = false;
+err:
+ return ret;
+}
+
+static struct platform_driver da9063_rtc_driver = {
+ .probe = da9063_rtc_probe,
+ .driver = {
+ .name = DA9063_DRVNAME_RTC,
+ .owner = THIS_MODULE,
+ },
+};
+
+module_platform_driver(da9063_rtc_driver);
+
+MODULE_AUTHOR("S Twiss <stwiss.opensource@diasemi.com>");
+MODULE_DESCRIPTION("Real time clock device driver for Dialog DA9063");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:" DA9063_DRVNAME_RTC);
diff --git a/drivers/rtc/rtc-ds1343.c b/drivers/rtc/rtc-ds1343.c
new file mode 100644
index 0000000..c371918
--- /dev/null
+++ b/drivers/rtc/rtc-ds1343.c
@@ -0,0 +1,689 @@
+/* rtc-ds1343.c
+ *
+ * Driver for Dallas Semiconductor DS1343 Low Current, SPI Compatible
+ * Real Time Clock
+ *
+ * Author : Raghavendra Chandra Ganiga <ravi23ganiga@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/interrupt.h>
+#include <linux/device.h>
+#include <linux/spi/spi.h>
+#include <linux/regmap.h>
+#include <linux/rtc.h>
+#include <linux/bcd.h>
+#include <linux/pm.h>
+#include <linux/slab.h>
+
+#define DS1343_DRV_VERSION "01.00"
+#define DALLAS_MAXIM_DS1343 0
+#define DALLAS_MAXIM_DS1344 1
+
+/* RTC DS1343 Registers */
+#define DS1343_SECONDS_REG 0x00
+#define DS1343_MINUTES_REG 0x01
+#define DS1343_HOURS_REG 0x02
+#define DS1343_DAY_REG 0x03
+#define DS1343_DATE_REG 0x04
+#define DS1343_MONTH_REG 0x05
+#define DS1343_YEAR_REG 0x06
+#define DS1343_ALM0_SEC_REG 0x07
+#define DS1343_ALM0_MIN_REG 0x08
+#define DS1343_ALM0_HOUR_REG 0x09
+#define DS1343_ALM0_DAY_REG 0x0A
+#define DS1343_ALM1_SEC_REG 0x0B
+#define DS1343_ALM1_MIN_REG 0x0C
+#define DS1343_ALM1_HOUR_REG 0x0D
+#define DS1343_ALM1_DAY_REG 0x0E
+#define DS1343_CONTROL_REG 0x0F
+#define DS1343_STATUS_REG 0x10
+#define DS1343_TRICKLE_REG 0x11
+
+/* DS1343 Control Registers bits */
+#define DS1343_EOSC 0x80
+#define DS1343_DOSF 0x20
+#define DS1343_EGFIL 0x10
+#define DS1343_SQW 0x08
+#define DS1343_INTCN 0x04
+#define DS1343_A1IE 0x02
+#define DS1343_A0IE 0x01
+
+/* DS1343 Status Registers bits */
+#define DS1343_OSF 0x80
+#define DS1343_IRQF1 0x02
+#define DS1343_IRQF0 0x01
+
+/* DS1343 Trickle Charger Registers bits */
+#define DS1343_TRICKLE_MAGIC 0xa0
+#define DS1343_TRICKLE_DS1 0x08
+#define DS1343_TRICKLE_1K 0x01
+#define DS1343_TRICKLE_2K 0x02
+#define DS1343_TRICKLE_4K 0x03
+
+static const struct spi_device_id ds1343_id[] = {
+ { "ds1343", DALLAS_MAXIM_DS1343 },
+ { "ds1344", DALLAS_MAXIM_DS1344 },
+ { }
+};
+MODULE_DEVICE_TABLE(spi, ds1343_id);
+
+struct ds1343_priv {
+ struct spi_device *spi;
+ struct rtc_device *rtc;
+ struct regmap *map;
+ struct mutex mutex;
+ unsigned int irqen;
+ int irq;
+ int alarm_sec;
+ int alarm_min;
+ int alarm_hour;
+ int alarm_mday;
+};
+
+static int ds1343_ioctl(struct device *dev, unsigned int cmd, unsigned long arg)
+{
+ switch (cmd) {
+#ifdef RTC_SET_CHARGE
+ case RTC_SET_CHARGE:
+ {
+ int val;
+
+ if (copy_from_user(&val, (int __user *)arg, sizeof(int)))
+ return -EFAULT;
+
+ return regmap_write(priv->map, DS1343_TRICKLE_REG, val);
+ }
+ break;
+#endif
+ }
+
+ return -ENOIOCTLCMD;
+}
+
+static ssize_t ds1343_show_glitchfilter(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ int glitch_filt_status, data;
+
+ regmap_read(priv->map, DS1343_CONTROL_REG, &data);
+
+ glitch_filt_status = !!(data & DS1343_EGFIL);
+
+ if (glitch_filt_status)
+ return sprintf(buf, "enabled\n");
+ else
+ return sprintf(buf, "disabled\n");
+}
+
+static ssize_t ds1343_store_glitchfilter(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ int data;
+
+ regmap_read(priv->map, DS1343_CONTROL_REG, &data);
+
+ if (strncmp(buf, "enabled", 7) == 0)
+ data |= DS1343_EGFIL;
+
+ else if (strncmp(buf, "disabled", 8) == 0)
+ data &= ~(DS1343_EGFIL);
+
+ else
+ return -EINVAL;
+
+ regmap_write(priv->map, DS1343_CONTROL_REG, data);
+
+ return count;
+}
+
+static DEVICE_ATTR(glitch_filter, S_IRUGO | S_IWUSR, ds1343_show_glitchfilter,
+ ds1343_store_glitchfilter);
+
+static ssize_t ds1343_show_alarmstatus(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ int alarmstatus, data;
+
+ regmap_read(priv->map, DS1343_CONTROL_REG, &data);
+
+ alarmstatus = !!(data & DS1343_A0IE);
+
+ if (alarmstatus)
+ return sprintf(buf, "enabled\n");
+ else
+ return sprintf(buf, "disabled\n");
+}
+
+static DEVICE_ATTR(alarm_status, S_IRUGO, ds1343_show_alarmstatus, NULL);
+
+static ssize_t ds1343_show_alarmmode(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ int alarm_mode, data;
+ char *alarm_str;
+
+ regmap_read(priv->map, DS1343_ALM0_SEC_REG, &data);
+ alarm_mode = (data & 0x80) >> 4;
+
+ regmap_read(priv->map, DS1343_ALM0_MIN_REG, &data);
+ alarm_mode |= (data & 0x80) >> 5;
+
+ regmap_read(priv->map, DS1343_ALM0_HOUR_REG, &data);
+ alarm_mode |= (data & 0x80) >> 6;
+
+ regmap_read(priv->map, DS1343_ALM0_DAY_REG, &data);
+ alarm_mode |= (data & 0x80) >> 7;
+
+ switch (alarm_mode) {
+ case 15:
+ alarm_str = "each second";
+ break;
+
+ case 7:
+ alarm_str = "seconds match";
+ break;
+
+ case 3:
+ alarm_str = "minutes and seconds match";
+ break;
+
+ case 1:
+ alarm_str = "hours, minutes and seconds match";
+ break;
+
+ case 0:
+ alarm_str = "day, hours, minutes and seconds match";
+ break;
+
+ default:
+ alarm_str = "invalid";
+ break;
+ }
+
+ return sprintf(buf, "%s\n", alarm_str);
+}
+
+static DEVICE_ATTR(alarm_mode, S_IRUGO, ds1343_show_alarmmode, NULL);
+
+static ssize_t ds1343_show_tricklecharger(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ int data;
+ char *diodes = "disabled", *resistors = " ";
+
+ regmap_read(priv->map, DS1343_TRICKLE_REG, &data);
+
+ if ((data & 0xf0) == DS1343_TRICKLE_MAGIC) {
+ switch (data & 0x0c) {
+ case DS1343_TRICKLE_DS1:
+ diodes = "one diode,";
+ break;
+
+ default:
+ diodes = "no diode,";
+ break;
+ }
+
+ switch (data & 0x03) {
+ case DS1343_TRICKLE_1K:
+ resistors = "1k Ohm";
+ break;
+
+ case DS1343_TRICKLE_2K:
+ resistors = "2k Ohm";
+ break;
+
+ case DS1343_TRICKLE_4K:
+ resistors = "4k Ohm";
+ break;
+
+ default:
+ diodes = "disabled";
+ break;
+ }
+ }
+
+ return sprintf(buf, "%s %s\n", diodes, resistors);
+}
+
+static DEVICE_ATTR(trickle_charger, S_IRUGO, ds1343_show_tricklecharger, NULL);
+
+static int ds1343_sysfs_register(struct device *dev)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ int err;
+
+ err = device_create_file(dev, &dev_attr_glitch_filter);
+ if (err)
+ return err;
+
+ err = device_create_file(dev, &dev_attr_trickle_charger);
+ if (err)
+ goto error1;
+
+ if (priv->irq <= 0)
+ return err;
+
+ err = device_create_file(dev, &dev_attr_alarm_mode);
+ if (err)
+ goto error2;
+
+ err = device_create_file(dev, &dev_attr_alarm_status);
+ if (!err)
+ return err;
+
+ device_remove_file(dev, &dev_attr_alarm_mode);
+
+error2:
+ device_remove_file(dev, &dev_attr_trickle_charger);
+
+error1:
+ device_remove_file(dev, &dev_attr_glitch_filter);
+
+ return err;
+}
+
+static void ds1343_sysfs_unregister(struct device *dev)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+
+ device_remove_file(dev, &dev_attr_glitch_filter);
+ device_remove_file(dev, &dev_attr_trickle_charger);
+
+ if (priv->irq <= 0)
+ return;
+
+ device_remove_file(dev, &dev_attr_alarm_status);
+ device_remove_file(dev, &dev_attr_alarm_mode);
+}
+
+static int ds1343_read_time(struct device *dev, struct rtc_time *dt)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ unsigned char buf[7];
+ int res;
+
+ res = regmap_bulk_read(priv->map, DS1343_SECONDS_REG, buf, 7);
+ if (res)
+ return res;
+
+ dt->tm_sec = bcd2bin(buf[0]);
+ dt->tm_min = bcd2bin(buf[1]);
+ dt->tm_hour = bcd2bin(buf[2] & 0x3F);
+ dt->tm_wday = bcd2bin(buf[3]) - 1;
+ dt->tm_mday = bcd2bin(buf[4]);
+ dt->tm_mon = bcd2bin(buf[5] & 0x1F) - 1;
+ dt->tm_year = bcd2bin(buf[6]) + 100; /* year offset from 1900 */
+
+ return rtc_valid_tm(dt);
+}
+
+static int ds1343_set_time(struct device *dev, struct rtc_time *dt)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ int res;
+
+ res = regmap_write(priv->map, DS1343_SECONDS_REG,
+ bin2bcd(dt->tm_sec));
+ if (res)
+ return res;
+
+ res = regmap_write(priv->map, DS1343_MINUTES_REG,
+ bin2bcd(dt->tm_min));
+ if (res)
+ return res;
+
+ res = regmap_write(priv->map, DS1343_HOURS_REG,
+ bin2bcd(dt->tm_hour) & 0x3F);
+ if (res)
+ return res;
+
+ res = regmap_write(priv->map, DS1343_DAY_REG,
+ bin2bcd(dt->tm_wday + 1));
+ if (res)
+ return res;
+
+ res = regmap_write(priv->map, DS1343_DATE_REG,
+ bin2bcd(dt->tm_mday));
+ if (res)
+ return res;
+
+ res = regmap_write(priv->map, DS1343_MONTH_REG,
+ bin2bcd(dt->tm_mon + 1));
+ if (res)
+ return res;
+
+ dt->tm_year %= 100;
+
+ res = regmap_write(priv->map, DS1343_YEAR_REG,
+ bin2bcd(dt->tm_year));
+ if (res)
+ return res;
+
+ return 0;
+}
+
+static int ds1343_update_alarm(struct device *dev)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ unsigned int control, stat;
+ unsigned char buf[4];
+ int res = 0;
+
+ res = regmap_read(priv->map, DS1343_CONTROL_REG, &control);
+ if (res)
+ return res;
+
+ res = regmap_read(priv->map, DS1343_STATUS_REG, &stat);
+ if (res)
+ return res;
+
+ control &= ~(DS1343_A0IE);
+ stat &= ~(DS1343_IRQF0);
+
+ res = regmap_write(priv->map, DS1343_CONTROL_REG, control);
+ if (res)
+ return res;
+
+ res = regmap_write(priv->map, DS1343_STATUS_REG, stat);
+ if (res)
+ return res;
+
+ buf[0] = priv->alarm_sec < 0 || (priv->irqen & RTC_UF) ?
+ 0x80 : bin2bcd(priv->alarm_sec) & 0x7F;
+ buf[1] = priv->alarm_min < 0 || (priv->irqen & RTC_UF) ?
+ 0x80 : bin2bcd(priv->alarm_min) & 0x7F;
+ buf[2] = priv->alarm_hour < 0 || (priv->irqen & RTC_UF) ?
+ 0x80 : bin2bcd(priv->alarm_hour) & 0x3F;
+ buf[3] = priv->alarm_mday < 0 || (priv->irqen & RTC_UF) ?
+ 0x80 : bin2bcd(priv->alarm_mday) & 0x7F;
+
+ res = regmap_bulk_write(priv->map, DS1343_ALM0_SEC_REG, buf, 4);
+ if (res)
+ return res;
+
+ if (priv->irqen) {
+ control |= DS1343_A0IE;
+ res = regmap_write(priv->map, DS1343_CONTROL_REG, control);
+ }
+
+ return res;
+}
+
+static int ds1343_read_alarm(struct device *dev, struct rtc_wkalrm *alarm)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ int res = 0;
+ unsigned int stat;
+
+ if (priv->irq <= 0)
+ return -EINVAL;
+
+ mutex_lock(&priv->mutex);
+
+ res = regmap_read(priv->map, DS1343_STATUS_REG, &stat);
+ if (res)
+ goto out;
+
+ alarm->enabled = !!(priv->irqen & RTC_AF);
+ alarm->pending = !!(stat & DS1343_IRQF0);
+
+ alarm->time.tm_sec = priv->alarm_sec < 0 ? 0 : priv->alarm_sec;
+ alarm->time.tm_min = priv->alarm_min < 0 ? 0 : priv->alarm_min;
+ alarm->time.tm_hour = priv->alarm_hour < 0 ? 0 : priv->alarm_hour;
+ alarm->time.tm_mday = priv->alarm_mday < 0 ? 0 : priv->alarm_mday;
+
+ alarm->time.tm_mon = -1;
+ alarm->time.tm_year = -1;
+ alarm->time.tm_wday = -1;
+ alarm->time.tm_yday = -1;
+ alarm->time.tm_isdst = -1;
+
+out:
+ mutex_unlock(&priv->mutex);
+ return res;
+}
+
+static int ds1343_set_alarm(struct device *dev, struct rtc_wkalrm *alarm)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ int res = 0;
+
+ if (priv->irq <= 0)
+ return -EINVAL;
+
+ mutex_lock(&priv->mutex);
+
+ priv->alarm_sec = alarm->time.tm_sec;
+ priv->alarm_min = alarm->time.tm_min;
+ priv->alarm_hour = alarm->time.tm_hour;
+ priv->alarm_mday = alarm->time.tm_mday;
+
+ if (alarm->enabled)
+ priv->irqen |= RTC_AF;
+
+ res = ds1343_update_alarm(dev);
+
+ mutex_unlock(&priv->mutex);
+
+ return res;
+}
+
+static int ds1343_alarm_irq_enable(struct device *dev, unsigned int enabled)
+{
+ struct ds1343_priv *priv = dev_get_drvdata(dev);
+ int res = 0;
+
+ if (priv->irq <= 0)
+ return -EINVAL;
+
+ mutex_lock(&priv->mutex);
+
+ if (enabled)
+ priv->irqen |= RTC_AF;
+ else
+ priv->irqen &= ~RTC_AF;
+
+ res = ds1343_update_alarm(dev);
+
+ mutex_unlock(&priv->mutex);
+
+ return res;
+}
+
+static irqreturn_t ds1343_thread(int irq, void *dev_id)
+{
+ struct ds1343_priv *priv = dev_id;
+ unsigned int stat, control;
+ int res = 0;
+
+ mutex_lock(&priv->mutex);
+
+ res = regmap_read(priv->map, DS1343_STATUS_REG, &stat);
+ if (res)
+ goto out;
+
+ if (stat & DS1343_IRQF0) {
+ stat &= ~DS1343_IRQF0;
+ regmap_write(priv->map, DS1343_STATUS_REG, stat);
+
+ res = regmap_read(priv->map, DS1343_CONTROL_REG, &control);
+ if (res)
+ goto out;
+
+ control &= ~DS1343_A0IE;
+ regmap_write(priv->map, DS1343_CONTROL_REG, control);
+
+ rtc_update_irq(priv->rtc, 1, RTC_AF | RTC_IRQF);
+ }
+
+out:
+ mutex_unlock(&priv->mutex);
+ return IRQ_HANDLED;
+}
+
+static const struct rtc_class_ops ds1343_rtc_ops = {
+ .ioctl = ds1343_ioctl,
+ .read_time = ds1343_read_time,
+ .set_time = ds1343_set_time,
+ .read_alarm = ds1343_read_alarm,
+ .set_alarm = ds1343_set_alarm,
+ .alarm_irq_enable = ds1343_alarm_irq_enable,
+};
+
+static int ds1343_probe(struct spi_device *spi)
+{
+ struct ds1343_priv *priv;
+ struct regmap_config config;
+ unsigned int data;
+ int res;
+
+ memset(&config, 0, sizeof(config));
+ config.reg_bits = 8;
+ config.val_bits = 8;
+ config.write_flag_mask = 0x80;
+
+ priv = devm_kzalloc(&spi->dev, sizeof(struct ds1343_priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->spi = spi;
+ mutex_init(&priv->mutex);
+
+ /* RTC DS1347 works in spi mode 3 and
+ * its chip select is active high
+ */
+ spi->mode = SPI_MODE_3 | SPI_CS_HIGH;
+ spi->bits_per_word = 8;
+ res = spi_setup(spi);
+ if (res)
+ return res;
+
+ spi_set_drvdata(spi, priv);
+
+ priv->map = devm_regmap_init_spi(spi, &config);
+
+ if (IS_ERR(priv->map)) {
+ dev_err(&spi->dev, "spi regmap init failed for rtc ds1343\n");
+ return PTR_ERR(priv->map);
+ }
+
+ res = regmap_read(priv->map, DS1343_SECONDS_REG, &data);
+ if (res)
+ return res;
+
+ regmap_read(priv->map, DS1343_CONTROL_REG, &data);
+ data |= DS1343_INTCN;
+ data &= ~(DS1343_EOSC | DS1343_A1IE | DS1343_A0IE);
+ regmap_write(priv->map, DS1343_CONTROL_REG, data);
+
+ regmap_read(priv->map, DS1343_STATUS_REG, &data);
+ data &= ~(DS1343_OSF | DS1343_IRQF1 | DS1343_IRQF0);
+ regmap_write(priv->map, DS1343_STATUS_REG, data);
+
+ priv->rtc = devm_rtc_device_register(&spi->dev, "ds1343",
+ &ds1343_rtc_ops, THIS_MODULE);
+ if (IS_ERR(priv->rtc)) {
+ dev_err(&spi->dev, "unable to register rtc ds1343\n");
+ return PTR_ERR(priv->rtc);
+ }
+
+ priv->irq = spi->irq;
+
+ if (priv->irq >= 0) {
+ res = devm_request_threaded_irq(&spi->dev, spi->irq, NULL,
+ ds1343_thread,
+ IRQF_NO_SUSPEND | IRQF_ONESHOT,
+ "ds1343", priv);
+ if (res) {
+ priv->irq = -1;
+ dev_err(&spi->dev,
+ "unable to request irq for rtc ds1343\n");
+ } else {
+ device_set_wakeup_capable(&spi->dev, 1);
+ }
+ }
+
+ res = ds1343_sysfs_register(&spi->dev);
+ if (res)
+ dev_err(&spi->dev,
+ "unable to create sysfs entries for rtc ds1343\n");
+
+ return 0;
+}
+
+static int ds1343_remove(struct spi_device *spi)
+{
+ struct ds1343_priv *priv = spi_get_drvdata(spi);
+
+ if (spi->irq) {
+ mutex_lock(&priv->mutex);
+ priv->irqen &= ~RTC_AF;
+ mutex_unlock(&priv->mutex);
+
+ devm_free_irq(&spi->dev, spi->irq, priv);
+ }
+
+ spi_set_drvdata(spi, NULL);
+
+ ds1343_sysfs_unregister(&spi->dev);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP
+
+static int ds1343_suspend(struct device *dev)
+{
+ struct spi_device *spi = to_spi_device(dev);
+
+ if (spi->irq >= 0 && device_may_wakeup(dev))
+ enable_irq_wake(spi->irq);
+
+ return 0;
+}
+
+static int ds1343_resume(struct device *dev)
+{
+ struct spi_device *spi = to_spi_device(dev);
+
+ if (spi->irq >= 0 && device_may_wakeup(dev))
+ disable_irq_wake(spi->irq);
+
+ return 0;
+}
+
+#endif
+
+static SIMPLE_DEV_PM_OPS(ds1343_pm, ds1343_suspend, ds1343_resume);
+
+static struct spi_driver ds1343_driver = {
+ .driver = {
+ .name = "ds1343",
+ .owner = THIS_MODULE,
+ .pm = &ds1343_pm,
+ },
+ .probe = ds1343_probe,
+ .remove = ds1343_remove,
+ .id_table = ds1343_id,
+};
+
+module_spi_driver(ds1343_driver);
+
+MODULE_DESCRIPTION("DS1343 RTC SPI Driver");
+MODULE_AUTHOR("Raghavendra Chandra Ganiga <ravi23ganiga@gmail.com>");
+MODULE_LICENSE("GPL v2");
+MODULE_VERSION(DS1343_DRV_VERSION);
diff --git a/drivers/rtc/rtc-ds1742.c b/drivers/rtc/rtc-ds1742.c
index 942103d..c6b2191 100644
--- a/drivers/rtc/rtc-ds1742.c
+++ b/drivers/rtc/rtc-ds1742.c
@@ -219,7 +219,7 @@
return 0;
}
-static struct of_device_id __maybe_unused ds1742_rtc_of_match[] = {
+static const struct of_device_id __maybe_unused ds1742_rtc_of_match[] = {
{ .compatible = "maxim,ds1742", },
{ }
};
diff --git a/drivers/rtc/rtc-efi.c b/drivers/rtc/rtc-efi.c
index 797aa02..c4c3843 100644
--- a/drivers/rtc/rtc-efi.c
+++ b/drivers/rtc/rtc-efi.c
@@ -35,7 +35,7 @@
compute_yday(efi_time_t *eft)
{
/* efi_time_t.month is in the [1-12] so, we need -1 */
- return rtc_year_days(eft->day - 1, eft->month - 1, eft->year);
+ return rtc_year_days(eft->day, eft->month - 1, eft->year);
}
/*
* returns day of the week [0-6] 0=Sunday
diff --git a/drivers/rtc/rtc-hym8563.c b/drivers/rtc/rtc-hym8563.c
index e5f13c4..b936bb4 100644
--- a/drivers/rtc/rtc-hym8563.c
+++ b/drivers/rtc/rtc-hym8563.c
@@ -418,6 +418,9 @@
init.num_parents = 0;
hym8563->clkout_hw.init = &init;
+ /* optional override of the clockname */
+ of_property_read_string(node, "clock-output-names", &init.name);
+
/* register the clock */
clk = clk_register(&client->dev, &hym8563->clkout_hw);
@@ -585,7 +588,7 @@
};
MODULE_DEVICE_TABLE(i2c, hym8563_id);
-static struct of_device_id hym8563_dt_idtable[] = {
+static const struct of_device_id hym8563_dt_idtable[] = {
{ .compatible = "haoyu,hym8563" },
{},
};
diff --git a/drivers/rtc/rtc-isl12057.c b/drivers/rtc/rtc-isl12057.c
index 41bd76a..455b601 100644
--- a/drivers/rtc/rtc-isl12057.c
+++ b/drivers/rtc/rtc-isl12057.c
@@ -278,7 +278,7 @@
}
#ifdef CONFIG_OF
-static struct of_device_id isl12057_dt_match[] = {
+static const struct of_device_id isl12057_dt_match[] = {
{ .compatible = "isl,isl12057" },
{ },
};
diff --git a/drivers/rtc/rtc-m41t80.c b/drivers/rtc/rtc-m41t80.c
index a5248aa..7ff7427 100644
--- a/drivers/rtc/rtc-m41t80.c
+++ b/drivers/rtc/rtc-m41t80.c
@@ -66,8 +66,6 @@
#define M41T80_FEATURE_WD (1 << 3) /* Extra watchdog resolution */
#define M41T80_FEATURE_SQ_ALT (1 << 4) /* RSx bits are in reg 4 */
-#define DRV_VERSION "0.05"
-
static DEFINE_MUTEX(m41t80_rtc_mutex);
static const struct i2c_device_id m41t80_id[] = {
{ "m41t62", M41T80_FEATURE_SQ | M41T80_FEATURE_SQ_ALT },
@@ -80,6 +78,7 @@
{ "m41st84", M41T80_FEATURE_HT | M41T80_FEATURE_BL | M41T80_FEATURE_SQ },
{ "m41st85", M41T80_FEATURE_HT | M41T80_FEATURE_BL | M41T80_FEATURE_SQ },
{ "m41st87", M41T80_FEATURE_HT | M41T80_FEATURE_BL | M41T80_FEATURE_SQ },
+ { "rv4162", M41T80_FEATURE_SQ | M41T80_FEATURE_WD | M41T80_FEATURE_SQ_ALT },
{ }
};
MODULE_DEVICE_TABLE(i2c, m41t80_id);
@@ -232,7 +231,7 @@
val = i2c_smbus_read_byte_data(client, M41T80_REG_FLAGS);
if (val < 0)
- return -EIO;
+ return val;
return sprintf(buf, "%#x\n", val);
}
static DEVICE_ATTR(flags, S_IRUGO, m41t80_sysfs_show_flags, NULL);
@@ -252,7 +251,7 @@
reg_sqw = M41T80_REG_WDAY;
val = i2c_smbus_read_byte_data(client, reg_sqw);
if (val < 0)
- return -EIO;
+ return val;
val = (val >> 4) & 0xf;
switch (val) {
case 0:
@@ -271,7 +270,7 @@
{
struct i2c_client *client = to_i2c_client(dev);
struct m41t80_data *clientdata = i2c_get_clientdata(client);
- int almon, sqw, reg_sqw;
+ int almon, sqw, reg_sqw, rc;
int val = simple_strtoul(buf, NULL, 0);
if (!(clientdata->features & M41T80_FEATURE_SQ))
@@ -291,21 +290,30 @@
/* disable SQW, set SQW frequency & re-enable */
almon = i2c_smbus_read_byte_data(client, M41T80_REG_ALARM_MON);
if (almon < 0)
- return -EIO;
+ return almon;
reg_sqw = M41T80_REG_SQW;
if (clientdata->features & M41T80_FEATURE_SQ_ALT)
reg_sqw = M41T80_REG_WDAY;
sqw = i2c_smbus_read_byte_data(client, reg_sqw);
if (sqw < 0)
- return -EIO;
+ return sqw;
sqw = (sqw & 0x0f) | (val << 4);
- if (i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_MON,
- almon & ~M41T80_ALMON_SQWE) < 0 ||
- i2c_smbus_write_byte_data(client, reg_sqw, sqw) < 0)
- return -EIO;
- if (val && i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_MON,
- almon | M41T80_ALMON_SQWE) < 0)
- return -EIO;
+
+ rc = i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_MON,
+ almon & ~M41T80_ALMON_SQWE);
+ if (rc < 0)
+ return rc;
+
+ if (val) {
+ rc = i2c_smbus_write_byte_data(client, reg_sqw, sqw);
+ if (rc < 0)
+ return rc;
+
+ rc = i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_MON,
+ almon | M41T80_ALMON_SQWE);
+ if (rc <0)
+ return rc;
+ }
return count;
}
static DEVICE_ATTR(sqwfreq, S_IRUGO | S_IWUSR,
@@ -629,40 +637,28 @@
struct m41t80_data *clientdata = NULL;
if (!i2c_check_functionality(client->adapter, I2C_FUNC_I2C
- | I2C_FUNC_SMBUS_BYTE_DATA)) {
- rc = -ENODEV;
- goto exit;
- }
-
- dev_info(&client->dev,
- "chip found, driver version " DRV_VERSION "\n");
+ | I2C_FUNC_SMBUS_BYTE_DATA))
+ return -ENODEV;
clientdata = devm_kzalloc(&client->dev, sizeof(*clientdata),
GFP_KERNEL);
- if (!clientdata) {
- rc = -ENOMEM;
- goto exit;
- }
+ if (!clientdata)
+ return -ENOMEM;
clientdata->features = id->driver_data;
i2c_set_clientdata(client, clientdata);
rtc = devm_rtc_device_register(&client->dev, client->name,
&m41t80_rtc_ops, THIS_MODULE);
- if (IS_ERR(rtc)) {
- rc = PTR_ERR(rtc);
- rtc = NULL;
- goto exit;
- }
+ if (IS_ERR(rtc))
+ return PTR_ERR(rtc);
clientdata->rtc = rtc;
/* Make sure HT (Halt Update) bit is cleared */
rc = i2c_smbus_read_byte_data(client, M41T80_REG_ALARM_HOUR);
- if (rc < 0)
- goto ht_err;
- if (rc & M41T80_ALHOUR_HT) {
+ if (rc >= 0 && rc & M41T80_ALHOUR_HT) {
if (clientdata->features & M41T80_FEATURE_HT) {
m41t80_get_datetime(client, &tm);
dev_info(&client->dev, "HT bit was set!\n");
@@ -673,53 +669,44 @@
tm.tm_mon + 1, tm.tm_mday, tm.tm_hour,
tm.tm_min, tm.tm_sec);
}
- if (i2c_smbus_write_byte_data(client,
- M41T80_REG_ALARM_HOUR,
- rc & ~M41T80_ALHOUR_HT) < 0)
- goto ht_err;
+ rc = i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_HOUR,
+ rc & ~M41T80_ALHOUR_HT);
+ }
+
+ if (rc < 0) {
+ dev_err(&client->dev, "Can't clear HT bit\n");
+ return rc;
}
/* Make sure ST (stop) bit is cleared */
rc = i2c_smbus_read_byte_data(client, M41T80_REG_SEC);
- if (rc < 0)
- goto st_err;
- if (rc & M41T80_SEC_ST) {
- if (i2c_smbus_write_byte_data(client, M41T80_REG_SEC,
- rc & ~M41T80_SEC_ST) < 0)
- goto st_err;
+ if (rc >= 0 && rc & M41T80_SEC_ST)
+ rc = i2c_smbus_write_byte_data(client, M41T80_REG_SEC,
+ rc & ~M41T80_SEC_ST);
+ if (rc < 0) {
+ dev_err(&client->dev, "Can't clear ST bit\n");
+ return rc;
}
rc = m41t80_sysfs_register(&client->dev);
if (rc)
- goto exit;
+ return rc;
#ifdef CONFIG_RTC_DRV_M41T80_WDT
if (clientdata->features & M41T80_FEATURE_HT) {
save_client = client;
rc = misc_register(&wdt_dev);
if (rc)
- goto exit;
+ return rc;
rc = register_reboot_notifier(&wdt_notifier);
if (rc) {
misc_deregister(&wdt_dev);
- goto exit;
+ return rc;
}
}
#endif
return 0;
-
-st_err:
- rc = -EIO;
- dev_err(&client->dev, "Can't clear ST bit\n");
- goto exit;
-ht_err:
- rc = -EIO;
- dev_err(&client->dev, "Can't clear HT bit\n");
- goto exit;
-
-exit:
- return rc;
}
static int m41t80_remove(struct i2c_client *client)
@@ -750,4 +737,3 @@
MODULE_AUTHOR("Alexander Bigga <ab@mycable.de>");
MODULE_DESCRIPTION("ST Microelectronics M41T80 series RTC I2C Client Driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-mcp795.c b/drivers/rtc/rtc-mcp795.c
new file mode 100644
index 0000000..34295bf
--- /dev/null
+++ b/drivers/rtc/rtc-mcp795.c
@@ -0,0 +1,199 @@
+/*
+ * SPI Driver for Microchip MCP795 RTC
+ *
+ * Copyright (C) Josef Gajdusek <atx@atx.name>
+ *
+ * based on other Linux RTC drivers
+ *
+ * Device datasheet:
+ * http://ww1.microchip.com/downloads/en/DeviceDoc/22280A.pdf
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * */
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/device.h>
+#include <linux/printk.h>
+#include <linux/spi/spi.h>
+#include <linux/rtc.h>
+
+/* MCP795 Instructions, see datasheet table 3-1 */
+#define MCP795_EEREAD 0x03
+#define MCP795_EEWRITE 0x02
+#define MCP795_EEWRDI 0x04
+#define MCP795_EEWREN 0x06
+#define MCP795_SRREAD 0x05
+#define MCP795_SRWRITE 0x01
+#define MCP795_READ 0x13
+#define MCP795_WRITE 0x12
+#define MCP795_UNLOCK 0x14
+#define MCP795_IDWRITE 0x32
+#define MCP795_IDREAD 0x33
+#define MCP795_CLRWDT 0x44
+#define MCP795_CLRRAM 0x54
+
+#define MCP795_ST_BIT 0x80
+#define MCP795_24_BIT 0x40
+
+static int mcp795_rtcc_read(struct device *dev, u8 addr, u8 *buf, u8 count)
+{
+ struct spi_device *spi = to_spi_device(dev);
+ int ret;
+ u8 tx[2];
+
+ tx[0] = MCP795_READ;
+ tx[1] = addr;
+ ret = spi_write_then_read(spi, tx, sizeof(tx), buf, count);
+
+ if (ret)
+ dev_err(dev, "Failed reading %d bytes from address %x.\n",
+ count, addr);
+
+ return ret;
+}
+
+static int mcp795_rtcc_write(struct device *dev, u8 addr, u8 *data, u8 count)
+{
+ struct spi_device *spi = to_spi_device(dev);
+ int ret;
+ u8 tx[2 + count];
+
+ tx[0] = MCP795_WRITE;
+ tx[1] = addr;
+ memcpy(&tx[2], data, count);
+
+ ret = spi_write(spi, tx, 2 + count);
+
+ if (ret)
+ dev_err(dev, "Failed to write %d bytes to address %x.\n",
+ count, addr);
+
+ return ret;
+}
+
+static int mcp795_rtcc_set_bits(struct device *dev, u8 addr, u8 mask, u8 state)
+{
+ int ret;
+ u8 tmp;
+
+ ret = mcp795_rtcc_read(dev, addr, &tmp, 1);
+ if (ret)
+ return ret;
+
+ if ((tmp & mask) != state) {
+ tmp = (tmp & ~mask) | state;
+ ret = mcp795_rtcc_write(dev, addr, &tmp, 1);
+ }
+
+ return ret;
+}
+
+static int mcp795_set_time(struct device *dev, struct rtc_time *tim)
+{
+ int ret;
+ u8 data[7];
+
+ /* Read first, so we can leave config bits untouched */
+ ret = mcp795_rtcc_read(dev, 0x01, data, sizeof(data));
+
+ if (ret)
+ return ret;
+
+ data[0] = (data[0] & 0x80) | ((tim->tm_sec / 10) << 4) | (tim->tm_sec % 10);
+ data[1] = (data[1] & 0x80) | ((tim->tm_min / 10) << 4) | (tim->tm_min % 10);
+ data[2] = ((tim->tm_hour / 10) << 4) | (tim->tm_hour % 10);
+ data[4] = ((tim->tm_mday / 10) << 4) | ((tim->tm_mday) % 10);
+ data[5] = (data[5] & 0x10) | (tim->tm_mon / 10) | (tim->tm_mon % 10);
+
+ if (tim->tm_year > 100)
+ tim->tm_year -= 100;
+
+ data[6] = ((tim->tm_year / 10) << 4) | (tim->tm_year % 10);
+
+ ret = mcp795_rtcc_write(dev, 0x01, data, sizeof(data));
+
+ if (ret)
+ return ret;
+
+ dev_dbg(dev, "Set mcp795: %04d-%02d-%02d %02d:%02d:%02d\n",
+ tim->tm_year + 1900, tim->tm_mon, tim->tm_mday,
+ tim->tm_hour, tim->tm_min, tim->tm_sec);
+
+ return 0;
+}
+
+static int mcp795_read_time(struct device *dev, struct rtc_time *tim)
+{
+ int ret;
+ u8 data[7];
+
+ ret = mcp795_rtcc_read(dev, 0x01, data, sizeof(data));
+
+ if (ret)
+ return ret;
+
+ tim->tm_sec = ((data[0] & 0x70) >> 4) * 10 + (data[0] & 0x0f);
+ tim->tm_min = ((data[1] & 0x70) >> 4) * 10 + (data[1] & 0x0f);
+ tim->tm_hour = ((data[2] & 0x30) >> 4) * 10 + (data[2] & 0x0f);
+ tim->tm_mday = ((data[4] & 0x30) >> 4) * 10 + (data[4] & 0x0f);
+ tim->tm_mon = ((data[5] & 0x10) >> 4) * 10 + (data[5] & 0x0f);
+ tim->tm_year = ((data[6] & 0xf0) >> 4) * 10 + (data[6] & 0x0f) + 100; /* Assume we are in 20xx */
+
+ dev_dbg(dev, "Read from mcp795: %04d-%02d-%02d %02d:%02d:%02d\n",
+ tim->tm_year + 1900, tim->tm_mon, tim->tm_mday,
+ tim->tm_hour, tim->tm_min, tim->tm_sec);
+
+ return rtc_valid_tm(tim);
+}
+
+static struct rtc_class_ops mcp795_rtc_ops = {
+ .read_time = mcp795_read_time,
+ .set_time = mcp795_set_time
+};
+
+static int mcp795_probe(struct spi_device *spi)
+{
+ struct rtc_device *rtc;
+ int ret;
+
+ spi->mode = SPI_MODE_0;
+ spi->bits_per_word = 8;
+ ret = spi_setup(spi);
+ if (ret) {
+ dev_err(&spi->dev, "Unable to setup SPI\n");
+ return ret;
+ }
+
+ /* Start the oscillator */
+ mcp795_rtcc_set_bits(&spi->dev, 0x01, MCP795_ST_BIT, MCP795_ST_BIT);
+ /* Clear the 12 hour mode flag*/
+ mcp795_rtcc_set_bits(&spi->dev, 0x03, MCP795_24_BIT, 0);
+
+ rtc = devm_rtc_device_register(&spi->dev, "rtc-mcp795",
+ &mcp795_rtc_ops, THIS_MODULE);
+ if (IS_ERR(rtc))
+ return PTR_ERR(rtc);
+
+ spi_set_drvdata(spi, rtc);
+
+ return 0;
+}
+
+static struct spi_driver mcp795_driver = {
+ .driver = {
+ .name = "rtc-mcp795",
+ .owner = THIS_MODULE,
+ },
+ .probe = mcp795_probe,
+};
+
+module_spi_driver(mcp795_driver);
+
+MODULE_DESCRIPTION("MCP795 RTC SPI Driver");
+MODULE_AUTHOR("Josef Gajdusek <atx@atx.name>");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("spi:mcp795");
diff --git a/drivers/rtc/rtc-mv.c b/drivers/rtc/rtc-mv.c
index d15a999..6aaec2f 100644
--- a/drivers/rtc/rtc-mv.c
+++ b/drivers/rtc/rtc-mv.c
@@ -319,7 +319,7 @@
}
#ifdef CONFIG_OF
-static struct of_device_id rtc_mv_of_match_table[] = {
+static const struct of_device_id rtc_mv_of_match_table[] = {
{ .compatible = "marvell,orion-rtc", },
{}
};
diff --git a/drivers/rtc/rtc-omap.c b/drivers/rtc/rtc-omap.c
index 26de5f8..21142e6 100644
--- a/drivers/rtc/rtc-omap.c
+++ b/drivers/rtc/rtc-omap.c
@@ -73,43 +73,52 @@
#define OMAP_RTC_IRQWAKEEN 0x7c
/* OMAP_RTC_CTRL_REG bit fields: */
-#define OMAP_RTC_CTRL_SPLIT (1<<7)
-#define OMAP_RTC_CTRL_DISABLE (1<<6)
-#define OMAP_RTC_CTRL_SET_32_COUNTER (1<<5)
-#define OMAP_RTC_CTRL_TEST (1<<4)
-#define OMAP_RTC_CTRL_MODE_12_24 (1<<3)
-#define OMAP_RTC_CTRL_AUTO_COMP (1<<2)
-#define OMAP_RTC_CTRL_ROUND_30S (1<<1)
-#define OMAP_RTC_CTRL_STOP (1<<0)
+#define OMAP_RTC_CTRL_SPLIT BIT(7)
+#define OMAP_RTC_CTRL_DISABLE BIT(6)
+#define OMAP_RTC_CTRL_SET_32_COUNTER BIT(5)
+#define OMAP_RTC_CTRL_TEST BIT(4)
+#define OMAP_RTC_CTRL_MODE_12_24 BIT(3)
+#define OMAP_RTC_CTRL_AUTO_COMP BIT(2)
+#define OMAP_RTC_CTRL_ROUND_30S BIT(1)
+#define OMAP_RTC_CTRL_STOP BIT(0)
/* OMAP_RTC_STATUS_REG bit fields: */
-#define OMAP_RTC_STATUS_POWER_UP (1<<7)
-#define OMAP_RTC_STATUS_ALARM (1<<6)
-#define OMAP_RTC_STATUS_1D_EVENT (1<<5)
-#define OMAP_RTC_STATUS_1H_EVENT (1<<4)
-#define OMAP_RTC_STATUS_1M_EVENT (1<<3)
-#define OMAP_RTC_STATUS_1S_EVENT (1<<2)
-#define OMAP_RTC_STATUS_RUN (1<<1)
-#define OMAP_RTC_STATUS_BUSY (1<<0)
+#define OMAP_RTC_STATUS_POWER_UP BIT(7)
+#define OMAP_RTC_STATUS_ALARM BIT(6)
+#define OMAP_RTC_STATUS_1D_EVENT BIT(5)
+#define OMAP_RTC_STATUS_1H_EVENT BIT(4)
+#define OMAP_RTC_STATUS_1M_EVENT BIT(3)
+#define OMAP_RTC_STATUS_1S_EVENT BIT(2)
+#define OMAP_RTC_STATUS_RUN BIT(1)
+#define OMAP_RTC_STATUS_BUSY BIT(0)
/* OMAP_RTC_INTERRUPTS_REG bit fields: */
-#define OMAP_RTC_INTERRUPTS_IT_ALARM (1<<3)
-#define OMAP_RTC_INTERRUPTS_IT_TIMER (1<<2)
+#define OMAP_RTC_INTERRUPTS_IT_ALARM BIT(3)
+#define OMAP_RTC_INTERRUPTS_IT_TIMER BIT(2)
+
+/* OMAP_RTC_OSC_REG bit fields: */
+#define OMAP_RTC_OSC_32KCLK_EN BIT(6)
/* OMAP_RTC_IRQWAKEEN bit fields: */
-#define OMAP_RTC_IRQWAKEEN_ALARM_WAKEEN (1<<1)
+#define OMAP_RTC_IRQWAKEEN_ALARM_WAKEEN BIT(1)
/* OMAP_RTC_KICKER values */
#define KICK0_VALUE 0x83e70b13
#define KICK1_VALUE 0x95a4f1e0
-#define OMAP_RTC_HAS_KICKER 0x1
+#define OMAP_RTC_HAS_KICKER BIT(0)
/*
* Few RTC IP revisions has special WAKE-EN Register to enable Wakeup
* generation for event Alarm.
*/
-#define OMAP_RTC_HAS_IRQWAKEEN 0x2
+#define OMAP_RTC_HAS_IRQWAKEEN BIT(1)
+
+/*
+ * Some RTC IP revisions (like those in AM335x and DRA7x) need
+ * the 32KHz clock to be explicitly enabled.
+ */
+#define OMAP_RTC_HAS_32KCLK_EN BIT(2)
static void __iomem *rtc_base;
@@ -162,17 +171,28 @@
static int omap_rtc_alarm_irq_enable(struct device *dev, unsigned int enabled)
{
- u8 reg;
+ u8 reg, irqwake_reg = 0;
+ struct platform_device *pdev = to_platform_device(dev);
+ const struct platform_device_id *id_entry =
+ platform_get_device_id(pdev);
local_irq_disable();
rtc_wait_not_busy();
reg = rtc_read(OMAP_RTC_INTERRUPTS_REG);
- if (enabled)
+ if (id_entry->driver_data & OMAP_RTC_HAS_IRQWAKEEN)
+ irqwake_reg = rtc_read(OMAP_RTC_IRQWAKEEN);
+
+ if (enabled) {
reg |= OMAP_RTC_INTERRUPTS_IT_ALARM;
- else
+ irqwake_reg |= OMAP_RTC_IRQWAKEEN_ALARM_WAKEEN;
+ } else {
reg &= ~OMAP_RTC_INTERRUPTS_IT_ALARM;
+ irqwake_reg &= ~OMAP_RTC_IRQWAKEEN_ALARM_WAKEEN;
+ }
rtc_wait_not_busy();
rtc_write(reg, OMAP_RTC_INTERRUPTS_REG);
+ if (id_entry->driver_data & OMAP_RTC_HAS_IRQWAKEEN)
+ rtc_write(irqwake_reg, OMAP_RTC_IRQWAKEEN);
local_irq_enable();
return 0;
@@ -272,7 +292,10 @@
static int omap_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alm)
{
- u8 reg;
+ u8 reg, irqwake_reg = 0;
+ struct platform_device *pdev = to_platform_device(dev);
+ const struct platform_device_id *id_entry =
+ platform_get_device_id(pdev);
if (tm2bcd(&alm->time) < 0)
return -EINVAL;
@@ -288,11 +311,19 @@
rtc_write(alm->time.tm_sec, OMAP_RTC_ALARM_SECONDS_REG);
reg = rtc_read(OMAP_RTC_INTERRUPTS_REG);
- if (alm->enabled)
+ if (id_entry->driver_data & OMAP_RTC_HAS_IRQWAKEEN)
+ irqwake_reg = rtc_read(OMAP_RTC_IRQWAKEEN);
+
+ if (alm->enabled) {
reg |= OMAP_RTC_INTERRUPTS_IT_ALARM;
- else
+ irqwake_reg |= OMAP_RTC_IRQWAKEEN_ALARM_WAKEEN;
+ } else {
reg &= ~OMAP_RTC_INTERRUPTS_IT_ALARM;
+ irqwake_reg &= ~OMAP_RTC_IRQWAKEEN_ALARM_WAKEEN;
+ }
rtc_write(reg, OMAP_RTC_INTERRUPTS_REG);
+ if (id_entry->driver_data & OMAP_RTC_HAS_IRQWAKEEN)
+ rtc_write(irqwake_reg, OMAP_RTC_IRQWAKEEN);
local_irq_enable();
@@ -319,7 +350,8 @@
},
[OMAP_RTC_DATA_AM3352_IDX] = {
.name = "am3352-rtc",
- .driver_data = OMAP_RTC_HAS_KICKER | OMAP_RTC_HAS_IRQWAKEEN,
+ .driver_data = OMAP_RTC_HAS_KICKER | OMAP_RTC_HAS_IRQWAKEEN |
+ OMAP_RTC_HAS_32KCLK_EN,
},
[OMAP_RTC_DATA_DA830_IDX] = {
.name = "da830-rtc",
@@ -352,6 +384,12 @@
if (of_id)
pdev->id_entry = of_id->data;
+ id_entry = platform_get_device_id(pdev);
+ if (!id_entry) {
+ dev_err(&pdev->dev, "no matching device entry\n");
+ return -ENODEV;
+ }
+
omap_rtc_timer = platform_get_irq(pdev, 0);
if (omap_rtc_timer <= 0) {
pr_debug("%s: no update irq?\n", pdev->name);
@@ -373,8 +411,7 @@
pm_runtime_enable(&pdev->dev);
pm_runtime_get_sync(&pdev->dev);
- id_entry = platform_get_device_id(pdev);
- if (id_entry && (id_entry->driver_data & OMAP_RTC_HAS_KICKER)) {
+ if (id_entry->driver_data & OMAP_RTC_HAS_KICKER) {
rtc_writel(KICK0_VALUE, OMAP_RTC_KICK0_REG);
rtc_writel(KICK1_VALUE, OMAP_RTC_KICK1_REG);
}
@@ -393,6 +430,10 @@
*/
rtc_write(0, OMAP_RTC_INTERRUPTS_REG);
+ /* enable RTC functional clock */
+ if (id_entry->driver_data & OMAP_RTC_HAS_32KCLK_EN)
+ rtc_writel(OMAP_RTC_OSC_32KCLK_EN, OMAP_RTC_OSC_REG);
+
/* clear old status */
reg = rtc_read(OMAP_RTC_STATUS_REG);
if (reg & (u8) OMAP_RTC_STATUS_POWER_UP) {
@@ -452,7 +493,7 @@
return 0;
fail0:
- if (id_entry && (id_entry->driver_data & OMAP_RTC_HAS_KICKER))
+ if (id_entry->driver_data & OMAP_RTC_HAS_KICKER)
rtc_writel(0, OMAP_RTC_KICK0_REG);
pm_runtime_put_sync(&pdev->dev);
pm_runtime_disable(&pdev->dev);
@@ -469,7 +510,7 @@
/* leave rtc running, but disable irqs */
rtc_write(0, OMAP_RTC_INTERRUPTS_REG);
- if (id_entry && (id_entry->driver_data & OMAP_RTC_HAS_KICKER))
+ if (id_entry->driver_data & OMAP_RTC_HAS_KICKER)
rtc_writel(0, OMAP_RTC_KICK0_REG);
/* Disable the clock/module */
@@ -484,28 +525,16 @@
static int omap_rtc_suspend(struct device *dev)
{
- u8 irqwake_stat;
- struct platform_device *pdev = to_platform_device(dev);
- const struct platform_device_id *id_entry =
- platform_get_device_id(pdev);
-
irqstat = rtc_read(OMAP_RTC_INTERRUPTS_REG);
/* FIXME the RTC alarm is not currently acting as a wakeup event
* source on some platforms, and in fact this enable() call is just
* saving a flag that's never used...
*/
- if (device_may_wakeup(dev)) {
+ if (device_may_wakeup(dev))
enable_irq_wake(omap_rtc_alarm);
-
- if (id_entry->driver_data & OMAP_RTC_HAS_IRQWAKEEN) {
- irqwake_stat = rtc_read(OMAP_RTC_IRQWAKEEN);
- irqwake_stat |= OMAP_RTC_IRQWAKEEN_ALARM_WAKEEN;
- rtc_write(irqwake_stat, OMAP_RTC_IRQWAKEEN);
- }
- } else {
+ else
rtc_write(0, OMAP_RTC_INTERRUPTS_REG);
- }
/* Disable the clock/module */
pm_runtime_put_sync(dev);
@@ -515,25 +544,14 @@
static int omap_rtc_resume(struct device *dev)
{
- u8 irqwake_stat;
- struct platform_device *pdev = to_platform_device(dev);
- const struct platform_device_id *id_entry =
- platform_get_device_id(pdev);
-
/* Enable the clock/module so that we can access the registers */
pm_runtime_get_sync(dev);
- if (device_may_wakeup(dev)) {
+ if (device_may_wakeup(dev))
disable_irq_wake(omap_rtc_alarm);
-
- if (id_entry->driver_data & OMAP_RTC_HAS_IRQWAKEEN) {
- irqwake_stat = rtc_read(OMAP_RTC_IRQWAKEEN);
- irqwake_stat &= ~OMAP_RTC_IRQWAKEEN_ALARM_WAKEEN;
- rtc_write(irqwake_stat, OMAP_RTC_IRQWAKEEN);
- }
- } else {
+ else
rtc_write(irqstat, OMAP_RTC_INTERRUPTS_REG);
- }
+
return 0;
}
#endif
diff --git a/drivers/rtc/rtc-palmas.c b/drivers/rtc/rtc-palmas.c
index c360d62..4dfe2d7 100644
--- a/drivers/rtc/rtc-palmas.c
+++ b/drivers/rtc/rtc-palmas.c
@@ -352,7 +352,7 @@
palmas_rtc_resume);
#ifdef CONFIG_OF
-static struct of_device_id of_palmas_rtc_match[] = {
+static const struct of_device_id of_palmas_rtc_match[] = {
{ .compatible = "ti,palmas-rtc"},
{ },
};
diff --git a/drivers/rtc/rtc-pxa.c b/drivers/rtc/rtc-pxa.c
index cccbf9d..4561f37 100644
--- a/drivers/rtc/rtc-pxa.c
+++ b/drivers/rtc/rtc-pxa.c
@@ -389,7 +389,7 @@
}
#ifdef CONFIG_OF
-static struct of_device_id pxa_rtc_dt_ids[] = {
+static const struct of_device_id pxa_rtc_dt_ids[] = {
{ .compatible = "marvell,pxa-rtc" },
{}
};
diff --git a/drivers/rtc/rtc-s5m.c b/drivers/rtc/rtc-s5m.c
index 476af93..8f06250 100644
--- a/drivers/rtc/rtc-s5m.c
+++ b/drivers/rtc/rtc-s5m.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2013 Samsung Electronics Co., Ltd
+ * Copyright (c) 2013-2014 Samsung Electronics Co., Ltd
* http://www.samsung.com
*
* Copyright (C) 2013 Google, Inc
@@ -17,29 +17,79 @@
#include <linux/module.h>
#include <linux/i2c.h>
-#include <linux/slab.h>
#include <linux/bcd.h>
-#include <linux/bitops.h>
#include <linux/regmap.h>
#include <linux/rtc.h>
-#include <linux/delay.h>
#include <linux/platform_device.h>
#include <linux/mfd/samsung/core.h>
#include <linux/mfd/samsung/irq.h>
#include <linux/mfd/samsung/rtc.h>
+#include <linux/mfd/samsung/s2mps14.h>
/*
* Maximum number of retries for checking changes in UDR field
- * of SEC_RTC_UDR_CON register (to limit possible endless loop).
+ * of S5M_RTC_UDR_CON register (to limit possible endless loop).
*
* After writing to RTC registers (setting time or alarm) read the UDR field
- * in SEC_RTC_UDR_CON register. UDR is auto-cleared when data have
+ * in S5M_RTC_UDR_CON register. UDR is auto-cleared when data have
* been transferred.
*/
#define UDR_READ_RETRY_CNT 5
+/* Registers used by the driver which are different between chipsets. */
+struct s5m_rtc_reg_config {
+ /* Number of registers used for setting time/alarm0/alarm1 */
+ unsigned int regs_count;
+ /* First register for time, seconds */
+ unsigned int time;
+ /* RTC control register */
+ unsigned int ctrl;
+ /* First register for alarm 0, seconds */
+ unsigned int alarm0;
+ /* First register for alarm 1, seconds */
+ unsigned int alarm1;
+ /* SMPL/WTSR register */
+ unsigned int smpl_wtsr;
+ /*
+ * Register for update flag (UDR). Typically setting UDR field to 1
+ * will enable update of time or alarm register. Then it will be
+ * auto-cleared after successful update.
+ */
+ unsigned int rtc_udr_update;
+ /* Mask for UDR field in 'rtc_udr_update' register */
+ unsigned int rtc_udr_mask;
+};
+
+/* Register map for S5M8763 and S5M8767 */
+static const struct s5m_rtc_reg_config s5m_rtc_regs = {
+ .regs_count = 8,
+ .time = S5M_RTC_SEC,
+ .ctrl = S5M_ALARM1_CONF,
+ .alarm0 = S5M_ALARM0_SEC,
+ .alarm1 = S5M_ALARM1_SEC,
+ .smpl_wtsr = S5M_WTSR_SMPL_CNTL,
+ .rtc_udr_update = S5M_RTC_UDR_CON,
+ .rtc_udr_mask = S5M_RTC_UDR_MASK,
+};
+
+/*
+ * Register map for S2MPS14.
+ * It may be also suitable for S2MPS11 but this was not tested.
+ */
+static const struct s5m_rtc_reg_config s2mps_rtc_regs = {
+ .regs_count = 7,
+ .time = S2MPS_RTC_SEC,
+ .ctrl = S2MPS_RTC_CTRL,
+ .alarm0 = S2MPS_ALARM0_SEC,
+ .alarm1 = S2MPS_ALARM1_SEC,
+ .smpl_wtsr = S2MPS_WTSR_SMPL_CNTL,
+ .rtc_udr_update = S2MPS_RTC_UDR_CON,
+ .rtc_udr_mask = S2MPS_RTC_WUDR_MASK,
+};
+
struct s5m_rtc_info {
struct device *dev;
+ struct i2c_client *i2c;
struct sec_pmic_dev *s5m87xx;
struct regmap *regmap;
struct rtc_device *rtc_dev;
@@ -47,6 +97,21 @@
int device_type;
int rtc_24hr_mode;
bool wtsr_smpl;
+ const struct s5m_rtc_reg_config *regs;
+};
+
+static const struct regmap_config s5m_rtc_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+
+ .max_register = S5M_RTC_REG_MAX,
+};
+
+static const struct regmap_config s2mps14_rtc_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+
+ .max_register = S2MPS_RTC_REG_MAX,
};
static void s5m8767_data_to_tm(u8 *data, struct rtc_time *tm,
@@ -104,8 +169,9 @@
unsigned int data;
do {
- ret = regmap_read(info->regmap, SEC_RTC_UDR_CON, &data);
- } while (--retry && (data & RTC_UDR_MASK) && !ret);
+ ret = regmap_read(info->regmap, info->regs->rtc_udr_update,
+ &data);
+ } while (--retry && (data & info->regs->rtc_udr_mask) && !ret);
if (!retry)
dev_err(info->dev, "waiting for UDR update, reached max number of retries\n");
@@ -113,21 +179,53 @@
return ret;
}
+static inline int s5m_check_peding_alarm_interrupt(struct s5m_rtc_info *info,
+ struct rtc_wkalrm *alarm)
+{
+ int ret;
+ unsigned int val;
+
+ switch (info->device_type) {
+ case S5M8767X:
+ case S5M8763X:
+ ret = regmap_read(info->regmap, S5M_RTC_STATUS, &val);
+ val &= S5M_ALARM0_STATUS;
+ break;
+ case S2MPS14X:
+ ret = regmap_read(info->s5m87xx->regmap_pmic, S2MPS14_REG_ST2,
+ &val);
+ val &= S2MPS_ALARM0_STATUS;
+ break;
+ default:
+ return -EINVAL;
+ }
+ if (ret < 0)
+ return ret;
+
+ if (val)
+ alarm->pending = 1;
+ else
+ alarm->pending = 0;
+
+ return 0;
+}
+
static inline int s5m8767_rtc_set_time_reg(struct s5m_rtc_info *info)
{
int ret;
unsigned int data;
- ret = regmap_read(info->regmap, SEC_RTC_UDR_CON, &data);
+ ret = regmap_read(info->regmap, info->regs->rtc_udr_update, &data);
if (ret < 0) {
dev_err(info->dev, "failed to read update reg(%d)\n", ret);
return ret;
}
- data |= RTC_TIME_EN_MASK;
- data |= RTC_UDR_MASK;
+ data |= info->regs->rtc_udr_mask;
+ if (info->device_type == S5M8763X || info->device_type == S5M8767X)
+ data |= S5M_RTC_TIME_EN_MASK;
- ret = regmap_write(info->regmap, SEC_RTC_UDR_CON, data);
+ ret = regmap_write(info->regmap, info->regs->rtc_udr_update, data);
if (ret < 0) {
dev_err(info->dev, "failed to write update reg(%d)\n", ret);
return ret;
@@ -143,17 +241,27 @@
int ret;
unsigned int data;
- ret = regmap_read(info->regmap, SEC_RTC_UDR_CON, &data);
+ ret = regmap_read(info->regmap, info->regs->rtc_udr_update, &data);
if (ret < 0) {
dev_err(info->dev, "%s: fail to read update reg(%d)\n",
__func__, ret);
return ret;
}
- data &= ~RTC_TIME_EN_MASK;
- data |= RTC_UDR_MASK;
+ data |= info->regs->rtc_udr_mask;
+ switch (info->device_type) {
+ case S5M8763X:
+ case S5M8767X:
+ data &= ~S5M_RTC_TIME_EN_MASK;
+ break;
+ case S2MPS14X:
+ data |= S2MPS_RTC_RUDR_MASK;
+ break;
+ default:
+ return -EINVAL;
+ }
- ret = regmap_write(info->regmap, SEC_RTC_UDR_CON, data);
+ ret = regmap_write(info->regmap, info->regs->rtc_udr_update, data);
if (ret < 0) {
dev_err(info->dev, "%s: fail to write update reg(%d)\n",
__func__, ret);
@@ -200,10 +308,22 @@
static int s5m_rtc_read_time(struct device *dev, struct rtc_time *tm)
{
struct s5m_rtc_info *info = dev_get_drvdata(dev);
- u8 data[8];
+ u8 data[info->regs->regs_count];
int ret;
- ret = regmap_bulk_read(info->regmap, SEC_RTC_SEC, data, 8);
+ if (info->device_type == S2MPS14X) {
+ ret = regmap_update_bits(info->regmap,
+ info->regs->rtc_udr_update,
+ S2MPS_RTC_RUDR_MASK, S2MPS_RTC_RUDR_MASK);
+ if (ret) {
+ dev_err(dev,
+ "Failed to prepare registers for time reading: %d\n",
+ ret);
+ return ret;
+ }
+ }
+ ret = regmap_bulk_read(info->regmap, info->regs->time, data,
+ info->regs->regs_count);
if (ret < 0)
return ret;
@@ -213,6 +333,7 @@
break;
case S5M8767X:
+ case S2MPS14X:
s5m8767_data_to_tm(data, tm, info->rtc_24hr_mode);
break;
@@ -230,7 +351,7 @@
static int s5m_rtc_set_time(struct device *dev, struct rtc_time *tm)
{
struct s5m_rtc_info *info = dev_get_drvdata(dev);
- u8 data[8];
+ u8 data[info->regs->regs_count];
int ret = 0;
switch (info->device_type) {
@@ -238,6 +359,7 @@
s5m8763_tm_to_data(tm, data);
break;
case S5M8767X:
+ case S2MPS14X:
ret = s5m8767_tm_to_data(tm, data);
break;
default:
@@ -251,7 +373,8 @@
1900 + tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
tm->tm_hour, tm->tm_min, tm->tm_sec, tm->tm_wday);
- ret = regmap_raw_write(info->regmap, SEC_RTC_SEC, data, 8);
+ ret = regmap_raw_write(info->regmap, info->regs->time, data,
+ info->regs->regs_count);
if (ret < 0)
return ret;
@@ -263,70 +386,60 @@
static int s5m_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *alrm)
{
struct s5m_rtc_info *info = dev_get_drvdata(dev);
- u8 data[8];
+ u8 data[info->regs->regs_count];
unsigned int val;
int ret, i;
- ret = regmap_bulk_read(info->regmap, SEC_ALARM0_SEC, data, 8);
+ ret = regmap_bulk_read(info->regmap, info->regs->alarm0, data,
+ info->regs->regs_count);
if (ret < 0)
return ret;
switch (info->device_type) {
case S5M8763X:
s5m8763_data_to_tm(data, &alrm->time);
- ret = regmap_read(info->regmap, SEC_ALARM0_CONF, &val);
+ ret = regmap_read(info->regmap, S5M_ALARM0_CONF, &val);
if (ret < 0)
return ret;
alrm->enabled = !!val;
-
- ret = regmap_read(info->regmap, SEC_RTC_STATUS, &val);
- if (ret < 0)
- return ret;
-
break;
case S5M8767X:
+ case S2MPS14X:
s5m8767_data_to_tm(data, &alrm->time, info->rtc_24hr_mode);
- dev_dbg(dev, "%s: %d/%d/%d %d:%d:%d(%d)\n", __func__,
- 1900 + alrm->time.tm_year, 1 + alrm->time.tm_mon,
- alrm->time.tm_mday, alrm->time.tm_hour,
- alrm->time.tm_min, alrm->time.tm_sec,
- alrm->time.tm_wday);
-
alrm->enabled = 0;
- for (i = 0; i < 7; i++) {
+ for (i = 0; i < info->regs->regs_count; i++) {
if (data[i] & ALARM_ENABLE_MASK) {
alrm->enabled = 1;
break;
}
}
-
- alrm->pending = 0;
- ret = regmap_read(info->regmap, SEC_RTC_STATUS, &val);
- if (ret < 0)
- return ret;
break;
default:
return -EINVAL;
}
- if (val & ALARM0_STATUS)
- alrm->pending = 1;
- else
- alrm->pending = 0;
+ dev_dbg(dev, "%s: %d/%d/%d %d:%d:%d(%d)\n", __func__,
+ 1900 + alrm->time.tm_year, 1 + alrm->time.tm_mon,
+ alrm->time.tm_mday, alrm->time.tm_hour,
+ alrm->time.tm_min, alrm->time.tm_sec,
+ alrm->time.tm_wday);
+
+ ret = s5m_check_peding_alarm_interrupt(info, alrm);
return 0;
}
static int s5m_rtc_stop_alarm(struct s5m_rtc_info *info)
{
- u8 data[8];
+ u8 data[info->regs->regs_count];
int ret, i;
struct rtc_time tm;
- ret = regmap_bulk_read(info->regmap, SEC_ALARM0_SEC, data, 8);
+ ret = regmap_bulk_read(info->regmap, info->regs->alarm0, data,
+ info->regs->regs_count);
if (ret < 0)
return ret;
@@ -337,14 +450,16 @@
switch (info->device_type) {
case S5M8763X:
- ret = regmap_write(info->regmap, SEC_ALARM0_CONF, 0);
+ ret = regmap_write(info->regmap, S5M_ALARM0_CONF, 0);
break;
case S5M8767X:
- for (i = 0; i < 7; i++)
+ case S2MPS14X:
+ for (i = 0; i < info->regs->regs_count; i++)
data[i] &= ~ALARM_ENABLE_MASK;
- ret = regmap_raw_write(info->regmap, SEC_ALARM0_SEC, data, 8);
+ ret = regmap_raw_write(info->regmap, info->regs->alarm0, data,
+ info->regs->regs_count);
if (ret < 0)
return ret;
@@ -362,11 +477,12 @@
static int s5m_rtc_start_alarm(struct s5m_rtc_info *info)
{
int ret;
- u8 data[8];
+ u8 data[info->regs->regs_count];
u8 alarm0_conf;
struct rtc_time tm;
- ret = regmap_bulk_read(info->regmap, SEC_ALARM0_SEC, data, 8);
+ ret = regmap_bulk_read(info->regmap, info->regs->alarm0, data,
+ info->regs->regs_count);
if (ret < 0)
return ret;
@@ -378,10 +494,11 @@
switch (info->device_type) {
case S5M8763X:
alarm0_conf = 0x77;
- ret = regmap_write(info->regmap, SEC_ALARM0_CONF, alarm0_conf);
+ ret = regmap_write(info->regmap, S5M_ALARM0_CONF, alarm0_conf);
break;
case S5M8767X:
+ case S2MPS14X:
data[RTC_SEC] |= ALARM_ENABLE_MASK;
data[RTC_MIN] |= ALARM_ENABLE_MASK;
data[RTC_HOUR] |= ALARM_ENABLE_MASK;
@@ -393,7 +510,8 @@
if (data[RTC_YEAR1] & 0x7f)
data[RTC_YEAR1] |= ALARM_ENABLE_MASK;
- ret = regmap_raw_write(info->regmap, SEC_ALARM0_SEC, data, 8);
+ ret = regmap_raw_write(info->regmap, info->regs->alarm0, data,
+ info->regs->regs_count);
if (ret < 0)
return ret;
ret = s5m8767_rtc_set_alarm_reg(info);
@@ -410,7 +528,7 @@
static int s5m_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alrm)
{
struct s5m_rtc_info *info = dev_get_drvdata(dev);
- u8 data[8];
+ u8 data[info->regs->regs_count];
int ret;
switch (info->device_type) {
@@ -419,6 +537,7 @@
break;
case S5M8767X:
+ case S2MPS14X:
s5m8767_tm_to_data(&alrm->time, data);
break;
@@ -435,7 +554,8 @@
if (ret < 0)
return ret;
- ret = regmap_raw_write(info->regmap, SEC_ALARM0_SEC, data, 8);
+ ret = regmap_raw_write(info->regmap, info->regs->alarm0, data,
+ info->regs->regs_count);
if (ret < 0)
return ret;
@@ -480,7 +600,7 @@
static void s5m_rtc_enable_wtsr(struct s5m_rtc_info *info, bool enable)
{
int ret;
- ret = regmap_update_bits(info->regmap, SEC_WTSR_SMPL_CNTL,
+ ret = regmap_update_bits(info->regmap, info->regs->smpl_wtsr,
WTSR_ENABLE_MASK,
enable ? WTSR_ENABLE_MASK : 0);
if (ret < 0)
@@ -491,7 +611,7 @@
static void s5m_rtc_enable_smpl(struct s5m_rtc_info *info, bool enable)
{
int ret;
- ret = regmap_update_bits(info->regmap, SEC_WTSR_SMPL_CNTL,
+ ret = regmap_update_bits(info->regmap, info->regs->smpl_wtsr,
SMPL_ENABLE_MASK,
enable ? SMPL_ENABLE_MASK : 0);
if (ret < 0)
@@ -502,50 +622,41 @@
static int s5m8767_rtc_init_reg(struct s5m_rtc_info *info)
{
u8 data[2];
- unsigned int tp_read;
int ret;
- struct rtc_time tm;
- ret = regmap_read(info->regmap, SEC_RTC_UDR_CON, &tp_read);
- if (ret < 0) {
- dev_err(info->dev, "%s: fail to read control reg(%d)\n",
- __func__, ret);
- return ret;
+ switch (info->device_type) {
+ case S5M8763X:
+ case S5M8767X:
+ /* UDR update time. Default of 7.32 ms is too long. */
+ ret = regmap_update_bits(info->regmap, S5M_RTC_UDR_CON,
+ S5M_RTC_UDR_T_MASK, S5M_RTC_UDR_T_450_US);
+ if (ret < 0)
+ dev_err(info->dev, "%s: fail to change UDR time: %d\n",
+ __func__, ret);
+
+ /* Set RTC control register : Binary mode, 24hour mode */
+ data[0] = (1 << BCD_EN_SHIFT) | (1 << MODEL24_SHIFT);
+ data[1] = (0 << BCD_EN_SHIFT) | (1 << MODEL24_SHIFT);
+
+ ret = regmap_raw_write(info->regmap, S5M_ALARM0_CONF, data, 2);
+ break;
+
+ case S2MPS14X:
+ data[0] = (0 << BCD_EN_SHIFT) | (1 << MODEL24_SHIFT);
+ ret = regmap_write(info->regmap, info->regs->ctrl, data[0]);
+ break;
+
+ default:
+ return -EINVAL;
}
- /* Set RTC control register : Binary mode, 24hour mode */
- data[0] = (1 << BCD_EN_SHIFT) | (1 << MODEL24_SHIFT);
- data[1] = (0 << BCD_EN_SHIFT) | (1 << MODEL24_SHIFT);
-
info->rtc_24hr_mode = 1;
- ret = regmap_raw_write(info->regmap, SEC_ALARM0_CONF, data, 2);
if (ret < 0) {
dev_err(info->dev, "%s: fail to write controlm reg(%d)\n",
__func__, ret);
return ret;
}
- /* In first boot time, Set rtc time to 1/1/2012 00:00:00(SUN) */
- if ((tp_read & RTC_TCON_MASK) == 0) {
- dev_dbg(info->dev, "rtc init\n");
- tm.tm_sec = 0;
- tm.tm_min = 0;
- tm.tm_hour = 0;
- tm.tm_wday = 0;
- tm.tm_mday = 1;
- tm.tm_mon = 0;
- tm.tm_year = 112;
- tm.tm_yday = 0;
- tm.tm_isdst = 0;
- ret = s5m_rtc_set_time(info->dev, &tm);
- }
-
- ret = regmap_update_bits(info->regmap, SEC_RTC_UDR_CON,
- RTC_TCON_MASK, tp_read | RTC_TCON_MASK);
- if (ret < 0)
- dev_err(info->dev, "%s: fail to update TCON reg(%d)\n",
- __func__, ret);
-
return ret;
}
@@ -554,7 +665,8 @@
struct sec_pmic_dev *s5m87xx = dev_get_drvdata(pdev->dev.parent);
struct sec_platform_data *pdata = s5m87xx->pdata;
struct s5m_rtc_info *info;
- int ret;
+ const struct regmap_config *regmap_cfg;
+ int ret, alarm_irq;
if (!pdata) {
dev_err(pdev->dev.parent, "Platform data not supplied\n");
@@ -565,27 +677,52 @@
if (!info)
return -ENOMEM;
+ switch (pdata->device_type) {
+ case S2MPS14X:
+ regmap_cfg = &s2mps14_rtc_regmap_config;
+ info->regs = &s2mps_rtc_regs;
+ alarm_irq = S2MPS14_IRQ_RTCA0;
+ break;
+ case S5M8763X:
+ regmap_cfg = &s5m_rtc_regmap_config;
+ info->regs = &s5m_rtc_regs;
+ alarm_irq = S5M8763_IRQ_ALARM0;
+ break;
+ case S5M8767X:
+ regmap_cfg = &s5m_rtc_regmap_config;
+ info->regs = &s5m_rtc_regs;
+ alarm_irq = S5M8767_IRQ_RTCA1;
+ break;
+ default:
+ dev_err(&pdev->dev, "Device type is not supported by RTC driver\n");
+ return -ENODEV;
+ }
+
+ info->i2c = i2c_new_dummy(s5m87xx->i2c->adapter, RTC_I2C_ADDR);
+ if (!info->i2c) {
+ dev_err(&pdev->dev, "Failed to allocate I2C for RTC\n");
+ return -ENODEV;
+ }
+
+ info->regmap = devm_regmap_init_i2c(info->i2c, regmap_cfg);
+ if (IS_ERR(info->regmap)) {
+ ret = PTR_ERR(info->regmap);
+ dev_err(&pdev->dev, "Failed to allocate RTC register map: %d\n",
+ ret);
+ goto err;
+ }
+
info->dev = &pdev->dev;
info->s5m87xx = s5m87xx;
- info->regmap = s5m87xx->regmap_rtc;
info->device_type = s5m87xx->device_type;
info->wtsr_smpl = s5m87xx->wtsr_smpl;
- switch (pdata->device_type) {
- case S5M8763X:
- info->irq = regmap_irq_get_virq(s5m87xx->irq_data,
- S5M8763_IRQ_ALARM0);
- break;
-
- case S5M8767X:
- info->irq = regmap_irq_get_virq(s5m87xx->irq_data,
- S5M8767_IRQ_RTCA1);
- break;
-
- default:
+ info->irq = regmap_irq_get_virq(s5m87xx->irq_data, alarm_irq);
+ if (info->irq <= 0) {
ret = -EINVAL;
- dev_err(&pdev->dev, "Unsupported device type: %d\n", ret);
- return ret;
+ dev_err(&pdev->dev, "Failed to get virtual IRQ %d\n",
+ alarm_irq);
+ goto err;
}
platform_set_drvdata(pdev, info);
@@ -602,15 +739,24 @@
info->rtc_dev = devm_rtc_device_register(&pdev->dev, "s5m-rtc",
&s5m_rtc_ops, THIS_MODULE);
- if (IS_ERR(info->rtc_dev))
- return PTR_ERR(info->rtc_dev);
+ if (IS_ERR(info->rtc_dev)) {
+ ret = PTR_ERR(info->rtc_dev);
+ goto err;
+ }
ret = devm_request_threaded_irq(&pdev->dev, info->irq, NULL,
s5m_rtc_alarm_irq, 0, "rtc-alarm0",
info);
- if (ret < 0)
+ if (ret < 0) {
dev_err(&pdev->dev, "Failed to request alarm IRQ: %d: %d\n",
info->irq, ret);
+ goto err;
+ }
+
+ return 0;
+
+err:
+ i2c_unregister_device(info->i2c);
return ret;
}
@@ -623,7 +769,7 @@
if (info->wtsr_smpl) {
for (i = 0; i < 3; i++) {
s5m_rtc_enable_wtsr(info, false);
- regmap_read(info->regmap, SEC_WTSR_SMPL_CNTL, &val);
+ regmap_read(info->regmap, info->regs->smpl_wtsr, &val);
pr_debug("%s: WTSR_SMPL reg(0x%02x)\n", __func__, val);
if (val & WTSR_ENABLE_MASK)
pr_emerg("%s: fail to disable WTSR\n",
@@ -639,6 +785,17 @@
s5m_rtc_enable_smpl(info, false);
}
+static int s5m_rtc_remove(struct platform_device *pdev)
+{
+ struct s5m_rtc_info *info = platform_get_drvdata(pdev);
+
+ /* Perform also all shutdown steps when removing */
+ s5m_rtc_shutdown(pdev);
+ i2c_unregister_device(info->i2c);
+
+ return 0;
+}
+
#ifdef CONFIG_PM_SLEEP
static int s5m_rtc_resume(struct device *dev)
{
@@ -666,7 +823,8 @@
static SIMPLE_DEV_PM_OPS(s5m_rtc_pm_ops, s5m_rtc_suspend, s5m_rtc_resume);
static const struct platform_device_id s5m_rtc_id[] = {
- { "s5m-rtc", 0 },
+ { "s5m-rtc", S5M8767X },
+ { "s2mps14-rtc", S2MPS14X },
};
static struct platform_driver s5m_rtc_driver = {
@@ -676,6 +834,7 @@
.pm = &s5m_rtc_pm_ops,
},
.probe = s5m_rtc_probe,
+ .remove = s5m_rtc_remove,
.shutdown = s5m_rtc_shutdown,
.id_table = s5m_rtc_id,
};
@@ -684,6 +843,6 @@
/* Module information */
MODULE_AUTHOR("Sangbeom Kim <sbkim73@samsung.com>");
-MODULE_DESCRIPTION("Samsung S5M RTC driver");
+MODULE_DESCRIPTION("Samsung S5M/S2MPS14 RTC driver");
MODULE_LICENSE("GPL");
MODULE_ALIAS("platform:s5m-rtc");
diff --git a/drivers/rtc/rtc-sa1100.c b/drivers/rtc/rtc-sa1100.c
index 0f7adeb..b6e1ca0 100644
--- a/drivers/rtc/rtc-sa1100.c
+++ b/drivers/rtc/rtc-sa1100.c
@@ -338,7 +338,7 @@
sa1100_rtc_resume);
#ifdef CONFIG_OF
-static struct of_device_id sa1100_rtc_dt_ids[] = {
+static const struct of_device_id sa1100_rtc_dt_ids[] = {
{ .compatible = "mrvl,sa1100-rtc", },
{ .compatible = "mrvl,mmp-rtc", },
{}
diff --git a/drivers/rtc/rtc-xgene.c b/drivers/rtc/rtc-xgene.c
new file mode 100644
index 0000000..14129cc
--- /dev/null
+++ b/drivers/rtc/rtc-xgene.c
@@ -0,0 +1,278 @@
+/*
+ * APM X-Gene SoC Real Time Clock Driver
+ *
+ * Copyright (c) 2014, Applied Micro Circuits Corporation
+ * Author: Rameshwar Prasad Sahu <rsahu@apm.com>
+ * Loc Ho <lho@apm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/io.h>
+#include <linux/slab.h>
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/rtc.h>
+
+/* RTC CSR Registers */
+#define RTC_CCVR 0x00
+#define RTC_CMR 0x04
+#define RTC_CLR 0x08
+#define RTC_CCR 0x0C
+#define RTC_CCR_IE BIT(0)
+#define RTC_CCR_MASK BIT(1)
+#define RTC_CCR_EN BIT(2)
+#define RTC_CCR_WEN BIT(3)
+#define RTC_STAT 0x10
+#define RTC_STAT_BIT BIT(0)
+#define RTC_RSTAT 0x14
+#define RTC_EOI 0x18
+#define RTC_VER 0x1C
+
+struct xgene_rtc_dev {
+ struct rtc_device *rtc;
+ struct device *dev;
+ unsigned long alarm_time;
+ void __iomem *csr_base;
+ struct clk *clk;
+ unsigned int irq_wake;
+};
+
+static int xgene_rtc_read_time(struct device *dev, struct rtc_time *tm)
+{
+ struct xgene_rtc_dev *pdata = dev_get_drvdata(dev);
+
+ rtc_time_to_tm(readl(pdata->csr_base + RTC_CCVR), tm);
+ return rtc_valid_tm(tm);
+}
+
+static int xgene_rtc_set_mmss(struct device *dev, unsigned long secs)
+{
+ struct xgene_rtc_dev *pdata = dev_get_drvdata(dev);
+
+ /*
+ * NOTE: After the following write, the RTC_CCVR is only reflected
+ * after the update cycle of 1 seconds.
+ */
+ writel((u32) secs, pdata->csr_base + RTC_CLR);
+ readl(pdata->csr_base + RTC_CLR); /* Force a barrier */
+
+ return 0;
+}
+
+static int xgene_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *alrm)
+{
+ struct xgene_rtc_dev *pdata = dev_get_drvdata(dev);
+
+ rtc_time_to_tm(pdata->alarm_time, &alrm->time);
+ alrm->enabled = readl(pdata->csr_base + RTC_CCR) & RTC_CCR_IE;
+
+ return 0;
+}
+
+static int xgene_rtc_alarm_irq_enable(struct device *dev, u32 enabled)
+{
+ struct xgene_rtc_dev *pdata = dev_get_drvdata(dev);
+ u32 ccr;
+
+ ccr = readl(pdata->csr_base + RTC_CCR);
+ if (enabled) {
+ ccr &= ~RTC_CCR_MASK;
+ ccr |= RTC_CCR_IE;
+ } else {
+ ccr &= ~RTC_CCR_IE;
+ ccr |= RTC_CCR_MASK;
+ }
+ writel(ccr, pdata->csr_base + RTC_CCR);
+
+ return 0;
+}
+
+static int xgene_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alrm)
+{
+ struct xgene_rtc_dev *pdata = dev_get_drvdata(dev);
+ unsigned long rtc_time;
+ unsigned long alarm_time;
+
+ rtc_time = readl(pdata->csr_base + RTC_CCVR);
+ rtc_tm_to_time(&alrm->time, &alarm_time);
+
+ pdata->alarm_time = alarm_time;
+ writel((u32) pdata->alarm_time, pdata->csr_base + RTC_CMR);
+
+ xgene_rtc_alarm_irq_enable(dev, alrm->enabled);
+
+ return 0;
+}
+
+static const struct rtc_class_ops xgene_rtc_ops = {
+ .read_time = xgene_rtc_read_time,
+ .set_mmss = xgene_rtc_set_mmss,
+ .read_alarm = xgene_rtc_read_alarm,
+ .set_alarm = xgene_rtc_set_alarm,
+ .alarm_irq_enable = xgene_rtc_alarm_irq_enable,
+};
+
+static irqreturn_t xgene_rtc_interrupt(int irq, void *id)
+{
+ struct xgene_rtc_dev *pdata = (struct xgene_rtc_dev *) id;
+
+ /* Check if interrupt asserted */
+ if (!(readl(pdata->csr_base + RTC_STAT) & RTC_STAT_BIT))
+ return IRQ_NONE;
+
+ /* Clear interrupt */
+ readl(pdata->csr_base + RTC_EOI);
+
+ rtc_update_irq(pdata->rtc, 1, RTC_IRQF | RTC_AF);
+
+ return IRQ_HANDLED;
+}
+
+static int xgene_rtc_probe(struct platform_device *pdev)
+{
+ struct xgene_rtc_dev *pdata;
+ struct resource *res;
+ int ret;
+ int irq;
+
+ pdata = devm_kzalloc(&pdev->dev, sizeof(*pdata), GFP_KERNEL);
+ if (!pdata)
+ return -ENOMEM;
+ platform_set_drvdata(pdev, pdata);
+ pdata->dev = &pdev->dev;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ pdata->csr_base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(pdata->csr_base))
+ return PTR_ERR(pdata->csr_base);
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0) {
+ dev_err(&pdev->dev, "No IRQ resource\n");
+ return irq;
+ }
+ ret = devm_request_irq(&pdev->dev, irq, xgene_rtc_interrupt, 0,
+ dev_name(&pdev->dev), pdata);
+ if (ret) {
+ dev_err(&pdev->dev, "Could not request IRQ\n");
+ return ret;
+ }
+
+ pdata->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(pdata->clk)) {
+ dev_err(&pdev->dev, "Couldn't get the clock for RTC\n");
+ return -ENODEV;
+ }
+ clk_prepare_enable(pdata->clk);
+
+ /* Turn on the clock and the crystal */
+ writel(RTC_CCR_EN, pdata->csr_base + RTC_CCR);
+
+ device_init_wakeup(&pdev->dev, 1);
+
+ pdata->rtc = devm_rtc_device_register(&pdev->dev, pdev->name,
+ &xgene_rtc_ops, THIS_MODULE);
+ if (IS_ERR(pdata->rtc)) {
+ clk_disable_unprepare(pdata->clk);
+ return PTR_ERR(pdata->rtc);
+ }
+
+ /* HW does not support update faster than 1 seconds */
+ pdata->rtc->uie_unsupported = 1;
+
+ return 0;
+}
+
+static int xgene_rtc_remove(struct platform_device *pdev)
+{
+ struct xgene_rtc_dev *pdata = platform_get_drvdata(pdev);
+
+ xgene_rtc_alarm_irq_enable(&pdev->dev, 0);
+ device_init_wakeup(&pdev->dev, 0);
+ clk_disable_unprepare(pdata->clk);
+ return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int xgene_rtc_suspend(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct xgene_rtc_dev *pdata = platform_get_drvdata(pdev);
+ int irq;
+
+ irq = platform_get_irq(pdev, 0);
+ if (device_may_wakeup(&pdev->dev)) {
+ if (!enable_irq_wake(irq))
+ pdata->irq_wake = 1;
+ } else {
+ xgene_rtc_alarm_irq_enable(dev, 0);
+ clk_disable(pdata->clk);
+ }
+
+ return 0;
+}
+
+static int xgene_rtc_resume(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct xgene_rtc_dev *pdata = platform_get_drvdata(pdev);
+ int irq;
+
+ irq = platform_get_irq(pdev, 0);
+ if (device_may_wakeup(&pdev->dev)) {
+ if (pdata->irq_wake) {
+ disable_irq_wake(irq);
+ pdata->irq_wake = 0;
+ }
+ } else {
+ clk_enable(pdata->clk);
+ xgene_rtc_alarm_irq_enable(dev, 1);
+ }
+
+ return 0;
+}
+#endif
+
+static SIMPLE_DEV_PM_OPS(xgene_rtc_pm_ops, xgene_rtc_suspend, xgene_rtc_resume);
+
+#ifdef CONFIG_OF
+static const struct of_device_id xgene_rtc_of_match[] = {
+ {.compatible = "apm,xgene-rtc" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, xgene_rtc_of_match);
+#endif
+
+static struct platform_driver xgene_rtc_driver = {
+ .probe = xgene_rtc_probe,
+ .remove = xgene_rtc_remove,
+ .driver = {
+ .owner = THIS_MODULE,
+ .name = "xgene-rtc",
+ .pm = &xgene_rtc_pm_ops,
+ .of_match_table = of_match_ptr(xgene_rtc_of_match),
+ },
+};
+
+module_platform_driver(xgene_rtc_driver);
+
+MODULE_DESCRIPTION("APM X-Gene SoC RTC driver");
+MODULE_AUTHOR("Rameshwar Sahu <rsahu@apm.com>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/scsi/NCR5380.c b/drivers/scsi/NCR5380.c
index bcd2238..93d13fc 100644
--- a/drivers/scsi/NCR5380.c
+++ b/drivers/scsi/NCR5380.c
@@ -27,8 +27,6 @@
*/
/*
- * $Log: NCR5380.c,v $
-
* Revision 1.10 1998/9/2 Alan Cox
* (alan@lxorguk.ukuu.org.uk)
* Fixed up the timer lockups reported so far. Things still suck. Looking
@@ -89,13 +87,6 @@
#include <scsi/scsi_dbg.h>
#include <scsi/scsi_transport_spi.h>
-#ifndef NDEBUG
-#define NDEBUG 0
-#endif
-#ifndef NDEBUG_ABORT
-#define NDEBUG_ABORT 0
-#endif
-
#if (NDEBUG & NDEBUG_LISTS)
#define LIST(x,y) {printk("LINE:%d Adding %p to %p\n", __LINE__, (void*)(x), (void*)(y)); if ((x)==(y)) udelay(5); }
#define REMOVE(w,x,y,z) {printk("LINE:%d Removing: %p->%p %p->%p \n", __LINE__, (void*)(w), (void*)(x), (void*)(y), (void*)(z)); if ((x)==(y)) udelay(5); }
@@ -1005,7 +996,7 @@
LIST(cmd, tmp);
tmp->host_scribble = (unsigned char *) cmd;
}
- dprintk(NDEBUG_QUEUES, ("scsi%d : command added to %s of queue\n", instance->host_no, (cmd->cmnd[0] == REQUEST_SENSE) ? "head" : "tail"));
+ dprintk(NDEBUG_QUEUES, "scsi%d : command added to %s of queue\n", instance->host_no, (cmd->cmnd[0] == REQUEST_SENSE) ? "head" : "tail");
/* Run the coroutine if it isn't already running. */
/* Kick off command processing */
@@ -1040,7 +1031,7 @@
/* Lock held here */
done = 1;
if (!hostdata->connected && !hostdata->selecting) {
- dprintk(NDEBUG_MAIN, ("scsi%d : not connected\n", instance->host_no));
+ dprintk(NDEBUG_MAIN, "scsi%d : not connected\n", instance->host_no);
/*
* Search through the issue_queue for a command destined
* for a target that's not busy.
@@ -1048,7 +1039,7 @@
for (tmp = (Scsi_Cmnd *) hostdata->issue_queue, prev = NULL; tmp; prev = tmp, tmp = (Scsi_Cmnd *) tmp->host_scribble)
{
if (prev != tmp)
- dprintk(NDEBUG_LISTS, ("MAIN tmp=%p target=%d busy=%d lun=%d\n", tmp, tmp->target, hostdata->busy[tmp->target], tmp->lun));
+ dprintk(NDEBUG_LISTS, "MAIN tmp=%p target=%d busy=%d lun=%d\n", tmp, tmp->device->id, hostdata->busy[tmp->device->id], tmp->device->lun);
/* When we find one, remove it from the issue queue. */
if (!(hostdata->busy[tmp->device->id] & (1 << tmp->device->lun))) {
if (prev) {
@@ -1066,7 +1057,7 @@
* On failure, we must add the command back to the
* issue queue so we can keep trying.
*/
- dprintk(NDEBUG_MAIN|NDEBUG_QUEUES, ("scsi%d : main() : command for target %d lun %d removed from issue_queue\n", instance->host_no, tmp->target, tmp->lun));
+ dprintk(NDEBUG_MAIN|NDEBUG_QUEUES, "scsi%d : main() : command for target %d lun %d removed from issue_queue\n", instance->host_no, tmp->device->id, tmp->device->lun);
/*
* A successful selection is defined as one that
@@ -1095,7 +1086,7 @@
tmp->host_scribble = (unsigned char *) hostdata->issue_queue;
hostdata->issue_queue = tmp;
done = 0;
- dprintk(NDEBUG_MAIN|NDEBUG_QUEUES, ("scsi%d : main(): select() failed, returned to issue_queue\n", instance->host_no));
+ dprintk(NDEBUG_MAIN|NDEBUG_QUEUES, "scsi%d : main(): select() failed, returned to issue_queue\n", instance->host_no);
}
/* lock held here still */
} /* if target/lun is not busy */
@@ -1125,9 +1116,9 @@
#endif
&& (!hostdata->time_expires || time_before_eq(hostdata->time_expires, jiffies))
) {
- dprintk(NDEBUG_MAIN, ("scsi%d : main() : performing information transfer\n", instance->host_no));
+ dprintk(NDEBUG_MAIN, "scsi%d : main() : performing information transfer\n", instance->host_no);
NCR5380_information_transfer(instance);
- dprintk(NDEBUG_MAIN, ("scsi%d : main() : done set false\n", instance->host_no));
+ dprintk(NDEBUG_MAIN, "scsi%d : main() : done set false\n", instance->host_no);
done = 0;
} else
break;
@@ -1159,8 +1150,8 @@
unsigned char basr;
unsigned long flags;
- dprintk(NDEBUG_INTR, ("scsi : NCR5380 irq %d triggered\n",
- instance->irq));
+ dprintk(NDEBUG_INTR, "scsi : NCR5380 irq %d triggered\n",
+ instance->irq);
do {
done = 1;
@@ -1173,14 +1164,14 @@
NCR5380_dprint(NDEBUG_INTR, instance);
if ((NCR5380_read(STATUS_REG) & (SR_SEL | SR_IO)) == (SR_SEL | SR_IO)) {
done = 0;
- dprintk(NDEBUG_INTR, ("scsi%d : SEL interrupt\n", instance->host_no));
+ dprintk(NDEBUG_INTR, "scsi%d : SEL interrupt\n", instance->host_no);
NCR5380_reselect(instance);
(void) NCR5380_read(RESET_PARITY_INTERRUPT_REG);
} else if (basr & BASR_PARITY_ERROR) {
- dprintk(NDEBUG_INTR, ("scsi%d : PARITY interrupt\n", instance->host_no));
+ dprintk(NDEBUG_INTR, "scsi%d : PARITY interrupt\n", instance->host_no);
(void) NCR5380_read(RESET_PARITY_INTERRUPT_REG);
} else if ((NCR5380_read(STATUS_REG) & SR_RST) == SR_RST) {
- dprintk(NDEBUG_INTR, ("scsi%d : RESET interrupt\n", instance->host_no));
+ dprintk(NDEBUG_INTR, "scsi%d : RESET interrupt\n", instance->host_no);
(void) NCR5380_read(RESET_PARITY_INTERRUPT_REG);
} else {
#if defined(REAL_DMA)
@@ -1210,7 +1201,7 @@
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
}
#else
- dprintk(NDEBUG_INTR, ("scsi : unknown interrupt, BASR 0x%X, MR 0x%X, SR 0x%x\n", basr, NCR5380_read(MODE_REG), NCR5380_read(STATUS_REG)));
+ dprintk(NDEBUG_INTR, "scsi : unknown interrupt, BASR 0x%X, MR 0x%X, SR 0x%x\n", basr, NCR5380_read(MODE_REG), NCR5380_read(STATUS_REG));
(void) NCR5380_read(RESET_PARITY_INTERRUPT_REG);
#endif
}
@@ -1304,7 +1295,7 @@
hostdata->restart_select = 0;
NCR5380_dprint(NDEBUG_ARBITRATION, instance);
- dprintk(NDEBUG_ARBITRATION, ("scsi%d : starting arbitration, id = %d\n", instance->host_no, instance->this_id));
+ dprintk(NDEBUG_ARBITRATION, "scsi%d : starting arbitration, id = %d\n", instance->host_no, instance->this_id);
/*
* Set the phase bits to 0, otherwise the NCR5380 won't drive the
@@ -1333,7 +1324,7 @@
goto failed;
}
- dprintk(NDEBUG_ARBITRATION, ("scsi%d : arbitration complete\n", instance->host_no));
+ dprintk(NDEBUG_ARBITRATION, "scsi%d : arbitration complete\n", instance->host_no);
/*
* The arbitration delay is 2.2us, but this is a minimum and there is
@@ -1347,7 +1338,7 @@
/* Check for lost arbitration */
if ((NCR5380_read(INITIATOR_COMMAND_REG) & ICR_ARBITRATION_LOST) || (NCR5380_read(CURRENT_SCSI_DATA_REG) & hostdata->id_higher_mask) || (NCR5380_read(INITIATOR_COMMAND_REG) & ICR_ARBITRATION_LOST)) {
NCR5380_write(MODE_REG, MR_BASE);
- dprintk(NDEBUG_ARBITRATION, ("scsi%d : lost arbitration, deasserting MR_ARBITRATE\n", instance->host_no));
+ dprintk(NDEBUG_ARBITRATION, "scsi%d : lost arbitration, deasserting MR_ARBITRATE\n", instance->host_no);
goto failed;
}
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_SEL);
@@ -1360,7 +1351,7 @@
(NCR5380_read(INITIATOR_COMMAND_REG) & ICR_ARBITRATION_LOST)) {
NCR5380_write(MODE_REG, MR_BASE);
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- dprintk(NDEBUG_ARBITRATION, ("scsi%d : lost arbitration, deasserting ICR_ASSERT_SEL\n", instance->host_no));
+ dprintk(NDEBUG_ARBITRATION, "scsi%d : lost arbitration, deasserting ICR_ASSERT_SEL\n", instance->host_no);
goto failed;
}
/*
@@ -1370,7 +1361,7 @@
udelay(2);
- dprintk(NDEBUG_ARBITRATION, ("scsi%d : won arbitration\n", instance->host_no));
+ dprintk(NDEBUG_ARBITRATION, "scsi%d : won arbitration\n", instance->host_no);
/*
* Now that we have won arbitration, start Selection process, asserting
@@ -1422,7 +1413,7 @@
udelay(1);
- dprintk(NDEBUG_SELECTION, ("scsi%d : selecting target %d\n", instance->host_no, scmd_id(cmd)));
+ dprintk(NDEBUG_SELECTION, "scsi%d : selecting target %d\n", instance->host_no, scmd_id(cmd));
/*
* The SCSI specification calls for a 250 ms timeout for the actual
@@ -1487,7 +1478,7 @@
collect_stats(hostdata, cmd);
cmd->scsi_done(cmd);
NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
- dprintk(NDEBUG_SELECTION, ("scsi%d : target did not respond within 250ms\n", instance->host_no));
+ dprintk(NDEBUG_SELECTION, "scsi%d : target did not respond within 250ms\n", instance->host_no);
NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
return 0;
}
@@ -1520,7 +1511,7 @@
goto failed;
}
- dprintk(NDEBUG_SELECTION, ("scsi%d : target %d selected, going into MESSAGE OUT phase.\n", instance->host_no, cmd->device->id));
+ dprintk(NDEBUG_SELECTION, "scsi%d : target %d selected, going into MESSAGE OUT phase.\n", instance->host_no, cmd->device->id);
tmp[0] = IDENTIFY(((instance->irq == SCSI_IRQ_NONE) ? 0 : 1), cmd->device->lun);
len = 1;
@@ -1530,7 +1521,7 @@
data = tmp;
phase = PHASE_MSGOUT;
NCR5380_transfer_pio(instance, &phase, &len, &data);
- dprintk(NDEBUG_SELECTION, ("scsi%d : nexus established.\n", instance->host_no));
+ dprintk(NDEBUG_SELECTION, "scsi%d : nexus established.\n", instance->host_no);
/* XXX need to handle errors here */
hostdata->connected = cmd;
hostdata->busy[cmd->device->id] |= (1 << cmd->device->lun);
@@ -1583,9 +1574,9 @@
NCR5380_setup(instance);
if (!(p & SR_IO))
- dprintk(NDEBUG_PIO, ("scsi%d : pio write %d bytes\n", instance->host_no, c));
+ dprintk(NDEBUG_PIO, "scsi%d : pio write %d bytes\n", instance->host_no, c);
else
- dprintk(NDEBUG_PIO, ("scsi%d : pio read %d bytes\n", instance->host_no, c));
+ dprintk(NDEBUG_PIO, "scsi%d : pio read %d bytes\n", instance->host_no, c);
/*
* The NCR5380 chip will only drive the SCSI bus when the
@@ -1620,11 +1611,11 @@
break;
}
- dprintk(NDEBUG_HANDSHAKE, ("scsi%d : REQ detected\n", instance->host_no));
+ dprintk(NDEBUG_HANDSHAKE, "scsi%d : REQ detected\n", instance->host_no);
/* Check for phase mismatch */
if ((tmp & PHASE_MASK) != p) {
- dprintk(NDEBUG_HANDSHAKE, ("scsi%d : phase mismatch\n", instance->host_no));
+ dprintk(NDEBUG_HANDSHAKE, "scsi%d : phase mismatch\n", instance->host_no);
NCR5380_dprint_phase(NDEBUG_HANDSHAKE, instance);
break;
}
@@ -1660,7 +1651,7 @@
/* FIXME - if this fails bus reset ?? */
NCR5380_poll_politely(instance, STATUS_REG, SR_REQ, 0, 5*HZ);
- dprintk(NDEBUG_HANDSHAKE, ("scsi%d : req false, handshake complete\n", instance->host_no));
+ dprintk(NDEBUG_HANDSHAKE, "scsi%d : req false, handshake complete\n", instance->host_no);
/*
* We have several special cases to consider during REQ/ACK handshaking :
@@ -1681,7 +1672,7 @@
}
} while (--c);
- dprintk(NDEBUG_PIO, ("scsi%d : residual %d\n", instance->host_no, c));
+ dprintk(NDEBUG_PIO, "scsi%d : residual %d\n", instance->host_no, c);
*count = c;
*data = d;
@@ -1828,7 +1819,7 @@
c -= 2;
}
#endif
- dprintk(NDEBUG_DMA, ("scsi%d : initializing DMA channel %d for %s, %d bytes %s %0x\n", instance->host_no, instance->dma_channel, (p & SR_IO) ? "reading" : "writing", c, (p & SR_IO) ? "to" : "from", (unsigned) d));
+ dprintk(NDEBUG_DMA, "scsi%d : initializing DMA channel %d for %s, %d bytes %s %0x\n", instance->host_no, instance->dma_channel, (p & SR_IO) ? "reading" : "writing", c, (p & SR_IO) ? "to" : "from", (unsigned) d);
hostdata->dma_len = (p & SR_IO) ? NCR5380_dma_read_setup(instance, d, c) : NCR5380_dma_write_setup(instance, d, c);
#endif
@@ -1857,7 +1848,7 @@
NCR5380_write(MODE_REG, MR_BASE | MR_DMA_MODE);
#endif /* def REAL_DMA */
- dprintk(NDEBUG_DMA, ("scsi%d : mode reg = 0x%X\n", instance->host_no, NCR5380_read(MODE_REG)));
+ dprintk(NDEBUG_DMA, "scsi%d : mode reg = 0x%X\n", instance->host_no, NCR5380_read(MODE_REG));
/*
* On the PAS16 at least I/O recovery delays are not needed here.
@@ -1934,7 +1925,7 @@
}
}
- dprintk(NDEBUG_DMA, ("scsi%d : polled DMA transfer complete, basr 0x%X, sr 0x%X\n", instance->host_no, tmp, NCR5380_read(STATUS_REG)));
+ dprintk(NDEBUG_DMA, "scsi%d : polled DMA transfer complete, basr 0x%X, sr 0x%X\n", instance->host_no, tmp, NCR5380_read(STATUS_REG));
NCR5380_write(MODE_REG, MR_BASE);
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
@@ -1948,7 +1939,7 @@
#ifdef READ_OVERRUNS
if (*phase == p && (p & SR_IO) && residue == 0) {
if (overrun) {
- dprintk(NDEBUG_DMA, ("Got an input overrun, using saved byte\n"));
+ dprintk(NDEBUG_DMA, "Got an input overrun, using saved byte\n");
**data = saved_data;
*data += 1;
*count -= 1;
@@ -1957,13 +1948,13 @@
printk("No overrun??\n");
cnt = toPIO = 2;
}
- dprintk(NDEBUG_DMA, ("Doing %d-byte PIO to 0x%X\n", cnt, *data));
+ dprintk(NDEBUG_DMA, "Doing %d-byte PIO to 0x%X\n", cnt, *data);
NCR5380_transfer_pio(instance, phase, &cnt, data);
*count -= toPIO - cnt;
}
#endif
- dprintk(NDEBUG_DMA, ("Return with data ptr = 0x%X, count %d, last 0x%X, next 0x%X\n", *data, *count, *(*data + *count - 1), *(*data + *count)));
+ dprintk(NDEBUG_DMA, "Return with data ptr = 0x%X, count %d, last 0x%X, next 0x%X\n", *data, *count, *(*data + *count - 1), *(*data + *count));
return 0;
#elif defined(REAL_DMA)
@@ -2013,7 +2004,7 @@
foo = NCR5380_pwrite(instance, d, c);
#else
int timeout;
- dprintk(NDEBUG_C400_PWRITE, ("About to pwrite %d bytes\n", c));
+ dprintk(NDEBUG_C400_PWRITE, "About to pwrite %d bytes\n", c);
if (!(foo = NCR5380_pwrite(instance, d, c))) {
/*
* Wait for the last byte to be sent. If REQ is being asserted for
@@ -2024,19 +2015,19 @@
while (!(NCR5380_read(BUS_AND_STATUS_REG) & BASR_DRQ) && (NCR5380_read(BUS_AND_STATUS_REG) & BASR_PHASE_MATCH));
if (!timeout)
- dprintk(NDEBUG_LAST_BYTE_SENT, ("scsi%d : timed out on last byte\n", instance->host_no));
+ dprintk(NDEBUG_LAST_BYTE_SENT, "scsi%d : timed out on last byte\n", instance->host_no);
if (hostdata->flags & FLAG_CHECK_LAST_BYTE_SENT) {
hostdata->flags &= ~FLAG_CHECK_LAST_BYTE_SENT;
if (NCR5380_read(TARGET_COMMAND_REG) & TCR_LAST_BYTE_SENT) {
hostdata->flags |= FLAG_HAS_LAST_BYTE_SENT;
- dprintk(NDEBUG_LAST_WRITE_SENT, ("scsi%d : last bit sent works\n", instance->host_no));
+ dprintk(NDEBUG_LAST_BYTE_SENT, "scsi%d : last byte sent works\n", instance->host_no);
}
}
} else {
- dprintk(NDEBUG_C400_PWRITE, ("Waiting for LASTBYTE\n"));
+ dprintk(NDEBUG_C400_PWRITE, "Waiting for LASTBYTE\n");
while (!(NCR5380_read(TARGET_COMMAND_REG) & TCR_LAST_BYTE_SENT));
- dprintk(NDEBUG_C400_PWRITE, ("Got LASTBYTE\n"));
+ dprintk(NDEBUG_C400_PWRITE, "Got LASTBYTE\n");
}
}
#endif
@@ -2045,9 +2036,9 @@
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
if ((!(p & SR_IO)) && (hostdata->flags & FLAG_NCR53C400)) {
- dprintk(NDEBUG_C400_PWRITE, ("53C400w: Checking for IRQ\n"));
+ dprintk(NDEBUG_C400_PWRITE, "53C400w: Checking for IRQ\n");
if (NCR5380_read(BUS_AND_STATUS_REG) & BASR_IRQ) {
- dprintk(NDEBUG_C400_PWRITE, ("53C400w: got it, reading reset interrupt reg\n"));
+ dprintk(NDEBUG_C400_PWRITE, "53C400w: got it, reading reset interrupt reg\n");
NCR5380_read(RESET_PARITY_INTERRUPT_REG);
} else {
printk("53C400w: IRQ NOT THERE!\n");
@@ -2139,7 +2130,7 @@
--cmd->SCp.buffers_residual;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
cmd->SCp.ptr = sg_virt(cmd->SCp.buffer);
- dprintk(NDEBUG_INFORMATION, ("scsi%d : %d bytes and %d buffers left\n", instance->host_no, cmd->SCp.this_residual, cmd->SCp.buffers_residual));
+ dprintk(NDEBUG_INFORMATION, "scsi%d : %d bytes and %d buffers left\n", instance->host_no, cmd->SCp.this_residual, cmd->SCp.buffers_residual);
}
/*
* The preferred transfer method is going to be
@@ -2219,7 +2210,7 @@
case LINKED_FLG_CMD_COMPLETE:
/* Accept message by clearing ACK */
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- dprintk(NDEBUG_LINKED, ("scsi%d : target %d lun %d linked command complete.\n", instance->host_no, cmd->device->id, cmd->device->lun));
+ dprintk(NDEBUG_LINKED, "scsi%d : target %d lun %d linked command complete.\n", instance->host_no, cmd->device->id, cmd->device->lun);
/*
* Sanity check : A linked command should only terminate with
* one of these messages if there are more linked commands
@@ -2235,7 +2226,7 @@
/* The next command is still part of this process */
cmd->next_link->tag = cmd->tag;
cmd->result = cmd->SCp.Status | (cmd->SCp.Message << 8);
- dprintk(NDEBUG_LINKED, ("scsi%d : target %d lun %d linked request done, calling scsi_done().\n", instance->host_no, cmd->device->id, cmd->device->lun));
+ dprintk(NDEBUG_LINKED, "scsi%d : target %d lun %d linked request done, calling scsi_done().\n", instance->host_no, cmd->device->id, cmd->device->lun);
collect_stats(hostdata, cmd);
cmd->scsi_done(cmd);
cmd = hostdata->connected;
@@ -2247,7 +2238,7 @@
sink = 1;
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
hostdata->connected = NULL;
- dprintk(NDEBUG_QUEUES, ("scsi%d : command for target %d, lun %d completed\n", instance->host_no, cmd->device->id, cmd->device->lun));
+ dprintk(NDEBUG_QUEUES, "scsi%d : command for target %d, lun %d completed\n", instance->host_no, cmd->device->id, cmd->device->lun);
hostdata->busy[cmd->device->id] &= ~(1 << cmd->device->lun);
/*
@@ -2281,13 +2272,13 @@
if ((cmd->cmnd[0] != REQUEST_SENSE) && (status_byte(cmd->SCp.Status) == CHECK_CONDITION)) {
scsi_eh_prep_cmnd(cmd, &hostdata->ses, NULL, 0, ~0);
- dprintk(NDEBUG_AUTOSENSE, ("scsi%d : performing request sense\n", instance->host_no));
+ dprintk(NDEBUG_AUTOSENSE, "scsi%d : performing request sense\n", instance->host_no);
LIST(cmd, hostdata->issue_queue);
cmd->host_scribble = (unsigned char *)
hostdata->issue_queue;
hostdata->issue_queue = (Scsi_Cmnd *) cmd;
- dprintk(NDEBUG_QUEUES, ("scsi%d : REQUEST SENSE added to head of issue queue\n", instance->host_no));
+ dprintk(NDEBUG_QUEUES, "scsi%d : REQUEST SENSE added to head of issue queue\n", instance->host_no);
} else
#endif /* def AUTOSENSE */
{
@@ -2327,7 +2318,7 @@
hostdata->disconnected_queue;
hostdata->connected = NULL;
hostdata->disconnected_queue = cmd;
- dprintk(NDEBUG_QUEUES, ("scsi%d : command for target %d lun %d was moved from connected to" " the disconnected_queue\n", instance->host_no, cmd->device->id, cmd->device->lun));
+ dprintk(NDEBUG_QUEUES, "scsi%d : command for target %d lun %d was moved from connected to" " the disconnected_queue\n", instance->host_no, cmd->device->id, cmd->device->lun);
/*
* Restore phase bits to 0 so an interrupted selection,
* arbitration can resume.
@@ -2373,14 +2364,14 @@
extended_msg[0] = EXTENDED_MESSAGE;
/* Accept first byte by clearing ACK */
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- dprintk(NDEBUG_EXTENDED, ("scsi%d : receiving extended message\n", instance->host_no));
+ dprintk(NDEBUG_EXTENDED, "scsi%d : receiving extended message\n", instance->host_no);
len = 2;
data = extended_msg + 1;
phase = PHASE_MSGIN;
NCR5380_transfer_pio(instance, &phase, &len, &data);
- dprintk(NDEBUG_EXTENDED, ("scsi%d : length=%d, code=0x%02x\n", instance->host_no, (int) extended_msg[1], (int) extended_msg[2]));
+ dprintk(NDEBUG_EXTENDED, "scsi%d : length=%d, code=0x%02x\n", instance->host_no, (int) extended_msg[1], (int) extended_msg[2]);
if (!len && extended_msg[1] <= (sizeof(extended_msg) - 1)) {
/* Accept third byte by clearing ACK */
@@ -2390,7 +2381,7 @@
phase = PHASE_MSGIN;
NCR5380_transfer_pio(instance, &phase, &len, &data);
- dprintk(NDEBUG_EXTENDED, ("scsi%d : message received, residual %d\n", instance->host_no, len));
+ dprintk(NDEBUG_EXTENDED, "scsi%d : message received, residual %d\n", instance->host_no, len);
switch (extended_msg[2]) {
case EXTENDED_SDTR:
@@ -2456,7 +2447,7 @@
NCR5380_transfer_pio(instance, &phase, &len, &data);
if (!cmd->device->disconnect && should_disconnect(cmd->cmnd[0])) {
NCR5380_set_timer(hostdata, USLEEP_SLEEP);
- dprintk(NDEBUG_USLEEP, ("scsi%d : issued command, sleeping until %ul\n", instance->host_no, hostdata->time_expires));
+ dprintk(NDEBUG_USLEEP, "scsi%d : issued command, sleeping until %lu\n", instance->host_no, hostdata->time_expires);
return;
}
break;
@@ -2468,7 +2459,7 @@
break;
default:
printk("scsi%d : unknown phase\n", instance->host_no);
- NCR5380_dprint(NDEBUG_ALL, instance);
+ NCR5380_dprint(NDEBUG_ANY, instance);
} /* switch(phase) */
} /* if (tmp * SR_REQ) */
else {
@@ -2476,7 +2467,7 @@
*/
if (!cmd->device->disconnect && time_after_eq(jiffies, poll_time)) {
NCR5380_set_timer(hostdata, USLEEP_SLEEP);
- dprintk(NDEBUG_USLEEP, ("scsi%d : poll timed out, sleeping until %ul\n", instance->host_no, hostdata->time_expires));
+ dprintk(NDEBUG_USLEEP, "scsi%d : poll timed out, sleeping until %lu\n", instance->host_no, hostdata->time_expires);
return;
}
}
@@ -2517,7 +2508,7 @@
hostdata->restart_select = 1;
target_mask = NCR5380_read(CURRENT_SCSI_DATA_REG) & ~(hostdata->id_mask);
- dprintk(NDEBUG_SELECTION, ("scsi%d : reselect\n", instance->host_no));
+ dprintk(NDEBUG_SELECTION, "scsi%d : reselect\n", instance->host_no);
/*
* At this point, we have detected that our SCSI ID is on the bus,
@@ -2597,7 +2588,7 @@
do_abort(instance);
} else {
hostdata->connected = tmp;
- dprintk(NDEBUG_RESELECTION, ("scsi%d : nexus established, target = %d, lun = %d, tag = %d\n", instance->host_no, tmp->target, tmp->lun, tmp->tag));
+ dprintk(NDEBUG_RESELECTION, "scsi%d : nexus established, target = %d, lun = %d, tag = %d\n", instance->host_no, tmp->device->id, tmp->device->lun, tmp->tag);
}
}
@@ -2682,8 +2673,8 @@
NCR5380_setup(instance);
- dprintk(NDEBUG_ABORT, ("scsi%d : abort called\n", instance->host_no));
- dprintk(NDEBUG_ABORT, (" basr 0x%X, sr 0x%X\n", NCR5380_read(BUS_AND_STATUS_REG), NCR5380_read(STATUS_REG)));
+ dprintk(NDEBUG_ABORT, "scsi%d : abort called\n", instance->host_no);
+ dprintk(NDEBUG_ABORT, " basr 0x%X, sr 0x%X\n", NCR5380_read(BUS_AND_STATUS_REG), NCR5380_read(STATUS_REG));
#if 0
/*
@@ -2693,7 +2684,7 @@
*/
if (hostdata->connected == cmd) {
- dprintk(NDEBUG_ABORT, ("scsi%d : aborting connected command\n", instance->host_no));
+ dprintk(NDEBUG_ABORT, "scsi%d : aborting connected command\n", instance->host_no);
hostdata->aborted = 1;
/*
* We should perform BSY checking, and make sure we haven't slipped
@@ -2721,14 +2712,14 @@
* from the issue queue.
*/
- dprintk(NDEBUG_ABORT, ("scsi%d : abort going into loop.\n", instance->host_no));
+ dprintk(NDEBUG_ABORT, "scsi%d : abort going into loop.\n", instance->host_no);
for (prev = (Scsi_Cmnd **) & (hostdata->issue_queue), tmp = (Scsi_Cmnd *) hostdata->issue_queue; tmp; prev = (Scsi_Cmnd **) & (tmp->host_scribble), tmp = (Scsi_Cmnd *) tmp->host_scribble)
if (cmd == tmp) {
REMOVE(5, *prev, tmp, tmp->host_scribble);
(*prev) = (Scsi_Cmnd *) tmp->host_scribble;
tmp->host_scribble = NULL;
tmp->result = DID_ABORT << 16;
- dprintk(NDEBUG_ABORT, ("scsi%d : abort removed command from issue queue.\n", instance->host_no));
+ dprintk(NDEBUG_ABORT, "scsi%d : abort removed command from issue queue.\n", instance->host_no);
tmp->scsi_done(tmp);
return SUCCESS;
}
@@ -2750,7 +2741,7 @@
*/
if (hostdata->connected) {
- dprintk(NDEBUG_ABORT, ("scsi%d : abort failed, command connected.\n", instance->host_no));
+ dprintk(NDEBUG_ABORT, "scsi%d : abort failed, command connected.\n", instance->host_no);
return FAILED;
}
/*
@@ -2780,11 +2771,11 @@
for (tmp = (Scsi_Cmnd *) hostdata->disconnected_queue; tmp; tmp = (Scsi_Cmnd *) tmp->host_scribble)
if (cmd == tmp) {
- dprintk(NDEBUG_ABORT, ("scsi%d : aborting disconnected command.\n", instance->host_no));
+ dprintk(NDEBUG_ABORT, "scsi%d : aborting disconnected command.\n", instance->host_no);
if (NCR5380_select(instance, cmd, (int) cmd->tag))
return FAILED;
- dprintk(NDEBUG_ABORT, ("scsi%d : nexus reestablished.\n", instance->host_no));
+ dprintk(NDEBUG_ABORT, "scsi%d : nexus reestablished.\n", instance->host_no);
do_abort(instance);
diff --git a/drivers/scsi/NCR5380.h b/drivers/scsi/NCR5380.h
index 14964d0..c79ddfa 100644
--- a/drivers/scsi/NCR5380.h
+++ b/drivers/scsi/NCR5380.h
@@ -21,10 +21,6 @@
* 1+ (800) 334-5454
*/
-/*
- * $Log: NCR5380.h,v $
- */
-
#ifndef NCR5380_H
#define NCR5380_H
@@ -60,6 +56,9 @@
#define NDEBUG_C400_PREAD 0x100000
#define NDEBUG_C400_PWRITE 0x200000
#define NDEBUG_LISTS 0x400000
+#define NDEBUG_ABORT 0x800000
+#define NDEBUG_TAGS 0x1000000
+#define NDEBUG_MERGING 0x2000000
#define NDEBUG_ANY 0xFFFFFFFFUL
@@ -292,9 +291,24 @@
#ifdef __KERNEL__
-#define dprintk(a,b) do {} while(0)
-#define NCR5380_dprint(a,b) do {} while(0)
-#define NCR5380_dprint_phase(a,b) do {} while(0)
+#ifndef NDEBUG
+#define NDEBUG (0)
+#endif
+
+#define dprintk(flg, fmt, ...) \
+ do { if ((NDEBUG) & (flg)) pr_debug(fmt, ## __VA_ARGS__); } while (0)
+
+#if NDEBUG
+#define NCR5380_dprint(flg, arg) \
+ do { if ((NDEBUG) & (flg)) NCR5380_print(arg); } while (0)
+#define NCR5380_dprint_phase(flg, arg) \
+ do { if ((NDEBUG) & (flg)) NCR5380_print_phase(arg); } while (0)
+static void NCR5380_print_phase(struct Scsi_Host *instance);
+static void NCR5380_print(struct Scsi_Host *instance);
+#else
+#define NCR5380_dprint(flg, arg) do {} while (0)
+#define NCR5380_dprint_phase(flg, arg) do {} while (0)
+#endif
#if defined(AUTOPROBE_IRQ)
static int NCR5380_probe_irq(struct Scsi_Host *instance, int possible);
@@ -307,10 +321,6 @@
#endif
static void NCR5380_main(struct work_struct *work);
static void __maybe_unused NCR5380_print_options(struct Scsi_Host *instance);
-#ifdef NDEBUG
-static void NCR5380_print_phase(struct Scsi_Host *instance);
-static void NCR5380_print(struct Scsi_Host *instance);
-#endif
static int NCR5380_abort(Scsi_Cmnd * cmd);
static int NCR5380_bus_reset(Scsi_Cmnd * cmd);
static int NCR5380_queue_command(struct Scsi_Host *, struct scsi_cmnd *);
diff --git a/drivers/scsi/aic7xxx/aic79xx_pci.c b/drivers/scsi/aic7xxx/aic79xx_pci.c
index 14b5f8d..cc9bd26 100644
--- a/drivers/scsi/aic7xxx/aic79xx_pci.c
+++ b/drivers/scsi/aic7xxx/aic79xx_pci.c
@@ -827,7 +827,7 @@
for (bit = 0; bit < 8; bit++) {
if ((pci_status[i] & (0x1 << bit)) != 0) {
- static const char *s;
+ const char *s;
s = pci_status_strings[bit];
if (i == 7/*TARG*/ && bit == 3)
@@ -887,23 +887,15 @@
for (bit = 0; bit < 8; bit++) {
- if ((split_status[i] & (0x1 << bit)) != 0) {
- static const char *s;
-
- s = split_status_strings[bit];
- printk(s, ahd_name(ahd),
+ if ((split_status[i] & (0x1 << bit)) != 0)
+ printk(split_status_strings[bit], ahd_name(ahd),
split_status_source[i]);
- }
if (i > 1)
continue;
- if ((sg_split_status[i] & (0x1 << bit)) != 0) {
- static const char *s;
-
- s = split_status_strings[bit];
- printk(s, ahd_name(ahd), "SG");
- }
+ if ((sg_split_status[i] & (0x1 << bit)) != 0)
+ printk(split_status_strings[bit], ahd_name(ahd), "SG");
}
}
/*
diff --git a/drivers/scsi/arm/acornscsi.c b/drivers/scsi/arm/acornscsi.c
index 059ff47..2e797a3 100644
--- a/drivers/scsi/arm/acornscsi.c
+++ b/drivers/scsi/arm/acornscsi.c
@@ -62,13 +62,6 @@
*/
#undef CONFIG_SCSI_ACORNSCSI_TAGGED_QUEUE
/*
- * SCSI-II Linked command support.
- *
- * The higher level code doesn't support linked commands yet, and so the option
- * is undef'd here.
- */
-#undef CONFIG_SCSI_ACORNSCSI_LINK
-/*
* SCSI-II Synchronous transfer support.
*
* Tried and tested...
@@ -160,10 +153,6 @@
#error "Yippee! ABORT TAG is now defined! Remove this error!"
#endif
-#ifdef CONFIG_SCSI_ACORNSCSI_LINK
-#error SCSI2 LINKed commands not supported (yet)!
-#endif
-
#ifdef USE_DMAC
/*
* DMAC setup parameters
@@ -1668,42 +1657,6 @@
}
break;
-#ifdef CONFIG_SCSI_ACORNSCSI_LINK
- case LINKED_CMD_COMPLETE:
- case LINKED_FLG_CMD_COMPLETE:
- /*
- * We don't support linked commands yet
- */
- if (0) {
-#if (DEBUG & DEBUG_LINK)
- printk("scsi%d.%c: lun %d tag %d linked command complete\n",
- host->host->host_no, acornscsi_target(host), host->SCpnt->tag);
-#endif
- /*
- * A linked command should only terminate with one of these messages
- * if there are more linked commands available.
- */
- if (!host->SCpnt->next_link) {
- printk(KERN_WARNING "scsi%d.%c: lun %d tag %d linked command complete, but no next_link\n",
- instance->host_no, acornscsi_target(host), host->SCpnt->tag);
- acornscsi_sbic_issuecmd(host, CMND_ASSERTATN);
- msgqueue_addmsg(&host->scsi.msgs, 1, ABORT);
- } else {
- struct scsi_cmnd *SCpnt = host->SCpnt;
-
- acornscsi_dma_cleanup(host);
-
- host->SCpnt = host->SCpnt->next_link;
- host->SCpnt->tag = SCpnt->tag;
- SCpnt->result = DID_OK | host->scsi.SCp.Message << 8 | host->Scsi.SCp.Status;
- SCpnt->done(SCpnt);
-
- /* initialise host->SCpnt->SCp */
- }
- break;
- }
-#endif
-
default: /* reject message */
printk(KERN_ERR "scsi%d.%c: unrecognised message %02X, rejecting\n",
host->host->host_no, acornscsi_target(host),
@@ -2825,9 +2778,6 @@
#ifdef CONFIG_SCSI_ACORNSCSI_TAGGED_QUEUE
" TAG"
#endif
-#ifdef CONFIG_SCSI_ACORNSCSI_LINK
- " LINK"
-#endif
#if (DEBUG & DEBUG_NO_WRITE)
" NOWRITE (" __stringify(NO_WRITE) ")"
#endif
@@ -2851,9 +2801,6 @@
#ifdef CONFIG_SCSI_ACORNSCSI_TAGGED_QUEUE
" TAG"
#endif
-#ifdef CONFIG_SCSI_ACORNSCSI_LINK
- " LINK"
-#endif
#if (DEBUG & DEBUG_NO_WRITE)
" NOWRITE (" __stringify(NO_WRITE) ")"
#endif
diff --git a/drivers/scsi/arm/cumana_1.c b/drivers/scsi/arm/cumana_1.c
index f8e0609..8ef810a 100644
--- a/drivers/scsi/arm/cumana_1.c
+++ b/drivers/scsi/arm/cumana_1.c
@@ -36,9 +36,6 @@
void __iomem *base; \
void __iomem *dma
-#define BOARD_NORMAL 0
-#define BOARD_NCR53C400 1
-
#include "../NCR5380.h"
void cumanascsi_setup(char *str, int *ints)
diff --git a/drivers/scsi/arm/oak.c b/drivers/scsi/arm/oak.c
index 4266eef..188e734 100644
--- a/drivers/scsi/arm/oak.c
+++ b/drivers/scsi/arm/oak.c
@@ -37,9 +37,6 @@
#define NCR5380_implementation_fields \
void __iomem *base
-#define BOARD_NORMAL 0
-#define BOARD_NCR53C400 1
-
#include "../NCR5380.h"
#undef START_DMA_INITIATOR_RECEIVE_REG
diff --git a/drivers/scsi/atari_NCR5380.c b/drivers/scsi/atari_NCR5380.c
index 0f3cdbc..1814aa2 100644
--- a/drivers/scsi/atari_NCR5380.c
+++ b/drivers/scsi/atari_NCR5380.c
@@ -370,7 +370,7 @@
return 0;
if (TagAlloc[cmd->device->id][cmd->device->lun].nr_allocated >=
TagAlloc[cmd->device->id][cmd->device->lun].queue_size) {
- TAG_PRINTK("scsi%d: target %d lun %d: no free tags\n",
+ dprintk(NDEBUG_TAGS, "scsi%d: target %d lun %d: no free tags\n",
H_NO(cmd), cmd->device->id, cmd->device->lun);
return 1;
}
@@ -394,7 +394,7 @@
!setup_use_tagged_queuing || !cmd->device->tagged_supported) {
cmd->tag = TAG_NONE;
hostdata->busy[cmd->device->id] |= (1 << cmd->device->lun);
- TAG_PRINTK("scsi%d: target %d lun %d now allocated by untagged "
+ dprintk(NDEBUG_TAGS, "scsi%d: target %d lun %d now allocated by untagged "
"command\n", H_NO(cmd), cmd->device->id, cmd->device->lun);
} else {
TAG_ALLOC *ta = &TagAlloc[cmd->device->id][cmd->device->lun];
@@ -402,7 +402,7 @@
cmd->tag = find_first_zero_bit(ta->allocated, MAX_TAGS);
set_bit(cmd->tag, ta->allocated);
ta->nr_allocated++;
- TAG_PRINTK("scsi%d: using tag %d for target %d lun %d "
+ dprintk(NDEBUG_TAGS, "scsi%d: using tag %d for target %d lun %d "
"(now %d tags in use)\n",
H_NO(cmd), cmd->tag, cmd->device->id,
cmd->device->lun, ta->nr_allocated);
@@ -420,7 +420,7 @@
if (cmd->tag == TAG_NONE) {
hostdata->busy[cmd->device->id] &= ~(1 << cmd->device->lun);
- TAG_PRINTK("scsi%d: target %d lun %d untagged cmd finished\n",
+ dprintk(NDEBUG_TAGS, "scsi%d: target %d lun %d untagged cmd finished\n",
H_NO(cmd), cmd->device->id, cmd->device->lun);
} else if (cmd->tag >= MAX_TAGS) {
printk(KERN_NOTICE "scsi%d: trying to free bad tag %d!\n",
@@ -429,7 +429,7 @@
TAG_ALLOC *ta = &TagAlloc[cmd->device->id][cmd->device->lun];
clear_bit(cmd->tag, ta->allocated);
ta->nr_allocated--;
- TAG_PRINTK("scsi%d: freed tag %d for target %d lun %d\n",
+ dprintk(NDEBUG_TAGS, "scsi%d: freed tag %d for target %d lun %d\n",
H_NO(cmd), cmd->tag, cmd->device->id, cmd->device->lun);
}
}
@@ -478,7 +478,7 @@
for (endaddr = virt_to_phys(cmd->SCp.ptr + cmd->SCp.this_residual - 1) + 1;
cmd->SCp.buffers_residual &&
virt_to_phys(sg_virt(&cmd->SCp.buffer[1])) == endaddr;) {
- MER_PRINTK("VTOP(%p) == %08lx -> merging\n",
+ dprintk(NDEBUG_MERGING, "VTOP(%p) == %08lx -> merging\n",
page_address(sg_page(&cmd->SCp.buffer[1])), endaddr);
#if (NDEBUG & NDEBUG_MERGING)
++cnt;
@@ -490,7 +490,7 @@
}
#if (NDEBUG & NDEBUG_MERGING)
if (oldlen != cmd->SCp.this_residual)
- MER_PRINTK("merged %d buffers from %p, new length %08x\n",
+ dprintk(NDEBUG_MERGING, "merged %d buffers from %p, new length %08x\n",
cnt, cmd->SCp.ptr, cmd->SCp.this_residual);
#endif
}
@@ -626,16 +626,6 @@
}
}
-#else /* !NDEBUG */
-
-/* dummies... */
-static inline void NCR5380_print(struct Scsi_Host *instance)
-{
-};
-static inline void NCR5380_print_phase(struct Scsi_Host *instance)
-{
-};
-
#endif
/*
@@ -676,7 +666,7 @@
{
static int done = 0;
if (!done) {
- INI_PRINTK("scsi : NCR5380_all_init()\n");
+ dprintk(NDEBUG_INIT, "scsi : NCR5380_all_init()\n");
done = 1;
}
}
@@ -739,8 +729,8 @@
Scsi_Cmnd *ptr;
unsigned long flags;
- NCR_PRINT(NDEBUG_ANY);
- NCR_PRINT_PHASE(NDEBUG_ANY);
+ NCR5380_dprint(NDEBUG_ANY, instance);
+ NCR5380_dprint_phase(NDEBUG_ANY, instance);
hostdata = (struct NCR5380_hostdata *)instance->hostdata;
@@ -984,7 +974,7 @@
}
local_irq_restore(flags);
- QU_PRINTK("scsi%d: command added to %s of queue\n", H_NO(cmd),
+ dprintk(NDEBUG_QUEUES, "scsi%d: command added to %s of queue\n", H_NO(cmd),
(cmd->cmnd[0] == REQUEST_SENSE) ? "head" : "tail");
/* If queue_command() is called from an interrupt (real one or bottom
@@ -1054,7 +1044,7 @@
done = 1;
if (!hostdata->connected) {
- MAIN_PRINTK("scsi%d: not connected\n", HOSTNO);
+ dprintk(NDEBUG_MAIN, "scsi%d: not connected\n", HOSTNO);
/*
* Search through the issue_queue for a command destined
* for a target that's not busy.
@@ -1107,7 +1097,7 @@
* On failure, we must add the command back to the
* issue queue so we can keep trying.
*/
- MAIN_PRINTK("scsi%d: main(): command for target %d "
+ dprintk(NDEBUG_MAIN, "scsi%d: main(): command for target %d "
"lun %d removed from issue_queue\n",
HOSTNO, tmp->device->id, tmp->device->lun);
/*
@@ -1140,7 +1130,7 @@
#endif
falcon_dont_release--;
local_irq_restore(flags);
- MAIN_PRINTK("scsi%d: main(): select() failed, "
+ dprintk(NDEBUG_MAIN, "scsi%d: main(): select() failed, "
"returned to issue_queue\n", HOSTNO);
if (hostdata->connected)
break;
@@ -1155,10 +1145,10 @@
#endif
) {
local_irq_restore(flags);
- MAIN_PRINTK("scsi%d: main: performing information transfer\n",
+ dprintk(NDEBUG_MAIN, "scsi%d: main: performing information transfer\n",
HOSTNO);
NCR5380_information_transfer(instance);
- MAIN_PRINTK("scsi%d: main: done set false\n", HOSTNO);
+ dprintk(NDEBUG_MAIN, "scsi%d: main: done set false\n", HOSTNO);
done = 0;
}
} while (!done);
@@ -1204,12 +1194,12 @@
(BASR_PHASE_MATCH|BASR_ACK)) {
saved_data = NCR5380_read(INPUT_DATA_REG);
overrun = 1;
- DMA_PRINTK("scsi%d: read overrun handled\n", HOSTNO);
+ dprintk(NDEBUG_DMA, "scsi%d: read overrun handled\n", HOSTNO);
}
}
}
- DMA_PRINTK("scsi%d: real DMA transfer complete, basr 0x%X, sr 0x%X\n",
+ dprintk(NDEBUG_DMA, "scsi%d: real DMA transfer complete, basr 0x%X, sr 0x%X\n",
HOSTNO, NCR5380_read(BUS_AND_STATUS_REG),
NCR5380_read(STATUS_REG));
@@ -1229,13 +1219,13 @@
if ((NCR5380_read(STATUS_REG) & PHASE_MASK) == p && (p & SR_IO)) {
cnt = toPIO = atari_read_overruns;
if (overrun) {
- DMA_PRINTK("Got an input overrun, using saved byte\n");
+ dprintk(NDEBUG_DMA, "Got an input overrun, using saved byte\n");
*(*data)++ = saved_data;
(*count)--;
cnt--;
toPIO--;
}
- DMA_PRINTK("Doing %d-byte PIO to 0x%08lx\n", cnt, (long)*data);
+ dprintk(NDEBUG_DMA, "Doing %d-byte PIO to 0x%08lx\n", cnt, (long)*data);
NCR5380_transfer_pio(instance, &p, &cnt, data);
*count -= toPIO - cnt;
}
@@ -1261,25 +1251,25 @@
int done = 1, handled = 0;
unsigned char basr;
- INT_PRINTK("scsi%d: NCR5380 irq triggered\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: NCR5380 irq triggered\n", HOSTNO);
/* Look for pending interrupts */
basr = NCR5380_read(BUS_AND_STATUS_REG);
- INT_PRINTK("scsi%d: BASR=%02x\n", HOSTNO, basr);
+ dprintk(NDEBUG_INTR, "scsi%d: BASR=%02x\n", HOSTNO, basr);
/* dispatch to appropriate routine if found and done=0 */
if (basr & BASR_IRQ) {
- NCR_PRINT(NDEBUG_INTR);
+ NCR5380_dprint(NDEBUG_INTR, instance);
if ((NCR5380_read(STATUS_REG) & (SR_SEL|SR_IO)) == (SR_SEL|SR_IO)) {
done = 0;
ENABLE_IRQ();
- INT_PRINTK("scsi%d: SEL interrupt\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: SEL interrupt\n", HOSTNO);
NCR5380_reselect(instance);
(void)NCR5380_read(RESET_PARITY_INTERRUPT_REG);
} else if (basr & BASR_PARITY_ERROR) {
- INT_PRINTK("scsi%d: PARITY interrupt\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: PARITY interrupt\n", HOSTNO);
(void)NCR5380_read(RESET_PARITY_INTERRUPT_REG);
} else if ((NCR5380_read(STATUS_REG) & SR_RST) == SR_RST) {
- INT_PRINTK("scsi%d: RESET interrupt\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: RESET interrupt\n", HOSTNO);
(void)NCR5380_read(RESET_PARITY_INTERRUPT_REG);
} else {
/*
@@ -1298,7 +1288,7 @@
((basr & BASR_END_DMA_TRANSFER) ||
!(basr & BASR_PHASE_MATCH))) {
- INT_PRINTK("scsi%d: PHASE MISM or EOP interrupt\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: PHASE MISM or EOP interrupt\n", HOSTNO);
NCR5380_dma_complete( instance );
done = 0;
ENABLE_IRQ();
@@ -1323,7 +1313,7 @@
}
if (!done) {
- INT_PRINTK("scsi%d: in int routine, calling main\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: in int routine, calling main\n", HOSTNO);
/* Put a call to NCR5380_main() on the queue... */
queue_main();
}
@@ -1396,8 +1386,8 @@
unsigned long flags;
hostdata->restart_select = 0;
- NCR_PRINT(NDEBUG_ARBITRATION);
- ARB_PRINTK("scsi%d: starting arbitration, id = %d\n", HOSTNO,
+ NCR5380_dprint(NDEBUG_ARBITRATION, instance);
+ dprintk(NDEBUG_ARBITRATION, "scsi%d: starting arbitration, id = %d\n", HOSTNO,
instance->this_id);
/*
@@ -1442,7 +1432,7 @@
;
#endif
- ARB_PRINTK("scsi%d: arbitration complete\n", HOSTNO);
+ dprintk(NDEBUG_ARBITRATION, "scsi%d: arbitration complete\n", HOSTNO);
if (hostdata->connected) {
NCR5380_write(MODE_REG, MR_BASE);
@@ -1463,7 +1453,7 @@
(NCR5380_read(INITIATOR_COMMAND_REG) & ICR_ARBITRATION_LOST) ||
hostdata->connected) {
NCR5380_write(MODE_REG, MR_BASE);
- ARB_PRINTK("scsi%d: lost arbitration, deasserting MR_ARBITRATE\n",
+ dprintk(NDEBUG_ARBITRATION, "scsi%d: lost arbitration, deasserting MR_ARBITRATE\n",
HOSTNO);
return -1;
}
@@ -1478,7 +1468,7 @@
hostdata->connected) {
NCR5380_write(MODE_REG, MR_BASE);
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- ARB_PRINTK("scsi%d: lost arbitration, deasserting ICR_ASSERT_SEL\n",
+ dprintk(NDEBUG_ARBITRATION, "scsi%d: lost arbitration, deasserting ICR_ASSERT_SEL\n",
HOSTNO);
return -1;
}
@@ -1501,7 +1491,7 @@
return -1;
}
- ARB_PRINTK("scsi%d: won arbitration\n", HOSTNO);
+ dprintk(NDEBUG_ARBITRATION, "scsi%d: won arbitration\n", HOSTNO);
/*
* Now that we have won arbitration, start Selection process, asserting
@@ -1561,7 +1551,7 @@
udelay(1);
- SEL_PRINTK("scsi%d: selecting target %d\n", HOSTNO, cmd->device->id);
+ dprintk(NDEBUG_SELECTION, "scsi%d: selecting target %d\n", HOSTNO, cmd->device->id);
/*
* The SCSI specification calls for a 250 ms timeout for the actual
@@ -1617,7 +1607,7 @@
printk(KERN_ERR "scsi%d: weirdness\n", HOSTNO);
if (hostdata->restart_select)
printk(KERN_NOTICE "\trestart select\n");
- NCR_PRINT(NDEBUG_ANY);
+ NCR5380_dprint(NDEBUG_ANY, instance);
NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
return -1;
}
@@ -1630,7 +1620,7 @@
#endif
cmd->scsi_done(cmd);
NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
- SEL_PRINTK("scsi%d: target did not respond within 250ms\n", HOSTNO);
+ dprintk(NDEBUG_SELECTION, "scsi%d: target did not respond within 250ms\n", HOSTNO);
NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
return 0;
}
@@ -1656,7 +1646,7 @@
while (!(NCR5380_read(STATUS_REG) & SR_REQ))
;
- SEL_PRINTK("scsi%d: target %d selected, going into MESSAGE OUT phase.\n",
+ dprintk(NDEBUG_SELECTION, "scsi%d: target %d selected, going into MESSAGE OUT phase.\n",
HOSTNO, cmd->device->id);
tmp[0] = IDENTIFY(1, cmd->device->lun);
@@ -1676,7 +1666,7 @@
data = tmp;
phase = PHASE_MSGOUT;
NCR5380_transfer_pio(instance, &phase, &len, &data);
- SEL_PRINTK("scsi%d: nexus established.\n", HOSTNO);
+ dprintk(NDEBUG_SELECTION, "scsi%d: nexus established.\n", HOSTNO);
/* XXX need to handle errors here */
hostdata->connected = cmd;
#ifndef SUPPORT_TAGS
@@ -1737,12 +1727,12 @@
while (!((tmp = NCR5380_read(STATUS_REG)) & SR_REQ))
;
- HSH_PRINTK("scsi%d: REQ detected\n", HOSTNO);
+ dprintk(NDEBUG_HANDSHAKE, "scsi%d: REQ detected\n", HOSTNO);
/* Check for phase mismatch */
if ((tmp & PHASE_MASK) != p) {
- PIO_PRINTK("scsi%d: phase mismatch\n", HOSTNO);
- NCR_PRINT_PHASE(NDEBUG_PIO);
+ dprintk(NDEBUG_PIO, "scsi%d: phase mismatch\n", HOSTNO);
+ NCR5380_dprint_phase(NDEBUG_PIO, instance);
break;
}
@@ -1764,25 +1754,25 @@
if (!(p & SR_IO)) {
if (!((p & SR_MSG) && c > 1)) {
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_DATA);
- NCR_PRINT(NDEBUG_PIO);
+ NCR5380_dprint(NDEBUG_PIO, instance);
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE |
ICR_ASSERT_DATA | ICR_ASSERT_ACK);
} else {
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE |
ICR_ASSERT_DATA | ICR_ASSERT_ATN);
- NCR_PRINT(NDEBUG_PIO);
+ NCR5380_dprint(NDEBUG_PIO, instance);
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE |
ICR_ASSERT_DATA | ICR_ASSERT_ATN | ICR_ASSERT_ACK);
}
} else {
- NCR_PRINT(NDEBUG_PIO);
+ NCR5380_dprint(NDEBUG_PIO, instance);
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_ACK);
}
while (NCR5380_read(STATUS_REG) & SR_REQ)
;
- HSH_PRINTK("scsi%d: req false, handshake complete\n", HOSTNO);
+ dprintk(NDEBUG_HANDSHAKE, "scsi%d: req false, handshake complete\n", HOSTNO);
/*
* We have several special cases to consider during REQ/ACK handshaking :
@@ -1803,7 +1793,7 @@
}
} while (--c);
- PIO_PRINTK("scsi%d: residual %d\n", HOSTNO, c);
+ dprintk(NDEBUG_PIO, "scsi%d: residual %d\n", HOSTNO, c);
*count = c;
*data = d;
@@ -1917,7 +1907,7 @@
if (atari_read_overruns && (p & SR_IO))
c -= atari_read_overruns;
- DMA_PRINTK("scsi%d: initializing DMA for %s, %d bytes %s %p\n",
+ dprintk(NDEBUG_DMA, "scsi%d: initializing DMA for %s, %d bytes %s %p\n",
HOSTNO, (p & SR_IO) ? "reading" : "writing",
c, (p & SR_IO) ? "to" : "from", d);
@@ -1997,7 +1987,7 @@
phase = (tmp & PHASE_MASK);
if (phase != old_phase) {
old_phase = phase;
- NCR_PRINT_PHASE(NDEBUG_INFORMATION);
+ NCR5380_dprint_phase(NDEBUG_INFORMATION, instance);
}
if (sink && (phase != PHASE_MSGOUT)) {
@@ -2039,7 +2029,7 @@
* they are at contiguous physical addresses.
*/
merge_contiguous_buffers(cmd);
- INF_PRINTK("scsi%d: %d bytes and %d buffers left\n",
+ dprintk(NDEBUG_INFORMATION, "scsi%d: %d bytes and %d buffers left\n",
HOSTNO, cmd->SCp.this_residual,
cmd->SCp.buffers_residual);
}
@@ -2123,7 +2113,7 @@
/* Accept message by clearing ACK */
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- LNK_PRINTK("scsi%d: target %d lun %d linked command "
+ dprintk(NDEBUG_LINKED, "scsi%d: target %d lun %d linked command "
"complete.\n", HOSTNO, cmd->device->id, cmd->device->lun);
/* Enable reselect interrupts */
@@ -2148,7 +2138,7 @@
* and don't free it! */
cmd->next_link->tag = cmd->tag;
cmd->result = cmd->SCp.Status | (cmd->SCp.Message << 8);
- LNK_PRINTK("scsi%d: target %d lun %d linked request "
+ dprintk(NDEBUG_LINKED, "scsi%d: target %d lun %d linked request "
"done, calling scsi_done().\n",
HOSTNO, cmd->device->id, cmd->device->lun);
#ifdef NCR5380_STATS
@@ -2165,7 +2155,7 @@
/* ++guenther: possible race with Falcon locking */
falcon_dont_release++;
hostdata->connected = NULL;
- QU_PRINTK("scsi%d: command for target %d, lun %d "
+ dprintk(NDEBUG_QUEUES, "scsi%d: command for target %d, lun %d "
"completed\n", HOSTNO, cmd->device->id, cmd->device->lun);
#ifdef SUPPORT_TAGS
cmd_free_tag(cmd);
@@ -2179,7 +2169,7 @@
/* ++Andreas: the mid level code knows about
QUEUE_FULL now. */
TAG_ALLOC *ta = &TagAlloc[cmd->device->id][cmd->device->lun];
- TAG_PRINTK("scsi%d: target %d lun %d returned "
+ dprintk(NDEBUG_TAGS, "scsi%d: target %d lun %d returned "
"QUEUE_FULL after %d commands\n",
HOSTNO, cmd->device->id, cmd->device->lun,
ta->nr_allocated);
@@ -2224,14 +2214,14 @@
(status_byte(cmd->SCp.Status) == CHECK_CONDITION)) {
scsi_eh_prep_cmnd(cmd, &hostdata->ses, NULL, 0, ~0);
- ASEN_PRINTK("scsi%d: performing request sense\n", HOSTNO);
+ dprintk(NDEBUG_AUTOSENSE, "scsi%d: performing request sense\n", HOSTNO);
local_irq_save(flags);
LIST(cmd,hostdata->issue_queue);
SET_NEXT(cmd, hostdata->issue_queue);
hostdata->issue_queue = (Scsi_Cmnd *) cmd;
local_irq_restore(flags);
- QU_PRINTK("scsi%d: REQUEST SENSE added to head of "
+ dprintk(NDEBUG_QUEUES, "scsi%d: REQUEST SENSE added to head of "
"issue queue\n", H_NO(cmd));
} else
#endif /* def AUTOSENSE */
@@ -2277,7 +2267,7 @@
cmd->device->tagged_supported = 0;
hostdata->busy[cmd->device->id] |= (1 << cmd->device->lun);
cmd->tag = TAG_NONE;
- TAG_PRINTK("scsi%d: target %d lun %d rejected "
+ dprintk(NDEBUG_TAGS, "scsi%d: target %d lun %d rejected "
"QUEUE_TAG message; tagged queuing "
"disabled\n",
HOSTNO, cmd->device->id, cmd->device->lun);
@@ -2294,7 +2284,7 @@
hostdata->connected = NULL;
hostdata->disconnected_queue = cmd;
local_irq_restore(flags);
- QU_PRINTK("scsi%d: command for target %d lun %d was "
+ dprintk(NDEBUG_QUEUES, "scsi%d: command for target %d lun %d was "
"moved from connected to the "
"disconnected_queue\n", HOSTNO,
cmd->device->id, cmd->device->lun);
@@ -2344,13 +2334,13 @@
/* Accept first byte by clearing ACK */
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- EXT_PRINTK("scsi%d: receiving extended message\n", HOSTNO);
+ dprintk(NDEBUG_EXTENDED, "scsi%d: receiving extended message\n", HOSTNO);
len = 2;
data = extended_msg + 1;
phase = PHASE_MSGIN;
NCR5380_transfer_pio(instance, &phase, &len, &data);
- EXT_PRINTK("scsi%d: length=%d, code=0x%02x\n", HOSTNO,
+ dprintk(NDEBUG_EXTENDED, "scsi%d: length=%d, code=0x%02x\n", HOSTNO,
(int)extended_msg[1], (int)extended_msg[2]);
if (!len && extended_msg[1] <=
@@ -2362,7 +2352,7 @@
phase = PHASE_MSGIN;
NCR5380_transfer_pio(instance, &phase, &len, &data);
- EXT_PRINTK("scsi%d: message received, residual %d\n",
+ dprintk(NDEBUG_EXTENDED, "scsi%d: message received, residual %d\n",
HOSTNO, len);
switch (extended_msg[2]) {
@@ -2451,7 +2441,7 @@
break;
default:
printk("scsi%d: unknown phase\n", HOSTNO);
- NCR_PRINT(NDEBUG_ANY);
+ NCR5380_dprint(NDEBUG_ANY, instance);
} /* switch(phase) */
} /* if (tmp * SR_REQ) */
} /* while (1) */
@@ -2493,7 +2483,7 @@
target_mask = NCR5380_read(CURRENT_SCSI_DATA_REG) & ~(hostdata->id_mask);
- RSL_PRINTK("scsi%d: reselect\n", HOSTNO);
+ dprintk(NDEBUG_RESELECTION, "scsi%d: reselect\n", HOSTNO);
/*
* At this point, we have detected that our SCSI ID is on the bus,
@@ -2544,7 +2534,7 @@
if (!NCR5380_transfer_pio(instance, &phase, &len, &data) &&
msg[1] == SIMPLE_QUEUE_TAG)
tag = msg[2];
- TAG_PRINTK("scsi%d: target mask %02x, lun %d sent tag %d at "
+ dprintk(NDEBUG_TAGS, "scsi%d: target mask %02x, lun %d sent tag %d at "
"reselection\n", HOSTNO, target_mask, lun, tag);
}
#endif
@@ -2598,7 +2588,7 @@
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
hostdata->connected = tmp;
- RSL_PRINTK("scsi%d: nexus established, target = %d, lun = %d, tag = %d\n",
+ dprintk(NDEBUG_RESELECTION, "scsi%d: nexus established, target = %d, lun = %d, tag = %d\n",
HOSTNO, tmp->device->id, tmp->device->lun, tmp->tag);
falcon_dont_release--;
}
@@ -2640,7 +2630,7 @@
printk(KERN_ERR "scsi%d: !!BINGO!! Falcon has no lock in NCR5380_abort\n",
HOSTNO);
- ABRT_PRINTK("scsi%d: abort called basr 0x%02x, sr 0x%02x\n", HOSTNO,
+ dprintk(NDEBUG_ABORT, "scsi%d: abort called basr 0x%02x, sr 0x%02x\n", HOSTNO,
NCR5380_read(BUS_AND_STATUS_REG),
NCR5380_read(STATUS_REG));
@@ -2653,7 +2643,7 @@
if (hostdata->connected == cmd) {
- ABRT_PRINTK("scsi%d: aborting connected command\n", HOSTNO);
+ dprintk(NDEBUG_ABORT, "scsi%d: aborting connected command\n", HOSTNO);
/*
* We should perform BSY checking, and make sure we haven't slipped
* into BUS FREE.
@@ -2683,11 +2673,11 @@
local_irq_restore(flags);
cmd->scsi_done(cmd);
falcon_release_lock_if_possible(hostdata);
- return SCSI_ABORT_SUCCESS;
+ return SUCCESS;
} else {
/* local_irq_restore(flags); */
printk("scsi%d: abort of connected command failed!\n", HOSTNO);
- return SCSI_ABORT_ERROR;
+ return FAILED;
}
}
#endif
@@ -2705,13 +2695,13 @@
SET_NEXT(tmp, NULL);
tmp->result = DID_ABORT << 16;
local_irq_restore(flags);
- ABRT_PRINTK("scsi%d: abort removed command from issue queue.\n",
+ dprintk(NDEBUG_ABORT, "scsi%d: abort removed command from issue queue.\n",
HOSTNO);
/* Tagged queuing note: no tag to free here, hasn't been assigned
* yet... */
tmp->scsi_done(tmp);
falcon_release_lock_if_possible(hostdata);
- return SCSI_ABORT_SUCCESS;
+ return SUCCESS;
}
}
@@ -2728,8 +2718,8 @@
if (hostdata->connected) {
local_irq_restore(flags);
- ABRT_PRINTK("scsi%d: abort failed, command connected.\n", HOSTNO);
- return SCSI_ABORT_SNOOZE;
+ dprintk(NDEBUG_ABORT, "scsi%d: abort failed, command connected.\n", HOSTNO);
+ return FAILED;
}
/*
@@ -2761,12 +2751,12 @@
tmp = NEXT(tmp)) {
if (cmd == tmp) {
local_irq_restore(flags);
- ABRT_PRINTK("scsi%d: aborting disconnected command.\n", HOSTNO);
+ dprintk(NDEBUG_ABORT, "scsi%d: aborting disconnected command.\n", HOSTNO);
if (NCR5380_select(instance, cmd, (int)cmd->tag))
- return SCSI_ABORT_BUSY;
+ return FAILED;
- ABRT_PRINTK("scsi%d: nexus reestablished.\n", HOSTNO);
+ dprintk(NDEBUG_ABORT, "scsi%d: nexus reestablished.\n", HOSTNO);
do_abort(instance);
@@ -2791,7 +2781,7 @@
local_irq_restore(flags);
tmp->scsi_done(tmp);
falcon_release_lock_if_possible(hostdata);
- return SCSI_ABORT_SUCCESS;
+ return SUCCESS;
}
}
}
@@ -2816,7 +2806,7 @@
*/
falcon_release_lock_if_possible(hostdata);
- return SCSI_ABORT_NOT_RUNNING;
+ return FAILED;
}
@@ -2825,7 +2815,7 @@
*
* Purpose : reset the SCSI bus.
*
- * Returns : SCSI_RESET_WAKEUP
+ * Returns : SUCCESS or FAILURE
*
*/
@@ -2834,7 +2824,7 @@
SETUP_HOSTDATA(cmd->device->host);
int i;
unsigned long flags;
-#if 1
+#if defined(RESET_RUN_DONE)
Scsi_Cmnd *connected, *disconnected_queue;
#endif
@@ -2859,7 +2849,14 @@
* through anymore ... */
(void)NCR5380_read(RESET_PARITY_INTERRUPT_REG);
-#if 1 /* XXX Should now be done by midlevel code, but it's broken XXX */
+ /* MSch 20140115 - looking at the generic NCR5380 driver, all of this
+ * should go.
+ * Catch-22: if we don't clear all queues, the SCSI driver lock will
+ * not be reset by atari_scsi_reset()!
+ */
+
+#if defined(RESET_RUN_DONE)
+ /* XXX Should now be done by midlevel code, but it's broken XXX */
/* XXX see below XXX */
/* MSch: old-style reset: actually abort all command processing here */
@@ -2890,7 +2887,7 @@
*/
if ((cmd = connected)) {
- ABRT_PRINTK("scsi%d: reset aborted a connected command\n", H_NO(cmd));
+ dprintk(NDEBUG_ABORT, "scsi%d: reset aborted a connected command\n", H_NO(cmd));
cmd->result = (cmd->result & 0xffff) | (DID_RESET << 16);
cmd->scsi_done(cmd);
}
@@ -2902,7 +2899,7 @@
cmd->scsi_done(cmd);
}
if (i > 0)
- ABRT_PRINTK("scsi: reset aborted %d disconnected command(s)\n", i);
+ dprintk(NDEBUG_ABORT, "scsi: reset aborted %d disconnected command(s)\n", i);
/* The Falcon lock should be released after a reset...
*/
@@ -2915,7 +2912,7 @@
* the midlevel code that the reset was SUCCESSFUL, and there is no
* need to 'wake up' the commands by a request_sense
*/
- return SCSI_RESET_SUCCESS | SCSI_RESET_BUS_RESET;
+ return SUCCESS;
#else /* 1 */
/* MSch: new-style reset handling: let the mid-level do what it can */
@@ -2942,11 +2939,11 @@
*/
if (hostdata->issue_queue)
- ABRT_PRINTK("scsi%d: reset aborted issued command(s)\n", H_NO(cmd));
+ dprintk(NDEBUG_ABORT, "scsi%d: reset aborted issued command(s)\n", H_NO(cmd));
if (hostdata->connected)
- ABRT_PRINTK("scsi%d: reset aborted a connected command\n", H_NO(cmd));
+ dprintk(NDEBUG_ABORT, "scsi%d: reset aborted a connected command\n", H_NO(cmd));
if (hostdata->disconnected_queue)
- ABRT_PRINTK("scsi%d: reset aborted disconnected command(s)\n", H_NO(cmd));
+ dprintk(NDEBUG_ABORT, "scsi%d: reset aborted disconnected command(s)\n", H_NO(cmd));
local_irq_save(flags);
hostdata->issue_queue = NULL;
@@ -2963,6 +2960,6 @@
local_irq_restore(flags);
/* we did no complete reset of all commands, so a wakeup is required */
- return SCSI_RESET_WAKEUP | SCSI_RESET_BUS_RESET;
+ return SUCCESS;
#endif /* 1 */
}
diff --git a/drivers/scsi/atari_scsi.c b/drivers/scsi/atari_scsi.c
index a8d721f..b522134 100644
--- a/drivers/scsi/atari_scsi.c
+++ b/drivers/scsi/atari_scsi.c
@@ -67,12 +67,6 @@
#include <linux/module.h>
-#define NDEBUG (0)
-
-#define NDEBUG_ABORT 0x00100000
-#define NDEBUG_TAGS 0x00200000
-#define NDEBUG_MERGING 0x00400000
-
#define AUTOSENSE
/* For the Atari version, use only polled IO or REAL_DMA */
#define REAL_DMA
@@ -314,7 +308,7 @@
dma_stat = tt_scsi_dma.dma_ctrl;
- INT_PRINTK("scsi%d: NCR5380 interrupt, DMA status = %02x\n",
+ dprintk(NDEBUG_INTR, "scsi%d: NCR5380 interrupt, DMA status = %02x\n",
atari_scsi_host->host_no, dma_stat & 0xff);
/* Look if it was the DMA that has interrupted: First possibility
@@ -340,7 +334,7 @@
if ((dma_stat & 0x02) && !(dma_stat & 0x40)) {
atari_dma_residual = HOSTDATA_DMALEN - (SCSI_DMA_READ_P(dma_addr) - atari_dma_startaddr);
- DMA_PRINTK("SCSI DMA: There are %ld residual bytes.\n",
+ dprintk(NDEBUG_DMA, "SCSI DMA: There are %ld residual bytes.\n",
atari_dma_residual);
if ((signed int)atari_dma_residual < 0)
@@ -371,7 +365,7 @@
* other command. These shouldn't disconnect anyway.
*/
if (atari_dma_residual & 0x1ff) {
- DMA_PRINTK("SCSI DMA: DMA bug corrected, "
+ dprintk(NDEBUG_DMA, "SCSI DMA: DMA bug corrected, "
"difference %ld bytes\n",
512 - (atari_dma_residual & 0x1ff));
atari_dma_residual = (atari_dma_residual + 511) & ~0x1ff;
@@ -438,7 +432,7 @@
"ST-DMA fifo\n", transferred & 15);
atari_dma_residual = HOSTDATA_DMALEN - transferred;
- DMA_PRINTK("SCSI DMA: There are %ld residual bytes.\n",
+ dprintk(NDEBUG_DMA, "SCSI DMA: There are %ld residual bytes.\n",
atari_dma_residual);
} else
atari_dma_residual = 0;
@@ -474,11 +468,11 @@
/* there are 'nr' bytes left for the last long address
before the DMA pointer */
phys_dst ^= nr;
- DMA_PRINTK("SCSI DMA: there are %d rest bytes for phys addr 0x%08lx",
+ dprintk(NDEBUG_DMA, "SCSI DMA: there are %d rest bytes for phys addr 0x%08lx",
nr, phys_dst);
/* The content of the DMA pointer is a physical address! */
dst = phys_to_virt(phys_dst);
- DMA_PRINTK(" = virt addr %p\n", dst);
+ dprintk(NDEBUG_DMA, " = virt addr %p\n", dst);
for (src = (char *)&tt_scsi_dma.dma_restdata; nr != 0; --nr)
*dst++ = *src++;
}
@@ -827,7 +821,7 @@
} else {
atari_turnon_irq(IRQ_MFP_FSCSI);
}
- if ((rv & SCSI_RESET_ACTION) == SCSI_RESET_SUCCESS)
+ if (rv == SUCCESS)
falcon_release_lock_if_possible(hostdata);
return rv;
@@ -883,7 +877,7 @@
{
unsigned long addr = virt_to_phys(data);
- DMA_PRINTK("scsi%d: setting up dma, data = %p, phys = %lx, count = %ld, "
+ dprintk(NDEBUG_DMA, "scsi%d: setting up dma, data = %p, phys = %lx, count = %ld, "
"dir = %d\n", instance->host_no, data, addr, count, dir);
if (!IS_A_TT() && !STRAM_ADDR(addr)) {
@@ -1063,7 +1057,7 @@
possible_len = limit;
if (possible_len != wanted_len)
- DMA_PRINTK("Sorry, must cut DMA transfer size to %ld bytes "
+ dprintk(NDEBUG_DMA, "Sorry, must cut DMA transfer size to %ld bytes "
"instead of %ld\n", possible_len, wanted_len);
return possible_len;
diff --git a/drivers/scsi/atari_scsi.h b/drivers/scsi/atari_scsi.h
index 11c624b..3299d91 100644
--- a/drivers/scsi/atari_scsi.h
+++ b/drivers/scsi/atari_scsi.h
@@ -54,125 +54,6 @@
#define NCR5380_dma_xfer_len(i,cmd,phase) \
atari_dma_xfer_len(cmd->SCp.this_residual,cmd,((phase) & SR_IO) ? 0 : 1)
-/* former generic SCSI error handling stuff */
-
-#define SCSI_ABORT_SNOOZE 0
-#define SCSI_ABORT_SUCCESS 1
-#define SCSI_ABORT_PENDING 2
-#define SCSI_ABORT_BUSY 3
-#define SCSI_ABORT_NOT_RUNNING 4
-#define SCSI_ABORT_ERROR 5
-
-#define SCSI_RESET_SNOOZE 0
-#define SCSI_RESET_PUNT 1
-#define SCSI_RESET_SUCCESS 2
-#define SCSI_RESET_PENDING 3
-#define SCSI_RESET_WAKEUP 4
-#define SCSI_RESET_NOT_RUNNING 5
-#define SCSI_RESET_ERROR 6
-
-#define SCSI_RESET_SYNCHRONOUS 0x01
-#define SCSI_RESET_ASYNCHRONOUS 0x02
-#define SCSI_RESET_SUGGEST_BUS_RESET 0x04
-#define SCSI_RESET_SUGGEST_HOST_RESET 0x08
-
-#define SCSI_RESET_BUS_RESET 0x100
-#define SCSI_RESET_HOST_RESET 0x200
-#define SCSI_RESET_ACTION 0xff
-
-/* Debugging printk definitions:
- *
- * ARB -> arbitration
- * ASEN -> auto-sense
- * DMA -> DMA
- * HSH -> PIO handshake
- * INF -> information transfer
- * INI -> initialization
- * INT -> interrupt
- * LNK -> linked commands
- * MAIN -> NCR5380_main() control flow
- * NDAT -> no data-out phase
- * NWR -> no write commands
- * PIO -> PIO transfers
- * PDMA -> pseudo DMA (unused on Atari)
- * QU -> queues
- * RSL -> reselections
- * SEL -> selections
- * USL -> usleep cpde (unused on Atari)
- * LBS -> last byte sent (unused on Atari)
- * RSS -> restarting of selections
- * EXT -> extended messages
- * ABRT -> aborting and resetting
- * TAG -> queue tag handling
- * MER -> merging of consec. buffers
- *
- */
-
-#define dprint(flg, format...) \
-({ \
- if (NDEBUG & (flg)) \
- printk(KERN_DEBUG format); \
-})
-
-#define ARB_PRINTK(format, args...) \
- dprint(NDEBUG_ARBITRATION, format , ## args)
-#define ASEN_PRINTK(format, args...) \
- dprint(NDEBUG_AUTOSENSE, format , ## args)
-#define DMA_PRINTK(format, args...) \
- dprint(NDEBUG_DMA, format , ## args)
-#define HSH_PRINTK(format, args...) \
- dprint(NDEBUG_HANDSHAKE, format , ## args)
-#define INF_PRINTK(format, args...) \
- dprint(NDEBUG_INFORMATION, format , ## args)
-#define INI_PRINTK(format, args...) \
- dprint(NDEBUG_INIT, format , ## args)
-#define INT_PRINTK(format, args...) \
- dprint(NDEBUG_INTR, format , ## args)
-#define LNK_PRINTK(format, args...) \
- dprint(NDEBUG_LINKED, format , ## args)
-#define MAIN_PRINTK(format, args...) \
- dprint(NDEBUG_MAIN, format , ## args)
-#define NDAT_PRINTK(format, args...) \
- dprint(NDEBUG_NO_DATAOUT, format , ## args)
-#define NWR_PRINTK(format, args...) \
- dprint(NDEBUG_NO_WRITE, format , ## args)
-#define PIO_PRINTK(format, args...) \
- dprint(NDEBUG_PIO, format , ## args)
-#define PDMA_PRINTK(format, args...) \
- dprint(NDEBUG_PSEUDO_DMA, format , ## args)
-#define QU_PRINTK(format, args...) \
- dprint(NDEBUG_QUEUES, format , ## args)
-#define RSL_PRINTK(format, args...) \
- dprint(NDEBUG_RESELECTION, format , ## args)
-#define SEL_PRINTK(format, args...) \
- dprint(NDEBUG_SELECTION, format , ## args)
-#define USL_PRINTK(format, args...) \
- dprint(NDEBUG_USLEEP, format , ## args)
-#define LBS_PRINTK(format, args...) \
- dprint(NDEBUG_LAST_BYTE_SENT, format , ## args)
-#define RSS_PRINTK(format, args...) \
- dprint(NDEBUG_RESTART_SELECT, format , ## args)
-#define EXT_PRINTK(format, args...) \
- dprint(NDEBUG_EXTENDED, format , ## args)
-#define ABRT_PRINTK(format, args...) \
- dprint(NDEBUG_ABORT, format , ## args)
-#define TAG_PRINTK(format, args...) \
- dprint(NDEBUG_TAGS, format , ## args)
-#define MER_PRINTK(format, args...) \
- dprint(NDEBUG_MERGING, format , ## args)
-
-/* conditional macros for NCR5380_print_{,phase,status} */
-
-#define NCR_PRINT(mask) \
- ((NDEBUG & (mask)) ? NCR5380_print(instance) : (void)0)
-
-#define NCR_PRINT_PHASE(mask) \
- ((NDEBUG & (mask)) ? NCR5380_print_phase(instance) : (void)0)
-
-#define NCR_PRINT_STATUS(mask) \
- ((NDEBUG & (mask)) ? NCR5380_print_status(instance) : (void)0)
-
-
#endif /* ndef ASM */
#endif /* ATARI_SCSI_H */
diff --git a/drivers/scsi/be2iscsi/be.h b/drivers/scsi/be2iscsi/be.h
index 1bfb0bd..860f527 100644
--- a/drivers/scsi/be2iscsi/be.h
+++ b/drivers/scsi/be2iscsi/be.h
@@ -83,9 +83,20 @@
/*ISCSI */
+struct be_aic_obj { /* Adaptive interrupt coalescing (AIC) info */
+ bool enable;
+ u32 min_eqd; /* in usecs */
+ u32 max_eqd; /* in usecs */
+ u32 prev_eqd; /* in usecs */
+ u32 et_eqd; /* configured val when aic is off */
+ ulong jiffs;
+ u64 eq_prev; /* Used to calculate eqe */
+};
+
struct be_eq_obj {
bool todo_mcc_cq;
bool todo_cq;
+ u32 cq_count;
struct be_queue_info q;
struct beiscsi_hba *phba;
struct be_queue_info *cq;
diff --git a/drivers/scsi/be2iscsi/be_cmds.h b/drivers/scsi/be2iscsi/be_cmds.h
index 7cf7f99..cc7405c 100644
--- a/drivers/scsi/be2iscsi/be_cmds.h
+++ b/drivers/scsi/be2iscsi/be_cmds.h
@@ -71,6 +71,7 @@
#define BEISCSI_FW_MBX_TIMEOUT 100
/* MBOX Command VER */
+#define MBX_CMD_VER1 0x01
#define MBX_CMD_VER2 0x02
struct be_mcc_compl {
@@ -271,6 +272,12 @@
u16 rsvd0; /* sword */
} __packed;
+struct be_set_eqd {
+ u32 eq_id;
+ u32 phase;
+ u32 delay_multiplier;
+} __packed;
+
struct mgmt_chap_format {
u32 flags;
u8 intr_chap_name[256];
@@ -622,7 +629,7 @@
u32 eq_id;
u32 phase;
u32 delay_multiplier;
- } delay[8];
+ } delay[MAX_CPUS];
} __packed;
/******************** Get MAC ADDR *******************/
@@ -708,6 +715,8 @@
void free_mcc_tag(struct be_ctrl_info *ctrl, unsigned int tag);
+int be_cmd_modify_eq_delay(struct beiscsi_hba *phba, struct be_set_eqd *,
+ int num);
int beiscsi_mccq_compl(struct beiscsi_hba *phba,
uint32_t tag, struct be_mcc_wrb **wrb,
struct be_dma_mem *mbx_cmd_mem);
@@ -1005,6 +1014,26 @@
u8 rsvd0[3];
} __packed;
+struct tcp_connect_and_offload_in_v1 {
+ struct be_cmd_req_hdr hdr;
+ struct ip_addr_format ip_address;
+ u16 tcp_port;
+ u16 cid;
+ u16 cq_id;
+ u16 defq_id;
+ struct phys_addr dataout_template_pa;
+ u16 hdr_ring_id;
+ u16 data_ring_id;
+ u8 do_offload;
+ u8 ifd_state;
+ u8 rsvd0[2];
+ u16 tcp_window_size;
+ u8 tcp_window_scale_count;
+ u8 rsvd1;
+ u32 tcp_mss:24;
+ u8 rsvd2;
+} __packed;
+
struct tcp_connect_and_offload_out {
struct be_cmd_resp_hdr hdr;
u32 connection_handle;
diff --git a/drivers/scsi/be2iscsi/be_iscsi.c b/drivers/scsi/be2iscsi/be_iscsi.c
index a3df433..fd284ff 100644
--- a/drivers/scsi/be2iscsi/be_iscsi.c
+++ b/drivers/scsi/be2iscsi/be_iscsi.c
@@ -1106,7 +1106,7 @@
struct beiscsi_hba *phba = beiscsi_ep->phba;
struct tcp_connect_and_offload_out *ptcpcnct_out;
struct be_dma_mem nonemb_cmd;
- unsigned int tag;
+ unsigned int tag, req_memsize;
int ret = -ENOMEM;
beiscsi_log(phba, KERN_INFO, BEISCSI_LOG_CONFIG,
@@ -1127,8 +1127,14 @@
(beiscsi_ep->ep_cid)] = ep;
beiscsi_ep->cid_vld = 0;
+
+ if (is_chip_be2_be3r(phba))
+ req_memsize = sizeof(struct tcp_connect_and_offload_in);
+ else
+ req_memsize = sizeof(struct tcp_connect_and_offload_in_v1);
+
nonemb_cmd.va = pci_alloc_consistent(phba->ctrl.pdev,
- sizeof(struct tcp_connect_and_offload_in),
+ req_memsize,
&nonemb_cmd.dma);
if (nonemb_cmd.va == NULL) {
@@ -1139,7 +1145,7 @@
beiscsi_free_ep(beiscsi_ep);
return -ENOMEM;
}
- nonemb_cmd.size = sizeof(struct tcp_connect_and_offload_in);
+ nonemb_cmd.size = req_memsize;
memset(nonemb_cmd.va, 0, nonemb_cmd.size);
tag = mgmt_open_connection(phba, dst_addr, beiscsi_ep, &nonemb_cmd);
if (tag <= 0) {
diff --git a/drivers/scsi/be2iscsi/be_main.c b/drivers/scsi/be2iscsi/be_main.c
index 0d82229..5543490 100644
--- a/drivers/scsi/be2iscsi/be_main.c
+++ b/drivers/scsi/be2iscsi/be_main.c
@@ -599,15 +599,7 @@
pci_set_drvdata(pcidev, phba);
phba->interface_handle = 0xFFFFFFFF;
- if (iscsi_host_add(shost, &phba->pcidev->dev))
- goto free_devices;
-
return phba;
-
-free_devices:
- pci_dev_put(phba->pcidev);
- iscsi_host_free(phba->shost);
- return NULL;
}
static void beiscsi_unmap_pci_function(struct beiscsi_hba *phba)
@@ -2279,6 +2271,7 @@
pbe_eq = container_of(iop, struct be_eq_obj, iopoll);
ret = beiscsi_process_cq(pbe_eq);
+ pbe_eq->cq_count += ret;
if (ret < budget) {
phba = pbe_eq->phba;
blk_iopoll_complete(iop);
@@ -3692,7 +3685,7 @@
struct hwi_controller *phwi_ctrlr;
struct hwi_context_memory *phwi_context;
struct hwi_async_pdu_context *pasync_ctx;
- int i, eq_num, ulp_num;
+ int i, eq_for_mcc, ulp_num;
phwi_ctrlr = phba->phwi_ctrlr;
phwi_context = phwi_ctrlr->phwi_ctxt;
@@ -3729,16 +3722,17 @@
if (q->created)
beiscsi_cmd_q_destroy(ctrl, q, QTYPE_CQ);
}
+
+ be_mcc_queues_destroy(phba);
if (phba->msix_enabled)
- eq_num = 1;
+ eq_for_mcc = 1;
else
- eq_num = 0;
- for (i = 0; i < (phba->num_cpus + eq_num); i++) {
+ eq_for_mcc = 0;
+ for (i = 0; i < (phba->num_cpus + eq_for_mcc); i++) {
q = &phwi_context->be_eq[i].q;
if (q->created)
beiscsi_cmd_q_destroy(ctrl, q, QTYPE_EQ);
}
- be_mcc_queues_destroy(phba);
be_cmd_fw_uninit(ctrl);
}
@@ -3833,9 +3827,9 @@
phwi_ctrlr = phba->phwi_ctrlr;
phwi_context = phwi_ctrlr->phwi_ctxt;
- phwi_context->max_eqd = 0;
+ phwi_context->max_eqd = 128;
phwi_context->min_eqd = 0;
- phwi_context->cur_eqd = 64;
+ phwi_context->cur_eqd = 0;
be_cmd_fw_initialize(&phba->ctrl);
status = beiscsi_create_eqs(phba, phwi_context);
@@ -5290,6 +5284,57 @@
return;
}
+static void be_eqd_update(struct beiscsi_hba *phba)
+{
+ struct be_set_eqd set_eqd[MAX_CPUS];
+ struct be_aic_obj *aic;
+ struct be_eq_obj *pbe_eq;
+ struct hwi_controller *phwi_ctrlr;
+ struct hwi_context_memory *phwi_context;
+ int eqd, i, num = 0;
+ ulong now;
+ u32 pps, delta;
+ unsigned int tag;
+
+ phwi_ctrlr = phba->phwi_ctrlr;
+ phwi_context = phwi_ctrlr->phwi_ctxt;
+
+ for (i = 0; i <= phba->num_cpus; i++) {
+ aic = &phba->aic_obj[i];
+ pbe_eq = &phwi_context->be_eq[i];
+ now = jiffies;
+ if (!aic->jiffs || time_before(now, aic->jiffs) ||
+ pbe_eq->cq_count < aic->eq_prev) {
+ aic->jiffs = now;
+ aic->eq_prev = pbe_eq->cq_count;
+ continue;
+ }
+ delta = jiffies_to_msecs(now - aic->jiffs);
+ pps = (((u32)(pbe_eq->cq_count - aic->eq_prev) * 1000) / delta);
+ eqd = (pps / 1500) << 2;
+
+ if (eqd < 8)
+ eqd = 0;
+ eqd = min_t(u32, eqd, phwi_context->max_eqd);
+ eqd = max_t(u32, eqd, phwi_context->min_eqd);
+
+ aic->jiffs = now;
+ aic->eq_prev = pbe_eq->cq_count;
+
+ if (eqd != aic->prev_eqd) {
+ set_eqd[num].delay_multiplier = (eqd * 65)/100;
+ set_eqd[num].eq_id = pbe_eq->q.id;
+ aic->prev_eqd = eqd;
+ num++;
+ }
+ }
+ if (num) {
+ tag = be_cmd_modify_eq_delay(phba, set_eqd, num);
+ if (tag)
+ beiscsi_mccq_compl(phba, tag, NULL, NULL);
+ }
+}
+
/*
* beiscsi_hw_health_check()- Check adapter health
* @work: work item to check HW health
@@ -5303,6 +5348,8 @@
container_of(work, struct beiscsi_hba,
beiscsi_hw_check_task.work);
+ be_eqd_update(phba);
+
beiscsi_ue_detect(phba);
schedule_delayed_work(&phba->beiscsi_hw_check_task,
@@ -5579,7 +5626,7 @@
phba->ctrl.mcc_numtag[i + 1] = 0;
phba->ctrl.mcc_tag_available++;
memset(&phba->ctrl.ptag_state[i].tag_mem_state, 0,
- sizeof(struct beiscsi_mcc_tag_state));
+ sizeof(struct be_dma_mem));
}
phba->ctrl.mcc_alloc_index = phba->ctrl.mcc_free_index = 0;
@@ -5621,6 +5668,9 @@
}
hwi_enable_intr(phba);
+ if (iscsi_host_add(phba->shost, &phba->pcidev->dev))
+ goto free_blkenbld;
+
if (beiscsi_setup_boot_info(phba))
/*
* log error but continue, because we may not be using
diff --git a/drivers/scsi/be2iscsi/be_main.h b/drivers/scsi/be2iscsi/be_main.h
index 9380b55b..9ceab42 100644
--- a/drivers/scsi/be2iscsi/be_main.h
+++ b/drivers/scsi/be2iscsi/be_main.h
@@ -36,7 +36,7 @@
#include <scsi/scsi_transport_iscsi.h>
#define DRV_NAME "be2iscsi"
-#define BUILD_STR "10.2.125.0"
+#define BUILD_STR "10.2.273.0"
#define BE_NAME "Emulex OneConnect" \
"Open-iSCSI Driver version" BUILD_STR
#define DRV_DESC BE_NAME " " "Driver"
@@ -71,8 +71,8 @@
#define BEISCSI_SGLIST_ELEMENTS 30
-#define BEISCSI_CMD_PER_LUN 128 /* scsi_host->cmd_per_lun */
-#define BEISCSI_MAX_SECTORS 2048 /* scsi_host->max_sectors */
+#define BEISCSI_CMD_PER_LUN 128 /* scsi_host->cmd_per_lun */
+#define BEISCSI_MAX_SECTORS 1024 /* scsi_host->max_sectors */
#define BEISCSI_TEMPLATE_HDR_PER_CXN_SIZE 128 /* Template size per cxn */
#define BEISCSI_MAX_CMD_LEN 16 /* scsi_host->max_cmd_len */
@@ -427,6 +427,7 @@
struct mgmt_session_info boot_sess;
struct invalidate_command_table inv_tbl[128];
+ struct be_aic_obj aic_obj[MAX_CPUS];
unsigned int attr_log_enable;
int (*iotask_fn)(struct iscsi_task *,
struct scatterlist *sg,
diff --git a/drivers/scsi/be2iscsi/be_mgmt.c b/drivers/scsi/be2iscsi/be_mgmt.c
index 088bdf7..6045aa7 100644
--- a/drivers/scsi/be2iscsi/be_mgmt.c
+++ b/drivers/scsi/be2iscsi/be_mgmt.c
@@ -155,6 +155,43 @@
}
}
+int be_cmd_modify_eq_delay(struct beiscsi_hba *phba,
+ struct be_set_eqd *set_eqd, int num)
+{
+ struct be_ctrl_info *ctrl = &phba->ctrl;
+ struct be_mcc_wrb *wrb;
+ struct be_cmd_req_modify_eq_delay *req;
+ unsigned int tag = 0;
+ int i;
+
+ spin_lock(&ctrl->mbox_lock);
+ tag = alloc_mcc_tag(phba);
+ if (!tag) {
+ spin_unlock(&ctrl->mbox_lock);
+ return tag;
+ }
+
+ wrb = wrb_from_mccq(phba);
+ req = embedded_payload(wrb);
+
+ wrb->tag0 |= tag;
+ be_wrb_hdr_prepare(wrb, sizeof(*req), true, 0);
+ be_cmd_hdr_prepare(&req->hdr, CMD_SUBSYSTEM_COMMON,
+ OPCODE_COMMON_MODIFY_EQ_DELAY, sizeof(*req));
+
+ req->num_eq = cpu_to_le32(num);
+ for (i = 0; i < num; i++) {
+ req->delay[i].eq_id = cpu_to_le32(set_eqd[i].eq_id);
+ req->delay[i].phase = 0;
+ req->delay[i].delay_multiplier =
+ cpu_to_le32(set_eqd[i].delay_multiplier);
+ }
+
+ be_mcc_notify(phba);
+ spin_unlock(&ctrl->mbox_lock);
+ return tag;
+}
+
/**
* mgmt_reopen_session()- Reopen a session based on reopen_type
* @phba: Device priv structure instance
@@ -447,8 +484,8 @@
struct be_dma_mem *nonemb_cmd)
{
struct be_cmd_resp_hdr *resp;
- struct be_mcc_wrb *wrb = wrb_from_mccq(phba);
- struct be_sge *mcc_sge = nonembedded_sgl(wrb);
+ struct be_mcc_wrb *wrb;
+ struct be_sge *mcc_sge;
unsigned int tag = 0;
struct iscsi_bsg_request *bsg_req = job->request;
struct be_bsg_vendor_cmd *req = nonemb_cmd->va;
@@ -465,7 +502,6 @@
req->sector = sector;
req->offset = offset;
spin_lock(&ctrl->mbox_lock);
- memset(wrb, 0, sizeof(*wrb));
switch (bsg_req->rqst_data.h_vendor.vendor_cmd[0]) {
case BEISCSI_WRITE_FLASH:
@@ -495,6 +531,8 @@
return tag;
}
+ wrb = wrb_from_mccq(phba);
+ mcc_sge = nonembedded_sgl(wrb);
be_wrb_hdr_prepare(wrb, nonemb_cmd->size, false,
job->request_payload.sg_cnt);
mcc_sge->pa_hi = cpu_to_le32(upper_32_bits(nonemb_cmd->dma));
@@ -525,7 +563,6 @@
int status = 0;
spin_lock(&ctrl->mbox_lock);
- memset(wrb, 0, sizeof(*wrb));
be_wrb_hdr_prepare(wrb, sizeof(*req), true, 0);
be_cmd_hdr_prepare(&req->hdr, CMD_SUBSYSTEM_ISCSI,
@@ -675,7 +712,7 @@
struct sockaddr_in6 *daddr_in6 = (struct sockaddr_in6 *)dst_addr;
struct be_ctrl_info *ctrl = &phba->ctrl;
struct be_mcc_wrb *wrb;
- struct tcp_connect_and_offload_in *req;
+ struct tcp_connect_and_offload_in_v1 *req;
unsigned short def_hdr_id;
unsigned short def_data_id;
struct phys_addr template_address = { 0, 0 };
@@ -702,17 +739,16 @@
return tag;
}
wrb = wrb_from_mccq(phba);
- memset(wrb, 0, sizeof(*wrb));
sge = nonembedded_sgl(wrb);
req = nonemb_cmd->va;
memset(req, 0, sizeof(*req));
wrb->tag0 |= tag;
- be_wrb_hdr_prepare(wrb, sizeof(*req), false, 1);
+ be_wrb_hdr_prepare(wrb, nonemb_cmd->size, false, 1);
be_cmd_hdr_prepare(&req->hdr, CMD_SUBSYSTEM_ISCSI,
OPCODE_COMMON_ISCSI_TCP_CONNECT_AND_OFFLOAD,
- sizeof(*req));
+ nonemb_cmd->size);
if (dst_addr->sa_family == PF_INET) {
__be32 s_addr = daddr_in->sin_addr.s_addr;
req->ip_address.ip_type = BE2_IPV4;
@@ -758,6 +794,13 @@
sge->pa_hi = cpu_to_le32(upper_32_bits(nonemb_cmd->dma));
sge->pa_lo = cpu_to_le32(nonemb_cmd->dma & 0xFFFFFFFF);
sge->len = cpu_to_le32(nonemb_cmd->size);
+
+ if (!is_chip_be2_be3r(phba)) {
+ req->hdr.version = MBX_CMD_VER1;
+ req->tcp_window_size = 0;
+ req->tcp_window_scale_count = 2;
+ }
+
be_mcc_notify(phba);
spin_unlock(&ctrl->mbox_lock);
return tag;
@@ -804,7 +847,7 @@
int resp_buf_len)
{
struct be_ctrl_info *ctrl = &phba->ctrl;
- struct be_mcc_wrb *wrb = wrb_from_mccq(phba);
+ struct be_mcc_wrb *wrb;
struct be_sge *sge;
unsigned int tag;
int rc = 0;
@@ -816,7 +859,8 @@
rc = -ENOMEM;
goto free_cmd;
}
- memset(wrb, 0, sizeof(*wrb));
+
+ wrb = wrb_from_mccq(phba);
wrb->tag0 |= tag;
sge = nonembedded_sgl(wrb);
diff --git a/drivers/scsi/be2iscsi/be_mgmt.h b/drivers/scsi/be2iscsi/be_mgmt.h
index 01b8c97..24a8fc5 100644
--- a/drivers/scsi/be2iscsi/be_mgmt.h
+++ b/drivers/scsi/be2iscsi/be_mgmt.h
@@ -335,5 +335,7 @@
void beiscsi_offload_cxn_v2(struct beiscsi_offload_params *params,
struct wrb_handle *pwrb_handle);
void beiscsi_ue_detect(struct beiscsi_hba *phba);
+int be_cmd_modify_eq_delay(struct beiscsi_hba *phba,
+ struct be_set_eqd *, int num);
#endif
diff --git a/drivers/scsi/bfa/bfad.c b/drivers/scsi/bfa/bfad.c
index cc0fbcd..7593b7c 100644
--- a/drivers/scsi/bfa/bfad.c
+++ b/drivers/scsi/bfa/bfad.c
@@ -507,7 +507,7 @@
struct bfad_vport_s *vport;
int rc;
- vport = kzalloc(sizeof(struct bfad_vport_s), GFP_KERNEL);
+ vport = kzalloc(sizeof(struct bfad_vport_s), GFP_ATOMIC);
if (!vport) {
bfa_trc(bfad, 0);
return;
diff --git a/drivers/scsi/bnx2fc/bnx2fc_hwi.c b/drivers/scsi/bnx2fc/bnx2fc_hwi.c
index 46a3765..512aed3 100644
--- a/drivers/scsi/bnx2fc/bnx2fc_hwi.c
+++ b/drivers/scsi/bnx2fc/bnx2fc_hwi.c
@@ -1966,26 +1966,29 @@
{
int i;
int segment_count;
- int hash_table_size;
u32 *pbl;
- segment_count = hba->hash_tbl_segment_count;
- hash_table_size = BNX2FC_NUM_MAX_SESS * BNX2FC_MAX_ROWS_IN_HASH_TBL *
- sizeof(struct fcoe_hash_table_entry);
+ if (hba->hash_tbl_segments) {
- pbl = hba->hash_tbl_pbl;
- for (i = 0; i < segment_count; ++i) {
- dma_addr_t dma_address;
+ pbl = hba->hash_tbl_pbl;
+ if (pbl) {
+ segment_count = hba->hash_tbl_segment_count;
+ for (i = 0; i < segment_count; ++i) {
+ dma_addr_t dma_address;
- dma_address = le32_to_cpu(*pbl);
- ++pbl;
- dma_address += ((u64)le32_to_cpu(*pbl)) << 32;
- ++pbl;
- dma_free_coherent(&hba->pcidev->dev,
- BNX2FC_HASH_TBL_CHUNK_SIZE,
- hba->hash_tbl_segments[i],
- dma_address);
+ dma_address = le32_to_cpu(*pbl);
+ ++pbl;
+ dma_address += ((u64)le32_to_cpu(*pbl)) << 32;
+ ++pbl;
+ dma_free_coherent(&hba->pcidev->dev,
+ BNX2FC_HASH_TBL_CHUNK_SIZE,
+ hba->hash_tbl_segments[i],
+ dma_address);
+ }
+ }
+ kfree(hba->hash_tbl_segments);
+ hba->hash_tbl_segments = NULL;
}
if (hba->hash_tbl_pbl) {
@@ -2023,7 +2026,7 @@
dma_segment_array = kzalloc(dma_segment_array_size, GFP_KERNEL);
if (!dma_segment_array) {
printk(KERN_ERR PFX "hash table pointers (dma) alloc failed\n");
- return -ENOMEM;
+ goto cleanup_ht;
}
for (i = 0; i < segment_count; ++i) {
@@ -2034,15 +2037,7 @@
GFP_KERNEL);
if (!hba->hash_tbl_segments[i]) {
printk(KERN_ERR PFX "hash segment alloc failed\n");
- while (--i >= 0) {
- dma_free_coherent(&hba->pcidev->dev,
- BNX2FC_HASH_TBL_CHUNK_SIZE,
- hba->hash_tbl_segments[i],
- dma_segment_array[i]);
- hba->hash_tbl_segments[i] = NULL;
- }
- kfree(dma_segment_array);
- return -ENOMEM;
+ goto cleanup_dma;
}
memset(hba->hash_tbl_segments[i], 0,
BNX2FC_HASH_TBL_CHUNK_SIZE);
@@ -2054,8 +2049,7 @@
GFP_KERNEL);
if (!hba->hash_tbl_pbl) {
printk(KERN_ERR PFX "hash table pbl alloc failed\n");
- kfree(dma_segment_array);
- return -ENOMEM;
+ goto cleanup_dma;
}
memset(hba->hash_tbl_pbl, 0, PAGE_SIZE);
@@ -2080,6 +2074,22 @@
}
kfree(dma_segment_array);
return 0;
+
+cleanup_dma:
+ for (i = 0; i < segment_count; ++i) {
+ if (hba->hash_tbl_segments[i])
+ dma_free_coherent(&hba->pcidev->dev,
+ BNX2FC_HASH_TBL_CHUNK_SIZE,
+ hba->hash_tbl_segments[i],
+ dma_segment_array[i]);
+ }
+
+ kfree(dma_segment_array);
+
+cleanup_ht:
+ kfree(hba->hash_tbl_segments);
+ hba->hash_tbl_segments = NULL;
+ return -ENOMEM;
}
/**
diff --git a/drivers/scsi/dtc.c b/drivers/scsi/dtc.c
index eb29fe7..0a667fe 100644
--- a/drivers/scsi/dtc.c
+++ b/drivers/scsi/dtc.c
@@ -3,8 +3,6 @@
#define PSEUDO_DMA
#define DONT_USE_INTR
#define UNSAFE /* Leave interrupts enabled during pseudo-dma I/O */
-#define xNDEBUG (NDEBUG_INTR+NDEBUG_RESELECTION+\
- NDEBUG_SELECTION+NDEBUG_ARBITRATION)
#define DMA_WORKS_RIGHT
diff --git a/drivers/scsi/esas2r/esas2r_main.c b/drivers/scsi/esas2r/esas2r_main.c
index f37f3e3..6504a19 100644
--- a/drivers/scsi/esas2r/esas2r_main.c
+++ b/drivers/scsi/esas2r/esas2r_main.c
@@ -390,7 +390,7 @@
esas2r_log_dev(ESAS2R_LOG_INFO, &(pcid->dev),
"pci_enable_device() OK");
esas2r_log_dev(ESAS2R_LOG_INFO, &(pcid->dev),
- "after pci_device_enable() enable_cnt: %d",
+ "after pci_enable_device() enable_cnt: %d",
pcid->enable_cnt.counter);
host = scsi_host_alloc(&driver_template, host_alloc_size);
diff --git a/drivers/scsi/fnic/fnic.h b/drivers/scsi/fnic/fnic.h
index 528d43b..1d3521e 100644
--- a/drivers/scsi/fnic/fnic.h
+++ b/drivers/scsi/fnic/fnic.h
@@ -39,14 +39,15 @@
#define DRV_NAME "fnic"
#define DRV_DESCRIPTION "Cisco FCoE HBA Driver"
-#define DRV_VERSION "1.5.0.45"
+#define DRV_VERSION "1.6.0.10"
#define PFX DRV_NAME ": "
#define DFX DRV_NAME "%d: "
#define DESC_CLEAN_LOW_WATERMARK 8
#define FNIC_UCSM_DFLT_THROTTLE_CNT_BLD 16 /* UCSM default throttle count */
#define FNIC_MIN_IO_REQ 256 /* Min IO throttle count */
-#define FNIC_MAX_IO_REQ 2048 /* scsi_cmnd tag map entries */
+#define FNIC_MAX_IO_REQ 1024 /* scsi_cmnd tag map entries */
+#define FNIC_DFLT_IO_REQ 256 /* Default scsi_cmnd tag map entries */
#define FNIC_IO_LOCKS 64 /* IO locks: power of 2 */
#define FNIC_DFLT_QUEUE_DEPTH 32
#define FNIC_STATS_RATE_LIMIT 4 /* limit rate at which stats are pulled up */
diff --git a/drivers/scsi/fnic/fnic_debugfs.c b/drivers/scsi/fnic/fnic_debugfs.c
index b6073f8..2c613bd 100644
--- a/drivers/scsi/fnic/fnic_debugfs.c
+++ b/drivers/scsi/fnic/fnic_debugfs.c
@@ -25,6 +25,21 @@
static struct dentry *fnic_trace_enable;
static struct dentry *fnic_stats_debugfs_root;
+static struct dentry *fnic_fc_trace_debugfs_file;
+static struct dentry *fnic_fc_rdata_trace_debugfs_file;
+static struct dentry *fnic_fc_trace_enable;
+static struct dentry *fnic_fc_trace_clear;
+
+struct fc_trace_flag_type {
+ u8 fc_row_file;
+ u8 fc_normal_file;
+ u8 fnic_trace;
+ u8 fc_trace;
+ u8 fc_clear;
+};
+
+static struct fc_trace_flag_type *fc_trc_flag;
+
/*
* fnic_debugfs_init - Initialize debugfs for fnic debug logging
*
@@ -56,6 +71,18 @@
return rc;
}
+ /* Allocate memory to structure */
+ fc_trc_flag = (struct fc_trace_flag_type *)
+ vmalloc(sizeof(struct fc_trace_flag_type));
+
+ if (fc_trc_flag) {
+ fc_trc_flag->fc_row_file = 0;
+ fc_trc_flag->fc_normal_file = 1;
+ fc_trc_flag->fnic_trace = 2;
+ fc_trc_flag->fc_trace = 3;
+ fc_trc_flag->fc_clear = 4;
+ }
+
rc = 0;
return rc;
}
@@ -74,15 +101,19 @@
debugfs_remove(fnic_trace_debugfs_root);
fnic_trace_debugfs_root = NULL;
+
+ if (fc_trc_flag)
+ vfree(fc_trc_flag);
}
/*
- * fnic_trace_ctrl_open - Open the trace_enable file
+ * fnic_trace_ctrl_open - Open the trace_enable file for fnic_trace
+ * Or Open fc_trace_enable file for fc_trace
* @inode: The inode pointer.
* @file: The file pointer to attach the trace enable/disable flag.
*
* Description:
- * This routine opens a debugsfs file trace_enable.
+ * This routine opens a debugsfs file trace_enable or fc_trace_enable.
*
* Returns:
* This function returns zero if successful.
@@ -94,15 +125,19 @@
}
/*
- * fnic_trace_ctrl_read - Read a trace_enable debugfs file
+ * fnic_trace_ctrl_read -
+ * Read trace_enable ,fc_trace_enable
+ * or fc_trace_clear debugfs file
* @filp: The file pointer to read from.
* @ubuf: The buffer to copy the data to.
* @cnt: The number of bytes to read.
* @ppos: The position in the file to start reading from.
*
* Description:
- * This routine reads value of variable fnic_tracing_enabled
- * and stores into local @buf. It will start reading file at @ppos and
+ * This routine reads value of variable fnic_tracing_enabled or
+ * fnic_fc_tracing_enabled or fnic_fc_trace_cleared
+ * and stores into local @buf.
+ * It will start reading file at @ppos and
* copy up to @cnt of data to @ubuf from @buf.
*
* Returns:
@@ -114,13 +149,25 @@
{
char buf[64];
int len;
- len = sprintf(buf, "%u\n", fnic_tracing_enabled);
+ u8 *trace_type;
+ len = 0;
+ trace_type = (u8 *)filp->private_data;
+ if (*trace_type == fc_trc_flag->fnic_trace)
+ len = sprintf(buf, "%u\n", fnic_tracing_enabled);
+ else if (*trace_type == fc_trc_flag->fc_trace)
+ len = sprintf(buf, "%u\n", fnic_fc_tracing_enabled);
+ else if (*trace_type == fc_trc_flag->fc_clear)
+ len = sprintf(buf, "%u\n", fnic_fc_trace_cleared);
+ else
+ pr_err("fnic: Cannot read to any debugfs file\n");
return simple_read_from_buffer(ubuf, cnt, ppos, buf, len);
}
/*
- * fnic_trace_ctrl_write - Write to trace_enable debugfs file
+ * fnic_trace_ctrl_write -
+ * Write to trace_enable, fc_trace_enable or
+ * fc_trace_clear debugfs file
* @filp: The file pointer to write from.
* @ubuf: The buffer to copy the data from.
* @cnt: The number of bytes to write.
@@ -128,7 +175,8 @@
*
* Description:
* This routine writes data from user buffer @ubuf to buffer @buf and
- * sets fnic_tracing_enabled value as per user input.
+ * sets fc_trace_enable ,tracing_enable or fnic_fc_trace_cleared
+ * value as per user input.
*
* Returns:
* This function returns the amount of data that was written.
@@ -140,6 +188,8 @@
char buf[64];
unsigned long val;
int ret;
+ u8 *trace_type;
+ trace_type = (u8 *)filp->private_data;
if (cnt >= sizeof(buf))
return -EINVAL;
@@ -153,12 +203,27 @@
if (ret < 0)
return ret;
- fnic_tracing_enabled = val;
+ if (*trace_type == fc_trc_flag->fnic_trace)
+ fnic_tracing_enabled = val;
+ else if (*trace_type == fc_trc_flag->fc_trace)
+ fnic_fc_tracing_enabled = val;
+ else if (*trace_type == fc_trc_flag->fc_clear)
+ fnic_fc_trace_cleared = val;
+ else
+ pr_err("fnic: cannot write to any debufs file\n");
+
(*ppos)++;
return cnt;
}
+static const struct file_operations fnic_trace_ctrl_fops = {
+ .owner = THIS_MODULE,
+ .open = fnic_trace_ctrl_open,
+ .read = fnic_trace_ctrl_read,
+ .write = fnic_trace_ctrl_write,
+};
+
/*
* fnic_trace_debugfs_open - Open the fnic trace log
* @inode: The inode pointer
@@ -178,19 +243,36 @@
struct file *file)
{
fnic_dbgfs_t *fnic_dbg_prt;
+ u8 *rdata_ptr;
+ rdata_ptr = (u8 *)inode->i_private;
fnic_dbg_prt = kzalloc(sizeof(fnic_dbgfs_t), GFP_KERNEL);
if (!fnic_dbg_prt)
return -ENOMEM;
- fnic_dbg_prt->buffer = vmalloc((3*(trace_max_pages * PAGE_SIZE)));
- if (!fnic_dbg_prt->buffer) {
- kfree(fnic_dbg_prt);
- return -ENOMEM;
+ if (*rdata_ptr == fc_trc_flag->fnic_trace) {
+ fnic_dbg_prt->buffer = vmalloc(3 *
+ (trace_max_pages * PAGE_SIZE));
+ if (!fnic_dbg_prt->buffer) {
+ kfree(fnic_dbg_prt);
+ return -ENOMEM;
+ }
+ memset((void *)fnic_dbg_prt->buffer, 0,
+ 3 * (trace_max_pages * PAGE_SIZE));
+ fnic_dbg_prt->buffer_len = fnic_get_trace_data(fnic_dbg_prt);
+ } else {
+ fnic_dbg_prt->buffer =
+ vmalloc(3 * (fnic_fc_trace_max_pages * PAGE_SIZE));
+ if (!fnic_dbg_prt->buffer) {
+ kfree(fnic_dbg_prt);
+ return -ENOMEM;
+ }
+ memset((void *)fnic_dbg_prt->buffer, 0,
+ 3 * (fnic_fc_trace_max_pages * PAGE_SIZE));
+ fnic_dbg_prt->buffer_len =
+ fnic_fc_trace_get_data(fnic_dbg_prt, *rdata_ptr);
}
- memset((void *)fnic_dbg_prt->buffer, 0,
- (3*(trace_max_pages * PAGE_SIZE)));
- fnic_dbg_prt->buffer_len = fnic_get_trace_data(fnic_dbg_prt);
file->private_data = fnic_dbg_prt;
+
return 0;
}
@@ -272,13 +354,6 @@
return 0;
}
-static const struct file_operations fnic_trace_ctrl_fops = {
- .owner = THIS_MODULE,
- .open = fnic_trace_ctrl_open,
- .read = fnic_trace_ctrl_read,
- .write = fnic_trace_ctrl_write,
-};
-
static const struct file_operations fnic_trace_debugfs_fops = {
.owner = THIS_MODULE,
.open = fnic_trace_debugfs_open,
@@ -306,9 +381,10 @@
return rc;
}
fnic_trace_enable = debugfs_create_file("tracing_enable",
- S_IFREG|S_IRUGO|S_IWUSR,
- fnic_trace_debugfs_root,
- NULL, &fnic_trace_ctrl_fops);
+ S_IFREG|S_IRUGO|S_IWUSR,
+ fnic_trace_debugfs_root,
+ &(fc_trc_flag->fnic_trace),
+ &fnic_trace_ctrl_fops);
if (!fnic_trace_enable) {
printk(KERN_DEBUG
@@ -317,10 +393,10 @@
}
fnic_trace_debugfs_file = debugfs_create_file("trace",
- S_IFREG|S_IRUGO|S_IWUSR,
- fnic_trace_debugfs_root,
- NULL,
- &fnic_trace_debugfs_fops);
+ S_IFREG|S_IRUGO|S_IWUSR,
+ fnic_trace_debugfs_root,
+ &(fc_trc_flag->fnic_trace),
+ &fnic_trace_debugfs_fops);
if (!fnic_trace_debugfs_file) {
printk(KERN_DEBUG
@@ -340,14 +416,104 @@
*/
void fnic_trace_debugfs_terminate(void)
{
- if (fnic_trace_debugfs_file) {
- debugfs_remove(fnic_trace_debugfs_file);
- fnic_trace_debugfs_file = NULL;
+ debugfs_remove(fnic_trace_debugfs_file);
+ fnic_trace_debugfs_file = NULL;
+
+ debugfs_remove(fnic_trace_enable);
+ fnic_trace_enable = NULL;
+}
+
+/*
+ * fnic_fc_trace_debugfs_init -
+ * Initialize debugfs for fnic control frame trace logging
+ *
+ * Description:
+ * When Debugfs is configured this routine sets up the fnic_fc debugfs
+ * file system. If not already created, this routine will create the
+ * create file trace to log fnic fc trace buffer output into debugfs and
+ * it will also create file fc_trace_enable to control enable/disable of
+ * trace logging into trace buffer.
+ */
+
+int fnic_fc_trace_debugfs_init(void)
+{
+ int rc = -1;
+
+ if (!fnic_trace_debugfs_root) {
+ pr_err("fnic:Debugfs root directory doesn't exist\n");
+ return rc;
}
- if (fnic_trace_enable) {
- debugfs_remove(fnic_trace_enable);
- fnic_trace_enable = NULL;
+
+ fnic_fc_trace_enable = debugfs_create_file("fc_trace_enable",
+ S_IFREG|S_IRUGO|S_IWUSR,
+ fnic_trace_debugfs_root,
+ &(fc_trc_flag->fc_trace),
+ &fnic_trace_ctrl_fops);
+
+ if (!fnic_fc_trace_enable) {
+ pr_err("fnic: Failed create fc_trace_enable file\n");
+ return rc;
}
+
+ fnic_fc_trace_clear = debugfs_create_file("fc_trace_clear",
+ S_IFREG|S_IRUGO|S_IWUSR,
+ fnic_trace_debugfs_root,
+ &(fc_trc_flag->fc_clear),
+ &fnic_trace_ctrl_fops);
+
+ if (!fnic_fc_trace_clear) {
+ pr_err("fnic: Failed to create fc_trace_enable file\n");
+ return rc;
+ }
+
+ fnic_fc_rdata_trace_debugfs_file =
+ debugfs_create_file("fc_trace_rdata",
+ S_IFREG|S_IRUGO|S_IWUSR,
+ fnic_trace_debugfs_root,
+ &(fc_trc_flag->fc_normal_file),
+ &fnic_trace_debugfs_fops);
+
+ if (!fnic_fc_rdata_trace_debugfs_file) {
+ pr_err("fnic: Failed create fc_rdata_trace file\n");
+ return rc;
+ }
+
+ fnic_fc_trace_debugfs_file =
+ debugfs_create_file("fc_trace",
+ S_IFREG|S_IRUGO|S_IWUSR,
+ fnic_trace_debugfs_root,
+ &(fc_trc_flag->fc_row_file),
+ &fnic_trace_debugfs_fops);
+
+ if (!fnic_fc_trace_debugfs_file) {
+ pr_err("fnic: Failed to create fc_trace file\n");
+ return rc;
+ }
+ rc = 0;
+ return rc;
+}
+
+/*
+ * fnic_fc_trace_debugfs_terminate - Tear down debugfs infrastructure
+ *
+ * Description:
+ * When Debugfs is configured this routine removes debugfs file system
+ * elements that are specific to fnic_fc trace logging.
+ */
+
+void fnic_fc_trace_debugfs_terminate(void)
+{
+ debugfs_remove(fnic_fc_trace_debugfs_file);
+ fnic_fc_trace_debugfs_file = NULL;
+
+ debugfs_remove(fnic_fc_rdata_trace_debugfs_file);
+ fnic_fc_rdata_trace_debugfs_file = NULL;
+
+ debugfs_remove(fnic_fc_trace_enable);
+ fnic_fc_trace_enable = NULL;
+
+ debugfs_remove(fnic_fc_trace_clear);
+ fnic_fc_trace_clear = NULL;
}
/*
diff --git a/drivers/scsi/fnic/fnic_fcs.c b/drivers/scsi/fnic/fnic_fcs.c
index 1671325..1b948f6 100644
--- a/drivers/scsi/fnic/fnic_fcs.c
+++ b/drivers/scsi/fnic/fnic_fcs.c
@@ -66,19 +66,35 @@
fnic->link_down_cnt = vnic_dev_link_down_cnt(fnic->vdev);
if (old_link_status == fnic->link_status) {
- if (!fnic->link_status)
+ if (!fnic->link_status) {
/* DOWN -> DOWN */
spin_unlock_irqrestore(&fnic->fnic_lock, flags);
- else {
+ fnic_fc_trace_set_data(fnic->lport->host->host_no,
+ FNIC_FC_LE, "Link Status: DOWN->DOWN",
+ strlen("Link Status: DOWN->DOWN"));
+ } else {
if (old_link_down_cnt != fnic->link_down_cnt) {
/* UP -> DOWN -> UP */
fnic->lport->host_stats.link_failure_count++;
spin_unlock_irqrestore(&fnic->fnic_lock, flags);
+ fnic_fc_trace_set_data(
+ fnic->lport->host->host_no,
+ FNIC_FC_LE,
+ "Link Status:UP_DOWN_UP",
+ strlen("Link_Status:UP_DOWN_UP")
+ );
FNIC_FCS_DBG(KERN_DEBUG, fnic->lport->host,
"link down\n");
fcoe_ctlr_link_down(&fnic->ctlr);
if (fnic->config.flags & VFCF_FIP_CAPABLE) {
/* start FCoE VLAN discovery */
+ fnic_fc_trace_set_data(
+ fnic->lport->host->host_no,
+ FNIC_FC_LE,
+ "Link Status: UP_DOWN_UP_VLAN",
+ strlen(
+ "Link Status: UP_DOWN_UP_VLAN")
+ );
fnic_fcoe_send_vlan_req(fnic);
return;
}
@@ -88,22 +104,36 @@
} else
/* UP -> UP */
spin_unlock_irqrestore(&fnic->fnic_lock, flags);
+ fnic_fc_trace_set_data(
+ fnic->lport->host->host_no, FNIC_FC_LE,
+ "Link Status: UP_UP",
+ strlen("Link Status: UP_UP"));
}
} else if (fnic->link_status) {
/* DOWN -> UP */
spin_unlock_irqrestore(&fnic->fnic_lock, flags);
if (fnic->config.flags & VFCF_FIP_CAPABLE) {
/* start FCoE VLAN discovery */
+ fnic_fc_trace_set_data(
+ fnic->lport->host->host_no,
+ FNIC_FC_LE, "Link Status: DOWN_UP_VLAN",
+ strlen("Link Status: DOWN_UP_VLAN"));
fnic_fcoe_send_vlan_req(fnic);
return;
}
FNIC_FCS_DBG(KERN_DEBUG, fnic->lport->host, "link up\n");
+ fnic_fc_trace_set_data(fnic->lport->host->host_no, FNIC_FC_LE,
+ "Link Status: DOWN_UP", strlen("Link Status: DOWN_UP"));
fcoe_ctlr_link_up(&fnic->ctlr);
} else {
/* UP -> DOWN */
fnic->lport->host_stats.link_failure_count++;
spin_unlock_irqrestore(&fnic->fnic_lock, flags);
FNIC_FCS_DBG(KERN_DEBUG, fnic->lport->host, "link down\n");
+ fnic_fc_trace_set_data(
+ fnic->lport->host->host_no, FNIC_FC_LE,
+ "Link Status: UP_DOWN",
+ strlen("Link Status: UP_DOWN"));
fcoe_ctlr_link_down(&fnic->ctlr);
}
@@ -267,11 +297,6 @@
if (desc->fip_dtype == FIP_DT_FLOGI) {
- shost_printk(KERN_DEBUG, lport->host,
- " FIP TYPE FLOGI: fab name:%llx "
- "vfid:%d map:%x\n",
- fip->sel_fcf->fabric_name, fip->sel_fcf->vfid,
- fip->sel_fcf->fc_map);
if (dlen < sizeof(*els) + sizeof(*fh) + 1)
return 0;
@@ -616,6 +641,10 @@
"using UCSM\n");
goto drop;
}
+ if ((fnic_fc_trace_set_data(fnic->lport->host->host_no,
+ FNIC_FC_RECV|0x80, (char *)skb->data, skb->len)) != 0) {
+ printk(KERN_ERR "fnic ctlr frame trace error!!!");
+ }
skb_queue_tail(&fnic->fip_frame_queue, skb);
queue_work(fnic_fip_queue, &fnic->fip_frame_work);
return 1; /* let caller know packet was used */
@@ -844,6 +873,10 @@
}
fr_dev(fp) = fnic->lport;
spin_unlock_irqrestore(&fnic->fnic_lock, flags);
+ if ((fnic_fc_trace_set_data(fnic->lport->host->host_no, FNIC_FC_RECV,
+ (char *)skb->data, skb->len)) != 0) {
+ printk(KERN_ERR "fnic ctlr frame trace error!!!");
+ }
skb_queue_tail(&fnic->frame_queue, skb);
queue_work(fnic_event_queue, &fnic->frame_work);
@@ -951,6 +984,15 @@
vlan_hdr->h_vlan_proto = htons(ETH_P_8021Q);
vlan_hdr->h_vlan_encapsulated_proto = eth_hdr->h_proto;
vlan_hdr->h_vlan_TCI = htons(fnic->vlan_id);
+ if ((fnic_fc_trace_set_data(fnic->lport->host->host_no,
+ FNIC_FC_SEND|0x80, (char *)eth_hdr, skb->len)) != 0) {
+ printk(KERN_ERR "fnic ctlr frame trace error!!!");
+ }
+ } else {
+ if ((fnic_fc_trace_set_data(fnic->lport->host->host_no,
+ FNIC_FC_SEND|0x80, (char *)skb->data, skb->len)) != 0) {
+ printk(KERN_ERR "fnic ctlr frame trace error!!!");
+ }
}
pa = pci_map_single(fnic->pdev, skb->data, skb->len, PCI_DMA_TODEVICE);
@@ -1023,6 +1065,11 @@
pa = pci_map_single(fnic->pdev, eth_hdr, tot_len, PCI_DMA_TODEVICE);
+ if ((fnic_fc_trace_set_data(fnic->lport->host->host_no, FNIC_FC_SEND,
+ (char *)eth_hdr, tot_len)) != 0) {
+ printk(KERN_ERR "fnic ctlr frame trace error!!!");
+ }
+
spin_lock_irqsave(&fnic->wq_lock[0], flags);
if (!vnic_wq_desc_avail(wq)) {
diff --git a/drivers/scsi/fnic/fnic_main.c b/drivers/scsi/fnic/fnic_main.c
index 33e4ec2..8c56fdc 100644
--- a/drivers/scsi/fnic/fnic_main.c
+++ b/drivers/scsi/fnic/fnic_main.c
@@ -74,6 +74,11 @@
MODULE_PARM_DESC(fnic_trace_max_pages, "Total allocated memory pages "
"for fnic trace buffer");
+unsigned int fnic_fc_trace_max_pages = 64;
+module_param(fnic_fc_trace_max_pages, uint, S_IRUGO|S_IWUSR);
+MODULE_PARM_DESC(fnic_fc_trace_max_pages,
+ "Total allocated memory pages for fc trace buffer");
+
static unsigned int fnic_max_qdepth = FNIC_DFLT_QUEUE_DEPTH;
module_param(fnic_max_qdepth, uint, S_IRUGO|S_IWUSR);
MODULE_PARM_DESC(fnic_max_qdepth, "Queue depth to report for each LUN");
@@ -111,7 +116,7 @@
.change_queue_type = fc_change_queue_type,
.this_id = -1,
.cmd_per_lun = 3,
- .can_queue = FNIC_MAX_IO_REQ,
+ .can_queue = FNIC_DFLT_IO_REQ,
.use_clustering = ENABLE_CLUSTERING,
.sg_tablesize = FNIC_MAX_SG_DESC_CNT,
.max_sectors = 0xffff,
@@ -773,6 +778,7 @@
shost_printk(KERN_INFO, fnic->lport->host,
"firmware uses non-FIP mode\n");
fcoe_ctlr_init(&fnic->ctlr, FIP_MODE_NON_FIP);
+ fnic->ctlr.state = FIP_ST_NON_FIP;
}
fnic->state = FNIC_IN_FC_MODE;
@@ -1033,11 +1039,20 @@
/* Allocate memory for trace buffer */
err = fnic_trace_buf_init();
if (err < 0) {
- printk(KERN_ERR PFX "Trace buffer initialization Failed "
- "Fnic Tracing utility is disabled\n");
+ printk(KERN_ERR PFX
+ "Trace buffer initialization Failed. "
+ "Fnic Tracing utility is disabled\n");
fnic_trace_free();
}
+ /* Allocate memory for fc trace buffer */
+ err = fnic_fc_trace_init();
+ if (err < 0) {
+ printk(KERN_ERR PFX "FC trace buffer initialization Failed "
+ "FC frame tracing utility is disabled\n");
+ fnic_fc_trace_free();
+ }
+
/* Create a cache for allocation of default size sgls */
len = sizeof(struct fnic_dflt_sgl_list);
fnic_sgl_cache[FNIC_SGL_CACHE_DFLT] = kmem_cache_create
@@ -1118,6 +1133,7 @@
kmem_cache_destroy(fnic_sgl_cache[FNIC_SGL_CACHE_DFLT]);
err_create_fnic_sgl_slab_dflt:
fnic_trace_free();
+ fnic_fc_trace_free();
fnic_debugfs_terminate();
return err;
}
@@ -1135,6 +1151,7 @@
kmem_cache_destroy(fnic_io_req_cache);
fc_release_transport(fnic_fc_transport);
fnic_trace_free();
+ fnic_fc_trace_free();
fnic_debugfs_terminate();
}
diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c
index 0521436d..ea28b5c 100644
--- a/drivers/scsi/fnic/fnic_scsi.c
+++ b/drivers/scsi/fnic/fnic_scsi.c
@@ -1312,8 +1312,9 @@
cleanup_scsi_cmd:
sc->result = DID_TRANSPORT_DISRUPTED << 16;
- FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, "fnic_cleanup_io:"
- " DID_TRANSPORT_DISRUPTED\n");
+ FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+ "%s: sc duration = %lu DID_TRANSPORT_DISRUPTED\n",
+ __func__, (jiffies - start_time));
if (atomic64_read(&fnic->io_cmpl_skip))
atomic64_dec(&fnic->io_cmpl_skip);
@@ -1733,6 +1734,7 @@
struct fnic_stats *fnic_stats;
struct abort_stats *abts_stats;
struct terminate_stats *term_stats;
+ enum fnic_ioreq_state old_ioreq_state;
int tag;
DECLARE_COMPLETION_ONSTACK(tm_done);
@@ -1793,6 +1795,7 @@
* the completion wont be done till mid-layer, since abort
* has already started.
*/
+ old_ioreq_state = CMD_STATE(sc);
CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE;
@@ -1816,6 +1819,8 @@
if (fnic_queue_abort_io_req(fnic, sc->request->tag, task_req,
fc_lun.scsi_lun, io_req)) {
spin_lock_irqsave(io_lock, flags);
+ if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
+ CMD_STATE(sc) = old_ioreq_state;
io_req = (struct fnic_io_req *)CMD_SP(sc);
if (io_req)
io_req->abts_done = NULL;
@@ -1859,12 +1864,8 @@
if (CMD_ABTS_STATUS(sc) == FCPIO_INVALID_CODE) {
spin_unlock_irqrestore(io_lock, flags);
if (task_req == FCPIO_ITMF_ABT_TASK) {
- FNIC_SCSI_DBG(KERN_INFO,
- fnic->lport->host, "Abort Driver Timeout\n");
atomic64_inc(&abts_stats->abort_drv_timeouts);
} else {
- FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
- "Terminate Driver Timeout\n");
atomic64_inc(&term_stats->terminate_drv_timeouts);
}
CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_TIMED_OUT;
diff --git a/drivers/scsi/fnic/fnic_trace.c b/drivers/scsi/fnic/fnic_trace.c
index e002e71..c772859 100644
--- a/drivers/scsi/fnic/fnic_trace.c
+++ b/drivers/scsi/fnic/fnic_trace.c
@@ -20,6 +20,7 @@
#include <linux/errno.h>
#include <linux/spinlock.h>
#include <linux/kallsyms.h>
+#include <linux/time.h>
#include "fnic_io.h"
#include "fnic.h"
@@ -32,6 +33,16 @@
static fnic_trace_dbg_t fnic_trace_entries;
int fnic_tracing_enabled = 1;
+/* static char *fnic_fc_ctlr_trace_buf_p; */
+
+static int fc_trace_max_entries;
+static unsigned long fnic_fc_ctlr_trace_buf_p;
+static fnic_trace_dbg_t fc_trace_entries;
+int fnic_fc_tracing_enabled = 1;
+int fnic_fc_trace_cleared = 1;
+static DEFINE_SPINLOCK(fnic_fc_trace_lock);
+
+
/*
* fnic_trace_get_buf - Give buffer pointer to user to fill up trace information
*
@@ -428,10 +439,10 @@
}
err = fnic_trace_debugfs_init();
if (err < 0) {
- printk(KERN_ERR PFX "Failed to initialize debugfs for tracing\n");
+ pr_err("fnic: Failed to initialize debugfs for tracing\n");
goto err_fnic_trace_debugfs_init;
}
- printk(KERN_INFO PFX "Successfully Initialized Trace Buffer\n");
+ pr_info("fnic: Successfully Initialized Trace Buffer\n");
return err;
err_fnic_trace_debugfs_init:
fnic_trace_free();
@@ -456,3 +467,314 @@
}
printk(KERN_INFO PFX "Successfully Freed Trace Buffer\n");
}
+
+/*
+ * fnic_fc_ctlr_trace_buf_init -
+ * Initialize trace buffer to log fnic control frames
+ * Description:
+ * Initialize trace buffer data structure by allocating
+ * required memory for trace data as well as for Indexes.
+ * Frame size is 256 bytes and
+ * memory is allocated for 1024 entries of 256 bytes.
+ * Page_offset(Index) is set to the address of trace entry
+ * and page_offset is initialized by adding frame size
+ * to the previous page_offset entry.
+ */
+
+int fnic_fc_trace_init(void)
+{
+ unsigned long fc_trace_buf_head;
+ int err = 0;
+ int i;
+
+ fc_trace_max_entries = (fnic_fc_trace_max_pages * PAGE_SIZE)/
+ FC_TRC_SIZE_BYTES;
+ fnic_fc_ctlr_trace_buf_p = (unsigned long)vmalloc(
+ fnic_fc_trace_max_pages * PAGE_SIZE);
+ if (!fnic_fc_ctlr_trace_buf_p) {
+ pr_err("fnic: Failed to allocate memory for "
+ "FC Control Trace Buf\n");
+ err = -ENOMEM;
+ goto err_fnic_fc_ctlr_trace_buf_init;
+ }
+
+ memset((void *)fnic_fc_ctlr_trace_buf_p, 0,
+ fnic_fc_trace_max_pages * PAGE_SIZE);
+
+ /* Allocate memory for page offset */
+ fc_trace_entries.page_offset = vmalloc(fc_trace_max_entries *
+ sizeof(unsigned long));
+ if (!fc_trace_entries.page_offset) {
+ pr_err("fnic:Failed to allocate memory for page_offset\n");
+ if (fnic_fc_ctlr_trace_buf_p) {
+ pr_err("fnic: Freeing FC Control Trace Buf\n");
+ vfree((void *)fnic_fc_ctlr_trace_buf_p);
+ fnic_fc_ctlr_trace_buf_p = 0;
+ }
+ err = -ENOMEM;
+ goto err_fnic_fc_ctlr_trace_buf_init;
+ }
+ memset((void *)fc_trace_entries.page_offset, 0,
+ (fc_trace_max_entries * sizeof(unsigned long)));
+
+ fc_trace_entries.rd_idx = fc_trace_entries.wr_idx = 0;
+ fc_trace_buf_head = fnic_fc_ctlr_trace_buf_p;
+
+ /*
+ * Set up fc_trace_entries.page_offset field with memory location
+ * for every trace entry
+ */
+ for (i = 0; i < fc_trace_max_entries; i++) {
+ fc_trace_entries.page_offset[i] = fc_trace_buf_head;
+ fc_trace_buf_head += FC_TRC_SIZE_BYTES;
+ }
+ err = fnic_fc_trace_debugfs_init();
+ if (err < 0) {
+ pr_err("fnic: Failed to initialize FC_CTLR tracing.\n");
+ goto err_fnic_fc_ctlr_trace_debugfs_init;
+ }
+ pr_info("fnic: Successfully Initialized FC_CTLR Trace Buffer\n");
+ return err;
+
+err_fnic_fc_ctlr_trace_debugfs_init:
+ fnic_fc_trace_free();
+err_fnic_fc_ctlr_trace_buf_init:
+ return err;
+}
+
+/*
+ * Fnic_fc_ctlr_trace_free - Free memory of fnic_fc_ctlr trace data structures.
+ */
+void fnic_fc_trace_free(void)
+{
+ fnic_fc_tracing_enabled = 0;
+ fnic_fc_trace_debugfs_terminate();
+ if (fc_trace_entries.page_offset) {
+ vfree((void *)fc_trace_entries.page_offset);
+ fc_trace_entries.page_offset = NULL;
+ }
+ if (fnic_fc_ctlr_trace_buf_p) {
+ vfree((void *)fnic_fc_ctlr_trace_buf_p);
+ fnic_fc_ctlr_trace_buf_p = 0;
+ }
+ pr_info("fnic:Successfully FC_CTLR Freed Trace Buffer\n");
+}
+
+/*
+ * fnic_fc_ctlr_set_trace_data:
+ * Maintain rd & wr idx accordingly and set data
+ * Passed parameters:
+ * host_no: host number accociated with fnic
+ * frame_type: send_frame, rece_frame or link event
+ * fc_frame: pointer to fc_frame
+ * frame_len: Length of the fc_frame
+ * Description:
+ * This routine will get next available wr_idx and
+ * copy all passed trace data to the buffer pointed by wr_idx
+ * and increment wr_idx. It will also make sure that we dont
+ * overwrite the entry which we are reading and also
+ * wrap around if we reach the maximum entries.
+ * Returned Value:
+ * It will return 0 for success or -1 for failure
+ */
+int fnic_fc_trace_set_data(u32 host_no, u8 frame_type,
+ char *frame, u32 fc_trc_frame_len)
+{
+ unsigned long flags;
+ struct fc_trace_hdr *fc_buf;
+ unsigned long eth_fcoe_hdr_len;
+ char *fc_trace;
+
+ if (fnic_fc_tracing_enabled == 0)
+ return 0;
+
+ spin_lock_irqsave(&fnic_fc_trace_lock, flags);
+
+ if (fnic_fc_trace_cleared == 1) {
+ fc_trace_entries.rd_idx = fc_trace_entries.wr_idx = 0;
+ pr_info("fnic: Reseting the read idx\n");
+ memset((void *)fnic_fc_ctlr_trace_buf_p, 0,
+ fnic_fc_trace_max_pages * PAGE_SIZE);
+ fnic_fc_trace_cleared = 0;
+ }
+
+ fc_buf = (struct fc_trace_hdr *)
+ fc_trace_entries.page_offset[fc_trace_entries.wr_idx];
+
+ fc_trace_entries.wr_idx++;
+
+ if (fc_trace_entries.wr_idx >= fc_trace_max_entries)
+ fc_trace_entries.wr_idx = 0;
+
+ if (fc_trace_entries.wr_idx == fc_trace_entries.rd_idx) {
+ fc_trace_entries.rd_idx++;
+ if (fc_trace_entries.rd_idx >= fc_trace_max_entries)
+ fc_trace_entries.rd_idx = 0;
+ }
+
+ fc_buf->time_stamp = CURRENT_TIME;
+ fc_buf->host_no = host_no;
+ fc_buf->frame_type = frame_type;
+
+ fc_trace = (char *)FC_TRACE_ADDRESS(fc_buf);
+
+ /* During the receive path, we do not have eth hdr as well as fcoe hdr
+ * at trace entry point so we will stuff 0xff just to make it generic.
+ */
+ if (frame_type == FNIC_FC_RECV) {
+ eth_fcoe_hdr_len = sizeof(struct ethhdr) +
+ sizeof(struct fcoe_hdr);
+ fc_trc_frame_len = fc_trc_frame_len + eth_fcoe_hdr_len;
+ memset((char *)fc_trace, 0xff, eth_fcoe_hdr_len);
+ /* Copy the rest of data frame */
+ memcpy((char *)(fc_trace + eth_fcoe_hdr_len), (void *)frame,
+ min_t(u8, fc_trc_frame_len,
+ (u8)(FC_TRC_SIZE_BYTES - FC_TRC_HEADER_SIZE)));
+ } else {
+ memcpy((char *)fc_trace, (void *)frame,
+ min_t(u8, fc_trc_frame_len,
+ (u8)(FC_TRC_SIZE_BYTES - FC_TRC_HEADER_SIZE)));
+ }
+
+ /* Store the actual received length */
+ fc_buf->frame_len = fc_trc_frame_len;
+
+ spin_unlock_irqrestore(&fnic_fc_trace_lock, flags);
+ return 0;
+}
+
+/*
+ * fnic_fc_ctlr_get_trace_data: Copy trace buffer to a memory file
+ * Passed parameter:
+ * @fnic_dbgfs_t: pointer to debugfs trace buffer
+ * rdata_flag: 1 => Unformated file
+ * 0 => formated file
+ * Description:
+ * This routine will copy the trace data to memory file with
+ * proper formatting and also copy to another memory
+ * file without formatting for further procesing.
+ * Retrun Value:
+ * Number of bytes that were dumped into fnic_dbgfs_t
+ */
+
+int fnic_fc_trace_get_data(fnic_dbgfs_t *fnic_dbgfs_prt, u8 rdata_flag)
+{
+ int rd_idx, wr_idx;
+ unsigned long flags;
+ int len = 0, j;
+ struct fc_trace_hdr *tdata;
+ char *fc_trace;
+
+ spin_lock_irqsave(&fnic_fc_trace_lock, flags);
+ if (fc_trace_entries.wr_idx == fc_trace_entries.rd_idx) {
+ spin_unlock_irqrestore(&fnic_fc_trace_lock, flags);
+ pr_info("fnic: Buffer is empty\n");
+ return 0;
+ }
+ rd_idx = fc_trace_entries.rd_idx;
+ wr_idx = fc_trace_entries.wr_idx;
+ if (rdata_flag == 0) {
+ len += snprintf(fnic_dbgfs_prt->buffer + len,
+ (fnic_fc_trace_max_pages * PAGE_SIZE * 3) - len,
+ "Time Stamp (UTC)\t\t"
+ "Host No: F Type: len: FCoE_FRAME:\n");
+ }
+
+ while (rd_idx != wr_idx) {
+ tdata = (struct fc_trace_hdr *)
+ fc_trace_entries.page_offset[rd_idx];
+ if (!tdata) {
+ pr_info("fnic: Rd data is NULL\n");
+ spin_unlock_irqrestore(&fnic_fc_trace_lock, flags);
+ return 0;
+ }
+ if (rdata_flag == 0) {
+ copy_and_format_trace_data(tdata,
+ fnic_dbgfs_prt, &len, rdata_flag);
+ } else {
+ fc_trace = (char *)tdata;
+ for (j = 0; j < FC_TRC_SIZE_BYTES; j++) {
+ len += snprintf(fnic_dbgfs_prt->buffer + len,
+ (fnic_fc_trace_max_pages * PAGE_SIZE * 3)
+ - len, "%02x", fc_trace[j] & 0xff);
+ } /* for loop */
+ len += snprintf(fnic_dbgfs_prt->buffer + len,
+ (fnic_fc_trace_max_pages * PAGE_SIZE * 3) - len,
+ "\n");
+ }
+ rd_idx++;
+ if (rd_idx > (fc_trace_max_entries - 1))
+ rd_idx = 0;
+ }
+
+ spin_unlock_irqrestore(&fnic_fc_trace_lock, flags);
+ return len;
+}
+
+/*
+ * copy_and_format_trace_data: Copy formatted data to char * buffer
+ * Passed Parameter:
+ * @fc_trace_hdr_t: pointer to trace data
+ * @fnic_dbgfs_t: pointer to debugfs trace buffer
+ * @orig_len: pointer to len
+ * rdata_flag: 0 => Formated file, 1 => Unformated file
+ * Description:
+ * This routine will format and copy the passed trace data
+ * for formated file or unformated file accordingly.
+ */
+
+void copy_and_format_trace_data(struct fc_trace_hdr *tdata,
+ fnic_dbgfs_t *fnic_dbgfs_prt, int *orig_len,
+ u8 rdata_flag)
+{
+ struct tm tm;
+ int j, i = 1, len;
+ char *fc_trace, *fmt;
+ int ethhdr_len = sizeof(struct ethhdr) - 1;
+ int fcoehdr_len = sizeof(struct fcoe_hdr);
+ int fchdr_len = sizeof(struct fc_frame_header);
+ int max_size = fnic_fc_trace_max_pages * PAGE_SIZE * 3;
+
+ tdata->frame_type = tdata->frame_type & 0x7F;
+
+ len = *orig_len;
+
+ time_to_tm(tdata->time_stamp.tv_sec, 0, &tm);
+
+ fmt = "%02d:%02d:%04ld %02d:%02d:%02d.%09lu ns%8x %c%8x\t";
+ len += snprintf(fnic_dbgfs_prt->buffer + len,
+ (fnic_fc_trace_max_pages * PAGE_SIZE * 3) - len,
+ fmt,
+ tm.tm_mon + 1, tm.tm_mday, tm.tm_year + 1900,
+ tm.tm_hour, tm.tm_min, tm.tm_sec,
+ tdata->time_stamp.tv_nsec, tdata->host_no,
+ tdata->frame_type, tdata->frame_len);
+
+ fc_trace = (char *)FC_TRACE_ADDRESS(tdata);
+
+ for (j = 0; j < min_t(u8, tdata->frame_len,
+ (u8)(FC_TRC_SIZE_BYTES - FC_TRC_HEADER_SIZE)); j++) {
+ if (tdata->frame_type == FNIC_FC_LE) {
+ len += snprintf(fnic_dbgfs_prt->buffer + len,
+ max_size - len, "%c", fc_trace[j]);
+ } else {
+ len += snprintf(fnic_dbgfs_prt->buffer + len,
+ max_size - len, "%02x", fc_trace[j] & 0xff);
+ len += snprintf(fnic_dbgfs_prt->buffer + len,
+ max_size - len, " ");
+ if (j == ethhdr_len ||
+ j == ethhdr_len + fcoehdr_len ||
+ j == ethhdr_len + fcoehdr_len + fchdr_len ||
+ (i > 3 && j%fchdr_len == 0)) {
+ len += snprintf(fnic_dbgfs_prt->buffer
+ + len, (fnic_fc_trace_max_pages
+ * PAGE_SIZE * 3) - len,
+ "\n\t\t\t\t\t\t\t\t");
+ i++;
+ }
+ } /* end of else*/
+ } /* End of for loop*/
+ len += snprintf(fnic_dbgfs_prt->buffer + len,
+ max_size - len, "\n");
+ *orig_len = len;
+}
diff --git a/drivers/scsi/fnic/fnic_trace.h b/drivers/scsi/fnic/fnic_trace.h
index d412f2e..a8aa057 100644
--- a/drivers/scsi/fnic/fnic_trace.h
+++ b/drivers/scsi/fnic/fnic_trace.h
@@ -19,6 +19,17 @@
#define __FNIC_TRACE_H__
#define FNIC_ENTRY_SIZE_BYTES 64
+#define FC_TRC_SIZE_BYTES 256
+#define FC_TRC_HEADER_SIZE sizeof(struct fc_trace_hdr)
+
+/*
+ * Fisrt bit of FNIC_FC_RECV and FNIC_FC_SEND is used to represent the type
+ * of frame 1 => Eth frame, 0=> FC frame
+ */
+
+#define FNIC_FC_RECV 0x52 /* Character R */
+#define FNIC_FC_SEND 0x54 /* Character T */
+#define FNIC_FC_LE 0x4C /* Character L */
extern ssize_t simple_read_from_buffer(void __user *to,
size_t count,
@@ -30,6 +41,10 @@
extern int fnic_tracing_enabled;
extern unsigned int trace_max_pages;
+extern unsigned int fnic_fc_trace_max_pages;
+extern int fnic_fc_tracing_enabled;
+extern int fnic_fc_trace_cleared;
+
typedef struct fnic_trace_dbg {
int wr_idx;
int rd_idx;
@@ -56,6 +71,16 @@
typedef struct fnic_trace_data fnic_trace_data_t;
+struct fc_trace_hdr {
+ struct timespec time_stamp;
+ u32 host_no;
+ u8 frame_type;
+ u8 frame_len;
+} __attribute__((__packed__));
+
+#define FC_TRACE_ADDRESS(a) \
+ ((unsigned long)(a) + sizeof(struct fc_trace_hdr))
+
#define FNIC_TRACE_ENTRY_SIZE \
(FNIC_ENTRY_SIZE_BYTES - sizeof(fnic_trace_data_t))
@@ -88,4 +113,17 @@
void fnic_debugfs_terminate(void);
int fnic_trace_debugfs_init(void);
void fnic_trace_debugfs_terminate(void);
+
+/* Fnic FC CTLR Trace releated function */
+int fnic_fc_trace_init(void);
+void fnic_fc_trace_free(void);
+int fnic_fc_trace_set_data(u32 host_no, u8 frame_type,
+ char *frame, u32 fc_frame_len);
+int fnic_fc_trace_get_data(fnic_dbgfs_t *fnic_dbgfs_prt, u8 rdata_flag);
+void copy_and_format_trace_data(struct fc_trace_hdr *tdata,
+ fnic_dbgfs_t *fnic_dbgfs_prt,
+ int *len, u8 rdata_flag);
+int fnic_fc_trace_debugfs_init(void);
+void fnic_fc_trace_debugfs_terminate(void);
+
#endif
diff --git a/drivers/scsi/g_NCR5380.c b/drivers/scsi/g_NCR5380.c
index 7176365..a1bc8ca9 100644
--- a/drivers/scsi/g_NCR5380.c
+++ b/drivers/scsi/g_NCR5380.c
@@ -78,10 +78,6 @@
*
*/
-/*
- * $Log: generic_NCR5380.c,v $
- */
-
/* settings for DTC3181E card with only Mustek scanner attached */
#define USLEEP
#define USLEEP_POLL 1
diff --git a/drivers/scsi/g_NCR5380.h b/drivers/scsi/g_NCR5380.h
index 1bcdb7b..703adf7 100644
--- a/drivers/scsi/g_NCR5380.h
+++ b/drivers/scsi/g_NCR5380.h
@@ -25,10 +25,6 @@
* 1+ (800) 334-5454
*/
-/*
- * $Log: generic_NCR5380.h,v $
- */
-
#ifndef GENERIC_NCR5380_H
#define GENERIC_NCR5380_H
@@ -58,8 +54,6 @@
#define CAN_QUEUE 16
#endif
-#ifndef HOSTS_C
-
#define __STRVAL(x) #x
#define STRVAL(x) __STRVAL(x)
@@ -131,7 +125,6 @@
#define BOARD_NCR53C400A 2
#define BOARD_DTC3181E 3
-#endif /* else def HOSTS_C */
#endif /* ndef ASM */
#endif /* GENERIC_NCR5380_H */
diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
index 9a6e4a2..5858600b 100644
--- a/drivers/scsi/hpsa.c
+++ b/drivers/scsi/hpsa.c
@@ -115,9 +115,15 @@
{PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21C3},
{PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21C4},
{PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21C5},
+ {PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21C6},
{PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21C7},
{PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21C8},
{PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21C9},
+ {PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21CA},
+ {PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21CB},
+ {PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21CC},
+ {PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21CD},
+ {PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_CISSI, 0x103C, 0x21CE},
{PCI_VENDOR_ID_HP_3PAR, 0x0075, 0x1590, 0x0076},
{PCI_VENDOR_ID_HP_3PAR, 0x0075, 0x1590, 0x0087},
{PCI_VENDOR_ID_HP_3PAR, 0x0075, 0x1590, 0x007D},
@@ -165,9 +171,15 @@
{0x21C3103C, "Smart Array", &SA5_access},
{0x21C4103C, "Smart Array", &SA5_access},
{0x21C5103C, "Smart Array", &SA5_access},
+ {0x21C6103C, "Smart Array", &SA5_access},
{0x21C7103C, "Smart Array", &SA5_access},
{0x21C8103C, "Smart Array", &SA5_access},
{0x21C9103C, "Smart Array", &SA5_access},
+ {0x21CA103C, "Smart Array", &SA5_access},
+ {0x21CB103C, "Smart Array", &SA5_access},
+ {0x21CC103C, "Smart Array", &SA5_access},
+ {0x21CD103C, "Smart Array", &SA5_access},
+ {0x21CE103C, "Smart Array", &SA5_access},
{0x00761590, "HP Storage P1224 Array Controller", &SA5_access},
{0x00871590, "HP Storage P1224e Array Controller", &SA5_access},
{0x007D1590, "HP Storage P1228 Array Controller", &SA5_access},
@@ -2836,6 +2848,8 @@
/* Get the list of physical devices */
physicals = kzalloc(reportsize, GFP_KERNEL);
+ if (physicals == NULL)
+ return 0;
if (hpsa_scsi_do_report_phys_luns(h, (struct ReportLUNdata *) physicals,
reportsize, extended)) {
dev_err(&h->pdev->dev,
@@ -2963,19 +2977,24 @@
static int hpsa_hba_mode_enabled(struct ctlr_info *h)
{
int rc;
+ int hba_mode_enabled;
struct bmic_controller_parameters *ctlr_params;
ctlr_params = kzalloc(sizeof(struct bmic_controller_parameters),
GFP_KERNEL);
if (!ctlr_params)
- return 0;
+ return -ENOMEM;
rc = hpsa_bmic_ctrl_mode_sense(h, RAID_CTLR_LUNID, 0, ctlr_params,
sizeof(struct bmic_controller_parameters));
- if (rc != 0) {
+ if (rc) {
kfree(ctlr_params);
- return 0;
+ return rc;
}
- return ctlr_params->nvram_flags & (1 << 3) ? 1 : 0;
+
+ hba_mode_enabled =
+ ((ctlr_params->nvram_flags & HBA_MODE_ENABLED_FLAG) != 0);
+ kfree(ctlr_params);
+ return hba_mode_enabled;
}
static void hpsa_update_scsi_devices(struct ctlr_info *h, int hostno)
@@ -3001,7 +3020,7 @@
int reportlunsize = sizeof(*physdev_list) + HPSA_MAX_PHYS_LUN * 24;
int i, n_ext_target_devs, ndevs_to_allocate;
int raid_ctlr_position;
- u8 rescan_hba_mode;
+ int rescan_hba_mode;
DECLARE_BITMAP(lunzerobits, MAX_EXT_TARGETS);
currentsd = kzalloc(sizeof(*currentsd) * HPSA_MAX_DEVICES, GFP_KERNEL);
@@ -3016,6 +3035,8 @@
memset(lunzerobits, 0, sizeof(lunzerobits));
rescan_hba_mode = hpsa_hba_mode_enabled(h);
+ if (rescan_hba_mode < 0)
+ goto out;
if (!h->hba_mode_enabled && rescan_hba_mode)
dev_warn(&h->pdev->dev, "HBA mode enabled\n");
diff --git a/drivers/scsi/hpsa.h b/drivers/scsi/hpsa.h
index 44235a2..1e3cf33 100644
--- a/drivers/scsi/hpsa.h
+++ b/drivers/scsi/hpsa.h
@@ -90,6 +90,7 @@
u8 automatic_drive_slamming;
u8 reserved1;
u8 nvram_flags;
+#define HBA_MODE_ENABLED_FLAG (1 << 3)
u8 cache_nvram_flags;
u8 drive_config_flags;
u16 reserved2;
diff --git a/drivers/scsi/libiscsi.c b/drivers/scsi/libiscsi.c
index 26dc005b..ecd7bd3 100644
--- a/drivers/scsi/libiscsi.c
+++ b/drivers/scsi/libiscsi.c
@@ -1442,9 +1442,9 @@
conn->task = NULL;
}
/* regular RX path uses back_lock */
- spin_lock_bh(&conn->session->back_lock);
+ spin_lock(&conn->session->back_lock);
__iscsi_put_task(task);
- spin_unlock_bh(&conn->session->back_lock);
+ spin_unlock(&conn->session->back_lock);
return rc;
}
diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index 6bb51f8e..393662c 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -265,6 +265,16 @@
return NULL;
q->hba_index = idx;
+
+ /*
+ * insert barrier for instruction interlock : data from the hardware
+ * must have the valid bit checked before it can be copied and acted
+ * upon. Given what was seen in lpfc_sli4_cq_get() of speculative
+ * instructions allowing action on content before valid bit checked,
+ * add barrier here as well. May not be needed as "content" is a
+ * single 32-bit entity here (vs multi word structure for cq's).
+ */
+ mb();
return eqe;
}
@@ -370,6 +380,17 @@
cqe = q->qe[q->hba_index].cqe;
q->hba_index = idx;
+
+ /*
+ * insert barrier for instruction interlock : data from the hardware
+ * must have the valid bit checked before it can be copied and acted
+ * upon. Speculative instructions were allowing a bcopy at the start
+ * of lpfc_sli4_fp_handle_wcqe(), which is called immediately
+ * after our return, to copy data before the valid bit check above
+ * was done. As such, some of the copied data was stale. The barrier
+ * ensures the check is before any data is copied.
+ */
+ mb();
return cqe;
}
diff --git a/drivers/scsi/mac_scsi.c b/drivers/scsi/mac_scsi.c
index f5cdc68..6a039eb 100644
--- a/drivers/scsi/mac_scsi.c
+++ b/drivers/scsi/mac_scsi.c
@@ -25,10 +25,6 @@
* 1+ (800) 334-5454
*/
-/*
- * $Log: mac_NCR5380.c,v $
- */
-
#include <linux/types.h>
#include <linux/stddef.h>
#include <linux/ctype.h>
@@ -58,12 +54,6 @@
#include "NCR5380.h"
-#if 0
-#define NDEBUG (NDEBUG_INTR | NDEBUG_PSEUDO_DMA | NDEBUG_ARBITRATION | NDEBUG_SELECTION | NDEBUG_RESELECTION)
-#else
-#define NDEBUG (NDEBUG_ABORT)
-#endif
-
#define RESET_BOOT
#define DRIVER_SETUP
diff --git a/drivers/scsi/mac_scsi.h b/drivers/scsi/mac_scsi.h
index 7dc62fc..06969b0 100644
--- a/drivers/scsi/mac_scsi.h
+++ b/drivers/scsi/mac_scsi.h
@@ -22,10 +22,6 @@
* 1+ (800) 334-5454
*/
-/*
- * $Log: cumana_NCR5380.h,v $
- */
-
#ifndef MAC_NCR5380_H
#define MAC_NCR5380_H
@@ -51,8 +47,6 @@
#include <scsi/scsicam.h>
-#ifndef HOSTS_C
-
#define NCR5380_implementation_fields \
int port, ctrl
@@ -75,10 +69,6 @@
#define NCR5380_show_info macscsi_show_info
#define NCR5380_write_info macscsi_write_info
-#define BOARD_NORMAL 0
-#define BOARD_NCR53C400 1
-
-#endif /* ndef HOSTS_C */
#endif /* ndef ASM */
#endif /* MAC_NCR5380_H */
diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index d84d02c..112799b 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -3061,7 +3061,8 @@
u32 cur_state;
u32 abs_state, curr_abs_state;
- fw_state = instance->instancet->read_fw_status_reg(instance->reg_set) & MFI_STATE_MASK;
+ abs_state = instance->instancet->read_fw_status_reg(instance->reg_set);
+ fw_state = abs_state & MFI_STATE_MASK;
if (fw_state != MFI_STATE_READY)
printk(KERN_INFO "megasas: Waiting for FW to come to ready"
@@ -3069,9 +3070,6 @@
while (fw_state != MFI_STATE_READY) {
- abs_state =
- instance->instancet->read_fw_status_reg(instance->reg_set);
-
switch (fw_state) {
case MFI_STATE_FAULT:
@@ -3223,10 +3221,8 @@
* The cur_state should not last for more than max_wait secs
*/
for (i = 0; i < (max_wait * 1000); i++) {
- fw_state = instance->instancet->read_fw_status_reg(instance->reg_set) &
- MFI_STATE_MASK ;
- curr_abs_state =
- instance->instancet->read_fw_status_reg(instance->reg_set);
+ curr_abs_state = instance->instancet->
+ read_fw_status_reg(instance->reg_set);
if (abs_state == curr_abs_state) {
msleep(1);
@@ -3242,6 +3238,9 @@
"in %d secs\n", fw_state, max_wait);
return -ENODEV;
}
+
+ abs_state = curr_abs_state;
+ fw_state = curr_abs_state & MFI_STATE_MASK;
}
printk(KERN_INFO "megasas: FW now in Ready state\n");
diff --git a/drivers/scsi/mpt2sas/mpt2sas_base.c b/drivers/scsi/mpt2sas/mpt2sas_base.c
index bde63f7..8b88118 100644
--- a/drivers/scsi/mpt2sas/mpt2sas_base.c
+++ b/drivers/scsi/mpt2sas/mpt2sas_base.c
@@ -1739,14 +1739,14 @@
list_for_each_entry_safe(chain_req, next,
&ioc->scsi_lookup[i].chain_list, tracker_list) {
list_del_init(&chain_req->tracker_list);
- list_add_tail(&chain_req->tracker_list,
+ list_add(&chain_req->tracker_list,
&ioc->free_chain_list);
}
}
ioc->scsi_lookup[i].cb_idx = 0xFF;
ioc->scsi_lookup[i].scmd = NULL;
ioc->scsi_lookup[i].direct_io = 0;
- list_add_tail(&ioc->scsi_lookup[i].tracker_list,
+ list_add(&ioc->scsi_lookup[i].tracker_list,
&ioc->free_list);
spin_unlock_irqrestore(&ioc->scsi_lookup_lock, flags);
@@ -1764,13 +1764,13 @@
/* hi-priority */
i = smid - ioc->hi_priority_smid;
ioc->hpr_lookup[i].cb_idx = 0xFF;
- list_add_tail(&ioc->hpr_lookup[i].tracker_list,
+ list_add(&ioc->hpr_lookup[i].tracker_list,
&ioc->hpr_free_list);
} else if (smid <= ioc->hba_queue_depth) {
/* internal queue */
i = smid - ioc->internal_smid;
ioc->internal_lookup[i].cb_idx = 0xFF;
- list_add_tail(&ioc->internal_lookup[i].tracker_list,
+ list_add(&ioc->internal_lookup[i].tracker_list,
&ioc->internal_free_list);
}
spin_unlock_irqrestore(&ioc->scsi_lookup_lock, flags);
diff --git a/drivers/scsi/mpt2sas/mpt2sas_base.h b/drivers/scsi/mpt2sas/mpt2sas_base.h
index 1f2ac3a..fd3b998 100644
--- a/drivers/scsi/mpt2sas/mpt2sas_base.h
+++ b/drivers/scsi/mpt2sas/mpt2sas_base.h
@@ -1065,7 +1065,7 @@
u32 reply);
int mpt2sas_scsih_issue_tm(struct MPT2SAS_ADAPTER *ioc, u16 handle,
uint channel, uint id, uint lun, u8 type, u16 smid_task,
- ulong timeout, unsigned long serial_number, enum mutex_type m_type);
+ ulong timeout, enum mutex_type m_type);
void mpt2sas_scsih_set_tm_flag(struct MPT2SAS_ADAPTER *ioc, u16 handle);
void mpt2sas_scsih_clear_tm_flag(struct MPT2SAS_ADAPTER *ioc, u16 handle);
void mpt2sas_expander_remove(struct MPT2SAS_ADAPTER *ioc, u64 sas_address);
diff --git a/drivers/scsi/mpt2sas/mpt2sas_ctl.c b/drivers/scsi/mpt2sas/mpt2sas_ctl.c
index b7f887c..62df8f9 100644
--- a/drivers/scsi/mpt2sas/mpt2sas_ctl.c
+++ b/drivers/scsi/mpt2sas/mpt2sas_ctl.c
@@ -987,7 +987,7 @@
mpt2sas_scsih_issue_tm(ioc,
le16_to_cpu(mpi_request->FunctionDependent1), 0, 0,
0, MPI2_SCSITASKMGMT_TASKTYPE_TARGET_RESET, 0, 10,
- 0, TM_MUTEX_ON);
+ TM_MUTEX_ON);
ioc->tm_cmds.status = MPT2_CMD_NOT_USED;
} else
mpt2sas_base_hard_reset_handler(ioc, CAN_SLEEP,
diff --git a/drivers/scsi/mpt2sas/mpt2sas_scsih.c b/drivers/scsi/mpt2sas/mpt2sas_scsih.c
index 6fd7d40..5055f92 100644
--- a/drivers/scsi/mpt2sas/mpt2sas_scsih.c
+++ b/drivers/scsi/mpt2sas/mpt2sas_scsih.c
@@ -2368,7 +2368,6 @@
* @type: MPI2_SCSITASKMGMT_TASKTYPE__XXX (defined in mpi2_init.h)
* @smid_task: smid assigned to the task
* @timeout: timeout in seconds
- * @serial_number: the serial_number from scmd
* @m_type: TM_MUTEX_ON or TM_MUTEX_OFF
* Context: user
*
@@ -2381,7 +2380,7 @@
int
mpt2sas_scsih_issue_tm(struct MPT2SAS_ADAPTER *ioc, u16 handle, uint channel,
uint id, uint lun, u8 type, u16 smid_task, ulong timeout,
- unsigned long serial_number, enum mutex_type m_type)
+ enum mutex_type m_type)
{
Mpi2SCSITaskManagementRequest_t *mpi_request;
Mpi2SCSITaskManagementReply_t *mpi_reply;
@@ -2634,8 +2633,7 @@
handle = sas_device_priv_data->sas_target->handle;
r = mpt2sas_scsih_issue_tm(ioc, handle, scmd->device->channel,
scmd->device->id, scmd->device->lun,
- MPI2_SCSITASKMGMT_TASKTYPE_ABORT_TASK, smid, 30,
- scmd->serial_number, TM_MUTEX_ON);
+ MPI2_SCSITASKMGMT_TASKTYPE_ABORT_TASK, smid, 30, TM_MUTEX_ON);
out:
sdev_printk(KERN_INFO, scmd->device, "task abort: %s scmd(%p)\n",
@@ -2696,8 +2694,7 @@
r = mpt2sas_scsih_issue_tm(ioc, handle, scmd->device->channel,
scmd->device->id, scmd->device->lun,
- MPI2_SCSITASKMGMT_TASKTYPE_LOGICAL_UNIT_RESET, 0, 30, 0,
- TM_MUTEX_ON);
+ MPI2_SCSITASKMGMT_TASKTYPE_LOGICAL_UNIT_RESET, 0, 30, TM_MUTEX_ON);
out:
sdev_printk(KERN_INFO, scmd->device, "device reset: %s scmd(%p)\n",
@@ -2757,7 +2754,7 @@
r = mpt2sas_scsih_issue_tm(ioc, handle, scmd->device->channel,
scmd->device->id, 0, MPI2_SCSITASKMGMT_TASKTYPE_TARGET_RESET, 0,
- 30, 0, TM_MUTEX_ON);
+ 30, TM_MUTEX_ON);
out:
starget_printk(KERN_INFO, starget, "target reset: %s scmd(%p)\n",
@@ -3953,9 +3950,9 @@
* SCSI_MLQUEUE_HOST_BUSY if the entire host queue is full
*/
static int
-_scsih_qcmd_lck(struct scsi_cmnd *scmd, void (*done)(struct scsi_cmnd *))
+_scsih_qcmd(struct Scsi_Host *shost, struct scsi_cmnd *scmd)
{
- struct MPT2SAS_ADAPTER *ioc = shost_priv(scmd->device->host);
+ struct MPT2SAS_ADAPTER *ioc = shost_priv(shost);
struct MPT2SAS_DEVICE *sas_device_priv_data;
struct MPT2SAS_TARGET *sas_target_priv_data;
struct _raid_device *raid_device;
@@ -3963,7 +3960,6 @@
u32 mpi_control;
u16 smid;
- scmd->scsi_done = done;
sas_device_priv_data = scmd->device->hostdata;
if (!sas_device_priv_data || !sas_device_priv_data->sas_target) {
scmd->result = DID_NO_CONNECT << 16;
@@ -4039,7 +4035,7 @@
MPT_TARGET_FLAGS_RAID_COMPONENT)
mpi_request->Function = MPI2_FUNCTION_RAID_SCSI_IO_PASSTHROUGH;
else
- mpi_request->Function = MPI2_FUNCTION_SCSI_IO_REQUEST;
+ mpi_request->Function = MPI2_FUNCTION_SCSI_IO_REQUEST;
mpi_request->DevHandle =
cpu_to_le16(sas_device_priv_data->sas_target->handle);
mpi_request->DataLength = cpu_to_le32(scsi_bufflen(scmd));
@@ -4083,8 +4079,6 @@
return SCSI_MLQUEUE_HOST_BUSY;
}
-static DEF_SCSI_QCMD(_scsih_qcmd)
-
/**
* _scsih_normalize_sense - normalize descriptor and fixed format sense data
* @sense_buffer: sense data returned by target
@@ -5880,7 +5874,7 @@
spin_unlock_irqrestore(&ioc->scsi_lookup_lock, flags);
r = mpt2sas_scsih_issue_tm(ioc, handle, 0, 0, lun,
- MPI2_SCSITASKMGMT_TASKTYPE_QUERY_TASK, smid, 30, 0,
+ MPI2_SCSITASKMGMT_TASKTYPE_QUERY_TASK, smid, 30,
TM_MUTEX_OFF);
if (r == FAILED) {
sdev_printk(KERN_WARNING, sdev,
@@ -5922,7 +5916,7 @@
r = mpt2sas_scsih_issue_tm(ioc, handle, sdev->channel, sdev->id,
sdev->lun, MPI2_SCSITASKMGMT_TASKTYPE_ABORT_TASK, smid, 30,
- scmd->serial_number, TM_MUTEX_OFF);
+ TM_MUTEX_OFF);
if (r == FAILED) {
sdev_printk(KERN_WARNING, sdev,
"mpt2sas_scsih_issue_tm: ABORT_TASK: FAILED : "
diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h
index 0ebf5d9..9b90a6f 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_base.h
+++ b/drivers/scsi/mpt3sas/mpt3sas_base.h
@@ -993,7 +993,7 @@
int mpt3sas_scsih_issue_tm(struct MPT3SAS_ADAPTER *ioc, u16 handle,
uint channel, uint id, uint lun, u8 type, u16 smid_task,
- ulong timeout, unsigned long serial_number, enum mutex_type m_type);
+ ulong timeout, enum mutex_type m_type);
void mpt3sas_scsih_set_tm_flag(struct MPT3SAS_ADAPTER *ioc, u16 handle);
void mpt3sas_scsih_clear_tm_flag(struct MPT3SAS_ADAPTER *ioc, u16 handle);
void mpt3sas_expander_remove(struct MPT3SAS_ADAPTER *ioc, u64 sas_address);
diff --git a/drivers/scsi/mpt3sas/mpt3sas_ctl.c b/drivers/scsi/mpt3sas/mpt3sas_ctl.c
index 9b89de1..ba9cbe5 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_ctl.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_ctl.c
@@ -980,7 +980,7 @@
mpt3sas_scsih_issue_tm(ioc,
le16_to_cpu(mpi_request->FunctionDependent1), 0, 0,
0, MPI2_SCSITASKMGMT_TASKTYPE_TARGET_RESET, 0, 30,
- 0, TM_MUTEX_ON);
+ TM_MUTEX_ON);
} else
mpt3sas_base_hard_reset_handler(ioc, CAN_SLEEP,
FORCE_BIG_HAMMER);
diff --git a/drivers/scsi/mpt3sas/mpt3sas_scsih.c b/drivers/scsi/mpt3sas/mpt3sas_scsih.c
index a961fe1..18e713d 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_scsih.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_scsih.c
@@ -2029,7 +2029,6 @@
* @type: MPI2_SCSITASKMGMT_TASKTYPE__XXX (defined in mpi2_init.h)
* @smid_task: smid assigned to the task
* @timeout: timeout in seconds
- * @serial_number: the serial_number from scmd
* @m_type: TM_MUTEX_ON or TM_MUTEX_OFF
* Context: user
*
@@ -2042,7 +2041,7 @@
int
mpt3sas_scsih_issue_tm(struct MPT3SAS_ADAPTER *ioc, u16 handle, uint channel,
uint id, uint lun, u8 type, u16 smid_task, ulong timeout,
- unsigned long serial_number, enum mutex_type m_type)
+ enum mutex_type m_type)
{
Mpi2SCSITaskManagementRequest_t *mpi_request;
Mpi2SCSITaskManagementReply_t *mpi_reply;
@@ -2293,8 +2292,7 @@
handle = sas_device_priv_data->sas_target->handle;
r = mpt3sas_scsih_issue_tm(ioc, handle, scmd->device->channel,
scmd->device->id, scmd->device->lun,
- MPI2_SCSITASKMGMT_TASKTYPE_ABORT_TASK, smid, 30,
- scmd->serial_number, TM_MUTEX_ON);
+ MPI2_SCSITASKMGMT_TASKTYPE_ABORT_TASK, smid, 30, TM_MUTEX_ON);
out:
sdev_printk(KERN_INFO, scmd->device, "task abort: %s scmd(%p)\n",
@@ -2353,8 +2351,7 @@
r = mpt3sas_scsih_issue_tm(ioc, handle, scmd->device->channel,
scmd->device->id, scmd->device->lun,
- MPI2_SCSITASKMGMT_TASKTYPE_LOGICAL_UNIT_RESET, 0, 30, 0,
- TM_MUTEX_ON);
+ MPI2_SCSITASKMGMT_TASKTYPE_LOGICAL_UNIT_RESET, 0, 30, TM_MUTEX_ON);
out:
sdev_printk(KERN_INFO, scmd->device, "device reset: %s scmd(%p)\n",
@@ -2414,7 +2411,7 @@
r = mpt3sas_scsih_issue_tm(ioc, handle, scmd->device->channel,
scmd->device->id, 0, MPI2_SCSITASKMGMT_TASKTYPE_TARGET_RESET, 0,
- 30, 0, TM_MUTEX_ON);
+ 30, TM_MUTEX_ON);
out:
starget_printk(KERN_INFO, starget, "target reset: %s scmd(%p)\n",
@@ -3518,7 +3515,7 @@
/**
- * _scsih_qcmd_lck - main scsi request entry point
+ * _scsih_qcmd - main scsi request entry point
* @scmd: pointer to scsi command object
* @done: function pointer to be invoked on completion
*
@@ -3529,9 +3526,9 @@
* SCSI_MLQUEUE_HOST_BUSY if the entire host queue is full
*/
static int
-_scsih_qcmd_lck(struct scsi_cmnd *scmd, void (*done)(struct scsi_cmnd *))
+_scsih_qcmd(struct Scsi_Host *shost, struct scsi_cmnd *scmd)
{
- struct MPT3SAS_ADAPTER *ioc = shost_priv(scmd->device->host);
+ struct MPT3SAS_ADAPTER *ioc = shost_priv(shost);
struct MPT3SAS_DEVICE *sas_device_priv_data;
struct MPT3SAS_TARGET *sas_target_priv_data;
Mpi2SCSIIORequest_t *mpi_request;
@@ -3544,7 +3541,6 @@
scsi_print_command(scmd);
#endif
- scmd->scsi_done = done;
sas_device_priv_data = scmd->device->hostdata;
if (!sas_device_priv_data || !sas_device_priv_data->sas_target) {
scmd->result = DID_NO_CONNECT << 16;
@@ -3659,8 +3655,6 @@
out:
return SCSI_MLQUEUE_HOST_BUSY;
}
-static DEF_SCSI_QCMD(_scsih_qcmd)
-
/**
* _scsih_normalize_sense - normalize descriptor and fixed format sense data
@@ -5425,7 +5419,7 @@
spin_unlock_irqrestore(&ioc->scsi_lookup_lock, flags);
r = mpt3sas_scsih_issue_tm(ioc, handle, 0, 0, lun,
- MPI2_SCSITASKMGMT_TASKTYPE_QUERY_TASK, smid, 30, 0,
+ MPI2_SCSITASKMGMT_TASKTYPE_QUERY_TASK, smid, 30,
TM_MUTEX_OFF);
if (r == FAILED) {
sdev_printk(KERN_WARNING, sdev,
@@ -5467,7 +5461,7 @@
r = mpt3sas_scsih_issue_tm(ioc, handle, sdev->channel, sdev->id,
sdev->lun, MPI2_SCSITASKMGMT_TASKTYPE_ABORT_TASK, smid, 30,
- scmd->serial_number, TM_MUTEX_OFF);
+ TM_MUTEX_OFF);
if (r == FAILED) {
sdev_printk(KERN_WARNING, sdev,
"mpt3sas_scsih_issue_tm: ABORT_TASK: FAILED : "
diff --git a/drivers/scsi/mvsas/mv_init.c b/drivers/scsi/mvsas/mv_init.c
index 5ff978b..eacee48 100644
--- a/drivers/scsi/mvsas/mv_init.c
+++ b/drivers/scsi/mvsas/mv_init.c
@@ -728,6 +728,15 @@
.class_mask = 0,
.driver_data = chip_9485,
},
+ {
+ .vendor = PCI_VENDOR_ID_MARVELL_EXT,
+ .device = 0x9485,
+ .subvendor = PCI_ANY_ID,
+ .subdevice = 0x9485,
+ .class = 0,
+ .class_mask = 0,
+ .driver_data = chip_9485,
+ },
{ PCI_VDEVICE(OCZ, 0x1021), chip_9485}, /* OCZ RevoDrive3 */
{ PCI_VDEVICE(OCZ, 0x1022), chip_9485}, /* OCZ RevoDrive3/zDriveR4 (exact model unknown) */
{ PCI_VDEVICE(OCZ, 0x1040), chip_9485}, /* OCZ RevoDrive3/zDriveR4 (exact model unknown) */
diff --git a/drivers/scsi/pas16.h b/drivers/scsi/pas16.h
index 3721342..aa528f5 100644
--- a/drivers/scsi/pas16.h
+++ b/drivers/scsi/pas16.h
@@ -129,8 +129,6 @@
#define CAN_QUEUE 32
#endif
-#ifndef HOSTS_C
-
#define NCR5380_implementation_fields \
volatile unsigned short io_port
@@ -171,6 +169,5 @@
#define PAS16_IRQS 0xd4a8
-#endif /* else def HOSTS_C */
#endif /* ndef ASM */
#endif /* PAS16_H */
diff --git a/drivers/scsi/pm8001/pm8001_ctl.c b/drivers/scsi/pm8001/pm8001_ctl.c
index 28b4e81..fe5eee4 100644
--- a/drivers/scsi/pm8001/pm8001_ctl.c
+++ b/drivers/scsi/pm8001/pm8001_ctl.c
@@ -395,6 +395,8 @@
payload.offset = 0;
payload.length = 4096;
payload.func_specific = kzalloc(4096, GFP_KERNEL);
+ if (!payload.func_specific)
+ return -ENOMEM;
PM8001_CHIP_DISP->get_nvmd_req(pm8001_ha, &payload);
wait_for_completion(&completion);
virt_addr = pm8001_ha->memoryMap.region[NVMD].virt_ptr;
@@ -402,6 +404,7 @@
bios_index++)
str += sprintf(str, "%c",
*((u8 *)((u8 *)virt_addr+bios_index)));
+ kfree(payload.func_specific);
return str - buf;
}
static DEVICE_ATTR(bios_version, S_IRUGO, pm8001_ctl_bios_version_show, NULL);
diff --git a/drivers/scsi/qla2xxx/qla_attr.c b/drivers/scsi/qla2xxx/qla_attr.c
index 07befcf..16fe519 100644
--- a/drivers/scsi/qla2xxx/qla_attr.c
+++ b/drivers/scsi/qla2xxx/qla_attr.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -664,7 +664,7 @@
}
rval = qla2x00_read_sfp(vha, ha->sfp_data_dma, ha->sfp_data,
- addr, offset, SFP_BLOCK_SIZE, 0);
+ addr, offset, SFP_BLOCK_SIZE, BIT_1);
if (rval != QLA_SUCCESS) {
ql_log(ql_log_warn, vha, 0x706d,
"Unable to read SFP data (%x/%x/%x).\n", rval,
@@ -1495,7 +1495,7 @@
if (!ha->fw_dumped)
size = 0;
- else if (IS_QLA82XX(ha))
+ else if (IS_P3P_TYPE(ha))
size = ha->md_template_size + ha->md_dump_size;
else
size = ha->fw_dump_len;
diff --git a/drivers/scsi/qla2xxx/qla_bsg.c b/drivers/scsi/qla2xxx/qla_bsg.c
index 71ff340..524f9eb 100644
--- a/drivers/scsi/qla2xxx/qla_bsg.c
+++ b/drivers/scsi/qla2xxx/qla_bsg.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2012 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -2054,9 +2054,49 @@
bsg_job->reply->reply_payload_rcv_len = sizeof(sr);
break;
default:
- ql_log(ql_log_warn, vha, 0x708c,
+ ql_dbg(ql_dbg_user, vha, 0x708c,
"Unknown serdes cmd %x.\n", sr.cmd);
- rval = -EDOM;
+ rval = -EINVAL;
+ break;
+ }
+
+ bsg_job->reply->reply_data.vendor_reply.vendor_rsp[0] =
+ rval ? EXT_STATUS_MAILBOX : 0;
+
+ bsg_job->reply_len = sizeof(struct fc_bsg_reply);
+ bsg_job->reply->result = DID_OK << 16;
+ bsg_job->job_done(bsg_job);
+ return 0;
+}
+
+static int
+qla8044_serdes_op(struct fc_bsg_job *bsg_job)
+{
+ struct Scsi_Host *host = bsg_job->shost;
+ scsi_qla_host_t *vha = shost_priv(host);
+ int rval = 0;
+ struct qla_serdes_reg_ex sr;
+
+ memset(&sr, 0, sizeof(sr));
+
+ sg_copy_to_buffer(bsg_job->request_payload.sg_list,
+ bsg_job->request_payload.sg_cnt, &sr, sizeof(sr));
+
+ switch (sr.cmd) {
+ case INT_SC_SERDES_WRITE_REG:
+ rval = qla8044_write_serdes_word(vha, sr.addr, sr.val);
+ bsg_job->reply->reply_payload_rcv_len = 0;
+ break;
+ case INT_SC_SERDES_READ_REG:
+ rval = qla8044_read_serdes_word(vha, sr.addr, &sr.val);
+ sg_copy_from_buffer(bsg_job->reply_payload.sg_list,
+ bsg_job->reply_payload.sg_cnt, &sr, sizeof(sr));
+ bsg_job->reply->reply_payload_rcv_len = sizeof(sr);
+ break;
+ default:
+ ql_dbg(ql_dbg_user, vha, 0x70cf,
+ "Unknown serdes cmd %x.\n", sr.cmd);
+ rval = -EINVAL;
break;
}
@@ -2121,6 +2161,9 @@
case QL_VND_SERDES_OP:
return qla26xx_serdes_op(bsg_job);
+ case QL_VND_SERDES_OP_EX:
+ return qla8044_serdes_op(bsg_job);
+
default:
return -ENOSYS;
}
diff --git a/drivers/scsi/qla2xxx/qla_bsg.h b/drivers/scsi/qla2xxx/qla_bsg.h
index e5c2126..d38f9ef 100644
--- a/drivers/scsi/qla2xxx/qla_bsg.h
+++ b/drivers/scsi/qla2xxx/qla_bsg.h
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -24,6 +24,7 @@
#define QL_VND_READ_I2C 0x11
#define QL_VND_FX00_MGMT_CMD 0x12
#define QL_VND_SERDES_OP 0x13
+#define QL_VND_SERDES_OP_EX 0x14
/* BSG Vendor specific subcode returns */
#define EXT_STATUS_OK 0
@@ -225,4 +226,10 @@
uint16_t val;
} __packed;
+struct qla_serdes_reg_ex {
+ uint16_t cmd;
+ uint32_t addr;
+ uint32_t val;
+} __packed;
+
#endif
diff --git a/drivers/scsi/qla2xxx/qla_dbg.c b/drivers/scsi/qla2xxx/qla_dbg.c
index 97255f7..c72ee97b 100644
--- a/drivers/scsi/qla2xxx/qla_dbg.c
+++ b/drivers/scsi/qla2xxx/qla_dbg.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -15,7 +15,7 @@
* | | | 0x0144,0x0146 |
* | | | 0x015b-0x0160 |
* | | | 0x016e-0x0170 |
- * | Mailbox commands | 0x1187 | 0x1018-0x1019 |
+ * | Mailbox commands | 0x118d | 0x1018-0x1019 |
* | | | 0x10ca |
* | | | 0x1115-0x1116 |
* | | | 0x111a-0x111b |
@@ -45,12 +45,16 @@
* | | | 0x70ad-0x70ae |
* | | | 0x70d7-0x70db |
* | | | 0x70de-0x70df |
- * | Task Management | 0x803d | 0x8025-0x8026 |
- * | | | 0x800b,0x8039 |
+ * | Task Management | 0x803d | 0x8000,0x800b |
+ * | | | 0x8019 |
+ * | | | 0x8025,0x8026 |
+ * | | | 0x8031,0x8032 |
+ * | | | 0x8039,0x803c |
* | AER/EEH | 0x9011 | |
* | Virtual Port | 0xa007 | |
- * | ISP82XX Specific | 0xb14c | 0xb002,0xb024 |
+ * | ISP82XX Specific | 0xb157 | 0xb002,0xb024 |
* | | | 0xb09e,0xb0ae |
+ * | | | 0xb0c3,0xb0c6 |
* | | | 0xb0e0-0xb0ef |
* | | | 0xb085,0xb0dc |
* | | | 0xb107,0xb108 |
@@ -60,12 +64,12 @@
* | | | 0xb13c-0xb140 |
* | | | 0xb149 |
* | MultiQ | 0xc00c | |
- * | Misc | 0xd2ff | 0xd017-0xd019 |
+ * | Misc | 0xd212 | 0xd017-0xd019 |
* | | | 0xd020 |
- * | | | 0xd02e-0xd0ff |
+ * | | | 0xd030-0xd0ff |
* | | | 0xd101-0xd1fe |
- * | | | 0xd212-0xd2fe |
- * | Target Mode | 0xe070 | 0xe021 |
+ * | | | 0xd213-0xd2fe |
+ * | Target Mode | 0xe078 | |
* | Target Mode Management | 0xf072 | 0xf002-0xf003 |
* | | | 0xf046-0xf049 |
* | Target Mode Task Management | 0x1000b | |
@@ -277,9 +281,15 @@
if (rval != QLA_SUCCESS)
return rval;
+ set_bit(RISC_SRAM_DUMP_CMPL, &ha->fw_dump_cap_flags);
+
/* External Memory. */
- return qla24xx_dump_ram(ha, 0x100000, *nxt,
+ rval = qla24xx_dump_ram(ha, 0x100000, *nxt,
ha->fw_memory_size - 0x100000 + 1, nxt);
+ if (rval == QLA_SUCCESS)
+ set_bit(RISC_EXT_MEM_DUMP_CMPL, &ha->fw_dump_cap_flags);
+
+ return rval;
}
static uint32_t *
@@ -296,23 +306,15 @@
return buf;
}
-int
-qla24xx_pause_risc(struct device_reg_24xx __iomem *reg)
+void
+qla24xx_pause_risc(struct device_reg_24xx __iomem *reg, struct qla_hw_data *ha)
{
- int rval = QLA_SUCCESS;
- uint32_t cnt;
-
WRT_REG_DWORD(®->hccr, HCCRX_SET_RISC_PAUSE);
- for (cnt = 30000;
- ((RD_REG_DWORD(®->host_status) & HSRX_RISC_PAUSED) == 0) &&
- rval == QLA_SUCCESS; cnt--) {
- if (cnt)
- udelay(100);
- else
- rval = QLA_FUNCTION_TIMEOUT;
- }
- return rval;
+ /* 100 usec delay is sufficient enough for hardware to pause RISC */
+ udelay(100);
+ if (RD_REG_DWORD(®->host_status) & HSRX_RISC_PAUSED)
+ set_bit(RISC_PAUSE_CMPL, &ha->fw_dump_cap_flags);
}
int
@@ -320,10 +322,14 @@
{
int rval = QLA_SUCCESS;
uint32_t cnt;
- uint16_t mb0, wd;
+ uint16_t wd;
struct device_reg_24xx __iomem *reg = &ha->iobase->isp24;
- /* Reset RISC. */
+ /*
+ * Reset RISC. The delay is dependent on system architecture.
+ * Driver can proceed with the reset sequence after waiting
+ * for a timeout period.
+ */
WRT_REG_DWORD(®->ctrl_status, CSRX_DMA_SHUTDOWN|MWB_4096_BYTES);
for (cnt = 0; cnt < 30000; cnt++) {
if ((RD_REG_DWORD(®->ctrl_status) & CSRX_DMA_ACTIVE) == 0)
@@ -331,19 +337,14 @@
udelay(10);
}
+ if (!(RD_REG_DWORD(®->ctrl_status) & CSRX_DMA_ACTIVE))
+ set_bit(DMA_SHUTDOWN_CMPL, &ha->fw_dump_cap_flags);
WRT_REG_DWORD(®->ctrl_status,
CSRX_ISP_SOFT_RESET|CSRX_DMA_SHUTDOWN|MWB_4096_BYTES);
pci_read_config_word(ha->pdev, PCI_COMMAND, &wd);
udelay(100);
- /* Wait for firmware to complete NVRAM accesses. */
- mb0 = (uint32_t) RD_REG_WORD(®->mailbox0);
- for (cnt = 10000 ; cnt && mb0; cnt--) {
- udelay(5);
- mb0 = (uint32_t) RD_REG_WORD(®->mailbox0);
- barrier();
- }
/* Wait for soft-reset to complete. */
for (cnt = 0; cnt < 30000; cnt++) {
@@ -353,16 +354,21 @@
udelay(10);
}
+ if (!(RD_REG_DWORD(®->ctrl_status) & CSRX_ISP_SOFT_RESET))
+ set_bit(ISP_RESET_CMPL, &ha->fw_dump_cap_flags);
+
WRT_REG_DWORD(®->hccr, HCCRX_CLR_RISC_RESET);
RD_REG_DWORD(®->hccr); /* PCI Posting. */
- for (cnt = 30000; RD_REG_WORD(®->mailbox0) != 0 &&
+ for (cnt = 10000; RD_REG_WORD(®->mailbox0) != 0 &&
rval == QLA_SUCCESS; cnt--) {
if (cnt)
- udelay(100);
+ udelay(10);
else
rval = QLA_FUNCTION_TIMEOUT;
}
+ if (rval == QLA_SUCCESS)
+ set_bit(RISC_RDY_AFT_RESET, &ha->fw_dump_cap_flags);
return rval;
}
@@ -659,12 +665,13 @@
if (rval != QLA_SUCCESS) {
ql_log(ql_log_warn, vha, 0xd000,
- "Failed to dump firmware (%x).\n", rval);
+ "Failed to dump firmware (%x), dump status flags (0x%lx).\n",
+ rval, ha->fw_dump_cap_flags);
ha->fw_dumped = 0;
} else {
ql_log(ql_log_info, vha, 0xd001,
- "Firmware dump saved to temp buffer (%ld/%p).\n",
- vha->host_no, ha->fw_dump);
+ "Firmware dump saved to temp buffer (%ld/%p), dump status flags (0x%lx).\n",
+ vha->host_no, ha->fw_dump, ha->fw_dump_cap_flags);
ha->fw_dumped = 1;
qla2x00_post_uevent_work(vha, QLA_UEVENT_CODE_FW_DUMP);
}
@@ -1053,6 +1060,7 @@
risc_address = ext_mem_cnt = 0;
flags = 0;
+ ha->fw_dump_cap_flags = 0;
if (!hardware_locked)
spin_lock_irqsave(&ha->hardware_lock, flags);
@@ -1075,10 +1083,11 @@
fw->host_status = htonl(RD_REG_DWORD(®->host_status));
- /* Pause RISC. */
- rval = qla24xx_pause_risc(reg);
- if (rval != QLA_SUCCESS)
- goto qla24xx_fw_dump_failed_0;
+ /*
+ * Pause RISC. No need to track timeout, as resetting the chip
+ * is the right approach incase of pause timeout
+ */
+ qla24xx_pause_risc(reg, ha);
/* Host interface registers. */
dmp_reg = ®->flash_addr;
@@ -1302,6 +1311,7 @@
risc_address = ext_mem_cnt = 0;
flags = 0;
+ ha->fw_dump_cap_flags = 0;
if (!hardware_locked)
spin_lock_irqsave(&ha->hardware_lock, flags);
@@ -1325,10 +1335,11 @@
fw->host_status = htonl(RD_REG_DWORD(®->host_status));
- /* Pause RISC. */
- rval = qla24xx_pause_risc(reg);
- if (rval != QLA_SUCCESS)
- goto qla25xx_fw_dump_failed_0;
+ /*
+ * Pause RISC. No need to track timeout, as resetting the chip
+ * is the right approach incase of pause timeout
+ */
+ qla24xx_pause_risc(reg, ha);
/* Host/Risc registers. */
iter_reg = fw->host_risc_reg;
@@ -1619,6 +1630,7 @@
risc_address = ext_mem_cnt = 0;
flags = 0;
+ ha->fw_dump_cap_flags = 0;
if (!hardware_locked)
spin_lock_irqsave(&ha->hardware_lock, flags);
@@ -1641,10 +1653,11 @@
fw->host_status = htonl(RD_REG_DWORD(®->host_status));
- /* Pause RISC. */
- rval = qla24xx_pause_risc(reg);
- if (rval != QLA_SUCCESS)
- goto qla81xx_fw_dump_failed_0;
+ /*
+ * Pause RISC. No need to track timeout, as resetting the chip
+ * is the right approach incase of pause timeout
+ */
+ qla24xx_pause_risc(reg, ha);
/* Host/Risc registers. */
iter_reg = fw->host_risc_reg;
@@ -1938,6 +1951,7 @@
risc_address = ext_mem_cnt = 0;
flags = 0;
+ ha->fw_dump_cap_flags = 0;
if (!hardware_locked)
spin_lock_irqsave(&ha->hardware_lock, flags);
@@ -1959,10 +1973,11 @@
fw->host_status = htonl(RD_REG_DWORD(®->host_status));
- /* Pause RISC. */
- rval = qla24xx_pause_risc(reg);
- if (rval != QLA_SUCCESS)
- goto qla83xx_fw_dump_failed_0;
+ /*
+ * Pause RISC. No need to track timeout, as resetting the chip
+ * is the right approach incase of pause timeout
+ */
+ qla24xx_pause_risc(reg, ha);
WRT_REG_DWORD(®->iobase_addr, 0x6000);
dmp_reg = ®->iobase_window;
@@ -2385,9 +2400,11 @@
nxt += sizeof(fw->code_ram);
nxt += (ha->fw_memory_size - 0x100000 + 1);
goto copy_queue;
- } else
+ } else {
+ set_bit(RISC_RDY_AFT_RESET, &ha->fw_dump_cap_flags);
ql_log(ql_log_warn, vha, 0xd010,
"bigger hammer success?\n");
+ }
}
rval = qla24xx_dump_memory(ha, fw->code_ram, sizeof(fw->code_ram),
diff --git a/drivers/scsi/qla2xxx/qla_dbg.h b/drivers/scsi/qla2xxx/qla_dbg.h
index cc96104..e1fc4e6 100644
--- a/drivers/scsi/qla2xxx/qla_dbg.h
+++ b/drivers/scsi/qla2xxx/qla_dbg.h
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -353,5 +353,6 @@
uint32_t, void **);
extern int qla24xx_dump_ram(struct qla_hw_data *, uint32_t, uint32_t *,
uint32_t, void **);
-extern int qla24xx_pause_risc(struct device_reg_24xx __iomem *);
+extern void qla24xx_pause_risc(struct device_reg_24xx __iomem *,
+ struct qla_hw_data *);
extern int qla24xx_soft_reset(struct qla_hw_data *);
diff --git a/drivers/scsi/qla2xxx/qla_def.h b/drivers/scsi/qla2xxx/qla_def.h
index 6a10613..1fa0104 100644
--- a/drivers/scsi/qla2xxx/qla_def.h
+++ b/drivers/scsi/qla2xxx/qla_def.h
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -965,6 +965,13 @@
*/
#define MBC_WRITE_MPI_REGISTER 0x01 /* Write MPI Register. */
+/*
+ * ISP8044 mailbox commands
+ */
+#define MBC_SET_GET_ETH_SERDES_REG 0x150
+#define HCS_WRITE_SERDES 0x3
+#define HCS_READ_SERDES 0x4
+
/* Firmware return data sizes */
#define FCAL_MAP_SIZE 128
@@ -1622,10 +1629,20 @@
#define PO_MODE_DIF_PASS 2
#define PO_MODE_DIF_REPLACE 3
#define PO_MODE_DIF_TCP_CKSUM 6
-#define PO_ENABLE_DIF_BUNDLING BIT_8
#define PO_ENABLE_INCR_GUARD_SEED BIT_3
-#define PO_DISABLE_INCR_REF_TAG BIT_5
#define PO_DISABLE_GUARD_CHECK BIT_4
+#define PO_DISABLE_INCR_REF_TAG BIT_5
+#define PO_DIS_HEADER_MODE BIT_7
+#define PO_ENABLE_DIF_BUNDLING BIT_8
+#define PO_DIS_FRAME_MODE BIT_9
+#define PO_DIS_VALD_APP_ESC BIT_10 /* Dis validation for escape tag/ffffh */
+#define PO_DIS_VALD_APP_REF_ESC BIT_11
+
+#define PO_DIS_APP_TAG_REPL BIT_12 /* disable REG Tag replacement */
+#define PO_DIS_REF_TAG_REPL BIT_13
+#define PO_DIS_APP_TAG_VALD BIT_14 /* disable REF Tag validation */
+#define PO_DIS_REF_TAG_VALD BIT_15
+
/*
* ISP queue - 64-Bit addressing, continuation crc entry structure definition.
*/
@@ -1748,6 +1765,8 @@
#define CS_PORT_CONFIG_CHG 0x2A /* Port Configuration Changed */
#define CS_PORT_BUSY 0x2B /* Port Busy */
#define CS_COMPLETE_CHKCOND 0x30 /* Error? */
+#define CS_IOCB_ERROR 0x31 /* Generic error for IOCB request
+ failure */
#define CS_BAD_PAYLOAD 0x80 /* Driver defined */
#define CS_UNKNOWN 0x81 /* Driver defined */
#define CS_RETRY 0x82 /* Driver defined */
@@ -2676,6 +2695,7 @@
uint32_t __iomem *rsp_q_out;
uint16_t ring_index;
uint16_t out_ptr;
+ uint16_t *in_ptr; /* queue shadow in index */
uint16_t length;
uint16_t options;
uint16_t rid;
@@ -2702,6 +2722,7 @@
uint32_t __iomem *req_q_out;
uint16_t ring_index;
uint16_t in_ptr;
+ uint16_t *out_ptr; /* queue shadow out index */
uint16_t cnt;
uint16_t length;
uint16_t options;
@@ -2907,6 +2928,8 @@
#define PCI_DEVICE_ID_QLOGIC_ISP8031 0x8031
#define PCI_DEVICE_ID_QLOGIC_ISP2031 0x2031
#define PCI_DEVICE_ID_QLOGIC_ISP2071 0x2071
+#define PCI_DEVICE_ID_QLOGIC_ISP2271 0x2271
+
uint32_t device_type;
#define DT_ISP2100 BIT_0
#define DT_ISP2200 BIT_1
@@ -2928,7 +2951,8 @@
#define DT_ISPFX00 BIT_17
#define DT_ISP8044 BIT_18
#define DT_ISP2071 BIT_19
-#define DT_ISP_LAST (DT_ISP2071 << 1)
+#define DT_ISP2271 BIT_20
+#define DT_ISP_LAST (DT_ISP2271 << 1)
#define DT_T10_PI BIT_25
#define DT_IIDMA BIT_26
@@ -2959,6 +2983,7 @@
#define IS_QLA8031(ha) (DT_MASK(ha) & DT_ISP8031)
#define IS_QLAFX00(ha) (DT_MASK(ha) & DT_ISPFX00)
#define IS_QLA2071(ha) (DT_MASK(ha) & DT_ISP2071)
+#define IS_QLA2271(ha) (DT_MASK(ha) & DT_ISP2271)
#define IS_QLA23XX(ha) (IS_QLA2300(ha) || IS_QLA2312(ha) || IS_QLA2322(ha) || \
IS_QLA6312(ha) || IS_QLA6322(ha))
@@ -2967,7 +2992,7 @@
#define IS_QLA25XX(ha) (IS_QLA2532(ha))
#define IS_QLA83XX(ha) (IS_QLA2031(ha) || IS_QLA8031(ha))
#define IS_QLA84XX(ha) (IS_QLA8432(ha))
-#define IS_QLA27XX(ha) (IS_QLA2071(ha))
+#define IS_QLA27XX(ha) (IS_QLA2071(ha) || IS_QLA2271(ha))
#define IS_QLA24XX_TYPE(ha) (IS_QLA24XX(ha) || IS_QLA54XX(ha) || \
IS_QLA84XX(ha))
#define IS_CNA_CAPABLE(ha) (IS_QLA81XX(ha) || IS_QLA82XX(ha) || \
@@ -3006,6 +3031,7 @@
(((ha)->fw_attributes_h << 16 | (ha)->fw_attributes) & BIT_22))
#define IS_ATIO_MSIX_CAPABLE(ha) (IS_QLA83XX(ha))
#define IS_TGT_MODE_CAPABLE(ha) (ha->tgt.atio_q_length)
+#define IS_SHADOW_REG_CAPABLE(ha) (IS_QLA27XX(ha))
/* HBA serial number */
uint8_t serial0;
@@ -3136,7 +3162,15 @@
struct qla2xxx_fw_dump *fw_dump;
uint32_t fw_dump_len;
int fw_dumped;
+ unsigned long fw_dump_cap_flags;
+#define RISC_PAUSE_CMPL 0
+#define DMA_SHUTDOWN_CMPL 1
+#define ISP_RESET_CMPL 2
+#define RISC_RDY_AFT_RESET 3
+#define RISC_SRAM_DUMP_CMPL 4
+#define RISC_EXT_MEM_DUMP_CMPL 5
int fw_dump_reading;
+ int prev_minidump_failed;
dma_addr_t eft_dma;
void *eft;
/* Current size of mctp dump is 0x086064 bytes */
diff --git a/drivers/scsi/qla2xxx/qla_dfs.c b/drivers/scsi/qla2xxx/qla_dfs.c
index 32ab809..2ca39b8 100644
--- a/drivers/scsi/qla2xxx/qla_dfs.c
+++ b/drivers/scsi/qla2xxx/qla_dfs.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
diff --git a/drivers/scsi/qla2xxx/qla_fw.h b/drivers/scsi/qla2xxx/qla_fw.h
index 3a7353e..eb8f572 100644
--- a/drivers/scsi/qla2xxx/qla_fw.h
+++ b/drivers/scsi/qla2xxx/qla_fw.h
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -371,7 +371,10 @@
* BIT 14 = Data Rate bit 1
* BIT 15 = Data Rate bit 2
* BIT 16 = Enable 75 ohm Termination Select
- * BIT 17-31 = Reserved
+ * BIT 17-28 = Reserved
+ * BIT 29 = Enable response queue 0 in index shadowing
+ * BIT 30 = Enable request queue 0 out index shadowing
+ * BIT 31 = Reserved
*/
uint32_t firmware_options_3;
uint16_t qos;
@@ -1134,13 +1137,6 @@
#define MIN_MULTI_ID_FABRIC 64 /* Must be power-of-2. */
#define MAX_MULTI_ID_FABRIC 256 /* ... */
-#define for_each_mapped_vp_idx(_ha, _idx) \
- for (_idx = find_next_bit((_ha)->vp_idx_map, \
- (_ha)->max_npiv_vports + 1, 1); \
- _idx <= (_ha)->max_npiv_vports; \
- _idx = find_next_bit((_ha)->vp_idx_map, \
- (_ha)->max_npiv_vports + 1, _idx + 1)) \
-
struct mid_conf_entry_24xx {
uint16_t reserved_1;
diff --git a/drivers/scsi/qla2xxx/qla_gbl.h b/drivers/scsi/qla2xxx/qla_gbl.h
index e665e81..d48dea8 100644
--- a/drivers/scsi/qla2xxx/qla_gbl.h
+++ b/drivers/scsi/qla2xxx/qla_gbl.h
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -220,6 +220,13 @@
extern void *qla2x00_alloc_iocbs(scsi_qla_host_t *, srb_t *);
extern int qla2x00_issue_marker(scsi_qla_host_t *, int);
+extern int qla24xx_walk_and_build_sglist_no_difb(struct qla_hw_data *, srb_t *,
+ uint32_t *, uint16_t, struct qla_tgt_cmd *);
+extern int qla24xx_walk_and_build_sglist(struct qla_hw_data *, srb_t *,
+ uint32_t *, uint16_t, struct qla_tgt_cmd *);
+extern int qla24xx_walk_and_build_prot_sglist(struct qla_hw_data *, srb_t *,
+ uint32_t *, uint16_t, struct qla_tgt_cmd *);
+
/*
* Global Function Prototypes in qla_mbx.c source file.
@@ -347,6 +354,11 @@
qla2x00_read_serdes_word(scsi_qla_host_t *, uint16_t, uint16_t *);
extern int
+qla8044_write_serdes_word(scsi_qla_host_t *, uint32_t, uint32_t);
+extern int
+qla8044_read_serdes_word(scsi_qla_host_t *, uint32_t, uint32_t *);
+
+extern int
qla2x00_set_serdes_params(scsi_qla_host_t *, uint16_t, uint16_t, uint16_t);
extern int
diff --git a/drivers/scsi/qla2xxx/qla_gs.c b/drivers/scsi/qla2xxx/qla_gs.c
index e377f9d2..a0df3b1 100644
--- a/drivers/scsi/qla2xxx/qla_gs.c
+++ b/drivers/scsi/qla2xxx/qla_gs.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
diff --git a/drivers/scsi/qla2xxx/qla_init.c b/drivers/scsi/qla2xxx/qla_init.c
index 38aeb54..e218441 100644
--- a/drivers/scsi/qla2xxx/qla_init.c
+++ b/drivers/scsi/qla2xxx/qla_init.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -1476,6 +1476,7 @@
}
ha->fw_dumped = 0;
+ ha->fw_dump_cap_flags = 0;
dump_size = fixed_size = mem_size = eft_size = fce_size = mq_size = 0;
req_q_size = rsp_q_size = 0;
@@ -2061,6 +2062,10 @@
icb->atio_q_address[0] = cpu_to_le32(LSD(ha->tgt.atio_dma));
icb->atio_q_address[1] = cpu_to_le32(MSD(ha->tgt.atio_dma));
+ if (IS_SHADOW_REG_CAPABLE(ha))
+ icb->firmware_options_2 |=
+ __constant_cpu_to_le32(BIT_30|BIT_29);
+
if (ha->mqenable || IS_QLA83XX(ha) || IS_QLA27XX(ha)) {
icb->qos = __constant_cpu_to_le16(QLA_DEFAULT_QUE_QOS);
icb->rid = __constant_cpu_to_le16(rid);
@@ -2138,6 +2143,8 @@
req = ha->req_q_map[que];
if (!req)
continue;
+ req->out_ptr = (void *)(req->ring + req->length);
+ *req->out_ptr = 0;
for (cnt = 1; cnt < req->num_outstanding_cmds; cnt++)
req->outstanding_cmds[cnt] = NULL;
@@ -2153,6 +2160,8 @@
rsp = ha->rsp_q_map[que];
if (!rsp)
continue;
+ rsp->in_ptr = (void *)(rsp->ring + rsp->length);
+ *rsp->in_ptr = 0;
/* Initialize response queue entries */
if (IS_QLAFX00(ha))
qlafx00_init_response_q_entries(rsp);
@@ -3406,7 +3415,7 @@
fcport->d_id.b.domain,
fcport->d_id.b.area,
fcport->d_id.b.al_pa);
- fcport->loop_id = FC_NO_LOOP_ID;
+ qla2x00_clear_loop_id(fcport);
}
}
}
@@ -4727,7 +4736,6 @@
qla2x00_restart_isp(scsi_qla_host_t *vha)
{
int status = 0;
- uint32_t wait_time;
struct qla_hw_data *ha = vha->hw;
struct req_que *req = ha->req_q_map[0];
struct rsp_que *rsp = ha->rsp_q_map[0];
@@ -4744,14 +4752,12 @@
if (!status && !(status = qla2x00_init_rings(vha))) {
clear_bit(RESET_MARKER_NEEDED, &vha->dpc_flags);
ha->flags.chip_reset_done = 1;
+
/* Initialize the queues in use */
qla25xx_init_queues(ha);
status = qla2x00_fw_ready(vha);
if (!status) {
- ql_dbg(ql_dbg_taskm, vha, 0x8031,
- "Start configure loop status = %d.\n", status);
-
/* Issue a marker after FW becomes ready. */
qla2x00_marker(vha, req, rsp, 0, 0, MK_SYNC_ALL);
@@ -4766,24 +4772,12 @@
qlt_24xx_process_atio_queue(vha);
spin_unlock_irqrestore(&ha->hardware_lock, flags);
- /* Wait at most MAX_TARGET RSCNs for a stable link. */
- wait_time = 256;
- do {
- clear_bit(LOOP_RESYNC_NEEDED, &vha->dpc_flags);
- qla2x00_configure_loop(vha);
- wait_time--;
- } while (!atomic_read(&vha->loop_down_timer) &&
- !(test_bit(ISP_ABORT_NEEDED, &vha->dpc_flags))
- && wait_time && (test_bit(LOOP_RESYNC_NEEDED,
- &vha->dpc_flags)));
+ set_bit(LOOP_RESYNC_NEEDED, &vha->dpc_flags);
}
/* if no cable then assume it's good */
if ((vha->device_flags & DFLG_NO_CABLE))
status = 0;
-
- ql_dbg(ql_dbg_taskm, vha, 0x8032,
- "Configure loop done, status = 0x%x.\n", status);
}
return (status);
}
@@ -6130,7 +6124,6 @@
qla82xx_restart_isp(scsi_qla_host_t *vha)
{
int status, rval;
- uint32_t wait_time;
struct qla_hw_data *ha = vha->hw;
struct req_que *req = ha->req_q_map[0];
struct rsp_que *rsp = ha->rsp_q_map[0];
@@ -6144,31 +6137,15 @@
status = qla2x00_fw_ready(vha);
if (!status) {
- ql_log(ql_log_info, vha, 0x803c,
- "Start configure loop, status =%d.\n", status);
-
/* Issue a marker after FW becomes ready. */
qla2x00_marker(vha, req, rsp, 0, 0, MK_SYNC_ALL);
-
vha->flags.online = 1;
- /* Wait at most MAX_TARGET RSCNs for a stable link. */
- wait_time = 256;
- do {
- clear_bit(LOOP_RESYNC_NEEDED, &vha->dpc_flags);
- qla2x00_configure_loop(vha);
- wait_time--;
- } while (!atomic_read(&vha->loop_down_timer) &&
- !(test_bit(ISP_ABORT_NEEDED, &vha->dpc_flags)) &&
- wait_time &&
- (test_bit(LOOP_RESYNC_NEEDED, &vha->dpc_flags)));
+ set_bit(LOOP_RESYNC_NEEDED, &vha->dpc_flags);
}
/* if no cable then assume it's good */
if ((vha->device_flags & DFLG_NO_CABLE))
status = 0;
-
- ql_log(ql_log_info, vha, 0x8000,
- "Configure loop done, status = 0x%x.\n", status);
}
if (!status) {
@@ -6182,8 +6159,6 @@
vha->marker_needed = 1;
}
- vha->flags.online = 1;
-
ha->isp_ops->enable_intrs(ha);
ha->isp_abort_cnt = 0;
diff --git a/drivers/scsi/qla2xxx/qla_inline.h b/drivers/scsi/qla2xxx/qla_inline.h
index ce8b5fb..b3b1d6f 100644
--- a/drivers/scsi/qla2xxx/qla_inline.h
+++ b/drivers/scsi/qla2xxx/qla_inline.h
@@ -1,10 +1,11 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
+#include "qla_target.h"
/**
* qla24xx_calc_iocbs() - Determine number of Command Type 3 and
* Continuation Type 1 IOCBs to allocate.
@@ -128,12 +129,20 @@
}
static inline void
-qla2x00_clean_dsd_pool(struct qla_hw_data *ha, srb_t *sp)
+qla2x00_clean_dsd_pool(struct qla_hw_data *ha, srb_t *sp,
+ struct qla_tgt_cmd *tc)
{
struct dsd_dma *dsd_ptr, *tdsd_ptr;
struct crc_context *ctx;
- ctx = (struct crc_context *)GET_CMD_CTX_SP(sp);
+ if (sp)
+ ctx = (struct crc_context *)GET_CMD_CTX_SP(sp);
+ else if (tc)
+ ctx = (struct crc_context *)tc->ctx;
+ else {
+ BUG();
+ return;
+ }
/* clean up allocated prev pool */
list_for_each_entry_safe(dsd_ptr, tdsd_ptr,
diff --git a/drivers/scsi/qla2xxx/qla_iocb.c b/drivers/scsi/qla2xxx/qla_iocb.c
index e607568..7609315 100644
--- a/drivers/scsi/qla2xxx/qla_iocb.c
+++ b/drivers/scsi/qla2xxx/qla_iocb.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -936,9 +936,9 @@
return 1;
}
-static int
+int
qla24xx_walk_and_build_sglist_no_difb(struct qla_hw_data *ha, srb_t *sp,
- uint32_t *dsd, uint16_t tot_dsds)
+ uint32_t *dsd, uint16_t tot_dsds, struct qla_tgt_cmd *tc)
{
void *next_dsd;
uint8_t avail_dsds = 0;
@@ -948,21 +948,35 @@
uint32_t *cur_dsd = dsd;
uint16_t used_dsds = tot_dsds;
- uint32_t prot_int;
+ uint32_t prot_int; /* protection interval */
uint32_t partial;
struct qla2_sgx sgx;
dma_addr_t sle_dma;
uint32_t sle_dma_len, tot_prot_dma_len = 0;
- struct scsi_cmnd *cmd = GET_CMD_SP(sp);
-
- prot_int = cmd->device->sector_size;
+ struct scsi_cmnd *cmd;
+ struct scsi_qla_host *vha;
memset(&sgx, 0, sizeof(struct qla2_sgx));
- sgx.tot_bytes = scsi_bufflen(cmd);
- sgx.cur_sg = scsi_sglist(cmd);
- sgx.sp = sp;
+ if (sp) {
+ vha = sp->fcport->vha;
+ cmd = GET_CMD_SP(sp);
+ prot_int = cmd->device->sector_size;
- sg_prot = scsi_prot_sglist(cmd);
+ sgx.tot_bytes = scsi_bufflen(cmd);
+ sgx.cur_sg = scsi_sglist(cmd);
+ sgx.sp = sp;
+
+ sg_prot = scsi_prot_sglist(cmd);
+ } else if (tc) {
+ vha = tc->vha;
+ prot_int = tc->blk_sz;
+ sgx.tot_bytes = tc->bufflen;
+ sgx.cur_sg = tc->sg;
+ sg_prot = tc->prot_sg;
+ } else {
+ BUG();
+ return 1;
+ }
while (qla24xx_get_one_block_sg(prot_int, &sgx, &partial)) {
@@ -995,10 +1009,18 @@
return 1;
}
- list_add_tail(&dsd_ptr->list,
- &((struct crc_context *)sp->u.scmd.ctx)->dsd_list);
+ if (sp) {
+ list_add_tail(&dsd_ptr->list,
+ &((struct crc_context *)
+ sp->u.scmd.ctx)->dsd_list);
- sp->flags |= SRB_CRC_CTX_DSD_VALID;
+ sp->flags |= SRB_CRC_CTX_DSD_VALID;
+ } else {
+ list_add_tail(&dsd_ptr->list,
+ &(tc->ctx->dsd_list));
+ tc->ctx_dsd_alloced = 1;
+ }
+
/* add new list to cmd iocb or last list */
*cur_dsd++ = cpu_to_le32(LSD(dsd_ptr->dsd_list_dma));
@@ -1033,21 +1055,35 @@
return 0;
}
-static int
+int
qla24xx_walk_and_build_sglist(struct qla_hw_data *ha, srb_t *sp, uint32_t *dsd,
- uint16_t tot_dsds)
+ uint16_t tot_dsds, struct qla_tgt_cmd *tc)
{
void *next_dsd;
uint8_t avail_dsds = 0;
uint32_t dsd_list_len;
struct dsd_dma *dsd_ptr;
- struct scatterlist *sg;
+ struct scatterlist *sg, *sgl;
uint32_t *cur_dsd = dsd;
int i;
uint16_t used_dsds = tot_dsds;
- struct scsi_cmnd *cmd = GET_CMD_SP(sp);
+ struct scsi_cmnd *cmd;
+ struct scsi_qla_host *vha;
- scsi_for_each_sg(cmd, sg, tot_dsds, i) {
+ if (sp) {
+ cmd = GET_CMD_SP(sp);
+ sgl = scsi_sglist(cmd);
+ vha = sp->fcport->vha;
+ } else if (tc) {
+ sgl = tc->sg;
+ vha = tc->vha;
+ } else {
+ BUG();
+ return 1;
+ }
+
+
+ for_each_sg(sgl, sg, tot_dsds, i) {
dma_addr_t sle_dma;
/* Allocate additional continuation packets? */
@@ -1076,10 +1112,17 @@
return 1;
}
- list_add_tail(&dsd_ptr->list,
- &((struct crc_context *)sp->u.scmd.ctx)->dsd_list);
+ if (sp) {
+ list_add_tail(&dsd_ptr->list,
+ &((struct crc_context *)
+ sp->u.scmd.ctx)->dsd_list);
- sp->flags |= SRB_CRC_CTX_DSD_VALID;
+ sp->flags |= SRB_CRC_CTX_DSD_VALID;
+ } else {
+ list_add_tail(&dsd_ptr->list,
+ &(tc->ctx->dsd_list));
+ tc->ctx_dsd_alloced = 1;
+ }
/* add new list to cmd iocb or last list */
*cur_dsd++ = cpu_to_le32(LSD(dsd_ptr->dsd_list_dma));
@@ -1102,23 +1145,37 @@
return 0;
}
-static int
+int
qla24xx_walk_and_build_prot_sglist(struct qla_hw_data *ha, srb_t *sp,
- uint32_t *dsd,
- uint16_t tot_dsds)
+ uint32_t *dsd, uint16_t tot_dsds, struct qla_tgt_cmd *tc)
{
void *next_dsd;
uint8_t avail_dsds = 0;
uint32_t dsd_list_len;
struct dsd_dma *dsd_ptr;
- struct scatterlist *sg;
+ struct scatterlist *sg, *sgl;
int i;
struct scsi_cmnd *cmd;
uint32_t *cur_dsd = dsd;
- uint16_t used_dsds = tot_dsds;
+ uint16_t used_dsds = tot_dsds;
+ struct scsi_qla_host *vha;
- cmd = GET_CMD_SP(sp);
- scsi_for_each_prot_sg(cmd, sg, tot_dsds, i) {
+ if (sp) {
+ cmd = GET_CMD_SP(sp);
+ sgl = scsi_prot_sglist(cmd);
+ vha = sp->fcport->vha;
+ } else if (tc) {
+ vha = tc->vha;
+ sgl = tc->prot_sg;
+ } else {
+ BUG();
+ return 1;
+ }
+
+ ql_dbg(ql_dbg_tgt, vha, 0xe021,
+ "%s: enter\n", __func__);
+
+ for_each_sg(sgl, sg, tot_dsds, i) {
dma_addr_t sle_dma;
/* Allocate additional continuation packets? */
@@ -1147,10 +1204,17 @@
return 1;
}
- list_add_tail(&dsd_ptr->list,
- &((struct crc_context *)sp->u.scmd.ctx)->dsd_list);
+ if (sp) {
+ list_add_tail(&dsd_ptr->list,
+ &((struct crc_context *)
+ sp->u.scmd.ctx)->dsd_list);
- sp->flags |= SRB_CRC_CTX_DSD_VALID;
+ sp->flags |= SRB_CRC_CTX_DSD_VALID;
+ } else {
+ list_add_tail(&dsd_ptr->list,
+ &(tc->ctx->dsd_list));
+ tc->ctx_dsd_alloced = 1;
+ }
/* add new list to cmd iocb or last list */
*cur_dsd++ = cpu_to_le32(LSD(dsd_ptr->dsd_list_dma));
@@ -1386,10 +1450,10 @@
if (!bundling && tot_prot_dsds) {
if (qla24xx_walk_and_build_sglist_no_difb(ha, sp,
- cur_dsd, tot_dsds))
+ cur_dsd, tot_dsds, NULL))
goto crc_queuing_error;
} else if (qla24xx_walk_and_build_sglist(ha, sp, cur_dsd,
- (tot_dsds - tot_prot_dsds)))
+ (tot_dsds - tot_prot_dsds), NULL))
goto crc_queuing_error;
if (bundling && tot_prot_dsds) {
@@ -1398,7 +1462,7 @@
__constant_cpu_to_le16(CF_DIF_SEG_DESCR_ENABLE);
cur_dsd = (uint32_t *) &crc_ctx_pkt->u.bundling.dif_address;
if (qla24xx_walk_and_build_prot_sglist(ha, sp, cur_dsd,
- tot_prot_dsds))
+ tot_prot_dsds, NULL))
goto crc_queuing_error;
}
return QLA_SUCCESS;
@@ -1478,8 +1542,8 @@
tot_dsds = nseg;
req_cnt = qla24xx_calc_iocbs(vha, tot_dsds);
if (req->cnt < (req_cnt + 2)) {
- cnt = RD_REG_DWORD_RELAXED(req->req_q_out);
-
+ cnt = IS_SHADOW_REG_CAPABLE(ha) ? *req->out_ptr :
+ RD_REG_DWORD_RELAXED(req->req_q_out);
if (req->ring_index < cnt)
req->cnt = cnt - req->ring_index;
else
@@ -1697,8 +1761,8 @@
tot_prot_dsds = nseg;
tot_dsds += nseg;
if (req->cnt < (req_cnt + 2)) {
- cnt = RD_REG_DWORD_RELAXED(req->req_q_out);
-
+ cnt = IS_SHADOW_REG_CAPABLE(ha) ? *req->out_ptr :
+ RD_REG_DWORD_RELAXED(req->req_q_out);
if (req->ring_index < cnt)
req->cnt = cnt - req->ring_index;
else
@@ -2825,8 +2889,8 @@
/* Check for room on request queue. */
if (req->cnt < req_cnt + 2) {
- cnt = RD_REG_DWORD_RELAXED(req->req_q_out);
-
+ cnt = IS_SHADOW_REG_CAPABLE(ha) ? *req->out_ptr :
+ RD_REG_DWORD_RELAXED(req->req_q_out);
if (req->ring_index < cnt)
req->cnt = cnt - req->ring_index;
else
diff --git a/drivers/scsi/qla2xxx/qla_isr.c b/drivers/scsi/qla2xxx/qla_isr.c
index 95314ef..a56825c 100644
--- a/drivers/scsi/qla2xxx/qla_isr.c
+++ b/drivers/scsi/qla2xxx/qla_isr.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -2009,11 +2009,13 @@
ql_dbg(ql_dbg_io, vha, 0x3017,
"Invalid status handle (0x%x).\n", sts->handle);
- if (IS_P3P_TYPE(ha))
- set_bit(FCOE_CTX_RESET_NEEDED, &vha->dpc_flags);
- else
- set_bit(ISP_ABORT_NEEDED, &vha->dpc_flags);
- qla2xxx_wake_dpc(vha);
+ if (!test_bit(ABORT_ISP_ACTIVE, &vha->dpc_flags)) {
+ if (IS_P3P_TYPE(ha))
+ set_bit(FCOE_CTX_RESET_NEEDED, &vha->dpc_flags);
+ else
+ set_bit(ISP_ABORT_NEEDED, &vha->dpc_flags);
+ qla2xxx_wake_dpc(vha);
+ }
return;
}
@@ -2472,12 +2474,14 @@
if (pkt->entry_status != 0) {
qla2x00_error_entry(vha, rsp, (sts_entry_t *) pkt);
- (void)qlt_24xx_process_response_error(vha, pkt);
+ if (qlt_24xx_process_response_error(vha, pkt))
+ goto process_err;
((response_t *)pkt)->signature = RESPONSE_PROCESSED;
wmb();
continue;
}
+process_err:
switch (pkt->entry_type) {
case STATUS_TYPE:
@@ -2494,10 +2498,10 @@
qla24xx_logio_entry(vha, rsp->req,
(struct logio_entry_24xx *)pkt);
break;
- case CT_IOCB_TYPE:
+ case CT_IOCB_TYPE:
qla24xx_els_ct_entry(vha, rsp->req, pkt, CT_IOCB_TYPE);
break;
- case ELS_IOCB_TYPE:
+ case ELS_IOCB_TYPE:
qla24xx_els_ct_entry(vha, rsp->req, pkt, ELS_IOCB_TYPE);
break;
case ABTS_RECV_24XX:
@@ -2506,6 +2510,7 @@
case ABTS_RESP_24XX:
case CTIO_TYPE7:
case NOTIFY_ACK_TYPE:
+ case CTIO_CRC2:
qlt_response_pkt_all_vps(vha, (response_t *)pkt);
break;
case MARKER_TYPE:
diff --git a/drivers/scsi/qla2xxx/qla_mbx.c b/drivers/scsi/qla2xxx/qla_mbx.c
index 2528709..1c33a77 100644
--- a/drivers/scsi/qla2xxx/qla_mbx.c
+++ b/drivers/scsi/qla2xxx/qla_mbx.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -1319,7 +1319,7 @@
left = 0;
- list = kzalloc(dma_size, GFP_KERNEL);
+ list = kmemdup(pmap, dma_size, GFP_KERNEL);
if (!list) {
ql_log(ql_log_warn, vha, 0x1140,
"%s(%ld): failed to allocate node names list "
@@ -1328,7 +1328,6 @@
goto out_free;
}
- memcpy(list, pmap, dma_size);
restart:
dma_free_coherent(&ha->pdev->dev, dma_size, pmap, pmap_dma);
}
@@ -2644,7 +2643,10 @@
ql_dbg(ql_dbg_mbx, vha, 0x1090,
"Failed to complete IOCB -- completion status (%x).\n",
le16_to_cpu(abt->nport_handle));
- rval = QLA_FUNCTION_FAILED;
+ if (abt->nport_handle == CS_IOCB_ERROR)
+ rval = QLA_FUNCTION_PARAMETER_ERROR;
+ else
+ rval = QLA_FUNCTION_FAILED;
} else {
ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x1091,
"Done %s.\n", __func__);
@@ -2879,6 +2881,78 @@
return rval;
}
+int
+qla8044_write_serdes_word(scsi_qla_host_t *vha, uint32_t addr, uint32_t data)
+{
+ int rval;
+ mbx_cmd_t mc;
+ mbx_cmd_t *mcp = &mc;
+
+ if (!IS_QLA8044(vha->hw))
+ return QLA_FUNCTION_FAILED;
+
+ ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x1186,
+ "Entered %s.\n", __func__);
+
+ mcp->mb[0] = MBC_SET_GET_ETH_SERDES_REG;
+ mcp->mb[1] = HCS_WRITE_SERDES;
+ mcp->mb[3] = LSW(addr);
+ mcp->mb[4] = MSW(addr);
+ mcp->mb[5] = LSW(data);
+ mcp->mb[6] = MSW(data);
+ mcp->out_mb = MBX_6|MBX_5|MBX_4|MBX_3|MBX_1|MBX_0;
+ mcp->in_mb = MBX_0;
+ mcp->tov = MBX_TOV_SECONDS;
+ mcp->flags = 0;
+ rval = qla2x00_mailbox_command(vha, mcp);
+
+ if (rval != QLA_SUCCESS) {
+ ql_dbg(ql_dbg_mbx, vha, 0x1187,
+ "Failed=%x mb[0]=%x.\n", rval, mcp->mb[0]);
+ } else {
+ ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x1188,
+ "Done %s.\n", __func__);
+ }
+
+ return rval;
+}
+
+int
+qla8044_read_serdes_word(scsi_qla_host_t *vha, uint32_t addr, uint32_t *data)
+{
+ int rval;
+ mbx_cmd_t mc;
+ mbx_cmd_t *mcp = &mc;
+
+ if (!IS_QLA8044(vha->hw))
+ return QLA_FUNCTION_FAILED;
+
+ ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x1189,
+ "Entered %s.\n", __func__);
+
+ mcp->mb[0] = MBC_SET_GET_ETH_SERDES_REG;
+ mcp->mb[1] = HCS_READ_SERDES;
+ mcp->mb[3] = LSW(addr);
+ mcp->mb[4] = MSW(addr);
+ mcp->out_mb = MBX_4|MBX_3|MBX_1|MBX_0;
+ mcp->in_mb = MBX_2|MBX_1|MBX_0;
+ mcp->tov = MBX_TOV_SECONDS;
+ mcp->flags = 0;
+ rval = qla2x00_mailbox_command(vha, mcp);
+
+ *data = mcp->mb[2] << 16 | mcp->mb[1];
+
+ if (rval != QLA_SUCCESS) {
+ ql_dbg(ql_dbg_mbx, vha, 0x118a,
+ "Failed=%x mb[0]=%x.\n", rval, mcp->mb[0]);
+ } else {
+ ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x118b,
+ "Done %s.\n", __func__);
+ }
+
+ return rval;
+}
+
/**
* qla2x00_set_serdes_params() -
* @ha: HA context
@@ -3660,6 +3734,9 @@
ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x10d3,
"Entered %s.\n", __func__);
+ if (IS_SHADOW_REG_CAPABLE(ha))
+ req->options |= BIT_13;
+
mcp->mb[0] = MBC_INITIALIZE_MULTIQ;
mcp->mb[1] = req->options;
mcp->mb[2] = MSW(LSD(req->dma));
@@ -3679,7 +3756,7 @@
/* que in ptr index */
mcp->mb[8] = 0;
/* que out ptr index */
- mcp->mb[9] = 0;
+ mcp->mb[9] = *req->out_ptr = 0;
mcp->out_mb = MBX_14|MBX_13|MBX_12|MBX_11|MBX_10|MBX_9|MBX_8|MBX_7|
MBX_6|MBX_5|MBX_4|MBX_3|MBX_2|MBX_1|MBX_0;
mcp->in_mb = MBX_0;
@@ -3688,7 +3765,7 @@
if (IS_QLA81XX(ha) || IS_QLA83XX(ha) || IS_QLA27XX(ha))
mcp->in_mb |= MBX_1;
- if (IS_QLA83XX(ha) || !IS_QLA27XX(ha)) {
+ if (IS_QLA83XX(ha) || IS_QLA27XX(ha)) {
mcp->out_mb |= MBX_15;
/* debug q create issue in SR-IOV */
mcp->in_mb |= MBX_9 | MBX_8 | MBX_7;
@@ -3697,7 +3774,7 @@
spin_lock_irqsave(&ha->hardware_lock, flags);
if (!(req->options & BIT_0)) {
WRT_REG_DWORD(req->req_q_in, 0);
- if (!IS_QLA83XX(ha) || !IS_QLA27XX(ha))
+ if (!IS_QLA83XX(ha) && !IS_QLA27XX(ha))
WRT_REG_DWORD(req->req_q_out, 0);
}
spin_unlock_irqrestore(&ha->hardware_lock, flags);
@@ -3726,6 +3803,9 @@
ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x10d6,
"Entered %s.\n", __func__);
+ if (IS_SHADOW_REG_CAPABLE(ha))
+ rsp->options |= BIT_13;
+
mcp->mb[0] = MBC_INITIALIZE_MULTIQ;
mcp->mb[1] = rsp->options;
mcp->mb[2] = MSW(LSD(rsp->dma));
@@ -3740,7 +3820,7 @@
mcp->mb[4] = rsp->id;
/* que in ptr index */
- mcp->mb[8] = 0;
+ mcp->mb[8] = *rsp->in_ptr = 0;
/* que out ptr index */
mcp->mb[9] = 0;
mcp->out_mb = MBX_14|MBX_13|MBX_9|MBX_8|MBX_7
diff --git a/drivers/scsi/qla2xxx/qla_mid.c b/drivers/scsi/qla2xxx/qla_mid.c
index f0a8522..8999824 100644
--- a/drivers/scsi/qla2xxx/qla_mid.c
+++ b/drivers/scsi/qla2xxx/qla_mid.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
diff --git a/drivers/scsi/qla2xxx/qla_mr.c b/drivers/scsi/qla2xxx/qla_mr.c
index 0aaf6a9..abeb390 100644
--- a/drivers/scsi/qla2xxx/qla_mr.c
+++ b/drivers/scsi/qla2xxx/qla_mr.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -527,21 +527,63 @@
struct qla_hw_data *ha = vha->hw;
int i, core;
uint32_t cnt;
+ uint32_t reg_val;
+
+ spin_lock_irqsave(&ha->hardware_lock, flags);
+
+ QLAFX00_SET_HBA_SOC_REG(ha, 0x80004, 0);
+ QLAFX00_SET_HBA_SOC_REG(ha, 0x82004, 0);
+
+ /* stop the XOR DMA engines */
+ QLAFX00_SET_HBA_SOC_REG(ha, 0x60920, 0x02);
+ QLAFX00_SET_HBA_SOC_REG(ha, 0x60924, 0x02);
+ QLAFX00_SET_HBA_SOC_REG(ha, 0xf0920, 0x02);
+ QLAFX00_SET_HBA_SOC_REG(ha, 0xf0924, 0x02);
+
+ /* stop the IDMA engines */
+ reg_val = QLAFX00_GET_HBA_SOC_REG(ha, 0x60840);
+ reg_val &= ~(1<<12);
+ QLAFX00_SET_HBA_SOC_REG(ha, 0x60840, reg_val);
+
+ reg_val = QLAFX00_GET_HBA_SOC_REG(ha, 0x60844);
+ reg_val &= ~(1<<12);
+ QLAFX00_SET_HBA_SOC_REG(ha, 0x60844, reg_val);
+
+ reg_val = QLAFX00_GET_HBA_SOC_REG(ha, 0x60848);
+ reg_val &= ~(1<<12);
+ QLAFX00_SET_HBA_SOC_REG(ha, 0x60848, reg_val);
+
+ reg_val = QLAFX00_GET_HBA_SOC_REG(ha, 0x6084C);
+ reg_val &= ~(1<<12);
+ QLAFX00_SET_HBA_SOC_REG(ha, 0x6084C, reg_val);
+
+ for (i = 0; i < 100000; i++) {
+ if ((QLAFX00_GET_HBA_SOC_REG(ha, 0xd0000) & 0x10000000) == 0 &&
+ (QLAFX00_GET_HBA_SOC_REG(ha, 0x10600) & 0x1) == 0)
+ break;
+ udelay(100);
+ }
/* Set all 4 cores in reset */
for (i = 0; i < 4; i++) {
QLAFX00_SET_HBA_SOC_REG(ha,
(SOC_SW_RST_CONTROL_REG_CORE0 + 8*i), (0xF01));
- }
-
- /* Set all 4 core Clock gating control */
- for (i = 0; i < 4; i++) {
QLAFX00_SET_HBA_SOC_REG(ha,
(SOC_SW_RST_CONTROL_REG_CORE0 + 4 + 8*i), (0x01010101));
}
/* Reset all units in Fabric */
- QLAFX00_SET_HBA_SOC_REG(ha, SOC_FABRIC_RST_CONTROL_REG, (0x11F0101));
+ QLAFX00_SET_HBA_SOC_REG(ha, SOC_FABRIC_RST_CONTROL_REG, (0x011f0101));
+
+ /* */
+ QLAFX00_SET_HBA_SOC_REG(ha, 0x10610, 1);
+ QLAFX00_SET_HBA_SOC_REG(ha, 0x10600, 0);
+
+ /* Set all 4 core Memory Power Down Registers */
+ for (i = 0; i < 5; i++) {
+ QLAFX00_SET_HBA_SOC_REG(ha,
+ (SOC_PWR_MANAGEMENT_PWR_DOWN_REG + 4*i), (0x0));
+ }
/* Reset all interrupt control registers */
for (i = 0; i < 115; i++) {
@@ -564,20 +606,19 @@
QLAFX00_SET_HBA_SOC_REG(ha, SOC_FABRIC_CONTROL_REG, (0x2));
QLAFX00_SET_HBA_SOC_REG(ha, SOC_FABRIC_CONFIG_REG, (0x3));
- spin_lock_irqsave(&ha->hardware_lock, flags);
-
/* Kick in Fabric units */
QLAFX00_SET_HBA_SOC_REG(ha, SOC_FABRIC_RST_CONTROL_REG, (0x0));
/* Kick in Core0 to start boot process */
QLAFX00_SET_HBA_SOC_REG(ha, SOC_SW_RST_CONTROL_REG_CORE0, (0xF00));
+ spin_unlock_irqrestore(&ha->hardware_lock, flags);
+
/* Wait 10secs for soft-reset to complete. */
for (cnt = 10; cnt; cnt--) {
msleep(1000);
barrier();
}
- spin_unlock_irqrestore(&ha->hardware_lock, flags);
}
/**
@@ -597,7 +638,6 @@
ha->isp_ops->disable_intrs(ha);
qlafx00_soc_cpu_reset(vha);
- ha->isp_ops->enable_intrs(ha);
}
/**
@@ -2675,7 +2715,7 @@
uint16_t lreq_q_out = 0;
lreq_q_in = RD_REG_DWORD(rsp->rsp_q_in);
- lreq_q_out = RD_REG_DWORD(rsp->rsp_q_out);
+ lreq_q_out = rsp->ring_index;
while (lreq_q_in != lreq_q_out) {
lptr = rsp->ring_ptr;
@@ -3426,7 +3466,7 @@
sp->fcport->vha, 0x3047,
(uint8_t *)&fx_iocb, sizeof(struct fxdisc_entry_fx00));
- memcpy((void *)pfxiocb, &fx_iocb,
+ memcpy_toio((void __iomem *)pfxiocb, &fx_iocb,
sizeof(struct fxdisc_entry_fx00));
wmb();
}
diff --git a/drivers/scsi/qla2xxx/qla_mr.h b/drivers/scsi/qla2xxx/qla_mr.h
index e529dfa..aeaa1b4 100644
--- a/drivers/scsi/qla2xxx/qla_mr.h
+++ b/drivers/scsi/qla2xxx/qla_mr.h
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -351,6 +351,7 @@
#define SOC_FABRIC_RST_CONTROL_REG 0x0020840
#define SOC_FABRIC_CONTROL_REG 0x0020200
#define SOC_FABRIC_CONFIG_REG 0x0020204
+#define SOC_PWR_MANAGEMENT_PWR_DOWN_REG 0x001820C
#define SOC_INTERRUPT_SOURCE_I_CONTROL_REG 0x0020B00
#define SOC_CORE_TIMER_REG 0x0021850
diff --git a/drivers/scsi/qla2xxx/qla_nx.c b/drivers/scsi/qla2xxx/qla_nx.c
index 5511e24..58f3c91 100644
--- a/drivers/scsi/qla2xxx/qla_nx.c
+++ b/drivers/scsi/qla2xxx/qla_nx.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -848,6 +848,7 @@
{
int done = 0, timeout = 0;
uint32_t lock_owner = 0;
+ scsi_qla_host_t *vha = pci_get_drvdata(ha->pdev);
while (!done) {
/* acquire semaphore2 from PCI HW block */
@@ -856,17 +857,21 @@
break;
if (timeout >= qla82xx_rom_lock_timeout) {
lock_owner = qla82xx_rd_32(ha, QLA82XX_ROM_LOCK_ID);
+ ql_log(ql_log_warn, vha, 0xb157,
+ "%s: Simultaneous flash access by following ports, active port = %d: accessing port = %d",
+ __func__, ha->portnum, lock_owner);
return -1;
}
timeout++;
}
- qla82xx_wr_32(ha, QLA82XX_ROM_LOCK_ID, ROM_LOCK_DRIVER);
+ qla82xx_wr_32(ha, QLA82XX_ROM_LOCK_ID, ha->portnum);
return 0;
}
static void
qla82xx_rom_unlock(struct qla_hw_data *ha)
{
+ qla82xx_wr_32(ha, QLA82XX_ROM_LOCK_ID, 0xffffffff);
qla82xx_rd_32(ha, QLA82XX_PCIE_REG(PCIE_SEM2_UNLOCK));
}
@@ -950,6 +955,7 @@
qla82xx_rom_fast_read(struct qla_hw_data *ha, int addr, int *valp)
{
int ret, loops = 0;
+ uint32_t lock_owner = 0;
scsi_qla_host_t *vha = pci_get_drvdata(ha->pdev);
while ((qla82xx_rom_lock(ha) != 0) && (loops < 50000)) {
@@ -958,8 +964,10 @@
loops++;
}
if (loops >= 50000) {
+ lock_owner = qla82xx_rd_32(ha, QLA82XX_ROM_LOCK_ID);
ql_log(ql_log_fatal, vha, 0x00b9,
- "Failed to acquire SEM2 lock.\n");
+ "Failed to acquire SEM2 lock, Lock Owner %u.\n",
+ lock_owner);
return -1;
}
ret = qla82xx_do_rom_fast_read(ha, addr, valp);
@@ -1057,6 +1065,7 @@
ql82xx_rom_lock_d(struct qla_hw_data *ha)
{
int loops = 0;
+ uint32_t lock_owner = 0;
scsi_qla_host_t *vha = pci_get_drvdata(ha->pdev);
while ((qla82xx_rom_lock(ha) != 0) && (loops < 50000)) {
@@ -1065,8 +1074,9 @@
loops++;
}
if (loops >= 50000) {
+ lock_owner = qla82xx_rd_32(ha, QLA82XX_ROM_LOCK_ID);
ql_log(ql_log_warn, vha, 0xb010,
- "ROM lock failed.\n");
+ "ROM lock failed, Lock Owner %u.\n", lock_owner);
return -1;
}
return 0;
@@ -2811,12 +2821,14 @@
qla82xx_rom_lock_recovery(struct qla_hw_data *ha)
{
scsi_qla_host_t *vha = pci_get_drvdata(ha->pdev);
+ uint32_t lock_owner = 0;
- if (qla82xx_rom_lock(ha))
+ if (qla82xx_rom_lock(ha)) {
+ lock_owner = qla82xx_rd_32(ha, QLA82XX_ROM_LOCK_ID);
/* Someone else is holding the lock. */
ql_log(ql_log_info, vha, 0xb022,
- "Resetting rom_lock.\n");
-
+ "Resetting rom_lock, Lock Owner %u.\n", lock_owner);
+ }
/*
* Either we got the lock, or someone
* else died while holding it.
@@ -2840,47 +2852,30 @@
qla82xx_device_bootstrap(scsi_qla_host_t *vha)
{
int rval = QLA_SUCCESS;
- int i, timeout;
+ int i;
uint32_t old_count, count;
struct qla_hw_data *ha = vha->hw;
- int need_reset = 0, peg_stuck = 1;
+ int need_reset = 0;
need_reset = qla82xx_need_reset(ha);
- old_count = qla82xx_rd_32(ha, QLA82XX_PEG_ALIVE_COUNTER);
-
- for (i = 0; i < 10; i++) {
- timeout = msleep_interruptible(200);
- if (timeout) {
- qla82xx_wr_32(ha, QLA82XX_CRB_DEV_STATE,
- QLA8XXX_DEV_FAILED);
- return QLA_FUNCTION_FAILED;
- }
-
- count = qla82xx_rd_32(ha, QLA82XX_PEG_ALIVE_COUNTER);
- if (count != old_count)
- peg_stuck = 0;
- }
-
if (need_reset) {
/* We are trying to perform a recovery here. */
- if (peg_stuck)
+ if (ha->flags.isp82xx_fw_hung)
qla82xx_rom_lock_recovery(ha);
- goto dev_initialize;
} else {
- /* Start of day for this ha context. */
- if (peg_stuck) {
- /* Either we are the first or recovery in progress. */
- qla82xx_rom_lock_recovery(ha);
- goto dev_initialize;
- } else
- /* Firmware already running. */
- goto dev_ready;
+ old_count = qla82xx_rd_32(ha, QLA82XX_PEG_ALIVE_COUNTER);
+ for (i = 0; i < 10; i++) {
+ msleep(200);
+ count = qla82xx_rd_32(ha, QLA82XX_PEG_ALIVE_COUNTER);
+ if (count != old_count) {
+ rval = QLA_SUCCESS;
+ goto dev_ready;
+ }
+ }
+ qla82xx_rom_lock_recovery(ha);
}
- return rval;
-
-dev_initialize:
/* set to DEV_INITIALIZING */
ql_log(ql_log_info, vha, 0x009e,
"HW State: INITIALIZING.\n");
@@ -3142,18 +3137,18 @@
if (ql2xmdenable) {
if (!ha->fw_dumped) {
- if (fw_major_version != ha->fw_major_version ||
+ if ((fw_major_version != ha->fw_major_version ||
fw_minor_version != ha->fw_minor_version ||
- fw_subminor_version != ha->fw_subminor_version) {
+ fw_subminor_version != ha->fw_subminor_version) ||
+ (ha->prev_minidump_failed)) {
ql_dbg(ql_dbg_p3p, vha, 0xb02d,
- "Firmware version differs "
- "Previous version: %d:%d:%d - "
- "New version: %d:%d:%d\n",
+ "Firmware version differs Previous version: %d:%d:%d - New version: %d:%d:%d, prev_minidump_failed: %d.\n",
fw_major_version, fw_minor_version,
fw_subminor_version,
ha->fw_major_version,
ha->fw_minor_version,
- ha->fw_subminor_version);
+ ha->fw_subminor_version,
+ ha->prev_minidump_failed);
/* Release MiniDump resources */
qla82xx_md_free(vha);
/* ALlocate MiniDump resources */
@@ -3682,8 +3677,10 @@
for (cnt = 1; cnt < req->num_outstanding_cmds; cnt++) {
sp = req->outstanding_cmds[cnt];
if (sp) {
- if (!sp->u.scmd.ctx ||
- (sp->flags & SRB_FCP_CMND_DMA_VALID)) {
+ if ((!sp->u.scmd.ctx ||
+ (sp->flags &
+ SRB_FCP_CMND_DMA_VALID)) &&
+ !ha->flags.isp82xx_fw_hung) {
spin_unlock_irqrestore(
&ha->hardware_lock, flags);
if (ha->isp_ops->abort_command(sp)) {
diff --git a/drivers/scsi/qla2xxx/qla_nx.h b/drivers/scsi/qla2xxx/qla_nx.h
index 1bb93db..59c4778 100644
--- a/drivers/scsi/qla2xxx/qla_nx.h
+++ b/drivers/scsi/qla2xxx/qla_nx.h
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -333,9 +333,6 @@
#define QLA82XX_ROMUSB_ROM_INSTR_OPCODE (ROMUSB_ROM + 0x0004)
#define QLA82XX_ROMUSB_GLB_CAS_RST (ROMUSB_GLB + 0x0038)
-/* Lock IDs for ROM lock */
-#define ROM_LOCK_DRIVER 0x0d417340
-
#define QLA82XX_PCI_CRB_WINDOWSIZE 0x00100000 /* all are 1MB windows */
#define QLA82XX_PCI_CRB_WINDOW(A) \
(QLA82XX_PCI_CRBSPACE + (A)*QLA82XX_PCI_CRB_WINDOWSIZE)
@@ -1186,6 +1183,7 @@
#define CRB_NIU_XG_PAUSE_CTL_P1 0x8
#define qla82xx_get_temp_val(x) ((x) >> 16)
+#define qla82xx_get_temp_val1(x) ((x) && 0x0000FFFF)
#define qla82xx_get_temp_state(x) ((x) & 0xffff)
#define qla82xx_encode_temp(val, state) (((val) << 16) | (state))
diff --git a/drivers/scsi/qla2xxx/qla_nx2.c b/drivers/scsi/qla2xxx/qla_nx2.c
index 86cf108..da9e390 100644
--- a/drivers/scsi/qla2xxx/qla_nx2.c
+++ b/drivers/scsi/qla2xxx/qla_nx2.c
@@ -1,17 +1,20 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
#include <linux/vmalloc.h>
+#include <linux/delay.h>
#include "qla_def.h"
#include "qla_gbl.h"
#include <linux/delay.h>
+#define TIMEOUT_100_MS 100
+
/* 8044 Flash Read/Write functions */
uint32_t
qla8044_rd_reg(struct qla_hw_data *ha, ulong addr)
@@ -117,6 +120,95 @@
qla8044_wr_reg_indirect(vha, waddr, value);
}
+static int
+qla8044_poll_wait_for_ready(struct scsi_qla_host *vha, uint32_t addr1,
+ uint32_t mask)
+{
+ unsigned long timeout;
+ uint32_t temp;
+
+ /* jiffies after 100ms */
+ timeout = jiffies + msecs_to_jiffies(TIMEOUT_100_MS);
+ do {
+ qla8044_rd_reg_indirect(vha, addr1, &temp);
+ if ((temp & mask) != 0)
+ break;
+ if (time_after_eq(jiffies, timeout)) {
+ ql_log(ql_log_warn, vha, 0xb151,
+ "Error in processing rdmdio entry\n");
+ return -1;
+ }
+ } while (1);
+
+ return 0;
+}
+
+static uint32_t
+qla8044_ipmdio_rd_reg(struct scsi_qla_host *vha,
+ uint32_t addr1, uint32_t addr3, uint32_t mask, uint32_t addr)
+{
+ uint32_t temp;
+ int ret = 0;
+
+ ret = qla8044_poll_wait_for_ready(vha, addr1, mask);
+ if (ret == -1)
+ return -1;
+
+ temp = (0x40000000 | addr);
+ qla8044_wr_reg_indirect(vha, addr1, temp);
+
+ ret = qla8044_poll_wait_for_ready(vha, addr1, mask);
+ if (ret == -1)
+ return 0;
+
+ qla8044_rd_reg_indirect(vha, addr3, &ret);
+
+ return ret;
+}
+
+
+static int
+qla8044_poll_wait_ipmdio_bus_idle(struct scsi_qla_host *vha,
+ uint32_t addr1, uint32_t addr2, uint32_t addr3, uint32_t mask)
+{
+ unsigned long timeout;
+ uint32_t temp;
+
+ /* jiffies after 100 msecs */
+ timeout = jiffies + msecs_to_jiffies(TIMEOUT_100_MS);
+ do {
+ temp = qla8044_ipmdio_rd_reg(vha, addr1, addr3, mask, addr2);
+ if ((temp & 0x1) != 1)
+ break;
+ if (time_after_eq(jiffies, timeout)) {
+ ql_log(ql_log_warn, vha, 0xb152,
+ "Error in processing mdiobus idle\n");
+ return -1;
+ }
+ } while (1);
+
+ return 0;
+}
+
+static int
+qla8044_ipmdio_wr_reg(struct scsi_qla_host *vha, uint32_t addr1,
+ uint32_t addr3, uint32_t mask, uint32_t addr, uint32_t value)
+{
+ int ret = 0;
+
+ ret = qla8044_poll_wait_for_ready(vha, addr1, mask);
+ if (ret == -1)
+ return -1;
+
+ qla8044_wr_reg_indirect(vha, addr3, value);
+ qla8044_wr_reg_indirect(vha, addr1, addr);
+
+ ret = qla8044_poll_wait_for_ready(vha, addr1, mask);
+ if (ret == -1)
+ return -1;
+
+ return 0;
+}
/*
* qla8044_rmw_crb_reg - Read value from raddr, AND with test_mask,
* Shift Left,Right/OR/XOR with values RMW header and write value to waddr.
@@ -356,8 +448,8 @@
lock_owner = qla8044_rd_reg(ha,
QLA8044_FLASH_LOCK_ID);
ql_log(ql_log_warn, vha, 0xb113,
- "%s: flash lock by %d failed, held by %d\n",
- __func__, ha->portnum, lock_owner);
+ "%s: Simultaneous flash access by following ports, active port = %d: accessing port = %d",
+ __func__, ha->portnum, lock_owner);
ret_val = QLA_FUNCTION_FAILED;
break;
}
@@ -1541,7 +1633,7 @@
qla8044_need_reset_handler(struct scsi_qla_host *vha)
{
uint32_t dev_state = 0, drv_state, drv_active;
- unsigned long reset_timeout, dev_init_timeout;
+ unsigned long reset_timeout;
struct qla_hw_data *ha = vha->hw;
ql_log(ql_log_fatal, vha, 0xb0c2,
@@ -1555,84 +1647,78 @@
qla8044_idc_lock(ha);
}
+ dev_state = qla8044_rd_direct(vha,
+ QLA8044_CRB_DEV_STATE_INDEX);
drv_state = qla8044_rd_direct(vha,
QLA8044_CRB_DRV_STATE_INDEX);
drv_active = qla8044_rd_direct(vha,
QLA8044_CRB_DRV_ACTIVE_INDEX);
ql_log(ql_log_info, vha, 0xb0c5,
- "%s(%ld): drv_state = 0x%x, drv_active = 0x%x\n",
- __func__, vha->host_no, drv_state, drv_active);
+ "%s(%ld): drv_state = 0x%x, drv_active = 0x%x dev_state = 0x%x\n",
+ __func__, vha->host_no, drv_state, drv_active, dev_state);
- if (!ha->flags.nic_core_reset_owner) {
- ql_dbg(ql_dbg_p3p, vha, 0xb0c3,
- "%s(%ld): reset acknowledged\n",
- __func__, vha->host_no);
- qla8044_set_rst_ready(vha);
+ qla8044_set_rst_ready(vha);
- /* Non-reset owners ACK Reset and wait for device INIT state
- * as part of Reset Recovery by Reset Owner
- */
- dev_init_timeout = jiffies + (ha->fcoe_reset_timeout * HZ);
+ /* wait for 10 seconds for reset ack from all functions */
+ reset_timeout = jiffies + (ha->fcoe_reset_timeout * HZ);
- do {
- if (time_after_eq(jiffies, dev_init_timeout)) {
- ql_log(ql_log_info, vha, 0xb0c4,
- "%s: Non Reset owner: Reset Ack Timeout!\n",
- __func__);
- break;
- }
+ do {
+ if (time_after_eq(jiffies, reset_timeout)) {
+ ql_log(ql_log_info, vha, 0xb0c4,
+ "%s: Function %d: Reset Ack Timeout!, drv_state: 0x%08x, drv_active: 0x%08x\n",
+ __func__, ha->portnum, drv_state, drv_active);
+ break;
+ }
- qla8044_idc_unlock(ha);
- msleep(1000);
- qla8044_idc_lock(ha);
+ qla8044_idc_unlock(ha);
+ msleep(1000);
+ qla8044_idc_lock(ha);
- dev_state = qla8044_rd_direct(vha,
- QLA8044_CRB_DEV_STATE_INDEX);
- } while (((drv_state & drv_active) != drv_active) &&
- (dev_state == QLA8XXX_DEV_NEED_RESET));
+ dev_state = qla8044_rd_direct(vha,
+ QLA8044_CRB_DEV_STATE_INDEX);
+ drv_state = qla8044_rd_direct(vha,
+ QLA8044_CRB_DRV_STATE_INDEX);
+ drv_active = qla8044_rd_direct(vha,
+ QLA8044_CRB_DRV_ACTIVE_INDEX);
+ } while (((drv_state & drv_active) != drv_active) &&
+ (dev_state == QLA8XXX_DEV_NEED_RESET));
+
+ /* Remove IDC participation of functions not acknowledging */
+ if (drv_state != drv_active) {
+ ql_log(ql_log_info, vha, 0xb0c7,
+ "%s(%ld): Function %d turning off drv_active of non-acking function 0x%x\n",
+ __func__, vha->host_no, ha->portnum,
+ (drv_active ^ drv_state));
+ drv_active = drv_active & drv_state;
+ qla8044_wr_direct(vha, QLA8044_CRB_DRV_ACTIVE_INDEX,
+ drv_active);
} else {
- qla8044_set_rst_ready(vha);
-
- /* wait for 10 seconds for reset ack from all functions */
- reset_timeout = jiffies + (ha->fcoe_reset_timeout * HZ);
-
- while ((drv_state & drv_active) != drv_active) {
- if (time_after_eq(jiffies, reset_timeout)) {
- ql_log(ql_log_info, vha, 0xb0c6,
- "%s: RESET TIMEOUT!"
- "drv_state: 0x%08x, drv_active: 0x%08x\n",
- QLA2XXX_DRIVER_NAME, drv_state, drv_active);
- break;
- }
-
- qla8044_idc_unlock(ha);
- msleep(1000);
- qla8044_idc_lock(ha);
-
- drv_state = qla8044_rd_direct(vha,
- QLA8044_CRB_DRV_STATE_INDEX);
- drv_active = qla8044_rd_direct(vha,
- QLA8044_CRB_DRV_ACTIVE_INDEX);
- }
-
- if (drv_state != drv_active) {
- ql_log(ql_log_info, vha, 0xb0c7,
- "%s(%ld): Reset_owner turning off drv_active "
- "of non-acking function 0x%x\n", __func__,
- vha->host_no, (drv_active ^ drv_state));
- drv_active = drv_active & drv_state;
- qla8044_wr_direct(vha, QLA8044_CRB_DRV_ACTIVE_INDEX,
- drv_active);
- }
-
/*
- * Clear RESET OWNER, will be set at next reset
- * by next RST_OWNER
- */
- ha->flags.nic_core_reset_owner = 0;
+ * Reset owner should execute reset recovery,
+ * if all functions acknowledged
+ */
+ if ((ha->flags.nic_core_reset_owner) &&
+ (dev_state == QLA8XXX_DEV_NEED_RESET)) {
+ ha->flags.nic_core_reset_owner = 0;
+ qla8044_device_bootstrap(vha);
+ return;
+ }
+ }
- /* Start Reset Recovery */
+ /* Exit if non active function */
+ if (!(drv_active & (1 << ha->portnum))) {
+ ha->flags.nic_core_reset_owner = 0;
+ return;
+ }
+
+ /*
+ * Execute Reset Recovery if Reset Owner or Function 7
+ * is the only active function
+ */
+ if (ha->flags.nic_core_reset_owner ||
+ ((drv_state & drv_active) == QLA8044_FUN7_ACTIVE_INDEX)) {
+ ha->flags.nic_core_reset_owner = 0;
qla8044_device_bootstrap(vha);
}
}
@@ -1655,6 +1741,19 @@
qla8044_wr_direct(vha, QLA8044_CRB_DRV_ACTIVE_INDEX, drv_active);
}
+static int
+qla8044_check_drv_active(struct scsi_qla_host *vha)
+{
+ uint32_t drv_active;
+ struct qla_hw_data *ha = vha->hw;
+
+ drv_active = qla8044_rd_direct(vha, QLA8044_CRB_DRV_ACTIVE_INDEX);
+ if (drv_active & (1 << ha->portnum))
+ return QLA_SUCCESS;
+ else
+ return QLA_TEST_FAILED;
+}
+
static void
qla8044_clear_idc_dontreset(struct scsi_qla_host *vha)
{
@@ -1837,14 +1936,16 @@
while (1) {
if (time_after_eq(jiffies, dev_init_timeout)) {
- ql_log(ql_log_warn, vha, 0xb0cf,
- "%s: Device Init Failed 0x%x = %s\n",
- QLA2XXX_DRIVER_NAME, dev_state,
- dev_state < MAX_STATES ?
- qdev_state(dev_state) : "Unknown");
-
- qla8044_wr_direct(vha, QLA8044_CRB_DEV_STATE_INDEX,
- QLA8XXX_DEV_FAILED);
+ if (qla8044_check_drv_active(vha) == QLA_SUCCESS) {
+ ql_log(ql_log_warn, vha, 0xb0cf,
+ "%s: Device Init Failed 0x%x = %s\n",
+ QLA2XXX_DRIVER_NAME, dev_state,
+ dev_state < MAX_STATES ?
+ qdev_state(dev_state) : "Unknown");
+ qla8044_wr_direct(vha,
+ QLA8044_CRB_DEV_STATE_INDEX,
+ QLA8XXX_DEV_FAILED);
+ }
}
dev_state = qla8044_rd_direct(vha, QLA8044_CRB_DEV_STATE_INDEX);
@@ -2017,6 +2118,13 @@
test_bit(FCOE_CTX_RESET_NEEDED, &vha->dpc_flags))) {
dev_state = qla8044_rd_direct(vha, QLA8044_CRB_DEV_STATE_INDEX);
+ if (qla8044_check_fw_alive(vha)) {
+ ha->flags.isp82xx_fw_hung = 1;
+ ql_log(ql_log_warn, vha, 0xb10a,
+ "Firmware hung.\n");
+ qla82xx_clear_pending_mbx(vha);
+ }
+
if (qla8044_check_temp(vha)) {
set_bit(ISP_UNRECOVERABLE, &vha->dpc_flags);
ha->flags.isp82xx_fw_hung = 1;
@@ -2037,7 +2145,7 @@
qla2xxx_wake_dpc(vha);
} else {
/* Check firmware health */
- if (qla8044_check_fw_alive(vha)) {
+ if (ha->flags.isp82xx_fw_hung) {
halt_status = qla8044_rd_direct(vha,
QLA8044_PEG_HALT_STATUS1_INDEX);
if (halt_status &
@@ -2073,12 +2181,8 @@
__func__);
set_bit(ISP_ABORT_NEEDED,
&vha->dpc_flags);
- qla82xx_clear_pending_mbx(vha);
}
}
- ha->flags.isp82xx_fw_hung = 1;
- ql_log(ql_log_warn, vha, 0xb10a,
- "Firmware hung.\n");
qla2xxx_wake_dpc(vha);
}
}
@@ -2286,8 +2390,6 @@
}
if (j >= MAX_CTL_CHECK) {
- printk_ratelimited(KERN_ERR
- "%s: failed to read through agent\n", __func__);
write_unlock_irqrestore(&ha->hw_lock, flags);
return QLA_SUCCESS;
}
@@ -2882,6 +2984,231 @@
return rval;
}
+static uint32_t
+qla8044_minidump_process_rddfe(struct scsi_qla_host *vha,
+ struct qla8044_minidump_entry_hdr *entry_hdr, uint32_t **d_ptr)
+{
+ int loop_cnt;
+ uint32_t addr1, addr2, value, data, temp, wrVal;
+ uint8_t stride, stride2;
+ uint16_t count;
+ uint32_t poll, mask, data_size, modify_mask;
+ uint32_t wait_count = 0;
+
+ uint32_t *data_ptr = *d_ptr;
+
+ struct qla8044_minidump_entry_rddfe *rddfe;
+ rddfe = (struct qla8044_minidump_entry_rddfe *) entry_hdr;
+
+ addr1 = rddfe->addr_1;
+ value = rddfe->value;
+ stride = rddfe->stride;
+ stride2 = rddfe->stride2;
+ count = rddfe->count;
+
+ poll = rddfe->poll;
+ mask = rddfe->mask;
+ modify_mask = rddfe->modify_mask;
+ data_size = rddfe->data_size;
+
+ addr2 = addr1 + stride;
+
+ for (loop_cnt = 0x0; loop_cnt < count; loop_cnt++) {
+ qla8044_wr_reg_indirect(vha, addr1, (0x40000000 | value));
+
+ wait_count = 0;
+ while (wait_count < poll) {
+ qla8044_rd_reg_indirect(vha, addr1, &temp);
+ if ((temp & mask) != 0)
+ break;
+ wait_count++;
+ }
+
+ if (wait_count == poll) {
+ ql_log(ql_log_warn, vha, 0xb153,
+ "%s: TIMEOUT\n", __func__);
+ goto error;
+ } else {
+ qla8044_rd_reg_indirect(vha, addr2, &temp);
+ temp = temp & modify_mask;
+ temp = (temp | ((loop_cnt << 16) | loop_cnt));
+ wrVal = ((temp << 16) | temp);
+
+ qla8044_wr_reg_indirect(vha, addr2, wrVal);
+ qla8044_wr_reg_indirect(vha, addr1, value);
+
+ wait_count = 0;
+ while (wait_count < poll) {
+ qla8044_rd_reg_indirect(vha, addr1, &temp);
+ if ((temp & mask) != 0)
+ break;
+ wait_count++;
+ }
+ if (wait_count == poll) {
+ ql_log(ql_log_warn, vha, 0xb154,
+ "%s: TIMEOUT\n", __func__);
+ goto error;
+ }
+
+ qla8044_wr_reg_indirect(vha, addr1,
+ ((0x40000000 | value) + stride2));
+ wait_count = 0;
+ while (wait_count < poll) {
+ qla8044_rd_reg_indirect(vha, addr1, &temp);
+ if ((temp & mask) != 0)
+ break;
+ wait_count++;
+ }
+
+ if (wait_count == poll) {
+ ql_log(ql_log_warn, vha, 0xb155,
+ "%s: TIMEOUT\n", __func__);
+ goto error;
+ }
+
+ qla8044_rd_reg_indirect(vha, addr2, &data);
+
+ *data_ptr++ = wrVal;
+ *data_ptr++ = data;
+ }
+
+ }
+
+ *d_ptr = data_ptr;
+ return QLA_SUCCESS;
+
+error:
+ return -1;
+
+}
+
+static uint32_t
+qla8044_minidump_process_rdmdio(struct scsi_qla_host *vha,
+ struct qla8044_minidump_entry_hdr *entry_hdr, uint32_t **d_ptr)
+{
+ int ret = 0;
+ uint32_t addr1, addr2, value1, value2, data, selVal;
+ uint8_t stride1, stride2;
+ uint32_t addr3, addr4, addr5, addr6, addr7;
+ uint16_t count, loop_cnt;
+ uint32_t poll, mask;
+ uint32_t *data_ptr = *d_ptr;
+
+ struct qla8044_minidump_entry_rdmdio *rdmdio;
+
+ rdmdio = (struct qla8044_minidump_entry_rdmdio *) entry_hdr;
+
+ addr1 = rdmdio->addr_1;
+ addr2 = rdmdio->addr_2;
+ value1 = rdmdio->value_1;
+ stride1 = rdmdio->stride_1;
+ stride2 = rdmdio->stride_2;
+ count = rdmdio->count;
+
+ poll = rdmdio->poll;
+ mask = rdmdio->mask;
+ value2 = rdmdio->value_2;
+
+ addr3 = addr1 + stride1;
+
+ for (loop_cnt = 0; loop_cnt < count; loop_cnt++) {
+ ret = qla8044_poll_wait_ipmdio_bus_idle(vha, addr1, addr2,
+ addr3, mask);
+ if (ret == -1)
+ goto error;
+
+ addr4 = addr2 - stride1;
+ ret = qla8044_ipmdio_wr_reg(vha, addr1, addr3, mask, addr4,
+ value2);
+ if (ret == -1)
+ goto error;
+
+ addr5 = addr2 - (2 * stride1);
+ ret = qla8044_ipmdio_wr_reg(vha, addr1, addr3, mask, addr5,
+ value1);
+ if (ret == -1)
+ goto error;
+
+ addr6 = addr2 - (3 * stride1);
+ ret = qla8044_ipmdio_wr_reg(vha, addr1, addr3, mask,
+ addr6, 0x2);
+ if (ret == -1)
+ goto error;
+
+ ret = qla8044_poll_wait_ipmdio_bus_idle(vha, addr1, addr2,
+ addr3, mask);
+ if (ret == -1)
+ goto error;
+
+ addr7 = addr2 - (4 * stride1);
+ data = qla8044_ipmdio_rd_reg(vha, addr1, addr3,
+ mask, addr7);
+ if (data == -1)
+ goto error;
+
+ selVal = (value2 << 18) | (value1 << 2) | 2;
+
+ stride2 = rdmdio->stride_2;
+ *data_ptr++ = selVal;
+ *data_ptr++ = data;
+
+ value1 = value1 + stride2;
+ *d_ptr = data_ptr;
+ }
+
+ return 0;
+
+error:
+ return -1;
+}
+
+static uint32_t qla8044_minidump_process_pollwr(struct scsi_qla_host *vha,
+ struct qla8044_minidump_entry_hdr *entry_hdr, uint32_t **d_ptr)
+{
+ uint32_t addr1, addr2, value1, value2, poll, mask, r_value;
+ uint32_t wait_count = 0;
+ struct qla8044_minidump_entry_pollwr *pollwr_hdr;
+
+ pollwr_hdr = (struct qla8044_minidump_entry_pollwr *)entry_hdr;
+ addr1 = pollwr_hdr->addr_1;
+ addr2 = pollwr_hdr->addr_2;
+ value1 = pollwr_hdr->value_1;
+ value2 = pollwr_hdr->value_2;
+
+ poll = pollwr_hdr->poll;
+ mask = pollwr_hdr->mask;
+
+ while (wait_count < poll) {
+ qla8044_rd_reg_indirect(vha, addr1, &r_value);
+
+ if ((r_value & poll) != 0)
+ break;
+ wait_count++;
+ }
+
+ if (wait_count == poll) {
+ ql_log(ql_log_warn, vha, 0xb156, "%s: TIMEOUT\n", __func__);
+ goto error;
+ }
+
+ qla8044_wr_reg_indirect(vha, addr2, value2);
+ qla8044_wr_reg_indirect(vha, addr1, value1);
+
+ wait_count = 0;
+ while (wait_count < poll) {
+ qla8044_rd_reg_indirect(vha, addr1, &r_value);
+
+ if ((r_value & poll) != 0)
+ break;
+ wait_count++;
+ }
+
+ return QLA_SUCCESS;
+
+error:
+ return -1;
+}
+
/*
*
* qla8044_collect_md_data - Retrieve firmware minidump data.
@@ -3089,6 +3416,24 @@
if (rval != QLA_SUCCESS)
qla8044_mark_entry_skipped(vha, entry_hdr, i);
break;
+ case QLA8044_RDDFE:
+ rval = qla8044_minidump_process_rddfe(vha, entry_hdr,
+ &data_ptr);
+ if (rval != QLA_SUCCESS)
+ qla8044_mark_entry_skipped(vha, entry_hdr, i);
+ break;
+ case QLA8044_RDMDIO:
+ rval = qla8044_minidump_process_rdmdio(vha, entry_hdr,
+ &data_ptr);
+ if (rval != QLA_SUCCESS)
+ qla8044_mark_entry_skipped(vha, entry_hdr, i);
+ break;
+ case QLA8044_POLLWR:
+ rval = qla8044_minidump_process_pollwr(vha, entry_hdr,
+ &data_ptr);
+ if (rval != QLA_SUCCESS)
+ qla8044_mark_entry_skipped(vha, entry_hdr, i);
+ break;
case QLA82XX_RDNOP:
default:
qla8044_mark_entry_skipped(vha, entry_hdr, i);
@@ -3110,6 +3455,7 @@
"Dump data mismatch: Data collected: "
"[0x%x], total_data_size:[0x%x]\n",
data_collected, ha->md_dump_size);
+ rval = QLA_FUNCTION_FAILED;
goto md_failed;
}
@@ -3134,10 +3480,12 @@
if (!qla8044_collect_md_data(vha)) {
ha->fw_dumped = 1;
+ ha->prev_minidump_failed = 0;
} else {
ql_log(ql_log_fatal, vha, 0xb0db,
"%s: Unable to collect minidump\n",
__func__);
+ ha->prev_minidump_failed = 1;
}
}
diff --git a/drivers/scsi/qla2xxx/qla_nx2.h b/drivers/scsi/qla2xxx/qla_nx2.h
index 2ab2eab..ada3605 100644
--- a/drivers/scsi/qla2xxx/qla_nx2.h
+++ b/drivers/scsi/qla2xxx/qla_nx2.h
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -133,6 +133,7 @@
#define QLA8044_LINK_SPEED(f) (0x36E0+(((f) >> 2) * 4))
#define QLA8044_MAX_LINK_SPEED(f) (0x36F0+(((f) / 4) * 4))
#define QLA8044_LINK_SPEED_FACTOR 10
+#define QLA8044_FUN7_ACTIVE_INDEX 0x80
/* FLASH API Defines */
#define QLA8044_FLASH_MAX_WAIT_USEC 100
@@ -431,6 +432,50 @@
uint32_t rsvd_1;
} __packed;
+struct qla8044_minidump_entry_rddfe {
+ struct qla8044_minidump_entry_hdr h;
+ uint32_t addr_1;
+ uint32_t value;
+ uint8_t stride;
+ uint8_t stride2;
+ uint16_t count;
+ uint32_t poll;
+ uint32_t mask;
+ uint32_t modify_mask;
+ uint32_t data_size;
+ uint32_t rsvd;
+
+} __packed;
+
+struct qla8044_minidump_entry_rdmdio {
+ struct qla8044_minidump_entry_hdr h;
+
+ uint32_t addr_1;
+ uint32_t addr_2;
+ uint32_t value_1;
+ uint8_t stride_1;
+ uint8_t stride_2;
+ uint16_t count;
+ uint32_t poll;
+ uint32_t mask;
+ uint32_t value_2;
+ uint32_t data_size;
+
+} __packed;
+
+struct qla8044_minidump_entry_pollwr {
+ struct qla8044_minidump_entry_hdr h;
+ uint32_t addr_1;
+ uint32_t addr_2;
+ uint32_t value_1;
+ uint32_t value_2;
+ uint32_t poll;
+ uint32_t mask;
+ uint32_t data_size;
+ uint32_t rsvd;
+
+} __packed;
+
/* RDMUX2 Entry */
struct qla8044_minidump_entry_rdmux2 {
struct qla8044_minidump_entry_hdr h;
@@ -516,6 +561,9 @@
#define QLA8044_DBG_RSVD_ARRAY_LEN 8
#define QLA8044_DBG_OCM_WNDREG_ARRAY_LEN 16
#define QLA8044_SS_PCI_INDEX 0
+#define QLA8044_RDDFE 38
+#define QLA8044_RDMDIO 39
+#define QLA8044_POLLWR 40
struct qla8044_minidump_template_hdr {
uint32_t entry_type;
diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c
index afc8481..d96bfb5 100644
--- a/drivers/scsi/qla2xxx/qla_os.c
+++ b/drivers/scsi/qla2xxx/qla_os.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -616,7 +616,7 @@
if (sp->flags & SRB_CRC_CTX_DSD_VALID) {
/* List assured to be having elements */
- qla2x00_clean_dsd_pool(ha, sp);
+ qla2x00_clean_dsd_pool(ha, sp, NULL);
sp->flags &= ~SRB_CRC_CTX_DSD_VALID;
}
@@ -781,7 +781,7 @@
qla2x00_eh_wait_on_command(struct scsi_cmnd *cmd)
{
#define ABORT_POLLING_PERIOD 1000
-#define ABORT_WAIT_ITER ((10 * 1000) / (ABORT_POLLING_PERIOD))
+#define ABORT_WAIT_ITER ((2 * 1000) / (ABORT_POLLING_PERIOD))
unsigned long wait_iter = ABORT_WAIT_ITER;
scsi_qla_host_t *vha = shost_priv(cmd->device->host);
struct qla_hw_data *ha = vha->hw;
@@ -844,11 +844,8 @@
}
/*
- * qla2x00_wait_for_reset_ready
- * Wait till the HBA is online after going through
- * <= MAX_RETRIES_OF_ISP_ABORT or
- * finally HBA is disabled ie marked offline or flash
- * operations are in progress.
+ * qla2x00_wait_for_hba_ready
+ * Wait till the HBA is ready before doing driver unload
*
* Input:
* ha - pointer to host adapter structure
@@ -857,35 +854,15 @@
* Does context switching-Release SPIN_LOCK
* (if any) before calling this routine.
*
- * Return:
- * Success (Adapter is online/no flash ops) : 0
- * Failed (Adapter is offline/disabled/flash ops in progress) : 1
*/
-static int
-qla2x00_wait_for_reset_ready(scsi_qla_host_t *vha)
+static void
+qla2x00_wait_for_hba_ready(scsi_qla_host_t *vha)
{
- int return_status;
- unsigned long wait_online;
struct qla_hw_data *ha = vha->hw;
- scsi_qla_host_t *base_vha = pci_get_drvdata(ha->pdev);
- wait_online = jiffies + (MAX_LOOP_TIMEOUT * HZ);
- while (((test_bit(ISP_ABORT_NEEDED, &base_vha->dpc_flags)) ||
- test_bit(ABORT_ISP_ACTIVE, &base_vha->dpc_flags) ||
- test_bit(ISP_ABORT_RETRY, &base_vha->dpc_flags) ||
- ha->optrom_state != QLA_SWAITING ||
- ha->dpc_active) && time_before(jiffies, wait_online))
+ while ((!(vha->flags.online) || ha->dpc_active ||
+ ha->flags.mbox_busy))
msleep(1000);
-
- if (base_vha->flags.online && ha->optrom_state == QLA_SWAITING)
- return_status = QLA_SUCCESS;
- else
- return_status = QLA_FUNCTION_FAILED;
-
- ql_dbg(ql_dbg_taskm, vha, 0x8019,
- "%s return status=%d.\n", __func__, return_status);
-
- return return_status;
}
int
@@ -945,7 +922,7 @@
int ret;
unsigned int id, lun;
unsigned long flags;
- int wait = 0;
+ int rval, wait = 0;
struct qla_hw_data *ha = vha->hw;
if (!CMD_SP(cmd))
@@ -974,10 +951,20 @@
sp_get(sp);
spin_unlock_irqrestore(&ha->hardware_lock, flags);
- if (ha->isp_ops->abort_command(sp)) {
- ret = FAILED;
+ rval = ha->isp_ops->abort_command(sp);
+ if (rval) {
+ if (rval == QLA_FUNCTION_PARAMETER_ERROR) {
+ /*
+ * Decrement the ref_count since we can't find the
+ * command
+ */
+ atomic_dec(&sp->ref_count);
+ ret = SUCCESS;
+ } else
+ ret = FAILED;
+
ql_dbg(ql_dbg_taskm, vha, 0x8003,
- "Abort command mbx failed cmd=%p.\n", cmd);
+ "Abort command mbx failed cmd=%p, rval=%x.\n", cmd, rval);
} else {
ql_dbg(ql_dbg_taskm, vha, 0x8004,
"Abort command mbx success cmd=%p.\n", cmd);
@@ -985,6 +972,12 @@
}
spin_lock_irqsave(&ha->hardware_lock, flags);
+ /*
+ * Clear the slot in the oustanding_cmds array if we can't find the
+ * command to reclaim the resources.
+ */
+ if (rval == QLA_FUNCTION_PARAMETER_ERROR)
+ vha->req->outstanding_cmds[sp->handle] = NULL;
sp->done(ha, sp, 0);
spin_unlock_irqrestore(&ha->hardware_lock, flags);
@@ -1236,7 +1229,11 @@
ql_log(ql_log_info, vha, 0x8018,
"ADAPTER RESET ISSUED nexus=%ld:%d:%d.\n", vha->host_no, id, lun);
- if (qla2x00_wait_for_reset_ready(vha) != QLA_SUCCESS)
+ /*
+ * No point in issuing another reset if one is active. Also do not
+ * attempt a reset if we are updating flash.
+ */
+ if (qla2x00_reset_active(vha) || ha->optrom_state != QLA_SWAITING)
goto eh_host_reset_lock;
if (vha != base_vha) {
@@ -2270,6 +2267,13 @@
ha->device_type |= DT_IIDMA;
ha->fw_srisc_address = RISC_START_ADDRESS_2400;
break;
+ case PCI_DEVICE_ID_QLOGIC_ISP2271:
+ ha->device_type |= DT_ISP2271;
+ ha->device_type |= DT_ZIO_SUPPORTED;
+ ha->device_type |= DT_FWI2;
+ ha->device_type |= DT_IIDMA;
+ ha->fw_srisc_address = RISC_START_ADDRESS_2400;
+ break;
}
if (IS_QLA82XX(ha))
@@ -2346,7 +2350,8 @@
pdev->device == PCI_DEVICE_ID_QLOGIC_ISP8031 ||
pdev->device == PCI_DEVICE_ID_QLOGIC_ISPF001 ||
pdev->device == PCI_DEVICE_ID_QLOGIC_ISP8044 ||
- pdev->device == PCI_DEVICE_ID_QLOGIC_ISP2071) {
+ pdev->device == PCI_DEVICE_ID_QLOGIC_ISP2071 ||
+ pdev->device == PCI_DEVICE_ID_QLOGIC_ISP2271) {
bars = pci_select_bars(pdev, IORESOURCE_MEM);
mem_only = 1;
ql_dbg_pci(ql_dbg_init, pdev, 0x0007,
@@ -2877,6 +2882,7 @@
base_vha->flags.init_done = 1;
base_vha->flags.online = 1;
+ ha->prev_minidump_failed = 0;
ql_dbg(ql_dbg_init, base_vha, 0x00f2,
"Init done and hba is online.\n");
@@ -3136,6 +3142,8 @@
base_vha = pci_get_drvdata(pdev);
ha = base_vha->hw;
+ qla2x00_wait_for_hba_ready(base_vha);
+
set_bit(UNLOADING, &base_vha->dpc_flags);
if (IS_QLAFX00(ha))
@@ -3645,6 +3653,7 @@
ha->eft = NULL;
ha->eft_dma = 0;
ha->fw_dumped = 0;
+ ha->fw_dump_cap_flags = 0;
ha->fw_dump_reading = 0;
ha->fw_dump = NULL;
ha->fw_dump_len = 0;
@@ -4913,12 +4922,13 @@
if (qlafx00_reset_initialize(base_vha)) {
/* Failed. Abort isp later. */
if (!test_bit(UNLOADING,
- &base_vha->dpc_flags))
+ &base_vha->dpc_flags)) {
set_bit(ISP_UNRECOVERABLE,
&base_vha->dpc_flags);
ql_dbg(ql_dbg_dpc, base_vha,
0x4021,
"Reset Recovery Failed\n");
+ }
}
}
@@ -5077,8 +5087,10 @@
ha->isp_ops->enable_intrs(ha);
if (test_and_clear_bit(BEACON_BLINK_NEEDED,
- &base_vha->dpc_flags))
- ha->isp_ops->beacon_blink(base_vha);
+ &base_vha->dpc_flags)) {
+ if (ha->beacon_blink_led == 1)
+ ha->isp_ops->beacon_blink(base_vha);
+ }
if (!IS_QLAFX00(ha))
qla2x00_do_dpc_all_vps(base_vha);
@@ -5325,7 +5337,7 @@
#define FW_ISP82XX 7
#define FW_ISP2031 8
#define FW_ISP8031 9
-#define FW_ISP2071 10
+#define FW_ISP27XX 10
#define FW_FILE_ISP21XX "ql2100_fw.bin"
#define FW_FILE_ISP22XX "ql2200_fw.bin"
@@ -5337,7 +5349,7 @@
#define FW_FILE_ISP82XX "ql8200_fw.bin"
#define FW_FILE_ISP2031 "ql2600_fw.bin"
#define FW_FILE_ISP8031 "ql8300_fw.bin"
-#define FW_FILE_ISP2071 "ql2700_fw.bin"
+#define FW_FILE_ISP27XX "ql2700_fw.bin"
static DEFINE_MUTEX(qla_fw_lock);
@@ -5353,7 +5365,7 @@
{ .name = FW_FILE_ISP82XX, },
{ .name = FW_FILE_ISP2031, },
{ .name = FW_FILE_ISP8031, },
- { .name = FW_FILE_ISP2071, },
+ { .name = FW_FILE_ISP27XX, },
};
struct fw_blob *
@@ -5382,8 +5394,8 @@
blob = &qla_fw_blobs[FW_ISP2031];
} else if (IS_QLA8031(ha)) {
blob = &qla_fw_blobs[FW_ISP8031];
- } else if (IS_QLA2071(ha)) {
- blob = &qla_fw_blobs[FW_ISP2071];
+ } else if (IS_QLA27XX(ha)) {
+ blob = &qla_fw_blobs[FW_ISP27XX];
} else {
return NULL;
}
@@ -5714,6 +5726,7 @@
{ PCI_DEVICE(PCI_VENDOR_ID_QLOGIC, PCI_DEVICE_ID_QLOGIC_ISPF001) },
{ PCI_DEVICE(PCI_VENDOR_ID_QLOGIC, PCI_DEVICE_ID_QLOGIC_ISP8044) },
{ PCI_DEVICE(PCI_VENDOR_ID_QLOGIC, PCI_DEVICE_ID_QLOGIC_ISP2071) },
+ { PCI_DEVICE(PCI_VENDOR_ID_QLOGIC, PCI_DEVICE_ID_QLOGIC_ISP2271) },
{ 0 },
};
MODULE_DEVICE_TABLE(pci, qla2xxx_pci_tbl);
diff --git a/drivers/scsi/qla2xxx/qla_settings.h b/drivers/scsi/qla2xxx/qla_settings.h
index 46ef0ac..2fb7ebf 100644
--- a/drivers/scsi/qla2xxx/qla_settings.h
+++ b/drivers/scsi/qla2xxx/qla_settings.h
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
diff --git a/drivers/scsi/qla2xxx/qla_sup.c b/drivers/scsi/qla2xxx/qla_sup.c
index f28123e..bca173e 100644
--- a/drivers/scsi/qla2xxx/qla_sup.c
+++ b/drivers/scsi/qla2xxx/qla_sup.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -1727,11 +1727,8 @@
if (IS_QLA2031(ha)) {
led_select_value = qla83xx_select_led_port(ha);
- qla83xx_wr_reg(vha, led_select_value, 0x40002000);
- qla83xx_wr_reg(vha, led_select_value + 4, 0x40002000);
- msleep(1000);
- qla83xx_wr_reg(vha, led_select_value, 0x40004000);
- qla83xx_wr_reg(vha, led_select_value + 4, 0x40004000);
+ qla83xx_wr_reg(vha, led_select_value, 0x40000230);
+ qla83xx_wr_reg(vha, led_select_value + 4, 0x40000230);
} else if (IS_QLA8031(ha)) {
led_select_value = qla83xx_select_led_port(ha);
diff --git a/drivers/scsi/qla2xxx/qla_target.c b/drivers/scsi/qla2xxx/qla_target.c
index 0cb7307..b1d10f9 100644
--- a/drivers/scsi/qla2xxx/qla_target.c
+++ b/drivers/scsi/qla2xxx/qla_target.c
@@ -182,6 +182,11 @@
void qlt_24xx_atio_pkt_all_vps(struct scsi_qla_host *vha,
struct atio_from_isp *atio)
{
+ ql_dbg(ql_dbg_tgt, vha, 0xe072,
+ "%s: qla_target(%d): type %x ox_id %04x\n",
+ __func__, vha->vp_idx, atio->u.raw.entry_type,
+ be16_to_cpu(atio->u.isp24.fcp_hdr.ox_id));
+
switch (atio->u.raw.entry_type) {
case ATIO_TYPE7:
{
@@ -236,6 +241,10 @@
void qlt_response_pkt_all_vps(struct scsi_qla_host *vha, response_t *pkt)
{
switch (pkt->entry_type) {
+ case CTIO_CRC2:
+ ql_dbg(ql_dbg_tgt, vha, 0xe073,
+ "qla_target(%d):%s: CRC2 Response pkt\n",
+ vha->vp_idx, __func__);
case CTIO_TYPE7:
{
struct ctio7_from_24xx *entry = (struct ctio7_from_24xx *)pkt;
@@ -1350,13 +1359,42 @@
prm->cmd->sg_mapped = 1;
- /*
- * If greater than four sg entries then we need to allocate
- * the continuation entries
- */
- if (prm->seg_cnt > prm->tgt->datasegs_per_cmd)
- prm->req_cnt += DIV_ROUND_UP(prm->seg_cnt -
- prm->tgt->datasegs_per_cmd, prm->tgt->datasegs_per_cont);
+ if (cmd->se_cmd.prot_op == TARGET_PROT_NORMAL) {
+ /*
+ * If greater than four sg entries then we need to allocate
+ * the continuation entries
+ */
+ if (prm->seg_cnt > prm->tgt->datasegs_per_cmd)
+ prm->req_cnt += DIV_ROUND_UP(prm->seg_cnt -
+ prm->tgt->datasegs_per_cmd,
+ prm->tgt->datasegs_per_cont);
+ } else {
+ /* DIF */
+ if ((cmd->se_cmd.prot_op == TARGET_PROT_DIN_INSERT) ||
+ (cmd->se_cmd.prot_op == TARGET_PROT_DOUT_STRIP)) {
+ prm->seg_cnt = DIV_ROUND_UP(cmd->bufflen, cmd->blk_sz);
+ prm->tot_dsds = prm->seg_cnt;
+ } else
+ prm->tot_dsds = prm->seg_cnt;
+
+ if (cmd->prot_sg_cnt) {
+ prm->prot_sg = cmd->prot_sg;
+ prm->prot_seg_cnt = pci_map_sg(prm->tgt->ha->pdev,
+ cmd->prot_sg, cmd->prot_sg_cnt,
+ cmd->dma_data_direction);
+ if (unlikely(prm->prot_seg_cnt == 0))
+ goto out_err;
+
+ if ((cmd->se_cmd.prot_op == TARGET_PROT_DIN_INSERT) ||
+ (cmd->se_cmd.prot_op == TARGET_PROT_DOUT_STRIP)) {
+ /* Dif Bundling not support here */
+ prm->prot_seg_cnt = DIV_ROUND_UP(cmd->bufflen,
+ cmd->blk_sz);
+ prm->tot_dsds += prm->prot_seg_cnt;
+ } else
+ prm->tot_dsds += prm->prot_seg_cnt;
+ }
+ }
ql_dbg(ql_dbg_tgt, prm->cmd->vha, 0xe009, "seg_cnt=%d, req_cnt=%d\n",
prm->seg_cnt, prm->req_cnt);
@@ -1377,6 +1415,16 @@
BUG_ON(!cmd->sg_mapped);
pci_unmap_sg(ha->pdev, cmd->sg, cmd->sg_cnt, cmd->dma_data_direction);
cmd->sg_mapped = 0;
+
+ if (cmd->prot_sg_cnt)
+ pci_unmap_sg(ha->pdev, cmd->prot_sg, cmd->prot_sg_cnt,
+ cmd->dma_data_direction);
+
+ if (cmd->ctx_dsd_alloced)
+ qla2x00_clean_dsd_pool(ha, NULL, cmd);
+
+ if (cmd->ctx)
+ dma_pool_free(ha->dl_dma_pool, cmd->ctx, cmd->ctx->crc_ctx_dma);
}
static int qlt_check_reserve_free_req(struct scsi_qla_host *vha,
@@ -1665,8 +1713,9 @@
return QLA_TGT_PRE_XMIT_RESP_CMD_ABORTED;
}
- ql_dbg(ql_dbg_tgt, vha, 0xe011, "qla_target(%d): tag=%u\n",
- vha->vp_idx, cmd->tag);
+ ql_dbg(ql_dbg_tgt, vha, 0xe011, "qla_target(%d): tag=%u ox_id %04x\n",
+ vha->vp_idx, cmd->tag,
+ be16_to_cpu(cmd->atio.u.isp24.fcp_hdr.ox_id));
prm->cmd = cmd;
prm->tgt = tgt;
@@ -1902,6 +1951,323 @@
/* Sense with len > 24, is it possible ??? */
}
+
+
+/* diff */
+static inline int
+qlt_hba_err_chk_enabled(struct se_cmd *se_cmd)
+{
+ /*
+ * Uncomment when corresponding SCSI changes are done.
+ *
+ if (!sp->cmd->prot_chk)
+ return 0;
+ *
+ */
+ switch (se_cmd->prot_op) {
+ case TARGET_PROT_DOUT_INSERT:
+ case TARGET_PROT_DIN_STRIP:
+ if (ql2xenablehba_err_chk >= 1)
+ return 1;
+ break;
+ case TARGET_PROT_DOUT_PASS:
+ case TARGET_PROT_DIN_PASS:
+ if (ql2xenablehba_err_chk >= 2)
+ return 1;
+ break;
+ case TARGET_PROT_DIN_INSERT:
+ case TARGET_PROT_DOUT_STRIP:
+ return 1;
+ default:
+ break;
+ }
+ return 0;
+}
+
+/*
+ * qla24xx_set_t10dif_tags_from_cmd - Extract Ref and App tags from SCSI command
+ *
+ */
+static inline void
+qlt_set_t10dif_tags(struct se_cmd *se_cmd, struct crc_context *ctx)
+{
+ uint32_t lba = 0xffffffff & se_cmd->t_task_lba;
+
+ /* wait til Mode Sense/Select cmd, modepage Ah, subpage 2
+ * have been immplemented by TCM, before AppTag is avail.
+ * Look for modesense_handlers[]
+ */
+ ctx->app_tag = __constant_cpu_to_le16(0);
+ ctx->app_tag_mask[0] = 0x0;
+ ctx->app_tag_mask[1] = 0x0;
+
+ switch (se_cmd->prot_type) {
+ case TARGET_DIF_TYPE0_PROT:
+ /*
+ * No check for ql2xenablehba_err_chk, as it would be an
+ * I/O error if hba tag generation is not done.
+ */
+ ctx->ref_tag = cpu_to_le32(lba);
+
+ if (!qlt_hba_err_chk_enabled(se_cmd))
+ break;
+
+ /* enable ALL bytes of the ref tag */
+ ctx->ref_tag_mask[0] = 0xff;
+ ctx->ref_tag_mask[1] = 0xff;
+ ctx->ref_tag_mask[2] = 0xff;
+ ctx->ref_tag_mask[3] = 0xff;
+ break;
+ /*
+ * For TYpe 1 protection: 16 bit GUARD tag, 32 bit REF tag, and
+ * 16 bit app tag.
+ */
+ case TARGET_DIF_TYPE1_PROT:
+ ctx->ref_tag = cpu_to_le32(lba);
+
+ if (!qlt_hba_err_chk_enabled(se_cmd))
+ break;
+
+ /* enable ALL bytes of the ref tag */
+ ctx->ref_tag_mask[0] = 0xff;
+ ctx->ref_tag_mask[1] = 0xff;
+ ctx->ref_tag_mask[2] = 0xff;
+ ctx->ref_tag_mask[3] = 0xff;
+ break;
+ /*
+ * For TYPE 2 protection: 16 bit GUARD + 32 bit REF tag has to
+ * match LBA in CDB + N
+ */
+ case TARGET_DIF_TYPE2_PROT:
+ ctx->ref_tag = cpu_to_le32(lba);
+
+ if (!qlt_hba_err_chk_enabled(se_cmd))
+ break;
+
+ /* enable ALL bytes of the ref tag */
+ ctx->ref_tag_mask[0] = 0xff;
+ ctx->ref_tag_mask[1] = 0xff;
+ ctx->ref_tag_mask[2] = 0xff;
+ ctx->ref_tag_mask[3] = 0xff;
+ break;
+
+ /* For Type 3 protection: 16 bit GUARD only */
+ case TARGET_DIF_TYPE3_PROT:
+ ctx->ref_tag_mask[0] = ctx->ref_tag_mask[1] =
+ ctx->ref_tag_mask[2] = ctx->ref_tag_mask[3] = 0x00;
+ break;
+ }
+}
+
+
+static inline int
+qlt_build_ctio_crc2_pkt(struct qla_tgt_prm *prm, scsi_qla_host_t *vha)
+{
+ uint32_t *cur_dsd;
+ int sgc;
+ uint32_t transfer_length = 0;
+ uint32_t data_bytes;
+ uint32_t dif_bytes;
+ uint8_t bundling = 1;
+ uint8_t *clr_ptr;
+ struct crc_context *crc_ctx_pkt = NULL;
+ struct qla_hw_data *ha;
+ struct ctio_crc2_to_fw *pkt;
+ dma_addr_t crc_ctx_dma;
+ uint16_t fw_prot_opts = 0;
+ struct qla_tgt_cmd *cmd = prm->cmd;
+ struct se_cmd *se_cmd = &cmd->se_cmd;
+ uint32_t h;
+ struct atio_from_isp *atio = &prm->cmd->atio;
+
+ sgc = 0;
+ ha = vha->hw;
+
+ pkt = (struct ctio_crc2_to_fw *)vha->req->ring_ptr;
+ prm->pkt = pkt;
+ memset(pkt, 0, sizeof(*pkt));
+
+ ql_dbg(ql_dbg_tgt, vha, 0xe071,
+ "qla_target(%d):%s: se_cmd[%p] CRC2 prot_op[0x%x] cmd prot sg:cnt[%p:%x] lba[%llu]\n",
+ vha->vp_idx, __func__, se_cmd, se_cmd->prot_op,
+ prm->prot_sg, prm->prot_seg_cnt, se_cmd->t_task_lba);
+
+ if ((se_cmd->prot_op == TARGET_PROT_DIN_INSERT) ||
+ (se_cmd->prot_op == TARGET_PROT_DOUT_STRIP))
+ bundling = 0;
+
+ /* Compute dif len and adjust data len to incude protection */
+ data_bytes = cmd->bufflen;
+ dif_bytes = (data_bytes / cmd->blk_sz) * 8;
+
+ switch (se_cmd->prot_op) {
+ case TARGET_PROT_DIN_INSERT:
+ case TARGET_PROT_DOUT_STRIP:
+ transfer_length = data_bytes;
+ data_bytes += dif_bytes;
+ break;
+
+ case TARGET_PROT_DIN_STRIP:
+ case TARGET_PROT_DOUT_INSERT:
+ case TARGET_PROT_DIN_PASS:
+ case TARGET_PROT_DOUT_PASS:
+ transfer_length = data_bytes + dif_bytes;
+ break;
+
+ default:
+ BUG();
+ break;
+ }
+
+ if (!qlt_hba_err_chk_enabled(se_cmd))
+ fw_prot_opts |= 0x10; /* Disable Guard tag checking */
+ /* HBA error checking enabled */
+ else if (IS_PI_UNINIT_CAPABLE(ha)) {
+ if ((se_cmd->prot_type == TARGET_DIF_TYPE1_PROT) ||
+ (se_cmd->prot_type == TARGET_DIF_TYPE2_PROT))
+ fw_prot_opts |= PO_DIS_VALD_APP_ESC;
+ else if (se_cmd->prot_type == TARGET_DIF_TYPE3_PROT)
+ fw_prot_opts |= PO_DIS_VALD_APP_REF_ESC;
+ }
+
+ switch (se_cmd->prot_op) {
+ case TARGET_PROT_DIN_INSERT:
+ case TARGET_PROT_DOUT_INSERT:
+ fw_prot_opts |= PO_MODE_DIF_INSERT;
+ break;
+ case TARGET_PROT_DIN_STRIP:
+ case TARGET_PROT_DOUT_STRIP:
+ fw_prot_opts |= PO_MODE_DIF_REMOVE;
+ break;
+ case TARGET_PROT_DIN_PASS:
+ case TARGET_PROT_DOUT_PASS:
+ fw_prot_opts |= PO_MODE_DIF_PASS;
+ /* FUTURE: does tcm require T10CRC<->IPCKSUM conversion? */
+ break;
+ default:/* Normal Request */
+ fw_prot_opts |= PO_MODE_DIF_PASS;
+ break;
+ }
+
+
+ /* ---- PKT ---- */
+ /* Update entry type to indicate Command Type CRC_2 IOCB */
+ pkt->entry_type = CTIO_CRC2;
+ pkt->entry_count = 1;
+ pkt->vp_index = vha->vp_idx;
+
+ h = qlt_make_handle(vha);
+ if (unlikely(h == QLA_TGT_NULL_HANDLE)) {
+ /*
+ * CTIO type 7 from the firmware doesn't provide a way to
+ * know the initiator's LOOP ID, hence we can't find
+ * the session and, so, the command.
+ */
+ return -EAGAIN;
+ } else
+ ha->tgt.cmds[h-1] = prm->cmd;
+
+
+ pkt->handle = h | CTIO_COMPLETION_HANDLE_MARK;
+ pkt->nport_handle = prm->cmd->loop_id;
+ pkt->timeout = __constant_cpu_to_le16(QLA_TGT_TIMEOUT);
+ pkt->initiator_id[0] = atio->u.isp24.fcp_hdr.s_id[2];
+ pkt->initiator_id[1] = atio->u.isp24.fcp_hdr.s_id[1];
+ pkt->initiator_id[2] = atio->u.isp24.fcp_hdr.s_id[0];
+ pkt->exchange_addr = atio->u.isp24.exchange_addr;
+ pkt->ox_id = swab16(atio->u.isp24.fcp_hdr.ox_id);
+ pkt->flags |= (atio->u.isp24.attr << 9);
+ pkt->relative_offset = cpu_to_le32(prm->cmd->offset);
+
+ /* Set transfer direction */
+ if (cmd->dma_data_direction == DMA_TO_DEVICE)
+ pkt->flags = __constant_cpu_to_le16(CTIO7_FLAGS_DATA_IN);
+ else if (cmd->dma_data_direction == DMA_FROM_DEVICE)
+ pkt->flags = __constant_cpu_to_le16(CTIO7_FLAGS_DATA_OUT);
+
+
+ pkt->dseg_count = prm->tot_dsds;
+ /* Fibre channel byte count */
+ pkt->transfer_length = cpu_to_le32(transfer_length);
+
+
+ /* ----- CRC context -------- */
+
+ /* Allocate CRC context from global pool */
+ crc_ctx_pkt = cmd->ctx =
+ dma_pool_alloc(ha->dl_dma_pool, GFP_ATOMIC, &crc_ctx_dma);
+
+ if (!crc_ctx_pkt)
+ goto crc_queuing_error;
+
+ /* Zero out CTX area. */
+ clr_ptr = (uint8_t *)crc_ctx_pkt;
+ memset(clr_ptr, 0, sizeof(*crc_ctx_pkt));
+
+ crc_ctx_pkt->crc_ctx_dma = crc_ctx_dma;
+ INIT_LIST_HEAD(&crc_ctx_pkt->dsd_list);
+
+ /* Set handle */
+ crc_ctx_pkt->handle = pkt->handle;
+
+ qlt_set_t10dif_tags(se_cmd, crc_ctx_pkt);
+
+ pkt->crc_context_address[0] = cpu_to_le32(LSD(crc_ctx_dma));
+ pkt->crc_context_address[1] = cpu_to_le32(MSD(crc_ctx_dma));
+ pkt->crc_context_len = CRC_CONTEXT_LEN_FW;
+
+
+ if (!bundling) {
+ cur_dsd = (uint32_t *) &crc_ctx_pkt->u.nobundling.data_address;
+ } else {
+ /*
+ * Configure Bundling if we need to fetch interlaving
+ * protection PCI accesses
+ */
+ fw_prot_opts |= PO_ENABLE_DIF_BUNDLING;
+ crc_ctx_pkt->u.bundling.dif_byte_count = cpu_to_le32(dif_bytes);
+ crc_ctx_pkt->u.bundling.dseg_count =
+ cpu_to_le16(prm->tot_dsds - prm->prot_seg_cnt);
+ cur_dsd = (uint32_t *) &crc_ctx_pkt->u.bundling.data_address;
+ }
+
+ /* Finish the common fields of CRC pkt */
+ crc_ctx_pkt->blk_size = cpu_to_le16(cmd->blk_sz);
+ crc_ctx_pkt->prot_opts = cpu_to_le16(fw_prot_opts);
+ crc_ctx_pkt->byte_count = cpu_to_le32(data_bytes);
+ crc_ctx_pkt->guard_seed = __constant_cpu_to_le16(0);
+
+
+ /* Walks data segments */
+ pkt->flags |= __constant_cpu_to_le16(CTIO7_FLAGS_DSD_PTR);
+
+ if (!bundling && prm->prot_seg_cnt) {
+ if (qla24xx_walk_and_build_sglist_no_difb(ha, NULL, cur_dsd,
+ prm->tot_dsds, cmd))
+ goto crc_queuing_error;
+ } else if (qla24xx_walk_and_build_sglist(ha, NULL, cur_dsd,
+ (prm->tot_dsds - prm->prot_seg_cnt), cmd))
+ goto crc_queuing_error;
+
+ if (bundling && prm->prot_seg_cnt) {
+ /* Walks dif segments */
+ pkt->add_flags |=
+ __constant_cpu_to_le16(CTIO_CRC2_AF_DIF_DSD_ENA);
+
+ cur_dsd = (uint32_t *) &crc_ctx_pkt->u.bundling.dif_address;
+ if (qla24xx_walk_and_build_prot_sglist(ha, NULL, cur_dsd,
+ prm->prot_seg_cnt, cmd))
+ goto crc_queuing_error;
+ }
+ return QLA_SUCCESS;
+
+crc_queuing_error:
+ /* Cleanup will be performed by the caller */
+
+ return QLA_FUNCTION_FAILED;
+}
+
+
/*
* Callback to setup response of xmit_type of QLA_TGT_XMIT_DATA and *
* QLA_TGT_XMIT_STATUS for >= 24xx silicon
@@ -1921,9 +2287,10 @@
qlt_check_srr_debug(cmd, &xmit_type);
ql_dbg(ql_dbg_tgt, cmd->vha, 0xe018,
- "is_send_status=%d, cmd->bufflen=%d, cmd->sg_cnt=%d, "
- "cmd->dma_data_direction=%d\n", (xmit_type & QLA_TGT_XMIT_STATUS) ?
- 1 : 0, cmd->bufflen, cmd->sg_cnt, cmd->dma_data_direction);
+ "is_send_status=%d, cmd->bufflen=%d, cmd->sg_cnt=%d, cmd->dma_data_direction=%d se_cmd[%p]\n",
+ (xmit_type & QLA_TGT_XMIT_STATUS) ?
+ 1 : 0, cmd->bufflen, cmd->sg_cnt, cmd->dma_data_direction,
+ &cmd->se_cmd);
res = qlt_pre_xmit_response(cmd, &prm, xmit_type, scsi_status,
&full_req_cnt);
@@ -1941,7 +2308,10 @@
if (unlikely(res))
goto out_unmap_unlock;
- res = qlt_24xx_build_ctio_pkt(&prm, vha);
+ if (cmd->se_cmd.prot_op && (xmit_type & QLA_TGT_XMIT_DATA))
+ res = qlt_build_ctio_crc2_pkt(&prm, vha);
+ else
+ res = qlt_24xx_build_ctio_pkt(&prm, vha);
if (unlikely(res != 0))
goto out_unmap_unlock;
@@ -1953,7 +2323,8 @@
__constant_cpu_to_le16(CTIO7_FLAGS_DATA_IN |
CTIO7_FLAGS_STATUS_MODE_0);
- qlt_load_data_segments(&prm, vha);
+ if (cmd->se_cmd.prot_op == TARGET_PROT_NORMAL)
+ qlt_load_data_segments(&prm, vha);
if (prm.add_status_pkt == 0) {
if (xmit_type & QLA_TGT_XMIT_STATUS) {
@@ -1983,8 +2354,14 @@
ql_dbg(ql_dbg_tgt, vha, 0xe019,
"Building additional status packet\n");
+ /*
+ * T10Dif: ctio_crc2_to_fw overlay ontop of
+ * ctio7_to_24xx
+ */
memcpy(ctio, pkt, sizeof(*ctio));
+ /* reset back to CTIO7 */
ctio->entry_count = 1;
+ ctio->entry_type = CTIO_TYPE7;
ctio->dseg_count = 0;
ctio->u.status1.flags &= ~__constant_cpu_to_le16(
CTIO7_FLAGS_DATA_IN);
@@ -1993,6 +2370,11 @@
pkt->handle |= CTIO_INTERMEDIATE_HANDLE_MARK;
pkt->u.status0.flags |= __constant_cpu_to_le16(
CTIO7_FLAGS_DONT_RET_CTIO);
+
+ /* qlt_24xx_init_ctio_to_isp will correct
+ * all neccessary fields that's part of CTIO7.
+ * There should be no residual of CTIO-CRC2 data.
+ */
qlt_24xx_init_ctio_to_isp((struct ctio7_to_24xx *)ctio,
&prm);
pr_debug("Status CTIO7: %p\n", ctio);
@@ -2041,8 +2423,10 @@
if (qlt_issue_marker(vha, 0) != QLA_SUCCESS)
return -EIO;
- ql_dbg(ql_dbg_tgt, vha, 0xe01b, "CTIO_start: vha(%d)",
- (int)vha->vp_idx);
+ ql_dbg(ql_dbg_tgt, vha, 0xe01b,
+ "%s: CTIO_start: vha(%d) se_cmd %p ox_id %04x\n",
+ __func__, (int)vha->vp_idx, &cmd->se_cmd,
+ be16_to_cpu(cmd->atio.u.isp24.fcp_hdr.ox_id));
/* Calculate number of entries and segments required */
if (qlt_pci_map_calc_cnt(&prm) != 0)
@@ -2054,14 +2438,19 @@
res = qlt_check_reserve_free_req(vha, prm.req_cnt);
if (res != 0)
goto out_unlock_free_unmap;
+ if (cmd->se_cmd.prot_op)
+ res = qlt_build_ctio_crc2_pkt(&prm, vha);
+ else
+ res = qlt_24xx_build_ctio_pkt(&prm, vha);
- res = qlt_24xx_build_ctio_pkt(&prm, vha);
if (unlikely(res != 0))
goto out_unlock_free_unmap;
pkt = (struct ctio7_to_24xx *)prm.pkt;
pkt->u.status0.flags |= __constant_cpu_to_le16(CTIO7_FLAGS_DATA_OUT |
CTIO7_FLAGS_STATUS_MODE_0);
- qlt_load_data_segments(&prm, vha);
+
+ if (cmd->se_cmd.prot_op == TARGET_PROT_NORMAL)
+ qlt_load_data_segments(&prm, vha);
cmd->state = QLA_TGT_STATE_NEED_DATA;
@@ -2079,6 +2468,143 @@
}
EXPORT_SYMBOL(qlt_rdy_to_xfer);
+
+/*
+ * Checks the guard or meta-data for the type of error
+ * detected by the HBA.
+ */
+static inline int
+qlt_handle_dif_error(struct scsi_qla_host *vha, struct qla_tgt_cmd *cmd,
+ struct ctio_crc_from_fw *sts)
+{
+ uint8_t *ap = &sts->actual_dif[0];
+ uint8_t *ep = &sts->expected_dif[0];
+ uint32_t e_ref_tag, a_ref_tag;
+ uint16_t e_app_tag, a_app_tag;
+ uint16_t e_guard, a_guard;
+ uint64_t lba = cmd->se_cmd.t_task_lba;
+
+ a_guard = be16_to_cpu(*(uint16_t *)(ap + 0));
+ a_app_tag = be16_to_cpu(*(uint16_t *)(ap + 2));
+ a_ref_tag = be32_to_cpu(*(uint32_t *)(ap + 4));
+
+ e_guard = be16_to_cpu(*(uint16_t *)(ep + 0));
+ e_app_tag = be16_to_cpu(*(uint16_t *)(ep + 2));
+ e_ref_tag = be32_to_cpu(*(uint32_t *)(ep + 4));
+
+ ql_dbg(ql_dbg_tgt, vha, 0xe075,
+ "iocb(s) %p Returned STATUS.\n", sts);
+
+ ql_dbg(ql_dbg_tgt, vha, 0xf075,
+ "dif check TGT cdb 0x%x lba 0x%llu: [Actual|Expected] Ref Tag[0x%x|0x%x], App Tag [0x%x|0x%x], Guard [0x%x|0x%x]\n",
+ cmd->atio.u.isp24.fcp_cmnd.cdb[0], lba,
+ a_ref_tag, e_ref_tag, a_app_tag, e_app_tag, a_guard, e_guard);
+
+ /*
+ * Ignore sector if:
+ * For type 3: ref & app tag is all 'f's
+ * For type 0,1,2: app tag is all 'f's
+ */
+ if ((a_app_tag == 0xffff) &&
+ ((cmd->se_cmd.prot_type != TARGET_DIF_TYPE3_PROT) ||
+ (a_ref_tag == 0xffffffff))) {
+ uint32_t blocks_done;
+
+ /* 2TB boundary case covered automatically with this */
+ blocks_done = e_ref_tag - (uint32_t)lba + 1;
+ cmd->se_cmd.bad_sector = e_ref_tag;
+ cmd->se_cmd.pi_err = 0;
+ ql_dbg(ql_dbg_tgt, vha, 0xf074,
+ "need to return scsi good\n");
+
+ /* Update protection tag */
+ if (cmd->prot_sg_cnt) {
+ uint32_t i, j = 0, k = 0, num_ent;
+ struct scatterlist *sg, *sgl;
+
+
+ sgl = cmd->prot_sg;
+
+ /* Patch the corresponding protection tags */
+ for_each_sg(sgl, sg, cmd->prot_sg_cnt, i) {
+ num_ent = sg_dma_len(sg) / 8;
+ if (k + num_ent < blocks_done) {
+ k += num_ent;
+ continue;
+ }
+ j = blocks_done - k - 1;
+ k = blocks_done;
+ break;
+ }
+
+ if (k != blocks_done) {
+ ql_log(ql_log_warn, vha, 0xf076,
+ "unexpected tag values tag:lba=%u:%llu)\n",
+ e_ref_tag, (unsigned long long)lba);
+ goto out;
+ }
+
+#if 0
+ struct sd_dif_tuple *spt;
+ /* TODO:
+ * This section came from initiator. Is it valid here?
+ * should ulp be override with actual val???
+ */
+ spt = page_address(sg_page(sg)) + sg->offset;
+ spt += j;
+
+ spt->app_tag = 0xffff;
+ if (cmd->se_cmd.prot_type == SCSI_PROT_DIF_TYPE3)
+ spt->ref_tag = 0xffffffff;
+#endif
+ }
+
+ return 0;
+ }
+
+ /* check guard */
+ if (e_guard != a_guard) {
+ cmd->se_cmd.pi_err = TCM_LOGICAL_BLOCK_GUARD_CHECK_FAILED;
+ cmd->se_cmd.bad_sector = cmd->se_cmd.t_task_lba;
+
+ ql_log(ql_log_warn, vha, 0xe076,
+ "Guard ERR: cdb 0x%x lba 0x%llx: [Actual|Expected] Ref Tag[0x%x|0x%x], App Tag [0x%x|0x%x], Guard [0x%x|0x%x] cmd=%p\n",
+ cmd->atio.u.isp24.fcp_cmnd.cdb[0], lba,
+ a_ref_tag, e_ref_tag, a_app_tag, e_app_tag,
+ a_guard, e_guard, cmd);
+ goto out;
+ }
+
+ /* check ref tag */
+ if (e_ref_tag != a_ref_tag) {
+ cmd->se_cmd.pi_err = TCM_LOGICAL_BLOCK_REF_TAG_CHECK_FAILED;
+ cmd->se_cmd.bad_sector = e_ref_tag;
+
+ ql_log(ql_log_warn, vha, 0xe077,
+ "Ref Tag ERR: cdb 0x%x lba 0x%llx: [Actual|Expected] Ref Tag[0x%x|0x%x], App Tag [0x%x|0x%x], Guard [0x%x|0x%x] cmd=%p\n",
+ cmd->atio.u.isp24.fcp_cmnd.cdb[0], lba,
+ a_ref_tag, e_ref_tag, a_app_tag, e_app_tag,
+ a_guard, e_guard, cmd);
+ goto out;
+ }
+
+ /* check appl tag */
+ if (e_app_tag != a_app_tag) {
+ cmd->se_cmd.pi_err = TCM_LOGICAL_BLOCK_APP_TAG_CHECK_FAILED;
+ cmd->se_cmd.bad_sector = cmd->se_cmd.t_task_lba;
+
+ ql_log(ql_log_warn, vha, 0xe078,
+ "App Tag ERR: cdb 0x%x lba 0x%llx: [Actual|Expected] Ref Tag[0x%x|0x%x], App Tag [0x%x|0x%x], Guard [0x%x|0x%x] cmd=%p\n",
+ cmd->atio.u.isp24.fcp_cmnd.cdb[0], lba,
+ a_ref_tag, e_ref_tag, a_app_tag, e_app_tag,
+ a_guard, e_guard, cmd);
+ goto out;
+ }
+out:
+ return 1;
+}
+
+
/* If hardware_lock held on entry, might drop it, then reaquire */
/* This function sends the appropriate CTIO to ISP 2xxx or 24xx */
static int __qlt_send_term_exchange(struct scsi_qla_host *vha,
@@ -2155,18 +2681,36 @@
rc = __qlt_send_term_exchange(vha, cmd, atio);
spin_unlock_irqrestore(&vha->hw->hardware_lock, flags);
done:
- if (rc == 1) {
+ /*
+ * Terminate exchange will tell fw to release any active CTIO
+ * that's in FW posession and cleanup the exchange.
+ *
+ * "cmd->state == QLA_TGT_STATE_ABORTED" means CTIO is still
+ * down at FW. Free the cmd later when CTIO comes back later
+ * w/aborted(0x2) status.
+ *
+ * "cmd->state != QLA_TGT_STATE_ABORTED" means CTIO is already
+ * back w/some err. Free the cmd now.
+ */
+ if ((rc == 1) && (cmd->state != QLA_TGT_STATE_ABORTED)) {
if (!ha_locked && !in_interrupt())
msleep(250); /* just in case */
+ if (cmd->sg_mapped)
+ qlt_unmap_sg(vha, cmd);
vha->hw->tgt.tgt_ops->free_cmd(cmd);
}
+ return;
}
void qlt_free_cmd(struct qla_tgt_cmd *cmd)
{
- BUG_ON(cmd->sg_mapped);
+ ql_dbg(ql_dbg_tgt, cmd->vha, 0xe074,
+ "%s: se_cmd[%p] ox_id %04x\n",
+ __func__, &cmd->se_cmd,
+ be16_to_cpu(cmd->atio.u.isp24.fcp_hdr.ox_id));
+ BUG_ON(cmd->sg_mapped);
if (unlikely(cmd->free_sg))
kfree(cmd->sg);
kmem_cache_free(qla_tgt_cmd_cachep, cmd);
@@ -2374,6 +2918,7 @@
case CTIO_LIP_RESET:
case CTIO_TARGET_RESET:
case CTIO_ABORTED:
+ /* driver request abort via Terminate exchange */
case CTIO_TIMEOUT:
case CTIO_INVALID_RX_ID:
/* They are OK */
@@ -2404,18 +2949,58 @@
else
return;
+ case CTIO_DIF_ERROR: {
+ struct ctio_crc_from_fw *crc =
+ (struct ctio_crc_from_fw *)ctio;
+ ql_dbg(ql_dbg_tgt_mgt, vha, 0xf073,
+ "qla_target(%d): CTIO with DIF_ERROR status %x received (state %x, se_cmd %p) actual_dif[0x%llx] expect_dif[0x%llx]\n",
+ vha->vp_idx, status, cmd->state, se_cmd,
+ *((u64 *)&crc->actual_dif[0]),
+ *((u64 *)&crc->expected_dif[0]));
+
+ if (qlt_handle_dif_error(vha, cmd, ctio)) {
+ if (cmd->state == QLA_TGT_STATE_NEED_DATA) {
+ /* scsi Write/xfer rdy complete */
+ goto skip_term;
+ } else {
+ /* scsi read/xmit respond complete
+ * call handle dif to send scsi status
+ * rather than terminate exchange.
+ */
+ cmd->state = QLA_TGT_STATE_PROCESSED;
+ ha->tgt.tgt_ops->handle_dif_err(cmd);
+ return;
+ }
+ } else {
+ /* Need to generate a SCSI good completion.
+ * because FW did not send scsi status.
+ */
+ status = 0;
+ goto skip_term;
+ }
+ break;
+ }
default:
ql_dbg(ql_dbg_tgt_mgt, vha, 0xf05b,
- "qla_target(%d): CTIO with error status "
- "0x%x received (state %x, se_cmd %p\n",
+ "qla_target(%d): CTIO with error status 0x%x received (state %x, se_cmd %p\n",
vha->vp_idx, status, cmd->state, se_cmd);
break;
}
- if (cmd->state != QLA_TGT_STATE_NEED_DATA)
+
+ /* "cmd->state == QLA_TGT_STATE_ABORTED" means
+ * cmd is already aborted/terminated, we don't
+ * need to terminate again. The exchange is already
+ * cleaned up/freed at FW level. Just cleanup at driver
+ * level.
+ */
+ if ((cmd->state != QLA_TGT_STATE_NEED_DATA) &&
+ (cmd->state != QLA_TGT_STATE_ABORTED)) {
if (qlt_term_ctio_exchange(vha, ctio, cmd, status))
return;
+ }
}
+skip_term:
if (cmd->state == QLA_TGT_STATE_PROCESSED) {
ql_dbg(ql_dbg_tgt, vha, 0xe01f, "Command %p finished\n", cmd);
@@ -2444,7 +3029,8 @@
"not return a CTIO complete\n", vha->vp_idx, cmd->state);
}
- if (unlikely(status != CTIO_SUCCESS)) {
+ if (unlikely(status != CTIO_SUCCESS) &&
+ (cmd->state != QLA_TGT_STATE_ABORTED)) {
ql_dbg(ql_dbg_tgt_mgt, vha, 0xf01f, "Finishing failed CTIO\n");
dump_stack();
}
@@ -2563,8 +3149,9 @@
atio->u.isp24.fcp_cmnd.add_cdb_len]));
ql_dbg(ql_dbg_tgt, vha, 0xe022,
- "qla_target: START qla command: %p lun: 0x%04x (tag %d)\n",
- cmd, cmd->unpacked_lun, cmd->tag);
+ "qla_target: START qla cmd: %p se_cmd %p lun: 0x%04x (tag %d) len(%d) ox_id %x\n",
+ cmd, &cmd->se_cmd, cmd->unpacked_lun, cmd->tag, data_length,
+ cmd->atio.u.isp24.fcp_hdr.ox_id);
ret = vha->hw->tgt.tgt_ops->handle_cmd(vha, cmd, cdb, data_length,
fcp_task_attr, data_dir, bidi);
@@ -3527,11 +4114,11 @@
switch (atio->u.raw.entry_type) {
case ATIO_TYPE7:
ql_dbg(ql_dbg_tgt, vha, 0xe02d,
- "ATIO_TYPE7 instance %d, lun %Lx, read/write %d/%d, "
- "add_cdb_len %d, data_length %04x, s_id %x:%x:%x\n",
+ "ATIO_TYPE7 instance %d, lun %Lx, read/write %d/%d, cdb %x, add_cdb_len %x, data_length %04x, s_id %02x%02x%02x\n",
vha->vp_idx, atio->u.isp24.fcp_cmnd.lun,
atio->u.isp24.fcp_cmnd.rddata,
atio->u.isp24.fcp_cmnd.wrdata,
+ atio->u.isp24.fcp_cmnd.cdb[0],
atio->u.isp24.fcp_cmnd.add_cdb_len,
be32_to_cpu(get_unaligned((uint32_t *)
&atio->u.isp24.fcp_cmnd.add_cdb[
@@ -3629,11 +4216,13 @@
tgt->irq_cmd_count++;
switch (pkt->entry_type) {
+ case CTIO_CRC2:
case CTIO_TYPE7:
{
struct ctio7_from_24xx *entry = (struct ctio7_from_24xx *)pkt;
- ql_dbg(ql_dbg_tgt, vha, 0xe030, "CTIO_TYPE7: instance %d\n",
- vha->vp_idx);
+ ql_dbg(ql_dbg_tgt, vha, 0xe030,
+ "CTIO[0x%x] 12/CTIO7 7A/CRC2: instance %d\n",
+ entry->entry_type, vha->vp_idx);
qlt_do_ctio_completion(vha, entry->handle,
le16_to_cpu(entry->status)|(pkt->entry_status << 16),
entry);
@@ -4768,6 +5357,7 @@
case ABTS_RESP_24XX:
case CTIO_TYPE7:
case NOTIFY_ACK_TYPE:
+ case CTIO_CRC2:
return 1;
default:
return 0;
diff --git a/drivers/scsi/qla2xxx/qla_target.h b/drivers/scsi/qla2xxx/qla_target.h
index ce33d8c..f873e10 100644
--- a/drivers/scsi/qla2xxx/qla_target.h
+++ b/drivers/scsi/qla2xxx/qla_target.h
@@ -293,6 +293,7 @@
#define CTIO_ABORTED 0x02
#define CTIO_INVALID_RX_ID 0x08
#define CTIO_TIMEOUT 0x0B
+#define CTIO_DIF_ERROR 0x0C /* DIF error detected */
#define CTIO_LIP_RESET 0x0E
#define CTIO_TARGET_RESET 0x17
#define CTIO_PORT_UNAVAILABLE 0x28
@@ -498,11 +499,12 @@
#define CTIO7_FLAGS_DONT_RET_CTIO BIT_8
#define CTIO7_FLAGS_STATUS_MODE_0 0
#define CTIO7_FLAGS_STATUS_MODE_1 BIT_6
+#define CTIO7_FLAGS_STATUS_MODE_2 BIT_7
#define CTIO7_FLAGS_EXPLICIT_CONFORM BIT_5
#define CTIO7_FLAGS_CONFIRM_SATISF BIT_4
#define CTIO7_FLAGS_DSD_PTR BIT_2
-#define CTIO7_FLAGS_DATA_IN BIT_1
-#define CTIO7_FLAGS_DATA_OUT BIT_0
+#define CTIO7_FLAGS_DATA_IN BIT_1 /* data to initiator */
+#define CTIO7_FLAGS_DATA_OUT BIT_0 /* data from initiator */
#define ELS_PLOGI 0x3
#define ELS_FLOGI 0x4
@@ -514,6 +516,68 @@
#define ELS_ADISC 0x52
/*
+ *CTIO Type CRC_2 IOCB
+ */
+struct ctio_crc2_to_fw {
+ uint8_t entry_type; /* Entry type. */
+#define CTIO_CRC2 0x7A
+ uint8_t entry_count; /* Entry count. */
+ uint8_t sys_define; /* System defined. */
+ uint8_t entry_status; /* Entry Status. */
+
+ uint32_t handle; /* System handle. */
+ uint16_t nport_handle; /* N_PORT handle. */
+ uint16_t timeout; /* Command timeout. */
+
+ uint16_t dseg_count; /* Data segment count. */
+ uint8_t vp_index;
+ uint8_t add_flags; /* additional flags */
+#define CTIO_CRC2_AF_DIF_DSD_ENA BIT_3
+
+ uint8_t initiator_id[3]; /* initiator ID */
+ uint8_t reserved1;
+ uint32_t exchange_addr; /* rcv exchange address */
+ uint16_t reserved2;
+ uint16_t flags; /* refer to CTIO7 flags values */
+ uint32_t residual;
+ uint16_t ox_id;
+ uint16_t scsi_status;
+ uint32_t relative_offset;
+ uint32_t reserved5;
+ uint32_t transfer_length; /* total fc transfer length */
+ uint32_t reserved6;
+ uint32_t crc_context_address[2];/* Data segment address. */
+ uint16_t crc_context_len; /* Data segment length. */
+ uint16_t reserved_1; /* MUST be set to 0. */
+} __packed;
+
+/* CTIO Type CRC_x Status IOCB */
+struct ctio_crc_from_fw {
+ uint8_t entry_type; /* Entry type. */
+ uint8_t entry_count; /* Entry count. */
+ uint8_t sys_define; /* System defined. */
+ uint8_t entry_status; /* Entry Status. */
+
+ uint32_t handle; /* System handle. */
+ uint16_t status;
+ uint16_t timeout; /* Command timeout. */
+ uint16_t dseg_count; /* Data segment count. */
+ uint32_t reserved1;
+ uint16_t state_flags;
+#define CTIO_CRC_SF_DIF_CHOPPED BIT_4
+
+ uint32_t exchange_address; /* rcv exchange address */
+ uint16_t reserved2;
+ uint16_t flags;
+ uint32_t resid_xfer_length;
+ uint16_t ox_id;
+ uint8_t reserved3[12];
+ uint16_t runt_guard; /* reported runt blk guard */
+ uint8_t actual_dif[8];
+ uint8_t expected_dif[8];
+} __packed;
+
+/*
* ISP queue - ABTS received/response entries structure definition for 24xx.
*/
#define ABTS_RECV_24XX 0x54 /* ABTS received (for 24xx) */
@@ -641,6 +705,7 @@
int (*handle_cmd)(struct scsi_qla_host *, struct qla_tgt_cmd *,
unsigned char *, uint32_t, int, int, int);
void (*handle_data)(struct qla_tgt_cmd *);
+ void (*handle_dif_err)(struct qla_tgt_cmd *);
int (*handle_tmr)(struct qla_tgt_mgmt_cmd *, uint32_t, uint8_t,
uint32_t);
void (*free_cmd)(struct qla_tgt_cmd *);
@@ -829,9 +894,9 @@
};
struct qla_tgt_cmd {
+ struct se_cmd se_cmd;
struct qla_tgt_sess *sess;
int state;
- struct se_cmd se_cmd;
struct work_struct free_work;
struct work_struct work;
/* Sense buffer that will be mapped into outgoing status */
@@ -843,6 +908,7 @@
unsigned int free_sg:1;
unsigned int aborted:1; /* Needed in case of SRR */
unsigned int write_data_transferred:1;
+ unsigned int ctx_dsd_alloced:1;
struct scatterlist *sg; /* cmd data buffer SG vector */
int sg_cnt; /* SG segments count */
@@ -857,6 +923,12 @@
struct scsi_qla_host *vha;
struct atio_from_isp atio;
+ /* t10dif */
+ struct scatterlist *prot_sg;
+ uint32_t prot_sg_cnt;
+ uint32_t blk_sz;
+ struct crc_context *ctx;
+
};
struct qla_tgt_sess_work_param {
@@ -901,6 +973,10 @@
int sense_buffer_len;
int residual;
int add_status_pkt;
+ /* dif */
+ struct scatterlist *prot_sg;
+ uint16_t prot_seg_cnt;
+ uint16_t tot_dsds;
};
struct qla_tgt_srr_imm {
@@ -976,6 +1052,8 @@
extern void qlt_response_pkt_all_vps(struct scsi_qla_host *, response_t *);
extern int qlt_rdy_to_xfer(struct qla_tgt_cmd *);
extern int qlt_xmit_response(struct qla_tgt_cmd *, int, uint8_t);
+extern int qlt_rdy_to_xfer_dif(struct qla_tgt_cmd *);
+extern int qlt_xmit_response_dif(struct qla_tgt_cmd *, int, uint8_t);
extern void qlt_xmit_tm_rsp(struct qla_tgt_mgmt_cmd *);
extern void qlt_free_mcmd(struct qla_tgt_mgmt_cmd *);
extern void qlt_free_cmd(struct qla_tgt_cmd *cmd);
diff --git a/drivers/scsi/qla2xxx/qla_tmpl.c b/drivers/scsi/qla2xxx/qla_tmpl.c
index a804e9b..cb9a0c4 100644
--- a/drivers/scsi/qla2xxx/qla_tmpl.c
+++ b/drivers/scsi/qla2xxx/qla_tmpl.c
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -201,7 +201,6 @@
ql_dbg(ql_dbg_misc, NULL, 0xd014,
"%s: @%x\n", __func__, offset);
}
- qla27xx_insert32(offset, buf, len);
qla27xx_read32(window, buf, len);
}
@@ -220,7 +219,7 @@
static inline void
qla27xx_read_window(__iomem struct device_reg_24xx *reg,
- uint32_t base, uint offset, uint count, uint width, void *buf,
+ uint32_t addr, uint offset, uint count, uint width, void *buf,
ulong *len)
{
void *window = (void *)reg + offset;
@@ -229,14 +228,14 @@
if (buf) {
ql_dbg(ql_dbg_misc, NULL, 0xd016,
"%s: base=%x offset=%x count=%x width=%x\n",
- __func__, base, offset, count, width);
+ __func__, addr, offset, count, width);
}
- qla27xx_write_reg(reg, IOBASE_ADDR, base, buf);
+ qla27xx_write_reg(reg, IOBASE_ADDR, addr, buf);
while (count--) {
- qla27xx_insert32(base, buf, len);
+ qla27xx_insert32(addr, buf, len);
readn(window, buf, len);
window += width;
- base += width;
+ addr++;
}
}
@@ -336,7 +335,8 @@
ql_dbg(ql_dbg_misc, vha, 0xd204,
"%s: rdpci [%lx]\n", __func__, *len);
- qla27xx_read_reg(reg, ent->t260.pci_addr, buf, len);
+ qla27xx_insert32(ent->t260.pci_offset, buf, len);
+ qla27xx_read_reg(reg, ent->t260.pci_offset, buf, len);
return false;
}
@@ -349,7 +349,7 @@
ql_dbg(ql_dbg_misc, vha, 0xd205,
"%s: wrpci [%lx]\n", __func__, *len);
- qla27xx_write_reg(reg, ent->t261.pci_addr, ent->t261.write_data, buf);
+ qla27xx_write_reg(reg, ent->t261.pci_offset, ent->t261.write_data, buf);
return false;
}
@@ -392,9 +392,9 @@
goto done;
}
- if (end < start) {
+ if (end < start || end == 0) {
ql_dbg(ql_dbg_misc, vha, 0xd023,
- "%s: bad range (start=%x end=%x)\n", __func__,
+ "%s: unusable range (start=%x end=%x)\n", __func__,
ent->t262.end_addr, ent->t262.start_addr);
qla27xx_skip_entry(ent, buf);
goto done;
@@ -452,17 +452,15 @@
ql_dbg(ql_dbg_misc, vha, 0xd025,
"%s: unsupported atio queue\n", __func__);
qla27xx_skip_entry(ent, buf);
- goto done;
} else {
ql_dbg(ql_dbg_misc, vha, 0xd026,
"%s: unknown queue %u\n", __func__, ent->t263.queue_type);
qla27xx_skip_entry(ent, buf);
- goto done;
}
if (buf)
ent->t263.num_queues = count;
-done:
+
return false;
}
@@ -503,7 +501,7 @@
ql_dbg(ql_dbg_misc, vha, 0xd209,
"%s: pause risc [%lx]\n", __func__, *len);
if (buf)
- qla24xx_pause_risc(reg);
+ qla24xx_pause_risc(reg, vha->hw);
return false;
}
@@ -590,7 +588,6 @@
struct qla27xx_fwdt_entry *ent, void *buf, ulong *len)
{
struct device_reg_24xx __iomem *reg = qla27xx_isp_reg(vha);
- void *window = (void *)reg + 0xc4;
ulong dwords = ent->t270.count;
ulong addr = ent->t270.addr;
@@ -599,10 +596,9 @@
qla27xx_write_reg(reg, IOBASE_ADDR, 0x40, buf);
while (dwords--) {
qla27xx_write_reg(reg, 0xc0, addr|0x80000000, buf);
- qla27xx_read_reg(reg, 0xc4, buf, len);
qla27xx_insert32(addr, buf, len);
- qla27xx_read32(window, buf, len);
- addr++;
+ qla27xx_read_reg(reg, 0xc4, buf, len);
+ addr += sizeof(uint32_t);
}
return false;
@@ -614,12 +610,12 @@
{
struct device_reg_24xx __iomem *reg = qla27xx_isp_reg(vha);
ulong addr = ent->t271.addr;
+ ulong data = ent->t271.data;
ql_dbg(ql_dbg_misc, vha, 0xd20f,
"%s: wrremreg [%lx]\n", __func__, *len);
qla27xx_write_reg(reg, IOBASE_ADDR, 0x40, buf);
- qla27xx_read_reg(reg, 0xc4, buf, len);
- qla27xx_insert32(addr, buf, len);
+ qla27xx_write_reg(reg, 0xc4, data, buf);
qla27xx_write_reg(reg, 0xc0, addr, buf);
return false;
@@ -662,13 +658,63 @@
"%s: failed pcicfg read at %lx\n", __func__, addr);
qla27xx_insert32(addr, buf, len);
qla27xx_insert32(value, buf, len);
- addr += 4;
+ addr += sizeof(uint32_t);
}
return false;
}
static int
+qla27xx_fwdt_entry_t274(struct scsi_qla_host *vha,
+ struct qla27xx_fwdt_entry *ent, void *buf, ulong *len)
+{
+ uint count = 0;
+ uint i;
+
+ ql_dbg(ql_dbg_misc, vha, 0xd212,
+ "%s: getqsh(%x) [%lx]\n", __func__, ent->t274.queue_type, *len);
+ if (ent->t274.queue_type == T274_QUEUE_TYPE_REQ_SHAD) {
+ for (i = 0; i < vha->hw->max_req_queues; i++) {
+ struct req_que *req = vha->hw->req_q_map[i];
+ if (req || !buf) {
+ qla27xx_insert16(i, buf, len);
+ qla27xx_insert16(1, buf, len);
+ qla27xx_insert32(req && req->out_ptr ?
+ *req->out_ptr : 0, buf, len);
+ count++;
+ }
+ }
+ } else if (ent->t274.queue_type == T274_QUEUE_TYPE_RSP_SHAD) {
+ for (i = 0; i < vha->hw->max_rsp_queues; i++) {
+ struct rsp_que *rsp = vha->hw->rsp_q_map[i];
+ if (rsp || !buf) {
+ qla27xx_insert16(i, buf, len);
+ qla27xx_insert16(1, buf, len);
+ qla27xx_insert32(rsp && rsp->in_ptr ?
+ *rsp->in_ptr : 0, buf, len);
+ count++;
+ }
+ }
+ } else if (ent->t274.queue_type == T274_QUEUE_TYPE_ATIO_SHAD) {
+ ql_dbg(ql_dbg_misc, vha, 0xd02e,
+ "%s: unsupported atio queue\n", __func__);
+ qla27xx_skip_entry(ent, buf);
+ } else {
+ ql_dbg(ql_dbg_misc, vha, 0xd02f,
+ "%s: unknown queue %u\n", __func__, ent->t274.queue_type);
+ qla27xx_skip_entry(ent, buf);
+ }
+
+ if (buf)
+ ent->t274.num_queues = count;
+
+ if (!count)
+ qla27xx_skip_entry(ent, buf);
+
+ return false;
+}
+
+static int
qla27xx_fwdt_entry_other(struct scsi_qla_host *vha,
struct qla27xx_fwdt_entry *ent, void *buf, ulong *len)
{
@@ -709,6 +755,7 @@
{ ENTRY_TYPE_WRREMREG , qla27xx_fwdt_entry_t271 } ,
{ ENTRY_TYPE_RDREMRAM , qla27xx_fwdt_entry_t272 } ,
{ ENTRY_TYPE_PCICFG , qla27xx_fwdt_entry_t273 } ,
+ { ENTRY_TYPE_GET_SHADOW , qla27xx_fwdt_entry_t274 } ,
{ -1 , qla27xx_fwdt_entry_other }
};
diff --git a/drivers/scsi/qla2xxx/qla_tmpl.h b/drivers/scsi/qla2xxx/qla_tmpl.h
index c9d2fff..1967424 100644
--- a/drivers/scsi/qla2xxx/qla_tmpl.h
+++ b/drivers/scsi/qla2xxx/qla_tmpl.h
@@ -1,6 +1,6 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
@@ -52,6 +52,7 @@
#define ENTRY_TYPE_WRREMREG 271
#define ENTRY_TYPE_RDREMRAM 272
#define ENTRY_TYPE_PCICFG 273
+#define ENTRY_TYPE_GET_SHADOW 274
#define CAPTURE_FLAG_PHYS_ONLY BIT_0
#define CAPTURE_FLAG_PHYS_VIRT BIT_1
@@ -109,12 +110,12 @@
} t259;
struct __packed {
- uint8_t pci_addr;
+ uint8_t pci_offset;
uint8_t reserved[3];
} t260;
struct __packed {
- uint8_t pci_addr;
+ uint8_t pci_offset;
uint8_t reserved[3];
uint32_t write_data;
} t261;
@@ -186,6 +187,12 @@
uint32_t addr;
uint32_t count;
} t273;
+
+ struct __packed {
+ uint32_t num_queues;
+ uint8_t queue_type;
+ uint8_t reserved[3];
+ } t274;
};
};
@@ -202,4 +209,8 @@
#define T268_BUF_TYPE_EXCH_BUFOFF 2
#define T268_BUF_TYPE_EXTD_LOGIN 3
+#define T274_QUEUE_TYPE_REQ_SHAD 1
+#define T274_QUEUE_TYPE_RSP_SHAD 2
+#define T274_QUEUE_TYPE_ATIO_SHAD 3
+
#endif
diff --git a/drivers/scsi/qla2xxx/qla_version.h b/drivers/scsi/qla2xxx/qla_version.h
index e36b947..4d2c98c 100644
--- a/drivers/scsi/qla2xxx/qla_version.h
+++ b/drivers/scsi/qla2xxx/qla_version.h
@@ -1,13 +1,13 @@
/*
* QLogic Fibre Channel HBA Driver
- * Copyright (c) 2003-2013 QLogic Corporation
+ * Copyright (c) 2003-2014 QLogic Corporation
*
* See LICENSE.qla2xxx for copyright and licensing details.
*/
/*
* Driver version
*/
-#define QLA2XXX_VERSION "8.07.00.02-k"
+#define QLA2XXX_VERSION "8.07.00.08-k"
#define QLA_DRIVER_MAJOR_VER 8
#define QLA_DRIVER_MINOR_VER 7
diff --git a/drivers/scsi/qla2xxx/tcm_qla2xxx.c b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
index 68fb66fd..896cb23 100644
--- a/drivers/scsi/qla2xxx/tcm_qla2xxx.c
+++ b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
@@ -472,6 +472,11 @@
cmd->sg_cnt = se_cmd->t_data_nents;
cmd->sg = se_cmd->t_data_sg;
+ cmd->prot_sg_cnt = se_cmd->t_prot_nents;
+ cmd->prot_sg = se_cmd->t_prot_sg;
+ cmd->blk_sz = se_cmd->se_dev->dev_attrib.block_size;
+ se_cmd->pi_err = 0;
+
/*
* qla_target.c:qlt_rdy_to_xfer() will call pci_map_sg() to setup
* the SGL mappings into PCIe memory for incoming FCP WRITE data.
@@ -567,8 +572,13 @@
return;
}
- transport_generic_request_failure(&cmd->se_cmd,
- TCM_CHECK_CONDITION_ABORT_CMD);
+ if (cmd->se_cmd.pi_err)
+ transport_generic_request_failure(&cmd->se_cmd,
+ cmd->se_cmd.pi_err);
+ else
+ transport_generic_request_failure(&cmd->se_cmd,
+ TCM_CHECK_CONDITION_ABORT_CMD);
+
return;
}
@@ -584,6 +594,27 @@
queue_work(tcm_qla2xxx_free_wq, &cmd->work);
}
+static void tcm_qla2xxx_handle_dif_work(struct work_struct *work)
+{
+ struct qla_tgt_cmd *cmd = container_of(work, struct qla_tgt_cmd, work);
+
+ /* take an extra kref to prevent cmd free too early.
+ * need to wait for SCSI status/check condition to
+ * finish responding generate by transport_generic_request_failure.
+ */
+ kref_get(&cmd->se_cmd.cmd_kref);
+ transport_generic_request_failure(&cmd->se_cmd, cmd->se_cmd.pi_err);
+}
+
+/*
+ * Called from qla_target.c:qlt_do_ctio_completion()
+ */
+static void tcm_qla2xxx_handle_dif_err(struct qla_tgt_cmd *cmd)
+{
+ INIT_WORK(&cmd->work, tcm_qla2xxx_handle_dif_work);
+ queue_work(tcm_qla2xxx_free_wq, &cmd->work);
+}
+
/*
* Called from qla_target.c:qlt_issue_task_mgmt()
*/
@@ -610,6 +641,11 @@
cmd->sg = se_cmd->t_data_sg;
cmd->offset = 0;
+ cmd->prot_sg_cnt = se_cmd->t_prot_nents;
+ cmd->prot_sg = se_cmd->t_prot_sg;
+ cmd->blk_sz = se_cmd->se_dev->dev_attrib.block_size;
+ se_cmd->pi_err = 0;
+
/*
* Now queue completed DATA_IN the qla2xxx LLD and response ring
*/
@@ -1600,6 +1636,7 @@
static struct qla_tgt_func_tmpl tcm_qla2xxx_template = {
.handle_cmd = tcm_qla2xxx_handle_cmd,
.handle_data = tcm_qla2xxx_handle_data,
+ .handle_dif_err = tcm_qla2xxx_handle_dif_err,
.handle_tmr = tcm_qla2xxx_handle_tmr,
.free_cmd = tcm_qla2xxx_free_cmd,
.free_mcmd = tcm_qla2xxx_free_mcmd,
diff --git a/drivers/scsi/qla4xxx/ql4_83xx.c b/drivers/scsi/qla4xxx/ql4_83xx.c
index 2eba353..556c152 100644
--- a/drivers/scsi/qla4xxx/ql4_83xx.c
+++ b/drivers/scsi/qla4xxx/ql4_83xx.c
@@ -249,110 +249,6 @@
qla4_83xx_flash_unlock(ha);
}
-/**
- * qla4_83xx_ms_mem_write_128b - Writes data to MS/off-chip memory
- * @ha: Pointer to adapter structure
- * @addr: Flash address to write to
- * @data: Data to be written
- * @count: word_count to be written
- *
- * Return: On success return QLA_SUCCESS
- * On error return QLA_ERROR
- **/
-int qla4_83xx_ms_mem_write_128b(struct scsi_qla_host *ha, uint64_t addr,
- uint32_t *data, uint32_t count)
-{
- int i, j;
- uint32_t agt_ctrl;
- unsigned long flags;
- int ret_val = QLA_SUCCESS;
-
- /* Only 128-bit aligned access */
- if (addr & 0xF) {
- ret_val = QLA_ERROR;
- goto exit_ms_mem_write;
- }
-
- write_lock_irqsave(&ha->hw_lock, flags);
-
- /* Write address */
- ret_val = qla4_83xx_wr_reg_indirect(ha, MD_MIU_TEST_AGT_ADDR_HI, 0);
- if (ret_val == QLA_ERROR) {
- ql4_printk(KERN_ERR, ha, "%s: write to AGT_ADDR_HI failed\n",
- __func__);
- goto exit_ms_mem_write_unlock;
- }
-
- for (i = 0; i < count; i++, addr += 16) {
- if (!((QLA8XXX_ADDR_IN_RANGE(addr, QLA8XXX_ADDR_QDR_NET,
- QLA8XXX_ADDR_QDR_NET_MAX)) ||
- (QLA8XXX_ADDR_IN_RANGE(addr, QLA8XXX_ADDR_DDR_NET,
- QLA8XXX_ADDR_DDR_NET_MAX)))) {
- ret_val = QLA_ERROR;
- goto exit_ms_mem_write_unlock;
- }
-
- ret_val = qla4_83xx_wr_reg_indirect(ha, MD_MIU_TEST_AGT_ADDR_LO,
- addr);
- /* Write data */
- ret_val |= qla4_83xx_wr_reg_indirect(ha,
- MD_MIU_TEST_AGT_WRDATA_LO,
- *data++);
- ret_val |= qla4_83xx_wr_reg_indirect(ha,
- MD_MIU_TEST_AGT_WRDATA_HI,
- *data++);
- ret_val |= qla4_83xx_wr_reg_indirect(ha,
- MD_MIU_TEST_AGT_WRDATA_ULO,
- *data++);
- ret_val |= qla4_83xx_wr_reg_indirect(ha,
- MD_MIU_TEST_AGT_WRDATA_UHI,
- *data++);
- if (ret_val == QLA_ERROR) {
- ql4_printk(KERN_ERR, ha, "%s: write to AGT_WRDATA failed\n",
- __func__);
- goto exit_ms_mem_write_unlock;
- }
-
- /* Check write status */
- ret_val = qla4_83xx_wr_reg_indirect(ha, MD_MIU_TEST_AGT_CTRL,
- MIU_TA_CTL_WRITE_ENABLE);
- ret_val |= qla4_83xx_wr_reg_indirect(ha, MD_MIU_TEST_AGT_CTRL,
- MIU_TA_CTL_WRITE_START);
- if (ret_val == QLA_ERROR) {
- ql4_printk(KERN_ERR, ha, "%s: write to AGT_CTRL failed\n",
- __func__);
- goto exit_ms_mem_write_unlock;
- }
-
- for (j = 0; j < MAX_CTL_CHECK; j++) {
- ret_val = qla4_83xx_rd_reg_indirect(ha,
- MD_MIU_TEST_AGT_CTRL,
- &agt_ctrl);
- if (ret_val == QLA_ERROR) {
- ql4_printk(KERN_ERR, ha, "%s: failed to read MD_MIU_TEST_AGT_CTRL\n",
- __func__);
- goto exit_ms_mem_write_unlock;
- }
- if ((agt_ctrl & MIU_TA_CTL_BUSY) == 0)
- break;
- }
-
- /* Status check failed */
- if (j >= MAX_CTL_CHECK) {
- printk_ratelimited(KERN_ERR "%s: MS memory write failed!\n",
- __func__);
- ret_val = QLA_ERROR;
- goto exit_ms_mem_write_unlock;
- }
- }
-
-exit_ms_mem_write_unlock:
- write_unlock_irqrestore(&ha->hw_lock, flags);
-
-exit_ms_mem_write:
- return ret_val;
-}
-
#define INTENT_TO_RECOVER 0x01
#define PROCEED_TO_RECOVER 0x02
@@ -760,7 +656,7 @@
__func__));
/* 128 bit/16 byte write to MS memory */
- ret_val = qla4_83xx_ms_mem_write_128b(ha, dest, (uint32_t *)p_cache,
+ ret_val = qla4_8xxx_ms_mem_write_128b(ha, dest, (uint32_t *)p_cache,
count);
if (ret_val == QLA_ERROR) {
ql4_printk(KERN_ERR, ha, "%s: Error writing firmware to MS\n",
diff --git a/drivers/scsi/qla4xxx/ql4_83xx.h b/drivers/scsi/qla4xxx/ql4_83xx.h
index a0de6e2..775fdf9 100644
--- a/drivers/scsi/qla4xxx/ql4_83xx.h
+++ b/drivers/scsi/qla4xxx/ql4_83xx.h
@@ -254,6 +254,50 @@
uint32_t rsvd_1;
};
+struct qla8044_minidump_entry_rddfe {
+ struct qla8xxx_minidump_entry_hdr h;
+ uint32_t addr_1;
+ uint32_t value;
+ uint8_t stride;
+ uint8_t stride2;
+ uint16_t count;
+ uint32_t poll;
+ uint32_t mask;
+ uint32_t modify_mask;
+ uint32_t data_size;
+ uint32_t rsvd;
+
+} __packed;
+
+struct qla8044_minidump_entry_rdmdio {
+ struct qla8xxx_minidump_entry_hdr h;
+
+ uint32_t addr_1;
+ uint32_t addr_2;
+ uint32_t value_1;
+ uint8_t stride_1;
+ uint8_t stride_2;
+ uint16_t count;
+ uint32_t poll;
+ uint32_t mask;
+ uint32_t value_2;
+ uint32_t data_size;
+
+} __packed;
+
+struct qla8044_minidump_entry_pollwr {
+ struct qla8xxx_minidump_entry_hdr h;
+ uint32_t addr_1;
+ uint32_t addr_2;
+ uint32_t value_1;
+ uint32_t value_2;
+ uint32_t poll;
+ uint32_t mask;
+ uint32_t data_size;
+ uint32_t rsvd;
+
+} __packed;
+
/* RDMUX2 Entry */
struct qla83xx_minidump_entry_rdmux2 {
struct qla8xxx_minidump_entry_hdr h;
diff --git a/drivers/scsi/qla4xxx/ql4_def.h b/drivers/scsi/qla4xxx/ql4_def.h
index 73a5022..8f6d0fb 100644
--- a/drivers/scsi/qla4xxx/ql4_def.h
+++ b/drivers/scsi/qla4xxx/ql4_def.h
@@ -601,6 +601,7 @@
#define DPC_HA_NEED_QUIESCENT 22 /* 0x00400000 ISP-82xx only*/
#define DPC_POST_IDC_ACK 23 /* 0x00800000 */
#define DPC_RESTORE_ACB 24 /* 0x01000000 */
+#define DPC_SYSFS_DDB_EXPORT 25 /* 0x02000000 */
struct Scsi_Host *host; /* pointer to host data */
uint32_t tot_ddbs;
diff --git a/drivers/scsi/qla4xxx/ql4_fw.h b/drivers/scsi/qla4xxx/ql4_fw.h
index 209853c..699575e 100644
--- a/drivers/scsi/qla4xxx/ql4_fw.h
+++ b/drivers/scsi/qla4xxx/ql4_fw.h
@@ -1415,6 +1415,9 @@
#define QLA83XX_DBG_OCM_WNDREG_ARRAY_LEN 16
#define QLA83XX_SS_OCM_WNDREG_INDEX 3
#define QLA83XX_SS_PCI_INDEX 0
+#define QLA8022_TEMPLATE_CAP_OFFSET 172
+#define QLA83XX_TEMPLATE_CAP_OFFSET 268
+#define QLA80XX_TEMPLATE_RESERVED_BITS 16
struct qla4_8xxx_minidump_template_hdr {
uint32_t entry_type;
@@ -1434,6 +1437,7 @@
uint32_t saved_state_array[QLA8XXX_DBG_STATE_ARRAY_LEN];
uint32_t capture_size_array[QLA8XXX_DBG_CAP_SIZE_ARRAY_LEN];
uint32_t ocm_window_reg[QLA83XX_DBG_OCM_WNDREG_ARRAY_LEN];
+ uint32_t capabilities[QLA80XX_TEMPLATE_RESERVED_BITS];
};
#endif /* _QLA4X_FW_H */
diff --git a/drivers/scsi/qla4xxx/ql4_glbl.h b/drivers/scsi/qla4xxx/ql4_glbl.h
index b1a19cd..5f58b45 100644
--- a/drivers/scsi/qla4xxx/ql4_glbl.h
+++ b/drivers/scsi/qla4xxx/ql4_glbl.h
@@ -274,13 +274,14 @@
int qla4xxx_get_acb(struct scsi_qla_host *ha, dma_addr_t acb_dma,
uint32_t acb_type, uint32_t len);
int qla4_84xx_config_acb(struct scsi_qla_host *ha, int acb_config);
-int qla4_83xx_ms_mem_write_128b(struct scsi_qla_host *ha,
+int qla4_8xxx_ms_mem_write_128b(struct scsi_qla_host *ha,
uint64_t addr, uint32_t *data, uint32_t count);
uint8_t qla4xxx_set_ipaddr_state(uint8_t fw_ipaddr_state);
int qla4_83xx_get_port_config(struct scsi_qla_host *ha, uint32_t *config);
int qla4_83xx_set_port_config(struct scsi_qla_host *ha, uint32_t *config);
int qla4_8xxx_check_init_adapter_retry(struct scsi_qla_host *ha);
int qla4_83xx_is_detached(struct scsi_qla_host *ha);
+int qla4xxx_sysfs_ddb_export(struct scsi_qla_host *ha);
extern int ql4xextended_error_logging;
extern int ql4xdontresethba;
diff --git a/drivers/scsi/qla4xxx/ql4_init.c b/drivers/scsi/qla4xxx/ql4_init.c
index 28fbece..6f12f85 100644
--- a/drivers/scsi/qla4xxx/ql4_init.c
+++ b/drivers/scsi/qla4xxx/ql4_init.c
@@ -282,6 +282,25 @@
return ipv4_wait|ipv6_wait;
}
+static int qla4_80xx_is_minidump_dma_capable(struct scsi_qla_host *ha,
+ struct qla4_8xxx_minidump_template_hdr *md_hdr)
+{
+ int offset = (is_qla8022(ha)) ? QLA8022_TEMPLATE_CAP_OFFSET :
+ QLA83XX_TEMPLATE_CAP_OFFSET;
+ int rval = 1;
+ uint32_t *cap_offset;
+
+ cap_offset = (uint32_t *)((char *)md_hdr + offset);
+
+ if (!(le32_to_cpu(*cap_offset) & BIT_0)) {
+ ql4_printk(KERN_INFO, ha, "PEX DMA Not supported %d\n",
+ *cap_offset);
+ rval = 0;
+ }
+
+ return rval;
+}
+
/**
* qla4xxx_alloc_fw_dump - Allocate memory for minidump data.
* @ha: pointer to host adapter structure.
@@ -294,6 +313,7 @@
void *md_tmp;
dma_addr_t md_tmp_dma;
struct qla4_8xxx_minidump_template_hdr *md_hdr;
+ int dma_capable;
if (ha->fw_dump) {
ql4_printk(KERN_WARNING, ha,
@@ -326,13 +346,19 @@
md_hdr = (struct qla4_8xxx_minidump_template_hdr *)md_tmp;
+ dma_capable = qla4_80xx_is_minidump_dma_capable(ha, md_hdr);
+
capture_debug_level = md_hdr->capture_debug_level;
/* Get capture mask based on module loadtime setting. */
- if (ql4xmdcapmask >= 0x3 && ql4xmdcapmask <= 0x7F)
+ if ((ql4xmdcapmask >= 0x3 && ql4xmdcapmask <= 0x7F) ||
+ (ql4xmdcapmask == 0xFF && dma_capable)) {
ha->fw_dump_capture_mask = ql4xmdcapmask;
- else
+ } else {
+ if (ql4xmdcapmask == 0xFF)
+ ql4_printk(KERN_INFO, ha, "Falling back to default capture mask, as PEX DMA is not supported\n");
ha->fw_dump_capture_mask = capture_debug_level;
+ }
md_hdr->driver_capture_mask = ha->fw_dump_capture_mask;
@@ -864,6 +890,8 @@
if (status == QLA_SUCCESS) {
if (test_and_clear_bit(AF_GET_CRASH_RECORD, &ha->flags))
qla4xxx_get_crash_record(ha);
+
+ qla4xxx_init_rings(ha);
} else {
DEBUG(printk("scsi%ld: %s: Firmware has NOT started\n",
ha->host_no, __func__));
diff --git a/drivers/scsi/qla4xxx/ql4_isr.c b/drivers/scsi/qla4xxx/ql4_isr.c
index b1925d1..081b6b7 100644
--- a/drivers/scsi/qla4xxx/ql4_isr.c
+++ b/drivers/scsi/qla4xxx/ql4_isr.c
@@ -1526,7 +1526,7 @@
int qla4xxx_request_irqs(struct scsi_qla_host *ha)
{
- int ret;
+ int ret = 0;
int rval = QLA_ERROR;
if (is_qla40XX(ha))
@@ -1580,15 +1580,13 @@
}
}
- /*
- * Prevent interrupts from falling back to INTx mode in cases where
- * interrupts cannot get acquired through MSI-X or MSI mode.
- */
+try_intx:
if (is_qla8022(ha)) {
- ql4_printk(KERN_WARNING, ha, "IRQ not attached -- %d.\n", ret);
+ ql4_printk(KERN_WARNING, ha, "%s: ISP82xx Legacy interrupt not supported\n",
+ __func__);
goto irq_not_attached;
}
-try_intx:
+
/* Trying INTx */
ret = request_irq(ha->pdev->irq, ha->isp_ops->intr_handler,
IRQF_SHARED, DRIVER_NAME, ha);
diff --git a/drivers/scsi/qla4xxx/ql4_mbx.c b/drivers/scsi/qla4xxx/ql4_mbx.c
index 0a6b782..0a3312c 100644
--- a/drivers/scsi/qla4xxx/ql4_mbx.c
+++ b/drivers/scsi/qla4xxx/ql4_mbx.c
@@ -2381,7 +2381,7 @@
ql4_printk(KERN_ERR, ha, "%s: Unable to alloc acb\n",
__func__);
rval = QLA_ERROR;
- goto exit_config_acb;
+ goto exit_free_acb;
}
memcpy(ha->saved_acb, acb, acb_len);
break;
@@ -2395,8 +2395,6 @@
}
memcpy(acb, ha->saved_acb, acb_len);
- kfree(ha->saved_acb);
- ha->saved_acb = NULL;
rval = qla4xxx_set_acb(ha, &mbox_cmd[0], &mbox_sts[0], acb_dma);
if (rval != QLA_SUCCESS)
@@ -2412,6 +2410,10 @@
dma_free_coherent(&ha->pdev->dev, sizeof(struct addr_ctrl_blk), acb,
acb_dma);
exit_config_acb:
+ if ((acb_config == ACB_CONFIG_SET) && ha->saved_acb) {
+ kfree(ha->saved_acb);
+ ha->saved_acb = NULL;
+ }
DEBUG2(ql4_printk(KERN_INFO, ha,
"%s %s\n", __func__,
rval == QLA_SUCCESS ? "SUCCEEDED" : "FAILED"));
diff --git a/drivers/scsi/qla4xxx/ql4_nx.c b/drivers/scsi/qla4xxx/ql4_nx.c
index 63328c8..9dbdb4b 100644
--- a/drivers/scsi/qla4xxx/ql4_nx.c
+++ b/drivers/scsi/qla4xxx/ql4_nx.c
@@ -14,6 +14,7 @@
#include <asm-generic/io-64-nonatomic-lo-hi.h>
+#define TIMEOUT_100_MS 100
#define MASK(n) DMA_BIT_MASK(n)
#define MN_WIN(addr) (((addr & 0x1fc0000) >> 1) | ((addr >> 25) & 0x3ff))
#define OCM_WIN(addr) (((addr & 0x1ff0000) >> 1) | ((addr >> 25) & 0x3ff))
@@ -1176,6 +1177,112 @@
return 0;
}
+/**
+ * qla4_8xxx_ms_mem_write_128b - Writes data to MS/off-chip memory
+ * @ha: Pointer to adapter structure
+ * @addr: Flash address to write to
+ * @data: Data to be written
+ * @count: word_count to be written
+ *
+ * Return: On success return QLA_SUCCESS
+ * On error return QLA_ERROR
+ **/
+int qla4_8xxx_ms_mem_write_128b(struct scsi_qla_host *ha, uint64_t addr,
+ uint32_t *data, uint32_t count)
+{
+ int i, j;
+ uint32_t agt_ctrl;
+ unsigned long flags;
+ int ret_val = QLA_SUCCESS;
+
+ /* Only 128-bit aligned access */
+ if (addr & 0xF) {
+ ret_val = QLA_ERROR;
+ goto exit_ms_mem_write;
+ }
+
+ write_lock_irqsave(&ha->hw_lock, flags);
+
+ /* Write address */
+ ret_val = ha->isp_ops->wr_reg_indirect(ha, MD_MIU_TEST_AGT_ADDR_HI, 0);
+ if (ret_val == QLA_ERROR) {
+ ql4_printk(KERN_ERR, ha, "%s: write to AGT_ADDR_HI failed\n",
+ __func__);
+ goto exit_ms_mem_write_unlock;
+ }
+
+ for (i = 0; i < count; i++, addr += 16) {
+ if (!((QLA8XXX_ADDR_IN_RANGE(addr, QLA8XXX_ADDR_QDR_NET,
+ QLA8XXX_ADDR_QDR_NET_MAX)) ||
+ (QLA8XXX_ADDR_IN_RANGE(addr, QLA8XXX_ADDR_DDR_NET,
+ QLA8XXX_ADDR_DDR_NET_MAX)))) {
+ ret_val = QLA_ERROR;
+ goto exit_ms_mem_write_unlock;
+ }
+
+ ret_val = ha->isp_ops->wr_reg_indirect(ha,
+ MD_MIU_TEST_AGT_ADDR_LO,
+ addr);
+ /* Write data */
+ ret_val |= ha->isp_ops->wr_reg_indirect(ha,
+ MD_MIU_TEST_AGT_WRDATA_LO,
+ *data++);
+ ret_val |= ha->isp_ops->wr_reg_indirect(ha,
+ MD_MIU_TEST_AGT_WRDATA_HI,
+ *data++);
+ ret_val |= ha->isp_ops->wr_reg_indirect(ha,
+ MD_MIU_TEST_AGT_WRDATA_ULO,
+ *data++);
+ ret_val |= ha->isp_ops->wr_reg_indirect(ha,
+ MD_MIU_TEST_AGT_WRDATA_UHI,
+ *data++);
+ if (ret_val == QLA_ERROR) {
+ ql4_printk(KERN_ERR, ha, "%s: write to AGT_WRDATA failed\n",
+ __func__);
+ goto exit_ms_mem_write_unlock;
+ }
+
+ /* Check write status */
+ ret_val = ha->isp_ops->wr_reg_indirect(ha, MD_MIU_TEST_AGT_CTRL,
+ MIU_TA_CTL_WRITE_ENABLE);
+ ret_val |= ha->isp_ops->wr_reg_indirect(ha,
+ MD_MIU_TEST_AGT_CTRL,
+ MIU_TA_CTL_WRITE_START);
+ if (ret_val == QLA_ERROR) {
+ ql4_printk(KERN_ERR, ha, "%s: write to AGT_CTRL failed\n",
+ __func__);
+ goto exit_ms_mem_write_unlock;
+ }
+
+ for (j = 0; j < MAX_CTL_CHECK; j++) {
+ ret_val = ha->isp_ops->rd_reg_indirect(ha,
+ MD_MIU_TEST_AGT_CTRL,
+ &agt_ctrl);
+ if (ret_val == QLA_ERROR) {
+ ql4_printk(KERN_ERR, ha, "%s: failed to read MD_MIU_TEST_AGT_CTRL\n",
+ __func__);
+ goto exit_ms_mem_write_unlock;
+ }
+ if ((agt_ctrl & MIU_TA_CTL_BUSY) == 0)
+ break;
+ }
+
+ /* Status check failed */
+ if (j >= MAX_CTL_CHECK) {
+ printk_ratelimited(KERN_ERR "%s: MS memory write failed!\n",
+ __func__);
+ ret_val = QLA_ERROR;
+ goto exit_ms_mem_write_unlock;
+ }
+ }
+
+exit_ms_mem_write_unlock:
+ write_unlock_irqrestore(&ha->hw_lock, flags);
+
+exit_ms_mem_write:
+ return ret_val;
+}
+
static int
qla4_82xx_load_from_flash(struct scsi_qla_host *ha, uint32_t image_start)
{
@@ -1714,6 +1821,101 @@
qla4_82xx_rom_unlock(ha);
}
+static uint32_t ql4_84xx_poll_wait_for_ready(struct scsi_qla_host *ha,
+ uint32_t addr1, uint32_t mask)
+{
+ unsigned long timeout;
+ uint32_t rval = QLA_SUCCESS;
+ uint32_t temp;
+
+ timeout = jiffies + msecs_to_jiffies(TIMEOUT_100_MS);
+ do {
+ ha->isp_ops->rd_reg_indirect(ha, addr1, &temp);
+ if ((temp & mask) != 0)
+ break;
+
+ if (time_after_eq(jiffies, timeout)) {
+ ql4_printk(KERN_INFO, ha, "Error in processing rdmdio entry\n");
+ return QLA_ERROR;
+ }
+ } while (1);
+
+ return rval;
+}
+
+uint32_t ql4_84xx_ipmdio_rd_reg(struct scsi_qla_host *ha, uint32_t addr1,
+ uint32_t addr3, uint32_t mask, uint32_t addr,
+ uint32_t *data_ptr)
+{
+ int rval = QLA_SUCCESS;
+ uint32_t temp;
+ uint32_t data;
+
+ rval = ql4_84xx_poll_wait_for_ready(ha, addr1, mask);
+ if (rval)
+ goto exit_ipmdio_rd_reg;
+
+ temp = (0x40000000 | addr);
+ ha->isp_ops->wr_reg_indirect(ha, addr1, temp);
+
+ rval = ql4_84xx_poll_wait_for_ready(ha, addr1, mask);
+ if (rval)
+ goto exit_ipmdio_rd_reg;
+
+ ha->isp_ops->rd_reg_indirect(ha, addr3, &data);
+ *data_ptr = data;
+
+exit_ipmdio_rd_reg:
+ return rval;
+}
+
+
+static uint32_t ql4_84xx_poll_wait_ipmdio_bus_idle(struct scsi_qla_host *ha,
+ uint32_t addr1,
+ uint32_t addr2,
+ uint32_t addr3,
+ uint32_t mask)
+{
+ unsigned long timeout;
+ uint32_t temp;
+ uint32_t rval = QLA_SUCCESS;
+
+ timeout = jiffies + msecs_to_jiffies(TIMEOUT_100_MS);
+ do {
+ ql4_84xx_ipmdio_rd_reg(ha, addr1, addr3, mask, addr2, &temp);
+ if ((temp & 0x1) != 1)
+ break;
+ if (time_after_eq(jiffies, timeout)) {
+ ql4_printk(KERN_INFO, ha, "Error in processing mdiobus idle\n");
+ return QLA_ERROR;
+ }
+ } while (1);
+
+ return rval;
+}
+
+static int ql4_84xx_ipmdio_wr_reg(struct scsi_qla_host *ha,
+ uint32_t addr1, uint32_t addr3,
+ uint32_t mask, uint32_t addr,
+ uint32_t value)
+{
+ int rval = QLA_SUCCESS;
+
+ rval = ql4_84xx_poll_wait_for_ready(ha, addr1, mask);
+ if (rval)
+ goto exit_ipmdio_wr_reg;
+
+ ha->isp_ops->wr_reg_indirect(ha, addr3, value);
+ ha->isp_ops->wr_reg_indirect(ha, addr1, addr);
+
+ rval = ql4_84xx_poll_wait_for_ready(ha, addr1, mask);
+ if (rval)
+ goto exit_ipmdio_wr_reg;
+
+exit_ipmdio_wr_reg:
+ return rval;
+}
+
static void qla4_8xxx_minidump_process_rdcrb(struct scsi_qla_host *ha,
struct qla8xxx_minidump_entry_hdr *entry_hdr,
uint32_t **d_ptr)
@@ -1822,7 +2024,7 @@
return rval;
}
-static int qla4_83xx_minidump_pex_dma_read(struct scsi_qla_host *ha,
+static int qla4_8xxx_minidump_pex_dma_read(struct scsi_qla_host *ha,
struct qla8xxx_minidump_entry_hdr *entry_hdr,
uint32_t **d_ptr)
{
@@ -1899,11 +2101,11 @@
dma_desc.cmd.read_data_size = size;
/* Prepare: Write pex-dma descriptor to MS memory. */
- rval = qla4_83xx_ms_mem_write_128b(ha,
+ rval = qla4_8xxx_ms_mem_write_128b(ha,
(uint64_t)m_hdr->desc_card_addr,
(uint32_t *)&dma_desc,
(sizeof(struct qla4_83xx_pex_dma_descriptor)/16));
- if (rval == -1) {
+ if (rval != QLA_SUCCESS) {
ql4_printk(KERN_INFO, ha,
"%s: Error writing rdmem-dma-init to MS !!!\n",
__func__);
@@ -2359,17 +2561,10 @@
uint32_t *data_ptr = *d_ptr;
int rval = QLA_SUCCESS;
- if (is_qla8032(ha) || is_qla8042(ha)) {
- rval = qla4_83xx_minidump_pex_dma_read(ha, entry_hdr,
- &data_ptr);
- if (rval != QLA_SUCCESS) {
- rval = __qla4_8xxx_minidump_process_rdmem(ha, entry_hdr,
- &data_ptr);
- }
- } else {
+ rval = qla4_8xxx_minidump_pex_dma_read(ha, entry_hdr, &data_ptr);
+ if (rval != QLA_SUCCESS)
rval = __qla4_8xxx_minidump_process_rdmem(ha, entry_hdr,
&data_ptr);
- }
*d_ptr = data_ptr;
return rval;
}
@@ -2440,6 +2635,227 @@
return rval;
}
+static uint32_t qla4_84xx_minidump_process_rddfe(struct scsi_qla_host *ha,
+ struct qla8xxx_minidump_entry_hdr *entry_hdr,
+ uint32_t **d_ptr)
+{
+ int loop_cnt;
+ uint32_t addr1, addr2, value, data, temp, wrval;
+ uint8_t stride, stride2;
+ uint16_t count;
+ uint32_t poll, mask, data_size, modify_mask;
+ uint32_t wait_count = 0;
+ uint32_t *data_ptr = *d_ptr;
+ struct qla8044_minidump_entry_rddfe *rddfe;
+ uint32_t rval = QLA_SUCCESS;
+
+ rddfe = (struct qla8044_minidump_entry_rddfe *)entry_hdr;
+ addr1 = le32_to_cpu(rddfe->addr_1);
+ value = le32_to_cpu(rddfe->value);
+ stride = le32_to_cpu(rddfe->stride);
+ stride2 = le32_to_cpu(rddfe->stride2);
+ count = le32_to_cpu(rddfe->count);
+
+ poll = le32_to_cpu(rddfe->poll);
+ mask = le32_to_cpu(rddfe->mask);
+ modify_mask = le32_to_cpu(rddfe->modify_mask);
+ data_size = le32_to_cpu(rddfe->data_size);
+
+ addr2 = addr1 + stride;
+
+ for (loop_cnt = 0x0; loop_cnt < count; loop_cnt++) {
+ ha->isp_ops->wr_reg_indirect(ha, addr1, (0x40000000 | value));
+
+ wait_count = 0;
+ while (wait_count < poll) {
+ ha->isp_ops->rd_reg_indirect(ha, addr1, &temp);
+ if ((temp & mask) != 0)
+ break;
+ wait_count++;
+ }
+
+ if (wait_count == poll) {
+ ql4_printk(KERN_ERR, ha, "%s: TIMEOUT\n", __func__);
+ rval = QLA_ERROR;
+ goto exit_process_rddfe;
+ } else {
+ ha->isp_ops->rd_reg_indirect(ha, addr2, &temp);
+ temp = temp & modify_mask;
+ temp = (temp | ((loop_cnt << 16) | loop_cnt));
+ wrval = ((temp << 16) | temp);
+
+ ha->isp_ops->wr_reg_indirect(ha, addr2, wrval);
+ ha->isp_ops->wr_reg_indirect(ha, addr1, value);
+
+ wait_count = 0;
+ while (wait_count < poll) {
+ ha->isp_ops->rd_reg_indirect(ha, addr1, &temp);
+ if ((temp & mask) != 0)
+ break;
+ wait_count++;
+ }
+ if (wait_count == poll) {
+ ql4_printk(KERN_ERR, ha, "%s: TIMEOUT\n",
+ __func__);
+ rval = QLA_ERROR;
+ goto exit_process_rddfe;
+ }
+
+ ha->isp_ops->wr_reg_indirect(ha, addr1,
+ ((0x40000000 | value) +
+ stride2));
+ wait_count = 0;
+ while (wait_count < poll) {
+ ha->isp_ops->rd_reg_indirect(ha, addr1, &temp);
+ if ((temp & mask) != 0)
+ break;
+ wait_count++;
+ }
+
+ if (wait_count == poll) {
+ ql4_printk(KERN_ERR, ha, "%s: TIMEOUT\n",
+ __func__);
+ rval = QLA_ERROR;
+ goto exit_process_rddfe;
+ }
+
+ ha->isp_ops->rd_reg_indirect(ha, addr2, &data);
+
+ *data_ptr++ = cpu_to_le32(wrval);
+ *data_ptr++ = cpu_to_le32(data);
+ }
+ }
+
+ *d_ptr = data_ptr;
+exit_process_rddfe:
+ return rval;
+}
+
+static uint32_t qla4_84xx_minidump_process_rdmdio(struct scsi_qla_host *ha,
+ struct qla8xxx_minidump_entry_hdr *entry_hdr,
+ uint32_t **d_ptr)
+{
+ int rval = QLA_SUCCESS;
+ uint32_t addr1, addr2, value1, value2, data, selval;
+ uint8_t stride1, stride2;
+ uint32_t addr3, addr4, addr5, addr6, addr7;
+ uint16_t count, loop_cnt;
+ uint32_t poll, mask;
+ uint32_t *data_ptr = *d_ptr;
+ struct qla8044_minidump_entry_rdmdio *rdmdio;
+
+ rdmdio = (struct qla8044_minidump_entry_rdmdio *)entry_hdr;
+ addr1 = le32_to_cpu(rdmdio->addr_1);
+ addr2 = le32_to_cpu(rdmdio->addr_2);
+ value1 = le32_to_cpu(rdmdio->value_1);
+ stride1 = le32_to_cpu(rdmdio->stride_1);
+ stride2 = le32_to_cpu(rdmdio->stride_2);
+ count = le32_to_cpu(rdmdio->count);
+
+ poll = le32_to_cpu(rdmdio->poll);
+ mask = le32_to_cpu(rdmdio->mask);
+ value2 = le32_to_cpu(rdmdio->value_2);
+
+ addr3 = addr1 + stride1;
+
+ for (loop_cnt = 0; loop_cnt < count; loop_cnt++) {
+ rval = ql4_84xx_poll_wait_ipmdio_bus_idle(ha, addr1, addr2,
+ addr3, mask);
+ if (rval)
+ goto exit_process_rdmdio;
+
+ addr4 = addr2 - stride1;
+ rval = ql4_84xx_ipmdio_wr_reg(ha, addr1, addr3, mask, addr4,
+ value2);
+ if (rval)
+ goto exit_process_rdmdio;
+
+ addr5 = addr2 - (2 * stride1);
+ rval = ql4_84xx_ipmdio_wr_reg(ha, addr1, addr3, mask, addr5,
+ value1);
+ if (rval)
+ goto exit_process_rdmdio;
+
+ addr6 = addr2 - (3 * stride1);
+ rval = ql4_84xx_ipmdio_wr_reg(ha, addr1, addr3, mask,
+ addr6, 0x2);
+ if (rval)
+ goto exit_process_rdmdio;
+
+ rval = ql4_84xx_poll_wait_ipmdio_bus_idle(ha, addr1, addr2,
+ addr3, mask);
+ if (rval)
+ goto exit_process_rdmdio;
+
+ addr7 = addr2 - (4 * stride1);
+ rval = ql4_84xx_ipmdio_rd_reg(ha, addr1, addr3,
+ mask, addr7, &data);
+ if (rval)
+ goto exit_process_rdmdio;
+
+ selval = (value2 << 18) | (value1 << 2) | 2;
+
+ stride2 = le32_to_cpu(rdmdio->stride_2);
+ *data_ptr++ = cpu_to_le32(selval);
+ *data_ptr++ = cpu_to_le32(data);
+
+ value1 = value1 + stride2;
+ *d_ptr = data_ptr;
+ }
+
+exit_process_rdmdio:
+ return rval;
+}
+
+static uint32_t qla4_84xx_minidump_process_pollwr(struct scsi_qla_host *ha,
+ struct qla8xxx_minidump_entry_hdr *entry_hdr,
+ uint32_t **d_ptr)
+{
+ uint32_t addr1, addr2, value1, value2, poll, mask, r_value;
+ struct qla8044_minidump_entry_pollwr *pollwr_hdr;
+ uint32_t wait_count = 0;
+ uint32_t rval = QLA_SUCCESS;
+
+ pollwr_hdr = (struct qla8044_minidump_entry_pollwr *)entry_hdr;
+ addr1 = le32_to_cpu(pollwr_hdr->addr_1);
+ addr2 = le32_to_cpu(pollwr_hdr->addr_2);
+ value1 = le32_to_cpu(pollwr_hdr->value_1);
+ value2 = le32_to_cpu(pollwr_hdr->value_2);
+
+ poll = le32_to_cpu(pollwr_hdr->poll);
+ mask = le32_to_cpu(pollwr_hdr->mask);
+
+ while (wait_count < poll) {
+ ha->isp_ops->rd_reg_indirect(ha, addr1, &r_value);
+
+ if ((r_value & poll) != 0)
+ break;
+
+ wait_count++;
+ }
+
+ if (wait_count == poll) {
+ ql4_printk(KERN_ERR, ha, "%s: TIMEOUT\n", __func__);
+ rval = QLA_ERROR;
+ goto exit_process_pollwr;
+ }
+
+ ha->isp_ops->wr_reg_indirect(ha, addr2, value2);
+ ha->isp_ops->wr_reg_indirect(ha, addr1, value1);
+
+ wait_count = 0;
+ while (wait_count < poll) {
+ ha->isp_ops->rd_reg_indirect(ha, addr1, &r_value);
+
+ if ((r_value & poll) != 0)
+ break;
+ wait_count++;
+ }
+
+exit_process_pollwr:
+ return rval;
+}
+
static void qla83xx_minidump_process_rdmux2(struct scsi_qla_host *ha,
struct qla8xxx_minidump_entry_hdr *entry_hdr,
uint32_t **d_ptr)
@@ -2753,6 +3169,24 @@
if (rval != QLA_SUCCESS)
qla4_8xxx_mark_entry_skipped(ha, entry_hdr, i);
break;
+ case QLA8044_RDDFE:
+ rval = qla4_84xx_minidump_process_rddfe(ha, entry_hdr,
+ &data_ptr);
+ if (rval != QLA_SUCCESS)
+ qla4_8xxx_mark_entry_skipped(ha, entry_hdr, i);
+ break;
+ case QLA8044_RDMDIO:
+ rval = qla4_84xx_minidump_process_rdmdio(ha, entry_hdr,
+ &data_ptr);
+ if (rval != QLA_SUCCESS)
+ qla4_8xxx_mark_entry_skipped(ha, entry_hdr, i);
+ break;
+ case QLA8044_POLLWR:
+ rval = qla4_84xx_minidump_process_pollwr(ha, entry_hdr,
+ &data_ptr);
+ if (rval != QLA_SUCCESS)
+ qla4_8xxx_mark_entry_skipped(ha, entry_hdr, i);
+ break;
case QLA8XXX_RDNOP:
default:
qla4_8xxx_mark_entry_skipped(ha, entry_hdr, i);
diff --git a/drivers/scsi/qla4xxx/ql4_nx.h b/drivers/scsi/qla4xxx/ql4_nx.h
index 14500a0..337d9fcf 100644
--- a/drivers/scsi/qla4xxx/ql4_nx.h
+++ b/drivers/scsi/qla4xxx/ql4_nx.h
@@ -858,6 +858,9 @@
#define QLA83XX_POLLRD 35
#define QLA83XX_RDMUX2 36
#define QLA83XX_POLLRDMWR 37
+#define QLA8044_RDDFE 38
+#define QLA8044_RDMDIO 39
+#define QLA8044_POLLWR 40
#define QLA8XXX_RDROM 71
#define QLA8XXX_RDMEM 72
#define QLA8XXX_CNTRL 98
diff --git a/drivers/scsi/qla4xxx/ql4_os.c b/drivers/scsi/qla4xxx/ql4_os.c
index 459b9f7..3202063 100644
--- a/drivers/scsi/qla4xxx/ql4_os.c
+++ b/drivers/scsi/qla4xxx/ql4_os.c
@@ -83,12 +83,12 @@
" Target Session Recovery Timeout.\n"
"\t\t Default: 120 sec.");
-int ql4xmdcapmask = 0x1F;
+int ql4xmdcapmask = 0;
module_param(ql4xmdcapmask, int, S_IRUGO);
MODULE_PARM_DESC(ql4xmdcapmask,
" Set the Minidump driver capture mask level.\n"
- "\t\t Default is 0x1F.\n"
- "\t\t Can be set to 0x3, 0x7, 0xF, 0x1F, 0x3F, 0x7F");
+ "\t\t Default is 0 (firmware default capture mask)\n"
+ "\t\t Can be set to 0x3, 0x7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF");
int ql4xenablemd = 1;
module_param(ql4xenablemd, int, S_IRUGO | S_IWUSR);
@@ -1742,6 +1742,9 @@
struct sockaddr *dst_addr;
struct scsi_qla_host *ha;
+ if (!qla_ep)
+ return -ENOTCONN;
+
ha = to_qla_host(qla_ep->host);
DEBUG2(ql4_printk(KERN_INFO, ha, "%s: host: %ld\n", __func__,
ha->host_no));
@@ -1749,9 +1752,6 @@
switch (param) {
case ISCSI_PARAM_CONN_PORT:
case ISCSI_PARAM_CONN_ADDRESS:
- if (!qla_ep)
- return -ENOTCONN;
-
dst_addr = (struct sockaddr *)&qla_ep->dst_addr;
if (!dst_addr)
return -ENOTCONN;
@@ -2879,7 +2879,6 @@
struct iscsi_conn *conn;
struct qla_conn *qla_conn;
struct sockaddr *dst_addr;
- int len = 0;
conn = cls_conn->dd_data;
qla_conn = conn->dd_data;
@@ -2893,9 +2892,6 @@
default:
return iscsi_conn_get_param(cls_conn, param, buf);
}
-
- return len;
-
}
int qla4xxx_get_ddb_index(struct scsi_qla_host *ha, uint16_t *ddb_index)
@@ -3569,14 +3565,13 @@
if (test_bit(OPT_IPV6_DEVICE, &options)) {
conn->ipv6_traffic_class = fw_ddb_entry->ipv4_tos;
- conn->link_local_ipv6_addr = kzalloc(IPv6_ADDR_LEN, GFP_KERNEL);
+ conn->link_local_ipv6_addr = kmemdup(
+ fw_ddb_entry->link_local_ipv6_addr,
+ IPv6_ADDR_LEN, GFP_KERNEL);
if (!conn->link_local_ipv6_addr) {
rc = -ENOMEM;
goto exit_copy;
}
-
- memcpy(conn->link_local_ipv6_addr,
- fw_ddb_entry->link_local_ipv6_addr, IPv6_ADDR_LEN);
} else {
conn->ipv4_tos = fw_ddb_entry->ipv4_tos;
}
@@ -4565,6 +4560,7 @@
test_bit(DPC_LINK_CHANGED, &ha->dpc_flags) ||
test_bit(DPC_HA_UNRECOVERABLE, &ha->dpc_flags) ||
test_bit(DPC_HA_NEED_QUIESCENT, &ha->dpc_flags) ||
+ test_bit(DPC_SYSFS_DDB_EXPORT, &ha->dpc_flags) ||
test_bit(DPC_AEN, &ha->dpc_flags)) {
DEBUG2(printk("scsi%ld: %s: scheduling dpc routine"
" - dpc flags = 0x%lx\n",
@@ -4862,9 +4858,6 @@
ha->host_no, __func__));
status = ha->isp_ops->reset_firmware(ha);
if (status == QLA_SUCCESS) {
- if (!test_bit(AF_FW_RECOVERY, &ha->flags))
- qla4xxx_cmd_wait(ha);
-
ha->isp_ops->disable_intrs(ha);
qla4xxx_process_aen(ha, FLUSH_DDB_CHANGED_AENS);
qla4xxx_abort_active_cmds(ha, DID_RESET << 16);
@@ -5432,6 +5425,11 @@
qla4xxx_relogin_all_devices(ha);
}
}
+ if (test_and_clear_bit(DPC_SYSFS_DDB_EXPORT, &ha->dpc_flags)) {
+ if (qla4xxx_sysfs_ddb_export(ha))
+ ql4_printk(KERN_ERR, ha, "%s: Error exporting ddb to sysfs\n",
+ __func__);
+ }
}
/**
@@ -8409,7 +8407,7 @@
*
* Export the firmware DDB for all send targets and normal targets to sysfs.
**/
-static int qla4xxx_sysfs_ddb_export(struct scsi_qla_host *ha)
+int qla4xxx_sysfs_ddb_export(struct scsi_qla_host *ha)
{
struct dev_db_entry *fw_ddb_entry = NULL;
dma_addr_t fw_ddb_entry_dma;
@@ -8847,11 +8845,8 @@
ql4_printk(KERN_ERR, ha,
"%s: No iSCSI boot target configured\n", __func__);
- if (qla4xxx_sysfs_ddb_export(ha))
- ql4_printk(KERN_ERR, ha,
- "%s: Error exporting ddb to sysfs\n", __func__);
-
- /* Perform the build ddb list and login to each */
+ set_bit(DPC_SYSFS_DDB_EXPORT, &ha->dpc_flags);
+ /* Perform the build ddb list and login to each */
qla4xxx_build_ddb_list(ha, INIT_ADAPTER);
iscsi_host_for_each_session(ha->host, qla4xxx_login_flash_ddb);
qla4xxx_wait_login_resp_boot_tgt(ha);
diff --git a/drivers/scsi/qla4xxx/ql4_version.h b/drivers/scsi/qla4xxx/ql4_version.h
index c6ba0a6..f11eaa7 100644
--- a/drivers/scsi/qla4xxx/ql4_version.h
+++ b/drivers/scsi/qla4xxx/ql4_version.h
@@ -5,4 +5,4 @@
* See LICENSE.qla4xxx for copyright and licensing details.
*/
-#define QLA4XXX_DRIVER_VERSION "5.04.00-k4"
+#define QLA4XXX_DRIVER_VERSION "5.04.00-k6"
diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c
index f3e9cc0..1328a26 100644
--- a/drivers/scsi/scsi_debug.c
+++ b/drivers/scsi/scsi_debug.c
@@ -130,6 +130,7 @@
#define SCSI_DEBUG_OPT_DIF_ERR 32
#define SCSI_DEBUG_OPT_DIX_ERR 64
#define SCSI_DEBUG_OPT_MAC_TIMEOUT 128
+#define SCSI_DEBUG_OPT_SHORT_TRANSFER 256
/* When "every_nth" > 0 then modulo "every_nth" commands:
* - a no response is simulated if SCSI_DEBUG_OPT_TIMEOUT is set
* - a RECOVERED_ERROR is simulated on successful read and write
@@ -3583,6 +3584,7 @@
int inj_transport = 0;
int inj_dif = 0;
int inj_dix = 0;
+ int inj_short = 0;
int delay_override = 0;
int unmap = 0;
@@ -3628,6 +3630,8 @@
inj_dif = 1; /* to reads and writes below */
else if (SCSI_DEBUG_OPT_DIX_ERR & scsi_debug_opts)
inj_dix = 1; /* to reads and writes below */
+ else if (SCSI_DEBUG_OPT_SHORT_TRANSFER & scsi_debug_opts)
+ inj_short = 1;
}
if (devip->wlun) {
@@ -3744,6 +3748,10 @@
if (scsi_debug_fake_rw)
break;
get_data_transfer_info(cmd, &lba, &num, &ei_lba);
+
+ if (inj_short)
+ num /= 2;
+
errsts = resp_read(SCpnt, lba, num, devip, ei_lba);
if (inj_recovered && (0 == errsts)) {
mk_sense_buffer(devip, RECOVERED_ERROR,
diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c
index af62461..cbe38e5 100644
--- a/drivers/scsi/scsi_error.c
+++ b/drivers/scsi/scsi_error.c
@@ -1029,6 +1029,7 @@
rtn = NEEDS_RETRY;
} else {
timeleft = wait_for_completion_timeout(&done, timeout);
+ rtn = SUCCESS;
}
shost->eh_action = NULL;
@@ -2307,6 +2308,12 @@
}
scmd = scsi_get_command(dev, GFP_KERNEL);
+ if (!scmd) {
+ rtn = FAILED;
+ put_device(&dev->sdev_gendev);
+ goto out_put_autopm_host;
+ }
+
blk_rq_init(NULL, &req);
scmd->request = &req;
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index c3c1697..f7e3163 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -512,68 +512,6 @@
scsi_run_queue(sdev->request_queue);
}
-static void __scsi_release_buffers(struct scsi_cmnd *, int);
-
-/*
- * Function: scsi_end_request()
- *
- * Purpose: Post-processing of completed commands (usually invoked at end
- * of upper level post-processing and scsi_io_completion).
- *
- * Arguments: cmd - command that is complete.
- * error - 0 if I/O indicates success, < 0 for I/O error.
- * bytes - number of bytes of completed I/O
- * requeue - indicates whether we should requeue leftovers.
- *
- * Lock status: Assumed that lock is not held upon entry.
- *
- * Returns: cmd if requeue required, NULL otherwise.
- *
- * Notes: This is called for block device requests in order to
- * mark some number of sectors as complete.
- *
- * We are guaranteeing that the request queue will be goosed
- * at some point during this call.
- * Notes: If cmd was requeued, upon return it will be a stale pointer.
- */
-static struct scsi_cmnd *scsi_end_request(struct scsi_cmnd *cmd, int error,
- int bytes, int requeue)
-{
- struct request_queue *q = cmd->device->request_queue;
- struct request *req = cmd->request;
-
- /*
- * If there are blocks left over at the end, set up the command
- * to queue the remainder of them.
- */
- if (blk_end_request(req, error, bytes)) {
- /* kill remainder if no retrys */
- if (error && scsi_noretry_cmd(cmd))
- blk_end_request_all(req, error);
- else {
- if (requeue) {
- /*
- * Bleah. Leftovers again. Stick the
- * leftovers in the front of the
- * queue, and goose the queue again.
- */
- scsi_release_buffers(cmd);
- scsi_requeue_command(q, cmd);
- cmd = NULL;
- }
- return cmd;
- }
- }
-
- /*
- * This will goose the queue request function at the end, so we don't
- * need to worry about launching another command.
- */
- __scsi_release_buffers(cmd, 0);
- scsi_next_command(cmd);
- return NULL;
-}
-
static inline unsigned int scsi_sgtable_index(unsigned short nents)
{
unsigned int index;
@@ -625,30 +563,10 @@
__sg_free_table(&sdb->table, SCSI_MAX_SG_SEGMENTS, scsi_sg_free);
}
-static void __scsi_release_buffers(struct scsi_cmnd *cmd, int do_bidi_check)
-{
-
- if (cmd->sdb.table.nents)
- scsi_free_sgtable(&cmd->sdb);
-
- memset(&cmd->sdb, 0, sizeof(cmd->sdb));
-
- if (do_bidi_check && scsi_bidi_cmnd(cmd)) {
- struct scsi_data_buffer *bidi_sdb =
- cmd->request->next_rq->special;
- scsi_free_sgtable(bidi_sdb);
- kmem_cache_free(scsi_sdb_cache, bidi_sdb);
- cmd->request->next_rq->special = NULL;
- }
-
- if (scsi_prot_sg_count(cmd))
- scsi_free_sgtable(cmd->prot_sdb);
-}
-
/*
* Function: scsi_release_buffers()
*
- * Purpose: Completion processing for block device I/O requests.
+ * Purpose: Free resources allocate for a scsi_command.
*
* Arguments: cmd - command that we are bailing.
*
@@ -659,15 +577,29 @@
* Notes: In the event that an upper level driver rejects a
* command, we must release resources allocated during
* the __init_io() function. Primarily this would involve
- * the scatter-gather table, and potentially any bounce
- * buffers.
+ * the scatter-gather table.
*/
void scsi_release_buffers(struct scsi_cmnd *cmd)
{
- __scsi_release_buffers(cmd, 1);
+ if (cmd->sdb.table.nents)
+ scsi_free_sgtable(&cmd->sdb);
+
+ memset(&cmd->sdb, 0, sizeof(cmd->sdb));
+
+ if (scsi_prot_sg_count(cmd))
+ scsi_free_sgtable(cmd->prot_sdb);
}
EXPORT_SYMBOL(scsi_release_buffers);
+static void scsi_release_bidi_buffers(struct scsi_cmnd *cmd)
+{
+ struct scsi_data_buffer *bidi_sdb = cmd->request->next_rq->special;
+
+ scsi_free_sgtable(bidi_sdb);
+ kmem_cache_free(scsi_sdb_cache, bidi_sdb);
+ cmd->request->next_rq->special = NULL;
+}
+
/**
* __scsi_error_from_host_byte - translate SCSI error code into errno
* @cmd: SCSI command (unused)
@@ -725,16 +657,9 @@
*
* Returns: Nothing
*
- * Notes: This function is matched in terms of capabilities to
- * the function that created the scatter-gather list.
- * In other words, if there are no bounce buffers
- * (the normal case for most drivers), we don't need
- * the logic to deal with cleaning up afterwards.
- *
- * We must call scsi_end_request(). This will finish off
- * the specified number of sectors. If we are done, the
- * command block will be released and the queue function
- * will be goosed. If we are not done then we have to
+ * Notes: We will finish off the specified number of sectors. If we
+ * are done, the command block will be released and the queue
+ * function will be goosed. If we are not done then we have to
* figure out what to do next:
*
* a) We can call scsi_requeue_command(). The request
@@ -743,7 +668,7 @@
* be used if we made forward progress, or if we want
* to switch from READ(10) to READ(6) for example.
*
- * b) We can call scsi_queue_insert(). The request will
+ * b) We can call __scsi_queue_insert(). The request will
* be put back on the queue and retried using the same
* command as before, possibly after a delay.
*
@@ -801,6 +726,8 @@
req->next_rq->resid_len = scsi_in(cmd)->resid;
scsi_release_buffers(cmd);
+ scsi_release_bidi_buffers(cmd);
+
blk_end_request_all(req, 0);
scsi_next_command(cmd);
@@ -840,12 +767,25 @@
}
/*
- * A number of bytes were successfully read. If there
- * are leftovers and there is some kind of error
- * (result != 0), retry the rest.
+ * If we finished all bytes in the request we are done now.
*/
- if (scsi_end_request(cmd, error, good_bytes, result == 0) == NULL)
- return;
+ if (!blk_end_request(req, error, good_bytes))
+ goto next_command;
+
+ /*
+ * Kill remainder if no retrys.
+ */
+ if (error && scsi_noretry_cmd(cmd)) {
+ blk_end_request_all(req, error);
+ goto next_command;
+ }
+
+ /*
+ * If there had been no error, but we have leftover bytes in the
+ * requeues just queue the command up again.
+ */
+ if (result == 0)
+ goto requeue;
error = __scsi_error_from_host_byte(cmd, result);
@@ -973,7 +913,6 @@
switch (action) {
case ACTION_FAIL:
/* Give up and fail the remainder of the request */
- scsi_release_buffers(cmd);
if (!(req->cmd_flags & REQ_QUIET)) {
if (description)
scmd_printk(KERN_INFO, cmd, "%s\n",
@@ -983,12 +922,11 @@
scsi_print_sense("", cmd);
scsi_print_command(cmd);
}
- if (blk_end_request_err(req, error))
- scsi_requeue_command(q, cmd);
- else
- scsi_next_command(cmd);
- break;
+ if (!blk_end_request_err(req, error))
+ goto next_command;
+ /*FALLTHRU*/
case ACTION_REPREP:
+ requeue:
/* Unprep the request and put it back at the head of the queue.
* A new command will be prepared and issued.
*/
@@ -1004,6 +942,11 @@
__scsi_queue_insert(cmd, SCSI_MLQUEUE_DEVICE_BUSY, 0);
break;
}
+ return;
+
+next_command:
+ scsi_release_buffers(cmd);
+ scsi_next_command(cmd);
}
static int scsi_init_sgtable(struct request *req, struct scsi_data_buffer *sdb,
@@ -1128,15 +1071,7 @@
int scsi_setup_blk_pc_cmnd(struct scsi_device *sdev, struct request *req)
{
- struct scsi_cmnd *cmd;
- int ret = scsi_prep_state_check(sdev, req);
-
- if (ret != BLKPREP_OK)
- return ret;
-
- cmd = scsi_get_cmd_from_req(sdev, req);
- if (unlikely(!cmd))
- return BLKPREP_DEFER;
+ struct scsi_cmnd *cmd = req->special;
/*
* BLOCK_PC requests may transfer data, in which case they must
@@ -1179,15 +1114,11 @@
*/
int scsi_setup_fs_cmnd(struct scsi_device *sdev, struct request *req)
{
- struct scsi_cmnd *cmd;
- int ret = scsi_prep_state_check(sdev, req);
-
- if (ret != BLKPREP_OK)
- return ret;
+ struct scsi_cmnd *cmd = req->special;
if (unlikely(sdev->scsi_dh_data && sdev->scsi_dh_data->scsi_dh
&& sdev->scsi_dh_data->scsi_dh->prep_fn)) {
- ret = sdev->scsi_dh_data->scsi_dh->prep_fn(sdev, req);
+ int ret = sdev->scsi_dh_data->scsi_dh->prep_fn(sdev, req);
if (ret != BLKPREP_OK)
return ret;
}
@@ -1197,16 +1128,13 @@
*/
BUG_ON(!req->nr_phys_segments);
- cmd = scsi_get_cmd_from_req(sdev, req);
- if (unlikely(!cmd))
- return BLKPREP_DEFER;
-
memset(cmd->cmnd, 0, BLK_MAX_CDB);
return scsi_init_io(cmd, GFP_ATOMIC);
}
EXPORT_SYMBOL(scsi_setup_fs_cmnd);
-int scsi_prep_state_check(struct scsi_device *sdev, struct request *req)
+static int
+scsi_prep_state_check(struct scsi_device *sdev, struct request *req)
{
int ret = BLKPREP_OK;
@@ -1258,9 +1186,9 @@
}
return ret;
}
-EXPORT_SYMBOL(scsi_prep_state_check);
-int scsi_prep_return(struct request_queue *q, struct request *req, int ret)
+static int
+scsi_prep_return(struct request_queue *q, struct request *req, int ret)
{
struct scsi_device *sdev = q->queuedata;
@@ -1291,18 +1219,44 @@
return ret;
}
-EXPORT_SYMBOL(scsi_prep_return);
-int scsi_prep_fn(struct request_queue *q, struct request *req)
+static int scsi_prep_fn(struct request_queue *q, struct request *req)
{
struct scsi_device *sdev = q->queuedata;
- int ret = BLKPREP_KILL;
+ struct scsi_cmnd *cmd;
+ int ret;
- if (req->cmd_type == REQ_TYPE_BLOCK_PC)
+ ret = scsi_prep_state_check(sdev, req);
+ if (ret != BLKPREP_OK)
+ goto out;
+
+ cmd = scsi_get_cmd_from_req(sdev, req);
+ if (unlikely(!cmd)) {
+ ret = BLKPREP_DEFER;
+ goto out;
+ }
+
+ if (req->cmd_type == REQ_TYPE_FS)
+ ret = scsi_cmd_to_driver(cmd)->init_command(cmd);
+ else if (req->cmd_type == REQ_TYPE_BLOCK_PC)
ret = scsi_setup_blk_pc_cmnd(sdev, req);
+ else
+ ret = BLKPREP_KILL;
+
+out:
return scsi_prep_return(q, req, ret);
}
-EXPORT_SYMBOL(scsi_prep_fn);
+
+static void scsi_unprep_fn(struct request_queue *q, struct request *req)
+{
+ if (req->cmd_type == REQ_TYPE_FS) {
+ struct scsi_cmnd *cmd = req->special;
+ struct scsi_driver *drv = scsi_cmd_to_driver(cmd);
+
+ if (drv->uninit_command)
+ drv->uninit_command(cmd);
+ }
+}
/*
* scsi_dev_queue_ready: if we can send requests to sdev, return 1 else
@@ -1723,6 +1677,7 @@
return NULL;
blk_queue_prep_rq(q, scsi_prep_fn);
+ blk_queue_unprep_rq(q, scsi_unprep_fn);
blk_queue_softirq_done(q, scsi_softirq_done);
blk_queue_rq_timed_out(q, scsi_times_out);
blk_queue_lld_busy(q, scsi_lld_busy);
diff --git a/drivers/scsi/scsi_sysctl.c b/drivers/scsi/scsi_sysctl.c
index 2b6b93f..546f162 100644
--- a/drivers/scsi/scsi_sysctl.c
+++ b/drivers/scsi/scsi_sysctl.c
@@ -12,7 +12,7 @@
#include "scsi_priv.h"
-static ctl_table scsi_table[] = {
+static struct ctl_table scsi_table[] = {
{ .procname = "logging_level",
.data = &scsi_logging_level,
.maxlen = sizeof(scsi_logging_level),
@@ -21,14 +21,14 @@
{ }
};
-static ctl_table scsi_dir_table[] = {
+static struct ctl_table scsi_dir_table[] = {
{ .procname = "scsi",
.mode = 0555,
.child = scsi_table },
{ }
};
-static ctl_table scsi_root_table[] = {
+static struct ctl_table scsi_root_table[] = {
{ .procname = "dev",
.mode = 0555,
.child = scsi_dir_table },
diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 96af195..e9689d5 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -109,6 +109,8 @@
static int sd_suspend_runtime(struct device *);
static int sd_resume(struct device *);
static void sd_rescan(struct device *);
+static int sd_init_command(struct scsi_cmnd *SCpnt);
+static void sd_uninit_command(struct scsi_cmnd *SCpnt);
static int sd_done(struct scsi_cmnd *);
static int sd_eh_action(struct scsi_cmnd *, int);
static void sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer);
@@ -503,6 +505,8 @@
.pm = &sd_pm_ops,
},
.rescan = sd_rescan,
+ .init_command = sd_init_command,
+ .uninit_command = sd_uninit_command,
.done = sd_done,
.eh_action = sd_eh_action,
};
@@ -836,9 +840,9 @@
return scsi_setup_blk_pc_cmnd(sdp, rq);
}
-static void sd_unprep_fn(struct request_queue *q, struct request *rq)
+static void sd_uninit_command(struct scsi_cmnd *SCpnt)
{
- struct scsi_cmnd *SCpnt = rq->special;
+ struct request *rq = SCpnt->request;
if (rq->cmd_flags & REQ_DISCARD)
__free_page(rq->completion_data);
@@ -850,18 +854,10 @@
}
}
-/**
- * sd_prep_fn - build a scsi (read or write) command from
- * information in the request structure.
- * @SCpnt: pointer to mid-level's per scsi command structure that
- * contains request and into which the scsi command is written
- *
- * Returns 1 if successful and 0 if error (or cannot be done now).
- **/
-static int sd_prep_fn(struct request_queue *q, struct request *rq)
+static int sd_init_command(struct scsi_cmnd *SCpnt)
{
- struct scsi_cmnd *SCpnt;
- struct scsi_device *sdp = q->queuedata;
+ struct request *rq = SCpnt->request;
+ struct scsi_device *sdp = SCpnt->device;
struct gendisk *disk = rq->rq_disk;
struct scsi_disk *sdkp;
sector_t block = blk_rq_pos(rq);
@@ -883,12 +879,6 @@
} else if (rq->cmd_flags & REQ_FLUSH) {
ret = scsi_setup_flush_cmnd(sdp, rq);
goto out;
- } else if (rq->cmd_type == REQ_TYPE_BLOCK_PC) {
- ret = scsi_setup_blk_pc_cmnd(sdp, rq);
- goto out;
- } else if (rq->cmd_type != REQ_TYPE_FS) {
- ret = BLKPREP_KILL;
- goto out;
}
ret = scsi_setup_fs_cmnd(sdp, rq);
if (ret != BLKPREP_OK)
@@ -900,11 +890,10 @@
* is used for a killable error condition */
ret = BLKPREP_KILL;
- SCSI_LOG_HLQUEUE(1, scmd_printk(KERN_INFO, SCpnt,
- "sd_prep_fn: block=%llu, "
- "count=%d\n",
- (unsigned long long)block,
- this_count));
+ SCSI_LOG_HLQUEUE(1,
+ scmd_printk(KERN_INFO, SCpnt,
+ "%s: block=%llu, count=%d\n",
+ __func__, (unsigned long long)block, this_count));
if (!sdp || !scsi_device_online(sdp) ||
block + blk_rq_sectors(rq) > get_capacity(disk)) {
@@ -1124,7 +1113,7 @@
*/
ret = BLKPREP_OK;
out:
- return scsi_prep_return(q, rq, ret);
+ return ret;
}
/**
@@ -1686,12 +1675,12 @@
sshdr.ascq));
}
#endif
+ sdkp->medium_access_timed_out = 0;
+
if (driver_byte(result) != DRIVER_SENSE &&
(!sense_valid || sense_deferred))
goto out;
- sdkp->medium_access_timed_out = 0;
-
switch (sshdr.sense_key) {
case HARDWARE_ERROR:
case MEDIUM_ERROR:
@@ -2875,9 +2864,6 @@
sd_revalidate_disk(gd);
- blk_queue_prep_rq(sdp->request_queue, sd_prep_fn);
- blk_queue_unprep_rq(sdp->request_queue, sd_unprep_fn);
-
gd->driverfs_dev = &sdp->sdev_gendev;
gd->flags = GENHD_FL_EXT_DEVT;
if (sdp->removable) {
@@ -3025,8 +3011,6 @@
async_synchronize_full_domain(&scsi_sd_pm_domain);
async_synchronize_full_domain(&scsi_sd_probe_domain);
- blk_queue_prep_rq(sdkp->device->request_queue, scsi_prep_fn);
- blk_queue_unprep_rq(sdkp->device->request_queue, NULL);
device_del(&sdkp->dev);
del_gendisk(sdkp->disk);
sd_shutdown(dev);
diff --git a/drivers/scsi/sr.c b/drivers/scsi/sr.c
index 40d8592..93cbd36 100644
--- a/drivers/scsi/sr.c
+++ b/drivers/scsi/sr.c
@@ -79,6 +79,7 @@
static DEFINE_MUTEX(sr_mutex);
static int sr_probe(struct device *);
static int sr_remove(struct device *);
+static int sr_init_command(struct scsi_cmnd *SCpnt);
static int sr_done(struct scsi_cmnd *);
static int sr_runtime_suspend(struct device *dev);
@@ -94,6 +95,7 @@
.remove = sr_remove,
.pm = &sr_pm_ops,
},
+ .init_command = sr_init_command,
.done = sr_done,
};
@@ -378,21 +380,14 @@
return good_bytes;
}
-static int sr_prep_fn(struct request_queue *q, struct request *rq)
+static int sr_init_command(struct scsi_cmnd *SCpnt)
{
int block = 0, this_count, s_size;
struct scsi_cd *cd;
- struct scsi_cmnd *SCpnt;
- struct scsi_device *sdp = q->queuedata;
+ struct request *rq = SCpnt->request;
+ struct scsi_device *sdp = SCpnt->device;
int ret;
- if (rq->cmd_type == REQ_TYPE_BLOCK_PC) {
- ret = scsi_setup_blk_pc_cmnd(sdp, rq);
- goto out;
- } else if (rq->cmd_type != REQ_TYPE_FS) {
- ret = BLKPREP_KILL;
- goto out;
- }
ret = scsi_setup_fs_cmnd(sdp, rq);
if (ret != BLKPREP_OK)
goto out;
@@ -517,7 +512,7 @@
*/
ret = BLKPREP_OK;
out:
- return scsi_prep_return(q, rq, ret);
+ return ret;
}
static int sr_block_open(struct block_device *bdev, fmode_t mode)
@@ -718,7 +713,6 @@
/* FIXME: need to handle a get_capabilities failure properly ?? */
get_capabilities(cd);
- blk_queue_prep_rq(sdev->request_queue, sr_prep_fn);
sr_vendor_init(cd);
disk->driverfs_dev = &sdev->sdev_gendev;
@@ -993,7 +987,6 @@
scsi_autopm_get_device(cd->device);
- blk_queue_prep_rq(cd->device->request_queue, scsi_prep_fn);
del_gendisk(cd->disk);
mutex_lock(&sr_ref_mutex);
diff --git a/drivers/scsi/sun3_NCR5380.c b/drivers/scsi/sun3_NCR5380.c
index 636bbe0..8822079 100644
--- a/drivers/scsi/sun3_NCR5380.c
+++ b/drivers/scsi/sun3_NCR5380.c
@@ -364,7 +364,7 @@
return( 0 );
if (TagAlloc[cmd->device->id][cmd->device->lun].nr_allocated >=
TagAlloc[cmd->device->id][cmd->device->lun].queue_size ) {
- TAG_PRINTK( "scsi%d: target %d lun %d: no free tags\n",
+ dprintk(NDEBUG_TAGS, "scsi%d: target %d lun %d: no free tags\n",
H_NO(cmd), cmd->device->id, cmd->device->lun );
return( 1 );
}
@@ -388,7 +388,7 @@
!setup_use_tagged_queuing || !cmd->device->tagged_supported) {
cmd->tag = TAG_NONE;
hostdata->busy[cmd->device->id] |= (1 << cmd->device->lun);
- TAG_PRINTK( "scsi%d: target %d lun %d now allocated by untagged "
+ dprintk(NDEBUG_TAGS, "scsi%d: target %d lun %d now allocated by untagged "
"command\n", H_NO(cmd), cmd->device->id, cmd->device->lun );
}
else {
@@ -397,7 +397,7 @@
cmd->tag = find_first_zero_bit( &ta->allocated, MAX_TAGS );
set_bit( cmd->tag, &ta->allocated );
ta->nr_allocated++;
- TAG_PRINTK( "scsi%d: using tag %d for target %d lun %d "
+ dprintk(NDEBUG_TAGS, "scsi%d: using tag %d for target %d lun %d "
"(now %d tags in use)\n",
H_NO(cmd), cmd->tag, cmd->device->id, cmd->device->lun,
ta->nr_allocated );
@@ -415,7 +415,7 @@
if (cmd->tag == TAG_NONE) {
hostdata->busy[cmd->device->id] &= ~(1 << cmd->device->lun);
- TAG_PRINTK( "scsi%d: target %d lun %d untagged cmd finished\n",
+ dprintk(NDEBUG_TAGS, "scsi%d: target %d lun %d untagged cmd finished\n",
H_NO(cmd), cmd->device->id, cmd->device->lun );
}
else if (cmd->tag >= MAX_TAGS) {
@@ -426,7 +426,7 @@
TAG_ALLOC *ta = &TagAlloc[cmd->device->id][cmd->device->lun];
clear_bit( cmd->tag, &ta->allocated );
ta->nr_allocated--;
- TAG_PRINTK( "scsi%d: freed tag %d for target %d lun %d\n",
+ dprintk(NDEBUG_TAGS, "scsi%d: freed tag %d for target %d lun %d\n",
H_NO(cmd), cmd->tag, cmd->device->id, cmd->device->lun );
}
}
@@ -484,7 +484,7 @@
#include <linux/delay.h>
-#if 1
+#if NDEBUG
static struct {
unsigned char mask;
const char * name;}
@@ -572,12 +572,6 @@
}
}
-#else /* !NDEBUG */
-
-/* dummies... */
-__inline__ void NCR5380_print(struct Scsi_Host *instance) { };
-__inline__ void NCR5380_print_phase(struct Scsi_Host *instance) { };
-
#endif
/*
@@ -618,7 +612,7 @@
{
static int done = 0;
if (!done) {
- INI_PRINTK("scsi : NCR5380_all_init()\n");
+ dprintk(NDEBUG_INIT, "scsi : NCR5380_all_init()\n");
done = 1;
}
}
@@ -681,8 +675,8 @@
Scsi_Cmnd *ptr;
unsigned long flags;
- NCR_PRINT(NDEBUG_ANY);
- NCR_PRINT_PHASE(NDEBUG_ANY);
+ NCR5380_dprint(NDEBUG_ANY, instance);
+ NCR5380_dprint_phase(NDEBUG_ANY, instance);
hostdata = (struct NCR5380_hostdata *)instance->hostdata;
@@ -928,7 +922,7 @@
local_irq_restore(flags);
- QU_PRINTK("scsi%d: command added to %s of queue\n", H_NO(cmd),
+ dprintk(NDEBUG_QUEUES, "scsi%d: command added to %s of queue\n", H_NO(cmd),
(cmd->cmnd[0] == REQUEST_SENSE) ? "head" : "tail");
/* If queue_command() is called from an interrupt (real one or bottom
@@ -998,7 +992,7 @@
done = 1;
if (!hostdata->connected) {
- MAIN_PRINTK( "scsi%d: not connected\n", HOSTNO );
+ dprintk(NDEBUG_MAIN, "scsi%d: not connected\n", HOSTNO );
/*
* Search through the issue_queue for a command destined
* for a target that's not busy.
@@ -1012,12 +1006,8 @@
for (tmp = (struct scsi_cmnd *) hostdata->issue_queue,
prev = NULL; tmp; prev = tmp, tmp = NEXT(tmp) ) {
-#if (NDEBUG & NDEBUG_LISTS)
if (prev != tmp)
- printk("MAIN tmp=%p target=%d busy=%d lun=%d\n",
- tmp, tmp->target, hostdata->busy[tmp->target],
- tmp->lun);
-#endif
+ dprintk(NDEBUG_LISTS, "MAIN tmp=%p target=%d busy=%d lun=%d\n", tmp, tmp->device->id, hostdata->busy[tmp->device->id], tmp->device->lun);
/* When we find one, remove it from the issue queue. */
/* ++guenther: possible race with Falcon locking */
if (
@@ -1047,9 +1037,9 @@
* On failure, we must add the command back to the
* issue queue so we can keep trying.
*/
- MAIN_PRINTK("scsi%d: main(): command for target %d "
+ dprintk(NDEBUG_MAIN, "scsi%d: main(): command for target %d "
"lun %d removed from issue_queue\n",
- HOSTNO, tmp->target, tmp->lun);
+ HOSTNO, tmp->device->id, tmp->device->lun);
/*
* REQUEST SENSE commands are issued without tagged
* queueing, even on SCSI-II devices because the
@@ -1076,7 +1066,7 @@
cmd_free_tag( tmp );
#endif
local_irq_restore(flags);
- MAIN_PRINTK("scsi%d: main(): select() failed, "
+ dprintk(NDEBUG_MAIN, "scsi%d: main(): select() failed, "
"returned to issue_queue\n", HOSTNO);
if (hostdata->connected)
break;
@@ -1090,10 +1080,10 @@
#endif
) {
local_irq_restore(flags);
- MAIN_PRINTK("scsi%d: main: performing information transfer\n",
+ dprintk(NDEBUG_MAIN, "scsi%d: main: performing information transfer\n",
HOSTNO);
NCR5380_information_transfer(instance);
- MAIN_PRINTK("scsi%d: main: done set false\n", HOSTNO);
+ dprintk(NDEBUG_MAIN, "scsi%d: main: done set false\n", HOSTNO);
done = 0;
}
} while (!done);
@@ -1130,7 +1120,7 @@
return;
}
- DMA_PRINTK("scsi%d: real DMA transfer complete, basr 0x%X, sr 0x%X\n",
+ dprintk(NDEBUG_DMA, "scsi%d: real DMA transfer complete, basr 0x%X, sr 0x%X\n",
HOSTNO, NCR5380_read(BUS_AND_STATUS_REG),
NCR5380_read(STATUS_REG));
@@ -1189,27 +1179,27 @@
int done = 1, handled = 0;
unsigned char basr;
- INT_PRINTK("scsi%d: NCR5380 irq triggered\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: NCR5380 irq triggered\n", HOSTNO);
/* Look for pending interrupts */
basr = NCR5380_read(BUS_AND_STATUS_REG);
- INT_PRINTK("scsi%d: BASR=%02x\n", HOSTNO, basr);
+ dprintk(NDEBUG_INTR, "scsi%d: BASR=%02x\n", HOSTNO, basr);
/* dispatch to appropriate routine if found and done=0 */
if (basr & BASR_IRQ) {
- NCR_PRINT(NDEBUG_INTR);
+ NCR5380_dprint(NDEBUG_INTR, instance);
if ((NCR5380_read(STATUS_REG) & (SR_SEL|SR_IO)) == (SR_SEL|SR_IO)) {
done = 0;
// ENABLE_IRQ();
- INT_PRINTK("scsi%d: SEL interrupt\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: SEL interrupt\n", HOSTNO);
NCR5380_reselect(instance);
(void) NCR5380_read(RESET_PARITY_INTERRUPT_REG);
}
else if (basr & BASR_PARITY_ERROR) {
- INT_PRINTK("scsi%d: PARITY interrupt\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: PARITY interrupt\n", HOSTNO);
(void) NCR5380_read(RESET_PARITY_INTERRUPT_REG);
}
else if ((NCR5380_read(STATUS_REG) & SR_RST) == SR_RST) {
- INT_PRINTK("scsi%d: RESET interrupt\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: RESET interrupt\n", HOSTNO);
(void)NCR5380_read(RESET_PARITY_INTERRUPT_REG);
}
else {
@@ -1229,7 +1219,7 @@
((basr & BASR_END_DMA_TRANSFER) ||
!(basr & BASR_PHASE_MATCH))) {
- INT_PRINTK("scsi%d: PHASE MISM or EOP interrupt\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: PHASE MISM or EOP interrupt\n", HOSTNO);
NCR5380_dma_complete( instance );
done = 0;
// ENABLE_IRQ();
@@ -1238,7 +1228,7 @@
{
/* MS: Ignore unknown phase mismatch interrupts (caused by EOP interrupt) */
if (basr & BASR_PHASE_MATCH)
- INT_PRINTK("scsi%d: unknown interrupt, "
+ dprintk(NDEBUG_INTR, "scsi%d: unknown interrupt, "
"BASR 0x%x, MR 0x%x, SR 0x%x\n",
HOSTNO, basr, NCR5380_read(MODE_REG),
NCR5380_read(STATUS_REG));
@@ -1262,7 +1252,7 @@
}
if (!done) {
- INT_PRINTK("scsi%d: in int routine, calling main\n", HOSTNO);
+ dprintk(NDEBUG_INTR, "scsi%d: in int routine, calling main\n", HOSTNO);
/* Put a call to NCR5380_main() on the queue... */
queue_main();
}
@@ -1338,8 +1328,8 @@
unsigned long flags;
hostdata->restart_select = 0;
- NCR_PRINT(NDEBUG_ARBITRATION);
- ARB_PRINTK("scsi%d: starting arbitration, id = %d\n", HOSTNO,
+ NCR5380_dprint(NDEBUG_ARBITRATION, instance);
+ dprintk(NDEBUG_ARBITRATION, "scsi%d: starting arbitration, id = %d\n", HOSTNO,
instance->this_id);
/*
@@ -1385,7 +1375,7 @@
&& !hostdata->connected);
#endif
- ARB_PRINTK("scsi%d: arbitration complete\n", HOSTNO);
+ dprintk(NDEBUG_ARBITRATION, "scsi%d: arbitration complete\n", HOSTNO);
if (hostdata->connected) {
NCR5380_write(MODE_REG, MR_BASE);
@@ -1406,7 +1396,7 @@
(NCR5380_read(INITIATOR_COMMAND_REG) & ICR_ARBITRATION_LOST) ||
hostdata->connected) {
NCR5380_write(MODE_REG, MR_BASE);
- ARB_PRINTK("scsi%d: lost arbitration, deasserting MR_ARBITRATE\n",
+ dprintk(NDEBUG_ARBITRATION, "scsi%d: lost arbitration, deasserting MR_ARBITRATE\n",
HOSTNO);
return -1;
}
@@ -1421,7 +1411,7 @@
hostdata->connected) {
NCR5380_write(MODE_REG, MR_BASE);
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- ARB_PRINTK("scsi%d: lost arbitration, deasserting ICR_ASSERT_SEL\n",
+ dprintk(NDEBUG_ARBITRATION, "scsi%d: lost arbitration, deasserting ICR_ASSERT_SEL\n",
HOSTNO);
return -1;
}
@@ -1444,7 +1434,7 @@
return -1;
}
- ARB_PRINTK("scsi%d: won arbitration\n", HOSTNO);
+ dprintk(NDEBUG_ARBITRATION, "scsi%d: won arbitration\n", HOSTNO);
/*
* Now that we have won arbitration, start Selection process, asserting
@@ -1504,7 +1494,7 @@
udelay(1);
- SEL_PRINTK("scsi%d: selecting target %d\n", HOSTNO, cmd->device->id);
+ dprintk(NDEBUG_SELECTION, "scsi%d: selecting target %d\n", HOSTNO, cmd->device->id);
/*
* The SCSI specification calls for a 250 ms timeout for the actual
@@ -1559,7 +1549,7 @@
printk(KERN_ERR "scsi%d: weirdness\n", HOSTNO);
if (hostdata->restart_select)
printk(KERN_NOTICE "\trestart select\n");
- NCR_PRINT(NDEBUG_ANY);
+ NCR5380_dprint(NDEBUG_ANY, instance);
NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
return -1;
}
@@ -1572,7 +1562,7 @@
#endif
cmd->scsi_done(cmd);
NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
- SEL_PRINTK("scsi%d: target did not respond within 250ms\n", HOSTNO);
+ dprintk(NDEBUG_SELECTION, "scsi%d: target did not respond within 250ms\n", HOSTNO);
NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
return 0;
}
@@ -1597,7 +1587,7 @@
/* Wait for start of REQ/ACK handshake */
while (!(NCR5380_read(STATUS_REG) & SR_REQ));
- SEL_PRINTK("scsi%d: target %d selected, going into MESSAGE OUT phase.\n",
+ dprintk(NDEBUG_SELECTION, "scsi%d: target %d selected, going into MESSAGE OUT phase.\n",
HOSTNO, cmd->device->id);
tmp[0] = IDENTIFY(1, cmd->device->lun);
@@ -1617,7 +1607,7 @@
data = tmp;
phase = PHASE_MSGOUT;
NCR5380_transfer_pio(instance, &phase, &len, &data);
- SEL_PRINTK("scsi%d: nexus established.\n", HOSTNO);
+ dprintk(NDEBUG_SELECTION, "scsi%d: nexus established.\n", HOSTNO);
/* XXX need to handle errors here */
hostdata->connected = cmd;
#ifndef SUPPORT_TAGS
@@ -1680,12 +1670,12 @@
*/
while (!((tmp = NCR5380_read(STATUS_REG)) & SR_REQ));
- HSH_PRINTK("scsi%d: REQ detected\n", HOSTNO);
+ dprintk(NDEBUG_HANDSHAKE, "scsi%d: REQ detected\n", HOSTNO);
/* Check for phase mismatch */
if ((tmp & PHASE_MASK) != p) {
- PIO_PRINTK("scsi%d: phase mismatch\n", HOSTNO);
- NCR_PRINT_PHASE(NDEBUG_PIO);
+ dprintk(NDEBUG_PIO, "scsi%d: phase mismatch\n", HOSTNO);
+ NCR5380_dprint_phase(NDEBUG_PIO, instance);
break;
}
@@ -1708,24 +1698,24 @@
if (!((p & SR_MSG) && c > 1)) {
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE |
ICR_ASSERT_DATA);
- NCR_PRINT(NDEBUG_PIO);
+ NCR5380_dprint(NDEBUG_PIO, instance);
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE |
ICR_ASSERT_DATA | ICR_ASSERT_ACK);
} else {
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE |
ICR_ASSERT_DATA | ICR_ASSERT_ATN);
- NCR_PRINT(NDEBUG_PIO);
+ NCR5380_dprint(NDEBUG_PIO, instance);
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE |
ICR_ASSERT_DATA | ICR_ASSERT_ATN | ICR_ASSERT_ACK);
}
} else {
- NCR_PRINT(NDEBUG_PIO);
+ NCR5380_dprint(NDEBUG_PIO, instance);
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_ACK);
}
while (NCR5380_read(STATUS_REG) & SR_REQ);
- HSH_PRINTK("scsi%d: req false, handshake complete\n", HOSTNO);
+ dprintk(NDEBUG_HANDSHAKE, "scsi%d: req false, handshake complete\n", HOSTNO);
/*
* We have several special cases to consider during REQ/ACK handshaking :
@@ -1746,7 +1736,7 @@
}
} while (--c);
- PIO_PRINTK("scsi%d: residual %d\n", HOSTNO, c);
+ dprintk(NDEBUG_PIO, "scsi%d: residual %d\n", HOSTNO, c);
*count = c;
*data = d;
@@ -1854,7 +1844,7 @@
}
hostdata->dma_len = c;
- DMA_PRINTK("scsi%d: initializing DMA for %s, %d bytes %s %p\n",
+ dprintk(NDEBUG_DMA, "scsi%d: initializing DMA for %s, %d bytes %s %p\n",
HOSTNO, (p & SR_IO) ? "reading" : "writing",
c, (p & SR_IO) ? "to" : "from", *data);
@@ -1931,7 +1921,7 @@
phase = (tmp & PHASE_MASK);
if (phase != old_phase) {
old_phase = phase;
- NCR_PRINT_PHASE(NDEBUG_INFORMATION);
+ NCR5380_dprint_phase(NDEBUG_INFORMATION, instance);
}
if(phase == PHASE_CMDOUT) {
@@ -1996,7 +1986,7 @@
--cmd->SCp.buffers_residual;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
cmd->SCp.ptr = SGADDR(cmd->SCp.buffer);
- INF_PRINTK("scsi%d: %d bytes and %d buffers left\n",
+ dprintk(NDEBUG_INFORMATION, "scsi%d: %d bytes and %d buffers left\n",
HOSTNO, cmd->SCp.this_residual,
cmd->SCp.buffers_residual);
}
@@ -2088,7 +2078,7 @@
/* Accept message by clearing ACK */
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- LNK_PRINTK("scsi%d: target %d lun %d linked command "
+ dprintk(NDEBUG_LINKED, "scsi%d: target %d lun %d linked command "
"complete.\n", HOSTNO, cmd->device->id, cmd->device->lun);
/* Enable reselect interrupts */
@@ -2113,7 +2103,7 @@
* and don't free it! */
cmd->next_link->tag = cmd->tag;
cmd->result = cmd->SCp.Status | (cmd->SCp.Message << 8);
- LNK_PRINTK("scsi%d: target %d lun %d linked request "
+ dprintk(NDEBUG_LINKED, "scsi%d: target %d lun %d linked request "
"done, calling scsi_done().\n",
HOSTNO, cmd->device->id, cmd->device->lun);
#ifdef NCR5380_STATS
@@ -2128,7 +2118,7 @@
/* Accept message by clearing ACK */
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
hostdata->connected = NULL;
- QU_PRINTK("scsi%d: command for target %d, lun %d "
+ dprintk(NDEBUG_QUEUES, "scsi%d: command for target %d, lun %d "
"completed\n", HOSTNO, cmd->device->id, cmd->device->lun);
#ifdef SUPPORT_TAGS
cmd_free_tag( cmd );
@@ -2142,7 +2132,7 @@
/* ++Andreas: the mid level code knows about
QUEUE_FULL now. */
TAG_ALLOC *ta = &TagAlloc[cmd->device->id][cmd->device->lun];
- TAG_PRINTK("scsi%d: target %d lun %d returned "
+ dprintk(NDEBUG_TAGS, "scsi%d: target %d lun %d returned "
"QUEUE_FULL after %d commands\n",
HOSTNO, cmd->device->id, cmd->device->lun,
ta->nr_allocated);
@@ -2186,7 +2176,7 @@
if ((cmd->cmnd[0] != REQUEST_SENSE) &&
(status_byte(cmd->SCp.Status) == CHECK_CONDITION)) {
scsi_eh_prep_cmnd(cmd, &hostdata->ses, NULL, 0, ~0);
- ASEN_PRINTK("scsi%d: performing request sense\n",
+ dprintk(NDEBUG_AUTOSENSE, "scsi%d: performing request sense\n",
HOSTNO);
/* this is initialized from initialize_SCp
cmd->SCp.buffer = NULL;
@@ -2198,7 +2188,7 @@
SET_NEXT(cmd, hostdata->issue_queue);
hostdata->issue_queue = (struct scsi_cmnd *) cmd;
local_irq_restore(flags);
- QU_PRINTK("scsi%d: REQUEST SENSE added to head of "
+ dprintk(NDEBUG_QUEUES, "scsi%d: REQUEST SENSE added to head of "
"issue queue\n", H_NO(cmd));
} else
#endif /* def AUTOSENSE */
@@ -2238,7 +2228,7 @@
cmd->device->tagged_supported = 0;
hostdata->busy[cmd->device->id] |= (1 << cmd->device->lun);
cmd->tag = TAG_NONE;
- TAG_PRINTK("scsi%d: target %d lun %d rejected "
+ dprintk(NDEBUG_TAGS, "scsi%d: target %d lun %d rejected "
"QUEUE_TAG message; tagged queuing "
"disabled\n",
HOSTNO, cmd->device->id, cmd->device->lun);
@@ -2255,7 +2245,7 @@
hostdata->connected = NULL;
hostdata->disconnected_queue = cmd;
local_irq_restore(flags);
- QU_PRINTK("scsi%d: command for target %d lun %d was "
+ dprintk(NDEBUG_QUEUES, "scsi%d: command for target %d lun %d was "
"moved from connected to the "
"disconnected_queue\n", HOSTNO,
cmd->device->id, cmd->device->lun);
@@ -2308,13 +2298,13 @@
/* Accept first byte by clearing ACK */
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- EXT_PRINTK("scsi%d: receiving extended message\n", HOSTNO);
+ dprintk(NDEBUG_EXTENDED, "scsi%d: receiving extended message\n", HOSTNO);
len = 2;
data = extended_msg + 1;
phase = PHASE_MSGIN;
NCR5380_transfer_pio(instance, &phase, &len, &data);
- EXT_PRINTK("scsi%d: length=%d, code=0x%02x\n", HOSTNO,
+ dprintk(NDEBUG_EXTENDED, "scsi%d: length=%d, code=0x%02x\n", HOSTNO,
(int)extended_msg[1], (int)extended_msg[2]);
if (!len && extended_msg[1] <=
@@ -2326,7 +2316,7 @@
phase = PHASE_MSGIN;
NCR5380_transfer_pio(instance, &phase, &len, &data);
- EXT_PRINTK("scsi%d: message received, residual %d\n",
+ dprintk(NDEBUG_EXTENDED, "scsi%d: message received, residual %d\n",
HOSTNO, len);
switch (extended_msg[2]) {
@@ -2416,7 +2406,7 @@
break;
default:
printk("scsi%d: unknown phase\n", HOSTNO);
- NCR_PRINT(NDEBUG_ANY);
+ NCR5380_dprint(NDEBUG_ANY, instance);
} /* switch(phase) */
} /* if (tmp * SR_REQ) */
} /* while (1) */
@@ -2458,7 +2448,7 @@
target_mask = NCR5380_read(CURRENT_SCSI_DATA_REG) & ~(hostdata->id_mask);
- RSL_PRINTK("scsi%d: reselect\n", HOSTNO);
+ dprintk(NDEBUG_RESELECTION, "scsi%d: reselect\n", HOSTNO);
/*
* At this point, we have detected that our SCSI ID is on the bus,
@@ -2580,14 +2570,14 @@
if (!NCR5380_transfer_pio(instance, &phase, &len, &data) &&
msg[1] == SIMPLE_QUEUE_TAG)
tag = msg[2];
- TAG_PRINTK("scsi%d: target mask %02x, lun %d sent tag %d at "
+ dprintk(NDEBUG_TAGS, "scsi%d: target mask %02x, lun %d sent tag %d at "
"reselection\n", HOSTNO, target_mask, lun, tag);
}
#endif
hostdata->connected = tmp;
- RSL_PRINTK("scsi%d: nexus established, target = %d, lun = %d, tag = %d\n",
- HOSTNO, tmp->target, tmp->lun, tmp->tag);
+ dprintk(NDEBUG_RESELECTION, "scsi%d: nexus established, target = %d, lun = %d, tag = %d\n",
+ HOSTNO, tmp->device->id, tmp->device->lun, tmp->tag);
}
@@ -2622,7 +2612,7 @@
local_irq_save(flags);
- ABRT_PRINTK("scsi%d: abort called basr 0x%02x, sr 0x%02x\n", HOSTNO,
+ dprintk(NDEBUG_ABORT, "scsi%d: abort called basr 0x%02x, sr 0x%02x\n", HOSTNO,
NCR5380_read(BUS_AND_STATUS_REG),
NCR5380_read(STATUS_REG));
@@ -2635,7 +2625,7 @@
if (hostdata->connected == cmd) {
- ABRT_PRINTK("scsi%d: aborting connected command\n", HOSTNO);
+ dprintk(NDEBUG_ABORT, "scsi%d: aborting connected command\n", HOSTNO);
/*
* We should perform BSY checking, and make sure we haven't slipped
* into BUS FREE.
@@ -2664,11 +2654,11 @@
#endif
local_irq_restore(flags);
cmd->scsi_done(cmd);
- return SCSI_ABORT_SUCCESS;
+ return SUCCESS;
} else {
/* local_irq_restore(flags); */
printk("scsi%d: abort of connected command failed!\n", HOSTNO);
- return SCSI_ABORT_ERROR;
+ return FAILED;
}
}
#endif
@@ -2686,12 +2676,12 @@
SET_NEXT(tmp, NULL);
tmp->result = DID_ABORT << 16;
local_irq_restore(flags);
- ABRT_PRINTK("scsi%d: abort removed command from issue queue.\n",
+ dprintk(NDEBUG_ABORT, "scsi%d: abort removed command from issue queue.\n",
HOSTNO);
/* Tagged queuing note: no tag to free here, hasn't been assigned
* yet... */
tmp->scsi_done(tmp);
- return SCSI_ABORT_SUCCESS;
+ return SUCCESS;
}
/*
@@ -2707,8 +2697,8 @@
if (hostdata->connected) {
local_irq_restore(flags);
- ABRT_PRINTK("scsi%d: abort failed, command connected.\n", HOSTNO);
- return SCSI_ABORT_SNOOZE;
+ dprintk(NDEBUG_ABORT, "scsi%d: abort failed, command connected.\n", HOSTNO);
+ return FAILED;
}
/*
@@ -2740,12 +2730,12 @@
tmp = NEXT(tmp))
if (cmd == tmp) {
local_irq_restore(flags);
- ABRT_PRINTK("scsi%d: aborting disconnected command.\n", HOSTNO);
+ dprintk(NDEBUG_ABORT, "scsi%d: aborting disconnected command.\n", HOSTNO);
if (NCR5380_select (instance, cmd, (int) cmd->tag))
- return SCSI_ABORT_BUSY;
+ return FAILED;
- ABRT_PRINTK("scsi%d: nexus reestablished.\n", HOSTNO);
+ dprintk(NDEBUG_ABORT, "scsi%d: nexus reestablished.\n", HOSTNO);
do_abort (instance);
@@ -2769,7 +2759,7 @@
#endif
local_irq_restore(flags);
tmp->scsi_done(tmp);
- return SCSI_ABORT_SUCCESS;
+ return SUCCESS;
}
}
@@ -2786,7 +2776,7 @@
local_irq_restore(flags);
printk(KERN_INFO "scsi%d: warning : SCSI command probably completed successfully before abortion\n", HOSTNO);
- return SCSI_ABORT_NOT_RUNNING;
+ return FAILED;
}
@@ -2795,7 +2785,7 @@
*
* Purpose : reset the SCSI bus.
*
- * Returns : SCSI_RESET_WAKEUP
+ * Returns : SUCCESS or FAILURE
*
*/
@@ -2804,7 +2794,7 @@
SETUP_HOSTDATA(cmd->device->host);
int i;
unsigned long flags;
-#if 1
+#if defined(RESET_RUN_DONE)
struct scsi_cmnd *connected, *disconnected_queue;
#endif
@@ -2826,8 +2816,15 @@
* through anymore ... */
(void)NCR5380_read( RESET_PARITY_INTERRUPT_REG );
-#if 1 /* XXX Should now be done by midlevel code, but it's broken XXX */
- /* XXX see below XXX */
+ /* MSch 20140115 - looking at the generic NCR5380 driver, all of this
+ * should go.
+ * Catch-22: if we don't clear all queues, the SCSI driver lock will
+ * not be released by atari_scsi_reset()!
+ */
+
+#if defined(RESET_RUN_DONE)
+ /* XXX Should now be done by midlevel code, but it's broken XXX */
+ /* XXX see below XXX */
/* MSch: old-style reset: actually abort all command processing here */
@@ -2857,7 +2854,7 @@
*/
if ((cmd = connected)) {
- ABRT_PRINTK("scsi%d: reset aborted a connected command\n", H_NO(cmd));
+ dprintk(NDEBUG_ABORT, "scsi%d: reset aborted a connected command\n", H_NO(cmd));
cmd->result = (cmd->result & 0xffff) | (DID_RESET << 16);
cmd->scsi_done( cmd );
}
@@ -2869,14 +2866,14 @@
cmd->scsi_done( cmd );
}
if (i > 0)
- ABRT_PRINTK("scsi: reset aborted %d disconnected command(s)\n", i);
+ dprintk(NDEBUG_ABORT, "scsi: reset aborted %d disconnected command(s)\n", i);
/* since all commands have been explicitly terminated, we need to tell
* the midlevel code that the reset was SUCCESSFUL, and there is no
* need to 'wake up' the commands by a request_sense
*/
- return SCSI_RESET_SUCCESS | SCSI_RESET_BUS_RESET;
+ return SUCCESS;
#else /* 1 */
/* MSch: new-style reset handling: let the mid-level do what it can */
@@ -2903,11 +2900,11 @@
*/
if (hostdata->issue_queue)
- ABRT_PRINTK("scsi%d: reset aborted issued command(s)\n", H_NO(cmd));
+ dprintk(NDEBUG_ABORT, "scsi%d: reset aborted issued command(s)\n", H_NO(cmd));
if (hostdata->connected)
- ABRT_PRINTK("scsi%d: reset aborted a connected command\n", H_NO(cmd));
+ dprintk(NDEBUG_ABORT, "scsi%d: reset aborted a connected command\n", H_NO(cmd));
if (hostdata->disconnected_queue)
- ABRT_PRINTK("scsi%d: reset aborted disconnected command(s)\n", H_NO(cmd));
+ dprintk(NDEBUG_ABORT, "scsi%d: reset aborted disconnected command(s)\n", H_NO(cmd));
local_irq_save(flags);
hostdata->issue_queue = NULL;
@@ -2924,7 +2921,7 @@
local_irq_restore(flags);
/* we did no complete reset of all commands, so a wakeup is required */
- return SCSI_RESET_WAKEUP | SCSI_RESET_BUS_RESET;
+ return SUCCESS;
#endif /* 1 */
}
diff --git a/drivers/scsi/sun3_scsi.c b/drivers/scsi/sun3_scsi.c
index e2c009b..9707b74 100644
--- a/drivers/scsi/sun3_scsi.c
+++ b/drivers/scsi/sun3_scsi.c
@@ -3,6 +3,10 @@
*
* Sun3 DMA routines added by Sam Creasey (sammy@sammy.net)
*
+ * VME support added by Sam Creasey
+ *
+ * TODO: modify this driver to support multiple Sun3 SCSI VME boards
+ *
* Adapted from mac_scsinew.c:
*/
/*
@@ -45,10 +49,6 @@
* USLEEP - enable support for devices that don't disconnect. Untested.
*/
-/*
- * $Log: sun3_NCR5380.c,v $
- */
-
#define AUTOSENSE
#include <linux/types.h>
@@ -69,23 +69,15 @@
#include <asm/idprom.h>
#include <asm/machines.h>
-#define NDEBUG 0
-
-#define NDEBUG_ABORT 0x00100000
-#define NDEBUG_TAGS 0x00200000
-#define NDEBUG_MERGING 0x00400000
-
/* dma on! */
#define REAL_DMA
#include "scsi.h"
-#include "initio.h"
#include <scsi/scsi_host.h>
#include "sun3_scsi.h"
+#include "NCR5380.h"
-static void NCR5380_print(struct Scsi_Host *instance);
-
-/* #define OLDDMA */
+extern int sun3_map_test(unsigned long, char *);
#define USE_WRAPPER
/*#define RESET_BOOT */
@@ -101,7 +93,11 @@
/* #define SUPPORT_TAGS */
+#ifdef SUN3_SCSI_VME
+#define ENABLE_IRQ()
+#else
#define ENABLE_IRQ() enable_irq( IRQ_SUN3_SCSI );
+#endif
static irqreturn_t scsi_sun3_intr(int irq, void *dummy);
@@ -123,6 +119,8 @@
static struct scsi_cmnd *sun3_dma_setup_done = NULL;
+#define RESET_RUN_DONE
+
#define AFTER_RESET_DELAY (HZ/2)
/* ms to wait after hitting dma regs */
@@ -136,10 +134,9 @@
static volatile unsigned char *sun3_scsi_regp;
static volatile struct sun3_dma_regs *dregs;
-#ifdef OLDDMA
-static unsigned char *dmabuf = NULL; /* dma memory buffer */
-#endif
+#ifndef SUN3_SCSI_VME
static struct sun3_udc_regs *udc_regs = NULL;
+#endif
static unsigned char *sun3_dma_orig_addr = NULL;
static unsigned long sun3_dma_orig_count = 0;
static int sun3_dma_active = 0;
@@ -159,6 +156,7 @@
sun3_scsi_regp[reg] = value;
}
+#ifndef SUN3_SCSI_VME
/* dma controller register access functions */
static inline unsigned short sun3_udc_read(unsigned char reg)
@@ -180,6 +178,7 @@
dregs->udc_data = val;
udelay(SUN3_DMA_DELAY);
}
+#endif
/*
* XXX: status debug
@@ -198,17 +197,32 @@
*
*/
-int __init sun3scsi_detect(struct scsi_host_template * tpnt)
+static int __init sun3scsi_detect(struct scsi_host_template *tpnt)
{
- unsigned long ioaddr;
+ unsigned long ioaddr, irq;
static int called = 0;
struct Scsi_Host *instance;
+#ifdef SUN3_SCSI_VME
+ int i;
+ unsigned long addrs[3] = { IOBASE_SUN3_VMESCSI,
+ IOBASE_SUN3_VMESCSI + 0x4000,
+ 0 };
+ unsigned long vecs[3] = { SUN3_VEC_VMESCSI0,
+ SUN3_VEC_VMESCSI1,
+ 0 };
+#endif
/* check that this machine has an onboard 5380 */
switch(idprom->id_machtype) {
+#ifdef SUN3_SCSI_VME
+ case SM_SUN3|SM_3_160:
+ case SM_SUN3|SM_3_260:
+ break;
+#else
case SM_SUN3|SM_3_50:
case SM_SUN3|SM_3_60:
break;
+#endif
default:
return 0;
@@ -217,7 +231,11 @@
if(called)
return 0;
+#ifdef SUN3_SCSI_VME
+ tpnt->proc_name = "Sun3 5380 VME SCSI";
+#else
tpnt->proc_name = "Sun3 5380 SCSI";
+#endif
/* setup variables */
tpnt->can_queue =
@@ -234,6 +252,38 @@
tpnt->this_id = 7;
}
+#ifdef SUN3_SCSI_VME
+ ioaddr = 0;
+ for (i = 0; addrs[i] != 0; i++) {
+ unsigned char x;
+
+ ioaddr = (unsigned long)sun3_ioremap(addrs[i], PAGE_SIZE,
+ SUN3_PAGE_TYPE_VME16);
+ irq = vecs[i];
+ sun3_scsi_regp = (unsigned char *)ioaddr;
+
+ dregs = (struct sun3_dma_regs *)(((unsigned char *)ioaddr) + 8);
+
+ if (sun3_map_test((unsigned long)dregs, &x)) {
+ unsigned short oldcsr;
+
+ oldcsr = dregs->csr;
+ dregs->csr = 0;
+ udelay(SUN3_DMA_DELAY);
+ if (dregs->csr == 0x1400)
+ break;
+
+ dregs->csr = oldcsr;
+ }
+
+ iounmap((void *)ioaddr);
+ ioaddr = 0;
+ }
+
+ if (!ioaddr)
+ return 0;
+#else
+ irq = IRQ_SUN3_SCSI;
ioaddr = (unsigned long)ioremap(IOBASE_SUN3_SCSI, PAGE_SIZE);
sun3_scsi_regp = (unsigned char *)ioaddr;
@@ -244,11 +294,6 @@
printk("SUN3 Scsi couldn't allocate DVMA memory!\n");
return 0;
}
-#ifdef OLDDMA
- if((dmabuf = dvma_malloc_align(SUN3_DVMA_BUFSIZE, 0x10000)) == NULL) {
- printk("SUN3 Scsi couldn't allocate DVMA memory!\n");
- return 0;
- }
#endif
#ifdef SUPPORT_TAGS
if (setup_use_tagged_queuing < 0)
@@ -262,7 +307,7 @@
default_instance = instance;
instance->io_port = (unsigned long) ioaddr;
- instance->irq = IRQ_SUN3_SCSI;
+ instance->irq = irq;
NCR5380_init(instance, 0);
@@ -283,7 +328,8 @@
#endif
}
- printk("scsi%d: Sun3 5380 at port %lX irq", instance->host_no, instance->io_port);
+ pr_info("scsi%d: %s at port %lX irq", instance->host_no,
+ tpnt->proc_name, instance->io_port);
if (instance->irq == SCSI_IRQ_NONE)
printk ("s disabled");
else
@@ -300,6 +346,15 @@
dregs->csr = CSR_SCSI | CSR_FIFO | CSR_INTR;
udelay(SUN3_DMA_DELAY);
dregs->fifo_count = 0;
+#ifdef SUN3_SCSI_VME
+ dregs->fifo_count_hi = 0;
+ dregs->dma_addr_hi = 0;
+ dregs->dma_addr_lo = 0;
+ dregs->dma_count_hi = 0;
+ dregs->dma_count_lo = 0;
+
+ dregs->ivect = VME_DATA24 | (instance->irq & 0xff);
+#endif
called = 1;
@@ -367,7 +422,8 @@
}
#endif
-const char * sun3scsi_info (struct Scsi_Host *spnt) {
+static const char *sun3scsi_info(struct Scsi_Host *spnt)
+{
return "";
}
@@ -379,6 +435,10 @@
unsigned short csr = dregs->csr;
int handled = 0;
+#ifdef SUN3_SCSI_VME
+ dregs->csr &= ~CSR_DMA_ENABLE;
+#endif
+
if(csr & ~CSR_GOOD) {
if(csr & CSR_DMA_BUSERR) {
printk("scsi%d: bus error in dma\n", default_instance->host_no);
@@ -422,31 +482,28 @@
/* sun3scsi_dma_setup() -- initialize the dma controller for a read/write */
static unsigned long sun3scsi_dma_setup(void *data, unsigned long count, int write_flag)
{
-#ifdef OLDDMA
- if(write_flag)
- memcpy(dmabuf, data, count);
- else {
- sun3_dma_orig_addr = data;
- sun3_dma_orig_count = count;
- }
-#else
void *addr;
if(sun3_dma_orig_addr != NULL)
dvma_unmap(sun3_dma_orig_addr);
-// addr = sun3_dvma_page((unsigned long)data, (unsigned long)dmabuf);
+#ifdef SUN3_SCSI_VME
+ addr = (void *)dvma_map_vme((unsigned long) data, count);
+#else
addr = (void *)dvma_map((unsigned long) data, count);
+#endif
sun3_dma_orig_addr = addr;
sun3_dma_orig_count = count;
-#endif
+
+#ifndef SUN3_SCSI_VME
dregs->fifo_count = 0;
sun3_udc_write(UDC_RESET, UDC_CSR);
/* reset fifo */
dregs->csr &= ~CSR_FIFO;
dregs->csr |= CSR_FIFO;
+#endif
/* set direction */
if(write_flag)
@@ -454,6 +511,17 @@
else
dregs->csr &= ~CSR_SEND;
+#ifdef SUN3_SCSI_VME
+ dregs->csr |= CSR_PACK_ENABLE;
+
+ dregs->dma_addr_hi = ((unsigned long)addr >> 16);
+ dregs->dma_addr_lo = ((unsigned long)addr & 0xffff);
+
+ dregs->dma_count_hi = 0;
+ dregs->dma_count_lo = 0;
+ dregs->fifo_count_hi = 0;
+ dregs->fifo_count = 0;
+#else
/* byte count for fifo */
dregs->fifo_count = count;
@@ -467,17 +535,12 @@
printk("scsi%d: fifo_mismatch %04x not %04x\n",
default_instance->host_no, dregs->fifo_count,
(unsigned int) count);
- NCR5380_print(default_instance);
+ NCR5380_dprint(NDEBUG_DMA, default_instance);
}
/* setup udc */
-#ifdef OLDDMA
- udc_regs->addr_hi = ((dvma_vtob(dmabuf) & 0xff0000) >> 8);
- udc_regs->addr_lo = (dvma_vtob(dmabuf) & 0xffff);
-#else
udc_regs->addr_hi = (((unsigned long)(addr) & 0xff0000) >> 8);
udc_regs->addr_lo = ((unsigned long)(addr) & 0xffff);
-#endif
udc_regs->count = count/2; /* count in words */
udc_regs->mode_hi = UDC_MODE_HIWORD;
if(write_flag) {
@@ -501,11 +564,13 @@
/* interrupt enable */
sun3_udc_write(UDC_INT_ENABLE, UDC_CSR);
+#endif
return count;
}
+#ifndef SUN3_SCSI_VME
static inline unsigned long sun3scsi_dma_count(struct Scsi_Host *instance)
{
unsigned short resid;
@@ -518,6 +583,7 @@
return (unsigned long) resid;
}
+#endif
static inline unsigned long sun3scsi_dma_residual(struct Scsi_Host *instance)
{
@@ -536,8 +602,23 @@
static inline int sun3scsi_dma_start(unsigned long count, unsigned char *data)
{
+#ifdef SUN3_SCSI_VME
+ unsigned short csr;
+ csr = dregs->csr;
+
+ dregs->dma_count_hi = (sun3_dma_orig_count >> 16);
+ dregs->dma_count_lo = (sun3_dma_orig_count & 0xffff);
+
+ dregs->fifo_count_hi = (sun3_dma_orig_count >> 16);
+ dregs->fifo_count = (sun3_dma_orig_count & 0xffff);
+
+/* if(!(csr & CSR_DMA_ENABLE))
+ * dregs->csr |= CSR_DMA_ENABLE;
+ */
+#else
sun3_udc_write(UDC_CHN_START, UDC_CSR);
+#endif
return 0;
}
@@ -545,12 +626,46 @@
/* clean up after our dma is done */
static int sun3scsi_dma_finish(int write_flag)
{
- unsigned short count;
+ unsigned short __maybe_unused count;
unsigned short fifo;
int ret = 0;
sun3_dma_active = 0;
-#if 1
+
+#ifdef SUN3_SCSI_VME
+ dregs->csr &= ~CSR_DMA_ENABLE;
+
+ fifo = dregs->fifo_count;
+ if (write_flag) {
+ if ((fifo > 0) && (fifo < sun3_dma_orig_count))
+ fifo++;
+ }
+
+ last_residual = fifo;
+ /* empty bytes from the fifo which didn't make it */
+ if ((!write_flag) && (dregs->csr & CSR_LEFT)) {
+ unsigned char *vaddr;
+
+ vaddr = (unsigned char *)dvma_vmetov(sun3_dma_orig_addr);
+
+ vaddr += (sun3_dma_orig_count - fifo);
+ vaddr--;
+
+ switch (dregs->csr & CSR_LEFT) {
+ case CSR_LEFT_3:
+ *vaddr = (dregs->bpack_lo & 0xff00) >> 8;
+ vaddr--;
+
+ case CSR_LEFT_2:
+ *vaddr = (dregs->bpack_hi & 0x00ff);
+ vaddr--;
+
+ case CSR_LEFT_1:
+ *vaddr = (dregs->bpack_hi & 0xff00) >> 8;
+ break;
+ }
+ }
+#else
// check to empty the fifo on a read
if(!write_flag) {
int tmo = 20000; /* .2 sec */
@@ -566,28 +681,8 @@
udelay(10);
}
}
-
-#endif
count = sun3scsi_dma_count(default_instance);
-#ifdef OLDDMA
-
- /* if we've finished a read, copy out the data we read */
- if(sun3_dma_orig_addr) {
- /* check for residual bytes after dma end */
- if(count && (NCR5380_read(BUS_AND_STATUS_REG) &
- (BASR_PHASE_MATCH | BASR_ACK))) {
- printk("scsi%d: sun3_scsi_finish: read overrun baby... ", default_instance->host_no);
- printk("basr now %02x\n", NCR5380_read(BUS_AND_STATUS_REG));
- ret = count;
- }
-
- /* copy in what we dma'd no matter what */
- memcpy(sun3_dma_orig_addr, dmabuf, sun3_dma_orig_count);
- sun3_dma_orig_addr = NULL;
-
- }
-#else
fifo = dregs->fifo_count;
last_residual = fifo;
@@ -605,10 +700,23 @@
vaddr[-2] = (data & 0xff00) >> 8;
vaddr[-1] = (data & 0xff);
}
+#endif
dvma_unmap(sun3_dma_orig_addr);
sun3_dma_orig_addr = NULL;
-#endif
+
+#ifdef SUN3_SCSI_VME
+ dregs->dma_addr_hi = 0;
+ dregs->dma_addr_lo = 0;
+ dregs->dma_count_hi = 0;
+ dregs->dma_count_lo = 0;
+
+ dregs->fifo_count = 0;
+ dregs->fifo_count_hi = 0;
+
+ dregs->csr &= ~CSR_SEND;
+/* dregs->csr |= CSR_DMA_ENABLE; */
+#else
sun3_udc_write(UDC_RESET, UDC_CSR);
dregs->fifo_count = 0;
dregs->csr &= ~CSR_SEND;
@@ -616,6 +724,7 @@
/* reset fifo */
dregs->csr &= ~CSR_FIFO;
dregs->csr |= CSR_FIFO;
+#endif
sun3_dma_setup_done = NULL;
diff --git a/drivers/scsi/sun3_scsi.h b/drivers/scsi/sun3_scsi.h
index a8da9c7..e96a37c 100644
--- a/drivers/scsi/sun3_scsi.h
+++ b/drivers/scsi/sun3_scsi.h
@@ -29,12 +29,8 @@
* 1+ (800) 334-5454
*/
-/*
- * $Log: cumana_NCR5380.h,v $
- */
-
-#ifndef SUN3_NCR5380_H
-#define SUN3_NCR5380_H
+#ifndef SUN3_SCSI_H
+#define SUN3_SCSI_H
#define SUN3SCSI_PUBLIC_RELEASE 1
@@ -82,8 +78,6 @@
#define SUN3_SCSI_NAME "Sun3 NCR5380 SCSI"
#endif
-#ifndef HOSTS_C
-
#define NCR5380_implementation_fields \
int port, ctrl
@@ -108,9 +102,6 @@
#define NCR5380_dma_read_setup(instance, data, count) sun3scsi_dma_setup(data, count, 0)
#define NCR5380_dma_residual sun3scsi_dma_residual
-#define BOARD_NORMAL 0
-#define BOARD_NCR53C400 1
-
/* additional registers - mainly DMA control regs */
/* these start at regbase + 8 -- directly after the NCR regs */
struct sun3_dma_regs {
@@ -191,189 +182,5 @@
#define VME_DATA24 0x3d00
-// debugging printk's, taken from atari_scsi.h
-/* Debugging printk definitions:
- *
- * ARB -> arbitration
- * ASEN -> auto-sense
- * DMA -> DMA
- * HSH -> PIO handshake
- * INF -> information transfer
- * INI -> initialization
- * INT -> interrupt
- * LNK -> linked commands
- * MAIN -> NCR5380_main() control flow
- * NDAT -> no data-out phase
- * NWR -> no write commands
- * PIO -> PIO transfers
- * PDMA -> pseudo DMA (unused on Atari)
- * QU -> queues
- * RSL -> reselections
- * SEL -> selections
- * USL -> usleep cpde (unused on Atari)
- * LBS -> last byte sent (unused on Atari)
- * RSS -> restarting of selections
- * EXT -> extended messages
- * ABRT -> aborting and resetting
- * TAG -> queue tag handling
- * MER -> merging of consec. buffers
- *
- */
-
-#include "NCR5380.h"
-
-#if NDEBUG & NDEBUG_ARBITRATION
-#define ARB_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define ARB_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_AUTOSENSE
-#define ASEN_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define ASEN_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_DMA
-#define DMA_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define DMA_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_HANDSHAKE
-#define HSH_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define HSH_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_INFORMATION
-#define INF_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define INF_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_INIT
-#define INI_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define INI_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_INTR
-#define INT_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define INT_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_LINKED
-#define LNK_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define LNK_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_MAIN
-#define MAIN_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define MAIN_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_NO_DATAOUT
-#define NDAT_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define NDAT_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_NO_WRITE
-#define NWR_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define NWR_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_PIO
-#define PIO_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define PIO_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_PSEUDO_DMA
-#define PDMA_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define PDMA_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_QUEUES
-#define QU_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define QU_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_RESELECTION
-#define RSL_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define RSL_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_SELECTION
-#define SEL_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define SEL_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_USLEEP
-#define USL_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define USL_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_LAST_BYTE_SENT
-#define LBS_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define LBS_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_RESTART_SELECT
-#define RSS_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define RSS_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_EXTENDED
-#define EXT_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define EXT_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_ABORT
-#define ABRT_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define ABRT_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_TAGS
-#define TAG_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define TAG_PRINTK(format, args...)
-#endif
-#if NDEBUG & NDEBUG_MERGING
-#define MER_PRINTK(format, args...) \
- printk(KERN_DEBUG format , ## args)
-#else
-#define MER_PRINTK(format, args...)
-#endif
-
-/* conditional macros for NCR5380_print_{,phase,status} */
-
-#define NCR_PRINT(mask) \
- ((NDEBUG & (mask)) ? NCR5380_print(instance) : (void)0)
-
-#define NCR_PRINT_PHASE(mask) \
- ((NDEBUG & (mask)) ? NCR5380_print_phase(instance) : (void)0)
-
-#define NCR_PRINT_STATUS(mask) \
- ((NDEBUG & (mask)) ? NCR5380_print_status(instance) : (void)0)
-
-
-
-#endif /* ndef HOSTS_C */
-#endif /* SUN3_NCR5380_H */
+#endif /* SUN3_SCSI_H */
diff --git a/drivers/scsi/sun3_scsi_vme.c b/drivers/scsi/sun3_scsi_vme.c
index a3dd55d..1eeece6 100644
--- a/drivers/scsi/sun3_scsi_vme.c
+++ b/drivers/scsi/sun3_scsi_vme.c
@@ -1,589 +1,3 @@
- /*
- * Sun3 SCSI stuff by Erik Verbruggen (erik@bigmama.xtdnet.nl)
- *
- * Sun3 DMA routines added by Sam Creasey (sammy@sammy.net)
- *
- * VME support added by Sam Creasey
- *
- * Adapted from sun3_scsi.c -- see there for other headers
- *
- * TODO: modify this driver to support multiple Sun3 SCSI VME boards
- *
- */
-
-#define AUTOSENSE
-
-#include <linux/types.h>
-#include <linux/stddef.h>
-#include <linux/ctype.h>
-#include <linux/delay.h>
-
-#include <linux/module.h>
-#include <linux/signal.h>
-#include <linux/ioport.h>
-#include <linux/init.h>
-#include <linux/blkdev.h>
-
-#include <asm/io.h>
-
-#include <asm/sun3ints.h>
-#include <asm/dvma.h>
-#include <asm/idprom.h>
-#include <asm/machines.h>
-
#define SUN3_SCSI_VME
-#undef SUN3_SCSI_DEBUG
-
-/* dma on! */
-#define REAL_DMA
-
-#define NDEBUG 0
-
-#define NDEBUG_ABORT 0x00100000
-#define NDEBUG_TAGS 0x00200000
-#define NDEBUG_MERGING 0x00400000
-
-#include "scsi.h"
-#include "initio.h"
-#include <scsi/scsi_host.h>
-#include "sun3_scsi.h"
-
-extern int sun3_map_test(unsigned long, char *);
-
-#define USE_WRAPPER
-/*#define RESET_BOOT */
-#define DRIVER_SETUP
-
-/*
- * BUG can be used to trigger a strange code-size related hang on 2.1 kernels
- */
-#ifdef BUG
-#undef RESET_BOOT
-#undef DRIVER_SETUP
-#endif
-
-/* #define SUPPORT_TAGS */
-
-//#define ENABLE_IRQ() enable_irq( SUN3_VEC_VMESCSI0 );
-#define ENABLE_IRQ()
-
-
-static irqreturn_t scsi_sun3_intr(int irq, void *dummy);
-static inline unsigned char sun3scsi_read(int reg);
-static inline void sun3scsi_write(int reg, int value);
-
-static int setup_can_queue = -1;
-module_param(setup_can_queue, int, 0);
-static int setup_cmd_per_lun = -1;
-module_param(setup_cmd_per_lun, int, 0);
-static int setup_sg_tablesize = -1;
-module_param(setup_sg_tablesize, int, 0);
-#ifdef SUPPORT_TAGS
-static int setup_use_tagged_queuing = -1;
-module_param(setup_use_tagged_queuing, int, 0);
-#endif
-static int setup_hostid = -1;
-module_param(setup_hostid, int, 0);
-
-static struct scsi_cmnd *sun3_dma_setup_done = NULL;
-
-#define AFTER_RESET_DELAY (HZ/2)
-
-/* ms to wait after hitting dma regs */
-#define SUN3_DMA_DELAY 10
-
-/* dvma buffer to allocate -- 32k should hopefully be more than sufficient */
-#define SUN3_DVMA_BUFSIZE 0xe000
-
-/* minimum number of bytes to do dma on */
-#define SUN3_DMA_MINSIZE 128
-
-static volatile unsigned char *sun3_scsi_regp;
-static volatile struct sun3_dma_regs *dregs;
-#ifdef OLDDMA
-static unsigned char *dmabuf = NULL; /* dma memory buffer */
-#endif
-static unsigned char *sun3_dma_orig_addr = NULL;
-static unsigned long sun3_dma_orig_count = 0;
-static int sun3_dma_active = 0;
-static unsigned long last_residual = 0;
-
-/*
- * NCR 5380 register access functions
- */
-
-static inline unsigned char sun3scsi_read(int reg)
-{
- return( sun3_scsi_regp[reg] );
-}
-
-static inline void sun3scsi_write(int reg, int value)
-{
- sun3_scsi_regp[reg] = value;
-}
-
-/*
- * XXX: status debug
- */
-static struct Scsi_Host *default_instance;
-
-/*
- * Function : int sun3scsi_detect(struct scsi_host_template * tpnt)
- *
- * Purpose : initializes mac NCR5380 driver based on the
- * command line / compile time port and irq definitions.
- *
- * Inputs : tpnt - template for this SCSI adapter.
- *
- * Returns : 1 if a host adapter was found, 0 if not.
- *
- */
-
-static int __init sun3scsi_detect(struct scsi_host_template * tpnt)
-{
- unsigned long ioaddr, irq = 0;
- static int called = 0;
- struct Scsi_Host *instance;
- int i;
- unsigned long addrs[3] = { IOBASE_SUN3_VMESCSI,
- IOBASE_SUN3_VMESCSI + 0x4000,
- 0 };
- unsigned long vecs[3] = { SUN3_VEC_VMESCSI0,
- SUN3_VEC_VMESCSI1,
- 0 };
- /* check that this machine has an onboard 5380 */
- switch(idprom->id_machtype) {
- case SM_SUN3|SM_3_160:
- case SM_SUN3|SM_3_260:
- break;
-
- default:
- return 0;
- }
-
- if(called)
- return 0;
-
- tpnt->proc_name = "Sun3 5380 VME SCSI";
-
- /* setup variables */
- tpnt->can_queue =
- (setup_can_queue > 0) ? setup_can_queue : CAN_QUEUE;
- tpnt->cmd_per_lun =
- (setup_cmd_per_lun > 0) ? setup_cmd_per_lun : CMD_PER_LUN;
- tpnt->sg_tablesize =
- (setup_sg_tablesize >= 0) ? setup_sg_tablesize : SG_TABLESIZE;
-
- if (setup_hostid >= 0)
- tpnt->this_id = setup_hostid;
- else {
- /* use 7 as default */
- tpnt->this_id = 7;
- }
-
- ioaddr = 0;
- for(i = 0; addrs[i] != 0; i++) {
- unsigned char x;
-
- ioaddr = (unsigned long)sun3_ioremap(addrs[i], PAGE_SIZE,
- SUN3_PAGE_TYPE_VME16);
- irq = vecs[i];
- sun3_scsi_regp = (unsigned char *)ioaddr;
-
- dregs = (struct sun3_dma_regs *)(((unsigned char *)ioaddr) + 8);
-
- if(sun3_map_test((unsigned long)dregs, &x)) {
- unsigned short oldcsr;
-
- oldcsr = dregs->csr;
- dregs->csr = 0;
- udelay(SUN3_DMA_DELAY);
- if(dregs->csr == 0x1400)
- break;
-
- dregs->csr = oldcsr;
- }
-
- iounmap((void *)ioaddr);
- ioaddr = 0;
- }
-
- if(!ioaddr)
- return 0;
-
-#ifdef SUPPORT_TAGS
- if (setup_use_tagged_queuing < 0)
- setup_use_tagged_queuing = USE_TAGGED_QUEUING;
-#endif
-
- instance = scsi_register (tpnt, sizeof(struct NCR5380_hostdata));
- if(instance == NULL)
- return 0;
-
- default_instance = instance;
-
- instance->io_port = (unsigned long) ioaddr;
- instance->irq = irq;
-
- NCR5380_init(instance, 0);
-
- instance->n_io_port = 32;
-
- ((struct NCR5380_hostdata *)instance->hostdata)->ctrl = 0;
-
- if (request_irq(instance->irq, scsi_sun3_intr,
- 0, "Sun3SCSI-5380VME", instance)) {
-#ifndef REAL_DMA
- printk("scsi%d: IRQ%d not free, interrupts disabled\n",
- instance->host_no, instance->irq);
- instance->irq = SCSI_IRQ_NONE;
-#else
- printk("scsi%d: IRQ%d not free, bailing out\n",
- instance->host_no, instance->irq);
- return 0;
-#endif
- }
-
- printk("scsi%d: Sun3 5380 VME at port %lX irq", instance->host_no, instance->io_port);
- if (instance->irq == SCSI_IRQ_NONE)
- printk ("s disabled");
- else
- printk (" %d", instance->irq);
- printk(" options CAN_QUEUE=%d CMD_PER_LUN=%d release=%d",
- instance->can_queue, instance->cmd_per_lun,
- SUN3SCSI_PUBLIC_RELEASE);
- printk("\nscsi%d:", instance->host_no);
- NCR5380_print_options(instance);
- printk("\n");
-
- dregs->csr = 0;
- udelay(SUN3_DMA_DELAY);
- dregs->csr = CSR_SCSI | CSR_FIFO | CSR_INTR;
- udelay(SUN3_DMA_DELAY);
- dregs->fifo_count = 0;
- dregs->fifo_count_hi = 0;
- dregs->dma_addr_hi = 0;
- dregs->dma_addr_lo = 0;
- dregs->dma_count_hi = 0;
- dregs->dma_count_lo = 0;
-
- dregs->ivect = VME_DATA24 | (instance->irq & 0xff);
-
- called = 1;
-
-#ifdef RESET_BOOT
- sun3_scsi_reset_boot(instance);
-#endif
-
- return 1;
-}
-
-int sun3scsi_release (struct Scsi_Host *shpnt)
-{
- if (shpnt->irq != SCSI_IRQ_NONE)
- free_irq(shpnt->irq, shpnt);
-
- iounmap((void *)sun3_scsi_regp);
-
- NCR5380_exit(shpnt);
- return 0;
-}
-
-#ifdef RESET_BOOT
-/*
- * Our 'bus reset on boot' function
- */
-
-static void sun3_scsi_reset_boot(struct Scsi_Host *instance)
-{
- unsigned long end;
-
- NCR5380_local_declare();
- NCR5380_setup(instance);
-
- /*
- * Do a SCSI reset to clean up the bus during initialization. No
- * messing with the queues, interrupts, or locks necessary here.
- */
-
- printk( "Sun3 SCSI: resetting the SCSI bus..." );
-
- /* switch off SCSI IRQ - catch an interrupt without IRQ bit set else */
-// sun3_disable_irq( IRQ_SUN3_SCSI );
-
- /* get in phase */
- NCR5380_write( TARGET_COMMAND_REG,
- PHASE_SR_TO_TCR( NCR5380_read(STATUS_REG) ));
-
- /* assert RST */
- NCR5380_write( INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_RST );
-
- /* The min. reset hold time is 25us, so 40us should be enough */
- udelay( 50 );
-
- /* reset RST and interrupt */
- NCR5380_write( INITIATOR_COMMAND_REG, ICR_BASE );
- NCR5380_read( RESET_PARITY_INTERRUPT_REG );
-
- for( end = jiffies + AFTER_RESET_DELAY; time_before(jiffies, end); )
- barrier();
-
- /* switch on SCSI IRQ again */
-// sun3_enable_irq( IRQ_SUN3_SCSI );
-
- printk( " done\n" );
-}
-#endif
-
-static const char * sun3scsi_info (struct Scsi_Host *spnt) {
- return "";
-}
-
-// safe bits for the CSR
-#define CSR_GOOD 0x060f
-
-static irqreturn_t scsi_sun3_intr(int irq, void *dummy)
-{
- unsigned short csr = dregs->csr;
- int handled = 0;
-
- dregs->csr &= ~CSR_DMA_ENABLE;
-
-
-#ifdef SUN3_SCSI_DEBUG
- printk("scsi_intr csr %x\n", csr);
-#endif
-
- if(csr & ~CSR_GOOD) {
- if(csr & CSR_DMA_BUSERR) {
- printk("scsi%d: bus error in dma\n", default_instance->host_no);
-#ifdef SUN3_SCSI_DEBUG
- printk("scsi: residual %x count %x addr %p dmaaddr %x\n",
- dregs->fifo_count,
- dregs->dma_count_lo | (dregs->dma_count_hi << 16),
- sun3_dma_orig_addr,
- dregs->dma_addr_lo | (dregs->dma_addr_hi << 16));
-#endif
- }
-
- if(csr & CSR_DMA_CONFLICT) {
- printk("scsi%d: dma conflict\n", default_instance->host_no);
- }
- handled = 1;
- }
-
- if(csr & (CSR_SDB_INT | CSR_DMA_INT)) {
- NCR5380_intr(irq, dummy);
- handled = 1;
- }
-
- return IRQ_RETVAL(handled);
-}
-
-/*
- * Debug stuff - to be called on NMI, or sysrq key. Use at your own risk;
- * reentering NCR5380_print_status seems to have ugly side effects
- */
-
-/* this doesn't seem to get used at all -- sam */
-#if 0
-void sun3_sun3_debug (void)
-{
- unsigned long flags;
- NCR5380_local_declare();
-
- if (default_instance) {
- local_irq_save(flags);
- NCR5380_print_status(default_instance);
- local_irq_restore(flags);
- }
-}
-#endif
-
-
-/* sun3scsi_dma_setup() -- initialize the dma controller for a read/write */
-static unsigned long sun3scsi_dma_setup(void *data, unsigned long count, int write_flag)
-{
- void *addr;
-
- if(sun3_dma_orig_addr != NULL)
- dvma_unmap(sun3_dma_orig_addr);
-
-// addr = sun3_dvma_page((unsigned long)data, (unsigned long)dmabuf);
- addr = (void *)dvma_map_vme((unsigned long) data, count);
-
- sun3_dma_orig_addr = addr;
- sun3_dma_orig_count = count;
-
-#ifdef SUN3_SCSI_DEBUG
- printk("scsi: dma_setup addr %p count %x\n", addr, count);
-#endif
-
-// dregs->fifo_count = 0;
-#if 0
- /* reset fifo */
- dregs->csr &= ~CSR_FIFO;
- dregs->csr |= CSR_FIFO;
-#endif
- /* set direction */
- if(write_flag)
- dregs->csr |= CSR_SEND;
- else
- dregs->csr &= ~CSR_SEND;
-
- /* reset fifo */
-// dregs->csr &= ~CSR_FIFO;
-// dregs->csr |= CSR_FIFO;
-
- dregs->csr |= CSR_PACK_ENABLE;
-
- dregs->dma_addr_hi = ((unsigned long)addr >> 16);
- dregs->dma_addr_lo = ((unsigned long)addr & 0xffff);
-
- dregs->dma_count_hi = 0;
- dregs->dma_count_lo = 0;
- dregs->fifo_count_hi = 0;
- dregs->fifo_count = 0;
-
-#ifdef SUN3_SCSI_DEBUG
- printk("scsi: dma_setup done csr %x\n", dregs->csr);
-#endif
- return count;
-
-}
-
-static inline unsigned long sun3scsi_dma_residual(struct Scsi_Host *instance)
-{
- return last_residual;
-}
-
-static inline unsigned long sun3scsi_dma_xfer_len(unsigned long wanted,
- struct scsi_cmnd *cmd,
- int write_flag)
-{
- if (cmd->request->cmd_type == REQ_TYPE_FS)
- return wanted;
- else
- return 0;
-}
-
-static int sun3scsi_dma_start(unsigned long count, char *data)
-{
-
- unsigned short csr;
-
- csr = dregs->csr;
-#ifdef SUN3_SCSI_DEBUG
- printk("scsi: dma_start data %p count %x csr %x fifo %x\n", data, count, csr, dregs->fifo_count);
-#endif
-
- dregs->dma_count_hi = (sun3_dma_orig_count >> 16);
- dregs->dma_count_lo = (sun3_dma_orig_count & 0xffff);
-
- dregs->fifo_count_hi = (sun3_dma_orig_count >> 16);
- dregs->fifo_count = (sun3_dma_orig_count & 0xffff);
-
-// if(!(csr & CSR_DMA_ENABLE))
-// dregs->csr |= CSR_DMA_ENABLE;
-
- return 0;
-}
-
-/* clean up after our dma is done */
-static int sun3scsi_dma_finish(int write_flag)
-{
- unsigned short fifo;
- int ret = 0;
-
- sun3_dma_active = 0;
-
- dregs->csr &= ~CSR_DMA_ENABLE;
-
- fifo = dregs->fifo_count;
- if(write_flag) {
- if((fifo > 0) && (fifo < sun3_dma_orig_count))
- fifo++;
- }
-
- last_residual = fifo;
-#ifdef SUN3_SCSI_DEBUG
- printk("scsi: residual %x total %x\n", fifo, sun3_dma_orig_count);
-#endif
- /* empty bytes from the fifo which didn't make it */
- if((!write_flag) && (dregs->csr & CSR_LEFT)) {
- unsigned char *vaddr;
-
-#ifdef SUN3_SCSI_DEBUG
- printk("scsi: got left over bytes\n");
-#endif
-
- vaddr = (unsigned char *)dvma_vmetov(sun3_dma_orig_addr);
-
- vaddr += (sun3_dma_orig_count - fifo);
- vaddr--;
-
- switch(dregs->csr & CSR_LEFT) {
- case CSR_LEFT_3:
- *vaddr = (dregs->bpack_lo & 0xff00) >> 8;
- vaddr--;
-
- case CSR_LEFT_2:
- *vaddr = (dregs->bpack_hi & 0x00ff);
- vaddr--;
-
- case CSR_LEFT_1:
- *vaddr = (dregs->bpack_hi & 0xff00) >> 8;
- break;
- }
-
-
- }
-
- dvma_unmap(sun3_dma_orig_addr);
- sun3_dma_orig_addr = NULL;
-
- dregs->dma_addr_hi = 0;
- dregs->dma_addr_lo = 0;
- dregs->dma_count_hi = 0;
- dregs->dma_count_lo = 0;
-
- dregs->fifo_count = 0;
- dregs->fifo_count_hi = 0;
-
- dregs->csr &= ~CSR_SEND;
-
-// dregs->csr |= CSR_DMA_ENABLE;
-
-#if 0
- /* reset fifo */
- dregs->csr &= ~CSR_FIFO;
- dregs->csr |= CSR_FIFO;
-#endif
- sun3_dma_setup_done = NULL;
-
- return ret;
-
-}
-
-#include "sun3_NCR5380.c"
-
-static struct scsi_host_template driver_template = {
- .name = SUN3_SCSI_NAME,
- .detect = sun3scsi_detect,
- .release = sun3scsi_release,
- .info = sun3scsi_info,
- .queuecommand = sun3scsi_queue_command,
- .eh_abort_handler = sun3scsi_abort,
- .eh_bus_reset_handler = sun3scsi_bus_reset,
- .can_queue = CAN_QUEUE,
- .this_id = 7,
- .sg_tablesize = SG_TABLESIZE,
- .cmd_per_lun = CMD_PER_LUN,
- .use_clustering = DISABLE_CLUSTERING
-};
-
-
-#include "scsi_module.c"
-
-MODULE_LICENSE("GPL");
+#include "sun3_scsi.c"
diff --git a/drivers/scsi/t128.c b/drivers/scsi/t128.c
index a4abce9..8cc8093 100644
--- a/drivers/scsi/t128.c
+++ b/drivers/scsi/t128.c
@@ -102,10 +102,6 @@
* 15 9-11
*/
-/*
- * $Log: t128.c,v $
- */
-
#include <linux/signal.h>
#include <linux/io.h>
#include <linux/blkdev.h>
diff --git a/drivers/scsi/t128.h b/drivers/scsi/t128.h
index 1df82c28..fd68cec 100644
--- a/drivers/scsi/t128.h
+++ b/drivers/scsi/t128.h
@@ -34,10 +34,6 @@
* 1+ (800) 334-5454
*/
-/*
- * $Log: t128.h,v $
- */
-
#ifndef T128_H
#define T128_H
@@ -107,8 +103,6 @@
#define CAN_QUEUE 32
#endif
-#ifndef HOSTS_C
-
#define NCR5380_implementation_fields \
void __iomem *base
@@ -148,6 +142,5 @@
#define T128_IRQS 0xc4a8
-#endif /* else def HOSTS_C */
#endif /* ndef ASM */
#endif /* T128_H */
diff --git a/drivers/scsi/ufs/ufs.h b/drivers/scsi/ufs/ufs.h
index 7210500..f42d1ce 100644
--- a/drivers/scsi/ufs/ufs.h
+++ b/drivers/scsi/ufs/ufs.h
@@ -196,9 +196,9 @@
* @dword_2: UPIU header DW-2
*/
struct utp_upiu_header {
- u32 dword_0;
- u32 dword_1;
- u32 dword_2;
+ __be32 dword_0;
+ __be32 dword_1;
+ __be32 dword_2;
};
/**
@@ -207,7 +207,7 @@
* @cdb: Command Descriptor Block CDB DW-4 to DW-7
*/
struct utp_upiu_cmd {
- u32 exp_data_transfer_len;
+ __be32 exp_data_transfer_len;
u8 cdb[MAX_CDB_SIZE];
};
@@ -228,10 +228,10 @@
u8 idn;
u8 index;
u8 selector;
- u16 reserved_osf;
- u16 length;
- u32 value;
- u32 reserved[2];
+ __be16 reserved_osf;
+ __be16 length;
+ __be32 value;
+ __be32 reserved[2];
};
/**
@@ -256,9 +256,9 @@
* @sense_data: Sense data field DW-8 to DW-12
*/
struct utp_cmd_rsp {
- u32 residual_transfer_count;
- u32 reserved[4];
- u16 sense_data_len;
+ __be32 residual_transfer_count;
+ __be32 reserved[4];
+ __be16 sense_data_len;
u8 sense_data[18];
};
@@ -286,10 +286,10 @@
*/
struct utp_upiu_task_req {
struct utp_upiu_header header;
- u32 input_param1;
- u32 input_param2;
- u32 input_param3;
- u32 reserved[2];
+ __be32 input_param1;
+ __be32 input_param2;
+ __be32 input_param3;
+ __be32 reserved[2];
};
/**
@@ -301,9 +301,9 @@
*/
struct utp_upiu_task_rsp {
struct utp_upiu_header header;
- u32 output_param1;
- u32 output_param2;
- u32 reserved[3];
+ __be32 output_param1;
+ __be32 output_param2;
+ __be32 reserved[3];
};
/**
diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
index 04884d6..0c28772 100644
--- a/drivers/scsi/ufs/ufshcd.c
+++ b/drivers/scsi/ufs/ufshcd.c
@@ -55,6 +55,9 @@
/* Query request timeout */
#define QUERY_REQ_TIMEOUT 30 /* msec */
+/* Task management command timeout */
+#define TM_CMD_TIMEOUT 100 /* msecs */
+
/* Expose the flag value from utp_upiu_query.value */
#define MASK_QUERY_UPIU_FLAG_LOC 0xFF
@@ -71,9 +74,22 @@
/* UFSHCD states */
enum {
- UFSHCD_STATE_OPERATIONAL,
UFSHCD_STATE_RESET,
UFSHCD_STATE_ERROR,
+ UFSHCD_STATE_OPERATIONAL,
+};
+
+/* UFSHCD error handling flags */
+enum {
+ UFSHCD_EH_IN_PROGRESS = (1 << 0),
+};
+
+/* UFSHCD UIC layer error flags */
+enum {
+ UFSHCD_UIC_DL_PA_INIT_ERROR = (1 << 0), /* Data link layer error */
+ UFSHCD_UIC_NL_ERROR = (1 << 1), /* Network layer error */
+ UFSHCD_UIC_TL_ERROR = (1 << 2), /* Transport Layer error */
+ UFSHCD_UIC_DME_ERROR = (1 << 3), /* DME error */
};
/* Interrupt configuration options */
@@ -83,6 +99,18 @@
UFSHCD_INT_CLEAR,
};
+#define ufshcd_set_eh_in_progress(h) \
+ (h->eh_flags |= UFSHCD_EH_IN_PROGRESS)
+#define ufshcd_eh_in_progress(h) \
+ (h->eh_flags & UFSHCD_EH_IN_PROGRESS)
+#define ufshcd_clear_eh_in_progress(h) \
+ (h->eh_flags &= ~UFSHCD_EH_IN_PROGRESS)
+
+static void ufshcd_tmc_handler(struct ufs_hba *hba);
+static void ufshcd_async_scan(void *data, async_cookie_t cookie);
+static int ufshcd_reset_and_restore(struct ufs_hba *hba);
+static int ufshcd_clear_tm_cmd(struct ufs_hba *hba, int tag);
+
/*
* ufshcd_wait_for_register - wait for register value to change
* @hba - per-adapter interface
@@ -163,7 +191,7 @@
*/
static inline int ufshcd_get_tr_ocs(struct ufshcd_lrb *lrbp)
{
- return lrbp->utr_descriptor_ptr->header.dword_2 & MASK_OCS;
+ return le32_to_cpu(lrbp->utr_descriptor_ptr->header.dword_2) & MASK_OCS;
}
/**
@@ -176,19 +204,41 @@
static inline int
ufshcd_get_tmr_ocs(struct utp_task_req_desc *task_req_descp)
{
- return task_req_descp->header.dword_2 & MASK_OCS;
+ return le32_to_cpu(task_req_descp->header.dword_2) & MASK_OCS;
}
/**
* ufshcd_get_tm_free_slot - get a free slot for task management request
* @hba: per adapter instance
+ * @free_slot: pointer to variable with available slot value
*
- * Returns maximum number of task management request slots in case of
- * task management queue full or returns the free slot number
+ * Get a free tag and lock it until ufshcd_put_tm_slot() is called.
+ * Returns 0 if free slot is not available, else return 1 with tag value
+ * in @free_slot.
*/
-static inline int ufshcd_get_tm_free_slot(struct ufs_hba *hba)
+static bool ufshcd_get_tm_free_slot(struct ufs_hba *hba, int *free_slot)
{
- return find_first_zero_bit(&hba->outstanding_tasks, hba->nutmrs);
+ int tag;
+ bool ret = false;
+
+ if (!free_slot)
+ goto out;
+
+ do {
+ tag = find_first_zero_bit(&hba->tm_slots_in_use, hba->nutmrs);
+ if (tag >= hba->nutmrs)
+ goto out;
+ } while (test_and_set_bit_lock(tag, &hba->tm_slots_in_use));
+
+ *free_slot = tag;
+ ret = true;
+out:
+ return ret;
+}
+
+static inline void ufshcd_put_tm_slot(struct ufs_hba *hba, int slot)
+{
+ clear_bit_unlock(slot, &hba->tm_slots_in_use);
}
/**
@@ -390,26 +440,6 @@
}
/**
- * ufshcd_query_to_cpu() - formats the buffer to native cpu endian
- * @response: upiu query response to convert
- */
-static inline void ufshcd_query_to_cpu(struct utp_upiu_query *response)
-{
- response->length = be16_to_cpu(response->length);
- response->value = be32_to_cpu(response->value);
-}
-
-/**
- * ufshcd_query_to_be() - formats the buffer to big endian
- * @request: upiu query request to convert
- */
-static inline void ufshcd_query_to_be(struct utp_upiu_query *request)
-{
- request->length = cpu_to_be16(request->length);
- request->value = cpu_to_be32(request->value);
-}
-
-/**
* ufshcd_copy_query_response() - Copy the Query Response and the data
* descriptor
* @hba: per adapter instance
@@ -425,7 +455,6 @@
UPIU_RSP_CODE_OFFSET;
memcpy(&query_res->upiu_res, &lrbp->ucd_rsp_ptr->qr, QUERY_OSF_SIZE);
- ufshcd_query_to_cpu(&query_res->upiu_res);
/* Get the descriptor */
@@ -749,7 +778,7 @@
{
struct utp_upiu_req *ucd_req_ptr = lrbp->ucd_req_ptr;
struct ufs_query *query = &hba->dev_cmd.query;
- u16 len = query->request.upiu_req.length;
+ u16 len = be16_to_cpu(query->request.upiu_req.length);
u8 *descp = (u8 *)lrbp->ucd_req_ptr + GENERAL_UPIU_REQUEST_SIZE;
/* Query request header */
@@ -766,7 +795,6 @@
/* Copy the Query Request buffer as is */
memcpy(&ucd_req_ptr->qr, &query->request.upiu_req,
QUERY_OSF_SIZE);
- ufshcd_query_to_be(&ucd_req_ptr->qr);
/* Copy the Descriptor */
if ((len > 0) && (query->request.upiu_req.opcode ==
@@ -853,10 +881,25 @@
tag = cmd->request->tag;
- if (hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL) {
+ spin_lock_irqsave(hba->host->host_lock, flags);
+ switch (hba->ufshcd_state) {
+ case UFSHCD_STATE_OPERATIONAL:
+ break;
+ case UFSHCD_STATE_RESET:
err = SCSI_MLQUEUE_HOST_BUSY;
- goto out;
+ goto out_unlock;
+ case UFSHCD_STATE_ERROR:
+ set_host_byte(cmd, DID_ERROR);
+ cmd->scsi_done(cmd);
+ goto out_unlock;
+ default:
+ dev_WARN_ONCE(hba->dev, 1, "%s: invalid state %d\n",
+ __func__, hba->ufshcd_state);
+ set_host_byte(cmd, DID_BAD_TARGET);
+ cmd->scsi_done(cmd);
+ goto out_unlock;
}
+ spin_unlock_irqrestore(hba->host->host_lock, flags);
/* acquire the tag to make sure device cmds don't use it */
if (test_and_set_bit_lock(tag, &hba->lrb_in_use)) {
@@ -893,6 +936,7 @@
/* issue command to the controller */
spin_lock_irqsave(hba->host->host_lock, flags);
ufshcd_send_command(hba, tag);
+out_unlock:
spin_unlock_irqrestore(hba->host->host_lock, flags);
out:
return err;
@@ -1151,7 +1195,7 @@
}
if (flag_res)
- *flag_res = (response->upiu_res.value &
+ *flag_res = (be32_to_cpu(response->upiu_res.value) &
MASK_QUERY_UPIU_FLAG_LOC) & 0x1;
out_unlock:
@@ -1170,7 +1214,7 @@
*
* Returns 0 for success, non-zero in case of failure
*/
-int ufshcd_query_attr(struct ufs_hba *hba, enum query_opcode opcode,
+static int ufshcd_query_attr(struct ufs_hba *hba, enum query_opcode opcode,
enum attr_idn idn, u8 index, u8 selector, u32 *attr_val)
{
struct ufs_query_req *request;
@@ -1195,7 +1239,7 @@
switch (opcode) {
case UPIU_QUERY_OPCODE_WRITE_ATTR:
request->query_func = UPIU_QUERY_FUNC_STANDARD_WRITE_REQUEST;
- request->upiu_req.value = *attr_val;
+ request->upiu_req.value = cpu_to_be32(*attr_val);
break;
case UPIU_QUERY_OPCODE_READ_ATTR:
request->query_func = UPIU_QUERY_FUNC_STANDARD_READ_REQUEST;
@@ -1222,7 +1266,7 @@
goto out_unlock;
}
- *attr_val = response->upiu_res.value;
+ *attr_val = be32_to_cpu(response->upiu_res.value);
out_unlock:
mutex_unlock(&hba->dev_cmd.lock);
@@ -1481,7 +1525,7 @@
*
* Returns 0 on success, non-zero value on failure
*/
-int ufshcd_uic_change_pwr_mode(struct ufs_hba *hba, u8 mode)
+static int ufshcd_uic_change_pwr_mode(struct ufs_hba *hba, u8 mode)
{
struct uic_command uic_cmd = {0};
struct completion pwr_done;
@@ -1701,11 +1745,6 @@
goto out;
}
- if (hba->ufshcd_state == UFSHCD_STATE_RESET)
- scsi_unblock_requests(hba->host);
-
- hba->ufshcd_state = UFSHCD_STATE_OPERATIONAL;
-
out:
return err;
}
@@ -1831,66 +1870,6 @@
}
/**
- * ufshcd_do_reset - reset the host controller
- * @hba: per adapter instance
- *
- * Returns SUCCESS/FAILED
- */
-static int ufshcd_do_reset(struct ufs_hba *hba)
-{
- struct ufshcd_lrb *lrbp;
- unsigned long flags;
- int tag;
-
- /* block commands from midlayer */
- scsi_block_requests(hba->host);
-
- spin_lock_irqsave(hba->host->host_lock, flags);
- hba->ufshcd_state = UFSHCD_STATE_RESET;
-
- /* send controller to reset state */
- ufshcd_hba_stop(hba);
- spin_unlock_irqrestore(hba->host->host_lock, flags);
-
- /* abort outstanding commands */
- for (tag = 0; tag < hba->nutrs; tag++) {
- if (test_bit(tag, &hba->outstanding_reqs)) {
- lrbp = &hba->lrb[tag];
- if (lrbp->cmd) {
- scsi_dma_unmap(lrbp->cmd);
- lrbp->cmd->result = DID_RESET << 16;
- lrbp->cmd->scsi_done(lrbp->cmd);
- lrbp->cmd = NULL;
- clear_bit_unlock(tag, &hba->lrb_in_use);
- }
- }
- }
-
- /* complete device management command */
- if (hba->dev_cmd.complete)
- complete(hba->dev_cmd.complete);
-
- /* clear outstanding request/task bit maps */
- hba->outstanding_reqs = 0;
- hba->outstanding_tasks = 0;
-
- /* Host controller enable */
- if (ufshcd_hba_enable(hba)) {
- dev_err(hba->dev,
- "Reset: Controller initialization failed\n");
- return FAILED;
- }
-
- if (ufshcd_link_startup(hba)) {
- dev_err(hba->dev,
- "Reset: Link start-up failed\n");
- return FAILED;
- }
-
- return SUCCESS;
-}
-
-/**
* ufshcd_slave_alloc - handle initial SCSI device configurations
* @sdev: pointer to SCSI device
*
@@ -1907,6 +1886,9 @@
sdev->use_10_for_ms = 1;
scsi_set_tag_type(sdev, MSG_SIMPLE_TAG);
+ /* allow SCSI layer to restart the device in case of errors */
+ sdev->allow_restart = 1;
+
/*
* Inform SCSI Midlayer that the LUN queue depth is same as the
* controller queue depth. If a LUN queue depth is less than the
@@ -1934,10 +1916,11 @@
* ufshcd_task_req_compl - handle task management request completion
* @hba: per adapter instance
* @index: index of the completed request
+ * @resp: task management service response
*
- * Returns SUCCESS/FAILED
+ * Returns non-zero value on error, zero on success
*/
-static int ufshcd_task_req_compl(struct ufs_hba *hba, u32 index)
+static int ufshcd_task_req_compl(struct ufs_hba *hba, u32 index, u8 *resp)
{
struct utp_task_req_desc *task_req_descp;
struct utp_upiu_task_rsp *task_rsp_upiup;
@@ -1958,19 +1941,15 @@
task_req_descp[index].task_rsp_upiu;
task_result = be32_to_cpu(task_rsp_upiup->header.dword_1);
task_result = ((task_result & MASK_TASK_RESPONSE) >> 8);
-
- if (task_result != UPIU_TASK_MANAGEMENT_FUNC_COMPL &&
- task_result != UPIU_TASK_MANAGEMENT_FUNC_SUCCEEDED)
- task_result = FAILED;
- else
- task_result = SUCCESS;
+ if (resp)
+ *resp = (u8)task_result;
} else {
- task_result = FAILED;
- dev_err(hba->dev,
- "trc: Invalid ocs = %x\n", ocs_value);
+ dev_err(hba->dev, "%s: failed, ocs = 0x%x\n",
+ __func__, ocs_value);
}
spin_unlock_irqrestore(hba->host->host_lock, flags);
- return task_result;
+
+ return ocs_value;
}
/**
@@ -2105,6 +2084,9 @@
case OCS_ABORTED:
result |= DID_ABORT << 16;
break;
+ case OCS_INVALID_COMMAND_STATUS:
+ result |= DID_REQUEUE << 16;
+ break;
case OCS_INVALID_CMD_TABLE_ATTR:
case OCS_INVALID_PRDT_ATTR:
case OCS_MISMATCH_DATA_BUF_SIZE:
@@ -2422,41 +2404,145 @@
}
/**
- * ufshcd_fatal_err_handler - handle fatal errors
- * @hba: per adapter instance
+ * ufshcd_err_handler - handle UFS errors that require s/w attention
+ * @work: pointer to work structure
*/
-static void ufshcd_fatal_err_handler(struct work_struct *work)
+static void ufshcd_err_handler(struct work_struct *work)
{
struct ufs_hba *hba;
- hba = container_of(work, struct ufs_hba, feh_workq);
+ unsigned long flags;
+ u32 err_xfer = 0;
+ u32 err_tm = 0;
+ int err = 0;
+ int tag;
+
+ hba = container_of(work, struct ufs_hba, eh_work);
pm_runtime_get_sync(hba->dev);
- /* check if reset is already in progress */
- if (hba->ufshcd_state != UFSHCD_STATE_RESET)
- ufshcd_do_reset(hba);
+
+ spin_lock_irqsave(hba->host->host_lock, flags);
+ if (hba->ufshcd_state == UFSHCD_STATE_RESET) {
+ spin_unlock_irqrestore(hba->host->host_lock, flags);
+ goto out;
+ }
+
+ hba->ufshcd_state = UFSHCD_STATE_RESET;
+ ufshcd_set_eh_in_progress(hba);
+
+ /* Complete requests that have door-bell cleared by h/w */
+ ufshcd_transfer_req_compl(hba);
+ ufshcd_tmc_handler(hba);
+ spin_unlock_irqrestore(hba->host->host_lock, flags);
+
+ /* Clear pending transfer requests */
+ for_each_set_bit(tag, &hba->outstanding_reqs, hba->nutrs)
+ if (ufshcd_clear_cmd(hba, tag))
+ err_xfer |= 1 << tag;
+
+ /* Clear pending task management requests */
+ for_each_set_bit(tag, &hba->outstanding_tasks, hba->nutmrs)
+ if (ufshcd_clear_tm_cmd(hba, tag))
+ err_tm |= 1 << tag;
+
+ /* Complete the requests that are cleared by s/w */
+ spin_lock_irqsave(hba->host->host_lock, flags);
+ ufshcd_transfer_req_compl(hba);
+ ufshcd_tmc_handler(hba);
+ spin_unlock_irqrestore(hba->host->host_lock, flags);
+
+ /* Fatal errors need reset */
+ if (err_xfer || err_tm || (hba->saved_err & INT_FATAL_ERRORS) ||
+ ((hba->saved_err & UIC_ERROR) &&
+ (hba->saved_uic_err & UFSHCD_UIC_DL_PA_INIT_ERROR))) {
+ err = ufshcd_reset_and_restore(hba);
+ if (err) {
+ dev_err(hba->dev, "%s: reset and restore failed\n",
+ __func__);
+ hba->ufshcd_state = UFSHCD_STATE_ERROR;
+ }
+ /*
+ * Inform scsi mid-layer that we did reset and allow to handle
+ * Unit Attention properly.
+ */
+ scsi_report_bus_reset(hba->host, 0);
+ hba->saved_err = 0;
+ hba->saved_uic_err = 0;
+ }
+ ufshcd_clear_eh_in_progress(hba);
+
+out:
+ scsi_unblock_requests(hba->host);
pm_runtime_put_sync(hba->dev);
}
/**
- * ufshcd_err_handler - Check for fatal errors
- * @work: pointer to a work queue structure
+ * ufshcd_update_uic_error - check and set fatal UIC error flags.
+ * @hba: per-adapter instance
*/
-static void ufshcd_err_handler(struct ufs_hba *hba)
+static void ufshcd_update_uic_error(struct ufs_hba *hba)
{
u32 reg;
+ /* PA_INIT_ERROR is fatal and needs UIC reset */
+ reg = ufshcd_readl(hba, REG_UIC_ERROR_CODE_DATA_LINK_LAYER);
+ if (reg & UIC_DATA_LINK_LAYER_ERROR_PA_INIT)
+ hba->uic_error |= UFSHCD_UIC_DL_PA_INIT_ERROR;
+
+ /* UIC NL/TL/DME errors needs software retry */
+ reg = ufshcd_readl(hba, REG_UIC_ERROR_CODE_NETWORK_LAYER);
+ if (reg)
+ hba->uic_error |= UFSHCD_UIC_NL_ERROR;
+
+ reg = ufshcd_readl(hba, REG_UIC_ERROR_CODE_TRANSPORT_LAYER);
+ if (reg)
+ hba->uic_error |= UFSHCD_UIC_TL_ERROR;
+
+ reg = ufshcd_readl(hba, REG_UIC_ERROR_CODE_DME);
+ if (reg)
+ hba->uic_error |= UFSHCD_UIC_DME_ERROR;
+
+ dev_dbg(hba->dev, "%s: UIC error flags = 0x%08x\n",
+ __func__, hba->uic_error);
+}
+
+/**
+ * ufshcd_check_errors - Check for errors that need s/w attention
+ * @hba: per-adapter instance
+ */
+static void ufshcd_check_errors(struct ufs_hba *hba)
+{
+ bool queue_eh_work = false;
+
if (hba->errors & INT_FATAL_ERRORS)
- goto fatal_eh;
+ queue_eh_work = true;
if (hba->errors & UIC_ERROR) {
- reg = ufshcd_readl(hba, REG_UIC_ERROR_CODE_DATA_LINK_LAYER);
- if (reg & UIC_DATA_LINK_LAYER_ERROR_PA_INIT)
- goto fatal_eh;
+ hba->uic_error = 0;
+ ufshcd_update_uic_error(hba);
+ if (hba->uic_error)
+ queue_eh_work = true;
}
- return;
-fatal_eh:
- hba->ufshcd_state = UFSHCD_STATE_ERROR;
- schedule_work(&hba->feh_workq);
+
+ if (queue_eh_work) {
+ /* handle fatal errors only when link is functional */
+ if (hba->ufshcd_state == UFSHCD_STATE_OPERATIONAL) {
+ /* block commands from scsi mid-layer */
+ scsi_block_requests(hba->host);
+
+ /* transfer error masks to sticky bits */
+ hba->saved_err |= hba->errors;
+ hba->saved_uic_err |= hba->uic_error;
+
+ hba->ufshcd_state = UFSHCD_STATE_ERROR;
+ schedule_work(&hba->eh_work);
+ }
+ }
+ /*
+ * if (!queue_eh_work) -
+ * Other errors are either non-fatal where host recovers
+ * itself without s/w intervention or errors that will be
+ * handled by the SCSI core layer.
+ */
}
/**
@@ -2469,7 +2555,7 @@
tm_doorbell = ufshcd_readl(hba, REG_UTP_TASK_REQ_DOOR_BELL);
hba->tm_condition = tm_doorbell ^ hba->outstanding_tasks;
- wake_up_interruptible(&hba->ufshcd_tm_wait_queue);
+ wake_up(&hba->tm_wq);
}
/**
@@ -2481,7 +2567,7 @@
{
hba->errors = UFSHCD_ERROR_MASK & intr_status;
if (hba->errors)
- ufshcd_err_handler(hba);
+ ufshcd_check_errors(hba);
if (intr_status & UFSHCD_UIC_MASK)
ufshcd_uic_cmd_compl(hba, intr_status);
@@ -2519,38 +2605,58 @@
return retval;
}
+static int ufshcd_clear_tm_cmd(struct ufs_hba *hba, int tag)
+{
+ int err = 0;
+ u32 mask = 1 << tag;
+ unsigned long flags;
+
+ if (!test_bit(tag, &hba->outstanding_tasks))
+ goto out;
+
+ spin_lock_irqsave(hba->host->host_lock, flags);
+ ufshcd_writel(hba, ~(1 << tag), REG_UTP_TASK_REQ_LIST_CLEAR);
+ spin_unlock_irqrestore(hba->host->host_lock, flags);
+
+ /* poll for max. 1 sec to clear door bell register by h/w */
+ err = ufshcd_wait_for_register(hba,
+ REG_UTP_TASK_REQ_DOOR_BELL,
+ mask, 0, 1000, 1000);
+out:
+ return err;
+}
+
/**
* ufshcd_issue_tm_cmd - issues task management commands to controller
* @hba: per adapter instance
- * @lrbp: pointer to local reference block
+ * @lun_id: LUN ID to which TM command is sent
+ * @task_id: task ID to which the TM command is applicable
+ * @tm_function: task management function opcode
+ * @tm_response: task management service response return value
*
- * Returns SUCCESS/FAILED
+ * Returns non-zero value on error, zero on success.
*/
-static int
-ufshcd_issue_tm_cmd(struct ufs_hba *hba,
- struct ufshcd_lrb *lrbp,
- u8 tm_function)
+static int ufshcd_issue_tm_cmd(struct ufs_hba *hba, int lun_id, int task_id,
+ u8 tm_function, u8 *tm_response)
{
struct utp_task_req_desc *task_req_descp;
struct utp_upiu_task_req *task_req_upiup;
struct Scsi_Host *host;
unsigned long flags;
- int free_slot = 0;
+ int free_slot;
int err;
+ int task_tag;
host = hba->host;
+ /*
+ * Get free slot, sleep if slots are unavailable.
+ * Even though we use wait_event() which sleeps indefinitely,
+ * the maximum wait time is bounded by %TM_CMD_TIMEOUT.
+ */
+ wait_event(hba->tm_tag_wq, ufshcd_get_tm_free_slot(hba, &free_slot));
+
spin_lock_irqsave(host->host_lock, flags);
-
- /* If task management queue is full */
- free_slot = ufshcd_get_tm_free_slot(hba);
- if (free_slot >= hba->nutmrs) {
- spin_unlock_irqrestore(host->host_lock, flags);
- dev_err(hba->dev, "Task management queue full\n");
- err = FAILED;
- goto out;
- }
-
task_req_descp = hba->utmrdl_base_addr;
task_req_descp += free_slot;
@@ -2562,18 +2668,15 @@
/* Configure task request UPIU */
task_req_upiup =
(struct utp_upiu_task_req *) task_req_descp->task_req_upiu;
+ task_tag = hba->nutrs + free_slot;
task_req_upiup->header.dword_0 =
UPIU_HEADER_DWORD(UPIU_TRANSACTION_TASK_REQ, 0,
- lrbp->lun, lrbp->task_tag);
+ lun_id, task_tag);
task_req_upiup->header.dword_1 =
UPIU_HEADER_DWORD(0, tm_function, 0, 0);
- task_req_upiup->input_param1 = lrbp->lun;
- task_req_upiup->input_param1 =
- cpu_to_be32(task_req_upiup->input_param1);
- task_req_upiup->input_param2 = lrbp->task_tag;
- task_req_upiup->input_param2 =
- cpu_to_be32(task_req_upiup->input_param2);
+ task_req_upiup->input_param1 = cpu_to_be32(lun_id);
+ task_req_upiup->input_param2 = cpu_to_be32(task_id);
/* send command to the controller */
__set_bit(free_slot, &hba->outstanding_tasks);
@@ -2582,91 +2685,88 @@
spin_unlock_irqrestore(host->host_lock, flags);
/* wait until the task management command is completed */
- err =
- wait_event_interruptible_timeout(hba->ufshcd_tm_wait_queue,
- (test_bit(free_slot,
- &hba->tm_condition) != 0),
- 60 * HZ);
+ err = wait_event_timeout(hba->tm_wq,
+ test_bit(free_slot, &hba->tm_condition),
+ msecs_to_jiffies(TM_CMD_TIMEOUT));
if (!err) {
- dev_err(hba->dev,
- "Task management command timed-out\n");
- err = FAILED;
- goto out;
+ dev_err(hba->dev, "%s: task management cmd 0x%.2x timed-out\n",
+ __func__, tm_function);
+ if (ufshcd_clear_tm_cmd(hba, free_slot))
+ dev_WARN(hba->dev, "%s: unable clear tm cmd (slot %d) after timeout\n",
+ __func__, free_slot);
+ err = -ETIMEDOUT;
+ } else {
+ err = ufshcd_task_req_compl(hba, free_slot, tm_response);
}
+
clear_bit(free_slot, &hba->tm_condition);
- err = ufshcd_task_req_compl(hba, free_slot);
-out:
+ ufshcd_put_tm_slot(hba, free_slot);
+ wake_up(&hba->tm_tag_wq);
+
return err;
}
/**
- * ufshcd_device_reset - reset device and abort all the pending commands
+ * ufshcd_eh_device_reset_handler - device reset handler registered to
+ * scsi layer.
* @cmd: SCSI command pointer
*
* Returns SUCCESS/FAILED
*/
-static int ufshcd_device_reset(struct scsi_cmnd *cmd)
+static int ufshcd_eh_device_reset_handler(struct scsi_cmnd *cmd)
{
struct Scsi_Host *host;
struct ufs_hba *hba;
unsigned int tag;
u32 pos;
int err;
+ u8 resp = 0xF;
+ struct ufshcd_lrb *lrbp;
+ unsigned long flags;
host = cmd->device->host;
hba = shost_priv(host);
tag = cmd->request->tag;
- err = ufshcd_issue_tm_cmd(hba, &hba->lrb[tag], UFS_LOGICAL_RESET);
- if (err == FAILED)
+ lrbp = &hba->lrb[tag];
+ err = ufshcd_issue_tm_cmd(hba, lrbp->lun, 0, UFS_LOGICAL_RESET, &resp);
+ if (err || resp != UPIU_TASK_MANAGEMENT_FUNC_COMPL) {
+ if (!err)
+ err = resp;
goto out;
+ }
- for (pos = 0; pos < hba->nutrs; pos++) {
- if (test_bit(pos, &hba->outstanding_reqs) &&
- (hba->lrb[tag].lun == hba->lrb[pos].lun)) {
-
- /* clear the respective UTRLCLR register bit */
- ufshcd_utrl_clear(hba, pos);
-
- clear_bit(pos, &hba->outstanding_reqs);
-
- if (hba->lrb[pos].cmd) {
- scsi_dma_unmap(hba->lrb[pos].cmd);
- hba->lrb[pos].cmd->result =
- DID_ABORT << 16;
- hba->lrb[pos].cmd->scsi_done(cmd);
- hba->lrb[pos].cmd = NULL;
- clear_bit_unlock(pos, &hba->lrb_in_use);
- wake_up(&hba->dev_cmd.tag_wq);
- }
+ /* clear the commands that were pending for corresponding LUN */
+ for_each_set_bit(pos, &hba->outstanding_reqs, hba->nutrs) {
+ if (hba->lrb[pos].lun == lrbp->lun) {
+ err = ufshcd_clear_cmd(hba, pos);
+ if (err)
+ break;
}
- } /* end of for */
+ }
+ spin_lock_irqsave(host->host_lock, flags);
+ ufshcd_transfer_req_compl(hba);
+ spin_unlock_irqrestore(host->host_lock, flags);
out:
+ if (!err) {
+ err = SUCCESS;
+ } else {
+ dev_err(hba->dev, "%s: failed with err %d\n", __func__, err);
+ err = FAILED;
+ }
return err;
}
/**
- * ufshcd_host_reset - Main reset function registered with scsi layer
- * @cmd: SCSI command pointer
- *
- * Returns SUCCESS/FAILED
- */
-static int ufshcd_host_reset(struct scsi_cmnd *cmd)
-{
- struct ufs_hba *hba;
-
- hba = shost_priv(cmd->device->host);
-
- if (hba->ufshcd_state == UFSHCD_STATE_RESET)
- return SUCCESS;
-
- return ufshcd_do_reset(hba);
-}
-
-/**
* ufshcd_abort - abort a specific command
* @cmd: SCSI command pointer
*
+ * Abort the pending command in device by sending UFS_ABORT_TASK task management
+ * command, and in host controller by clearing the door-bell register. There can
+ * be race between controller sending the command to the device while abort is
+ * issued. To avoid that, first issue UFS_QUERY_TASK to check if the command is
+ * really issued and then try to abort it.
+ *
* Returns SUCCESS/FAILED
*/
static int ufshcd_abort(struct scsi_cmnd *cmd)
@@ -2675,33 +2775,68 @@
struct ufs_hba *hba;
unsigned long flags;
unsigned int tag;
- int err;
+ int err = 0;
+ int poll_cnt;
+ u8 resp = 0xF;
+ struct ufshcd_lrb *lrbp;
host = cmd->device->host;
hba = shost_priv(host);
tag = cmd->request->tag;
- spin_lock_irqsave(host->host_lock, flags);
+ /* If command is already aborted/completed, return SUCCESS */
+ if (!(test_bit(tag, &hba->outstanding_reqs)))
+ goto out;
- /* check if command is still pending */
- if (!(test_bit(tag, &hba->outstanding_reqs))) {
- err = FAILED;
- spin_unlock_irqrestore(host->host_lock, flags);
+ lrbp = &hba->lrb[tag];
+ for (poll_cnt = 100; poll_cnt; poll_cnt--) {
+ err = ufshcd_issue_tm_cmd(hba, lrbp->lun, lrbp->task_tag,
+ UFS_QUERY_TASK, &resp);
+ if (!err && resp == UPIU_TASK_MANAGEMENT_FUNC_SUCCEEDED) {
+ /* cmd pending in the device */
+ break;
+ } else if (!err && resp == UPIU_TASK_MANAGEMENT_FUNC_COMPL) {
+ u32 reg;
+
+ /*
+ * cmd not pending in the device, check if it is
+ * in transition.
+ */
+ reg = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL);
+ if (reg & (1 << tag)) {
+ /* sleep for max. 200us to stabilize */
+ usleep_range(100, 200);
+ continue;
+ }
+ /* command completed already */
+ goto out;
+ } else {
+ if (!err)
+ err = resp; /* service response error */
+ goto out;
+ }
+ }
+
+ if (!poll_cnt) {
+ err = -EBUSY;
goto out;
}
- spin_unlock_irqrestore(host->host_lock, flags);
- err = ufshcd_issue_tm_cmd(hba, &hba->lrb[tag], UFS_ABORT_TASK);
- if (err == FAILED)
+ err = ufshcd_issue_tm_cmd(hba, lrbp->lun, lrbp->task_tag,
+ UFS_ABORT_TASK, &resp);
+ if (err || resp != UPIU_TASK_MANAGEMENT_FUNC_COMPL) {
+ if (!err)
+ err = resp; /* service response error */
+ goto out;
+ }
+
+ err = ufshcd_clear_cmd(hba, tag);
+ if (err)
goto out;
scsi_dma_unmap(cmd);
spin_lock_irqsave(host->host_lock, flags);
-
- /* clear the respective UTRLCLR register bit */
- ufshcd_utrl_clear(hba, tag);
-
__clear_bit(tag, &hba->outstanding_reqs);
hba->lrb[tag].cmd = NULL;
spin_unlock_irqrestore(host->host_lock, flags);
@@ -2709,6 +2844,129 @@
clear_bit_unlock(tag, &hba->lrb_in_use);
wake_up(&hba->dev_cmd.tag_wq);
out:
+ if (!err) {
+ err = SUCCESS;
+ } else {
+ dev_err(hba->dev, "%s: failed with err %d\n", __func__, err);
+ err = FAILED;
+ }
+
+ return err;
+}
+
+/**
+ * ufshcd_host_reset_and_restore - reset and restore host controller
+ * @hba: per-adapter instance
+ *
+ * Note that host controller reset may issue DME_RESET to
+ * local and remote (device) Uni-Pro stack and the attributes
+ * are reset to default state.
+ *
+ * Returns zero on success, non-zero on failure
+ */
+static int ufshcd_host_reset_and_restore(struct ufs_hba *hba)
+{
+ int err;
+ async_cookie_t cookie;
+ unsigned long flags;
+
+ /* Reset the host controller */
+ spin_lock_irqsave(hba->host->host_lock, flags);
+ ufshcd_hba_stop(hba);
+ spin_unlock_irqrestore(hba->host->host_lock, flags);
+
+ err = ufshcd_hba_enable(hba);
+ if (err)
+ goto out;
+
+ /* Establish the link again and restore the device */
+ cookie = async_schedule(ufshcd_async_scan, hba);
+ /* wait for async scan to be completed */
+ async_synchronize_cookie(++cookie);
+ if (hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL)
+ err = -EIO;
+out:
+ if (err)
+ dev_err(hba->dev, "%s: Host init failed %d\n", __func__, err);
+
+ return err;
+}
+
+/**
+ * ufshcd_reset_and_restore - reset and re-initialize host/device
+ * @hba: per-adapter instance
+ *
+ * Reset and recover device, host and re-establish link. This
+ * is helpful to recover the communication in fatal error conditions.
+ *
+ * Returns zero on success, non-zero on failure
+ */
+static int ufshcd_reset_and_restore(struct ufs_hba *hba)
+{
+ int err = 0;
+ unsigned long flags;
+
+ err = ufshcd_host_reset_and_restore(hba);
+
+ /*
+ * After reset the door-bell might be cleared, complete
+ * outstanding requests in s/w here.
+ */
+ spin_lock_irqsave(hba->host->host_lock, flags);
+ ufshcd_transfer_req_compl(hba);
+ ufshcd_tmc_handler(hba);
+ spin_unlock_irqrestore(hba->host->host_lock, flags);
+
+ return err;
+}
+
+/**
+ * ufshcd_eh_host_reset_handler - host reset handler registered to scsi layer
+ * @cmd - SCSI command pointer
+ *
+ * Returns SUCCESS/FAILED
+ */
+static int ufshcd_eh_host_reset_handler(struct scsi_cmnd *cmd)
+{
+ int err;
+ unsigned long flags;
+ struct ufs_hba *hba;
+
+ hba = shost_priv(cmd->device->host);
+
+ /*
+ * Check if there is any race with fatal error handling.
+ * If so, wait for it to complete. Even though fatal error
+ * handling does reset and restore in some cases, don't assume
+ * anything out of it. We are just avoiding race here.
+ */
+ do {
+ spin_lock_irqsave(hba->host->host_lock, flags);
+ if (!(work_pending(&hba->eh_work) ||
+ hba->ufshcd_state == UFSHCD_STATE_RESET))
+ break;
+ spin_unlock_irqrestore(hba->host->host_lock, flags);
+ dev_dbg(hba->dev, "%s: reset in progress\n", __func__);
+ flush_work(&hba->eh_work);
+ } while (1);
+
+ hba->ufshcd_state = UFSHCD_STATE_RESET;
+ ufshcd_set_eh_in_progress(hba);
+ spin_unlock_irqrestore(hba->host->host_lock, flags);
+
+ err = ufshcd_reset_and_restore(hba);
+
+ spin_lock_irqsave(hba->host->host_lock, flags);
+ if (!err) {
+ err = SUCCESS;
+ hba->ufshcd_state = UFSHCD_STATE_OPERATIONAL;
+ } else {
+ err = FAILED;
+ hba->ufshcd_state = UFSHCD_STATE_ERROR;
+ }
+ ufshcd_clear_eh_in_progress(hba);
+ spin_unlock_irqrestore(hba->host->host_lock, flags);
+
return err;
}
@@ -2737,8 +2995,13 @@
goto out;
ufshcd_force_reset_auto_bkops(hba);
- scsi_scan_host(hba->host);
- pm_runtime_put_sync(hba->dev);
+ hba->ufshcd_state = UFSHCD_STATE_OPERATIONAL;
+
+ /* If we are in error handling context no need to scan the host */
+ if (!ufshcd_eh_in_progress(hba)) {
+ scsi_scan_host(hba->host);
+ pm_runtime_put_sync(hba->dev);
+ }
out:
return;
}
@@ -2751,8 +3014,8 @@
.slave_alloc = ufshcd_slave_alloc,
.slave_destroy = ufshcd_slave_destroy,
.eh_abort_handler = ufshcd_abort,
- .eh_device_reset_handler = ufshcd_device_reset,
- .eh_host_reset_handler = ufshcd_host_reset,
+ .eh_device_reset_handler = ufshcd_eh_device_reset_handler,
+ .eh_host_reset_handler = ufshcd_eh_host_reset_handler,
.this_id = -1,
.sg_tablesize = SG_ALL,
.cmd_per_lun = UFSHCD_CMD_PER_LUN,
@@ -2916,10 +3179,11 @@
host->max_cmd_len = MAX_CDB_SIZE;
/* Initailize wait queue for task management */
- init_waitqueue_head(&hba->ufshcd_tm_wait_queue);
+ init_waitqueue_head(&hba->tm_wq);
+ init_waitqueue_head(&hba->tm_tag_wq);
/* Initialize work queues */
- INIT_WORK(&hba->feh_workq, ufshcd_fatal_err_handler);
+ INIT_WORK(&hba->eh_work, ufshcd_err_handler);
INIT_WORK(&hba->eeh_work, ufshcd_exception_event_handler);
/* Initialize UIC command mutex */
diff --git a/drivers/scsi/ufs/ufshcd.h b/drivers/scsi/ufs/ufshcd.h
index 577679a..acf318e3 100644
--- a/drivers/scsi/ufs/ufshcd.h
+++ b/drivers/scsi/ufs/ufshcd.h
@@ -174,15 +174,21 @@
* @irq: Irq number of the controller
* @active_uic_cmd: handle of active UIC command
* @uic_cmd_mutex: mutex for uic command
- * @ufshcd_tm_wait_queue: wait queue for task management
+ * @tm_wq: wait queue for task management
+ * @tm_tag_wq: wait queue for free task management slots
+ * @tm_slots_in_use: bit map of task management request slots in use
* @pwr_done: completion for power mode change
* @tm_condition: condition variable for task management
* @ufshcd_state: UFSHCD states
+ * @eh_flags: Error handling flags
* @intr_mask: Interrupt Mask Bits
* @ee_ctrl_mask: Exception event control mask
- * @feh_workq: Work queue for fatal controller error handling
+ * @eh_work: Worker to handle UFS errors that require s/w attention
* @eeh_work: Worker to handle exception events
* @errors: HBA errors
+ * @uic_error: UFS interconnect layer error status
+ * @saved_err: sticky error mask
+ * @saved_uic_err: sticky UIC error mask
* @dev_cmd: ufs device management command information
* @auto_bkops_enabled: to track whether bkops is enabled in device
*/
@@ -217,21 +223,27 @@
struct uic_command *active_uic_cmd;
struct mutex uic_cmd_mutex;
- wait_queue_head_t ufshcd_tm_wait_queue;
+ wait_queue_head_t tm_wq;
+ wait_queue_head_t tm_tag_wq;
unsigned long tm_condition;
+ unsigned long tm_slots_in_use;
struct completion *pwr_done;
u32 ufshcd_state;
+ u32 eh_flags;
u32 intr_mask;
u16 ee_ctrl_mask;
/* Work Queues */
- struct work_struct feh_workq;
+ struct work_struct eh_work;
struct work_struct eeh_work;
/* HBA Errors */
u32 errors;
+ u32 uic_error;
+ u32 saved_err;
+ u32 saved_uic_err;
/* Device management request data */
struct ufs_dev_cmd dev_cmd;
@@ -263,6 +275,8 @@
GENERAL_UPIU_REQUEST_SIZE + QUERY_DESC_MAX_SIZE);
}
+extern int ufshcd_suspend(struct ufs_hba *hba, pm_message_t state);
+extern int ufshcd_resume(struct ufs_hba *hba);
extern int ufshcd_runtime_suspend(struct ufs_hba *hba);
extern int ufshcd_runtime_resume(struct ufs_hba *hba);
extern int ufshcd_runtime_idle(struct ufs_hba *hba);
diff --git a/drivers/scsi/ufs/ufshci.h b/drivers/scsi/ufs/ufshci.h
index 0475c66..9abc7e3 100644
--- a/drivers/scsi/ufs/ufshci.h
+++ b/drivers/scsi/ufs/ufshci.h
@@ -304,10 +304,10 @@
* @size: size of physical segment DW-3
*/
struct ufshcd_sg_entry {
- u32 base_addr;
- u32 upper_addr;
- u32 reserved;
- u32 size;
+ __le32 base_addr;
+ __le32 upper_addr;
+ __le32 reserved;
+ __le32 size;
};
/**
@@ -330,10 +330,10 @@
* @dword3: Descriptor Header DW3
*/
struct request_desc_header {
- u32 dword_0;
- u32 dword_1;
- u32 dword_2;
- u32 dword_3;
+ __le32 dword_0;
+ __le32 dword_1;
+ __le32 dword_2;
+ __le32 dword_3;
};
/**
@@ -352,16 +352,16 @@
struct request_desc_header header;
/* DW 4-5*/
- u32 command_desc_base_addr_lo;
- u32 command_desc_base_addr_hi;
+ __le32 command_desc_base_addr_lo;
+ __le32 command_desc_base_addr_hi;
/* DW 6 */
- u16 response_upiu_length;
- u16 response_upiu_offset;
+ __le16 response_upiu_length;
+ __le16 response_upiu_offset;
/* DW 7 */
- u16 prd_table_length;
- u16 prd_table_offset;
+ __le16 prd_table_length;
+ __le16 prd_table_offset;
};
/**
@@ -376,10 +376,10 @@
struct request_desc_header header;
/* DW 4-11 */
- u32 task_req_upiu[TASK_REQ_UPIU_SIZE_DWORDS];
+ __le32 task_req_upiu[TASK_REQ_UPIU_SIZE_DWORDS];
/* DW 12-19 */
- u32 task_rsp_upiu[TASK_RSP_UPIU_SIZE_DWORDS];
+ __le32 task_rsp_upiu[TASK_RSP_UPIU_SIZE_DWORDS];
};
#endif /* End of Header */
diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
index db3b494..d4727b3 100644
--- a/drivers/scsi/virtio_scsi.c
+++ b/drivers/scsi/virtio_scsi.c
@@ -73,17 +73,12 @@
* queue, and also lets the driver optimize the IRQ affinity for the virtqueues
* (each virtqueue's affinity is set to the CPU that "owns" the queue).
*
- * An interesting effect of this policy is that only writes to req_vq need to
- * take the tgt_lock. Read can be done outside the lock because:
+ * tgt_lock is held to serialize reading and writing req_vq. Reading req_vq
+ * could be done locklessly, but we do not do it yet.
*
- * - writes of req_vq only occur when atomic_inc_return(&tgt->reqs) returns 1.
- * In that case, no other CPU is reading req_vq: even if they were in
- * virtscsi_queuecommand_multi, they would be spinning on tgt_lock.
- *
- * - reads of req_vq only occur when the target is not idle (reqs != 0).
- * A CPU that enters virtscsi_queuecommand_multi will not modify req_vq.
- *
- * Similarly, decrements of reqs are never concurrent with writes of req_vq.
+ * Decrements of reqs are never concurrent with writes of req_vq: before the
+ * decrement reqs will be != 0; after the decrement the virtqueue completion
+ * routine will not use the req_vq so it can be changed by a new request.
* Thus they can happen outside the tgt_lock, provided of course we make reqs
* an atomic_t.
*/
@@ -204,7 +199,6 @@
set_driver_byte(sc, DRIVER_SENSE);
}
- mempool_free(cmd, virtscsi_cmd_pool);
sc->scsi_done(sc);
atomic_dec(&tgt->reqs);
@@ -238,38 +232,6 @@
int index = vq->index - VIRTIO_SCSI_VQ_BASE;
struct virtio_scsi_vq *req_vq = &vscsi->req_vqs[index];
- /*
- * Read req_vq before decrementing the reqs field in
- * virtscsi_complete_cmd.
- *
- * With barriers:
- *
- * CPU #0 virtscsi_queuecommand_multi (CPU #1)
- * ------------------------------------------------------------
- * lock vq_lock
- * read req_vq
- * read reqs (reqs = 1)
- * write reqs (reqs = 0)
- * increment reqs (reqs = 1)
- * write req_vq
- *
- * Possible reordering without barriers:
- *
- * CPU #0 virtscsi_queuecommand_multi (CPU #1)
- * ------------------------------------------------------------
- * lock vq_lock
- * read reqs (reqs = 1)
- * write reqs (reqs = 0)
- * increment reqs (reqs = 1)
- * write req_vq
- * read (wrong) req_vq
- *
- * We do not need a full smp_rmb, because req_vq is required to get
- * to tgt->reqs: tgt is &vscsi->tgt[sc->device->id], where sc is stored
- * in the virtqueue as the user token.
- */
- smp_read_barrier_depends();
-
virtscsi_vq_done(vscsi, req_vq, virtscsi_complete_cmd);
};
@@ -279,8 +241,6 @@
if (cmd->comp)
complete_all(cmd->comp);
- else
- mempool_free(cmd, virtscsi_cmd_pool);
}
static void virtscsi_ctrl_done(struct virtqueue *vq)
@@ -496,10 +456,9 @@
struct virtio_scsi_vq *req_vq,
struct scsi_cmnd *sc)
{
- struct virtio_scsi_cmd *cmd;
- int ret;
-
struct Scsi_Host *shost = virtio_scsi_host(vscsi->vdev);
+ struct virtio_scsi_cmd *cmd = scsi_cmd_priv(sc);
+
BUG_ON(scsi_sg_count(sc) > shost->sg_tablesize);
/* TODO: check feature bit and fail if unsupported? */
@@ -508,11 +467,6 @@
dev_dbg(&sc->device->sdev_gendev,
"cmd %p CDB: %#02x\n", sc, sc->cmnd[0]);
- ret = SCSI_MLQUEUE_HOST_BUSY;
- cmd = mempool_alloc(virtscsi_cmd_pool, GFP_ATOMIC);
- if (!cmd)
- goto out;
-
memset(cmd, 0, sizeof(*cmd));
cmd->sc = sc;
cmd->req.cmd = (struct virtio_scsi_cmd_req){
@@ -531,13 +485,9 @@
if (virtscsi_kick_cmd(req_vq, cmd,
sizeof cmd->req.cmd, sizeof cmd->resp.cmd,
- GFP_ATOMIC) == 0)
- ret = 0;
- else
- mempool_free(cmd, virtscsi_cmd_pool);
-
-out:
- return ret;
+ GFP_ATOMIC) != 0)
+ return SCSI_MLQUEUE_HOST_BUSY;
+ return 0;
}
static int virtscsi_queuecommand_single(struct Scsi_Host *sh,
@@ -560,12 +510,8 @@
spin_lock_irqsave(&tgt->tgt_lock, flags);
- /*
- * The memory barrier after atomic_inc_return matches
- * the smp_read_barrier_depends() in virtscsi_req_done.
- */
if (atomic_inc_return(&tgt->reqs) > 1)
- vq = ACCESS_ONCE(tgt->req_vq);
+ vq = tgt->req_vq;
else {
queue_num = smp_processor_id();
while (unlikely(queue_num >= vscsi->num_queues))
@@ -683,6 +629,7 @@
.name = "Virtio SCSI HBA",
.proc_name = "virtio_scsi",
.this_id = -1,
+ .cmd_size = sizeof(struct virtio_scsi_cmd),
.queuecommand = virtscsi_queuecommand_single,
.eh_abort_handler = virtscsi_abort,
.eh_device_reset_handler = virtscsi_device_reset,
@@ -699,6 +646,7 @@
.name = "Virtio SCSI HBA",
.proc_name = "virtio_scsi",
.this_id = -1,
+ .cmd_size = sizeof(struct virtio_scsi_cmd),
.queuecommand = virtscsi_queuecommand_multi,
.eh_abort_handler = virtscsi_abort,
.eh_device_reset_handler = virtscsi_device_reset,
diff --git a/drivers/sh/intc/Kconfig b/drivers/sh/intc/Kconfig
index f7d9061..60228fa 100644
--- a/drivers/sh/intc/Kconfig
+++ b/drivers/sh/intc/Kconfig
@@ -6,7 +6,7 @@
config INTC_USERIMASK
bool "Userspace interrupt masking support"
- depends on ARCH_SHMOBILE || (SUPERH && CPU_SH4A) || COMPILE_TEST
+ depends on (SUPERH && CPU_SH4A) || COMPILE_TEST
help
This enables support for hardware-assisted userspace hardirq
masking.
diff --git a/drivers/sh/pm_runtime.c b/drivers/sh/pm_runtime.c
index 10c65eb..72f6381 100644
--- a/drivers/sh/pm_runtime.c
+++ b/drivers/sh/pm_runtime.c
@@ -21,18 +21,43 @@
#include <linux/slab.h>
#ifdef CONFIG_PM_RUNTIME
-
-static int default_platform_runtime_idle(struct device *dev)
+static int sh_pm_runtime_suspend(struct device *dev)
{
- /* suspend synchronously to disable clocks immediately */
+ int ret;
+
+ ret = pm_generic_runtime_suspend(dev);
+ if (ret) {
+ dev_err(dev, "failed to suspend device\n");
+ return ret;
+ }
+
+ ret = pm_clk_suspend(dev);
+ if (ret) {
+ dev_err(dev, "failed to suspend clock\n");
+ pm_generic_runtime_resume(dev);
+ return ret;
+ }
+
return 0;
}
+static int sh_pm_runtime_resume(struct device *dev)
+{
+ int ret;
+
+ ret = pm_clk_resume(dev);
+ if (ret) {
+ dev_err(dev, "failed to resume clock\n");
+ return ret;
+ }
+
+ return pm_generic_runtime_resume(dev);
+}
+
static struct dev_pm_domain default_pm_domain = {
.ops = {
- .runtime_suspend = pm_clk_suspend,
- .runtime_resume = pm_clk_resume,
- .runtime_idle = default_platform_runtime_idle,
+ .runtime_suspend = sh_pm_runtime_suspend,
+ .runtime_resume = sh_pm_runtime_resume,
USE_PLATFORM_PM_SLEEP_OPS
},
};
@@ -63,6 +88,9 @@
!of_machine_is_compatible("renesas,r8a7779") &&
!of_machine_is_compatible("renesas,r8a7790") &&
!of_machine_is_compatible("renesas,r8a7791") &&
+ !of_machine_is_compatible("renesas,r8a7792") &&
+ !of_machine_is_compatible("renesas,r8a7793") &&
+ !of_machine_is_compatible("renesas,r8a7794") &&
!of_machine_is_compatible("renesas,sh7372") &&
!of_machine_is_compatible("renesas,sh73a0"))
return 0;
diff --git a/drivers/staging/comedi/drivers/ni_daq_700.c b/drivers/staging/comedi/drivers/ni_daq_700.c
index 171a71d..728bf7f 100644
--- a/drivers/staging/comedi/drivers/ni_daq_700.c
+++ b/drivers/staging/comedi/drivers/ni_daq_700.c
@@ -139,6 +139,8 @@
/* write channel to multiplexer */
/* set mask scan bit high to disable scanning */
outb(chan | 0x80, dev->iobase + CMD_R1);
+ /* mux needs 2us to really settle [Fred Brooks]. */
+ udelay(2);
/* convert n samples */
for (n = 0; n < insn->n; n++) {
diff --git a/drivers/staging/rtl8192e/rtllib_tx.c b/drivers/staging/rtl8192e/rtllib_tx.c
index 11d0a9d..b7dd153 100644
--- a/drivers/staging/rtl8192e/rtllib_tx.c
+++ b/drivers/staging/rtl8192e/rtllib_tx.c
@@ -171,7 +171,7 @@
snap->oui[1] = oui[1];
snap->oui[2] = oui[2];
- *(u16 *)(data + SNAP_SIZE) = h_proto;
+ *(__be16 *)(data + SNAP_SIZE) = htons(h_proto);
return SNAP_SIZE + sizeof(u16);
}
diff --git a/drivers/staging/speakup/main.c b/drivers/staging/speakup/main.c
index 3b6e535..7de79d5 100644
--- a/drivers/staging/speakup/main.c
+++ b/drivers/staging/speakup/main.c
@@ -2218,6 +2218,7 @@
unregister_keyboard_notifier(&keyboard_notifier_block);
unregister_vt_notifier(&vt_notifier_block);
speakup_unregister_devsynth();
+ speakup_cancel_paste();
del_timer(&cursor_timer);
kthread_stop(speakup_task);
speakup_task = NULL;
diff --git a/drivers/staging/speakup/selection.c b/drivers/staging/speakup/selection.c
index f0fb003..ca04d36 100644
--- a/drivers/staging/speakup/selection.c
+++ b/drivers/staging/speakup/selection.c
@@ -4,6 +4,10 @@
#include <linux/sched.h>
#include <linux/device.h> /* for dev_warn */
#include <linux/selection.h>
+#include <linux/workqueue.h>
+#include <linux/tty.h>
+#include <linux/tty_flip.h>
+#include <asm/cmpxchg.h>
#include "speakup.h"
@@ -121,31 +125,61 @@
return 0;
}
-/* TODO: move to some helper thread, probably. That'd fix having to check for
- * in_atomic(). */
-int speakup_paste_selection(struct tty_struct *tty)
+struct speakup_paste_work {
+ struct work_struct work;
+ struct tty_struct *tty;
+};
+
+static void __speakup_paste_selection(struct work_struct *work)
{
+ struct speakup_paste_work *spw =
+ container_of(work, struct speakup_paste_work, work);
+ struct tty_struct *tty = xchg(&spw->tty, NULL);
struct vc_data *vc = (struct vc_data *) tty->driver_data;
int pasted = 0, count;
+ struct tty_ldisc *ld;
DECLARE_WAITQUEUE(wait, current);
+
+ ld = tty_ldisc_ref_wait(tty);
+ tty_buffer_lock_exclusive(&vc->port);
+
add_wait_queue(&vc->paste_wait, &wait);
while (sel_buffer && sel_buffer_lth > pasted) {
set_current_state(TASK_INTERRUPTIBLE);
if (test_bit(TTY_THROTTLED, &tty->flags)) {
- if (in_atomic())
- /* if we are in an interrupt handler, abort */
- break;
schedule();
continue;
}
count = sel_buffer_lth - pasted;
- count = min_t(int, count, tty->receive_room);
- tty->ldisc->ops->receive_buf(tty, sel_buffer + pasted,
- NULL, count);
+ count = tty_ldisc_receive_buf(ld, sel_buffer + pasted, NULL,
+ count);
pasted += count;
}
remove_wait_queue(&vc->paste_wait, &wait);
current->state = TASK_RUNNING;
+
+ tty_buffer_unlock_exclusive(&vc->port);
+ tty_ldisc_deref(ld);
+ tty_kref_put(tty);
+}
+
+static struct speakup_paste_work speakup_paste_work = {
+ .work = __WORK_INITIALIZER(speakup_paste_work.work,
+ __speakup_paste_selection)
+};
+
+int speakup_paste_selection(struct tty_struct *tty)
+{
+ if (cmpxchg(&speakup_paste_work.tty, NULL, tty) != NULL)
+ return -EBUSY;
+
+ tty_kref_get(tty);
+ schedule_work_on(WORK_CPU_UNBOUND, &speakup_paste_work.work);
return 0;
}
+void speakup_cancel_paste(void)
+{
+ cancel_work_sync(&speakup_paste_work.work);
+ tty_kref_put(speakup_paste_work.tty);
+}
diff --git a/drivers/staging/speakup/speakup.h b/drivers/staging/speakup/speakup.h
index a7bccee..898dce5 100644
--- a/drivers/staging/speakup/speakup.h
+++ b/drivers/staging/speakup/speakup.h
@@ -75,6 +75,7 @@
extern void speakup_clear_selection(void);
extern int speakup_set_selection(struct tty_struct *tty);
extern int speakup_paste_selection(struct tty_struct *tty);
+extern void speakup_cancel_paste(void);
extern void speakup_register_devsynth(void);
extern void speakup_unregister_devsynth(void);
extern void synth_write(const char *buf, size_t count);
diff --git a/drivers/staging/speakup/speakup_acntsa.c b/drivers/staging/speakup/speakup_acntsa.c
index 1f37484..3f2b5698 100644
--- a/drivers/staging/speakup/speakup_acntsa.c
+++ b/drivers/staging/speakup/speakup_acntsa.c
@@ -60,15 +60,15 @@
__ATTR(vol, S_IWUSR|S_IRUGO, spk_var_show, spk_var_store);
static struct kobj_attribute delay_time_attribute =
- __ATTR(delay_time, S_IRUSR|S_IRUGO, spk_var_show, spk_var_store);
+ __ATTR(delay_time, S_IWUSR|S_IRUGO, spk_var_show, spk_var_store);
static struct kobj_attribute direct_attribute =
__ATTR(direct, S_IWUSR|S_IRUGO, spk_var_show, spk_var_store);
static struct kobj_attribute full_time_attribute =
- __ATTR(full_time, S_IRUSR|S_IRUGO, spk_var_show, spk_var_store);
+ __ATTR(full_time, S_IWUSR|S_IRUGO, spk_var_show, spk_var_store);
static struct kobj_attribute jiffy_delta_attribute =
- __ATTR(jiffy_delta, S_IRUSR|S_IRUGO, spk_var_show, spk_var_store);
+ __ATTR(jiffy_delta, S_IWUSR|S_IRUGO, spk_var_show, spk_var_store);
static struct kobj_attribute trigger_time_attribute =
- __ATTR(trigger_time, S_IRUSR|S_IRUGO, spk_var_show, spk_var_store);
+ __ATTR(trigger_time, S_IWUSR|S_IRUGO, spk_var_show, spk_var_store);
/*
* Create a group of attributes so that we can create and destroy them all
diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
index 46588c8..9189bc0 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -460,6 +460,7 @@
spin_lock_bh(&np->np_thread_lock);
np->np_exports--;
if (np->np_exports) {
+ np->enabled = true;
spin_unlock_bh(&np->np_thread_lock);
return 0;
}
diff --git a/drivers/target/iscsi/iscsi_target_login.c b/drivers/target/iscsi/iscsi_target_login.c
index ca31fa1..d9b1d88 100644
--- a/drivers/target/iscsi/iscsi_target_login.c
+++ b/drivers/target/iscsi/iscsi_target_login.c
@@ -249,6 +249,28 @@
mutex_unlock(&auth_id_lock);
}
+static __printf(2, 3) int iscsi_change_param_sprintf(
+ struct iscsi_conn *conn,
+ const char *fmt, ...)
+{
+ va_list args;
+ unsigned char buf[64];
+
+ memset(buf, 0, sizeof buf);
+
+ va_start(args, fmt);
+ vsnprintf(buf, sizeof buf, fmt, args);
+ va_end(args);
+
+ if (iscsi_change_param_value(buf, conn->param_list, 0) < 0) {
+ iscsit_tx_login_rsp(conn, ISCSI_STATUS_CLS_TARGET_ERR,
+ ISCSI_LOGIN_STATUS_NO_RESOURCES);
+ return -1;
+ }
+
+ return 0;
+}
+
/*
* This is the leading connection of a new session,
* or session reinstatement.
@@ -339,7 +361,6 @@
{
struct iscsi_node_attrib *na;
struct iscsi_session *sess = conn->sess;
- unsigned char buf[32];
bool iser = false;
sess->tpg = conn->tpg;
@@ -380,26 +401,16 @@
*
* In our case, we have already located the struct iscsi_tiqn at this point.
*/
- memset(buf, 0, 32);
- sprintf(buf, "TargetPortalGroupTag=%hu", sess->tpg->tpgt);
- if (iscsi_change_param_value(buf, conn->param_list, 0) < 0) {
- iscsit_tx_login_rsp(conn, ISCSI_STATUS_CLS_TARGET_ERR,
- ISCSI_LOGIN_STATUS_NO_RESOURCES);
+ if (iscsi_change_param_sprintf(conn, "TargetPortalGroupTag=%hu", sess->tpg->tpgt))
return -1;
- }
/*
* Workaround for Initiators that have broken connection recovery logic.
*
* "We would really like to get rid of this." Linux-iSCSI.org team
*/
- memset(buf, 0, 32);
- sprintf(buf, "ErrorRecoveryLevel=%d", na->default_erl);
- if (iscsi_change_param_value(buf, conn->param_list, 0) < 0) {
- iscsit_tx_login_rsp(conn, ISCSI_STATUS_CLS_TARGET_ERR,
- ISCSI_LOGIN_STATUS_NO_RESOURCES);
+ if (iscsi_change_param_sprintf(conn, "ErrorRecoveryLevel=%d", na->default_erl))
return -1;
- }
if (iscsi_login_disable_FIM_keys(conn->param_list, conn) < 0)
return -1;
@@ -411,12 +422,9 @@
unsigned long mrdsl, off;
int rc;
- sprintf(buf, "RDMAExtensions=Yes");
- if (iscsi_change_param_value(buf, conn->param_list, 0) < 0) {
- iscsit_tx_login_rsp(conn, ISCSI_STATUS_CLS_TARGET_ERR,
- ISCSI_LOGIN_STATUS_NO_RESOURCES);
+ if (iscsi_change_param_sprintf(conn, "RDMAExtensions=Yes"))
return -1;
- }
+
/*
* Make MaxRecvDataSegmentLength PAGE_SIZE aligned for
* Immediate Data + Unsolicitied Data-OUT if necessary..
@@ -446,12 +454,8 @@
pr_warn("Aligning ISER MaxRecvDataSegmentLength: %lu down"
" to PAGE_SIZE\n", mrdsl);
- sprintf(buf, "MaxRecvDataSegmentLength=%lu\n", mrdsl);
- if (iscsi_change_param_value(buf, conn->param_list, 0) < 0) {
- iscsit_tx_login_rsp(conn, ISCSI_STATUS_CLS_TARGET_ERR,
- ISCSI_LOGIN_STATUS_NO_RESOURCES);
+ if (iscsi_change_param_sprintf(conn, "MaxRecvDataSegmentLength=%lu\n", mrdsl))
return -1;
- }
/*
* ISER currently requires that ImmediateData + Unsolicited
* Data be disabled when protection / signature MRs are enabled.
@@ -461,19 +465,12 @@
(TARGET_PROT_DOUT_STRIP | TARGET_PROT_DOUT_PASS |
TARGET_PROT_DOUT_INSERT)) {
- sprintf(buf, "ImmediateData=No");
- if (iscsi_change_param_value(buf, conn->param_list, 0) < 0) {
- iscsit_tx_login_rsp(conn, ISCSI_STATUS_CLS_TARGET_ERR,
- ISCSI_LOGIN_STATUS_NO_RESOURCES);
+ if (iscsi_change_param_sprintf(conn, "ImmediateData=No"))
return -1;
- }
- sprintf(buf, "InitialR2T=Yes");
- if (iscsi_change_param_value(buf, conn->param_list, 0) < 0) {
- iscsit_tx_login_rsp(conn, ISCSI_STATUS_CLS_TARGET_ERR,
- ISCSI_LOGIN_STATUS_NO_RESOURCES);
+ if (iscsi_change_param_sprintf(conn, "InitialR2T=Yes"))
return -1;
- }
+
pr_debug("Forcing ImmediateData=No + InitialR2T=Yes for"
" T10-PI enabled ISER session\n");
}
@@ -618,13 +615,8 @@
*
* In our case, we have already located the struct iscsi_tiqn at this point.
*/
- memset(buf, 0, 32);
- sprintf(buf, "TargetPortalGroupTag=%hu", sess->tpg->tpgt);
- if (iscsi_change_param_value(buf, conn->param_list, 0) < 0) {
- iscsit_tx_login_rsp(conn, ISCSI_STATUS_CLS_TARGET_ERR,
- ISCSI_LOGIN_STATUS_NO_RESOURCES);
+ if (iscsi_change_param_sprintf(conn, "TargetPortalGroupTag=%hu", sess->tpg->tpgt))
return -1;
- }
return iscsi_login_disable_FIM_keys(conn->param_list, conn);
}
diff --git a/drivers/target/iscsi/iscsi_target_tpg.c b/drivers/target/iscsi/iscsi_target_tpg.c
index ca18118..1431e84 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.c
+++ b/drivers/target/iscsi/iscsi_target_tpg.c
@@ -184,7 +184,8 @@
return;
}
- tpg_np->tpg_np->enabled = false;
+ if (shutdown)
+ tpg_np->tpg_np->enabled = false;
iscsit_reset_np_thread(tpg_np->tpg_np, tpg_np, tpg, shutdown);
}
diff --git a/drivers/target/target_core_alua.c b/drivers/target/target_core_alua.c
index 0b79b85..fbc5ebb 100644
--- a/drivers/target/target_core_alua.c
+++ b/drivers/target/target_core_alua.c
@@ -576,7 +576,16 @@
case REPORT_LUNS:
case RECEIVE_DIAGNOSTIC:
case SEND_DIAGNOSTIC:
+ case READ_CAPACITY:
return 0;
+ case SERVICE_ACTION_IN:
+ switch (cdb[1] & 0x1f) {
+ case SAI_READ_CAPACITY_16:
+ return 0;
+ default:
+ set_ascq(cmd, ASCQ_04H_ALUA_TG_PT_STANDBY);
+ return 1;
+ }
case MAINTENANCE_IN:
switch (cdb[1] & 0x1f) {
case MI_REPORT_TARGET_PGS:
diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c
index 60a9ae6..bf55c5a 100644
--- a/drivers/target/target_core_configfs.c
+++ b/drivers/target/target_core_configfs.c
@@ -2227,6 +2227,11 @@
" tg_pt_gp ID: %hu\n", tg_pt_gp->tg_pt_gp_valid_id);
return -EINVAL;
}
+ if (!(dev->dev_flags & DF_CONFIGURED)) {
+ pr_err("Unable to set alua_access_state while device is"
+ " not configured\n");
+ return -ENODEV;
+ }
ret = kstrtoul(page, 0, &tmp);
if (ret < 0) {
diff --git a/drivers/tty/sysrq.c b/drivers/tty/sysrq.c
index b767a64..454b658 100644
--- a/drivers/tty/sysrq.c
+++ b/drivers/tty/sysrq.c
@@ -46,6 +46,7 @@
#include <linux/jiffies.h>
#include <linux/syscalls.h>
#include <linux/of.h>
+#include <linux/rcupdate.h>
#include <asm/ptrace.h>
#include <asm/irq_regs.h>
@@ -510,9 +511,9 @@
struct sysrq_key_op *op_p;
int orig_log_level;
int i;
- unsigned long flags;
- spin_lock_irqsave(&sysrq_key_table_lock, flags);
+ rcu_sysrq_start();
+ rcu_read_lock();
/*
* Raise the apparent loglevel to maximum so that the sysrq header
* is shown to provide the user with positive feedback. We do not
@@ -554,7 +555,8 @@
printk("\n");
console_loglevel = orig_log_level;
}
- spin_unlock_irqrestore(&sysrq_key_table_lock, flags);
+ rcu_read_unlock();
+ rcu_sysrq_end();
}
void handle_sysrq(int key)
@@ -1043,16 +1045,23 @@
struct sysrq_key_op *remove_op_p)
{
int retval;
- unsigned long flags;
- spin_lock_irqsave(&sysrq_key_table_lock, flags);
+ spin_lock(&sysrq_key_table_lock);
if (__sysrq_get_key_op(key) == remove_op_p) {
__sysrq_put_key_op(key, insert_op_p);
retval = 0;
} else {
retval = -1;
}
- spin_unlock_irqrestore(&sysrq_key_table_lock, flags);
+ spin_unlock(&sysrq_key_table_lock);
+
+ /*
+ * A concurrent __handle_sysrq either got the old op or the new op.
+ * Wait for it to go away before returning, so the code for an old
+ * op is not freed (eg. on module unload) while it is in use.
+ */
+ synchronize_rcu();
+
return retval;
}
diff --git a/drivers/tty/tty_buffer.c b/drivers/tty/tty_buffer.c
index cf78d19..143deb6 100644
--- a/drivers/tty/tty_buffer.c
+++ b/drivers/tty/tty_buffer.c
@@ -60,6 +60,7 @@
atomic_inc(&buf->priority);
mutex_lock(&buf->lock);
}
+EXPORT_SYMBOL_GPL(tty_buffer_lock_exclusive);
void tty_buffer_unlock_exclusive(struct tty_port *port)
{
@@ -73,6 +74,7 @@
if (restart)
queue_work(system_unbound_wq, &buf->work);
}
+EXPORT_SYMBOL_GPL(tty_buffer_unlock_exclusive);
/**
* tty_buffer_space_avail - return unused buffer space
diff --git a/drivers/usb/core/driver.c b/drivers/usb/core/driver.c
index 888881e..4aeb100 100644
--- a/drivers/usb/core/driver.c
+++ b/drivers/usb/core/driver.c
@@ -1822,10 +1822,13 @@
if (status == -EAGAIN || status == -EBUSY)
usb_mark_last_busy(udev);
- /* The PM core reacts badly unless the return code is 0,
- * -EAGAIN, or -EBUSY, so always return -EBUSY on an error.
+ /*
+ * The PM core reacts badly unless the return code is 0,
+ * -EAGAIN, or -EBUSY, so always return -EBUSY on an error
+ * (except for root hubs, because they don't suspend through
+ * an upstream port like other USB devices).
*/
- if (status != 0)
+ if (status != 0 && udev->parent)
return -EBUSY;
return status;
}
diff --git a/drivers/usb/core/hub.c b/drivers/usb/core/hub.c
index db62870..879b66e 100644
--- a/drivers/usb/core/hub.c
+++ b/drivers/usb/core/hub.c
@@ -1706,8 +1706,19 @@
*/
pm_runtime_set_autosuspend_delay(&hdev->dev, 0);
- /* Hubs have proper suspend/resume support. */
- usb_enable_autosuspend(hdev);
+ /*
+ * Hubs have proper suspend/resume support, except for root hubs
+ * where the controller driver doesn't have bus_suspend and
+ * bus_resume methods.
+ */
+ if (hdev->parent) { /* normal device */
+ usb_enable_autosuspend(hdev);
+ } else { /* root hub */
+ const struct hc_driver *drv = bus_to_hcd(hdev->bus)->driver;
+
+ if (drv->bus_suspend && drv->bus_resume)
+ usb_enable_autosuspend(hdev);
+ }
if (hdev->level == MAX_TOPO_LEVEL) {
dev_err(&intf->dev,
diff --git a/drivers/usb/host/ehci-pmcmsp.c b/drivers/usb/host/ehci-pmcmsp.c
index af3974a..7d75465 100644
--- a/drivers/usb/host/ehci-pmcmsp.c
+++ b/drivers/usb/host/ehci-pmcmsp.c
@@ -68,9 +68,6 @@
/* set TWI GPIO USB_HOST_DEV pin high */
gpio_direction_output(MSP_PIN_USB0_HOST_DEV, 1);
-#ifdef CONFIG_MSP_HAS_DUAL_USB
- gpio_direction_output(MSP_PIN_USB1_HOST_DEV, 1);
-#endif
}
/* called during probe() after chip reset completes */
@@ -248,33 +245,6 @@
usb_put_hcd(hcd);
}
-#ifdef CONFIG_MSP_HAS_DUAL_USB
-/*
- * Wrapper around the main ehci_irq. Since both USB host controllers are
- * sharing the same IRQ, need to first determine whether we're the intended
- * recipient of this interrupt.
- */
-static irqreturn_t ehci_msp_irq(struct usb_hcd *hcd)
-{
- u32 int_src;
- struct device *dev = hcd->self.controller;
- struct platform_device *pdev;
- struct mspusb_device *mdev;
- struct ehci_hcd *ehci = hcd_to_ehci(hcd);
- /* need to reverse-map a couple of containers to get our device */
- pdev = to_platform_device(dev);
- mdev = to_mspusb_device(pdev);
-
- /* Check to see if this interrupt is for this host controller */
- int_src = ehci_readl(ehci, &mdev->mab_regs->int_stat);
- if (int_src & (1 << pdev->id))
- return ehci_irq(hcd);
-
- /* Not for this device */
- return IRQ_NONE;
-}
-#endif /* DUAL_USB */
-
static const struct hc_driver ehci_msp_hc_driver = {
.description = hcd_name,
.product_desc = "PMC MSP EHCI",
@@ -283,11 +253,7 @@
/*
* generic hardware linkage
*/
-#ifdef CONFIG_MSP_HAS_DUAL_USB
- .irq = ehci_msp_irq,
-#else
.irq = ehci_irq,
-#endif
.flags = HCD_MEMORY | HCD_USB2 | HCD_BH,
/*
@@ -334,9 +300,6 @@
return -ENODEV;
gpio_request(MSP_PIN_USB0_HOST_DEV, "USB0_HOST_DEV_GPIO");
-#ifdef CONFIG_MSP_HAS_DUAL_USB
- gpio_request(MSP_PIN_USB1_HOST_DEV, "USB1_HOST_DEV_GPIO");
-#endif
ret = usb_hcd_msp_probe(&ehci_msp_hc_driver, pdev);
@@ -351,9 +314,6 @@
/* free TWI GPIO USB_HOST_DEV pin */
gpio_free(MSP_PIN_USB0_HOST_DEV);
-#ifdef CONFIG_MSP_HAS_DUAL_USB
- gpio_free(MSP_PIN_USB1_HOST_DEV);
-#endif
return 0;
}
diff --git a/drivers/usb/host/pci-quirks.c b/drivers/usb/host/pci-quirks.c
index 00661d3..4a6d3dd 100644
--- a/drivers/usb/host/pci-quirks.c
+++ b/drivers/usb/host/pci-quirks.c
@@ -847,6 +847,13 @@
bool ehci_found = false;
struct pci_dev *companion = NULL;
+ /* Sony VAIO t-series with subsystem device ID 90a8 is not capable of
+ * switching ports from EHCI to xHCI
+ */
+ if (xhci_pdev->subsystem_vendor == PCI_VENDOR_ID_SONY &&
+ xhci_pdev->subsystem_device == 0x90a8)
+ return;
+
/* make sure an intel EHCI controller exists */
for_each_pci_dev(companion) {
if (companion->class == PCI_CLASS_SERIAL_USB_EHCI &&
diff --git a/drivers/usb/host/xhci-mem.c b/drivers/usb/host/xhci-mem.c
index 6a57e81..8056d90 100644
--- a/drivers/usb/host/xhci-mem.c
+++ b/drivers/usb/host/xhci-mem.c
@@ -1818,6 +1818,16 @@
xhci_dbg_trace(xhci, trace_xhci_dbg_init, "Freed command ring");
xhci_cleanup_command_queue(xhci);
+ num_ports = HCS_MAX_PORTS(xhci->hcs_params1);
+ for (i = 0; i < num_ports; i++) {
+ struct xhci_interval_bw_table *bwt = &xhci->rh_bw[i].bw_table;
+ for (j = 0; j < XHCI_MAX_INTERVAL; j++) {
+ struct list_head *ep = &bwt->interval_bw[j].endpoints;
+ while (!list_empty(ep))
+ list_del_init(ep->next);
+ }
+ }
+
for (i = 1; i < MAX_HC_SLOTS; ++i)
xhci_free_virt_device(xhci, i);
@@ -1853,16 +1863,6 @@
if (!xhci->rh_bw)
goto no_bw;
- num_ports = HCS_MAX_PORTS(xhci->hcs_params1);
- for (i = 0; i < num_ports; i++) {
- struct xhci_interval_bw_table *bwt = &xhci->rh_bw[i].bw_table;
- for (j = 0; j < XHCI_MAX_INTERVAL; j++) {
- struct list_head *ep = &bwt->interval_bw[j].endpoints;
- while (!list_empty(ep))
- list_del_init(ep->next);
- }
- }
-
for (i = 0; i < num_ports; i++) {
struct xhci_tt_bw_info *tt, *n;
list_for_each_entry_safe(tt, n, &xhci->rh_bw[i].tts, tt_list) {
diff --git a/drivers/usb/serial/ftdi_sio.c b/drivers/usb/serial/ftdi_sio.c
index 7c6e1de..edf3b12 100644
--- a/drivers/usb/serial/ftdi_sio.c
+++ b/drivers/usb/serial/ftdi_sio.c
@@ -580,6 +580,8 @@
{ USB_DEVICE(FTDI_VID, FTDI_TAVIR_STK500_PID) },
{ USB_DEVICE(FTDI_VID, FTDI_TIAO_UMPA_PID),
.driver_info = (kernel_ulong_t)&ftdi_jtag_quirk },
+ { USB_DEVICE(FTDI_VID, FTDI_NT_ORIONLXM_PID),
+ .driver_info = (kernel_ulong_t)&ftdi_jtag_quirk },
/*
* ELV devices:
*/
diff --git a/drivers/usb/serial/ftdi_sio_ids.h b/drivers/usb/serial/ftdi_sio_ids.h
index 993c93d..500474c 100644
--- a/drivers/usb/serial/ftdi_sio_ids.h
+++ b/drivers/usb/serial/ftdi_sio_ids.h
@@ -538,6 +538,11 @@
*/
#define FTDI_TIAO_UMPA_PID 0x8a98 /* TIAO/DIYGADGET USB Multi-Protocol Adapter */
+/*
+ * NovaTech product ids (FTDI_VID)
+ */
+#define FTDI_NT_ORIONLXM_PID 0x7c90 /* OrionLXm Substation Automation Platform */
+
/********************************/
/** third-party VID/PID combos **/
diff --git a/drivers/usb/serial/io_ti.c b/drivers/usb/serial/io_ti.c
index df90dae..c0a42e9 100644
--- a/drivers/usb/serial/io_ti.c
+++ b/drivers/usb/serial/io_ti.c
@@ -821,7 +821,7 @@
firmware_rec = (struct ti_i2c_firmware_rec*)i2c_header->Data;
i2c_header->Type = I2C_DESC_TYPE_FIRMWARE_BLANK;
- i2c_header->Size = (__u16)buffer_size;
+ i2c_header->Size = cpu_to_le16(buffer_size);
i2c_header->CheckSum = cs;
firmware_rec->Ver_Major = OperationalMajorVersion;
firmware_rec->Ver_Minor = OperationalMinorVersion;
diff --git a/drivers/usb/serial/io_usbvend.h b/drivers/usb/serial/io_usbvend.h
index 51f83fbb7..6f6a856 100644
--- a/drivers/usb/serial/io_usbvend.h
+++ b/drivers/usb/serial/io_usbvend.h
@@ -594,7 +594,7 @@
struct ti_i2c_desc {
__u8 Type; // Type of descriptor
- __u16 Size; // Size of data only not including header
+ __le16 Size; // Size of data only not including header
__u8 CheckSum; // Checksum (8 bit sum of data only)
__u8 Data[0]; // Data starts here
} __attribute__((packed));
diff --git a/drivers/usb/serial/option.c b/drivers/usb/serial/option.c
index 51e3074..59c3108 100644
--- a/drivers/usb/serial/option.c
+++ b/drivers/usb/serial/option.c
@@ -161,6 +161,7 @@
#define NOVATELWIRELESS_PRODUCT_HSPA_EMBEDDED_FULLSPEED 0x9000
#define NOVATELWIRELESS_PRODUCT_HSPA_EMBEDDED_HIGHSPEED 0x9001
#define NOVATELWIRELESS_PRODUCT_E362 0x9010
+#define NOVATELWIRELESS_PRODUCT_E371 0x9011
#define NOVATELWIRELESS_PRODUCT_G2 0xA010
#define NOVATELWIRELESS_PRODUCT_MC551 0xB001
@@ -1012,6 +1013,7 @@
/* Novatel Ovation MC551 a.k.a. Verizon USB551L */
{ USB_DEVICE_AND_INTERFACE_INFO(NOVATELWIRELESS_VENDOR_ID, NOVATELWIRELESS_PRODUCT_MC551, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(NOVATELWIRELESS_VENDOR_ID, NOVATELWIRELESS_PRODUCT_E362, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(NOVATELWIRELESS_VENDOR_ID, NOVATELWIRELESS_PRODUCT_E371, 0xff, 0xff, 0xff) },
{ USB_DEVICE(AMOI_VENDOR_ID, AMOI_PRODUCT_H01) },
{ USB_DEVICE(AMOI_VENDOR_ID, AMOI_PRODUCT_H01A) },
diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 7ba0424..010e0f8 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -57,7 +57,8 @@
ret = vfio_config_init(vdev);
if (ret) {
- pci_load_and_free_saved_state(pdev, &vdev->pci_saved_state);
+ kfree(vdev->pci_saved_state);
+ vdev->pci_saved_state = NULL;
pci_disable_device(pdev);
return ret;
}
@@ -196,8 +197,7 @@
if (pos) {
pci_read_config_word(vdev->pdev,
pos + PCI_MSI_FLAGS, &flags);
-
- return 1 << (flags & PCI_MSI_FLAGS_QMASK);
+ return 1 << ((flags & PCI_MSI_FLAGS_QMASK) >> 1);
}
} else if (irq_type == VFIO_PCI_MSIX_IRQ_INDEX) {
u8 pos;
diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
index 83cd157..e50790e 100644
--- a/drivers/vfio/pci/vfio_pci_config.c
+++ b/drivers/vfio/pci/vfio_pci_config.c
@@ -1126,8 +1126,7 @@
return pcibios_err_to_errno(ret);
byte &= PCI_DPA_CAP_SUBSTATE_MASK;
- byte = round_up(byte + 1, 4);
- return PCI_DPA_BASE_SIZEOF + byte;
+ return PCI_DPA_BASE_SIZEOF + byte + 1;
case PCI_EXT_CAP_ID_TPH:
ret = pci_read_config_dword(pdev, epos + PCI_TPH_CAP, &dword);
if (ret)
@@ -1136,9 +1135,9 @@
if ((dword & PCI_TPH_CAP_LOC_MASK) == PCI_TPH_LOC_CAP) {
int sts;
- sts = byte & PCI_TPH_CAP_ST_MASK;
+ sts = dword & PCI_TPH_CAP_ST_MASK;
sts >>= PCI_TPH_CAP_ST_SHIFT;
- return PCI_TPH_BASE_SIZEOF + round_up(sts * 2, 4);
+ return PCI_TPH_BASE_SIZEOF + (sts * 2) + 2;
}
return PCI_TPH_BASE_SIZEOF;
default:
diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index 6673e7b..0734fbe 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -524,7 +524,7 @@
static int vfio_dma_do_map(struct vfio_iommu *iommu,
struct vfio_iommu_type1_dma_map *map)
{
- dma_addr_t end, iova;
+ dma_addr_t iova = map->iova;
unsigned long vaddr = map->vaddr;
size_t size = map->size;
long npage;
@@ -533,39 +533,30 @@
struct vfio_dma *dma;
unsigned long pfn;
- end = map->iova + map->size;
+ /* Verify that none of our __u64 fields overflow */
+ if (map->size != size || map->vaddr != vaddr || map->iova != iova)
+ return -EINVAL;
mask = ((uint64_t)1 << __ffs(vfio_pgsize_bitmap(iommu))) - 1;
+ WARN_ON(mask & PAGE_MASK);
+
/* READ/WRITE from device perspective */
if (map->flags & VFIO_DMA_MAP_FLAG_WRITE)
prot |= IOMMU_WRITE;
if (map->flags & VFIO_DMA_MAP_FLAG_READ)
prot |= IOMMU_READ;
- if (!prot)
- return -EINVAL; /* No READ/WRITE? */
-
- if (vaddr & mask)
- return -EINVAL;
- if (map->iova & mask)
- return -EINVAL;
- if (!map->size || map->size & mask)
+ if (!prot || !size || (size | iova | vaddr) & mask)
return -EINVAL;
- WARN_ON(mask & PAGE_MASK);
-
- /* Don't allow IOVA wrap */
- if (end && end < map->iova)
- return -EINVAL;
-
- /* Don't allow virtual address wrap */
- if (vaddr + map->size && vaddr + map->size < vaddr)
+ /* Don't allow IOVA or virtual address wrap */
+ if (iova + size - 1 < iova || vaddr + size - 1 < vaddr)
return -EINVAL;
mutex_lock(&iommu->lock);
- if (vfio_find_dma(iommu, map->iova, map->size)) {
+ if (vfio_find_dma(iommu, iova, size)) {
mutex_unlock(&iommu->lock);
return -EEXIST;
}
@@ -576,17 +567,17 @@
return -ENOMEM;
}
- dma->iova = map->iova;
- dma->vaddr = map->vaddr;
+ dma->iova = iova;
+ dma->vaddr = vaddr;
dma->prot = prot;
/* Insert zero-sized and grow as we map chunks of it */
vfio_link_dma(iommu, dma);
- for (iova = map->iova; iova < end; iova += size, vaddr += size) {
+ while (size) {
/* Pin a contiguous chunk of memory */
- npage = vfio_pin_pages(vaddr, (end - iova) >> PAGE_SHIFT,
- prot, &pfn);
+ npage = vfio_pin_pages(vaddr + dma->size,
+ size >> PAGE_SHIFT, prot, &pfn);
if (npage <= 0) {
WARN_ON(!npage);
ret = (int)npage;
@@ -594,14 +585,14 @@
}
/* Map it! */
- ret = vfio_iommu_map(iommu, iova, pfn, npage, prot);
+ ret = vfio_iommu_map(iommu, iova + dma->size, pfn, npage, prot);
if (ret) {
vfio_unpin_pages(pfn, npage, prot, true);
break;
}
- size = npage << PAGE_SHIFT;
- dma->size += size;
+ size -= npage << PAGE_SHIFT;
+ dma->size += npage << PAGE_SHIFT;
}
if (ret)
diff --git a/drivers/watchdog/Kconfig b/drivers/watchdog/Kconfig
index 74ec8fc..c845527 100644
--- a/drivers/watchdog/Kconfig
+++ b/drivers/watchdog/Kconfig
@@ -272,7 +272,7 @@
config IOP_WATCHDOG
tristate "IOP Watchdog"
- depends on PLAT_IOP
+ depends on ARCH_IOP13XX
select WATCHDOG_NOWAYOUT if (ARCH_IOP32X || ARCH_IOP33X)
help
Say Y here if to include support for the watchdog timer
@@ -378,6 +378,8 @@
config IMX2_WDT
tristate "IMX2+ Watchdog"
depends on ARCH_MXC
+ select REGMAP_MMIO
+ select WATCHDOG_CORE
help
This is the driver for the hardware watchdog
on the Freescale IMX2 and later processors.
@@ -663,6 +665,19 @@
To compile this driver as a module, choose M here.
+config INTEL_MID_WATCHDOG
+ tristate "Intel MID Watchdog Timer"
+ depends on X86_INTEL_MID
+ select WATCHDOG_CORE
+ ---help---
+ Watchdog timer driver built into the Intel SCU for Intel MID
+ Platforms.
+
+ This driver currently supports only the watchdog evolution
+ implementation in SCU, available for Merrifield generation.
+
+ To compile this driver as a module, choose M here.
+
config ITCO_WDT
tristate "Intel TCO Timer/Watchdog"
depends on (X86 || IA64) && PCI
@@ -835,7 +850,7 @@
config SBC8360_WDT
tristate "SBC8360 Watchdog Timer"
- depends on X86
+ depends on X86_32
---help---
This is the driver for the hardware watchdog on the SBC8360 Single
@@ -938,36 +953,6 @@
Most people will say N.
-config W83697HF_WDT
- tristate "W83697HF/W83697HG Watchdog Timer"
- depends on X86
- ---help---
- This is the driver for the hardware watchdog on the W83697HF/HG
- chipset as used in Dedibox/VIA motherboards (and likely others).
- This watchdog simply watches your kernel to make sure it doesn't
- freeze, and if it does, it reboots your computer after a certain
- amount of time.
-
- To compile this driver as a module, choose M here: the
- module will be called w83697hf_wdt.
-
- Most people will say N.
-
-config W83697UG_WDT
- tristate "W83697UG/W83697UF Watchdog Timer"
- depends on X86
- ---help---
- This is the driver for the hardware watchdog on the W83697UG/UF
- chipset as used in MSI Fuzzy CX700 VIA motherboards (and likely others).
- This watchdog simply watches your kernel to make sure it doesn't
- freeze, and if it does, it reboots your computer after a certain
- amount of time.
-
- To compile this driver as a module, choose M here: the
- module will be called w83697ug_wdt.
-
- Most people will say N.
-
config W83877F_WDT
tristate "W83877F (EMACS) Watchdog Timer"
depends on X86
diff --git a/drivers/watchdog/Makefile b/drivers/watchdog/Makefile
index 1b5f3d5..7b8a91e 100644
--- a/drivers/watchdog/Makefile
+++ b/drivers/watchdog/Makefile
@@ -107,13 +107,12 @@
obj-$(CONFIG_SMSC37B787_WDT) += smsc37b787_wdt.o
obj-$(CONFIG_VIA_WDT) += via_wdt.o
obj-$(CONFIG_W83627HF_WDT) += w83627hf_wdt.o
-obj-$(CONFIG_W83697HF_WDT) += w83697hf_wdt.o
-obj-$(CONFIG_W83697UG_WDT) += w83697ug_wdt.o
obj-$(CONFIG_W83877F_WDT) += w83877f_wdt.o
obj-$(CONFIG_W83977F_WDT) += w83977f_wdt.o
obj-$(CONFIG_MACHZ_WDT) += machzwd.o
obj-$(CONFIG_SBC_EPX_C3_WATCHDOG) += sbc_epx_c3.o
obj-$(CONFIG_INTEL_SCU_WATCHDOG) += intel_scu_watchdog.o
+obj-$(CONFIG_INTEL_MID_WATCHDOG) += intel-mid_wdt.o
# M32R Architecture
diff --git a/drivers/watchdog/ath79_wdt.c b/drivers/watchdog/ath79_wdt.c
index 399c3fd..41ac466 100644
--- a/drivers/watchdog/ath79_wdt.c
+++ b/drivers/watchdog/ath79_wdt.c
@@ -20,6 +20,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/bitops.h>
+#include <linux/delay.h>
#include <linux/errno.h>
#include <linux/fs.h>
#include <linux/io.h>
@@ -90,6 +91,15 @@
static inline void ath79_wdt_enable(void)
{
ath79_wdt_keepalive();
+
+ /*
+ * Updating the TIMER register requires a few microseconds
+ * on the AR934x SoCs at least. Use a small delay to ensure
+ * that the TIMER register is updated within the hardware
+ * before enabling the watchdog.
+ */
+ udelay(2);
+
ath79_wdt_wr(WDOG_REG_CTRL, WDOG_CTRL_ACTION_FCR);
/* flush write */
ath79_wdt_rr(WDOG_REG_CTRL);
@@ -255,7 +265,7 @@
if (IS_ERR(wdt_clk))
return PTR_ERR(wdt_clk);
- err = clk_enable(wdt_clk);
+ err = clk_prepare_enable(wdt_clk);
if (err)
return err;
@@ -286,14 +296,14 @@
return 0;
err_clk_disable:
- clk_disable(wdt_clk);
+ clk_disable_unprepare(wdt_clk);
return err;
}
static int ath79_wdt_remove(struct platform_device *pdev)
{
misc_deregister(&ath79_wdt_miscdev);
- clk_disable(wdt_clk);
+ clk_disable_unprepare(wdt_clk);
return 0;
}
diff --git a/drivers/watchdog/booke_wdt.c b/drivers/watchdog/booke_wdt.c
index a8dbceb3..08a7853 100644
--- a/drivers/watchdog/booke_wdt.c
+++ b/drivers/watchdog/booke_wdt.c
@@ -41,6 +41,28 @@
#define WDTP_MASK (TCR_WP_MASK)
#endif
+/* Checks wdt=x and wdt_period=xx command-line option */
+notrace int __init early_parse_wdt(char *p)
+{
+ if (p && strncmp(p, "0", 1) != 0)
+ booke_wdt_enabled = 1;
+
+ return 0;
+}
+early_param("wdt", early_parse_wdt);
+
+int __init early_parse_wdt_period(char *p)
+{
+ unsigned long ret;
+ if (p) {
+ if (!kstrtol(p, 0, &ret))
+ booke_wdt_period = ret;
+ }
+
+ return 0;
+}
+early_param("wdt_period", early_parse_wdt_period);
+
#ifdef CONFIG_PPC_FSL_BOOK3E
/* For the specified period, determine the number of seconds
@@ -103,17 +125,18 @@
static void __booke_wdt_set(void *data)
{
u32 val;
+ struct watchdog_device *wdog = data;
val = mfspr(SPRN_TCR);
val &= ~WDTP_MASK;
- val |= WDTP(booke_wdt_period);
+ val |= WDTP(sec_to_period(wdog->timeout));
mtspr(SPRN_TCR, val);
}
-static void booke_wdt_set(void)
+static void booke_wdt_set(void *data)
{
- on_each_cpu(__booke_wdt_set, NULL, 0);
+ on_each_cpu(__booke_wdt_set, data, 0);
}
static void __booke_wdt_ping(void *data)
@@ -131,12 +154,13 @@
static void __booke_wdt_enable(void *data)
{
u32 val;
+ struct watchdog_device *wdog = data;
/* clear status before enabling watchdog */
__booke_wdt_ping(NULL);
val = mfspr(SPRN_TCR);
val &= ~WDTP_MASK;
- val |= (TCR_WIE|TCR_WRC(WRC_CHIP)|WDTP(booke_wdt_period));
+ val |= (TCR_WIE|TCR_WRC(WRC_CHIP)|WDTP(sec_to_period(wdog->timeout)));
mtspr(SPRN_TCR, val);
}
@@ -162,25 +186,17 @@
}
-static void __booke_wdt_start(struct watchdog_device *wdog)
-{
- on_each_cpu(__booke_wdt_enable, NULL, 0);
- pr_debug("watchdog enabled (timeout = %u sec)\n", wdog->timeout);
-}
-
static int booke_wdt_start(struct watchdog_device *wdog)
{
- if (booke_wdt_enabled == 0) {
- booke_wdt_enabled = 1;
- __booke_wdt_start(wdog);
- }
+ on_each_cpu(__booke_wdt_enable, wdog, 0);
+ pr_debug("watchdog enabled (timeout = %u sec)\n", wdog->timeout);
+
return 0;
}
static int booke_wdt_stop(struct watchdog_device *wdog)
{
on_each_cpu(__booke_wdt_disable, NULL, 0);
- booke_wdt_enabled = 0;
pr_debug("watchdog disabled\n");
return 0;
@@ -191,9 +207,8 @@
{
if (timeout > MAX_WDT_TIMEOUT)
return -EINVAL;
- booke_wdt_period = sec_to_period(timeout);
wdt_dev->timeout = timeout;
- booke_wdt_set();
+ booke_wdt_set(wdt_dev);
return 0;
}
@@ -231,10 +246,10 @@
pr_info("powerpc book-e watchdog driver loaded\n");
booke_wdt_info.firmware_version = cur_cpu_spec->pvr_value;
booke_wdt_set_timeout(&booke_wdt_dev,
- period_to_sec(CONFIG_BOOKE_WDT_DEFAULT_TIMEOUT));
+ period_to_sec(booke_wdt_period));
watchdog_set_nowayout(&booke_wdt_dev, nowayout);
if (booke_wdt_enabled)
- __booke_wdt_start(&booke_wdt_dev);
+ booke_wdt_start(&booke_wdt_dev);
ret = watchdog_register_device(&booke_wdt_dev);
diff --git a/drivers/watchdog/imx2_wdt.c b/drivers/watchdog/imx2_wdt.c
index dd51d95..9d4874f 100644
--- a/drivers/watchdog/imx2_wdt.c
+++ b/drivers/watchdog/imx2_wdt.c
@@ -21,19 +21,17 @@
* Halt on suspend: Manual Can be automatic
*/
+#include <linux/clk.h>
#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/jiffies.h>
#include <linux/kernel.h>
-#include <linux/miscdevice.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/platform_device.h>
-#include <linux/watchdog.h>
-#include <linux/clk.h>
-#include <linux/fs.h>
-#include <linux/io.h>
-#include <linux/uaccess.h>
+#include <linux/regmap.h>
#include <linux/timer.h>
-#include <linux/jiffies.h>
+#include <linux/watchdog.h>
#define DRIVER_NAME "imx2-wdt"
@@ -55,19 +53,12 @@
#define WDOG_SEC_TO_COUNT(s) ((s * 2 - 1) << 8)
-#define IMX2_WDT_STATUS_OPEN 0
-#define IMX2_WDT_STATUS_STARTED 1
-#define IMX2_WDT_EXPECT_CLOSE 2
-
-static struct {
+struct imx2_wdt_device {
struct clk *clk;
- void __iomem *base;
- unsigned timeout;
- unsigned long status;
+ struct regmap *regmap;
struct timer_list timer; /* Pings the watchdog when closed */
-} imx2_wdt;
-
-static struct miscdevice imx2_wdt_miscdev;
+ struct watchdog_device wdog;
+};
static bool nowayout = WATCHDOG_NOWAYOUT;
module_param(nowayout, bool, 0);
@@ -85,9 +76,12 @@
.options = WDIOF_KEEPALIVEPING | WDIOF_SETTIMEOUT | WDIOF_MAGICCLOSE,
};
-static inline void imx2_wdt_setup(void)
+static inline void imx2_wdt_setup(struct watchdog_device *wdog)
{
- u16 val = __raw_readw(imx2_wdt.base + IMX2_WDT_WCR);
+ struct imx2_wdt_device *wdev = watchdog_get_drvdata(wdog);
+ u32 val;
+
+ regmap_read(wdev->regmap, IMX2_WDT_WCR, &val);
/* Suspend timer in low power mode, write once-only */
val |= IMX2_WDT_WCR_WDZST;
@@ -98,227 +92,199 @@
/* Keep Watchdog Disabled */
val &= ~IMX2_WDT_WCR_WDE;
/* Set the watchdog's Time-Out value */
- val |= WDOG_SEC_TO_COUNT(imx2_wdt.timeout);
+ val |= WDOG_SEC_TO_COUNT(wdog->timeout);
- __raw_writew(val, imx2_wdt.base + IMX2_WDT_WCR);
+ regmap_write(wdev->regmap, IMX2_WDT_WCR, val);
/* enable the watchdog */
val |= IMX2_WDT_WCR_WDE;
- __raw_writew(val, imx2_wdt.base + IMX2_WDT_WCR);
+ regmap_write(wdev->regmap, IMX2_WDT_WCR, val);
}
-static inline void imx2_wdt_ping(void)
+static inline bool imx2_wdt_is_running(struct imx2_wdt_device *wdev)
{
- __raw_writew(IMX2_WDT_SEQ1, imx2_wdt.base + IMX2_WDT_WSR);
- __raw_writew(IMX2_WDT_SEQ2, imx2_wdt.base + IMX2_WDT_WSR);
+ u32 val;
+
+ regmap_read(wdev->regmap, IMX2_WDT_WCR, &val);
+
+ return val & IMX2_WDT_WCR_WDE;
+}
+
+static int imx2_wdt_ping(struct watchdog_device *wdog)
+{
+ struct imx2_wdt_device *wdev = watchdog_get_drvdata(wdog);
+
+ regmap_write(wdev->regmap, IMX2_WDT_WSR, IMX2_WDT_SEQ1);
+ regmap_write(wdev->regmap, IMX2_WDT_WSR, IMX2_WDT_SEQ2);
+ return 0;
}
static void imx2_wdt_timer_ping(unsigned long arg)
{
- /* ping it every imx2_wdt.timeout / 2 seconds to prevent reboot */
- imx2_wdt_ping();
- mod_timer(&imx2_wdt.timer, jiffies + imx2_wdt.timeout * HZ / 2);
+ struct watchdog_device *wdog = (struct watchdog_device *)arg;
+ struct imx2_wdt_device *wdev = watchdog_get_drvdata(wdog);
+
+ /* ping it every wdog->timeout / 2 seconds to prevent reboot */
+ imx2_wdt_ping(wdog);
+ mod_timer(&wdev->timer, jiffies + wdog->timeout * HZ / 2);
}
-static void imx2_wdt_start(void)
+static int imx2_wdt_set_timeout(struct watchdog_device *wdog,
+ unsigned int new_timeout)
{
- if (!test_and_set_bit(IMX2_WDT_STATUS_STARTED, &imx2_wdt.status)) {
- /* at our first start we enable clock and do initialisations */
- clk_prepare_enable(imx2_wdt.clk);
+ struct imx2_wdt_device *wdev = watchdog_get_drvdata(wdog);
- imx2_wdt_setup();
- } else /* delete the timer that pings the watchdog after close */
- del_timer_sync(&imx2_wdt.timer);
-
- /* Watchdog is enabled - time to reload the timeout value */
- imx2_wdt_ping();
-}
-
-static void imx2_wdt_stop(void)
-{
- /* we don't need a clk_disable, it cannot be disabled once started.
- * We use a timer to ping the watchdog while /dev/watchdog is closed */
- imx2_wdt_timer_ping(0);
-}
-
-static void imx2_wdt_set_timeout(int new_timeout)
-{
- u16 val = __raw_readw(imx2_wdt.base + IMX2_WDT_WCR);
-
- /* set the new timeout value in the WSR */
- val &= ~IMX2_WDT_WCR_WT;
- val |= WDOG_SEC_TO_COUNT(new_timeout);
- __raw_writew(val, imx2_wdt.base + IMX2_WDT_WCR);
-}
-
-static int imx2_wdt_open(struct inode *inode, struct file *file)
-{
- if (test_and_set_bit(IMX2_WDT_STATUS_OPEN, &imx2_wdt.status))
- return -EBUSY;
-
- imx2_wdt_start();
- return nonseekable_open(inode, file);
-}
-
-static int imx2_wdt_close(struct inode *inode, struct file *file)
-{
- if (test_bit(IMX2_WDT_EXPECT_CLOSE, &imx2_wdt.status) && !nowayout)
- imx2_wdt_stop();
- else {
- dev_crit(imx2_wdt_miscdev.parent,
- "Unexpected close: Expect reboot!\n");
- imx2_wdt_ping();
- }
-
- clear_bit(IMX2_WDT_EXPECT_CLOSE, &imx2_wdt.status);
- clear_bit(IMX2_WDT_STATUS_OPEN, &imx2_wdt.status);
+ regmap_update_bits(wdev->regmap, IMX2_WDT_WCR, IMX2_WDT_WCR_WT,
+ WDOG_SEC_TO_COUNT(new_timeout));
return 0;
}
-static long imx2_wdt_ioctl(struct file *file, unsigned int cmd,
- unsigned long arg)
+static int imx2_wdt_start(struct watchdog_device *wdog)
{
- void __user *argp = (void __user *)arg;
- int __user *p = argp;
- int new_value;
- u16 val;
+ struct imx2_wdt_device *wdev = watchdog_get_drvdata(wdog);
- switch (cmd) {
- case WDIOC_GETSUPPORT:
- return copy_to_user(argp, &imx2_wdt_info,
- sizeof(struct watchdog_info)) ? -EFAULT : 0;
+ if (imx2_wdt_is_running(wdev)) {
+ /* delete the timer that pings the watchdog after close */
+ del_timer_sync(&wdev->timer);
+ imx2_wdt_set_timeout(wdog, wdog->timeout);
+ } else
+ imx2_wdt_setup(wdog);
- case WDIOC_GETSTATUS:
- return put_user(0, p);
+ return imx2_wdt_ping(wdog);
+}
- case WDIOC_GETBOOTSTATUS:
- val = __raw_readw(imx2_wdt.base + IMX2_WDT_WRSR);
- new_value = val & IMX2_WDT_WRSR_TOUT ? WDIOF_CARDRESET : 0;
- return put_user(new_value, p);
+static int imx2_wdt_stop(struct watchdog_device *wdog)
+{
+ /*
+ * We don't need a clk_disable, it cannot be disabled once started.
+ * We use a timer to ping the watchdog while /dev/watchdog is closed
+ */
+ imx2_wdt_timer_ping((unsigned long)wdog);
+ return 0;
+}
- case WDIOC_KEEPALIVE:
- imx2_wdt_ping();
- return 0;
+static inline void imx2_wdt_ping_if_active(struct watchdog_device *wdog)
+{
+ struct imx2_wdt_device *wdev = watchdog_get_drvdata(wdog);
- case WDIOC_SETTIMEOUT:
- if (get_user(new_value, p))
- return -EFAULT;
- if ((new_value < 1) || (new_value > IMX2_WDT_MAX_TIME))
- return -EINVAL;
- imx2_wdt_set_timeout(new_value);
- imx2_wdt.timeout = new_value;
- imx2_wdt_ping();
-
- /* Fallthrough to return current value */
- case WDIOC_GETTIMEOUT:
- return put_user(imx2_wdt.timeout, p);
-
- default:
- return -ENOTTY;
+ if (imx2_wdt_is_running(wdev)) {
+ imx2_wdt_set_timeout(wdog, wdog->timeout);
+ imx2_wdt_timer_ping((unsigned long)wdog);
}
}
-static ssize_t imx2_wdt_write(struct file *file, const char __user *data,
- size_t len, loff_t *ppos)
-{
- size_t i;
- char c;
-
- if (len == 0) /* Can we see this even ? */
- return 0;
-
- clear_bit(IMX2_WDT_EXPECT_CLOSE, &imx2_wdt.status);
- /* scan to see whether or not we got the magic character */
- for (i = 0; i != len; i++) {
- if (get_user(c, data + i))
- return -EFAULT;
- if (c == 'V')
- set_bit(IMX2_WDT_EXPECT_CLOSE, &imx2_wdt.status);
- }
-
- imx2_wdt_ping();
- return len;
-}
-
-static const struct file_operations imx2_wdt_fops = {
+static struct watchdog_ops imx2_wdt_ops = {
.owner = THIS_MODULE,
- .llseek = no_llseek,
- .unlocked_ioctl = imx2_wdt_ioctl,
- .open = imx2_wdt_open,
- .release = imx2_wdt_close,
- .write = imx2_wdt_write,
+ .start = imx2_wdt_start,
+ .stop = imx2_wdt_stop,
+ .ping = imx2_wdt_ping,
+ .set_timeout = imx2_wdt_set_timeout,
};
-static struct miscdevice imx2_wdt_miscdev = {
- .minor = WATCHDOG_MINOR,
- .name = "watchdog",
- .fops = &imx2_wdt_fops,
+static struct regmap_config imx2_wdt_regmap_config = {
+ .reg_bits = 16,
+ .reg_stride = 2,
+ .val_bits = 16,
+ .max_register = 0x8,
};
static int __init imx2_wdt_probe(struct platform_device *pdev)
{
- int ret;
+ struct imx2_wdt_device *wdev;
+ struct watchdog_device *wdog;
struct resource *res;
+ void __iomem *base;
+ int ret;
+ u32 val;
+
+ wdev = devm_kzalloc(&pdev->dev, sizeof(*wdev), GFP_KERNEL);
+ if (!wdev)
+ return -ENOMEM;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- imx2_wdt.base = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(imx2_wdt.base))
- return PTR_ERR(imx2_wdt.base);
+ base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
- imx2_wdt.clk = devm_clk_get(&pdev->dev, NULL);
- if (IS_ERR(imx2_wdt.clk)) {
- dev_err(&pdev->dev, "can't get Watchdog clock\n");
- return PTR_ERR(imx2_wdt.clk);
+ wdev->regmap = devm_regmap_init_mmio_clk(&pdev->dev, NULL, base,
+ &imx2_wdt_regmap_config);
+ if (IS_ERR(wdev->regmap)) {
+ dev_err(&pdev->dev, "regmap init failed\n");
+ return PTR_ERR(wdev->regmap);
}
- imx2_wdt.timeout = clamp_t(unsigned, timeout, 1, IMX2_WDT_MAX_TIME);
- if (imx2_wdt.timeout != timeout)
- dev_warn(&pdev->dev, "Initial timeout out of range! "
- "Clamped from %u to %u\n", timeout, imx2_wdt.timeout);
+ wdev->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(wdev->clk)) {
+ dev_err(&pdev->dev, "can't get Watchdog clock\n");
+ return PTR_ERR(wdev->clk);
+ }
- setup_timer(&imx2_wdt.timer, imx2_wdt_timer_ping, 0);
+ wdog = &wdev->wdog;
+ wdog->info = &imx2_wdt_info;
+ wdog->ops = &imx2_wdt_ops;
+ wdog->min_timeout = 1;
+ wdog->max_timeout = IMX2_WDT_MAX_TIME;
- imx2_wdt_miscdev.parent = &pdev->dev;
- ret = misc_register(&imx2_wdt_miscdev);
- if (ret)
- goto fail;
+ clk_prepare_enable(wdev->clk);
- dev_info(&pdev->dev,
- "IMX2+ Watchdog Timer enabled. timeout=%ds (nowayout=%d)\n",
- imx2_wdt.timeout, nowayout);
+ regmap_read(wdev->regmap, IMX2_WDT_WRSR, &val);
+ wdog->bootstatus = val & IMX2_WDT_WRSR_TOUT ? WDIOF_CARDRESET : 0;
+
+ wdog->timeout = clamp_t(unsigned, timeout, 1, IMX2_WDT_MAX_TIME);
+ if (wdog->timeout != timeout)
+ dev_warn(&pdev->dev, "Initial timeout out of range! Clamped from %u to %u\n",
+ timeout, wdog->timeout);
+
+ platform_set_drvdata(pdev, wdog);
+ watchdog_set_drvdata(wdog, wdev);
+ watchdog_set_nowayout(wdog, nowayout);
+ watchdog_init_timeout(wdog, timeout, &pdev->dev);
+
+ setup_timer(&wdev->timer, imx2_wdt_timer_ping, (unsigned long)wdog);
+
+ imx2_wdt_ping_if_active(wdog);
+
+ ret = watchdog_register_device(wdog);
+ if (ret) {
+ dev_err(&pdev->dev, "cannot register watchdog device\n");
+ return ret;
+ }
+
+ dev_info(&pdev->dev, "timeout %d sec (nowayout=%d)\n",
+ wdog->timeout, nowayout);
+
return 0;
-
-fail:
- imx2_wdt_miscdev.parent = NULL;
- return ret;
}
static int __exit imx2_wdt_remove(struct platform_device *pdev)
{
- misc_deregister(&imx2_wdt_miscdev);
+ struct watchdog_device *wdog = platform_get_drvdata(pdev);
+ struct imx2_wdt_device *wdev = watchdog_get_drvdata(wdog);
- if (test_bit(IMX2_WDT_STATUS_STARTED, &imx2_wdt.status)) {
- del_timer_sync(&imx2_wdt.timer);
+ watchdog_unregister_device(wdog);
- dev_crit(imx2_wdt_miscdev.parent,
- "Device removed: Expect reboot!\n");
+ if (imx2_wdt_is_running(wdev)) {
+ del_timer_sync(&wdev->timer);
+ imx2_wdt_ping(wdog);
+ dev_crit(&pdev->dev, "Device removed: Expect reboot!\n");
}
-
- imx2_wdt_miscdev.parent = NULL;
return 0;
}
static void imx2_wdt_shutdown(struct platform_device *pdev)
{
- if (test_bit(IMX2_WDT_STATUS_STARTED, &imx2_wdt.status)) {
- /* we are running, we need to delete the timer but will give
- * max timeout before reboot will take place */
- del_timer_sync(&imx2_wdt.timer);
- imx2_wdt_set_timeout(IMX2_WDT_MAX_TIME);
- imx2_wdt_ping();
+ struct watchdog_device *wdog = platform_get_drvdata(pdev);
+ struct imx2_wdt_device *wdev = watchdog_get_drvdata(wdog);
- dev_crit(imx2_wdt_miscdev.parent,
- "Device shutdown: Expect reboot!\n");
+ if (imx2_wdt_is_running(wdev)) {
+ /*
+ * We are running, we need to delete the timer but will
+ * give max timeout before reboot will take place
+ */
+ del_timer_sync(&wdev->timer);
+ imx2_wdt_set_timeout(wdog, IMX2_WDT_MAX_TIME);
+ imx2_wdt_ping(wdog);
+ dev_crit(&pdev->dev, "Device shutdown: Expect reboot!\n");
}
}
diff --git a/drivers/watchdog/intel-mid_wdt.c b/drivers/watchdog/intel-mid_wdt.c
new file mode 100644
index 0000000..ca66e8e
--- /dev/null
+++ b/drivers/watchdog/intel-mid_wdt.c
@@ -0,0 +1,184 @@
+/*
+ * intel-mid_wdt: generic Intel MID SCU watchdog driver
+ *
+ * Platforms supported so far:
+ * - Merrifield only
+ *
+ * Copyright (C) 2014 Intel Corporation. All rights reserved.
+ * Contact: David Cohen <david.a.cohen@linux.intel.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of version 2 of the GNU General
+ * Public License as published by the Free Software Foundation.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/nmi.h>
+#include <linux/platform_device.h>
+#include <linux/watchdog.h>
+#include <linux/platform_data/intel-mid_wdt.h>
+
+#include <asm/intel_scu_ipc.h>
+#include <asm/intel-mid.h>
+
+#define IPC_WATCHDOG 0xf8
+
+#define MID_WDT_PRETIMEOUT 15
+#define MID_WDT_TIMEOUT_MIN (1 + MID_WDT_PRETIMEOUT)
+#define MID_WDT_TIMEOUT_MAX 170
+#define MID_WDT_DEFAULT_TIMEOUT 90
+
+/* SCU watchdog messages */
+enum {
+ SCU_WATCHDOG_START = 0,
+ SCU_WATCHDOG_STOP,
+ SCU_WATCHDOG_KEEPALIVE,
+};
+
+static inline int wdt_command(int sub, u32 *in, int inlen)
+{
+ return intel_scu_ipc_command(IPC_WATCHDOG, sub, in, inlen, NULL, 0);
+}
+
+static int wdt_start(struct watchdog_device *wd)
+{
+ int ret, in_size;
+ int timeout = wd->timeout;
+ struct ipc_wd_start {
+ u32 pretimeout;
+ u32 timeout;
+ } ipc_wd_start = { timeout - MID_WDT_PRETIMEOUT, timeout };
+
+ /*
+ * SCU expects the input size for watchdog IPC to
+ * be based on 4 bytes
+ */
+ in_size = DIV_ROUND_UP(sizeof(ipc_wd_start), 4);
+
+ ret = wdt_command(SCU_WATCHDOG_START, (u32 *)&ipc_wd_start, in_size);
+ if (ret) {
+ struct device *dev = watchdog_get_drvdata(wd);
+ dev_crit(dev, "error starting watchdog: %d\n", ret);
+ }
+
+ return ret;
+}
+
+static int wdt_ping(struct watchdog_device *wd)
+{
+ int ret;
+
+ ret = wdt_command(SCU_WATCHDOG_KEEPALIVE, NULL, 0);
+ if (ret) {
+ struct device *dev = watchdog_get_drvdata(wd);
+ dev_crit(dev, "Error executing keepalive: 0x%x\n", ret);
+ }
+
+ return ret;
+}
+
+static int wdt_stop(struct watchdog_device *wd)
+{
+ int ret;
+
+ ret = wdt_command(SCU_WATCHDOG_STOP, NULL, 0);
+ if (ret) {
+ struct device *dev = watchdog_get_drvdata(wd);
+ dev_crit(dev, "Error stopping watchdog: 0x%x\n", ret);
+ }
+
+ return ret;
+}
+
+static irqreturn_t mid_wdt_irq(int irq, void *dev_id)
+{
+ panic("Kernel Watchdog");
+
+ /* This code should not be reached */
+ return IRQ_HANDLED;
+}
+
+static const struct watchdog_info mid_wdt_info = {
+ .identity = "Intel MID SCU watchdog",
+ .options = WDIOF_KEEPALIVEPING | WDIOF_SETTIMEOUT,
+};
+
+static const struct watchdog_ops mid_wdt_ops = {
+ .owner = THIS_MODULE,
+ .start = wdt_start,
+ .stop = wdt_stop,
+ .ping = wdt_ping,
+};
+
+static int mid_wdt_probe(struct platform_device *pdev)
+{
+ struct watchdog_device *wdt_dev;
+ struct intel_mid_wdt_pdata *pdata = pdev->dev.platform_data;
+ int ret;
+
+ if (!pdata) {
+ dev_err(&pdev->dev, "missing platform data\n");
+ return -EINVAL;
+ }
+
+ if (pdata->probe) {
+ ret = pdata->probe(pdev);
+ if (ret)
+ return ret;
+ }
+
+ wdt_dev = devm_kzalloc(&pdev->dev, sizeof(*wdt_dev), GFP_KERNEL);
+ if (!wdt_dev)
+ return -ENOMEM;
+
+ wdt_dev->info = &mid_wdt_info;
+ wdt_dev->ops = &mid_wdt_ops;
+ wdt_dev->min_timeout = MID_WDT_TIMEOUT_MIN;
+ wdt_dev->max_timeout = MID_WDT_TIMEOUT_MAX;
+ wdt_dev->timeout = MID_WDT_DEFAULT_TIMEOUT;
+
+ watchdog_set_drvdata(wdt_dev, &pdev->dev);
+ platform_set_drvdata(pdev, wdt_dev);
+
+ ret = devm_request_irq(&pdev->dev, pdata->irq, mid_wdt_irq,
+ IRQF_SHARED | IRQF_NO_SUSPEND, "watchdog",
+ wdt_dev);
+ if (ret) {
+ dev_err(&pdev->dev, "error requesting warning irq %d\n",
+ pdata->irq);
+ return ret;
+ }
+
+ ret = watchdog_register_device(wdt_dev);
+ if (ret) {
+ dev_err(&pdev->dev, "error registering watchdog device\n");
+ return ret;
+ }
+
+ dev_info(&pdev->dev, "Intel MID watchdog device probed\n");
+
+ return 0;
+}
+
+static int mid_wdt_remove(struct platform_device *pdev)
+{
+ struct watchdog_device *wd = platform_get_drvdata(pdev);
+ watchdog_unregister_device(wd);
+ return 0;
+}
+
+static struct platform_driver mid_wdt_driver = {
+ .probe = mid_wdt_probe,
+ .remove = mid_wdt_remove,
+ .driver = {
+ .owner = THIS_MODULE,
+ .name = "intel_mid_wdt",
+ },
+};
+
+module_platform_driver(mid_wdt_driver);
+
+MODULE_AUTHOR("David Cohen <david.a.cohen@linux.intel.com>");
+MODULE_DESCRIPTION("Watchdog Driver for Intel MID platform");
+MODULE_LICENSE("GPL");
diff --git a/drivers/watchdog/kempld_wdt.c b/drivers/watchdog/kempld_wdt.c
index 20dc738..d9c1a16 100644
--- a/drivers/watchdog/kempld_wdt.c
+++ b/drivers/watchdog/kempld_wdt.c
@@ -162,7 +162,7 @@
kempld_get_mutex(pld);
stage_cfg = kempld_read8(pld, KEMPLD_WDT_STAGE_CFG(stage->id));
stage_cfg &= ~STAGE_CFG_PRESCALER_MASK;
- stage_cfg |= STAGE_CFG_SET_PRESCALER(prescaler);
+ stage_cfg |= STAGE_CFG_SET_PRESCALER(PRESCALER_21);
kempld_write8(pld, KEMPLD_WDT_STAGE_CFG(stage->id), stage_cfg);
kempld_write32(pld, KEMPLD_WDT_STAGE_TIMEOUT(stage->id),
stage_timeout);
diff --git a/drivers/watchdog/of_xilinx_wdt.c b/drivers/watchdog/of_xilinx_wdt.c
index 57ccae8..1e6e28d 100644
--- a/drivers/watchdog/of_xilinx_wdt.c
+++ b/drivers/watchdog/of_xilinx_wdt.c
@@ -225,7 +225,7 @@
}
/* Match table for of_platform binding */
-static struct of_device_id xwdt_of_match[] = {
+static const struct of_device_id xwdt_of_match[] = {
{ .compatible = "xlnx,xps-timebase-wdt-1.00.a", },
{ .compatible = "xlnx,xps-timebase-wdt-1.01.a", },
{},
diff --git a/drivers/watchdog/orion_wdt.c b/drivers/watchdog/orion_wdt.c
index 9b3c41d..00d0741 100644
--- a/drivers/watchdog/orion_wdt.c
+++ b/drivers/watchdog/orion_wdt.c
@@ -55,15 +55,19 @@
int wdt_counter_offset;
int wdt_enable_bit;
int rstout_enable_bit;
+ int rstout_mask_bit;
int (*clock_init)(struct platform_device *,
struct orion_watchdog *);
+ int (*enabled)(struct orion_watchdog *);
int (*start)(struct watchdog_device *);
+ int (*stop)(struct watchdog_device *);
};
struct orion_watchdog {
struct watchdog_device wdt;
void __iomem *reg;
void __iomem *rstout;
+ void __iomem *rstout_mask;
unsigned long clk_rate;
struct clk *clk;
const struct orion_watchdog_data *data;
@@ -142,9 +146,10 @@
return 0;
}
-static int armada370_start(struct watchdog_device *wdt_dev)
+static int armada375_start(struct watchdog_device *wdt_dev)
{
struct orion_watchdog *dev = watchdog_get_drvdata(wdt_dev);
+ u32 reg;
/* Set watchdog duration */
writel(dev->clk_rate * wdt_dev->timeout,
@@ -157,8 +162,35 @@
atomic_io_modify(dev->reg + TIMER_CTRL, dev->data->wdt_enable_bit,
dev->data->wdt_enable_bit);
- atomic_io_modify(dev->rstout, dev->data->rstout_enable_bit,
- dev->data->rstout_enable_bit);
+ /* Enable reset on watchdog */
+ reg = readl(dev->rstout);
+ reg |= dev->data->rstout_enable_bit;
+ writel(reg, dev->rstout);
+
+ atomic_io_modify(dev->rstout_mask, dev->data->rstout_mask_bit, 0);
+ return 0;
+}
+
+static int armada370_start(struct watchdog_device *wdt_dev)
+{
+ struct orion_watchdog *dev = watchdog_get_drvdata(wdt_dev);
+ u32 reg;
+
+ /* Set watchdog duration */
+ writel(dev->clk_rate * wdt_dev->timeout,
+ dev->reg + dev->data->wdt_counter_offset);
+
+ /* Clear the watchdog expiration bit */
+ atomic_io_modify(dev->reg + TIMER_A370_STATUS, WDT_A370_EXPIRED, 0);
+
+ /* Enable watchdog timer */
+ atomic_io_modify(dev->reg + TIMER_CTRL, dev->data->wdt_enable_bit,
+ dev->data->wdt_enable_bit);
+
+ /* Enable reset on watchdog */
+ reg = readl(dev->rstout);
+ reg |= dev->data->rstout_enable_bit;
+ writel(reg, dev->rstout);
return 0;
}
@@ -189,7 +221,7 @@
return dev->data->start(wdt_dev);
}
-static int orion_wdt_stop(struct watchdog_device *wdt_dev)
+static int orion_stop(struct watchdog_device *wdt_dev)
{
struct orion_watchdog *dev = watchdog_get_drvdata(wdt_dev);
@@ -202,7 +234,48 @@
return 0;
}
-static int orion_wdt_enabled(struct orion_watchdog *dev)
+static int armada375_stop(struct watchdog_device *wdt_dev)
+{
+ struct orion_watchdog *dev = watchdog_get_drvdata(wdt_dev);
+ u32 reg;
+
+ /* Disable reset on watchdog */
+ atomic_io_modify(dev->rstout_mask, dev->data->rstout_mask_bit,
+ dev->data->rstout_mask_bit);
+ reg = readl(dev->rstout);
+ reg &= ~dev->data->rstout_enable_bit;
+ writel(reg, dev->rstout);
+
+ /* Disable watchdog timer */
+ atomic_io_modify(dev->reg + TIMER_CTRL, dev->data->wdt_enable_bit, 0);
+
+ return 0;
+}
+
+static int armada370_stop(struct watchdog_device *wdt_dev)
+{
+ struct orion_watchdog *dev = watchdog_get_drvdata(wdt_dev);
+ u32 reg;
+
+ /* Disable reset on watchdog */
+ reg = readl(dev->rstout);
+ reg &= ~dev->data->rstout_enable_bit;
+ writel(reg, dev->rstout);
+
+ /* Disable watchdog timer */
+ atomic_io_modify(dev->reg + TIMER_CTRL, dev->data->wdt_enable_bit, 0);
+
+ return 0;
+}
+
+static int orion_wdt_stop(struct watchdog_device *wdt_dev)
+{
+ struct orion_watchdog *dev = watchdog_get_drvdata(wdt_dev);
+
+ return dev->data->stop(wdt_dev);
+}
+
+static int orion_enabled(struct orion_watchdog *dev)
{
bool enabled, running;
@@ -212,6 +285,24 @@
return enabled && running;
}
+static int armada375_enabled(struct orion_watchdog *dev)
+{
+ bool masked, enabled, running;
+
+ masked = readl(dev->rstout_mask) & dev->data->rstout_mask_bit;
+ enabled = readl(dev->rstout) & dev->data->rstout_enable_bit;
+ running = readl(dev->reg + TIMER_CTRL) & dev->data->wdt_enable_bit;
+
+ return !masked && enabled && running;
+}
+
+static int orion_wdt_enabled(struct watchdog_device *wdt_dev)
+{
+ struct orion_watchdog *dev = watchdog_get_drvdata(wdt_dev);
+
+ return dev->data->enabled(dev);
+}
+
static unsigned int orion_wdt_get_timeleft(struct watchdog_device *wdt_dev)
{
struct orion_watchdog *dev = watchdog_get_drvdata(wdt_dev);
@@ -262,10 +353,6 @@
return devm_ioremap(&pdev->dev, res->start,
resource_size(res));
- /* This workaround works only for "orion-wdt", DT-enabled */
- if (!of_device_is_compatible(pdev->dev.of_node, "marvell,orion-wdt"))
- return NULL;
-
rstout = internal_regs + ORION_RSTOUT_MASK_OFFSET;
WARN(1, FW_BUG "falling back to harcoded RSTOUT reg %pa\n", &rstout);
@@ -277,7 +364,9 @@
.wdt_enable_bit = BIT(4),
.wdt_counter_offset = 0x24,
.clock_init = orion_wdt_clock_init,
+ .enabled = orion_enabled,
.start = orion_start,
+ .stop = orion_stop,
};
static const struct orion_watchdog_data armada370_data = {
@@ -285,7 +374,9 @@
.wdt_enable_bit = BIT(8),
.wdt_counter_offset = 0x34,
.clock_init = armada370_wdt_clock_init,
+ .enabled = orion_enabled,
.start = armada370_start,
+ .stop = armada370_stop,
};
static const struct orion_watchdog_data armadaxp_data = {
@@ -293,7 +384,31 @@
.wdt_enable_bit = BIT(8),
.wdt_counter_offset = 0x34,
.clock_init = armadaxp_wdt_clock_init,
+ .enabled = orion_enabled,
.start = armada370_start,
+ .stop = armada370_stop,
+};
+
+static const struct orion_watchdog_data armada375_data = {
+ .rstout_enable_bit = BIT(8),
+ .rstout_mask_bit = BIT(10),
+ .wdt_enable_bit = BIT(8),
+ .wdt_counter_offset = 0x34,
+ .clock_init = armada370_wdt_clock_init,
+ .enabled = armada375_enabled,
+ .start = armada375_start,
+ .stop = armada375_stop,
+};
+
+static const struct orion_watchdog_data armada380_data = {
+ .rstout_enable_bit = BIT(8),
+ .rstout_mask_bit = BIT(10),
+ .wdt_enable_bit = BIT(8),
+ .wdt_counter_offset = 0x34,
+ .clock_init = armadaxp_wdt_clock_init,
+ .enabled = armada375_enabled,
+ .start = armada375_start,
+ .stop = armada375_stop,
};
static const struct of_device_id orion_wdt_of_match_table[] = {
@@ -309,16 +424,78 @@
.compatible = "marvell,armada-xp-wdt",
.data = &armadaxp_data,
},
+ {
+ .compatible = "marvell,armada-375-wdt",
+ .data = &armada375_data,
+ },
+ {
+ .compatible = "marvell,armada-380-wdt",
+ .data = &armada380_data,
+ },
{},
};
MODULE_DEVICE_TABLE(of, orion_wdt_of_match_table);
+static int orion_wdt_get_regs(struct platform_device *pdev,
+ struct orion_watchdog *dev)
+{
+ struct device_node *node = pdev->dev.of_node;
+ struct resource *res;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res)
+ return -ENODEV;
+ dev->reg = devm_ioremap(&pdev->dev, res->start,
+ resource_size(res));
+ if (!dev->reg)
+ return -ENOMEM;
+
+ /* Each supported compatible has some RSTOUT register quirk */
+ if (of_device_is_compatible(node, "marvell,orion-wdt")) {
+
+ dev->rstout = orion_wdt_ioremap_rstout(pdev, res->start &
+ INTERNAL_REGS_MASK);
+ if (!dev->rstout)
+ return -ENODEV;
+
+ } else if (of_device_is_compatible(node, "marvell,armada-370-wdt") ||
+ of_device_is_compatible(node, "marvell,armada-xp-wdt")) {
+
+ /* Dedicated RSTOUT register, can be requested. */
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+ dev->rstout = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(dev->rstout))
+ return PTR_ERR(dev->rstout);
+
+ } else if (of_device_is_compatible(node, "marvell,armada-375-wdt") ||
+ of_device_is_compatible(node, "marvell,armada-380-wdt")) {
+
+ /* Dedicated RSTOUT register, can be requested. */
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+ dev->rstout = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(dev->rstout))
+ return PTR_ERR(dev->rstout);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 2);
+ if (!res)
+ return -ENODEV;
+ dev->rstout_mask = devm_ioremap(&pdev->dev, res->start,
+ resource_size(res));
+ if (!dev->rstout_mask)
+ return -ENOMEM;
+
+ } else {
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
static int orion_wdt_probe(struct platform_device *pdev)
{
struct orion_watchdog *dev;
const struct of_device_id *match;
unsigned int wdt_max_duration; /* (seconds) */
- struct resource *res;
int ret, irq;
dev = devm_kzalloc(&pdev->dev, sizeof(struct orion_watchdog),
@@ -336,19 +513,9 @@
dev->wdt.min_timeout = 1;
dev->data = match->data;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (!res)
- return -ENODEV;
-
- dev->reg = devm_ioremap(&pdev->dev, res->start,
- resource_size(res));
- if (!dev->reg)
- return -ENOMEM;
-
- dev->rstout = orion_wdt_ioremap_rstout(pdev, res->start &
- INTERNAL_REGS_MASK);
- if (!dev->rstout)
- return -ENODEV;
+ ret = orion_wdt_get_regs(pdev, dev);
+ if (ret)
+ return ret;
ret = dev->data->clock_init(pdev, dev);
if (ret) {
@@ -371,7 +538,7 @@
* removed and re-insterted, or if the bootloader explicitly
* set a running watchdog before booting the kernel.
*/
- if (!orion_wdt_enabled(dev))
+ if (!orion_wdt_enabled(&dev->wdt))
orion_wdt_stop(&dev->wdt);
/* Request the IRQ only after the watchdog is disabled */
diff --git a/drivers/watchdog/shwdt.c b/drivers/watchdog/shwdt.c
index d04d02b..061756e 100644
--- a/drivers/watchdog/shwdt.c
+++ b/drivers/watchdog/shwdt.c
@@ -282,8 +282,6 @@
wdt->timer.data = (unsigned long)wdt;
wdt->timer.expires = next_ping_period(clock_division_ratio);
- platform_set_drvdata(pdev, wdt);
-
dev_info(&pdev->dev, "initialized.\n");
pm_runtime_enable(&pdev->dev);
diff --git a/drivers/watchdog/sp805_wdt.c b/drivers/watchdog/sp805_wdt.c
index 47629d2..c1b03f4 100644
--- a/drivers/watchdog/sp805_wdt.c
+++ b/drivers/watchdog/sp805_wdt.c
@@ -59,7 +59,6 @@
* @adev: amba device structure of wdt
* @status: current status of wdt
* @load_val: load value to be set for current timeout
- * @timeout: current programmed timeout
*/
struct sp805_wdt {
struct watchdog_device wdd;
@@ -68,7 +67,6 @@
struct clk *clk;
struct amba_device *adev;
unsigned int load_val;
- unsigned int timeout;
};
static bool nowayout = WATCHDOG_NOWAYOUT;
@@ -98,7 +96,7 @@
spin_lock(&wdt->lock);
wdt->load_val = load;
/* roundup timeout to closest positive integer value */
- wdt->timeout = div_u64((load + 1) * 2 + (rate / 2), rate);
+ wdd->timeout = div_u64((load + 1) * 2 + (rate / 2), rate);
spin_unlock(&wdt->lock);
return 0;
diff --git a/drivers/watchdog/sunxi_wdt.c b/drivers/watchdog/sunxi_wdt.c
index cd00a78..693b9d2c 100644
--- a/drivers/watchdog/sunxi_wdt.c
+++ b/drivers/watchdog/sunxi_wdt.c
@@ -57,17 +57,17 @@
*/
static const int wdt_timeout_map[] = {
- [1] = 0b0001, /* 1s */
- [2] = 0b0010, /* 2s */
- [3] = 0b0011, /* 3s */
- [4] = 0b0100, /* 4s */
- [5] = 0b0101, /* 5s */
- [6] = 0b0110, /* 6s */
- [8] = 0b0111, /* 8s */
- [10] = 0b1000, /* 10s */
- [12] = 0b1001, /* 12s */
- [14] = 0b1010, /* 14s */
- [16] = 0b1011, /* 16s */
+ [1] = 0x1, /* 1s */
+ [2] = 0x2, /* 2s */
+ [3] = 0x3, /* 3s */
+ [4] = 0x4, /* 4s */
+ [5] = 0x5, /* 5s */
+ [6] = 0x6, /* 6s */
+ [8] = 0x7, /* 8s */
+ [10] = 0x8, /* 10s */
+ [12] = 0x9, /* 12s */
+ [14] = 0xA, /* 14s */
+ [16] = 0xB, /* 16s */
};
static int sunxi_wdt_ping(struct watchdog_device *wdt_dev)
diff --git a/drivers/watchdog/via_wdt.c b/drivers/watchdog/via_wdt.c
index d2cd9f0..56369c4 100644
--- a/drivers/watchdog/via_wdt.c
+++ b/drivers/watchdog/via_wdt.c
@@ -232,7 +232,7 @@
static void wdt_remove(struct pci_dev *pdev)
{
watchdog_unregister_device(&wdt_dev);
- del_timer(&timer);
+ del_timer_sync(&timer);
iounmap(wdt_mem);
release_mem_region(mmio, VIA_WDT_MMIO_LEN);
release_resource(&wdt_res);
diff --git a/drivers/watchdog/w83627hf_wdt.c b/drivers/watchdog/w83627hf_wdt.c
index b1da0c1..7165704 100644
--- a/drivers/watchdog/w83627hf_wdt.c
+++ b/drivers/watchdog/w83627hf_wdt.c
@@ -64,6 +64,10 @@
"Watchdog cannot be stopped once started (default="
__MODULE_STRING(WATCHDOG_NOWAYOUT) ")");
+static int early_disable;
+module_param(early_disable, int, 0);
+MODULE_PARM_DESC(early_disable, "Disable watchdog at boot time (default=0)");
+
/*
* Kernel methods.
*/
@@ -208,9 +212,14 @@
t = superio_inb(cr_wdt_timeout);
if (t != 0) {
- pr_info("Watchdog already running. Resetting timeout to %d sec\n",
- wdog->timeout);
- superio_outb(cr_wdt_timeout, wdog->timeout);
+ if (early_disable) {
+ pr_warn("Stopping previously enabled watchdog until userland kicks in\n");
+ superio_outb(cr_wdt_timeout, 0);
+ } else {
+ pr_info("Watchdog already running. Resetting timeout to %d sec\n",
+ wdog->timeout);
+ superio_outb(cr_wdt_timeout, wdog->timeout);
+ }
}
/* set second mode & disable keyboard turning off watchdog */
diff --git a/drivers/watchdog/w83697hf_wdt.c b/drivers/watchdog/w83697hf_wdt.c
deleted file mode 100644
index e9ea856..0000000
--- a/drivers/watchdog/w83697hf_wdt.c
+++ /dev/null
@@ -1,460 +0,0 @@
-/*
- * w83697hf/hg WDT driver
- *
- * (c) Copyright 2006 Samuel Tardieu <sam@rfc1149.net>
- * (c) Copyright 2006 Marcus Junker <junker@anduras.de>
- *
- * Based on w83627hf_wdt.c which is based on advantechwdt.c
- * which is based on wdt.c.
- * Original copyright messages:
- *
- * (c) Copyright 2003 Pádraig Brady <P@draigBrady.com>
- *
- * (c) Copyright 2000-2001 Marek Michalkiewicz <marekm@linux.org.pl>
- *
- * (c) Copyright 1996 Alan Cox <alan@lxorguk.ukuu.org.uk>,
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
- * Neither Marcus Junker nor ANDURAS AG admit liability nor provide
- * warranty for any of this software. This material is provided
- * "AS-IS" and at no charge.
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <linux/module.h>
-#include <linux/moduleparam.h>
-#include <linux/types.h>
-#include <linux/miscdevice.h>
-#include <linux/watchdog.h>
-#include <linux/fs.h>
-#include <linux/ioport.h>
-#include <linux/notifier.h>
-#include <linux/reboot.h>
-#include <linux/init.h>
-#include <linux/spinlock.h>
-#include <linux/io.h>
-#include <linux/uaccess.h>
-
-
-#define WATCHDOG_NAME "w83697hf/hg WDT"
-#define WATCHDOG_TIMEOUT 60 /* 60 sec default timeout */
-#define WATCHDOG_EARLY_DISABLE 1 /* Disable until userland kicks in */
-
-static unsigned long wdt_is_open;
-static char expect_close;
-static DEFINE_SPINLOCK(io_lock);
-
-/* You must set this - there is no sane way to probe for this board. */
-static int wdt_io = 0x2e;
-module_param(wdt_io, int, 0);
-MODULE_PARM_DESC(wdt_io,
- "w83697hf/hg WDT io port (default 0x2e, 0 = autodetect)");
-
-static int timeout = WATCHDOG_TIMEOUT; /* in seconds */
-module_param(timeout, int, 0);
-MODULE_PARM_DESC(timeout,
- "Watchdog timeout in seconds. 1<= timeout <=255 (default="
- __MODULE_STRING(WATCHDOG_TIMEOUT) ")");
-
-static bool nowayout = WATCHDOG_NOWAYOUT;
-module_param(nowayout, bool, 0);
-MODULE_PARM_DESC(nowayout,
- "Watchdog cannot be stopped once started (default="
- __MODULE_STRING(WATCHDOG_NOWAYOUT) ")");
-
-static int early_disable = WATCHDOG_EARLY_DISABLE;
-module_param(early_disable, int, 0);
-MODULE_PARM_DESC(early_disable,
- "Watchdog gets disabled at boot time (default="
- __MODULE_STRING(WATCHDOG_EARLY_DISABLE) ")");
-
-/*
- * Kernel methods.
- */
-
-#define W83697HF_EFER (wdt_io + 0) /* Extended Function Enable Register */
-#define W83697HF_EFIR (wdt_io + 0) /* Extended Function Index Register
- (same as EFER) */
-#define W83697HF_EFDR (wdt_io + 1) /* Extended Function Data Register */
-
-static inline void w83697hf_unlock(void)
-{
- outb_p(0x87, W83697HF_EFER); /* Enter extended function mode */
- outb_p(0x87, W83697HF_EFER); /* Again according to manual */
-}
-
-static inline void w83697hf_lock(void)
-{
- outb_p(0xAA, W83697HF_EFER); /* Leave extended function mode */
-}
-
-/*
- * The three functions w83697hf_get_reg(), w83697hf_set_reg() and
- * w83697hf_write_timeout() must be called with the device unlocked.
- */
-
-static unsigned char w83697hf_get_reg(unsigned char reg)
-{
- outb_p(reg, W83697HF_EFIR);
- return inb_p(W83697HF_EFDR);
-}
-
-static void w83697hf_set_reg(unsigned char reg, unsigned char data)
-{
- outb_p(reg, W83697HF_EFIR);
- outb_p(data, W83697HF_EFDR);
-}
-
-static void w83697hf_write_timeout(int timeout)
-{
- /* Write Timeout counter to CRF4 */
- w83697hf_set_reg(0xF4, timeout);
-}
-
-static void w83697hf_select_wdt(void)
-{
- w83697hf_unlock();
- w83697hf_set_reg(0x07, 0x08); /* Switch to logic device 8 (GPIO2) */
-}
-
-static inline void w83697hf_deselect_wdt(void)
-{
- w83697hf_lock();
-}
-
-static void w83697hf_init(void)
-{
- unsigned char bbuf;
-
- w83697hf_select_wdt();
-
- bbuf = w83697hf_get_reg(0x29);
- bbuf &= ~0x60;
- bbuf |= 0x20;
-
- /* Set pin 119 to WDTO# mode (= CR29, WDT0) */
- w83697hf_set_reg(0x29, bbuf);
-
- bbuf = w83697hf_get_reg(0xF3);
- bbuf &= ~0x04;
- w83697hf_set_reg(0xF3, bbuf); /* Count mode is seconds */
-
- w83697hf_deselect_wdt();
-}
-
-static void wdt_ping(void)
-{
- spin_lock(&io_lock);
- w83697hf_select_wdt();
-
- w83697hf_write_timeout(timeout);
-
- w83697hf_deselect_wdt();
- spin_unlock(&io_lock);
-}
-
-static void wdt_enable(void)
-{
- spin_lock(&io_lock);
- w83697hf_select_wdt();
-
- w83697hf_write_timeout(timeout);
- w83697hf_set_reg(0x30, 1); /* Enable timer */
-
- w83697hf_deselect_wdt();
- spin_unlock(&io_lock);
-}
-
-static void wdt_disable(void)
-{
- spin_lock(&io_lock);
- w83697hf_select_wdt();
-
- w83697hf_set_reg(0x30, 0); /* Disable timer */
- w83697hf_write_timeout(0);
-
- w83697hf_deselect_wdt();
- spin_unlock(&io_lock);
-}
-
-static unsigned char wdt_running(void)
-{
- unsigned char t;
-
- spin_lock(&io_lock);
- w83697hf_select_wdt();
-
- t = w83697hf_get_reg(0xF4); /* Read timer */
-
- w83697hf_deselect_wdt();
- spin_unlock(&io_lock);
-
- return t;
-}
-
-static int wdt_set_heartbeat(int t)
-{
- if (t < 1 || t > 255)
- return -EINVAL;
-
- timeout = t;
- return 0;
-}
-
-static ssize_t wdt_write(struct file *file, const char __user *buf,
- size_t count, loff_t *ppos)
-{
- if (count) {
- if (!nowayout) {
- size_t i;
-
- expect_close = 0;
-
- for (i = 0; i != count; i++) {
- char c;
- if (get_user(c, buf + i))
- return -EFAULT;
- if (c == 'V')
- expect_close = 42;
- }
- }
- wdt_ping();
- }
- return count;
-}
-
-static long wdt_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
-{
- void __user *argp = (void __user *)arg;
- int __user *p = argp;
- int new_timeout;
- static const struct watchdog_info ident = {
- .options = WDIOF_KEEPALIVEPING | WDIOF_SETTIMEOUT
- | WDIOF_MAGICCLOSE,
- .firmware_version = 1,
- .identity = "W83697HF WDT",
- };
-
- switch (cmd) {
- case WDIOC_GETSUPPORT:
- if (copy_to_user(argp, &ident, sizeof(ident)))
- return -EFAULT;
- break;
-
- case WDIOC_GETSTATUS:
- case WDIOC_GETBOOTSTATUS:
- return put_user(0, p);
-
- case WDIOC_SETOPTIONS:
- {
- int options, retval = -EINVAL;
-
- if (get_user(options, p))
- return -EFAULT;
-
- if (options & WDIOS_DISABLECARD) {
- wdt_disable();
- retval = 0;
- }
-
- if (options & WDIOS_ENABLECARD) {
- wdt_enable();
- retval = 0;
- }
-
- return retval;
- }
-
- case WDIOC_KEEPALIVE:
- wdt_ping();
- break;
-
- case WDIOC_SETTIMEOUT:
- if (get_user(new_timeout, p))
- return -EFAULT;
- if (wdt_set_heartbeat(new_timeout))
- return -EINVAL;
- wdt_ping();
- /* Fall */
-
- case WDIOC_GETTIMEOUT:
- return put_user(timeout, p);
-
- default:
- return -ENOTTY;
- }
- return 0;
-}
-
-static int wdt_open(struct inode *inode, struct file *file)
-{
- if (test_and_set_bit(0, &wdt_is_open))
- return -EBUSY;
- /*
- * Activate
- */
-
- wdt_enable();
- return nonseekable_open(inode, file);
-}
-
-static int wdt_close(struct inode *inode, struct file *file)
-{
- if (expect_close == 42)
- wdt_disable();
- else {
- pr_crit("Unexpected close, not stopping watchdog!\n");
- wdt_ping();
- }
- expect_close = 0;
- clear_bit(0, &wdt_is_open);
- return 0;
-}
-
-/*
- * Notifier for system down
- */
-
-static int wdt_notify_sys(struct notifier_block *this, unsigned long code,
- void *unused)
-{
- if (code == SYS_DOWN || code == SYS_HALT)
- wdt_disable(); /* Turn the WDT off */
-
- return NOTIFY_DONE;
-}
-
-/*
- * Kernel Interfaces
- */
-
-static const struct file_operations wdt_fops = {
- .owner = THIS_MODULE,
- .llseek = no_llseek,
- .write = wdt_write,
- .unlocked_ioctl = wdt_ioctl,
- .open = wdt_open,
- .release = wdt_close,
-};
-
-static struct miscdevice wdt_miscdev = {
- .minor = WATCHDOG_MINOR,
- .name = "watchdog",
- .fops = &wdt_fops,
-};
-
-/*
- * The WDT needs to learn about soft shutdowns in order to
- * turn the timebomb registers off.
- */
-
-static struct notifier_block wdt_notifier = {
- .notifier_call = wdt_notify_sys,
-};
-
-static int w83697hf_check_wdt(void)
-{
- if (!request_region(wdt_io, 2, WATCHDOG_NAME)) {
- pr_err("I/O address 0x%x already in use\n", wdt_io);
- return -EIO;
- }
-
- pr_debug("Looking for watchdog at address 0x%x\n", wdt_io);
- w83697hf_unlock();
- if (w83697hf_get_reg(0x20) == 0x60) {
- pr_info("watchdog found at address 0x%x\n", wdt_io);
- w83697hf_lock();
- return 0;
- }
- /* Reprotect in case it was a compatible device */
- w83697hf_lock();
-
- pr_info("watchdog not found at address 0x%x\n", wdt_io);
- release_region(wdt_io, 2);
- return -EIO;
-}
-
-static int w83697hf_ioports[] = { 0x2e, 0x4e, 0x00 };
-
-static int __init wdt_init(void)
-{
- int ret, i, found = 0;
-
- pr_info("WDT driver for W83697HF/HG initializing\n");
-
- if (wdt_io == 0) {
- /* we will autodetect the W83697HF/HG watchdog */
- for (i = 0; ((!found) && (w83697hf_ioports[i] != 0)); i++) {
- wdt_io = w83697hf_ioports[i];
- if (!w83697hf_check_wdt())
- found++;
- }
- } else {
- if (!w83697hf_check_wdt())
- found++;
- }
-
- if (!found) {
- pr_err("No W83697HF/HG could be found\n");
- ret = -ENODEV;
- goto out;
- }
-
- w83697hf_init();
- if (early_disable) {
- if (wdt_running())
- pr_warn("Stopping previously enabled watchdog until userland kicks in\n");
- wdt_disable();
- }
-
- if (wdt_set_heartbeat(timeout)) {
- wdt_set_heartbeat(WATCHDOG_TIMEOUT);
- pr_info("timeout value must be 1 <= timeout <= 255, using %d\n",
- WATCHDOG_TIMEOUT);
- }
-
- ret = register_reboot_notifier(&wdt_notifier);
- if (ret != 0) {
- pr_err("cannot register reboot notifier (err=%d)\n", ret);
- goto unreg_regions;
- }
-
- ret = misc_register(&wdt_miscdev);
- if (ret != 0) {
- pr_err("cannot register miscdev on minor=%d (err=%d)\n",
- WATCHDOG_MINOR, ret);
- goto unreg_reboot;
- }
-
- pr_info("initialized. timeout=%d sec (nowayout=%d)\n",
- timeout, nowayout);
-
-out:
- return ret;
-unreg_reboot:
- unregister_reboot_notifier(&wdt_notifier);
-unreg_regions:
- release_region(wdt_io, 2);
- goto out;
-}
-
-static void __exit wdt_exit(void)
-{
- misc_deregister(&wdt_miscdev);
- unregister_reboot_notifier(&wdt_notifier);
- release_region(wdt_io, 2);
-}
-
-module_init(wdt_init);
-module_exit(wdt_exit);
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Marcus Junker <junker@anduras.de>");
-MODULE_AUTHOR("Samuel Tardieu <sam@rfc1149.net>");
-MODULE_DESCRIPTION("w83697hf/hg WDT driver");
diff --git a/drivers/watchdog/w83697ug_wdt.c b/drivers/watchdog/w83697ug_wdt.c
deleted file mode 100644
index ff58cb7..0000000
--- a/drivers/watchdog/w83697ug_wdt.c
+++ /dev/null
@@ -1,397 +0,0 @@
-/*
- * w83697ug/uf WDT driver
- *
- * (c) Copyright 2008 Flemming Fransen <ff@nrvissing.net>
- * reused original code to support w83697ug/uf.
- *
- * Based on w83627hf_wdt.c which is based on advantechwdt.c
- * which is based on wdt.c.
- * Original copyright messages:
- *
- * (c) Copyright 2007 Vlad Drukker <vlad@storewiz.com>
- * added support for W83627THF.
- *
- * (c) Copyright 2003 Pádraig Brady <P@draigBrady.com>
- *
- * (c) Copyright 2000-2001 Marek Michalkiewicz <marekm@linux.org.pl>
- *
- * (c) Copyright 1996 Alan Cox <alan@redhat.com>, All Rights Reserved.
- * http://www.redhat.com
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
- * Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
- * warranty for any of this software. This material is provided
- * "AS-IS" and at no charge.
- *
- * (c) Copyright 1995 Alan Cox <alan@redhat.com>
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <linux/module.h>
-#include <linux/moduleparam.h>
-#include <linux/types.h>
-#include <linux/miscdevice.h>
-#include <linux/watchdog.h>
-#include <linux/fs.h>
-#include <linux/ioport.h>
-#include <linux/notifier.h>
-#include <linux/reboot.h>
-#include <linux/init.h>
-#include <linux/spinlock.h>
-#include <linux/io.h>
-#include <linux/uaccess.h>
-
-
-#define WATCHDOG_NAME "w83697ug/uf WDT"
-#define WATCHDOG_TIMEOUT 60 /* 60 sec default timeout */
-
-static unsigned long wdt_is_open;
-static char expect_close;
-static DEFINE_SPINLOCK(io_lock);
-
-static int wdt_io = 0x2e;
-module_param(wdt_io, int, 0);
-MODULE_PARM_DESC(wdt_io, "w83697ug/uf WDT io port (default 0x2e)");
-
-static int timeout = WATCHDOG_TIMEOUT; /* in seconds */
-module_param(timeout, int, 0);
-MODULE_PARM_DESC(timeout,
- "Watchdog timeout in seconds. 1<= timeout <=255 (default="
- __MODULE_STRING(WATCHDOG_TIMEOUT) ")");
-
-static bool nowayout = WATCHDOG_NOWAYOUT;
-module_param(nowayout, bool, 0);
-MODULE_PARM_DESC(nowayout,
- "Watchdog cannot be stopped once started (default="
- __MODULE_STRING(WATCHDOG_NOWAYOUT) ")");
-
-/*
- * Kernel methods.
- */
-
-#define WDT_EFER (wdt_io+0) /* Extended Function Enable Registers */
-#define WDT_EFIR (wdt_io+0) /* Extended Function Index Register
- (same as EFER) */
-#define WDT_EFDR (WDT_EFIR+1) /* Extended Function Data Register */
-
-static int w83697ug_select_wd_register(void)
-{
- unsigned char c;
- unsigned char version;
-
- outb_p(0x87, WDT_EFER); /* Enter extended function mode */
- outb_p(0x87, WDT_EFER); /* Again according to manual */
-
- outb(0x20, WDT_EFER); /* check chip version */
- version = inb(WDT_EFDR);
-
- if (version == 0x68) { /* W83697UG */
- pr_info("Watchdog chip version 0x%02x = W83697UG/UF found at 0x%04x\n",
- version, wdt_io);
-
- outb_p(0x2b, WDT_EFER);
- c = inb_p(WDT_EFDR); /* select WDT0 */
- c &= ~0x04;
- outb_p(0x2b, WDT_EFER);
- outb_p(c, WDT_EFDR); /* set pin118 to WDT0 */
-
- } else {
- pr_err("No W83697UG/UF could be found\n");
- return -ENODEV;
- }
-
- outb_p(0x07, WDT_EFER); /* point to logical device number reg */
- outb_p(0x08, WDT_EFDR); /* select logical device 8 (GPIO2) */
- outb_p(0x30, WDT_EFER); /* select CR30 */
- c = inb_p(WDT_EFDR);
- outb_p(c | 0x01, WDT_EFDR); /* set bit 0 to activate GPIO2 */
-
- return 0;
-}
-
-static void w83697ug_unselect_wd_register(void)
-{
- outb_p(0xAA, WDT_EFER); /* Leave extended function mode */
-}
-
-static int w83697ug_init(void)
-{
- int ret;
- unsigned char t;
-
- ret = w83697ug_select_wd_register();
- if (ret != 0)
- return ret;
-
- outb_p(0xF6, WDT_EFER); /* Select CRF6 */
- t = inb_p(WDT_EFDR); /* read CRF6 */
- if (t != 0) {
- pr_info("Watchdog already running. Resetting timeout to %d sec\n",
- timeout);
- outb_p(timeout, WDT_EFDR); /* Write back to CRF6 */
- }
- outb_p(0xF5, WDT_EFER); /* Select CRF5 */
- t = inb_p(WDT_EFDR); /* read CRF5 */
- t &= ~0x0C; /* set second mode &
- disable keyboard turning off watchdog */
- outb_p(t, WDT_EFDR); /* Write back to CRF5 */
-
- w83697ug_unselect_wd_register();
- return 0;
-}
-
-static void wdt_ctrl(int timeout)
-{
- spin_lock(&io_lock);
-
- if (w83697ug_select_wd_register() < 0) {
- spin_unlock(&io_lock);
- return;
- }
-
- outb_p(0xF4, WDT_EFER); /* Select CRF4 */
- outb_p(timeout, WDT_EFDR); /* Write Timeout counter to CRF4 */
-
- w83697ug_unselect_wd_register();
-
- spin_unlock(&io_lock);
-}
-
-static int wdt_ping(void)
-{
- wdt_ctrl(timeout);
- return 0;
-}
-
-static int wdt_disable(void)
-{
- wdt_ctrl(0);
- return 0;
-}
-
-static int wdt_set_heartbeat(int t)
-{
- if (t < 1 || t > 255)
- return -EINVAL;
-
- timeout = t;
- return 0;
-}
-
-static ssize_t wdt_write(struct file *file, const char __user *buf,
- size_t count, loff_t *ppos)
-{
- if (count) {
- if (!nowayout) {
- size_t i;
-
- expect_close = 0;
-
- for (i = 0; i != count; i++) {
- char c;
- if (get_user(c, buf + i))
- return -EFAULT;
- if (c == 'V')
- expect_close = 42;
- }
- }
- wdt_ping();
- }
- return count;
-}
-
-static long wdt_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
-{
- void __user *argp = (void __user *)arg;
- int __user *p = argp;
- int new_timeout;
- static const struct watchdog_info ident = {
- .options = WDIOF_KEEPALIVEPING |
- WDIOF_SETTIMEOUT |
- WDIOF_MAGICCLOSE,
- .firmware_version = 1,
- .identity = "W83697UG WDT",
- };
-
- switch (cmd) {
- case WDIOC_GETSUPPORT:
- if (copy_to_user(argp, &ident, sizeof(ident)))
- return -EFAULT;
- break;
-
- case WDIOC_GETSTATUS:
- case WDIOC_GETBOOTSTATUS:
- return put_user(0, p);
-
- case WDIOC_SETOPTIONS:
- {
- int options, retval = -EINVAL;
-
- if (get_user(options, p))
- return -EFAULT;
-
- if (options & WDIOS_DISABLECARD) {
- wdt_disable();
- retval = 0;
- }
-
- if (options & WDIOS_ENABLECARD) {
- wdt_ping();
- retval = 0;
- }
-
- return retval;
- }
-
- case WDIOC_KEEPALIVE:
- wdt_ping();
- break;
-
- case WDIOC_SETTIMEOUT:
- if (get_user(new_timeout, p))
- return -EFAULT;
- if (wdt_set_heartbeat(new_timeout))
- return -EINVAL;
- wdt_ping();
- /* Fall */
-
- case WDIOC_GETTIMEOUT:
- return put_user(timeout, p);
-
- default:
- return -ENOTTY;
- }
- return 0;
-}
-
-static int wdt_open(struct inode *inode, struct file *file)
-{
- if (test_and_set_bit(0, &wdt_is_open))
- return -EBUSY;
- /*
- * Activate
- */
-
- wdt_ping();
- return nonseekable_open(inode, file);
-}
-
-static int wdt_close(struct inode *inode, struct file *file)
-{
- if (expect_close == 42)
- wdt_disable();
- else {
- pr_crit("Unexpected close, not stopping watchdog!\n");
- wdt_ping();
- }
- expect_close = 0;
- clear_bit(0, &wdt_is_open);
- return 0;
-}
-
-/*
- * Notifier for system down
- */
-
-static int wdt_notify_sys(struct notifier_block *this, unsigned long code,
- void *unused)
-{
- if (code == SYS_DOWN || code == SYS_HALT)
- wdt_disable(); /* Turn the WDT off */
-
- return NOTIFY_DONE;
-}
-
-/*
- * Kernel Interfaces
- */
-
-static const struct file_operations wdt_fops = {
- .owner = THIS_MODULE,
- .llseek = no_llseek,
- .write = wdt_write,
- .unlocked_ioctl = wdt_ioctl,
- .open = wdt_open,
- .release = wdt_close,
-};
-
-static struct miscdevice wdt_miscdev = {
- .minor = WATCHDOG_MINOR,
- .name = "watchdog",
- .fops = &wdt_fops,
-};
-
-/*
- * The WDT needs to learn about soft shutdowns in order to
- * turn the timebomb registers off.
- */
-
-static struct notifier_block wdt_notifier = {
- .notifier_call = wdt_notify_sys,
-};
-
-static int __init wdt_init(void)
-{
- int ret;
-
- pr_info("WDT driver for the Winbond(TM) W83697UG/UF Super I/O chip initialising\n");
-
- if (wdt_set_heartbeat(timeout)) {
- wdt_set_heartbeat(WATCHDOG_TIMEOUT);
- pr_info("timeout value must be 1<=timeout<=255, using %d\n",
- WATCHDOG_TIMEOUT);
- }
-
- if (!request_region(wdt_io, 1, WATCHDOG_NAME)) {
- pr_err("I/O address 0x%04x already in use\n", wdt_io);
- ret = -EIO;
- goto out;
- }
-
- ret = w83697ug_init();
- if (ret != 0)
- goto unreg_regions;
-
- ret = register_reboot_notifier(&wdt_notifier);
- if (ret != 0) {
- pr_err("cannot register reboot notifier (err=%d)\n", ret);
- goto unreg_regions;
- }
-
- ret = misc_register(&wdt_miscdev);
- if (ret != 0) {
- pr_err("cannot register miscdev on minor=%d (err=%d)\n",
- WATCHDOG_MINOR, ret);
- goto unreg_reboot;
- }
-
- pr_info("initialized. timeout=%d sec (nowayout=%d)\n",
- timeout, nowayout);
-
-out:
- return ret;
-unreg_reboot:
- unregister_reboot_notifier(&wdt_notifier);
-unreg_regions:
- release_region(wdt_io, 1);
- goto out;
-}
-
-static void __exit wdt_exit(void)
-{
- misc_deregister(&wdt_miscdev);
- unregister_reboot_notifier(&wdt_notifier);
- release_region(wdt_io, 1);
-}
-
-module_init(wdt_init);
-module_exit(wdt_exit);
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Flemming Frandsen <ff@nrvissing.net>");
-MODULE_DESCRIPTION("w83697ug/uf WDT driver");
diff --git a/fs/9p/vfs_inode.c b/fs/9p/vfs_inode.c
index 00d140f..7fa4f7a 100644
--- a/fs/9p/vfs_inode.c
+++ b/fs/9p/vfs_inode.c
@@ -147,7 +147,7 @@
int major = -1, minor = -1;
strlcpy(ext, stat->extension, sizeof(ext));
- sscanf(ext, "%c %u %u", &type, &major, &minor);
+ sscanf(ext, "%c %i %i", &type, &major, &minor);
switch (type) {
case 'c':
res |= S_IFCHR;
diff --git a/fs/9p/xattr.c b/fs/9p/xattr.c
index 04133a1..f95e01e 100644
--- a/fs/9p/xattr.c
+++ b/fs/9p/xattr.c
@@ -156,7 +156,7 @@
offset += write_count;
value_len -= write_count;
}
- retval = offset;
+ retval = 0;
err:
p9_client_clunk(fid);
return retval;
diff --git a/fs/affs/affs.h b/fs/affs/affs.h
index 25b23b1..9bca881 100644
--- a/fs/affs/affs.h
+++ b/fs/affs/affs.h
@@ -1,3 +1,9 @@
+#ifdef pr_fmt
+#undef pr_fmt
+#endif
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/types.h>
#include <linux/fs.h>
#include <linux/buffer_head.h>
@@ -206,7 +212,7 @@
static inline struct buffer_head *
affs_bread(struct super_block *sb, int block)
{
- pr_debug("affs_bread: %d\n", block);
+ pr_debug("%s: %d\n", __func__, block);
if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size)
return sb_bread(sb, block);
return NULL;
@@ -214,7 +220,7 @@
static inline struct buffer_head *
affs_getblk(struct super_block *sb, int block)
{
- pr_debug("affs_getblk: %d\n", block);
+ pr_debug("%s: %d\n", __func__, block);
if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size)
return sb_getblk(sb, block);
return NULL;
@@ -223,7 +229,7 @@
affs_getzeroblk(struct super_block *sb, int block)
{
struct buffer_head *bh;
- pr_debug("affs_getzeroblk: %d\n", block);
+ pr_debug("%s: %d\n", __func__, block);
if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size) {
bh = sb_getblk(sb, block);
lock_buffer(bh);
@@ -238,7 +244,7 @@
affs_getemptyblk(struct super_block *sb, int block)
{
struct buffer_head *bh;
- pr_debug("affs_getemptyblk: %d\n", block);
+ pr_debug("%s: %d\n", __func__, block);
if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size) {
bh = sb_getblk(sb, block);
wait_on_buffer(bh);
@@ -251,7 +257,7 @@
affs_brelse(struct buffer_head *bh)
{
if (bh)
- pr_debug("affs_brelse: %lld\n", (long long) bh->b_blocknr);
+ pr_debug("%s: %lld\n", __func__, (long long) bh->b_blocknr);
brelse(bh);
}
diff --git a/fs/affs/amigaffs.c b/fs/affs/amigaffs.c
index 533a322..406b298 100644
--- a/fs/affs/amigaffs.c
+++ b/fs/affs/amigaffs.c
@@ -34,7 +34,7 @@
ino = bh->b_blocknr;
offset = affs_hash_name(sb, AFFS_TAIL(sb, bh)->name + 1, AFFS_TAIL(sb, bh)->name[0]);
- pr_debug("AFFS: insert_hash(dir=%u, ino=%d)\n", (u32)dir->i_ino, ino);
+ pr_debug("%s(dir=%u, ino=%d)\n", __func__, (u32)dir->i_ino, ino);
dir_bh = affs_bread(sb, dir->i_ino);
if (!dir_bh)
@@ -84,7 +84,8 @@
sb = dir->i_sb;
rem_ino = rem_bh->b_blocknr;
offset = affs_hash_name(sb, AFFS_TAIL(sb, rem_bh)->name+1, AFFS_TAIL(sb, rem_bh)->name[0]);
- pr_debug("AFFS: remove_hash(dir=%d, ino=%d, hashval=%d)\n", (u32)dir->i_ino, rem_ino, offset);
+ pr_debug("%s(dir=%d, ino=%d, hashval=%d)\n",
+ __func__, (u32)dir->i_ino, rem_ino, offset);
bh = affs_bread(sb, dir->i_ino);
if (!bh)
@@ -147,7 +148,7 @@
u32 link_ino, ino;
int retval;
- pr_debug("AFFS: remove_link(key=%ld)\n", inode->i_ino);
+ pr_debug("%s(key=%ld)\n", __func__, inode->i_ino);
retval = -EIO;
bh = affs_bread(sb, inode->i_ino);
if (!bh)
@@ -279,7 +280,7 @@
if (!inode)
goto done;
- pr_debug("AFFS: remove_header(key=%ld)\n", inode->i_ino);
+ pr_debug("%s(key=%ld)\n", __func__, inode->i_ino);
retval = -EIO;
bh = affs_bread(sb, (u32)(long)dentry->d_fsdata);
if (!bh)
@@ -451,10 +452,10 @@
vsnprintf(ErrorBuffer,sizeof(ErrorBuffer),fmt,args);
va_end(args);
- printk(KERN_CRIT "AFFS error (device %s): %s(): %s\n", sb->s_id,
+ pr_crit("error (device %s): %s(): %s\n", sb->s_id,
function,ErrorBuffer);
if (!(sb->s_flags & MS_RDONLY))
- printk(KERN_WARNING "AFFS: Remounting filesystem read-only\n");
+ pr_warn("Remounting filesystem read-only\n");
sb->s_flags |= MS_RDONLY;
}
@@ -467,7 +468,7 @@
vsnprintf(ErrorBuffer,sizeof(ErrorBuffer),fmt,args);
va_end(args);
- printk(KERN_WARNING "AFFS warning (device %s): %s(): %s\n", sb->s_id,
+ pr_warn("(device %s): %s(): %s\n", sb->s_id,
function,ErrorBuffer);
}
diff --git a/fs/affs/bitmap.c b/fs/affs/bitmap.c
index a32246b..c8de511 100644
--- a/fs/affs/bitmap.c
+++ b/fs/affs/bitmap.c
@@ -17,7 +17,7 @@
u32 free;
int i;
- pr_debug("AFFS: count_free_blocks()\n");
+ pr_debug("%s()\n", __func__);
if (sb->s_flags & MS_RDONLY)
return 0;
@@ -43,7 +43,7 @@
u32 blk, bmap, bit, mask, tmp;
__be32 *data;
- pr_debug("AFFS: free_block(%u)\n", block);
+ pr_debug("%s(%u)\n", __func__, block);
if (block > sbi->s_partition_size)
goto err_range;
@@ -125,7 +125,7 @@
sb = inode->i_sb;
sbi = AFFS_SB(sb);
- pr_debug("AFFS: balloc(inode=%lu,goal=%u): ", inode->i_ino, goal);
+ pr_debug("balloc(inode=%lu,goal=%u): ", inode->i_ino, goal);
if (AFFS_I(inode)->i_pa_cnt) {
pr_debug("%d\n", AFFS_I(inode)->i_lastalloc+1);
@@ -254,8 +254,7 @@
return 0;
if (!AFFS_ROOT_TAIL(sb, sbi->s_root_bh)->bm_flag) {
- printk(KERN_NOTICE "AFFS: Bitmap invalid - mounting %s read only\n",
- sb->s_id);
+ pr_notice("Bitmap invalid - mounting %s read only\n", sb->s_id);
*flags |= MS_RDONLY;
return 0;
}
@@ -268,7 +267,7 @@
size = sbi->s_bmap_count * sizeof(*bm);
bm = sbi->s_bitmap = kzalloc(size, GFP_KERNEL);
if (!sbi->s_bitmap) {
- printk(KERN_ERR "AFFS: Bitmap allocation failed\n");
+ pr_err("Bitmap allocation failed\n");
return -ENOMEM;
}
@@ -282,17 +281,17 @@
bm->bm_key = be32_to_cpu(bmap_blk[blk]);
bh = affs_bread(sb, bm->bm_key);
if (!bh) {
- printk(KERN_ERR "AFFS: Cannot read bitmap\n");
+ pr_err("Cannot read bitmap\n");
res = -EIO;
goto out;
}
if (affs_checksum_block(sb, bh)) {
- printk(KERN_WARNING "AFFS: Bitmap %u invalid - mounting %s read only.\n",
- bm->bm_key, sb->s_id);
+ pr_warn("Bitmap %u invalid - mounting %s read only.\n",
+ bm->bm_key, sb->s_id);
*flags |= MS_RDONLY;
goto out;
}
- pr_debug("AFFS: read bitmap block %d: %d\n", blk, bm->bm_key);
+ pr_debug("read bitmap block %d: %d\n", blk, bm->bm_key);
bm->bm_free = memweight(bh->b_data + 4, sb->s_blocksize - 4);
/* Don't try read the extension if this is the last block,
@@ -304,7 +303,7 @@
affs_brelse(bmap_bh);
bmap_bh = affs_bread(sb, be32_to_cpu(bmap_blk[blk]));
if (!bmap_bh) {
- printk(KERN_ERR "AFFS: Cannot read bitmap extension\n");
+ pr_err("Cannot read bitmap extension\n");
res = -EIO;
goto out;
}
diff --git a/fs/affs/dir.c b/fs/affs/dir.c
index cbbda47..59f07be 100644
--- a/fs/affs/dir.c
+++ b/fs/affs/dir.c
@@ -54,8 +54,8 @@
u32 ino;
int error = 0;
- pr_debug("AFFS: readdir(ino=%lu,f_pos=%lx)\n",
- inode->i_ino, (unsigned long)ctx->pos);
+ pr_debug("%s(ino=%lu,f_pos=%lx)\n",
+ __func__, inode->i_ino, (unsigned long)ctx->pos);
if (ctx->pos < 2) {
file->private_data = (void *)0;
@@ -81,7 +81,7 @@
*/
ino = (u32)(long)file->private_data;
if (ino && file->f_version == inode->i_version) {
- pr_debug("AFFS: readdir() left off=%d\n", ino);
+ pr_debug("readdir() left off=%d\n", ino);
goto inside;
}
@@ -117,7 +117,7 @@
namelen = min(AFFS_TAIL(sb, fh_bh)->name[0], (u8)30);
name = AFFS_TAIL(sb, fh_bh)->name + 1;
- pr_debug("AFFS: readdir(): dir_emit(\"%.*s\", "
+ pr_debug("readdir(): dir_emit(\"%.*s\", "
"ino=%u), hash=%d, f_pos=%x\n",
namelen, name, ino, hash_pos, (u32)ctx->pos);
diff --git a/fs/affs/file.c b/fs/affs/file.c
index 8669b6ec..0270303 100644
--- a/fs/affs/file.c
+++ b/fs/affs/file.c
@@ -45,7 +45,7 @@
static int
affs_file_open(struct inode *inode, struct file *filp)
{
- pr_debug("AFFS: open(%lu,%d)\n",
+ pr_debug("open(%lu,%d)\n",
inode->i_ino, atomic_read(&AFFS_I(inode)->i_opencnt));
atomic_inc(&AFFS_I(inode)->i_opencnt);
return 0;
@@ -54,7 +54,7 @@
static int
affs_file_release(struct inode *inode, struct file *filp)
{
- pr_debug("AFFS: release(%lu, %d)\n",
+ pr_debug("release(%lu, %d)\n",
inode->i_ino, atomic_read(&AFFS_I(inode)->i_opencnt));
if (atomic_dec_and_test(&AFFS_I(inode)->i_opencnt)) {
@@ -324,7 +324,8 @@
struct buffer_head *ext_bh;
u32 ext;
- pr_debug("AFFS: get_block(%u, %lu)\n", (u32)inode->i_ino, (unsigned long)block);
+ pr_debug("%s(%u, %lu)\n",
+ __func__, (u32)inode->i_ino, (unsigned long)block);
BUG_ON(block > (sector_t)0x7fffffffUL);
@@ -498,34 +499,36 @@
}
static int
-affs_do_readpage_ofs(struct file *file, struct page *page, unsigned from, unsigned to)
+affs_do_readpage_ofs(struct page *page, unsigned to)
{
struct inode *inode = page->mapping->host;
struct super_block *sb = inode->i_sb;
struct buffer_head *bh;
char *data;
+ unsigned pos = 0;
u32 bidx, boff, bsize;
u32 tmp;
- pr_debug("AFFS: read_page(%u, %ld, %d, %d)\n", (u32)inode->i_ino, page->index, from, to);
- BUG_ON(from > to || to > PAGE_CACHE_SIZE);
+ pr_debug("%s(%u, %ld, 0, %d)\n", __func__, (u32)inode->i_ino,
+ page->index, to);
+ BUG_ON(to > PAGE_CACHE_SIZE);
kmap(page);
data = page_address(page);
bsize = AFFS_SB(sb)->s_data_blksize;
- tmp = (page->index << PAGE_CACHE_SHIFT) + from;
+ tmp = page->index << PAGE_CACHE_SHIFT;
bidx = tmp / bsize;
boff = tmp % bsize;
- while (from < to) {
+ while (pos < to) {
bh = affs_bread_ino(inode, bidx, 0);
if (IS_ERR(bh))
return PTR_ERR(bh);
- tmp = min(bsize - boff, to - from);
- BUG_ON(from + tmp > to || tmp > bsize);
- memcpy(data + from, AFFS_DATA(bh) + boff, tmp);
+ tmp = min(bsize - boff, to - pos);
+ BUG_ON(pos + tmp > to || tmp > bsize);
+ memcpy(data + pos, AFFS_DATA(bh) + boff, tmp);
affs_brelse(bh);
bidx++;
- from += tmp;
+ pos += tmp;
boff = 0;
}
flush_dcache_page(page);
@@ -542,7 +545,7 @@
u32 size, bsize;
u32 tmp;
- pr_debug("AFFS: extent_file(%u, %d)\n", (u32)inode->i_ino, newsize);
+ pr_debug("%s(%u, %d)\n", __func__, (u32)inode->i_ino, newsize);
bsize = AFFS_SB(sb)->s_data_blksize;
bh = NULL;
size = AFFS_I(inode)->mmu_private;
@@ -608,14 +611,14 @@
u32 to;
int err;
- pr_debug("AFFS: read_page(%u, %ld)\n", (u32)inode->i_ino, page->index);
+ pr_debug("%s(%u, %ld)\n", __func__, (u32)inode->i_ino, page->index);
to = PAGE_CACHE_SIZE;
if (((page->index + 1) << PAGE_CACHE_SHIFT) > inode->i_size) {
to = inode->i_size & ~PAGE_CACHE_MASK;
memset(page_address(page) + to, 0, PAGE_CACHE_SIZE - to);
}
- err = affs_do_readpage_ofs(file, page, 0, to);
+ err = affs_do_readpage_ofs(page, to);
if (!err)
SetPageUptodate(page);
unlock_page(page);
@@ -631,7 +634,8 @@
pgoff_t index;
int err = 0;
- pr_debug("AFFS: write_begin(%u, %llu, %llu)\n", (u32)inode->i_ino, (unsigned long long)pos, (unsigned long long)pos + len);
+ pr_debug("%s(%u, %llu, %llu)\n", __func__, (u32)inode->i_ino,
+ (unsigned long long)pos, (unsigned long long)pos + len);
if (pos > AFFS_I(inode)->mmu_private) {
/* XXX: this probably leaves a too-big i_size in case of
* failure. Should really be updating i_size at write_end time
@@ -651,7 +655,7 @@
return 0;
/* XXX: inefficient but safe in the face of short writes */
- err = affs_do_readpage_ofs(file, page, 0, PAGE_CACHE_SIZE);
+ err = affs_do_readpage_ofs(page, PAGE_CACHE_SIZE);
if (err) {
unlock_page(page);
page_cache_release(page);
@@ -680,7 +684,9 @@
* due to write_begin.
*/
- pr_debug("AFFS: write_begin(%u, %llu, %llu)\n", (u32)inode->i_ino, (unsigned long long)pos, (unsigned long long)pos + len);
+ pr_debug("%s(%u, %llu, %llu)\n",
+ __func__, (u32)inode->i_ino, (unsigned long long)pos,
+ (unsigned long long)pos + len);
bsize = AFFS_SB(sb)->s_data_blksize;
data = page_address(page);
@@ -802,7 +808,7 @@
{
struct super_block *sb = inode->i_sb;
- pr_debug("AFFS: free_prealloc(ino=%lu)\n", inode->i_ino);
+ pr_debug("free_prealloc(ino=%lu)\n", inode->i_ino);
while (AFFS_I(inode)->i_pa_cnt) {
AFFS_I(inode)->i_pa_cnt--;
@@ -822,7 +828,7 @@
struct buffer_head *ext_bh;
int i;
- pr_debug("AFFS: truncate(inode=%d, oldsize=%u, newsize=%u)\n",
+ pr_debug("truncate(inode=%d, oldsize=%u, newsize=%u)\n",
(u32)inode->i_ino, (u32)AFFS_I(inode)->mmu_private, (u32)inode->i_size);
last_blk = 0;
diff --git a/fs/affs/inode.c b/fs/affs/inode.c
index 96df91e..bec2d1a 100644
--- a/fs/affs/inode.c
+++ b/fs/affs/inode.c
@@ -34,7 +34,7 @@
if (!(inode->i_state & I_NEW))
return inode;
- pr_debug("AFFS: affs_iget(%lu)\n", inode->i_ino);
+ pr_debug("affs_iget(%lu)\n", inode->i_ino);
block = inode->i_ino;
bh = affs_bread(sb, block);
@@ -175,7 +175,7 @@
uid_t uid;
gid_t gid;
- pr_debug("AFFS: write_inode(%lu)\n",inode->i_ino);
+ pr_debug("write_inode(%lu)\n", inode->i_ino);
if (!inode->i_nlink)
// possibly free block
@@ -220,7 +220,7 @@
struct inode *inode = dentry->d_inode;
int error;
- pr_debug("AFFS: notify_change(%lu,0x%x)\n",inode->i_ino,attr->ia_valid);
+ pr_debug("notify_change(%lu,0x%x)\n", inode->i_ino, attr->ia_valid);
error = inode_change_ok(inode,attr);
if (error)
@@ -258,7 +258,8 @@
affs_evict_inode(struct inode *inode)
{
unsigned long cache_page;
- pr_debug("AFFS: evict_inode(ino=%lu, nlink=%u)\n", inode->i_ino, inode->i_nlink);
+ pr_debug("evict_inode(ino=%lu, nlink=%u)\n",
+ inode->i_ino, inode->i_nlink);
truncate_inode_pages_final(&inode->i_data);
if (!inode->i_nlink) {
@@ -271,7 +272,7 @@
affs_free_prealloc(inode);
cache_page = (unsigned long)AFFS_I(inode)->i_lc;
if (cache_page) {
- pr_debug("AFFS: freeing ext cache\n");
+ pr_debug("freeing ext cache\n");
AFFS_I(inode)->i_lc = NULL;
AFFS_I(inode)->i_ac = NULL;
free_page(cache_page);
@@ -350,7 +351,8 @@
u32 block = 0;
int retval;
- pr_debug("AFFS: add_entry(dir=%u, inode=%u, \"%*s\", type=%d)\n", (u32)dir->i_ino,
+ pr_debug("%s(dir=%u, inode=%u, \"%*s\", type=%d)\n",
+ __func__, (u32)dir->i_ino,
(u32)inode->i_ino, (int)dentry->d_name.len, dentry->d_name.name, type);
retval = -EIO;
diff --git a/fs/affs/namei.c b/fs/affs/namei.c
index 6dae1cc..035bd31 100644
--- a/fs/affs/namei.c
+++ b/fs/affs/namei.c
@@ -190,7 +190,8 @@
toupper_t toupper = affs_get_toupper(sb);
u32 key;
- pr_debug("AFFS: find_entry(\"%.*s\")\n", (int)dentry->d_name.len, dentry->d_name.name);
+ pr_debug("%s(\"%.*s\")\n",
+ __func__, (int)dentry->d_name.len, dentry->d_name.name);
bh = affs_bread(sb, dir->i_ino);
if (!bh)
@@ -218,7 +219,8 @@
struct buffer_head *bh;
struct inode *inode = NULL;
- pr_debug("AFFS: lookup(\"%.*s\")\n",(int)dentry->d_name.len,dentry->d_name.name);
+ pr_debug("%s(\"%.*s\")\n",
+ __func__, (int)dentry->d_name.len, dentry->d_name.name);
affs_lock_dir(dir);
bh = affs_find_entry(dir, dentry);
@@ -248,9 +250,9 @@
int
affs_unlink(struct inode *dir, struct dentry *dentry)
{
- pr_debug("AFFS: unlink(dir=%d, %lu \"%.*s\")\n", (u32)dir->i_ino,
- dentry->d_inode->i_ino,
- (int)dentry->d_name.len, dentry->d_name.name);
+ pr_debug("%s(dir=%d, %lu \"%.*s\")\n",
+ __func__, (u32)dir->i_ino, dentry->d_inode->i_ino,
+ (int)dentry->d_name.len, dentry->d_name.name);
return affs_remove_header(dentry);
}
@@ -262,7 +264,8 @@
struct inode *inode;
int error;
- pr_debug("AFFS: create(%lu,\"%.*s\",0%ho)\n",dir->i_ino,(int)dentry->d_name.len,
+ pr_debug("%s(%lu,\"%.*s\",0%ho)\n",
+ __func__, dir->i_ino, (int)dentry->d_name.len,
dentry->d_name.name,mode);
inode = affs_new_inode(dir);
@@ -291,8 +294,9 @@
struct inode *inode;
int error;
- pr_debug("AFFS: mkdir(%lu,\"%.*s\",0%ho)\n",dir->i_ino,
- (int)dentry->d_name.len,dentry->d_name.name,mode);
+ pr_debug("%s(%lu,\"%.*s\",0%ho)\n",
+ __func__, dir->i_ino, (int)dentry->d_name.len,
+ dentry->d_name.name, mode);
inode = affs_new_inode(dir);
if (!inode)
@@ -317,8 +321,8 @@
int
affs_rmdir(struct inode *dir, struct dentry *dentry)
{
- pr_debug("AFFS: rmdir(dir=%u, %lu \"%.*s\")\n", (u32)dir->i_ino,
- dentry->d_inode->i_ino,
+ pr_debug("%s(dir=%u, %lu \"%.*s\")\n",
+ __func__, (u32)dir->i_ino, dentry->d_inode->i_ino,
(int)dentry->d_name.len, dentry->d_name.name);
return affs_remove_header(dentry);
@@ -334,8 +338,9 @@
int i, maxlen, error;
char c, lc;
- pr_debug("AFFS: symlink(%lu,\"%.*s\" -> \"%s\")\n",dir->i_ino,
- (int)dentry->d_name.len,dentry->d_name.name,symname);
+ pr_debug("%s(%lu,\"%.*s\" -> \"%s\")\n",
+ __func__, dir->i_ino, (int)dentry->d_name.len,
+ dentry->d_name.name, symname);
maxlen = AFFS_SB(sb)->s_hashsize * sizeof(u32) - 1;
inode = affs_new_inode(dir);
@@ -404,7 +409,8 @@
{
struct inode *inode = old_dentry->d_inode;
- pr_debug("AFFS: link(%u, %u, \"%.*s\")\n", (u32)inode->i_ino, (u32)dir->i_ino,
+ pr_debug("%s(%u, %u, \"%.*s\")\n",
+ __func__, (u32)inode->i_ino, (u32)dir->i_ino,
(int)dentry->d_name.len,dentry->d_name.name);
return affs_add_entry(dir, inode, dentry, ST_LINKFILE);
@@ -418,9 +424,10 @@
struct buffer_head *bh = NULL;
int retval;
- pr_debug("AFFS: rename(old=%u,\"%*s\" to new=%u,\"%*s\")\n",
- (u32)old_dir->i_ino, (int)old_dentry->d_name.len, old_dentry->d_name.name,
- (u32)new_dir->i_ino, (int)new_dentry->d_name.len, new_dentry->d_name.name);
+ pr_debug("%s(old=%u,\"%*s\" to new=%u,\"%*s\")\n",
+ __func__, (u32)old_dir->i_ino, (int)old_dentry->d_name.len,
+ old_dentry->d_name.name, (u32)new_dir->i_ino,
+ (int)new_dentry->d_name.len, new_dentry->d_name.name);
retval = affs_check_name(new_dentry->d_name.name,
new_dentry->d_name.len,
diff --git a/fs/affs/super.c b/fs/affs/super.c
index 895ac7d..51f1a95 100644
--- a/fs/affs/super.c
+++ b/fs/affs/super.c
@@ -46,7 +46,7 @@
affs_put_super(struct super_block *sb)
{
struct affs_sb_info *sbi = AFFS_SB(sb);
- pr_debug("AFFS: put_super()\n");
+ pr_debug("%s()\n", __func__);
cancel_delayed_work_sync(&sbi->sb_work);
}
@@ -220,7 +220,7 @@
return 0;
if (n != 512 && n != 1024 && n != 2048
&& n != 4096) {
- printk ("AFFS: Invalid blocksize (512, 1024, 2048, 4096 allowed)\n");
+ pr_warn("Invalid blocksize (512, 1024, 2048, 4096 allowed)\n");
return 0;
}
*blocksize = n;
@@ -285,8 +285,8 @@
/* Silently ignore the quota options */
break;
default:
- printk("AFFS: Unrecognized mount option \"%s\" "
- "or missing value\n", p);
+ pr_warn("Unrecognized mount option \"%s\" or missing value\n",
+ p);
return 0;
}
}
@@ -319,7 +319,7 @@
save_mount_options(sb, data);
- pr_debug("AFFS: read_super(%s)\n",data ? (const char *)data : "no options");
+ pr_debug("read_super(%s)\n", data ? (const char *)data : "no options");
sb->s_magic = AFFS_SUPER_MAGIC;
sb->s_op = &affs_sops;
@@ -339,7 +339,7 @@
if (!parse_options(data,&uid,&gid,&i,&reserved,&root_block,
&blocksize,&sbi->s_prefix,
sbi->s_volume, &mount_flags)) {
- printk(KERN_ERR "AFFS: Error parsing options\n");
+ pr_err("Error parsing options\n");
return -EINVAL;
}
/* N.B. after this point s_prefix must be released */
@@ -356,7 +356,7 @@
*/
size = sb->s_bdev->bd_inode->i_size >> 9;
- pr_debug("AFFS: initial blocksize=%d, #blocks=%d\n", 512, size);
+ pr_debug("initial blocksize=%d, #blocks=%d\n", 512, size);
affs_set_blocksize(sb, PAGE_SIZE);
/* Try to find root block. Its location depends on the block size. */
@@ -371,7 +371,7 @@
sbi->s_root_block = root_block;
if (root_block < 0)
sbi->s_root_block = (reserved + size - 1) / 2;
- pr_debug("AFFS: setting blocksize to %d\n", blocksize);
+ pr_debug("setting blocksize to %d\n", blocksize);
affs_set_blocksize(sb, blocksize);
sbi->s_partition_size = size;
@@ -386,7 +386,7 @@
* block behind the calculated one. So we check this one, too.
*/
for (num_bm = 0; num_bm < 2; num_bm++) {
- pr_debug("AFFS: Dev %s, trying root=%u, bs=%d, "
+ pr_debug("Dev %s, trying root=%u, bs=%d, "
"size=%d, reserved=%d\n",
sb->s_id,
sbi->s_root_block + num_bm,
@@ -407,8 +407,7 @@
}
}
if (!silent)
- printk(KERN_ERR "AFFS: No valid root block on device %s\n",
- sb->s_id);
+ pr_err("No valid root block on device %s\n", sb->s_id);
return -EINVAL;
/* N.B. after this point bh must be released */
@@ -420,7 +419,7 @@
/* Find out which kind of FS we have */
boot_bh = sb_bread(sb, 0);
if (!boot_bh) {
- printk(KERN_ERR "AFFS: Cannot read boot block\n");
+ pr_err("Cannot read boot block\n");
return -EINVAL;
}
memcpy(sig, boot_bh->b_data, 4);
@@ -433,8 +432,7 @@
*/
if ((chksum == FS_DCFFS || chksum == MUFS_DCFFS || chksum == FS_DCOFS
|| chksum == MUFS_DCOFS) && !(sb->s_flags & MS_RDONLY)) {
- printk(KERN_NOTICE "AFFS: Dircache FS - mounting %s read only\n",
- sb->s_id);
+ pr_notice("Dircache FS - mounting %s read only\n", sb->s_id);
sb->s_flags |= MS_RDONLY;
}
switch (chksum) {
@@ -468,14 +466,14 @@
sb->s_flags |= MS_NOEXEC;
break;
default:
- printk(KERN_ERR "AFFS: Unknown filesystem on device %s: %08X\n",
- sb->s_id, chksum);
+ pr_err("Unknown filesystem on device %s: %08X\n",
+ sb->s_id, chksum);
return -EINVAL;
}
if (mount_flags & SF_VERBOSE) {
u8 len = AFFS_ROOT_TAIL(sb, root_bh)->disk_name[0];
- printk(KERN_NOTICE "AFFS: Mounting volume \"%.*s\": Type=%.3s\\%c, Blocksize=%d\n",
+ pr_notice("Mounting volume \"%.*s\": Type=%.3s\\%c, Blocksize=%d\n",
len > 31 ? 31 : len,
AFFS_ROOT_TAIL(sb, root_bh)->disk_name + 1,
sig, sig[3] + '0', blocksize);
@@ -506,11 +504,11 @@
sb->s_root = d_make_root(root_inode);
if (!sb->s_root) {
- printk(KERN_ERR "AFFS: Get root inode failed\n");
+ pr_err("AFFS: Get root inode failed\n");
return -ENOMEM;
}
- pr_debug("AFFS: s_flags=%lX\n",sb->s_flags);
+ pr_debug("s_flags=%lX\n", sb->s_flags);
return 0;
}
@@ -530,7 +528,7 @@
char volume[32];
char *prefix = NULL;
- pr_debug("AFFS: remount(flags=0x%x,opts=\"%s\")\n",*flags,data);
+ pr_debug("%s(flags=0x%x,opts=\"%s\")\n", __func__, *flags, data);
sync_filesystem(sb);
*flags |= MS_NODIRATIME;
@@ -578,8 +576,9 @@
int free;
u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
- pr_debug("AFFS: statfs() partsize=%d, reserved=%d\n",AFFS_SB(sb)->s_partition_size,
- AFFS_SB(sb)->s_reserved);
+ pr_debug("%s() partsize=%d, reserved=%d\n",
+ __func__, AFFS_SB(sb)->s_partition_size,
+ AFFS_SB(sb)->s_reserved);
free = affs_count_free_blocks(sb);
buf->f_type = AFFS_SUPER_MAGIC;
diff --git a/fs/affs/symlink.c b/fs/affs/symlink.c
index ee00f08..f39b71c 100644
--- a/fs/affs/symlink.c
+++ b/fs/affs/symlink.c
@@ -21,7 +21,7 @@
char c;
char lc;
- pr_debug("AFFS: follow_link(ino=%lu)\n",inode->i_ino);
+ pr_debug("follow_link(ino=%lu)\n", inode->i_ino);
err = -EIO;
bh = affs_bread(inode->i_sb, inode->i_ino);
diff --git a/fs/attr.c b/fs/attr.c
index 5d4e59d..6530ced 100644
--- a/fs/attr.c
+++ b/fs/attr.c
@@ -50,14 +50,14 @@
if ((ia_valid & ATTR_UID) &&
(!uid_eq(current_fsuid(), inode->i_uid) ||
!uid_eq(attr->ia_uid, inode->i_uid)) &&
- !inode_capable(inode, CAP_CHOWN))
+ !capable_wrt_inode_uidgid(inode, CAP_CHOWN))
return -EPERM;
/* Make sure caller can chgrp. */
if ((ia_valid & ATTR_GID) &&
(!uid_eq(current_fsuid(), inode->i_uid) ||
(!in_group_p(attr->ia_gid) && !gid_eq(attr->ia_gid, inode->i_gid))) &&
- !inode_capable(inode, CAP_CHOWN))
+ !capable_wrt_inode_uidgid(inode, CAP_CHOWN))
return -EPERM;
/* Make sure a caller can chmod. */
@@ -67,7 +67,7 @@
/* Also check the setgid bit! */
if (!in_group_p((ia_valid & ATTR_GID) ? attr->ia_gid :
inode->i_gid) &&
- !inode_capable(inode, CAP_FSETID))
+ !capable_wrt_inode_uidgid(inode, CAP_FSETID))
attr->ia_mode &= ~S_ISGID;
}
@@ -160,7 +160,7 @@
umode_t mode = attr->ia_mode;
if (!in_group_p(inode->i_gid) &&
- !inode_capable(inode, CAP_FSETID))
+ !capable_wrt_inode_uidgid(inode, CAP_FSETID))
mode &= ~S_ISGID;
inode->i_mode = mode;
}
diff --git a/fs/befs/btree.c b/fs/befs/btree.c
index a2cd305..9c7faa8 100644
--- a/fs/befs/btree.c
+++ b/fs/befs/btree.c
@@ -318,7 +318,7 @@
* befs_find_key - Search for a key within a node
* @sb: Filesystem superblock
* @node: Node to find the key within
- * @key: Keystring to search for
+ * @findkey: Keystring to search for
* @value: If key is found, the value stored with the key is put here
*
* finds exact match if one exists, and returns BEFS_BT_MATCH
@@ -405,7 +405,7 @@
* Heres how it works: Key_no is the index of the key/value pair to
* return in keybuf/value.
* Bufsize is the size of keybuf (BEFS_NAME_LEN+1 is a good size). Keysize is
- * the number of charecters in the key (just a convenience).
+ * the number of characters in the key (just a convenience).
*
* Algorithm:
* Get the first leafnode of the tree. See if the requested key is in that
@@ -502,12 +502,11 @@
"for key of size %d", __func__, bufsize, keylen);
brelse(this_node->bh);
goto error_alloc;
- };
+ }
- strncpy(keybuf, keystart, keylen);
+ strlcpy(keybuf, keystart, keylen + 1);
*value = fs64_to_cpu(sb, valarray[cur_key]);
*keysize = keylen;
- keybuf[keylen] = '\0';
befs_debug(sb, "Read [%llu,%d]: Key \"%.*s\", Value %llu", node_off,
cur_key, keylen, keybuf, *value);
@@ -707,7 +706,7 @@
* @key1: pointer to the first key to be compared
* @keylen1: length in bytes of key1
* @key2: pointer to the second key to be compared
- * @kelen2: length in bytes of key2
+ * @keylen2: length in bytes of key2
*
* Returns 0 if @key1 and @key2 are equal.
* Returns >0 if @key1 is greater.
diff --git a/fs/befs/datastream.c b/fs/befs/datastream.c
index c467beb..1e8e0b8 100644
--- a/fs/befs/datastream.c
+++ b/fs/befs/datastream.c
@@ -116,7 +116,7 @@
* befs_read_lsmylink - read long symlink from datastream.
* @sb: Filesystem superblock
* @ds: Datastrem to read from
- * @buf: Buffer in which to place long symlink data
+ * @buff: Buffer in which to place long symlink data
* @len: Length of the long symlink in bytes
*
* Returns the number of bytes read
diff --git a/fs/befs/linuxvfs.c b/fs/befs/linuxvfs.c
index d626756..a16fbd4 100644
--- a/fs/befs/linuxvfs.c
+++ b/fs/befs/linuxvfs.c
@@ -133,14 +133,6 @@
befs_debug(sb, "---> befs_get_block() for inode %lu, block %ld",
(unsigned long)inode->i_ino, (long)block);
-
- if (block < 0) {
- befs_error(sb, "befs_get_block() was asked for a block "
- "number less than zero: block %ld in inode %lu",
- (long)block, (unsigned long)inode->i_ino);
- return -EIO;
- }
-
if (create) {
befs_error(sb, "befs_get_block() was asked to write to "
"block %ld in inode %lu", (long)block,
@@ -396,9 +388,8 @@
if (S_ISLNK(inode->i_mode) && !(befs_ino->i_flags & BEFS_LONG_SYMLINK)){
inode->i_size = 0;
inode->i_blocks = befs_sb->block_size / VFS_BLOCK_SIZE;
- strncpy(befs_ino->i_data.symlink, raw_inode->data.symlink,
- BEFS_SYMLINK_LEN - 1);
- befs_ino->i_data.symlink[BEFS_SYMLINK_LEN - 1] = '\0';
+ strlcpy(befs_ino->i_data.symlink, raw_inode->data.symlink,
+ BEFS_SYMLINK_LEN);
} else {
int num_blks;
@@ -591,21 +582,21 @@
/**
* befs_nls2utf - Convert NLS string to utf8 encodeing
* @sb: Superblock
- * @src: Input string buffer in NLS format
- * @srclen: Length of input string in bytes
- * @dest: The output string in UTF-8 format
- * @destlen: Length of the output buffer
+ * @in: Input string buffer in NLS format
+ * @in_len: Length of input string in bytes
+ * @out: The output string in UTF-8 format
+ * @out_len: Length of the output buffer
*
- * Converts input string @src, which is in the format of the loaded NLS map,
+ * Converts input string @in, which is in the format of the loaded NLS map,
* into a utf8 string.
*
- * The destination string @dest is allocated by this function and the caller is
+ * The destination string @out is allocated by this function and the caller is
* responsible for freeing it with kfree()
*
- * On return, *@destlen is the length of @dest in bytes.
+ * On return, *@out_len is the length of @out in bytes.
*
* On success, the return value is the number of utf8 characters written to
- * the output buffer @dest.
+ * the output buffer @out.
*
* On Failure, a negative number coresponding to the error code is returned.
*/
diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index fd38b50..484aaca 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -360,10 +360,13 @@
/*
* First time the inline_buf does not suffice
*/
- if (p->buf == p->inline_buf)
+ if (p->buf == p->inline_buf) {
tmp_buf = kmalloc(len, GFP_NOFS);
- else
+ if (tmp_buf)
+ memcpy(tmp_buf, p->buf, old_buf_len);
+ } else {
tmp_buf = krealloc(p->buf, len, GFP_NOFS);
+ }
if (!tmp_buf)
return -ENOMEM;
p->buf = tmp_buf;
diff --git a/fs/cachefiles/bind.c b/fs/cachefiles/bind.c
index 5b99baf..d749731 100644
--- a/fs/cachefiles/bind.c
+++ b/fs/cachefiles/bind.c
@@ -50,18 +50,18 @@
cache->brun_percent < 100);
if (*args) {
- kerror("'bind' command doesn't take an argument");
+ pr_err("'bind' command doesn't take an argument");
return -EINVAL;
}
if (!cache->rootdirname) {
- kerror("No cache directory specified");
+ pr_err("No cache directory specified");
return -EINVAL;
}
/* don't permit already bound caches to be re-bound */
if (test_bit(CACHEFILES_READY, &cache->flags)) {
- kerror("Cache already bound");
+ pr_err("Cache already bound");
return -EBUSY;
}
@@ -228,9 +228,7 @@
set_bit(CACHEFILES_READY, &cache->flags);
dput(root);
- printk(KERN_INFO "CacheFiles:"
- " File cache on %s registered\n",
- cache->cache.identifier);
+ pr_info("File cache on %s registered\n", cache->cache.identifier);
/* check how much space the cache has */
cachefiles_has_space(cache, 0, 0);
@@ -250,7 +248,7 @@
kmem_cache_free(cachefiles_object_jar, fsdef);
error_root_object:
cachefiles_end_secure(cache, saved_cred);
- kerror("Failed to register: %d", ret);
+ pr_err("Failed to register: %d", ret);
return ret;
}
@@ -262,9 +260,8 @@
_enter("");
if (test_bit(CACHEFILES_READY, &cache->flags)) {
- printk(KERN_INFO "CacheFiles:"
- " File cache on %s unregistering\n",
- cache->cache.identifier);
+ pr_info("File cache on %s unregistering\n",
+ cache->cache.identifier);
fscache_withdraw_cache(&cache->cache);
}
diff --git a/fs/cachefiles/daemon.c b/fs/cachefiles/daemon.c
index 0a1467b..b078d308 100644
--- a/fs/cachefiles/daemon.c
+++ b/fs/cachefiles/daemon.c
@@ -315,8 +315,7 @@
static int cachefiles_daemon_range_error(struct cachefiles_cache *cache,
char *args)
{
- kerror("Free space limits must be in range"
- " 0%%<=stop<cull<run<100%%");
+ pr_err("Free space limits must be in range 0%%<=stop<cull<run<100%%");
return -EINVAL;
}
@@ -476,12 +475,12 @@
_enter(",%s", args);
if (!*args) {
- kerror("Empty directory specified");
+ pr_err("Empty directory specified");
return -EINVAL;
}
if (cache->rootdirname) {
- kerror("Second cache directory specified");
+ pr_err("Second cache directory specified");
return -EEXIST;
}
@@ -504,12 +503,12 @@
_enter(",%s", args);
if (!*args) {
- kerror("Empty security context specified");
+ pr_err("Empty security context specified");
return -EINVAL;
}
if (cache->secctx) {
- kerror("Second security context specified");
+ pr_err("Second security context specified");
return -EINVAL;
}
@@ -532,7 +531,7 @@
_enter(",%s", args);
if (!*args) {
- kerror("Empty tag specified");
+ pr_err("Empty tag specified");
return -EINVAL;
}
@@ -563,12 +562,12 @@
goto inval;
if (!test_bit(CACHEFILES_READY, &cache->flags)) {
- kerror("cull applied to unready cache");
+ pr_err("cull applied to unready cache");
return -EIO;
}
if (test_bit(CACHEFILES_DEAD, &cache->flags)) {
- kerror("cull applied to dead cache");
+ pr_err("cull applied to dead cache");
return -EIO;
}
@@ -588,11 +587,11 @@
notdir:
path_put(&path);
- kerror("cull command requires dirfd to be a directory");
+ pr_err("cull command requires dirfd to be a directory");
return -ENOTDIR;
inval:
- kerror("cull command requires dirfd and filename");
+ pr_err("cull command requires dirfd and filename");
return -EINVAL;
}
@@ -615,7 +614,7 @@
return 0;
inval:
- kerror("debug command requires mask");
+ pr_err("debug command requires mask");
return -EINVAL;
}
@@ -635,12 +634,12 @@
goto inval;
if (!test_bit(CACHEFILES_READY, &cache->flags)) {
- kerror("inuse applied to unready cache");
+ pr_err("inuse applied to unready cache");
return -EIO;
}
if (test_bit(CACHEFILES_DEAD, &cache->flags)) {
- kerror("inuse applied to dead cache");
+ pr_err("inuse applied to dead cache");
return -EIO;
}
@@ -660,11 +659,11 @@
notdir:
path_put(&path);
- kerror("inuse command requires dirfd to be a directory");
+ pr_err("inuse command requires dirfd to be a directory");
return -ENOTDIR;
inval:
- kerror("inuse command requires dirfd and filename");
+ pr_err("inuse command requires dirfd and filename");
return -EINVAL;
}
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 57e17fe..584743d 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -146,8 +146,7 @@
if (ret < 0 && ret != -ETIMEDOUT) {
if (ret != -ENOBUFS)
- printk(KERN_WARNING
- "CacheFiles: Lookup failed error %d\n", ret);
+ pr_warn("Lookup failed error %d\n", ret);
fscache_object_lookup_error(&object->fscache);
}
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index 5349473..3d50998 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -9,6 +9,13 @@
* 2 of the Licence, or (at your option) any later version.
*/
+#ifdef pr_fmt
+#undef pr_fmt
+#endif
+
+#define pr_fmt(fmt) "CacheFiles: " fmt
+
+
#include <linux/fscache-cache.h>
#include <linux/timer.h>
#include <linux/wait.h>
@@ -245,11 +252,10 @@
/*
* error handling
*/
-#define kerror(FMT, ...) printk(KERN_ERR "CacheFiles: "FMT"\n", ##__VA_ARGS__)
#define cachefiles_io_error(___cache, FMT, ...) \
do { \
- kerror("I/O Error: " FMT, ##__VA_ARGS__); \
+ pr_err("I/O Error: " FMT, ##__VA_ARGS__); \
fscache_io_error(&(___cache)->cache); \
set_bit(CACHEFILES_DEAD, &(___cache)->flags); \
} while (0)
@@ -310,8 +316,8 @@
#define ASSERT(X) \
do { \
if (unlikely(!(X))) { \
- printk(KERN_ERR "\n"); \
- printk(KERN_ERR "CacheFiles: Assertion failed\n"); \
+ pr_err("\n"); \
+ pr_err("Assertion failed\n"); \
BUG(); \
} \
} while (0)
@@ -319,9 +325,9 @@
#define ASSERTCMP(X, OP, Y) \
do { \
if (unlikely(!((X) OP (Y)))) { \
- printk(KERN_ERR "\n"); \
- printk(KERN_ERR "CacheFiles: Assertion failed\n"); \
- printk(KERN_ERR "%lx " #OP " %lx is false\n", \
+ pr_err("\n"); \
+ pr_err("Assertion failed\n"); \
+ pr_err("%lx " #OP " %lx is false\n", \
(unsigned long)(X), (unsigned long)(Y)); \
BUG(); \
} \
@@ -330,8 +336,8 @@
#define ASSERTIF(C, X) \
do { \
if (unlikely((C) && !(X))) { \
- printk(KERN_ERR "\n"); \
- printk(KERN_ERR "CacheFiles: Assertion failed\n"); \
+ pr_err("\n"); \
+ pr_err("Assertion failed\n"); \
BUG(); \
} \
} while (0)
@@ -339,9 +345,9 @@
#define ASSERTIFCMP(C, X, OP, Y) \
do { \
if (unlikely((C) && !((X) OP (Y)))) { \
- printk(KERN_ERR "\n"); \
- printk(KERN_ERR "CacheFiles: Assertion failed\n"); \
- printk(KERN_ERR "%lx " #OP " %lx is false\n", \
+ pr_err("\n"); \
+ pr_err("Assertion failed\n"); \
+ pr_err("%lx " #OP " %lx is false\n", \
(unsigned long)(X), (unsigned long)(Y)); \
BUG(); \
} \
diff --git a/fs/cachefiles/main.c b/fs/cachefiles/main.c
index 4bfa8cf..180edfb 100644
--- a/fs/cachefiles/main.c
+++ b/fs/cachefiles/main.c
@@ -68,8 +68,7 @@
SLAB_HWCACHE_ALIGN,
cachefiles_object_init_once);
if (!cachefiles_object_jar) {
- printk(KERN_NOTICE
- "CacheFiles: Failed to allocate an object jar\n");
+ pr_notice("Failed to allocate an object jar\n");
goto error_object_jar;
}
@@ -77,7 +76,7 @@
if (ret < 0)
goto error_proc;
- printk(KERN_INFO "CacheFiles: Loaded\n");
+ pr_info("Loaded\n");
return 0;
error_proc:
@@ -85,7 +84,7 @@
error_object_jar:
misc_deregister(&cachefiles_dev);
error_dev:
- kerror("failed to register: %d", ret);
+ pr_err("failed to register: %d", ret);
return ret;
}
@@ -96,7 +95,7 @@
*/
static void __exit cachefiles_exit(void)
{
- printk(KERN_INFO "CacheFiles: Unloading\n");
+ pr_info("Unloading\n");
cachefiles_proc_cleanup();
kmem_cache_destroy(cachefiles_object_jar);
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index c0a6817..5bf2b41 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -35,22 +35,21 @@
struct fscache_cookie *cookie;
unsigned keylen, loop;
- printk(KERN_ERR "%sobject: OBJ%x\n",
- prefix, object->fscache.debug_id);
- printk(KERN_ERR "%sobjstate=%s fl=%lx wbusy=%x ev=%lx[%lx]\n",
+ pr_err("%sobject: OBJ%x\n", prefix, object->fscache.debug_id);
+ pr_err("%sobjstate=%s fl=%lx wbusy=%x ev=%lx[%lx]\n",
prefix, object->fscache.state->name,
object->fscache.flags, work_busy(&object->fscache.work),
object->fscache.events, object->fscache.event_mask);
- printk(KERN_ERR "%sops=%u inp=%u exc=%u\n",
+ pr_err("%sops=%u inp=%u exc=%u\n",
prefix, object->fscache.n_ops, object->fscache.n_in_progress,
object->fscache.n_exclusive);
- printk(KERN_ERR "%sparent=%p\n",
+ pr_err("%sparent=%p\n",
prefix, object->fscache.parent);
spin_lock(&object->fscache.lock);
cookie = object->fscache.cookie;
if (cookie) {
- printk(KERN_ERR "%scookie=%p [pr=%p nd=%p fl=%lx]\n",
+ pr_err("%scookie=%p [pr=%p nd=%p fl=%lx]\n",
prefix,
object->fscache.cookie,
object->fscache.cookie->parent,
@@ -62,16 +61,16 @@
else
keylen = 0;
} else {
- printk(KERN_ERR "%scookie=NULL\n", prefix);
+ pr_err("%scookie=NULL\n", prefix);
keylen = 0;
}
spin_unlock(&object->fscache.lock);
if (keylen) {
- printk(KERN_ERR "%skey=[%u] '", prefix, keylen);
+ pr_err("%skey=[%u] '", prefix, keylen);
for (loop = 0; loop < keylen; loop++)
- printk("%02x", keybuf[loop]);
- printk("'\n");
+ pr_cont("%02x", keybuf[loop]);
+ pr_cont("'\n");
}
}
@@ -131,13 +130,11 @@
dentry);
if (fscache_object_is_live(&object->fscache)) {
- printk(KERN_ERR "\n");
- printk(KERN_ERR "CacheFiles: Error:"
- " Can't preemptively bury live object\n");
+ pr_err("\n");
+ pr_err("Error: Can't preemptively bury live object\n");
cachefiles_printk_object(object, NULL);
} else if (test_and_set_bit(CACHEFILES_OBJECT_BURIED, &object->flags)) {
- printk(KERN_ERR "CacheFiles: Error:"
- " Object already preemptively buried\n");
+ pr_err("Error: Object already preemptively buried\n");
}
write_unlock(&cache->active_lock);
@@ -160,7 +157,7 @@
write_lock(&cache->active_lock);
if (test_and_set_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags)) {
- printk(KERN_ERR "CacheFiles: Error: Object already active\n");
+ pr_err("Error: Object already active\n");
cachefiles_printk_object(object, NULL);
BUG();
}
@@ -193,9 +190,8 @@
* need to wait for it to be destroyed */
wait_for_old_object:
if (fscache_object_is_live(&object->fscache)) {
- printk(KERN_ERR "\n");
- printk(KERN_ERR "CacheFiles: Error:"
- " Unexpected object collision\n");
+ pr_err("\n");
+ pr_err("Error: Unexpected object collision\n");
cachefiles_printk_object(object, xobject);
BUG();
}
@@ -241,9 +237,8 @@
}
if (timeout <= 0) {
- printk(KERN_ERR "\n");
- printk(KERN_ERR "CacheFiles: Error: Overlong"
- " wait for old active object to go away\n");
+ pr_err("\n");
+ pr_err("Error: Overlong wait for old active object to go away\n");
cachefiles_printk_object(object, xobject);
goto requeue;
}
@@ -548,7 +543,7 @@
next, next->d_inode, next->d_inode->i_ino);
} else if (!S_ISDIR(next->d_inode->i_mode)) {
- kerror("inode %lu is not a directory",
+ pr_err("inode %lu is not a directory",
next->d_inode->i_ino);
ret = -ENOBUFS;
goto error;
@@ -579,7 +574,7 @@
} else if (!S_ISDIR(next->d_inode->i_mode) &&
!S_ISREG(next->d_inode->i_mode)
) {
- kerror("inode %lu is not a file or directory",
+ pr_err("inode %lu is not a file or directory",
next->d_inode->i_ino);
ret = -ENOBUFS;
goto error;
@@ -773,7 +768,7 @@
ASSERT(subdir->d_inode);
if (!S_ISDIR(subdir->d_inode->i_mode)) {
- kerror("%s is not a directory", dirname);
+ pr_err("%s is not a directory", dirname);
ret = -EIO;
goto check_error;
}
@@ -800,13 +795,13 @@
mkdir_error:
mutex_unlock(&dir->d_inode->i_mutex);
dput(subdir);
- kerror("mkdir %s failed with error %d", dirname, ret);
+ pr_err("mkdir %s failed with error %d", dirname, ret);
return ERR_PTR(ret);
lookup_error:
mutex_unlock(&dir->d_inode->i_mutex);
ret = PTR_ERR(subdir);
- kerror("Lookup %s failed with error %d", dirname, ret);
+ pr_err("Lookup %s failed with error %d", dirname, ret);
return ERR_PTR(ret);
nomem_d_alloc:
@@ -896,7 +891,7 @@
if (ret == -EIO) {
cachefiles_io_error(cache, "Lookup failed");
} else if (ret != -ENOMEM) {
- kerror("Internal error: %d", ret);
+ pr_err("Internal error: %d", ret);
ret = -EIO;
}
@@ -955,7 +950,7 @@
}
if (ret != -ENOMEM) {
- kerror("Internal error: %d", ret);
+ pr_err("Internal error: %d", ret);
ret = -EIO;
}
diff --git a/fs/cachefiles/security.c b/fs/cachefiles/security.c
index 039b501..396c18e 100644
--- a/fs/cachefiles/security.c
+++ b/fs/cachefiles/security.c
@@ -34,9 +34,7 @@
ret = set_security_override_from_ctx(new, cache->secctx);
if (ret < 0) {
put_cred(new);
- printk(KERN_ERR "CacheFiles:"
- " Security denies permission to nominate"
- " security context: error %d\n",
+ pr_err("Security denies permission to nominate security context: error %d\n",
ret);
goto error;
}
@@ -59,16 +57,14 @@
ret = security_inode_mkdir(root->d_inode, root, 0);
if (ret < 0) {
- printk(KERN_ERR "CacheFiles:"
- " Security denies permission to make dirs: error %d",
+ pr_err("Security denies permission to make dirs: error %d",
ret);
return ret;
}
ret = security_inode_create(root->d_inode, root, 0);
if (ret < 0)
- printk(KERN_ERR "CacheFiles:"
- " Security denies permission to create files: error %d",
+ pr_err("Security denies permission to create files: error %d",
ret);
return ret;
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index 12b0eef..1ad51ffb 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -51,7 +51,7 @@
}
if (ret != -EEXIST) {
- kerror("Can't set xattr on %*.*s [%lu] (err %d)",
+ pr_err("Can't set xattr on %*.*s [%lu] (err %d)",
dentry->d_name.len, dentry->d_name.len,
dentry->d_name.name, dentry->d_inode->i_ino,
-ret);
@@ -64,7 +64,7 @@
if (ret == -ERANGE)
goto bad_type_length;
- kerror("Can't read xattr on %*.*s [%lu] (err %d)",
+ pr_err("Can't read xattr on %*.*s [%lu] (err %d)",
dentry->d_name.len, dentry->d_name.len,
dentry->d_name.name, dentry->d_inode->i_ino,
-ret);
@@ -85,14 +85,14 @@
return ret;
bad_type_length:
- kerror("Cache object %lu type xattr length incorrect",
+ pr_err("Cache object %lu type xattr length incorrect",
dentry->d_inode->i_ino);
ret = -EIO;
goto error;
bad_type:
xtype[2] = 0;
- kerror("Cache object %*.*s [%lu] type %s not %s",
+ pr_err("Cache object %*.*s [%lu] type %s not %s",
dentry->d_name.len, dentry->d_name.len,
dentry->d_name.name, dentry->d_inode->i_ino,
xtype, type);
@@ -293,7 +293,7 @@
return ret;
bad_type_length:
- kerror("Cache object %lu xattr length incorrect",
+ pr_err("Cache object %lu xattr length incorrect",
dentry->d_inode->i_ino);
ret = -EIO;
goto error;
diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
index b53278c..65a30e8 100644
--- a/fs/ceph/addr.c
+++ b/fs/ceph/addr.c
@@ -694,7 +694,7 @@
(wbc->sync_mode == WB_SYNC_ALL ? "ALL" : "HOLD"));
if (fsc->mount_state == CEPH_MOUNT_SHUTDOWN) {
- pr_warning("writepage_start %p on forced umount\n", inode);
+ pr_warn("writepage_start %p on forced umount\n", inode);
return -EIO; /* we're in a forced umount, don't write! */
}
if (fsc->mount_options->wsize && fsc->mount_options->wsize < wsize)
diff --git a/fs/ceph/debugfs.c b/fs/ceph/debugfs.c
index 16b54aa..5a743ac 100644
--- a/fs/ceph/debugfs.c
+++ b/fs/ceph/debugfs.c
@@ -71,9 +71,9 @@
seq_printf(s, "%s", ceph_mds_op_name(req->r_op));
if (req->r_got_unsafe)
- seq_printf(s, "\t(unsafe)");
+ seq_puts(s, "\t(unsafe)");
else
- seq_printf(s, "\t");
+ seq_puts(s, "\t");
if (req->r_inode) {
seq_printf(s, " #%llx", ceph_ino(req->r_inode));
@@ -119,7 +119,7 @@
seq_printf(s, " %s", req->r_path2);
}
- seq_printf(s, "\n");
+ seq_puts(s, "\n");
}
mutex_unlock(&mdsc->mutex);
diff --git a/fs/ceph/inode.c b/fs/ceph/inode.c
index 233c6f9..e4fff9f 100644
--- a/fs/ceph/inode.c
+++ b/fs/ceph/inode.c
@@ -821,7 +821,7 @@
spin_unlock(&ci->i_ceph_lock);
}
} else if (cap_fmode >= 0) {
- pr_warning("mds issued no caps on %llx.%llx\n",
+ pr_warn("mds issued no caps on %llx.%llx\n",
ceph_vinop(inode));
__ceph_get_fmode(ci, cap_fmode);
}
diff --git a/fs/ceph/mds_client.c b/fs/ceph/mds_client.c
index 2b4d093..9a33b98 100644
--- a/fs/ceph/mds_client.c
+++ b/fs/ceph/mds_client.c
@@ -2218,13 +2218,13 @@
/* dup? */
if ((req->r_got_unsafe && !head->safe) ||
(req->r_got_safe && head->safe)) {
- pr_warning("got a dup %s reply on %llu from mds%d\n",
+ pr_warn("got a dup %s reply on %llu from mds%d\n",
head->safe ? "safe" : "unsafe", tid, mds);
mutex_unlock(&mdsc->mutex);
goto out;
}
if (req->r_got_safe && !head->safe) {
- pr_warning("got unsafe after safe on %llu from mds%d\n",
+ pr_warn("got unsafe after safe on %llu from mds%d\n",
tid, mds);
mutex_unlock(&mdsc->mutex);
goto out;
@@ -3525,7 +3525,7 @@
struct ceph_mds_session *s = con->private;
struct ceph_mds_client *mdsc = s->s_mdsc;
- pr_warning("mds%d closed our session\n", s->s_mds);
+ pr_warn("mds%d closed our session\n", s->s_mds);
send_mds_reconnect(mdsc, s);
}
diff --git a/fs/ceph/mdsmap.c b/fs/ceph/mdsmap.c
index 132b64e..261531e 100644
--- a/fs/ceph/mdsmap.c
+++ b/fs/ceph/mdsmap.c
@@ -62,7 +62,7 @@
ceph_decode_16_safe(p, end, version, bad);
if (version > 3) {
- pr_warning("got mdsmap version %d > 3, failing", version);
+ pr_warn("got mdsmap version %d > 3, failing", version);
goto bad;
}
diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c
index 5be1f997..6aaa811 100644
--- a/fs/cifs/cifsfs.c
+++ b/fs/cifs/cifsfs.c
@@ -87,10 +87,6 @@
struct workqueue_struct *cifsiod_wq;
-#ifdef CONFIG_CIFS_SMB2
-__u8 cifs_client_guid[SMB2_CLIENT_GUID_SIZE];
-#endif
-
/*
* Bumps refcount for cifs super block.
* Note that it should be only called if a referece to VFS super block is
@@ -251,11 +247,7 @@
* server, can not assume caching of file data or metadata.
*/
cifs_set_oplock_level(cifs_inode, 0);
- cifs_inode->delete_pending = false;
- cifs_inode->invalid_mapping = false;
- clear_bit(CIFS_INODE_PENDING_OPLOCK_BREAK, &cifs_inode->flags);
- clear_bit(CIFS_INODE_PENDING_WRITERS, &cifs_inode->flags);
- clear_bit(CIFS_INODE_DOWNGRADE_OPLOCK_TO_L2, &cifs_inode->flags);
+ cifs_inode->flags = 0;
spin_lock_init(&cifs_inode->writers_lock);
cifs_inode->writers = 0;
cifs_inode->vfs_inode.i_blkbits = 14; /* 2**14 = CIFS_MAX_MSGSIZE */
@@ -302,7 +294,7 @@
struct sockaddr_in *sa = (struct sockaddr_in *) &server->dstaddr;
struct sockaddr_in6 *sa6 = (struct sockaddr_in6 *) &server->dstaddr;
- seq_printf(s, ",addr=");
+ seq_puts(s, ",addr=");
switch (server->dstaddr.ss_family) {
case AF_INET:
@@ -314,7 +306,7 @@
seq_printf(s, "%%%u", sa6->sin6_scope_id);
break;
default:
- seq_printf(s, "(unknown)");
+ seq_puts(s, "(unknown)");
}
}
@@ -324,45 +316,45 @@
if (ses->sectype == Unspecified)
return;
- seq_printf(s, ",sec=");
+ seq_puts(s, ",sec=");
switch (ses->sectype) {
case LANMAN:
- seq_printf(s, "lanman");
+ seq_puts(s, "lanman");
break;
case NTLMv2:
- seq_printf(s, "ntlmv2");
+ seq_puts(s, "ntlmv2");
break;
case NTLM:
- seq_printf(s, "ntlm");
+ seq_puts(s, "ntlm");
break;
case Kerberos:
- seq_printf(s, "krb5");
+ seq_puts(s, "krb5");
break;
case RawNTLMSSP:
- seq_printf(s, "ntlmssp");
+ seq_puts(s, "ntlmssp");
break;
default:
/* shouldn't ever happen */
- seq_printf(s, "unknown");
+ seq_puts(s, "unknown");
break;
}
if (ses->sign)
- seq_printf(s, "i");
+ seq_puts(s, "i");
}
static void
cifs_show_cache_flavor(struct seq_file *s, struct cifs_sb_info *cifs_sb)
{
- seq_printf(s, ",cache=");
+ seq_puts(s, ",cache=");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_STRICT_IO)
- seq_printf(s, "strict");
+ seq_puts(s, "strict");
else if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_DIRECT_IO)
- seq_printf(s, "none");
+ seq_puts(s, "none");
else
- seq_printf(s, "loose");
+ seq_puts(s, "loose");
}
static void
@@ -395,7 +387,7 @@
cifs_show_cache_flavor(s, cifs_sb);
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_MULTIUSER)
- seq_printf(s, ",multiuser");
+ seq_puts(s, ",multiuser");
else if (tcon->ses->user_name)
seq_printf(s, ",username=%s", tcon->ses->user_name);
@@ -421,16 +413,16 @@
seq_printf(s, ",uid=%u",
from_kuid_munged(&init_user_ns, cifs_sb->mnt_uid));
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_OVERR_UID)
- seq_printf(s, ",forceuid");
+ seq_puts(s, ",forceuid");
else
- seq_printf(s, ",noforceuid");
+ seq_puts(s, ",noforceuid");
seq_printf(s, ",gid=%u",
from_kgid_munged(&init_user_ns, cifs_sb->mnt_gid));
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_OVERR_GID)
- seq_printf(s, ",forcegid");
+ seq_puts(s, ",forcegid");
else
- seq_printf(s, ",noforcegid");
+ seq_puts(s, ",noforcegid");
cifs_show_address(s, tcon->ses->server);
@@ -442,47 +434,47 @@
cifs_show_nls(s, cifs_sb->local_nls);
if (tcon->seal)
- seq_printf(s, ",seal");
+ seq_puts(s, ",seal");
if (tcon->nocase)
- seq_printf(s, ",nocase");
+ seq_puts(s, ",nocase");
if (tcon->retry)
- seq_printf(s, ",hard");
+ seq_puts(s, ",hard");
if (tcon->unix_ext)
- seq_printf(s, ",unix");
+ seq_puts(s, ",unix");
else
- seq_printf(s, ",nounix");
+ seq_puts(s, ",nounix");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_POSIX_PATHS)
- seq_printf(s, ",posixpaths");
+ seq_puts(s, ",posixpaths");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_SET_UID)
- seq_printf(s, ",setuids");
+ seq_puts(s, ",setuids");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_SERVER_INUM)
- seq_printf(s, ",serverino");
+ seq_puts(s, ",serverino");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_RWPIDFORWARD)
- seq_printf(s, ",rwpidforward");
+ seq_puts(s, ",rwpidforward");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_NOPOSIXBRL)
- seq_printf(s, ",forcemand");
+ seq_puts(s, ",forcemand");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_NO_XATTR)
- seq_printf(s, ",nouser_xattr");
+ seq_puts(s, ",nouser_xattr");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_MAP_SPECIAL_CHR)
- seq_printf(s, ",mapchars");
+ seq_puts(s, ",mapchars");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_UNX_EMUL)
- seq_printf(s, ",sfu");
+ seq_puts(s, ",sfu");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_NO_BRL)
- seq_printf(s, ",nobrl");
+ seq_puts(s, ",nobrl");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_CIFS_ACL)
- seq_printf(s, ",cifsacl");
+ seq_puts(s, ",cifsacl");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_DYNPERM)
- seq_printf(s, ",dynperm");
+ seq_puts(s, ",dynperm");
if (root->d_sb->s_flags & MS_POSIXACL)
- seq_printf(s, ",acl");
+ seq_puts(s, ",acl");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_MF_SYMLINKS)
- seq_printf(s, ",mfsymlinks");
+ seq_puts(s, ",mfsymlinks");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_FSCACHE)
- seq_printf(s, ",fsc");
+ seq_puts(s, ",fsc");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_NOSSYNC)
- seq_printf(s, ",nostrictsync");
+ seq_puts(s, ",nostrictsync");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_NO_PERM)
- seq_printf(s, ",noperm");
+ seq_puts(s, ",noperm");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_CIFS_BACKUPUID)
seq_printf(s, ",backupuid=%u",
from_kuid_munged(&init_user_ns,
@@ -1192,10 +1184,6 @@
spin_lock_init(&cifs_file_list_lock);
spin_lock_init(&GlobalMid_Lock);
-#ifdef CONFIG_CIFS_SMB2
- get_random_bytes(cifs_client_guid, SMB2_CLIENT_GUID_SIZE);
-#endif
-
if (cifs_max_pending < 2) {
cifs_max_pending = 2;
cifs_dbg(FYI, "cifs_max_pending set to min of 2\n");
diff --git a/fs/cifs/cifsfs.h b/fs/cifs/cifsfs.h
index 26a754f..8fe5116 100644
--- a/fs/cifs/cifsfs.h
+++ b/fs/cifs/cifsfs.h
@@ -22,20 +22,28 @@
#ifndef _CIFSFS_H
#define _CIFSFS_H
+#include <linux/hash.h>
+
#define ROOT_I 2
/*
* ino_t is 32-bits on 32-bit arch. We have to squash the 64-bit value down
- * so that it will fit.
+ * so that it will fit. We use hash_64 to convert the value to 31 bits, and
+ * then add 1, to ensure that we don't end up with a 0 as the value.
*/
+#if BITS_PER_LONG == 64
static inline ino_t
cifs_uniqueid_to_ino_t(u64 fileid)
{
- ino_t ino = (ino_t) fileid;
- if (sizeof(ino_t) < sizeof(u64))
- ino ^= fileid >> (sizeof(u64)-sizeof(ino_t)) * 8;
- return ino;
+ return (ino_t)fileid;
}
+#else
+static inline ino_t
+cifs_uniqueid_to_ino_t(u64 fileid)
+{
+ return (ino_t)hash_64(fileid, (sizeof(ino_t) * 8) - 1) + 1;
+}
+#endif
extern struct file_system_type cifs_fs_type;
extern const struct address_space_operations cifs_addr_ops;
@@ -67,6 +75,8 @@
extern int cifs_revalidate_file(struct file *filp);
extern int cifs_revalidate_dentry(struct dentry *);
extern int cifs_invalidate_mapping(struct inode *inode);
+extern int cifs_revalidate_mapping(struct inode *inode);
+extern int cifs_zap_mapping(struct inode *inode);
extern int cifs_getattr(struct vfsmount *, struct dentry *, struct kstat *);
extern int cifs_setattr(struct dentry *, struct iattr *);
@@ -130,5 +140,5 @@
extern const struct export_operations cifs_export_ops;
#endif /* CONFIG_CIFS_NFSD_EXPORT */
-#define CIFS_VERSION "2.02"
+#define CIFS_VERSION "2.03"
#endif /* _CIFSFS_H */
diff --git a/fs/cifs/cifsglob.h b/fs/cifs/cifsglob.h
index 30f6e92..de6aed8 100644
--- a/fs/cifs/cifsglob.h
+++ b/fs/cifs/cifsglob.h
@@ -559,6 +559,7 @@
int echo_credits; /* echo reserved slots */
int oplock_credits; /* oplock break reserved slots */
bool echoes:1; /* enable echoes */
+ __u8 client_guid[SMB2_CLIENT_GUID_SIZE]; /* Client GUID */
#endif
u16 dialect; /* dialect index that server chose */
bool oplocks:1; /* enable oplocks */
@@ -1113,12 +1114,13 @@
__u32 cifsAttrs; /* e.g. DOS archive bit, sparse, compressed, system */
unsigned int oplock; /* oplock/lease level we have */
unsigned int epoch; /* used to track lease state changes */
- bool delete_pending; /* DELETE_ON_CLOSE is set */
- bool invalid_mapping; /* pagecache is invalid */
- unsigned long flags;
#define CIFS_INODE_PENDING_OPLOCK_BREAK (0) /* oplock break in progress */
#define CIFS_INODE_PENDING_WRITERS (1) /* Writes in progress */
#define CIFS_INODE_DOWNGRADE_OPLOCK_TO_L2 (2) /* Downgrade oplock to L2 */
+#define CIFS_INO_DELETE_PENDING (3) /* delete pending on server */
+#define CIFS_INO_INVALID_MAPPING (4) /* pagecache is invalid */
+#define CIFS_INO_LOCK (5) /* lock bit for synchronization */
+ unsigned long flags;
spinlock_t writers_lock;
unsigned int writers; /* Number of writers on this inode */
unsigned long time; /* jiffies of last update of inode */
diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index 8813ff7..20d75b8 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -2144,6 +2144,9 @@
sizeof(tcp_ses->srcaddr));
memcpy(&tcp_ses->dstaddr, &volume_info->dstaddr,
sizeof(tcp_ses->dstaddr));
+#ifdef CONFIG_CIFS_SMB2
+ get_random_bytes(tcp_ses->client_guid, SMB2_CLIENT_GUID_SIZE);
+#endif
/*
* at this point we are the only ones with the pointer
* to the struct since the kernel thread not created yet
@@ -2225,7 +2228,7 @@
vol->username ? vol->username : "",
CIFS_MAX_USERNAME_LEN))
return 0;
- if (strlen(vol->username) != 0 &&
+ if ((vol->username && strlen(vol->username) != 0) &&
ses->password != NULL &&
strncmp(ses->password,
vol->password ? vol->password : "",
diff --git a/fs/cifs/file.c b/fs/cifs/file.c
index 5ed03e0..208f56e 100644
--- a/fs/cifs/file.c
+++ b/fs/cifs/file.c
@@ -335,7 +335,7 @@
spin_unlock(&cifs_file_list_lock);
if (fid->purge_cache)
- cifs_invalidate_mapping(inode);
+ cifs_zap_mapping(inode);
file->private_data = cfile;
return cfile;
@@ -392,7 +392,7 @@
* again and get at least level II oplock.
*/
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_STRICT_IO)
- CIFS_I(inode)->invalid_mapping = true;
+ set_bit(CIFS_INO_INVALID_MAPPING, &cifsi->flags);
cifs_set_oplock_level(cifsi, 0);
}
spin_unlock(&cifs_file_list_lock);
@@ -1529,7 +1529,7 @@
*/
if (!CIFS_CACHE_WRITE(CIFS_I(inode)) &&
CIFS_CACHE_READ(CIFS_I(inode))) {
- cifs_invalidate_mapping(inode);
+ cifs_zap_mapping(inode);
cifs_dbg(FYI, "Set no oplock for inode=%p due to mand locks\n",
inode);
CIFS_I(inode)->oplock = 0;
@@ -2218,7 +2218,7 @@
file->f_path.dentry->d_name.name, datasync);
if (!CIFS_CACHE_READ(CIFS_I(inode))) {
- rc = cifs_invalidate_mapping(inode);
+ rc = cifs_zap_mapping(inode);
if (rc) {
cifs_dbg(FYI, "rc: %d during invalidate phase\n", rc);
rc = 0; /* don't care about it in fsync */
@@ -2562,7 +2562,7 @@
written = cifs_iovec_write(iocb->ki_filp, iov, nr_segs, &pos);
if (written > 0) {
- CIFS_I(inode)->invalid_mapping = true;
+ set_bit(CIFS_INO_INVALID_MAPPING, &CIFS_I(inode)->flags);
iocb->ki_pos = pos;
}
@@ -2649,7 +2649,7 @@
* request comes - break it on the client to prevent reading
* an old data.
*/
- cifs_invalidate_mapping(inode);
+ cifs_zap_mapping(inode);
cifs_dbg(FYI, "Set no oplock for inode=%p after a write operation\n",
inode);
cinode->oplock = 0;
@@ -3112,7 +3112,7 @@
xid = get_xid();
if (!CIFS_CACHE_READ(CIFS_I(inode))) {
- rc = cifs_invalidate_mapping(inode);
+ rc = cifs_zap_mapping(inode);
if (rc)
return rc;
}
@@ -3670,7 +3670,7 @@
if (!CIFS_CACHE_READ(cinode)) {
rc = filemap_fdatawait(inode->i_mapping);
mapping_set_error(inode->i_mapping, rc);
- cifs_invalidate_mapping(inode);
+ cifs_zap_mapping(inode);
}
cifs_dbg(FYI, "Oplock flush inode %p rc %d\n", inode, rc);
}
diff --git a/fs/cifs/inode.c b/fs/cifs/inode.c
index a22d667..a174605 100644
--- a/fs/cifs/inode.c
+++ b/fs/cifs/inode.c
@@ -22,6 +22,7 @@
#include <linux/stat.h>
#include <linux/slab.h>
#include <linux/pagemap.h>
+#include <linux/freezer.h>
#include <asm/div64.h>
#include "cifsfs.h"
#include "cifspdu.h"
@@ -117,7 +118,7 @@
cifs_dbg(FYI, "%s: invalidating inode %llu mapping\n",
__func__, cifs_i->uniqueid);
- cifs_i->invalid_mapping = true;
+ set_bit(CIFS_INO_INVALID_MAPPING, &cifs_i->flags);
}
/*
@@ -177,7 +178,10 @@
else
cifs_i->time = jiffies;
- cifs_i->delete_pending = fattr->cf_flags & CIFS_FATTR_DELETE_PENDING;
+ if (fattr->cf_flags & CIFS_FATTR_DELETE_PENDING)
+ set_bit(CIFS_INO_DELETE_PENDING, &cifs_i->flags);
+ else
+ clear_bit(CIFS_INO_DELETE_PENDING, &cifs_i->flags);
cifs_i->server_eof = fattr->cf_eof;
/*
@@ -1121,7 +1125,7 @@
}
/* try to set DELETE_ON_CLOSE */
- if (!cifsInode->delete_pending) {
+ if (!test_bit(CIFS_INO_DELETE_PENDING, &cifsInode->flags)) {
rc = CIFSSMBSetFileDisposition(xid, tcon, true, fid.netfid,
current->tgid);
/*
@@ -1138,7 +1142,7 @@
rc = -EBUSY;
goto undo_rename;
}
- cifsInode->delete_pending = true;
+ set_bit(CIFS_INO_DELETE_PENDING, &cifsInode->flags);
}
out_close:
@@ -1759,23 +1763,62 @@
cifs_invalidate_mapping(struct inode *inode)
{
int rc = 0;
- struct cifsInodeInfo *cifs_i = CIFS_I(inode);
-
- cifs_i->invalid_mapping = false;
if (inode->i_mapping && inode->i_mapping->nrpages != 0) {
rc = invalidate_inode_pages2(inode->i_mapping);
- if (rc) {
+ if (rc)
cifs_dbg(VFS, "%s: could not invalidate inode %p\n",
__func__, inode);
- cifs_i->invalid_mapping = true;
- }
}
cifs_fscache_reset_inode_cookie(inode);
return rc;
}
+/**
+ * cifs_wait_bit_killable - helper for functions that are sleeping on bit locks
+ * @word: long word containing the bit lock
+ */
+static int
+cifs_wait_bit_killable(void *word)
+{
+ if (fatal_signal_pending(current))
+ return -ERESTARTSYS;
+ freezable_schedule_unsafe();
+ return 0;
+}
+
+int
+cifs_revalidate_mapping(struct inode *inode)
+{
+ int rc;
+ unsigned long *flags = &CIFS_I(inode)->flags;
+
+ rc = wait_on_bit_lock(flags, CIFS_INO_LOCK, cifs_wait_bit_killable,
+ TASK_KILLABLE);
+ if (rc)
+ return rc;
+
+ if (test_and_clear_bit(CIFS_INO_INVALID_MAPPING, flags)) {
+ rc = cifs_invalidate_mapping(inode);
+ if (rc)
+ set_bit(CIFS_INO_INVALID_MAPPING, flags);
+ }
+
+ clear_bit_unlock(CIFS_INO_LOCK, flags);
+ smp_mb__after_atomic();
+ wake_up_bit(flags, CIFS_INO_LOCK);
+
+ return rc;
+}
+
+int
+cifs_zap_mapping(struct inode *inode)
+{
+ set_bit(CIFS_INO_INVALID_MAPPING, &CIFS_I(inode)->flags);
+ return cifs_revalidate_mapping(inode);
+}
+
int cifs_revalidate_file_attr(struct file *filp)
{
int rc = 0;
@@ -1842,9 +1885,7 @@
if (rc)
return rc;
- if (CIFS_I(inode)->invalid_mapping)
- rc = cifs_invalidate_mapping(inode);
- return rc;
+ return cifs_revalidate_mapping(inode);
}
/* revalidate a dentry's inode attributes */
@@ -1857,9 +1898,7 @@
if (rc)
return rc;
- if (CIFS_I(inode)->invalid_mapping)
- rc = cifs_invalidate_mapping(inode);
- return rc;
+ return cifs_revalidate_mapping(inode);
}
int cifs_getattr(struct vfsmount *mnt, struct dentry *dentry,
diff --git a/fs/cifs/ioctl.c b/fs/cifs/ioctl.c
index 7749230..45cb59b 100644
--- a/fs/cifs/ioctl.c
+++ b/fs/cifs/ioctl.c
@@ -85,7 +85,7 @@
goto out_fput;
}
- src_inode = src_file.file->f_dentry->d_inode;
+ src_inode = file_inode(src_file.file);
/*
* Note: cifs case is easier than btrfs since server responsible for
diff --git a/fs/cifs/netmisc.c b/fs/cifs/netmisc.c
index 0498845..6834b9c 100644
--- a/fs/cifs/netmisc.c
+++ b/fs/cifs/netmisc.c
@@ -795,8 +795,8 @@
while (nt_errs[idx].nt_errstr != NULL) {
if (((nt_errs[idx].nt_errcode) & 0xFFFFFF) ==
(status_code & 0xFFFFFF)) {
- printk(KERN_NOTICE "Status code returned 0x%08x %s\n",
- status_code, nt_errs[idx].nt_errstr);
+ pr_notice("Status code returned 0x%08x %s\n",
+ status_code, nt_errs[idx].nt_errstr);
}
idx++;
}
@@ -941,8 +941,9 @@
return (u64) t.tv_sec * 10000000 + t.tv_nsec/100 + NTFS_TIME_OFFSET;
}
-static int total_days_of_prev_months[] =
-{0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334};
+static const int total_days_of_prev_months[] = {
+ 0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334
+};
struct timespec cnvrtDosUnixTm(__le16 le_date, __le16 le_time, int offset)
{
diff --git a/fs/cifs/smb2ops.c b/fs/cifs/smb2ops.c
index 35ddc3e..787844b 100644
--- a/fs/cifs/smb2ops.c
+++ b/fs/cifs/smb2ops.c
@@ -1047,6 +1047,7 @@
buf->ccontext.NameOffset = cpu_to_le16(offsetof
(struct create_lease, Name));
buf->ccontext.NameLength = cpu_to_le16(4);
+ /* SMB2_CREATE_REQUEST_LEASE is "RqLs" */
buf->Name[0] = 'R';
buf->Name[1] = 'q';
buf->Name[2] = 'L';
@@ -1073,6 +1074,7 @@
buf->ccontext.NameOffset = cpu_to_le16(offsetof
(struct create_lease_v2, Name));
buf->ccontext.NameLength = cpu_to_le16(4);
+ /* SMB2_CREATE_REQUEST_LEASE is "RqLs" */
buf->Name[0] = 'R';
buf->Name[1] = 'q';
buf->Name[2] = 'L';
diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
index 3802f8c..b0b260d 100644
--- a/fs/cifs/smb2pdu.c
+++ b/fs/cifs/smb2pdu.c
@@ -375,7 +375,12 @@
req->Capabilities = cpu_to_le32(ses->server->vals->req_capabilities);
- memcpy(req->ClientGUID, cifs_client_guid, SMB2_CLIENT_GUID_SIZE);
+ /* ClientGUID must be zero for SMB2.02 dialect */
+ if (ses->server->vals->protocol_id == SMB20_PROT_ID)
+ memset(req->ClientGUID, 0, SMB2_CLIENT_GUID_SIZE);
+ else
+ memcpy(req->ClientGUID, server->client_guid,
+ SMB2_CLIENT_GUID_SIZE);
iov[0].iov_base = (char *)req;
/* 4 for rfc1002 length field */
@@ -478,7 +483,8 @@
vneg_inbuf.Capabilities =
cpu_to_le32(tcon->ses->server->vals->req_capabilities);
- memcpy(vneg_inbuf.Guid, cifs_client_guid, SMB2_CLIENT_GUID_SIZE);
+ memcpy(vneg_inbuf.Guid, tcon->ses->server->client_guid,
+ SMB2_CLIENT_GUID_SIZE);
if (tcon->ses->sign)
vneg_inbuf.SecurityMode =
@@ -966,6 +972,7 @@
buf->ccontext.NameOffset = cpu_to_le16(offsetof
(struct create_durable, Name));
buf->ccontext.NameLength = cpu_to_le16(4);
+ /* SMB2_CREATE_DURABLE_HANDLE_REQUEST is "DHnQ" */
buf->Name[0] = 'D';
buf->Name[1] = 'H';
buf->Name[2] = 'n';
@@ -990,6 +997,7 @@
buf->ccontext.NameLength = cpu_to_le16(4);
buf->Data.Fid.PersistentFileId = fid->persistent_fid;
buf->Data.Fid.VolatileFileId = fid->volatile_fid;
+ /* SMB2_CREATE_DURABLE_HANDLE_RECONNECT is "DHnC" */
buf->Name[0] = 'D';
buf->Name[1] = 'H';
buf->Name[2] = 'n';
@@ -1089,6 +1097,7 @@
int rc = 0;
unsigned int num_iovecs = 2;
__u32 file_attributes = 0;
+ char *dhc_buf = NULL, *lc_buf = NULL;
cifs_dbg(FYI, "create/open\n");
@@ -1155,6 +1164,7 @@
kfree(copy_path);
return rc;
}
+ lc_buf = iov[num_iovecs-1].iov_base;
}
if (*oplock == SMB2_OPLOCK_LEVEL_BATCH) {
@@ -1169,9 +1179,10 @@
if (rc) {
cifs_small_buf_release(req);
kfree(copy_path);
- kfree(iov[num_iovecs-1].iov_base);
+ kfree(lc_buf);
return rc;
}
+ dhc_buf = iov[num_iovecs-1].iov_base;
}
rc = SendReceive2(xid, ses, iov, num_iovecs, &resp_buftype, 0);
@@ -1203,6 +1214,8 @@
*oplock = rsp->OplockLevel;
creat_exit:
kfree(copy_path);
+ kfree(lc_buf);
+ kfree(dhc_buf);
free_rsp_buf(resp_buftype, rsp);
return rc;
}
diff --git a/fs/cifs/smb2pdu.h b/fs/cifs/smb2pdu.h
index 2022c54..69f3595 100644
--- a/fs/cifs/smb2pdu.h
+++ b/fs/cifs/smb2pdu.h
@@ -183,8 +183,6 @@
#define SMB2_CLIENT_GUID_SIZE 16
-extern __u8 cifs_client_guid[SMB2_CLIENT_GUID_SIZE];
-
struct smb2_negotiate_req {
struct smb2_hdr hdr;
__le16 StructureSize; /* Must be 36 */
@@ -437,11 +435,15 @@
#define SMB2_CREATE_SD_BUFFER "SecD" /* security descriptor */
#define SMB2_CREATE_DURABLE_HANDLE_REQUEST "DHnQ"
#define SMB2_CREATE_DURABLE_HANDLE_RECONNECT "DHnC"
-#define SMB2_CREATE_ALLOCATION_SIZE "AlSi"
+#define SMB2_CREATE_ALLOCATION_SIZE "AISi"
#define SMB2_CREATE_QUERY_MAXIMAL_ACCESS_REQUEST "MxAc"
#define SMB2_CREATE_TIMEWARP_REQUEST "TWrp"
#define SMB2_CREATE_QUERY_ON_DISK_ID "QFid"
#define SMB2_CREATE_REQUEST_LEASE "RqLs"
+#define SMB2_CREATE_DURABLE_HANDLE_REQUEST_V2 "DH2Q"
+#define SMB2_CREATE_DURABLE_HANDLE_RECONNECT_V2 "DH2C"
+#define SMB2_CREATE_APP_INSTANCE_ID 0x45BCA66AEFA7F74A9008FA462E144D74
+#define SVHDX_OPEN_DEVICE_CONTEXT 0x83CE6F1AD851E0986E34401CC9BCFCE9
struct smb2_create_req {
struct smb2_hdr hdr;
diff --git a/fs/coda/cnode.c b/fs/coda/cnode.c
index 911cf30..7740b1c 100644
--- a/fs/coda/cnode.c
+++ b/fs/coda/cnode.c
@@ -101,7 +101,7 @@
inode = coda_iget(sb, fid, &attr);
if (IS_ERR(inode))
- printk("coda_cnode_make: coda_iget failed\n");
+ pr_warn("%s: coda_iget failed\n", __func__);
return inode;
}
@@ -137,7 +137,7 @@
unsigned long hash = coda_f2i(fid);
if ( !sb ) {
- printk("coda_fid_to_inode: no sb!\n");
+ pr_warn("%s: no sb!\n", __func__);
return NULL;
}
diff --git a/fs/coda/coda_linux.h b/fs/coda/coda_linux.h
index e7550cb..d42b725 100644
--- a/fs/coda/coda_linux.h
+++ b/fs/coda/coda_linux.h
@@ -12,6 +12,12 @@
#ifndef _LINUX_CODA_FS
#define _LINUX_CODA_FS
+#ifdef pr_fmt
+#undef pr_fmt
+#endif
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/kernel.h>
#include <linux/param.h>
#include <linux/mm.h>
@@ -63,7 +69,7 @@
else \
ptr = (cast)vzalloc((unsigned long) size); \
if (!ptr) \
- printk("kernel malloc returns 0 at %s:%d\n", __FILE__, __LINE__); \
+ pr_warn("kernel malloc returns 0 at %s:%d\n", __FILE__, __LINE__); \
} while (0)
diff --git a/fs/coda/dir.c b/fs/coda/dir.c
index 5efbb5e..cd8a632 100644
--- a/fs/coda/dir.c
+++ b/fs/coda/dir.c
@@ -102,7 +102,7 @@
int type = 0;
if (length > CODA_MAXNAMLEN) {
- printk(KERN_ERR "name too long: lookup, %s (%*s)\n",
+ pr_err("name too long: lookup, %s (%*s)\n",
coda_i2s(dir), (int)length, name);
return ERR_PTR(-ENAMETOOLONG);
}
@@ -453,23 +453,23 @@
ret = kernel_read(host_file, ctx->pos - 2, (char *)vdir,
sizeof(*vdir));
if (ret < 0) {
- printk(KERN_ERR "coda readdir: read dir %s failed %d\n",
- coda_f2s(&cii->c_fid), ret);
+ pr_err("%s: read dir %s failed %d\n",
+ __func__, coda_f2s(&cii->c_fid), ret);
break;
}
if (ret == 0) break; /* end of directory file reached */
/* catch truncated reads */
if (ret < vdir_size || ret < vdir_size + vdir->d_namlen) {
- printk(KERN_ERR "coda readdir: short read on %s\n",
- coda_f2s(&cii->c_fid));
+ pr_err("%s: short read on %s\n",
+ __func__, coda_f2s(&cii->c_fid));
ret = -EBADF;
break;
}
/* validate whether the directory file actually makes sense */
if (vdir->d_reclen < vdir_size + vdir->d_namlen) {
- printk(KERN_ERR "coda readdir: invalid dir %s\n",
- coda_f2s(&cii->c_fid));
+ pr_err("%s: invalid dir %s\n",
+ __func__, coda_f2s(&cii->c_fid));
ret = -EBADF;
break;
}
@@ -589,8 +589,8 @@
coda_vattr_to_iattr(inode, &attr);
if ((old_mode & S_IFMT) != (inode->i_mode & S_IFMT)) {
- printk("Coda: inode %ld, fid %s changed type!\n",
- inode->i_ino, coda_f2s(&(cii->c_fid)));
+ pr_warn("inode %ld, fid %s changed type!\n",
+ inode->i_ino, coda_f2s(&(cii->c_fid)));
}
/* the following can happen when a local fid is replaced
diff --git a/fs/coda/inode.c b/fs/coda/inode.c
index d9c7751..fe3afb2 100644
--- a/fs/coda/inode.c
+++ b/fs/coda/inode.c
@@ -119,12 +119,12 @@
int idx;
if (data == NULL) {
- printk("coda_read_super: Bad mount data\n");
+ pr_warn("%s: Bad mount data\n", __func__);
return -1;
}
if (data->version != CODA_MOUNT_VERSION) {
- printk("coda_read_super: Bad mount version\n");
+ pr_warn("%s: Bad mount version\n", __func__);
return -1;
}
@@ -141,13 +141,13 @@
fdput(f);
if (idx < 0 || idx >= MAX_CODADEVS) {
- printk("coda_read_super: Bad minor number\n");
+ pr_warn("%s: Bad minor number\n", __func__);
return -1;
}
return idx;
Ebadf:
- printk("coda_read_super: Bad file\n");
+ pr_warn("%s: Bad file\n", __func__);
return -1;
}
@@ -168,19 +168,19 @@
if(idx == -1)
idx = 0;
- printk(KERN_INFO "coda_read_super: device index: %i\n", idx);
+ pr_info("%s: device index: %i\n", __func__, idx);
vc = &coda_comms[idx];
mutex_lock(&vc->vc_mutex);
if (!vc->vc_inuse) {
- printk("coda_read_super: No pseudo device\n");
+ pr_warn("%s: No pseudo device\n", __func__);
error = -EINVAL;
goto unlock_out;
}
if (vc->vc_sb) {
- printk("coda_read_super: Device already mounted\n");
+ pr_warn("%s: Device already mounted\n", __func__);
error = -EBUSY;
goto unlock_out;
}
@@ -204,22 +204,23 @@
/* get root fid from Venus: this needs the root inode */
error = venus_rootfid(sb, &fid);
if ( error ) {
- printk("coda_read_super: coda_get_rootfid failed with %d\n",
- error);
+ pr_warn("%s: coda_get_rootfid failed with %d\n",
+ __func__, error);
goto error;
}
- printk("coda_read_super: rootfid is %s\n", coda_f2s(&fid));
+ pr_info("%s: rootfid is %s\n", __func__, coda_f2s(&fid));
/* make root inode */
root = coda_cnode_make(&fid, sb);
if (IS_ERR(root)) {
error = PTR_ERR(root);
- printk("Failure of coda_cnode_make for root: error %d\n", error);
+ pr_warn("Failure of coda_cnode_make for root: error %d\n",
+ error);
goto error;
}
- printk("coda_read_super: rootinode is %ld dev %s\n",
- root->i_ino, root->i_sb->s_id);
+ pr_info("%s: rootinode is %ld dev %s\n",
+ __func__, root->i_ino, root->i_sb->s_id);
sb->s_root = d_make_root(root);
if (!sb->s_root) {
error = -EINVAL;
@@ -246,7 +247,7 @@
sb->s_fs_info = NULL;
mutex_unlock(&vcp->vc_mutex);
- printk("Coda: Bye bye.\n");
+ pr_info("Bye bye.\n");
}
static void coda_evict_inode(struct inode *inode)
diff --git a/fs/coda/psdev.c b/fs/coda/psdev.c
index ebc2bae..5c1e424 100644
--- a/fs/coda/psdev.c
+++ b/fs/coda/psdev.c
@@ -114,14 +114,14 @@
int size = sizeof(*dcbuf);
if ( nbytes < sizeof(struct coda_out_hdr) ) {
- printk("coda_downcall opc %d uniq %d, not enough!\n",
- hdr.opcode, hdr.unique);
+ pr_warn("coda_downcall opc %d uniq %d, not enough!\n",
+ hdr.opcode, hdr.unique);
count = nbytes;
goto out;
}
if ( nbytes > size ) {
- printk("Coda: downcall opc %d, uniq %d, too much!",
- hdr.opcode, hdr.unique);
+ pr_warn("downcall opc %d, uniq %d, too much!",
+ hdr.opcode, hdr.unique);
nbytes = size;
}
CODA_ALLOC(dcbuf, union outputArgs *, nbytes);
@@ -136,7 +136,8 @@
CODA_FREE(dcbuf, nbytes);
if (error) {
- printk("psdev_write: coda_downcall error: %d\n", error);
+ pr_warn("%s: coda_downcall error: %d\n",
+ __func__, error);
retval = error;
goto out;
}
@@ -157,16 +158,17 @@
mutex_unlock(&vcp->vc_mutex);
if (!req) {
- printk("psdev_write: msg (%d, %d) not found\n",
- hdr.opcode, hdr.unique);
+ pr_warn("%s: msg (%d, %d) not found\n",
+ __func__, hdr.opcode, hdr.unique);
retval = -ESRCH;
goto out;
}
/* move data into response buffer. */
if (req->uc_outSize < nbytes) {
- printk("psdev_write: too much cnt: %d, cnt: %ld, opc: %d, uniq: %d.\n",
- req->uc_outSize, (long)nbytes, hdr.opcode, hdr.unique);
+ pr_warn("%s: too much cnt: %d, cnt: %ld, opc: %d, uniq: %d.\n",
+ __func__, req->uc_outSize, (long)nbytes,
+ hdr.opcode, hdr.unique);
nbytes = req->uc_outSize; /* don't have more space! */
}
if (copy_from_user(req->uc_data, buf, nbytes)) {
@@ -240,8 +242,8 @@
/* Move the input args into userspace */
count = req->uc_inSize;
if (nbytes < req->uc_inSize) {
- printk ("psdev_read: Venus read %ld bytes of %d in message\n",
- (long)nbytes, req->uc_inSize);
+ pr_warn("%s: Venus read %ld bytes of %d in message\n",
+ __func__, (long)nbytes, req->uc_inSize);
count = nbytes;
}
@@ -305,7 +307,7 @@
struct upc_req *req, *tmp;
if (!vcp || !vcp->vc_inuse ) {
- printk("psdev_release: Not open.\n");
+ pr_warn("%s: Not open.\n", __func__);
return -1;
}
@@ -354,8 +356,8 @@
{
int i, err = 0;
if (register_chrdev(CODA_PSDEV_MAJOR, "coda", &coda_psdev_fops)) {
- printk(KERN_ERR "coda_psdev: unable to get major %d\n",
- CODA_PSDEV_MAJOR);
+ pr_err("%s: unable to get major %d\n",
+ __func__, CODA_PSDEV_MAJOR);
return -EIO;
}
coda_psdev_class = class_create(THIS_MODULE, "coda");
@@ -393,13 +395,13 @@
goto out2;
status = init_coda_psdev();
if ( status ) {
- printk("Problem (%d) in init_coda_psdev\n", status);
+ pr_warn("Problem (%d) in init_coda_psdev\n", status);
goto out1;
}
status = register_filesystem(&coda_fs_type);
if (status) {
- printk("coda: failed to register filesystem!\n");
+ pr_warn("failed to register filesystem!\n");
goto out;
}
return 0;
@@ -420,9 +422,8 @@
int err, i;
err = unregister_filesystem(&coda_fs_type);
- if ( err != 0 ) {
- printk("coda: failed to unregister filesystem\n");
- }
+ if (err != 0)
+ pr_warn("failed to unregister filesystem\n");
for (i = 0; i < MAX_CODADEVS; i++)
device_destroy(coda_psdev_class, MKDEV(CODA_PSDEV_MAJOR, i));
class_destroy(coda_psdev_class);
diff --git a/fs/coda/sysctl.c b/fs/coda/sysctl.c
index af56ad5..34218a8 100644
--- a/fs/coda/sysctl.c
+++ b/fs/coda/sysctl.c
@@ -14,7 +14,7 @@
#ifdef CONFIG_SYSCTL
static struct ctl_table_header *fs_table_header;
-static ctl_table coda_table[] = {
+static struct ctl_table coda_table[] = {
{
.procname = "timeout",
.data = &coda_timeout,
@@ -39,7 +39,7 @@
{}
};
-static ctl_table fs_table[] = {
+static struct ctl_table fs_table[] = {
{
.procname = "coda",
.mode = 0555,
diff --git a/fs/coda/upcall.c b/fs/coda/upcall.c
index 3a731976..21fcf8d 100644
--- a/fs/coda/upcall.c
+++ b/fs/coda/upcall.c
@@ -508,8 +508,8 @@
inp->coda_ioctl.data = (char *)(INSIZE(ioctl));
/* get the data out of user space */
- if ( copy_from_user((char*)inp + (long)inp->coda_ioctl.data,
- data->vi.in, data->vi.in_size) ) {
+ if (copy_from_user((char *)inp + (long)inp->coda_ioctl.data,
+ data->vi.in, data->vi.in_size)) {
error = -EINVAL;
goto exit;
}
@@ -518,8 +518,8 @@
&outsize, inp);
if (error) {
- printk("coda_pioctl: Venus returns: %d for %s\n",
- error, coda_f2s(fid));
+ pr_warn("%s: Venus returns: %d for %s\n",
+ __func__, error, coda_f2s(fid));
goto exit;
}
@@ -675,7 +675,7 @@
mutex_lock(&vcp->vc_mutex);
if (!vcp->vc_inuse) {
- printk(KERN_NOTICE "coda: Venus dead, not sending upcall\n");
+ pr_notice("Venus dead, not sending upcall\n");
error = -ENXIO;
goto exit;
}
@@ -725,7 +725,7 @@
error = -EINTR;
if ((req->uc_flags & CODA_REQ_ABORT) || !signal_pending(current)) {
- printk(KERN_WARNING "coda: Unexpected interruption.\n");
+ pr_warn("Unexpected interruption.\n");
goto exit;
}
@@ -735,7 +735,7 @@
/* Venus saw the upcall, make sure we can send interrupt signal */
if (!vcp->vc_inuse) {
- printk(KERN_INFO "coda: Venus dead, not sending signal.\n");
+ pr_info("Venus dead, not sending signal.\n");
goto exit;
}
diff --git a/fs/dcache.c b/fs/dcache.c
index be2bea8..1792d60 100644
--- a/fs/dcache.c
+++ b/fs/dcache.c
@@ -150,7 +150,7 @@
return sum < 0 ? 0 : sum;
}
-int proc_nr_dentry(ctl_table *table, int write, void __user *buffer,
+int proc_nr_dentry(struct ctl_table *table, int write, void __user *buffer,
size_t *lenp, loff_t *ppos)
{
dentry_stat.nr_dentry = get_nr_dentry();
diff --git a/fs/devpts/inode.c b/fs/devpts/inode.c
index c710380..cfe8466 100644
--- a/fs/devpts/inode.c
+++ b/fs/devpts/inode.c
@@ -10,6 +10,8 @@
*
* ------------------------------------------------------------------------- */
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/module.h>
#include <linux/init.h>
#include <linux/fs.h>
@@ -148,10 +150,10 @@
/*
* parse_mount_options():
- * Set @opts to mount options specified in @data. If an option is not
- * specified in @data, set it to its default value. The exception is
- * 'newinstance' option which can only be set/cleared on a mount (i.e.
- * cannot be changed during remount).
+ * Set @opts to mount options specified in @data. If an option is not
+ * specified in @data, set it to its default value. The exception is
+ * 'newinstance' option which can only be set/cleared on a mount (i.e.
+ * cannot be changed during remount).
*
* Note: @data may be NULL (in which case all options are set to default).
*/
@@ -225,7 +227,7 @@
break;
#endif
default:
- printk(KERN_ERR "devpts: called with bogus options\n");
+ pr_err("called with bogus options\n");
return -EINVAL;
}
}
@@ -261,7 +263,7 @@
dentry = d_alloc_name(root, "ptmx");
if (!dentry) {
- printk(KERN_NOTICE "Unable to alloc dentry for ptmx node\n");
+ pr_err("Unable to alloc dentry for ptmx node\n");
goto out;
}
@@ -270,7 +272,7 @@
*/
inode = new_inode(sb);
if (!inode) {
- printk(KERN_ERR "Unable to alloc inode for ptmx node\n");
+ pr_err("Unable to alloc inode for ptmx node\n");
dput(dentry);
goto out;
}
@@ -303,7 +305,7 @@
#else
static inline void update_ptmx_mode(struct pts_fs_info *fsi)
{
- return;
+ return;
}
#endif
@@ -333,9 +335,11 @@
struct pts_mount_opts *opts = &fsi->mount_opts;
if (opts->setuid)
- seq_printf(seq, ",uid=%u", from_kuid_munged(&init_user_ns, opts->uid));
+ seq_printf(seq, ",uid=%u",
+ from_kuid_munged(&init_user_ns, opts->uid));
if (opts->setgid)
- seq_printf(seq, ",gid=%u", from_kgid_munged(&init_user_ns, opts->gid));
+ seq_printf(seq, ",gid=%u",
+ from_kgid_munged(&init_user_ns, opts->gid));
seq_printf(seq, ",mode=%03o", opts->mode);
#ifdef CONFIG_DEVPTS_MULTIPLE_INSTANCES
seq_printf(seq, ",ptmxmode=%03o", opts->ptmxmode);
@@ -396,7 +400,7 @@
if (s->s_root)
return 0;
- printk(KERN_ERR "devpts: get root dentry failed\n");
+ pr_err("get root dentry failed\n");
fail:
return -ENOMEM;
diff --git a/fs/dlm/config.c b/fs/dlm/config.c
index 76feb4b..d521bdd 100644
--- a/fs/dlm/config.c
+++ b/fs/dlm/config.c
@@ -157,11 +157,13 @@
const char *buf, size_t len)
{
unsigned int x;
+ int rc;
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
-
- x = simple_strtoul(buf, NULL, 0);
+ rc = kstrtouint(buf, 0, &x);
+ if (rc)
+ return rc;
if (check_zero && !x)
return -EINVAL;
@@ -730,7 +732,10 @@
static ssize_t comm_nodeid_write(struct dlm_comm *cm, const char *buf,
size_t len)
{
- cm->nodeid = simple_strtol(buf, NULL, 0);
+ int rc = kstrtoint(buf, 0, &cm->nodeid);
+
+ if (rc)
+ return rc;
return len;
}
@@ -742,7 +747,10 @@
static ssize_t comm_local_write(struct dlm_comm *cm, const char *buf,
size_t len)
{
- cm->local= simple_strtol(buf, NULL, 0);
+ int rc = kstrtoint(buf, 0, &cm->local);
+
+ if (rc)
+ return rc;
if (cm->local && !local_comm)
local_comm = cm;
return len;
@@ -846,7 +854,10 @@
size_t len)
{
uint32_t seq = 0;
- nd->nodeid = simple_strtol(buf, NULL, 0);
+ int rc = kstrtoint(buf, 0, &nd->nodeid);
+
+ if (rc)
+ return rc;
dlm_comm_seq(nd->nodeid, &seq);
nd->comm_seq = seq;
return len;
@@ -860,7 +871,10 @@
static ssize_t node_weight_write(struct dlm_node *nd, const char *buf,
size_t len)
{
- nd->weight = simple_strtol(buf, NULL, 0);
+ int rc = kstrtoint(buf, 0, &nd->weight);
+
+ if (rc)
+ return rc;
return len;
}
diff --git a/fs/dlm/debug_fs.c b/fs/dlm/debug_fs.c
index b969dee..8d77ba7 100644
--- a/fs/dlm/debug_fs.c
+++ b/fs/dlm/debug_fs.c
@@ -68,7 +68,7 @@
if (lkb->lkb_wait_type)
seq_printf(s, " wait_type: %d", lkb->lkb_wait_type);
- return seq_printf(s, "\n");
+ return seq_puts(s, "\n");
}
static int print_format1(struct dlm_rsb *res, struct seq_file *s)
@@ -92,31 +92,31 @@
}
if (res->res_nodeid > 0)
- rv = seq_printf(s, "\" \nLocal Copy, Master is node %d\n",
+ rv = seq_printf(s, "\"\nLocal Copy, Master is node %d\n",
res->res_nodeid);
else if (res->res_nodeid == 0)
- rv = seq_printf(s, "\" \nMaster Copy\n");
+ rv = seq_puts(s, "\"\nMaster Copy\n");
else if (res->res_nodeid == -1)
- rv = seq_printf(s, "\" \nLooking up master (lkid %x)\n",
+ rv = seq_printf(s, "\"\nLooking up master (lkid %x)\n",
res->res_first_lkid);
else
- rv = seq_printf(s, "\" \nInvalid master %d\n",
+ rv = seq_printf(s, "\"\nInvalid master %d\n",
res->res_nodeid);
if (rv)
goto out;
/* Print the LVB: */
if (res->res_lvbptr) {
- seq_printf(s, "LVB: ");
+ seq_puts(s, "LVB: ");
for (i = 0; i < lvblen; i++) {
if (i == lvblen / 2)
- seq_printf(s, "\n ");
+ seq_puts(s, "\n ");
seq_printf(s, "%02x ",
(unsigned char) res->res_lvbptr[i]);
}
if (rsb_flag(res, RSB_VALNOTVALID))
- seq_printf(s, " (INVALID)");
- rv = seq_printf(s, "\n");
+ seq_puts(s, " (INVALID)");
+ rv = seq_puts(s, "\n");
if (rv)
goto out;
}
@@ -133,21 +133,21 @@
}
/* Print the locks attached to this resource */
- seq_printf(s, "Granted Queue\n");
+ seq_puts(s, "Granted Queue\n");
list_for_each_entry(lkb, &res->res_grantqueue, lkb_statequeue) {
rv = print_format1_lock(s, lkb, res);
if (rv)
goto out;
}
- seq_printf(s, "Conversion Queue\n");
+ seq_puts(s, "Conversion Queue\n");
list_for_each_entry(lkb, &res->res_convertqueue, lkb_statequeue) {
rv = print_format1_lock(s, lkb, res);
if (rv)
goto out;
}
- seq_printf(s, "Waiting Queue\n");
+ seq_puts(s, "Waiting Queue\n");
list_for_each_entry(lkb, &res->res_waitqueue, lkb_statequeue) {
rv = print_format1_lock(s, lkb, res);
if (rv)
@@ -157,13 +157,13 @@
if (list_empty(&res->res_lookup))
goto out;
- seq_printf(s, "Lookup Queue\n");
+ seq_puts(s, "Lookup Queue\n");
list_for_each_entry(lkb, &res->res_lookup, lkb_rsb_lookup) {
rv = seq_printf(s, "%08x %s", lkb->lkb_id,
print_lockmode(lkb->lkb_rqmode));
if (lkb->lkb_wait_type)
seq_printf(s, " wait_type: %d", lkb->lkb_wait_type);
- rv = seq_printf(s, "\n");
+ rv = seq_puts(s, "\n");
}
out:
unlock_rsb(res);
@@ -300,7 +300,7 @@
else
seq_printf(s, " %02x", (unsigned char)r->res_name[i]);
}
- rv = seq_printf(s, "\n");
+ rv = seq_puts(s, "\n");
if (rv)
goto out;
@@ -311,7 +311,7 @@
for (i = 0; i < lvblen; i++)
seq_printf(s, " %02x", (unsigned char)r->res_lvbptr[i]);
- rv = seq_printf(s, "\n");
+ rv = seq_puts(s, "\n");
if (rv)
goto out;
@@ -377,7 +377,7 @@
else
seq_printf(s, " %02x", (unsigned char)r->res_name[i]);
}
- rv = seq_printf(s, "\n");
+ rv = seq_puts(s, "\n");
out:
unlock_rsb(r);
return rv;
diff --git a/fs/dlm/lockspace.c b/fs/dlm/lockspace.c
index 04d6398..f3e7278 100644
--- a/fs/dlm/lockspace.c
+++ b/fs/dlm/lockspace.c
@@ -35,8 +35,11 @@
static ssize_t dlm_control_store(struct dlm_ls *ls, const char *buf, size_t len)
{
ssize_t ret = len;
- int n = simple_strtol(buf, NULL, 0);
+ int n;
+ int rc = kstrtoint(buf, 0, &n);
+ if (rc)
+ return rc;
ls = dlm_find_lockspace_local(ls->ls_local_handle);
if (!ls)
return -EINVAL;
@@ -57,7 +60,10 @@
static ssize_t dlm_event_store(struct dlm_ls *ls, const char *buf, size_t len)
{
- ls->ls_uevent_result = simple_strtol(buf, NULL, 0);
+ int rc = kstrtoint(buf, 0, &ls->ls_uevent_result);
+
+ if (rc)
+ return rc;
set_bit(LSFL_UEVENT_WAIT, &ls->ls_flags);
wake_up(&ls->ls_uevent_wait);
return len;
@@ -70,7 +76,10 @@
static ssize_t dlm_id_store(struct dlm_ls *ls, const char *buf, size_t len)
{
- ls->ls_global_id = simple_strtoul(buf, NULL, 0);
+ int rc = kstrtouint(buf, 0, &ls->ls_global_id);
+
+ if (rc)
+ return rc;
return len;
}
@@ -81,7 +90,11 @@
static ssize_t dlm_nodir_store(struct dlm_ls *ls, const char *buf, size_t len)
{
- int val = simple_strtoul(buf, NULL, 0);
+ int val;
+ int rc = kstrtoint(buf, 0, &val);
+
+ if (rc)
+ return rc;
if (val == 1)
set_bit(LSFL_NODIR, &ls->ls_flags);
return len;
diff --git a/fs/drop_caches.c b/fs/drop_caches.c
index 9280202..1de7294 100644
--- a/fs/drop_caches.c
+++ b/fs/drop_caches.c
@@ -50,7 +50,7 @@
} while (nr_objects > 10);
}
-int drop_caches_sysctl_handler(ctl_table *table, int write,
+int drop_caches_sysctl_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *length, loff_t *ppos)
{
int ret;
diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index af90312..b73e062 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -293,7 +293,7 @@
static long zero;
static long long_max = LONG_MAX;
-ctl_table epoll_table[] = {
+struct ctl_table epoll_table[] = {
{
.procname = "max_user_watches",
.data = &max_user_watches,
diff --git a/fs/exofs/Kconfig.ore b/fs/exofs/Kconfig.ore
index 1ca7fb7..2daf232 100644
--- a/fs/exofs/Kconfig.ore
+++ b/fs/exofs/Kconfig.ore
@@ -9,4 +9,6 @@
tristate
depends on EXOFS_FS || PNFS_OBJLAYOUT
select ASYNC_XOR
+ select RAID6_PQ
+ select ASYNC_PQ
default SCSI_OSD_ULD
diff --git a/fs/exofs/ore.c b/fs/exofs/ore.c
index dae8846..cfc0205 100644
--- a/fs/exofs/ore.c
+++ b/fs/exofs/ore.c
@@ -58,9 +58,12 @@
layout->parity = 1;
break;
case PNFS_OSD_RAID_PQ:
+ layout->parity = 2;
+ break;
case PNFS_OSD_RAID_4:
default:
- ORE_ERR("Only RAID_0/5 for now\n");
+ ORE_ERR("Only RAID_0/5/6 for now received-enum=%d\n",
+ layout->raid_algorithm);
return -EINVAL;
}
if (0 != (layout->stripe_unit & ~PAGE_MASK)) {
@@ -112,6 +115,8 @@
layout->max_io_length /= stripe_length;
layout->max_io_length *= stripe_length;
}
+ ORE_DBGMSG("max_io_length=0x%lx\n", layout->max_io_length);
+
return 0;
}
EXPORT_SYMBOL(ore_verify_layout);
@@ -545,21 +550,24 @@
/* "H - (N * U)" is just "H % U" so it's bound to u32 */
u32 C = (u32)(H - (N * U)) / stripe_unit + G * group_width;
+ u32 first_dev = C - C % group_width;
div_u64_rem(file_offset, stripe_unit, &si->unit_off);
si->obj_offset = si->unit_off + (N * stripe_unit) +
(M * group_depth * stripe_unit);
+ si->cur_comp = C - first_dev;
+ si->cur_pg = si->unit_off / PAGE_SIZE;
if (parity) {
u32 LCMdP = lcm(group_width, parity) / parity;
/* R = N % LCMdP; */
u32 RxP = (N % LCMdP) * parity;
- u32 first_dev = C - C % group_width;
si->par_dev = (group_width + group_width - parity - RxP) %
group_width + first_dev;
- si->dev = (group_width + C - RxP) % group_width + first_dev;
+ si->dev = (group_width + group_width + C - RxP) %
+ group_width + first_dev;
si->bytes_in_stripe = U;
si->first_stripe_start = M * S + G * T + N * U;
} else {
@@ -649,6 +657,43 @@
return ret;
}
+static int _add_parity_units(struct ore_io_state *ios,
+ struct ore_striping_info *si,
+ unsigned dev, unsigned first_dev,
+ unsigned mirrors_p1, unsigned devs_in_group,
+ unsigned cur_len)
+{
+ unsigned do_parity;
+ int ret = 0;
+
+ for (do_parity = ios->layout->parity; do_parity; --do_parity) {
+ struct ore_per_dev_state *per_dev;
+
+ per_dev = &ios->per_dev[dev - first_dev];
+ if (!per_dev->length && !per_dev->offset) {
+ /* Only/always the parity unit of the first
+ * stripe will be empty. So this is a chance to
+ * initialize the per_dev info.
+ */
+ per_dev->dev = dev;
+ per_dev->offset = si->obj_offset - si->unit_off;
+ }
+
+ ret = _ore_add_parity_unit(ios, si, per_dev, cur_len,
+ do_parity == 1);
+ if (unlikely(ret))
+ break;
+
+ if (do_parity != 1) {
+ dev = ((dev + mirrors_p1) % devs_in_group) + first_dev;
+ si->cur_comp = (si->cur_comp + 1) %
+ ios->layout->group_width;
+ }
+ }
+
+ return ret;
+}
+
static int _prepare_for_striping(struct ore_io_state *ios)
{
struct ore_striping_info *si = &ios->si;
@@ -658,7 +703,6 @@
unsigned devs_in_group = group_width * mirrors_p1;
unsigned dev = si->dev;
unsigned first_dev = dev - (dev % devs_in_group);
- unsigned dev_order;
unsigned cur_pg = ios->pages_consumed;
u64 length = ios->length;
int ret = 0;
@@ -670,16 +714,13 @@
BUG_ON(length > si->length);
- dev_order = _dev_order(devs_in_group, mirrors_p1, si->par_dev, dev);
- si->cur_comp = dev_order;
- si->cur_pg = si->unit_off / PAGE_SIZE;
-
while (length) {
- unsigned comp = dev - first_dev;
- struct ore_per_dev_state *per_dev = &ios->per_dev[comp];
+ struct ore_per_dev_state *per_dev =
+ &ios->per_dev[dev - first_dev];
unsigned cur_len, page_off = 0;
- if (!per_dev->length) {
+ if (!per_dev->length && !per_dev->offset) {
+ /* First time initialize the per_dev info. */
per_dev->dev = dev;
if (dev == si->dev) {
WARN_ON(dev == si->par_dev);
@@ -688,13 +729,7 @@
page_off = si->unit_off & ~PAGE_MASK;
BUG_ON(page_off && (page_off != ios->pgbase));
} else {
- if (si->cur_comp > dev_order)
- per_dev->offset =
- si->obj_offset - si->unit_off;
- else /* si->cur_comp < dev_order */
- per_dev->offset =
- si->obj_offset + stripe_unit -
- si->unit_off;
+ per_dev->offset = si->obj_offset - si->unit_off;
cur_len = stripe_unit;
}
} else {
@@ -708,11 +743,9 @@
if (unlikely(ret))
goto out;
- dev += mirrors_p1;
- dev = (dev % devs_in_group) + first_dev;
-
length -= cur_len;
+ dev = ((dev + mirrors_p1) % devs_in_group) + first_dev;
si->cur_comp = (si->cur_comp + 1) % group_width;
if (unlikely((dev == si->par_dev) || (!length && ios->sp2d))) {
if (!length && ios->sp2d) {
@@ -720,23 +753,16 @@
* stripe. then operate on parity dev.
*/
dev = si->par_dev;
- }
- if (ios->sp2d)
- /* In writes cur_len just means if it's the
- * last one. See _ore_add_parity_unit.
- */
- cur_len = length;
- per_dev = &ios->per_dev[dev - first_dev];
- if (!per_dev->length) {
- /* Only/always the parity unit of the first
- * stripe will be empty. So this is a chance to
- * initialize the per_dev info.
- */
- per_dev->dev = dev;
- per_dev->offset = si->obj_offset - si->unit_off;
+ /* If last stripe operate on parity comp */
+ si->cur_comp = group_width - ios->layout->parity;
}
- ret = _ore_add_parity_unit(ios, si, per_dev, cur_len);
+ /* In writes cur_len just means if it's the
+ * last one. See _ore_add_parity_unit.
+ */
+ ret = _add_parity_units(ios, si, dev, first_dev,
+ mirrors_p1, devs_in_group,
+ ios->sp2d ? length : cur_len);
if (unlikely(ret))
goto out;
@@ -747,6 +773,8 @@
/* Next stripe, start fresh */
si->cur_comp = 0;
si->cur_pg = 0;
+ si->obj_offset += cur_len;
+ si->unit_off = 0;
}
}
out:
diff --git a/fs/exofs/ore_raid.c b/fs/exofs/ore_raid.c
index 4e2c032..7f20f25 100644
--- a/fs/exofs/ore_raid.c
+++ b/fs/exofs/ore_raid.c
@@ -218,22 +218,28 @@
static void _gen_xor_unit(struct __stripe_pages_2d *sp2d)
{
unsigned p;
+ unsigned tx_flags = ASYNC_TX_ACK;
+
+ if (sp2d->parity == 1)
+ tx_flags |= ASYNC_TX_XOR_ZERO_DST;
+
for (p = 0; p < sp2d->pages_in_unit; p++) {
struct __1_page_stripe *_1ps = &sp2d->_1p_stripes[p];
if (!_1ps->write_count)
continue;
- init_async_submit(&_1ps->submit,
- ASYNC_TX_XOR_ZERO_DST | ASYNC_TX_ACK,
- NULL,
- NULL, NULL,
- (addr_conv_t *)_1ps->scribble);
+ init_async_submit(&_1ps->submit, tx_flags,
+ NULL, NULL, NULL, (addr_conv_t *)_1ps->scribble);
- /* TODO: raid6 */
- _1ps->tx = async_xor(_1ps->pages[sp2d->data_devs], _1ps->pages,
- 0, sp2d->data_devs, PAGE_SIZE,
- &_1ps->submit);
+ if (sp2d->parity == 1)
+ _1ps->tx = async_xor(_1ps->pages[sp2d->data_devs],
+ _1ps->pages, 0, sp2d->data_devs,
+ PAGE_SIZE, &_1ps->submit);
+ else /* parity == 2 */
+ _1ps->tx = async_gen_syndrome(_1ps->pages, 0,
+ sp2d->data_devs + sp2d->parity,
+ PAGE_SIZE, &_1ps->submit);
}
for (p = 0; p < sp2d->pages_in_unit; p++) {
@@ -404,9 +410,8 @@
ore_calc_stripe_info(ios->layout, *offset, 0, &si);
- p = si.unit_off / PAGE_SIZE;
- c = _dev_order(ios->layout->group_width * ios->layout->mirrors_p1,
- ios->layout->mirrors_p1, si.par_dev, si.dev);
+ p = si.cur_pg;
+ c = si.cur_comp;
page = ios->sp2d->_1p_stripes[p].pages[c];
pg_len = PAGE_SIZE - (si.unit_off % PAGE_SIZE);
@@ -534,9 +539,8 @@
goto read_it;
ore_calc_stripe_info(ios->layout, offset, 0, &read_si);
- p = read_si.unit_off / PAGE_SIZE;
- c = _dev_order(ios->layout->group_width * ios->layout->mirrors_p1,
- ios->layout->mirrors_p1, read_si.par_dev, read_si.dev);
+ p = read_si.cur_pg;
+ c = read_si.cur_comp;
if (min_p == sp2d->pages_in_unit) {
/* Didn't do it yet */
@@ -620,7 +624,7 @@
int _ore_add_parity_unit(struct ore_io_state *ios,
struct ore_striping_info *si,
struct ore_per_dev_state *per_dev,
- unsigned cur_len)
+ unsigned cur_len, bool do_xor)
{
if (ios->reading) {
if (per_dev->cur_sg >= ios->sgs_per_dev) {
@@ -640,17 +644,16 @@
si->cur_pg = _sp2d_min_pg(sp2d);
num_pages = _sp2d_max_pg(sp2d) + 1 - si->cur_pg;
- if (!cur_len) /* If last stripe operate on parity comp */
- si->cur_comp = sp2d->data_devs;
-
if (!per_dev->length) {
per_dev->offset += si->cur_pg * PAGE_SIZE;
/* If first stripe, Read in all read4write pages
* (if needed) before we calculate the first parity.
*/
- _read_4_write_first_stripe(ios);
+ if (do_xor)
+ _read_4_write_first_stripe(ios);
}
- if (!cur_len) /* If last stripe r4w pages of last stripe */
+ if (!cur_len && do_xor)
+ /* If last stripe r4w pages of last stripe */
_read_4_write_last_stripe(ios);
_read_4_write_execute(ios);
@@ -662,7 +665,7 @@
++(ios->cur_par_page);
}
- BUG_ON(si->cur_comp != sp2d->data_devs);
+ BUG_ON(si->cur_comp < sp2d->data_devs);
BUG_ON(si->cur_pg + num_pages > sp2d->pages_in_unit);
ret = _ore_add_stripe_unit(ios, &array_start, 0, pages,
@@ -670,9 +673,10 @@
if (unlikely(ret))
return ret;
- /* TODO: raid6 if (last_parity_dev) */
- _gen_xor_unit(sp2d);
- _sp2d_reset(sp2d, ios->r4w, ios->private);
+ if (do_xor) {
+ _gen_xor_unit(sp2d);
+ _sp2d_reset(sp2d, ios->r4w, ios->private);
+ }
}
return 0;
}
diff --git a/fs/exofs/ore_raid.h b/fs/exofs/ore_raid.h
index 2ffd2c3..cf6375d 100644
--- a/fs/exofs/ore_raid.h
+++ b/fs/exofs/ore_raid.h
@@ -31,24 +31,6 @@
#define ORE_DBGMSG2(M...) do {} while (0)
/* #define ORE_DBGMSG2 ORE_DBGMSG */
-/* Calculate the component order in a stripe. eg the logical data unit
- * address within the stripe of @dev given the @par_dev of this stripe.
- */
-static inline unsigned _dev_order(unsigned devs_in_group, unsigned mirrors_p1,
- unsigned par_dev, unsigned dev)
-{
- unsigned first_dev = dev - dev % devs_in_group;
-
- dev -= first_dev;
- par_dev -= first_dev;
-
- if (devs_in_group == par_dev) /* The raid 0 case */
- return dev / mirrors_p1;
- /* raid4/5/6 case */
- return ((devs_in_group + dev - par_dev - mirrors_p1) % devs_in_group) /
- mirrors_p1;
-}
-
/* ios_raid.c stuff needed by ios.c */
int _ore_post_alloc_raid_stuff(struct ore_io_state *ios);
void _ore_free_raid_stuff(struct ore_io_state *ios);
@@ -56,7 +38,8 @@
void _ore_add_sg_seg(struct ore_per_dev_state *per_dev, unsigned cur_len,
bool not_last);
int _ore_add_parity_unit(struct ore_io_state *ios, struct ore_striping_info *si,
- struct ore_per_dev_state *per_dev, unsigned cur_len);
+ struct ore_per_dev_state *per_dev, unsigned cur_len,
+ bool do_xor);
void _ore_add_stripe_page(struct __stripe_pages_2d *sp2d,
struct ore_striping_info *si, struct page *page);
static inline void _add_stripe_page(struct __stripe_pages_2d *sp2d,
diff --git a/fs/ext4/balloc.c b/fs/ext4/balloc.c
index 5c56785..0762d14 100644
--- a/fs/ext4/balloc.c
+++ b/fs/ext4/balloc.c
@@ -83,9 +83,9 @@
/* Return the number of clusters used for file system metadata; this
* represents the overhead needed by the file system.
*/
-unsigned ext4_num_overhead_clusters(struct super_block *sb,
- ext4_group_t block_group,
- struct ext4_group_desc *gdp)
+static unsigned ext4_num_overhead_clusters(struct super_block *sb,
+ ext4_group_t block_group,
+ struct ext4_group_desc *gdp)
{
unsigned num_clusters;
int block_cluster = -1, inode_cluster = -1, itbl_cluster = -1, i, c;
@@ -176,9 +176,10 @@
}
/* Initializes an uninitialized block bitmap */
-void ext4_init_block_bitmap(struct super_block *sb, struct buffer_head *bh,
- ext4_group_t block_group,
- struct ext4_group_desc *gdp)
+static void ext4_init_block_bitmap(struct super_block *sb,
+ struct buffer_head *bh,
+ ext4_group_t block_group,
+ struct ext4_group_desc *gdp)
{
unsigned int bit, bit_max;
struct ext4_sb_info *sbi = EXT4_SB(sb);
@@ -307,6 +308,7 @@
ext4_group_t block_group,
struct buffer_head *bh)
{
+ struct ext4_sb_info *sbi = EXT4_SB(sb);
ext4_grpblk_t offset;
ext4_grpblk_t next_zero_bit;
ext4_fsblk_t blk;
@@ -326,14 +328,14 @@
/* check whether block bitmap block number is set */
blk = ext4_block_bitmap(sb, desc);
offset = blk - group_first_block;
- if (!ext4_test_bit(offset, bh->b_data))
+ if (!ext4_test_bit(EXT4_B2C(sbi, offset), bh->b_data))
/* bad block bitmap */
return blk;
/* check whether the inode bitmap block number is set */
blk = ext4_inode_bitmap(sb, desc);
offset = blk - group_first_block;
- if (!ext4_test_bit(offset, bh->b_data))
+ if (!ext4_test_bit(EXT4_B2C(sbi, offset), bh->b_data))
/* bad block bitmap */
return blk;
@@ -341,18 +343,19 @@
blk = ext4_inode_table(sb, desc);
offset = blk - group_first_block;
next_zero_bit = ext4_find_next_zero_bit(bh->b_data,
- offset + EXT4_SB(sb)->s_itb_per_group,
- offset);
- if (next_zero_bit < offset + EXT4_SB(sb)->s_itb_per_group)
+ EXT4_B2C(sbi, offset + EXT4_SB(sb)->s_itb_per_group),
+ EXT4_B2C(sbi, offset));
+ if (next_zero_bit <
+ EXT4_B2C(sbi, offset + EXT4_SB(sb)->s_itb_per_group))
/* bad bitmap for inode tables */
return blk;
return 0;
}
-void ext4_validate_block_bitmap(struct super_block *sb,
- struct ext4_group_desc *desc,
- ext4_group_t block_group,
- struct buffer_head *bh)
+static void ext4_validate_block_bitmap(struct super_block *sb,
+ struct ext4_group_desc *desc,
+ ext4_group_t block_group,
+ struct buffer_head *bh)
{
ext4_fsblk_t blk;
struct ext4_group_info *grp = ext4_get_group_info(sb, block_group);
@@ -708,16 +711,6 @@
}
}
-static int ext4_group_sparse(ext4_group_t group)
-{
- if (group <= 1)
- return 1;
- if (!(group & 1))
- return 0;
- return (test_root(group, 7) || test_root(group, 5) ||
- test_root(group, 3));
-}
-
/**
* ext4_bg_has_super - number of blocks used by the superblock in group
* @sb: superblock for filesystem
@@ -728,11 +721,26 @@
*/
int ext4_bg_has_super(struct super_block *sb, ext4_group_t group)
{
- if (EXT4_HAS_RO_COMPAT_FEATURE(sb,
- EXT4_FEATURE_RO_COMPAT_SPARSE_SUPER) &&
- !ext4_group_sparse(group))
+ struct ext4_super_block *es = EXT4_SB(sb)->s_es;
+
+ if (group == 0)
+ return 1;
+ if (EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_SPARSE_SUPER2)) {
+ if (group == le32_to_cpu(es->s_backup_bgs[0]) ||
+ group == le32_to_cpu(es->s_backup_bgs[1]))
+ return 1;
return 0;
- return 1;
+ }
+ if ((group <= 1) || !EXT4_HAS_RO_COMPAT_FEATURE(sb,
+ EXT4_FEATURE_RO_COMPAT_SPARSE_SUPER))
+ return 1;
+ if (!(group & 1))
+ return 0;
+ if (test_root(group, 3) || (test_root(group, 5)) ||
+ test_root(group, 7))
+ return 1;
+
+ return 0;
}
static unsigned long ext4_bg_num_gdb_meta(struct super_block *sb,
diff --git a/fs/ext4/dir.c b/fs/ext4/dir.c
index d638c57..ef1bed6 100644
--- a/fs/ext4/dir.c
+++ b/fs/ext4/dir.c
@@ -105,7 +105,7 @@
static int ext4_readdir(struct file *file, struct dir_context *ctx)
{
unsigned int offset;
- int i, stored;
+ int i;
struct ext4_dir_entry_2 *de;
int err;
struct inode *inode = file_inode(file);
@@ -133,7 +133,6 @@
return ret;
}
- stored = 0;
offset = ctx->pos & (sb->s_blocksize - 1);
while (ctx->pos < inode->i_size) {
diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 66946aa..1479e2a 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -158,7 +158,6 @@
#define EXT4_MAP_MAPPED (1 << BH_Mapped)
#define EXT4_MAP_UNWRITTEN (1 << BH_Unwritten)
#define EXT4_MAP_BOUNDARY (1 << BH_Boundary)
-#define EXT4_MAP_UNINIT (1 << BH_Uninit)
/* Sometimes (in the bigalloc case, from ext4_da_get_block_prep) the caller of
* ext4_map_blocks wants to know whether or not the underlying cluster has
* already been accounted for. EXT4_MAP_FROM_CLUSTER conveys to the caller that
@@ -169,7 +168,7 @@
#define EXT4_MAP_FROM_CLUSTER (1 << BH_AllocFromCluster)
#define EXT4_MAP_FLAGS (EXT4_MAP_NEW | EXT4_MAP_MAPPED |\
EXT4_MAP_UNWRITTEN | EXT4_MAP_BOUNDARY |\
- EXT4_MAP_UNINIT | EXT4_MAP_FROM_CLUSTER)
+ EXT4_MAP_FROM_CLUSTER)
struct ext4_map_blocks {
ext4_fsblk_t m_pblk;
@@ -184,7 +183,7 @@
#define EXT4_IO_END_UNWRITTEN 0x0001
/*
- * For converting uninitialized extents on a work queue. 'handle' is used for
+ * For converting unwritten extents on a work queue. 'handle' is used for
* buffered writeback.
*/
typedef struct ext4_io_end {
@@ -537,26 +536,26 @@
/*
* Flags used by ext4_map_blocks()
*/
- /* Allocate any needed blocks and/or convert an unitialized
+ /* Allocate any needed blocks and/or convert an unwritten
extent to be an initialized ext4 */
#define EXT4_GET_BLOCKS_CREATE 0x0001
- /* Request the creation of an unitialized extent */
-#define EXT4_GET_BLOCKS_UNINIT_EXT 0x0002
-#define EXT4_GET_BLOCKS_CREATE_UNINIT_EXT (EXT4_GET_BLOCKS_UNINIT_EXT|\
+ /* Request the creation of an unwritten extent */
+#define EXT4_GET_BLOCKS_UNWRIT_EXT 0x0002
+#define EXT4_GET_BLOCKS_CREATE_UNWRIT_EXT (EXT4_GET_BLOCKS_UNWRIT_EXT|\
EXT4_GET_BLOCKS_CREATE)
/* Caller is from the delayed allocation writeout path
* finally doing the actual allocation of delayed blocks */
#define EXT4_GET_BLOCKS_DELALLOC_RESERVE 0x0004
/* caller is from the direct IO path, request to creation of an
- unitialized extents if not allocated, split the uninitialized
+ unwritten extents if not allocated, split the unwritten
extent if blocks has been preallocated already*/
#define EXT4_GET_BLOCKS_PRE_IO 0x0008
#define EXT4_GET_BLOCKS_CONVERT 0x0010
#define EXT4_GET_BLOCKS_IO_CREATE_EXT (EXT4_GET_BLOCKS_PRE_IO|\
- EXT4_GET_BLOCKS_CREATE_UNINIT_EXT)
+ EXT4_GET_BLOCKS_CREATE_UNWRIT_EXT)
/* Convert extent to initialized after IO complete */
#define EXT4_GET_BLOCKS_IO_CONVERT_EXT (EXT4_GET_BLOCKS_CONVERT|\
- EXT4_GET_BLOCKS_CREATE_UNINIT_EXT)
+ EXT4_GET_BLOCKS_CREATE_UNWRIT_EXT)
/* Eventual metadata allocation (due to growing extent tree)
* should not fail, so try to use reserved blocks for that.*/
#define EXT4_GET_BLOCKS_METADATA_NOFAIL 0x0020
@@ -876,6 +875,8 @@
struct inode vfs_inode;
struct jbd2_inode *jinode;
+ spinlock_t i_raw_lock; /* protects updates to the raw inode */
+
/*
* File creation time. Its function is same as that of
* struct timespec i_{a,c,m}time in the generic inode.
@@ -1159,7 +1160,8 @@
__le32 s_usr_quota_inum; /* inode for tracking user quota */
__le32 s_grp_quota_inum; /* inode for tracking group quota */
__le32 s_overhead_clusters; /* overhead blocks/clusters in fs */
- __le32 s_reserved[108]; /* Padding to the end of the block */
+ __le32 s_backup_bgs[2]; /* groups with sparse_super2 SBs */
+ __le32 s_reserved[106]; /* Padding to the end of the block */
__le32 s_checksum; /* crc32c(superblock) */
};
@@ -1505,6 +1507,7 @@
#define EXT4_FEATURE_COMPAT_EXT_ATTR 0x0008
#define EXT4_FEATURE_COMPAT_RESIZE_INODE 0x0010
#define EXT4_FEATURE_COMPAT_DIR_INDEX 0x0020
+#define EXT4_FEATURE_COMPAT_SPARSE_SUPER2 0x0200
#define EXT4_FEATURE_RO_COMPAT_SPARSE_SUPER 0x0001
#define EXT4_FEATURE_RO_COMPAT_LARGE_FILE 0x0002
@@ -1953,10 +1956,6 @@
extern ext4_group_t ext4_get_group_number(struct super_block *sb,
ext4_fsblk_t block);
-extern void ext4_validate_block_bitmap(struct super_block *sb,
- struct ext4_group_desc *desc,
- ext4_group_t block_group,
- struct buffer_head *bh);
extern unsigned int ext4_block_group(struct super_block *sb,
ext4_fsblk_t blocknr);
extern ext4_grpblk_t ext4_block_group_offset(struct super_block *sb,
@@ -1985,16 +1984,9 @@
struct buffer_head *bh);
extern struct buffer_head *ext4_read_block_bitmap(struct super_block *sb,
ext4_group_t block_group);
-extern void ext4_init_block_bitmap(struct super_block *sb,
- struct buffer_head *bh,
- ext4_group_t group,
- struct ext4_group_desc *desc);
extern unsigned ext4_free_clusters_after_init(struct super_block *sb,
ext4_group_t block_group,
struct ext4_group_desc *gdp);
-extern unsigned ext4_num_overhead_clusters(struct super_block *sb,
- ext4_group_t block_group,
- struct ext4_group_desc *gdp);
ext4_fsblk_t ext4_inode_to_goal_block(struct inode *);
/* dir.c */
@@ -2137,8 +2129,6 @@
extern void ext4_set_aops(struct inode *inode);
extern int ext4_writepage_trans_blocks(struct inode *);
extern int ext4_chunk_trans_blocks(struct inode *, int nrblocks);
-extern int ext4_block_truncate_page(handle_t *handle,
- struct address_space *mapping, loff_t from);
extern int ext4_zero_partial_blocks(handle_t *handle, struct inode *inode,
loff_t lstart, loff_t lend);
extern int ext4_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf);
@@ -2198,8 +2188,6 @@
/* super.c */
extern int ext4_calculate_overhead(struct super_block *sb);
-extern int ext4_superblock_csum_verify(struct super_block *sb,
- struct ext4_super_block *es);
extern void ext4_superblock_csum_set(struct super_block *sb);
extern void *ext4_kvmalloc(size_t size, gfp_t flags);
extern void *ext4_kvzalloc(size_t size, gfp_t flags);
@@ -2571,19 +2559,11 @@
extern const struct inode_operations ext4_file_inode_operations;
extern const struct file_operations ext4_file_operations;
extern loff_t ext4_llseek(struct file *file, loff_t offset, int origin);
-extern void ext4_unwritten_wait(struct inode *inode);
/* inline.c */
extern int ext4_has_inline_data(struct inode *inode);
-extern int ext4_get_inline_size(struct inode *inode);
extern int ext4_get_max_inline_size(struct inode *inode);
extern int ext4_find_inline_data_nolock(struct inode *inode);
-extern void ext4_write_inline_data(struct inode *inode,
- struct ext4_iloc *iloc,
- void *buffer, loff_t pos,
- unsigned int len);
-extern int ext4_prepare_inline_data(handle_t *handle, struct inode *inode,
- unsigned int len);
extern int ext4_init_inline_data(handle_t *handle, struct inode *inode,
unsigned int len);
extern int ext4_destroy_inline_data(handle_t *handle, struct inode *inode);
@@ -2771,23 +2751,20 @@
extern int ext4_bio_write_page(struct ext4_io_submit *io,
struct page *page,
int len,
- struct writeback_control *wbc);
+ struct writeback_control *wbc,
+ bool keep_towrite);
/* mmp.c */
extern int ext4_multi_mount_protect(struct super_block *, ext4_fsblk_t);
-extern void ext4_mmp_csum_set(struct super_block *sb, struct mmp_struct *mmp);
-extern int ext4_mmp_csum_verify(struct super_block *sb,
- struct mmp_struct *mmp);
/*
* Note that these flags will never ever appear in a buffer_head's state flag.
* See EXT4_MAP_... to see where this is used.
*/
enum ext4_state_bits {
- BH_Uninit /* blocks are allocated but uninitialized on disk */
- = BH_JBDPrivateStart,
- BH_AllocFromCluster, /* allocated blocks were part of already
+ BH_AllocFromCluster /* allocated blocks were part of already
* allocated cluster. */
+ = BH_JBDPrivateStart
};
/*
diff --git a/fs/ext4/ext4_extents.h b/fs/ext4/ext4_extents.h
index 5074fe2..a867f5c 100644
--- a/fs/ext4/ext4_extents.h
+++ b/fs/ext4/ext4_extents.h
@@ -137,21 +137,21 @@
* EXT_INIT_MAX_LEN is the maximum number of blocks we can have in an
* initialized extent. This is 2^15 and not (2^16 - 1), since we use the
* MSB of ee_len field in the extent datastructure to signify if this
- * particular extent is an initialized extent or an uninitialized (i.e.
+ * particular extent is an initialized extent or an unwritten (i.e.
* preallocated).
- * EXT_UNINIT_MAX_LEN is the maximum number of blocks we can have in an
- * uninitialized extent.
+ * EXT_UNWRITTEN_MAX_LEN is the maximum number of blocks we can have in an
+ * unwritten extent.
* If ee_len is <= 0x8000, it is an initialized extent. Otherwise, it is an
- * uninitialized one. In other words, if MSB of ee_len is set, it is an
- * uninitialized extent with only one special scenario when ee_len = 0x8000.
- * In this case we can not have an uninitialized extent of zero length and
+ * unwritten one. In other words, if MSB of ee_len is set, it is an
+ * unwritten extent with only one special scenario when ee_len = 0x8000.
+ * In this case we can not have an unwritten extent of zero length and
* thus we make it as a special case of initialized extent with 0x8000 length.
* This way we get better extent-to-group alignment for initialized extents.
* Hence, the maximum number of blocks we can have in an *initialized*
- * extent is 2^15 (32768) and in an *uninitialized* extent is 2^15-1 (32767).
+ * extent is 2^15 (32768) and in an *unwritten* extent is 2^15-1 (32767).
*/
#define EXT_INIT_MAX_LEN (1UL << 15)
-#define EXT_UNINIT_MAX_LEN (EXT_INIT_MAX_LEN - 1)
+#define EXT_UNWRITTEN_MAX_LEN (EXT_INIT_MAX_LEN - 1)
#define EXT_FIRST_EXTENT(__hdr__) \
@@ -187,14 +187,14 @@
return le16_to_cpu(ext_inode_hdr(inode)->eh_depth);
}
-static inline void ext4_ext_mark_uninitialized(struct ext4_extent *ext)
+static inline void ext4_ext_mark_unwritten(struct ext4_extent *ext)
{
- /* We can not have an uninitialized extent of zero length! */
+ /* We can not have an unwritten extent of zero length! */
BUG_ON((le16_to_cpu(ext->ee_len) & ~EXT_INIT_MAX_LEN) == 0);
ext->ee_len |= cpu_to_le16(EXT_INIT_MAX_LEN);
}
-static inline int ext4_ext_is_uninitialized(struct ext4_extent *ext)
+static inline int ext4_ext_is_unwritten(struct ext4_extent *ext)
{
/* Extent with ee_len of 0x8000 is treated as an initialized extent */
return (le16_to_cpu(ext->ee_len) > EXT_INIT_MAX_LEN);
diff --git a/fs/ext4/ext4_jbd2.c b/fs/ext4/ext4_jbd2.c
index c3fb607..0074e0d2 100644
--- a/fs/ext4/ext4_jbd2.c
+++ b/fs/ext4/ext4_jbd2.c
@@ -122,9 +122,10 @@
return handle;
}
-void ext4_journal_abort_handle(const char *caller, unsigned int line,
- const char *err_fn, struct buffer_head *bh,
- handle_t *handle, int err)
+static void ext4_journal_abort_handle(const char *caller, unsigned int line,
+ const char *err_fn,
+ struct buffer_head *bh,
+ handle_t *handle, int err)
{
char nbuf[16];
const char *errstr = ext4_decode_error(NULL, err, nbuf);
diff --git a/fs/ext4/ext4_jbd2.h b/fs/ext4/ext4_jbd2.h
index 81cfefa..17c00ff 100644
--- a/fs/ext4/ext4_jbd2.h
+++ b/fs/ext4/ext4_jbd2.h
@@ -231,10 +231,6 @@
/*
* Wrapper functions with which ext4 calls into JBD.
*/
-void ext4_journal_abort_handle(const char *caller, unsigned int line,
- const char *err_fn,
- struct buffer_head *bh, handle_t *handle, int err);
-
int __ext4_journal_get_write_access(const char *where, unsigned int line,
handle_t *handle, struct buffer_head *bh);
diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 01b0c20..4da228a 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -50,8 +50,8 @@
*/
#define EXT4_EXT_MAY_ZEROOUT 0x1 /* safe to zeroout if split fails \
due to ENOSPC */
-#define EXT4_EXT_MARK_UNINIT1 0x2 /* mark first half uninitialized */
-#define EXT4_EXT_MARK_UNINIT2 0x4 /* mark second half uninitialized */
+#define EXT4_EXT_MARK_UNWRIT1 0x2 /* mark first half unwritten */
+#define EXT4_EXT_MARK_UNWRIT2 0x4 /* mark second half unwritten */
#define EXT4_EXT_DATA_VALID1 0x8 /* first half contains valid data */
#define EXT4_EXT_DATA_VALID2 0x10 /* second half contains valid data */
@@ -143,6 +143,7 @@
{
if (path->p_bh) {
/* path points to block */
+ BUFFER_TRACE(path->p_bh, "get_write_access");
return ext4_journal_get_write_access(handle, path->p_bh);
}
/* path points to leaf/index in inode body */
@@ -524,7 +525,7 @@
lblk - prev, ~0,
EXTENT_STATUS_HOLE);
- if (ext4_ext_is_uninitialized(ex))
+ if (ext4_ext_is_unwritten(ex))
status = EXTENT_STATUS_UNWRITTEN;
ext4_es_cache_extent(inode, lblk, len,
ext4_ext_pblock(ex), status);
@@ -620,7 +621,7 @@
} else if (path->p_ext) {
ext_debug(" %d:[%d]%d:%llu ",
le32_to_cpu(path->p_ext->ee_block),
- ext4_ext_is_uninitialized(path->p_ext),
+ ext4_ext_is_unwritten(path->p_ext),
ext4_ext_get_actual_len(path->p_ext),
ext4_ext_pblock(path->p_ext));
} else
@@ -646,7 +647,7 @@
for (i = 0; i < le16_to_cpu(eh->eh_entries); i++, ex++) {
ext_debug("%d:[%d]%d:%llu ", le32_to_cpu(ex->ee_block),
- ext4_ext_is_uninitialized(ex),
+ ext4_ext_is_unwritten(ex),
ext4_ext_get_actual_len(ex), ext4_ext_pblock(ex));
}
ext_debug("\n");
@@ -677,7 +678,7 @@
ext_debug("move %d:%llu:[%d]%d in new leaf %llu\n",
le32_to_cpu(ex->ee_block),
ext4_ext_pblock(ex),
- ext4_ext_is_uninitialized(ex),
+ ext4_ext_is_unwritten(ex),
ext4_ext_get_actual_len(ex),
newblock);
ex++;
@@ -802,7 +803,7 @@
ext_debug(" -> %d:%llu:[%d]%d ",
le32_to_cpu(path->p_ext->ee_block),
ext4_ext_pblock(path->p_ext),
- ext4_ext_is_uninitialized(path->p_ext),
+ ext4_ext_is_unwritten(path->p_ext),
ext4_ext_get_actual_len(path->p_ext));
#ifdef CHECK_BINSEARCH
@@ -1686,11 +1687,11 @@
/*
* Make sure that both extents are initialized. We don't merge
- * uninitialized extents so that we can be sure that end_io code has
+ * unwritten extents so that we can be sure that end_io code has
* the extent that was written properly split out and conversion to
* initialized is trivial.
*/
- if (ext4_ext_is_uninitialized(ex1) != ext4_ext_is_uninitialized(ex2))
+ if (ext4_ext_is_unwritten(ex1) != ext4_ext_is_unwritten(ex2))
return 0;
ext1_ee_len = ext4_ext_get_actual_len(ex1);
@@ -1707,10 +1708,10 @@
*/
if (ext1_ee_len + ext2_ee_len > EXT_INIT_MAX_LEN)
return 0;
- if (ext4_ext_is_uninitialized(ex1) &&
+ if (ext4_ext_is_unwritten(ex1) &&
(ext4_test_inode_state(inode, EXT4_STATE_DIO_UNWRITTEN) ||
atomic_read(&EXT4_I(inode)->i_unwritten) ||
- (ext1_ee_len + ext2_ee_len > EXT_UNINIT_MAX_LEN)))
+ (ext1_ee_len + ext2_ee_len > EXT_UNWRITTEN_MAX_LEN)))
return 0;
#ifdef AGGRESSIVE_TEST
if (ext1_ee_len >= 4)
@@ -1735,7 +1736,7 @@
{
struct ext4_extent_header *eh;
unsigned int depth, len;
- int merge_done = 0, uninit;
+ int merge_done = 0, unwritten;
depth = ext_depth(inode);
BUG_ON(path[depth].p_hdr == NULL);
@@ -1745,11 +1746,11 @@
if (!ext4_can_extents_be_merged(inode, ex, ex + 1))
break;
/* merge with next extent! */
- uninit = ext4_ext_is_uninitialized(ex);
+ unwritten = ext4_ext_is_unwritten(ex);
ex->ee_len = cpu_to_le16(ext4_ext_get_actual_len(ex)
+ ext4_ext_get_actual_len(ex + 1));
- if (uninit)
- ext4_ext_mark_uninitialized(ex);
+ if (unwritten)
+ ext4_ext_mark_unwritten(ex);
if (ex + 1 < EXT_LAST_EXTENT(eh)) {
len = (EXT_LAST_EXTENT(eh) - ex - 1)
@@ -1903,7 +1904,7 @@
struct ext4_ext_path *npath = NULL;
int depth, len, err;
ext4_lblk_t next;
- int mb_flags = 0, uninit;
+ int mb_flags = 0, unwritten;
if (unlikely(ext4_ext_get_actual_len(newext) == 0)) {
EXT4_ERROR_INODE(inode, "ext4_ext_get_actual_len(newext) == 0");
@@ -1943,21 +1944,21 @@
if (ext4_can_extents_be_merged(inode, ex, newext)) {
ext_debug("append [%d]%d block to %u:[%d]%d"
"(from %llu)\n",
- ext4_ext_is_uninitialized(newext),
+ ext4_ext_is_unwritten(newext),
ext4_ext_get_actual_len(newext),
le32_to_cpu(ex->ee_block),
- ext4_ext_is_uninitialized(ex),
+ ext4_ext_is_unwritten(ex),
ext4_ext_get_actual_len(ex),
ext4_ext_pblock(ex));
err = ext4_ext_get_access(handle, inode,
path + depth);
if (err)
return err;
- uninit = ext4_ext_is_uninitialized(ex);
+ unwritten = ext4_ext_is_unwritten(ex);
ex->ee_len = cpu_to_le16(ext4_ext_get_actual_len(ex)
+ ext4_ext_get_actual_len(newext));
- if (uninit)
- ext4_ext_mark_uninitialized(ex);
+ if (unwritten)
+ ext4_ext_mark_unwritten(ex);
eh = path[depth].p_hdr;
nearex = ex;
goto merge;
@@ -1969,10 +1970,10 @@
ext_debug("prepend %u[%d]%d block to %u:[%d]%d"
"(from %llu)\n",
le32_to_cpu(newext->ee_block),
- ext4_ext_is_uninitialized(newext),
+ ext4_ext_is_unwritten(newext),
ext4_ext_get_actual_len(newext),
le32_to_cpu(ex->ee_block),
- ext4_ext_is_uninitialized(ex),
+ ext4_ext_is_unwritten(ex),
ext4_ext_get_actual_len(ex),
ext4_ext_pblock(ex));
err = ext4_ext_get_access(handle, inode,
@@ -1980,13 +1981,13 @@
if (err)
return err;
- uninit = ext4_ext_is_uninitialized(ex);
+ unwritten = ext4_ext_is_unwritten(ex);
ex->ee_block = newext->ee_block;
ext4_ext_store_pblock(ex, ext4_ext_pblock(newext));
ex->ee_len = cpu_to_le16(ext4_ext_get_actual_len(ex)
+ ext4_ext_get_actual_len(newext));
- if (uninit)
- ext4_ext_mark_uninitialized(ex);
+ if (unwritten)
+ ext4_ext_mark_unwritten(ex);
eh = path[depth].p_hdr;
nearex = ex;
goto merge;
@@ -2046,7 +2047,7 @@
ext_debug("first extent in the leaf: %u:%llu:[%d]%d\n",
le32_to_cpu(newext->ee_block),
ext4_ext_pblock(newext),
- ext4_ext_is_uninitialized(newext),
+ ext4_ext_is_unwritten(newext),
ext4_ext_get_actual_len(newext));
nearex = EXT_FIRST_EXTENT(eh);
} else {
@@ -2057,7 +2058,7 @@
"nearest %p\n",
le32_to_cpu(newext->ee_block),
ext4_ext_pblock(newext),
- ext4_ext_is_uninitialized(newext),
+ ext4_ext_is_unwritten(newext),
ext4_ext_get_actual_len(newext),
nearex);
nearex++;
@@ -2068,7 +2069,7 @@
"nearest %p\n",
le32_to_cpu(newext->ee_block),
ext4_ext_pblock(newext),
- ext4_ext_is_uninitialized(newext),
+ ext4_ext_is_unwritten(newext),
ext4_ext_get_actual_len(newext),
nearex);
}
@@ -2078,7 +2079,7 @@
"move %d extents from 0x%p to 0x%p\n",
le32_to_cpu(newext->ee_block),
ext4_ext_pblock(newext),
- ext4_ext_is_uninitialized(newext),
+ ext4_ext_is_unwritten(newext),
ext4_ext_get_actual_len(newext),
len, nearex, nearex + 1);
memmove(nearex + 1, nearex,
@@ -2200,7 +2201,7 @@
es.es_lblk = le32_to_cpu(ex->ee_block);
es.es_len = ext4_ext_get_actual_len(ex);
es.es_pblk = ext4_ext_pblock(ex);
- if (ext4_ext_is_uninitialized(ex))
+ if (ext4_ext_is_unwritten(ex))
flags |= FIEMAP_EXTENT_UNWRITTEN;
}
@@ -2576,7 +2577,7 @@
unsigned num;
ext4_lblk_t ex_ee_block;
unsigned short ex_ee_len;
- unsigned uninitialized = 0;
+ unsigned unwritten = 0;
struct ext4_extent *ex;
ext4_fsblk_t pblk;
@@ -2623,13 +2624,13 @@
while (ex >= EXT_FIRST_EXTENT(eh) &&
ex_ee_block + ex_ee_len > start) {
- if (ext4_ext_is_uninitialized(ex))
- uninitialized = 1;
+ if (ext4_ext_is_unwritten(ex))
+ unwritten = 1;
else
- uninitialized = 0;
+ unwritten = 0;
ext_debug("remove ext %u:[%d]%d\n", ex_ee_block,
- uninitialized, ex_ee_len);
+ unwritten, ex_ee_len);
path[depth].p_ext = ex;
a = ex_ee_block > start ? ex_ee_block : start;
@@ -2701,11 +2702,11 @@
ex->ee_len = cpu_to_le16(num);
/*
- * Do not mark uninitialized if all the blocks in the
+ * Do not mark unwritten if all the blocks in the
* extent have been removed.
*/
- if (uninitialized && num)
- ext4_ext_mark_uninitialized(ex);
+ if (unwritten && num)
+ ext4_ext_mark_unwritten(ex);
/*
* If the extent was completely released,
* we need to remove it from the leaf
@@ -2854,9 +2855,9 @@
end < ee_block + ext4_ext_get_actual_len(ex) - 1) {
int split_flag = 0;
- if (ext4_ext_is_uninitialized(ex))
- split_flag = EXT4_EXT_MARK_UNINIT1 |
- EXT4_EXT_MARK_UNINIT2;
+ if (ext4_ext_is_unwritten(ex))
+ split_flag = EXT4_EXT_MARK_UNWRIT1 |
+ EXT4_EXT_MARK_UNWRIT2;
/*
* Split the extent in two so that 'end' is the last
@@ -3113,7 +3114,7 @@
* @path: the path to the extent
* @split: the logical block where the extent is splitted.
* @split_flags: indicates if the extent could be zeroout if split fails, and
- * the states(init or uninit) of new extents.
+ * the states(init or unwritten) of new extents.
* @flags: flags used to insert new extent to extent tree.
*
*
@@ -3155,10 +3156,10 @@
newblock = split - ee_block + ext4_ext_pblock(ex);
BUG_ON(split < ee_block || split >= (ee_block + ee_len));
- BUG_ON(!ext4_ext_is_uninitialized(ex) &&
+ BUG_ON(!ext4_ext_is_unwritten(ex) &&
split_flag & (EXT4_EXT_MAY_ZEROOUT |
- EXT4_EXT_MARK_UNINIT1 |
- EXT4_EXT_MARK_UNINIT2));
+ EXT4_EXT_MARK_UNWRIT1 |
+ EXT4_EXT_MARK_UNWRIT2));
err = ext4_ext_get_access(handle, inode, path + depth);
if (err)
@@ -3170,8 +3171,8 @@
* then we just change the state of the extent, and splitting
* is not needed.
*/
- if (split_flag & EXT4_EXT_MARK_UNINIT2)
- ext4_ext_mark_uninitialized(ex);
+ if (split_flag & EXT4_EXT_MARK_UNWRIT2)
+ ext4_ext_mark_unwritten(ex);
else
ext4_ext_mark_initialized(ex);
@@ -3185,8 +3186,8 @@
/* case a */
memcpy(&orig_ex, ex, sizeof(orig_ex));
ex->ee_len = cpu_to_le16(split - ee_block);
- if (split_flag & EXT4_EXT_MARK_UNINIT1)
- ext4_ext_mark_uninitialized(ex);
+ if (split_flag & EXT4_EXT_MARK_UNWRIT1)
+ ext4_ext_mark_unwritten(ex);
/*
* path may lead to new leaf, not to original leaf any more
@@ -3200,8 +3201,8 @@
ex2->ee_block = cpu_to_le32(split);
ex2->ee_len = cpu_to_le16(ee_len - (split - ee_block));
ext4_ext_store_pblock(ex2, newblock);
- if (split_flag & EXT4_EXT_MARK_UNINIT2)
- ext4_ext_mark_uninitialized(ex2);
+ if (split_flag & EXT4_EXT_MARK_UNWRIT2)
+ ext4_ext_mark_unwritten(ex2);
err = ext4_ext_insert_extent(handle, inode, path, &newex, flags);
if (err == -ENOSPC && (EXT4_EXT_MAY_ZEROOUT & split_flag)) {
@@ -3278,7 +3279,7 @@
struct ext4_extent *ex;
unsigned int ee_len, depth;
int err = 0;
- int uninitialized;
+ int unwritten;
int split_flag1, flags1;
int allocated = map->m_len;
@@ -3286,14 +3287,14 @@
ex = path[depth].p_ext;
ee_block = le32_to_cpu(ex->ee_block);
ee_len = ext4_ext_get_actual_len(ex);
- uninitialized = ext4_ext_is_uninitialized(ex);
+ unwritten = ext4_ext_is_unwritten(ex);
if (map->m_lblk + map->m_len < ee_block + ee_len) {
split_flag1 = split_flag & EXT4_EXT_MAY_ZEROOUT;
flags1 = flags | EXT4_GET_BLOCKS_PRE_IO;
- if (uninitialized)
- split_flag1 |= EXT4_EXT_MARK_UNINIT1 |
- EXT4_EXT_MARK_UNINIT2;
+ if (unwritten)
+ split_flag1 |= EXT4_EXT_MARK_UNWRIT1 |
+ EXT4_EXT_MARK_UNWRIT2;
if (split_flag & EXT4_EXT_DATA_VALID2)
split_flag1 |= EXT4_EXT_DATA_VALID1;
err = ext4_split_extent_at(handle, inode, path,
@@ -3318,15 +3319,15 @@
(unsigned long) map->m_lblk);
return -EIO;
}
- uninitialized = ext4_ext_is_uninitialized(ex);
+ unwritten = ext4_ext_is_unwritten(ex);
split_flag1 = 0;
if (map->m_lblk >= ee_block) {
split_flag1 = split_flag & EXT4_EXT_DATA_VALID2;
- if (uninitialized) {
- split_flag1 |= EXT4_EXT_MARK_UNINIT1;
+ if (unwritten) {
+ split_flag1 |= EXT4_EXT_MARK_UNWRIT1;
split_flag1 |= split_flag & (EXT4_EXT_MAY_ZEROOUT |
- EXT4_EXT_MARK_UNINIT2);
+ EXT4_EXT_MARK_UNWRIT2);
}
err = ext4_split_extent_at(handle, inode, path,
map->m_lblk, split_flag1, flags);
@@ -3341,16 +3342,16 @@
/*
* This function is called by ext4_ext_map_blocks() if someone tries to write
- * to an uninitialized extent. It may result in splitting the uninitialized
+ * to an unwritten extent. It may result in splitting the unwritten
* extent into multiple extents (up to three - one initialized and two
- * uninitialized).
+ * unwritten).
* There are three possibilities:
* a> There is no split required: Entire extent should be initialized
* b> Splits in two extents: Write is happening at either end of the extent
* c> Splits in three extents: Somone is writing in middle of the extent
*
* Pre-conditions:
- * - The extent pointed to by 'path' is uninitialized.
+ * - The extent pointed to by 'path' is unwritten.
* - The extent pointed to by 'path' contains a superset
* of the logical span [map->m_lblk, map->m_lblk + map->m_len).
*
@@ -3396,12 +3397,12 @@
trace_ext4_ext_convert_to_initialized_enter(inode, map, ex);
/* Pre-conditions */
- BUG_ON(!ext4_ext_is_uninitialized(ex));
+ BUG_ON(!ext4_ext_is_unwritten(ex));
BUG_ON(!in_range(map->m_lblk, ee_block, ee_len));
/*
* Attempt to transfer newly initialized blocks from the currently
- * uninitialized extent to its neighbor. This is much cheaper
+ * unwritten extent to its neighbor. This is much cheaper
* than an insertion followed by a merge as those involve costly
* memmove() calls. Transferring to the left is the common case in
* steady state for workloads doing fallocate(FALLOC_FL_KEEP_SIZE)
@@ -3437,7 +3438,7 @@
* - C4: abut_ex can receive the additional blocks without
* overflowing the (initialized) length limit.
*/
- if ((!ext4_ext_is_uninitialized(abut_ex)) && /*C1*/
+ if ((!ext4_ext_is_unwritten(abut_ex)) && /*C1*/
((prev_lblk + prev_len) == ee_block) && /*C2*/
((prev_pblk + prev_len) == ee_pblk) && /*C3*/
(prev_len < (EXT_INIT_MAX_LEN - map_len))) { /*C4*/
@@ -3452,7 +3453,7 @@
ex->ee_block = cpu_to_le32(ee_block + map_len);
ext4_ext_store_pblock(ex, ee_pblk + map_len);
ex->ee_len = cpu_to_le16(ee_len - map_len);
- ext4_ext_mark_uninitialized(ex); /* Restore the flag */
+ ext4_ext_mark_unwritten(ex); /* Restore the flag */
/* Extend abut_ex by 'map_len' blocks */
abut_ex->ee_len = cpu_to_le16(prev_len + map_len);
@@ -3483,7 +3484,7 @@
* - C4: abut_ex can receive the additional blocks without
* overflowing the (initialized) length limit.
*/
- if ((!ext4_ext_is_uninitialized(abut_ex)) && /*C1*/
+ if ((!ext4_ext_is_unwritten(abut_ex)) && /*C1*/
((map->m_lblk + map_len) == next_lblk) && /*C2*/
((ee_pblk + ee_len) == next_pblk) && /*C3*/
(next_len < (EXT_INIT_MAX_LEN - map_len))) { /*C4*/
@@ -3498,7 +3499,7 @@
abut_ex->ee_block = cpu_to_le32(next_lblk - map_len);
ext4_ext_store_pblock(abut_ex, next_pblk - map_len);
ex->ee_len = cpu_to_le16(ee_len - map_len);
- ext4_ext_mark_uninitialized(ex); /* Restore the flag */
+ ext4_ext_mark_unwritten(ex); /* Restore the flag */
/* Extend abut_ex by 'map_len' blocks */
abut_ex->ee_len = cpu_to_le16(next_len + map_len);
@@ -3603,26 +3604,26 @@
/*
* This function is called by ext4_ext_map_blocks() from
* ext4_get_blocks_dio_write() when DIO to write
- * to an uninitialized extent.
+ * to an unwritten extent.
*
- * Writing to an uninitialized extent may result in splitting the uninitialized
- * extent into multiple initialized/uninitialized extents (up to three)
+ * Writing to an unwritten extent may result in splitting the unwritten
+ * extent into multiple initialized/unwritten extents (up to three)
* There are three possibilities:
- * a> There is no split required: Entire extent should be uninitialized
+ * a> There is no split required: Entire extent should be unwritten
* b> Splits in two extents: Write is happening at either end of the extent
* c> Splits in three extents: Somone is writing in middle of the extent
*
* This works the same way in the case of initialized -> unwritten conversion.
*
* One of more index blocks maybe needed if the extent tree grow after
- * the uninitialized extent split. To prevent ENOSPC occur at the IO
- * complete, we need to split the uninitialized extent before DIO submit
- * the IO. The uninitialized extent called at this time will be split
- * into three uninitialized extent(at most). After IO complete, the part
+ * the unwritten extent split. To prevent ENOSPC occur at the IO
+ * complete, we need to split the unwritten extent before DIO submit
+ * the IO. The unwritten extent called at this time will be split
+ * into three unwritten extent(at most). After IO complete, the part
* being filled will be convert to initialized by the end_io callback function
* via ext4_convert_unwritten_extents().
*
- * Returns the size of uninitialized extent to be written on success.
+ * Returns the size of unwritten extent to be written on success.
*/
static int ext4_split_convert_extents(handle_t *handle,
struct inode *inode,
@@ -3660,7 +3661,7 @@
} else if (flags & EXT4_GET_BLOCKS_CONVERT) {
split_flag |= ee_block + ee_len <= eof_block ?
EXT4_EXT_MAY_ZEROOUT : 0;
- split_flag |= (EXT4_EXT_MARK_UNINIT2 | EXT4_EXT_DATA_VALID2);
+ split_flag |= (EXT4_EXT_MARK_UNWRIT2 | EXT4_EXT_DATA_VALID2);
}
flags |= EXT4_GET_BLOCKS_PRE_IO;
return ext4_split_extent(handle, inode, path, map, split_flag, flags);
@@ -3710,8 +3711,8 @@
err = ext4_ext_get_access(handle, inode, path + depth);
if (err)
goto out;
- /* first mark the extent as uninitialized */
- ext4_ext_mark_uninitialized(ex);
+ /* first mark the extent as unwritten */
+ ext4_ext_mark_unwritten(ex);
/* note: ext4_ext_correct_indexes() isn't needed here because
* borders are not changed
@@ -3971,10 +3972,10 @@
/*
* Make sure that the extent is no bigger than we support with
- * uninitialized extent
+ * unwritten extent
*/
- if (map->m_len > EXT_UNINIT_MAX_LEN)
- map->m_len = EXT_UNINIT_MAX_LEN / 2;
+ if (map->m_len > EXT_UNWRITTEN_MAX_LEN)
+ map->m_len = EXT_UNWRITTEN_MAX_LEN / 2;
ret = ext4_convert_initialized_extents(handle, inode, map,
path);
@@ -3993,7 +3994,7 @@
}
static int
-ext4_ext_handle_uninitialized_extents(handle_t *handle, struct inode *inode,
+ext4_ext_handle_unwritten_extents(handle_t *handle, struct inode *inode,
struct ext4_map_blocks *map,
struct ext4_ext_path *path, int flags,
unsigned int allocated, ext4_fsblk_t newblock)
@@ -4002,23 +4003,23 @@
int err = 0;
ext4_io_end_t *io = ext4_inode_aio(inode);
- ext_debug("ext4_ext_handle_uninitialized_extents: inode %lu, logical "
+ ext_debug("ext4_ext_handle_unwritten_extents: inode %lu, logical "
"block %llu, max_blocks %u, flags %x, allocated %u\n",
inode->i_ino, (unsigned long long)map->m_lblk, map->m_len,
flags, allocated);
ext4_ext_show_leaf(inode, path);
/*
- * When writing into uninitialized space, we should not fail to
+ * When writing into unwritten space, we should not fail to
* allocate metadata blocks for the new extent block if needed.
*/
flags |= EXT4_GET_BLOCKS_METADATA_NOFAIL;
- trace_ext4_ext_handle_uninitialized_extents(inode, map, flags,
+ trace_ext4_ext_handle_unwritten_extents(inode, map, flags,
allocated, newblock);
/* get_block() before submit the IO, split the extent */
- if ((flags & EXT4_GET_BLOCKS_PRE_IO)) {
+ if (flags & EXT4_GET_BLOCKS_PRE_IO) {
ret = ext4_split_convert_extents(handle, inode, map,
path, flags | EXT4_GET_BLOCKS_CONVERT);
if (ret <= 0)
@@ -4033,12 +4034,10 @@
else
ext4_set_inode_state(inode, EXT4_STATE_DIO_UNWRITTEN);
map->m_flags |= EXT4_MAP_UNWRITTEN;
- if (ext4_should_dioread_nolock(inode))
- map->m_flags |= EXT4_MAP_UNINIT;
goto out;
}
/* IO end_io complete, convert the filled extent to written */
- if ((flags & EXT4_GET_BLOCKS_CONVERT)) {
+ if (flags & EXT4_GET_BLOCKS_CONVERT) {
ret = ext4_convert_unwritten_extents_endio(handle, inode, map,
path);
if (ret >= 0) {
@@ -4059,7 +4058,7 @@
* repeat fallocate creation request
* we already have an unwritten extent
*/
- if (flags & EXT4_GET_BLOCKS_UNINIT_EXT) {
+ if (flags & EXT4_GET_BLOCKS_UNWRIT_EXT) {
map->m_flags |= EXT4_MAP_UNWRITTEN;
goto map_out;
}
@@ -4310,7 +4309,7 @@
/*
- * Uninitialized extents are treated as holes, except that
+ * unwritten extents are treated as holes, except that
* we split out initialized portions during a write.
*/
ee_len = ext4_ext_get_actual_len(ex);
@@ -4329,16 +4328,16 @@
* If the extent is initialized check whether the
* caller wants to convert it to unwritten.
*/
- if ((!ext4_ext_is_uninitialized(ex)) &&
+ if ((!ext4_ext_is_unwritten(ex)) &&
(flags & EXT4_GET_BLOCKS_CONVERT_UNWRITTEN)) {
allocated = ext4_ext_convert_initialized_extent(
handle, inode, map, path, flags,
allocated, newblock);
goto out2;
- } else if (!ext4_ext_is_uninitialized(ex))
+ } else if (!ext4_ext_is_unwritten(ex))
goto out;
- ret = ext4_ext_handle_uninitialized_extents(
+ ret = ext4_ext_handle_unwritten_extents(
handle, inode, map, path, flags,
allocated, newblock);
if (ret < 0)
@@ -4410,15 +4409,15 @@
/*
* See if request is beyond maximum number of blocks we can have in
* a single extent. For an initialized extent this limit is
- * EXT_INIT_MAX_LEN and for an uninitialized extent this limit is
- * EXT_UNINIT_MAX_LEN.
+ * EXT_INIT_MAX_LEN and for an unwritten extent this limit is
+ * EXT_UNWRITTEN_MAX_LEN.
*/
if (map->m_len > EXT_INIT_MAX_LEN &&
- !(flags & EXT4_GET_BLOCKS_UNINIT_EXT))
+ !(flags & EXT4_GET_BLOCKS_UNWRIT_EXT))
map->m_len = EXT_INIT_MAX_LEN;
- else if (map->m_len > EXT_UNINIT_MAX_LEN &&
- (flags & EXT4_GET_BLOCKS_UNINIT_EXT))
- map->m_len = EXT_UNINIT_MAX_LEN;
+ else if (map->m_len > EXT_UNWRITTEN_MAX_LEN &&
+ (flags & EXT4_GET_BLOCKS_UNWRIT_EXT))
+ map->m_len = EXT_UNWRITTEN_MAX_LEN;
/* Check if we can really insert (m_lblk)::(m_lblk + m_len) extent */
newex.ee_len = cpu_to_le16(map->m_len);
@@ -4466,21 +4465,19 @@
/* try to insert new extent into found leaf and return */
ext4_ext_store_pblock(&newex, newblock + offset);
newex.ee_len = cpu_to_le16(ar.len);
- /* Mark uninitialized */
- if (flags & EXT4_GET_BLOCKS_UNINIT_EXT){
- ext4_ext_mark_uninitialized(&newex);
+ /* Mark unwritten */
+ if (flags & EXT4_GET_BLOCKS_UNWRIT_EXT){
+ ext4_ext_mark_unwritten(&newex);
map->m_flags |= EXT4_MAP_UNWRITTEN;
/*
* io_end structure was created for every IO write to an
- * uninitialized extent. To avoid unnecessary conversion,
+ * unwritten extent. To avoid unnecessary conversion,
* here we flag the IO that really needs the conversion.
* For non asycn direct IO case, flag the inode state
* that we need to perform conversion when IO is done.
*/
- if ((flags & EXT4_GET_BLOCKS_PRE_IO))
+ if (flags & EXT4_GET_BLOCKS_PRE_IO)
set_unwritten = 1;
- if (ext4_should_dioread_nolock(inode))
- map->m_flags |= EXT4_MAP_UNINIT;
}
err = 0;
@@ -4607,9 +4604,9 @@
/*
* Cache the extent and update transaction to commit on fdatasync only
- * when it is _not_ an uninitialized extent.
+ * when it is _not_ an unwritten extent.
*/
- if ((flags & EXT4_GET_BLOCKS_UNINIT_EXT) == 0)
+ if ((flags & EXT4_GET_BLOCKS_UNWRIT_EXT) == 0)
ext4_update_inode_fsync_trans(handle, inode, 1);
else
ext4_update_inode_fsync_trans(handle, inode, 0);
@@ -4683,7 +4680,7 @@
* that it doesn't get unnecessarily split into multiple
* extents.
*/
- if (len <= EXT_UNINIT_MAX_LEN)
+ if (len <= EXT_UNWRITTEN_MAX_LEN)
flags |= EXT4_GET_BLOCKS_NO_NORMALIZE;
/*
@@ -4744,6 +4741,13 @@
if (!S_ISREG(inode->i_mode))
return -EINVAL;
+ /* Call ext4_force_commit to flush all data in case of data=journal. */
+ if (ext4_should_journal_data(inode)) {
+ ret = ext4_force_commit(inode->i_sb);
+ if (ret)
+ return ret;
+ }
+
/*
* Write out all dirty pages to avoid race conditions
* Then release them.
@@ -4775,7 +4779,7 @@
else
max_blocks -= lblk;
- flags = EXT4_GET_BLOCKS_CREATE_UNINIT_EXT |
+ flags = EXT4_GET_BLOCKS_CREATE_UNWRIT_EXT |
EXT4_GET_BLOCKS_CONVERT_UNWRITTEN;
if (mode & FALLOC_FL_KEEP_SIZE)
flags |= EXT4_GET_BLOCKS_KEEP_SIZE;
@@ -4918,7 +4922,7 @@
max_blocks = (EXT4_BLOCK_ALIGN(len + offset, blkbits) >> blkbits)
- lblk;
- flags = EXT4_GET_BLOCKS_CREATE_UNINIT_EXT;
+ flags = EXT4_GET_BLOCKS_CREATE_UNWRIT_EXT;
if (mode & FALLOC_FL_KEEP_SIZE)
flags |= EXT4_GET_BLOCKS_KEEP_SIZE;
diff --git a/fs/ext4/extents_status.c b/fs/ext4/extents_status.c
index 0ebc212..3f5c188 100644
--- a/fs/ext4/extents_status.c
+++ b/fs/ext4/extents_status.c
@@ -344,8 +344,14 @@
if (ext4_es_status(es1) != ext4_es_status(es2))
return 0;
- if (((__u64) es1->es_len) + es2->es_len > 0xFFFFFFFFULL)
+ if (((__u64) es1->es_len) + es2->es_len > EXT_MAX_BLOCKS) {
+ pr_warn("ES assertion failed when merging extents. "
+ "The sum of lengths of es1 (%d) and es2 (%d) "
+ "is bigger than allowed file size (%d)\n",
+ es1->es_len, es2->es_len, EXT_MAX_BLOCKS);
+ WARN_ON(1);
return 0;
+ }
if (((__u64) es1->es_lblk) + es1->es_len != es2->es_lblk)
return 0;
@@ -433,7 +439,7 @@
ee_start = ext4_ext_pblock(ex);
ee_len = ext4_ext_get_actual_len(ex);
- ee_status = ext4_ext_is_uninitialized(ex) ? 1 : 0;
+ ee_status = ext4_ext_is_unwritten(ex) ? 1 : 0;
es_status = ext4_es_is_unwritten(es) ? 1 : 0;
/*
diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 063fc15..4e8bc284 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -57,7 +57,7 @@
return 0;
}
-void ext4_unwritten_wait(struct inode *inode)
+static void ext4_unwritten_wait(struct inode *inode)
{
wait_queue_head_t *wq = ext4_ioend_wq(inode);
@@ -92,58 +92,91 @@
}
static ssize_t
-ext4_file_dio_write(struct kiocb *iocb, const struct iovec *iov,
- unsigned long nr_segs, loff_t pos)
+ext4_file_write(struct kiocb *iocb, const struct iovec *iov,
+ unsigned long nr_segs, loff_t pos)
{
struct file *file = iocb->ki_filp;
- struct inode *inode = file->f_mapping->host;
+ struct inode *inode = file_inode(iocb->ki_filp);
+ struct mutex *aio_mutex = NULL;
struct blk_plug plug;
- int unaligned_aio = 0;
- ssize_t ret;
+ int o_direct = file->f_flags & O_DIRECT;
int overwrite = 0;
size_t length = iov_length(iov, nr_segs);
-
- if (ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS) &&
- !is_sync_kiocb(iocb))
- unaligned_aio = ext4_unaligned_aio(inode, iov, nr_segs, pos);
-
- /* Unaligned direct AIO must be serialized; see comment above */
- if (unaligned_aio) {
- mutex_lock(ext4_aio_mutex(inode));
- ext4_unwritten_wait(inode);
- }
+ ssize_t ret;
BUG_ON(iocb->ki_pos != pos);
+ /*
+ * Unaligned direct AIO must be serialized; see comment above
+ * In the case of O_APPEND, assume that we must always serialize
+ */
+ if (o_direct &&
+ ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS) &&
+ !is_sync_kiocb(iocb) &&
+ (file->f_flags & O_APPEND ||
+ ext4_unaligned_aio(inode, iov, nr_segs, pos))) {
+ aio_mutex = ext4_aio_mutex(inode);
+ mutex_lock(aio_mutex);
+ ext4_unwritten_wait(inode);
+ }
+
mutex_lock(&inode->i_mutex);
- blk_start_plug(&plug);
+ if (file->f_flags & O_APPEND)
+ iocb->ki_pos = pos = i_size_read(inode);
- iocb->private = &overwrite;
+ /*
+ * If we have encountered a bitmap-format file, the size limit
+ * is smaller than s_maxbytes, which is for extent-mapped files.
+ */
+ if (!(ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS))) {
+ struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
- /* check whether we do a DIO overwrite or not */
- if (ext4_should_dioread_nolock(inode) && !unaligned_aio &&
- !file->f_mapping->nrpages && pos + length <= i_size_read(inode)) {
- struct ext4_map_blocks map;
- unsigned int blkbits = inode->i_blkbits;
- int err, len;
+ if ((pos > sbi->s_bitmap_maxbytes) ||
+ (pos == sbi->s_bitmap_maxbytes && length > 0)) {
+ mutex_unlock(&inode->i_mutex);
+ ret = -EFBIG;
+ goto errout;
+ }
- map.m_lblk = pos >> blkbits;
- map.m_len = (EXT4_BLOCK_ALIGN(pos + length, blkbits) >> blkbits)
- - map.m_lblk;
- len = map.m_len;
+ if (pos + length > sbi->s_bitmap_maxbytes) {
+ nr_segs = iov_shorten((struct iovec *)iov, nr_segs,
+ sbi->s_bitmap_maxbytes - pos);
+ }
+ }
- err = ext4_map_blocks(NULL, inode, &map, 0);
- /*
- * 'err==len' means that all of blocks has been preallocated no
- * matter they are initialized or not. For excluding
- * uninitialized extents, we need to check m_flags. There are
- * two conditions that indicate for initialized extents.
- * 1) If we hit extent cache, EXT4_MAP_MAPPED flag is returned;
- * 2) If we do a real lookup, non-flags are returned.
- * So we should check these two conditions.
- */
- if (err == len && (map.m_flags & EXT4_MAP_MAPPED))
- overwrite = 1;
+ if (o_direct) {
+ blk_start_plug(&plug);
+
+ iocb->private = &overwrite;
+
+ /* check whether we do a DIO overwrite or not */
+ if (ext4_should_dioread_nolock(inode) && !aio_mutex &&
+ !file->f_mapping->nrpages && pos + length <= i_size_read(inode)) {
+ struct ext4_map_blocks map;
+ unsigned int blkbits = inode->i_blkbits;
+ int err, len;
+
+ map.m_lblk = pos >> blkbits;
+ map.m_len = (EXT4_BLOCK_ALIGN(pos + length, blkbits) >> blkbits)
+ - map.m_lblk;
+ len = map.m_len;
+
+ err = ext4_map_blocks(NULL, inode, &map, 0);
+ /*
+ * 'err==len' means that all of blocks has
+ * been preallocated no matter they are
+ * initialized or not. For excluding
+ * unwritten extents, we need to check
+ * m_flags. There are two conditions that
+ * indicate for initialized extents. 1) If we
+ * hit extent cache, EXT4_MAP_MAPPED flag is
+ * returned; 2) If we do a real lookup,
+ * non-flags are returned. So we should check
+ * these two conditions.
+ */
+ if (err == len && (map.m_flags & EXT4_MAP_MAPPED))
+ overwrite = 1;
+ }
}
ret = __generic_file_aio_write(iocb, iov, nr_segs);
@@ -156,45 +189,12 @@
if (err < 0)
ret = err;
}
- blk_finish_plug(&plug);
+ if (o_direct)
+ blk_finish_plug(&plug);
- if (unaligned_aio)
- mutex_unlock(ext4_aio_mutex(inode));
-
- return ret;
-}
-
-static ssize_t
-ext4_file_write(struct kiocb *iocb, const struct iovec *iov,
- unsigned long nr_segs, loff_t pos)
-{
- struct inode *inode = file_inode(iocb->ki_filp);
- ssize_t ret;
-
- /*
- * If we have encountered a bitmap-format file, the size limit
- * is smaller than s_maxbytes, which is for extent-mapped files.
- */
-
- if (!(ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS))) {
- struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
- size_t length = iov_length(iov, nr_segs);
-
- if ((pos > sbi->s_bitmap_maxbytes ||
- (pos == sbi->s_bitmap_maxbytes && length > 0)))
- return -EFBIG;
-
- if (pos + length > sbi->s_bitmap_maxbytes) {
- nr_segs = iov_shorten((struct iovec *)iov, nr_segs,
- sbi->s_bitmap_maxbytes - pos);
- }
- }
-
- if (unlikely(iocb->ki_filp->f_flags & O_DIRECT))
- ret = ext4_file_dio_write(iocb, iov, nr_segs, pos);
- else
- ret = generic_file_aio_write(iocb, iov, nr_segs, pos);
-
+errout:
+ if (aio_mutex)
+ mutex_unlock(aio_mutex);
return ret;
}
@@ -244,6 +244,7 @@
handle = ext4_journal_start_sb(sb, EXT4_HT_MISC, 1);
if (IS_ERR(handle))
return PTR_ERR(handle);
+ BUFFER_TRACE(sbi->s_sbh, "get_write_access");
err = ext4_journal_get_write_access(handle, sbi->s_sbh);
if (err) {
ext4_journal_stop(handle);
diff --git a/fs/ext4/inline.c b/fs/ext4/inline.c
index 82edf5b..645205d 100644
--- a/fs/ext4/inline.c
+++ b/fs/ext4/inline.c
@@ -22,7 +22,7 @@
#define EXT4_INLINE_DOTDOT_OFFSET 2
#define EXT4_INLINE_DOTDOT_SIZE 4
-int ext4_get_inline_size(struct inode *inode)
+static int ext4_get_inline_size(struct inode *inode)
{
if (EXT4_I(inode)->i_inline_off)
return EXT4_I(inode)->i_inline_size;
@@ -211,8 +211,8 @@
* value since it is already handled by ext4_xattr_ibody_inline_set.
* That saves us one memcpy.
*/
-void ext4_write_inline_data(struct inode *inode, struct ext4_iloc *iloc,
- void *buffer, loff_t pos, unsigned int len)
+static void ext4_write_inline_data(struct inode *inode, struct ext4_iloc *iloc,
+ void *buffer, loff_t pos, unsigned int len)
{
struct ext4_xattr_entry *entry;
struct ext4_xattr_ibody_header *header;
@@ -264,6 +264,7 @@
if (error)
return error;
+ BUFFER_TRACE(is.iloc.bh, "get_write_access");
error = ext4_journal_get_write_access(handle, is.iloc.bh);
if (error)
goto out;
@@ -347,6 +348,7 @@
if (error == -ENODATA)
goto out;
+ BUFFER_TRACE(is.iloc.bh, "get_write_access");
error = ext4_journal_get_write_access(handle, is.iloc.bh);
if (error)
goto out;
@@ -373,8 +375,8 @@
return error;
}
-int ext4_prepare_inline_data(handle_t *handle, struct inode *inode,
- unsigned int len)
+static int ext4_prepare_inline_data(handle_t *handle, struct inode *inode,
+ unsigned int len)
{
int ret, size;
struct ext4_inode_info *ei = EXT4_I(inode);
@@ -424,6 +426,7 @@
if (error)
goto out;
+ BUFFER_TRACE(is.iloc.bh, "get_write_access");
error = ext4_journal_get_write_access(handle, is.iloc.bh);
if (error)
goto out;
@@ -1007,6 +1010,7 @@
if (err)
return err;
+ BUFFER_TRACE(iloc->bh, "get_write_access");
err = ext4_journal_get_write_access(handle, iloc->bh);
if (err)
return err;
@@ -1669,6 +1673,7 @@
EXT4_MIN_INLINE_DATA_SIZE;
}
+ BUFFER_TRACE(bh, "get_write_access");
err = ext4_journal_get_write_access(handle, bh);
if (err)
goto out;
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index d7b7462..7fcd68e 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -148,6 +148,9 @@
int ea_blocks = EXT4_I(inode)->i_file_acl ?
EXT4_CLUSTER_SIZE(inode->i_sb) >> 9 : 0;
+ if (ext4_has_inline_data(inode))
+ return 0;
+
return (S_ISLNK(inode->i_mode) && inode->i_blocks - ea_blocks == 0);
}
@@ -443,7 +446,7 @@
* could be converted.
*/
if (!(flags & EXT4_GET_BLOCKS_NO_LOCK))
- down_read((&EXT4_I(inode)->i_data_sem));
+ down_read(&EXT4_I(inode)->i_data_sem);
if (ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS)) {
retval = ext4_ext_map_blocks(handle, inode, map, flags &
EXT4_GET_BLOCKS_KEEP_SIZE);
@@ -489,8 +492,8 @@
* Otherwise, call with ext4_ind_map_blocks() to handle indirect mapping
* based files
*
- * On success, it returns the number of blocks being mapped or allocate.
- * if create==0 and the blocks are pre-allocated and uninitialized block,
+ * On success, it returns the number of blocks being mapped or allocated.
+ * if create==0 and the blocks are pre-allocated and unwritten block,
* the result buffer head is unmapped. If the create ==1, it will make sure
* the buffer head is mapped.
*
@@ -555,7 +558,7 @@
* file system block.
*/
if (!(flags & EXT4_GET_BLOCKS_NO_LOCK))
- down_read((&EXT4_I(inode)->i_data_sem));
+ down_read(&EXT4_I(inode)->i_data_sem);
if (ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS)) {
retval = ext4_ext_map_blocks(handle, inode, map, flags &
EXT4_GET_BLOCKS_KEEP_SIZE);
@@ -622,12 +625,12 @@
map->m_flags &= ~EXT4_MAP_FLAGS;
/*
- * New blocks allocate and/or writing to uninitialized extent
+ * New blocks allocate and/or writing to unwritten extent
* will possibly result in updating i_data, so we take
* the write lock of i_data_sem, and call get_blocks()
* with create == 1 flag.
*/
- down_write((&EXT4_I(inode)->i_data_sem));
+ down_write(&EXT4_I(inode)->i_data_sem);
/*
* if the caller is from delayed allocation writeout path
@@ -922,6 +925,7 @@
*/
if (dirty)
clear_buffer_dirty(bh);
+ BUFFER_TRACE(bh, "get write access");
ret = ext4_journal_get_write_access(handle, bh);
if (!ret && dirty)
ret = ext4_handle_dirty_metadata(handle, NULL, bh);
@@ -1540,7 +1544,7 @@
ext4_es_lru_add(inode);
if (ext4_es_is_hole(&es)) {
retval = 0;
- down_read((&EXT4_I(inode)->i_data_sem));
+ down_read(&EXT4_I(inode)->i_data_sem);
goto add_delayed;
}
@@ -1577,7 +1581,7 @@
* Try to see if we can get the block without requesting a new
* file system block.
*/
- down_read((&EXT4_I(inode)->i_data_sem));
+ down_read(&EXT4_I(inode)->i_data_sem);
if (ext4_has_inline_data(inode)) {
/*
* We will soon create blocks for this page, and let
@@ -1769,6 +1773,7 @@
BUG_ON(!ext4_handle_valid(handle));
if (inline_data) {
+ BUFFER_TRACE(inode_bh, "get write access");
ret = ext4_journal_get_write_access(handle, inode_bh);
err = ext4_handle_dirty_metadata(handle, inode, inode_bh);
@@ -1846,6 +1851,7 @@
struct buffer_head *page_bufs = NULL;
struct inode *inode = page->mapping->host;
struct ext4_io_submit io_submit;
+ bool keep_towrite = false;
trace_ext4_writepage(page);
size = i_size_read(inode);
@@ -1876,6 +1882,7 @@
unlock_page(page);
return 0;
}
+ keep_towrite = true;
}
if (PageChecked(page) && ext4_should_journal_data(inode))
@@ -1892,7 +1899,7 @@
unlock_page(page);
return -ENOMEM;
}
- ret = ext4_bio_write_page(&io_submit, page, len, wbc);
+ ret = ext4_bio_write_page(&io_submit, page, len, wbc, keep_towrite);
ext4_io_submit(&io_submit);
/* Drop io_end reference we got from init */
ext4_put_io_end_defer(io_submit.io_end);
@@ -1911,7 +1918,7 @@
else
len = PAGE_CACHE_SIZE;
clear_page_dirty_for_io(page);
- err = ext4_bio_write_page(&mpd->io_submit, page, len, mpd->wbc);
+ err = ext4_bio_write_page(&mpd->io_submit, page, len, mpd->wbc, false);
if (!err)
mpd->wbc->nr_to_write--;
mpd->first_page++;
@@ -2032,7 +2039,7 @@
* Scan buffers corresponding to changed extent (we expect corresponding pages
* to be already locked) and update buffer state according to new extent state.
* We map delalloc buffers to their physical location, clear unwritten bits,
- * and mark buffers as uninit when we perform writes to uninitialized extents
+ * and mark buffers as uninit when we perform writes to unwritten extents
* and do extent conversion after IO is finished. If the last page is not fully
* mapped, we update @map to the next extent in the last page that needs
* mapping. Otherwise we submit the page for IO.
@@ -2126,12 +2133,12 @@
struct inode *inode = mpd->inode;
struct ext4_map_blocks *map = &mpd->map;
int get_blocks_flags;
- int err;
+ int err, dioread_nolock;
trace_ext4_da_write_pages_extent(inode, map);
/*
* Call ext4_map_blocks() to allocate any delayed allocation blocks, or
- * to convert an uninitialized extent to be initialized (in the case
+ * to convert an unwritten extent to be initialized (in the case
* where we have written into one or more preallocated blocks). It is
* possible that we're going to need more metadata blocks than
* previously reserved. However we must not fail because we're in
@@ -2148,7 +2155,8 @@
*/
get_blocks_flags = EXT4_GET_BLOCKS_CREATE |
EXT4_GET_BLOCKS_METADATA_NOFAIL;
- if (ext4_should_dioread_nolock(inode))
+ dioread_nolock = ext4_should_dioread_nolock(inode);
+ if (dioread_nolock)
get_blocks_flags |= EXT4_GET_BLOCKS_IO_CREATE_EXT;
if (map->m_flags & (1 << BH_Delay))
get_blocks_flags |= EXT4_GET_BLOCKS_DELALLOC_RESERVE;
@@ -2156,7 +2164,7 @@
err = ext4_map_blocks(handle, inode, map, get_blocks_flags);
if (err < 0)
return err;
- if (map->m_flags & EXT4_MAP_UNINIT) {
+ if (dioread_nolock && (map->m_flags & EXT4_MAP_UNWRITTEN)) {
if (!mpd->io_submit.io_end->handle &&
ext4_handle_valid(handle)) {
mpd->io_submit.io_end->handle = handle->h_rsv_handle;
@@ -3070,9 +3078,9 @@
* preallocated extents, and those write extend the file, no need to
* fall back to buffered IO.
*
- * For holes, we fallocate those blocks, mark them as uninitialized
+ * For holes, we fallocate those blocks, mark them as unwritten
* If those blocks were preallocated, we mark sure they are split, but
- * still keep the range to write as uninitialized.
+ * still keep the range to write as unwritten.
*
* The unwritten extents will be converted to written when DIO is completed.
* For async direct IO, since the IO may still pending when return, we
@@ -3124,12 +3132,12 @@
* We could direct write to holes and fallocate.
*
* Allocated blocks to fill the hole are marked as
- * uninitialized to prevent parallel buffered read to expose
+ * unwritten to prevent parallel buffered read to expose
* the stale data before DIO complete the data IO.
*
* As to previously fallocated extents, ext4 get_block will
* just simply mark the buffer mapped but still keep the
- * extents uninitialized.
+ * extents unwritten.
*
* For non AIO case, we will convert those unwritten extents
* to written after return back from blockdev_direct_IO.
@@ -3440,7 +3448,7 @@
* This required during truncate. We need to physically zero the tail end
* of that block so it doesn't yield old data if the file is later grown.
*/
-int ext4_block_truncate_page(handle_t *handle,
+static int ext4_block_truncate_page(handle_t *handle,
struct address_space *mapping, loff_t from)
{
unsigned offset = from & (PAGE_CACHE_SIZE-1);
@@ -4304,12 +4312,15 @@
struct ext4_inode *raw_inode = ext4_raw_inode(iloc);
struct ext4_inode_info *ei = EXT4_I(inode);
struct buffer_head *bh = iloc->bh;
+ struct super_block *sb = inode->i_sb;
int err = 0, rc, block;
- int need_datasync = 0;
+ int need_datasync = 0, set_large_file = 0;
uid_t i_uid;
gid_t i_gid;
- /* For fields not not tracking in the in-memory inode,
+ spin_lock(&ei->i_raw_lock);
+
+ /* For fields not tracked in the in-memory inode,
* initialise them to zero for new inodes. */
if (ext4_test_inode_state(inode, EXT4_STATE_NEW))
memset(raw_inode, 0, EXT4_SB(inode->i_sb)->s_inode_size);
@@ -4347,8 +4358,10 @@
EXT4_INODE_SET_XTIME(i_atime, inode, raw_inode);
EXT4_EINODE_SET_XTIME(i_crtime, ei, raw_inode);
- if (ext4_inode_blocks_set(handle, raw_inode, ei))
+ if (ext4_inode_blocks_set(handle, raw_inode, ei)) {
+ spin_unlock(&ei->i_raw_lock);
goto out_brelse;
+ }
raw_inode->i_dtime = cpu_to_le32(ei->i_dtime);
raw_inode->i_flags = cpu_to_le32(ei->i_flags & 0xFFFFFFFF);
if (likely(!test_opt2(inode->i_sb, HURD_COMPAT)))
@@ -4360,24 +4373,11 @@
need_datasync = 1;
}
if (ei->i_disksize > 0x7fffffffULL) {
- struct super_block *sb = inode->i_sb;
if (!EXT4_HAS_RO_COMPAT_FEATURE(sb,
EXT4_FEATURE_RO_COMPAT_LARGE_FILE) ||
EXT4_SB(sb)->s_es->s_rev_level ==
- cpu_to_le32(EXT4_GOOD_OLD_REV)) {
- /* If this is the first large file
- * created, add a flag to the superblock.
- */
- err = ext4_journal_get_write_access(handle,
- EXT4_SB(sb)->s_sbh);
- if (err)
- goto out_brelse;
- ext4_update_dynamic_rev(sb);
- EXT4_SET_RO_COMPAT_FEATURE(sb,
- EXT4_FEATURE_RO_COMPAT_LARGE_FILE);
- ext4_handle_sync(handle);
- err = ext4_handle_dirty_super(handle, sb);
- }
+ cpu_to_le32(EXT4_GOOD_OLD_REV))
+ set_large_file = 1;
}
raw_inode->i_generation = cpu_to_le32(inode->i_generation);
if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode)) {
@@ -4409,12 +4409,24 @@
ext4_inode_csum_set(inode, raw_inode, ei);
+ spin_unlock(&ei->i_raw_lock);
+
BUFFER_TRACE(bh, "call ext4_handle_dirty_metadata");
rc = ext4_handle_dirty_metadata(handle, NULL, bh);
if (!err)
err = rc;
ext4_clear_inode_state(inode, EXT4_STATE_NEW);
-
+ if (set_large_file) {
+ BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get write access");
+ err = ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh);
+ if (err)
+ goto out_brelse;
+ ext4_update_dynamic_rev(sb);
+ EXT4_SET_RO_COMPAT_FEATURE(sb,
+ EXT4_FEATURE_RO_COMPAT_LARGE_FILE);
+ ext4_handle_sync(handle);
+ err = ext4_handle_dirty_super(handle, sb);
+ }
ext4_update_inode_fsync_trans(handle, inode, need_datasync);
out_brelse:
brelse(bh);
diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
index afe8a13..59e3162 100644
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -2619,7 +2619,7 @@
sbi->s_locality_groups = alloc_percpu(struct ext4_locality_group);
if (sbi->s_locality_groups == NULL) {
ret = -ENOMEM;
- goto out_free_groupinfo_slab;
+ goto out;
}
for_each_possible_cpu(i) {
struct ext4_locality_group *lg;
@@ -2644,8 +2644,6 @@
out_free_locality_groups:
free_percpu(sbi->s_locality_groups);
sbi->s_locality_groups = NULL;
-out_free_groupinfo_slab:
- ext4_groupinfo_destroy_slabs();
out:
kfree(sbi->s_mb_offsets);
sbi->s_mb_offsets = NULL;
@@ -2878,6 +2876,7 @@
if (!bitmap_bh)
goto out_err;
+ BUFFER_TRACE(bitmap_bh, "getting write access");
err = ext4_journal_get_write_access(handle, bitmap_bh);
if (err)
goto out_err;
@@ -2890,6 +2889,7 @@
ext4_debug("using block group %u(%d)\n", ac->ac_b_ex.fe_group,
ext4_free_group_clusters(sb, gdp));
+ BUFFER_TRACE(gdp_bh, "get_write_access");
err = ext4_journal_get_write_access(handle, gdp_bh);
if (err)
goto out_err;
@@ -3147,7 +3147,7 @@
}
BUG_ON(start + size <= ac->ac_o_ex.fe_logical &&
start > ac->ac_o_ex.fe_logical);
- BUG_ON(size <= 0 || size > EXT4_CLUSTERS_PER_GROUP(ac->ac_sb));
+ BUG_ON(size <= 0 || size > EXT4_BLOCKS_PER_GROUP(ac->ac_sb));
/* now prepare goal request */
diff --git a/fs/ext4/migrate.c b/fs/ext4/migrate.c
index 2ae73a8..ec09243 100644
--- a/fs/ext4/migrate.c
+++ b/fs/ext4/migrate.c
@@ -505,7 +505,7 @@
* with i_data_sem held to prevent racing with block
* allocation.
*/
- down_read((&EXT4_I(inode)->i_data_sem));
+ down_read(&EXT4_I(inode)->i_data_sem);
ext4_set_inode_state(inode, EXT4_STATE_EXT_MIGRATE);
up_read((&EXT4_I(inode)->i_data_sem));
diff --git a/fs/ext4/mmp.c b/fs/ext4/mmp.c
index 04434ad..32bce84 100644
--- a/fs/ext4/mmp.c
+++ b/fs/ext4/mmp.c
@@ -18,7 +18,7 @@
return cpu_to_le32(csum);
}
-int ext4_mmp_csum_verify(struct super_block *sb, struct mmp_struct *mmp)
+static int ext4_mmp_csum_verify(struct super_block *sb, struct mmp_struct *mmp)
{
if (!EXT4_HAS_RO_COMPAT_FEATURE(sb,
EXT4_FEATURE_RO_COMPAT_METADATA_CSUM))
@@ -27,7 +27,7 @@
return mmp->mmp_checksum == ext4_mmp_csum(sb, mmp);
}
-void ext4_mmp_csum_set(struct super_block *sb, struct mmp_struct *mmp)
+static void ext4_mmp_csum_set(struct super_block *sb, struct mmp_struct *mmp)
{
if (!EXT4_HAS_RO_COMPAT_FEATURE(sb,
EXT4_FEATURE_RO_COMPAT_METADATA_CSUM))
diff --git a/fs/ext4/move_extent.c b/fs/ext4/move_extent.c
index 58ee7dc..2484c7e 100644
--- a/fs/ext4/move_extent.c
+++ b/fs/ext4/move_extent.c
@@ -57,8 +57,8 @@
static void
copy_extent_status(struct ext4_extent *src, struct ext4_extent *dest)
{
- if (ext4_ext_is_uninitialized(src))
- ext4_ext_mark_uninitialized(dest);
+ if (ext4_ext_is_unwritten(src))
+ ext4_ext_mark_unwritten(dest);
else
dest->ee_len = cpu_to_le16(ext4_ext_get_actual_len(dest));
}
@@ -391,6 +391,7 @@
if (depth) {
/* Register to journal */
+ BUFFER_TRACE(orig_path->p_bh, "get_write_access");
ret = ext4_journal_get_write_access(handle, orig_path->p_bh);
if (ret)
return ret;
@@ -593,14 +594,14 @@
* @inode: inode in question
* @from: block offset of inode
* @count: block count to be checked
- * @uninit: extents expected to be uninitialized
+ * @unwritten: extents expected to be unwritten
* @err: pointer to save error value
*
* Return 1 if all extents in range has expected type, and zero otherwise.
*/
static int
mext_check_coverage(struct inode *inode, ext4_lblk_t from, ext4_lblk_t count,
- int uninit, int *err)
+ int unwritten, int *err)
{
struct ext4_ext_path *path = NULL;
struct ext4_extent *ext;
@@ -611,7 +612,7 @@
if (*err)
goto out;
ext = path[ext_depth(inode)].p_ext;
- if (uninit != ext4_ext_is_uninitialized(ext))
+ if (unwritten != ext4_ext_is_unwritten(ext))
goto out;
from += ext4_ext_get_actual_len(ext);
ext4_ext_drop_refs(path);
@@ -894,7 +895,7 @@
* @orig_page_offset: page index on original file
* @data_offset_in_page: block index where data swapping starts
* @block_len_in_page: the number of blocks to be swapped
- * @uninit: orig extent is uninitialized or not
+ * @unwritten: orig extent is unwritten or not
* @err: pointer to save return value
*
* Save the data in original inode blocks and replace original inode extents
@@ -905,7 +906,7 @@
static int
move_extent_per_page(struct file *o_filp, struct inode *donor_inode,
pgoff_t orig_page_offset, int data_offset_in_page,
- int block_len_in_page, int uninit, int *err)
+ int block_len_in_page, int unwritten, int *err)
{
struct inode *orig_inode = file_inode(o_filp);
struct page *pagep[2] = {NULL, NULL};
@@ -962,27 +963,27 @@
if (unlikely(*err < 0))
goto stop_journal;
/*
- * If orig extent was uninitialized it can become initialized
+ * If orig extent was unwritten it can become initialized
* at any time after i_data_sem was dropped, in order to
* serialize with delalloc we have recheck extent while we
* hold page's lock, if it is still the case data copy is not
* necessary, just swap data blocks between orig and donor.
*/
- if (uninit) {
+ if (unwritten) {
ext4_double_down_write_data_sem(orig_inode, donor_inode);
/* If any of extents in range became initialized we have to
* fallback to data copying */
- uninit = mext_check_coverage(orig_inode, orig_blk_offset,
- block_len_in_page, 1, err);
+ unwritten = mext_check_coverage(orig_inode, orig_blk_offset,
+ block_len_in_page, 1, err);
if (*err)
goto drop_data_sem;
- uninit &= mext_check_coverage(donor_inode, orig_blk_offset,
- block_len_in_page, 1, err);
+ unwritten &= mext_check_coverage(donor_inode, orig_blk_offset,
+ block_len_in_page, 1, err);
if (*err)
goto drop_data_sem;
- if (!uninit) {
+ if (!unwritten) {
ext4_double_up_write_data_sem(orig_inode, donor_inode);
goto data_copy;
}
@@ -1259,7 +1260,7 @@
int blocks_per_page = PAGE_CACHE_SIZE >> orig_inode->i_blkbits;
int data_offset_in_page;
int block_len_in_page;
- int uninit;
+ int unwritten;
if (orig_inode->i_sb != donor_inode->i_sb) {
ext4_debug("ext4 move extent: The argument files "
@@ -1391,8 +1392,8 @@
!last_extent)
continue;
- /* Is original extent is uninitialized */
- uninit = ext4_ext_is_uninitialized(ext_prev);
+ /* Is original extent is unwritten */
+ unwritten = ext4_ext_is_unwritten(ext_prev);
data_offset_in_page = seq_start % blocks_per_page;
@@ -1432,8 +1433,8 @@
o_filp, donor_inode,
orig_page_offset,
data_offset_in_page,
- block_len_in_page, uninit,
- &ret);
+ block_len_in_page,
+ unwritten, &ret);
/* Count how many blocks we have exchanged */
*moved_len += block_len_in_page;
diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c
index 1cb84f7..3520ab8 100644
--- a/fs/ext4/namei.c
+++ b/fs/ext4/namei.c
@@ -67,6 +67,7 @@
return ERR_PTR(err);
inode->i_size += inode->i_sb->s_blocksize;
EXT4_I(inode)->i_disksize = inode->i_size;
+ BUFFER_TRACE(bh, "get_write_access");
err = ext4_journal_get_write_access(handle, bh);
if (err) {
brelse(bh);
@@ -1778,6 +1779,7 @@
blocksize = dir->i_sb->s_blocksize;
dxtrace(printk(KERN_DEBUG "Creating index: inode %lu\n", dir->i_ino));
+ BUFFER_TRACE(bh, "get_write_access");
retval = ext4_journal_get_write_access(handle, bh);
if (retval) {
ext4_std_error(dir->i_sb, retval);
@@ -2510,8 +2512,7 @@
ext4_rec_len_from_disk(de1->rec_len, sb->s_blocksize);
de = ext4_next_entry(de1, sb->s_blocksize);
while (offset < inode->i_size) {
- if (!bh ||
- (void *) de >= (void *) (bh->b_data+sb->s_blocksize)) {
+ if ((void *) de >= (void *) (bh->b_data+sb->s_blocksize)) {
unsigned int lblock;
err = 0;
brelse(bh);
@@ -2539,26 +2540,37 @@
return 1;
}
-/* ext4_orphan_add() links an unlinked or truncated inode into a list of
+/*
+ * ext4_orphan_add() links an unlinked or truncated inode into a list of
* such inodes, starting at the superblock, in case we crash before the
* file is closed/deleted, or in case the inode truncate spans multiple
* transactions and the last transaction is not recovered after a crash.
*
* At filesystem recovery time, we walk this list deleting unlinked
* inodes and truncating linked inodes in ext4_orphan_cleanup().
+ *
+ * Orphan list manipulation functions must be called under i_mutex unless
+ * we are just creating the inode or deleting it.
*/
int ext4_orphan_add(handle_t *handle, struct inode *inode)
{
struct super_block *sb = inode->i_sb;
+ struct ext4_sb_info *sbi = EXT4_SB(sb);
struct ext4_iloc iloc;
int err = 0, rc;
+ bool dirty = false;
- if (!EXT4_SB(sb)->s_journal)
+ if (!sbi->s_journal)
return 0;
- mutex_lock(&EXT4_SB(sb)->s_orphan_lock);
+ WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) &&
+ !mutex_is_locked(&inode->i_mutex));
+ /*
+ * Exit early if inode already is on orphan list. This is a big speedup
+ * since we don't have to contend on the global s_orphan_lock.
+ */
if (!list_empty(&EXT4_I(inode)->i_orphan))
- goto out_unlock;
+ return 0;
/*
* Orphan handling is only valid for files with data blocks
@@ -2569,48 +2581,51 @@
J_ASSERT((S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) ||
S_ISLNK(inode->i_mode)) || inode->i_nlink == 0);
- BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get_write_access");
- err = ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh);
+ BUFFER_TRACE(sbi->s_sbh, "get_write_access");
+ err = ext4_journal_get_write_access(handle, sbi->s_sbh);
if (err)
- goto out_unlock;
+ goto out;
err = ext4_reserve_inode_write(handle, inode, &iloc);
if (err)
- goto out_unlock;
+ goto out;
+
+ mutex_lock(&sbi->s_orphan_lock);
/*
* Due to previous errors inode may be already a part of on-disk
* orphan list. If so skip on-disk list modification.
*/
- if (NEXT_ORPHAN(inode) && NEXT_ORPHAN(inode) <=
- (le32_to_cpu(EXT4_SB(sb)->s_es->s_inodes_count)))
- goto mem_insert;
+ if (!NEXT_ORPHAN(inode) || NEXT_ORPHAN(inode) >
+ (le32_to_cpu(sbi->s_es->s_inodes_count))) {
+ /* Insert this inode at the head of the on-disk orphan list */
+ NEXT_ORPHAN(inode) = le32_to_cpu(sbi->s_es->s_last_orphan);
+ sbi->s_es->s_last_orphan = cpu_to_le32(inode->i_ino);
+ dirty = true;
+ }
+ list_add(&EXT4_I(inode)->i_orphan, &sbi->s_orphan);
+ mutex_unlock(&sbi->s_orphan_lock);
- /* Insert this inode at the head of the on-disk orphan list... */
- NEXT_ORPHAN(inode) = le32_to_cpu(EXT4_SB(sb)->s_es->s_last_orphan);
- EXT4_SB(sb)->s_es->s_last_orphan = cpu_to_le32(inode->i_ino);
- err = ext4_handle_dirty_super(handle, sb);
- rc = ext4_mark_iloc_dirty(handle, inode, &iloc);
- if (!err)
- err = rc;
-
- /* Only add to the head of the in-memory list if all the
- * previous operations succeeded. If the orphan_add is going to
- * fail (possibly taking the journal offline), we can't risk
- * leaving the inode on the orphan list: stray orphan-list
- * entries can cause panics at unmount time.
- *
- * This is safe: on error we're going to ignore the orphan list
- * anyway on the next recovery. */
-mem_insert:
- if (!err)
- list_add(&EXT4_I(inode)->i_orphan, &EXT4_SB(sb)->s_orphan);
-
+ if (dirty) {
+ err = ext4_handle_dirty_super(handle, sb);
+ rc = ext4_mark_iloc_dirty(handle, inode, &iloc);
+ if (!err)
+ err = rc;
+ if (err) {
+ /*
+ * We have to remove inode from in-memory list if
+ * addition to on disk orphan list failed. Stray orphan
+ * list entries can cause panics at unmount time.
+ */
+ mutex_lock(&sbi->s_orphan_lock);
+ list_del(&EXT4_I(inode)->i_orphan);
+ mutex_unlock(&sbi->s_orphan_lock);
+ }
+ }
jbd_debug(4, "superblock will point to %lu\n", inode->i_ino);
jbd_debug(4, "orphan inode %lu will point to %d\n",
inode->i_ino, NEXT_ORPHAN(inode));
-out_unlock:
- mutex_unlock(&EXT4_SB(sb)->s_orphan_lock);
- ext4_std_error(inode->i_sb, err);
+out:
+ ext4_std_error(sb, err);
return err;
}
@@ -2622,45 +2637,51 @@
{
struct list_head *prev;
struct ext4_inode_info *ei = EXT4_I(inode);
- struct ext4_sb_info *sbi;
+ struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
__u32 ino_next;
struct ext4_iloc iloc;
int err = 0;
- if ((!EXT4_SB(inode->i_sb)->s_journal) &&
- !(EXT4_SB(inode->i_sb)->s_mount_state & EXT4_ORPHAN_FS))
+ if (!sbi->s_journal && !(sbi->s_mount_state & EXT4_ORPHAN_FS))
return 0;
- mutex_lock(&EXT4_SB(inode->i_sb)->s_orphan_lock);
+ WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) &&
+ !mutex_is_locked(&inode->i_mutex));
+ /* Do this quick check before taking global s_orphan_lock. */
if (list_empty(&ei->i_orphan))
- goto out;
+ return 0;
- ino_next = NEXT_ORPHAN(inode);
- prev = ei->i_orphan.prev;
- sbi = EXT4_SB(inode->i_sb);
+ if (handle) {
+ /* Grab inode buffer early before taking global s_orphan_lock */
+ err = ext4_reserve_inode_write(handle, inode, &iloc);
+ }
+ mutex_lock(&sbi->s_orphan_lock);
jbd_debug(4, "remove inode %lu from orphan list\n", inode->i_ino);
+ prev = ei->i_orphan.prev;
list_del_init(&ei->i_orphan);
/* If we're on an error path, we may not have a valid
* transaction handle with which to update the orphan list on
* disk, but we still need to remove the inode from the linked
* list in memory. */
- if (!handle)
- goto out;
-
- err = ext4_reserve_inode_write(handle, inode, &iloc);
- if (err)
+ if (!handle || err) {
+ mutex_unlock(&sbi->s_orphan_lock);
goto out_err;
+ }
+ ino_next = NEXT_ORPHAN(inode);
if (prev == &sbi->s_orphan) {
jbd_debug(4, "superblock will point to %u\n", ino_next);
BUFFER_TRACE(sbi->s_sbh, "get_write_access");
err = ext4_journal_get_write_access(handle, sbi->s_sbh);
- if (err)
+ if (err) {
+ mutex_unlock(&sbi->s_orphan_lock);
goto out_brelse;
+ }
sbi->s_es->s_last_orphan = cpu_to_le32(ino_next);
+ mutex_unlock(&sbi->s_orphan_lock);
err = ext4_handle_dirty_super(handle, inode->i_sb);
} else {
struct ext4_iloc iloc2;
@@ -2670,20 +2691,20 @@
jbd_debug(4, "orphan inode %lu will point to %u\n",
i_prev->i_ino, ino_next);
err = ext4_reserve_inode_write(handle, i_prev, &iloc2);
- if (err)
+ if (err) {
+ mutex_unlock(&sbi->s_orphan_lock);
goto out_brelse;
+ }
NEXT_ORPHAN(i_prev) = ino_next;
err = ext4_mark_iloc_dirty(handle, i_prev, &iloc2);
+ mutex_unlock(&sbi->s_orphan_lock);
}
if (err)
goto out_brelse;
NEXT_ORPHAN(inode) = 0;
err = ext4_mark_iloc_dirty(handle, inode, &iloc);
-
out_err:
ext4_std_error(inode->i_sb, err);
-out:
- mutex_unlock(&EXT4_SB(inode->i_sb)->s_orphan_lock);
return err;
out_brelse:
diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c
index 1a64e7a..b24a254 100644
--- a/fs/ext4/page-io.c
+++ b/fs/ext4/page-io.c
@@ -401,7 +401,8 @@
int ext4_bio_write_page(struct ext4_io_submit *io,
struct page *page,
int len,
- struct writeback_control *wbc)
+ struct writeback_control *wbc,
+ bool keep_towrite)
{
struct inode *inode = page->mapping->host;
unsigned block_start, blocksize;
@@ -414,10 +415,24 @@
BUG_ON(!PageLocked(page));
BUG_ON(PageWriteback(page));
- set_page_writeback(page);
+ if (keep_towrite)
+ set_page_writeback_keepwrite(page);
+ else
+ set_page_writeback(page);
ClearPageError(page);
/*
+ * Comments copied from block_write_full_page:
+ *
+ * The page straddles i_size. It must be zeroed out on each and every
+ * writepage invocation because it may be mmapped. "A file is mapped
+ * in multiples of the page size. For a file that is not a multiple of
+ * the page size, the remaining memory is zeroed when mapped, and
+ * writes to that region are not written out to the file."
+ */
+ if (len < PAGE_CACHE_SIZE)
+ zero_user_segment(page, len, PAGE_CACHE_SIZE);
+ /*
* In the first loop we prepare and mark buffers to submit. We have to
* mark all buffers in the page before submitting so that
* end_page_writeback() cannot be called from ext4_bio_end_io() when IO
@@ -428,19 +443,6 @@
do {
block_start = bh_offset(bh);
if (block_start >= len) {
- /*
- * Comments copied from block_write_full_page:
- *
- * The page straddles i_size. It must be zeroed out on
- * each and every writepage invocation because it may
- * be mmapped. "A file is mapped in multiples of the
- * page size. For a file that is not a multiple of
- * the page size, the remaining memory is zeroed when
- * mapped, and writes to that region are not written
- * out to the file."
- */
- zero_user_segment(page, block_start,
- block_start + blocksize);
clear_buffer_dirty(bh);
set_buffer_uptodate(bh);
continue;
diff --git a/fs/ext4/resize.c b/fs/ext4/resize.c
index 08b3c11..bb0e80f 100644
--- a/fs/ext4/resize.c
+++ b/fs/ext4/resize.c
@@ -348,6 +348,7 @@
bh = sb_getblk(sb, blk);
if (unlikely(!bh))
return ERR_PTR(-ENOMEM);
+ BUFFER_TRACE(bh, "get_write_access");
if ((err = ext4_journal_get_write_access(handle, bh))) {
brelse(bh);
bh = ERR_PTR(err);
@@ -426,6 +427,7 @@
if (unlikely(!bh))
return -ENOMEM;
+ BUFFER_TRACE(bh, "get_write_access");
err = ext4_journal_get_write_access(handle, bh);
if (err)
return err;
@@ -518,6 +520,7 @@
goto out;
}
+ BUFFER_TRACE(gdb, "get_write_access");
err = ext4_journal_get_write_access(handle, gdb);
if (err) {
brelse(gdb);
@@ -790,14 +793,17 @@
goto exit_dind;
}
+ BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get_write_access");
err = ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh);
if (unlikely(err))
goto exit_dind;
+ BUFFER_TRACE(gdb_bh, "get_write_access");
err = ext4_journal_get_write_access(handle, gdb_bh);
if (unlikely(err))
goto exit_dind;
+ BUFFER_TRACE(dind, "get_write_access");
err = ext4_journal_get_write_access(handle, dind);
if (unlikely(err))
ext4_std_error(sb, err);
@@ -902,6 +908,7 @@
EXT4_SB(sb)->s_group_desc = n_group_desc;
EXT4_SB(sb)->s_gdb_count++;
ext4_kvfree(o_group_desc);
+ BUFFER_TRACE(gdb_bh, "get_write_access");
err = ext4_journal_get_write_access(handle, gdb_bh);
if (unlikely(err))
brelse(gdb_bh);
@@ -977,6 +984,7 @@
}
for (i = 0; i < reserved_gdb; i++) {
+ BUFFER_TRACE(primary[i], "get_write_access");
if ((err = ext4_journal_get_write_access(handle, primary[i])))
goto exit_bh;
}
@@ -1084,6 +1092,7 @@
ext4_debug("update metadata backup %llu(+%llu)\n",
backup_block, backup_block -
ext4_group_first_block_no(sb, group));
+ BUFFER_TRACE(bh, "get_write_access");
if ((err = ext4_journal_get_write_access(handle, bh)))
break;
lock_buffer(bh);
@@ -1163,6 +1172,7 @@
*/
if (gdb_off) {
gdb_bh = sbi->s_group_desc[gdb_num];
+ BUFFER_TRACE(gdb_bh, "get_write_access");
err = ext4_journal_get_write_access(handle, gdb_bh);
if (!err && reserved_gdb && ext4_bg_num_gdb(sb, group))
@@ -1433,6 +1443,7 @@
goto exit;
}
+ BUFFER_TRACE(sbi->s_sbh, "get_write_access");
err = ext4_journal_get_write_access(handle, sbi->s_sbh);
if (err)
goto exit_journal;
@@ -1645,6 +1656,7 @@
return err;
}
+ BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get_write_access");
err = ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh);
if (err) {
ext4_warning(sb, "error %d on journal write access", err);
@@ -1804,6 +1816,7 @@
if (IS_ERR(handle))
return PTR_ERR(handle);
+ BUFFER_TRACE(sbi->s_sbh, "get_write_access");
err = ext4_journal_get_write_access(handle, sbi->s_sbh);
if (err)
goto errout;
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 6f9e6fa..b9b9aab 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -138,8 +138,8 @@
return cpu_to_le32(csum);
}
-int ext4_superblock_csum_verify(struct super_block *sb,
- struct ext4_super_block *es)
+static int ext4_superblock_csum_verify(struct super_block *sb,
+ struct ext4_super_block *es)
{
if (!EXT4_HAS_RO_COMPAT_FEATURE(sb,
EXT4_FEATURE_RO_COMPAT_METADATA_CSUM))
@@ -879,6 +879,7 @@
return NULL;
ei->vfs_inode.i_version = 1;
+ spin_lock_init(&ei->i_raw_lock);
INIT_LIST_HEAD(&ei->i_prealloc_list);
spin_lock_init(&ei->i_prealloc_lock);
ext4_es_init_tree(&ei->i_es_tree);
@@ -1903,7 +1904,7 @@
if (!(sbi->s_mount_state & EXT4_VALID_FS))
ext4_msg(sb, KERN_WARNING, "warning: mounting unchecked fs, "
"running e2fsck is recommended");
- else if ((sbi->s_mount_state & EXT4_ERROR_FS))
+ else if (sbi->s_mount_state & EXT4_ERROR_FS)
ext4_msg(sb, KERN_WARNING,
"warning: mounting fs with errors, "
"running e2fsck is recommended");
@@ -2404,6 +2405,16 @@
if (ext4_bg_has_super(sb, bg))
has_super = 1;
+ /*
+ * If we have a meta_bg fs with 1k blocks, group 0's GDT is at
+ * block 2, not 1. If s_first_data_block == 0 (bigalloc is enabled
+ * on modern mke2fs or blksize > 1k on older mke2fs) then we must
+ * compensate.
+ */
+ if (sb->s_blocksize == 1024 && nr == 0 &&
+ le32_to_cpu(EXT4_SB(sb)->s_es->s_first_data_block) == 0)
+ has_super++;
+
return (has_super + ext4_group_first_block_no(sb, bg));
}
@@ -3337,7 +3348,7 @@
* By default we reserve 2% or 4096 clusters, whichever is smaller.
* This should cover the situations where we can not afford to run
* out of space like for example punch hole, or converting
- * uninitialized extents in delalloc path. In most cases such
+ * unwritten extents in delalloc path. In most cases such
* allocation would require 1, or 2 blocks, higher numbers are
* very rare.
*/
@@ -5370,6 +5381,7 @@
bh = ext4_bread(handle, inode, blk, 1, &err);
if (!bh)
goto out;
+ BUFFER_TRACE(bh, "get write access");
err = ext4_journal_get_write_access(handle, bh);
if (err) {
brelse(bh);
diff --git a/fs/ext4/xattr.c b/fs/ext4/xattr.c
index 4eec399..e738733 100644
--- a/fs/ext4/xattr.c
+++ b/fs/ext4/xattr.c
@@ -369,6 +369,9 @@
{
int error;
+ if (strlen(name) > 255)
+ return -ERANGE;
+
down_read(&EXT4_I(inode)->xattr_sem);
error = ext4_xattr_ibody_get(inode, name_index, name, buffer,
buffer_size);
@@ -513,6 +516,7 @@
if (EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_EXT_ATTR))
return;
+ BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get_write_access");
if (ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh) == 0) {
EXT4_SET_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_EXT_ATTR);
ext4_handle_dirty_super(handle, sb);
@@ -532,6 +536,7 @@
struct mb_cache *ext4_mb_cache = EXT4_GET_MB_CACHE(inode);
ce = mb_cache_entry_get(ext4_mb_cache, bh->b_bdev, bh->b_blocknr);
+ BUFFER_TRACE(bh, "get_write_access");
error = ext4_journal_get_write_access(handle, bh);
if (error)
goto out;
@@ -774,6 +779,7 @@
if (s->base) {
ce = mb_cache_entry_get(ext4_mb_cache, bs->bh->b_bdev,
bs->bh->b_blocknr);
+ BUFFER_TRACE(bs->bh, "get_write_access");
error = ext4_journal_get_write_access(handle, bs->bh);
if (error)
goto cleanup;
@@ -859,6 +865,7 @@
EXT4_C2B(EXT4_SB(sb), 1));
if (error)
goto cleanup;
+ BUFFER_TRACE(new_bh, "get_write_access");
error = ext4_journal_get_write_access(handle,
new_bh);
if (error)
@@ -896,7 +903,7 @@
* take i_data_sem because we will test
* i_delalloc_reserved_flag in ext4_mb_new_blocks
*/
- down_read((&EXT4_I(inode)->i_data_sem));
+ down_read(&EXT4_I(inode)->i_data_sem);
block = ext4_new_meta_blocks(handle, inode, goal, 0,
NULL, &error);
up_read((&EXT4_I(inode)->i_data_sem));
diff --git a/fs/f2fs/acl.c b/fs/f2fs/acl.c
index e93e4ec..dbe2141 100644
--- a/fs/f2fs/acl.c
+++ b/fs/f2fs/acl.c
@@ -240,7 +240,7 @@
}
}
- error = f2fs_setxattr(inode, name_index, "", value, size, ipage);
+ error = f2fs_setxattr(inode, name_index, "", value, size, ipage, 0);
kfree(value);
if (!error)
diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
index c405b8f..0b4710c 100644
--- a/fs/f2fs/checkpoint.c
+++ b/fs/f2fs/checkpoint.c
@@ -33,12 +33,12 @@
struct address_space *mapping = META_MAPPING(sbi);
struct page *page = NULL;
repeat:
- page = grab_cache_page_write_begin(mapping, index, AOP_FLAG_NOFS);
+ page = grab_cache_page(mapping, index);
if (!page) {
cond_resched();
goto repeat;
}
-
+ f2fs_wait_on_page_writeback(page, META);
SetPageUptodate(page);
return page;
}
@@ -72,7 +72,7 @@
return page;
}
-inline int get_max_meta_blks(struct f2fs_sb_info *sbi, int type)
+static inline int get_max_meta_blks(struct f2fs_sb_info *sbi, int type)
{
switch (type) {
case META_NAT:
@@ -154,6 +154,8 @@
struct inode *inode = page->mapping->host;
struct f2fs_sb_info *sbi = F2FS_SB(inode->i_sb);
+ trace_f2fs_writepage(page, META);
+
if (unlikely(sbi->por_doing))
goto redirty_out;
if (wbc->for_reclaim)
@@ -171,10 +173,7 @@
return 0;
redirty_out:
- dec_page_count(sbi, F2FS_DIRTY_META);
- wbc->pages_skipped++;
- account_page_redirty(page);
- set_page_dirty(page);
+ redirty_page_for_writepage(wbc, page);
return AOP_WRITEPAGE_ACTIVATE;
}
@@ -184,6 +183,8 @@
struct f2fs_sb_info *sbi = F2FS_SB(mapping->host->i_sb);
long diff, written;
+ trace_f2fs_writepages(mapping->host, wbc, META);
+
/* collect a number of dirty meta pages and write together */
if (wbc->for_kupdate ||
get_pages(sbi, F2FS_DIRTY_META) < nr_pages_to_skip(sbi, META))
@@ -367,7 +368,9 @@
return;
sbi->por_doing = true;
- start_blk = __start_cp_addr(sbi) + 1;
+
+ start_blk = __start_cp_addr(sbi) + 1 +
+ le32_to_cpu(F2FS_RAW_SUPER(sbi)->cp_payload);
orphan_blkaddr = __start_sum_addr(sbi) - 1;
ra_meta_pages(sbi, start_blk, orphan_blkaddr, META_CP);
@@ -508,8 +511,11 @@
unsigned long blk_size = sbi->blocksize;
unsigned long long cp1_version = 0, cp2_version = 0;
unsigned long long cp_start_blk_no;
+ unsigned int cp_blks = 1 + le32_to_cpu(F2FS_RAW_SUPER(sbi)->cp_payload);
+ block_t cp_blk_no;
+ int i;
- sbi->ckpt = kzalloc(blk_size, GFP_KERNEL);
+ sbi->ckpt = kzalloc(cp_blks * blk_size, GFP_KERNEL);
if (!sbi->ckpt)
return -ENOMEM;
/*
@@ -540,6 +546,23 @@
cp_block = (struct f2fs_checkpoint *)page_address(cur_page);
memcpy(sbi->ckpt, cp_block, blk_size);
+ if (cp_blks <= 1)
+ goto done;
+
+ cp_blk_no = le32_to_cpu(fsb->cp_blkaddr);
+ if (cur_page == cp2)
+ cp_blk_no += 1 << le32_to_cpu(fsb->log_blocks_per_seg);
+
+ for (i = 1; i < cp_blks; i++) {
+ void *sit_bitmap_ptr;
+ unsigned char *ckpt = (unsigned char *)sbi->ckpt;
+
+ cur_page = get_meta_page(sbi, cp_blk_no + i);
+ sit_bitmap_ptr = page_address(cur_page);
+ memcpy(ckpt + i * blk_size, sit_bitmap_ptr, blk_size);
+ f2fs_put_page(cur_page, 1);
+ }
+done:
f2fs_put_page(cp1, 1);
f2fs_put_page(cp2, 1);
return 0;
@@ -552,14 +575,13 @@
static int __add_dirty_inode(struct inode *inode, struct dir_inode_entry *new)
{
struct f2fs_sb_info *sbi = F2FS_SB(inode->i_sb);
- struct list_head *head = &sbi->dir_inode_list;
- struct dir_inode_entry *entry;
- list_for_each_entry(entry, head, list)
- if (unlikely(entry->inode == inode))
- return -EEXIST;
+ if (is_inode_flag_set(F2FS_I(inode), FI_DIRTY_DIR))
+ return -EEXIST;
- list_add_tail(&new->list, head);
+ set_inode_flag(F2FS_I(inode), FI_DIRTY_DIR);
+ F2FS_I(inode)->dirty_dir = new;
+ list_add_tail(&new->list, &sbi->dir_inode_list);
stat_inc_dirty_dir(sbi);
return 0;
}
@@ -608,31 +630,26 @@
void remove_dirty_dir_inode(struct inode *inode)
{
struct f2fs_sb_info *sbi = F2FS_SB(inode->i_sb);
- struct list_head *head;
struct dir_inode_entry *entry;
if (!S_ISDIR(inode->i_mode))
return;
spin_lock(&sbi->dir_inode_lock);
- if (get_dirty_dents(inode)) {
+ if (get_dirty_dents(inode) ||
+ !is_inode_flag_set(F2FS_I(inode), FI_DIRTY_DIR)) {
spin_unlock(&sbi->dir_inode_lock);
return;
}
- head = &sbi->dir_inode_list;
- list_for_each_entry(entry, head, list) {
- if (entry->inode == inode) {
- list_del(&entry->list);
- stat_dec_dirty_dir(sbi);
- spin_unlock(&sbi->dir_inode_lock);
- kmem_cache_free(inode_entry_slab, entry);
- goto done;
- }
- }
+ entry = F2FS_I(inode)->dirty_dir;
+ list_del(&entry->list);
+ F2FS_I(inode)->dirty_dir = NULL;
+ clear_inode_flag(F2FS_I(inode), FI_DIRTY_DIR);
+ stat_dec_dirty_dir(sbi);
spin_unlock(&sbi->dir_inode_lock);
+ kmem_cache_free(inode_entry_slab, entry);
-done:
/* Only from the recovery routine */
if (is_inode_flag_set(F2FS_I(inode), FI_DELAY_IPUT)) {
clear_inode_flag(F2FS_I(inode), FI_DELAY_IPUT);
@@ -640,26 +657,6 @@
}
}
-struct inode *check_dirty_dir_inode(struct f2fs_sb_info *sbi, nid_t ino)
-{
-
- struct list_head *head;
- struct inode *inode = NULL;
- struct dir_inode_entry *entry;
-
- spin_lock(&sbi->dir_inode_lock);
-
- head = &sbi->dir_inode_list;
- list_for_each_entry(entry, head, list) {
- if (entry->inode->i_ino == ino) {
- inode = entry->inode;
- break;
- }
- }
- spin_unlock(&sbi->dir_inode_lock);
- return inode;
-}
-
void sync_dirty_dir_inodes(struct f2fs_sb_info *sbi)
{
struct list_head *head;
@@ -758,6 +755,13 @@
__u32 crc32 = 0;
void *kaddr;
int i;
+ int cp_payload_blks = le32_to_cpu(F2FS_RAW_SUPER(sbi)->cp_payload);
+
+ /*
+ * This avoids to conduct wrong roll-forward operations and uses
+ * metapages, so should be called prior to sync_meta_pages below.
+ */
+ discard_next_dnode(sbi);
/* Flush all the NAT/SIT pages */
while (get_pages(sbi, F2FS_DIRTY_META))
@@ -802,16 +806,19 @@
orphan_blocks = (sbi->n_orphans + F2FS_ORPHANS_PER_BLOCK - 1)
/ F2FS_ORPHANS_PER_BLOCK;
- ckpt->cp_pack_start_sum = cpu_to_le32(1 + orphan_blocks);
+ ckpt->cp_pack_start_sum = cpu_to_le32(1 + cp_payload_blks +
+ orphan_blocks);
if (is_umount) {
set_ckpt_flags(ckpt, CP_UMOUNT_FLAG);
ckpt->cp_pack_total_block_count = cpu_to_le32(2 +
- data_sum_blocks + orphan_blocks + NR_CURSEG_NODE_TYPE);
+ cp_payload_blks + data_sum_blocks +
+ orphan_blocks + NR_CURSEG_NODE_TYPE);
} else {
clear_ckpt_flags(ckpt, CP_UMOUNT_FLAG);
ckpt->cp_pack_total_block_count = cpu_to_le32(2 +
- data_sum_blocks + orphan_blocks);
+ cp_payload_blks + data_sum_blocks +
+ orphan_blocks);
}
if (sbi->n_orphans)
@@ -837,6 +844,15 @@
set_page_dirty(cp_page);
f2fs_put_page(cp_page, 1);
+ for (i = 1; i < 1 + cp_payload_blks; i++) {
+ cp_page = grab_meta_page(sbi, start_blk++);
+ kaddr = page_address(cp_page);
+ memcpy(kaddr, (char *)ckpt + i * F2FS_BLKSIZE,
+ (1 << sbi->log_blocksize));
+ set_page_dirty(cp_page);
+ f2fs_put_page(cp_page, 1);
+ }
+
if (sbi->n_orphans) {
write_orphan_inodes(sbi, start_blk);
start_blk += orphan_blocks;
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 45abd60e..c1fb6dd 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -417,7 +417,7 @@
if (unlikely(dn.data_blkaddr == NEW_ADDR))
return ERR_PTR(-EINVAL);
- page = grab_cache_page_write_begin(mapping, index, AOP_FLAG_NOFS);
+ page = grab_cache_page(mapping, index);
if (!page)
return ERR_PTR(-ENOMEM);
@@ -455,7 +455,7 @@
int err;
repeat:
- page = grab_cache_page_write_begin(mapping, index, AOP_FLAG_NOFS);
+ page = grab_cache_page(mapping, index);
if (!page)
return ERR_PTR(-ENOMEM);
@@ -652,8 +652,7 @@
goto put_out;
}
- end_offset = IS_INODE(dn.node_page) ?
- ADDRS_PER_INODE(F2FS_I(inode)) : ADDRS_PER_BLOCK;
+ end_offset = ADDRS_PER_PAGE(dn.node_page, F2FS_I(inode));
bh_result->b_size = (((size_t)1) << blkbits);
dn.ofs_in_node++;
pgofs++;
@@ -675,8 +674,7 @@
if (dn.data_blkaddr == NEW_ADDR)
goto put_out;
- end_offset = IS_INODE(dn.node_page) ?
- ADDRS_PER_INODE(F2FS_I(inode)) : ADDRS_PER_BLOCK;
+ end_offset = ADDRS_PER_PAGE(dn.node_page, F2FS_I(inode));
}
if (maxblocks > (bh_result->b_size >> blkbits)) {
@@ -710,11 +708,19 @@
return err;
}
+int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
+ u64 start, u64 len)
+{
+ return generic_block_fiemap(inode, fieinfo, start, len, get_data_block);
+}
+
static int f2fs_read_data_page(struct file *file, struct page *page)
{
struct inode *inode = page->mapping->host;
int ret;
+ trace_f2fs_readpage(page, DATA);
+
/* If the file has inline data, try to read it directlly */
if (f2fs_has_inline_data(inode))
ret = f2fs_read_inline_data(inode, page);
@@ -790,6 +796,8 @@
.rw = (wbc->sync_mode == WB_SYNC_ALL) ? WRITE_SYNC : WRITE,
};
+ trace_f2fs_writepage(page, DATA);
+
if (page->index < end_index)
goto write;
@@ -798,10 +806,8 @@
* this page does not have to be written to disk.
*/
offset = i_size & (PAGE_CACHE_SIZE - 1);
- if ((page->index >= end_index + 1) || !offset) {
- inode_dec_dirty_dents(inode);
+ if ((page->index >= end_index + 1) || !offset)
goto out;
- }
zero_user_segment(page, offset, PAGE_CACHE_SIZE);
write:
@@ -810,7 +816,6 @@
/* Dentry blocks are controlled by checkpoint */
if (S_ISDIR(inode->i_mode)) {
- inode_dec_dirty_dents(inode);
err = do_write_data_page(page, &fio);
goto done;
}
@@ -832,15 +837,16 @@
clear_cold_data(page);
out:
+ inode_dec_dirty_dents(inode);
unlock_page(page);
if (need_balance_fs)
f2fs_balance_fs(sbi);
+ if (wbc->for_reclaim)
+ f2fs_submit_merged_bio(sbi, DATA, WRITE);
return 0;
redirty_out:
- wbc->pages_skipped++;
- account_page_redirty(page);
- set_page_dirty(page);
+ redirty_page_for_writepage(wbc, page);
return AOP_WRITEPAGE_ACTIVATE;
}
@@ -862,12 +868,15 @@
int ret;
long diff;
+ trace_f2fs_writepages(mapping->host, wbc, DATA);
+
/* deal with chardevs and other special file */
if (!mapping->a_ops->writepage)
return 0;
if (S_ISDIR(inode->i_mode) && wbc->sync_mode == WB_SYNC_NONE &&
- get_dirty_dents(inode) < nr_pages_to_skip(sbi, DATA))
+ get_dirty_dents(inode) < nr_pages_to_skip(sbi, DATA) &&
+ available_free_memory(sbi, DIRTY_DENTS))
goto skip_write;
diff = nr_pages_to_write(sbi, DATA, wbc);
@@ -903,6 +912,8 @@
struct dnode_of_data dn;
int err = 0;
+ trace_f2fs_write_begin(inode, pos, len, flags);
+
f2fs_balance_fs(sbi);
repeat:
err = f2fs_convert_inline_data(inode, pos + len);
@@ -912,6 +923,10 @@
page = grab_cache_page_write_begin(mapping, index, flags);
if (!page)
return -ENOMEM;
+
+ /* to avoid latency during memory pressure */
+ unlock_page(page);
+
*pagep = page;
if (f2fs_has_inline_data(inode) && (pos + len) <= MAX_INLINE_DATA)
@@ -923,10 +938,18 @@
f2fs_unlock_op(sbi);
if (err) {
- f2fs_put_page(page, 1);
+ f2fs_put_page(page, 0);
return err;
}
inline_data:
+ lock_page(page);
+ if (unlikely(page->mapping != mapping)) {
+ f2fs_put_page(page, 1);
+ goto repeat;
+ }
+
+ f2fs_wait_on_page_writeback(page, DATA);
+
if ((len == PAGE_CACHE_SIZE) || PageUptodate(page))
return 0;
@@ -978,6 +1001,8 @@
{
struct inode *inode = page->mapping->host;
+ trace_f2fs_write_end(inode, pos, len, copied);
+
SetPageUptodate(page);
set_page_dirty(page);
@@ -1022,6 +1047,9 @@
if (check_direct_IO(inode, rw, iov, offset, nr_segs))
return 0;
+ /* clear fsync mark to recover these blocks */
+ fsync_mark_clear(F2FS_SB(inode->i_sb), inode->i_ino);
+
return blockdev_direct_IO(rw, iocb, inode, iov, offset, nr_segs,
get_data_block);
}
@@ -1061,6 +1089,11 @@
static sector_t f2fs_bmap(struct address_space *mapping, sector_t block)
{
+ struct inode *inode = mapping->host;
+
+ if (f2fs_has_inline_data(inode))
+ return 0;
+
return generic_block_bmap(mapping, block, get_data_block);
}
diff --git a/fs/f2fs/dir.c b/fs/f2fs/dir.c
index 972fd0e..966acb0 100644
--- a/fs/f2fs/dir.c
+++ b/fs/f2fs/dir.c
@@ -23,10 +23,10 @@
static unsigned int dir_buckets(unsigned int level, int dir_level)
{
- if (level < MAX_DIR_HASH_DEPTH / 2)
+ if (level + dir_level < MAX_DIR_HASH_DEPTH / 2)
return 1 << (level + dir_level);
else
- return 1 << ((MAX_DIR_HASH_DEPTH / 2 + dir_level) - 1);
+ return MAX_DIR_BUCKETS;
}
static unsigned int bucket_blocks(unsigned int level)
@@ -268,6 +268,8 @@
{
struct f2fs_inode *ri;
+ f2fs_wait_on_page_writeback(ipage, NODE);
+
/* copy name info. to this inode page */
ri = F2FS_INODE(ipage);
ri->i_namelen = cpu_to_le32(name->len);
@@ -637,11 +639,17 @@
struct f2fs_dentry_block *dentry_blk = NULL;
struct f2fs_dir_entry *de = NULL;
struct page *dentry_page = NULL;
+ struct file_ra_state *ra = &file->f_ra;
unsigned int n = ((unsigned long)ctx->pos / NR_DENTRY_IN_BLOCK);
unsigned char d_type = DT_UNKNOWN;
bit_pos = ((unsigned long)ctx->pos % NR_DENTRY_IN_BLOCK);
+ /* readahead for multi pages of dir */
+ if (npages - n > 1 && !ra_has_index(ra, n))
+ page_cache_sync_readahead(inode->i_mapping, ra, file, n,
+ min(npages - n, (pgoff_t)MAX_DIR_RA_PAGES));
+
for (; n < npages; n++) {
dentry_page = get_lock_data_page(inode, n);
if (IS_ERR(dentry_page))
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 2ecac83..e51c732 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -182,6 +182,8 @@
#define F2FS_LINK_MAX 32000 /* maximum link count per file */
+#define MAX_DIR_RA_PAGES 4 /* maximum ra pages of dir */
+
/* for in-memory extent cache entry */
#define F2FS_MIN_EXTENT_LEN 16 /* minimum extent length */
@@ -218,6 +220,7 @@
nid_t i_xattr_nid; /* node id that contains xattrs */
unsigned long long xattr_ver; /* cp version of xattr modification */
struct extent_info ext; /* in-memory extent cache entry */
+ struct dir_inode_entry *dirty_dir; /* the pointer of dirty dir */
};
static inline void get_extent_info(struct extent_info *ext,
@@ -243,6 +246,7 @@
struct f2fs_nm_info {
block_t nat_blkaddr; /* base disk address of NAT */
nid_t max_nid; /* maximum possible node ids */
+ nid_t available_nids; /* maximum available node ids */
nid_t next_scan_nid; /* the next nid to be scanned */
unsigned int ram_thresh; /* control the memory footprint */
@@ -323,6 +327,15 @@
int ret;
};
+struct flush_cmd_control {
+ struct task_struct *f2fs_issue_flush; /* flush thread */
+ wait_queue_head_t flush_wait_queue; /* waiting queue for wake-up */
+ struct flush_cmd *issue_list; /* list for command issue */
+ struct flush_cmd *dispatch_list; /* list for command dispatch */
+ spinlock_t issue_lock; /* for issue list lock */
+ struct flush_cmd *issue_tail; /* list tail of issue list */
+};
+
struct f2fs_sm_info {
struct sit_info *sit_info; /* whole segment information */
struct free_segmap_info *free_info; /* free segment information */
@@ -353,12 +366,8 @@
unsigned int min_ipu_util; /* in-place-update threshold */
/* for flush command control */
- struct task_struct *f2fs_issue_flush; /* flush thread */
- wait_queue_head_t flush_wait_queue; /* waiting queue for wake-up */
- struct flush_cmd *issue_list; /* list for command issue */
- struct flush_cmd *dispatch_list; /* list for command dispatch */
- spinlock_t issue_lock; /* for issue list lock */
- struct flush_cmd *issue_tail; /* list tail of issue list */
+ struct flush_cmd_control *cmd_control_info;
+
};
/*
@@ -755,9 +764,18 @@
static inline void *__bitmap_ptr(struct f2fs_sb_info *sbi, int flag)
{
struct f2fs_checkpoint *ckpt = F2FS_CKPT(sbi);
- int offset = (flag == NAT_BITMAP) ?
+ int offset;
+
+ if (le32_to_cpu(F2FS_RAW_SUPER(sbi)->cp_payload) > 0) {
+ if (flag == NAT_BITMAP)
+ return &ckpt->sit_nat_version_bitmap;
+ else
+ return ((unsigned char *)ckpt + F2FS_BLKSIZE);
+ } else {
+ offset = (flag == NAT_BITMAP) ?
le32_to_cpu(ckpt->sit_ver_bitmap_bytesize) : 0;
- return &ckpt->sit_nat_version_bitmap + offset;
+ return &ckpt->sit_nat_version_bitmap + offset;
+ }
}
static inline block_t __start_cp_addr(struct f2fs_sb_info *sbi)
@@ -958,6 +976,7 @@
enum {
FI_NEW_INODE, /* indicate newly allocated inode */
FI_DIRTY_INODE, /* indicate inode is dirty or not */
+ FI_DIRTY_DIR, /* indicate directory has dirty pages */
FI_INC_LINK, /* need to increment i_nlink */
FI_ACL_MODE, /* indicate acl mode */
FI_NO_ALLOC, /* should not allocate any blocks */
@@ -1071,6 +1090,12 @@
((is_inode_flag_set(F2FS_I(i), FI_ACL_MODE)) ? \
(F2FS_I(i)->i_acl_mode) : ((i)->i_mode))
+/* get offset of first page in next direct node */
+#define PGOFS_OF_NEXT_DNODE(pgofs, fi) \
+ ((pgofs < ADDRS_PER_INODE(fi)) ? ADDRS_PER_INODE(fi) : \
+ (pgofs - ADDRS_PER_INODE(fi) + ADDRS_PER_BLOCK) / \
+ ADDRS_PER_BLOCK * ADDRS_PER_BLOCK + ADDRS_PER_INODE(fi))
+
/*
* file.c
*/
@@ -1140,8 +1165,10 @@
struct dnode_of_data;
struct node_info;
+bool available_free_memory(struct f2fs_sb_info *, int);
int is_checkpointed_node(struct f2fs_sb_info *, nid_t);
bool fsync_mark_done(struct f2fs_sb_info *, nid_t);
+void fsync_mark_clear(struct f2fs_sb_info *, nid_t);
void get_node_info(struct f2fs_sb_info *, nid_t, struct node_info *);
int get_dnode_of_data(struct dnode_of_data *, pgoff_t, int);
int truncate_inode_blocks(struct inode *, pgoff_t);
@@ -1176,9 +1203,12 @@
void f2fs_balance_fs(struct f2fs_sb_info *);
void f2fs_balance_fs_bg(struct f2fs_sb_info *);
int f2fs_issue_flush(struct f2fs_sb_info *);
+int create_flush_cmd_control(struct f2fs_sb_info *);
+void destroy_flush_cmd_control(struct f2fs_sb_info *);
void invalidate_blocks(struct f2fs_sb_info *, block_t);
void refresh_sit_entry(struct f2fs_sb_info *, block_t, block_t);
void clear_prefree_segments(struct f2fs_sb_info *);
+void discard_next_dnode(struct f2fs_sb_info *);
int npages_for_summary_flush(struct f2fs_sb_info *);
void allocate_new_segments(struct f2fs_sb_info *);
struct page *get_sum_page(struct f2fs_sb_info *, unsigned int);
@@ -1221,7 +1251,6 @@
void set_dirty_dir_page(struct inode *, struct page *);
void add_dirty_dir_inode(struct inode *);
void remove_dirty_dir_inode(struct inode *);
-struct inode *check_dirty_dir_inode(struct f2fs_sb_info *, nid_t);
void sync_dirty_dir_inodes(struct f2fs_sb_info *);
void write_checkpoint(struct f2fs_sb_info *, bool);
void init_orphan_info(struct f2fs_sb_info *);
@@ -1242,6 +1271,7 @@
struct page *get_lock_data_page(struct inode *, pgoff_t);
struct page *get_new_data_page(struct inode *, struct page *, pgoff_t, bool);
int do_write_data_page(struct page *, struct f2fs_io_info *);
+int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *, u64, u64);
/*
* gc.c
@@ -1391,5 +1421,6 @@
int f2fs_read_inline_data(struct inode *, struct page *);
int f2fs_convert_inline_data(struct inode *, pgoff_t);
int f2fs_write_inline_data(struct inode *, struct page *, unsigned int);
+void truncate_inline_data(struct inode *, u64);
int recover_inline_data(struct inode *, struct page *);
#endif
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 60e7d54..9c49c59 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -19,6 +19,7 @@
#include <linux/compat.h>
#include <linux/uaccess.h>
#include <linux/mount.h>
+#include <linux/pagevec.h>
#include "f2fs.h"
#include "node.h"
@@ -194,6 +195,132 @@
return ret;
}
+static pgoff_t __get_first_dirty_index(struct address_space *mapping,
+ pgoff_t pgofs, int whence)
+{
+ struct pagevec pvec;
+ int nr_pages;
+
+ if (whence != SEEK_DATA)
+ return 0;
+
+ /* find first dirty page index */
+ pagevec_init(&pvec, 0);
+ nr_pages = pagevec_lookup_tag(&pvec, mapping, &pgofs, PAGECACHE_TAG_DIRTY, 1);
+ pgofs = nr_pages ? pvec.pages[0]->index: LONG_MAX;
+ pagevec_release(&pvec);
+ return pgofs;
+}
+
+static bool __found_offset(block_t blkaddr, pgoff_t dirty, pgoff_t pgofs,
+ int whence)
+{
+ switch (whence) {
+ case SEEK_DATA:
+ if ((blkaddr == NEW_ADDR && dirty == pgofs) ||
+ (blkaddr != NEW_ADDR && blkaddr != NULL_ADDR))
+ return true;
+ break;
+ case SEEK_HOLE:
+ if (blkaddr == NULL_ADDR)
+ return true;
+ break;
+ }
+ return false;
+}
+
+static loff_t f2fs_seek_block(struct file *file, loff_t offset, int whence)
+{
+ struct inode *inode = file->f_mapping->host;
+ loff_t maxbytes = inode->i_sb->s_maxbytes;
+ struct dnode_of_data dn;
+ pgoff_t pgofs, end_offset, dirty;
+ loff_t data_ofs = offset;
+ loff_t isize;
+ int err = 0;
+
+ mutex_lock(&inode->i_mutex);
+
+ isize = i_size_read(inode);
+ if (offset >= isize)
+ goto fail;
+
+ /* handle inline data case */
+ if (f2fs_has_inline_data(inode)) {
+ if (whence == SEEK_HOLE)
+ data_ofs = isize;
+ goto found;
+ }
+
+ pgofs = (pgoff_t)(offset >> PAGE_CACHE_SHIFT);
+
+ dirty = __get_first_dirty_index(inode->i_mapping, pgofs, whence);
+
+ for (; data_ofs < isize; data_ofs = pgofs << PAGE_CACHE_SHIFT) {
+ set_new_dnode(&dn, inode, NULL, NULL, 0);
+ err = get_dnode_of_data(&dn, pgofs, LOOKUP_NODE_RA);
+ if (err && err != -ENOENT) {
+ goto fail;
+ } else if (err == -ENOENT) {
+ /* direct node is not exist */
+ if (whence == SEEK_DATA) {
+ pgofs = PGOFS_OF_NEXT_DNODE(pgofs,
+ F2FS_I(inode));
+ continue;
+ } else {
+ goto found;
+ }
+ }
+
+ end_offset = IS_INODE(dn.node_page) ?
+ ADDRS_PER_INODE(F2FS_I(inode)) : ADDRS_PER_BLOCK;
+
+ /* find data/hole in dnode block */
+ for (; dn.ofs_in_node < end_offset;
+ dn.ofs_in_node++, pgofs++,
+ data_ofs = pgofs << PAGE_CACHE_SHIFT) {
+ block_t blkaddr;
+ blkaddr = datablock_addr(dn.node_page, dn.ofs_in_node);
+
+ if (__found_offset(blkaddr, dirty, pgofs, whence)) {
+ f2fs_put_dnode(&dn);
+ goto found;
+ }
+ }
+ f2fs_put_dnode(&dn);
+ }
+
+ if (whence == SEEK_DATA)
+ goto fail;
+found:
+ if (whence == SEEK_HOLE && data_ofs > isize)
+ data_ofs = isize;
+ mutex_unlock(&inode->i_mutex);
+ return vfs_setpos(file, data_ofs, maxbytes);
+fail:
+ mutex_unlock(&inode->i_mutex);
+ return -ENXIO;
+}
+
+static loff_t f2fs_llseek(struct file *file, loff_t offset, int whence)
+{
+ struct inode *inode = file->f_mapping->host;
+ loff_t maxbytes = inode->i_sb->s_maxbytes;
+
+ switch (whence) {
+ case SEEK_SET:
+ case SEEK_CUR:
+ case SEEK_END:
+ return generic_file_llseek_size(file, offset, whence,
+ maxbytes, i_size_read(inode));
+ case SEEK_DATA:
+ case SEEK_HOLE:
+ return f2fs_seek_block(file, offset, whence);
+ }
+
+ return -EINVAL;
+}
+
static int f2fs_file_mmap(struct file *file, struct vm_area_struct *vma)
{
file_accessed(file);
@@ -242,6 +369,9 @@
unsigned offset = from & (PAGE_CACHE_SIZE - 1);
struct page *page;
+ if (f2fs_has_inline_data(inode))
+ return truncate_inline_data(inode, from);
+
if (!offset)
return;
@@ -288,10 +418,7 @@
return err;
}
- if (IS_INODE(dn.node_page))
- count = ADDRS_PER_INODE(F2FS_I(inode));
- else
- count = ADDRS_PER_BLOCK;
+ count = ADDRS_PER_PAGE(dn.node_page, F2FS_I(inode));
count -= dn.ofs_in_node;
f2fs_bug_on(count < 0);
@@ -413,6 +540,7 @@
.listxattr = f2fs_listxattr,
.removexattr = generic_removexattr,
#endif
+ .fiemap = f2fs_fiemap,
};
static void fill_zero(struct inode *inode, pgoff_t index,
@@ -555,6 +683,7 @@
i_size_read(inode) < new_size) {
i_size_write(inode, new_size);
mark_inode_dirty(inode);
+ f2fs_write_inode(inode, NULL);
}
return ret;
@@ -678,7 +807,7 @@
#endif
const struct file_operations f2fs_file_operations = {
- .llseek = generic_file_llseek,
+ .llseek = f2fs_llseek,
.read = do_sync_read,
.write = do_sync_write,
.aio_read = generic_file_aio_read,
diff --git a/fs/f2fs/inline.c b/fs/f2fs/inline.c
index 383db1f..1bba522 100644
--- a/fs/f2fs/inline.c
+++ b/fs/f2fs/inline.c
@@ -81,8 +81,10 @@
f2fs_lock_op(sbi);
ipage = get_node_page(sbi, inode->i_ino);
- if (IS_ERR(ipage))
- return PTR_ERR(ipage);
+ if (IS_ERR(ipage)) {
+ err = PTR_ERR(ipage);
+ goto out;
+ }
/*
* i_addr[0] is not used for inline data,
@@ -90,11 +92,10 @@
*/
set_new_dnode(&dn, inode, ipage, NULL, 0);
err = f2fs_reserve_block(&dn, 0);
- if (err) {
- f2fs_unlock_op(sbi);
- return err;
- }
+ if (err)
+ goto out;
+ f2fs_wait_on_page_writeback(page, DATA);
zero_user_segment(page, MAX_INLINE_DATA, PAGE_CACHE_SIZE);
/* Copy the whole inline data block */
@@ -118,6 +119,7 @@
sync_inode_page(&dn);
f2fs_put_dnode(&dn);
+out:
f2fs_unlock_op(sbi);
return err;
}
@@ -132,7 +134,7 @@
else if (to_size <= MAX_INLINE_DATA)
return 0;
- page = grab_cache_page_write_begin(inode->i_mapping, 0, AOP_FLAG_NOFS);
+ page = grab_cache_page(inode->i_mapping, 0);
if (!page)
return -ENOMEM;
@@ -155,6 +157,7 @@
return err;
ipage = dn.inode_page;
+ f2fs_wait_on_page_writeback(ipage, NODE);
zero_user_segment(ipage, INLINE_DATA_OFFSET,
INLINE_DATA_OFFSET + MAX_INLINE_DATA);
src_addr = kmap(page);
@@ -175,6 +178,26 @@
return 0;
}
+void truncate_inline_data(struct inode *inode, u64 from)
+{
+ struct f2fs_sb_info *sbi = F2FS_SB(inode->i_sb);
+ struct page *ipage;
+
+ if (from >= MAX_INLINE_DATA)
+ return;
+
+ ipage = get_node_page(sbi, inode->i_ino);
+ if (IS_ERR(ipage))
+ return;
+
+ f2fs_wait_on_page_writeback(ipage, NODE);
+
+ zero_user_segment(ipage, INLINE_DATA_OFFSET + from,
+ INLINE_DATA_OFFSET + MAX_INLINE_DATA);
+ set_page_dirty(ipage);
+ f2fs_put_page(ipage, 1);
+}
+
int recover_inline_data(struct inode *inode, struct page *npage)
{
struct f2fs_sb_info *sbi = F2FS_SB(inode->i_sb);
@@ -199,6 +222,8 @@
ipage = get_node_page(sbi, inode->i_ino);
f2fs_bug_on(IS_ERR(ipage));
+ f2fs_wait_on_page_writeback(ipage, NODE);
+
src_addr = inline_data_addr(npage);
dst_addr = inline_data_addr(ipage);
memcpy(dst_addr, src_addr, MAX_INLINE_DATA);
@@ -210,6 +235,7 @@
if (f2fs_has_inline_data(inode)) {
ipage = get_node_page(sbi, inode->i_ino);
f2fs_bug_on(IS_ERR(ipage));
+ f2fs_wait_on_page_writeback(ipage, NODE);
zero_user_segment(ipage, INLINE_DATA_OFFSET,
INLINE_DATA_OFFSET + MAX_INLINE_DATA);
clear_inode_flag(F2FS_I(inode), FI_INLINE_DATA);
diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
index ee829d3..adc622c 100644
--- a/fs/f2fs/inode.c
+++ b/fs/f2fs/inode.c
@@ -12,6 +12,7 @@
#include <linux/f2fs_fs.h>
#include <linux/buffer_head.h>
#include <linux/writeback.h>
+#include <linux/bitops.h>
#include "f2fs.h"
#include "node.h"
@@ -21,20 +22,20 @@
void f2fs_set_inode_flags(struct inode *inode)
{
unsigned int flags = F2FS_I(inode)->i_flags;
-
- inode->i_flags &= ~(S_SYNC | S_APPEND | S_IMMUTABLE |
- S_NOATIME | S_DIRSYNC);
+ unsigned int new_fl = 0;
if (flags & FS_SYNC_FL)
- inode->i_flags |= S_SYNC;
+ new_fl |= S_SYNC;
if (flags & FS_APPEND_FL)
- inode->i_flags |= S_APPEND;
+ new_fl |= S_APPEND;
if (flags & FS_IMMUTABLE_FL)
- inode->i_flags |= S_IMMUTABLE;
+ new_fl |= S_IMMUTABLE;
if (flags & FS_NOATIME_FL)
- inode->i_flags |= S_NOATIME;
+ new_fl |= S_NOATIME;
if (flags & FS_DIRSYNC_FL)
- inode->i_flags |= S_DIRSYNC;
+ new_fl |= S_DIRSYNC;
+ set_mask_bits(&inode->i_flags,
+ S_SYNC|S_APPEND|S_IMMUTABLE|S_NOATIME|S_DIRSYNC, new_fl);
}
static void __get_inode_rdev(struct inode *inode, struct f2fs_inode *ri)
@@ -294,4 +295,5 @@
sb_end_intwrite(inode->i_sb);
no_delete:
clear_inode(inode);
+ invalidate_mapping_pages(NODE_MAPPING(sbi), inode->i_ino, inode->i_ino);
}
diff --git a/fs/f2fs/namei.c b/fs/f2fs/namei.c
index a9409d1..9138c32 100644
--- a/fs/f2fs/namei.c
+++ b/fs/f2fs/namei.c
@@ -41,18 +41,9 @@
}
f2fs_unlock_op(sbi);
- inode->i_uid = current_fsuid();
-
- if (dir->i_mode & S_ISGID) {
- inode->i_gid = dir->i_gid;
- if (S_ISDIR(mode))
- mode |= S_ISGID;
- } else {
- inode->i_gid = current_fsgid();
- }
+ inode_init_owner(inode, dir, mode);
inode->i_ino = ino;
- inode->i_mode = mode;
inode->i_blocks = 0;
inode->i_mtime = inode->i_atime = inode->i_ctime = CURRENT_TIME;
inode->i_generation = sbi->s_next_generation++;
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index 57caa6e..9dfb9a0 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -26,20 +26,26 @@
static struct kmem_cache *nat_entry_slab;
static struct kmem_cache *free_nid_slab;
-static inline bool available_free_memory(struct f2fs_nm_info *nm_i, int type)
+bool available_free_memory(struct f2fs_sb_info *sbi, int type)
{
+ struct f2fs_nm_info *nm_i = NM_I(sbi);
struct sysinfo val;
unsigned long mem_size = 0;
+ bool res = false;
si_meminfo(&val);
- if (type == FREE_NIDS)
- mem_size = nm_i->fcnt * sizeof(struct free_nid);
- else if (type == NAT_ENTRIES)
- mem_size += nm_i->nat_cnt * sizeof(struct nat_entry);
- mem_size >>= 12;
-
- /* give 50:50 memory for free nids and nat caches respectively */
- return (mem_size < ((val.totalram * nm_i->ram_thresh) >> 11));
+ /* give 25%, 25%, 50% memory for each components respectively */
+ if (type == FREE_NIDS) {
+ mem_size = (nm_i->fcnt * sizeof(struct free_nid)) >> 12;
+ res = mem_size < ((val.totalram * nm_i->ram_thresh / 100) >> 2);
+ } else if (type == NAT_ENTRIES) {
+ mem_size = (nm_i->nat_cnt * sizeof(struct nat_entry)) >> 12;
+ res = mem_size < ((val.totalram * nm_i->ram_thresh / 100) >> 2);
+ } else if (type == DIRTY_DENTS) {
+ mem_size = get_pages(sbi, F2FS_DIRTY_DENTS);
+ res = mem_size < ((val.totalram * nm_i->ram_thresh / 100) >> 1);
+ }
+ return res;
}
static void clear_node_page_dirty(struct page *page)
@@ -147,6 +153,18 @@
return fsync_done;
}
+void fsync_mark_clear(struct f2fs_sb_info *sbi, nid_t nid)
+{
+ struct f2fs_nm_info *nm_i = NM_I(sbi);
+ struct nat_entry *e;
+
+ write_lock(&nm_i->nat_tree_lock);
+ e = __lookup_nat_cache(nm_i, nid);
+ if (e)
+ e->fsync_done = false;
+ write_unlock(&nm_i->nat_tree_lock);
+}
+
static struct nat_entry *grab_nat_entry(struct f2fs_nm_info *nm_i, nid_t nid)
{
struct nat_entry *new;
@@ -179,9 +197,7 @@
write_unlock(&nm_i->nat_tree_lock);
goto retry;
}
- nat_set_blkaddr(e, le32_to_cpu(ne->block_addr));
- nat_set_ino(e, le32_to_cpu(ne->ino));
- nat_set_version(e, ne->version);
+ node_info_from_raw_nat(&e->ni, ne);
}
write_unlock(&nm_i->nat_tree_lock);
}
@@ -243,7 +259,7 @@
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
- if (available_free_memory(nm_i, NAT_ENTRIES))
+ if (available_free_memory(sbi, NAT_ENTRIES))
return 0;
write_lock(&nm_i->nat_tree_lock);
@@ -849,8 +865,7 @@
if (unlikely(is_inode_flag_set(F2FS_I(dn->inode), FI_NO_ALLOC)))
return ERR_PTR(-EPERM);
- page = grab_cache_page_write_begin(NODE_MAPPING(sbi),
- dn->nid, AOP_FLAG_NOFS);
+ page = grab_cache_page(NODE_MAPPING(sbi), dn->nid);
if (!page)
return ERR_PTR(-ENOMEM);
@@ -867,6 +882,7 @@
new_ni.ino = dn->inode->i_ino;
set_node_addr(sbi, &new_ni, NEW_ADDR, false);
+ f2fs_wait_on_page_writeback(page, NODE);
fill_node_footer(page, dn->nid, dn->inode->i_ino, ofs, true);
set_cold_node(dn->inode, page);
SetPageUptodate(page);
@@ -946,8 +962,7 @@
struct page *page;
int err;
repeat:
- page = grab_cache_page_write_begin(NODE_MAPPING(sbi),
- nid, AOP_FLAG_NOFS);
+ page = grab_cache_page(NODE_MAPPING(sbi), nid);
if (!page)
return ERR_PTR(-ENOMEM);
@@ -1194,6 +1209,8 @@
.rw = (wbc->sync_mode == WB_SYNC_ALL) ? WRITE_SYNC : WRITE,
};
+ trace_f2fs_writepage(page, NODE);
+
if (unlikely(sbi->por_doing))
goto redirty_out;
@@ -1225,10 +1242,7 @@
return 0;
redirty_out:
- dec_page_count(sbi, F2FS_DIRTY_NODES);
- wbc->pages_skipped++;
- account_page_redirty(page);
- set_page_dirty(page);
+ redirty_page_for_writepage(wbc, page);
return AOP_WRITEPAGE_ACTIVATE;
}
@@ -1238,6 +1252,8 @@
struct f2fs_sb_info *sbi = F2FS_SB(mapping->host->i_sb);
long diff;
+ trace_f2fs_writepages(mapping->host, wbc, NODE);
+
/* balancing f2fs's metadata in background */
f2fs_balance_fs_bg(sbi);
@@ -1313,13 +1329,14 @@
radix_tree_delete(&nm_i->free_nid_root, i->nid);
}
-static int add_free_nid(struct f2fs_nm_info *nm_i, nid_t nid, bool build)
+static int add_free_nid(struct f2fs_sb_info *sbi, nid_t nid, bool build)
{
+ struct f2fs_nm_info *nm_i = NM_I(sbi);
struct free_nid *i;
struct nat_entry *ne;
bool allocated = false;
- if (!available_free_memory(nm_i, FREE_NIDS))
+ if (!available_free_memory(sbi, FREE_NIDS))
return -1;
/* 0 nid should not be used */
@@ -1372,9 +1389,10 @@
kmem_cache_free(free_nid_slab, i);
}
-static void scan_nat_page(struct f2fs_nm_info *nm_i,
+static void scan_nat_page(struct f2fs_sb_info *sbi,
struct page *nat_page, nid_t start_nid)
{
+ struct f2fs_nm_info *nm_i = NM_I(sbi);
struct f2fs_nat_block *nat_blk = page_address(nat_page);
block_t blk_addr;
int i;
@@ -1389,7 +1407,7 @@
blk_addr = le32_to_cpu(nat_blk->entries[i].block_addr);
f2fs_bug_on(blk_addr == NEW_ADDR);
if (blk_addr == NULL_ADDR) {
- if (add_free_nid(nm_i, start_nid, true) < 0)
+ if (add_free_nid(sbi, start_nid, true) < 0)
break;
}
}
@@ -1413,7 +1431,7 @@
while (1) {
struct page *page = get_current_nat_page(sbi, nid);
- scan_nat_page(nm_i, page, nid);
+ scan_nat_page(sbi, page, nid);
f2fs_put_page(page, 1);
nid += (NAT_ENTRY_PER_BLOCK - (nid % NAT_ENTRY_PER_BLOCK));
@@ -1433,7 +1451,7 @@
block_t addr = le32_to_cpu(nat_in_journal(sum, i).block_addr);
nid = le32_to_cpu(nid_in_journal(sum, i));
if (addr == NULL_ADDR)
- add_free_nid(nm_i, nid, true);
+ add_free_nid(sbi, nid, true);
else
remove_free_nid(nm_i, nid);
}
@@ -1450,7 +1468,7 @@
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct free_nid *i = NULL;
retry:
- if (unlikely(sbi->total_valid_node_count + 1 >= nm_i->max_nid))
+ if (unlikely(sbi->total_valid_node_count + 1 > nm_i->available_nids))
return false;
spin_lock(&nm_i->free_nid_list_lock);
@@ -1510,7 +1528,7 @@
spin_lock(&nm_i->free_nid_list_lock);
i = __lookup_free_nid_list(nm_i, nid);
f2fs_bug_on(!i || i->state != NID_ALLOC);
- if (!available_free_memory(nm_i, FREE_NIDS)) {
+ if (!available_free_memory(sbi, FREE_NIDS)) {
__del_from_free_nid_list(nm_i, i);
need_free = true;
} else {
@@ -1532,7 +1550,7 @@
clear_node_page_dirty(page);
}
-void recover_inline_xattr(struct inode *inode, struct page *page)
+static void recover_inline_xattr(struct inode *inode, struct page *page)
{
struct f2fs_sb_info *sbi = F2FS_SB(inode->i_sb);
void *src_addr, *dst_addr;
@@ -1557,6 +1575,7 @@
src_addr = inline_xattr_addr(page);
inline_size = inline_xattr_size(inode);
+ f2fs_wait_on_page_writeback(ipage, NODE);
memcpy(dst_addr, src_addr, inline_size);
update_inode(inode, ipage);
@@ -1612,6 +1631,11 @@
struct node_info old_ni, new_ni;
struct page *ipage;
+ get_node_info(sbi, ino, &old_ni);
+
+ if (unlikely(old_ni.blk_addr != NULL_ADDR))
+ return -EINVAL;
+
ipage = grab_cache_page(NODE_MAPPING(sbi), ino);
if (!ipage)
return -ENOMEM;
@@ -1619,7 +1643,6 @@
/* Should not use this inode from free nid list */
remove_free_nid(NM_I(sbi), ino);
- get_node_info(sbi, ino, &old_ni);
SetPageUptodate(ipage);
fill_node_footer(ipage, ino, ino, 0, true);
@@ -1645,35 +1668,29 @@
/*
* ra_sum_pages() merge contiguous pages into one bio and submit.
- * these pre-readed pages are linked in pages list.
+ * these pre-readed pages are alloced in bd_inode's mapping tree.
*/
-static int ra_sum_pages(struct f2fs_sb_info *sbi, struct list_head *pages,
+static int ra_sum_pages(struct f2fs_sb_info *sbi, struct page **pages,
int start, int nrpages)
{
- struct page *page;
- int page_idx = start;
+ struct inode *inode = sbi->sb->s_bdev->bd_inode;
+ struct address_space *mapping = inode->i_mapping;
+ int i, page_idx = start;
struct f2fs_io_info fio = {
.type = META,
.rw = READ_SYNC | REQ_META | REQ_PRIO
};
- for (; page_idx < start + nrpages; page_idx++) {
- /* alloc temporal page for read node summary info*/
- page = alloc_page(GFP_F2FS_ZERO);
- if (!page)
+ for (i = 0; page_idx < start + nrpages; page_idx++, i++) {
+ /* alloc page in bd_inode for reading node summary info */
+ pages[i] = grab_cache_page(mapping, page_idx);
+ if (!pages[i])
break;
-
- lock_page(page);
- page->index = page_idx;
- list_add_tail(&page->lru, pages);
+ f2fs_submit_page_mbio(sbi, pages[i], page_idx, &fio);
}
- list_for_each_entry(page, pages, lru)
- f2fs_submit_page_mbio(sbi, page, page->index, &fio);
-
f2fs_submit_merged_bio(sbi, META, READ);
-
- return page_idx - start;
+ return i;
}
int restore_node_summary(struct f2fs_sb_info *sbi,
@@ -1681,11 +1698,11 @@
{
struct f2fs_node *rn;
struct f2fs_summary *sum_entry;
- struct page *page, *tmp;
+ struct inode *inode = sbi->sb->s_bdev->bd_inode;
block_t addr;
int bio_blocks = MAX_BIO_BLOCKS(max_hw_blocks(sbi));
- int i, last_offset, nrpages, err = 0;
- LIST_HEAD(page_list);
+ struct page *pages[bio_blocks];
+ int i, idx, last_offset, nrpages, err = 0;
/* scan the node segment */
last_offset = sbi->blocks_per_seg;
@@ -1696,29 +1713,31 @@
nrpages = min(last_offset - i, bio_blocks);
/* read ahead node pages */
- nrpages = ra_sum_pages(sbi, &page_list, addr, nrpages);
+ nrpages = ra_sum_pages(sbi, pages, addr, nrpages);
if (!nrpages)
return -ENOMEM;
- list_for_each_entry_safe(page, tmp, &page_list, lru) {
+ for (idx = 0; idx < nrpages; idx++) {
if (err)
goto skip;
- lock_page(page);
- if (unlikely(!PageUptodate(page))) {
+ lock_page(pages[idx]);
+ if (unlikely(!PageUptodate(pages[idx]))) {
err = -EIO;
} else {
- rn = F2FS_NODE(page);
+ rn = F2FS_NODE(pages[idx]);
sum_entry->nid = rn->footer.nid;
sum_entry->version = 0;
sum_entry->ofs_in_node = 0;
sum_entry++;
}
- unlock_page(page);
+ unlock_page(pages[idx]);
skip:
- list_del(&page->lru);
- __free_pages(page, 0);
+ page_cache_release(pages[idx]);
}
+
+ invalidate_mapping_pages(inode->i_mapping, addr,
+ addr + nrpages);
}
return err;
}
@@ -1756,9 +1775,7 @@
write_unlock(&nm_i->nat_tree_lock);
goto retry;
}
- nat_set_blkaddr(ne, le32_to_cpu(raw_ne.block_addr));
- nat_set_ino(ne, le32_to_cpu(raw_ne.ino));
- nat_set_version(ne, raw_ne.version);
+ node_info_from_raw_nat(&ne->ni, &raw_ne);
__set_nat_cache_dirty(nm_i, ne);
write_unlock(&nm_i->nat_tree_lock);
}
@@ -1791,7 +1808,6 @@
nid_t nid;
struct f2fs_nat_entry raw_ne;
int offset = -1;
- block_t new_blkaddr;
if (nat_get_blkaddr(ne) == NEW_ADDR)
continue;
@@ -1827,11 +1843,7 @@
f2fs_bug_on(!nat_blk);
raw_ne = nat_blk->entries[nid - start_nid];
flush_now:
- new_blkaddr = nat_get_blkaddr(ne);
-
- raw_ne.ino = cpu_to_le32(nat_get_ino(ne));
- raw_ne.block_addr = cpu_to_le32(new_blkaddr);
- raw_ne.version = nat_get_version(ne);
+ raw_nat_from_node_info(&raw_ne, &ne->ni);
if (offset < 0) {
nat_blk->entries[nid - start_nid] = raw_ne;
@@ -1841,7 +1853,7 @@
}
if (nat_get_blkaddr(ne) == NULL_ADDR &&
- add_free_nid(NM_I(sbi), nid, false) <= 0) {
+ add_free_nid(sbi, nid, false) <= 0) {
write_lock(&nm_i->nat_tree_lock);
__del_from_nat_cache(nm_i, ne);
write_unlock(&nm_i->nat_tree_lock);
@@ -1869,8 +1881,10 @@
nat_segs = le32_to_cpu(sb_raw->segment_count_nat) >> 1;
nat_blocks = nat_segs << le32_to_cpu(sb_raw->log_blocks_per_seg);
+ nm_i->max_nid = NAT_ENTRY_PER_BLOCK * nat_blocks;
+
/* not used nids: 0, node, meta, (and root counted as valid node) */
- nm_i->max_nid = NAT_ENTRY_PER_BLOCK * nat_blocks - 3;
+ nm_i->available_nids = nm_i->max_nid - 3;
nm_i->fcnt = 0;
nm_i->nat_cnt = 0;
nm_i->ram_thresh = DEF_RAM_THRESHOLD;
diff --git a/fs/f2fs/node.h b/fs/f2fs/node.h
index 5decc1a..7281112 100644
--- a/fs/f2fs/node.h
+++ b/fs/f2fs/node.h
@@ -59,12 +59,12 @@
do { \
ne->checkpointed = false; \
list_move_tail(&ne->list, &nm_i->dirty_nat_entries); \
- } while (0);
+ } while (0)
#define __clear_nat_cache_dirty(nm_i, ne) \
do { \
ne->checkpointed = true; \
list_move_tail(&ne->list, &nm_i->nat_entries); \
- } while (0);
+ } while (0)
#define inc_node_version(version) (++version)
static inline void node_info_from_raw_nat(struct node_info *ni,
@@ -75,9 +75,18 @@
ni->version = raw_ne->version;
}
-enum nid_type {
+static inline void raw_nat_from_node_info(struct f2fs_nat_entry *raw_ne,
+ struct node_info *ni)
+{
+ raw_ne->ino = cpu_to_le32(ni->ino);
+ raw_ne->block_addr = cpu_to_le32(ni->blk_addr);
+ raw_ne->version = ni->version;
+}
+
+enum mem_type {
FREE_NIDS, /* indicates the free nid list */
- NAT_ENTRIES /* indicates the cached nat entry */
+ NAT_ENTRIES, /* indicates the cached nat entry */
+ DIRTY_DENTS /* indicates dirty dentry pages */
};
/*
@@ -263,7 +272,7 @@
{
struct f2fs_node *rn = F2FS_NODE(p);
- wait_on_page_writeback(p);
+ f2fs_wait_on_page_writeback(p, NODE);
if (i)
rn->i.i_nid[off - NODE_DIR1_BLOCK] = cpu_to_le32(nid);
diff --git a/fs/f2fs/recovery.c b/fs/f2fs/recovery.c
index b1ae89f..a112368 100644
--- a/fs/f2fs/recovery.c
+++ b/fs/f2fs/recovery.c
@@ -46,15 +46,10 @@
struct inode *dir, *einode;
int err = 0;
- dir = check_dirty_dir_inode(F2FS_SB(inode->i_sb), pino);
- if (!dir) {
- dir = f2fs_iget(inode->i_sb, pino);
- if (IS_ERR(dir)) {
- err = PTR_ERR(dir);
- goto out;
- }
- set_inode_flag(F2FS_I(dir), FI_DELAY_IPUT);
- add_dirty_dir_inode(dir);
+ dir = f2fs_iget(inode->i_sb, pino);
+ if (IS_ERR(dir)) {
+ err = PTR_ERR(dir);
+ goto out;
}
name.len = le32_to_cpu(raw_inode->i_namelen);
@@ -63,7 +58,7 @@
if (unlikely(name.len > F2FS_NAME_LEN)) {
WARN_ON(1);
err = -ENAMETOOLONG;
- goto out;
+ goto out_err;
}
retry:
de = f2fs_find_entry(dir, &name, &page);
@@ -73,7 +68,8 @@
einode = f2fs_iget(inode->i_sb, le32_to_cpu(de->ino));
if (IS_ERR(einode)) {
WARN_ON(1);
- if (PTR_ERR(einode) == -ENOENT)
+ err = PTR_ERR(einode);
+ if (err == -ENOENT)
err = -EEXIST;
goto out_unmap_put;
}
@@ -87,11 +83,23 @@
goto retry;
}
err = __f2fs_add_link(dir, &name, inode);
+ if (err)
+ goto out_err;
+
+ if (is_inode_flag_set(F2FS_I(dir), FI_DELAY_IPUT)) {
+ iput(dir);
+ } else {
+ add_dirty_dir_inode(dir);
+ set_inode_flag(F2FS_I(dir), FI_DELAY_IPUT);
+ }
+
goto out;
out_unmap_put:
kunmap(page);
f2fs_put_page(page, 0);
+out_err:
+ iput(dir);
out:
f2fs_msg(inode->i_sb, KERN_NOTICE,
"%s: ino = %x, name = %s, dir = %lx, err = %d",
@@ -299,10 +307,7 @@
goto out;
start = start_bidx_of_node(ofs_of_node(page), fi);
- if (IS_INODE(page))
- end = start + ADDRS_PER_INODE(fi);
- else
- end = start + ADDRS_PER_BLOCK;
+ end = start + ADDRS_PER_PAGE(page, fi);
f2fs_lock_op(sbi);
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 085f548b..f25f0e0 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -25,7 +25,6 @@
#define __reverse_ffz(x) __reverse_ffs(~(x))
static struct kmem_cache *discard_entry_slab;
-static struct kmem_cache *flush_cmd_slab;
/*
* __reverse_ffs is copied from include/asm-generic/bitops/__ffs.h since
@@ -200,20 +199,20 @@
static int issue_flush_thread(void *data)
{
struct f2fs_sb_info *sbi = data;
- struct f2fs_sm_info *sm_i = SM_I(sbi);
- wait_queue_head_t *q = &sm_i->flush_wait_queue;
+ struct flush_cmd_control *fcc = SM_I(sbi)->cmd_control_info;
+ wait_queue_head_t *q = &fcc->flush_wait_queue;
repeat:
if (kthread_should_stop())
return 0;
- spin_lock(&sm_i->issue_lock);
- if (sm_i->issue_list) {
- sm_i->dispatch_list = sm_i->issue_list;
- sm_i->issue_list = sm_i->issue_tail = NULL;
+ spin_lock(&fcc->issue_lock);
+ if (fcc->issue_list) {
+ fcc->dispatch_list = fcc->issue_list;
+ fcc->issue_list = fcc->issue_tail = NULL;
}
- spin_unlock(&sm_i->issue_lock);
+ spin_unlock(&fcc->issue_lock);
- if (sm_i->dispatch_list) {
+ if (fcc->dispatch_list) {
struct bio *bio = bio_alloc(GFP_NOIO, 0);
struct flush_cmd *cmd, *next;
int ret;
@@ -221,47 +220,79 @@
bio->bi_bdev = sbi->sb->s_bdev;
ret = submit_bio_wait(WRITE_FLUSH, bio);
- for (cmd = sm_i->dispatch_list; cmd; cmd = next) {
+ for (cmd = fcc->dispatch_list; cmd; cmd = next) {
cmd->ret = ret;
next = cmd->next;
complete(&cmd->wait);
}
- sm_i->dispatch_list = NULL;
+ bio_put(bio);
+ fcc->dispatch_list = NULL;
}
- wait_event_interruptible(*q, kthread_should_stop() || sm_i->issue_list);
+ wait_event_interruptible(*q,
+ kthread_should_stop() || fcc->issue_list);
goto repeat;
}
int f2fs_issue_flush(struct f2fs_sb_info *sbi)
{
- struct f2fs_sm_info *sm_i = SM_I(sbi);
- struct flush_cmd *cmd;
- int ret;
+ struct flush_cmd_control *fcc = SM_I(sbi)->cmd_control_info;
+ struct flush_cmd cmd;
if (!test_opt(sbi, FLUSH_MERGE))
return blkdev_issue_flush(sbi->sb->s_bdev, GFP_KERNEL, NULL);
- cmd = f2fs_kmem_cache_alloc(flush_cmd_slab, GFP_ATOMIC);
- cmd->next = NULL;
- cmd->ret = 0;
- init_completion(&cmd->wait);
+ init_completion(&cmd.wait);
+ cmd.next = NULL;
- spin_lock(&sm_i->issue_lock);
- if (sm_i->issue_list)
- sm_i->issue_tail->next = cmd;
+ spin_lock(&fcc->issue_lock);
+ if (fcc->issue_list)
+ fcc->issue_tail->next = &cmd;
else
- sm_i->issue_list = cmd;
- sm_i->issue_tail = cmd;
- spin_unlock(&sm_i->issue_lock);
+ fcc->issue_list = &cmd;
+ fcc->issue_tail = &cmd;
+ spin_unlock(&fcc->issue_lock);
- if (!sm_i->dispatch_list)
- wake_up(&sm_i->flush_wait_queue);
+ if (!fcc->dispatch_list)
+ wake_up(&fcc->flush_wait_queue);
- wait_for_completion(&cmd->wait);
- ret = cmd->ret;
- kmem_cache_free(flush_cmd_slab, cmd);
- return ret;
+ wait_for_completion(&cmd.wait);
+
+ return cmd.ret;
+}
+
+int create_flush_cmd_control(struct f2fs_sb_info *sbi)
+{
+ dev_t dev = sbi->sb->s_bdev->bd_dev;
+ struct flush_cmd_control *fcc;
+ int err = 0;
+
+ fcc = kzalloc(sizeof(struct flush_cmd_control), GFP_KERNEL);
+ if (!fcc)
+ return -ENOMEM;
+ spin_lock_init(&fcc->issue_lock);
+ init_waitqueue_head(&fcc->flush_wait_queue);
+ fcc->f2fs_issue_flush = kthread_run(issue_flush_thread, sbi,
+ "f2fs_flush-%u:%u", MAJOR(dev), MINOR(dev));
+ if (IS_ERR(fcc->f2fs_issue_flush)) {
+ err = PTR_ERR(fcc->f2fs_issue_flush);
+ kfree(fcc);
+ return err;
+ }
+ sbi->sm_info->cmd_control_info = fcc;
+
+ return err;
+}
+
+void destroy_flush_cmd_control(struct f2fs_sb_info *sbi)
+{
+ struct flush_cmd_control *fcc =
+ sbi->sm_info->cmd_control_info;
+
+ if (fcc && fcc->f2fs_issue_flush)
+ kthread_stop(fcc->f2fs_issue_flush);
+ kfree(fcc);
+ sbi->sm_info->cmd_control_info = NULL;
}
static void __locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno,
@@ -336,13 +367,26 @@
mutex_unlock(&dirty_i->seglist_lock);
}
-static void f2fs_issue_discard(struct f2fs_sb_info *sbi,
+static int f2fs_issue_discard(struct f2fs_sb_info *sbi,
block_t blkstart, block_t blklen)
{
sector_t start = SECTOR_FROM_BLOCK(sbi, blkstart);
sector_t len = SECTOR_FROM_BLOCK(sbi, blklen);
- blkdev_issue_discard(sbi->sb->s_bdev, start, len, GFP_NOFS, 0);
trace_f2fs_issue_discard(sbi->sb, blkstart, blklen);
+ return blkdev_issue_discard(sbi->sb->s_bdev, start, len, GFP_NOFS, 0);
+}
+
+void discard_next_dnode(struct f2fs_sb_info *sbi)
+{
+ struct curseg_info *curseg = CURSEG_I(sbi, CURSEG_WARM_NODE);
+ block_t blkaddr = NEXT_FREE_BLKADDR(sbi, curseg);
+
+ if (f2fs_issue_discard(sbi, blkaddr, 1)) {
+ struct page *page = grab_meta_page(sbi, blkaddr);
+ /* zero-filled page */
+ set_page_dirty(page);
+ f2fs_put_page(page, 1);
+ }
}
static void add_discard_addrs(struct f2fs_sb_info *sbi,
@@ -1832,7 +1876,6 @@
{
struct f2fs_super_block *raw_super = F2FS_RAW_SUPER(sbi);
struct f2fs_checkpoint *ckpt = F2FS_CKPT(sbi);
- dev_t dev = sbi->sb->s_bdev->bd_dev;
struct f2fs_sm_info *sm_info;
int err;
@@ -1860,14 +1903,10 @@
sm_info->nr_discards = 0;
sm_info->max_discards = 0;
- if (test_opt(sbi, FLUSH_MERGE)) {
- spin_lock_init(&sm_info->issue_lock);
- init_waitqueue_head(&sm_info->flush_wait_queue);
-
- sm_info->f2fs_issue_flush = kthread_run(issue_flush_thread, sbi,
- "f2fs_flush-%u:%u", MAJOR(dev), MINOR(dev));
- if (IS_ERR(sm_info->f2fs_issue_flush))
- return PTR_ERR(sm_info->f2fs_issue_flush);
+ if (test_opt(sbi, FLUSH_MERGE) && !f2fs_readonly(sbi->sb)) {
+ err = create_flush_cmd_control(sbi);
+ if (err)
+ return err;
}
err = build_sit_info(sbi);
@@ -1976,10 +2015,10 @@
void destroy_segment_manager(struct f2fs_sb_info *sbi)
{
struct f2fs_sm_info *sm_info = SM_I(sbi);
+
if (!sm_info)
return;
- if (sm_info->f2fs_issue_flush)
- kthread_stop(sm_info->f2fs_issue_flush);
+ destroy_flush_cmd_control(sbi);
destroy_dirty_segmap(sbi);
destroy_curseg(sbi);
destroy_free_segmap(sbi);
@@ -1994,17 +2033,10 @@
sizeof(struct discard_entry));
if (!discard_entry_slab)
return -ENOMEM;
- flush_cmd_slab = f2fs_kmem_cache_create("flush_command",
- sizeof(struct flush_cmd));
- if (!flush_cmd_slab) {
- kmem_cache_destroy(discard_entry_slab);
- return -ENOMEM;
- }
return 0;
}
void destroy_segment_manager_caches(void)
{
kmem_cache_destroy(discard_entry_slab);
- kmem_cache_destroy(flush_cmd_slab);
}
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index c756923..b2b1863 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -514,7 +514,7 @@
{
struct f2fs_sb_info *sbi = F2FS_SB(root->d_sb);
- if (!(root->d_sb->s_flags & MS_RDONLY) && test_opt(sbi, BG_GC))
+ if (!f2fs_readonly(sbi->sb) && test_opt(sbi, BG_GC))
seq_printf(seq, ",background_gc=%s", "on");
else
seq_printf(seq, ",background_gc=%s", "off");
@@ -542,7 +542,7 @@
seq_puts(seq, ",disable_ext_identify");
if (test_opt(sbi, INLINE_DATA))
seq_puts(seq, ",inline_data");
- if (test_opt(sbi, FLUSH_MERGE))
+ if (!f2fs_readonly(sbi->sb) && test_opt(sbi, FLUSH_MERGE))
seq_puts(seq, ",flush_merge");
seq_printf(seq, ",active_logs=%u", sbi->active_logs);
@@ -594,6 +594,8 @@
struct f2fs_sb_info *sbi = F2FS_SB(sb);
struct f2fs_mount_info org_mount_opt;
int err, active_logs;
+ bool need_restart_gc = false;
+ bool need_stop_gc = false;
sync_filesystem(sb);
@@ -611,7 +613,7 @@
/*
* Previous and new state of filesystem is RO,
- * so no point in checking GC conditions.
+ * so skip checking GC and FLUSH_MERGE conditions.
*/
if ((sb->s_flags & MS_RDONLY) && (*flags & MS_RDONLY))
goto skip;
@@ -625,18 +627,40 @@
if (sbi->gc_thread) {
stop_gc_thread(sbi);
f2fs_sync_fs(sb, 1);
+ need_restart_gc = true;
}
} else if (test_opt(sbi, BG_GC) && !sbi->gc_thread) {
err = start_gc_thread(sbi);
if (err)
goto restore_opts;
+ need_stop_gc = true;
+ }
+
+ /*
+ * We stop issue flush thread if FS is mounted as RO
+ * or if flush_merge is not passed in mount option.
+ */
+ if ((*flags & MS_RDONLY) || !test_opt(sbi, FLUSH_MERGE)) {
+ destroy_flush_cmd_control(sbi);
+ } else if (test_opt(sbi, FLUSH_MERGE) &&
+ !sbi->sm_info->cmd_control_info) {
+ err = create_flush_cmd_control(sbi);
+ if (err)
+ goto restore_gc;
}
skip:
/* Update the POSIXACL Flag */
sb->s_flags = (sb->s_flags & ~MS_POSIXACL) |
(test_opt(sbi, POSIX_ACL) ? MS_POSIXACL : 0);
return 0;
-
+restore_gc:
+ if (need_restart_gc) {
+ if (start_gc_thread(sbi))
+ f2fs_msg(sbi->sb, KERN_WARNING,
+ "background gc thread is stop");
+ } else if (need_stop_gc) {
+ stop_gc_thread(sbi);
+ }
restore_opts:
sbi->mount_opt = org_mount_opt;
sbi->active_logs = active_logs;
diff --git a/fs/f2fs/xattr.c b/fs/f2fs/xattr.c
index 503c245..8bea941 100644
--- a/fs/f2fs/xattr.c
+++ b/fs/f2fs/xattr.c
@@ -26,7 +26,7 @@
#include "xattr.h"
static size_t f2fs_xattr_generic_list(struct dentry *dentry, char *list,
- size_t list_size, const char *name, size_t name_len, int type)
+ size_t list_size, const char *name, size_t len, int type)
{
struct f2fs_sb_info *sbi = F2FS_SB(dentry->d_sb);
int total_len, prefix_len = 0;
@@ -53,11 +53,11 @@
return -EINVAL;
}
- total_len = prefix_len + name_len + 1;
+ total_len = prefix_len + len + 1;
if (list && total_len <= list_size) {
memcpy(list, prefix, prefix_len);
- memcpy(list + prefix_len, name, name_len);
- list[prefix_len + name_len] = '\0';
+ memcpy(list + prefix_len, name, len);
+ list[prefix_len + len] = '\0';
}
return total_len;
}
@@ -108,11 +108,12 @@
if (strcmp(name, "") == 0)
return -EINVAL;
- return f2fs_setxattr(dentry->d_inode, type, name, value, size, NULL);
+ return f2fs_setxattr(dentry->d_inode, type, name,
+ value, size, NULL, flags);
}
static size_t f2fs_xattr_advise_list(struct dentry *dentry, char *list,
- size_t list_size, const char *name, size_t name_len, int type)
+ size_t list_size, const char *name, size_t len, int type)
{
const char *xname = F2FS_SYSTEM_ADVISE_PREFIX;
size_t size;
@@ -155,9 +156,6 @@
}
#ifdef CONFIG_F2FS_FS_SECURITY
-static int __f2fs_setxattr(struct inode *inode, int name_index,
- const char *name, const void *value, size_t value_len,
- struct page *ipage);
static int f2fs_initxattrs(struct inode *inode, const struct xattr *xattr_array,
void *page)
{
@@ -165,9 +163,9 @@
int err = 0;
for (xattr = xattr_array; xattr->name != NULL; xattr++) {
- err = __f2fs_setxattr(inode, F2FS_XATTR_INDEX_SECURITY,
+ err = f2fs_setxattr(inode, F2FS_XATTR_INDEX_SECURITY,
xattr->name, xattr->value,
- xattr->value_len, (struct page *)page);
+ xattr->value_len, (struct page *)page, 0);
if (err < 0)
break;
}
@@ -241,26 +239,26 @@
NULL,
};
-static inline const struct xattr_handler *f2fs_xattr_handler(int name_index)
+static inline const struct xattr_handler *f2fs_xattr_handler(int index)
{
const struct xattr_handler *handler = NULL;
- if (name_index > 0 && name_index < ARRAY_SIZE(f2fs_xattr_handler_map))
- handler = f2fs_xattr_handler_map[name_index];
+ if (index > 0 && index < ARRAY_SIZE(f2fs_xattr_handler_map))
+ handler = f2fs_xattr_handler_map[index];
return handler;
}
-static struct f2fs_xattr_entry *__find_xattr(void *base_addr, int name_index,
- size_t name_len, const char *name)
+static struct f2fs_xattr_entry *__find_xattr(void *base_addr, int index,
+ size_t len, const char *name)
{
struct f2fs_xattr_entry *entry;
list_for_each_xattr(entry, base_addr) {
- if (entry->e_name_index != name_index)
+ if (entry->e_name_index != index)
continue;
- if (entry->e_name_len != name_len)
+ if (entry->e_name_len != len)
continue;
- if (!memcmp(entry->e_name, name, name_len))
+ if (!memcmp(entry->e_name, name, len))
break;
}
return entry;
@@ -347,6 +345,7 @@
if (ipage) {
inline_addr = inline_xattr_addr(ipage);
+ f2fs_wait_on_page_writeback(ipage, NODE);
} else {
page = get_node_page(sbi, inode->i_ino);
if (IS_ERR(page)) {
@@ -354,6 +353,7 @@
return PTR_ERR(page);
}
inline_addr = inline_xattr_addr(page);
+ f2fs_wait_on_page_writeback(page, NODE);
}
memcpy(inline_addr, txattr_addr, inline_size);
f2fs_put_page(page, 1);
@@ -374,6 +374,7 @@
return PTR_ERR(xpage);
}
f2fs_bug_on(new_nid);
+ f2fs_wait_on_page_writeback(xpage, NODE);
} else {
struct dnode_of_data dn;
set_new_dnode(&dn, inode, NULL, NULL, new_nid);
@@ -396,42 +397,43 @@
return 0;
}
-int f2fs_getxattr(struct inode *inode, int name_index, const char *name,
+int f2fs_getxattr(struct inode *inode, int index, const char *name,
void *buffer, size_t buffer_size)
{
struct f2fs_xattr_entry *entry;
void *base_addr;
int error = 0;
- size_t value_len, name_len;
+ size_t size, len;
if (name == NULL)
return -EINVAL;
- name_len = strlen(name);
- if (name_len > F2FS_NAME_LEN)
+
+ len = strlen(name);
+ if (len > F2FS_NAME_LEN)
return -ERANGE;
base_addr = read_all_xattrs(inode, NULL);
if (!base_addr)
return -ENOMEM;
- entry = __find_xattr(base_addr, name_index, name_len, name);
+ entry = __find_xattr(base_addr, index, len, name);
if (IS_XATTR_LAST_ENTRY(entry)) {
error = -ENODATA;
goto cleanup;
}
- value_len = le16_to_cpu(entry->e_value_size);
+ size = le16_to_cpu(entry->e_value_size);
- if (buffer && value_len > buffer_size) {
+ if (buffer && size > buffer_size) {
error = -ERANGE;
goto cleanup;
}
if (buffer) {
char *pval = entry->e_name + entry->e_name_len;
- memcpy(buffer, pval, value_len);
+ memcpy(buffer, pval, size);
}
- error = value_len;
+ error = size;
cleanup:
kzfree(base_addr);
@@ -475,15 +477,15 @@
return error;
}
-static int __f2fs_setxattr(struct inode *inode, int name_index,
- const char *name, const void *value, size_t value_len,
- struct page *ipage)
+static int __f2fs_setxattr(struct inode *inode, int index,
+ const char *name, const void *value, size_t size,
+ struct page *ipage, int flags)
{
struct f2fs_inode_info *fi = F2FS_I(inode);
struct f2fs_xattr_entry *here, *last;
void *base_addr;
int found, newsize;
- size_t name_len;
+ size_t len;
__u32 new_hsize;
int error = -ENOMEM;
@@ -491,11 +493,11 @@
return -EINVAL;
if (value == NULL)
- value_len = 0;
+ size = 0;
- name_len = strlen(name);
+ len = strlen(name);
- if (name_len > F2FS_NAME_LEN || value_len > MAX_VALUE_LEN(inode))
+ if (len > F2FS_NAME_LEN || size > MAX_VALUE_LEN(inode))
return -ERANGE;
base_addr = read_all_xattrs(inode, ipage);
@@ -503,16 +505,23 @@
goto exit;
/* find entry with wanted name. */
- here = __find_xattr(base_addr, name_index, name_len, name);
+ here = __find_xattr(base_addr, index, len, name);
found = IS_XATTR_LAST_ENTRY(here) ? 0 : 1;
- last = here;
+ if ((flags & XATTR_REPLACE) && !found) {
+ error = -ENODATA;
+ goto exit;
+ } else if ((flags & XATTR_CREATE) && found) {
+ error = -EEXIST;
+ goto exit;
+ }
+
+ last = here;
while (!IS_XATTR_LAST_ENTRY(last))
last = XATTR_NEXT_ENTRY(last);
- newsize = XATTR_ALIGN(sizeof(struct f2fs_xattr_entry) +
- name_len + value_len);
+ newsize = XATTR_ALIGN(sizeof(struct f2fs_xattr_entry) + len + size);
/* 1. Check space */
if (value) {
@@ -555,12 +564,12 @@
* We just write new entry.
*/
memset(last, 0, newsize);
- last->e_name_index = name_index;
- last->e_name_len = name_len;
- memcpy(last->e_name, name, name_len);
- pval = last->e_name + name_len;
- memcpy(pval, value, value_len);
- last->e_value_size = cpu_to_le16(value_len);
+ last->e_name_index = index;
+ last->e_name_len = len;
+ memcpy(last->e_name, name, len);
+ pval = last->e_name + len;
+ memcpy(pval, value, size);
+ last->e_value_size = cpu_to_le16(size);
new_hsize += newsize;
}
@@ -583,18 +592,23 @@
return error;
}
-int f2fs_setxattr(struct inode *inode, int name_index, const char *name,
- const void *value, size_t value_len, struct page *ipage)
+int f2fs_setxattr(struct inode *inode, int index, const char *name,
+ const void *value, size_t size,
+ struct page *ipage, int flags)
{
struct f2fs_sb_info *sbi = F2FS_SB(inode->i_sb);
int err;
+ /* this case is only from init_inode_metadata */
+ if (ipage)
+ return __f2fs_setxattr(inode, index, name, value,
+ size, ipage, flags);
f2fs_balance_fs(sbi);
f2fs_lock_op(sbi);
/* protect xattr_ver */
down_write(&F2FS_I(inode)->i_sem);
- err = __f2fs_setxattr(inode, name_index, name, value, value_len, ipage);
+ err = __f2fs_setxattr(inode, index, name, value, size, ipage, flags);
up_write(&F2FS_I(inode)->i_sem);
f2fs_unlock_op(sbi);
diff --git a/fs/f2fs/xattr.h b/fs/f2fs/xattr.h
index b21d9eb..34ab7db 100644
--- a/fs/f2fs/xattr.h
+++ b/fs/f2fs/xattr.h
@@ -114,18 +114,18 @@
extern const struct xattr_handler *f2fs_xattr_handlers[];
extern int f2fs_setxattr(struct inode *, int, const char *,
- const void *, size_t, struct page *);
+ const void *, size_t, struct page *, int);
extern int f2fs_getxattr(struct inode *, int, const char *, void *, size_t);
extern ssize_t f2fs_listxattr(struct dentry *, char *, size_t);
#else
#define f2fs_xattr_handlers NULL
-static inline int f2fs_setxattr(struct inode *inode, int name_index,
- const char *name, const void *value, size_t value_len)
+static inline int f2fs_setxattr(struct inode *inode, int index,
+ const char *name, const void *value, size_t size, int flags)
{
return -EOPNOTSUPP;
}
-static inline int f2fs_getxattr(struct inode *inode, int name_index,
+static inline int f2fs_getxattr(struct inode *inode, int index,
const char *name, void *buffer, size_t buffer_size)
{
return -EOPNOTSUPP;
diff --git a/fs/fat/fat.h b/fs/fat/fat.h
index 7c31f4b..e0c4ba3 100644
--- a/fs/fat/fat.h
+++ b/fs/fat/fat.h
@@ -52,7 +52,8 @@
usefree:1, /* Use free_clusters for FAT32 */
tz_set:1, /* Filesystem timestamps' offset set */
rodir:1, /* allow ATTR_RO for directory */
- discard:1; /* Issue discard requests on deletions */
+ discard:1, /* Issue discard requests on deletions */
+ dos1xfloppy:1; /* Assume default BPB for DOS 1.x floppies */
};
#define FAT_HASH_BITS 8
diff --git a/fs/fat/inode.c b/fs/fat/inode.c
index b3361fe..9c83594 100644
--- a/fs/fat/inode.c
+++ b/fs/fat/inode.c
@@ -35,9 +35,71 @@
#define CONFIG_FAT_DEFAULT_IOCHARSET ""
#endif
+#define KB_IN_SECTORS 2
+
+/*
+ * A deserialized copy of the on-disk structure laid out in struct
+ * fat_boot_sector.
+ */
+struct fat_bios_param_block {
+ u16 fat_sector_size;
+ u8 fat_sec_per_clus;
+ u16 fat_reserved;
+ u8 fat_fats;
+ u16 fat_dir_entries;
+ u16 fat_sectors;
+ u16 fat_fat_length;
+ u32 fat_total_sect;
+
+ u8 fat16_state;
+ u32 fat16_vol_id;
+
+ u32 fat32_length;
+ u32 fat32_root_cluster;
+ u16 fat32_info_sector;
+ u8 fat32_state;
+ u32 fat32_vol_id;
+};
+
static int fat_default_codepage = CONFIG_FAT_DEFAULT_CODEPAGE;
static char fat_default_iocharset[] = CONFIG_FAT_DEFAULT_IOCHARSET;
+static struct fat_floppy_defaults {
+ unsigned nr_sectors;
+ unsigned sec_per_clus;
+ unsigned dir_entries;
+ unsigned media;
+ unsigned fat_length;
+} floppy_defaults[] = {
+{
+ .nr_sectors = 160 * KB_IN_SECTORS,
+ .sec_per_clus = 1,
+ .dir_entries = 64,
+ .media = 0xFE,
+ .fat_length = 1,
+},
+{
+ .nr_sectors = 180 * KB_IN_SECTORS,
+ .sec_per_clus = 1,
+ .dir_entries = 64,
+ .media = 0xFC,
+ .fat_length = 2,
+},
+{
+ .nr_sectors = 320 * KB_IN_SECTORS,
+ .sec_per_clus = 2,
+ .dir_entries = 112,
+ .media = 0xFF,
+ .fat_length = 1,
+},
+{
+ .nr_sectors = 360 * KB_IN_SECTORS,
+ .sec_per_clus = 2,
+ .dir_entries = 112,
+ .media = 0xFD,
+ .fat_length = 2,
+},
+};
static int fat_add_cluster(struct inode *inode)
{
@@ -359,7 +421,7 @@
static int is_exec(unsigned char *extension)
{
- unsigned char *exe_extensions = "EXECOMBAT", *walk;
+ unsigned char exe_extensions[] = "EXECOMBAT", *walk;
for (walk = exe_extensions; *walk; walk += 3)
if (!strncmp(extension, walk, 3))
@@ -853,6 +915,8 @@
seq_puts(m, ",nfs=stale_rw");
if (opts->discard)
seq_puts(m, ",discard");
+ if (opts->dos1xfloppy)
+ seq_puts(m, ",dos1xfloppy");
return 0;
}
@@ -867,7 +931,7 @@
Opt_uni_xl_no, Opt_uni_xl_yes, Opt_nonumtail_no, Opt_nonumtail_yes,
Opt_obsolete, Opt_flush, Opt_tz_utc, Opt_rodir, Opt_err_cont,
Opt_err_panic, Opt_err_ro, Opt_discard, Opt_nfs, Opt_time_offset,
- Opt_nfs_stale_rw, Opt_nfs_nostale_ro, Opt_err,
+ Opt_nfs_stale_rw, Opt_nfs_nostale_ro, Opt_err, Opt_dos1xfloppy,
};
static const match_table_t fat_tokens = {
@@ -900,6 +964,7 @@
{Opt_nfs_stale_rw, "nfs"},
{Opt_nfs_stale_rw, "nfs=stale_rw"},
{Opt_nfs_nostale_ro, "nfs=nostale_ro"},
+ {Opt_dos1xfloppy, "dos1xfloppy"},
{Opt_obsolete, "conv=binary"},
{Opt_obsolete, "conv=text"},
{Opt_obsolete, "conv=auto"},
@@ -1102,6 +1167,9 @@
case Opt_nfs_nostale_ro:
opts->nfs = FAT_NFS_NOSTALE_RO;
break;
+ case Opt_dos1xfloppy:
+ opts->dos1xfloppy = 1;
+ break;
/* msdos specific */
case Opt_dots:
@@ -1247,6 +1315,169 @@
return sbi->fat_length * sb->s_blocksize * 8 / sbi->fat_bits;
}
+static bool fat_bpb_is_zero(struct fat_boot_sector *b)
+{
+ if (get_unaligned_le16(&b->sector_size))
+ return false;
+ if (b->sec_per_clus)
+ return false;
+ if (b->reserved)
+ return false;
+ if (b->fats)
+ return false;
+ if (get_unaligned_le16(&b->dir_entries))
+ return false;
+ if (get_unaligned_le16(&b->sectors))
+ return false;
+ if (b->media)
+ return false;
+ if (b->fat_length)
+ return false;
+ if (b->secs_track)
+ return false;
+ if (b->heads)
+ return false;
+ return true;
+}
+
+static int fat_read_bpb(struct super_block *sb, struct fat_boot_sector *b,
+ int silent, struct fat_bios_param_block *bpb)
+{
+ int error = -EINVAL;
+
+ /* Read in BPB ... */
+ memset(bpb, 0, sizeof(*bpb));
+ bpb->fat_sector_size = get_unaligned_le16(&b->sector_size);
+ bpb->fat_sec_per_clus = b->sec_per_clus;
+ bpb->fat_reserved = le16_to_cpu(b->reserved);
+ bpb->fat_fats = b->fats;
+ bpb->fat_dir_entries = get_unaligned_le16(&b->dir_entries);
+ bpb->fat_sectors = get_unaligned_le16(&b->sectors);
+ bpb->fat_fat_length = le16_to_cpu(b->fat_length);
+ bpb->fat_total_sect = le32_to_cpu(b->total_sect);
+
+ bpb->fat16_state = b->fat16.state;
+ bpb->fat16_vol_id = get_unaligned_le32(b->fat16.vol_id);
+
+ bpb->fat32_length = le32_to_cpu(b->fat32.length);
+ bpb->fat32_root_cluster = le32_to_cpu(b->fat32.root_cluster);
+ bpb->fat32_info_sector = le16_to_cpu(b->fat32.info_sector);
+ bpb->fat32_state = b->fat32.state;
+ bpb->fat32_vol_id = get_unaligned_le32(b->fat32.vol_id);
+
+ /* Validate this looks like a FAT filesystem BPB */
+ if (!bpb->fat_reserved) {
+ if (!silent)
+ fat_msg(sb, KERN_ERR,
+ "bogus number of reserved sectors");
+ goto out;
+ }
+ if (!bpb->fat_fats) {
+ if (!silent)
+ fat_msg(sb, KERN_ERR, "bogus number of FAT structure");
+ goto out;
+ }
+
+ /*
+ * Earlier we checked here that b->secs_track and b->head are nonzero,
+ * but it turns out valid FAT filesystems can have zero there.
+ */
+
+ if (!fat_valid_media(b->media)) {
+ if (!silent)
+ fat_msg(sb, KERN_ERR, "invalid media value (0x%02x)",
+ (unsigned)b->media);
+ goto out;
+ }
+
+ if (!is_power_of_2(bpb->fat_sector_size)
+ || (bpb->fat_sector_size < 512)
+ || (bpb->fat_sector_size > 4096)) {
+ if (!silent)
+ fat_msg(sb, KERN_ERR, "bogus logical sector size %u",
+ (unsigned)bpb->fat_sector_size);
+ goto out;
+ }
+
+ if (!is_power_of_2(bpb->fat_sec_per_clus)) {
+ if (!silent)
+ fat_msg(sb, KERN_ERR, "bogus sectors per cluster %u",
+ (unsigned)bpb->fat_sec_per_clus);
+ goto out;
+ }
+
+ error = 0;
+
+out:
+ return error;
+}
+
+static int fat_read_static_bpb(struct super_block *sb,
+ struct fat_boot_sector *b, int silent,
+ struct fat_bios_param_block *bpb)
+{
+ static const char *notdos1x = "This doesn't look like a DOS 1.x volume";
+
+ struct fat_floppy_defaults *fdefaults = NULL;
+ int error = -EINVAL;
+ sector_t bd_sects;
+ unsigned i;
+
+ bd_sects = i_size_read(sb->s_bdev->bd_inode) / SECTOR_SIZE;
+
+ /* 16-bit DOS 1.x reliably wrote bootstrap short-jmp code */
+ if (b->ignored[0] != 0xeb || b->ignored[2] != 0x90) {
+ if (!silent)
+ fat_msg(sb, KERN_ERR,
+ "%s; no bootstrapping code", notdos1x);
+ goto out;
+ }
+
+ /*
+ * If any value in this region is non-zero, it isn't archaic
+ * DOS.
+ */
+ if (!fat_bpb_is_zero(b)) {
+ if (!silent)
+ fat_msg(sb, KERN_ERR,
+ "%s; DOS 2.x BPB is non-zero", notdos1x);
+ goto out;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(floppy_defaults); i++) {
+ if (floppy_defaults[i].nr_sectors == bd_sects) {
+ fdefaults = &floppy_defaults[i];
+ break;
+ }
+ }
+
+ if (fdefaults == NULL) {
+ if (!silent)
+ fat_msg(sb, KERN_WARNING,
+ "This looks like a DOS 1.x volume, but isn't a recognized floppy size (%llu sectors)",
+ (u64)bd_sects);
+ goto out;
+ }
+
+ if (!silent)
+ fat_msg(sb, KERN_INFO,
+ "This looks like a DOS 1.x volume; assuming default BPB values");
+
+ memset(bpb, 0, sizeof(*bpb));
+ bpb->fat_sector_size = SECTOR_SIZE;
+ bpb->fat_sec_per_clus = fdefaults->sec_per_clus;
+ bpb->fat_reserved = 1;
+ bpb->fat_fats = 2;
+ bpb->fat_dir_entries = fdefaults->dir_entries;
+ bpb->fat_sectors = fdefaults->nr_sectors;
+ bpb->fat_fat_length = fdefaults->fat_length;
+
+ error = 0;
+
+out:
+ return error;
+}
+
/*
* Read the super block of an MS-DOS FS.
*/
@@ -1256,12 +1487,11 @@
struct inode *root_inode = NULL, *fat_inode = NULL;
struct inode *fsinfo_inode = NULL;
struct buffer_head *bh;
- struct fat_boot_sector *b;
+ struct fat_bios_param_block bpb;
struct msdos_sb_info *sbi;
u16 logical_sector_size;
u32 total_sectors, total_clusters, fat_clusters, rootdir_sectors;
int debug;
- unsigned int media;
long error;
char buf[50];
@@ -1298,100 +1528,72 @@
goto out_fail;
}
- b = (struct fat_boot_sector *) bh->b_data;
- if (!b->reserved) {
- if (!silent)
- fat_msg(sb, KERN_ERR, "bogus number of reserved sectors");
- brelse(bh);
- goto out_invalid;
- }
- if (!b->fats) {
- if (!silent)
- fat_msg(sb, KERN_ERR, "bogus number of FAT structure");
- brelse(bh);
- goto out_invalid;
- }
+ error = fat_read_bpb(sb, (struct fat_boot_sector *)bh->b_data, silent,
+ &bpb);
+ if (error == -EINVAL && sbi->options.dos1xfloppy)
+ error = fat_read_static_bpb(sb,
+ (struct fat_boot_sector *)bh->b_data, silent, &bpb);
+ brelse(bh);
- /*
- * Earlier we checked here that b->secs_track and b->head are nonzero,
- * but it turns out valid FAT filesystems can have zero there.
- */
+ if (error == -EINVAL)
+ goto out_invalid;
+ else if (error)
+ goto out_fail;
- media = b->media;
- if (!fat_valid_media(media)) {
- if (!silent)
- fat_msg(sb, KERN_ERR, "invalid media value (0x%02x)",
- media);
- brelse(bh);
- goto out_invalid;
- }
- logical_sector_size = get_unaligned_le16(&b->sector_size);
- if (!is_power_of_2(logical_sector_size)
- || (logical_sector_size < 512)
- || (logical_sector_size > 4096)) {
- if (!silent)
- fat_msg(sb, KERN_ERR, "bogus logical sector size %u",
- logical_sector_size);
- brelse(bh);
- goto out_invalid;
- }
- sbi->sec_per_clus = b->sec_per_clus;
- if (!is_power_of_2(sbi->sec_per_clus)) {
- if (!silent)
- fat_msg(sb, KERN_ERR, "bogus sectors per cluster %u",
- sbi->sec_per_clus);
- brelse(bh);
- goto out_invalid;
- }
+ logical_sector_size = bpb.fat_sector_size;
+ sbi->sec_per_clus = bpb.fat_sec_per_clus;
+ error = -EIO;
if (logical_sector_size < sb->s_blocksize) {
fat_msg(sb, KERN_ERR, "logical sector size too small for device"
" (logical sector size = %u)", logical_sector_size);
- brelse(bh);
goto out_fail;
}
+
if (logical_sector_size > sb->s_blocksize) {
- brelse(bh);
+ struct buffer_head *bh_resize;
if (!sb_set_blocksize(sb, logical_sector_size)) {
fat_msg(sb, KERN_ERR, "unable to set blocksize %u",
logical_sector_size);
goto out_fail;
}
- bh = sb_bread(sb, 0);
- if (bh == NULL) {
+
+ /* Verify that the larger boot sector is fully readable */
+ bh_resize = sb_bread(sb, 0);
+ if (bh_resize == NULL) {
fat_msg(sb, KERN_ERR, "unable to read boot sector"
" (logical sector size = %lu)",
sb->s_blocksize);
goto out_fail;
}
- b = (struct fat_boot_sector *) bh->b_data;
+ brelse(bh_resize);
}
mutex_init(&sbi->s_lock);
sbi->cluster_size = sb->s_blocksize * sbi->sec_per_clus;
sbi->cluster_bits = ffs(sbi->cluster_size) - 1;
- sbi->fats = b->fats;
+ sbi->fats = bpb.fat_fats;
sbi->fat_bits = 0; /* Don't know yet */
- sbi->fat_start = le16_to_cpu(b->reserved);
- sbi->fat_length = le16_to_cpu(b->fat_length);
+ sbi->fat_start = bpb.fat_reserved;
+ sbi->fat_length = bpb.fat_fat_length;
sbi->root_cluster = 0;
sbi->free_clusters = -1; /* Don't know yet */
sbi->free_clus_valid = 0;
sbi->prev_free = FAT_START_ENT;
sb->s_maxbytes = 0xffffffff;
- if (!sbi->fat_length && b->fat32.length) {
+ if (!sbi->fat_length && bpb.fat32_length) {
struct fat_boot_fsinfo *fsinfo;
struct buffer_head *fsinfo_bh;
/* Must be FAT32 */
sbi->fat_bits = 32;
- sbi->fat_length = le32_to_cpu(b->fat32.length);
- sbi->root_cluster = le32_to_cpu(b->fat32.root_cluster);
+ sbi->fat_length = bpb.fat32_length;
+ sbi->root_cluster = bpb.fat32_root_cluster;
/* MC - if info_sector is 0, don't multiply by 0 */
- sbi->fsinfo_sector = le16_to_cpu(b->fat32.info_sector);
+ sbi->fsinfo_sector = bpb.fat32_info_sector;
if (sbi->fsinfo_sector == 0)
sbi->fsinfo_sector = 1;
@@ -1399,7 +1601,6 @@
if (fsinfo_bh == NULL) {
fat_msg(sb, KERN_ERR, "bread failed, FSINFO block"
" (sector = %lu)", sbi->fsinfo_sector);
- brelse(bh);
goto out_fail;
}
@@ -1422,35 +1623,28 @@
/* interpret volume ID as a little endian 32 bit integer */
if (sbi->fat_bits == 32)
- sbi->vol_id = (((u32)b->fat32.vol_id[0]) |
- ((u32)b->fat32.vol_id[1] << 8) |
- ((u32)b->fat32.vol_id[2] << 16) |
- ((u32)b->fat32.vol_id[3] << 24));
+ sbi->vol_id = bpb.fat32_vol_id;
else /* fat 16 or 12 */
- sbi->vol_id = (((u32)b->fat16.vol_id[0]) |
- ((u32)b->fat16.vol_id[1] << 8) |
- ((u32)b->fat16.vol_id[2] << 16) |
- ((u32)b->fat16.vol_id[3] << 24));
+ sbi->vol_id = bpb.fat16_vol_id;
sbi->dir_per_block = sb->s_blocksize / sizeof(struct msdos_dir_entry);
sbi->dir_per_block_bits = ffs(sbi->dir_per_block) - 1;
sbi->dir_start = sbi->fat_start + sbi->fats * sbi->fat_length;
- sbi->dir_entries = get_unaligned_le16(&b->dir_entries);
+ sbi->dir_entries = bpb.fat_dir_entries;
if (sbi->dir_entries & (sbi->dir_per_block - 1)) {
if (!silent)
fat_msg(sb, KERN_ERR, "bogus directory-entries per block"
" (%u)", sbi->dir_entries);
- brelse(bh);
goto out_invalid;
}
rootdir_sectors = sbi->dir_entries
* sizeof(struct msdos_dir_entry) / sb->s_blocksize;
sbi->data_start = sbi->dir_start + rootdir_sectors;
- total_sectors = get_unaligned_le16(&b->sectors);
+ total_sectors = bpb.fat_sectors;
if (total_sectors == 0)
- total_sectors = le32_to_cpu(b->total_sect);
+ total_sectors = bpb.fat_total_sect;
total_clusters = (total_sectors - sbi->data_start) / sbi->sec_per_clus;
@@ -1459,9 +1653,9 @@
/* some OSes set FAT_STATE_DIRTY and clean it on unmount. */
if (sbi->fat_bits == 32)
- sbi->dirty = b->fat32.state & FAT_STATE_DIRTY;
+ sbi->dirty = bpb.fat32_state & FAT_STATE_DIRTY;
else /* fat 16 or 12 */
- sbi->dirty = b->fat16.state & FAT_STATE_DIRTY;
+ sbi->dirty = bpb.fat16_state & FAT_STATE_DIRTY;
/* check that FAT table does not overflow */
fat_clusters = calc_fat_clusters(sb);
@@ -1470,7 +1664,6 @@
if (!silent)
fat_msg(sb, KERN_ERR, "count of clusters too big (%u)",
total_clusters);
- brelse(bh);
goto out_invalid;
}
@@ -1483,8 +1676,6 @@
if (sbi->prev_free < FAT_START_ENT)
sbi->prev_free = FAT_START_ENT;
- brelse(bh);
-
/* set up enough so that it can read an inode */
fat_hash_init(sb);
dir_hash_init(sb);
diff --git a/fs/file_table.c b/fs/file_table.c
index a374f50..40bf466 100644
--- a/fs/file_table.c
+++ b/fs/file_table.c
@@ -76,14 +76,14 @@
* Handle nr_files sysctl
*/
#if defined(CONFIG_SYSCTL) && defined(CONFIG_PROC_FS)
-int proc_nr_files(ctl_table *table, int write,
+int proc_nr_files(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
files_stat.nr_files = get_nr_files();
return proc_doulongvec_minmax(table, write, buffer, lenp, ppos);
}
#else
-int proc_nr_files(ctl_table *table, int write,
+int proc_nr_files(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
return -ENOSYS;
diff --git a/fs/fscache/main.c b/fs/fscache/main.c
index acd4bf1..63f868e 100644
--- a/fs/fscache/main.c
+++ b/fs/fscache/main.c
@@ -67,7 +67,7 @@
return ret;
}
-ctl_table fscache_sysctls[] = {
+struct ctl_table fscache_sysctls[] = {
{
.procname = "object_max_active",
.data = &fscache_object_max_active,
@@ -87,7 +87,7 @@
{}
};
-ctl_table fscache_sysctls_root[] = {
+struct ctl_table fscache_sysctls_root[] = {
{
.procname = "fscache",
.mode = 0555,
diff --git a/fs/hfsplus/attributes.c b/fs/hfsplus/attributes.c
index caf89a7..e5b221d 100644
--- a/fs/hfsplus/attributes.c
+++ b/fs/hfsplus/attributes.c
@@ -54,14 +54,11 @@
memset(key, 0, sizeof(struct hfsplus_attr_key));
key->attr.cnid = cpu_to_be32(cnid);
if (name) {
- len = strlen(name);
- if (len > HFSPLUS_ATTR_MAX_STRLEN) {
- pr_err("invalid xattr name's length\n");
- return -EINVAL;
- }
- hfsplus_asc2uni(sb,
+ int res = hfsplus_asc2uni(sb,
(struct hfsplus_unistr *)&key->attr.key_name,
- HFSPLUS_ATTR_MAX_STRLEN, name, len);
+ HFSPLUS_ATTR_MAX_STRLEN, name, strlen(name));
+ if (res)
+ return res;
len = be16_to_cpu(key->attr.key_name.length);
} else {
key->attr.key_name.length = 0;
@@ -82,31 +79,6 @@
return 0;
}
-void hfsplus_attr_build_key_uni(hfsplus_btree_key *key,
- u32 cnid,
- struct hfsplus_attr_unistr *name)
-{
- int ustrlen;
-
- memset(key, 0, sizeof(struct hfsplus_attr_key));
- ustrlen = be16_to_cpu(name->length);
- key->attr.cnid = cpu_to_be32(cnid);
- key->attr.key_name.length = cpu_to_be16(ustrlen);
- ustrlen *= 2;
- memcpy(key->attr.key_name.unicode, name->unicode, ustrlen);
-
- /* The length of the key, as stored in key_len field, does not include
- * the size of the key_len field itself.
- * So, offsetof(hfsplus_attr_key, key_name) is a trick because
- * it takes into consideration key_len field (__be16) of
- * hfsplus_attr_key structure instead of length field (__be16) of
- * hfsplus_attr_unistr structure.
- */
- key->key_len =
- cpu_to_be16(offsetof(struct hfsplus_attr_key, key_name) +
- ustrlen);
-}
-
hfsplus_attr_entry *hfsplus_alloc_attr_entry(void)
{
return kmem_cache_alloc(hfsplus_attr_tree_cachep, GFP_KERNEL);
diff --git a/fs/hfsplus/bnode.c b/fs/hfsplus/bnode.c
index 11c8602..759708f 100644
--- a/fs/hfsplus/bnode.c
+++ b/fs/hfsplus/bnode.c
@@ -27,13 +27,13 @@
pagep = node->page + (off >> PAGE_CACHE_SHIFT);
off &= ~PAGE_CACHE_MASK;
- l = min(len, (int)PAGE_CACHE_SIZE - off);
+ l = min_t(int, len, PAGE_CACHE_SIZE - off);
memcpy(buf, kmap(*pagep) + off, l);
kunmap(*pagep);
while ((len -= l) != 0) {
buf += l;
- l = min(len, (int)PAGE_CACHE_SIZE);
+ l = min_t(int, len, PAGE_CACHE_SIZE);
memcpy(buf, kmap(*++pagep), l);
kunmap(*pagep);
}
@@ -80,14 +80,14 @@
pagep = node->page + (off >> PAGE_CACHE_SHIFT);
off &= ~PAGE_CACHE_MASK;
- l = min(len, (int)PAGE_CACHE_SIZE - off);
+ l = min_t(int, len, PAGE_CACHE_SIZE - off);
memcpy(kmap(*pagep) + off, buf, l);
set_page_dirty(*pagep);
kunmap(*pagep);
while ((len -= l) != 0) {
buf += l;
- l = min(len, (int)PAGE_CACHE_SIZE);
+ l = min_t(int, len, PAGE_CACHE_SIZE);
memcpy(kmap(*++pagep), buf, l);
set_page_dirty(*pagep);
kunmap(*pagep);
@@ -110,13 +110,13 @@
pagep = node->page + (off >> PAGE_CACHE_SHIFT);
off &= ~PAGE_CACHE_MASK;
- l = min(len, (int)PAGE_CACHE_SIZE - off);
+ l = min_t(int, len, PAGE_CACHE_SIZE - off);
memset(kmap(*pagep) + off, 0, l);
set_page_dirty(*pagep);
kunmap(*pagep);
while ((len -= l) != 0) {
- l = min(len, (int)PAGE_CACHE_SIZE);
+ l = min_t(int, len, PAGE_CACHE_SIZE);
memset(kmap(*++pagep), 0, l);
set_page_dirty(*pagep);
kunmap(*pagep);
@@ -142,14 +142,14 @@
dst &= ~PAGE_CACHE_MASK;
if (src == dst) {
- l = min(len, (int)PAGE_CACHE_SIZE - src);
+ l = min_t(int, len, PAGE_CACHE_SIZE - src);
memcpy(kmap(*dst_page) + src, kmap(*src_page) + src, l);
kunmap(*src_page);
set_page_dirty(*dst_page);
kunmap(*dst_page);
while ((len -= l) != 0) {
- l = min(len, (int)PAGE_CACHE_SIZE);
+ l = min_t(int, len, PAGE_CACHE_SIZE);
memcpy(kmap(*++dst_page), kmap(*++src_page), l);
kunmap(*src_page);
set_page_dirty(*dst_page);
@@ -251,7 +251,7 @@
dst &= ~PAGE_CACHE_MASK;
if (src == dst) {
- l = min(len, (int)PAGE_CACHE_SIZE - src);
+ l = min_t(int, len, PAGE_CACHE_SIZE - src);
memmove(kmap(*dst_page) + src,
kmap(*src_page) + src, l);
kunmap(*src_page);
@@ -259,7 +259,7 @@
kunmap(*dst_page);
while ((len -= l) != 0) {
- l = min(len, (int)PAGE_CACHE_SIZE);
+ l = min_t(int, len, PAGE_CACHE_SIZE);
memmove(kmap(*++dst_page),
kmap(*++src_page), l);
kunmap(*src_page);
@@ -386,9 +386,8 @@
struct hfs_bnode *node;
if (cnid >= tree->node_count) {
- pr_err("request for non-existent node "
- "%d in B*Tree\n",
- cnid);
+ pr_err("request for non-existent node %d in B*Tree\n",
+ cnid);
return NULL;
}
@@ -409,9 +408,8 @@
loff_t off;
if (cnid >= tree->node_count) {
- pr_err("request for non-existent node "
- "%d in B*Tree\n",
- cnid);
+ pr_err("request for non-existent node %d in B*Tree\n",
+ cnid);
return NULL;
}
@@ -602,7 +600,7 @@
pagep = node->page;
memset(kmap(*pagep) + node->page_offset, 0,
- min((int)PAGE_CACHE_SIZE, (int)tree->node_size));
+ min_t(int, PAGE_CACHE_SIZE, tree->node_size));
set_page_dirty(*pagep);
kunmap(*pagep);
for (i = 1; i < tree->pages_per_bnode; i++) {
@@ -648,8 +646,8 @@
if (test_bit(HFS_BNODE_DELETED, &node->flags)) {
hfs_bnode_unhash(node);
spin_unlock(&tree->hash_lock);
- hfs_bnode_clear(node, 0,
- PAGE_CACHE_SIZE * tree->pages_per_bnode);
+ if (hfs_bnode_need_zeroout(tree))
+ hfs_bnode_clear(node, 0, tree->node_size);
hfs_bmap_free(node);
hfs_bnode_free(node);
return;
@@ -658,3 +656,16 @@
}
}
+/*
+ * Unused nodes have to be zeroed if this is the catalog tree and
+ * a corresponding flag in the volume header is set.
+ */
+bool hfs_bnode_need_zeroout(struct hfs_btree *tree)
+{
+ struct super_block *sb = tree->inode->i_sb;
+ struct hfsplus_sb_info *sbi = HFSPLUS_SB(sb);
+ const u32 volume_attr = be32_to_cpu(sbi->s_vhdr->attributes);
+
+ return tree->cnid == HFSPLUS_CAT_CNID &&
+ volume_attr & HFSPLUS_VOL_UNUSED_NODE_FIX;
+}
diff --git a/fs/hfsplus/btree.c b/fs/hfsplus/btree.c
index 0fcec8b..3345c75 100644
--- a/fs/hfsplus/btree.c
+++ b/fs/hfsplus/btree.c
@@ -358,7 +358,7 @@
u32 count;
int res;
- res = hfsplus_file_extend(inode);
+ res = hfsplus_file_extend(inode, hfs_bnode_need_zeroout(tree));
if (res)
return ERR_PTR(res);
hip->phys_size = inode->i_size =
diff --git a/fs/hfsplus/dir.c b/fs/hfsplus/dir.c
index bdec665..610a326 100644
--- a/fs/hfsplus/dir.c
+++ b/fs/hfsplus/dir.c
@@ -12,6 +12,7 @@
#include <linux/fs.h>
#include <linux/slab.h>
#include <linux/random.h>
+#include <linux/nls.h>
#include "hfsplus_fs.h"
#include "hfsplus_raw.h"
@@ -127,7 +128,7 @@
struct inode *inode = file_inode(file);
struct super_block *sb = inode->i_sb;
int len, err;
- char strbuf[HFSPLUS_MAX_STRLEN + 1];
+ char *strbuf;
hfsplus_cat_entry entry;
struct hfs_find_data fd;
struct hfsplus_readdir_data *rd;
@@ -139,6 +140,11 @@
err = hfs_find_init(HFSPLUS_SB(sb)->cat_tree, &fd);
if (err)
return err;
+ strbuf = kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_MAX_STRLEN + 1, GFP_KERNEL);
+ if (!strbuf) {
+ err = -ENOMEM;
+ goto out;
+ }
hfsplus_cat_build_key(sb, fd.search_key, inode->i_ino, NULL);
err = hfs_brec_find(&fd, hfs_find_rec_by_key);
if (err)
@@ -193,7 +199,7 @@
hfs_bnode_read(fd.bnode, &entry, fd.entryoffset,
fd.entrylength);
type = be16_to_cpu(entry.type);
- len = HFSPLUS_MAX_STRLEN;
+ len = NLS_MAX_CHARSET_SIZE * HFSPLUS_MAX_STRLEN;
err = hfsplus_uni2asc(sb, &fd.key->cat.name, strbuf, &len);
if (err)
goto out;
@@ -212,13 +218,31 @@
be32_to_cpu(entry.folder.id), DT_DIR))
break;
} else if (type == HFSPLUS_FILE) {
+ u16 mode;
+ unsigned type = DT_UNKNOWN;
+
if (fd.entrylength < sizeof(struct hfsplus_cat_file)) {
pr_err("small file entry\n");
err = -EIO;
goto out;
}
+
+ mode = be16_to_cpu(entry.file.permissions.mode);
+ if (S_ISREG(mode))
+ type = DT_REG;
+ else if (S_ISLNK(mode))
+ type = DT_LNK;
+ else if (S_ISFIFO(mode))
+ type = DT_FIFO;
+ else if (S_ISCHR(mode))
+ type = DT_CHR;
+ else if (S_ISBLK(mode))
+ type = DT_BLK;
+ else if (S_ISSOCK(mode))
+ type = DT_SOCK;
+
if (!dir_emit(ctx, strbuf, len,
- be32_to_cpu(entry.file.id), DT_REG))
+ be32_to_cpu(entry.file.id), type))
break;
} else {
pr_err("bad catalog entry type\n");
@@ -246,6 +270,7 @@
}
memcpy(&rd->key, fd.key, sizeof(struct hfsplus_cat_key));
out:
+ kfree(strbuf);
hfs_find_exit(&fd);
return err;
}
diff --git a/fs/hfsplus/extents.c b/fs/hfsplus/extents.c
index a7aafb35..feca524 100644
--- a/fs/hfsplus/extents.c
+++ b/fs/hfsplus/extents.c
@@ -235,7 +235,7 @@
if (iblock > hip->fs_blocks || !create)
return -EIO;
if (ablock >= hip->alloc_blocks) {
- res = hfsplus_file_extend(inode);
+ res = hfsplus_file_extend(inode, false);
if (res)
return res;
}
@@ -425,7 +425,7 @@
return res;
}
-int hfsplus_file_extend(struct inode *inode)
+int hfsplus_file_extend(struct inode *inode, bool zeroout)
{
struct super_block *sb = inode->i_sb;
struct hfsplus_sb_info *sbi = HFSPLUS_SB(sb);
@@ -436,10 +436,9 @@
if (sbi->alloc_file->i_size * 8 <
sbi->total_blocks - sbi->free_blocks + 8) {
/* extend alloc file */
- pr_err("extend alloc file! "
- "(%llu,%u,%u)\n",
- sbi->alloc_file->i_size * 8,
- sbi->total_blocks, sbi->free_blocks);
+ pr_err("extend alloc file! (%llu,%u,%u)\n",
+ sbi->alloc_file->i_size * 8,
+ sbi->total_blocks, sbi->free_blocks);
return -ENOSPC;
}
@@ -463,6 +462,12 @@
}
}
+ if (zeroout) {
+ res = sb_issue_zeroout(sb, start, len, GFP_NOFS);
+ if (res)
+ goto out;
+ }
+
hfs_dbg(EXTENT, "extend %lu: %u,%u\n", inode->i_ino, start, len);
if (hip->alloc_blocks <= hip->first_blocks) {
diff --git a/fs/hfsplus/hfsplus_fs.h b/fs/hfsplus/hfsplus_fs.h
index 83dc292..eb5e059 100644
--- a/fs/hfsplus/hfsplus_fs.h
+++ b/fs/hfsplus/hfsplus_fs.h
@@ -369,114 +369,119 @@
/* attributes.c */
int __init hfsplus_create_attr_tree_cache(void);
void hfsplus_destroy_attr_tree_cache(void);
+int hfsplus_attr_bin_cmp_key(const hfsplus_btree_key *k1,
+ const hfsplus_btree_key *k2);
+int hfsplus_attr_build_key(struct super_block *sb, hfsplus_btree_key *key,
+ u32 cnid, const char *name);
hfsplus_attr_entry *hfsplus_alloc_attr_entry(void);
-void hfsplus_destroy_attr_entry(hfsplus_attr_entry *entry_p);
-int hfsplus_attr_bin_cmp_key(const hfsplus_btree_key *,
- const hfsplus_btree_key *);
-int hfsplus_attr_build_key(struct super_block *, hfsplus_btree_key *,
- u32, const char *);
-void hfsplus_attr_build_key_uni(hfsplus_btree_key *key,
- u32 cnid,
- struct hfsplus_attr_unistr *name);
-int hfsplus_find_attr(struct super_block *, u32,
- const char *, struct hfs_find_data *);
+void hfsplus_destroy_attr_entry(hfsplus_attr_entry *entry);
+int hfsplus_find_attr(struct super_block *sb, u32 cnid, const char *name,
+ struct hfs_find_data *fd);
int hfsplus_attr_exists(struct inode *inode, const char *name);
-int hfsplus_create_attr(struct inode *, const char *, const void *, size_t);
-int hfsplus_delete_attr(struct inode *, const char *);
+int hfsplus_create_attr(struct inode *inode, const char *name,
+ const void *value, size_t size);
+int hfsplus_delete_attr(struct inode *inode, const char *name);
int hfsplus_delete_all_attrs(struct inode *dir, u32 cnid);
/* bitmap.c */
-int hfsplus_block_allocate(struct super_block *, u32, u32, u32 *);
-int hfsplus_block_free(struct super_block *, u32, u32);
+int hfsplus_block_allocate(struct super_block *sb, u32 size, u32 offset,
+ u32 *max);
+int hfsplus_block_free(struct super_block *sb, u32 offset, u32 count);
/* btree.c */
-u32 hfsplus_calc_btree_clump_size(u32, u32, u64, int);
-struct hfs_btree *hfs_btree_open(struct super_block *, u32);
-void hfs_btree_close(struct hfs_btree *);
-int hfs_btree_write(struct hfs_btree *);
-struct hfs_bnode *hfs_bmap_alloc(struct hfs_btree *);
-void hfs_bmap_free(struct hfs_bnode *);
+u32 hfsplus_calc_btree_clump_size(u32 block_size, u32 node_size, u64 sectors,
+ int file_id);
+struct hfs_btree *hfs_btree_open(struct super_block *sb, u32 id);
+void hfs_btree_close(struct hfs_btree *tree);
+int hfs_btree_write(struct hfs_btree *tree);
+struct hfs_bnode *hfs_bmap_alloc(struct hfs_btree *tree);
+void hfs_bmap_free(struct hfs_bnode *node);
/* bnode.c */
-void hfs_bnode_read(struct hfs_bnode *, void *, int, int);
-u16 hfs_bnode_read_u16(struct hfs_bnode *, int);
-u8 hfs_bnode_read_u8(struct hfs_bnode *, int);
-void hfs_bnode_read_key(struct hfs_bnode *, void *, int);
-void hfs_bnode_write(struct hfs_bnode *, void *, int, int);
-void hfs_bnode_write_u16(struct hfs_bnode *, int, u16);
-void hfs_bnode_clear(struct hfs_bnode *, int, int);
-void hfs_bnode_copy(struct hfs_bnode *, int,
- struct hfs_bnode *, int, int);
-void hfs_bnode_move(struct hfs_bnode *, int, int, int);
-void hfs_bnode_dump(struct hfs_bnode *);
-void hfs_bnode_unlink(struct hfs_bnode *);
-struct hfs_bnode *hfs_bnode_findhash(struct hfs_btree *, u32);
-struct hfs_bnode *hfs_bnode_find(struct hfs_btree *, u32);
-void hfs_bnode_unhash(struct hfs_bnode *);
-void hfs_bnode_free(struct hfs_bnode *);
-struct hfs_bnode *hfs_bnode_create(struct hfs_btree *, u32);
-void hfs_bnode_get(struct hfs_bnode *);
-void hfs_bnode_put(struct hfs_bnode *);
+void hfs_bnode_read(struct hfs_bnode *node, void *buf, int off, int len);
+u16 hfs_bnode_read_u16(struct hfs_bnode *node, int off);
+u8 hfs_bnode_read_u8(struct hfs_bnode *node, int off);
+void hfs_bnode_read_key(struct hfs_bnode *node, void *key, int off);
+void hfs_bnode_write(struct hfs_bnode *node, void *buf, int off, int len);
+void hfs_bnode_write_u16(struct hfs_bnode *node, int off, u16 data);
+void hfs_bnode_clear(struct hfs_bnode *node, int off, int len);
+void hfs_bnode_copy(struct hfs_bnode *dst_node, int dst,
+ struct hfs_bnode *src_node, int src, int len);
+void hfs_bnode_move(struct hfs_bnode *node, int dst, int src, int len);
+void hfs_bnode_dump(struct hfs_bnode *node);
+void hfs_bnode_unlink(struct hfs_bnode *node);
+struct hfs_bnode *hfs_bnode_findhash(struct hfs_btree *tree, u32 cnid);
+void hfs_bnode_unhash(struct hfs_bnode *node);
+struct hfs_bnode *hfs_bnode_find(struct hfs_btree *tree, u32 num);
+void hfs_bnode_free(struct hfs_bnode *node);
+struct hfs_bnode *hfs_bnode_create(struct hfs_btree *tree, u32 num);
+void hfs_bnode_get(struct hfs_bnode *node);
+void hfs_bnode_put(struct hfs_bnode *node);
+bool hfs_bnode_need_zeroout(struct hfs_btree *tree);
/* brec.c */
-u16 hfs_brec_lenoff(struct hfs_bnode *, u16, u16 *);
-u16 hfs_brec_keylen(struct hfs_bnode *, u16);
-int hfs_brec_insert(struct hfs_find_data *, void *, int);
-int hfs_brec_remove(struct hfs_find_data *);
+u16 hfs_brec_lenoff(struct hfs_bnode *node, u16 rec, u16 *off);
+u16 hfs_brec_keylen(struct hfs_bnode *node, u16 rec);
+int hfs_brec_insert(struct hfs_find_data *fd, void *entry, int entry_len);
+int hfs_brec_remove(struct hfs_find_data *fd);
/* bfind.c */
-int hfs_find_init(struct hfs_btree *, struct hfs_find_data *);
-void hfs_find_exit(struct hfs_find_data *);
-int hfs_find_1st_rec_by_cnid(struct hfs_bnode *,
- struct hfs_find_data *,
- int *, int *, int *);
-int hfs_find_rec_by_key(struct hfs_bnode *,
- struct hfs_find_data *,
- int *, int *, int *);
-int __hfs_brec_find(struct hfs_bnode *, struct hfs_find_data *,
- search_strategy_t);
-int hfs_brec_find(struct hfs_find_data *, search_strategy_t);
-int hfs_brec_read(struct hfs_find_data *, void *, int);
-int hfs_brec_goto(struct hfs_find_data *, int);
+int hfs_find_init(struct hfs_btree *tree, struct hfs_find_data *fd);
+void hfs_find_exit(struct hfs_find_data *fd);
+int hfs_find_1st_rec_by_cnid(struct hfs_bnode *bnode, struct hfs_find_data *fd,
+ int *begin, int *end, int *cur_rec);
+int hfs_find_rec_by_key(struct hfs_bnode *bnode, struct hfs_find_data *fd,
+ int *begin, int *end, int *cur_rec);
+int __hfs_brec_find(struct hfs_bnode *bnode, struct hfs_find_data *fd,
+ search_strategy_t rec_found);
+int hfs_brec_find(struct hfs_find_data *fd, search_strategy_t do_key_compare);
+int hfs_brec_read(struct hfs_find_data *fd, void *rec, int rec_len);
+int hfs_brec_goto(struct hfs_find_data *fd, int cnt);
/* catalog.c */
-int hfsplus_cat_case_cmp_key(const hfsplus_btree_key *,
- const hfsplus_btree_key *);
-int hfsplus_cat_bin_cmp_key(const hfsplus_btree_key *,
- const hfsplus_btree_key *);
-void hfsplus_cat_build_key(struct super_block *sb,
- hfsplus_btree_key *, u32, struct qstr *);
-int hfsplus_find_cat(struct super_block *, u32, struct hfs_find_data *);
-int hfsplus_create_cat(u32, struct inode *, struct qstr *, struct inode *);
-int hfsplus_delete_cat(u32, struct inode *, struct qstr *);
-int hfsplus_rename_cat(u32, struct inode *, struct qstr *,
- struct inode *, struct qstr *);
+int hfsplus_cat_case_cmp_key(const hfsplus_btree_key *k1,
+ const hfsplus_btree_key *k2);
+int hfsplus_cat_bin_cmp_key(const hfsplus_btree_key *k1,
+ const hfsplus_btree_key *k2);
+void hfsplus_cat_build_key(struct super_block *sb, hfsplus_btree_key *key,
+ u32 parent, struct qstr *str);
void hfsplus_cat_set_perms(struct inode *inode, struct hfsplus_perm *perms);
+int hfsplus_find_cat(struct super_block *sb, u32 cnid,
+ struct hfs_find_data *fd);
+int hfsplus_create_cat(u32 cnid, struct inode *dir, struct qstr *str,
+ struct inode *inode);
+int hfsplus_delete_cat(u32 cnid, struct inode *dir, struct qstr *str);
+int hfsplus_rename_cat(u32 cnid, struct inode *src_dir, struct qstr *src_name,
+ struct inode *dst_dir, struct qstr *dst_name);
/* dir.c */
extern const struct inode_operations hfsplus_dir_inode_operations;
extern const struct file_operations hfsplus_dir_operations;
/* extents.c */
-int hfsplus_ext_cmp_key(const hfsplus_btree_key *, const hfsplus_btree_key *);
-int hfsplus_ext_write_extent(struct inode *);
-int hfsplus_get_block(struct inode *, sector_t, struct buffer_head *, int);
-int hfsplus_free_fork(struct super_block *, u32,
- struct hfsplus_fork_raw *, int);
-int hfsplus_file_extend(struct inode *);
-void hfsplus_file_truncate(struct inode *);
+int hfsplus_ext_cmp_key(const hfsplus_btree_key *k1,
+ const hfsplus_btree_key *k2);
+int hfsplus_ext_write_extent(struct inode *inode);
+int hfsplus_get_block(struct inode *inode, sector_t iblock,
+ struct buffer_head *bh_result, int create);
+int hfsplus_free_fork(struct super_block *sb, u32 cnid,
+ struct hfsplus_fork_raw *fork, int type);
+int hfsplus_file_extend(struct inode *inode, bool zeroout);
+void hfsplus_file_truncate(struct inode *inode);
/* inode.c */
extern const struct address_space_operations hfsplus_aops;
extern const struct address_space_operations hfsplus_btree_aops;
extern const struct dentry_operations hfsplus_dentry_operations;
-void hfsplus_inode_read_fork(struct inode *, struct hfsplus_fork_raw *);
-void hfsplus_inode_write_fork(struct inode *, struct hfsplus_fork_raw *);
-int hfsplus_cat_read_inode(struct inode *, struct hfs_find_data *);
-int hfsplus_cat_write_inode(struct inode *);
-struct inode *hfsplus_new_inode(struct super_block *, umode_t);
-void hfsplus_delete_inode(struct inode *);
+struct inode *hfsplus_new_inode(struct super_block *sb, umode_t mode);
+void hfsplus_delete_inode(struct inode *inode);
+void hfsplus_inode_read_fork(struct inode *inode,
+ struct hfsplus_fork_raw *fork);
+void hfsplus_inode_write_fork(struct inode *inode,
+ struct hfsplus_fork_raw *fork);
+int hfsplus_cat_read_inode(struct inode *inode, struct hfs_find_data *fd);
+int hfsplus_cat_write_inode(struct inode *inode);
int hfsplus_file_fsync(struct file *file, loff_t start, loff_t end,
int datasync);
@@ -484,13 +489,17 @@
long hfsplus_ioctl(struct file *filp, unsigned int cmd, unsigned long arg);
/* options.c */
-int hfsplus_parse_options(char *, struct hfsplus_sb_info *);
+void hfsplus_fill_defaults(struct hfsplus_sb_info *opts);
int hfsplus_parse_options_remount(char *input, int *force);
-void hfsplus_fill_defaults(struct hfsplus_sb_info *);
-int hfsplus_show_options(struct seq_file *, struct dentry *);
+int hfsplus_parse_options(char *input, struct hfsplus_sb_info *sbi);
+int hfsplus_show_options(struct seq_file *seq, struct dentry *root);
+
+/* part_tbl.c */
+int hfs_part_find(struct super_block *sb, sector_t *part_start,
+ sector_t *part_size);
/* super.c */
-struct inode *hfsplus_iget(struct super_block *, unsigned long);
+struct inode *hfsplus_iget(struct super_block *sb, unsigned long ino);
void hfsplus_mark_mdb_dirty(struct super_block *sb);
/* tables.c */
@@ -499,23 +508,23 @@
extern u16 hfsplus_compose_table[];
/* unicode.c */
-int hfsplus_strcasecmp(const struct hfsplus_unistr *,
- const struct hfsplus_unistr *);
-int hfsplus_strcmp(const struct hfsplus_unistr *,
- const struct hfsplus_unistr *);
-int hfsplus_uni2asc(struct super_block *,
- const struct hfsplus_unistr *, char *, int *);
-int hfsplus_asc2uni(struct super_block *,
- struct hfsplus_unistr *, int, const char *, int);
+int hfsplus_strcasecmp(const struct hfsplus_unistr *s1,
+ const struct hfsplus_unistr *s2);
+int hfsplus_strcmp(const struct hfsplus_unistr *s1,
+ const struct hfsplus_unistr *s2);
+int hfsplus_uni2asc(struct super_block *sb, const struct hfsplus_unistr *ustr,
+ char *astr, int *len_p);
+int hfsplus_asc2uni(struct super_block *sb, struct hfsplus_unistr *ustr,
+ int max_unistr_len, const char *astr, int len);
int hfsplus_hash_dentry(const struct dentry *dentry, struct qstr *str);
-int hfsplus_compare_dentry(const struct dentry *parent, const struct dentry *dentry,
- unsigned int len, const char *str, const struct qstr *name);
+int hfsplus_compare_dentry(const struct dentry *parent,
+ const struct dentry *dentry, unsigned int len,
+ const char *str, const struct qstr *name);
/* wrapper.c */
-int hfsplus_read_wrapper(struct super_block *);
-int hfs_part_find(struct super_block *, sector_t *, sector_t *);
-int hfsplus_submit_bio(struct super_block *sb, sector_t sector,
- void *buf, void **data, int rw);
+int hfsplus_submit_bio(struct super_block *sb, sector_t sector, void *buf,
+ void **data, int rw);
+int hfsplus_read_wrapper(struct super_block *sb);
/* time macros */
#define __hfsp_mt2ut(t) (be32_to_cpu(t) - 2082844800U)
diff --git a/fs/hfsplus/hfsplus_raw.h b/fs/hfsplus/hfsplus_raw.h
index 5a12682..8298d09 100644
--- a/fs/hfsplus/hfsplus_raw.h
+++ b/fs/hfsplus/hfsplus_raw.h
@@ -144,6 +144,7 @@
#define HFSPLUS_VOL_NODEID_REUSED (1 << 12)
#define HFSPLUS_VOL_JOURNALED (1 << 13)
#define HFSPLUS_VOL_SOFTLOCK (1 << 15)
+#define HFSPLUS_VOL_UNUSED_NODE_FIX (1 << 31)
/* HFS+ BTree node descriptor */
struct hfs_bnode_desc {
diff --git a/fs/hfsplus/options.c b/fs/hfsplus/options.c
index 68537e8..c90b72e 100644
--- a/fs/hfsplus/options.c
+++ b/fs/hfsplus/options.c
@@ -173,9 +173,8 @@
if (p)
sbi->nls = load_nls(p);
if (!sbi->nls) {
- pr_err("unable to load "
- "nls mapping \"%s\"\n",
- p);
+ pr_err("unable to load nls mapping \"%s\"\n",
+ p);
kfree(p);
return 0;
}
@@ -232,8 +231,8 @@
if (sbi->nls)
seq_printf(seq, ",nls=%s", sbi->nls->charset);
if (test_bit(HFSPLUS_SB_NODECOMPOSE, &sbi->flags))
- seq_printf(seq, ",nodecompose");
+ seq_puts(seq, ",nodecompose");
if (test_bit(HFSPLUS_SB_NOBARRIER, &sbi->flags))
- seq_printf(seq, ",nobarrier");
+ seq_puts(seq, ",nobarrier");
return 0;
}
diff --git a/fs/hfsplus/super.c b/fs/hfsplus/super.c
index a513d2d..4cf2024 100644
--- a/fs/hfsplus/super.c
+++ b/fs/hfsplus/super.c
@@ -131,9 +131,10 @@
hfsplus_inode_write_fork(inode, fork);
if (tree) {
int err = hfs_btree_write(tree);
+
if (err) {
pr_err("b-tree write err: %d, ino %lu\n",
- err, inode->i_ino);
+ err, inode->i_ino);
return err;
}
}
diff --git a/fs/hfsplus/wrapper.c b/fs/hfsplus/wrapper.c
index 3f99964..cc62356 100644
--- a/fs/hfsplus/wrapper.c
+++ b/fs/hfsplus/wrapper.c
@@ -24,8 +24,8 @@
u16 embed_count;
};
-/*
- * hfsplus_submit_bio - Perfrom block I/O
+/**
+ * hfsplus_submit_bio - Perform block I/O
* @sb: super block of volume for I/O
* @sector: block to read or write, for blocks of HFSPLUS_SECTOR_SIZE bytes
* @buf: buffer for I/O
@@ -231,10 +231,8 @@
if (blocksize < HFSPLUS_SECTOR_SIZE || ((blocksize - 1) & blocksize))
goto out_free_backup_vhdr;
sbi->alloc_blksz = blocksize;
- sbi->alloc_blksz_shift = 0;
- while ((blocksize >>= 1) != 0)
- sbi->alloc_blksz_shift++;
- blocksize = min(sbi->alloc_blksz, (u32)PAGE_SIZE);
+ sbi->alloc_blksz_shift = ilog2(blocksize);
+ blocksize = min_t(u32, sbi->alloc_blksz, PAGE_SIZE);
/*
* Align block size to block offset.
diff --git a/fs/hfsplus/xattr.c b/fs/hfsplus/xattr.c
index 4e27edc..d98094a 100644
--- a/fs/hfsplus/xattr.c
+++ b/fs/hfsplus/xattr.c
@@ -8,6 +8,7 @@
#include "hfsplus_fs.h"
#include <linux/posix_acl_xattr.h>
+#include <linux/nls.h>
#include "xattr.h"
#include "acl.h"
@@ -66,10 +67,10 @@
char *bmp;
u32 used_nodes;
u32 used_bmp_bytes;
- loff_t tmp;
+ u64 tmp;
hfs_dbg(ATTR_MOD, "init_hdr_attr_file: clump %u, node_size %u\n",
- clump_size, node_size);
+ clump_size, node_size);
/* The end of the node contains list of record offsets */
rec_offsets = (__be16 *)(buf + node_size);
@@ -195,7 +196,7 @@
}
while (hip->alloc_blocks < hip->clump_blocks) {
- err = hfsplus_file_extend(attr_file);
+ err = hfsplus_file_extend(attr_file, false);
if (unlikely(err)) {
pr_err("failed to extend attributes file\n");
goto end_attr_file_creation;
@@ -645,8 +646,7 @@
struct hfs_find_data fd;
u16 key_len = 0;
struct hfsplus_attr_key attr_key;
- char strbuf[HFSPLUS_ATTR_MAX_STRLEN +
- XATTR_MAC_OSX_PREFIX_LEN + 1] = {0};
+ char *strbuf;
int xattr_name_len;
if ((!S_ISREG(inode->i_mode) &&
@@ -666,6 +666,13 @@
return err;
}
+ strbuf = kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MAX_STRLEN +
+ XATTR_MAC_OSX_PREFIX_LEN + 1, GFP_KERNEL);
+ if (!strbuf) {
+ res = -ENOMEM;
+ goto out;
+ }
+
err = hfsplus_find_attr(inode->i_sb, inode->i_ino, NULL, &fd);
if (err) {
if (err == -ENOENT) {
@@ -692,7 +699,7 @@
if (be32_to_cpu(attr_key.cnid) != inode->i_ino)
goto end_listxattr;
- xattr_name_len = HFSPLUS_ATTR_MAX_STRLEN;
+ xattr_name_len = NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MAX_STRLEN;
if (hfsplus_uni2asc(inode->i_sb,
(const struct hfsplus_unistr *)&fd.key->attr.key_name,
strbuf, &xattr_name_len)) {
@@ -718,6 +725,8 @@
}
end_listxattr:
+ kfree(strbuf);
+out:
hfs_find_exit(&fd);
return res;
}
@@ -797,47 +806,55 @@
static int hfsplus_osx_getxattr(struct dentry *dentry, const char *name,
void *buffer, size_t size, int type)
{
- char xattr_name[HFSPLUS_ATTR_MAX_STRLEN +
- XATTR_MAC_OSX_PREFIX_LEN + 1] = {0};
- size_t len = strlen(name);
+ char *xattr_name;
+ int res;
if (!strcmp(name, ""))
return -EINVAL;
- if (len > HFSPLUS_ATTR_MAX_STRLEN)
- return -EOPNOTSUPP;
-
/*
* Don't allow retrieving properly prefixed attributes
* by prepending them with "osx."
*/
if (is_known_namespace(name))
return -EOPNOTSUPP;
+ xattr_name = kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MAX_STRLEN
+ + XATTR_MAC_OSX_PREFIX_LEN + 1, GFP_KERNEL);
+ if (!xattr_name)
+ return -ENOMEM;
+ strcpy(xattr_name, XATTR_MAC_OSX_PREFIX);
+ strcpy(xattr_name + XATTR_MAC_OSX_PREFIX_LEN, name);
- return hfsplus_getxattr(dentry, xattr_name, buffer, size);
+ res = hfsplus_getxattr(dentry, xattr_name, buffer, size);
+ kfree(xattr_name);
+ return res;
}
static int hfsplus_osx_setxattr(struct dentry *dentry, const char *name,
const void *buffer, size_t size, int flags, int type)
{
- char xattr_name[HFSPLUS_ATTR_MAX_STRLEN +
- XATTR_MAC_OSX_PREFIX_LEN + 1] = {0};
- size_t len = strlen(name);
+ char *xattr_name;
+ int res;
if (!strcmp(name, ""))
return -EINVAL;
- if (len > HFSPLUS_ATTR_MAX_STRLEN)
- return -EOPNOTSUPP;
-
/*
* Don't allow setting properly prefixed attributes
* by prepending them with "osx."
*/
if (is_known_namespace(name))
return -EOPNOTSUPP;
+ xattr_name = kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MAX_STRLEN
+ + XATTR_MAC_OSX_PREFIX_LEN + 1, GFP_KERNEL);
+ if (!xattr_name)
+ return -ENOMEM;
+ strcpy(xattr_name, XATTR_MAC_OSX_PREFIX);
+ strcpy(xattr_name + XATTR_MAC_OSX_PREFIX_LEN, name);
- return hfsplus_setxattr(dentry, xattr_name, buffer, size, flags);
+ res = hfsplus_setxattr(dentry, xattr_name, buffer, size, flags);
+ kfree(xattr_name);
+ return res;
}
static size_t hfsplus_osx_listxattr(struct dentry *dentry, char *list,
diff --git a/fs/hfsplus/xattr_security.c b/fs/hfsplus/xattr_security.c
index 0072276..6ec5e10 100644
--- a/fs/hfsplus/xattr_security.c
+++ b/fs/hfsplus/xattr_security.c
@@ -7,6 +7,8 @@
*/
#include <linux/security.h>
+#include <linux/nls.h>
+
#include "hfsplus_fs.h"
#include "xattr.h"
#include "acl.h"
@@ -14,37 +16,43 @@
static int hfsplus_security_getxattr(struct dentry *dentry, const char *name,
void *buffer, size_t size, int type)
{
- char xattr_name[HFSPLUS_ATTR_MAX_STRLEN + 1] = {0};
- size_t len = strlen(name);
+ char *xattr_name;
+ int res;
if (!strcmp(name, ""))
return -EINVAL;
- if (len + XATTR_SECURITY_PREFIX_LEN > HFSPLUS_ATTR_MAX_STRLEN)
- return -EOPNOTSUPP;
-
+ xattr_name = kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MAX_STRLEN + 1,
+ GFP_KERNEL);
+ if (!xattr_name)
+ return -ENOMEM;
strcpy(xattr_name, XATTR_SECURITY_PREFIX);
strcpy(xattr_name + XATTR_SECURITY_PREFIX_LEN, name);
- return hfsplus_getxattr(dentry, xattr_name, buffer, size);
+ res = hfsplus_getxattr(dentry, xattr_name, buffer, size);
+ kfree(xattr_name);
+ return res;
}
static int hfsplus_security_setxattr(struct dentry *dentry, const char *name,
const void *buffer, size_t size, int flags, int type)
{
- char xattr_name[HFSPLUS_ATTR_MAX_STRLEN + 1] = {0};
- size_t len = strlen(name);
+ char *xattr_name;
+ int res;
if (!strcmp(name, ""))
return -EINVAL;
- if (len + XATTR_SECURITY_PREFIX_LEN > HFSPLUS_ATTR_MAX_STRLEN)
- return -EOPNOTSUPP;
-
+ xattr_name = kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MAX_STRLEN + 1,
+ GFP_KERNEL);
+ if (!xattr_name)
+ return -ENOMEM;
strcpy(xattr_name, XATTR_SECURITY_PREFIX);
strcpy(xattr_name + XATTR_SECURITY_PREFIX_LEN, name);
- return hfsplus_setxattr(dentry, xattr_name, buffer, size, flags);
+ res = hfsplus_setxattr(dentry, xattr_name, buffer, size, flags);
+ kfree(xattr_name);
+ return res;
}
static size_t hfsplus_security_listxattr(struct dentry *dentry, char *list,
@@ -62,31 +70,30 @@
void *fs_info)
{
const struct xattr *xattr;
- char xattr_name[HFSPLUS_ATTR_MAX_STRLEN + 1] = {0};
- size_t xattr_name_len;
+ char *xattr_name;
int err = 0;
+ xattr_name = kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MAX_STRLEN + 1,
+ GFP_KERNEL);
+ if (!xattr_name)
+ return -ENOMEM;
for (xattr = xattr_array; xattr->name != NULL; xattr++) {
- xattr_name_len = strlen(xattr->name);
- if (xattr_name_len == 0)
+ if (!strcmp(xattr->name, ""))
continue;
- if (xattr_name_len + XATTR_SECURITY_PREFIX_LEN >
- HFSPLUS_ATTR_MAX_STRLEN)
- return -EOPNOTSUPP;
-
strcpy(xattr_name, XATTR_SECURITY_PREFIX);
strcpy(xattr_name +
XATTR_SECURITY_PREFIX_LEN, xattr->name);
memset(xattr_name +
- XATTR_SECURITY_PREFIX_LEN + xattr_name_len, 0, 1);
+ XATTR_SECURITY_PREFIX_LEN + strlen(xattr->name), 0, 1);
err = __hfsplus_setxattr(inode, xattr_name,
xattr->value, xattr->value_len, 0);
if (err)
break;
}
+ kfree(xattr_name);
return err;
}
diff --git a/fs/hfsplus/xattr_trusted.c b/fs/hfsplus/xattr_trusted.c
index 426cee2..3c5f27e 100644
--- a/fs/hfsplus/xattr_trusted.c
+++ b/fs/hfsplus/xattr_trusted.c
@@ -6,43 +6,51 @@
* Handler for trusted extended attributes.
*/
+#include <linux/nls.h>
+
#include "hfsplus_fs.h"
#include "xattr.h"
static int hfsplus_trusted_getxattr(struct dentry *dentry, const char *name,
void *buffer, size_t size, int type)
{
- char xattr_name[HFSPLUS_ATTR_MAX_STRLEN + 1] = {0};
- size_t len = strlen(name);
+ char *xattr_name;
+ int res;
if (!strcmp(name, ""))
return -EINVAL;
- if (len + XATTR_TRUSTED_PREFIX_LEN > HFSPLUS_ATTR_MAX_STRLEN)
- return -EOPNOTSUPP;
-
+ xattr_name = kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MAX_STRLEN + 1,
+ GFP_KERNEL);
+ if (!xattr_name)
+ return -ENOMEM;
strcpy(xattr_name, XATTR_TRUSTED_PREFIX);
strcpy(xattr_name + XATTR_TRUSTED_PREFIX_LEN, name);
- return hfsplus_getxattr(dentry, xattr_name, buffer, size);
+ res = hfsplus_getxattr(dentry, xattr_name, buffer, size);
+ kfree(xattr_name);
+ return res;
}
static int hfsplus_trusted_setxattr(struct dentry *dentry, const char *name,
const void *buffer, size_t size, int flags, int type)
{
- char xattr_name[HFSPLUS_ATTR_MAX_STRLEN + 1] = {0};
- size_t len = strlen(name);
+ char *xattr_name;
+ int res;
if (!strcmp(name, ""))
return -EINVAL;
- if (len + XATTR_TRUSTED_PREFIX_LEN > HFSPLUS_ATTR_MAX_STRLEN)
- return -EOPNOTSUPP;
-
+ xattr_name = kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MAX_STRLEN + 1,
+ GFP_KERNEL);
+ if (!xattr_name)
+ return -ENOMEM;
strcpy(xattr_name, XATTR_TRUSTED_PREFIX);
strcpy(xattr_name + XATTR_TRUSTED_PREFIX_LEN, name);
- return hfsplus_setxattr(dentry, xattr_name, buffer, size, flags);
+ res = hfsplus_setxattr(dentry, xattr_name, buffer, size, flags);
+ kfree(xattr_name);
+ return res;
}
static size_t hfsplus_trusted_listxattr(struct dentry *dentry, char *list,
diff --git a/fs/hfsplus/xattr_user.c b/fs/hfsplus/xattr_user.c
index e340165..2b625a5 100644
--- a/fs/hfsplus/xattr_user.c
+++ b/fs/hfsplus/xattr_user.c
@@ -6,43 +6,51 @@
* Handler for user extended attributes.
*/
+#include <linux/nls.h>
+
#include "hfsplus_fs.h"
#include "xattr.h"
static int hfsplus_user_getxattr(struct dentry *dentry, const char *name,
void *buffer, size_t size, int type)
{
- char xattr_name[HFSPLUS_ATTR_MAX_STRLEN + 1] = {0};
- size_t len = strlen(name);
+ char *xattr_name;
+ int res;
if (!strcmp(name, ""))
return -EINVAL;
- if (len + XATTR_USER_PREFIX_LEN > HFSPLUS_ATTR_MAX_STRLEN)
- return -EOPNOTSUPP;
-
+ xattr_name = kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MAX_STRLEN + 1,
+ GFP_KERNEL);
+ if (!xattr_name)
+ return -ENOMEM;
strcpy(xattr_name, XATTR_USER_PREFIX);
strcpy(xattr_name + XATTR_USER_PREFIX_LEN, name);
- return hfsplus_getxattr(dentry, xattr_name, buffer, size);
+ res = hfsplus_getxattr(dentry, xattr_name, buffer, size);
+ kfree(xattr_name);
+ return res;
}
static int hfsplus_user_setxattr(struct dentry *dentry, const char *name,
const void *buffer, size_t size, int flags, int type)
{
- char xattr_name[HFSPLUS_ATTR_MAX_STRLEN + 1] = {0};
- size_t len = strlen(name);
+ char *xattr_name;
+ int res;
if (!strcmp(name, ""))
return -EINVAL;
- if (len + XATTR_USER_PREFIX_LEN > HFSPLUS_ATTR_MAX_STRLEN)
- return -EOPNOTSUPP;
-
+ xattr_name = kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MAX_STRLEN + 1,
+ GFP_KERNEL);
+ if (!xattr_name)
+ return -ENOMEM;
strcpy(xattr_name, XATTR_USER_PREFIX);
strcpy(xattr_name + XATTR_USER_PREFIX_LEN, name);
- return hfsplus_setxattr(dentry, xattr_name, buffer, size, flags);
+ res = hfsplus_setxattr(dentry, xattr_name, buffer, size, flags);
+ kfree(xattr_name);
+ return res;
}
static size_t hfsplus_user_listxattr(struct dentry *dentry, char *list,
diff --git a/fs/hpfs/alloc.c b/fs/hpfs/alloc.c
index 58b5106..f005046 100644
--- a/fs/hpfs/alloc.c
+++ b/fs/hpfs/alloc.c
@@ -316,7 +316,7 @@
struct quad_buffer_head qbh;
__le32 *bmp;
struct hpfs_sb_info *sbi = hpfs_sb(s);
- /*printk("2 - ");*/
+ /*pr_info("2 - ");*/
if (!n) return;
if (sec < 0x12) {
hpfs_error(s, "Trying to free reserved sector %08x", sec);
diff --git a/fs/hpfs/buffer.c b/fs/hpfs/buffer.c
index 139ef16..8057fe4 100644
--- a/fs/hpfs/buffer.c
+++ b/fs/hpfs/buffer.c
@@ -55,7 +55,7 @@
if (bh != NULL)
return bh->b_data;
else {
- printk("HPFS: hpfs_map_sector: read error\n");
+ pr_err("%s(): read error\n", __func__);
return NULL;
}
}
@@ -76,7 +76,7 @@
set_buffer_uptodate(bh);
return bh->b_data;
} else {
- printk("HPFS: hpfs_get_sector: getblk failed\n");
+ pr_err("%s(): getblk failed\n", __func__);
return NULL;
}
}
@@ -93,7 +93,7 @@
cond_resched();
if (secno & 3) {
- printk("HPFS: hpfs_map_4sectors: unaligned read\n");
+ pr_err("%s(): unaligned read\n", __func__);
return NULL;
}
@@ -112,7 +112,7 @@
qbh->data = data = kmalloc(2048, GFP_NOFS);
if (!data) {
- printk("HPFS: hpfs_map_4sectors: out of memory\n");
+ pr_err("%s(): out of memory\n", __func__);
goto bail4;
}
@@ -145,7 +145,7 @@
hpfs_lock_assert(s);
if (secno & 3) {
- printk("HPFS: hpfs_get_4sectors: unaligned read\n");
+ pr_err("%s(): unaligned read\n", __func__);
return NULL;
}
@@ -161,7 +161,7 @@
}
if (!(qbh->data = kmalloc(2048, GFP_NOFS))) {
- printk("HPFS: hpfs_get_4sectors: out of memory\n");
+ pr_err("%s(): out of memory\n", __func__);
goto bail4;
}
return qbh->data;
diff --git a/fs/hpfs/dir.c b/fs/hpfs/dir.c
index 292b1ac..2a8e074 100644
--- a/fs/hpfs/dir.c
+++ b/fs/hpfs/dir.c
@@ -36,7 +36,7 @@
mutex_lock(&i->i_mutex);
hpfs_lock(s);
- /*printk("dir lseek\n");*/
+ /*pr_info("dir lseek\n");*/
if (new_off == 0 || new_off == 1 || new_off == 11 || new_off == 12 || new_off == 13) goto ok;
pos = ((loff_t) hpfs_de_as_down_as_possible(s, hpfs_inode->i_dno) << 4) + 1;
while (pos != new_off) {
@@ -51,7 +51,7 @@
mutex_unlock(&i->i_mutex);
return new_off;
fail:
- /*printk("illegal lseek: %016llx\n", new_off);*/
+ /*pr_warn("illegal lseek: %016llx\n", new_off);*/
hpfs_unlock(s);
mutex_unlock(&i->i_mutex);
return -ESPIPE;
@@ -127,7 +127,7 @@
if (ctx->pos == 12)
goto out;
if (ctx->pos == 3 || ctx->pos == 4 || ctx->pos == 5) {
- printk("HPFS: warning: pos==%d\n",(int)ctx->pos);
+ pr_err("pos==%d\n", (int)ctx->pos);
goto out;
}
if (ctx->pos == 0) {
diff --git a/fs/hpfs/dnode.c b/fs/hpfs/dnode.c
index 4364b2a0..f36fc01 100644
--- a/fs/hpfs/dnode.c
+++ b/fs/hpfs/dnode.c
@@ -17,7 +17,7 @@
if (de == fde) return ((loff_t) le32_to_cpu(d->self) << 4) | (loff_t)i;
i++;
}
- printk("HPFS: get_pos: not_found\n");
+ pr_info("%s(): not_found\n", __func__);
return ((loff_t)le32_to_cpu(d->self) << 4) | (loff_t)1;
}
@@ -32,7 +32,7 @@
if (hpfs_inode->i_rddir_off[i] == pos) return;
if (!(i&0x0f)) {
if (!(ppos = kmalloc((i+0x11) * sizeof(loff_t*), GFP_NOFS))) {
- printk("HPFS: out of memory for position list\n");
+ pr_err("out of memory for position list\n");
return;
}
if (hpfs_inode->i_rddir_off) {
@@ -63,7 +63,8 @@
}
return;
not_f:
- /*printk("HPFS: warning: position pointer %p->%08x not found\n", pos, (int)*pos);*/
+ /*pr_warn("position pointer %p->%08x not found\n",
+ pos, (int)*pos);*/
return;
}
@@ -92,8 +93,11 @@
{
if ((*p & ~0x3f) == (d & ~0x3f) && (*p & 0x3f) >= (d & 0x3f)) {
int n = (*p & 0x3f) + c;
- if (n > 0x3f) printk("HPFS: hpfs_pos_ins: %08x + %d\n", (int)*p, (int)c >> 8);
- else *p = (*p & ~0x3f) | n;
+ if (n > 0x3f)
+ pr_err("%s(): %08x + %d\n",
+ __func__, (int)*p, (int)c >> 8);
+ else
+ *p = (*p & ~0x3f) | n;
}
}
@@ -101,8 +105,11 @@
{
if ((*p & ~0x3f) == (d & ~0x3f) && (*p & 0x3f) >= (d & 0x3f)) {
int n = (*p & 0x3f) - c;
- if (n < 1) printk("HPFS: hpfs_pos_ins: %08x - %d\n", (int)*p, (int)c >> 8);
- else *p = (*p & ~0x3f) | n;
+ if (n < 1)
+ pr_err("%s(): %08x - %d\n",
+ __func__, (int)*p, (int)c >> 8);
+ else
+ *p = (*p & ~0x3f) | n;
}
}
@@ -239,12 +246,12 @@
struct fnode *fnode;
int c1, c2 = 0;
if (!(nname = kmalloc(256, GFP_NOFS))) {
- printk("HPFS: out of memory, can't add to dnode\n");
+ pr_err("out of memory, can't add to dnode\n");
return 1;
}
go_up:
if (namelen >= 256) {
- hpfs_error(i->i_sb, "hpfs_add_to_dnode: namelen == %d", namelen);
+ hpfs_error(i->i_sb, "%s(): namelen == %d", __func__, namelen);
kfree(nd);
kfree(nname);
return 1;
@@ -281,7 +288,7 @@
not be any error while splitting dnodes, otherwise the
whole directory, not only file we're adding, would
be lost. */
- printk("HPFS: out of memory for dnode splitting\n");
+ pr_err("out of memory for dnode splitting\n");
hpfs_brelse4(&qbh);
kfree(nname);
return 1;
@@ -597,7 +604,7 @@
if (!de_next->down) goto endm;
ndown = de_down_pointer(de_next);
if (!(de_cp = kmalloc(le16_to_cpu(de->length), GFP_NOFS))) {
- printk("HPFS: out of memory for dtree balancing\n");
+ pr_err("out of memory for dtree balancing\n");
goto endm;
}
memcpy(de_cp, de, le16_to_cpu(de->length));
@@ -612,7 +619,8 @@
hpfs_brelse4(&qbh1);
}
hpfs_add_to_dnode(i, ndown, de_cp->name, de_cp->namelen, de_cp, de_cp->down ? de_down_pointer(de_cp) : 0);
- /*printk("UP-TO-DNODE: %08x (ndown = %08x, down = %08x, dno = %08x)\n", up, ndown, down, dno);*/
+ /*pr_info("UP-TO-DNODE: %08x (ndown = %08x, down = %08x, dno = %08x)\n",
+ up, ndown, down, dno);*/
dno = up;
kfree(de_cp);
goto try_it_again;
@@ -637,15 +645,15 @@
if (!dlp && down) {
if (le32_to_cpu(d1->first_free) > 2044) {
if (hpfs_sb(i->i_sb)->sb_chk >= 2) {
- printk("HPFS: warning: unbalanced dnode tree, see hpfs.txt 4 more info\n");
- printk("HPFS: warning: terminating balancing operation\n");
+ pr_err("unbalanced dnode tree, see hpfs.txt 4 more info\n");
+ pr_err("terminating balancing operation\n");
}
hpfs_brelse4(&qbh1);
goto endm;
}
if (hpfs_sb(i->i_sb)->sb_chk >= 2) {
- printk("HPFS: warning: unbalanced dnode tree, see hpfs.txt 4 more info\n");
- printk("HPFS: warning: goin'on\n");
+ pr_err("unbalanced dnode tree, see hpfs.txt 4 more info\n");
+ pr_err("goin'on\n");
}
le16_add_cpu(&del->length, 4);
del->down = 1;
@@ -659,7 +667,7 @@
*(__le32 *) ((void *) del + le16_to_cpu(del->length) - 4) = cpu_to_le32(down);
} else goto endm;
if (!(de_cp = kmalloc(le16_to_cpu(de_prev->length), GFP_NOFS))) {
- printk("HPFS: out of memory for dtree balancing\n");
+ pr_err("out of memory for dtree balancing\n");
hpfs_brelse4(&qbh1);
goto endm;
}
@@ -1000,7 +1008,7 @@
int d1, d2 = 0;
name1 = f->name;
if (!(name2 = kmalloc(256, GFP_NOFS))) {
- printk("HPFS: out of memory, can't map dirent\n");
+ pr_err("out of memory, can't map dirent\n");
return NULL;
}
if (f->len <= 15)
diff --git a/fs/hpfs/ea.c b/fs/hpfs/ea.c
index bcaafcd..ce3f98b 100644
--- a/fs/hpfs/ea.c
+++ b/fs/hpfs/ea.c
@@ -51,7 +51,7 @@
{
char *ret;
if (!(ret = kmalloc(size + 1, GFP_NOFS))) {
- printk("HPFS: out of memory for EA\n");
+ pr_err("out of memory for EA\n");
return NULL;
}
if (hpfs_ea_read(s, a, ano, 0, size, ret)) {
@@ -139,7 +139,7 @@
if (ea_indirect(ea))
return get_indirect_ea(s, ea_in_anode(ea), ea_sec(ea), *size = ea_len(ea));
if (!(ret = kmalloc((*size = ea_valuelen(ea)) + 1, GFP_NOFS))) {
- printk("HPFS: out of memory for EA\n");
+ pr_err("out of memory for EA\n");
return NULL;
}
memcpy(ret, ea_data(ea), ea_valuelen(ea));
@@ -165,7 +165,7 @@
if (ea_indirect(ea))
return get_indirect_ea(s, ea_in_anode(ea), ea_sec(ea), *size = ea_len(ea));
if (!(ret = kmalloc((*size = ea_valuelen(ea)) + 1, GFP_NOFS))) {
- printk("HPFS: out of memory for EA\n");
+ pr_err("out of memory for EA\n");
return NULL;
}
if (hpfs_ea_read(s, a, ano, pos + 4 + ea->namelen + 1, ea_valuelen(ea), ret)) {
diff --git a/fs/hpfs/hpfs_fn.h b/fs/hpfs/hpfs_fn.h
index 3ba49c0..b63b75f 100644
--- a/fs/hpfs/hpfs_fn.h
+++ b/fs/hpfs/hpfs_fn.h
@@ -8,6 +8,11 @@
//#define DBG
//#define DEBUG_LOCKS
+#ifdef pr_fmt
+#undef pr_fmt
+#endif
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/mutex.h>
#include <linux/pagemap.h>
diff --git a/fs/hpfs/inode.c b/fs/hpfs/inode.c
index 50a4273..7ce4b74 100644
--- a/fs/hpfs/inode.c
+++ b/fs/hpfs/inode.c
@@ -183,7 +183,8 @@
struct inode *parent;
if (i->i_ino == hpfs_sb(i->i_sb)->sb_root) return;
if (hpfs_inode->i_rddir_off && !atomic_read(&i->i_count)) {
- if (*hpfs_inode->i_rddir_off) printk("HPFS: write_inode: some position still there\n");
+ if (*hpfs_inode->i_rddir_off)
+ pr_err("write_inode: some position still there\n");
kfree(hpfs_inode->i_rddir_off);
hpfs_inode->i_rddir_off = NULL;
}
diff --git a/fs/hpfs/map.c b/fs/hpfs/map.c
index 3aa66ae..442770e 100644
--- a/fs/hpfs/map.c
+++ b/fs/hpfs/map.c
@@ -65,12 +65,13 @@
struct code_page_directory *cp = hpfs_map_sector(s, cps, &bh, 0);
if (!cp) return NULL;
if (le32_to_cpu(cp->magic) != CP_DIR_MAGIC) {
- printk("HPFS: Code page directory magic doesn't match (magic = %08x)\n", le32_to_cpu(cp->magic));
+ pr_err("Code page directory magic doesn't match (magic = %08x)\n",
+ le32_to_cpu(cp->magic));
brelse(bh);
return NULL;
}
if (!le32_to_cpu(cp->n_code_pages)) {
- printk("HPFS: n_code_pages == 0\n");
+ pr_err("n_code_pages == 0\n");
brelse(bh);
return NULL;
}
@@ -79,19 +80,19 @@
brelse(bh);
if (cpi >= 3) {
- printk("HPFS: Code page index out of array\n");
+ pr_err("Code page index out of array\n");
return NULL;
}
if (!(cpd = hpfs_map_sector(s, cpds, &bh, 0))) return NULL;
if (le16_to_cpu(cpd->offs[cpi]) > 0x178) {
- printk("HPFS: Code page index out of sector\n");
+ pr_err("Code page index out of sector\n");
brelse(bh);
return NULL;
}
ptr = (unsigned char *)cpd + le16_to_cpu(cpd->offs[cpi]) + 6;
if (!(cp_table = kmalloc(256, GFP_KERNEL))) {
- printk("HPFS: out of memory for code page table\n");
+ pr_err("out of memory for code page table\n");
brelse(bh);
return NULL;
}
@@ -114,7 +115,7 @@
int i;
__le32 *b;
if (!(b = kmalloc(n * 512, GFP_KERNEL))) {
- printk("HPFS: can't allocate memory for bitmap directory\n");
+ pr_err("can't allocate memory for bitmap directory\n");
return NULL;
}
for (i=0;i<n;i++) {
@@ -281,7 +282,9 @@
hpfs_error(s, "dnode %08x does not end with \\377 entry", secno);
goto bail;
}
- if (b == 3) printk("HPFS: warning: unbalanced dnode tree, dnode %08x; see hpfs.txt 4 more info\n", secno);
+ if (b == 3)
+ pr_err("unbalanced dnode tree, dnode %08x; see hpfs.txt 4 more info\n",
+ secno);
}
return dnode;
bail:
diff --git a/fs/hpfs/name.c b/fs/hpfs/name.c
index 9acdf33..b00d396 100644
--- a/fs/hpfs/name.c
+++ b/fs/hpfs/name.c
@@ -56,14 +56,15 @@
unsigned char *to;
int i;
if (hpfs_sb(s)->sb_chk >= 2) if (hpfs_is_name_long(from, len) != lng) {
- printk("HPFS: Long name flag mismatch - name ");
- for (i=0; i<len; i++) printk("%c", from[i]);
- printk(" misidentified as %s.\n", lng ? "short" : "long");
- printk("HPFS: It's nothing serious. It could happen because of bug in OS/2.\nHPFS: Set checks=normal to disable this message.\n");
+ pr_err("Long name flag mismatch - name ");
+ for (i = 0; i < len; i++)
+ pr_cont("%c", from[i]);
+ pr_cont(" misidentified as %s.\n", lng ? "short" : "long");
+ pr_err("It's nothing serious. It could happen because of bug in OS/2.\nSet checks=normal to disable this message.\n");
}
if (!lc) return from;
if (!(to = kmalloc(len, GFP_KERNEL))) {
- printk("HPFS: can't allocate memory for name conversion buffer\n");
+ pr_err("can't allocate memory for name conversion buffer\n");
return from;
}
for (i = 0; i < len; i++) to[i] = locase(hpfs_sb(s)->sb_cp_table,from[i]);
diff --git a/fs/hpfs/namei.c b/fs/hpfs/namei.c
index 1b39afd..bdbc2c3 100644
--- a/fs/hpfs/namei.c
+++ b/fs/hpfs/namei.c
@@ -404,7 +404,7 @@
d_rehash(dentry);
} else {
struct iattr newattrs;
- /*printk("HPFS: truncating file before delete.\n");*/
+ /*pr_info("truncating file before delete.\n");*/
newattrs.ia_size = 0;
newattrs.ia_valid = ATTR_SIZE | ATTR_CTIME;
err = notify_change(dentry, &newattrs, NULL);
diff --git a/fs/hpfs/super.c b/fs/hpfs/super.c
index fe3463a..7cd00d3 100644
--- a/fs/hpfs/super.c
+++ b/fs/hpfs/super.c
@@ -62,22 +62,26 @@
vsnprintf(err_buf, sizeof(err_buf), fmt, args);
va_end(args);
- printk("HPFS: filesystem error: %s", err_buf);
+ pr_err("filesystem error: %s", err_buf);
if (!hpfs_sb(s)->sb_was_error) {
if (hpfs_sb(s)->sb_err == 2) {
- printk("; crashing the system because you wanted it\n");
+ pr_cont("; crashing the system because you wanted it\n");
mark_dirty(s, 0);
panic("HPFS panic");
} else if (hpfs_sb(s)->sb_err == 1) {
- if (s->s_flags & MS_RDONLY) printk("; already mounted read-only\n");
+ if (s->s_flags & MS_RDONLY)
+ pr_cont("; already mounted read-only\n");
else {
- printk("; remounting read-only\n");
+ pr_cont("; remounting read-only\n");
mark_dirty(s, 0);
s->s_flags |= MS_RDONLY;
}
- } else if (s->s_flags & MS_RDONLY) printk("; going on - but anything won't be destroyed because it's read-only\n");
- else printk("; corrupted filesystem mounted read/write - your computer will explode within 20 seconds ... but you wanted it so!\n");
- } else printk("\n");
+ } else if (s->s_flags & MS_RDONLY)
+ pr_cont("; going on - but anything won't be destroyed because it's read-only\n");
+ else
+ pr_cont("; corrupted filesystem mounted read/write - your computer will explode within 20 seconds ... but you wanted it so!\n");
+ } else
+ pr_cont("\n");
hpfs_sb(s)->sb_was_error = 1;
}
@@ -292,7 +296,7 @@
if (!opts)
return 1;
- /*printk("Parsing opts: '%s'\n",opts);*/
+ /*pr_info("Parsing opts: '%s'\n",opts);*/
while ((p = strsep(&opts, ",")) != NULL) {
substring_t args[MAX_OPT_ARGS];
@@ -387,7 +391,7 @@
static inline void hpfs_help(void)
{
- printk("\n\
+ pr_info("\n\
HPFS filesystem options:\n\
help do not mount and display this text\n\
uid=xxx set uid of files that don't have uid specified in eas\n\
@@ -434,7 +438,7 @@
if (!(o = parse_opts(data, &uid, &gid, &umask, &lowercase,
&eas, &chk, &errs, &chkdsk, ×hift))) {
- printk("HPFS: bad mount options.\n");
+ pr_err("bad mount options.\n");
goto out_err;
}
if (o == 2) {
@@ -442,7 +446,7 @@
goto out_err;
}
if (timeshift != sbi->sb_timeshift) {
- printk("HPFS: timeshift can't be changed using remount.\n");
+ pr_err("timeshift can't be changed using remount.\n");
goto out_err;
}
@@ -523,7 +527,7 @@
if (!(o = parse_opts(options, &uid, &gid, &umask, &lowercase,
&eas, &chk, &errs, &chkdsk, ×hift))) {
- printk("HPFS: bad mount options.\n");
+ pr_err("bad mount options.\n");
goto bail0;
}
if (o==2) {
@@ -542,16 +546,17 @@
if (/*le16_to_cpu(bootblock->magic) != BB_MAGIC
||*/ le32_to_cpu(superblock->magic) != SB_MAGIC
|| le32_to_cpu(spareblock->magic) != SP_MAGIC) {
- if (!silent) printk("HPFS: Bad magic ... probably not HPFS\n");
+ if (!silent)
+ pr_err("Bad magic ... probably not HPFS\n");
goto bail4;
}
/* Check version */
if (!(s->s_flags & MS_RDONLY) &&
superblock->funcversion != 2 && superblock->funcversion != 3) {
- printk("HPFS: Bad version %d,%d. Mount readonly to go around\n",
+ pr_err("Bad version %d,%d. Mount readonly to go around\n",
(int)superblock->version, (int)superblock->funcversion);
- printk("HPFS: please try recent version of HPFS driver at http://artax.karlin.mff.cuni.cz/~mikulas/vyplody/hpfs/index-e.cgi and if it still can't understand this format, contact author - mikulas@artax.karlin.mff.cuni.cz\n");
+ pr_err("please try recent version of HPFS driver at http://artax.karlin.mff.cuni.cz/~mikulas/vyplody/hpfs/index-e.cgi and if it still can't understand this format, contact author - mikulas@artax.karlin.mff.cuni.cz\n");
goto bail4;
}
@@ -597,7 +602,7 @@
/* Check for general fs errors*/
if (spareblock->dirty && !spareblock->old_wrote) {
if (errs == 2) {
- printk("HPFS: Improperly stopped, not mounted\n");
+ pr_err("Improperly stopped, not mounted\n");
goto bail4;
}
hpfs_error(s, "improperly stopped");
@@ -611,22 +616,25 @@
if (spareblock->hotfixes_used || spareblock->n_spares_used) {
if (errs >= 2) {
- printk("HPFS: Hotfixes not supported here, try chkdsk\n");
+ pr_err("Hotfixes not supported here, try chkdsk\n");
mark_dirty(s, 0);
goto bail4;
}
hpfs_error(s, "hotfixes not supported here, try chkdsk");
- if (errs == 0) printk("HPFS: Proceeding, but your filesystem will be probably corrupted by this driver...\n");
- else printk("HPFS: This driver may read bad files or crash when operating on disk with hotfixes.\n");
+ if (errs == 0)
+ pr_err("Proceeding, but your filesystem will be probably corrupted by this driver...\n");
+ else
+ pr_err("This driver may read bad files or crash when operating on disk with hotfixes.\n");
}
if (le32_to_cpu(spareblock->n_dnode_spares) != le32_to_cpu(spareblock->n_dnode_spares_free)) {
if (errs >= 2) {
- printk("HPFS: Spare dnodes used, try chkdsk\n");
+ pr_err("Spare dnodes used, try chkdsk\n");
mark_dirty(s, 0);
goto bail4;
}
hpfs_error(s, "warning: spare dnodes used, try chkdsk");
- if (errs == 0) printk("HPFS: Proceeding, but your filesystem could be corrupted if you delete files or directories\n");
+ if (errs == 0)
+ pr_err("Proceeding, but your filesystem could be corrupted if you delete files or directories\n");
}
if (chk) {
unsigned a;
@@ -645,12 +653,13 @@
goto bail4;
}
sbi->sb_dirband_size = a;
- } else printk("HPFS: You really don't want any checks? You are crazy...\n");
+ } else
+ pr_err("You really don't want any checks? You are crazy...\n");
/* Load code page table */
if (le32_to_cpu(spareblock->n_code_pages))
if (!(sbi->sb_cp_table = hpfs_load_code_page(s, le32_to_cpu(spareblock->code_page_dir))))
- printk("HPFS: Warning: code page support is disabled\n");
+ pr_err("code page support is disabled\n");
brelse(bh2);
brelse(bh1);
diff --git a/fs/inode.c b/fs/inode.c
index f96d2a6..6eecb7f 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -105,7 +105,7 @@
* Handle nr_inode sysctl
*/
#ifdef CONFIG_SYSCTL
-int proc_nr_inodes(ctl_table *table, int write,
+int proc_nr_inodes(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
inodes_stat.nr_inodes = get_nr_inodes();
@@ -1839,14 +1839,18 @@
* inode_owner_or_capable - check current task permissions to inode
* @inode: inode being checked
*
- * Return true if current either has CAP_FOWNER to the inode, or
- * owns the file.
+ * Return true if current either has CAP_FOWNER in a namespace with the
+ * inode owner uid mapped, or owns the file.
*/
bool inode_owner_or_capable(const struct inode *inode)
{
+ struct user_namespace *ns;
+
if (uid_eq(current_fsuid(), inode->i_uid))
return true;
- if (inode_capable(inode, CAP_FOWNER))
+
+ ns = current_user_ns();
+ if (ns_capable(ns, CAP_FOWNER) && kuid_has_mapping(ns, inode->i_uid))
return true;
return false;
}
diff --git a/fs/jffs2/background.c b/fs/jffs2/background.c
index 2b60ce1..bb9cebc 100644
--- a/fs/jffs2/background.c
+++ b/fs/jffs2/background.c
@@ -75,10 +75,13 @@
static int jffs2_garbage_collect_thread(void *_c)
{
struct jffs2_sb_info *c = _c;
+ sigset_t hupmask;
+ siginitset(&hupmask, sigmask(SIGHUP));
allow_signal(SIGKILL);
allow_signal(SIGSTOP);
allow_signal(SIGCONT);
+ allow_signal(SIGHUP);
c->gc_task = current;
complete(&c->gc_thread_start);
@@ -87,7 +90,7 @@
set_freezable();
for (;;) {
- allow_signal(SIGHUP);
+ sigprocmask(SIG_UNBLOCK, &hupmask, NULL);
again:
spin_lock(&c->erase_completion_lock);
if (!jffs2_thread_should_wake(c)) {
@@ -95,10 +98,9 @@
spin_unlock(&c->erase_completion_lock);
jffs2_dbg(1, "%s(): sleeping...\n", __func__);
schedule();
- } else
+ } else {
spin_unlock(&c->erase_completion_lock);
-
-
+ }
/* Problem - immediately after bootup, the GCD spends a lot
* of time in places like jffs2_kill_fragtree(); so much so
* that userspace processes (like gdm and X) are starved
@@ -150,7 +152,7 @@
}
}
/* We don't want SIGHUP to interrupt us. STOP and KILL are OK though. */
- disallow_signal(SIGHUP);
+ sigprocmask(SIG_BLOCK, &hupmask, NULL);
jffs2_dbg(1, "%s(): pass\n", __func__);
if (jffs2_garbage_collect_pass(c) == -ENOSPC) {
diff --git a/fs/lockd/clnt4xdr.c b/fs/lockd/clnt4xdr.c
index 00ec0b9c..d3e40db 100644
--- a/fs/lockd/clnt4xdr.c
+++ b/fs/lockd/clnt4xdr.c
@@ -14,6 +14,8 @@
#include <linux/sunrpc/stats.h>
#include <linux/lockd/lockd.h>
+#include <uapi/linux/nfs3.h>
+
#define NLMDBG_FACILITY NLMDBG_XDR
#if (NLMCLNT_OHSIZE > XDR_MAX_NETOBJ)
diff --git a/fs/lockd/clntxdr.c b/fs/lockd/clntxdr.c
index 9a55797..3e9f787 100644
--- a/fs/lockd/clntxdr.c
+++ b/fs/lockd/clntxdr.c
@@ -15,6 +15,8 @@
#include <linux/sunrpc/stats.h>
#include <linux/lockd/lockd.h>
+#include <uapi/linux/nfs2.h>
+
#define NLMDBG_FACILITY NLMDBG_XDR
#if (NLMCLNT_OHSIZE > XDR_MAX_NETOBJ)
diff --git a/fs/lockd/svc.c b/fs/lockd/svc.c
index 6bf06a0..8f27c93 100644
--- a/fs/lockd/svc.c
+++ b/fs/lockd/svc.c
@@ -436,7 +436,7 @@
* Sysctl parameters (same as module parameters, different interface).
*/
-static ctl_table nlm_sysctls[] = {
+static struct ctl_table nlm_sysctls[] = {
{
.procname = "nlm_grace_period",
.data = &nlm_grace_period,
@@ -490,7 +490,7 @@
{ }
};
-static ctl_table nlm_sysctl_dir[] = {
+static struct ctl_table nlm_sysctl_dir[] = {
{
.procname = "nfs",
.mode = 0555,
@@ -499,7 +499,7 @@
{ }
};
-static ctl_table nlm_sysctl_root[] = {
+static struct ctl_table nlm_sysctl_root[] = {
{
.procname = "fs",
.mode = 0555,
@@ -622,8 +622,8 @@
err_pernet:
#ifdef CONFIG_SYSCTL
unregister_sysctl_table(nlm_sysctl_table);
-#endif
err_sysctl:
+#endif
return err;
}
diff --git a/fs/lockd/svcsubs.c b/fs/lockd/svcsubs.c
index dc5c759..b6f3b84 100644
--- a/fs/lockd/svcsubs.c
+++ b/fs/lockd/svcsubs.c
@@ -14,12 +14,11 @@
#include <linux/mutex.h>
#include <linux/sunrpc/svc.h>
#include <linux/sunrpc/addr.h>
-#include <linux/nfsd/nfsfh.h>
-#include <linux/nfsd/export.h>
#include <linux/lockd/lockd.h>
#include <linux/lockd/share.h>
#include <linux/module.h>
#include <linux/mount.h>
+#include <uapi/linux/nfs2.h>
#define NLMDBG_FACILITY NLMDBG_SVCSUBS
diff --git a/fs/lockd/xdr.c b/fs/lockd/xdr.c
index 964666c..9340e7e 100644
--- a/fs/lockd/xdr.c
+++ b/fs/lockd/xdr.c
@@ -16,6 +16,8 @@
#include <linux/sunrpc/stats.h>
#include <linux/lockd/lockd.h>
+#include <uapi/linux/nfs2.h>
+
#define NLMDBG_FACILITY NLMDBG_XDR
diff --git a/fs/namei.c b/fs/namei.c
index 8016827..985c6f3 100644
--- a/fs/namei.c
+++ b/fs/namei.c
@@ -332,10 +332,11 @@
if (S_ISDIR(inode->i_mode)) {
/* DACs are overridable for directories */
- if (inode_capable(inode, CAP_DAC_OVERRIDE))
+ if (capable_wrt_inode_uidgid(inode, CAP_DAC_OVERRIDE))
return 0;
if (!(mask & MAY_WRITE))
- if (inode_capable(inode, CAP_DAC_READ_SEARCH))
+ if (capable_wrt_inode_uidgid(inode,
+ CAP_DAC_READ_SEARCH))
return 0;
return -EACCES;
}
@@ -345,7 +346,7 @@
* at least one exec bit set.
*/
if (!(mask & MAY_EXEC) || (inode->i_mode & S_IXUGO))
- if (inode_capable(inode, CAP_DAC_OVERRIDE))
+ if (capable_wrt_inode_uidgid(inode, CAP_DAC_OVERRIDE))
return 0;
/*
@@ -353,7 +354,7 @@
*/
mask &= MAY_READ | MAY_WRITE | MAY_EXEC;
if (mask == MAY_READ)
- if (inode_capable(inode, CAP_DAC_READ_SEARCH))
+ if (capable_wrt_inode_uidgid(inode, CAP_DAC_READ_SEARCH))
return 0;
return -EACCES;
@@ -2379,7 +2380,7 @@
return 0;
if (uid_eq(dir->i_uid, fsuid))
return 0;
- return !inode_capable(inode, CAP_FOWNER);
+ return !capable_wrt_inode_uidgid(inode, CAP_FOWNER);
}
/*
diff --git a/fs/nfs/Makefile b/fs/nfs/Makefile
index 03192a6..4782e08 100644
--- a/fs/nfs/Makefile
+++ b/fs/nfs/Makefile
@@ -29,8 +29,6 @@
nfsv4-$(CONFIG_SYSCTL) += nfs4sysctl.o
nfsv4-$(CONFIG_NFS_V4_1) += pnfs.o pnfs_dev.o
-obj-$(CONFIG_PNFS_FILE_LAYOUT) += nfs_layout_nfsv41_files.o
-nfs_layout_nfsv41_files-y := nfs4filelayout.o nfs4filelayoutdev.o
-
+obj-$(CONFIG_PNFS_FILE_LAYOUT) += filelayout/
obj-$(CONFIG_PNFS_OBJLAYOUT) += objlayout/
obj-$(CONFIG_PNFS_BLOCK) += blocklayout/
diff --git a/fs/nfs/blocklayout/blocklayout.c b/fs/nfs/blocklayout/blocklayout.c
index 65d849b..9b431f4 100644
--- a/fs/nfs/blocklayout/blocklayout.c
+++ b/fs/nfs/blocklayout/blocklayout.c
@@ -210,7 +210,7 @@
SetPageUptodate(bvec->bv_page);
if (err) {
- struct nfs_read_data *rdata = par->data;
+ struct nfs_pgio_data *rdata = par->data;
struct nfs_pgio_header *header = rdata->header;
if (!header->pnfs_error)
@@ -224,17 +224,17 @@
static void bl_read_cleanup(struct work_struct *work)
{
struct rpc_task *task;
- struct nfs_read_data *rdata;
+ struct nfs_pgio_data *rdata;
dprintk("%s enter\n", __func__);
task = container_of(work, struct rpc_task, u.tk_work);
- rdata = container_of(task, struct nfs_read_data, task);
+ rdata = container_of(task, struct nfs_pgio_data, task);
pnfs_ld_read_done(rdata);
}
static void
bl_end_par_io_read(void *data, int unused)
{
- struct nfs_read_data *rdata = data;
+ struct nfs_pgio_data *rdata = data;
rdata->task.tk_status = rdata->header->pnfs_error;
INIT_WORK(&rdata->task.u.tk_work, bl_read_cleanup);
@@ -242,7 +242,7 @@
}
static enum pnfs_try_status
-bl_read_pagelist(struct nfs_read_data *rdata)
+bl_read_pagelist(struct nfs_pgio_data *rdata)
{
struct nfs_pgio_header *header = rdata->header;
int i, hole;
@@ -390,7 +390,7 @@
}
if (unlikely(err)) {
- struct nfs_write_data *data = par->data;
+ struct nfs_pgio_data *data = par->data;
struct nfs_pgio_header *header = data->header;
if (!header->pnfs_error)
@@ -405,7 +405,7 @@
{
struct parallel_io *par = bio->bi_private;
const int uptodate = test_bit(BIO_UPTODATE, &bio->bi_flags);
- struct nfs_write_data *data = par->data;
+ struct nfs_pgio_data *data = par->data;
struct nfs_pgio_header *header = data->header;
if (!uptodate) {
@@ -423,10 +423,10 @@
static void bl_write_cleanup(struct work_struct *work)
{
struct rpc_task *task;
- struct nfs_write_data *wdata;
+ struct nfs_pgio_data *wdata;
dprintk("%s enter\n", __func__);
task = container_of(work, struct rpc_task, u.tk_work);
- wdata = container_of(task, struct nfs_write_data, task);
+ wdata = container_of(task, struct nfs_pgio_data, task);
if (likely(!wdata->header->pnfs_error)) {
/* Marks for LAYOUTCOMMIT */
mark_extents_written(BLK_LSEG2EXT(wdata->header->lseg),
@@ -438,7 +438,7 @@
/* Called when last of bios associated with a bl_write_pagelist call finishes */
static void bl_end_par_io_write(void *data, int num_se)
{
- struct nfs_write_data *wdata = data;
+ struct nfs_pgio_data *wdata = data;
if (unlikely(wdata->header->pnfs_error)) {
bl_free_short_extents(&BLK_LSEG2EXT(wdata->header->lseg)->bl_inval,
@@ -673,7 +673,7 @@
}
static enum pnfs_try_status
-bl_write_pagelist(struct nfs_write_data *wdata, int sync)
+bl_write_pagelist(struct nfs_pgio_data *wdata, int sync)
{
struct nfs_pgio_header *header = wdata->header;
int i, ret, npg_zero, pg_index, last = 0;
@@ -1189,13 +1189,17 @@
pnfs_generic_pg_init_read(pgio, req);
}
-static bool
+/*
+ * Return 0 if @req cannot be coalesced into @pgio, otherwise return the number
+ * of bytes (maximum @req->wb_bytes) that can be coalesced.
+ */
+static size_t
bl_pg_test_read(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev,
struct nfs_page *req)
{
if (pgio->pg_dreq != NULL &&
!is_aligned_req(req, SECTOR_SIZE))
- return false;
+ return 0;
return pnfs_generic_pg_test(pgio, prev, req);
}
@@ -1241,13 +1245,17 @@
}
}
-static bool
+/*
+ * Return 0 if @req cannot be coalesced into @pgio, otherwise return the number
+ * of bytes (maximum @req->wb_bytes) that can be coalesced.
+ */
+static size_t
bl_pg_test_write(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev,
struct nfs_page *req)
{
if (pgio->pg_dreq != NULL &&
!is_aligned_req(req, PAGE_CACHE_SIZE))
- return false;
+ return 0;
return pnfs_generic_pg_test(pgio, prev, req);
}
diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c
index b8797ae..4ad7bc3 100644
--- a/fs/nfs/direct.c
+++ b/fs/nfs/direct.c
@@ -108,6 +108,97 @@
return atomic_dec_and_test(&dreq->io_count);
}
+/*
+ * nfs_direct_select_verf - select the right verifier
+ * @dreq - direct request possibly spanning multiple servers
+ * @ds_clp - nfs_client of data server or NULL if MDS / non-pnfs
+ * @ds_idx - index of data server in data server list, only valid if ds_clp set
+ *
+ * returns the correct verifier to use given the role of the server
+ */
+static struct nfs_writeverf *
+nfs_direct_select_verf(struct nfs_direct_req *dreq,
+ struct nfs_client *ds_clp,
+ int ds_idx)
+{
+ struct nfs_writeverf *verfp = &dreq->verf;
+
+#ifdef CONFIG_NFS_V4_1
+ if (ds_clp) {
+ /* pNFS is in use, use the DS verf */
+ if (ds_idx >= 0 && ds_idx < dreq->ds_cinfo.nbuckets)
+ verfp = &dreq->ds_cinfo.buckets[ds_idx].direct_verf;
+ else
+ WARN_ON_ONCE(1);
+ }
+#endif
+ return verfp;
+}
+
+
+/*
+ * nfs_direct_set_hdr_verf - set the write/commit verifier
+ * @dreq - direct request possibly spanning multiple servers
+ * @hdr - pageio header to validate against previously seen verfs
+ *
+ * Set the server's (MDS or DS) "seen" verifier
+ */
+static void nfs_direct_set_hdr_verf(struct nfs_direct_req *dreq,
+ struct nfs_pgio_header *hdr)
+{
+ struct nfs_writeverf *verfp;
+
+ verfp = nfs_direct_select_verf(dreq, hdr->data->ds_clp,
+ hdr->data->ds_idx);
+ WARN_ON_ONCE(verfp->committed >= 0);
+ memcpy(verfp, &hdr->verf, sizeof(struct nfs_writeverf));
+ WARN_ON_ONCE(verfp->committed < 0);
+}
+
+/*
+ * nfs_direct_cmp_hdr_verf - compare verifier for pgio header
+ * @dreq - direct request possibly spanning multiple servers
+ * @hdr - pageio header to validate against previously seen verf
+ *
+ * set the server's "seen" verf if not initialized.
+ * returns result of comparison between @hdr->verf and the "seen"
+ * verf of the server used by @hdr (DS or MDS)
+ */
+static int nfs_direct_set_or_cmp_hdr_verf(struct nfs_direct_req *dreq,
+ struct nfs_pgio_header *hdr)
+{
+ struct nfs_writeverf *verfp;
+
+ verfp = nfs_direct_select_verf(dreq, hdr->data->ds_clp,
+ hdr->data->ds_idx);
+ if (verfp->committed < 0) {
+ nfs_direct_set_hdr_verf(dreq, hdr);
+ return 0;
+ }
+ return memcmp(verfp, &hdr->verf, sizeof(struct nfs_writeverf));
+}
+
+#if IS_ENABLED(CONFIG_NFS_V3) || IS_ENABLED(CONFIG_NFS_V4)
+/*
+ * nfs_direct_cmp_commit_data_verf - compare verifier for commit data
+ * @dreq - direct request possibly spanning multiple servers
+ * @data - commit data to validate against previously seen verf
+ *
+ * returns result of comparison between @data->verf and the verf of
+ * the server used by @data (DS or MDS)
+ */
+static int nfs_direct_cmp_commit_data_verf(struct nfs_direct_req *dreq,
+ struct nfs_commit_data *data)
+{
+ struct nfs_writeverf *verfp;
+
+ verfp = nfs_direct_select_verf(dreq, data->ds_clp,
+ data->ds_commit_index);
+ WARN_ON_ONCE(verfp->committed < 0);
+ return memcmp(verfp, &data->verf, sizeof(struct nfs_writeverf));
+}
+#endif
+
/**
* nfs_direct_IO - NFS address space operation for direct I/O
* @rw: direction (read or write)
@@ -168,6 +259,7 @@
kref_get(&dreq->kref);
init_completion(&dreq->completion);
INIT_LIST_HEAD(&dreq->mds_cinfo.list);
+ dreq->verf.committed = NFS_INVALID_STABLE_HOW; /* not set yet */
INIT_WORK(&dreq->work, nfs_direct_write_schedule_work);
spin_lock_init(&dreq->lock);
@@ -380,8 +472,7 @@
struct nfs_page *req;
unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
/* XXX do we need to do the eof zeroing found in async_filler? */
- req = nfs_create_request(dreq->ctx, dreq->inode,
- pagevec[i],
+ req = nfs_create_request(dreq->ctx, pagevec[i], NULL,
pgbase, req_len);
if (IS_ERR(req)) {
result = PTR_ERR(req);
@@ -424,7 +515,7 @@
size_t requested_bytes = 0;
unsigned long seg;
- NFS_PROTO(dreq->inode)->read_pageio_init(&desc, dreq->inode,
+ nfs_pageio_init_read(&desc, dreq->inode, false,
&nfs_direct_read_completion_ops);
get_dreq(dreq);
desc.pg_dreq = dreq;
@@ -564,7 +655,7 @@
dreq->count = 0;
get_dreq(dreq);
- NFS_PROTO(dreq->inode)->write_pageio_init(&desc, dreq->inode, FLUSH_STABLE,
+ nfs_pageio_init_write(&desc, dreq->inode, FLUSH_STABLE, false,
&nfs_direct_write_completion_ops);
desc.pg_dreq = dreq;
@@ -603,7 +694,7 @@
dprintk("NFS: %5u commit failed with error %d.\n",
data->task.tk_pid, status);
dreq->flags = NFS_ODIRECT_RESCHED_WRITES;
- } else if (memcmp(&dreq->verf, &data->verf, sizeof(data->verf))) {
+ } else if (nfs_direct_cmp_commit_data_verf(dreq, data)) {
dprintk("NFS: %5u commit verify failed\n", data->task.tk_pid);
dreq->flags = NFS_ODIRECT_RESCHED_WRITES;
}
@@ -750,8 +841,7 @@
struct nfs_page *req;
unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
- req = nfs_create_request(dreq->ctx, dreq->inode,
- pagevec[i],
+ req = nfs_create_request(dreq->ctx, pagevec[i], NULL,
pgbase, req_len);
if (IS_ERR(req)) {
result = PTR_ERR(req);
@@ -813,13 +903,13 @@
if (dreq->flags == NFS_ODIRECT_RESCHED_WRITES)
bit = NFS_IOHDR_NEED_RESCHED;
else if (dreq->flags == 0) {
- memcpy(&dreq->verf, hdr->verf,
- sizeof(dreq->verf));
+ nfs_direct_set_hdr_verf(dreq, hdr);
bit = NFS_IOHDR_NEED_COMMIT;
dreq->flags = NFS_ODIRECT_DO_COMMIT;
} else if (dreq->flags == NFS_ODIRECT_DO_COMMIT) {
- if (memcmp(&dreq->verf, hdr->verf, sizeof(dreq->verf))) {
- dreq->flags = NFS_ODIRECT_RESCHED_WRITES;
+ if (nfs_direct_set_or_cmp_hdr_verf(dreq, hdr)) {
+ dreq->flags =
+ NFS_ODIRECT_RESCHED_WRITES;
bit = NFS_IOHDR_NEED_RESCHED;
} else
bit = NFS_IOHDR_NEED_COMMIT;
@@ -829,6 +919,8 @@
spin_unlock(&dreq->lock);
while (!list_empty(&hdr->pages)) {
+ bool do_destroy = true;
+
req = nfs_list_entry(hdr->pages.next);
nfs_list_remove_request(req);
switch (bit) {
@@ -836,6 +928,7 @@
case NFS_IOHDR_NEED_COMMIT:
kref_get(&req->wb_kref);
nfs_mark_request_commit(req, hdr->lseg, &cinfo);
+ do_destroy = false;
}
nfs_unlock_and_release_request(req);
}
@@ -874,7 +967,7 @@
size_t requested_bytes = 0;
unsigned long seg;
- NFS_PROTO(inode)->write_pageio_init(&desc, inode, FLUSH_COND_STABLE,
+ nfs_pageio_init_write(&desc, inode, FLUSH_COND_STABLE, false,
&nfs_direct_write_completion_ops);
desc.pg_dreq = dreq;
get_dreq(dreq);
diff --git a/fs/nfs/filelayout/Makefile b/fs/nfs/filelayout/Makefile
new file mode 100644
index 0000000..8516cdf
--- /dev/null
+++ b/fs/nfs/filelayout/Makefile
@@ -0,0 +1,5 @@
+#
+# Makefile for the pNFS Files Layout Driver kernel module
+#
+obj-$(CONFIG_PNFS_FILE_LAYOUT) += nfs_layout_nfsv41_files.o
+nfs_layout_nfsv41_files-y := filelayout.o filelayoutdev.o
diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/filelayout/filelayout.c
similarity index 90%
rename from fs/nfs/nfs4filelayout.c
rename to fs/nfs/filelayout/filelayout.c
index b9a35c0..d2eba1c 100644
--- a/fs/nfs/nfs4filelayout.c
+++ b/fs/nfs/filelayout/filelayout.c
@@ -35,11 +35,11 @@
#include <linux/sunrpc/metrics.h>
-#include "nfs4session.h"
-#include "internal.h"
-#include "delegation.h"
-#include "nfs4filelayout.h"
-#include "nfs4trace.h"
+#include "../nfs4session.h"
+#include "../internal.h"
+#include "../delegation.h"
+#include "filelayout.h"
+#include "../nfs4trace.h"
#define NFSDBG_FACILITY NFSDBG_PNFS_LD
@@ -84,7 +84,7 @@
BUG();
}
-static void filelayout_reset_write(struct nfs_write_data *data)
+static void filelayout_reset_write(struct nfs_pgio_data *data)
{
struct nfs_pgio_header *hdr = data->header;
struct rpc_task *task = &data->task;
@@ -105,7 +105,7 @@
}
}
-static void filelayout_reset_read(struct nfs_read_data *data)
+static void filelayout_reset_read(struct nfs_pgio_data *data)
{
struct nfs_pgio_header *hdr = data->header;
struct rpc_task *task = &data->task;
@@ -243,7 +243,7 @@
/* NFS_PROTO call done callback routines */
static int filelayout_read_done_cb(struct rpc_task *task,
- struct nfs_read_data *data)
+ struct nfs_pgio_data *data)
{
struct nfs_pgio_header *hdr = data->header;
int err;
@@ -270,7 +270,7 @@
* rfc5661 is not clear about which credential should be used.
*/
static void
-filelayout_set_layoutcommit(struct nfs_write_data *wdata)
+filelayout_set_layoutcommit(struct nfs_pgio_data *wdata)
{
struct nfs_pgio_header *hdr = wdata->header;
@@ -279,7 +279,7 @@
return;
pnfs_set_layoutcommit(wdata);
- dprintk("%s ionde %lu pls_end_pos %lu\n", __func__, hdr->inode->i_ino,
+ dprintk("%s inode %lu pls_end_pos %lu\n", __func__, hdr->inode->i_ino,
(unsigned long) NFS_I(hdr->inode)->layout->plh_lwb);
}
@@ -305,7 +305,7 @@
*/
static void filelayout_read_prepare(struct rpc_task *task, void *data)
{
- struct nfs_read_data *rdata = data;
+ struct nfs_pgio_data *rdata = data;
if (unlikely(test_bit(NFS_CONTEXT_BAD, &rdata->args.context->flags))) {
rpc_exit(task, -EIO);
@@ -317,7 +317,7 @@
rpc_exit(task, 0);
return;
}
- rdata->read_done_cb = filelayout_read_done_cb;
+ rdata->pgio_done_cb = filelayout_read_done_cb;
if (nfs41_setup_sequence(rdata->ds_clp->cl_session,
&rdata->args.seq_args,
@@ -331,7 +331,7 @@
static void filelayout_read_call_done(struct rpc_task *task, void *data)
{
- struct nfs_read_data *rdata = data;
+ struct nfs_pgio_data *rdata = data;
dprintk("--> %s task->tk_status %d\n", __func__, task->tk_status);
@@ -347,14 +347,14 @@
static void filelayout_read_count_stats(struct rpc_task *task, void *data)
{
- struct nfs_read_data *rdata = data;
+ struct nfs_pgio_data *rdata = data;
rpc_count_iostats(task, NFS_SERVER(rdata->header->inode)->client->cl_metrics);
}
static void filelayout_read_release(void *data)
{
- struct nfs_read_data *rdata = data;
+ struct nfs_pgio_data *rdata = data;
struct pnfs_layout_hdr *lo = rdata->header->lseg->pls_layout;
filelayout_fenceme(lo->plh_inode, lo);
@@ -363,7 +363,7 @@
}
static int filelayout_write_done_cb(struct rpc_task *task,
- struct nfs_write_data *data)
+ struct nfs_pgio_data *data)
{
struct nfs_pgio_header *hdr = data->header;
int err;
@@ -419,7 +419,7 @@
static void filelayout_write_prepare(struct rpc_task *task, void *data)
{
- struct nfs_write_data *wdata = data;
+ struct nfs_pgio_data *wdata = data;
if (unlikely(test_bit(NFS_CONTEXT_BAD, &wdata->args.context->flags))) {
rpc_exit(task, -EIO);
@@ -443,7 +443,7 @@
static void filelayout_write_call_done(struct rpc_task *task, void *data)
{
- struct nfs_write_data *wdata = data;
+ struct nfs_pgio_data *wdata = data;
if (test_bit(NFS_IOHDR_REDO, &wdata->header->flags) &&
task->tk_status == 0) {
@@ -457,14 +457,14 @@
static void filelayout_write_count_stats(struct rpc_task *task, void *data)
{
- struct nfs_write_data *wdata = data;
+ struct nfs_pgio_data *wdata = data;
rpc_count_iostats(task, NFS_SERVER(wdata->header->inode)->client->cl_metrics);
}
static void filelayout_write_release(void *data)
{
- struct nfs_write_data *wdata = data;
+ struct nfs_pgio_data *wdata = data;
struct pnfs_layout_hdr *lo = wdata->header->lseg->pls_layout;
filelayout_fenceme(lo->plh_inode, lo);
@@ -529,7 +529,7 @@
};
static enum pnfs_try_status
-filelayout_read_pagelist(struct nfs_read_data *data)
+filelayout_read_pagelist(struct nfs_pgio_data *data)
{
struct nfs_pgio_header *hdr = data->header;
struct pnfs_layout_segment *lseg = hdr->lseg;
@@ -560,6 +560,7 @@
/* No multipath support. Use first DS */
atomic_inc(&ds->ds_clp->cl_count);
data->ds_clp = ds->ds_clp;
+ data->ds_idx = idx;
fh = nfs4_fl_select_ds_fh(lseg, j);
if (fh)
data->args.fh = fh;
@@ -568,14 +569,14 @@
data->mds_offset = offset;
/* Perform an asynchronous read to ds */
- nfs_initiate_read(ds_clnt, data,
- &filelayout_read_call_ops, RPC_TASK_SOFTCONN);
+ nfs_initiate_pgio(ds_clnt, data,
+ &filelayout_read_call_ops, 0, RPC_TASK_SOFTCONN);
return PNFS_ATTEMPTED;
}
/* Perform async writes. */
static enum pnfs_try_status
-filelayout_write_pagelist(struct nfs_write_data *data, int sync)
+filelayout_write_pagelist(struct nfs_pgio_data *data, int sync)
{
struct nfs_pgio_header *hdr = data->header;
struct pnfs_layout_segment *lseg = hdr->lseg;
@@ -600,20 +601,18 @@
__func__, hdr->inode->i_ino, sync, (size_t) data->args.count,
offset, ds->ds_remotestr, atomic_read(&ds->ds_clp->cl_count));
- data->write_done_cb = filelayout_write_done_cb;
+ data->pgio_done_cb = filelayout_write_done_cb;
atomic_inc(&ds->ds_clp->cl_count);
data->ds_clp = ds->ds_clp;
+ data->ds_idx = idx;
fh = nfs4_fl_select_ds_fh(lseg, j);
if (fh)
data->args.fh = fh;
- /*
- * Get the file offset on the dserver. Set the write offset to
- * this offset and save the original offset.
- */
+
data->args.offset = filelayout_get_dserver_offset(lseg, offset);
/* Perform an asynchronous write */
- nfs_initiate_write(ds_clnt, data,
+ nfs_initiate_pgio(ds_clnt, data,
&filelayout_write_call_ops, sync,
RPC_TASK_SOFTCONN);
return PNFS_ATTEMPTED;
@@ -637,7 +636,6 @@
struct nfs4_deviceid_node *d;
struct nfs4_file_layout_dsaddr *dsaddr;
int status = -EINVAL;
- struct nfs_server *nfss = NFS_SERVER(lo->plh_inode);
dprintk("--> %s\n", __func__);
@@ -655,7 +653,7 @@
goto out;
}
- if (!fl->stripe_unit || fl->stripe_unit % PAGE_SIZE) {
+ if (!fl->stripe_unit) {
dprintk("%s Invalid stripe unit (%u)\n",
__func__, fl->stripe_unit);
goto out;
@@ -692,12 +690,6 @@
goto out_put;
}
- if (fl->stripe_unit % nfss->rsize || fl->stripe_unit % nfss->wsize) {
- dprintk("%s Stripe unit (%u) not aligned with rsize %u "
- "wsize %u\n", __func__, fl->stripe_unit, nfss->rsize,
- nfss->wsize);
- }
-
status = 0;
out:
dprintk("--> %s returns %d\n", __func__, status);
@@ -850,11 +842,15 @@
{
struct nfs4_filelayout_segment *fl = FILELAYOUT_LSEG(lseg);
struct pnfs_commit_bucket *buckets;
- int size;
+ int size, i;
if (fl->commit_through_mds)
return 0;
- if (cinfo->ds->nbuckets != 0) {
+
+ size = (fl->stripe_type == STRIPE_SPARSE) ?
+ fl->dsaddr->ds_num : fl->dsaddr->stripe_count;
+
+ if (cinfo->ds->nbuckets >= size) {
/* This assumes there is only one IOMODE_RW lseg. What
* we really want to do is have a layout_hdr level
* dictionary of <multipath_list4, fh> keys, each
@@ -864,30 +860,36 @@
return 0;
}
- size = (fl->stripe_type == STRIPE_SPARSE) ?
- fl->dsaddr->ds_num : fl->dsaddr->stripe_count;
-
buckets = kcalloc(size, sizeof(struct pnfs_commit_bucket),
gfp_flags);
if (!buckets)
return -ENOMEM;
- else {
- int i;
-
- spin_lock(cinfo->lock);
- if (cinfo->ds->nbuckets != 0)
- kfree(buckets);
- else {
- cinfo->ds->buckets = buckets;
- cinfo->ds->nbuckets = size;
- for (i = 0; i < size; i++) {
- INIT_LIST_HEAD(&buckets[i].written);
- INIT_LIST_HEAD(&buckets[i].committing);
- }
- }
- spin_unlock(cinfo->lock);
- return 0;
+ for (i = 0; i < size; i++) {
+ INIT_LIST_HEAD(&buckets[i].written);
+ INIT_LIST_HEAD(&buckets[i].committing);
+ /* mark direct verifier as unset */
+ buckets[i].direct_verf.committed = NFS_INVALID_STABLE_HOW;
}
+
+ spin_lock(cinfo->lock);
+ if (cinfo->ds->nbuckets >= size)
+ goto out;
+ for (i = 0; i < cinfo->ds->nbuckets; i++) {
+ list_splice(&cinfo->ds->buckets[i].written,
+ &buckets[i].written);
+ list_splice(&cinfo->ds->buckets[i].committing,
+ &buckets[i].committing);
+ buckets[i].direct_verf.committed =
+ cinfo->ds->buckets[i].direct_verf.committed;
+ buckets[i].wlseg = cinfo->ds->buckets[i].wlseg;
+ buckets[i].clseg = cinfo->ds->buckets[i].clseg;
+ }
+ swap(cinfo->ds->buckets, buckets);
+ cinfo->ds->nbuckets = size;
+out:
+ spin_unlock(cinfo->lock);
+ kfree(buckets);
+ return 0;
}
static struct pnfs_layout_segment *
@@ -915,47 +917,51 @@
/*
* filelayout_pg_test(). Called by nfs_can_coalesce_requests()
*
- * return true : coalesce page
- * return false : don't coalesce page
+ * Return 0 if @req cannot be coalesced into @pgio, otherwise return the number
+ * of bytes (maximum @req->wb_bytes) that can be coalesced.
*/
-static bool
+static size_t
filelayout_pg_test(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev,
struct nfs_page *req)
{
+ unsigned int size;
u64 p_stripe, r_stripe;
- u32 stripe_unit;
+ u32 stripe_offset;
+ u64 segment_offset = pgio->pg_lseg->pls_range.offset;
+ u32 stripe_unit = FILELAYOUT_LSEG(pgio->pg_lseg)->stripe_unit;
- if (!pnfs_generic_pg_test(pgio, prev, req) ||
- !nfs_generic_pg_test(pgio, prev, req))
- return false;
+ /* calls nfs_generic_pg_test */
+ size = pnfs_generic_pg_test(pgio, prev, req);
+ if (!size)
+ return 0;
- p_stripe = (u64)req_offset(prev);
- r_stripe = (u64)req_offset(req);
- stripe_unit = FILELAYOUT_LSEG(pgio->pg_lseg)->stripe_unit;
+ /* see if req and prev are in the same stripe */
+ if (prev) {
+ p_stripe = (u64)req_offset(prev) - segment_offset;
+ r_stripe = (u64)req_offset(req) - segment_offset;
+ do_div(p_stripe, stripe_unit);
+ do_div(r_stripe, stripe_unit);
- do_div(p_stripe, stripe_unit);
- do_div(r_stripe, stripe_unit);
+ if (p_stripe != r_stripe)
+ return 0;
+ }
- return (p_stripe == r_stripe);
+ /* calculate remaining bytes in the current stripe */
+ div_u64_rem((u64)req_offset(req) - segment_offset,
+ stripe_unit,
+ &stripe_offset);
+ WARN_ON_ONCE(stripe_offset > stripe_unit);
+ if (stripe_offset >= stripe_unit)
+ return 0;
+ return min(stripe_unit - (unsigned int)stripe_offset, size);
}
static void
filelayout_pg_init_read(struct nfs_pageio_descriptor *pgio,
struct nfs_page *req)
{
- WARN_ON_ONCE(pgio->pg_lseg != NULL);
-
- if (req->wb_offset != req->wb_pgbase) {
- /*
- * Handling unaligned pages is difficult, because have to
- * somehow split a req in two in certain cases in the
- * pg.test code. Avoid this by just not using pnfs
- * in this case.
- */
- nfs_pageio_reset_read_mds(pgio);
- return;
- }
- pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
+ if (!pgio->pg_lseg)
+ pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
req->wb_context,
0,
NFS4_MAX_UINT64,
@@ -973,11 +979,8 @@
struct nfs_commit_info cinfo;
int status;
- WARN_ON_ONCE(pgio->pg_lseg != NULL);
-
- if (req->wb_offset != req->wb_pgbase)
- goto out_mds;
- pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
+ if (!pgio->pg_lseg)
+ pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
req->wb_context,
0,
NFS4_MAX_UINT64,
@@ -1067,6 +1070,7 @@
*/
j = nfs4_fl_calc_j_index(lseg, req_offset(req));
i = select_bucket_index(fl, j);
+ spin_lock(cinfo->lock);
buckets = cinfo->ds->buckets;
list = &buckets[i].written;
if (list_empty(list)) {
@@ -1080,6 +1084,7 @@
}
set_bit(PG_COMMIT_TO_DS, &req->wb_flags);
cinfo->ds->nwritten++;
+ spin_unlock(cinfo->lock);
return list;
}
@@ -1176,6 +1181,7 @@
return ret;
}
+/* Note called with cinfo->lock held. */
static int
filelayout_scan_ds_commit_list(struct pnfs_commit_bucket *bucket,
struct nfs_commit_info *cinfo,
@@ -1220,15 +1226,18 @@
struct nfs_commit_info *cinfo)
{
struct pnfs_commit_bucket *b;
+ struct pnfs_layout_segment *freeme;
int i;
+restart:
spin_lock(cinfo->lock);
for (i = 0, b = cinfo->ds->buckets; i < cinfo->ds->nbuckets; i++, b++) {
if (transfer_commit_list(&b->written, dst, cinfo, 0)) {
- spin_unlock(cinfo->lock);
- pnfs_put_lseg(b->wlseg);
+ freeme = b->wlseg;
b->wlseg = NULL;
- spin_lock(cinfo->lock);
+ spin_unlock(cinfo->lock);
+ pnfs_put_lseg(freeme);
+ goto restart;
}
}
cinfo->ds->nwritten = 0;
@@ -1243,6 +1252,7 @@
struct nfs_commit_data *data;
int i, j;
unsigned int nreq = 0;
+ struct pnfs_layout_segment *freeme;
fl_cinfo = cinfo->ds;
bucket = fl_cinfo->buckets;
@@ -1253,8 +1263,10 @@
if (!data)
break;
data->ds_commit_index = i;
+ spin_lock(cinfo->lock);
data->lseg = bucket->clseg;
bucket->clseg = NULL;
+ spin_unlock(cinfo->lock);
list_add(&data->pages, list);
nreq++;
}
@@ -1264,8 +1276,11 @@
if (list_empty(&bucket->committing))
continue;
nfs_retry_commit(&bucket->committing, bucket->clseg, cinfo);
- pnfs_put_lseg(bucket->clseg);
+ spin_lock(cinfo->lock);
+ freeme = bucket->clseg;
bucket->clseg = NULL;
+ spin_unlock(cinfo->lock);
+ pnfs_put_lseg(freeme);
}
/* Caller will clean up entries put on list */
return nreq;
@@ -1330,7 +1345,7 @@
struct nfs4_filelayout *flo;
flo = kzalloc(sizeof(*flo), gfp_flags);
- return &flo->generic_hdr;
+ return flo != NULL ? &flo->generic_hdr : NULL;
}
static void
diff --git a/fs/nfs/nfs4filelayout.h b/fs/nfs/filelayout/filelayout.h
similarity index 99%
rename from fs/nfs/nfs4filelayout.h
rename to fs/nfs/filelayout/filelayout.h
index cebd20e..ffbddf2 100644
--- a/fs/nfs/nfs4filelayout.h
+++ b/fs/nfs/filelayout/filelayout.h
@@ -30,7 +30,7 @@
#ifndef FS_NFS_NFS4FILELAYOUT_H
#define FS_NFS_NFS4FILELAYOUT_H
-#include "pnfs.h"
+#include "../pnfs.h"
/*
* Default data server connection timeout and retrans vaules.
diff --git a/fs/nfs/nfs4filelayoutdev.c b/fs/nfs/filelayout/filelayoutdev.c
similarity index 99%
rename from fs/nfs/nfs4filelayoutdev.c
rename to fs/nfs/filelayout/filelayoutdev.c
index b9c61ef..44bf014 100644
--- a/fs/nfs/nfs4filelayoutdev.c
+++ b/fs/nfs/filelayout/filelayoutdev.c
@@ -33,9 +33,9 @@
#include <linux/module.h>
#include <linux/sunrpc/addr.h>
-#include "internal.h"
-#include "nfs4session.h"
-#include "nfs4filelayout.h"
+#include "../internal.h"
+#include "../nfs4session.h"
+#include "filelayout.h"
#define NFSDBG_FACILITY NFSDBG_PNFS_LD
diff --git a/fs/nfs/getroot.c b/fs/nfs/getroot.c
index 66984a9..b94f804 100644
--- a/fs/nfs/getroot.c
+++ b/fs/nfs/getroot.c
@@ -120,7 +120,8 @@
security_d_instantiate(ret, inode);
spin_lock(&ret->d_lock);
- if (IS_ROOT(ret) && !(ret->d_flags & DCACHE_NFSFS_RENAMED)) {
+ if (IS_ROOT(ret) && !ret->d_fsdata &&
+ !(ret->d_flags & DCACHE_NFSFS_RENAMED)) {
ret->d_fsdata = name;
name = NULL;
}
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index e6f7398..c496f8a 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -1575,18 +1575,20 @@
inode->i_version = fattr->change_attr;
}
} else if (server->caps & NFS_CAP_CHANGE_ATTR)
- invalid |= save_cache_validity;
+ nfsi->cache_validity |= save_cache_validity;
if (fattr->valid & NFS_ATTR_FATTR_MTIME) {
memcpy(&inode->i_mtime, &fattr->mtime, sizeof(inode->i_mtime));
} else if (server->caps & NFS_CAP_MTIME)
- invalid |= save_cache_validity & (NFS_INO_INVALID_ATTR
+ nfsi->cache_validity |= save_cache_validity &
+ (NFS_INO_INVALID_ATTR
| NFS_INO_REVAL_FORCED);
if (fattr->valid & NFS_ATTR_FATTR_CTIME) {
memcpy(&inode->i_ctime, &fattr->ctime, sizeof(inode->i_ctime));
} else if (server->caps & NFS_CAP_CTIME)
- invalid |= save_cache_validity & (NFS_INO_INVALID_ATTR
+ nfsi->cache_validity |= save_cache_validity &
+ (NFS_INO_INVALID_ATTR
| NFS_INO_REVAL_FORCED);
/* Check if our cached file size is stale */
@@ -1608,7 +1610,8 @@
(long long)new_isize);
}
} else
- invalid |= save_cache_validity & (NFS_INO_INVALID_ATTR
+ nfsi->cache_validity |= save_cache_validity &
+ (NFS_INO_INVALID_ATTR
| NFS_INO_REVAL_PAGECACHE
| NFS_INO_REVAL_FORCED);
@@ -1616,7 +1619,8 @@
if (fattr->valid & NFS_ATTR_FATTR_ATIME)
memcpy(&inode->i_atime, &fattr->atime, sizeof(inode->i_atime));
else if (server->caps & NFS_CAP_ATIME)
- invalid |= save_cache_validity & (NFS_INO_INVALID_ATIME
+ nfsi->cache_validity |= save_cache_validity &
+ (NFS_INO_INVALID_ATIME
| NFS_INO_REVAL_FORCED);
if (fattr->valid & NFS_ATTR_FATTR_MODE) {
@@ -1627,7 +1631,8 @@
invalid |= NFS_INO_INVALID_ATTR|NFS_INO_INVALID_ACCESS|NFS_INO_INVALID_ACL;
}
} else if (server->caps & NFS_CAP_MODE)
- invalid |= save_cache_validity & (NFS_INO_INVALID_ATTR
+ nfsi->cache_validity |= save_cache_validity &
+ (NFS_INO_INVALID_ATTR
| NFS_INO_INVALID_ACCESS
| NFS_INO_INVALID_ACL
| NFS_INO_REVAL_FORCED);
@@ -1638,7 +1643,8 @@
inode->i_uid = fattr->uid;
}
} else if (server->caps & NFS_CAP_OWNER)
- invalid |= save_cache_validity & (NFS_INO_INVALID_ATTR
+ nfsi->cache_validity |= save_cache_validity &
+ (NFS_INO_INVALID_ATTR
| NFS_INO_INVALID_ACCESS
| NFS_INO_INVALID_ACL
| NFS_INO_REVAL_FORCED);
@@ -1649,7 +1655,8 @@
inode->i_gid = fattr->gid;
}
} else if (server->caps & NFS_CAP_OWNER_GROUP)
- invalid |= save_cache_validity & (NFS_INO_INVALID_ATTR
+ nfsi->cache_validity |= save_cache_validity &
+ (NFS_INO_INVALID_ATTR
| NFS_INO_INVALID_ACCESS
| NFS_INO_INVALID_ACL
| NFS_INO_REVAL_FORCED);
@@ -1662,7 +1669,8 @@
set_nlink(inode, fattr->nlink);
}
} else if (server->caps & NFS_CAP_NLINK)
- invalid |= save_cache_validity & (NFS_INO_INVALID_ATTR
+ nfsi->cache_validity |= save_cache_validity &
+ (NFS_INO_INVALID_ATTR
| NFS_INO_REVAL_FORCED);
if (fattr->valid & NFS_ATTR_FATTR_SPACE_USED) {
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index dd8bfc2..8b69cba 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -231,13 +231,20 @@
extern int __init nfs_init_directcache(void);
extern void nfs_destroy_directcache(void);
-extern bool nfs_pgarray_set(struct nfs_page_array *p, unsigned int pagecount);
extern void nfs_pgheader_init(struct nfs_pageio_descriptor *desc,
struct nfs_pgio_header *hdr,
void (*release)(struct nfs_pgio_header *hdr));
void nfs_set_pgio_error(struct nfs_pgio_header *hdr, int error, loff_t pos);
int nfs_iocounter_wait(struct nfs_io_counter *c);
+extern const struct nfs_pageio_ops nfs_pgio_rw_ops;
+struct nfs_rw_header *nfs_rw_header_alloc(const struct nfs_rw_ops *);
+void nfs_rw_header_free(struct nfs_pgio_header *);
+void nfs_pgio_data_release(struct nfs_pgio_data *);
+int nfs_generic_pgio(struct nfs_pageio_descriptor *, struct nfs_pgio_header *);
+int nfs_initiate_pgio(struct rpc_clnt *, struct nfs_pgio_data *,
+ const struct rpc_call_ops *, int, int);
+
static inline void nfs_iocounter_init(struct nfs_io_counter *c)
{
c->flags = 0;
@@ -395,19 +402,11 @@
struct nfs_pgio_completion_ops;
/* read.c */
-extern struct nfs_read_header *nfs_readhdr_alloc(void);
-extern void nfs_readhdr_free(struct nfs_pgio_header *hdr);
extern void nfs_pageio_init_read(struct nfs_pageio_descriptor *pgio,
- struct inode *inode,
+ struct inode *inode, bool force_mds,
const struct nfs_pgio_completion_ops *compl_ops);
-extern int nfs_initiate_read(struct rpc_clnt *clnt,
- struct nfs_read_data *data,
- const struct rpc_call_ops *call_ops, int flags);
extern void nfs_read_prepare(struct rpc_task *task, void *calldata);
-extern int nfs_generic_pagein(struct nfs_pageio_descriptor *desc,
- struct nfs_pgio_header *hdr);
extern void nfs_pageio_reset_read_mds(struct nfs_pageio_descriptor *pgio);
-extern void nfs_readdata_release(struct nfs_read_data *rdata);
/* super.c */
void nfs_clone_super(struct super_block *, struct nfs_mount_info *);
@@ -422,19 +421,10 @@
/* write.c */
extern void nfs_pageio_init_write(struct nfs_pageio_descriptor *pgio,
- struct inode *inode, int ioflags,
+ struct inode *inode, int ioflags, bool force_mds,
const struct nfs_pgio_completion_ops *compl_ops);
-extern struct nfs_write_header *nfs_writehdr_alloc(void);
-extern void nfs_writehdr_free(struct nfs_pgio_header *hdr);
-extern int nfs_generic_flush(struct nfs_pageio_descriptor *desc,
- struct nfs_pgio_header *hdr);
extern void nfs_pageio_reset_write_mds(struct nfs_pageio_descriptor *pgio);
-extern void nfs_writedata_release(struct nfs_write_data *wdata);
extern void nfs_commit_free(struct nfs_commit_data *p);
-extern int nfs_initiate_write(struct rpc_clnt *clnt,
- struct nfs_write_data *data,
- const struct rpc_call_ops *call_ops,
- int how, int flags);
extern void nfs_write_prepare(struct rpc_task *task, void *calldata);
extern void nfs_commit_prepare(struct rpc_task *task, void *calldata);
extern int nfs_initiate_commit(struct rpc_clnt *clnt,
@@ -447,6 +437,7 @@
struct nfs_commit_info *cinfo);
int nfs_scan_commit_list(struct list_head *src, struct list_head *dst,
struct nfs_commit_info *cinfo, int max);
+unsigned long nfs_reqs_to_commit(struct nfs_commit_info *);
int nfs_scan_commit(struct inode *inode, struct list_head *dst,
struct nfs_commit_info *cinfo);
void nfs_mark_request_commit(struct nfs_page *req,
@@ -492,7 +483,7 @@
extern ssize_t nfs_dreq_bytes_left(struct nfs_direct_req *dreq);
/* nfs4proc.c */
-extern void __nfs4_read_done_cb(struct nfs_read_data *);
+extern void __nfs4_read_done_cb(struct nfs_pgio_data *);
extern struct nfs_client *nfs4_init_client(struct nfs_client *clp,
const struct rpc_timeout *timeparms,
const char *ip_addr);
diff --git a/fs/nfs/nfs2xdr.c b/fs/nfs/nfs2xdr.c
index 62db136..5f61b83 100644
--- a/fs/nfs/nfs2xdr.c
+++ b/fs/nfs/nfs2xdr.c
@@ -103,7 +103,7 @@
/*
* typedef opaque nfsdata<>;
*/
-static int decode_nfsdata(struct xdr_stream *xdr, struct nfs_readres *result)
+static int decode_nfsdata(struct xdr_stream *xdr, struct nfs_pgio_res *result)
{
u32 recvd, count;
__be32 *p;
@@ -613,7 +613,7 @@
* };
*/
static void encode_readargs(struct xdr_stream *xdr,
- const struct nfs_readargs *args)
+ const struct nfs_pgio_args *args)
{
u32 offset = args->offset;
u32 count = args->count;
@@ -629,7 +629,7 @@
static void nfs2_xdr_enc_readargs(struct rpc_rqst *req,
struct xdr_stream *xdr,
- const struct nfs_readargs *args)
+ const struct nfs_pgio_args *args)
{
encode_readargs(xdr, args);
prepare_reply_buffer(req, args->pages, args->pgbase,
@@ -649,7 +649,7 @@
* };
*/
static void encode_writeargs(struct xdr_stream *xdr,
- const struct nfs_writeargs *args)
+ const struct nfs_pgio_args *args)
{
u32 offset = args->offset;
u32 count = args->count;
@@ -669,7 +669,7 @@
static void nfs2_xdr_enc_writeargs(struct rpc_rqst *req,
struct xdr_stream *xdr,
- const struct nfs_writeargs *args)
+ const struct nfs_pgio_args *args)
{
encode_writeargs(xdr, args);
xdr->buf->flags |= XDRBUF_WRITE;
@@ -857,7 +857,7 @@
* };
*/
static int nfs2_xdr_dec_readres(struct rpc_rqst *req, struct xdr_stream *xdr,
- struct nfs_readres *result)
+ struct nfs_pgio_res *result)
{
enum nfs_stat status;
int error;
@@ -878,7 +878,7 @@
}
static int nfs2_xdr_dec_writeres(struct rpc_rqst *req, struct xdr_stream *xdr,
- struct nfs_writeres *result)
+ struct nfs_pgio_res *result)
{
/* All NFSv2 writes are "file sync" writes */
result->verf->committed = NFS_FILE_SYNC;
diff --git a/fs/nfs/nfs3proc.c b/fs/nfs/nfs3proc.c
index db60149..e7daa42 100644
--- a/fs/nfs/nfs3proc.c
+++ b/fs/nfs/nfs3proc.c
@@ -795,7 +795,7 @@
return status;
}
-static int nfs3_read_done(struct rpc_task *task, struct nfs_read_data *data)
+static int nfs3_read_done(struct rpc_task *task, struct nfs_pgio_data *data)
{
struct inode *inode = data->header->inode;
@@ -807,18 +807,18 @@
return 0;
}
-static void nfs3_proc_read_setup(struct nfs_read_data *data, struct rpc_message *msg)
+static void nfs3_proc_read_setup(struct nfs_pgio_data *data, struct rpc_message *msg)
{
msg->rpc_proc = &nfs3_procedures[NFS3PROC_READ];
}
-static int nfs3_proc_read_rpc_prepare(struct rpc_task *task, struct nfs_read_data *data)
+static int nfs3_proc_pgio_rpc_prepare(struct rpc_task *task, struct nfs_pgio_data *data)
{
rpc_call_start(task);
return 0;
}
-static int nfs3_write_done(struct rpc_task *task, struct nfs_write_data *data)
+static int nfs3_write_done(struct rpc_task *task, struct nfs_pgio_data *data)
{
struct inode *inode = data->header->inode;
@@ -829,17 +829,11 @@
return 0;
}
-static void nfs3_proc_write_setup(struct nfs_write_data *data, struct rpc_message *msg)
+static void nfs3_proc_write_setup(struct nfs_pgio_data *data, struct rpc_message *msg)
{
msg->rpc_proc = &nfs3_procedures[NFS3PROC_WRITE];
}
-static int nfs3_proc_write_rpc_prepare(struct rpc_task *task, struct nfs_write_data *data)
-{
- rpc_call_start(task);
- return 0;
-}
-
static void nfs3_proc_commit_rpc_prepare(struct rpc_task *task, struct nfs_commit_data *data)
{
rpc_call_start(task);
@@ -946,13 +940,10 @@
.fsinfo = nfs3_proc_fsinfo,
.pathconf = nfs3_proc_pathconf,
.decode_dirent = nfs3_decode_dirent,
+ .pgio_rpc_prepare = nfs3_proc_pgio_rpc_prepare,
.read_setup = nfs3_proc_read_setup,
- .read_pageio_init = nfs_pageio_init_read,
- .read_rpc_prepare = nfs3_proc_read_rpc_prepare,
.read_done = nfs3_read_done,
.write_setup = nfs3_proc_write_setup,
- .write_pageio_init = nfs_pageio_init_write,
- .write_rpc_prepare = nfs3_proc_write_rpc_prepare,
.write_done = nfs3_write_done,
.commit_setup = nfs3_proc_commit_setup,
.commit_rpc_prepare = nfs3_proc_commit_rpc_prepare,
diff --git a/fs/nfs/nfs3xdr.c b/fs/nfs/nfs3xdr.c
index fa6d721..8f4cbe7 100644
--- a/fs/nfs/nfs3xdr.c
+++ b/fs/nfs/nfs3xdr.c
@@ -953,7 +953,7 @@
* };
*/
static void encode_read3args(struct xdr_stream *xdr,
- const struct nfs_readargs *args)
+ const struct nfs_pgio_args *args)
{
__be32 *p;
@@ -966,7 +966,7 @@
static void nfs3_xdr_enc_read3args(struct rpc_rqst *req,
struct xdr_stream *xdr,
- const struct nfs_readargs *args)
+ const struct nfs_pgio_args *args)
{
encode_read3args(xdr, args);
prepare_reply_buffer(req, args->pages, args->pgbase,
@@ -992,7 +992,7 @@
* };
*/
static void encode_write3args(struct xdr_stream *xdr,
- const struct nfs_writeargs *args)
+ const struct nfs_pgio_args *args)
{
__be32 *p;
@@ -1008,7 +1008,7 @@
static void nfs3_xdr_enc_write3args(struct rpc_rqst *req,
struct xdr_stream *xdr,
- const struct nfs_writeargs *args)
+ const struct nfs_pgio_args *args)
{
encode_write3args(xdr, args);
xdr->buf->flags |= XDRBUF_WRITE;
@@ -1589,7 +1589,7 @@
* };
*/
static int decode_read3resok(struct xdr_stream *xdr,
- struct nfs_readres *result)
+ struct nfs_pgio_res *result)
{
u32 eof, count, ocount, recvd;
__be32 *p;
@@ -1625,7 +1625,7 @@
}
static int nfs3_xdr_dec_read3res(struct rpc_rqst *req, struct xdr_stream *xdr,
- struct nfs_readres *result)
+ struct nfs_pgio_res *result)
{
enum nfs_stat status;
int error;
@@ -1673,7 +1673,7 @@
* };
*/
static int decode_write3resok(struct xdr_stream *xdr,
- struct nfs_writeres *result)
+ struct nfs_pgio_res *result)
{
__be32 *p;
@@ -1697,7 +1697,7 @@
}
static int nfs3_xdr_dec_write3res(struct rpc_rqst *req, struct xdr_stream *xdr,
- struct nfs_writeres *result)
+ struct nfs_pgio_res *result)
{
enum nfs_stat status;
int error;
diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index e1d1bad..f63cb87 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -337,7 +337,7 @@
*/
static inline void
nfs4_state_protect_write(struct nfs_client *clp, struct rpc_clnt **clntp,
- struct rpc_message *msg, struct nfs_write_data *wdata)
+ struct rpc_message *msg, struct nfs_pgio_data *wdata)
{
if (_nfs4_state_protect(clp, NFS_SP4_MACH_CRED_WRITE, clntp, msg) &&
!test_bit(NFS_SP4_MACH_CRED_COMMIT, &clp->cl_sp4_flags))
@@ -369,7 +369,7 @@
static inline void
nfs4_state_protect_write(struct nfs_client *clp, struct rpc_clnt **clntp,
- struct rpc_message *msg, struct nfs_write_data *wdata)
+ struct rpc_message *msg, struct nfs_pgio_data *wdata)
{
}
#endif /* CONFIG_NFS_V4_1 */
diff --git a/fs/nfs/nfs4file.c b/fs/nfs/nfs4file.c
index 8de3407..464db9d 100644
--- a/fs/nfs/nfs4file.c
+++ b/fs/nfs/nfs4file.c
@@ -100,8 +100,7 @@
break;
mutex_lock(&inode->i_mutex);
ret = nfs_file_fsync_commit(file, start, end, datasync);
- if (!ret && !datasync)
- /* application has asked for meta-data sync */
+ if (!ret)
ret = pnfs_layoutcommit_inode(inode, true);
mutex_unlock(&inode->i_mutex);
/*
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 397be39..285ad53 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -2027,7 +2027,7 @@
return status;
}
if (!(o_res->f_attr->valid & NFS_ATTR_FATTR))
- _nfs4_proc_getattr(server, &o_res->fh, o_res->f_attr, o_res->f_label);
+ nfs4_proc_getattr(server, &o_res->fh, o_res->f_attr, o_res->f_label);
return 0;
}
@@ -2750,7 +2750,7 @@
#define FATTR4_WORD1_NFS40_MASK (2*FATTR4_WORD1_MOUNTED_ON_FILEID - 1UL)
#define FATTR4_WORD2_NFS41_MASK (2*FATTR4_WORD2_SUPPATTR_EXCLCREAT - 1UL)
-#define FATTR4_WORD2_NFS42_MASK (2*FATTR4_WORD2_CHANGE_SECURITY_LABEL - 1UL)
+#define FATTR4_WORD2_NFS42_MASK (2*FATTR4_WORD2_SECURITY_LABEL - 1UL)
static int _nfs4_server_capabilities(struct nfs_server *server, struct nfs_fh *fhandle)
{
@@ -4033,12 +4033,12 @@
return false;
}
-void __nfs4_read_done_cb(struct nfs_read_data *data)
+void __nfs4_read_done_cb(struct nfs_pgio_data *data)
{
nfs_invalidate_atime(data->header->inode);
}
-static int nfs4_read_done_cb(struct rpc_task *task, struct nfs_read_data *data)
+static int nfs4_read_done_cb(struct rpc_task *task, struct nfs_pgio_data *data)
{
struct nfs_server *server = NFS_SERVER(data->header->inode);
@@ -4055,7 +4055,7 @@
}
static bool nfs4_read_stateid_changed(struct rpc_task *task,
- struct nfs_readargs *args)
+ struct nfs_pgio_args *args)
{
if (!nfs4_error_stateid_expired(task->tk_status) ||
@@ -4068,7 +4068,7 @@
return true;
}
-static int nfs4_read_done(struct rpc_task *task, struct nfs_read_data *data)
+static int nfs4_read_done(struct rpc_task *task, struct nfs_pgio_data *data)
{
dprintk("--> %s\n", __func__);
@@ -4077,19 +4077,19 @@
return -EAGAIN;
if (nfs4_read_stateid_changed(task, &data->args))
return -EAGAIN;
- return data->read_done_cb ? data->read_done_cb(task, data) :
+ return data->pgio_done_cb ? data->pgio_done_cb(task, data) :
nfs4_read_done_cb(task, data);
}
-static void nfs4_proc_read_setup(struct nfs_read_data *data, struct rpc_message *msg)
+static void nfs4_proc_read_setup(struct nfs_pgio_data *data, struct rpc_message *msg)
{
data->timestamp = jiffies;
- data->read_done_cb = nfs4_read_done_cb;
+ data->pgio_done_cb = nfs4_read_done_cb;
msg->rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_READ];
nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 0);
}
-static int nfs4_proc_read_rpc_prepare(struct rpc_task *task, struct nfs_read_data *data)
+static int nfs4_proc_pgio_rpc_prepare(struct rpc_task *task, struct nfs_pgio_data *data)
{
if (nfs4_setup_sequence(NFS_SERVER(data->header->inode),
&data->args.seq_args,
@@ -4097,14 +4097,14 @@
task))
return 0;
if (nfs4_set_rw_stateid(&data->args.stateid, data->args.context,
- data->args.lock_context, FMODE_READ) == -EIO)
+ data->args.lock_context, data->header->rw_ops->rw_mode) == -EIO)
return -EIO;
if (unlikely(test_bit(NFS_CONTEXT_BAD, &data->args.context->flags)))
return -EIO;
return 0;
}
-static int nfs4_write_done_cb(struct rpc_task *task, struct nfs_write_data *data)
+static int nfs4_write_done_cb(struct rpc_task *task, struct nfs_pgio_data *data)
{
struct inode *inode = data->header->inode;
@@ -4121,7 +4121,7 @@
}
static bool nfs4_write_stateid_changed(struct rpc_task *task,
- struct nfs_writeargs *args)
+ struct nfs_pgio_args *args)
{
if (!nfs4_error_stateid_expired(task->tk_status) ||
@@ -4134,18 +4134,18 @@
return true;
}
-static int nfs4_write_done(struct rpc_task *task, struct nfs_write_data *data)
+static int nfs4_write_done(struct rpc_task *task, struct nfs_pgio_data *data)
{
if (!nfs4_sequence_done(task, &data->res.seq_res))
return -EAGAIN;
if (nfs4_write_stateid_changed(task, &data->args))
return -EAGAIN;
- return data->write_done_cb ? data->write_done_cb(task, data) :
+ return data->pgio_done_cb ? data->pgio_done_cb(task, data) :
nfs4_write_done_cb(task, data);
}
static
-bool nfs4_write_need_cache_consistency_data(const struct nfs_write_data *data)
+bool nfs4_write_need_cache_consistency_data(const struct nfs_pgio_data *data)
{
const struct nfs_pgio_header *hdr = data->header;
@@ -4158,7 +4158,7 @@
return nfs4_have_delegation(hdr->inode, FMODE_READ) == 0;
}
-static void nfs4_proc_write_setup(struct nfs_write_data *data, struct rpc_message *msg)
+static void nfs4_proc_write_setup(struct nfs_pgio_data *data, struct rpc_message *msg)
{
struct nfs_server *server = NFS_SERVER(data->header->inode);
@@ -4168,8 +4168,8 @@
} else
data->args.bitmask = server->cache_consistency_bitmask;
- if (!data->write_done_cb)
- data->write_done_cb = nfs4_write_done_cb;
+ if (!data->pgio_done_cb)
+ data->pgio_done_cb = nfs4_write_done_cb;
data->res.server = server;
data->timestamp = jiffies;
@@ -4177,21 +4177,6 @@
nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
}
-static int nfs4_proc_write_rpc_prepare(struct rpc_task *task, struct nfs_write_data *data)
-{
- if (nfs4_setup_sequence(NFS_SERVER(data->header->inode),
- &data->args.seq_args,
- &data->res.seq_res,
- task))
- return 0;
- if (nfs4_set_rw_stateid(&data->args.stateid, data->args.context,
- data->args.lock_context, FMODE_WRITE) == -EIO)
- return -EIO;
- if (unlikely(test_bit(NFS_CONTEXT_BAD, &data->args.context->flags)))
- return -EIO;
- return 0;
-}
-
static void nfs4_proc_commit_rpc_prepare(struct rpc_task *task, struct nfs_commit_data *data)
{
nfs4_setup_sequence(NFS_SERVER(data->inode),
@@ -8432,13 +8417,10 @@
.pathconf = nfs4_proc_pathconf,
.set_capabilities = nfs4_server_capabilities,
.decode_dirent = nfs4_decode_dirent,
+ .pgio_rpc_prepare = nfs4_proc_pgio_rpc_prepare,
.read_setup = nfs4_proc_read_setup,
- .read_pageio_init = pnfs_pageio_init_read,
- .read_rpc_prepare = nfs4_proc_read_rpc_prepare,
.read_done = nfs4_read_done,
.write_setup = nfs4_proc_write_setup,
- .write_pageio_init = pnfs_pageio_init_write,
- .write_rpc_prepare = nfs4_proc_write_rpc_prepare,
.write_done = nfs4_write_done,
.commit_setup = nfs4_proc_commit_setup,
.commit_rpc_prepare = nfs4_proc_commit_rpc_prepare,
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index c0583b9..848f6853 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -1456,7 +1456,7 @@
* server that doesn't support a grace period.
*/
spin_lock(&sp->so_lock);
- write_seqcount_begin(&sp->so_reclaim_seqcount);
+ raw_write_seqcount_begin(&sp->so_reclaim_seqcount);
restart:
list_for_each_entry(state, &sp->so_states, open_states) {
if (!test_and_clear_bit(ops->state_flag_bit, &state->flags))
@@ -1519,13 +1519,13 @@
spin_lock(&sp->so_lock);
goto restart;
}
- write_seqcount_end(&sp->so_reclaim_seqcount);
+ raw_write_seqcount_end(&sp->so_reclaim_seqcount);
spin_unlock(&sp->so_lock);
return 0;
out_err:
nfs4_put_open_state(state);
spin_lock(&sp->so_lock);
- write_seqcount_end(&sp->so_reclaim_seqcount);
+ raw_write_seqcount_end(&sp->so_reclaim_seqcount);
spin_unlock(&sp->so_lock);
return status;
}
diff --git a/fs/nfs/nfs4sysctl.c b/fs/nfs/nfs4sysctl.c
index 2628d92..b6ebe7e 100644
--- a/fs/nfs/nfs4sysctl.c
+++ b/fs/nfs/nfs4sysctl.c
@@ -16,7 +16,7 @@
static const int nfs_set_port_max = 65535;
static struct ctl_table_header *nfs4_callback_sysctl_table;
-static ctl_table nfs4_cb_sysctls[] = {
+static struct ctl_table nfs4_cb_sysctls[] = {
{
.procname = "nfs_callback_tcpport",
.data = &nfs_callback_set_tcpport,
@@ -36,7 +36,7 @@
{ }
};
-static ctl_table nfs4_cb_sysctl_dir[] = {
+static struct ctl_table nfs4_cb_sysctl_dir[] = {
{
.procname = "nfs",
.mode = 0555,
@@ -45,7 +45,7 @@
{ }
};
-static ctl_table nfs4_cb_sysctl_root[] = {
+static struct ctl_table nfs4_cb_sysctl_root[] = {
{
.procname = "fs",
.mode = 0555,
diff --git a/fs/nfs/nfs4trace.h b/fs/nfs/nfs4trace.h
index 849cf14..0a744f3 100644
--- a/fs/nfs/nfs4trace.h
+++ b/fs/nfs/nfs4trace.h
@@ -932,7 +932,7 @@
DECLARE_EVENT_CLASS(nfs4_read_event,
TP_PROTO(
- const struct nfs_read_data *data,
+ const struct nfs_pgio_data *data,
int error
),
@@ -972,7 +972,7 @@
#define DEFINE_NFS4_READ_EVENT(name) \
DEFINE_EVENT(nfs4_read_event, name, \
TP_PROTO( \
- const struct nfs_read_data *data, \
+ const struct nfs_pgio_data *data, \
int error \
), \
TP_ARGS(data, error))
@@ -983,7 +983,7 @@
DECLARE_EVENT_CLASS(nfs4_write_event,
TP_PROTO(
- const struct nfs_write_data *data,
+ const struct nfs_pgio_data *data,
int error
),
@@ -1024,7 +1024,7 @@
#define DEFINE_NFS4_WRITE_EVENT(name) \
DEFINE_EVENT(nfs4_write_event, name, \
TP_PROTO( \
- const struct nfs_write_data *data, \
+ const struct nfs_pgio_data *data, \
int error \
), \
TP_ARGS(data, error))
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index 73ce8d4..939ae60 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -1556,7 +1556,8 @@
encode_op_hdr(xdr, OP_PUTROOTFH, decode_putrootfh_maxsz, hdr);
}
-static void encode_read(struct xdr_stream *xdr, const struct nfs_readargs *args, struct compound_hdr *hdr)
+static void encode_read(struct xdr_stream *xdr, const struct nfs_pgio_args *args,
+ struct compound_hdr *hdr)
{
__be32 *p;
@@ -1701,7 +1702,8 @@
encode_nfs4_verifier(xdr, &arg->confirm);
}
-static void encode_write(struct xdr_stream *xdr, const struct nfs_writeargs *args, struct compound_hdr *hdr)
+static void encode_write(struct xdr_stream *xdr, const struct nfs_pgio_args *args,
+ struct compound_hdr *hdr)
{
__be32 *p;
@@ -2451,7 +2453,7 @@
* Encode a READ request
*/
static void nfs4_xdr_enc_read(struct rpc_rqst *req, struct xdr_stream *xdr,
- struct nfs_readargs *args)
+ struct nfs_pgio_args *args)
{
struct compound_hdr hdr = {
.minorversion = nfs4_xdr_minorversion(&args->seq_args),
@@ -2513,7 +2515,7 @@
* Encode a WRITE request
*/
static void nfs4_xdr_enc_write(struct rpc_rqst *req, struct xdr_stream *xdr,
- struct nfs_writeargs *args)
+ struct nfs_pgio_args *args)
{
struct compound_hdr hdr = {
.minorversion = nfs4_xdr_minorversion(&args->seq_args),
@@ -5085,7 +5087,8 @@
return decode_op_hdr(xdr, OP_PUTROOTFH);
}
-static int decode_read(struct xdr_stream *xdr, struct rpc_rqst *req, struct nfs_readres *res)
+static int decode_read(struct xdr_stream *xdr, struct rpc_rqst *req,
+ struct nfs_pgio_res *res)
{
__be32 *p;
uint32_t count, eof, recvd;
@@ -5339,7 +5342,7 @@
return decode_op_hdr(xdr, OP_SETCLIENTID_CONFIRM);
}
-static int decode_write(struct xdr_stream *xdr, struct nfs_writeres *res)
+static int decode_write(struct xdr_stream *xdr, struct nfs_pgio_res *res)
{
__be32 *p;
int status;
@@ -6636,7 +6639,7 @@
* Decode Read response
*/
static int nfs4_xdr_dec_read(struct rpc_rqst *rqstp, struct xdr_stream *xdr,
- struct nfs_readres *res)
+ struct nfs_pgio_res *res)
{
struct compound_hdr hdr;
int status;
@@ -6661,7 +6664,7 @@
* Decode WRITE response
*/
static int nfs4_xdr_dec_write(struct rpc_rqst *rqstp, struct xdr_stream *xdr,
- struct nfs_writeres *res)
+ struct nfs_pgio_res *res)
{
struct compound_hdr hdr;
int status;
diff --git a/fs/nfs/objlayout/objio_osd.c b/fs/nfs/objlayout/objio_osd.c
index 5457745..6113207 100644
--- a/fs/nfs/objlayout/objio_osd.c
+++ b/fs/nfs/objlayout/objio_osd.c
@@ -439,7 +439,7 @@
objlayout_read_done(&objios->oir, status, objios->sync);
}
-int objio_read_pagelist(struct nfs_read_data *rdata)
+int objio_read_pagelist(struct nfs_pgio_data *rdata)
{
struct nfs_pgio_header *hdr = rdata->header;
struct objio_state *objios;
@@ -487,7 +487,7 @@
static struct page *__r4w_get_page(void *priv, u64 offset, bool *uptodate)
{
struct objio_state *objios = priv;
- struct nfs_write_data *wdata = objios->oir.rpcdata;
+ struct nfs_pgio_data *wdata = objios->oir.rpcdata;
struct address_space *mapping = wdata->header->inode->i_mapping;
pgoff_t index = offset / PAGE_SIZE;
struct page *page;
@@ -531,7 +531,7 @@
.put_page = &__r4w_put_page,
};
-int objio_write_pagelist(struct nfs_write_data *wdata, int how)
+int objio_write_pagelist(struct nfs_pgio_data *wdata, int how)
{
struct nfs_pgio_header *hdr = wdata->header;
struct objio_state *objios;
@@ -564,14 +564,22 @@
return 0;
}
-static bool objio_pg_test(struct nfs_pageio_descriptor *pgio,
+/*
+ * Return 0 if @req cannot be coalesced into @pgio, otherwise return the number
+ * of bytes (maximum @req->wb_bytes) that can be coalesced.
+ */
+static size_t objio_pg_test(struct nfs_pageio_descriptor *pgio,
struct nfs_page *prev, struct nfs_page *req)
{
- if (!pnfs_generic_pg_test(pgio, prev, req))
- return false;
+ unsigned int size;
- return pgio->pg_count + req->wb_bytes <=
- (unsigned long)pgio->pg_layout_private;
+ size = pnfs_generic_pg_test(pgio, prev, req);
+
+ if (!size || pgio->pg_count + req->wb_bytes >
+ (unsigned long)pgio->pg_layout_private)
+ return 0;
+
+ return min(size, req->wb_bytes);
}
static void objio_init_read(struct nfs_pageio_descriptor *pgio, struct nfs_page *req)
diff --git a/fs/nfs/objlayout/objlayout.c b/fs/nfs/objlayout/objlayout.c
index e4f9cbf..765d3f5 100644
--- a/fs/nfs/objlayout/objlayout.c
+++ b/fs/nfs/objlayout/objlayout.c
@@ -53,10 +53,10 @@
struct objlayout *objlay;
objlay = kzalloc(sizeof(struct objlayout), gfp_flags);
- if (objlay) {
- spin_lock_init(&objlay->lock);
- INIT_LIST_HEAD(&objlay->err_list);
- }
+ if (!objlay)
+ return NULL;
+ spin_lock_init(&objlay->lock);
+ INIT_LIST_HEAD(&objlay->err_list);
dprintk("%s: Return %p\n", __func__, objlay);
return &objlay->pnfs_layout;
}
@@ -229,11 +229,11 @@
static void _rpc_read_complete(struct work_struct *work)
{
struct rpc_task *task;
- struct nfs_read_data *rdata;
+ struct nfs_pgio_data *rdata;
dprintk("%s enter\n", __func__);
task = container_of(work, struct rpc_task, u.tk_work);
- rdata = container_of(task, struct nfs_read_data, task);
+ rdata = container_of(task, struct nfs_pgio_data, task);
pnfs_ld_read_done(rdata);
}
@@ -241,7 +241,7 @@
void
objlayout_read_done(struct objlayout_io_res *oir, ssize_t status, bool sync)
{
- struct nfs_read_data *rdata = oir->rpcdata;
+ struct nfs_pgio_data *rdata = oir->rpcdata;
oir->status = rdata->task.tk_status = status;
if (status >= 0)
@@ -266,7 +266,7 @@
* Perform sync or async reads.
*/
enum pnfs_try_status
-objlayout_read_pagelist(struct nfs_read_data *rdata)
+objlayout_read_pagelist(struct nfs_pgio_data *rdata)
{
struct nfs_pgio_header *hdr = rdata->header;
struct inode *inode = hdr->inode;
@@ -312,11 +312,11 @@
static void _rpc_write_complete(struct work_struct *work)
{
struct rpc_task *task;
- struct nfs_write_data *wdata;
+ struct nfs_pgio_data *wdata;
dprintk("%s enter\n", __func__);
task = container_of(work, struct rpc_task, u.tk_work);
- wdata = container_of(task, struct nfs_write_data, task);
+ wdata = container_of(task, struct nfs_pgio_data, task);
pnfs_ld_write_done(wdata);
}
@@ -324,7 +324,7 @@
void
objlayout_write_done(struct objlayout_io_res *oir, ssize_t status, bool sync)
{
- struct nfs_write_data *wdata = oir->rpcdata;
+ struct nfs_pgio_data *wdata = oir->rpcdata;
oir->status = wdata->task.tk_status = status;
if (status >= 0) {
@@ -351,7 +351,7 @@
* Perform sync or async writes.
*/
enum pnfs_try_status
-objlayout_write_pagelist(struct nfs_write_data *wdata,
+objlayout_write_pagelist(struct nfs_pgio_data *wdata,
int how)
{
struct nfs_pgio_header *hdr = wdata->header;
diff --git a/fs/nfs/objlayout/objlayout.h b/fs/nfs/objlayout/objlayout.h
index 87aa1de..01e0410 100644
--- a/fs/nfs/objlayout/objlayout.h
+++ b/fs/nfs/objlayout/objlayout.h
@@ -119,8 +119,8 @@
*/
extern void objio_free_result(struct objlayout_io_res *oir);
-extern int objio_read_pagelist(struct nfs_read_data *rdata);
-extern int objio_write_pagelist(struct nfs_write_data *wdata, int how);
+extern int objio_read_pagelist(struct nfs_pgio_data *rdata);
+extern int objio_write_pagelist(struct nfs_pgio_data *wdata, int how);
/*
* callback API
@@ -168,10 +168,10 @@
extern void objlayout_free_lseg(struct pnfs_layout_segment *);
extern enum pnfs_try_status objlayout_read_pagelist(
- struct nfs_read_data *);
+ struct nfs_pgio_data *);
extern enum pnfs_try_status objlayout_write_pagelist(
- struct nfs_write_data *,
+ struct nfs_pgio_data *,
int how);
extern void objlayout_encode_layoutcommit(
diff --git a/fs/nfs/pagelist.c b/fs/nfs/pagelist.c
index 03ed984..b6ee3a6 100644
--- a/fs/nfs/pagelist.c
+++ b/fs/nfs/pagelist.c
@@ -24,9 +24,14 @@
#include "internal.h"
#include "pnfs.h"
-static struct kmem_cache *nfs_page_cachep;
+#define NFSDBG_FACILITY NFSDBG_PAGECACHE
-bool nfs_pgarray_set(struct nfs_page_array *p, unsigned int pagecount)
+static struct kmem_cache *nfs_page_cachep;
+static const struct rpc_call_ops nfs_pgio_common_ops;
+
+static void nfs_free_request(struct nfs_page *);
+
+static bool nfs_pgarray_set(struct nfs_page_array *p, unsigned int pagecount)
{
p->npages = pagecount;
if (pagecount <= ARRAY_SIZE(p->page_array))
@@ -133,11 +138,156 @@
return __nfs_iocounter_wait(c);
}
+static int nfs_wait_bit_uninterruptible(void *word)
+{
+ io_schedule();
+ return 0;
+}
+
+/*
+ * nfs_page_group_lock - lock the head of the page group
+ * @req - request in group that is to be locked
+ *
+ * this lock must be held if modifying the page group list
+ */
+void
+nfs_page_group_lock(struct nfs_page *req)
+{
+ struct nfs_page *head = req->wb_head;
+
+ WARN_ON_ONCE(head != head->wb_head);
+
+ wait_on_bit_lock(&head->wb_flags, PG_HEADLOCK,
+ nfs_wait_bit_uninterruptible,
+ TASK_UNINTERRUPTIBLE);
+}
+
+/*
+ * nfs_page_group_unlock - unlock the head of the page group
+ * @req - request in group that is to be unlocked
+ */
+void
+nfs_page_group_unlock(struct nfs_page *req)
+{
+ struct nfs_page *head = req->wb_head;
+
+ WARN_ON_ONCE(head != head->wb_head);
+
+ smp_mb__before_atomic();
+ clear_bit(PG_HEADLOCK, &head->wb_flags);
+ smp_mb__after_atomic();
+ wake_up_bit(&head->wb_flags, PG_HEADLOCK);
+}
+
+/*
+ * nfs_page_group_sync_on_bit_locked
+ *
+ * must be called with page group lock held
+ */
+static bool
+nfs_page_group_sync_on_bit_locked(struct nfs_page *req, unsigned int bit)
+{
+ struct nfs_page *head = req->wb_head;
+ struct nfs_page *tmp;
+
+ WARN_ON_ONCE(!test_bit(PG_HEADLOCK, &head->wb_flags));
+ WARN_ON_ONCE(test_and_set_bit(bit, &req->wb_flags));
+
+ tmp = req->wb_this_page;
+ while (tmp != req) {
+ if (!test_bit(bit, &tmp->wb_flags))
+ return false;
+ tmp = tmp->wb_this_page;
+ }
+
+ /* true! reset all bits */
+ tmp = req;
+ do {
+ clear_bit(bit, &tmp->wb_flags);
+ tmp = tmp->wb_this_page;
+ } while (tmp != req);
+
+ return true;
+}
+
+/*
+ * nfs_page_group_sync_on_bit - set bit on current request, but only
+ * return true if the bit is set for all requests in page group
+ * @req - request in page group
+ * @bit - PG_* bit that is used to sync page group
+ */
+bool nfs_page_group_sync_on_bit(struct nfs_page *req, unsigned int bit)
+{
+ bool ret;
+
+ nfs_page_group_lock(req);
+ ret = nfs_page_group_sync_on_bit_locked(req, bit);
+ nfs_page_group_unlock(req);
+
+ return ret;
+}
+
+/*
+ * nfs_page_group_init - Initialize the page group linkage for @req
+ * @req - a new nfs request
+ * @prev - the previous request in page group, or NULL if @req is the first
+ * or only request in the group (the head).
+ */
+static inline void
+nfs_page_group_init(struct nfs_page *req, struct nfs_page *prev)
+{
+ WARN_ON_ONCE(prev == req);
+
+ if (!prev) {
+ req->wb_head = req;
+ req->wb_this_page = req;
+ } else {
+ WARN_ON_ONCE(prev->wb_this_page != prev->wb_head);
+ WARN_ON_ONCE(!test_bit(PG_HEADLOCK, &prev->wb_head->wb_flags));
+ req->wb_head = prev->wb_head;
+ req->wb_this_page = prev->wb_this_page;
+ prev->wb_this_page = req;
+
+ /* grab extra ref if head request has extra ref from
+ * the write/commit path to handle handoff between write
+ * and commit lists */
+ if (test_bit(PG_INODE_REF, &prev->wb_head->wb_flags))
+ kref_get(&req->wb_kref);
+ }
+}
+
+/*
+ * nfs_page_group_destroy - sync the destruction of page groups
+ * @req - request that no longer needs the page group
+ *
+ * releases the page group reference from each member once all
+ * members have called this function.
+ */
+static void
+nfs_page_group_destroy(struct kref *kref)
+{
+ struct nfs_page *req = container_of(kref, struct nfs_page, wb_kref);
+ struct nfs_page *tmp, *next;
+
+ if (!nfs_page_group_sync_on_bit(req, PG_TEARDOWN))
+ return;
+
+ tmp = req;
+ do {
+ next = tmp->wb_this_page;
+ /* unlink and free */
+ tmp->wb_this_page = tmp;
+ tmp->wb_head = tmp;
+ nfs_free_request(tmp);
+ tmp = next;
+ } while (tmp != req);
+}
+
/**
* nfs_create_request - Create an NFS read/write request.
* @ctx: open context to use
- * @inode: inode to which the request is attached
* @page: page to write
+ * @last: last nfs request created for this page group or NULL if head
* @offset: starting offset within the page for the write
* @count: number of bytes to read/write
*
@@ -146,9 +296,9 @@
* User should ensure it is safe to sleep in this function.
*/
struct nfs_page *
-nfs_create_request(struct nfs_open_context *ctx, struct inode *inode,
- struct page *page,
- unsigned int offset, unsigned int count)
+nfs_create_request(struct nfs_open_context *ctx, struct page *page,
+ struct nfs_page *last, unsigned int offset,
+ unsigned int count)
{
struct nfs_page *req;
struct nfs_lock_context *l_ctx;
@@ -180,6 +330,7 @@
req->wb_bytes = count;
req->wb_context = get_nfs_open_context(ctx);
kref_init(&req->wb_kref);
+ nfs_page_group_init(req, last);
return req;
}
@@ -237,16 +388,22 @@
}
}
-
/**
* nfs_release_request - Release the count on an NFS read/write request
* @req: request to release
*
* Note: Should never be called with the spinlock held!
*/
-static void nfs_free_request(struct kref *kref)
+static void nfs_free_request(struct nfs_page *req)
{
- struct nfs_page *req = container_of(kref, struct nfs_page, wb_kref);
+ WARN_ON_ONCE(req->wb_this_page != req);
+
+ /* extra debug: make sure no sync bits are still set */
+ WARN_ON_ONCE(test_bit(PG_TEARDOWN, &req->wb_flags));
+ WARN_ON_ONCE(test_bit(PG_UNLOCKPAGE, &req->wb_flags));
+ WARN_ON_ONCE(test_bit(PG_UPTODATE, &req->wb_flags));
+ WARN_ON_ONCE(test_bit(PG_WB_END, &req->wb_flags));
+ WARN_ON_ONCE(test_bit(PG_REMOVE, &req->wb_flags));
/* Release struct file and open context */
nfs_clear_request(req);
@@ -255,13 +412,7 @@
void nfs_release_request(struct nfs_page *req)
{
- kref_put(&req->wb_kref, nfs_free_request);
-}
-
-static int nfs_wait_bit_uninterruptible(void *word)
-{
- io_schedule();
- return 0;
+ kref_put(&req->wb_kref, nfs_page_group_destroy);
}
/**
@@ -279,22 +430,249 @@
TASK_UNINTERRUPTIBLE);
}
-bool nfs_generic_pg_test(struct nfs_pageio_descriptor *desc, struct nfs_page *prev, struct nfs_page *req)
+/*
+ * nfs_generic_pg_test - determine if requests can be coalesced
+ * @desc: pointer to descriptor
+ * @prev: previous request in desc, or NULL
+ * @req: this request
+ *
+ * Returns zero if @req can be coalesced into @desc, otherwise it returns
+ * the size of the request.
+ */
+size_t nfs_generic_pg_test(struct nfs_pageio_descriptor *desc,
+ struct nfs_page *prev, struct nfs_page *req)
{
- /*
- * FIXME: ideally we should be able to coalesce all requests
- * that are not block boundary aligned, but currently this
- * is problematic for the case of bsize < PAGE_CACHE_SIZE,
- * since nfs_flush_multi and nfs_pagein_multi assume you
- * can have only one struct nfs_page.
- */
- if (desc->pg_bsize < PAGE_SIZE)
+ if (desc->pg_count > desc->pg_bsize) {
+ /* should never happen */
+ WARN_ON_ONCE(1);
return 0;
+ }
- return desc->pg_count + req->wb_bytes <= desc->pg_bsize;
+ return min(desc->pg_bsize - desc->pg_count, (size_t)req->wb_bytes);
}
EXPORT_SYMBOL_GPL(nfs_generic_pg_test);
+static inline struct nfs_rw_header *NFS_RW_HEADER(struct nfs_pgio_header *hdr)
+{
+ return container_of(hdr, struct nfs_rw_header, header);
+}
+
+/**
+ * nfs_rw_header_alloc - Allocate a header for a read or write
+ * @ops: Read or write function vector
+ */
+struct nfs_rw_header *nfs_rw_header_alloc(const struct nfs_rw_ops *ops)
+{
+ struct nfs_rw_header *header = ops->rw_alloc_header();
+
+ if (header) {
+ struct nfs_pgio_header *hdr = &header->header;
+
+ INIT_LIST_HEAD(&hdr->pages);
+ spin_lock_init(&hdr->lock);
+ atomic_set(&hdr->refcnt, 0);
+ hdr->rw_ops = ops;
+ }
+ return header;
+}
+EXPORT_SYMBOL_GPL(nfs_rw_header_alloc);
+
+/*
+ * nfs_rw_header_free - Free a read or write header
+ * @hdr: The header to free
+ */
+void nfs_rw_header_free(struct nfs_pgio_header *hdr)
+{
+ hdr->rw_ops->rw_free_header(NFS_RW_HEADER(hdr));
+}
+EXPORT_SYMBOL_GPL(nfs_rw_header_free);
+
+/**
+ * nfs_pgio_data_alloc - Allocate pageio data
+ * @hdr: The header making a request
+ * @pagecount: Number of pages to create
+ */
+static struct nfs_pgio_data *nfs_pgio_data_alloc(struct nfs_pgio_header *hdr,
+ unsigned int pagecount)
+{
+ struct nfs_pgio_data *data, *prealloc;
+
+ prealloc = &NFS_RW_HEADER(hdr)->rpc_data;
+ if (prealloc->header == NULL)
+ data = prealloc;
+ else
+ data = kzalloc(sizeof(*data), GFP_KERNEL);
+ if (!data)
+ goto out;
+
+ if (nfs_pgarray_set(&data->pages, pagecount)) {
+ data->header = hdr;
+ atomic_inc(&hdr->refcnt);
+ } else {
+ if (data != prealloc)
+ kfree(data);
+ data = NULL;
+ }
+out:
+ return data;
+}
+
+/**
+ * nfs_pgio_data_release - Properly free pageio data
+ * @data: The data to release
+ */
+void nfs_pgio_data_release(struct nfs_pgio_data *data)
+{
+ struct nfs_pgio_header *hdr = data->header;
+ struct nfs_rw_header *pageio_header = NFS_RW_HEADER(hdr);
+
+ put_nfs_open_context(data->args.context);
+ if (data->pages.pagevec != data->pages.page_array)
+ kfree(data->pages.pagevec);
+ if (data == &pageio_header->rpc_data) {
+ data->header = NULL;
+ data = NULL;
+ }
+ if (atomic_dec_and_test(&hdr->refcnt))
+ hdr->completion_ops->completion(hdr);
+ /* Note: we only free the rpc_task after callbacks are done.
+ * See the comment in rpc_free_task() for why
+ */
+ kfree(data);
+}
+EXPORT_SYMBOL_GPL(nfs_pgio_data_release);
+
+/**
+ * nfs_pgio_rpcsetup - Set up arguments for a pageio call
+ * @data: The pageio data
+ * @count: Number of bytes to read
+ * @offset: Initial offset
+ * @how: How to commit data (writes only)
+ * @cinfo: Commit information for the call (writes only)
+ */
+static void nfs_pgio_rpcsetup(struct nfs_pgio_data *data,
+ unsigned int count, unsigned int offset,
+ int how, struct nfs_commit_info *cinfo)
+{
+ struct nfs_page *req = data->header->req;
+
+ /* Set up the RPC argument and reply structs
+ * NB: take care not to mess about with data->commit et al. */
+
+ data->args.fh = NFS_FH(data->header->inode);
+ data->args.offset = req_offset(req) + offset;
+ /* pnfs_set_layoutcommit needs this */
+ data->mds_offset = data->args.offset;
+ data->args.pgbase = req->wb_pgbase + offset;
+ data->args.pages = data->pages.pagevec;
+ data->args.count = count;
+ data->args.context = get_nfs_open_context(req->wb_context);
+ data->args.lock_context = req->wb_lock_context;
+ data->args.stable = NFS_UNSTABLE;
+ switch (how & (FLUSH_STABLE | FLUSH_COND_STABLE)) {
+ case 0:
+ break;
+ case FLUSH_COND_STABLE:
+ if (nfs_reqs_to_commit(cinfo))
+ break;
+ default:
+ data->args.stable = NFS_FILE_SYNC;
+ }
+
+ data->res.fattr = &data->fattr;
+ data->res.count = count;
+ data->res.eof = 0;
+ data->res.verf = &data->verf;
+ nfs_fattr_init(&data->fattr);
+}
+
+/**
+ * nfs_pgio_prepare - Prepare pageio data to go over the wire
+ * @task: The current task
+ * @calldata: pageio data to prepare
+ */
+static void nfs_pgio_prepare(struct rpc_task *task, void *calldata)
+{
+ struct nfs_pgio_data *data = calldata;
+ int err;
+ err = NFS_PROTO(data->header->inode)->pgio_rpc_prepare(task, data);
+ if (err)
+ rpc_exit(task, err);
+}
+
+int nfs_initiate_pgio(struct rpc_clnt *clnt, struct nfs_pgio_data *data,
+ const struct rpc_call_ops *call_ops, int how, int flags)
+{
+ struct rpc_task *task;
+ struct rpc_message msg = {
+ .rpc_argp = &data->args,
+ .rpc_resp = &data->res,
+ .rpc_cred = data->header->cred,
+ };
+ struct rpc_task_setup task_setup_data = {
+ .rpc_client = clnt,
+ .task = &data->task,
+ .rpc_message = &msg,
+ .callback_ops = call_ops,
+ .callback_data = data,
+ .workqueue = nfsiod_workqueue,
+ .flags = RPC_TASK_ASYNC | flags,
+ };
+ int ret = 0;
+
+ data->header->rw_ops->rw_initiate(data, &msg, &task_setup_data, how);
+
+ dprintk("NFS: %5u initiated pgio call "
+ "(req %s/%llu, %u bytes @ offset %llu)\n",
+ data->task.tk_pid,
+ data->header->inode->i_sb->s_id,
+ (unsigned long long)NFS_FILEID(data->header->inode),
+ data->args.count,
+ (unsigned long long)data->args.offset);
+
+ task = rpc_run_task(&task_setup_data);
+ if (IS_ERR(task)) {
+ ret = PTR_ERR(task);
+ goto out;
+ }
+ if (how & FLUSH_SYNC) {
+ ret = rpc_wait_for_completion_task(task);
+ if (ret == 0)
+ ret = task->tk_status;
+ }
+ rpc_put_task(task);
+out:
+ return ret;
+}
+EXPORT_SYMBOL_GPL(nfs_initiate_pgio);
+
+/**
+ * nfs_pgio_error - Clean up from a pageio error
+ * @desc: IO descriptor
+ * @hdr: pageio header
+ */
+static int nfs_pgio_error(struct nfs_pageio_descriptor *desc,
+ struct nfs_pgio_header *hdr)
+{
+ set_bit(NFS_IOHDR_REDO, &hdr->flags);
+ nfs_pgio_data_release(hdr->data);
+ hdr->data = NULL;
+ desc->pg_completion_ops->error_cleanup(&desc->pg_list);
+ return -ENOMEM;
+}
+
+/**
+ * nfs_pgio_release - Release pageio data
+ * @calldata: The pageio data to release
+ */
+static void nfs_pgio_release(void *calldata)
+{
+ struct nfs_pgio_data *data = calldata;
+ if (data->header->rw_ops->rw_release)
+ data->header->rw_ops->rw_release(data);
+ nfs_pgio_data_release(data);
+}
+
/**
* nfs_pageio_init - initialise a page io descriptor
* @desc: pointer to descriptor
@@ -307,6 +685,7 @@
struct inode *inode,
const struct nfs_pageio_ops *pg_ops,
const struct nfs_pgio_completion_ops *compl_ops,
+ const struct nfs_rw_ops *rw_ops,
size_t bsize,
int io_flags)
{
@@ -320,6 +699,7 @@
desc->pg_inode = inode;
desc->pg_ops = pg_ops;
desc->pg_completion_ops = compl_ops;
+ desc->pg_rw_ops = rw_ops;
desc->pg_ioflags = io_flags;
desc->pg_error = 0;
desc->pg_lseg = NULL;
@@ -328,6 +708,94 @@
}
EXPORT_SYMBOL_GPL(nfs_pageio_init);
+/**
+ * nfs_pgio_result - Basic pageio error handling
+ * @task: The task that ran
+ * @calldata: Pageio data to check
+ */
+static void nfs_pgio_result(struct rpc_task *task, void *calldata)
+{
+ struct nfs_pgio_data *data = calldata;
+ struct inode *inode = data->header->inode;
+
+ dprintk("NFS: %s: %5u, (status %d)\n", __func__,
+ task->tk_pid, task->tk_status);
+
+ if (data->header->rw_ops->rw_done(task, data, inode) != 0)
+ return;
+ if (task->tk_status < 0)
+ nfs_set_pgio_error(data->header, task->tk_status, data->args.offset);
+ else
+ data->header->rw_ops->rw_result(task, data);
+}
+
+/*
+ * Create an RPC task for the given read or write request and kick it.
+ * The page must have been locked by the caller.
+ *
+ * It may happen that the page we're passed is not marked dirty.
+ * This is the case if nfs_updatepage detects a conflicting request
+ * that has been written but not committed.
+ */
+int nfs_generic_pgio(struct nfs_pageio_descriptor *desc,
+ struct nfs_pgio_header *hdr)
+{
+ struct nfs_page *req;
+ struct page **pages;
+ struct nfs_pgio_data *data;
+ struct list_head *head = &desc->pg_list;
+ struct nfs_commit_info cinfo;
+
+ data = nfs_pgio_data_alloc(hdr, nfs_page_array_len(desc->pg_base,
+ desc->pg_count));
+ if (!data)
+ return nfs_pgio_error(desc, hdr);
+
+ nfs_init_cinfo(&cinfo, desc->pg_inode, desc->pg_dreq);
+ pages = data->pages.pagevec;
+ while (!list_empty(head)) {
+ req = nfs_list_entry(head->next);
+ nfs_list_remove_request(req);
+ nfs_list_add_request(req, &hdr->pages);
+ *pages++ = req->wb_page;
+ }
+
+ if ((desc->pg_ioflags & FLUSH_COND_STABLE) &&
+ (desc->pg_moreio || nfs_reqs_to_commit(&cinfo)))
+ desc->pg_ioflags &= ~FLUSH_COND_STABLE;
+
+ /* Set up the argument struct */
+ nfs_pgio_rpcsetup(data, desc->pg_count, 0, desc->pg_ioflags, &cinfo);
+ hdr->data = data;
+ desc->pg_rpc_callops = &nfs_pgio_common_ops;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(nfs_generic_pgio);
+
+static int nfs_generic_pg_pgios(struct nfs_pageio_descriptor *desc)
+{
+ struct nfs_rw_header *rw_hdr;
+ struct nfs_pgio_header *hdr;
+ int ret;
+
+ rw_hdr = nfs_rw_header_alloc(desc->pg_rw_ops);
+ if (!rw_hdr) {
+ desc->pg_completion_ops->error_cleanup(&desc->pg_list);
+ return -ENOMEM;
+ }
+ hdr = &rw_hdr->header;
+ nfs_pgheader_init(desc, hdr, nfs_rw_header_free);
+ atomic_inc(&hdr->refcnt);
+ ret = nfs_generic_pgio(desc, hdr);
+ if (ret == 0)
+ ret = nfs_initiate_pgio(NFS_CLIENT(hdr->inode),
+ hdr->data, desc->pg_rpc_callops,
+ desc->pg_ioflags, 0);
+ if (atomic_dec_and_test(&hdr->refcnt))
+ hdr->completion_ops->completion(hdr);
+ return ret;
+}
+
static bool nfs_match_open_context(const struct nfs_open_context *ctx1,
const struct nfs_open_context *ctx2)
{
@@ -356,18 +824,23 @@
struct nfs_page *req,
struct nfs_pageio_descriptor *pgio)
{
- if (!nfs_match_open_context(req->wb_context, prev->wb_context))
- return false;
- if (req->wb_context->dentry->d_inode->i_flock != NULL &&
- !nfs_match_lock_context(req->wb_lock_context, prev->wb_lock_context))
- return false;
- if (req->wb_pgbase != 0)
- return false;
- if (prev->wb_pgbase + prev->wb_bytes != PAGE_CACHE_SIZE)
- return false;
- if (req_offset(req) != req_offset(prev) + prev->wb_bytes)
- return false;
- return pgio->pg_ops->pg_test(pgio, prev, req);
+ size_t size;
+
+ if (prev) {
+ if (!nfs_match_open_context(req->wb_context, prev->wb_context))
+ return false;
+ if (req->wb_context->dentry->d_inode->i_flock != NULL &&
+ !nfs_match_lock_context(req->wb_lock_context,
+ prev->wb_lock_context))
+ return false;
+ if (req_offset(req) != req_offset(prev) + prev->wb_bytes)
+ return false;
+ }
+ size = pgio->pg_ops->pg_test(pgio, prev, req);
+ WARN_ON_ONCE(size > req->wb_bytes);
+ if (size && size < req->wb_bytes)
+ req->wb_bytes = size;
+ return size > 0;
}
/**
@@ -381,17 +854,16 @@
static int nfs_pageio_do_add_request(struct nfs_pageio_descriptor *desc,
struct nfs_page *req)
{
+ struct nfs_page *prev = NULL;
if (desc->pg_count != 0) {
- struct nfs_page *prev;
-
prev = nfs_list_entry(desc->pg_list.prev);
- if (!nfs_can_coalesce_requests(prev, req, desc))
- return 0;
} else {
if (desc->pg_ops->pg_init)
desc->pg_ops->pg_init(desc, req);
desc->pg_base = req->wb_pgbase;
}
+ if (!nfs_can_coalesce_requests(prev, req, desc))
+ return 0;
nfs_list_remove_request(req);
nfs_list_add_request(req, &desc->pg_list);
desc->pg_count += req->wb_bytes;
@@ -421,22 +893,73 @@
* @desc: destination io descriptor
* @req: request
*
+ * This may split a request into subrequests which are all part of the
+ * same page group.
+ *
* Returns true if the request 'req' was successfully coalesced into the
* existing list of pages 'desc'.
*/
static int __nfs_pageio_add_request(struct nfs_pageio_descriptor *desc,
struct nfs_page *req)
{
- while (!nfs_pageio_do_add_request(desc, req)) {
- desc->pg_moreio = 1;
- nfs_pageio_doio(desc);
- if (desc->pg_error < 0)
- return 0;
- desc->pg_moreio = 0;
- if (desc->pg_recoalesce)
- return 0;
- }
+ struct nfs_page *subreq;
+ unsigned int bytes_left = 0;
+ unsigned int offset, pgbase;
+
+ nfs_page_group_lock(req);
+
+ subreq = req;
+ bytes_left = subreq->wb_bytes;
+ offset = subreq->wb_offset;
+ pgbase = subreq->wb_pgbase;
+
+ do {
+ if (!nfs_pageio_do_add_request(desc, subreq)) {
+ /* make sure pg_test call(s) did nothing */
+ WARN_ON_ONCE(subreq->wb_bytes != bytes_left);
+ WARN_ON_ONCE(subreq->wb_offset != offset);
+ WARN_ON_ONCE(subreq->wb_pgbase != pgbase);
+
+ nfs_page_group_unlock(req);
+ desc->pg_moreio = 1;
+ nfs_pageio_doio(desc);
+ if (desc->pg_error < 0)
+ return 0;
+ desc->pg_moreio = 0;
+ if (desc->pg_recoalesce)
+ return 0;
+ /* retry add_request for this subreq */
+ nfs_page_group_lock(req);
+ continue;
+ }
+
+ /* check for buggy pg_test call(s) */
+ WARN_ON_ONCE(subreq->wb_bytes + subreq->wb_pgbase > PAGE_SIZE);
+ WARN_ON_ONCE(subreq->wb_bytes > bytes_left);
+ WARN_ON_ONCE(subreq->wb_bytes == 0);
+
+ bytes_left -= subreq->wb_bytes;
+ offset += subreq->wb_bytes;
+ pgbase += subreq->wb_bytes;
+
+ if (bytes_left) {
+ subreq = nfs_create_request(req->wb_context,
+ req->wb_page,
+ subreq, pgbase, bytes_left);
+ if (IS_ERR(subreq))
+ goto err_ptr;
+ nfs_lock_request(subreq);
+ subreq->wb_offset = offset;
+ subreq->wb_index = req->wb_index;
+ }
+ } while (bytes_left > 0);
+
+ nfs_page_group_unlock(req);
return 1;
+err_ptr:
+ desc->pg_error = PTR_ERR(subreq);
+ nfs_page_group_unlock(req);
+ return 0;
}
static int nfs_do_recoalesce(struct nfs_pageio_descriptor *desc)
@@ -535,3 +1058,13 @@
kmem_cache_destroy(nfs_page_cachep);
}
+static const struct rpc_call_ops nfs_pgio_common_ops = {
+ .rpc_call_prepare = nfs_pgio_prepare,
+ .rpc_call_done = nfs_pgio_result,
+ .rpc_release = nfs_pgio_release,
+};
+
+const struct nfs_pageio_ops nfs_pgio_rw_ops = {
+ .pg_test = nfs_generic_pg_test,
+ .pg_doio = nfs_generic_pg_pgios,
+};
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index fd9536e..6fdcd23 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -1388,11 +1388,6 @@
WARN_ON_ONCE(pgio->pg_lseg != NULL);
- if (req->wb_offset != req->wb_pgbase) {
- nfs_pageio_reset_read_mds(pgio);
- return;
- }
-
if (pgio->pg_dreq == NULL)
rd_size = i_size_read(pgio->pg_inode) - req_offset(req);
else
@@ -1417,11 +1412,6 @@
{
WARN_ON_ONCE(pgio->pg_lseg != NULL);
- if (req->wb_offset != req->wb_pgbase) {
- nfs_pageio_reset_write_mds(pgio);
- return;
- }
-
pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
req->wb_context,
req_offset(req),
@@ -1434,56 +1424,49 @@
}
EXPORT_SYMBOL_GPL(pnfs_generic_pg_init_write);
-void
-pnfs_pageio_init_read(struct nfs_pageio_descriptor *pgio, struct inode *inode,
- const struct nfs_pgio_completion_ops *compl_ops)
-{
- struct nfs_server *server = NFS_SERVER(inode);
- struct pnfs_layoutdriver_type *ld = server->pnfs_curr_ld;
-
- if (ld == NULL)
- nfs_pageio_init_read(pgio, inode, compl_ops);
- else
- nfs_pageio_init(pgio, inode, ld->pg_read_ops, compl_ops, server->rsize, 0);
-}
-
-void
-pnfs_pageio_init_write(struct nfs_pageio_descriptor *pgio, struct inode *inode,
- int ioflags,
- const struct nfs_pgio_completion_ops *compl_ops)
-{
- struct nfs_server *server = NFS_SERVER(inode);
- struct pnfs_layoutdriver_type *ld = server->pnfs_curr_ld;
-
- if (ld == NULL)
- nfs_pageio_init_write(pgio, inode, ioflags, compl_ops);
- else
- nfs_pageio_init(pgio, inode, ld->pg_write_ops, compl_ops, server->wsize, ioflags);
-}
-
-bool
+/*
+ * Return 0 if @req cannot be coalesced into @pgio, otherwise return the number
+ * of bytes (maximum @req->wb_bytes) that can be coalesced.
+ */
+size_t
pnfs_generic_pg_test(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev,
struct nfs_page *req)
{
- if (pgio->pg_lseg == NULL)
- return nfs_generic_pg_test(pgio, prev, req);
+ unsigned int size;
+ u64 seg_end, req_start, seg_left;
+
+ size = nfs_generic_pg_test(pgio, prev, req);
+ if (!size)
+ return 0;
/*
- * Test if a nfs_page is fully contained in the pnfs_layout_range.
- * Note that this test makes several assumptions:
- * - that the previous nfs_page in the struct nfs_pageio_descriptor
- * is known to lie within the range.
- * - that the nfs_page being tested is known to be contiguous with the
- * previous nfs_page.
- * - Layout ranges are page aligned, so we only have to test the
- * start offset of the request.
+ * 'size' contains the number of bytes left in the current page (up
+ * to the original size asked for in @req->wb_bytes).
+ *
+ * Calculate how many bytes are left in the layout segment
+ * and if there are less bytes than 'size', return that instead.
*
* Please also note that 'end_offset' is actually the offset of the
* first byte that lies outside the pnfs_layout_range. FIXME?
*
*/
- return req_offset(req) < end_offset(pgio->pg_lseg->pls_range.offset,
- pgio->pg_lseg->pls_range.length);
+ if (pgio->pg_lseg) {
+ seg_end = end_offset(pgio->pg_lseg->pls_range.offset,
+ pgio->pg_lseg->pls_range.length);
+ req_start = req_offset(req);
+ WARN_ON_ONCE(req_start > seg_end);
+ /* start of request is past the last byte of this segment */
+ if (req_start >= seg_end)
+ return 0;
+
+ /* adjust 'size' iff there are fewer bytes left in the
+ * segment than what nfs_generic_pg_test returned */
+ seg_left = seg_end - req_start;
+ if (seg_left < size)
+ size = (unsigned int)seg_left;
+ }
+
+ return size;
}
EXPORT_SYMBOL_GPL(pnfs_generic_pg_test);
@@ -1496,7 +1479,7 @@
LIST_HEAD(failed);
/* Resend all requests through the MDS */
- nfs_pageio_init_write(&pgio, inode, FLUSH_STABLE, compl_ops);
+ nfs_pageio_init_write(&pgio, inode, FLUSH_STABLE, true, compl_ops);
pgio.pg_dreq = dreq;
while (!list_empty(head)) {
struct nfs_page *req = nfs_list_entry(head->next);
@@ -1519,7 +1502,7 @@
}
EXPORT_SYMBOL_GPL(pnfs_write_done_resend_to_mds);
-static void pnfs_ld_handle_write_error(struct nfs_write_data *data)
+static void pnfs_ld_handle_write_error(struct nfs_pgio_data *data)
{
struct nfs_pgio_header *hdr = data->header;
@@ -1538,7 +1521,7 @@
/*
* Called by non rpc-based layout drivers
*/
-void pnfs_ld_write_done(struct nfs_write_data *data)
+void pnfs_ld_write_done(struct nfs_pgio_data *data)
{
struct nfs_pgio_header *hdr = data->header;
@@ -1554,7 +1537,7 @@
static void
pnfs_write_through_mds(struct nfs_pageio_descriptor *desc,
- struct nfs_write_data *data)
+ struct nfs_pgio_data *data)
{
struct nfs_pgio_header *hdr = data->header;
@@ -1563,11 +1546,11 @@
nfs_pageio_reset_write_mds(desc);
desc->pg_recoalesce = 1;
}
- nfs_writedata_release(data);
+ nfs_pgio_data_release(data);
}
static enum pnfs_try_status
-pnfs_try_to_write_data(struct nfs_write_data *wdata,
+pnfs_try_to_write_data(struct nfs_pgio_data *wdata,
const struct rpc_call_ops *call_ops,
struct pnfs_layout_segment *lseg,
int how)
@@ -1589,41 +1572,36 @@
}
static void
-pnfs_do_multiple_writes(struct nfs_pageio_descriptor *desc, struct list_head *head, int how)
+pnfs_do_write(struct nfs_pageio_descriptor *desc,
+ struct nfs_pgio_header *hdr, int how)
{
- struct nfs_write_data *data;
+ struct nfs_pgio_data *data = hdr->data;
const struct rpc_call_ops *call_ops = desc->pg_rpc_callops;
struct pnfs_layout_segment *lseg = desc->pg_lseg;
+ enum pnfs_try_status trypnfs;
desc->pg_lseg = NULL;
- while (!list_empty(head)) {
- enum pnfs_try_status trypnfs;
-
- data = list_first_entry(head, struct nfs_write_data, list);
- list_del_init(&data->list);
-
- trypnfs = pnfs_try_to_write_data(data, call_ops, lseg, how);
- if (trypnfs == PNFS_NOT_ATTEMPTED)
- pnfs_write_through_mds(desc, data);
- }
+ trypnfs = pnfs_try_to_write_data(data, call_ops, lseg, how);
+ if (trypnfs == PNFS_NOT_ATTEMPTED)
+ pnfs_write_through_mds(desc, data);
pnfs_put_lseg(lseg);
}
static void pnfs_writehdr_free(struct nfs_pgio_header *hdr)
{
pnfs_put_lseg(hdr->lseg);
- nfs_writehdr_free(hdr);
+ nfs_rw_header_free(hdr);
}
EXPORT_SYMBOL_GPL(pnfs_writehdr_free);
int
pnfs_generic_pg_writepages(struct nfs_pageio_descriptor *desc)
{
- struct nfs_write_header *whdr;
+ struct nfs_rw_header *whdr;
struct nfs_pgio_header *hdr;
int ret;
- whdr = nfs_writehdr_alloc();
+ whdr = nfs_rw_header_alloc(desc->pg_rw_ops);
if (!whdr) {
desc->pg_completion_ops->error_cleanup(&desc->pg_list);
pnfs_put_lseg(desc->pg_lseg);
@@ -1634,12 +1612,12 @@
nfs_pgheader_init(desc, hdr, pnfs_writehdr_free);
hdr->lseg = pnfs_get_lseg(desc->pg_lseg);
atomic_inc(&hdr->refcnt);
- ret = nfs_generic_flush(desc, hdr);
+ ret = nfs_generic_pgio(desc, hdr);
if (ret != 0) {
pnfs_put_lseg(desc->pg_lseg);
desc->pg_lseg = NULL;
} else
- pnfs_do_multiple_writes(desc, &hdr->rpc_list, desc->pg_ioflags);
+ pnfs_do_write(desc, hdr, desc->pg_ioflags);
if (atomic_dec_and_test(&hdr->refcnt))
hdr->completion_ops->completion(hdr);
return ret;
@@ -1655,7 +1633,7 @@
LIST_HEAD(failed);
/* Resend all requests through the MDS */
- nfs_pageio_init_read(&pgio, inode, compl_ops);
+ nfs_pageio_init_read(&pgio, inode, true, compl_ops);
pgio.pg_dreq = dreq;
while (!list_empty(head)) {
struct nfs_page *req = nfs_list_entry(head->next);
@@ -1674,7 +1652,7 @@
}
EXPORT_SYMBOL_GPL(pnfs_read_done_resend_to_mds);
-static void pnfs_ld_handle_read_error(struct nfs_read_data *data)
+static void pnfs_ld_handle_read_error(struct nfs_pgio_data *data)
{
struct nfs_pgio_header *hdr = data->header;
@@ -1693,7 +1671,7 @@
/*
* Called by non rpc-based layout drivers
*/
-void pnfs_ld_read_done(struct nfs_read_data *data)
+void pnfs_ld_read_done(struct nfs_pgio_data *data)
{
struct nfs_pgio_header *hdr = data->header;
@@ -1709,7 +1687,7 @@
static void
pnfs_read_through_mds(struct nfs_pageio_descriptor *desc,
- struct nfs_read_data *data)
+ struct nfs_pgio_data *data)
{
struct nfs_pgio_header *hdr = data->header;
@@ -1718,14 +1696,14 @@
nfs_pageio_reset_read_mds(desc);
desc->pg_recoalesce = 1;
}
- nfs_readdata_release(data);
+ nfs_pgio_data_release(data);
}
/*
* Call the appropriate parallel I/O subsystem read function.
*/
static enum pnfs_try_status
-pnfs_try_to_read_data(struct nfs_read_data *rdata,
+pnfs_try_to_read_data(struct nfs_pgio_data *rdata,
const struct rpc_call_ops *call_ops,
struct pnfs_layout_segment *lseg)
{
@@ -1747,41 +1725,35 @@
}
static void
-pnfs_do_multiple_reads(struct nfs_pageio_descriptor *desc, struct list_head *head)
+pnfs_do_read(struct nfs_pageio_descriptor *desc, struct nfs_pgio_header *hdr)
{
- struct nfs_read_data *data;
+ struct nfs_pgio_data *data = hdr->data;
const struct rpc_call_ops *call_ops = desc->pg_rpc_callops;
struct pnfs_layout_segment *lseg = desc->pg_lseg;
+ enum pnfs_try_status trypnfs;
desc->pg_lseg = NULL;
- while (!list_empty(head)) {
- enum pnfs_try_status trypnfs;
-
- data = list_first_entry(head, struct nfs_read_data, list);
- list_del_init(&data->list);
-
- trypnfs = pnfs_try_to_read_data(data, call_ops, lseg);
- if (trypnfs == PNFS_NOT_ATTEMPTED)
- pnfs_read_through_mds(desc, data);
- }
+ trypnfs = pnfs_try_to_read_data(data, call_ops, lseg);
+ if (trypnfs == PNFS_NOT_ATTEMPTED)
+ pnfs_read_through_mds(desc, data);
pnfs_put_lseg(lseg);
}
static void pnfs_readhdr_free(struct nfs_pgio_header *hdr)
{
pnfs_put_lseg(hdr->lseg);
- nfs_readhdr_free(hdr);
+ nfs_rw_header_free(hdr);
}
EXPORT_SYMBOL_GPL(pnfs_readhdr_free);
int
pnfs_generic_pg_readpages(struct nfs_pageio_descriptor *desc)
{
- struct nfs_read_header *rhdr;
+ struct nfs_rw_header *rhdr;
struct nfs_pgio_header *hdr;
int ret;
- rhdr = nfs_readhdr_alloc();
+ rhdr = nfs_rw_header_alloc(desc->pg_rw_ops);
if (!rhdr) {
desc->pg_completion_ops->error_cleanup(&desc->pg_list);
ret = -ENOMEM;
@@ -1793,12 +1765,12 @@
nfs_pgheader_init(desc, hdr, pnfs_readhdr_free);
hdr->lseg = pnfs_get_lseg(desc->pg_lseg);
atomic_inc(&hdr->refcnt);
- ret = nfs_generic_pagein(desc, hdr);
+ ret = nfs_generic_pgio(desc, hdr);
if (ret != 0) {
pnfs_put_lseg(desc->pg_lseg);
desc->pg_lseg = NULL;
} else
- pnfs_do_multiple_reads(desc, &hdr->rpc_list);
+ pnfs_do_read(desc, hdr);
if (atomic_dec_and_test(&hdr->refcnt))
hdr->completion_ops->completion(hdr);
return ret;
@@ -1848,7 +1820,7 @@
EXPORT_SYMBOL_GPL(pnfs_set_lo_fail);
void
-pnfs_set_layoutcommit(struct nfs_write_data *wdata)
+pnfs_set_layoutcommit(struct nfs_pgio_data *wdata)
{
struct nfs_pgio_header *hdr = wdata->header;
struct inode *inode = hdr->inode;
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index c3058a0..4fb309a 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -113,8 +113,8 @@
* Return PNFS_ATTEMPTED to indicate the layout code has attempted
* I/O, else return PNFS_NOT_ATTEMPTED to fall back to normal NFS
*/
- enum pnfs_try_status (*read_pagelist) (struct nfs_read_data *nfs_data);
- enum pnfs_try_status (*write_pagelist) (struct nfs_write_data *nfs_data, int how);
+ enum pnfs_try_status (*read_pagelist) (struct nfs_pgio_data *nfs_data);
+ enum pnfs_try_status (*write_pagelist) (struct nfs_pgio_data *nfs_data, int how);
void (*free_deviceid_node) (struct nfs4_deviceid_node *);
@@ -180,11 +180,6 @@
void pnfs_get_layout_hdr(struct pnfs_layout_hdr *lo);
void pnfs_put_lseg(struct pnfs_layout_segment *lseg);
-void pnfs_pageio_init_read(struct nfs_pageio_descriptor *, struct inode *,
- const struct nfs_pgio_completion_ops *);
-void pnfs_pageio_init_write(struct nfs_pageio_descriptor *, struct inode *,
- int, const struct nfs_pgio_completion_ops *);
-
void set_pnfs_layoutdriver(struct nfs_server *, const struct nfs_fh *, u32);
void unset_pnfs_layoutdriver(struct nfs_server *);
void pnfs_generic_pg_init_read(struct nfs_pageio_descriptor *, struct nfs_page *);
@@ -192,7 +187,8 @@
void pnfs_generic_pg_init_write(struct nfs_pageio_descriptor *pgio,
struct nfs_page *req, u64 wb_size);
int pnfs_generic_pg_writepages(struct nfs_pageio_descriptor *desc);
-bool pnfs_generic_pg_test(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev, struct nfs_page *req);
+size_t pnfs_generic_pg_test(struct nfs_pageio_descriptor *pgio,
+ struct nfs_page *prev, struct nfs_page *req);
void pnfs_set_lo_fail(struct pnfs_layout_segment *lseg);
struct pnfs_layout_segment *pnfs_layout_process(struct nfs4_layoutget *lgp);
void pnfs_free_lseg_list(struct list_head *tmp_list);
@@ -217,13 +213,13 @@
void pnfs_roc_release(struct inode *ino);
void pnfs_roc_set_barrier(struct inode *ino, u32 barrier);
bool pnfs_roc_drain(struct inode *ino, u32 *barrier, struct rpc_task *task);
-void pnfs_set_layoutcommit(struct nfs_write_data *wdata);
+void pnfs_set_layoutcommit(struct nfs_pgio_data *wdata);
void pnfs_cleanup_layoutcommit(struct nfs4_layoutcommit_data *data);
int pnfs_layoutcommit_inode(struct inode *inode, bool sync);
int _pnfs_return_layout(struct inode *);
int pnfs_commit_and_return_layout(struct inode *);
-void pnfs_ld_write_done(struct nfs_write_data *);
-void pnfs_ld_read_done(struct nfs_read_data *);
+void pnfs_ld_write_done(struct nfs_pgio_data *);
+void pnfs_ld_read_done(struct nfs_pgio_data *);
struct pnfs_layout_segment *pnfs_update_layout(struct inode *ino,
struct nfs_open_context *ctx,
loff_t pos,
@@ -461,18 +457,6 @@
{
}
-static inline void pnfs_pageio_init_read(struct nfs_pageio_descriptor *pgio, struct inode *inode,
- const struct nfs_pgio_completion_ops *compl_ops)
-{
- nfs_pageio_init_read(pgio, inode, compl_ops);
-}
-
-static inline void pnfs_pageio_init_write(struct nfs_pageio_descriptor *pgio, struct inode *inode, int ioflags,
- const struct nfs_pgio_completion_ops *compl_ops)
-{
- nfs_pageio_init_write(pgio, inode, ioflags, compl_ops);
-}
-
static inline int
pnfs_commit_list(struct inode *inode, struct list_head *mds_pages, int how,
struct nfs_commit_info *cinfo)
diff --git a/fs/nfs/proc.c b/fs/nfs/proc.c
index e55ce9e..c171ce1 100644
--- a/fs/nfs/proc.c
+++ b/fs/nfs/proc.c
@@ -578,7 +578,7 @@
return 0;
}
-static int nfs_read_done(struct rpc_task *task, struct nfs_read_data *data)
+static int nfs_read_done(struct rpc_task *task, struct nfs_pgio_data *data)
{
struct inode *inode = data->header->inode;
@@ -594,18 +594,18 @@
return 0;
}
-static void nfs_proc_read_setup(struct nfs_read_data *data, struct rpc_message *msg)
+static void nfs_proc_read_setup(struct nfs_pgio_data *data, struct rpc_message *msg)
{
msg->rpc_proc = &nfs_procedures[NFSPROC_READ];
}
-static int nfs_proc_read_rpc_prepare(struct rpc_task *task, struct nfs_read_data *data)
+static int nfs_proc_pgio_rpc_prepare(struct rpc_task *task, struct nfs_pgio_data *data)
{
rpc_call_start(task);
return 0;
}
-static int nfs_write_done(struct rpc_task *task, struct nfs_write_data *data)
+static int nfs_write_done(struct rpc_task *task, struct nfs_pgio_data *data)
{
struct inode *inode = data->header->inode;
@@ -614,19 +614,13 @@
return 0;
}
-static void nfs_proc_write_setup(struct nfs_write_data *data, struct rpc_message *msg)
+static void nfs_proc_write_setup(struct nfs_pgio_data *data, struct rpc_message *msg)
{
/* Note: NFSv2 ignores @stable and always uses NFS_FILE_SYNC */
data->args.stable = NFS_FILE_SYNC;
msg->rpc_proc = &nfs_procedures[NFSPROC_WRITE];
}
-static int nfs_proc_write_rpc_prepare(struct rpc_task *task, struct nfs_write_data *data)
-{
- rpc_call_start(task);
- return 0;
-}
-
static void nfs_proc_commit_rpc_prepare(struct rpc_task *task, struct nfs_commit_data *data)
{
BUG();
@@ -734,13 +728,10 @@
.fsinfo = nfs_proc_fsinfo,
.pathconf = nfs_proc_pathconf,
.decode_dirent = nfs2_decode_dirent,
+ .pgio_rpc_prepare = nfs_proc_pgio_rpc_prepare,
.read_setup = nfs_proc_read_setup,
- .read_pageio_init = nfs_pageio_init_read,
- .read_rpc_prepare = nfs_proc_read_rpc_prepare,
.read_done = nfs_read_done,
.write_setup = nfs_proc_write_setup,
- .write_pageio_init = nfs_pageio_init_write,
- .write_rpc_prepare = nfs_proc_write_rpc_prepare,
.write_done = nfs_write_done,
.commit_setup = nfs_proc_commit_setup,
.commit_rpc_prepare = nfs_proc_commit_rpc_prepare,
diff --git a/fs/nfs/read.c b/fs/nfs/read.c
index 411aedd..e818a47 100644
--- a/fs/nfs/read.c
+++ b/fs/nfs/read.c
@@ -24,85 +24,24 @@
#include "internal.h"
#include "iostat.h"
#include "fscache.h"
+#include "pnfs.h"
#define NFSDBG_FACILITY NFSDBG_PAGECACHE
-static const struct nfs_pageio_ops nfs_pageio_read_ops;
-static const struct rpc_call_ops nfs_read_common_ops;
static const struct nfs_pgio_completion_ops nfs_async_read_completion_ops;
+static const struct nfs_rw_ops nfs_rw_read_ops;
static struct kmem_cache *nfs_rdata_cachep;
-struct nfs_read_header *nfs_readhdr_alloc(void)
+static struct nfs_rw_header *nfs_readhdr_alloc(void)
{
- struct nfs_read_header *rhdr;
-
- rhdr = kmem_cache_zalloc(nfs_rdata_cachep, GFP_KERNEL);
- if (rhdr) {
- struct nfs_pgio_header *hdr = &rhdr->header;
-
- INIT_LIST_HEAD(&hdr->pages);
- INIT_LIST_HEAD(&hdr->rpc_list);
- spin_lock_init(&hdr->lock);
- atomic_set(&hdr->refcnt, 0);
- }
- return rhdr;
-}
-EXPORT_SYMBOL_GPL(nfs_readhdr_alloc);
-
-static struct nfs_read_data *nfs_readdata_alloc(struct nfs_pgio_header *hdr,
- unsigned int pagecount)
-{
- struct nfs_read_data *data, *prealloc;
-
- prealloc = &container_of(hdr, struct nfs_read_header, header)->rpc_data;
- if (prealloc->header == NULL)
- data = prealloc;
- else
- data = kzalloc(sizeof(*data), GFP_KERNEL);
- if (!data)
- goto out;
-
- if (nfs_pgarray_set(&data->pages, pagecount)) {
- data->header = hdr;
- atomic_inc(&hdr->refcnt);
- } else {
- if (data != prealloc)
- kfree(data);
- data = NULL;
- }
-out:
- return data;
+ return kmem_cache_zalloc(nfs_rdata_cachep, GFP_KERNEL);
}
-void nfs_readhdr_free(struct nfs_pgio_header *hdr)
+static void nfs_readhdr_free(struct nfs_rw_header *rhdr)
{
- struct nfs_read_header *rhdr = container_of(hdr, struct nfs_read_header, header);
-
kmem_cache_free(nfs_rdata_cachep, rhdr);
}
-EXPORT_SYMBOL_GPL(nfs_readhdr_free);
-
-void nfs_readdata_release(struct nfs_read_data *rdata)
-{
- struct nfs_pgio_header *hdr = rdata->header;
- struct nfs_read_header *read_header = container_of(hdr, struct nfs_read_header, header);
-
- put_nfs_open_context(rdata->args.context);
- if (rdata->pages.pagevec != rdata->pages.page_array)
- kfree(rdata->pages.pagevec);
- if (rdata == &read_header->rpc_data) {
- rdata->header = NULL;
- rdata = NULL;
- }
- if (atomic_dec_and_test(&hdr->refcnt))
- hdr->completion_ops->completion(hdr);
- /* Note: we only free the rpc_task after callbacks are done.
- * See the comment in rpc_free_task() for why
- */
- kfree(rdata);
-}
-EXPORT_SYMBOL_GPL(nfs_readdata_release);
static
int nfs_return_empty_page(struct page *page)
@@ -114,17 +53,24 @@
}
void nfs_pageio_init_read(struct nfs_pageio_descriptor *pgio,
- struct inode *inode,
+ struct inode *inode, bool force_mds,
const struct nfs_pgio_completion_ops *compl_ops)
{
- nfs_pageio_init(pgio, inode, &nfs_pageio_read_ops, compl_ops,
- NFS_SERVER(inode)->rsize, 0);
+ struct nfs_server *server = NFS_SERVER(inode);
+ const struct nfs_pageio_ops *pg_ops = &nfs_pgio_rw_ops;
+
+#ifdef CONFIG_NFS_V4_1
+ if (server->pnfs_curr_ld && !force_mds)
+ pg_ops = server->pnfs_curr_ld->pg_read_ops;
+#endif
+ nfs_pageio_init(pgio, inode, pg_ops, compl_ops, &nfs_rw_read_ops,
+ server->rsize, 0);
}
EXPORT_SYMBOL_GPL(nfs_pageio_init_read);
void nfs_pageio_reset_read_mds(struct nfs_pageio_descriptor *pgio)
{
- pgio->pg_ops = &nfs_pageio_read_ops;
+ pgio->pg_ops = &nfs_pgio_rw_ops;
pgio->pg_bsize = NFS_SERVER(pgio->pg_inode)->rsize;
}
EXPORT_SYMBOL_GPL(nfs_pageio_reset_read_mds);
@@ -139,7 +85,7 @@
len = nfs_page_length(page);
if (len == 0)
return nfs_return_empty_page(page);
- new = nfs_create_request(ctx, inode, page, 0, len);
+ new = nfs_create_request(ctx, page, NULL, 0, len);
if (IS_ERR(new)) {
unlock_page(page);
return PTR_ERR(new);
@@ -147,7 +93,8 @@
if (len < PAGE_CACHE_SIZE)
zero_user_segment(page, len, PAGE_CACHE_SIZE);
- NFS_PROTO(inode)->read_pageio_init(&pgio, inode, &nfs_async_read_completion_ops);
+ nfs_pageio_init_read(&pgio, inode, false,
+ &nfs_async_read_completion_ops);
nfs_pageio_add_request(&pgio, new);
nfs_pageio_complete(&pgio);
NFS_I(inode)->read_io += pgio.pg_bytes_written;
@@ -158,10 +105,16 @@
{
struct inode *d_inode = req->wb_context->dentry->d_inode;
- if (PageUptodate(req->wb_page))
- nfs_readpage_to_fscache(d_inode, req->wb_page, 0);
+ dprintk("NFS: read done (%s/%llu %d@%lld)\n", d_inode->i_sb->s_id,
+ (unsigned long long)NFS_FILEID(d_inode), req->wb_bytes,
+ (long long)req_offset(req));
- unlock_page(req->wb_page);
+ if (nfs_page_group_sync_on_bit(req, PG_UNLOCKPAGE)) {
+ if (PageUptodate(req->wb_page))
+ nfs_readpage_to_fscache(d_inode, req->wb_page, 0);
+
+ unlock_page(req->wb_page);
+ }
dprintk("NFS: read done (%s/%Lu %d@%Ld)\n",
req->wb_context->dentry->d_inode->i_sb->s_id,
@@ -171,7 +124,12 @@
nfs_release_request(req);
}
-/* Note io was page aligned */
+static void nfs_page_group_set_uptodate(struct nfs_page *req)
+{
+ if (nfs_page_group_sync_on_bit(req, PG_UPTODATE))
+ SetPageUptodate(req->wb_page);
+}
+
static void nfs_read_completion(struct nfs_pgio_header *hdr)
{
unsigned long bytes = 0;
@@ -181,21 +139,32 @@
while (!list_empty(&hdr->pages)) {
struct nfs_page *req = nfs_list_entry(hdr->pages.next);
struct page *page = req->wb_page;
+ unsigned long start = req->wb_pgbase;
+ unsigned long end = req->wb_pgbase + req->wb_bytes;
if (test_bit(NFS_IOHDR_EOF, &hdr->flags)) {
- if (bytes > hdr->good_bytes)
- zero_user(page, 0, PAGE_SIZE);
- else if (hdr->good_bytes - bytes < PAGE_SIZE)
- zero_user_segment(page,
- hdr->good_bytes & ~PAGE_MASK,
- PAGE_SIZE);
+ /* note: regions of the page not covered by a
+ * request are zeroed in nfs_readpage_async /
+ * readpage_async_filler */
+ if (bytes > hdr->good_bytes) {
+ /* nothing in this request was good, so zero
+ * the full extent of the request */
+ zero_user_segment(page, start, end);
+
+ } else if (hdr->good_bytes - bytes < req->wb_bytes) {
+ /* part of this request has good bytes, but
+ * not all. zero the bad bytes */
+ start += hdr->good_bytes - bytes;
+ WARN_ON(start < req->wb_pgbase);
+ zero_user_segment(page, start, end);
+ }
}
bytes += req->wb_bytes;
if (test_bit(NFS_IOHDR_ERROR, &hdr->flags)) {
if (bytes <= hdr->good_bytes)
- SetPageUptodate(page);
+ nfs_page_group_set_uptodate(req);
} else
- SetPageUptodate(page);
+ nfs_page_group_set_uptodate(req);
nfs_list_remove_request(req);
nfs_readpage_release(req);
}
@@ -203,95 +172,14 @@
hdr->release(hdr);
}
-int nfs_initiate_read(struct rpc_clnt *clnt,
- struct nfs_read_data *data,
- const struct rpc_call_ops *call_ops, int flags)
+static void nfs_initiate_read(struct nfs_pgio_data *data, struct rpc_message *msg,
+ struct rpc_task_setup *task_setup_data, int how)
{
struct inode *inode = data->header->inode;
int swap_flags = IS_SWAPFILE(inode) ? NFS_RPC_SWAPFLAGS : 0;
- struct rpc_task *task;
- struct rpc_message msg = {
- .rpc_argp = &data->args,
- .rpc_resp = &data->res,
- .rpc_cred = data->header->cred,
- };
- struct rpc_task_setup task_setup_data = {
- .task = &data->task,
- .rpc_client = clnt,
- .rpc_message = &msg,
- .callback_ops = call_ops,
- .callback_data = data,
- .workqueue = nfsiod_workqueue,
- .flags = RPC_TASK_ASYNC | swap_flags | flags,
- };
- /* Set up the initial task struct. */
- NFS_PROTO(inode)->read_setup(data, &msg);
-
- dprintk("NFS: %5u initiated read call (req %s/%llu, %u bytes @ "
- "offset %llu)\n",
- data->task.tk_pid,
- inode->i_sb->s_id,
- (unsigned long long)NFS_FILEID(inode),
- data->args.count,
- (unsigned long long)data->args.offset);
-
- task = rpc_run_task(&task_setup_data);
- if (IS_ERR(task))
- return PTR_ERR(task);
- rpc_put_task(task);
- return 0;
-}
-EXPORT_SYMBOL_GPL(nfs_initiate_read);
-
-/*
- * Set up the NFS read request struct
- */
-static void nfs_read_rpcsetup(struct nfs_read_data *data,
- unsigned int count, unsigned int offset)
-{
- struct nfs_page *req = data->header->req;
-
- data->args.fh = NFS_FH(data->header->inode);
- data->args.offset = req_offset(req) + offset;
- data->args.pgbase = req->wb_pgbase + offset;
- data->args.pages = data->pages.pagevec;
- data->args.count = count;
- data->args.context = get_nfs_open_context(req->wb_context);
- data->args.lock_context = req->wb_lock_context;
-
- data->res.fattr = &data->fattr;
- data->res.count = count;
- data->res.eof = 0;
- nfs_fattr_init(&data->fattr);
-}
-
-static int nfs_do_read(struct nfs_read_data *data,
- const struct rpc_call_ops *call_ops)
-{
- struct inode *inode = data->header->inode;
-
- return nfs_initiate_read(NFS_CLIENT(inode), data, call_ops, 0);
-}
-
-static int
-nfs_do_multiple_reads(struct list_head *head,
- const struct rpc_call_ops *call_ops)
-{
- struct nfs_read_data *data;
- int ret = 0;
-
- while (!list_empty(head)) {
- int ret2;
-
- data = list_first_entry(head, struct nfs_read_data, list);
- list_del_init(&data->list);
-
- ret2 = nfs_do_read(data, call_ops);
- if (ret == 0)
- ret = ret2;
- }
- return ret;
+ task_setup_data->flags |= swap_flags;
+ NFS_PROTO(inode)->read_setup(data, msg);
}
static void
@@ -311,143 +199,14 @@
.completion = nfs_read_completion,
};
-static void nfs_pagein_error(struct nfs_pageio_descriptor *desc,
- struct nfs_pgio_header *hdr)
-{
- set_bit(NFS_IOHDR_REDO, &hdr->flags);
- while (!list_empty(&hdr->rpc_list)) {
- struct nfs_read_data *data = list_first_entry(&hdr->rpc_list,
- struct nfs_read_data, list);
- list_del(&data->list);
- nfs_readdata_release(data);
- }
- desc->pg_completion_ops->error_cleanup(&desc->pg_list);
-}
-
-/*
- * Generate multiple requests to fill a single page.
- *
- * We optimize to reduce the number of read operations on the wire. If we
- * detect that we're reading a page, or an area of a page, that is past the
- * end of file, we do not generate NFS read operations but just clear the
- * parts of the page that would have come back zero from the server anyway.
- *
- * We rely on the cached value of i_size to make this determination; another
- * client can fill pages on the server past our cached end-of-file, but we
- * won't see the new data until our attribute cache is updated. This is more
- * or less conventional NFS client behavior.
- */
-static int nfs_pagein_multi(struct nfs_pageio_descriptor *desc,
- struct nfs_pgio_header *hdr)
-{
- struct nfs_page *req = hdr->req;
- struct page *page = req->wb_page;
- struct nfs_read_data *data;
- size_t rsize = desc->pg_bsize, nbytes;
- unsigned int offset;
-
- offset = 0;
- nbytes = desc->pg_count;
- do {
- size_t len = min(nbytes,rsize);
-
- data = nfs_readdata_alloc(hdr, 1);
- if (!data) {
- nfs_pagein_error(desc, hdr);
- return -ENOMEM;
- }
- data->pages.pagevec[0] = page;
- nfs_read_rpcsetup(data, len, offset);
- list_add(&data->list, &hdr->rpc_list);
- nbytes -= len;
- offset += len;
- } while (nbytes != 0);
-
- nfs_list_remove_request(req);
- nfs_list_add_request(req, &hdr->pages);
- desc->pg_rpc_callops = &nfs_read_common_ops;
- return 0;
-}
-
-static int nfs_pagein_one(struct nfs_pageio_descriptor *desc,
- struct nfs_pgio_header *hdr)
-{
- struct nfs_page *req;
- struct page **pages;
- struct nfs_read_data *data;
- struct list_head *head = &desc->pg_list;
-
- data = nfs_readdata_alloc(hdr, nfs_page_array_len(desc->pg_base,
- desc->pg_count));
- if (!data) {
- nfs_pagein_error(desc, hdr);
- return -ENOMEM;
- }
-
- pages = data->pages.pagevec;
- while (!list_empty(head)) {
- req = nfs_list_entry(head->next);
- nfs_list_remove_request(req);
- nfs_list_add_request(req, &hdr->pages);
- *pages++ = req->wb_page;
- }
-
- nfs_read_rpcsetup(data, desc->pg_count, 0);
- list_add(&data->list, &hdr->rpc_list);
- desc->pg_rpc_callops = &nfs_read_common_ops;
- return 0;
-}
-
-int nfs_generic_pagein(struct nfs_pageio_descriptor *desc,
- struct nfs_pgio_header *hdr)
-{
- if (desc->pg_bsize < PAGE_CACHE_SIZE)
- return nfs_pagein_multi(desc, hdr);
- return nfs_pagein_one(desc, hdr);
-}
-EXPORT_SYMBOL_GPL(nfs_generic_pagein);
-
-static int nfs_generic_pg_readpages(struct nfs_pageio_descriptor *desc)
-{
- struct nfs_read_header *rhdr;
- struct nfs_pgio_header *hdr;
- int ret;
-
- rhdr = nfs_readhdr_alloc();
- if (!rhdr) {
- desc->pg_completion_ops->error_cleanup(&desc->pg_list);
- return -ENOMEM;
- }
- hdr = &rhdr->header;
- nfs_pgheader_init(desc, hdr, nfs_readhdr_free);
- atomic_inc(&hdr->refcnt);
- ret = nfs_generic_pagein(desc, hdr);
- if (ret == 0)
- ret = nfs_do_multiple_reads(&hdr->rpc_list,
- desc->pg_rpc_callops);
- if (atomic_dec_and_test(&hdr->refcnt))
- hdr->completion_ops->completion(hdr);
- return ret;
-}
-
-static const struct nfs_pageio_ops nfs_pageio_read_ops = {
- .pg_test = nfs_generic_pg_test,
- .pg_doio = nfs_generic_pg_readpages,
-};
-
/*
* This is the callback from RPC telling us whether a reply was
* received or some error occurred (timeout or socket shutdown).
*/
-int nfs_readpage_result(struct rpc_task *task, struct nfs_read_data *data)
+static int nfs_readpage_done(struct rpc_task *task, struct nfs_pgio_data *data,
+ struct inode *inode)
{
- struct inode *inode = data->header->inode;
- int status;
-
- dprintk("NFS: %s: %5u, (status %d)\n", __func__, task->tk_pid,
- task->tk_status);
-
- status = NFS_PROTO(inode)->read_done(task, data);
+ int status = NFS_PROTO(inode)->read_done(task, data);
if (status != 0)
return status;
@@ -460,10 +219,10 @@
return 0;
}
-static void nfs_readpage_retry(struct rpc_task *task, struct nfs_read_data *data)
+static void nfs_readpage_retry(struct rpc_task *task, struct nfs_pgio_data *data)
{
- struct nfs_readargs *argp = &data->args;
- struct nfs_readres *resp = &data->res;
+ struct nfs_pgio_args *argp = &data->args;
+ struct nfs_pgio_res *resp = &data->res;
/* This is a short read! */
nfs_inc_stats(data->header->inode, NFSIOS_SHORTREAD);
@@ -480,17 +239,11 @@
rpc_restart_call_prepare(task);
}
-static void nfs_readpage_result_common(struct rpc_task *task, void *calldata)
+static void nfs_readpage_result(struct rpc_task *task, struct nfs_pgio_data *data)
{
- struct nfs_read_data *data = calldata;
struct nfs_pgio_header *hdr = data->header;
- /* Note the only returns of nfs_readpage_result are 0 and -EAGAIN */
- if (nfs_readpage_result(task, data) != 0)
- return;
- if (task->tk_status < 0)
- nfs_set_pgio_error(hdr, task->tk_status, data->args.offset);
- else if (data->res.eof) {
+ if (data->res.eof) {
loff_t bound;
bound = data->args.offset + data->res.count;
@@ -505,26 +258,6 @@
nfs_readpage_retry(task, data);
}
-static void nfs_readpage_release_common(void *calldata)
-{
- nfs_readdata_release(calldata);
-}
-
-void nfs_read_prepare(struct rpc_task *task, void *calldata)
-{
- struct nfs_read_data *data = calldata;
- int err;
- err = NFS_PROTO(data->header->inode)->read_rpc_prepare(task, data);
- if (err)
- rpc_exit(task, err);
-}
-
-static const struct rpc_call_ops nfs_read_common_ops = {
- .rpc_call_prepare = nfs_read_prepare,
- .rpc_call_done = nfs_readpage_result_common,
- .rpc_release = nfs_readpage_release_common,
-};
-
/*
* Read a page over NFS.
* We read the page synchronously in the following case:
@@ -592,7 +325,6 @@
readpage_async_filler(void *data, struct page *page)
{
struct nfs_readdesc *desc = (struct nfs_readdesc *)data;
- struct inode *inode = page_file_mapping(page)->host;
struct nfs_page *new;
unsigned int len;
int error;
@@ -601,7 +333,7 @@
if (len == 0)
return nfs_return_empty_page(page);
- new = nfs_create_request(desc->ctx, inode, page, 0, len);
+ new = nfs_create_request(desc->ctx, page, NULL, 0, len);
if (IS_ERR(new))
goto out_error;
@@ -654,7 +386,8 @@
if (ret == 0)
goto read_complete; /* all pages were read */
- NFS_PROTO(inode)->read_pageio_init(&pgio, inode, &nfs_async_read_completion_ops);
+ nfs_pageio_init_read(&pgio, inode, false,
+ &nfs_async_read_completion_ops);
ret = read_cache_pages(mapping, pages, readpage_async_filler, &desc);
@@ -671,7 +404,7 @@
int __init nfs_init_readpagecache(void)
{
nfs_rdata_cachep = kmem_cache_create("nfs_read_data",
- sizeof(struct nfs_read_header),
+ sizeof(struct nfs_rw_header),
0, SLAB_HWCACHE_ALIGN,
NULL);
if (nfs_rdata_cachep == NULL)
@@ -684,3 +417,12 @@
{
kmem_cache_destroy(nfs_rdata_cachep);
}
+
+static const struct nfs_rw_ops nfs_rw_read_ops = {
+ .rw_mode = FMODE_READ,
+ .rw_alloc_header = nfs_readhdr_alloc,
+ .rw_free_header = nfs_readhdr_free,
+ .rw_done = nfs_readpage_done,
+ .rw_result = nfs_readpage_result,
+ .rw_initiate = nfs_initiate_read,
+};
diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index 2cb5694..084af10 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -2180,11 +2180,23 @@
return -EINVAL;
}
+#define NFS_MOUNT_CMP_FLAGMASK ~(NFS_MOUNT_INTR \
+ | NFS_MOUNT_SECURE \
+ | NFS_MOUNT_TCP \
+ | NFS_MOUNT_VER3 \
+ | NFS_MOUNT_KERBEROS \
+ | NFS_MOUNT_NONLM \
+ | NFS_MOUNT_BROKEN_SUID \
+ | NFS_MOUNT_STRICTLOCK \
+ | NFS_MOUNT_UNSHARED \
+ | NFS_MOUNT_NORESVPORT \
+ | NFS_MOUNT_LEGACY_INTERFACE)
+
static int
nfs_compare_remount_data(struct nfs_server *nfss,
struct nfs_parsed_mount_data *data)
{
- if (data->flags != nfss->flags ||
+ if ((data->flags ^ nfss->flags) & NFS_MOUNT_CMP_FLAGMASK ||
data->rsize != nfss->rsize ||
data->wsize != nfss->wsize ||
data->version != nfss->nfs_client->rpc_ops->version ||
@@ -2248,6 +2260,7 @@
data->nfs_server.addrlen = nfss->nfs_client->cl_addrlen;
data->version = nfsvers;
data->minorversion = nfss->nfs_client->cl_minorversion;
+ data->net = current->nsproxy->net_ns;
memcpy(&data->nfs_server.address, &nfss->nfs_client->cl_addr,
data->nfs_server.addrlen);
@@ -2347,18 +2360,6 @@
nfs_initialise_sb(sb);
}
-#define NFS_MOUNT_CMP_FLAGMASK ~(NFS_MOUNT_INTR \
- | NFS_MOUNT_SECURE \
- | NFS_MOUNT_TCP \
- | NFS_MOUNT_VER3 \
- | NFS_MOUNT_KERBEROS \
- | NFS_MOUNT_NONLM \
- | NFS_MOUNT_BROKEN_SUID \
- | NFS_MOUNT_STRICTLOCK \
- | NFS_MOUNT_UNSHARED \
- | NFS_MOUNT_NORESVPORT \
- | NFS_MOUNT_LEGACY_INTERFACE)
-
static int nfs_compare_mount_options(const struct super_block *s, const struct nfs_server *b, int flags)
{
const struct nfs_server *a = s->s_fs_info;
diff --git a/fs/nfs/sysctl.c b/fs/nfs/sysctl.c
index 6b3f253..bb6ed81 100644
--- a/fs/nfs/sysctl.c
+++ b/fs/nfs/sysctl.c
@@ -13,7 +13,7 @@
static struct ctl_table_header *nfs_callback_sysctl_table;
-static ctl_table nfs_cb_sysctls[] = {
+static struct ctl_table nfs_cb_sysctls[] = {
{
.procname = "nfs_mountpoint_timeout",
.data = &nfs_mountpoint_expiry_timeout,
@@ -31,7 +31,7 @@
{ }
};
-static ctl_table nfs_cb_sysctl_dir[] = {
+static struct ctl_table nfs_cb_sysctl_dir[] = {
{
.procname = "nfs",
.mode = 0555,
@@ -40,7 +40,7 @@
{ }
};
-static ctl_table nfs_cb_sysctl_root[] = {
+static struct ctl_table nfs_cb_sysctl_root[] = {
{
.procname = "fs",
.mode = 0555,
diff --git a/fs/nfs/write.c b/fs/nfs/write.c
index ffb9459..3ee5af4 100644
--- a/fs/nfs/write.c
+++ b/fs/nfs/write.c
@@ -42,10 +42,10 @@
* Local function declarations
*/
static void nfs_redirty_request(struct nfs_page *req);
-static const struct rpc_call_ops nfs_write_common_ops;
static const struct rpc_call_ops nfs_commit_ops;
static const struct nfs_pgio_completion_ops nfs_async_write_completion_ops;
static const struct nfs_commit_completion_ops nfs_commit_completion_ops;
+static const struct nfs_rw_ops nfs_rw_write_ops;
static struct kmem_cache *nfs_wdata_cachep;
static mempool_t *nfs_wdata_mempool;
@@ -70,76 +70,19 @@
}
EXPORT_SYMBOL_GPL(nfs_commit_free);
-struct nfs_write_header *nfs_writehdr_alloc(void)
+static struct nfs_rw_header *nfs_writehdr_alloc(void)
{
- struct nfs_write_header *p = mempool_alloc(nfs_wdata_mempool, GFP_NOIO);
+ struct nfs_rw_header *p = mempool_alloc(nfs_wdata_mempool, GFP_NOIO);
- if (p) {
- struct nfs_pgio_header *hdr = &p->header;
-
+ if (p)
memset(p, 0, sizeof(*p));
- INIT_LIST_HEAD(&hdr->pages);
- INIT_LIST_HEAD(&hdr->rpc_list);
- spin_lock_init(&hdr->lock);
- atomic_set(&hdr->refcnt, 0);
- hdr->verf = &p->verf;
- }
return p;
}
-EXPORT_SYMBOL_GPL(nfs_writehdr_alloc);
-static struct nfs_write_data *nfs_writedata_alloc(struct nfs_pgio_header *hdr,
- unsigned int pagecount)
+static void nfs_writehdr_free(struct nfs_rw_header *whdr)
{
- struct nfs_write_data *data, *prealloc;
-
- prealloc = &container_of(hdr, struct nfs_write_header, header)->rpc_data;
- if (prealloc->header == NULL)
- data = prealloc;
- else
- data = kzalloc(sizeof(*data), GFP_KERNEL);
- if (!data)
- goto out;
-
- if (nfs_pgarray_set(&data->pages, pagecount)) {
- data->header = hdr;
- atomic_inc(&hdr->refcnt);
- } else {
- if (data != prealloc)
- kfree(data);
- data = NULL;
- }
-out:
- return data;
-}
-
-void nfs_writehdr_free(struct nfs_pgio_header *hdr)
-{
- struct nfs_write_header *whdr = container_of(hdr, struct nfs_write_header, header);
mempool_free(whdr, nfs_wdata_mempool);
}
-EXPORT_SYMBOL_GPL(nfs_writehdr_free);
-
-void nfs_writedata_release(struct nfs_write_data *wdata)
-{
- struct nfs_pgio_header *hdr = wdata->header;
- struct nfs_write_header *write_header = container_of(hdr, struct nfs_write_header, header);
-
- put_nfs_open_context(wdata->args.context);
- if (wdata->pages.pagevec != wdata->pages.page_array)
- kfree(wdata->pages.pagevec);
- if (wdata == &write_header->rpc_data) {
- wdata->header = NULL;
- wdata = NULL;
- }
- if (atomic_dec_and_test(&hdr->refcnt))
- hdr->completion_ops->completion(hdr);
- /* Note: we only free the rpc_task after callbacks are done.
- * See the comment in rpc_free_task() for why
- */
- kfree(wdata);
-}
-EXPORT_SYMBOL_GPL(nfs_writedata_release);
static void nfs_context_set_write_error(struct nfs_open_context *ctx, int error)
{
@@ -211,18 +154,78 @@
nfs_zap_mapping(page_file_mapping(page)->host, page_file_mapping(page));
}
+/*
+ * nfs_page_group_search_locked
+ * @head - head request of page group
+ * @page_offset - offset into page
+ *
+ * Search page group with head @head to find a request that contains the
+ * page offset @page_offset.
+ *
+ * Returns a pointer to the first matching nfs request, or NULL if no
+ * match is found.
+ *
+ * Must be called with the page group lock held
+ */
+static struct nfs_page *
+nfs_page_group_search_locked(struct nfs_page *head, unsigned int page_offset)
+{
+ struct nfs_page *req;
+
+ WARN_ON_ONCE(head != head->wb_head);
+ WARN_ON_ONCE(!test_bit(PG_HEADLOCK, &head->wb_head->wb_flags));
+
+ req = head;
+ do {
+ if (page_offset >= req->wb_pgbase &&
+ page_offset < (req->wb_pgbase + req->wb_bytes))
+ return req;
+
+ req = req->wb_this_page;
+ } while (req != head);
+
+ return NULL;
+}
+
+/*
+ * nfs_page_group_covers_page
+ * @head - head request of page group
+ *
+ * Return true if the page group with head @head covers the whole page,
+ * returns false otherwise
+ */
+static bool nfs_page_group_covers_page(struct nfs_page *req)
+{
+ struct nfs_page *tmp;
+ unsigned int pos = 0;
+ unsigned int len = nfs_page_length(req->wb_page);
+
+ nfs_page_group_lock(req);
+
+ do {
+ tmp = nfs_page_group_search_locked(req->wb_head, pos);
+ if (tmp) {
+ /* no way this should happen */
+ WARN_ON_ONCE(tmp->wb_pgbase != pos);
+ pos += tmp->wb_bytes - (pos - tmp->wb_pgbase);
+ }
+ } while (tmp && pos < len);
+
+ nfs_page_group_unlock(req);
+ WARN_ON_ONCE(pos > len);
+ return pos == len;
+}
+
/* We can set the PG_uptodate flag if we see that a write request
* covers the full page.
*/
-static void nfs_mark_uptodate(struct page *page, unsigned int base, unsigned int count)
+static void nfs_mark_uptodate(struct nfs_page *req)
{
- if (PageUptodate(page))
+ if (PageUptodate(req->wb_page))
return;
- if (base != 0)
+ if (!nfs_page_group_covers_page(req))
return;
- if (count != nfs_page_length(page))
- return;
- SetPageUptodate(page);
+ SetPageUptodate(req->wb_page);
}
static int wb_priority(struct writeback_control *wbc)
@@ -258,12 +261,15 @@
}
}
-static void nfs_end_page_writeback(struct page *page)
+static void nfs_end_page_writeback(struct nfs_page *req)
{
- struct inode *inode = page_file_mapping(page)->host;
+ struct inode *inode = page_file_mapping(req->wb_page)->host;
struct nfs_server *nfss = NFS_SERVER(inode);
- end_page_writeback(page);
+ if (!nfs_page_group_sync_on_bit(req, PG_WB_END))
+ return;
+
+ end_page_writeback(req->wb_page);
if (atomic_long_dec_return(&nfss->writeback) < NFS_CONGESTION_OFF_THRESH)
clear_bdi_congested(&nfss->backing_dev_info, BLK_RW_ASYNC);
}
@@ -354,10 +360,8 @@
struct nfs_pageio_descriptor pgio;
int err;
- NFS_PROTO(page_file_mapping(page)->host)->write_pageio_init(&pgio,
- page->mapping->host,
- wb_priority(wbc),
- &nfs_async_write_completion_ops);
+ nfs_pageio_init_write(&pgio, page->mapping->host, wb_priority(wbc),
+ false, &nfs_async_write_completion_ops);
err = nfs_do_writepage(page, wbc, &pgio);
nfs_pageio_complete(&pgio);
if (err < 0)
@@ -400,7 +404,8 @@
nfs_inc_stats(inode, NFSIOS_VFSWRITEPAGES);
- NFS_PROTO(inode)->write_pageio_init(&pgio, inode, wb_priority(wbc), &nfs_async_write_completion_ops);
+ nfs_pageio_init_write(&pgio, inode, wb_priority(wbc), false,
+ &nfs_async_write_completion_ops);
err = write_cache_pages(mapping, wbc, nfs_writepages_callback, &pgio);
nfs_pageio_complete(&pgio);
@@ -425,6 +430,8 @@
{
struct nfs_inode *nfsi = NFS_I(inode);
+ WARN_ON_ONCE(req->wb_this_page != req);
+
/* Lock the request! */
nfs_lock_request(req);
@@ -441,6 +448,7 @@
set_page_private(req->wb_page, (unsigned long)req);
}
nfsi->npages++;
+ set_bit(PG_INODE_REF, &req->wb_flags);
kref_get(&req->wb_kref);
spin_unlock(&inode->i_lock);
}
@@ -452,15 +460,20 @@
{
struct inode *inode = req->wb_context->dentry->d_inode;
struct nfs_inode *nfsi = NFS_I(inode);
+ struct nfs_page *head;
- spin_lock(&inode->i_lock);
- if (likely(!PageSwapCache(req->wb_page))) {
- set_page_private(req->wb_page, 0);
- ClearPagePrivate(req->wb_page);
- clear_bit(PG_MAPPED, &req->wb_flags);
+ if (nfs_page_group_sync_on_bit(req, PG_REMOVE)) {
+ head = req->wb_head;
+
+ spin_lock(&inode->i_lock);
+ if (likely(!PageSwapCache(head->wb_page))) {
+ set_page_private(head->wb_page, 0);
+ ClearPagePrivate(head->wb_page);
+ clear_bit(PG_MAPPED, &head->wb_flags);
+ }
+ nfsi->npages--;
+ spin_unlock(&inode->i_lock);
}
- nfsi->npages--;
- spin_unlock(&inode->i_lock);
nfs_release_request(req);
}
@@ -583,7 +596,7 @@
}
static inline
-int nfs_write_need_commit(struct nfs_write_data *data)
+int nfs_write_need_commit(struct nfs_pgio_data *data)
{
if (data->verf.committed == NFS_DATA_SYNC)
return data->header->lseg == NULL;
@@ -614,7 +627,7 @@
}
static inline
-int nfs_write_need_commit(struct nfs_write_data *data)
+int nfs_write_need_commit(struct nfs_pgio_data *data)
{
return 0;
}
@@ -625,6 +638,7 @@
{
struct nfs_commit_info cinfo;
unsigned long bytes = 0;
+ bool do_destroy;
if (test_bit(NFS_IOHDR_REDO, &hdr->flags))
goto out;
@@ -645,7 +659,7 @@
goto next;
}
if (test_bit(NFS_IOHDR_NEED_COMMIT, &hdr->flags)) {
- memcpy(&req->wb_verf, &hdr->verf->verifier, sizeof(req->wb_verf));
+ memcpy(&req->wb_verf, &hdr->verf.verifier, sizeof(req->wb_verf));
nfs_mark_request_commit(req, hdr->lseg, &cinfo);
goto next;
}
@@ -653,7 +667,8 @@
nfs_inode_remove_request(req);
next:
nfs_unlock_request(req);
- nfs_end_page_writeback(req->wb_page);
+ nfs_end_page_writeback(req);
+ do_destroy = !test_bit(NFS_IOHDR_NEED_COMMIT, &hdr->flags);
nfs_release_request(req);
}
out:
@@ -661,7 +676,7 @@
}
#if IS_ENABLED(CONFIG_NFS_V3) || IS_ENABLED(CONFIG_NFS_V4)
-static unsigned long
+unsigned long
nfs_reqs_to_commit(struct nfs_commit_info *cinfo)
{
return cinfo->mds->ncommit;
@@ -718,7 +733,7 @@
}
#else
-static unsigned long nfs_reqs_to_commit(struct nfs_commit_info *cinfo)
+unsigned long nfs_reqs_to_commit(struct nfs_commit_info *cinfo)
{
return 0;
}
@@ -758,6 +773,10 @@
if (req == NULL)
goto out_unlock;
+ /* should be handled by nfs_flush_incompatible */
+ WARN_ON_ONCE(req->wb_head != req);
+ WARN_ON_ONCE(req->wb_this_page != req);
+
rqend = req->wb_offset + req->wb_bytes;
/*
* Tell the caller to flush out the request if
@@ -819,7 +838,7 @@
req = nfs_try_to_update_request(inode, page, offset, bytes);
if (req != NULL)
goto out;
- req = nfs_create_request(ctx, inode, page, offset, bytes);
+ req = nfs_create_request(ctx, page, NULL, offset, bytes);
if (IS_ERR(req))
goto out;
nfs_inode_add_request(inode, req);
@@ -837,7 +856,7 @@
return PTR_ERR(req);
/* Update file length */
nfs_grow_file(page, offset, count);
- nfs_mark_uptodate(page, req->wb_pgbase, req->wb_bytes);
+ nfs_mark_uptodate(req);
nfs_mark_request_dirty(req);
nfs_unlock_and_release_request(req);
return 0;
@@ -863,6 +882,8 @@
return 0;
l_ctx = req->wb_lock_context;
do_flush = req->wb_page != page || req->wb_context != ctx;
+ /* for now, flush if more than 1 request in page_group */
+ do_flush |= req->wb_this_page != req;
if (l_ctx && ctx->dentry->d_inode->i_flock != NULL) {
do_flush |= l_ctx->lockowner.l_owner != current->files
|| l_ctx->lockowner.l_pid != current->tgid;
@@ -990,126 +1011,17 @@
return RPC_PRIORITY_NORMAL;
}
-int nfs_initiate_write(struct rpc_clnt *clnt,
- struct nfs_write_data *data,
- const struct rpc_call_ops *call_ops,
- int how, int flags)
+static void nfs_initiate_write(struct nfs_pgio_data *data, struct rpc_message *msg,
+ struct rpc_task_setup *task_setup_data, int how)
{
struct inode *inode = data->header->inode;
int priority = flush_task_priority(how);
- struct rpc_task *task;
- struct rpc_message msg = {
- .rpc_argp = &data->args,
- .rpc_resp = &data->res,
- .rpc_cred = data->header->cred,
- };
- struct rpc_task_setup task_setup_data = {
- .rpc_client = clnt,
- .task = &data->task,
- .rpc_message = &msg,
- .callback_ops = call_ops,
- .callback_data = data,
- .workqueue = nfsiod_workqueue,
- .flags = RPC_TASK_ASYNC | flags,
- .priority = priority,
- };
- int ret = 0;
- /* Set up the initial task struct. */
- NFS_PROTO(inode)->write_setup(data, &msg);
-
- dprintk("NFS: %5u initiated write call "
- "(req %s/%llu, %u bytes @ offset %llu)\n",
- data->task.tk_pid,
- inode->i_sb->s_id,
- (unsigned long long)NFS_FILEID(inode),
- data->args.count,
- (unsigned long long)data->args.offset);
+ task_setup_data->priority = priority;
+ NFS_PROTO(inode)->write_setup(data, msg);
nfs4_state_protect_write(NFS_SERVER(inode)->nfs_client,
- &task_setup_data.rpc_client, &msg, data);
-
- task = rpc_run_task(&task_setup_data);
- if (IS_ERR(task)) {
- ret = PTR_ERR(task);
- goto out;
- }
- if (how & FLUSH_SYNC) {
- ret = rpc_wait_for_completion_task(task);
- if (ret == 0)
- ret = task->tk_status;
- }
- rpc_put_task(task);
-out:
- return ret;
-}
-EXPORT_SYMBOL_GPL(nfs_initiate_write);
-
-/*
- * Set up the argument/result storage required for the RPC call.
- */
-static void nfs_write_rpcsetup(struct nfs_write_data *data,
- unsigned int count, unsigned int offset,
- int how, struct nfs_commit_info *cinfo)
-{
- struct nfs_page *req = data->header->req;
-
- /* Set up the RPC argument and reply structs
- * NB: take care not to mess about with data->commit et al. */
-
- data->args.fh = NFS_FH(data->header->inode);
- data->args.offset = req_offset(req) + offset;
- /* pnfs_set_layoutcommit needs this */
- data->mds_offset = data->args.offset;
- data->args.pgbase = req->wb_pgbase + offset;
- data->args.pages = data->pages.pagevec;
- data->args.count = count;
- data->args.context = get_nfs_open_context(req->wb_context);
- data->args.lock_context = req->wb_lock_context;
- data->args.stable = NFS_UNSTABLE;
- switch (how & (FLUSH_STABLE | FLUSH_COND_STABLE)) {
- case 0:
- break;
- case FLUSH_COND_STABLE:
- if (nfs_reqs_to_commit(cinfo))
- break;
- default:
- data->args.stable = NFS_FILE_SYNC;
- }
-
- data->res.fattr = &data->fattr;
- data->res.count = count;
- data->res.verf = &data->verf;
- nfs_fattr_init(&data->fattr);
-}
-
-static int nfs_do_write(struct nfs_write_data *data,
- const struct rpc_call_ops *call_ops,
- int how)
-{
- struct inode *inode = data->header->inode;
-
- return nfs_initiate_write(NFS_CLIENT(inode), data, call_ops, how, 0);
-}
-
-static int nfs_do_multiple_writes(struct list_head *head,
- const struct rpc_call_ops *call_ops,
- int how)
-{
- struct nfs_write_data *data;
- int ret = 0;
-
- while (!list_empty(head)) {
- int ret2;
-
- data = list_first_entry(head, struct nfs_write_data, list);
- list_del_init(&data->list);
-
- ret2 = nfs_do_write(data, call_ops, how);
- if (ret == 0)
- ret = ret2;
- }
- return ret;
+ &task_setup_data->rpc_client, msg, data);
}
/* If a nfs_flush_* function fails, it should remove reqs from @head and
@@ -1120,7 +1032,7 @@
{
nfs_mark_request_dirty(req);
nfs_unlock_request(req);
- nfs_end_page_writeback(req->wb_page);
+ nfs_end_page_writeback(req);
nfs_release_request(req);
}
@@ -1140,173 +1052,30 @@
.completion = nfs_write_completion,
};
-static void nfs_flush_error(struct nfs_pageio_descriptor *desc,
- struct nfs_pgio_header *hdr)
-{
- set_bit(NFS_IOHDR_REDO, &hdr->flags);
- while (!list_empty(&hdr->rpc_list)) {
- struct nfs_write_data *data = list_first_entry(&hdr->rpc_list,
- struct nfs_write_data, list);
- list_del(&data->list);
- nfs_writedata_release(data);
- }
- desc->pg_completion_ops->error_cleanup(&desc->pg_list);
-}
-
-/*
- * Generate multiple small requests to write out a single
- * contiguous dirty area on one page.
- */
-static int nfs_flush_multi(struct nfs_pageio_descriptor *desc,
- struct nfs_pgio_header *hdr)
-{
- struct nfs_page *req = hdr->req;
- struct page *page = req->wb_page;
- struct nfs_write_data *data;
- size_t wsize = desc->pg_bsize, nbytes;
- unsigned int offset;
- int requests = 0;
- struct nfs_commit_info cinfo;
-
- nfs_init_cinfo(&cinfo, desc->pg_inode, desc->pg_dreq);
-
- if ((desc->pg_ioflags & FLUSH_COND_STABLE) &&
- (desc->pg_moreio || nfs_reqs_to_commit(&cinfo) ||
- desc->pg_count > wsize))
- desc->pg_ioflags &= ~FLUSH_COND_STABLE;
-
-
- offset = 0;
- nbytes = desc->pg_count;
- do {
- size_t len = min(nbytes, wsize);
-
- data = nfs_writedata_alloc(hdr, 1);
- if (!data) {
- nfs_flush_error(desc, hdr);
- return -ENOMEM;
- }
- data->pages.pagevec[0] = page;
- nfs_write_rpcsetup(data, len, offset, desc->pg_ioflags, &cinfo);
- list_add(&data->list, &hdr->rpc_list);
- requests++;
- nbytes -= len;
- offset += len;
- } while (nbytes != 0);
- nfs_list_remove_request(req);
- nfs_list_add_request(req, &hdr->pages);
- desc->pg_rpc_callops = &nfs_write_common_ops;
- return 0;
-}
-
-/*
- * Create an RPC task for the given write request and kick it.
- * The page must have been locked by the caller.
- *
- * It may happen that the page we're passed is not marked dirty.
- * This is the case if nfs_updatepage detects a conflicting request
- * that has been written but not committed.
- */
-static int nfs_flush_one(struct nfs_pageio_descriptor *desc,
- struct nfs_pgio_header *hdr)
-{
- struct nfs_page *req;
- struct page **pages;
- struct nfs_write_data *data;
- struct list_head *head = &desc->pg_list;
- struct nfs_commit_info cinfo;
-
- data = nfs_writedata_alloc(hdr, nfs_page_array_len(desc->pg_base,
- desc->pg_count));
- if (!data) {
- nfs_flush_error(desc, hdr);
- return -ENOMEM;
- }
-
- nfs_init_cinfo(&cinfo, desc->pg_inode, desc->pg_dreq);
- pages = data->pages.pagevec;
- while (!list_empty(head)) {
- req = nfs_list_entry(head->next);
- nfs_list_remove_request(req);
- nfs_list_add_request(req, &hdr->pages);
- *pages++ = req->wb_page;
- }
-
- if ((desc->pg_ioflags & FLUSH_COND_STABLE) &&
- (desc->pg_moreio || nfs_reqs_to_commit(&cinfo)))
- desc->pg_ioflags &= ~FLUSH_COND_STABLE;
-
- /* Set up the argument struct */
- nfs_write_rpcsetup(data, desc->pg_count, 0, desc->pg_ioflags, &cinfo);
- list_add(&data->list, &hdr->rpc_list);
- desc->pg_rpc_callops = &nfs_write_common_ops;
- return 0;
-}
-
-int nfs_generic_flush(struct nfs_pageio_descriptor *desc,
- struct nfs_pgio_header *hdr)
-{
- if (desc->pg_bsize < PAGE_CACHE_SIZE)
- return nfs_flush_multi(desc, hdr);
- return nfs_flush_one(desc, hdr);
-}
-EXPORT_SYMBOL_GPL(nfs_generic_flush);
-
-static int nfs_generic_pg_writepages(struct nfs_pageio_descriptor *desc)
-{
- struct nfs_write_header *whdr;
- struct nfs_pgio_header *hdr;
- int ret;
-
- whdr = nfs_writehdr_alloc();
- if (!whdr) {
- desc->pg_completion_ops->error_cleanup(&desc->pg_list);
- return -ENOMEM;
- }
- hdr = &whdr->header;
- nfs_pgheader_init(desc, hdr, nfs_writehdr_free);
- atomic_inc(&hdr->refcnt);
- ret = nfs_generic_flush(desc, hdr);
- if (ret == 0)
- ret = nfs_do_multiple_writes(&hdr->rpc_list,
- desc->pg_rpc_callops,
- desc->pg_ioflags);
- if (atomic_dec_and_test(&hdr->refcnt))
- hdr->completion_ops->completion(hdr);
- return ret;
-}
-
-static const struct nfs_pageio_ops nfs_pageio_write_ops = {
- .pg_test = nfs_generic_pg_test,
- .pg_doio = nfs_generic_pg_writepages,
-};
-
void nfs_pageio_init_write(struct nfs_pageio_descriptor *pgio,
- struct inode *inode, int ioflags,
+ struct inode *inode, int ioflags, bool force_mds,
const struct nfs_pgio_completion_ops *compl_ops)
{
- nfs_pageio_init(pgio, inode, &nfs_pageio_write_ops, compl_ops,
- NFS_SERVER(inode)->wsize, ioflags);
+ struct nfs_server *server = NFS_SERVER(inode);
+ const struct nfs_pageio_ops *pg_ops = &nfs_pgio_rw_ops;
+
+#ifdef CONFIG_NFS_V4_1
+ if (server->pnfs_curr_ld && !force_mds)
+ pg_ops = server->pnfs_curr_ld->pg_write_ops;
+#endif
+ nfs_pageio_init(pgio, inode, pg_ops, compl_ops, &nfs_rw_write_ops,
+ server->wsize, ioflags);
}
EXPORT_SYMBOL_GPL(nfs_pageio_init_write);
void nfs_pageio_reset_write_mds(struct nfs_pageio_descriptor *pgio)
{
- pgio->pg_ops = &nfs_pageio_write_ops;
+ pgio->pg_ops = &nfs_pgio_rw_ops;
pgio->pg_bsize = NFS_SERVER(pgio->pg_inode)->wsize;
}
EXPORT_SYMBOL_GPL(nfs_pageio_reset_write_mds);
-void nfs_write_prepare(struct rpc_task *task, void *calldata)
-{
- struct nfs_write_data *data = calldata;
- int err;
- err = NFS_PROTO(data->header->inode)->write_rpc_prepare(task, data);
- if (err)
- rpc_exit(task, err);
-}
-
void nfs_commit_prepare(struct rpc_task *task, void *calldata)
{
struct nfs_commit_data *data = calldata;
@@ -1314,23 +1083,8 @@
NFS_PROTO(data->inode)->commit_rpc_prepare(task, data);
}
-/*
- * Handle a write reply that flushes a whole page.
- *
- * FIXME: There is an inherent race with invalidate_inode_pages and
- * writebacks since the page->count is kept > 1 for as long
- * as the page has a write request pending.
- */
-static void nfs_writeback_done_common(struct rpc_task *task, void *calldata)
+static void nfs_writeback_release_common(struct nfs_pgio_data *data)
{
- struct nfs_write_data *data = calldata;
-
- nfs_writeback_done(task, data);
-}
-
-static void nfs_writeback_release_common(void *calldata)
-{
- struct nfs_write_data *data = calldata;
struct nfs_pgio_header *hdr = data->header;
int status = data->task.tk_status;
@@ -1339,34 +1093,46 @@
if (test_bit(NFS_IOHDR_NEED_RESCHED, &hdr->flags))
; /* Do nothing */
else if (!test_and_set_bit(NFS_IOHDR_NEED_COMMIT, &hdr->flags))
- memcpy(hdr->verf, &data->verf, sizeof(*hdr->verf));
- else if (memcmp(hdr->verf, &data->verf, sizeof(*hdr->verf)))
+ memcpy(&hdr->verf, &data->verf, sizeof(hdr->verf));
+ else if (memcmp(&hdr->verf, &data->verf, sizeof(hdr->verf)))
set_bit(NFS_IOHDR_NEED_RESCHED, &hdr->flags);
spin_unlock(&hdr->lock);
}
- nfs_writedata_release(data);
}
-static const struct rpc_call_ops nfs_write_common_ops = {
- .rpc_call_prepare = nfs_write_prepare,
- .rpc_call_done = nfs_writeback_done_common,
- .rpc_release = nfs_writeback_release_common,
-};
+/*
+ * Special version of should_remove_suid() that ignores capabilities.
+ */
+static int nfs_should_remove_suid(const struct inode *inode)
+{
+ umode_t mode = inode->i_mode;
+ int kill = 0;
+ /* suid always must be killed */
+ if (unlikely(mode & S_ISUID))
+ kill = ATTR_KILL_SUID;
+
+ /*
+ * sgid without any exec bits is just a mandatory locking mark; leave
+ * it alone. If some exec bits are set, it's a real sgid; kill it.
+ */
+ if (unlikely((mode & S_ISGID) && (mode & S_IXGRP)))
+ kill |= ATTR_KILL_SGID;
+
+ if (unlikely(kill && S_ISREG(mode)))
+ return kill;
+
+ return 0;
+}
/*
* This function is called when the WRITE call is complete.
*/
-void nfs_writeback_done(struct rpc_task *task, struct nfs_write_data *data)
+static int nfs_writeback_done(struct rpc_task *task, struct nfs_pgio_data *data,
+ struct inode *inode)
{
- struct nfs_writeargs *argp = &data->args;
- struct nfs_writeres *resp = &data->res;
- struct inode *inode = data->header->inode;
int status;
- dprintk("NFS: %5u nfs_writeback_done (status %d)\n",
- task->tk_pid, task->tk_status);
-
/*
* ->write_done will attempt to use post-op attributes to detect
* conflicting writes by other clients. A strict interpretation
@@ -1376,11 +1142,11 @@
*/
status = NFS_PROTO(inode)->write_done(task, data);
if (status != 0)
- return;
- nfs_add_stats(inode, NFSIOS_SERVERWRITTENBYTES, resp->count);
+ return status;
+ nfs_add_stats(inode, NFSIOS_SERVERWRITTENBYTES, data->res.count);
#if IS_ENABLED(CONFIG_NFS_V3) || IS_ENABLED(CONFIG_NFS_V4)
- if (resp->verf->committed < argp->stable && task->tk_status >= 0) {
+ if (data->res.verf->committed < data->args.stable && task->tk_status >= 0) {
/* We tried a write call, but the server did not
* commit data to stable storage even though we
* requested it.
@@ -1396,18 +1162,31 @@
dprintk("NFS: faulty NFS server %s:"
" (committed = %d) != (stable = %d)\n",
NFS_SERVER(inode)->nfs_client->cl_hostname,
- resp->verf->committed, argp->stable);
+ data->res.verf->committed, data->args.stable);
complain = jiffies + 300 * HZ;
}
}
#endif
- if (task->tk_status < 0)
- nfs_set_pgio_error(data->header, task->tk_status, argp->offset);
- else if (resp->count < argp->count) {
+
+ /* Deal with the suid/sgid bit corner case */
+ if (nfs_should_remove_suid(inode))
+ nfs_mark_for_revalidate(inode);
+ return 0;
+}
+
+/*
+ * This function is called when the WRITE call is complete.
+ */
+static void nfs_writeback_result(struct rpc_task *task, struct nfs_pgio_data *data)
+{
+ struct nfs_pgio_args *argp = &data->args;
+ struct nfs_pgio_res *resp = &data->res;
+
+ if (resp->count < argp->count) {
static unsigned long complain;
/* This a short write! */
- nfs_inc_stats(inode, NFSIOS_SHORTWRITE);
+ nfs_inc_stats(data->header->inode, NFSIOS_SHORTWRITE);
/* Has the server at least made some progress? */
if (resp->count == 0) {
@@ -1874,7 +1653,7 @@
int __init nfs_init_writepagecache(void)
{
nfs_wdata_cachep = kmem_cache_create("nfs_write_data",
- sizeof(struct nfs_write_header),
+ sizeof(struct nfs_rw_header),
0, SLAB_HWCACHE_ALIGN,
NULL);
if (nfs_wdata_cachep == NULL)
@@ -1936,3 +1715,12 @@
kmem_cache_destroy(nfs_wdata_cachep);
}
+static const struct nfs_rw_ops nfs_rw_write_ops = {
+ .rw_mode = FMODE_WRITE,
+ .rw_alloc_header = nfs_writehdr_alloc,
+ .rw_free_header = nfs_writehdr_free,
+ .rw_release = nfs_writeback_release_common,
+ .rw_done = nfs_writeback_done,
+ .rw_result = nfs_writeback_result,
+ .rw_initiate = nfs_initiate_write,
+};
diff --git a/fs/nfsd/acl.h b/fs/nfsd/acl.h
index b481e1f..a986ceb 100644
--- a/fs/nfsd/acl.h
+++ b/fs/nfsd/acl.h
@@ -49,7 +49,7 @@
struct nfs4_acl *nfs4_acl_new(int);
int nfs4_acl_get_whotype(char *, u32);
-__be32 nfs4_acl_write_who(int who, __be32 **p, int *len);
+__be32 nfs4_acl_write_who(struct xdr_stream *xdr, int who);
int nfsd4_get_nfs4_acl(struct svc_rqst *rqstp, struct dentry *dentry,
struct nfs4_acl **acl);
diff --git a/fs/nfsd/auth.c b/fs/nfsd/auth.c
index 2645be4..72f4482 100644
--- a/fs/nfsd/auth.c
+++ b/fs/nfsd/auth.c
@@ -1,7 +1,6 @@
/* Copyright (C) 1995, 1996 Olaf Kirch <okir@monad.swb.de> */
#include <linux/sched.h>
-#include <linux/user_namespace.h>
#include "nfsd.h"
#include "auth.h"
@@ -25,7 +24,6 @@
struct cred *new;
int i;
int flags = nfsexp_flags(rqstp, exp);
- int ret;
validate_process_creds();
@@ -86,8 +84,7 @@
return 0;
oom:
- ret = -ENOMEM;
abort_creds(new);
- return ret;
+ return -ENOMEM;
}
diff --git a/fs/nfsd/export.c b/fs/nfsd/export.c
index 8513c59..13b85f9 100644
--- a/fs/nfsd/export.c
+++ b/fs/nfsd/export.c
@@ -17,17 +17,12 @@
#include <linux/exportfs.h>
#include <linux/sunrpc/svc_xprt.h>
-#include <net/ipv6.h>
-
#include "nfsd.h"
#include "nfsfh.h"
#include "netns.h"
#define NFSDDBG_FACILITY NFSDDBG_EXPORT
-typedef struct auth_domain svc_client;
-typedef struct svc_export svc_export;
-
/*
* We have two caches.
* One maps client+vfsmnt+dentry to export options - the export map
@@ -73,7 +68,7 @@
static int expkey_parse(struct cache_detail *cd, char *mesg, int mlen)
{
- /* client fsidtype fsid [path] */
+ /* client fsidtype fsid expiry [path] */
char *buf;
int len;
struct auth_domain *dom = NULL;
@@ -295,13 +290,19 @@
static void nfsd4_fslocs_free(struct nfsd4_fs_locations *fsloc)
{
+ struct nfsd4_fs_location *locations = fsloc->locations;
int i;
+ if (!locations)
+ return;
+
for (i = 0; i < fsloc->locations_count; i++) {
- kfree(fsloc->locations[i].path);
- kfree(fsloc->locations[i].hosts);
+ kfree(locations[i].path);
+ kfree(locations[i].hosts);
}
- kfree(fsloc->locations);
+
+ kfree(locations);
+ fsloc->locations = NULL;
}
static void svc_export_put(struct kref *ref)
@@ -388,6 +389,10 @@
int len;
int migrated, i, err;
+ /* more than one fsloc */
+ if (fsloc->locations)
+ return -EINVAL;
+
/* listsize */
err = get_uint(mesg, &fsloc->locations_count);
if (err)
@@ -437,13 +442,18 @@
static int secinfo_parse(char **mesg, char *buf, struct svc_export *exp)
{
- int listsize, err;
struct exp_flavor_info *f;
+ u32 listsize;
+ int err;
- err = get_int(mesg, &listsize);
+ /* more than one secinfo */
+ if (exp->ex_nflavors)
+ return -EINVAL;
+
+ err = get_uint(mesg, &listsize);
if (err)
return err;
- if (listsize < 0 || listsize > MAX_SECINFO_LIST)
+ if (listsize > MAX_SECINFO_LIST)
return -EINVAL;
for (f = exp->ex_flavors; f < exp->ex_flavors + listsize; f++) {
@@ -474,6 +484,27 @@
secinfo_parse(char **mesg, char *buf, struct svc_export *exp) { return 0; }
#endif
+static inline int
+uuid_parse(char **mesg, char *buf, unsigned char **puuid)
+{
+ int len;
+
+ /* more than one uuid */
+ if (*puuid)
+ return -EINVAL;
+
+ /* expect a 16 byte uuid encoded as \xXXXX... */
+ len = qword_get(mesg, buf, PAGE_SIZE);
+ if (len != EX_UUID_LEN)
+ return -EINVAL;
+
+ *puuid = kmemdup(buf, EX_UUID_LEN, GFP_KERNEL);
+ if (*puuid == NULL)
+ return -ENOMEM;
+
+ return 0;
+}
+
static int svc_export_parse(struct cache_detail *cd, char *mesg, int mlen)
{
/* client path expiry [flags anonuid anongid fsid] */
@@ -552,18 +583,9 @@
while ((len = qword_get(&mesg, buf, PAGE_SIZE)) > 0) {
if (strcmp(buf, "fsloc") == 0)
err = fsloc_parse(&mesg, buf, &exp.ex_fslocs);
- else if (strcmp(buf, "uuid") == 0) {
- /* expect a 16 byte uuid encoded as \xXXXX... */
- len = qword_get(&mesg, buf, PAGE_SIZE);
- if (len != 16)
- err = -EINVAL;
- else {
- exp.ex_uuid =
- kmemdup(buf, 16, GFP_KERNEL);
- if (exp.ex_uuid == NULL)
- err = -ENOMEM;
- }
- } else if (strcmp(buf, "secinfo") == 0)
+ else if (strcmp(buf, "uuid") == 0)
+ err = uuid_parse(&mesg, buf, &exp.ex_uuid);
+ else if (strcmp(buf, "secinfo") == 0)
err = secinfo_parse(&mesg, buf, &exp);
else
/* quietly ignore unknown words and anything
@@ -649,7 +671,7 @@
if (exp->ex_uuid) {
int i;
seq_puts(m, ",uuid=");
- for (i=0; i<16; i++) {
+ for (i = 0; i < EX_UUID_LEN; i++) {
if ((i&3) == 0 && i)
seq_putc(m, ':');
seq_printf(m, "%02x", exp->ex_uuid[i]);
@@ -771,7 +793,7 @@
static struct svc_expkey *
-exp_find_key(struct cache_detail *cd, svc_client *clp, int fsid_type,
+exp_find_key(struct cache_detail *cd, struct auth_domain *clp, int fsid_type,
u32 *fsidv, struct cache_req *reqp)
{
struct svc_expkey key, *ek;
@@ -793,9 +815,9 @@
return ek;
}
-
-static svc_export *exp_get_by_name(struct cache_detail *cd, svc_client *clp,
- const struct path *path, struct cache_req *reqp)
+static struct svc_export *
+exp_get_by_name(struct cache_detail *cd, struct auth_domain *clp,
+ const struct path *path, struct cache_req *reqp)
{
struct svc_export *exp, key;
int err;
@@ -819,11 +841,11 @@
/*
* Find the export entry for a given dentry.
*/
-static struct svc_export *exp_parent(struct cache_detail *cd, svc_client *clp,
- struct path *path)
+static struct svc_export *
+exp_parent(struct cache_detail *cd, struct auth_domain *clp, struct path *path)
{
struct dentry *saved = dget(path->dentry);
- svc_export *exp = exp_get_by_name(cd, clp, path, NULL);
+ struct svc_export *exp = exp_get_by_name(cd, clp, path, NULL);
while (PTR_ERR(exp) == -ENOENT && !IS_ROOT(path->dentry)) {
struct dentry *parent = dget_parent(path->dentry);
@@ -844,7 +866,7 @@
* since its harder to fool a kernel module than a user space program.
*/
int
-exp_rootfh(struct net *net, svc_client *clp, char *name,
+exp_rootfh(struct net *net, struct auth_domain *clp, char *name,
struct knfsd_fh *f, int maxsize)
{
struct svc_export *exp;
diff --git a/include/linux/nfsd/export.h b/fs/nfsd/export.h
similarity index 93%
rename from include/linux/nfsd/export.h
rename to fs/nfsd/export.h
index 7898c99..cfeea85 100644
--- a/include/linux/nfsd/export.h
+++ b/fs/nfsd/export.h
@@ -1,17 +1,16 @@
/*
- * include/linux/nfsd/export.h
- *
- * Public declarations for NFS exports. The definitions for the
- * syscall interface are in nfsctl.h
- *
* Copyright (C) 1995-1997 Olaf Kirch <okir@monad.swb.de>
*/
#ifndef NFSD_EXPORT_H
#define NFSD_EXPORT_H
-# include <linux/nfsd/nfsfh.h>
+#include <linux/sunrpc/cache.h>
#include <uapi/linux/nfsd/export.h>
+struct knfsd_fh;
+struct svc_fh;
+struct svc_rqst;
+
/*
* FS Locations
*/
@@ -38,6 +37,7 @@
* spkm3i, and spkm3p (and using all 8 at once should be rare).
*/
#define MAX_SECINFO_LIST 8
+#define EX_UUID_LEN 16
struct exp_flavor_info {
u32 pseudoflavor;
@@ -54,7 +54,7 @@
int ex_fsid;
unsigned char * ex_uuid; /* 16 byte fsid */
struct nfsd4_fs_locations ex_fslocs;
- int ex_nflavors;
+ uint32_t ex_nflavors;
struct exp_flavor_info ex_flavors[MAX_SECINFO_LIST];
struct cache_detail *cd;
};
diff --git a/fs/nfsd/fault_inject.c b/fs/nfsd/fault_inject.c
index d620e7f..2ed05c3 100644
--- a/fs/nfsd/fault_inject.c
+++ b/fs/nfsd/fault_inject.c
@@ -97,25 +97,14 @@
{
static u64 val;
char read_buf[25];
- size_t size, ret;
+ size_t size;
loff_t pos = *ppos;
if (!pos)
nfsd_inject_get(file_inode(file)->i_private, &val);
size = scnprintf(read_buf, sizeof(read_buf), "%llu\n", val);
- if (pos < 0)
- return -EINVAL;
- if (pos >= size || !len)
- return 0;
- if (len > size - pos)
- len = size - pos;
- ret = copy_to_user(buf, read_buf + pos, len);
- if (ret == len)
- return -EFAULT;
- len -= ret;
- *ppos = pos + len;
- return len;
+ return simple_read_from_buffer(buf, len, ppos, read_buf, size);
}
static ssize_t fault_inject_write(struct file *file, const char __user *buf,
diff --git a/fs/nfsd/idmap.h b/fs/nfsd/idmap.h
index 66e58db..a3f34900 100644
--- a/fs/nfsd/idmap.h
+++ b/fs/nfsd/idmap.h
@@ -56,7 +56,7 @@
__be32 nfsd_map_name_to_uid(struct svc_rqst *, const char *, size_t, kuid_t *);
__be32 nfsd_map_name_to_gid(struct svc_rqst *, const char *, size_t, kgid_t *);
-__be32 nfsd4_encode_user(struct svc_rqst *, kuid_t, __be32 **, int *);
-__be32 nfsd4_encode_group(struct svc_rqst *, kgid_t, __be32 **, int *);
+__be32 nfsd4_encode_user(struct xdr_stream *, struct svc_rqst *, kuid_t);
+__be32 nfsd4_encode_group(struct xdr_stream *, struct svc_rqst *, kgid_t);
#endif /* LINUX_NFSD_IDMAP_H */
diff --git a/fs/nfsd/nfs2acl.c b/fs/nfsd/nfs2acl.c
index 11c1fba..12b023a 100644
--- a/fs/nfsd/nfs2acl.c
+++ b/fs/nfsd/nfs2acl.c
@@ -182,7 +182,8 @@
static int nfsaclsvc_decode_getaclargs(struct svc_rqst *rqstp, __be32 *p,
struct nfsd3_getaclargs *argp)
{
- if (!(p = nfs2svc_decode_fh(p, &argp->fh)))
+ p = nfs2svc_decode_fh(p, &argp->fh);
+ if (!p)
return 0;
argp->mask = ntohl(*p); p++;
@@ -197,7 +198,8 @@
unsigned int base;
int n;
- if (!(p = nfs2svc_decode_fh(p, &argp->fh)))
+ p = nfs2svc_decode_fh(p, &argp->fh);
+ if (!p)
return 0;
argp->mask = ntohl(*p++);
if (argp->mask & ~(NFS_ACL|NFS_ACLCNT|NFS_DFACL|NFS_DFACLCNT) ||
@@ -218,7 +220,8 @@
static int nfsaclsvc_decode_fhandleargs(struct svc_rqst *rqstp, __be32 *p,
struct nfsd_fhandle *argp)
{
- if (!(p = nfs2svc_decode_fh(p, &argp->fh)))
+ p = nfs2svc_decode_fh(p, &argp->fh);
+ if (!p)
return 0;
return xdr_argsize_check(rqstp, p);
}
@@ -226,7 +229,8 @@
static int nfsaclsvc_decode_accessargs(struct svc_rqst *rqstp, __be32 *p,
struct nfsd3_accessargs *argp)
{
- if (!(p = nfs2svc_decode_fh(p, &argp->fh)))
+ p = nfs2svc_decode_fh(p, &argp->fh);
+ if (!p)
return 0;
argp->access = ntohl(*p++);
diff --git a/fs/nfsd/nfs3acl.c b/fs/nfsd/nfs3acl.c
index adc5f1b..2a514e2 100644
--- a/fs/nfsd/nfs3acl.c
+++ b/fs/nfsd/nfs3acl.c
@@ -128,7 +128,8 @@
static int nfs3svc_decode_getaclargs(struct svc_rqst *rqstp, __be32 *p,
struct nfsd3_getaclargs *args)
{
- if (!(p = nfs3svc_decode_fh(p, &args->fh)))
+ p = nfs3svc_decode_fh(p, &args->fh);
+ if (!p)
return 0;
args->mask = ntohl(*p); p++;
@@ -143,7 +144,8 @@
unsigned int base;
int n;
- if (!(p = nfs3svc_decode_fh(p, &args->fh)))
+ p = nfs3svc_decode_fh(p, &args->fh);
+ if (!p)
return 0;
args->mask = ntohl(*p++);
if (args->mask & ~(NFS_ACL|NFS_ACLCNT|NFS_DFACL|NFS_DFACLCNT) ||
diff --git a/fs/nfsd/nfs3xdr.c b/fs/nfsd/nfs3xdr.c
index de6e39e..e6c01e8 100644
--- a/fs/nfsd/nfs3xdr.c
+++ b/fs/nfsd/nfs3xdr.c
@@ -278,7 +278,8 @@
int
nfs3svc_decode_fhandle(struct svc_rqst *rqstp, __be32 *p, struct nfsd_fhandle *args)
{
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
return xdr_argsize_check(rqstp, p);
}
@@ -287,7 +288,8 @@
nfs3svc_decode_sattrargs(struct svc_rqst *rqstp, __be32 *p,
struct nfsd3_sattrargs *args)
{
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
p = decode_sattr3(p, &args->attrs);
@@ -315,7 +317,8 @@
nfs3svc_decode_accessargs(struct svc_rqst *rqstp, __be32 *p,
struct nfsd3_accessargs *args)
{
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
args->access = ntohl(*p++);
@@ -330,7 +333,8 @@
int v;
u32 max_blocksize = svc_max_payload(rqstp);
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
p = xdr_decode_hyper(p, &args->offset);
@@ -360,7 +364,8 @@
unsigned int len, v, hdr, dlen;
u32 max_blocksize = svc_max_payload(rqstp);
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
p = xdr_decode_hyper(p, &args->offset);
@@ -535,7 +540,8 @@
nfs3svc_decode_readlinkargs(struct svc_rqst *rqstp, __be32 *p,
struct nfsd3_readlinkargs *args)
{
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
args->buffer = page_address(*(rqstp->rq_next_page++));
@@ -558,7 +564,8 @@
nfs3svc_decode_readdirargs(struct svc_rqst *rqstp, __be32 *p,
struct nfsd3_readdirargs *args)
{
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
p = xdr_decode_hyper(p, &args->cookie);
args->verf = p; p += 2;
@@ -580,7 +587,8 @@
int len;
u32 max_blocksize = svc_max_payload(rqstp);
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
p = xdr_decode_hyper(p, &args->cookie);
args->verf = p; p += 2;
@@ -605,7 +613,8 @@
nfs3svc_decode_commitargs(struct svc_rqst *rqstp, __be32 *p,
struct nfsd3_commitargs *args)
{
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
p = xdr_decode_hyper(p, &args->offset);
args->count = ntohl(*p++);
diff --git a/fs/nfsd/nfs4acl.c b/fs/nfsd/nfs4acl.c
index f66c66b..d714156 100644
--- a/fs/nfsd/nfs4acl.c
+++ b/fs/nfsd/nfs4acl.c
@@ -36,7 +36,6 @@
#include <linux/slab.h>
#include <linux/nfs_fs.h>
-#include <linux/export.h>
#include "nfsfh.h"
#include "nfsd.h"
#include "acl.h"
@@ -920,20 +919,19 @@
return NFS4_ACL_WHO_NAMED;
}
-__be32 nfs4_acl_write_who(int who, __be32 **p, int *len)
+__be32 nfs4_acl_write_who(struct xdr_stream *xdr, int who)
{
+ __be32 *p;
int i;
- int bytes;
for (i = 0; i < ARRAY_SIZE(s2t_map); i++) {
if (s2t_map[i].type != who)
continue;
- bytes = 4 + (XDR_QUADLEN(s2t_map[i].stringlen) << 2);
- if (bytes > *len)
+ p = xdr_reserve_space(xdr, s2t_map[i].stringlen + 4);
+ if (!p)
return nfserr_resource;
- *p = xdr_encode_opaque(*p, s2t_map[i].string,
+ p = xdr_encode_opaque(p, s2t_map[i].string,
s2t_map[i].stringlen);
- *len -= bytes;
return 0;
}
WARN_ON_ONCE(1);
diff --git a/fs/nfsd/nfs4idmap.c b/fs/nfsd/nfs4idmap.c
index c0dfde6..a0ab0a8 100644
--- a/fs/nfsd/nfs4idmap.c
+++ b/fs/nfsd/nfs4idmap.c
@@ -551,44 +551,43 @@
return 0;
}
-static __be32 encode_ascii_id(u32 id, __be32 **p, int *buflen)
+static __be32 encode_ascii_id(struct xdr_stream *xdr, u32 id)
{
char buf[11];
int len;
- int bytes;
+ __be32 *p;
len = sprintf(buf, "%u", id);
- bytes = 4 + (XDR_QUADLEN(len) << 2);
- if (bytes > *buflen)
+ p = xdr_reserve_space(xdr, len + 4);
+ if (!p)
return nfserr_resource;
- *p = xdr_encode_opaque(*p, buf, len);
- *buflen -= bytes;
+ p = xdr_encode_opaque(p, buf, len);
return 0;
}
-static __be32 idmap_id_to_name(struct svc_rqst *rqstp, int type, u32 id, __be32 **p, int *buflen)
+static __be32 idmap_id_to_name(struct xdr_stream *xdr,
+ struct svc_rqst *rqstp, int type, u32 id)
{
struct ent *item, key = {
.id = id,
.type = type,
};
+ __be32 *p;
int ret;
- int bytes;
struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
strlcpy(key.authname, rqst_authname(rqstp), sizeof(key.authname));
ret = idmap_lookup(rqstp, idtoname_lookup, &key, nn->idtoname_cache, &item);
if (ret == -ENOENT)
- return encode_ascii_id(id, p, buflen);
+ return encode_ascii_id(xdr, id);
if (ret)
return nfserrno(ret);
ret = strlen(item->name);
WARN_ON_ONCE(ret > IDMAP_NAMESZ);
- bytes = 4 + (XDR_QUADLEN(ret) << 2);
- if (bytes > *buflen)
+ p = xdr_reserve_space(xdr, ret + 4);
+ if (!p)
return nfserr_resource;
- *p = xdr_encode_opaque(*p, item->name, ret);
- *buflen -= bytes;
+ p = xdr_encode_opaque(p, item->name, ret);
cache_put(&item->h, nn->idtoname_cache);
return 0;
}
@@ -622,11 +621,12 @@
return idmap_name_to_id(rqstp, type, name, namelen, id);
}
-static __be32 encode_name_from_id(struct svc_rqst *rqstp, int type, u32 id, __be32 **p, int *buflen)
+static __be32 encode_name_from_id(struct xdr_stream *xdr,
+ struct svc_rqst *rqstp, int type, u32 id)
{
if (nfs4_disable_idmapping && rqstp->rq_cred.cr_flavor < RPC_AUTH_GSS)
- return encode_ascii_id(id, p, buflen);
- return idmap_id_to_name(rqstp, type, id, p, buflen);
+ return encode_ascii_id(xdr, id);
+ return idmap_id_to_name(xdr, rqstp, type, id);
}
__be32
@@ -655,14 +655,16 @@
return status;
}
-__be32 nfsd4_encode_user(struct svc_rqst *rqstp, kuid_t uid, __be32 **p, int *buflen)
+__be32 nfsd4_encode_user(struct xdr_stream *xdr, struct svc_rqst *rqstp,
+ kuid_t uid)
{
u32 id = from_kuid(&init_user_ns, uid);
- return encode_name_from_id(rqstp, IDMAP_TYPE_USER, id, p, buflen);
+ return encode_name_from_id(xdr, rqstp, IDMAP_TYPE_USER, id);
}
-__be32 nfsd4_encode_group(struct svc_rqst *rqstp, kgid_t gid, __be32 **p, int *buflen)
+__be32 nfsd4_encode_group(struct xdr_stream *xdr, struct svc_rqst *rqstp,
+ kgid_t gid)
{
u32 id = from_kgid(&init_user_ns, gid);
- return encode_name_from_id(rqstp, IDMAP_TYPE_GROUP, id, p, buflen);
+ return encode_name_from_id(xdr, rqstp, IDMAP_TYPE_GROUP, id);
}
diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index d543222..6851b00 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -430,12 +430,12 @@
goto out;
break;
case NFS4_OPEN_CLAIM_PREVIOUS:
- open->op_openowner->oo_flags |= NFS4_OO_CONFIRMED;
status = nfs4_check_open_reclaim(&open->op_clientid,
cstate->minorversion,
nn);
if (status)
goto out;
+ open->op_openowner->oo_flags |= NFS4_OO_CONFIRMED;
case NFS4_OPEN_CLAIM_FH:
case NFS4_OPEN_CLAIM_DELEG_CUR_FH:
status = do_open_fhandle(rqstp, cstate, open);
@@ -445,7 +445,6 @@
break;
case NFS4_OPEN_CLAIM_DELEG_PREV_FH:
case NFS4_OPEN_CLAIM_DELEGATE_PREV:
- open->op_openowner->oo_flags |= NFS4_OO_CONFIRMED;
dprintk("NFSD: unsupported OPEN claim type %d\n",
open->op_claim_type);
status = nfserr_notsupp;
@@ -786,7 +785,6 @@
if (!nfsd4_last_compound_op(rqstp))
rqstp->rq_splice_ok = false;
- nfs4_lock_state();
/* check stateid */
if ((status = nfs4_preprocess_stateid_op(SVC_NET(rqstp),
cstate, &read->rd_stateid,
@@ -794,11 +792,8 @@
dprintk("NFSD: nfsd4_read: couldn't process stateid!\n");
goto out;
}
- if (read->rd_filp)
- get_file(read->rd_filp);
status = nfs_ok;
out:
- nfs4_unlock_state();
read->rd_rqstp = rqstp;
read->rd_fhp = &cstate->current_fh;
return status;
@@ -937,10 +932,8 @@
int err;
if (setattr->sa_iattr.ia_valid & ATTR_SIZE) {
- nfs4_lock_state();
status = nfs4_preprocess_stateid_op(SVC_NET(rqstp), cstate,
&setattr->sa_stateid, WR_STATE, NULL);
- nfs4_unlock_state();
if (status) {
dprintk("NFSD: nfsd4_setattr: couldn't process stateid!\n");
return status;
@@ -1006,17 +999,12 @@
if (write->wr_offset >= OFFSET_MAX)
return nfserr_inval;
- nfs4_lock_state();
status = nfs4_preprocess_stateid_op(SVC_NET(rqstp),
cstate, stateid, WR_STATE, &filp);
if (status) {
- nfs4_unlock_state();
dprintk("NFSD: nfsd4_write: couldn't process stateid!\n");
return status;
}
- if (filp)
- get_file(filp);
- nfs4_unlock_state();
cnt = write->wr_buflen;
write->wr_how_written = write->wr_stable_how;
@@ -1072,10 +1060,10 @@
return nfserr_jukebox;
p = buf;
- status = nfsd4_encode_fattr(&cstate->current_fh,
+ status = nfsd4_encode_fattr_to_buf(&p, count, &cstate->current_fh,
cstate->current_fh.fh_export,
- cstate->current_fh.fh_dentry, &p,
- count, verify->ve_bmval,
+ cstate->current_fh.fh_dentry,
+ verify->ve_bmval,
rqstp, 0);
/*
* If nfsd4_encode_fattr() ran out of space, assume that's because
@@ -1182,9 +1170,7 @@
static struct nfsd4_operation nfsd4_ops[];
-#ifdef NFSD_DEBUG
static const char *nfsd4_op_name(unsigned opnum);
-#endif
/*
* Enforce NFSv4.1 COMPOUND ordering rules:
@@ -1226,6 +1212,8 @@
bool nfsd4_cache_this_op(struct nfsd4_op *op)
{
+ if (op->opnum == OP_ILLEGAL)
+ return false;
return OPDESC(op)->op_flags & OP_CACHEME;
}
@@ -1262,6 +1250,25 @@
return !(nextd->op_flags & OP_HANDLES_WRONGSEC);
}
+static void svcxdr_init_encode(struct svc_rqst *rqstp,
+ struct nfsd4_compoundres *resp)
+{
+ struct xdr_stream *xdr = &resp->xdr;
+ struct xdr_buf *buf = &rqstp->rq_res;
+ struct kvec *head = buf->head;
+
+ xdr->buf = buf;
+ xdr->iov = head;
+ xdr->p = head->iov_base + head->iov_len;
+ xdr->end = head->iov_base + PAGE_SIZE - rqstp->rq_auth_slack;
+ /* Tail and page_len should be zero at this point: */
+ buf->len = buf->head[0].iov_len;
+ xdr->scratch.iov_len = 0;
+ xdr->page_ptr = buf->pages - 1;
+ buf->buflen = PAGE_SIZE * (1 + rqstp->rq_page_end - buf->pages)
+ - rqstp->rq_auth_slack;
+}
+
/*
* COMPOUND call.
*/
@@ -1275,24 +1282,16 @@
struct nfsd4_compound_state *cstate = &resp->cstate;
struct svc_fh *current_fh = &cstate->current_fh;
struct svc_fh *save_fh = &cstate->save_fh;
- int slack_bytes;
- u32 plen = 0;
__be32 status;
- resp->xbuf = &rqstp->rq_res;
- resp->p = rqstp->rq_res.head[0].iov_base +
- rqstp->rq_res.head[0].iov_len;
- resp->tagp = resp->p;
+ svcxdr_init_encode(rqstp, resp);
+ resp->tagp = resp->xdr.p;
/* reserve space for: taglen, tag, and opcnt */
- resp->p += 2 + XDR_QUADLEN(args->taglen);
- resp->end = rqstp->rq_res.head[0].iov_base + PAGE_SIZE;
+ xdr_reserve_space(&resp->xdr, 8 + args->taglen);
resp->taglen = args->taglen;
resp->tag = args->tag;
- resp->opcnt = 0;
resp->rqstp = rqstp;
cstate->minorversion = args->minorversion;
- cstate->replay_owner = NULL;
- cstate->session = NULL;
fh_init(current_fh, NFS4_FHSIZE);
fh_init(save_fh, NFS4_FHSIZE);
/*
@@ -1332,19 +1331,6 @@
goto encode_op;
}
- /* We must be able to encode a successful response to
- * this operation, with enough room left over to encode a
- * failed response to the next operation. If we don't
- * have enough room, fail with ERR_RESOURCE.
- */
- slack_bytes = (char *)resp->end - (char *)resp->p;
- if (slack_bytes < COMPOUND_SLACK_SPACE
- + COMPOUND_ERR_SLACK_SPACE) {
- BUG_ON(slack_bytes < COMPOUND_ERR_SLACK_SPACE);
- op->status = nfserr_resource;
- goto encode_op;
- }
-
opdesc = OPDESC(op);
if (!current_fh->fh_dentry) {
@@ -1362,9 +1348,13 @@
/* If op is non-idempotent */
if (opdesc->op_flags & OP_MODIFIES_SOMETHING) {
- plen = opdesc->op_rsize_bop(rqstp, op);
/*
- * If there's still another operation, make sure
+ * Don't execute this op if we couldn't encode a
+ * succesful reply:
+ */
+ u32 plen = opdesc->op_rsize_bop(rqstp, op);
+ /*
+ * Plus if there's another operation, make sure
* we'll have space to at least encode an error:
*/
if (resp->opcnt < args->opcnt)
@@ -1399,7 +1389,7 @@
}
if (op->status == nfserr_replay_me) {
op->replay = &cstate->replay_owner->so_replay;
- nfsd4_encode_replay(resp, op);
+ nfsd4_encode_replay(&resp->xdr, op);
status = op->status = op->replay->rp_status;
} else {
nfsd4_encode_operation(resp, op);
@@ -1438,7 +1428,8 @@
#define op_encode_change_info_maxsz (5)
#define nfs4_fattr_bitmap_maxsz (4)
-#define op_encode_lockowner_maxsz (1 + XDR_QUADLEN(IDMAP_NAMESZ))
+/* We'll fall back on returning no lockowner if run out of space: */
+#define op_encode_lockowner_maxsz (0)
#define op_encode_lock_denied_maxsz (8 + op_encode_lockowner_maxsz)
#define nfs4_owner_maxsz (1 + XDR_QUADLEN(IDMAP_NAMESZ))
@@ -1470,6 +1461,49 @@
+ nfs4_fattr_bitmap_maxsz) * sizeof(__be32);
}
+/*
+ * Note since this is an idempotent operation we won't insist on failing
+ * the op prematurely if the estimate is too large. We may turn off splice
+ * reads unnecessarily.
+ */
+static inline u32 nfsd4_getattr_rsize(struct svc_rqst *rqstp,
+ struct nfsd4_op *op)
+{
+ u32 *bmap = op->u.getattr.ga_bmval;
+ u32 bmap0 = bmap[0], bmap1 = bmap[1], bmap2 = bmap[2];
+ u32 ret = 0;
+
+ if (bmap0 & FATTR4_WORD0_ACL)
+ return svc_max_payload(rqstp);
+ if (bmap0 & FATTR4_WORD0_FS_LOCATIONS)
+ return svc_max_payload(rqstp);
+
+ if (bmap1 & FATTR4_WORD1_OWNER) {
+ ret += IDMAP_NAMESZ + 4;
+ bmap1 &= ~FATTR4_WORD1_OWNER;
+ }
+ if (bmap1 & FATTR4_WORD1_OWNER_GROUP) {
+ ret += IDMAP_NAMESZ + 4;
+ bmap1 &= ~FATTR4_WORD1_OWNER_GROUP;
+ }
+ if (bmap0 & FATTR4_WORD0_FILEHANDLE) {
+ ret += NFS4_FHSIZE + 4;
+ bmap0 &= ~FATTR4_WORD0_FILEHANDLE;
+ }
+ if (bmap2 & FATTR4_WORD2_SECURITY_LABEL) {
+ ret += NFSD4_MAX_SEC_LABEL_LEN + 12;
+ bmap2 &= ~FATTR4_WORD2_SECURITY_LABEL;
+ }
+ /*
+ * Largest of remaining attributes are 16 bytes (e.g.,
+ * supported_attributes)
+ */
+ ret += 16 * (hweight32(bmap0) + hweight32(bmap1) + hweight32(bmap2));
+ /* bitmask, length */
+ ret += 20;
+ return ret;
+}
+
static inline u32 nfsd4_link_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
{
return (op_encode_hdr_size + op_encode_change_info_maxsz)
@@ -1500,18 +1534,19 @@
if (rlen > maxcount)
rlen = maxcount;
- return (op_encode_hdr_size + 2) * sizeof(__be32) + rlen;
+ return (op_encode_hdr_size + 2 + XDR_QUADLEN(rlen)) * sizeof(__be32);
}
static inline u32 nfsd4_readdir_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
{
+ u32 maxcount = svc_max_payload(rqstp);
u32 rlen = op->u.readdir.rd_maxcount;
- if (rlen > PAGE_SIZE)
- rlen = PAGE_SIZE;
+ if (rlen > maxcount)
+ rlen = maxcount;
- return (op_encode_hdr_size + op_encode_verifier_maxsz)
- * sizeof(__be32) + rlen;
+ return (op_encode_hdr_size + op_encode_verifier_maxsz +
+ XDR_QUADLEN(rlen)) * sizeof(__be32);
}
static inline u32 nfsd4_remove_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
@@ -1526,6 +1561,12 @@
+ op_encode_change_info_maxsz) * sizeof(__be32);
}
+static inline u32 nfsd4_sequence_rsize(struct svc_rqst *rqstp,
+ struct nfsd4_op *op)
+{
+ return NFS4_MAX_SESSIONID_LEN + 20;
+}
+
static inline u32 nfsd4_setattr_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
{
return (op_encode_hdr_size + nfs4_fattr_bitmap_maxsz) * sizeof(__be32);
@@ -1539,7 +1580,7 @@
static inline u32 nfsd4_write_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
{
- return (op_encode_hdr_size + op_encode_verifier_maxsz) * sizeof(__be32);
+ return (op_encode_hdr_size + 2 + op_encode_verifier_maxsz) * sizeof(__be32);
}
static inline u32 nfsd4_exchange_id_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
@@ -1607,6 +1648,7 @@
[OP_GETATTR] = {
.op_func = (nfsd4op_func)nfsd4_getattr,
.op_flags = ALLOWED_ON_ABSENT_FS,
+ .op_rsize_bop = nfsd4_getattr_rsize,
.op_name = "OP_GETATTR",
},
[OP_GETFH] = {
@@ -1676,37 +1718,32 @@
[OP_PUTFH] = {
.op_func = (nfsd4op_func)nfsd4_putfh,
.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_ON_ABSENT_FS
- | OP_IS_PUTFH_LIKE | OP_MODIFIES_SOMETHING
- | OP_CLEAR_STATEID,
+ | OP_IS_PUTFH_LIKE | OP_CLEAR_STATEID,
.op_name = "OP_PUTFH",
.op_rsize_bop = (nfsd4op_rsize)nfsd4_only_status_rsize,
},
[OP_PUTPUBFH] = {
.op_func = (nfsd4op_func)nfsd4_putrootfh,
.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_ON_ABSENT_FS
- | OP_IS_PUTFH_LIKE | OP_MODIFIES_SOMETHING
- | OP_CLEAR_STATEID,
+ | OP_IS_PUTFH_LIKE | OP_CLEAR_STATEID,
.op_name = "OP_PUTPUBFH",
.op_rsize_bop = (nfsd4op_rsize)nfsd4_only_status_rsize,
},
[OP_PUTROOTFH] = {
.op_func = (nfsd4op_func)nfsd4_putrootfh,
.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_ON_ABSENT_FS
- | OP_IS_PUTFH_LIKE | OP_MODIFIES_SOMETHING
- | OP_CLEAR_STATEID,
+ | OP_IS_PUTFH_LIKE | OP_CLEAR_STATEID,
.op_name = "OP_PUTROOTFH",
.op_rsize_bop = (nfsd4op_rsize)nfsd4_only_status_rsize,
},
[OP_READ] = {
.op_func = (nfsd4op_func)nfsd4_read,
- .op_flags = OP_MODIFIES_SOMETHING,
.op_name = "OP_READ",
.op_rsize_bop = (nfsd4op_rsize)nfsd4_read_rsize,
.op_get_currentstateid = (stateid_getter)nfsd4_get_readstateid,
},
[OP_READDIR] = {
.op_func = (nfsd4op_func)nfsd4_readdir,
- .op_flags = OP_MODIFIES_SOMETHING,
.op_name = "OP_READDIR",
.op_rsize_bop = (nfsd4op_rsize)nfsd4_readdir_rsize,
},
@@ -1864,14 +1901,33 @@
},
};
-#ifdef NFSD_DEBUG
+int nfsd4_max_reply(struct svc_rqst *rqstp, struct nfsd4_op *op)
+{
+ struct nfsd4_operation *opdesc;
+ nfsd4op_rsize estimator;
+
+ if (op->opnum == OP_ILLEGAL)
+ return op_encode_hdr_size * sizeof(__be32);
+ opdesc = OPDESC(op);
+ estimator = opdesc->op_rsize_bop;
+ return estimator ? estimator(rqstp, op) : PAGE_SIZE;
+}
+
+void warn_on_nonidempotent_op(struct nfsd4_op *op)
+{
+ if (OPDESC(op)->op_flags & OP_MODIFIES_SOMETHING) {
+ pr_err("unable to encode reply to nonidempotent op %d (%s)\n",
+ op->opnum, nfsd4_op_name(op->opnum));
+ WARN_ON_ONCE(1);
+ }
+}
+
static const char *nfsd4_op_name(unsigned opnum)
{
if (opnum < ARRAY_SIZE(nfsd4_ops))
return nfsd4_ops[opnum].op_name;
return "unknown_operation";
}
-#endif
#define nfsd4_voidres nfsd4_voidargs
struct nfsd4_voidargs { int dummy; };
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 9a77a5a..c0d45ce 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -81,13 +81,13 @@
* effort to decrease the scope of the client_mutex, this spinlock may
* eventually cover more:
*/
-static DEFINE_SPINLOCK(recall_lock);
+static DEFINE_SPINLOCK(state_lock);
-static struct kmem_cache *openowner_slab = NULL;
-static struct kmem_cache *lockowner_slab = NULL;
-static struct kmem_cache *file_slab = NULL;
-static struct kmem_cache *stateid_slab = NULL;
-static struct kmem_cache *deleg_slab = NULL;
+static struct kmem_cache *openowner_slab;
+static struct kmem_cache *lockowner_slab;
+static struct kmem_cache *file_slab;
+static struct kmem_cache *stateid_slab;
+static struct kmem_cache *deleg_slab;
void
nfs4_lock_state(void)
@@ -235,9 +235,9 @@
static inline void
put_nfs4_file(struct nfs4_file *fi)
{
- if (atomic_dec_and_lock(&fi->fi_ref, &recall_lock)) {
+ if (atomic_dec_and_lock(&fi->fi_ref, &state_lock)) {
hlist_del(&fi->fi_hash);
- spin_unlock(&recall_lock);
+ spin_unlock(&state_lock);
iput(fi->fi_inode);
nfsd4_free_file(fi);
}
@@ -375,7 +375,6 @@
dp = delegstateid(nfs4_alloc_stid(clp, deleg_slab));
if (dp == NULL)
return dp;
- dp->dl_stid.sc_type = NFS4_DELEG_STID;
/*
* delegation seqid's are never incremented. The 4.1 special
* meaning of seqid 0 isn't meaningful, really, but let's avoid
@@ -418,6 +417,8 @@
static void nfs4_put_deleg_lease(struct nfs4_file *fp)
{
+ if (!fp->fi_lease)
+ return;
if (atomic_dec_and_test(&fp->fi_delegees)) {
vfs_setlease(fp->fi_deleg_file, F_UNLCK, &fp->fi_lease);
fp->fi_lease = NULL;
@@ -431,18 +432,30 @@
s->sc_type = 0;
}
+static void
+hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)
+{
+ lockdep_assert_held(&state_lock);
+
+ dp->dl_stid.sc_type = NFS4_DELEG_STID;
+ list_add(&dp->dl_perfile, &fp->fi_delegations);
+ list_add(&dp->dl_perclnt, &dp->dl_stid.sc_client->cl_delegations);
+}
+
/* Called under the state lock. */
static void
unhash_delegation(struct nfs4_delegation *dp)
{
+ spin_lock(&state_lock);
list_del_init(&dp->dl_perclnt);
- spin_lock(&recall_lock);
list_del_init(&dp->dl_perfile);
list_del_init(&dp->dl_recall_lru);
- spin_unlock(&recall_lock);
- nfs4_put_deleg_lease(dp->dl_file);
- put_nfs4_file(dp->dl_file);
- dp->dl_file = NULL;
+ spin_unlock(&state_lock);
+ if (dp->dl_file) {
+ nfs4_put_deleg_lease(dp->dl_file);
+ put_nfs4_file(dp->dl_file);
+ dp->dl_file = NULL;
+ }
}
@@ -645,6 +658,12 @@
}
}
+static void nfs4_free_lockowner(struct nfs4_lockowner *lo)
+{
+ kfree(lo->lo_owner.so_owner.data);
+ kmem_cache_free(lockowner_slab, lo);
+}
+
static void release_lockowner(struct nfs4_lockowner *lo)
{
unhash_lockowner(lo);
@@ -699,6 +718,12 @@
}
}
+static void nfs4_free_openowner(struct nfs4_openowner *oo)
+{
+ kfree(oo->oo_owner.so_owner.data);
+ kmem_cache_free(openowner_slab, oo);
+}
+
static void release_openowner(struct nfs4_openowner *oo)
{
unhash_openowner(oo);
@@ -1093,7 +1118,7 @@
return clp;
}
-static inline void
+static void
free_client(struct nfs4_client *clp)
{
struct nfsd_net __maybe_unused *nn = net_generic(clp->net, nfsd_net_id);
@@ -1136,13 +1161,13 @@
struct nfsd_net *nn = net_generic(clp->net, nfsd_net_id);
INIT_LIST_HEAD(&reaplist);
- spin_lock(&recall_lock);
+ spin_lock(&state_lock);
while (!list_empty(&clp->cl_delegations)) {
dp = list_entry(clp->cl_delegations.next, struct nfs4_delegation, dl_perclnt);
list_del_init(&dp->dl_perclnt);
list_move(&dp->dl_recall_lru, &reaplist);
}
- spin_unlock(&recall_lock);
+ spin_unlock(&state_lock);
while (!list_empty(&reaplist)) {
dp = list_entry(reaplist.next, struct nfs4_delegation, dl_recall_lru);
destroy_delegation(dp);
@@ -1544,6 +1569,7 @@
void
nfsd4_store_cache_entry(struct nfsd4_compoundres *resp)
{
+ struct xdr_buf *buf = resp->xdr.buf;
struct nfsd4_slot *slot = resp->cstate.slot;
unsigned int base;
@@ -1557,11 +1583,9 @@
slot->sl_datalen = 0;
return;
}
- slot->sl_datalen = (char *)resp->p - (char *)resp->cstate.datap;
- base = (char *)resp->cstate.datap -
- (char *)resp->xbuf->head[0].iov_base;
- if (read_bytes_from_xdr_buf(resp->xbuf, base, slot->sl_data,
- slot->sl_datalen))
+ base = resp->cstate.data_offset;
+ slot->sl_datalen = buf->len - base;
+ if (read_bytes_from_xdr_buf(buf, base, slot->sl_data, slot->sl_datalen))
WARN("%s: sessions DRC could not cache compound\n", __func__);
return;
}
@@ -1602,6 +1626,8 @@
struct nfsd4_sequence *seq)
{
struct nfsd4_slot *slot = resp->cstate.slot;
+ struct xdr_stream *xdr = &resp->xdr;
+ __be32 *p;
__be32 status;
dprintk("--> %s slot %p\n", __func__, slot);
@@ -1610,14 +1636,16 @@
if (status)
return status;
- /* The sequence operation has been encoded, cstate->datap set. */
- memcpy(resp->cstate.datap, slot->sl_data, slot->sl_datalen);
+ p = xdr_reserve_space(xdr, slot->sl_datalen);
+ if (!p) {
+ WARN_ON_ONCE(1);
+ return nfserr_serverfault;
+ }
+ xdr_encode_opaque_fixed(p, slot->sl_data, slot->sl_datalen);
+ xdr_commit_encode(xdr);
resp->opcnt = slot->sl_opcnt;
- resp->p = resp->cstate.datap + XDR_QUADLEN(slot->sl_datalen);
- status = slot->sl_status;
-
- return status;
+ return slot->sl_status;
}
/*
@@ -2189,11 +2217,13 @@
struct nfsd4_sequence *seq)
{
struct nfsd4_compoundres *resp = rqstp->rq_resp;
+ struct xdr_stream *xdr = &resp->xdr;
struct nfsd4_session *session;
struct nfs4_client *clp;
struct nfsd4_slot *slot;
struct nfsd4_conn *conn;
__be32 status;
+ int buflen;
struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
if (resp->opcnt != 1)
@@ -2262,6 +2292,16 @@
if (status)
goto out_put_session;
+ buflen = (seq->cachethis) ?
+ session->se_fchannel.maxresp_cached :
+ session->se_fchannel.maxresp_sz;
+ status = (seq->cachethis) ? nfserr_rep_too_big_to_cache :
+ nfserr_rep_too_big;
+ if (xdr_restrict_buflen(xdr, buflen - rqstp->rq_auth_slack))
+ goto out_put_session;
+ svc_reserve(rqstp, buflen);
+
+ status = nfs_ok;
/* Success! bump slot seqid */
slot->sl_seqid = seq->seqid;
slot->sl_flags |= NFSD4_SLOT_INUSE;
@@ -2499,28 +2539,19 @@
fp->fi_lease = NULL;
memset(fp->fi_fds, 0, sizeof(fp->fi_fds));
memset(fp->fi_access, 0, sizeof(fp->fi_access));
- spin_lock(&recall_lock);
+ spin_lock(&state_lock);
hlist_add_head(&fp->fi_hash, &file_hashtbl[hashval]);
- spin_unlock(&recall_lock);
-}
-
-static void
-nfsd4_free_slab(struct kmem_cache **slab)
-{
- if (*slab == NULL)
- return;
- kmem_cache_destroy(*slab);
- *slab = NULL;
+ spin_unlock(&state_lock);
}
void
nfsd4_free_slabs(void)
{
- nfsd4_free_slab(&openowner_slab);
- nfsd4_free_slab(&lockowner_slab);
- nfsd4_free_slab(&file_slab);
- nfsd4_free_slab(&stateid_slab);
- nfsd4_free_slab(&deleg_slab);
+ kmem_cache_destroy(openowner_slab);
+ kmem_cache_destroy(lockowner_slab);
+ kmem_cache_destroy(file_slab);
+ kmem_cache_destroy(stateid_slab);
+ kmem_cache_destroy(deleg_slab);
}
int
@@ -2529,42 +2560,38 @@
openowner_slab = kmem_cache_create("nfsd4_openowners",
sizeof(struct nfs4_openowner), 0, 0, NULL);
if (openowner_slab == NULL)
- goto out_nomem;
+ goto out;
lockowner_slab = kmem_cache_create("nfsd4_lockowners",
sizeof(struct nfs4_lockowner), 0, 0, NULL);
if (lockowner_slab == NULL)
- goto out_nomem;
+ goto out_free_openowner_slab;
file_slab = kmem_cache_create("nfsd4_files",
sizeof(struct nfs4_file), 0, 0, NULL);
if (file_slab == NULL)
- goto out_nomem;
+ goto out_free_lockowner_slab;
stateid_slab = kmem_cache_create("nfsd4_stateids",
sizeof(struct nfs4_ol_stateid), 0, 0, NULL);
if (stateid_slab == NULL)
- goto out_nomem;
+ goto out_free_file_slab;
deleg_slab = kmem_cache_create("nfsd4_delegations",
sizeof(struct nfs4_delegation), 0, 0, NULL);
if (deleg_slab == NULL)
- goto out_nomem;
+ goto out_free_stateid_slab;
return 0;
-out_nomem:
- nfsd4_free_slabs();
+
+out_free_stateid_slab:
+ kmem_cache_destroy(stateid_slab);
+out_free_file_slab:
+ kmem_cache_destroy(file_slab);
+out_free_lockowner_slab:
+ kmem_cache_destroy(lockowner_slab);
+out_free_openowner_slab:
+ kmem_cache_destroy(openowner_slab);
+out:
dprintk("nfsd4: out of memory while initializing nfsv4\n");
return -ENOMEM;
}
-void nfs4_free_openowner(struct nfs4_openowner *oo)
-{
- kfree(oo->oo_owner.so_owner.data);
- kmem_cache_free(openowner_slab, oo);
-}
-
-void nfs4_free_lockowner(struct nfs4_lockowner *lo)
-{
- kfree(lo->lo_owner.so_owner.data);
- kmem_cache_free(lockowner_slab, lo);
-}
-
static void init_nfs4_replay(struct nfs4_replay *rp)
{
rp->rp_status = nfserr_serverfault;
@@ -2685,15 +2712,15 @@
unsigned int hashval = file_hashval(ino);
struct nfs4_file *fp;
- spin_lock(&recall_lock);
+ spin_lock(&state_lock);
hlist_for_each_entry(fp, &file_hashtbl[hashval], fi_hash) {
if (fp->fi_inode == ino) {
get_nfs4_file(fp);
- spin_unlock(&recall_lock);
+ spin_unlock(&state_lock);
return fp;
}
}
- spin_unlock(&recall_lock);
+ spin_unlock(&state_lock);
return NULL;
}
@@ -2730,6 +2757,7 @@
struct nfs4_client *clp = dp->dl_stid.sc_client;
struct nfsd_net *nn = net_generic(clp->net, nfsd_net_id);
+ lockdep_assert_held(&state_lock);
/* We're assuming the state code never drops its reference
* without first removing the lease. Since we're in this lease
* callback (and since the lease code is serialized by the kernel
@@ -2766,11 +2794,11 @@
*/
fl->fl_break_time = 0;
- spin_lock(&recall_lock);
+ spin_lock(&state_lock);
fp->fi_had_conflict = true;
list_for_each_entry(dp, &fp->fi_delegations, dl_perfile)
nfsd_break_one_deleg(dp);
- spin_unlock(&recall_lock);
+ spin_unlock(&state_lock);
}
static
@@ -3047,11 +3075,12 @@
status = vfs_setlease(fl->fl_file, fl->fl_type, &fl);
if (status)
goto out_free;
- list_add(&dp->dl_perclnt, &dp->dl_stid.sc_client->cl_delegations);
fp->fi_lease = fl;
fp->fi_deleg_file = get_file(fl->fl_file);
atomic_set(&fp->fi_delegees, 1);
- list_add(&dp->dl_perfile, &fp->fi_delegations);
+ spin_lock(&state_lock);
+ hash_delegation_locked(dp, fp);
+ spin_unlock(&state_lock);
return 0;
out_free:
locks_free_lock(fl);
@@ -3060,33 +3089,21 @@
static int nfs4_set_delegation(struct nfs4_delegation *dp, struct nfs4_file *fp)
{
- int status;
-
if (fp->fi_had_conflict)
return -EAGAIN;
get_nfs4_file(fp);
dp->dl_file = fp;
- if (!fp->fi_lease) {
- status = nfs4_setlease(dp);
- if (status)
- goto out_free;
- return 0;
- }
- spin_lock(&recall_lock);
- if (fp->fi_had_conflict) {
- spin_unlock(&recall_lock);
- status = -EAGAIN;
- goto out_free;
- }
+ if (!fp->fi_lease)
+ return nfs4_setlease(dp);
+ spin_lock(&state_lock);
atomic_inc(&fp->fi_delegees);
- list_add(&dp->dl_perfile, &fp->fi_delegations);
- spin_unlock(&recall_lock);
- list_add(&dp->dl_perclnt, &dp->dl_stid.sc_client->cl_delegations);
+ if (fp->fi_had_conflict) {
+ spin_unlock(&state_lock);
+ return -EAGAIN;
+ }
+ hash_delegation_locked(dp, fp);
+ spin_unlock(&state_lock);
return 0;
-out_free:
- put_nfs4_file(fp);
- dp->dl_file = fp;
- return status;
}
static void nfsd4_open_deleg_none_ext(struct nfsd4_open *open, int status)
@@ -3173,8 +3190,7 @@
open->op_delegate_type = NFS4_OPEN_DELEGATE_READ;
return;
out_free:
- remove_stid(&dp->dl_stid);
- nfs4_put_delegation(dp);
+ destroy_delegation(dp);
out_no_deleg:
open->op_delegate_type = NFS4_OPEN_DELEGATE_NONE;
if (open->op_claim_type == NFS4_OPEN_CLAIM_PREVIOUS &&
@@ -3391,8 +3407,7 @@
struct nfs4_delegation *dp;
struct list_head *pos, *next, reaplist;
time_t cutoff = get_seconds() - nn->nfsd4_lease;
- time_t t, clientid_val = nn->nfsd4_lease;
- time_t u, test_val = nn->nfsd4_lease;
+ time_t t, new_timeo = nn->nfsd4_lease;
nfs4_lock_state();
@@ -3404,8 +3419,7 @@
clp = list_entry(pos, struct nfs4_client, cl_lru);
if (time_after((unsigned long)clp->cl_time, (unsigned long)cutoff)) {
t = clp->cl_time - cutoff;
- if (clientid_val > t)
- clientid_val = t;
+ new_timeo = min(new_timeo, t);
break;
}
if (mark_client_expired_locked(clp)) {
@@ -3422,39 +3436,35 @@
clp->cl_clientid.cl_id);
expire_client(clp);
}
- spin_lock(&recall_lock);
+ spin_lock(&state_lock);
list_for_each_safe(pos, next, &nn->del_recall_lru) {
dp = list_entry (pos, struct nfs4_delegation, dl_recall_lru);
if (net_generic(dp->dl_stid.sc_client->net, nfsd_net_id) != nn)
continue;
if (time_after((unsigned long)dp->dl_time, (unsigned long)cutoff)) {
- u = dp->dl_time - cutoff;
- if (test_val > u)
- test_val = u;
+ t = dp->dl_time - cutoff;
+ new_timeo = min(new_timeo, t);
break;
}
list_move(&dp->dl_recall_lru, &reaplist);
}
- spin_unlock(&recall_lock);
+ spin_unlock(&state_lock);
list_for_each_safe(pos, next, &reaplist) {
dp = list_entry (pos, struct nfs4_delegation, dl_recall_lru);
revoke_delegation(dp);
}
- test_val = nn->nfsd4_lease;
list_for_each_safe(pos, next, &nn->close_lru) {
oo = container_of(pos, struct nfs4_openowner, oo_close_lru);
if (time_after((unsigned long)oo->oo_time, (unsigned long)cutoff)) {
- u = oo->oo_time - cutoff;
- if (test_val > u)
- test_val = u;
+ t = oo->oo_time - cutoff;
+ new_timeo = min(new_timeo, t);
break;
}
release_openowner(oo);
}
- if (clientid_val < NFSD_LAUNDROMAT_MINTIMEOUT)
- clientid_val = NFSD_LAUNDROMAT_MINTIMEOUT;
+ new_timeo = max_t(time_t, new_timeo, NFSD_LAUNDROMAT_MINTIMEOUT);
nfs4_unlock_state();
- return clientid_val;
+ return new_timeo;
}
static struct workqueue_struct *laundry_wq;
@@ -3654,6 +3664,7 @@
struct svc_fh *current_fh = &cstate->current_fh;
struct inode *ino = current_fh->fh_dentry->d_inode;
struct nfsd_net *nn = net_generic(net, nfsd_net_id);
+ struct file *file = NULL;
__be32 status;
if (filpp)
@@ -3665,10 +3676,12 @@
if (ZERO_STATEID(stateid) || ONE_STATEID(stateid))
return check_special_stateids(net, current_fh, stateid, flags);
+ nfs4_lock_state();
+
status = nfsd4_lookup_stateid(stateid, NFS4_DELEG_STID|NFS4_OPEN_STID|NFS4_LOCK_STID,
&s, cstate->minorversion, nn);
if (status)
- return status;
+ goto out;
status = check_stateid_generation(stateid, &s->sc_stateid, nfsd4_has_session(cstate));
if (status)
goto out;
@@ -3679,8 +3692,8 @@
if (status)
goto out;
if (filpp) {
- *filpp = dp->dl_file->fi_deleg_file;
- if (!*filpp) {
+ file = dp->dl_file->fi_deleg_file;
+ if (!file) {
WARN_ON_ONCE(1);
status = nfserr_serverfault;
goto out;
@@ -3701,16 +3714,20 @@
goto out;
if (filpp) {
if (flags & RD_STATE)
- *filpp = find_readable_file(stp->st_file);
+ file = find_readable_file(stp->st_file);
else
- *filpp = find_writeable_file(stp->st_file);
+ file = find_writeable_file(stp->st_file);
}
break;
default:
- return nfserr_bad_stateid;
+ status = nfserr_bad_stateid;
+ goto out;
}
status = nfs_ok;
+ if (file)
+ *filpp = get_file(file);
out:
+ nfs4_unlock_state();
return status;
}
@@ -3726,7 +3743,7 @@
* correspondance, and we have to delete the lockowner when we
* delete the lock stateid:
*/
- unhash_lockowner(lo);
+ release_lockowner(lo);
return nfs_ok;
}
@@ -4896,6 +4913,7 @@
struct nfs4_delegation *dp, *next;
u64 count = 0;
+ lockdep_assert_held(&state_lock);
list_for_each_entry_safe(dp, next, &clp->cl_delegations, dl_perclnt) {
if (victims)
list_move(&dp->dl_recall_lru, victims);
@@ -4911,9 +4929,9 @@
LIST_HEAD(victims);
u64 count;
- spin_lock(&recall_lock);
+ spin_lock(&state_lock);
count = nfsd_find_all_delegations(clp, max, &victims);
- spin_unlock(&recall_lock);
+ spin_unlock(&state_lock);
list_for_each_entry_safe(dp, next, &victims, dl_recall_lru)
revoke_delegation(dp);
@@ -4927,11 +4945,11 @@
LIST_HEAD(victims);
u64 count;
- spin_lock(&recall_lock);
+ spin_lock(&state_lock);
count = nfsd_find_all_delegations(clp, max, &victims);
list_for_each_entry_safe(dp, next, &victims, dl_recall_lru)
nfsd_break_one_deleg(dp);
- spin_unlock(&recall_lock);
+ spin_unlock(&state_lock);
return count;
}
@@ -4940,9 +4958,9 @@
{
u64 count = 0;
- spin_lock(&recall_lock);
+ spin_lock(&state_lock);
count = nfsd_find_all_delegations(clp, max, NULL);
- spin_unlock(&recall_lock);
+ spin_unlock(&state_lock);
nfsd_print_count(clp, count, "delegations");
return count;
@@ -4983,13 +5001,6 @@
#endif /* CONFIG_NFSD_FAULT_INJECTION */
-/* initialization to perform at module load time: */
-
-void
-nfs4_state_init(void)
-{
-}
-
/*
* Since the lifetime of a delegation isn't limited to that of an open, a
* client may quite reasonably hang on to a delegation as long as it has
@@ -5160,12 +5171,12 @@
nfs4_lock_state();
INIT_LIST_HEAD(&reaplist);
- spin_lock(&recall_lock);
+ spin_lock(&state_lock);
list_for_each_safe(pos, next, &nn->del_recall_lru) {
dp = list_entry (pos, struct nfs4_delegation, dl_recall_lru);
list_move(&dp->dl_recall_lru, &reaplist);
}
- spin_unlock(&recall_lock);
+ spin_unlock(&state_lock);
list_for_each_safe(pos, next, &reaplist) {
dp = list_entry (pos, struct nfs4_delegation, dl_recall_lru);
destroy_delegation(dp);
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 18881f3..2d305a1 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -98,11 +98,6 @@
status = nfserr_bad_xdr; \
goto out
-#define READ32(x) (x) = ntohl(*p++)
-#define READ64(x) do { \
- (x) = (u64)ntohl(*p++) << 32; \
- (x) |= ntohl(*p++); \
-} while (0)
#define READMEM(x,nbytes) do { \
x = (char *)p; \
p += XDR_QUADLEN(nbytes); \
@@ -248,17 +243,17 @@
bmval[2] = 0;
READ_BUF(4);
- READ32(bmlen);
+ bmlen = be32_to_cpup(p++);
if (bmlen > 1000)
goto xdr_error;
READ_BUF(bmlen << 2);
if (bmlen > 0)
- READ32(bmval[0]);
+ bmval[0] = be32_to_cpup(p++);
if (bmlen > 1)
- READ32(bmval[1]);
+ bmval[1] = be32_to_cpup(p++);
if (bmlen > 2)
- READ32(bmval[2]);
+ bmval[2] = be32_to_cpup(p++);
DECODE_TAIL;
}
@@ -270,6 +265,7 @@
{
int expected_len, len = 0;
u32 dummy32;
+ u64 sec;
char *buf;
DECODE_HEAD;
@@ -278,12 +274,12 @@
return status;
READ_BUF(4);
- READ32(expected_len);
+ expected_len = be32_to_cpup(p++);
if (bmval[0] & FATTR4_WORD0_SIZE) {
READ_BUF(8);
len += 8;
- READ64(iattr->ia_size);
+ p = xdr_decode_hyper(p, &iattr->ia_size);
iattr->ia_valid |= ATTR_SIZE;
}
if (bmval[0] & FATTR4_WORD0_ACL) {
@@ -291,7 +287,7 @@
struct nfs4_ace *ace;
READ_BUF(4); len += 4;
- READ32(nace);
+ nace = be32_to_cpup(p++);
if (nace > NFS4_ACL_MAX)
return nfserr_fbig;
@@ -305,10 +301,10 @@
(*acl)->naces = nace;
for (ace = (*acl)->aces; ace < (*acl)->aces + nace; ace++) {
READ_BUF(16); len += 16;
- READ32(ace->type);
- READ32(ace->flag);
- READ32(ace->access_mask);
- READ32(dummy32);
+ ace->type = be32_to_cpup(p++);
+ ace->flag = be32_to_cpup(p++);
+ ace->access_mask = be32_to_cpup(p++);
+ dummy32 = be32_to_cpup(p++);
READ_BUF(dummy32);
len += XDR_QUADLEN(dummy32) << 2;
READMEM(buf, dummy32);
@@ -330,14 +326,14 @@
if (bmval[1] & FATTR4_WORD1_MODE) {
READ_BUF(4);
len += 4;
- READ32(iattr->ia_mode);
+ iattr->ia_mode = be32_to_cpup(p++);
iattr->ia_mode &= (S_IFMT | S_IALLUGO);
iattr->ia_valid |= ATTR_MODE;
}
if (bmval[1] & FATTR4_WORD1_OWNER) {
READ_BUF(4);
len += 4;
- READ32(dummy32);
+ dummy32 = be32_to_cpup(p++);
READ_BUF(dummy32);
len += (XDR_QUADLEN(dummy32) << 2);
READMEM(buf, dummy32);
@@ -348,7 +344,7 @@
if (bmval[1] & FATTR4_WORD1_OWNER_GROUP) {
READ_BUF(4);
len += 4;
- READ32(dummy32);
+ dummy32 = be32_to_cpup(p++);
READ_BUF(dummy32);
len += (XDR_QUADLEN(dummy32) << 2);
READMEM(buf, dummy32);
@@ -359,15 +355,16 @@
if (bmval[1] & FATTR4_WORD1_TIME_ACCESS_SET) {
READ_BUF(4);
len += 4;
- READ32(dummy32);
+ dummy32 = be32_to_cpup(p++);
switch (dummy32) {
case NFS4_SET_TO_CLIENT_TIME:
/* We require the high 32 bits of 'seconds' to be 0, and we ignore
all 32 bits of 'nseconds'. */
READ_BUF(12);
len += 12;
- READ64(iattr->ia_atime.tv_sec);
- READ32(iattr->ia_atime.tv_nsec);
+ p = xdr_decode_hyper(p, &sec);
+ iattr->ia_atime.tv_sec = (time_t)sec;
+ iattr->ia_atime.tv_nsec = be32_to_cpup(p++);
if (iattr->ia_atime.tv_nsec >= (u32)1000000000)
return nfserr_inval;
iattr->ia_valid |= (ATTR_ATIME | ATTR_ATIME_SET);
@@ -382,15 +379,16 @@
if (bmval[1] & FATTR4_WORD1_TIME_MODIFY_SET) {
READ_BUF(4);
len += 4;
- READ32(dummy32);
+ dummy32 = be32_to_cpup(p++);
switch (dummy32) {
case NFS4_SET_TO_CLIENT_TIME:
/* We require the high 32 bits of 'seconds' to be 0, and we ignore
all 32 bits of 'nseconds'. */
READ_BUF(12);
len += 12;
- READ64(iattr->ia_mtime.tv_sec);
- READ32(iattr->ia_mtime.tv_nsec);
+ p = xdr_decode_hyper(p, &sec);
+ iattr->ia_mtime.tv_sec = sec;
+ iattr->ia_mtime.tv_nsec = be32_to_cpup(p++);
if (iattr->ia_mtime.tv_nsec >= (u32)1000000000)
return nfserr_inval;
iattr->ia_valid |= (ATTR_MTIME | ATTR_MTIME_SET);
@@ -408,13 +406,13 @@
if (bmval[2] & FATTR4_WORD2_SECURITY_LABEL) {
READ_BUF(4);
len += 4;
- READ32(dummy32); /* lfs: we don't use it */
+ dummy32 = be32_to_cpup(p++); /* lfs: we don't use it */
READ_BUF(4);
len += 4;
- READ32(dummy32); /* pi: we don't use it either */
+ dummy32 = be32_to_cpup(p++); /* pi: we don't use it either */
READ_BUF(4);
len += 4;
- READ32(dummy32);
+ dummy32 = be32_to_cpup(p++);
READ_BUF(dummy32);
if (dummy32 > NFSD4_MAX_SEC_LABEL_LEN)
return nfserr_badlabel;
@@ -445,7 +443,7 @@
DECODE_HEAD;
READ_BUF(sizeof(stateid_t));
- READ32(sid->si_generation);
+ sid->si_generation = be32_to_cpup(p++);
COPYMEM(&sid->si_opaque, sizeof(stateid_opaque_t));
DECODE_TAIL;
@@ -457,7 +455,7 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(access->ac_req_access);
+ access->ac_req_access = be32_to_cpup(p++);
DECODE_TAIL;
}
@@ -472,7 +470,7 @@
/* callback_sec_params4 */
READ_BUF(4);
- READ32(nr_secflavs);
+ nr_secflavs = be32_to_cpup(p++);
if (nr_secflavs)
cbs->flavor = (u32)(-1);
else
@@ -480,7 +478,7 @@
cbs->flavor = 0;
for (i = 0; i < nr_secflavs; ++i) {
READ_BUF(4);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
switch (dummy) {
case RPC_AUTH_NULL:
/* Nothing to read */
@@ -490,21 +488,21 @@
case RPC_AUTH_UNIX:
READ_BUF(8);
/* stamp */
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
/* machine name */
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy);
SAVEMEM(machine_name, dummy);
/* uid, gid */
READ_BUF(8);
- READ32(uid);
- READ32(gid);
+ uid = be32_to_cpup(p++);
+ gid = be32_to_cpup(p++);
/* more gids */
READ_BUF(4);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy * 4);
if (cbs->flavor == (u32)(-1)) {
kuid_t kuid = make_kuid(&init_user_ns, uid);
@@ -524,14 +522,14 @@
"not supported!\n");
READ_BUF(8);
/* gcbp_service */
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
/* gcbp_handle_from_server */
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy);
p += XDR_QUADLEN(dummy);
/* gcbp_handle_from_client */
READ_BUF(4);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy);
break;
default:
@@ -547,7 +545,7 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(bc->bc_cb_program);
+ bc->bc_cb_program = be32_to_cpup(p++);
nfsd4_decode_cb_sec(argp, &bc->bc_cb_sec);
DECODE_TAIL;
@@ -559,7 +557,7 @@
READ_BUF(NFS4_MAX_SESSIONID_LEN + 8);
COPYMEM(bcts->sessionid.data, NFS4_MAX_SESSIONID_LEN);
- READ32(bcts->dir);
+ bcts->dir = be32_to_cpup(p++);
/* XXX: skipping ctsa_use_conn_in_rdma_mode. Perhaps Tom Tucker
* could help us figure out we should be using it. */
DECODE_TAIL;
@@ -571,7 +569,7 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(close->cl_seqid);
+ close->cl_seqid = be32_to_cpup(p++);
return nfsd4_decode_stateid(argp, &close->cl_stateid);
DECODE_TAIL;
@@ -584,8 +582,8 @@
DECODE_HEAD;
READ_BUF(12);
- READ64(commit->co_offset);
- READ32(commit->co_count);
+ p = xdr_decode_hyper(p, &commit->co_offset);
+ commit->co_count = be32_to_cpup(p++);
DECODE_TAIL;
}
@@ -596,19 +594,19 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(create->cr_type);
+ create->cr_type = be32_to_cpup(p++);
switch (create->cr_type) {
case NF4LNK:
READ_BUF(4);
- READ32(create->cr_linklen);
+ create->cr_linklen = be32_to_cpup(p++);
READ_BUF(create->cr_linklen);
SAVEMEM(create->cr_linkname, create->cr_linklen);
break;
case NF4BLK:
case NF4CHR:
READ_BUF(8);
- READ32(create->cr_specdata1);
- READ32(create->cr_specdata2);
+ create->cr_specdata1 = be32_to_cpup(p++);
+ create->cr_specdata2 = be32_to_cpup(p++);
break;
case NF4SOCK:
case NF4FIFO:
@@ -618,7 +616,7 @@
}
READ_BUF(4);
- READ32(create->cr_namelen);
+ create->cr_namelen = be32_to_cpup(p++);
READ_BUF(create->cr_namelen);
SAVEMEM(create->cr_name, create->cr_namelen);
if ((status = check_filename(create->cr_name, create->cr_namelen)))
@@ -650,7 +648,7 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(link->li_namelen);
+ link->li_namelen = be32_to_cpup(p++);
READ_BUF(link->li_namelen);
SAVEMEM(link->li_name, link->li_namelen);
if ((status = check_filename(link->li_name, link->li_namelen)))
@@ -668,24 +666,24 @@
* type, reclaim(boolean), offset, length, new_lock_owner(boolean)
*/
READ_BUF(28);
- READ32(lock->lk_type);
+ lock->lk_type = be32_to_cpup(p++);
if ((lock->lk_type < NFS4_READ_LT) || (lock->lk_type > NFS4_WRITEW_LT))
goto xdr_error;
- READ32(lock->lk_reclaim);
- READ64(lock->lk_offset);
- READ64(lock->lk_length);
- READ32(lock->lk_is_new);
+ lock->lk_reclaim = be32_to_cpup(p++);
+ p = xdr_decode_hyper(p, &lock->lk_offset);
+ p = xdr_decode_hyper(p, &lock->lk_length);
+ lock->lk_is_new = be32_to_cpup(p++);
if (lock->lk_is_new) {
READ_BUF(4);
- READ32(lock->lk_new_open_seqid);
+ lock->lk_new_open_seqid = be32_to_cpup(p++);
status = nfsd4_decode_stateid(argp, &lock->lk_new_open_stateid);
if (status)
return status;
READ_BUF(8 + sizeof(clientid_t));
- READ32(lock->lk_new_lock_seqid);
+ lock->lk_new_lock_seqid = be32_to_cpup(p++);
COPYMEM(&lock->lk_new_clientid, sizeof(clientid_t));
- READ32(lock->lk_new_owner.len);
+ lock->lk_new_owner.len = be32_to_cpup(p++);
READ_BUF(lock->lk_new_owner.len);
READMEM(lock->lk_new_owner.data, lock->lk_new_owner.len);
} else {
@@ -693,7 +691,7 @@
if (status)
return status;
READ_BUF(4);
- READ32(lock->lk_old_lock_seqid);
+ lock->lk_old_lock_seqid = be32_to_cpup(p++);
}
DECODE_TAIL;
@@ -705,13 +703,13 @@
DECODE_HEAD;
READ_BUF(32);
- READ32(lockt->lt_type);
+ lockt->lt_type = be32_to_cpup(p++);
if((lockt->lt_type < NFS4_READ_LT) || (lockt->lt_type > NFS4_WRITEW_LT))
goto xdr_error;
- READ64(lockt->lt_offset);
- READ64(lockt->lt_length);
+ p = xdr_decode_hyper(p, &lockt->lt_offset);
+ p = xdr_decode_hyper(p, &lockt->lt_length);
COPYMEM(&lockt->lt_clientid, 8);
- READ32(lockt->lt_owner.len);
+ lockt->lt_owner.len = be32_to_cpup(p++);
READ_BUF(lockt->lt_owner.len);
READMEM(lockt->lt_owner.data, lockt->lt_owner.len);
@@ -724,16 +722,16 @@
DECODE_HEAD;
READ_BUF(8);
- READ32(locku->lu_type);
+ locku->lu_type = be32_to_cpup(p++);
if ((locku->lu_type < NFS4_READ_LT) || (locku->lu_type > NFS4_WRITEW_LT))
goto xdr_error;
- READ32(locku->lu_seqid);
+ locku->lu_seqid = be32_to_cpup(p++);
status = nfsd4_decode_stateid(argp, &locku->lu_stateid);
if (status)
return status;
READ_BUF(16);
- READ64(locku->lu_offset);
- READ64(locku->lu_length);
+ p = xdr_decode_hyper(p, &locku->lu_offset);
+ p = xdr_decode_hyper(p, &locku->lu_length);
DECODE_TAIL;
}
@@ -744,7 +742,7 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(lookup->lo_len);
+ lookup->lo_len = be32_to_cpup(p++);
READ_BUF(lookup->lo_len);
SAVEMEM(lookup->lo_name, lookup->lo_len);
if ((status = check_filename(lookup->lo_name, lookup->lo_len)))
@@ -759,7 +757,7 @@
u32 w;
READ_BUF(4);
- READ32(w);
+ w = be32_to_cpup(p++);
*share_access = w & NFS4_SHARE_ACCESS_MASK;
*deleg_want = w & NFS4_SHARE_WANT_MASK;
if (deleg_when)
@@ -811,7 +809,7 @@
__be32 *p;
READ_BUF(4);
- READ32(*x);
+ *x = be32_to_cpup(p++);
/* Note: unlinke access bits, deny bits may be zero. */
if (*x & ~NFS4_SHARE_DENY_BOTH)
return nfserr_bad_xdr;
@@ -825,7 +823,7 @@
__be32 *p;
READ_BUF(4);
- READ32(o->len);
+ o->len = be32_to_cpup(p++);
if (o->len == 0 || o->len > NFS4_OPAQUE_LIMIT)
return nfserr_bad_xdr;
@@ -850,7 +848,7 @@
open->op_xdr_error = 0;
/* seqid, share_access, share_deny, clientid, ownerlen */
READ_BUF(4);
- READ32(open->op_seqid);
+ open->op_seqid = be32_to_cpup(p++);
/* decode, yet ignore deleg_when until supported */
status = nfsd4_decode_share_access(argp, &open->op_share_access,
&open->op_deleg_want, &dummy);
@@ -865,13 +863,13 @@
if (status)
goto xdr_error;
READ_BUF(4);
- READ32(open->op_create);
+ open->op_create = be32_to_cpup(p++);
switch (open->op_create) {
case NFS4_OPEN_NOCREATE:
break;
case NFS4_OPEN_CREATE:
READ_BUF(4);
- READ32(open->op_createmode);
+ open->op_createmode = be32_to_cpup(p++);
switch (open->op_createmode) {
case NFS4_CREATE_UNCHECKED:
case NFS4_CREATE_GUARDED:
@@ -904,12 +902,12 @@
/* open_claim */
READ_BUF(4);
- READ32(open->op_claim_type);
+ open->op_claim_type = be32_to_cpup(p++);
switch (open->op_claim_type) {
case NFS4_OPEN_CLAIM_NULL:
case NFS4_OPEN_CLAIM_DELEGATE_PREV:
READ_BUF(4);
- READ32(open->op_fname.len);
+ open->op_fname.len = be32_to_cpup(p++);
READ_BUF(open->op_fname.len);
SAVEMEM(open->op_fname.data, open->op_fname.len);
if ((status = check_filename(open->op_fname.data, open->op_fname.len)))
@@ -917,14 +915,14 @@
break;
case NFS4_OPEN_CLAIM_PREVIOUS:
READ_BUF(4);
- READ32(open->op_delegate_type);
+ open->op_delegate_type = be32_to_cpup(p++);
break;
case NFS4_OPEN_CLAIM_DELEGATE_CUR:
status = nfsd4_decode_stateid(argp, &open->op_delegate_stateid);
if (status)
return status;
READ_BUF(4);
- READ32(open->op_fname.len);
+ open->op_fname.len = be32_to_cpup(p++);
READ_BUF(open->op_fname.len);
SAVEMEM(open->op_fname.data, open->op_fname.len);
if ((status = check_filename(open->op_fname.data, open->op_fname.len)))
@@ -962,7 +960,7 @@
if (status)
return status;
READ_BUF(4);
- READ32(open_conf->oc_seqid);
+ open_conf->oc_seqid = be32_to_cpup(p++);
DECODE_TAIL;
}
@@ -976,7 +974,7 @@
if (status)
return status;
READ_BUF(4);
- READ32(open_down->od_seqid);
+ open_down->od_seqid = be32_to_cpup(p++);
status = nfsd4_decode_share_access(argp, &open_down->od_share_access,
&open_down->od_deleg_want, NULL);
if (status)
@@ -993,7 +991,7 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(putfh->pf_fhlen);
+ putfh->pf_fhlen = be32_to_cpup(p++);
if (putfh->pf_fhlen > NFS4_FHSIZE)
goto xdr_error;
READ_BUF(putfh->pf_fhlen);
@@ -1019,8 +1017,8 @@
if (status)
return status;
READ_BUF(12);
- READ64(read->rd_offset);
- READ32(read->rd_length);
+ p = xdr_decode_hyper(p, &read->rd_offset);
+ read->rd_length = be32_to_cpup(p++);
DECODE_TAIL;
}
@@ -1031,10 +1029,10 @@
DECODE_HEAD;
READ_BUF(24);
- READ64(readdir->rd_cookie);
+ p = xdr_decode_hyper(p, &readdir->rd_cookie);
COPYMEM(readdir->rd_verf.data, sizeof(readdir->rd_verf.data));
- READ32(readdir->rd_dircount); /* just in case you needed a useless field... */
- READ32(readdir->rd_maxcount);
+ readdir->rd_dircount = be32_to_cpup(p++);
+ readdir->rd_maxcount = be32_to_cpup(p++);
if ((status = nfsd4_decode_bitmap(argp, readdir->rd_bmval)))
goto out;
@@ -1047,7 +1045,7 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(remove->rm_namelen);
+ remove->rm_namelen = be32_to_cpup(p++);
READ_BUF(remove->rm_namelen);
SAVEMEM(remove->rm_name, remove->rm_namelen);
if ((status = check_filename(remove->rm_name, remove->rm_namelen)))
@@ -1062,10 +1060,10 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(rename->rn_snamelen);
+ rename->rn_snamelen = be32_to_cpup(p++);
READ_BUF(rename->rn_snamelen + 4);
SAVEMEM(rename->rn_sname, rename->rn_snamelen);
- READ32(rename->rn_tnamelen);
+ rename->rn_tnamelen = be32_to_cpup(p++);
READ_BUF(rename->rn_tnamelen);
SAVEMEM(rename->rn_tname, rename->rn_tnamelen);
if ((status = check_filename(rename->rn_sname, rename->rn_snamelen)))
@@ -1097,7 +1095,7 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(secinfo->si_namelen);
+ secinfo->si_namelen = be32_to_cpup(p++);
READ_BUF(secinfo->si_namelen);
SAVEMEM(secinfo->si_name, secinfo->si_namelen);
status = check_filename(secinfo->si_name, secinfo->si_namelen);
@@ -1113,7 +1111,7 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(sin->sin_style);
+ sin->sin_style = be32_to_cpup(p++);
DECODE_TAIL;
}
@@ -1144,16 +1142,16 @@
if (status)
return nfserr_bad_xdr;
READ_BUF(8);
- READ32(setclientid->se_callback_prog);
- READ32(setclientid->se_callback_netid_len);
+ setclientid->se_callback_prog = be32_to_cpup(p++);
+ setclientid->se_callback_netid_len = be32_to_cpup(p++);
READ_BUF(setclientid->se_callback_netid_len + 4);
SAVEMEM(setclientid->se_callback_netid_val, setclientid->se_callback_netid_len);
- READ32(setclientid->se_callback_addr_len);
+ setclientid->se_callback_addr_len = be32_to_cpup(p++);
READ_BUF(setclientid->se_callback_addr_len + 4);
SAVEMEM(setclientid->se_callback_addr_val, setclientid->se_callback_addr_len);
- READ32(setclientid->se_callback_ident);
+ setclientid->se_callback_ident = be32_to_cpup(p++);
DECODE_TAIL;
}
@@ -1186,7 +1184,7 @@
* nfsd4_proc_verify */
READ_BUF(4);
- READ32(verify->ve_attrlen);
+ verify->ve_attrlen = be32_to_cpup(p++);
READ_BUF(verify->ve_attrlen);
SAVEMEM(verify->ve_attrval, verify->ve_attrlen);
@@ -1204,11 +1202,11 @@
if (status)
return status;
READ_BUF(16);
- READ64(write->wr_offset);
- READ32(write->wr_stable_how);
+ p = xdr_decode_hyper(p, &write->wr_offset);
+ write->wr_stable_how = be32_to_cpup(p++);
if (write->wr_stable_how > 2)
goto xdr_error;
- READ32(write->wr_buflen);
+ write->wr_buflen = be32_to_cpup(p++);
/* Sorry .. no magic macros for this.. *
* READ_BUF(write->wr_buflen);
@@ -1254,7 +1252,7 @@
READ_BUF(12);
COPYMEM(&rlockowner->rl_clientid, sizeof(clientid_t));
- READ32(rlockowner->rl_owner.len);
+ rlockowner->rl_owner.len = be32_to_cpup(p++);
READ_BUF(rlockowner->rl_owner.len);
READMEM(rlockowner->rl_owner.data, rlockowner->rl_owner.len);
@@ -1278,63 +1276,63 @@
return nfserr_bad_xdr;
READ_BUF(4);
- READ32(exid->flags);
+ exid->flags = be32_to_cpup(p++);
/* Ignore state_protect4_a */
READ_BUF(4);
- READ32(exid->spa_how);
+ exid->spa_how = be32_to_cpup(p++);
switch (exid->spa_how) {
case SP4_NONE:
break;
case SP4_MACH_CRED:
/* spo_must_enforce */
READ_BUF(4);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy * 4);
p += dummy;
/* spo_must_allow */
READ_BUF(4);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy * 4);
p += dummy;
break;
case SP4_SSV:
/* ssp_ops */
READ_BUF(4);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy * 4);
p += dummy;
READ_BUF(4);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy * 4);
p += dummy;
/* ssp_hash_algs<> */
READ_BUF(4);
- READ32(tmp);
+ tmp = be32_to_cpup(p++);
while (tmp--) {
READ_BUF(4);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy);
p += XDR_QUADLEN(dummy);
}
/* ssp_encr_algs<> */
READ_BUF(4);
- READ32(tmp);
+ tmp = be32_to_cpup(p++);
while (tmp--) {
READ_BUF(4);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy);
p += XDR_QUADLEN(dummy);
}
/* ssp_window and ssp_num_gss_handles */
READ_BUF(8);
- READ32(dummy);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
+ dummy = be32_to_cpup(p++);
break;
default:
goto xdr_error;
@@ -1342,7 +1340,7 @@
/* Ignore Implementation ID */
READ_BUF(4); /* nfs_impl_id4 array length */
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
if (dummy > 1)
goto xdr_error;
@@ -1350,13 +1348,13 @@
if (dummy == 1) {
/* nii_domain */
READ_BUF(4);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy);
p += XDR_QUADLEN(dummy);
/* nii_name */
READ_BUF(4);
- READ32(dummy);
+ dummy = be32_to_cpup(p++);
READ_BUF(dummy);
p += XDR_QUADLEN(dummy);
@@ -1376,21 +1374,21 @@
READ_BUF(16);
COPYMEM(&sess->clientid, 8);
- READ32(sess->seqid);
- READ32(sess->flags);
+ sess->seqid = be32_to_cpup(p++);
+ sess->flags = be32_to_cpup(p++);
/* Fore channel attrs */
READ_BUF(28);
- READ32(dummy); /* headerpadsz is always 0 */
- READ32(sess->fore_channel.maxreq_sz);
- READ32(sess->fore_channel.maxresp_sz);
- READ32(sess->fore_channel.maxresp_cached);
- READ32(sess->fore_channel.maxops);
- READ32(sess->fore_channel.maxreqs);
- READ32(sess->fore_channel.nr_rdma_attrs);
+ dummy = be32_to_cpup(p++); /* headerpadsz is always 0 */
+ sess->fore_channel.maxreq_sz = be32_to_cpup(p++);
+ sess->fore_channel.maxresp_sz = be32_to_cpup(p++);
+ sess->fore_channel.maxresp_cached = be32_to_cpup(p++);
+ sess->fore_channel.maxops = be32_to_cpup(p++);
+ sess->fore_channel.maxreqs = be32_to_cpup(p++);
+ sess->fore_channel.nr_rdma_attrs = be32_to_cpup(p++);
if (sess->fore_channel.nr_rdma_attrs == 1) {
READ_BUF(4);
- READ32(sess->fore_channel.rdma_attrs);
+ sess->fore_channel.rdma_attrs = be32_to_cpup(p++);
} else if (sess->fore_channel.nr_rdma_attrs > 1) {
dprintk("Too many fore channel attr bitmaps!\n");
goto xdr_error;
@@ -1398,23 +1396,23 @@
/* Back channel attrs */
READ_BUF(28);
- READ32(dummy); /* headerpadsz is always 0 */
- READ32(sess->back_channel.maxreq_sz);
- READ32(sess->back_channel.maxresp_sz);
- READ32(sess->back_channel.maxresp_cached);
- READ32(sess->back_channel.maxops);
- READ32(sess->back_channel.maxreqs);
- READ32(sess->back_channel.nr_rdma_attrs);
+ dummy = be32_to_cpup(p++); /* headerpadsz is always 0 */
+ sess->back_channel.maxreq_sz = be32_to_cpup(p++);
+ sess->back_channel.maxresp_sz = be32_to_cpup(p++);
+ sess->back_channel.maxresp_cached = be32_to_cpup(p++);
+ sess->back_channel.maxops = be32_to_cpup(p++);
+ sess->back_channel.maxreqs = be32_to_cpup(p++);
+ sess->back_channel.nr_rdma_attrs = be32_to_cpup(p++);
if (sess->back_channel.nr_rdma_attrs == 1) {
READ_BUF(4);
- READ32(sess->back_channel.rdma_attrs);
+ sess->back_channel.rdma_attrs = be32_to_cpup(p++);
} else if (sess->back_channel.nr_rdma_attrs > 1) {
dprintk("Too many back channel attr bitmaps!\n");
goto xdr_error;
}
READ_BUF(4);
- READ32(sess->callback_prog);
+ sess->callback_prog = be32_to_cpup(p++);
nfsd4_decode_cb_sec(argp, &sess->cb_sec);
DECODE_TAIL;
}
@@ -1437,7 +1435,7 @@
DECODE_HEAD;
READ_BUF(sizeof(stateid_t));
- READ32(free_stateid->fr_stateid.si_generation);
+ free_stateid->fr_stateid.si_generation = be32_to_cpup(p++);
COPYMEM(&free_stateid->fr_stateid.si_opaque, sizeof(stateid_opaque_t));
DECODE_TAIL;
@@ -1451,10 +1449,10 @@
READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
COPYMEM(seq->sessionid.data, NFS4_MAX_SESSIONID_LEN);
- READ32(seq->seqid);
- READ32(seq->slotid);
- READ32(seq->maxslots);
- READ32(seq->cachethis);
+ seq->seqid = be32_to_cpup(p++);
+ seq->slotid = be32_to_cpup(p++);
+ seq->maxslots = be32_to_cpup(p++);
+ seq->cachethis = be32_to_cpup(p++);
DECODE_TAIL;
}
@@ -1511,7 +1509,7 @@
DECODE_HEAD;
READ_BUF(4);
- READ32(rc->rca_one_fs);
+ rc->rca_one_fs = be32_to_cpup(p++);
DECODE_TAIL;
}
@@ -1605,47 +1603,25 @@
return true;
}
-/*
- * Return a rough estimate of the maximum possible reply size. Note the
- * estimate includes rpc headers so is meant to be passed to
- * svc_reserve, not svc_reserve_auth.
- *
- * Also note the current compound encoding permits only one operation to
- * use pages beyond the first one, so the maximum possible length is the
- * maximum over these values, not the sum.
- */
-static int nfsd4_max_reply(u32 opnum)
-{
- switch (opnum) {
- case OP_READLINK:
- case OP_READDIR:
- /*
- * Both of these ops take a single page for data and put
- * the head and tail in another page:
- */
- return 2 * PAGE_SIZE;
- case OP_READ:
- return INT_MAX;
- default:
- return PAGE_SIZE;
- }
-}
-
static __be32
nfsd4_decode_compound(struct nfsd4_compoundargs *argp)
{
DECODE_HEAD;
struct nfsd4_op *op;
bool cachethis = false;
- int max_reply = PAGE_SIZE;
+ int auth_slack= argp->rqstp->rq_auth_slack;
+ int max_reply = auth_slack + 8; /* opcnt, status */
+ int readcount = 0;
+ int readbytes = 0;
int i;
READ_BUF(4);
- READ32(argp->taglen);
+ argp->taglen = be32_to_cpup(p++);
READ_BUF(argp->taglen + 8);
SAVEMEM(argp->tag, argp->taglen);
- READ32(argp->minorversion);
- READ32(argp->opcnt);
+ argp->minorversion = be32_to_cpup(p++);
+ argp->opcnt = be32_to_cpup(p++);
+ max_reply += 4 + (XDR_QUADLEN(argp->taglen) << 2);
if (argp->taglen > NFSD4_MAX_TAGLEN)
goto xdr_error;
@@ -1669,7 +1645,7 @@
op->replay = NULL;
READ_BUF(4);
- READ32(op->opnum);
+ op->opnum = be32_to_cpup(p++);
if (nfsd4_opnum_in_range(argp, op))
op->status = nfsd4_dec_ops[op->opnum](argp, &op->u);
@@ -1677,97 +1653,82 @@
op->opnum = OP_ILLEGAL;
op->status = nfserr_op_illegal;
}
-
- if (op->status) {
- argp->opcnt = i+1;
- break;
- }
/*
* We'll try to cache the result in the DRC if any one
* op in the compound wants to be cached:
*/
cachethis |= nfsd4_cache_this_op(op);
- max_reply = max(max_reply, nfsd4_max_reply(op->opnum));
+ if (op->opnum == OP_READ) {
+ readcount++;
+ readbytes += nfsd4_max_reply(argp->rqstp, op);
+ } else
+ max_reply += nfsd4_max_reply(argp->rqstp, op);
+
+ if (op->status) {
+ argp->opcnt = i+1;
+ break;
+ }
}
/* Sessions make the DRC unnecessary: */
if (argp->minorversion)
cachethis = false;
- if (max_reply != INT_MAX)
- svc_reserve(argp->rqstp, max_reply);
+ svc_reserve(argp->rqstp, max_reply + readbytes);
argp->rqstp->rq_cachetype = cachethis ? RC_REPLBUFF : RC_NOCACHE;
+ if (readcount > 1 || max_reply > PAGE_SIZE - auth_slack)
+ argp->rqstp->rq_splice_ok = false;
+
DECODE_TAIL;
}
-#define WRITE32(n) *p++ = htonl(n)
-#define WRITE64(n) do { \
- *p++ = htonl((u32)((n) >> 32)); \
- *p++ = htonl((u32)(n)); \
-} while (0)
-#define WRITEMEM(ptr,nbytes) do { if (nbytes > 0) { \
- *(p + XDR_QUADLEN(nbytes) -1) = 0; \
- memcpy(p, ptr, nbytes); \
- p += XDR_QUADLEN(nbytes); \
-}} while (0)
-
-static void write32(__be32 **p, u32 n)
-{
- *(*p)++ = htonl(n);
-}
-
-static void write64(__be32 **p, u64 n)
-{
- write32(p, (n >> 32));
- write32(p, (u32)n);
-}
-
-static void write_change(__be32 **p, struct kstat *stat, struct inode *inode)
+static __be32 *encode_change(__be32 *p, struct kstat *stat, struct inode *inode)
{
if (IS_I_VERSION(inode)) {
- write64(p, inode->i_version);
+ p = xdr_encode_hyper(p, inode->i_version);
} else {
- write32(p, stat->ctime.tv_sec);
- write32(p, stat->ctime.tv_nsec);
+ *p++ = cpu_to_be32(stat->ctime.tv_sec);
+ *p++ = cpu_to_be32(stat->ctime.tv_nsec);
}
+ return p;
}
-static void write_cinfo(__be32 **p, struct nfsd4_change_info *c)
+static __be32 *encode_cinfo(__be32 *p, struct nfsd4_change_info *c)
{
- write32(p, c->atomic);
+ *p++ = cpu_to_be32(c->atomic);
if (c->change_supported) {
- write64(p, c->before_change);
- write64(p, c->after_change);
+ p = xdr_encode_hyper(p, c->before_change);
+ p = xdr_encode_hyper(p, c->after_change);
} else {
- write32(p, c->before_ctime_sec);
- write32(p, c->before_ctime_nsec);
- write32(p, c->after_ctime_sec);
- write32(p, c->after_ctime_nsec);
+ *p++ = cpu_to_be32(c->before_ctime_sec);
+ *p++ = cpu_to_be32(c->before_ctime_nsec);
+ *p++ = cpu_to_be32(c->after_ctime_sec);
+ *p++ = cpu_to_be32(c->after_ctime_nsec);
}
+ return p;
}
-#define RESERVE_SPACE(nbytes) do { \
- p = resp->p; \
- BUG_ON(p + XDR_QUADLEN(nbytes) > resp->end); \
-} while (0)
-#define ADJUST_ARGS() resp->p = p
-
/* Encode as an array of strings the string given with components
* separated @sep, escaped with esc_enter and esc_exit.
*/
-static __be32 nfsd4_encode_components_esc(char sep, char *components,
- __be32 **pp, int *buflen,
- char esc_enter, char esc_exit)
+static __be32 nfsd4_encode_components_esc(struct xdr_stream *xdr, char sep,
+ char *components, char esc_enter,
+ char esc_exit)
{
- __be32 *p = *pp;
- __be32 *countp = p;
+ __be32 *p;
+ __be32 pathlen;
+ int pathlen_offset;
int strlen, count=0;
char *str, *end, *next;
dprintk("nfsd4_encode_components(%s)\n", components);
- if ((*buflen -= 4) < 0)
+
+ pathlen_offset = xdr->buf->len;
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
return nfserr_resource;
- WRITE32(0); /* We will fill this in with @count later */
+ p++; /* We will fill this in with @count later */
+
end = str = components;
while (*end) {
bool found_esc = false;
@@ -1789,59 +1750,57 @@
strlen = end - str;
if (strlen) {
- if ((*buflen -= ((XDR_QUADLEN(strlen) << 2) + 4)) < 0)
+ p = xdr_reserve_space(xdr, strlen + 4);
+ if (!p)
return nfserr_resource;
- WRITE32(strlen);
- WRITEMEM(str, strlen);
+ p = xdr_encode_opaque(p, str, strlen);
count++;
}
else
end++;
str = end;
}
- *pp = p;
- p = countp;
- WRITE32(count);
+ pathlen = htonl(xdr->buf->len - pathlen_offset);
+ write_bytes_to_xdr_buf(xdr->buf, pathlen_offset, &pathlen, 4);
return 0;
}
/* Encode as an array of strings the string given with components
* separated @sep.
*/
-static __be32 nfsd4_encode_components(char sep, char *components,
- __be32 **pp, int *buflen)
+static __be32 nfsd4_encode_components(struct xdr_stream *xdr, char sep,
+ char *components)
{
- return nfsd4_encode_components_esc(sep, components, pp, buflen, 0, 0);
+ return nfsd4_encode_components_esc(xdr, sep, components, 0, 0);
}
/*
* encode a location element of a fs_locations structure
*/
-static __be32 nfsd4_encode_fs_location4(struct nfsd4_fs_location *location,
- __be32 **pp, int *buflen)
+static __be32 nfsd4_encode_fs_location4(struct xdr_stream *xdr,
+ struct nfsd4_fs_location *location)
{
__be32 status;
- __be32 *p = *pp;
- status = nfsd4_encode_components_esc(':', location->hosts, &p, buflen,
+ status = nfsd4_encode_components_esc(xdr, ':', location->hosts,
'[', ']');
if (status)
return status;
- status = nfsd4_encode_components('/', location->path, &p, buflen);
+ status = nfsd4_encode_components(xdr, '/', location->path);
if (status)
return status;
- *pp = p;
return 0;
}
/*
* Encode a path in RFC3530 'pathname4' format
*/
-static __be32 nfsd4_encode_path(const struct path *root,
- const struct path *path, __be32 **pp, int *buflen)
+static __be32 nfsd4_encode_path(struct xdr_stream *xdr,
+ const struct path *root,
+ const struct path *path)
{
struct path cur = *path;
- __be32 *p = *pp;
+ __be32 *p;
struct dentry **components = NULL;
unsigned int ncomponents = 0;
__be32 err = nfserr_jukebox;
@@ -1872,11 +1831,11 @@
components[ncomponents++] = cur.dentry;
cur.dentry = dget_parent(cur.dentry);
}
-
- *buflen -= 4;
- if (*buflen < 0)
+ err = nfserr_resource;
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_free;
- WRITE32(ncomponents);
+ *p++ = cpu_to_be32(ncomponents);
while (ncomponents) {
struct dentry *dentry = components[ncomponents - 1];
@@ -1884,20 +1843,18 @@
spin_lock(&dentry->d_lock);
len = dentry->d_name.len;
- *buflen -= 4 + (XDR_QUADLEN(len) << 2);
- if (*buflen < 0) {
+ p = xdr_reserve_space(xdr, len + 4);
+ if (!p) {
spin_unlock(&dentry->d_lock);
goto out_free;
}
- WRITE32(len);
- WRITEMEM(dentry->d_name.name, len);
+ p = xdr_encode_opaque(p, dentry->d_name.name, len);
dprintk("/%s", dentry->d_name.name);
spin_unlock(&dentry->d_lock);
dput(dentry);
ncomponents--;
}
- *pp = p;
err = 0;
out_free:
dprintk(")\n");
@@ -1908,8 +1865,8 @@
return err;
}
-static __be32 nfsd4_encode_fsloc_fsroot(struct svc_rqst *rqstp,
- const struct path *path, __be32 **pp, int *buflen)
+static __be32 nfsd4_encode_fsloc_fsroot(struct xdr_stream *xdr,
+ struct svc_rqst *rqstp, const struct path *path)
{
struct svc_export *exp_ps;
__be32 res;
@@ -1917,7 +1874,7 @@
exp_ps = rqst_find_fsidzero_export(rqstp);
if (IS_ERR(exp_ps))
return nfserrno(PTR_ERR(exp_ps));
- res = nfsd4_encode_path(&exp_ps->ex_path, path, pp, buflen);
+ res = nfsd4_encode_path(xdr, &exp_ps->ex_path, path);
exp_put(exp_ps);
return res;
}
@@ -1925,28 +1882,26 @@
/*
* encode a fs_locations structure
*/
-static __be32 nfsd4_encode_fs_locations(struct svc_rqst *rqstp,
- struct svc_export *exp,
- __be32 **pp, int *buflen)
+static __be32 nfsd4_encode_fs_locations(struct xdr_stream *xdr,
+ struct svc_rqst *rqstp, struct svc_export *exp)
{
__be32 status;
int i;
- __be32 *p = *pp;
+ __be32 *p;
struct nfsd4_fs_locations *fslocs = &exp->ex_fslocs;
- status = nfsd4_encode_fsloc_fsroot(rqstp, &exp->ex_path, &p, buflen);
+ status = nfsd4_encode_fsloc_fsroot(xdr, rqstp, &exp->ex_path);
if (status)
return status;
- if ((*buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
return nfserr_resource;
- WRITE32(fslocs->locations_count);
+ *p++ = cpu_to_be32(fslocs->locations_count);
for (i=0; i<fslocs->locations_count; i++) {
- status = nfsd4_encode_fs_location4(&fslocs->locations[i],
- &p, buflen);
+ status = nfsd4_encode_fs_location4(xdr, &fslocs->locations[i]);
if (status)
return status;
}
- *pp = p;
return 0;
}
@@ -1965,15 +1920,15 @@
}
static inline __be32
-nfsd4_encode_aclname(struct svc_rqst *rqstp, struct nfs4_ace *ace,
- __be32 **p, int *buflen)
+nfsd4_encode_aclname(struct xdr_stream *xdr, struct svc_rqst *rqstp,
+ struct nfs4_ace *ace)
{
if (ace->whotype != NFS4_ACL_WHO_NAMED)
- return nfs4_acl_write_who(ace->whotype, p, buflen);
+ return nfs4_acl_write_who(xdr, ace->whotype);
else if (ace->flag & NFS4_ACE_IDENTIFIER_GROUP)
- return nfsd4_encode_group(rqstp, ace->who_gid, p, buflen);
+ return nfsd4_encode_group(xdr, rqstp, ace->who_gid);
else
- return nfsd4_encode_user(rqstp, ace->who_uid, p, buflen);
+ return nfsd4_encode_user(xdr, rqstp, ace->who_uid);
}
#define WORD0_ABSENT_FS_ATTRS (FATTR4_WORD0_FS_LOCATIONS | FATTR4_WORD0_FSID | \
@@ -1982,31 +1937,28 @@
#ifdef CONFIG_NFSD_V4_SECURITY_LABEL
static inline __be32
-nfsd4_encode_security_label(struct svc_rqst *rqstp, void *context, int len, __be32 **pp, int *buflen)
+nfsd4_encode_security_label(struct xdr_stream *xdr, struct svc_rqst *rqstp,
+ void *context, int len)
{
- __be32 *p = *pp;
+ __be32 *p;
- if (*buflen < ((XDR_QUADLEN(len) << 2) + 4 + 4 + 4))
+ p = xdr_reserve_space(xdr, len + 4 + 4 + 4);
+ if (!p)
return nfserr_resource;
/*
* For now we use a 0 here to indicate the null translation; in
* the future we may place a call to translation code here.
*/
- if ((*buflen -= 8) < 0)
- return nfserr_resource;
-
- WRITE32(0); /* lfs */
- WRITE32(0); /* pi */
+ *p++ = cpu_to_be32(0); /* lfs */
+ *p++ = cpu_to_be32(0); /* pi */
p = xdr_encode_opaque(p, context, len);
- *buflen -= (XDR_QUADLEN(len) << 2) + 4;
-
- *pp = p;
return 0;
}
#else
static inline __be32
-nfsd4_encode_security_label(struct svc_rqst *rqstp, void *context, int len, __be32 **pp, int *buflen)
+nfsd4_encode_security_label(struct xdr_stream *xdr, struct svc_rqst *rqstp,
+ void *context, int len)
{ return 0; }
#endif
@@ -2045,12 +1997,11 @@
/*
* Note: @fhp can be NULL; in this case, we might have to compose the filehandle
* ourselves.
- *
- * countp is the buffer size in _words_
*/
-__be32
-nfsd4_encode_fattr(struct svc_fh *fhp, struct svc_export *exp,
- struct dentry *dentry, __be32 **buffer, int count, u32 *bmval,
+static __be32
+nfsd4_encode_fattr(struct xdr_stream *xdr, struct svc_fh *fhp,
+ struct svc_export *exp,
+ struct dentry *dentry, u32 *bmval,
struct svc_rqst *rqstp, int ignore_crossmnt)
{
u32 bmval0 = bmval[0];
@@ -2059,12 +2010,13 @@
struct kstat stat;
struct svc_fh *tempfh = NULL;
struct kstatfs statfs;
- int buflen = count << 2;
- __be32 *attrlenp;
+ __be32 *p;
+ int starting_len = xdr->buf->len;
+ int attrlen_offset;
+ __be32 attrlen;
u32 dummy;
u64 dummy64;
u32 rdattr_err = 0;
- __be32 *p = *buffer;
__be32 status;
int err;
int aclsupport = 0;
@@ -2095,8 +2047,8 @@
err = vfs_getattr(&path, &stat);
if (err)
goto out_nfserr;
- if ((bmval0 & (FATTR4_WORD0_FILES_FREE | FATTR4_WORD0_FILES_TOTAL |
- FATTR4_WORD0_MAXNAME)) ||
+ if ((bmval0 & (FATTR4_WORD0_FILES_AVAIL | FATTR4_WORD0_FILES_FREE |
+ FATTR4_WORD0_FILES_TOTAL | FATTR4_WORD0_MAXNAME)) ||
(bmval1 & (FATTR4_WORD1_SPACE_AVAIL | FATTR4_WORD1_SPACE_FREE |
FATTR4_WORD1_SPACE_TOTAL))) {
err = vfs_statfs(&path, &statfs);
@@ -2145,25 +2097,33 @@
#endif /* CONFIG_NFSD_V4_SECURITY_LABEL */
if (bmval2) {
- if ((buflen -= 16) < 0)
+ p = xdr_reserve_space(xdr, 16);
+ if (!p)
goto out_resource;
- WRITE32(3);
- WRITE32(bmval0);
- WRITE32(bmval1);
- WRITE32(bmval2);
+ *p++ = cpu_to_be32(3);
+ *p++ = cpu_to_be32(bmval0);
+ *p++ = cpu_to_be32(bmval1);
+ *p++ = cpu_to_be32(bmval2);
} else if (bmval1) {
- if ((buflen -= 12) < 0)
+ p = xdr_reserve_space(xdr, 12);
+ if (!p)
goto out_resource;
- WRITE32(2);
- WRITE32(bmval0);
- WRITE32(bmval1);
+ *p++ = cpu_to_be32(2);
+ *p++ = cpu_to_be32(bmval0);
+ *p++ = cpu_to_be32(bmval1);
} else {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
- WRITE32(1);
- WRITE32(bmval0);
+ *p++ = cpu_to_be32(1);
+ *p++ = cpu_to_be32(bmval0);
}
- attrlenp = p++; /* to be backfilled later */
+
+ attrlen_offset = xdr->buf->len;
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
+ goto out_resource;
+ p++; /* to be backfilled later */
if (bmval0 & FATTR4_WORD0_SUPPORTED_ATTRS) {
u32 word0 = nfsd_suppattrs0(minorversion);
@@ -2175,296 +2135,343 @@
if (!contextsupport)
word2 &= ~FATTR4_WORD2_SECURITY_LABEL;
if (!word2) {
- if ((buflen -= 12) < 0)
+ p = xdr_reserve_space(xdr, 12);
+ if (!p)
goto out_resource;
- WRITE32(2);
- WRITE32(word0);
- WRITE32(word1);
+ *p++ = cpu_to_be32(2);
+ *p++ = cpu_to_be32(word0);
+ *p++ = cpu_to_be32(word1);
} else {
- if ((buflen -= 16) < 0)
+ p = xdr_reserve_space(xdr, 16);
+ if (!p)
goto out_resource;
- WRITE32(3);
- WRITE32(word0);
- WRITE32(word1);
- WRITE32(word2);
+ *p++ = cpu_to_be32(3);
+ *p++ = cpu_to_be32(word0);
+ *p++ = cpu_to_be32(word1);
+ *p++ = cpu_to_be32(word2);
}
}
if (bmval0 & FATTR4_WORD0_TYPE) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
dummy = nfs4_file_type(stat.mode);
if (dummy == NF4BAD) {
status = nfserr_serverfault;
goto out;
}
- WRITE32(dummy);
+ *p++ = cpu_to_be32(dummy);
}
if (bmval0 & FATTR4_WORD0_FH_EXPIRE_TYPE) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
if (exp->ex_flags & NFSEXP_NOSUBTREECHECK)
- WRITE32(NFS4_FH_PERSISTENT);
+ *p++ = cpu_to_be32(NFS4_FH_PERSISTENT);
else
- WRITE32(NFS4_FH_PERSISTENT|NFS4_FH_VOL_RENAME);
+ *p++ = cpu_to_be32(NFS4_FH_PERSISTENT|
+ NFS4_FH_VOL_RENAME);
}
if (bmval0 & FATTR4_WORD0_CHANGE) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
- write_change(&p, &stat, dentry->d_inode);
+ p = encode_change(p, &stat, dentry->d_inode);
}
if (bmval0 & FATTR4_WORD0_SIZE) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
- WRITE64(stat.size);
+ p = xdr_encode_hyper(p, stat.size);
}
if (bmval0 & FATTR4_WORD0_LINK_SUPPORT) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(1);
+ *p++ = cpu_to_be32(1);
}
if (bmval0 & FATTR4_WORD0_SYMLINK_SUPPORT) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(1);
+ *p++ = cpu_to_be32(1);
}
if (bmval0 & FATTR4_WORD0_NAMED_ATTR) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(0);
+ *p++ = cpu_to_be32(0);
}
if (bmval0 & FATTR4_WORD0_FSID) {
- if ((buflen -= 16) < 0)
+ p = xdr_reserve_space(xdr, 16);
+ if (!p)
goto out_resource;
if (exp->ex_fslocs.migrated) {
- WRITE64(NFS4_REFERRAL_FSID_MAJOR);
- WRITE64(NFS4_REFERRAL_FSID_MINOR);
+ p = xdr_encode_hyper(p, NFS4_REFERRAL_FSID_MAJOR);
+ p = xdr_encode_hyper(p, NFS4_REFERRAL_FSID_MINOR);
} else switch(fsid_source(fhp)) {
case FSIDSOURCE_FSID:
- WRITE64((u64)exp->ex_fsid);
- WRITE64((u64)0);
+ p = xdr_encode_hyper(p, (u64)exp->ex_fsid);
+ p = xdr_encode_hyper(p, (u64)0);
break;
case FSIDSOURCE_DEV:
- WRITE32(0);
- WRITE32(MAJOR(stat.dev));
- WRITE32(0);
- WRITE32(MINOR(stat.dev));
+ *p++ = cpu_to_be32(0);
+ *p++ = cpu_to_be32(MAJOR(stat.dev));
+ *p++ = cpu_to_be32(0);
+ *p++ = cpu_to_be32(MINOR(stat.dev));
break;
case FSIDSOURCE_UUID:
- WRITEMEM(exp->ex_uuid, 16);
+ p = xdr_encode_opaque_fixed(p, exp->ex_uuid,
+ EX_UUID_LEN);
break;
}
}
if (bmval0 & FATTR4_WORD0_UNIQUE_HANDLES) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(0);
+ *p++ = cpu_to_be32(0);
}
if (bmval0 & FATTR4_WORD0_LEASE_TIME) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(nn->nfsd4_lease);
+ *p++ = cpu_to_be32(nn->nfsd4_lease);
}
if (bmval0 & FATTR4_WORD0_RDATTR_ERROR) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(rdattr_err);
+ *p++ = cpu_to_be32(rdattr_err);
}
if (bmval0 & FATTR4_WORD0_ACL) {
struct nfs4_ace *ace;
if (acl == NULL) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(0);
+ *p++ = cpu_to_be32(0);
goto out_acl;
}
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(acl->naces);
+ *p++ = cpu_to_be32(acl->naces);
for (ace = acl->aces; ace < acl->aces + acl->naces; ace++) {
- if ((buflen -= 4*3) < 0)
+ p = xdr_reserve_space(xdr, 4*3);
+ if (!p)
goto out_resource;
- WRITE32(ace->type);
- WRITE32(ace->flag);
- WRITE32(ace->access_mask & NFS4_ACE_MASK_ALL);
- status = nfsd4_encode_aclname(rqstp, ace, &p, &buflen);
+ *p++ = cpu_to_be32(ace->type);
+ *p++ = cpu_to_be32(ace->flag);
+ *p++ = cpu_to_be32(ace->access_mask &
+ NFS4_ACE_MASK_ALL);
+ status = nfsd4_encode_aclname(xdr, rqstp, ace);
if (status)
goto out;
}
}
out_acl:
if (bmval0 & FATTR4_WORD0_ACLSUPPORT) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(aclsupport ?
+ *p++ = cpu_to_be32(aclsupport ?
ACL4_SUPPORT_ALLOW_ACL|ACL4_SUPPORT_DENY_ACL : 0);
}
if (bmval0 & FATTR4_WORD0_CANSETTIME) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(1);
+ *p++ = cpu_to_be32(1);
}
if (bmval0 & FATTR4_WORD0_CASE_INSENSITIVE) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(0);
+ *p++ = cpu_to_be32(0);
}
if (bmval0 & FATTR4_WORD0_CASE_PRESERVING) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(1);
+ *p++ = cpu_to_be32(1);
}
if (bmval0 & FATTR4_WORD0_CHOWN_RESTRICTED) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(1);
+ *p++ = cpu_to_be32(1);
}
if (bmval0 & FATTR4_WORD0_FILEHANDLE) {
- buflen -= (XDR_QUADLEN(fhp->fh_handle.fh_size) << 2) + 4;
- if (buflen < 0)
+ p = xdr_reserve_space(xdr, fhp->fh_handle.fh_size + 4);
+ if (!p)
goto out_resource;
- WRITE32(fhp->fh_handle.fh_size);
- WRITEMEM(&fhp->fh_handle.fh_base, fhp->fh_handle.fh_size);
+ p = xdr_encode_opaque(p, &fhp->fh_handle.fh_base,
+ fhp->fh_handle.fh_size);
}
if (bmval0 & FATTR4_WORD0_FILEID) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
- WRITE64(stat.ino);
+ p = xdr_encode_hyper(p, stat.ino);
}
if (bmval0 & FATTR4_WORD0_FILES_AVAIL) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
- WRITE64((u64) statfs.f_ffree);
+ p = xdr_encode_hyper(p, (u64) statfs.f_ffree);
}
if (bmval0 & FATTR4_WORD0_FILES_FREE) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
- WRITE64((u64) statfs.f_ffree);
+ p = xdr_encode_hyper(p, (u64) statfs.f_ffree);
}
if (bmval0 & FATTR4_WORD0_FILES_TOTAL) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
- WRITE64((u64) statfs.f_files);
+ p = xdr_encode_hyper(p, (u64) statfs.f_files);
}
if (bmval0 & FATTR4_WORD0_FS_LOCATIONS) {
- status = nfsd4_encode_fs_locations(rqstp, exp, &p, &buflen);
+ status = nfsd4_encode_fs_locations(xdr, rqstp, exp);
if (status)
goto out;
}
if (bmval0 & FATTR4_WORD0_HOMOGENEOUS) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(1);
+ *p++ = cpu_to_be32(1);
}
if (bmval0 & FATTR4_WORD0_MAXFILESIZE) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
- WRITE64(exp->ex_path.mnt->mnt_sb->s_maxbytes);
+ p = xdr_encode_hyper(p, exp->ex_path.mnt->mnt_sb->s_maxbytes);
}
if (bmval0 & FATTR4_WORD0_MAXLINK) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(255);
+ *p++ = cpu_to_be32(255);
}
if (bmval0 & FATTR4_WORD0_MAXNAME) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(statfs.f_namelen);
+ *p++ = cpu_to_be32(statfs.f_namelen);
}
if (bmval0 & FATTR4_WORD0_MAXREAD) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
- WRITE64((u64) svc_max_payload(rqstp));
+ p = xdr_encode_hyper(p, (u64) svc_max_payload(rqstp));
}
if (bmval0 & FATTR4_WORD0_MAXWRITE) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
- WRITE64((u64) svc_max_payload(rqstp));
+ p = xdr_encode_hyper(p, (u64) svc_max_payload(rqstp));
}
if (bmval1 & FATTR4_WORD1_MODE) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(stat.mode & S_IALLUGO);
+ *p++ = cpu_to_be32(stat.mode & S_IALLUGO);
}
if (bmval1 & FATTR4_WORD1_NO_TRUNC) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(1);
+ *p++ = cpu_to_be32(1);
}
if (bmval1 & FATTR4_WORD1_NUMLINKS) {
- if ((buflen -= 4) < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto out_resource;
- WRITE32(stat.nlink);
+ *p++ = cpu_to_be32(stat.nlink);
}
if (bmval1 & FATTR4_WORD1_OWNER) {
- status = nfsd4_encode_user(rqstp, stat.uid, &p, &buflen);
+ status = nfsd4_encode_user(xdr, rqstp, stat.uid);
if (status)
goto out;
}
if (bmval1 & FATTR4_WORD1_OWNER_GROUP) {
- status = nfsd4_encode_group(rqstp, stat.gid, &p, &buflen);
+ status = nfsd4_encode_group(xdr, rqstp, stat.gid);
if (status)
goto out;
}
if (bmval1 & FATTR4_WORD1_RAWDEV) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
- WRITE32((u32) MAJOR(stat.rdev));
- WRITE32((u32) MINOR(stat.rdev));
+ *p++ = cpu_to_be32((u32) MAJOR(stat.rdev));
+ *p++ = cpu_to_be32((u32) MINOR(stat.rdev));
}
if (bmval1 & FATTR4_WORD1_SPACE_AVAIL) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
dummy64 = (u64)statfs.f_bavail * (u64)statfs.f_bsize;
- WRITE64(dummy64);
+ p = xdr_encode_hyper(p, dummy64);
}
if (bmval1 & FATTR4_WORD1_SPACE_FREE) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
dummy64 = (u64)statfs.f_bfree * (u64)statfs.f_bsize;
- WRITE64(dummy64);
+ p = xdr_encode_hyper(p, dummy64);
}
if (bmval1 & FATTR4_WORD1_SPACE_TOTAL) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
dummy64 = (u64)statfs.f_blocks * (u64)statfs.f_bsize;
- WRITE64(dummy64);
+ p = xdr_encode_hyper(p, dummy64);
}
if (bmval1 & FATTR4_WORD1_SPACE_USED) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
dummy64 = (u64)stat.blocks << 9;
- WRITE64(dummy64);
+ p = xdr_encode_hyper(p, dummy64);
}
if (bmval1 & FATTR4_WORD1_TIME_ACCESS) {
- if ((buflen -= 12) < 0)
+ p = xdr_reserve_space(xdr, 12);
+ if (!p)
goto out_resource;
- WRITE64((s64)stat.atime.tv_sec);
- WRITE32(stat.atime.tv_nsec);
+ p = xdr_encode_hyper(p, (s64)stat.atime.tv_sec);
+ *p++ = cpu_to_be32(stat.atime.tv_nsec);
}
if (bmval1 & FATTR4_WORD1_TIME_DELTA) {
- if ((buflen -= 12) < 0)
+ p = xdr_reserve_space(xdr, 12);
+ if (!p)
goto out_resource;
- WRITE32(0);
- WRITE32(1);
- WRITE32(0);
+ *p++ = cpu_to_be32(0);
+ *p++ = cpu_to_be32(1);
+ *p++ = cpu_to_be32(0);
}
if (bmval1 & FATTR4_WORD1_TIME_METADATA) {
- if ((buflen -= 12) < 0)
+ p = xdr_reserve_space(xdr, 12);
+ if (!p)
goto out_resource;
- WRITE64((s64)stat.ctime.tv_sec);
- WRITE32(stat.ctime.tv_nsec);
+ p = xdr_encode_hyper(p, (s64)stat.ctime.tv_sec);
+ *p++ = cpu_to_be32(stat.ctime.tv_nsec);
}
if (bmval1 & FATTR4_WORD1_TIME_MODIFY) {
- if ((buflen -= 12) < 0)
+ p = xdr_reserve_space(xdr, 12);
+ if (!p)
goto out_resource;
- WRITE64((s64)stat.mtime.tv_sec);
- WRITE32(stat.mtime.tv_nsec);
+ p = xdr_encode_hyper(p, (s64)stat.mtime.tv_sec);
+ *p++ = cpu_to_be32(stat.mtime.tv_nsec);
}
if (bmval1 & FATTR4_WORD1_MOUNTED_ON_FILEID) {
- if ((buflen -= 8) < 0)
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
goto out_resource;
/*
* Get parent's attributes if not ignoring crossmount
@@ -2473,25 +2480,26 @@
if (ignore_crossmnt == 0 &&
dentry == exp->ex_path.mnt->mnt_root)
get_parent_attributes(exp, &stat);
- WRITE64(stat.ino);
+ p = xdr_encode_hyper(p, stat.ino);
}
if (bmval2 & FATTR4_WORD2_SECURITY_LABEL) {
- status = nfsd4_encode_security_label(rqstp, context,
- contextlen, &p, &buflen);
+ status = nfsd4_encode_security_label(xdr, rqstp, context,
+ contextlen);
if (status)
goto out;
}
if (bmval2 & FATTR4_WORD2_SUPPATTR_EXCLCREAT) {
- if ((buflen -= 16) < 0)
+ p = xdr_reserve_space(xdr, 16);
+ if (!p)
goto out_resource;
- WRITE32(3);
- WRITE32(NFSD_SUPPATTR_EXCLCREAT_WORD0);
- WRITE32(NFSD_SUPPATTR_EXCLCREAT_WORD1);
- WRITE32(NFSD_SUPPATTR_EXCLCREAT_WORD2);
+ *p++ = cpu_to_be32(3);
+ *p++ = cpu_to_be32(NFSD_SUPPATTR_EXCLCREAT_WORD0);
+ *p++ = cpu_to_be32(NFSD_SUPPATTR_EXCLCREAT_WORD1);
+ *p++ = cpu_to_be32(NFSD_SUPPATTR_EXCLCREAT_WORD2);
}
- *attrlenp = htonl((char *)p - (char *)attrlenp - 4);
- *buffer = p;
+ attrlen = htonl(xdr->buf->len - attrlen_offset - 4);
+ write_bytes_to_xdr_buf(xdr->buf, attrlen_offset, &attrlen, 4);
status = nfs_ok;
out:
@@ -2504,6 +2512,8 @@
fh_put(tempfh);
kfree(tempfh);
}
+ if (status)
+ xdr_truncate_encode(xdr, starting_len);
return status;
out_nfserr:
status = nfserrno(err);
@@ -2513,6 +2523,37 @@
goto out;
}
+static void svcxdr_init_encode_from_buffer(struct xdr_stream *xdr,
+ struct xdr_buf *buf, __be32 *p, int bytes)
+{
+ xdr->scratch.iov_len = 0;
+ memset(buf, 0, sizeof(struct xdr_buf));
+ buf->head[0].iov_base = p;
+ buf->head[0].iov_len = 0;
+ buf->len = 0;
+ xdr->buf = buf;
+ xdr->iov = buf->head;
+ xdr->p = p;
+ xdr->end = (void *)p + bytes;
+ buf->buflen = bytes;
+}
+
+__be32 nfsd4_encode_fattr_to_buf(__be32 **p, int words,
+ struct svc_fh *fhp, struct svc_export *exp,
+ struct dentry *dentry, u32 *bmval,
+ struct svc_rqst *rqstp, int ignore_crossmnt)
+{
+ struct xdr_buf dummy;
+ struct xdr_stream xdr;
+ __be32 ret;
+
+ svcxdr_init_encode_from_buffer(&xdr, &dummy, *p, words << 2);
+ ret = nfsd4_encode_fattr(&xdr, fhp, exp, dentry, bmval, rqstp,
+ ignore_crossmnt);
+ *p = xdr.p;
+ return ret;
+}
+
static inline int attributes_need_mount(u32 *bmval)
{
if (bmval[0] & ~(FATTR4_WORD0_RDATTR_ERROR | FATTR4_WORD0_LEASE_TIME))
@@ -2523,8 +2564,8 @@
}
static __be32
-nfsd4_encode_dirent_fattr(struct nfsd4_readdir *cd,
- const char *name, int namlen, __be32 **p, int buflen)
+nfsd4_encode_dirent_fattr(struct xdr_stream *xdr, struct nfsd4_readdir *cd,
+ const char *name, int namlen)
{
struct svc_export *exp = cd->rd_fhp->fh_export;
struct dentry *dentry;
@@ -2576,7 +2617,7 @@
}
out_encode:
- nfserr = nfsd4_encode_fattr(NULL, exp, dentry, p, buflen, cd->rd_bmval,
+ nfserr = nfsd4_encode_fattr(xdr, NULL, exp, dentry, cd->rd_bmval,
cd->rd_rqstp, ignore_crossmnt);
out_put:
dput(dentry);
@@ -2585,9 +2626,12 @@
}
static __be32 *
-nfsd4_encode_rdattr_error(__be32 *p, int buflen, __be32 nfserr)
+nfsd4_encode_rdattr_error(struct xdr_stream *xdr, __be32 nfserr)
{
- if (buflen < 6)
+ __be32 *p;
+
+ p = xdr_reserve_space(xdr, 6);
+ if (!p)
return NULL;
*p++ = htonl(2);
*p++ = htonl(FATTR4_WORD0_RDATTR_ERROR); /* bmval0 */
@@ -2604,10 +2648,13 @@
{
struct readdir_cd *ccd = ccdv;
struct nfsd4_readdir *cd = container_of(ccd, struct nfsd4_readdir, common);
- int buflen;
- __be32 *p = cd->buffer;
- __be32 *cookiep;
+ struct xdr_stream *xdr = cd->xdr;
+ int start_offset = xdr->buf->len;
+ int cookie_offset;
+ int entry_bytes;
__be32 nfserr = nfserr_toosmall;
+ __be64 wire_offset;
+ __be32 *p;
/* In nfsv4, "." and ".." never make it onto the wire.. */
if (name && isdotent(name, namlen)) {
@@ -2615,19 +2662,24 @@
return 0;
}
- if (cd->offset)
- xdr_encode_hyper(cd->offset, (u64) offset);
+ if (cd->cookie_offset) {
+ wire_offset = cpu_to_be64(offset);
+ write_bytes_to_xdr_buf(xdr->buf, cd->cookie_offset,
+ &wire_offset, 8);
+ }
- buflen = cd->buflen - 4 - XDR_QUADLEN(namlen);
- if (buflen < 0)
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
goto fail;
-
*p++ = xdr_one; /* mark entry present */
- cookiep = p;
+ cookie_offset = xdr->buf->len;
+ p = xdr_reserve_space(xdr, 3*4 + namlen);
+ if (!p)
+ goto fail;
p = xdr_encode_hyper(p, NFS_OFFSET_MAX); /* offset of next entry */
p = xdr_encode_array(p, name, namlen); /* name length & name */
- nfserr = nfsd4_encode_dirent_fattr(cd, name, namlen, &p, buflen);
+ nfserr = nfsd4_encode_dirent_fattr(xdr, cd, name, namlen);
switch (nfserr) {
case nfs_ok:
break;
@@ -2646,59 +2698,74 @@
*/
if (!(cd->rd_bmval[0] & FATTR4_WORD0_RDATTR_ERROR))
goto fail;
- p = nfsd4_encode_rdattr_error(p, buflen, nfserr);
+ p = nfsd4_encode_rdattr_error(xdr, nfserr);
if (p == NULL) {
nfserr = nfserr_toosmall;
goto fail;
}
}
- cd->buflen -= (p - cd->buffer);
- cd->buffer = p;
- cd->offset = cookiep;
+ nfserr = nfserr_toosmall;
+ entry_bytes = xdr->buf->len - start_offset;
+ if (entry_bytes > cd->rd_maxcount)
+ goto fail;
+ cd->rd_maxcount -= entry_bytes;
+ if (!cd->rd_dircount)
+ goto fail;
+ cd->rd_dircount--;
+ cd->cookie_offset = cookie_offset;
skip_entry:
cd->common.err = nfs_ok;
return 0;
fail:
+ xdr_truncate_encode(xdr, start_offset);
cd->common.err = nfserr;
return -EINVAL;
}
-static void
-nfsd4_encode_stateid(struct nfsd4_compoundres *resp, stateid_t *sid)
+static __be32
+nfsd4_encode_stateid(struct xdr_stream *xdr, stateid_t *sid)
{
__be32 *p;
- RESERVE_SPACE(sizeof(stateid_t));
- WRITE32(sid->si_generation);
- WRITEMEM(&sid->si_opaque, sizeof(stateid_opaque_t));
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, sizeof(stateid_t));
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(sid->si_generation);
+ p = xdr_encode_opaque_fixed(p, &sid->si_opaque,
+ sizeof(stateid_opaque_t));
+ return 0;
}
static __be32
nfsd4_encode_access(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_access *access)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (!nfserr) {
- RESERVE_SPACE(8);
- WRITE32(access->ac_supported);
- WRITE32(access->ac_resp_access);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(access->ac_supported);
+ *p++ = cpu_to_be32(access->ac_resp_access);
}
return nfserr;
}
static __be32 nfsd4_encode_bind_conn_to_session(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_bind_conn_to_session *bcts)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (!nfserr) {
- RESERVE_SPACE(NFS4_MAX_SESSIONID_LEN + 8);
- WRITEMEM(bcts->sessionid.data, NFS4_MAX_SESSIONID_LEN);
- WRITE32(bcts->dir);
+ p = xdr_reserve_space(xdr, NFS4_MAX_SESSIONID_LEN + 8);
+ if (!p)
+ return nfserr_resource;
+ p = xdr_encode_opaque_fixed(p, bcts->sessionid.data,
+ NFS4_MAX_SESSIONID_LEN);
+ *p++ = cpu_to_be32(bcts->dir);
/* Sorry, we do not yet support RDMA over 4.1: */
- WRITE32(0);
- ADJUST_ARGS();
+ *p++ = cpu_to_be32(0);
}
return nfserr;
}
@@ -2706,8 +2773,10 @@
static __be32
nfsd4_encode_close(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_close *close)
{
+ struct xdr_stream *xdr = &resp->xdr;
+
if (!nfserr)
- nfsd4_encode_stateid(resp, &close->cl_stateid);
+ nfserr = nfsd4_encode_stateid(xdr, &close->cl_stateid);
return nfserr;
}
@@ -2716,12 +2785,15 @@
static __be32
nfsd4_encode_commit(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_commit *commit)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (!nfserr) {
- RESERVE_SPACE(NFS4_VERIFIER_SIZE);
- WRITEMEM(commit->co_verf.data, NFS4_VERIFIER_SIZE);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, NFS4_VERIFIER_SIZE);
+ if (!p)
+ return nfserr_resource;
+ p = xdr_encode_opaque_fixed(p, commit->co_verf.data,
+ NFS4_VERIFIER_SIZE);
}
return nfserr;
}
@@ -2729,15 +2801,17 @@
static __be32
nfsd4_encode_create(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_create *create)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (!nfserr) {
- RESERVE_SPACE(32);
- write_cinfo(&p, &create->cr_cinfo);
- WRITE32(2);
- WRITE32(create->cr_bmval[0]);
- WRITE32(create->cr_bmval[1]);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 32);
+ if (!p)
+ return nfserr_resource;
+ p = encode_cinfo(p, &create->cr_cinfo);
+ *p++ = cpu_to_be32(2);
+ *p++ = cpu_to_be32(create->cr_bmval[0]);
+ *p++ = cpu_to_be32(create->cr_bmval[1]);
}
return nfserr;
}
@@ -2746,14 +2820,13 @@
nfsd4_encode_getattr(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_getattr *getattr)
{
struct svc_fh *fhp = getattr->ga_fhp;
- int buflen;
+ struct xdr_stream *xdr = &resp->xdr;
if (nfserr)
return nfserr;
- buflen = resp->end - resp->p - (COMPOUND_ERR_SLACK_SPACE >> 2);
- nfserr = nfsd4_encode_fattr(fhp, fhp->fh_export, fhp->fh_dentry,
- &resp->p, buflen, getattr->ga_bmval,
+ nfserr = nfsd4_encode_fattr(xdr, fhp, fhp->fh_export, fhp->fh_dentry,
+ getattr->ga_bmval,
resp->rqstp, 0);
return nfserr;
}
@@ -2761,16 +2834,17 @@
static __be32
nfsd4_encode_getfh(struct nfsd4_compoundres *resp, __be32 nfserr, struct svc_fh **fhpp)
{
+ struct xdr_stream *xdr = &resp->xdr;
struct svc_fh *fhp = *fhpp;
unsigned int len;
__be32 *p;
if (!nfserr) {
len = fhp->fh_handle.fh_size;
- RESERVE_SPACE(len + 4);
- WRITE32(len);
- WRITEMEM(&fhp->fh_handle.fh_base, len);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, len + 4);
+ if (!p)
+ return nfserr_resource;
+ p = xdr_encode_opaque(p, &fhp->fh_handle.fh_base, len);
}
return nfserr;
}
@@ -2779,52 +2853,69 @@
* Including all fields other than the name, a LOCK4denied structure requires
* 8(clientid) + 4(namelen) + 8(offset) + 8(length) + 4(type) = 32 bytes.
*/
-static void
-nfsd4_encode_lock_denied(struct nfsd4_compoundres *resp, struct nfsd4_lock_denied *ld)
+static __be32
+nfsd4_encode_lock_denied(struct xdr_stream *xdr, struct nfsd4_lock_denied *ld)
{
struct xdr_netobj *conf = &ld->ld_owner;
__be32 *p;
- RESERVE_SPACE(32 + XDR_LEN(conf->len));
- WRITE64(ld->ld_start);
- WRITE64(ld->ld_length);
- WRITE32(ld->ld_type);
- if (conf->len) {
- WRITEMEM(&ld->ld_clientid, 8);
- WRITE32(conf->len);
- WRITEMEM(conf->data, conf->len);
- kfree(conf->data);
- } else { /* non - nfsv4 lock in conflict, no clientid nor owner */
- WRITE64((u64)0); /* clientid */
- WRITE32(0); /* length of owner name */
+again:
+ p = xdr_reserve_space(xdr, 32 + XDR_LEN(conf->len));
+ if (!p) {
+ /*
+ * Don't fail to return the result just because we can't
+ * return the conflicting open:
+ */
+ if (conf->len) {
+ conf->len = 0;
+ conf->data = NULL;
+ goto again;
+ }
+ return nfserr_resource;
}
- ADJUST_ARGS();
+ p = xdr_encode_hyper(p, ld->ld_start);
+ p = xdr_encode_hyper(p, ld->ld_length);
+ *p++ = cpu_to_be32(ld->ld_type);
+ if (conf->len) {
+ p = xdr_encode_opaque_fixed(p, &ld->ld_clientid, 8);
+ p = xdr_encode_opaque(p, conf->data, conf->len);
+ } else { /* non - nfsv4 lock in conflict, no clientid nor owner */
+ p = xdr_encode_hyper(p, (u64)0); /* clientid */
+ *p++ = cpu_to_be32(0); /* length of owner name */
+ }
+ return nfserr_denied;
}
static __be32
nfsd4_encode_lock(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_lock *lock)
{
- if (!nfserr)
- nfsd4_encode_stateid(resp, &lock->lk_resp_stateid);
- else if (nfserr == nfserr_denied)
- nfsd4_encode_lock_denied(resp, &lock->lk_denied);
+ struct xdr_stream *xdr = &resp->xdr;
+ if (!nfserr)
+ nfserr = nfsd4_encode_stateid(xdr, &lock->lk_resp_stateid);
+ else if (nfserr == nfserr_denied)
+ nfserr = nfsd4_encode_lock_denied(xdr, &lock->lk_denied);
+ kfree(lock->lk_denied.ld_owner.data);
return nfserr;
}
static __be32
nfsd4_encode_lockt(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_lockt *lockt)
{
+ struct xdr_stream *xdr = &resp->xdr;
+
if (nfserr == nfserr_denied)
- nfsd4_encode_lock_denied(resp, &lockt->lt_denied);
+ nfsd4_encode_lock_denied(xdr, &lockt->lt_denied);
return nfserr;
}
static __be32
nfsd4_encode_locku(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_locku *locku)
{
+ struct xdr_stream *xdr = &resp->xdr;
+
if (!nfserr)
- nfsd4_encode_stateid(resp, &locku->lu_stateid);
+ nfserr = nfsd4_encode_stateid(xdr, &locku->lu_stateid);
return nfserr;
}
@@ -2833,12 +2924,14 @@
static __be32
nfsd4_encode_link(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_link *link)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (!nfserr) {
- RESERVE_SPACE(20);
- write_cinfo(&p, &link->li_cinfo);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 20);
+ if (!p)
+ return nfserr_resource;
+ p = encode_cinfo(p, &link->li_cinfo);
}
return nfserr;
}
@@ -2847,72 +2940,86 @@
static __be32
nfsd4_encode_open(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_open *open)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (nfserr)
goto out;
- nfsd4_encode_stateid(resp, &open->op_stateid);
- RESERVE_SPACE(40);
- write_cinfo(&p, &open->op_cinfo);
- WRITE32(open->op_rflags);
- WRITE32(2);
- WRITE32(open->op_bmval[0]);
- WRITE32(open->op_bmval[1]);
- WRITE32(open->op_delegate_type);
- ADJUST_ARGS();
+ nfserr = nfsd4_encode_stateid(xdr, &open->op_stateid);
+ if (nfserr)
+ goto out;
+ p = xdr_reserve_space(xdr, 40);
+ if (!p)
+ return nfserr_resource;
+ p = encode_cinfo(p, &open->op_cinfo);
+ *p++ = cpu_to_be32(open->op_rflags);
+ *p++ = cpu_to_be32(2);
+ *p++ = cpu_to_be32(open->op_bmval[0]);
+ *p++ = cpu_to_be32(open->op_bmval[1]);
+ *p++ = cpu_to_be32(open->op_delegate_type);
switch (open->op_delegate_type) {
case NFS4_OPEN_DELEGATE_NONE:
break;
case NFS4_OPEN_DELEGATE_READ:
- nfsd4_encode_stateid(resp, &open->op_delegate_stateid);
- RESERVE_SPACE(20);
- WRITE32(open->op_recall);
+ nfserr = nfsd4_encode_stateid(xdr, &open->op_delegate_stateid);
+ if (nfserr)
+ return nfserr;
+ p = xdr_reserve_space(xdr, 20);
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(open->op_recall);
/*
* TODO: ACE's in delegations
*/
- WRITE32(NFS4_ACE_ACCESS_ALLOWED_ACE_TYPE);
- WRITE32(0);
- WRITE32(0);
- WRITE32(0); /* XXX: is NULL principal ok? */
- ADJUST_ARGS();
+ *p++ = cpu_to_be32(NFS4_ACE_ACCESS_ALLOWED_ACE_TYPE);
+ *p++ = cpu_to_be32(0);
+ *p++ = cpu_to_be32(0);
+ *p++ = cpu_to_be32(0); /* XXX: is NULL principal ok? */
break;
case NFS4_OPEN_DELEGATE_WRITE:
- nfsd4_encode_stateid(resp, &open->op_delegate_stateid);
- RESERVE_SPACE(32);
- WRITE32(0);
+ nfserr = nfsd4_encode_stateid(xdr, &open->op_delegate_stateid);
+ if (nfserr)
+ return nfserr;
+ p = xdr_reserve_space(xdr, 32);
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(0);
/*
* TODO: space_limit's in delegations
*/
- WRITE32(NFS4_LIMIT_SIZE);
- WRITE32(~(u32)0);
- WRITE32(~(u32)0);
+ *p++ = cpu_to_be32(NFS4_LIMIT_SIZE);
+ *p++ = cpu_to_be32(~(u32)0);
+ *p++ = cpu_to_be32(~(u32)0);
/*
* TODO: ACE's in delegations
*/
- WRITE32(NFS4_ACE_ACCESS_ALLOWED_ACE_TYPE);
- WRITE32(0);
- WRITE32(0);
- WRITE32(0); /* XXX: is NULL principal ok? */
- ADJUST_ARGS();
+ *p++ = cpu_to_be32(NFS4_ACE_ACCESS_ALLOWED_ACE_TYPE);
+ *p++ = cpu_to_be32(0);
+ *p++ = cpu_to_be32(0);
+ *p++ = cpu_to_be32(0); /* XXX: is NULL principal ok? */
break;
case NFS4_OPEN_DELEGATE_NONE_EXT: /* 4.1 */
switch (open->op_why_no_deleg) {
case WND4_CONTENTION:
case WND4_RESOURCE:
- RESERVE_SPACE(8);
- WRITE32(open->op_why_no_deleg);
- WRITE32(0); /* deleg signaling not supported yet */
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(open->op_why_no_deleg);
+ /* deleg signaling not supported yet: */
+ *p++ = cpu_to_be32(0);
break;
default:
- RESERVE_SPACE(4);
- WRITE32(open->op_why_no_deleg);
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(open->op_why_no_deleg);
}
- ADJUST_ARGS();
break;
default:
BUG();
@@ -2925,8 +3032,10 @@
static __be32
nfsd4_encode_open_confirm(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_open_confirm *oc)
{
+ struct xdr_stream *xdr = &resp->xdr;
+
if (!nfserr)
- nfsd4_encode_stateid(resp, &oc->oc_resp_stateid);
+ nfserr = nfsd4_encode_stateid(xdr, &oc->oc_resp_stateid);
return nfserr;
}
@@ -2934,127 +3043,233 @@
static __be32
nfsd4_encode_open_downgrade(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_open_downgrade *od)
{
+ struct xdr_stream *xdr = &resp->xdr;
+
if (!nfserr)
- nfsd4_encode_stateid(resp, &od->od_stateid);
+ nfserr = nfsd4_encode_stateid(xdr, &od->od_stateid);
return nfserr;
}
+static __be32 nfsd4_encode_splice_read(
+ struct nfsd4_compoundres *resp,
+ struct nfsd4_read *read,
+ struct file *file, unsigned long maxcount)
+{
+ struct xdr_stream *xdr = &resp->xdr;
+ struct xdr_buf *buf = xdr->buf;
+ u32 eof;
+ int space_left;
+ __be32 nfserr;
+ __be32 *p = xdr->p - 2;
+
+ /*
+ * Don't inline pages unless we know there's room for eof,
+ * count, and possible padding:
+ */
+ if (xdr->end - xdr->p < 3)
+ return nfserr_resource;
+
+ nfserr = nfsd_splice_read(read->rd_rqstp, file,
+ read->rd_offset, &maxcount);
+ if (nfserr) {
+ /*
+ * nfsd_splice_actor may have already messed with the
+ * page length; reset it so as not to confuse
+ * xdr_truncate_encode:
+ */
+ buf->page_len = 0;
+ return nfserr;
+ }
+
+ eof = (read->rd_offset + maxcount >=
+ read->rd_fhp->fh_dentry->d_inode->i_size);
+
+ *(p++) = htonl(eof);
+ *(p++) = htonl(maxcount);
+
+ buf->page_len = maxcount;
+ buf->len += maxcount;
+ xdr->page_ptr += (maxcount + PAGE_SIZE - 1) / PAGE_SIZE;
+
+ /* Use rest of head for padding and remaining ops: */
+ buf->tail[0].iov_base = xdr->p;
+ buf->tail[0].iov_len = 0;
+ xdr->iov = buf->tail;
+ if (maxcount&3) {
+ int pad = 4 - (maxcount&3);
+
+ *(xdr->p++) = 0;
+
+ buf->tail[0].iov_base += maxcount&3;
+ buf->tail[0].iov_len = pad;
+ buf->len += pad;
+ }
+
+ space_left = min_t(int, (void *)xdr->end - (void *)xdr->p,
+ buf->buflen - buf->len);
+ buf->buflen = buf->len + space_left;
+ xdr->end = (__be32 *)((void *)xdr->end + space_left);
+
+ return 0;
+}
+
+static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
+ struct nfsd4_read *read,
+ struct file *file, unsigned long maxcount)
+{
+ struct xdr_stream *xdr = &resp->xdr;
+ u32 eof;
+ int v;
+ int starting_len = xdr->buf->len - 8;
+ long len;
+ int thislen;
+ __be32 nfserr;
+ __be32 tmp;
+ __be32 *p;
+ u32 zzz = 0;
+ int pad;
+
+ len = maxcount;
+ v = 0;
+
+ thislen = (void *)xdr->end - (void *)xdr->p;
+ if (len < thislen)
+ thislen = len;
+ p = xdr_reserve_space(xdr, (thislen+3)&~3);
+ WARN_ON_ONCE(!p);
+ resp->rqstp->rq_vec[v].iov_base = p;
+ resp->rqstp->rq_vec[v].iov_len = thislen;
+ v++;
+ len -= thislen;
+
+ while (len) {
+ thislen = min_t(long, len, PAGE_SIZE);
+ p = xdr_reserve_space(xdr, (thislen+3)&~3);
+ WARN_ON_ONCE(!p);
+ resp->rqstp->rq_vec[v].iov_base = p;
+ resp->rqstp->rq_vec[v].iov_len = thislen;
+ v++;
+ len -= thislen;
+ }
+ read->rd_vlen = v;
+
+ nfserr = nfsd_readv(file, read->rd_offset, resp->rqstp->rq_vec,
+ read->rd_vlen, &maxcount);
+ if (nfserr)
+ return nfserr;
+ xdr_truncate_encode(xdr, starting_len + 8 + ((maxcount+3)&~3));
+
+ eof = (read->rd_offset + maxcount >=
+ read->rd_fhp->fh_dentry->d_inode->i_size);
+
+ tmp = htonl(eof);
+ write_bytes_to_xdr_buf(xdr->buf, starting_len , &tmp, 4);
+ tmp = htonl(maxcount);
+ write_bytes_to_xdr_buf(xdr->buf, starting_len + 4, &tmp, 4);
+
+ pad = (maxcount&3) ? 4 - (maxcount&3) : 0;
+ write_bytes_to_xdr_buf(xdr->buf, starting_len + 8 + maxcount,
+ &zzz, pad);
+ return 0;
+
+}
+
static __be32
nfsd4_encode_read(struct nfsd4_compoundres *resp, __be32 nfserr,
struct nfsd4_read *read)
{
- u32 eof;
- int v;
- struct page *page;
- unsigned long maxcount;
- long len;
+ unsigned long maxcount;
+ struct xdr_stream *xdr = &resp->xdr;
+ struct file *file = read->rd_filp;
+ int starting_len = xdr->buf->len;
+ struct raparms *ra;
__be32 *p;
+ __be32 err;
if (nfserr)
return nfserr;
- if (resp->xbuf->page_len)
- return nfserr_resource;
- RESERVE_SPACE(8); /* eof flag and byte count */
+ p = xdr_reserve_space(xdr, 8); /* eof flag and byte count */
+ if (!p) {
+ WARN_ON_ONCE(resp->rqstp->rq_splice_ok);
+ return nfserr_resource;
+ }
+ if (resp->xdr.buf->page_len && resp->rqstp->rq_splice_ok) {
+ WARN_ON_ONCE(1);
+ return nfserr_resource;
+ }
+ xdr_commit_encode(xdr);
maxcount = svc_max_payload(resp->rqstp);
+ if (maxcount > xdr->buf->buflen - xdr->buf->len)
+ maxcount = xdr->buf->buflen - xdr->buf->len;
if (maxcount > read->rd_length)
maxcount = read->rd_length;
- len = maxcount;
- v = 0;
- while (len > 0) {
- page = *(resp->rqstp->rq_next_page);
- if (!page) { /* ran out of pages */
- maxcount -= len;
- break;
- }
- resp->rqstp->rq_vec[v].iov_base = page_address(page);
- resp->rqstp->rq_vec[v].iov_len =
- len < PAGE_SIZE ? len : PAGE_SIZE;
- resp->rqstp->rq_next_page++;
- v++;
- len -= PAGE_SIZE;
+ if (!read->rd_filp) {
+ err = nfsd_get_tmp_read_open(resp->rqstp, read->rd_fhp,
+ &file, &ra);
+ if (err)
+ goto err_truncate;
}
- read->rd_vlen = v;
- nfserr = nfsd_read_file(read->rd_rqstp, read->rd_fhp, read->rd_filp,
- read->rd_offset, resp->rqstp->rq_vec, read->rd_vlen,
- &maxcount);
+ if (file->f_op->splice_read && resp->rqstp->rq_splice_ok)
+ err = nfsd4_encode_splice_read(resp, read, file, maxcount);
+ else
+ err = nfsd4_encode_readv(resp, read, file, maxcount);
- if (nfserr)
- return nfserr;
- eof = (read->rd_offset + maxcount >=
- read->rd_fhp->fh_dentry->d_inode->i_size);
+ if (!read->rd_filp)
+ nfsd_put_tmp_read_open(file, ra);
- WRITE32(eof);
- WRITE32(maxcount);
- ADJUST_ARGS();
- resp->xbuf->head[0].iov_len = (char*)p
- - (char*)resp->xbuf->head[0].iov_base;
- resp->xbuf->page_len = maxcount;
-
- /* Use rest of head for padding and remaining ops: */
- resp->xbuf->tail[0].iov_base = p;
- resp->xbuf->tail[0].iov_len = 0;
- if (maxcount&3) {
- RESERVE_SPACE(4);
- WRITE32(0);
- resp->xbuf->tail[0].iov_base += maxcount&3;
- resp->xbuf->tail[0].iov_len = 4 - (maxcount&3);
- ADJUST_ARGS();
- }
- return 0;
+err_truncate:
+ if (err)
+ xdr_truncate_encode(xdr, starting_len);
+ return err;
}
static __be32
nfsd4_encode_readlink(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_readlink *readlink)
{
int maxcount;
- char *page;
+ __be32 wire_count;
+ int zero = 0;
+ struct xdr_stream *xdr = &resp->xdr;
+ int length_offset = xdr->buf->len;
__be32 *p;
if (nfserr)
return nfserr;
- if (resp->xbuf->page_len)
- return nfserr_resource;
- if (!*resp->rqstp->rq_next_page)
- return nfserr_resource;
- page = page_address(*(resp->rqstp->rq_next_page++));
-
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
+ return nfserr_resource;
maxcount = PAGE_SIZE;
- RESERVE_SPACE(4);
+ p = xdr_reserve_space(xdr, maxcount);
+ if (!p)
+ return nfserr_resource;
/*
* XXX: By default, the ->readlink() VFS op will truncate symlinks
* if they would overflow the buffer. Is this kosher in NFSv4? If
* not, one easy fix is: if ->readlink() precisely fills the buffer,
* assume that truncation occurred, and return NFS4ERR_RESOURCE.
*/
- nfserr = nfsd_readlink(readlink->rl_rqstp, readlink->rl_fhp, page, &maxcount);
+ nfserr = nfsd_readlink(readlink->rl_rqstp, readlink->rl_fhp,
+ (char *)p, &maxcount);
if (nfserr == nfserr_isdir)
- return nfserr_inval;
- if (nfserr)
+ nfserr = nfserr_inval;
+ if (nfserr) {
+ xdr_truncate_encode(xdr, length_offset);
return nfserr;
-
- WRITE32(maxcount);
- ADJUST_ARGS();
- resp->xbuf->head[0].iov_len = (char*)p
- - (char*)resp->xbuf->head[0].iov_base;
- resp->xbuf->page_len = maxcount;
-
- /* Use rest of head for padding and remaining ops: */
- resp->xbuf->tail[0].iov_base = p;
- resp->xbuf->tail[0].iov_len = 0;
- if (maxcount&3) {
- RESERVE_SPACE(4);
- WRITE32(0);
- resp->xbuf->tail[0].iov_base += maxcount&3;
- resp->xbuf->tail[0].iov_len = 4 - (maxcount&3);
- ADJUST_ARGS();
}
+
+ wire_count = htonl(maxcount);
+ write_bytes_to_xdr_buf(xdr->buf, length_offset, &wire_count, 4);
+ xdr_truncate_encode(xdr, length_offset + 4 + maxcount);
+ if (maxcount & 3)
+ write_bytes_to_xdr_buf(xdr->buf, length_offset + 4 + maxcount,
+ &zero, 4 - (maxcount&3));
return 0;
}
@@ -3062,47 +3277,52 @@
nfsd4_encode_readdir(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_readdir *readdir)
{
int maxcount;
+ int bytes_left;
loff_t offset;
- __be32 *page, *savep, *tailbase;
+ __be64 wire_offset;
+ struct xdr_stream *xdr = &resp->xdr;
+ int starting_len = xdr->buf->len;
__be32 *p;
if (nfserr)
return nfserr;
- if (resp->xbuf->page_len)
- return nfserr_resource;
- if (!*resp->rqstp->rq_next_page)
- return nfserr_resource;
- RESERVE_SPACE(NFS4_VERIFIER_SIZE);
- savep = p;
+ p = xdr_reserve_space(xdr, NFS4_VERIFIER_SIZE);
+ if (!p)
+ return nfserr_resource;
/* XXX: Following NFSv3, we ignore the READDIR verifier for now. */
- WRITE32(0);
- WRITE32(0);
- ADJUST_ARGS();
- resp->xbuf->head[0].iov_len = ((char*)resp->p) - (char*)resp->xbuf->head[0].iov_base;
- tailbase = p;
-
- maxcount = PAGE_SIZE;
- if (maxcount > readdir->rd_maxcount)
- maxcount = readdir->rd_maxcount;
+ *p++ = cpu_to_be32(0);
+ *p++ = cpu_to_be32(0);
+ resp->xdr.buf->head[0].iov_len = ((char *)resp->xdr.p)
+ - (char *)resp->xdr.buf->head[0].iov_base;
/*
- * Convert from bytes to words, account for the two words already
- * written, make sure to leave two words at the end for the next
- * pointer and eof field.
+ * Number of bytes left for directory entries allowing for the
+ * final 8 bytes of the readdir and a following failed op:
*/
- maxcount = (maxcount >> 2) - 4;
- if (maxcount < 0) {
- nfserr = nfserr_toosmall;
+ bytes_left = xdr->buf->buflen - xdr->buf->len
+ - COMPOUND_ERR_SLACK_SPACE - 8;
+ if (bytes_left < 0) {
+ nfserr = nfserr_resource;
goto err_no_verf;
}
+ maxcount = min_t(u32, readdir->rd_maxcount, INT_MAX);
+ /*
+ * Note the rfc defines rd_maxcount as the size of the
+ * READDIR4resok structure, which includes the verifier above
+ * and the 8 bytes encoded at the end of this function:
+ */
+ if (maxcount < 16) {
+ nfserr = nfserr_toosmall;
+ goto err_no_verf;
+ }
+ maxcount = min_t(int, maxcount-16, bytes_left);
- page = page_address(*(resp->rqstp->rq_next_page++));
+ readdir->xdr = xdr;
+ readdir->rd_maxcount = maxcount;
readdir->common.err = 0;
- readdir->buflen = maxcount;
- readdir->buffer = page;
- readdir->offset = NULL;
+ readdir->cookie_offset = 0;
offset = readdir->rd_cookie;
nfserr = nfsd_readdir(readdir->rd_rqstp, readdir->rd_fhp,
@@ -3110,42 +3330,49 @@
&readdir->common, nfsd4_encode_dirent);
if (nfserr == nfs_ok &&
readdir->common.err == nfserr_toosmall &&
- readdir->buffer == page)
- nfserr = nfserr_toosmall;
+ xdr->buf->len == starting_len + 8) {
+ /* nothing encoded; which limit did we hit?: */
+ if (maxcount - 16 < bytes_left)
+ /* It was the fault of rd_maxcount: */
+ nfserr = nfserr_toosmall;
+ else
+ /* We ran out of buffer space: */
+ nfserr = nfserr_resource;
+ }
if (nfserr)
goto err_no_verf;
- if (readdir->offset)
- xdr_encode_hyper(readdir->offset, offset);
+ if (readdir->cookie_offset) {
+ wire_offset = cpu_to_be64(offset);
+ write_bytes_to_xdr_buf(xdr->buf, readdir->cookie_offset,
+ &wire_offset, 8);
+ }
- p = readdir->buffer;
+ p = xdr_reserve_space(xdr, 8);
+ if (!p) {
+ WARN_ON_ONCE(1);
+ goto err_no_verf;
+ }
*p++ = 0; /* no more entries */
*p++ = htonl(readdir->common.err == nfserr_eof);
- resp->xbuf->page_len = ((char*)p) -
- (char*)page_address(*(resp->rqstp->rq_next_page-1));
-
- /* Use rest of head for padding and remaining ops: */
- resp->xbuf->tail[0].iov_base = tailbase;
- resp->xbuf->tail[0].iov_len = 0;
- resp->p = resp->xbuf->tail[0].iov_base;
- resp->end = resp->p + (PAGE_SIZE - resp->xbuf->head[0].iov_len)/4;
return 0;
err_no_verf:
- p = savep;
- ADJUST_ARGS();
+ xdr_truncate_encode(xdr, starting_len);
return nfserr;
}
static __be32
nfsd4_encode_remove(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_remove *remove)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (!nfserr) {
- RESERVE_SPACE(20);
- write_cinfo(&p, &remove->rm_cinfo);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 20);
+ if (!p)
+ return nfserr_resource;
+ p = encode_cinfo(p, &remove->rm_cinfo);
}
return nfserr;
}
@@ -3153,19 +3380,21 @@
static __be32
nfsd4_encode_rename(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_rename *rename)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (!nfserr) {
- RESERVE_SPACE(40);
- write_cinfo(&p, &rename->rn_sinfo);
- write_cinfo(&p, &rename->rn_tinfo);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 40);
+ if (!p)
+ return nfserr_resource;
+ p = encode_cinfo(p, &rename->rn_sinfo);
+ p = encode_cinfo(p, &rename->rn_tinfo);
}
return nfserr;
}
static __be32
-nfsd4_do_encode_secinfo(struct nfsd4_compoundres *resp,
+nfsd4_do_encode_secinfo(struct xdr_stream *xdr,
__be32 nfserr, struct svc_export *exp)
{
u32 i, nflavs, supported;
@@ -3176,6 +3405,7 @@
if (nfserr)
goto out;
+ nfserr = nfserr_resource;
if (exp->ex_nflavors) {
flavs = exp->ex_flavors;
nflavs = exp->ex_nflavors;
@@ -3197,9 +3427,10 @@
}
supported = 0;
- RESERVE_SPACE(4);
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
+ goto out;
flavorsp = p++; /* to be backfilled later */
- ADJUST_ARGS();
for (i = 0; i < nflavs; i++) {
rpc_authflavor_t pf = flavs[i].pseudoflavor;
@@ -3207,18 +3438,20 @@
if (rpcauth_get_gssinfo(pf, &info) == 0) {
supported++;
- RESERVE_SPACE(4 + 4 + XDR_LEN(info.oid.len) + 4 + 4);
- WRITE32(RPC_AUTH_GSS);
- WRITE32(info.oid.len);
- WRITEMEM(info.oid.data, info.oid.len);
- WRITE32(info.qop);
- WRITE32(info.service);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 4 + 4 +
+ XDR_LEN(info.oid.len) + 4 + 4);
+ if (!p)
+ goto out;
+ *p++ = cpu_to_be32(RPC_AUTH_GSS);
+ p = xdr_encode_opaque(p, info.oid.data, info.oid.len);
+ *p++ = cpu_to_be32(info.qop);
+ *p++ = cpu_to_be32(info.service);
} else if (pf < RPC_AUTH_MAXFLAVOR) {
supported++;
- RESERVE_SPACE(4);
- WRITE32(pf);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
+ goto out;
+ *p++ = cpu_to_be32(pf);
} else {
if (report)
pr_warn("NFS: SECINFO: security flavor %u "
@@ -3229,7 +3462,7 @@
if (nflavs != supported)
report = false;
*flavorsp = htonl(supported);
-
+ nfserr = 0;
out:
if (exp)
exp_put(exp);
@@ -3240,14 +3473,18 @@
nfsd4_encode_secinfo(struct nfsd4_compoundres *resp, __be32 nfserr,
struct nfsd4_secinfo *secinfo)
{
- return nfsd4_do_encode_secinfo(resp, nfserr, secinfo->si_exp);
+ struct xdr_stream *xdr = &resp->xdr;
+
+ return nfsd4_do_encode_secinfo(xdr, nfserr, secinfo->si_exp);
}
static __be32
nfsd4_encode_secinfo_no_name(struct nfsd4_compoundres *resp, __be32 nfserr,
struct nfsd4_secinfo_no_name *secinfo)
{
- return nfsd4_do_encode_secinfo(resp, nfserr, secinfo->sin_exp);
+ struct xdr_stream *xdr = &resp->xdr;
+
+ return nfsd4_do_encode_secinfo(xdr, nfserr, secinfo->sin_exp);
}
/*
@@ -3257,41 +3494,47 @@
static __be32
nfsd4_encode_setattr(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_setattr *setattr)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
- RESERVE_SPACE(16);
+ p = xdr_reserve_space(xdr, 16);
+ if (!p)
+ return nfserr_resource;
if (nfserr) {
- WRITE32(3);
- WRITE32(0);
- WRITE32(0);
- WRITE32(0);
+ *p++ = cpu_to_be32(3);
+ *p++ = cpu_to_be32(0);
+ *p++ = cpu_to_be32(0);
+ *p++ = cpu_to_be32(0);
}
else {
- WRITE32(3);
- WRITE32(setattr->sa_bmval[0]);
- WRITE32(setattr->sa_bmval[1]);
- WRITE32(setattr->sa_bmval[2]);
+ *p++ = cpu_to_be32(3);
+ *p++ = cpu_to_be32(setattr->sa_bmval[0]);
+ *p++ = cpu_to_be32(setattr->sa_bmval[1]);
+ *p++ = cpu_to_be32(setattr->sa_bmval[2]);
}
- ADJUST_ARGS();
return nfserr;
}
static __be32
nfsd4_encode_setclientid(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_setclientid *scd)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (!nfserr) {
- RESERVE_SPACE(8 + NFS4_VERIFIER_SIZE);
- WRITEMEM(&scd->se_clientid, 8);
- WRITEMEM(&scd->se_confirm, NFS4_VERIFIER_SIZE);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 8 + NFS4_VERIFIER_SIZE);
+ if (!p)
+ return nfserr_resource;
+ p = xdr_encode_opaque_fixed(p, &scd->se_clientid, 8);
+ p = xdr_encode_opaque_fixed(p, &scd->se_confirm,
+ NFS4_VERIFIER_SIZE);
}
else if (nfserr == nfserr_clid_inuse) {
- RESERVE_SPACE(8);
- WRITE32(0);
- WRITE32(0);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 8);
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(0);
+ *p++ = cpu_to_be32(0);
}
return nfserr;
}
@@ -3299,14 +3542,17 @@
static __be32
nfsd4_encode_write(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_write *write)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (!nfserr) {
- RESERVE_SPACE(16);
- WRITE32(write->wr_bytes_written);
- WRITE32(write->wr_how_written);
- WRITEMEM(write->wr_verifier.data, NFS4_VERIFIER_SIZE);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 16);
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(write->wr_bytes_written);
+ *p++ = cpu_to_be32(write->wr_how_written);
+ p = xdr_encode_opaque_fixed(p, write->wr_verifier.data,
+ NFS4_VERIFIER_SIZE);
}
return nfserr;
}
@@ -3323,6 +3569,7 @@
nfsd4_encode_exchange_id(struct nfsd4_compoundres *resp, __be32 nfserr,
struct nfsd4_exchange_id *exid)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
char *major_id;
char *server_scope;
@@ -3338,60 +3585,61 @@
server_scope = utsname()->nodename;
server_scope_sz = strlen(server_scope);
- RESERVE_SPACE(
+ p = xdr_reserve_space(xdr,
8 /* eir_clientid */ +
4 /* eir_sequenceid */ +
4 /* eir_flags */ +
4 /* spr_how */);
+ if (!p)
+ return nfserr_resource;
- WRITEMEM(&exid->clientid, 8);
- WRITE32(exid->seqid);
- WRITE32(exid->flags);
+ p = xdr_encode_opaque_fixed(p, &exid->clientid, 8);
+ *p++ = cpu_to_be32(exid->seqid);
+ *p++ = cpu_to_be32(exid->flags);
- WRITE32(exid->spa_how);
- ADJUST_ARGS();
+ *p++ = cpu_to_be32(exid->spa_how);
switch (exid->spa_how) {
case SP4_NONE:
break;
case SP4_MACH_CRED:
/* spo_must_enforce, spo_must_allow */
- RESERVE_SPACE(16);
+ p = xdr_reserve_space(xdr, 16);
+ if (!p)
+ return nfserr_resource;
/* spo_must_enforce bitmap: */
- WRITE32(2);
- WRITE32(nfs4_minimal_spo_must_enforce[0]);
- WRITE32(nfs4_minimal_spo_must_enforce[1]);
+ *p++ = cpu_to_be32(2);
+ *p++ = cpu_to_be32(nfs4_minimal_spo_must_enforce[0]);
+ *p++ = cpu_to_be32(nfs4_minimal_spo_must_enforce[1]);
/* empty spo_must_allow bitmap: */
- WRITE32(0);
+ *p++ = cpu_to_be32(0);
- ADJUST_ARGS();
break;
default:
WARN_ON_ONCE(1);
}
- RESERVE_SPACE(
+ p = xdr_reserve_space(xdr,
8 /* so_minor_id */ +
4 /* so_major_id.len */ +
(XDR_QUADLEN(major_id_sz) * 4) +
4 /* eir_server_scope.len */ +
(XDR_QUADLEN(server_scope_sz) * 4) +
4 /* eir_server_impl_id.count (0) */);
+ if (!p)
+ return nfserr_resource;
/* The server_owner struct */
- WRITE64(minor_id); /* Minor id */
+ p = xdr_encode_hyper(p, minor_id); /* Minor id */
/* major id */
- WRITE32(major_id_sz);
- WRITEMEM(major_id, major_id_sz);
+ p = xdr_encode_opaque(p, major_id, major_id_sz);
/* Server scope */
- WRITE32(server_scope_sz);
- WRITEMEM(server_scope, server_scope_sz);
+ p = xdr_encode_opaque(p, server_scope, server_scope_sz);
/* Implementation id */
- WRITE32(0); /* zero length nfs_impl_id4 array */
- ADJUST_ARGS();
+ *p++ = cpu_to_be32(0); /* zero length nfs_impl_id4 array */
return 0;
}
@@ -3399,47 +3647,54 @@
nfsd4_encode_create_session(struct nfsd4_compoundres *resp, __be32 nfserr,
struct nfsd4_create_session *sess)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (nfserr)
return nfserr;
- RESERVE_SPACE(24);
- WRITEMEM(sess->sessionid.data, NFS4_MAX_SESSIONID_LEN);
- WRITE32(sess->seqid);
- WRITE32(sess->flags);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 24);
+ if (!p)
+ return nfserr_resource;
+ p = xdr_encode_opaque_fixed(p, sess->sessionid.data,
+ NFS4_MAX_SESSIONID_LEN);
+ *p++ = cpu_to_be32(sess->seqid);
+ *p++ = cpu_to_be32(sess->flags);
- RESERVE_SPACE(28);
- WRITE32(0); /* headerpadsz */
- WRITE32(sess->fore_channel.maxreq_sz);
- WRITE32(sess->fore_channel.maxresp_sz);
- WRITE32(sess->fore_channel.maxresp_cached);
- WRITE32(sess->fore_channel.maxops);
- WRITE32(sess->fore_channel.maxreqs);
- WRITE32(sess->fore_channel.nr_rdma_attrs);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 28);
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(0); /* headerpadsz */
+ *p++ = cpu_to_be32(sess->fore_channel.maxreq_sz);
+ *p++ = cpu_to_be32(sess->fore_channel.maxresp_sz);
+ *p++ = cpu_to_be32(sess->fore_channel.maxresp_cached);
+ *p++ = cpu_to_be32(sess->fore_channel.maxops);
+ *p++ = cpu_to_be32(sess->fore_channel.maxreqs);
+ *p++ = cpu_to_be32(sess->fore_channel.nr_rdma_attrs);
if (sess->fore_channel.nr_rdma_attrs) {
- RESERVE_SPACE(4);
- WRITE32(sess->fore_channel.rdma_attrs);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(sess->fore_channel.rdma_attrs);
}
- RESERVE_SPACE(28);
- WRITE32(0); /* headerpadsz */
- WRITE32(sess->back_channel.maxreq_sz);
- WRITE32(sess->back_channel.maxresp_sz);
- WRITE32(sess->back_channel.maxresp_cached);
- WRITE32(sess->back_channel.maxops);
- WRITE32(sess->back_channel.maxreqs);
- WRITE32(sess->back_channel.nr_rdma_attrs);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 28);
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(0); /* headerpadsz */
+ *p++ = cpu_to_be32(sess->back_channel.maxreq_sz);
+ *p++ = cpu_to_be32(sess->back_channel.maxresp_sz);
+ *p++ = cpu_to_be32(sess->back_channel.maxresp_cached);
+ *p++ = cpu_to_be32(sess->back_channel.maxops);
+ *p++ = cpu_to_be32(sess->back_channel.maxreqs);
+ *p++ = cpu_to_be32(sess->back_channel.nr_rdma_attrs);
if (sess->back_channel.nr_rdma_attrs) {
- RESERVE_SPACE(4);
- WRITE32(sess->back_channel.rdma_attrs);
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 4);
+ if (!p)
+ return nfserr_resource;
+ *p++ = cpu_to_be32(sess->back_channel.rdma_attrs);
}
return 0;
}
@@ -3448,22 +3703,25 @@
nfsd4_encode_sequence(struct nfsd4_compoundres *resp, __be32 nfserr,
struct nfsd4_sequence *seq)
{
+ struct xdr_stream *xdr = &resp->xdr;
__be32 *p;
if (nfserr)
return nfserr;
- RESERVE_SPACE(NFS4_MAX_SESSIONID_LEN + 20);
- WRITEMEM(seq->sessionid.data, NFS4_MAX_SESSIONID_LEN);
- WRITE32(seq->seqid);
- WRITE32(seq->slotid);
+ p = xdr_reserve_space(xdr, NFS4_MAX_SESSIONID_LEN + 20);
+ if (!p)
+ return nfserr_resource;
+ p = xdr_encode_opaque_fixed(p, seq->sessionid.data,
+ NFS4_MAX_SESSIONID_LEN);
+ *p++ = cpu_to_be32(seq->seqid);
+ *p++ = cpu_to_be32(seq->slotid);
/* Note slotid's are numbered from zero: */
- WRITE32(seq->maxslots - 1); /* sr_highest_slotid */
- WRITE32(seq->maxslots - 1); /* sr_target_highest_slotid */
- WRITE32(seq->status_flags);
+ *p++ = cpu_to_be32(seq->maxslots - 1); /* sr_highest_slotid */
+ *p++ = cpu_to_be32(seq->maxslots - 1); /* sr_target_highest_slotid */
+ *p++ = cpu_to_be32(seq->status_flags);
- ADJUST_ARGS();
- resp->cstate.datap = p; /* DRC cache data pointer */
+ resp->cstate.data_offset = xdr->buf->len; /* DRC cache data pointer */
return 0;
}
@@ -3471,20 +3729,22 @@
nfsd4_encode_test_stateid(struct nfsd4_compoundres *resp, __be32 nfserr,
struct nfsd4_test_stateid *test_stateid)
{
+ struct xdr_stream *xdr = &resp->xdr;
struct nfsd4_test_stateid_id *stateid, *next;
__be32 *p;
if (nfserr)
return nfserr;
- RESERVE_SPACE(4 + (4 * test_stateid->ts_num_ids));
+ p = xdr_reserve_space(xdr, 4 + (4 * test_stateid->ts_num_ids));
+ if (!p)
+ return nfserr_resource;
*p++ = htonl(test_stateid->ts_num_ids);
list_for_each_entry_safe(stateid, next, &test_stateid->ts_stateid_list, ts_id_list) {
*p++ = stateid->ts_id_status;
}
- ADJUST_ARGS();
return nfserr;
}
@@ -3563,81 +3823,99 @@
};
/*
- * Calculate the total amount of memory that the compound response has taken
- * after encoding the current operation with pad.
+ * Calculate whether we still have space to encode repsize bytes.
+ * There are two considerations:
+ * - For NFS versions >=4.1, the size of the reply must stay within
+ * session limits
+ * - For all NFS versions, we must stay within limited preallocated
+ * buffer space.
*
- * pad: if operation is non-idempotent, pad was calculate by op_rsize_bop()
- * which was specified at nfsd4_operation, else pad is zero.
- *
- * Compare this length to the session se_fmaxresp_sz and se_fmaxresp_cached.
- *
- * Our se_fmaxresp_cached will always be a multiple of PAGE_SIZE, and so
- * will be at least a page and will therefore hold the xdr_buf head.
+ * This is called before the operation is processed, so can only provide
+ * an upper estimate. For some nonidempotent operations (such as
+ * getattr), it's not necessarily a problem if that estimate is wrong,
+ * as we can fail it after processing without significant side effects.
*/
-__be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 pad)
+__be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
{
- struct xdr_buf *xb = &resp->rqstp->rq_res;
- struct nfsd4_session *session = NULL;
+ struct xdr_buf *buf = &resp->rqstp->rq_res;
struct nfsd4_slot *slot = resp->cstate.slot;
- u32 length, tlen = 0;
+ if (buf->len + respsize <= buf->buflen)
+ return nfs_ok;
if (!nfsd4_has_session(&resp->cstate))
- return 0;
-
- session = resp->cstate.session;
-
- if (xb->page_len == 0) {
- length = (char *)resp->p - (char *)xb->head[0].iov_base + pad;
- } else {
- if (xb->tail[0].iov_base && xb->tail[0].iov_len > 0)
- tlen = (char *)resp->p - (char *)xb->tail[0].iov_base;
-
- length = xb->head[0].iov_len + xb->page_len + tlen + pad;
- }
- dprintk("%s length %u, xb->page_len %u tlen %u pad %u\n", __func__,
- length, xb->page_len, tlen, pad);
-
- if (length > session->se_fchannel.maxresp_sz)
- return nfserr_rep_too_big;
-
- if ((slot->sl_flags & NFSD4_SLOT_CACHETHIS) &&
- length > session->se_fchannel.maxresp_cached)
+ return nfserr_resource;
+ if (slot->sl_flags & NFSD4_SLOT_CACHETHIS) {
+ WARN_ON_ONCE(1);
return nfserr_rep_too_big_to_cache;
-
- return 0;
+ }
+ return nfserr_rep_too_big;
}
void
nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
{
+ struct xdr_stream *xdr = &resp->xdr;
struct nfs4_stateowner *so = resp->cstate.replay_owner;
- __be32 *statp;
+ struct svc_rqst *rqstp = resp->rqstp;
+ int post_err_offset;
+ nfsd4_enc encoder;
__be32 *p;
- RESERVE_SPACE(8);
- WRITE32(op->opnum);
- statp = p++; /* to be backfilled at the end */
- ADJUST_ARGS();
+ p = xdr_reserve_space(xdr, 8);
+ if (!p) {
+ WARN_ON_ONCE(1);
+ return;
+ }
+ *p++ = cpu_to_be32(op->opnum);
+ post_err_offset = xdr->buf->len;
if (op->opnum == OP_ILLEGAL)
goto status;
BUG_ON(op->opnum < 0 || op->opnum >= ARRAY_SIZE(nfsd4_enc_ops) ||
!nfsd4_enc_ops[op->opnum]);
- op->status = nfsd4_enc_ops[op->opnum](resp, op->status, &op->u);
+ encoder = nfsd4_enc_ops[op->opnum];
+ op->status = encoder(resp, op->status, &op->u);
+ xdr_commit_encode(xdr);
+
/* nfsd4_check_resp_size guarantees enough room for error status */
- if (!op->status)
- op->status = nfsd4_check_resp_size(resp, 0);
+ if (!op->status) {
+ int space_needed = 0;
+ if (!nfsd4_last_compound_op(rqstp))
+ space_needed = COMPOUND_ERR_SLACK_SPACE;
+ op->status = nfsd4_check_resp_size(resp, space_needed);
+ }
+ if (op->status == nfserr_resource && nfsd4_has_session(&resp->cstate)) {
+ struct nfsd4_slot *slot = resp->cstate.slot;
+
+ if (slot->sl_flags & NFSD4_SLOT_CACHETHIS)
+ op->status = nfserr_rep_too_big_to_cache;
+ else
+ op->status = nfserr_rep_too_big;
+ }
+ if (op->status == nfserr_resource ||
+ op->status == nfserr_rep_too_big ||
+ op->status == nfserr_rep_too_big_to_cache) {
+ /*
+ * The operation may have already been encoded or
+ * partially encoded. No op returns anything additional
+ * in the case of one of these three errors, so we can
+ * just truncate back to after the status. But it's a
+ * bug if we had to do this on a non-idempotent op:
+ */
+ warn_on_nonidempotent_op(op);
+ xdr_truncate_encode(xdr, post_err_offset);
+ }
if (so) {
+ int len = xdr->buf->len - post_err_offset;
+
so->so_replay.rp_status = op->status;
- so->so_replay.rp_buflen = (char *)resp->p - (char *)(statp+1);
- memcpy(so->so_replay.rp_buf, statp+1, so->so_replay.rp_buflen);
+ so->so_replay.rp_buflen = len;
+ read_bytes_from_xdr_buf(xdr->buf, post_err_offset,
+ so->so_replay.rp_buf, len);
}
status:
- /*
- * Note: We write the status directly, instead of using WRITE32(),
- * since it is already in network byte order.
- */
- *statp = op->status;
+ /* Note that op->status is already in network byte order: */
+ write_bytes_to_xdr_buf(xdr->buf, post_err_offset - 4, &op->status, 4);
}
/*
@@ -3649,21 +3927,22 @@
* called with nfs4_lock_state() held
*/
void
-nfsd4_encode_replay(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
+nfsd4_encode_replay(struct xdr_stream *xdr, struct nfsd4_op *op)
{
__be32 *p;
struct nfs4_replay *rp = op->replay;
BUG_ON(!rp);
- RESERVE_SPACE(8);
- WRITE32(op->opnum);
+ p = xdr_reserve_space(xdr, 8 + rp->rp_buflen);
+ if (!p) {
+ WARN_ON_ONCE(1);
+ return;
+ }
+ *p++ = cpu_to_be32(op->opnum);
*p++ = rp->rp_status; /* already xdr'ed */
- ADJUST_ARGS();
- RESERVE_SPACE(rp->rp_buflen);
- WRITEMEM(rp->rp_buf, rp->rp_buflen);
- ADJUST_ARGS();
+ p = xdr_encode_opaque_fixed(p, rp->rp_buf, rp->rp_buflen);
}
int
@@ -3720,19 +3999,19 @@
* All that remains is to write the tag and operation count...
*/
struct nfsd4_compound_state *cs = &resp->cstate;
- struct kvec *iov;
+ struct xdr_buf *buf = resp->xdr.buf;
+
+ WARN_ON_ONCE(buf->len != buf->head[0].iov_len + buf->page_len +
+ buf->tail[0].iov_len);
+
+ rqstp->rq_next_page = resp->xdr.page_ptr + 1;
+
p = resp->tagp;
*p++ = htonl(resp->taglen);
memcpy(p, resp->tag, resp->taglen);
p += XDR_QUADLEN(resp->taglen);
*p++ = htonl(resp->opcnt);
- if (rqstp->rq_res.page_len)
- iov = &rqstp->rq_res.tail[0];
- else
- iov = &rqstp->rq_res.head[0];
- iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
- BUG_ON(iov->iov_len > PAGE_SIZE);
if (nfsd4_has_session(cs)) {
struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
struct nfs4_client *clp = cs->session->se_client;
diff --git a/fs/nfsd/nfscache.c b/fs/nfsd/nfscache.c
index f8f060f..6040da88 100644
--- a/fs/nfsd/nfscache.c
+++ b/fs/nfsd/nfscache.c
@@ -224,13 +224,6 @@
hlist_add_head(&rp->c_hash, cache_hash + hash_32(rp->c_xid, maskbits));
}
-static inline bool
-nfsd_cache_entry_expired(struct svc_cacherep *rp)
-{
- return rp->c_state != RC_INPROG &&
- time_after(jiffies, rp->c_timestamp + RC_EXPIRE);
-}
-
/*
* Walk the LRU list and prune off entries that are older than RC_EXPIRE.
* Also prune the oldest ones when the total exceeds the max number of entries.
@@ -242,8 +235,14 @@
long freed = 0;
list_for_each_entry_safe(rp, tmp, &lru_head, c_lru) {
- if (!nfsd_cache_entry_expired(rp) &&
- num_drc_entries <= max_drc_entries)
+ /*
+ * Don't free entries attached to calls that are still
+ * in-progress, but do keep scanning the list.
+ */
+ if (rp->c_state == RC_INPROG)
+ continue;
+ if (num_drc_entries <= max_drc_entries &&
+ time_before(jiffies, rp->c_timestamp + RC_EXPIRE))
break;
nfsd_reply_cache_free_locked(rp);
freed++;
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index f34d9de..5184404 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -1179,7 +1179,6 @@
retval = nfsd4_init_slabs();
if (retval)
goto out_unregister_pernet;
- nfs4_state_init();
retval = nfsd_fault_inject_init(); /* nfsd fault injection controls */
if (retval)
goto out_free_slabs;
diff --git a/fs/nfsd/nfsd.h b/fs/nfsd/nfsd.h
index 479eb68..847daf3 100644
--- a/fs/nfsd/nfsd.h
+++ b/fs/nfsd/nfsd.h
@@ -15,11 +15,20 @@
#include <linux/nfs2.h>
#include <linux/nfs3.h>
#include <linux/nfs4.h>
+#include <linux/sunrpc/svc.h>
#include <linux/sunrpc/msg_prot.h>
-#include <linux/nfsd/debug.h>
-#include <linux/nfsd/export.h>
-#include <linux/nfsd/stats.h>
+#include <uapi/linux/nfsd/debug.h>
+
+#include "stats.h"
+#include "export.h"
+
+#undef ifdebug
+#ifdef NFSD_DEBUG
+# define ifdebug(flag) if (nfsd_debug & NFSDDBG_##flag)
+#else
+# define ifdebug(flag) if (0)
+#endif
/*
* nfsd version
@@ -106,7 +115,6 @@
*/
#ifdef CONFIG_NFSD_V4
extern unsigned long max_delegations;
-void nfs4_state_init(void);
int nfsd4_init_slabs(void);
void nfsd4_free_slabs(void);
int nfs4_state_start(void);
@@ -117,7 +125,6 @@
int nfs4_reset_recoverydir(char *recdir);
char * nfs4_recoverydir(void);
#else
-static inline void nfs4_state_init(void) { }
static inline int nfsd4_init_slabs(void) { return 0; }
static inline void nfsd4_free_slabs(void) { }
static inline int nfs4_state_start(void) { return 0; }
diff --git a/fs/nfsd/nfsfh.c b/fs/nfsd/nfsfh.c
index 3c37b16..ec83934 100644
--- a/fs/nfsd/nfsfh.c
+++ b/fs/nfsd/nfsfh.c
@@ -88,9 +88,8 @@
/* Check if the request originated from a secure port. */
if (!rqstp->rq_secure && !(flags & NFSEXP_INSECURE_PORT)) {
RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]);
- dprintk(KERN_WARNING
- "nfsd: request from insecure port %s!\n",
- svc_print_addr(rqstp, buf, sizeof(buf)));
+ dprintk("nfsd: request from insecure port %s!\n",
+ svc_print_addr(rqstp, buf, sizeof(buf)));
return nfserr_perm;
}
@@ -169,8 +168,8 @@
data_left -= len;
if (data_left < 0)
return error;
- exp = rqst_exp_find(rqstp, fh->fh_fsid_type, fh->fh_auth);
- fid = (struct fid *)(fh->fh_auth + len);
+ exp = rqst_exp_find(rqstp, fh->fh_fsid_type, fh->fh_fsid);
+ fid = (struct fid *)(fh->fh_fsid + len);
} else {
__u32 tfh[2];
dev_t xdev;
@@ -385,7 +384,7 @@
{
if (dentry != exp->ex_path.dentry) {
struct fid *fid = (struct fid *)
- (fhp->fh_handle.fh_auth + fhp->fh_handle.fh_size/4 - 1);
+ (fhp->fh_handle.fh_fsid + fhp->fh_handle.fh_size/4 - 1);
int maxsize = (fhp->fh_maxsize - fhp->fh_handle.fh_size)/4;
int subtreecheck = !(exp->ex_flags & NFSEXP_NOSUBTREECHECK);
@@ -513,7 +512,6 @@
*/
struct inode * inode = dentry->d_inode;
- __u32 *datap;
dev_t ex_dev = exp_sb(exp)->s_dev;
dprintk("nfsd: fh_compose(exp %02x:%02x/%ld %pd2, ino=%ld)\n",
@@ -557,17 +555,16 @@
if (inode)
_fh_update_old(dentry, exp, &fhp->fh_handle);
} else {
- int len;
+ fhp->fh_handle.fh_size =
+ key_len(fhp->fh_handle.fh_fsid_type) + 4;
fhp->fh_handle.fh_auth_type = 0;
- datap = fhp->fh_handle.fh_auth+0;
- mk_fsid(fhp->fh_handle.fh_fsid_type, datap, ex_dev,
+
+ mk_fsid(fhp->fh_handle.fh_fsid_type,
+ fhp->fh_handle.fh_fsid,
+ ex_dev,
exp->ex_path.dentry->d_inode->i_ino,
exp->ex_fsid, exp->ex_uuid);
- len = key_len(fhp->fh_handle.fh_fsid_type);
- datap += len/4;
- fhp->fh_handle.fh_size = 4 + len;
-
if (inode)
_fh_update(fhp, exp, dentry);
if (fhp->fh_handle.fh_fileid_type == FILEID_INVALID) {
diff --git a/fs/nfsd/nfsfh.h b/fs/nfsd/nfsfh.h
index ad67964..2e89e70 100644
--- a/fs/nfsd/nfsfh.h
+++ b/fs/nfsd/nfsfh.h
@@ -1,9 +1,58 @@
-/* Copyright (C) 1995, 1996, 1997 Olaf Kirch <okir@monad.swb.de> */
+/*
+ * Copyright (C) 1995, 1996, 1997 Olaf Kirch <okir@monad.swb.de>
+ *
+ * This file describes the layout of the file handles as passed
+ * over the wire.
+ */
+#ifndef _LINUX_NFSD_NFSFH_H
+#define _LINUX_NFSD_NFSFH_H
-#ifndef _LINUX_NFSD_FH_INT_H
-#define _LINUX_NFSD_FH_INT_H
+#include <linux/sunrpc/svc.h>
+#include <uapi/linux/nfsd/nfsfh.h>
-#include <linux/nfsd/nfsfh.h>
+static inline __u32 ino_t_to_u32(ino_t ino)
+{
+ return (__u32) ino;
+}
+
+static inline ino_t u32_to_ino_t(__u32 uino)
+{
+ return (ino_t) uino;
+}
+
+/*
+ * This is the internal representation of an NFS handle used in knfsd.
+ * pre_mtime/post_version will be used to support wcc_attr's in NFSv3.
+ */
+typedef struct svc_fh {
+ struct knfsd_fh fh_handle; /* FH data */
+ struct dentry * fh_dentry; /* validated dentry */
+ struct svc_export * fh_export; /* export pointer */
+ int fh_maxsize; /* max size for fh_handle */
+
+ unsigned char fh_locked; /* inode locked by us */
+ unsigned char fh_want_write; /* remount protection taken */
+
+#ifdef CONFIG_NFSD_V3
+ unsigned char fh_post_saved; /* post-op attrs saved */
+ unsigned char fh_pre_saved; /* pre-op attrs saved */
+
+ /* Pre-op attributes saved during fh_lock */
+ __u64 fh_pre_size; /* size before operation */
+ struct timespec fh_pre_mtime; /* mtime before oper */
+ struct timespec fh_pre_ctime; /* ctime before oper */
+ /*
+ * pre-op nfsv4 change attr: note must check IS_I_VERSION(inode)
+ * to find out if it is valid.
+ */
+ u64 fh_pre_change;
+
+ /* Post-op attributes saved in fh_unlock */
+ struct kstat fh_post_attr; /* full attrs after operation */
+ u64 fh_post_change; /* nfsv4 change; see above */
+#endif /* CONFIG_NFSD_V3 */
+
+} svc_fh;
enum nfsd_fsid {
FSID_DEV = 0,
@@ -215,4 +264,4 @@
}
}
-#endif /* _LINUX_NFSD_FH_INT_H */
+#endif /* _LINUX_NFSD_NFSFH_H */
diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index 9a4a5f9..1879e43 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -591,12 +591,6 @@
nfsdstats.th_cnt++;
mutex_unlock(&nfsd_mutex);
- /*
- * We want less throttling in balance_dirty_pages() so that nfs to
- * localhost doesn't cause nfsd to lock up due to all the client's
- * dirty pages.
- */
- current->flags |= PF_LESS_THROTTLE;
set_freezable();
/*
diff --git a/fs/nfsd/nfsxdr.c b/fs/nfsd/nfsxdr.c
index 9c769a4..1ac306b 100644
--- a/fs/nfsd/nfsxdr.c
+++ b/fs/nfsd/nfsxdr.c
@@ -214,7 +214,8 @@
int
nfssvc_decode_fhandle(struct svc_rqst *rqstp, __be32 *p, struct nfsd_fhandle *args)
{
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
return xdr_argsize_check(rqstp, p);
}
@@ -248,7 +249,8 @@
{
unsigned int len;
int v;
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
args->offset = ntohl(*p++);
@@ -281,7 +283,8 @@
unsigned int len, hdr, dlen;
int v;
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
p++; /* beginoffset */
@@ -355,7 +358,8 @@
int
nfssvc_decode_readlinkargs(struct svc_rqst *rqstp, __be32 *p, struct nfsd_readlinkargs *args)
{
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
args->buffer = page_address(*(rqstp->rq_next_page++));
@@ -391,7 +395,8 @@
nfssvc_decode_readdirargs(struct svc_rqst *rqstp, __be32 *p,
struct nfsd_readdirargs *args)
{
- if (!(p = decode_fh(p, &args->fh)))
+ p = decode_fh(p, &args->fh);
+ if (!p)
return 0;
args->cookie = ntohl(*p++);
args->count = ntohl(*p++);
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index 424d8f5..374c662 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -37,7 +37,6 @@
#include <linux/idr.h>
#include <linux/sunrpc/svc_xprt.h>
-#include <linux/nfsd/nfsfh.h>
#include "nfsfh.h"
typedef struct {
@@ -123,7 +122,7 @@
/* Maximum number of operations per session compound */
#define NFSD_MAX_OPS_PER_COMPOUND 16
/* Maximum session per slot cache size */
-#define NFSD_SLOT_CACHE_SIZE 1024
+#define NFSD_SLOT_CACHE_SIZE 2048
/* Maximum number of NFSD_SLOT_CACHE_SIZE slots per session */
#define NFSD_CACHE_SIZE_SLOTS_PER_SESSION 32
#define NFSD_MAX_MEM_PER_SESSION \
@@ -464,8 +463,6 @@
extern struct nfs4_client_reclaim *nfsd4_find_reclaim_client(const char *recdir,
struct nfsd_net *nn);
extern __be32 nfs4_check_open_reclaim(clientid_t *clid, bool sessions, struct nfsd_net *nn);
-extern void nfs4_free_openowner(struct nfs4_openowner *);
-extern void nfs4_free_lockowner(struct nfs4_lockowner *);
extern int set_callback_cred(void);
extern void nfsd4_init_callback(struct nfsd4_callback *);
extern void nfsd4_probe_callback(struct nfs4_client *clp);
diff --git a/fs/nfsd/stats.c b/fs/nfsd/stats.c
index 6d4521f..cd90878 100644
--- a/fs/nfsd/stats.c
+++ b/fs/nfsd/stats.c
@@ -24,7 +24,6 @@
#include <linux/seq_file.h>
#include <linux/module.h>
#include <linux/sunrpc/stats.h>
-#include <linux/nfsd/stats.h>
#include <net/net_namespace.h>
#include "nfsd.h"
diff --git a/include/linux/nfsd/stats.h b/fs/nfsd/stats.h
similarity index 91%
rename from include/linux/nfsd/stats.h
rename to fs/nfsd/stats.h
index e75b254..a5c944b 100644
--- a/include/linux/nfsd/stats.h
+++ b/fs/nfsd/stats.h
@@ -1,12 +1,10 @@
/*
- * linux/include/linux/nfsd/stats.h
- *
* Statistics for NFS server.
*
* Copyright (C) 1995, 1996 Olaf Kirch <okir@monad.swb.de>
*/
-#ifndef LINUX_NFSD_STATS_H
-#define LINUX_NFSD_STATS_H
+#ifndef _NFSD_STATS_H
+#define _NFSD_STATS_H
#include <uapi/linux/nfsd/stats.h>
@@ -42,4 +40,4 @@
void nfsd_stat_init(void);
void nfsd_stat_shutdown(void);
-#endif /* LINUX_NFSD_STATS_H */
+#endif /* _NFSD_STATS_H */
diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c
index 16f0673..140c496 100644
--- a/fs/nfsd/vfs.c
+++ b/fs/nfsd/vfs.c
@@ -820,55 +820,54 @@
return __splice_from_pipe(pipe, sd, nfsd_splice_actor);
}
-static __be32
-nfsd_vfs_read(struct svc_rqst *rqstp, struct svc_fh *fhp, struct file *file,
- loff_t offset, struct kvec *vec, int vlen, unsigned long *count)
+__be32 nfsd_finish_read(struct file *file, unsigned long *count, int host_err)
{
- mm_segment_t oldfs;
- __be32 err;
- int host_err;
-
- err = nfserr_perm;
-
- if (file->f_op->splice_read && rqstp->rq_splice_ok) {
- struct splice_desc sd = {
- .len = 0,
- .total_len = *count,
- .pos = offset,
- .u.data = rqstp,
- };
-
- rqstp->rq_next_page = rqstp->rq_respages + 1;
- host_err = splice_direct_to_actor(file, &sd, nfsd_direct_splice_actor);
- } else {
- oldfs = get_fs();
- set_fs(KERNEL_DS);
- host_err = vfs_readv(file, (struct iovec __user *)vec, vlen, &offset);
- set_fs(oldfs);
- }
-
if (host_err >= 0) {
nfsdstats.io_read += host_err;
*count = host_err;
- err = 0;
fsnotify_access(file);
+ return 0;
} else
- err = nfserrno(host_err);
- return err;
+ return nfserrno(host_err);
}
-static void kill_suid(struct dentry *dentry)
+int nfsd_splice_read(struct svc_rqst *rqstp,
+ struct file *file, loff_t offset, unsigned long *count)
{
- struct iattr ia;
- ia.ia_valid = ATTR_KILL_SUID | ATTR_KILL_SGID | ATTR_KILL_PRIV;
+ struct splice_desc sd = {
+ .len = 0,
+ .total_len = *count,
+ .pos = offset,
+ .u.data = rqstp,
+ };
+ int host_err;
- mutex_lock(&dentry->d_inode->i_mutex);
- /*
- * Note we call this on write, so notify_change will not
- * encounter any conflicting delegations:
- */
- notify_change(dentry, &ia, NULL);
- mutex_unlock(&dentry->d_inode->i_mutex);
+ rqstp->rq_next_page = rqstp->rq_respages + 1;
+ host_err = splice_direct_to_actor(file, &sd, nfsd_direct_splice_actor);
+ return nfsd_finish_read(file, count, host_err);
+}
+
+int nfsd_readv(struct file *file, loff_t offset, struct kvec *vec, int vlen,
+ unsigned long *count)
+{
+ mm_segment_t oldfs;
+ int host_err;
+
+ oldfs = get_fs();
+ set_fs(KERNEL_DS);
+ host_err = vfs_readv(file, (struct iovec __user *)vec, vlen, &offset);
+ set_fs(oldfs);
+ return nfsd_finish_read(file, count, host_err);
+}
+
+static __be32
+nfsd_vfs_read(struct svc_rqst *rqstp, struct file *file,
+ loff_t offset, struct kvec *vec, int vlen, unsigned long *count)
+{
+ if (file->f_op->splice_read && rqstp->rq_splice_ok)
+ return nfsd_splice_read(rqstp, file, offset, count);
+ else
+ return nfsd_readv(file, offset, vec, vlen, count);
}
/*
@@ -922,6 +921,16 @@
int stable = *stablep;
int use_wgather;
loff_t pos = offset;
+ unsigned int pflags = current->flags;
+
+ if (rqstp->rq_local)
+ /*
+ * We want less throttling in balance_dirty_pages()
+ * and shrink_inactive_list() so that nfs to
+ * localhost doesn't cause nfsd to lock up due to all
+ * the client's dirty pages or its congested queue.
+ */
+ current->flags |= PF_LESS_THROTTLE;
dentry = file->f_path.dentry;
inode = dentry->d_inode;
@@ -942,10 +951,6 @@
nfsdstats.io_write += host_err;
fsnotify_modify(file);
- /* clear setuid/setgid flag after write */
- if (inode->i_mode & (S_ISUID | S_ISGID))
- kill_suid(dentry);
-
if (stable) {
if (use_wgather)
host_err = wait_for_concurrent_writes(file);
@@ -959,9 +964,45 @@
err = 0;
else
err = nfserrno(host_err);
+ if (rqstp->rq_local)
+ tsk_restore_flags(current, pflags, PF_LESS_THROTTLE);
return err;
}
+__be32 nfsd_get_tmp_read_open(struct svc_rqst *rqstp, struct svc_fh *fhp,
+ struct file **file, struct raparms **ra)
+{
+ struct inode *inode;
+ __be32 err;
+
+ err = nfsd_open(rqstp, fhp, S_IFREG, NFSD_MAY_READ, file);
+ if (err)
+ return err;
+
+ inode = file_inode(*file);
+
+ /* Get readahead parameters */
+ *ra = nfsd_get_raparms(inode->i_sb->s_dev, inode->i_ino);
+
+ if (*ra && (*ra)->p_set)
+ (*file)->f_ra = (*ra)->p_ra;
+ return nfs_ok;
+}
+
+void nfsd_put_tmp_read_open(struct file *file, struct raparms *ra)
+{
+ /* Write back readahead params */
+ if (ra) {
+ struct raparm_hbucket *rab = &raparm_hash[ra->p_hindex];
+ spin_lock(&rab->pb_lock);
+ ra->p_ra = file->f_ra;
+ ra->p_set = 1;
+ ra->p_count--;
+ spin_unlock(&rab->pb_lock);
+ }
+ nfsd_close(file);
+}
+
/*
* Read data from a file. count must contain the requested read count
* on entry. On return, *count contains the number of bytes actually read.
@@ -971,55 +1012,17 @@
loff_t offset, struct kvec *vec, int vlen, unsigned long *count)
{
struct file *file;
- struct inode *inode;
struct raparms *ra;
__be32 err;
- err = nfsd_open(rqstp, fhp, S_IFREG, NFSD_MAY_READ, &file);
+ err = nfsd_get_tmp_read_open(rqstp, fhp, &file, &ra);
if (err)
return err;
- inode = file_inode(file);
+ err = nfsd_vfs_read(rqstp, file, offset, vec, vlen, count);
- /* Get readahead parameters */
- ra = nfsd_get_raparms(inode->i_sb->s_dev, inode->i_ino);
+ nfsd_put_tmp_read_open(file, ra);
- if (ra && ra->p_set)
- file->f_ra = ra->p_ra;
-
- err = nfsd_vfs_read(rqstp, fhp, file, offset, vec, vlen, count);
-
- /* Write back readahead params */
- if (ra) {
- struct raparm_hbucket *rab = &raparm_hash[ra->p_hindex];
- spin_lock(&rab->pb_lock);
- ra->p_ra = file->f_ra;
- ra->p_set = 1;
- ra->p_count--;
- spin_unlock(&rab->pb_lock);
- }
-
- nfsd_close(file);
- return err;
-}
-
-/* As above, but use the provided file descriptor. */
-__be32
-nfsd_read_file(struct svc_rqst *rqstp, struct svc_fh *fhp, struct file *file,
- loff_t offset, struct kvec *vec, int vlen,
- unsigned long *count)
-{
- __be32 err;
-
- if (file) {
- err = nfsd_permission(rqstp, fhp->fh_export, fhp->fh_dentry,
- NFSD_MAY_READ|NFSD_MAY_OWNER_OVERRIDE);
- if (err)
- goto out;
- err = nfsd_vfs_read(rqstp, fhp, file, offset, vec, vlen, count);
- } else /* Note file may still be NULL in NFSv4 special stateid case: */
- err = nfsd_read(rqstp, fhp, offset, vec, vlen, count);
-out:
return err;
}
diff --git a/fs/nfsd/vfs.h b/fs/nfsd/vfs.h
index fbe90bd..91b6ae3 100644
--- a/fs/nfsd/vfs.h
+++ b/fs/nfsd/vfs.h
@@ -70,10 +70,16 @@
__be32 nfsd_open(struct svc_rqst *, struct svc_fh *, umode_t,
int, struct file **);
void nfsd_close(struct file *);
+struct raparms;
+__be32 nfsd_get_tmp_read_open(struct svc_rqst *, struct svc_fh *,
+ struct file **, struct raparms **);
+void nfsd_put_tmp_read_open(struct file *, struct raparms *);
+int nfsd_splice_read(struct svc_rqst *,
+ struct file *, loff_t, unsigned long *);
+int nfsd_readv(struct file *, loff_t, struct kvec *, int,
+ unsigned long *);
__be32 nfsd_read(struct svc_rqst *, struct svc_fh *,
loff_t, struct kvec *, int, unsigned long *);
-__be32 nfsd_read_file(struct svc_rqst *, struct svc_fh *, struct file *,
- loff_t, struct kvec *, int, unsigned long *);
__be32 nfsd_write(struct svc_rqst *, struct svc_fh *,struct file *,
loff_t, struct kvec *,int, unsigned long *, int *);
__be32 nfsd_readlink(struct svc_rqst *, struct svc_fh *,
diff --git a/fs/nfsd/xdr4.h b/fs/nfsd/xdr4.h
index 5ea7df3..18cbb6d 100644
--- a/fs/nfsd/xdr4.h
+++ b/fs/nfsd/xdr4.h
@@ -58,7 +58,7 @@
/* For sessions DRC */
struct nfsd4_session *session;
struct nfsd4_slot *slot;
- __be32 *datap;
+ int data_offset;
size_t iovlen;
u32 minorversion;
__be32 status;
@@ -287,9 +287,8 @@
struct svc_fh * rd_fhp; /* response */
struct readdir_cd common;
- __be32 * buffer;
- int buflen;
- __be32 * offset;
+ struct xdr_stream *xdr;
+ int cookie_offset;
};
struct nfsd4_release_lockowner {
@@ -506,9 +505,7 @@
struct nfsd4_compoundres {
/* scratch variables for XDR encode */
- __be32 * p;
- __be32 * end;
- struct xdr_buf * xbuf;
+ struct xdr_stream xdr;
struct svc_rqst * rqstp;
u32 taglen;
@@ -538,6 +535,9 @@
return argp->opcnt == resp->opcnt;
}
+int nfsd4_max_reply(struct svc_rqst *rqstp, struct nfsd4_op *op);
+void warn_on_nonidempotent_op(struct nfsd4_op *op);
+
#define NFS4_SVC_XDRSIZE sizeof(struct nfsd4_compoundargs)
static inline void
@@ -563,10 +563,11 @@
struct nfsd4_compoundres *);
__be32 nfsd4_check_resp_size(struct nfsd4_compoundres *, u32);
void nfsd4_encode_operation(struct nfsd4_compoundres *, struct nfsd4_op *);
-void nfsd4_encode_replay(struct nfsd4_compoundres *resp, struct nfsd4_op *op);
-__be32 nfsd4_encode_fattr(struct svc_fh *fhp, struct svc_export *exp,
- struct dentry *dentry, __be32 **buffer, int countp,
- u32 *bmval, struct svc_rqst *, int ignore_crossmnt);
+void nfsd4_encode_replay(struct xdr_stream *xdr, struct nfsd4_op *op);
+__be32 nfsd4_encode_fattr_to_buf(__be32 **p, int words,
+ struct svc_fh *fhp, struct svc_export *exp,
+ struct dentry *dentry,
+ u32 *bmval, struct svc_rqst *, int ignore_crossmnt);
extern __be32 nfsd4_setclientid(struct svc_rqst *rqstp,
struct nfsd4_compound_state *,
struct nfsd4_setclientid *setclid);
diff --git a/fs/notify/inotify/inotify_user.c b/fs/notify/inotify/inotify_user.c
index 78a2ca3..cc423a3 100644
--- a/fs/notify/inotify/inotify_user.c
+++ b/fs/notify/inotify/inotify_user.c
@@ -57,7 +57,7 @@
static int zero;
-ctl_table inotify_table[] = {
+struct ctl_table inotify_table[] = {
{
.procname = "max_user_instances",
.data = &inotify_max_user_instances,
diff --git a/fs/ntfs/sysctl.c b/fs/ntfs/sysctl.c
index 1927170..a503156 100644
--- a/fs/ntfs/sysctl.c
+++ b/fs/ntfs/sysctl.c
@@ -34,7 +34,7 @@
#include "debug.h"
/* Definition of the ntfs sysctl. */
-static ctl_table ntfs_sysctls[] = {
+static struct ctl_table ntfs_sysctls[] = {
{
.procname = "ntfs-debug",
.data = &debug_msgs, /* Data pointer and size. */
@@ -46,7 +46,7 @@
};
/* Define the parent directory /proc/sys/fs. */
-static ctl_table sysctls_root[] = {
+static struct ctl_table sysctls_root[] = {
{
.procname = "fs",
.mode = 0555,
diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c
index a68e07a..681691b 100644
--- a/fs/ocfs2/cluster/tcp.c
+++ b/fs/ocfs2/cluster/tcp.c
@@ -1799,7 +1799,7 @@
/* ------------------------------------------------------------ */
-static int o2net_accept_one(struct socket *sock)
+static int o2net_accept_one(struct socket *sock, int *more)
{
int ret, slen;
struct sockaddr_in sin;
@@ -1810,6 +1810,7 @@
struct o2net_node *nn;
BUG_ON(sock == NULL);
+ *more = 0;
ret = sock_create_lite(sock->sk->sk_family, sock->sk->sk_type,
sock->sk->sk_protocol, &new_sock);
if (ret)
@@ -1821,6 +1822,7 @@
if (ret < 0)
goto out;
+ *more = 1;
new_sock->sk->sk_allocation = GFP_ATOMIC;
ret = o2net_set_nodelay(new_sock);
@@ -1919,11 +1921,36 @@
return ret;
}
+/*
+ * This function is invoked in response to one or more
+ * pending accepts at softIRQ level. We must drain the
+ * entire que before returning.
+ */
+
static void o2net_accept_many(struct work_struct *work)
{
struct socket *sock = o2net_listen_sock;
- while (o2net_accept_one(sock) == 0)
+ int more;
+ int err;
+
+ /*
+ * It is critical to note that due to interrupt moderation
+ * at the network driver level, we can't assume to get a
+ * softIRQ for every single conn since tcp SYN packets
+ * can arrive back-to-back, and therefore many pending
+ * accepts may result in just 1 softIRQ. If we terminate
+ * the o2net_accept_one() loop upon seeing an err, what happens
+ * to the rest of the conns in the queue? If no new SYN
+ * arrives for hours, no softIRQ will be delivered,
+ * and the connections will just sit in the queue.
+ */
+
+ for (;;) {
+ err = o2net_accept_one(sock, &more);
+ if (!more)
+ break;
cond_resched();
+ }
}
static void o2net_listen_data_ready(struct sock *sk)
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 48cbe4c..cfa63ee 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -1361,7 +1361,7 @@
struct numa_maps *md;
struct page *page;
- if (pte_none(*pte))
+ if (!pte_present(*pte))
return 0;
page = pte_page(*pte);
@@ -1418,10 +1418,10 @@
seq_printf(m, "%08lx %s", vma->vm_start, buffer);
if (file) {
- seq_printf(m, " file=");
+ seq_puts(m, " file=");
seq_path(m, &file->f_path, "\n\t= ");
} else if (vma->vm_start <= mm->brk && vma->vm_end >= mm->start_brk) {
- seq_printf(m, " heap");
+ seq_puts(m, " heap");
} else {
pid_t tid = vm_is_stack(task, vma, is_pid);
if (tid != 0) {
@@ -1431,14 +1431,14 @@
*/
if (!is_pid || (vma->vm_start <= mm->start_stack &&
vma->vm_end >= mm->start_stack))
- seq_printf(m, " stack");
+ seq_puts(m, " stack");
else
seq_printf(m, " stack:%d", tid);
}
}
if (is_vm_hugetlb_page(vma))
- seq_printf(m, " huge");
+ seq_puts(m, " huge");
walk_page_range(vma->vm_start, vma->vm_end, &walk);
diff --git a/fs/proc/vmcore.c b/fs/proc/vmcore.c
index 6a8e785..382aa89 100644
--- a/fs/proc/vmcore.c
+++ b/fs/proc/vmcore.c
@@ -42,7 +42,7 @@
/* Total size of vmcore file. */
static u64 vmcore_size;
-static struct proc_dir_entry *proc_vmcore = NULL;
+static struct proc_dir_entry *proc_vmcore;
/*
* Returns > 0 for RAM pages, 0 for non-RAM pages, < 0 on error
diff --git a/fs/pstore/platform.c b/fs/pstore/platform.c
index 46d269e..0a9b72c 100644
--- a/fs/pstore/platform.c
+++ b/fs/pstore/platform.c
@@ -18,6 +18,8 @@
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
+#define pr_fmt(fmt) "pstore: " fmt
+
#include <linux/atomic.h>
#include <linux/types.h>
#include <linux/errno.h>
@@ -224,14 +226,12 @@
zlib_inflate_workspacesize());
stream.workspace = kmalloc(size, GFP_KERNEL);
if (!stream.workspace) {
- pr_err("pstore: No memory for compression workspace; "
- "skipping compression\n");
+ pr_err("No memory for compression workspace; skipping compression\n");
kfree(big_oops_buf);
big_oops_buf = NULL;
}
} else {
- pr_err("No memory for uncompressed data; "
- "skipping compression\n");
+ pr_err("No memory for uncompressed data; skipping compression\n");
stream.workspace = NULL;
}
@@ -455,8 +455,7 @@
add_timer(&pstore_timer);
}
- pr_info("pstore: Registered %s as persistent store backend\n",
- psi->name);
+ pr_info("Registered %s as persistent store backend\n", psi->name);
return 0;
}
@@ -502,8 +501,8 @@
size = unzipped_len;
compressed = false;
} else {
- pr_err("pstore: decompression failed;"
- "returned %d\n", unzipped_len);
+ pr_err("decompression failed;returned %d\n",
+ unzipped_len);
compressed = true;
}
}
@@ -524,8 +523,8 @@
mutex_unlock(&psi->read_mutex);
if (failed)
- printk(KERN_WARNING "pstore: failed to load %d record(s) from '%s'\n",
- failed, psi->name);
+ pr_warn("failed to load %d record(s) from '%s'\n",
+ failed, psi->name);
}
static void pstore_dowork(struct work_struct *work)
diff --git a/fs/pstore/ram_core.c b/fs/pstore/ram_core.c
index ff7e3d4..34a1e5a 100644
--- a/fs/pstore/ram_core.c
+++ b/fs/pstore/ram_core.c
@@ -12,6 +12,8 @@
*
*/
+#define pr_fmt(fmt) "persistent_ram: " fmt
+
#include <linux/device.h>
#include <linux/err.h>
#include <linux/errno.h>
@@ -205,12 +207,10 @@
size = buffer->data + prz->buffer_size - block;
numerr = persistent_ram_decode_rs8(prz, block, size, par);
if (numerr > 0) {
- pr_devel("persistent_ram: error in block %p, %d\n",
- block, numerr);
+ pr_devel("error in block %p, %d\n", block, numerr);
prz->corrected_bytes += numerr;
} else if (numerr < 0) {
- pr_devel("persistent_ram: uncorrectable error in block %p\n",
- block);
+ pr_devel("uncorrectable error in block %p\n", block);
prz->bad_blocks++;
}
block += prz->ecc_info.block_size;
@@ -257,7 +257,7 @@
prz->rs_decoder = init_rs(prz->ecc_info.symsize, prz->ecc_info.poly,
0, 1, prz->ecc_info.ecc_size);
if (prz->rs_decoder == NULL) {
- pr_info("persistent_ram: init_rs failed\n");
+ pr_info("init_rs failed\n");
return -EINVAL;
}
@@ -267,10 +267,10 @@
numerr = persistent_ram_decode_rs8(prz, buffer, sizeof(*buffer),
prz->par_header);
if (numerr > 0) {
- pr_info("persistent_ram: error in header, %d\n", numerr);
+ pr_info("error in header, %d\n", numerr);
prz->corrected_bytes += numerr;
} else if (numerr < 0) {
- pr_info("persistent_ram: uncorrectable error in header\n");
+ pr_info("uncorrectable error in header\n");
prz->bad_blocks++;
}
@@ -317,7 +317,7 @@
prz->old_log = kmalloc(size, GFP_KERNEL);
}
if (!prz->old_log) {
- pr_err("persistent_ram: failed to allocate buffer\n");
+ pr_err("failed to allocate buffer\n");
return;
}
@@ -396,8 +396,8 @@
pages = kmalloc(sizeof(struct page *) * page_count, GFP_KERNEL);
if (!pages) {
- pr_err("%s: Failed to allocate array for %u pages\n", __func__,
- page_count);
+ pr_err("%s: Failed to allocate array for %u pages\n",
+ __func__, page_count);
return NULL;
}
@@ -462,19 +462,17 @@
if (prz->buffer->sig == sig) {
if (buffer_size(prz) > prz->buffer_size ||
buffer_start(prz) > buffer_size(prz))
- pr_info("persistent_ram: found existing invalid buffer,"
- " size %zu, start %zu\n",
- buffer_size(prz), buffer_start(prz));
+ pr_info("found existing invalid buffer, size %zu, start %zu\n",
+ buffer_size(prz), buffer_start(prz));
else {
- pr_debug("persistent_ram: found existing buffer,"
- " size %zu, start %zu\n",
- buffer_size(prz), buffer_start(prz));
+ pr_debug("found existing buffer, size %zu, start %zu\n",
+ buffer_size(prz), buffer_start(prz));
persistent_ram_save_old(prz);
return 0;
}
} else {
- pr_debug("persistent_ram: no valid data in buffer"
- " (sig = 0x%08x)\n", prz->buffer->sig);
+ pr_debug("no valid data in buffer (sig = 0x%08x)\n",
+ prz->buffer->sig);
}
prz->buffer->sig = sig;
@@ -509,7 +507,7 @@
prz = kzalloc(sizeof(struct persistent_ram_zone), GFP_KERNEL);
if (!prz) {
- pr_err("persistent_ram: failed to allocate persistent ram zone\n");
+ pr_err("failed to allocate persistent ram zone\n");
goto err;
}
diff --git a/fs/reiserfs/bitmap.c b/fs/reiserfs/bitmap.c
index dc9a682..1bcffea 100644
--- a/fs/reiserfs/bitmap.c
+++ b/fs/reiserfs/bitmap.c
@@ -142,7 +142,6 @@
int org = *beg;
BUG_ON(!th->t_trans_id);
-
RFALSE(bmap_n >= reiserfs_bmap_count(s), "Bitmap %u is out of "
"range (0..%u)", bmap_n, reiserfs_bmap_count(s) - 1);
PROC_INFO_INC(s, scan_bitmap.bmap);
@@ -321,7 +320,6 @@
unsigned int off_max = s->s_blocksize << 3;
BUG_ON(!th->t_trans_id);
-
PROC_INFO_INC(s, scan_bitmap.call);
if (SB_FREE_BLOCKS(s) <= 0)
return 0; // No point in looking for more free blocks
@@ -388,9 +386,7 @@
unsigned int nr, offset;
BUG_ON(!th->t_trans_id);
-
PROC_INFO_INC(s, free_block);
-
rs = SB_DISK_SUPER_BLOCK(s);
sbh = SB_BUFFER_WITH_SB(s);
apbi = SB_AP_BITMAP(s);
@@ -435,8 +431,8 @@
int for_unformatted)
{
struct super_block *s = th->t_super;
- BUG_ON(!th->t_trans_id);
+ BUG_ON(!th->t_trans_id);
RFALSE(!s, "vs-4061: trying to free block on nonexistent device");
if (!is_reusable(s, block, 1))
return;
@@ -471,6 +467,7 @@
unsigned long save = ei->i_prealloc_block;
int dirty = 0;
struct inode *inode = &ei->vfs_inode;
+
BUG_ON(!th->t_trans_id);
#ifdef CONFIG_REISERFS_CHECK
if (ei->i_prealloc_count < 0)
@@ -494,6 +491,7 @@
struct inode *inode)
{
struct reiserfs_inode_info *ei = REISERFS_I(inode);
+
BUG_ON(!th->t_trans_id);
if (ei->i_prealloc_count)
__discard_prealloc(th, ei);
@@ -504,7 +502,6 @@
struct list_head *plist = &SB_JOURNAL(th->t_super)->j_prealloc_list;
BUG_ON(!th->t_trans_id);
-
while (!list_empty(plist)) {
struct reiserfs_inode_info *ei;
ei = list_entry(plist->next, struct reiserfs_inode_info,
@@ -562,7 +559,7 @@
if (!strcmp(this_char, "displacing_new_packing_localities")) {
SET_OPTION(displacing_new_packing_localities);
continue;
- };
+ }
if (!strcmp(this_char, "old_hashed_relocation")) {
SET_OPTION(old_hashed_relocation);
@@ -729,6 +726,7 @@
static inline void new_hashed_relocation(reiserfs_blocknr_hint_t * hint)
{
char *hash_in;
+
if (hint->formatted_node) {
hash_in = (char *)&hint->key.k_dir_id;
} else {
@@ -757,6 +755,7 @@
__u32 dirid = 0;
int bm = 0;
struct super_block *sb = hint->th->t_super;
+
if (hint->inode)
dirid = le32_to_cpu(INODE_PKEY(hint->inode)->k_dir_id);
else if (hint->formatted_node)
diff --git a/fs/reiserfs/stree.c b/fs/reiserfs/stree.c
index b14706a..615cd9a 100644
--- a/fs/reiserfs/stree.c
+++ b/fs/reiserfs/stree.c
@@ -228,10 +228,10 @@
/* Maximal possible key. It is never in the tree. */
static const struct reiserfs_key MAX_KEY = {
- __constant_cpu_to_le32(0xffffffff),
- __constant_cpu_to_le32(0xffffffff),
- {{__constant_cpu_to_le32(0xffffffff),
- __constant_cpu_to_le32(0xffffffff)},}
+ cpu_to_le32(0xffffffff),
+ cpu_to_le32(0xffffffff),
+ {{cpu_to_le32(0xffffffff),
+ cpu_to_le32(0xffffffff)},}
};
/* Get delimiting key of the buffer by looking for it in the buffers in the path, starting from the bottom
diff --git a/fs/ubifs/budget.c b/fs/ubifs/budget.c
index e8e01d7..eb997e9 100644
--- a/fs/ubifs/budget.c
+++ b/fs/ubifs/budget.c
@@ -437,7 +437,6 @@
*/
int ubifs_budget_space(struct ubifs_info *c, struct ubifs_budget_req *req)
{
- int uninitialized_var(cmt_retries), uninitialized_var(wb_retries);
int err, idx_growth, data_growth, dd_growth, retried = 0;
ubifs_assert(req->new_page <= 1);
diff --git a/fs/ubifs/debug.c b/fs/ubifs/debug.c
index 5157b86..177b015 100644
--- a/fs/ubifs/debug.c
+++ b/fs/ubifs/debug.c
@@ -745,8 +745,10 @@
for (lnum = c->main_first; lnum < c->leb_cnt; lnum++) {
err = ubifs_read_one_lp(c, lnum, &lp);
- if (err)
+ if (err) {
ubifs_err("cannot read lprops for LEB %d", lnum);
+ continue;
+ }
ubifs_dump_lprop(c, &lp);
}
diff --git a/fs/ubifs/file.c b/fs/ubifs/file.c
index 4f34dba..0ab7f7d 100644
--- a/fs/ubifs/file.c
+++ b/fs/ubifs/file.c
@@ -903,8 +903,9 @@
struct ubifs_info *c = inode->i_sb->s_fs_info;
#ifdef UBIFS_DEBUG
+ struct ubifs_inode *ui = ubifs_inode(inode);
spin_lock(&ui->ui_lock);
- ubifs_assert(page->index <= ui->synced_i_size << PAGE_CACHE_SIZE);
+ ubifs_assert(page->index <= ui->synced_i_size >> PAGE_CACHE_SHIFT);
spin_unlock(&ui->ui_lock);
#endif
@@ -1525,8 +1526,7 @@
}
wait_for_stable_page(page);
- unlock_page(page);
- return 0;
+ return VM_FAULT_LOCKED;
out_unlock:
unlock_page(page);
diff --git a/fs/ubifs/io.c b/fs/ubifs/io.c
index e18b988..2290d58 100644
--- a/fs/ubifs/io.c
+++ b/fs/ubifs/io.c
@@ -988,30 +988,32 @@
return err;
if (type != ch->node_type) {
- ubifs_err("bad node type (%d but expected %d)",
- ch->node_type, type);
+ ubifs_errc(c, "bad node type (%d but expected %d)",
+ ch->node_type, type);
goto out;
}
err = ubifs_check_node(c, buf, lnum, offs, 0, 0);
if (err) {
- ubifs_err("expected node type %d", type);
+ ubifs_errc(c, "expected node type %d", type);
return err;
}
l = le32_to_cpu(ch->len);
if (l != len) {
- ubifs_err("bad node length %d, expected %d", l, len);
+ ubifs_errc(c, "bad node length %d, expected %d", l, len);
goto out;
}
return 0;
out:
- ubifs_err("bad node at LEB %d:%d, LEB mapping status %d", lnum, offs,
- ubi_is_mapped(c->ubi, lnum));
- ubifs_dump_node(c, buf);
- dump_stack();
+ ubifs_errc(c, "bad node at LEB %d:%d, LEB mapping status %d", lnum,
+ offs, ubi_is_mapped(c->ubi, lnum));
+ if (!c->probing) {
+ ubifs_dump_node(c, buf);
+ dump_stack();
+ }
return -EINVAL;
}
diff --git a/fs/ubifs/shrinker.c b/fs/ubifs/shrinker.c
index f35135e..9a9fb94a 100644
--- a/fs/ubifs/shrinker.c
+++ b/fs/ubifs/shrinker.c
@@ -128,7 +128,6 @@
freed = ubifs_destroy_tnc_subtree(znode);
atomic_long_sub(freed, &ubifs_clean_zn_cnt);
atomic_long_sub(freed, &c->clean_zn_cnt);
- ubifs_assert(atomic_long_read(&c->clean_zn_cnt) >= 0);
total_freed += freed;
znode = zprev;
}
diff --git a/fs/ubifs/super.c b/fs/ubifs/super.c
index a81c7b5..3904c85 100644
--- a/fs/ubifs/super.c
+++ b/fs/ubifs/super.c
@@ -1149,6 +1149,9 @@
size_t sz;
c->ro_mount = !!(c->vfs_sb->s_flags & MS_RDONLY);
+ /* Suppress error messages while probing if MS_SILENT is set */
+ c->probing = !!(c->vfs_sb->s_flags & MS_SILENT);
+
err = init_constants_early(c);
if (err)
return err;
@@ -1214,6 +1217,8 @@
if (err)
goto out_free;
+ c->probing = 0;
+
/*
* Make sure the compressor which is set as default in the superblock
* or overridden by mount options is actually compiled in.
diff --git a/fs/ubifs/tnc.c b/fs/ubifs/tnc.c
index 9083bc7..8a40cf9 100644
--- a/fs/ubifs/tnc.c
+++ b/fs/ubifs/tnc.c
@@ -2859,10 +2859,11 @@
{
tnc_destroy_cnext(c);
if (c->zroot.znode) {
- long n;
+ long n, freed;
- ubifs_destroy_tnc_subtree(c->zroot.znode);
n = atomic_long_read(&c->clean_zn_cnt);
+ freed = ubifs_destroy_tnc_subtree(c->zroot.znode);
+ ubifs_assert(freed == n);
atomic_long_sub(n, &ubifs_clean_zn_cnt);
}
kfree(c->gap_lebs);
diff --git a/fs/ubifs/ubifs.h b/fs/ubifs/ubifs.h
index e8c8cfe..c1f71fe 100644
--- a/fs/ubifs/ubifs.h
+++ b/fs/ubifs/ubifs.h
@@ -51,6 +51,15 @@
#define ubifs_warn(fmt, ...) \
pr_warn("UBIFS warning (pid %d): %s: " fmt "\n", \
current->pid, __func__, ##__VA_ARGS__)
+/*
+ * A variant of 'ubifs_err()' which takes the UBIFS file-sytem description
+ * object as an argument.
+ */
+#define ubifs_errc(c, fmt, ...) \
+ do { \
+ if (!(c)->probing) \
+ ubifs_err(fmt, ##__VA_ARGS__); \
+ } while (0)
/* UBIFS file system VFS magic number */
#define UBIFS_SUPER_MAGIC 0x24051905
@@ -1209,6 +1218,7 @@
* @need_recovery: %1 if the file-system needs recovery
* @replaying: %1 during journal replay
* @mounting: %1 while mounting
+ * @probing: %1 while attempting to mount if MS_SILENT mount flag is set
* @remounting_rw: %1 while re-mounting from R/O mode to R/W mode
* @replay_list: temporary list used during journal replay
* @replay_buds: list of buds to replay
@@ -1441,6 +1451,7 @@
unsigned int replaying:1;
unsigned int mounting:1;
unsigned int remounting_rw:1;
+ unsigned int probing:1;
struct list_head replay_list;
struct list_head replay_buds;
unsigned long long cs_sqnum;
diff --git a/fs/ufs/balloc.c b/fs/ufs/balloc.c
index 0ab1de4..7bc2080 100644
--- a/fs/ufs/balloc.c
+++ b/fs/ufs/balloc.c
@@ -24,7 +24,7 @@
#define INVBLOCK ((u64)-1L)
-static u64 ufs_add_fragments(struct inode *, u64, unsigned, unsigned, int *);
+static u64 ufs_add_fragments(struct inode *, u64, unsigned, unsigned);
static u64 ufs_alloc_fragments(struct inode *, unsigned, u64, unsigned, int *);
static u64 ufs_alloccg_block(struct inode *, struct ufs_cg_private_info *, u64, int *);
static u64 ufs_bitmap_search (struct super_block *, struct ufs_cg_private_info *, u64, unsigned);
@@ -52,7 +52,7 @@
if (ufs_fragnum(fragment) + count > uspi->s_fpg)
ufs_error (sb, "ufs_free_fragments", "internal error");
- mutex_lock(&UFS_SB(sb)->s_lock);
+ lock_ufs(sb);
cgno = ufs_dtog(uspi, fragment);
bit = ufs_dtogd(uspi, fragment);
@@ -116,12 +116,12 @@
ubh_sync_block(UCPI_UBH(ucpi));
ufs_mark_sb_dirty(sb);
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
UFSD("EXIT\n");
return;
failed:
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
UFSD("EXIT (FAILED)\n");
return;
}
@@ -151,7 +151,7 @@
goto failed;
}
- mutex_lock(&UFS_SB(sb)->s_lock);
+ lock_ufs(sb);
do_more:
overflow = 0;
@@ -211,12 +211,12 @@
}
ufs_mark_sb_dirty(sb);
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
UFSD("EXIT\n");
return;
failed_unlock:
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
failed:
UFSD("EXIT (FAILED)\n");
return;
@@ -357,7 +357,7 @@
usb1 = ubh_get_usb_first(uspi);
*err = -ENOSPC;
- mutex_lock(&UFS_SB(sb)->s_lock);
+ lock_ufs(sb);
tmp = ufs_data_ptr_to_cpu(sb, p);
if (count + ufs_fragnum(fragment) > uspi->s_fpb) {
@@ -378,19 +378,19 @@
"fragment %llu, tmp %llu\n",
(unsigned long long)fragment,
(unsigned long long)tmp);
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
return INVBLOCK;
}
if (fragment < UFS_I(inode)->i_lastfrag) {
UFSD("EXIT (ALREADY ALLOCATED)\n");
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
return 0;
}
}
else {
if (tmp) {
UFSD("EXIT (ALREADY ALLOCATED)\n");
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
return 0;
}
}
@@ -399,7 +399,7 @@
* There is not enough space for user on the device
*/
if (!capable(CAP_SYS_RESOURCE) && ufs_freespace(uspi, UFS_MINFREE) <= 0) {
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
UFSD("EXIT (FAILED)\n");
return 0;
}
@@ -424,7 +424,7 @@
ufs_clear_frags(inode, result + oldcount,
newcount - oldcount, locked_page != NULL);
}
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
UFSD("EXIT, result %llu\n", (unsigned long long)result);
return result;
}
@@ -432,14 +432,14 @@
/*
* resize block
*/
- result = ufs_add_fragments (inode, tmp, oldcount, newcount, err);
+ result = ufs_add_fragments(inode, tmp, oldcount, newcount);
if (result) {
*err = 0;
UFS_I(inode)->i_lastfrag = max(UFS_I(inode)->i_lastfrag,
fragment + count);
ufs_clear_frags(inode, result + oldcount, newcount - oldcount,
locked_page != NULL);
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
UFSD("EXIT, result %llu\n", (unsigned long long)result);
return result;
}
@@ -477,7 +477,7 @@
*err = 0;
UFS_I(inode)->i_lastfrag = max(UFS_I(inode)->i_lastfrag,
fragment + count);
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
if (newcount < request)
ufs_free_fragments (inode, result + newcount, request - newcount);
ufs_free_fragments (inode, tmp, oldcount);
@@ -485,13 +485,13 @@
return result;
}
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
UFSD("EXIT (FAILED)\n");
return 0;
}
static u64 ufs_add_fragments(struct inode *inode, u64 fragment,
- unsigned oldcount, unsigned newcount, int *err)
+ unsigned oldcount, unsigned newcount)
{
struct super_block * sb;
struct ufs_sb_private_info * uspi;
diff --git a/fs/ufs/ialloc.c b/fs/ufs/ialloc.c
index 98f7211..a9cc75f 100644
--- a/fs/ufs/ialloc.c
+++ b/fs/ufs/ialloc.c
@@ -69,11 +69,11 @@
ino = inode->i_ino;
- mutex_lock(&UFS_SB(sb)->s_lock);
+ lock_ufs(sb);
if (!((ino > 1) && (ino < (uspi->s_ncg * uspi->s_ipg )))) {
ufs_warning(sb, "ufs_free_inode", "reserved inode or nonexistent inode %u\n", ino);
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
return;
}
@@ -81,7 +81,7 @@
bit = ufs_inotocgoff (ino);
ucpi = ufs_load_cylinder (sb, cg);
if (!ucpi) {
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
return;
}
ucg = ubh_get_ucg(UCPI_UBH(ucpi));
@@ -115,7 +115,7 @@
ubh_sync_block(UCPI_UBH(ucpi));
ufs_mark_sb_dirty(sb);
- mutex_unlock(&UFS_SB(sb)->s_lock);
+ unlock_ufs(sb);
UFSD("EXIT\n");
}
@@ -193,7 +193,7 @@
sbi = UFS_SB(sb);
uspi = sbi->s_uspi;
- mutex_lock(&sbi->s_lock);
+ lock_ufs(sb);
/*
* Try to place the inode in its parent directory
@@ -328,21 +328,20 @@
sync_dirty_buffer(bh);
brelse(bh);
}
-
- mutex_unlock(&sbi->s_lock);
+ unlock_ufs(sb);
UFSD("allocating inode %lu\n", inode->i_ino);
UFSD("EXIT\n");
return inode;
fail_remove_inode:
- mutex_unlock(&sbi->s_lock);
+ unlock_ufs(sb);
clear_nlink(inode);
iput(inode);
UFSD("EXIT (FAILED): err %d\n", err);
return ERR_PTR(err);
failed:
- mutex_unlock(&sbi->s_lock);
+ unlock_ufs(sb);
make_bad_inode(inode);
iput (inode);
UFSD("EXIT (FAILED): err %d\n", err);
diff --git a/fs/ufs/super.c b/fs/ufs/super.c
index c1183f9..b879f1b 100644
--- a/fs/ufs/super.c
+++ b/fs/ufs/super.c
@@ -697,7 +697,6 @@
unsigned flags;
lock_ufs(sb);
- mutex_lock(&UFS_SB(sb)->s_lock);
UFSD("ENTER\n");
@@ -715,7 +714,6 @@
ufs_put_cstotal(sb);
UFSD("EXIT\n");
- mutex_unlock(&UFS_SB(sb)->s_lock);
unlock_ufs(sb);
return 0;
@@ -760,6 +758,7 @@
ubh_brelse_uspi (sbi->s_uspi);
kfree (sbi->s_uspi);
+ mutex_destroy(&sbi->mutex);
kfree (sbi);
sb->s_fs_info = NULL;
UFSD("EXIT\n");
@@ -786,6 +785,14 @@
flags = 0;
UFSD("ENTER\n");
+
+#ifndef CONFIG_UFS_FS_WRITE
+ if (!(sb->s_flags & MS_RDONLY)) {
+ printk("ufs was compiled with read-only support, "
+ "can't be mounted as read-write\n");
+ return -EROFS;
+ }
+#endif
sbi = kzalloc(sizeof(struct ufs_sb_info), GFP_KERNEL);
if (!sbi)
@@ -795,15 +802,7 @@
UFSD("flag %u\n", (int)(sb->s_flags & MS_RDONLY));
-#ifndef CONFIG_UFS_FS_WRITE
- if (!(sb->s_flags & MS_RDONLY)) {
- printk("ufs was compiled with read-only support, "
- "can't be mounted as read-write\n");
- goto failed;
- }
-#endif
mutex_init(&sbi->mutex);
- mutex_init(&sbi->s_lock);
spin_lock_init(&sbi->work_lock);
INIT_DELAYED_WORK(&sbi->sync_work, delayed_sync_fs);
/*
@@ -1257,6 +1256,7 @@
return 0;
failed:
+ mutex_destroy(&sbi->mutex);
if (ubh)
ubh_brelse_uspi (uspi);
kfree (uspi);
@@ -1280,7 +1280,6 @@
sync_filesystem(sb);
lock_ufs(sb);
- mutex_lock(&UFS_SB(sb)->s_lock);
uspi = UFS_SB(sb)->s_uspi;
flags = UFS_SB(sb)->s_flags;
usb1 = ubh_get_usb_first(uspi);
@@ -1294,7 +1293,6 @@
new_mount_opt = 0;
ufs_set_opt (new_mount_opt, ONERROR_LOCK);
if (!ufs_parse_options (data, &new_mount_opt)) {
- mutex_unlock(&UFS_SB(sb)->s_lock);
unlock_ufs(sb);
return -EINVAL;
}
@@ -1302,14 +1300,12 @@
new_mount_opt |= ufstype;
} else if ((new_mount_opt & UFS_MOUNT_UFSTYPE) != ufstype) {
printk("ufstype can't be changed during remount\n");
- mutex_unlock(&UFS_SB(sb)->s_lock);
unlock_ufs(sb);
return -EINVAL;
}
if ((*mount_flags & MS_RDONLY) == (sb->s_flags & MS_RDONLY)) {
UFS_SB(sb)->s_mount_opt = new_mount_opt;
- mutex_unlock(&UFS_SB(sb)->s_lock);
unlock_ufs(sb);
return 0;
}
@@ -1334,7 +1330,6 @@
#ifndef CONFIG_UFS_FS_WRITE
printk("ufs was compiled with read-only support, "
"can't be mounted as read-write\n");
- mutex_unlock(&UFS_SB(sb)->s_lock);
unlock_ufs(sb);
return -EINVAL;
#else
@@ -1344,13 +1339,11 @@
ufstype != UFS_MOUNT_UFSTYPE_SUNx86 &&
ufstype != UFS_MOUNT_UFSTYPE_UFS2) {
printk("this ufstype is read-only supported\n");
- mutex_unlock(&UFS_SB(sb)->s_lock);
unlock_ufs(sb);
return -EINVAL;
}
if (!ufs_read_cylinder_structures(sb)) {
printk("failed during remounting\n");
- mutex_unlock(&UFS_SB(sb)->s_lock);
unlock_ufs(sb);
return -EPERM;
}
@@ -1358,7 +1351,6 @@
#endif
}
UFS_SB(sb)->s_mount_opt = new_mount_opt;
- mutex_unlock(&UFS_SB(sb)->s_lock);
unlock_ufs(sb);
return 0;
}
diff --git a/fs/ufs/ufs.h b/fs/ufs/ufs.h
index ff2c15a..343e6fc 100644
--- a/fs/ufs/ufs.h
+++ b/fs/ufs/ufs.h
@@ -24,7 +24,6 @@
int work_queued; /* non-zero if the delayed work is queued */
struct delayed_work sync_work; /* FS sync delayed work */
spinlock_t work_lock; /* protects sync_work and work_queued */
- struct mutex s_lock;
};
struct ufs_inode_info {
diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_ioctl.c
index 0b18776..6152cbe 100644
--- a/fs/xfs/xfs_ioctl.c
+++ b/fs/xfs/xfs_ioctl.c
@@ -1215,7 +1215,7 @@
* cleared upon successful return from chown()
*/
if ((ip->i_d.di_mode & (S_ISUID|S_ISGID)) &&
- !inode_capable(VFS_I(ip), CAP_FSETID))
+ !capable_wrt_inode_uidgid(VFS_I(ip), CAP_FSETID))
ip->i_d.di_mode &= ~(S_ISUID|S_ISGID);
/*
diff --git a/include/asm-generic/ioctl.h b/include/asm-generic/ioctl.h
index d17295b..297fb0d 100644
--- a/include/asm-generic/ioctl.h
+++ b/include/asm-generic/ioctl.h
@@ -3,10 +3,15 @@
#include <uapi/asm-generic/ioctl.h>
+#ifdef __CHECKER__
+#define _IOC_TYPECHECK(t) (sizeof(t))
+#else
/* provoke compile error for invalid uses of size argument */
extern unsigned int __invalid_size_argument_for_IOC;
#define _IOC_TYPECHECK(t) \
((sizeof(t) == sizeof(t[1]) && \
sizeof(t) < (1 << _IOC_SIZEBITS)) ? \
sizeof(t) : __invalid_size_argument_for_IOC)
+#endif
+
#endif /* _ASM_GENERIC_IOCTL_H */
diff --git a/include/asm-generic/unaligned.h b/include/asm-generic/unaligned.h
index 03cf593..1ac0972 100644
--- a/include/asm-generic/unaligned.h
+++ b/include/asm-generic/unaligned.h
@@ -4,22 +4,27 @@
/*
* This is the most generic implementation of unaligned accesses
* and should work almost anywhere.
- *
- * If an architecture can handle unaligned accesses in hardware,
- * it may want to use the linux/unaligned/access_ok.h implementation
- * instead.
*/
#include <asm/byteorder.h>
+/* Set by the arch if it can handle unaligned accesses in hardware. */
+#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+# include <linux/unaligned/access_ok.h>
+#endif
+
#if defined(__LITTLE_ENDIAN)
-# include <linux/unaligned/le_struct.h>
-# include <linux/unaligned/be_byteshift.h>
+# ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+# include <linux/unaligned/le_struct.h>
+# include <linux/unaligned/be_byteshift.h>
+# endif
# include <linux/unaligned/generic.h>
# define get_unaligned __get_unaligned_le
# define put_unaligned __put_unaligned_le
#elif defined(__BIG_ENDIAN)
-# include <linux/unaligned/be_struct.h>
-# include <linux/unaligned/le_byteshift.h>
+# ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+# include <linux/unaligned/be_struct.h>
+# include <linux/unaligned/le_byteshift.h>
+# endif
# include <linux/unaligned/generic.h>
# define get_unaligned __get_unaligned_be
# define put_unaligned __put_unaligned_be
diff --git a/include/crypto/internal/hash.h b/include/crypto/internal/hash.h
index 821eae8..9b6f32a 100644
--- a/include/crypto/internal/hash.h
+++ b/include/crypto/internal/hash.h
@@ -55,15 +55,28 @@
int crypto_hash_walk_done(struct crypto_hash_walk *walk, int err);
int crypto_hash_walk_first(struct ahash_request *req,
struct crypto_hash_walk *walk);
+int crypto_ahash_walk_first(struct ahash_request *req,
+ struct crypto_hash_walk *walk);
int crypto_hash_walk_first_compat(struct hash_desc *hdesc,
struct crypto_hash_walk *walk,
struct scatterlist *sg, unsigned int len);
+static inline int crypto_ahash_walk_done(struct crypto_hash_walk *walk,
+ int err)
+{
+ return crypto_hash_walk_done(walk, err);
+}
+
static inline int crypto_hash_walk_last(struct crypto_hash_walk *walk)
{
return !(walk->entrylen | walk->total);
}
+static inline int crypto_ahash_walk_last(struct crypto_hash_walk *walk)
+{
+ return crypto_hash_walk_last(walk);
+}
+
int crypto_register_ahash(struct ahash_alg *alg);
int crypto_unregister_ahash(struct ahash_alg *alg);
int ahash_register_instance(struct crypto_template *tmpl,
diff --git a/include/dt-bindings/clock/bcm21664.h b/include/dt-bindings/clock/bcm21664.h
new file mode 100644
index 0000000..5a7f0e4
--- /dev/null
+++ b/include/dt-bindings/clock/bcm21664.h
@@ -0,0 +1,62 @@
+/*
+ * Copyright (C) 2013 Broadcom Corporation
+ * Copyright 2013 Linaro Limited
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _CLOCK_BCM21664_H
+#define _CLOCK_BCM21664_H
+
+/*
+ * This file defines the values used to specify clocks provided by
+ * the clock control units (CCUs) on Broadcom BCM21664 family SoCs.
+ */
+
+/* bcm21664 CCU device tree "compatible" strings */
+#define BCM21664_DT_ROOT_CCU_COMPAT "brcm,bcm21664-root-ccu"
+#define BCM21664_DT_AON_CCU_COMPAT "brcm,bcm21664-aon-ccu"
+#define BCM21664_DT_MASTER_CCU_COMPAT "brcm,bcm21664-master-ccu"
+#define BCM21664_DT_SLAVE_CCU_COMPAT "brcm,bcm21664-slave-ccu"
+
+/* root CCU clock ids */
+
+#define BCM21664_ROOT_CCU_FRAC_1M 0
+#define BCM21664_ROOT_CCU_CLOCK_COUNT 1
+
+/* aon CCU clock ids */
+
+#define BCM21664_AON_CCU_HUB_TIMER 0
+#define BCM21664_AON_CCU_CLOCK_COUNT 1
+
+/* master CCU clock ids */
+
+#define BCM21664_MASTER_CCU_SDIO1 0
+#define BCM21664_MASTER_CCU_SDIO2 1
+#define BCM21664_MASTER_CCU_SDIO3 2
+#define BCM21664_MASTER_CCU_SDIO4 3
+#define BCM21664_MASTER_CCU_SDIO1_SLEEP 4
+#define BCM21664_MASTER_CCU_SDIO2_SLEEP 5
+#define BCM21664_MASTER_CCU_SDIO3_SLEEP 6
+#define BCM21664_MASTER_CCU_SDIO4_SLEEP 7
+#define BCM21664_MASTER_CCU_CLOCK_COUNT 8
+
+/* slave CCU clock ids */
+
+#define BCM21664_SLAVE_CCU_UARTB 0
+#define BCM21664_SLAVE_CCU_UARTB2 1
+#define BCM21664_SLAVE_CCU_UARTB3 2
+#define BCM21664_SLAVE_CCU_BSC1 3
+#define BCM21664_SLAVE_CCU_BSC2 4
+#define BCM21664_SLAVE_CCU_BSC3 5
+#define BCM21664_SLAVE_CCU_BSC4 6
+#define BCM21664_SLAVE_CCU_CLOCK_COUNT 7
+
+#endif /* _CLOCK_BCM21664_H */
diff --git a/include/dt-bindings/clock/bcm281xx.h b/include/dt-bindings/clock/bcm281xx.h
index e009694..a763460 100644
--- a/include/dt-bindings/clock/bcm281xx.h
+++ b/include/dt-bindings/clock/bcm281xx.h
@@ -20,6 +20,18 @@
* the clock control units (CCUs) on Broadcom BCM281XX family SoCs.
*/
+/*
+ * These are the bcm281xx CCU device tree "compatible" strings.
+ * We're stuck with using "bcm11351" in the string because wild
+ * cards aren't allowed, and that name was the first one defined
+ * in this family of devices.
+ */
+#define BCM281XX_DT_ROOT_CCU_COMPAT "brcm,bcm11351-root-ccu"
+#define BCM281XX_DT_AON_CCU_COMPAT "brcm,bcm11351-aon-ccu"
+#define BCM281XX_DT_HUB_CCU_COMPAT "brcm,bcm11351-hub-ccu"
+#define BCM281XX_DT_MASTER_CCU_COMPAT "brcm,bcm11351-master-ccu"
+#define BCM281XX_DT_SLAVE_CCU_COMPAT "brcm,bcm11351-slave-ccu"
+
/* root CCU clock ids */
#define BCM281XX_ROOT_CCU_FRAC_1M 0
diff --git a/include/dt-bindings/clock/hix5hd2-clock.h b/include/dt-bindings/clock/hix5hd2-clock.h
new file mode 100644
index 0000000..aad579a
--- /dev/null
+++ b/include/dt-bindings/clock/hix5hd2-clock.h
@@ -0,0 +1,58 @@
+/*
+ * Copyright (c) 2014 Linaro Ltd.
+ * Copyright (c) 2014 Hisilicon Limited.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ */
+
+#ifndef __DTS_HIX5HD2_CLOCK_H
+#define __DTS_HIX5HD2_CLOCK_H
+
+/* fixed rate */
+#define HIX5HD2_FIXED_1200M 1
+#define HIX5HD2_FIXED_400M 2
+#define HIX5HD2_FIXED_48M 3
+#define HIX5HD2_FIXED_24M 4
+#define HIX5HD2_FIXED_600M 5
+#define HIX5HD2_FIXED_300M 6
+#define HIX5HD2_FIXED_75M 7
+#define HIX5HD2_FIXED_200M 8
+#define HIX5HD2_FIXED_100M 9
+#define HIX5HD2_FIXED_40M 10
+#define HIX5HD2_FIXED_150M 11
+#define HIX5HD2_FIXED_1728M 12
+#define HIX5HD2_FIXED_28P8M 13
+#define HIX5HD2_FIXED_432M 14
+#define HIX5HD2_FIXED_345P6M 15
+#define HIX5HD2_FIXED_288M 16
+#define HIX5HD2_FIXED_60M 17
+#define HIX5HD2_FIXED_750M 18
+#define HIX5HD2_FIXED_500M 19
+#define HIX5HD2_FIXED_54M 20
+#define HIX5HD2_FIXED_27M 21
+#define HIX5HD2_FIXED_1500M 22
+#define HIX5HD2_FIXED_375M 23
+#define HIX5HD2_FIXED_187M 24
+#define HIX5HD2_FIXED_250M 25
+#define HIX5HD2_FIXED_125M 26
+#define HIX5HD2_FIXED_2P02M 27
+#define HIX5HD2_FIXED_50M 28
+#define HIX5HD2_FIXED_25M 29
+#define HIX5HD2_FIXED_83M 30
+
+/* mux clocks */
+#define HIX5HD2_SFC_MUX 64
+#define HIX5HD2_MMC_MUX 65
+#define HIX5HD2_FEPHY_MUX 66
+
+/* gate clocks */
+#define HIX5HD2_SFC_RST 128
+#define HIX5HD2_SFC_CLK 129
+#define HIX5HD2_MMC_CIU_CLK 130
+#define HIX5HD2_MMC_BIU_CLK 131
+#define HIX5HD2_MMC_CIU_RST 132
+
+#define HIX5HD2_NR_CLKS 256
+#endif /* __DTS_HIX5HD2_CLOCK_H */
diff --git a/include/dt-bindings/clock/qcom,gcc-msm8960.h b/include/dt-bindings/clock/qcom,gcc-msm8960.h
index 03bbf49..f9f5471 100644
--- a/include/dt-bindings/clock/qcom,gcc-msm8960.h
+++ b/include/dt-bindings/clock/qcom,gcc-msm8960.h
@@ -51,7 +51,7 @@
#define QDSS_TSCTR_CLK 34
#define SFAB_ADM0_M0_A_CLK 35
#define SFAB_ADM0_M1_A_CLK 36
-#define SFAB_ADM0_M2_A_CLK 37
+#define SFAB_ADM0_M2_H_CLK 37
#define ADM0_CLK 38
#define ADM0_PBUS_CLK 39
#define MSS_XPU_CLK 40
@@ -99,7 +99,7 @@
#define CFPB2_H_CLK 82
#define SFAB_CFPB_M_H_CLK 83
#define CFPB_MASTER_H_CLK 84
-#define SFAB_CFPB_S_HCLK 85
+#define SFAB_CFPB_S_H_CLK 85
#define CFPB_SPLITTER_H_CLK 86
#define TSIF_H_CLK 87
#define TSIF_INACTIVITY_TIMERS_CLK 88
@@ -110,7 +110,6 @@
#define CE1_SLEEP_CLK 93
#define CE2_H_CLK 94
#define CE2_CORE_CLK 95
-#define CE2_SLEEP_CLK 96
#define SFPB_H_CLK_SRC 97
#define SFPB_H_CLK 98
#define SFAB_SFPB_M_H_CLK 99
@@ -252,7 +251,7 @@
#define MSS_S_H_CLK 235
#define MSS_CXO_SRC_CLK 236
#define SATA_H_CLK 237
-#define SATA_SRC_CLK 238
+#define SATA_CLK_SRC 238
#define SATA_RXOOB_CLK 239
#define SATA_PMALIVE_CLK 240
#define SATA_PHY_REF_CLK 241
diff --git a/include/dt-bindings/clock/qcom,gcc-msm8974.h b/include/dt-bindings/clock/qcom,gcc-msm8974.h
index 223ca17..51e51c8 100644
--- a/include/dt-bindings/clock/qcom,gcc-msm8974.h
+++ b/include/dt-bindings/clock/qcom,gcc-msm8974.h
@@ -316,5 +316,9 @@
#define GCC_CE2_CLK_SLEEP_ENA 299
#define GCC_CE2_AXI_CLK_SLEEP_ENA 300
#define GCC_CE2_AHB_CLK_SLEEP_ENA 301
+#define GPLL4 302
+#define GPLL4_VOTE 303
+#define GCC_SDCC1_CDCCAL_SLEEP_CLK 304
+#define GCC_SDCC1_CDCCAL_FF_CLK 305
#endif
diff --git a/include/dt-bindings/clock/r8a7779-clock.h b/include/dt-bindings/clock/r8a7779-clock.h
new file mode 100644
index 0000000..381a611
--- /dev/null
+++ b/include/dt-bindings/clock/r8a7779-clock.h
@@ -0,0 +1,64 @@
+/*
+ * Copyright (C) 2013 Horms Solutions Ltd.
+ *
+ * Contact: Simon Horman <horms@verge.net.au>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#ifndef __DT_BINDINGS_CLOCK_R8A7779_H__
+#define __DT_BINDINGS_CLOCK_R8A7779_H__
+
+/* CPG */
+#define R8A7779_CLK_PLLA 0
+#define R8A7779_CLK_Z 1
+#define R8A7779_CLK_ZS 2
+#define R8A7779_CLK_S 3
+#define R8A7779_CLK_S1 4
+#define R8A7779_CLK_P 5
+#define R8A7779_CLK_B 6
+#define R8A7779_CLK_OUT 7
+
+/* MSTP 0 */
+#define R8A7779_CLK_HSPI 7
+#define R8A7779_CLK_TMU2 14
+#define R8A7779_CLK_TMU1 15
+#define R8A7779_CLK_TMU0 16
+#define R8A7779_CLK_HSCIF1 18
+#define R8A7779_CLK_HSCIF0 19
+#define R8A7779_CLK_SCIF5 21
+#define R8A7779_CLK_SCIF4 22
+#define R8A7779_CLK_SCIF3 23
+#define R8A7779_CLK_SCIF2 24
+#define R8A7779_CLK_SCIF1 25
+#define R8A7779_CLK_SCIF0 26
+#define R8A7779_CLK_I2C3 27
+#define R8A7779_CLK_I2C2 28
+#define R8A7779_CLK_I2C1 29
+#define R8A7779_CLK_I2C0 30
+
+/* MSTP 1 */
+#define R8A7779_CLK_USB01 0
+#define R8A7779_CLK_USB2 1
+#define R8A7779_CLK_DU 3
+#define R8A7779_CLK_VIN2 8
+#define R8A7779_CLK_VIN1 9
+#define R8A7779_CLK_VIN0 10
+#define R8A7779_CLK_ETHER 14
+#define R8A7779_CLK_SATA 15
+#define R8A7779_CLK_PCIE 16
+#define R8A7779_CLK_VIN3 20
+
+/* MSTP 3 */
+#define R8A7779_CLK_SDHI3 20
+#define R8A7779_CLK_SDHI2 21
+#define R8A7779_CLK_SDHI1 22
+#define R8A7779_CLK_SDHI0 23
+#define R8A7779_CLK_MMC1 30
+#define R8A7779_CLK_MMC0 31
+
+
+#endif /* __DT_BINDINGS_CLOCK_R8A7779_H__ */
diff --git a/include/dt-bindings/clock/tegra114-car.h b/include/dt-bindings/clock/tegra114-car.h
index 6d0d8d8..fc12621 100644
--- a/include/dt-bindings/clock/tegra114-car.h
+++ b/include/dt-bindings/clock/tegra114-car.h
@@ -337,6 +337,7 @@
#define TEGRA114_CLK_CLK_OUT_3_MUX 308
#define TEGRA114_CLK_DSIA_MUX 309
#define TEGRA114_CLK_DSIB_MUX 310
-#define TEGRA114_CLK_CLK_MAX 311
+#define TEGRA114_CLK_XUSB_SS_DIV2 311
+#define TEGRA114_CLK_CLK_MAX 312
#endif /* _DT_BINDINGS_CLOCK_TEGRA114_CAR_H */
diff --git a/include/dt-bindings/clock/tegra124-car.h b/include/dt-bindings/clock/tegra124-car.h
index 433528a..8a4c589 100644
--- a/include/dt-bindings/clock/tegra124-car.h
+++ b/include/dt-bindings/clock/tegra124-car.h
@@ -336,6 +336,7 @@
#define TEGRA124_CLK_DSIA_MUX 309
#define TEGRA124_CLK_DSIB_MUX 310
#define TEGRA124_CLK_SOR0_LVDS 311
-#define TEGRA124_CLK_CLK_MAX 312
+#define TEGRA124_CLK_XUSB_SS_DIV2 312
+#define TEGRA124_CLK_CLK_MAX 313
#endif /* _DT_BINDINGS_CLOCK_TEGRA124_CAR_H */
diff --git a/include/dt-bindings/reset/qcom,gcc-msm8960.h b/include/dt-bindings/reset/qcom,gcc-msm8960.h
index a840e68..07edd0e 100644
--- a/include/dt-bindings/reset/qcom,gcc-msm8960.h
+++ b/include/dt-bindings/reset/qcom,gcc-msm8960.h
@@ -58,7 +58,7 @@
#define PPSS_PROC_RESET 41
#define PPSS_RESET 42
#define DMA_BAM_RESET 43
-#define SIC_TIC_RESET 44
+#define SPS_TIC_H_RESET 44
#define SLIMBUS_H_RESET 45
#define SFAB_CFPB_M_RESET 46
#define SFAB_CFPB_S_RESET 47
diff --git a/include/linux/ahci_platform.h b/include/linux/ahci_platform.h
index 1f16d50..6dfd51a 100644
--- a/include/linux/ahci_platform.h
+++ b/include/linux/ahci_platform.h
@@ -44,6 +44,7 @@
int ahci_platform_init_host(struct platform_device *pdev,
struct ahci_host_priv *hpriv,
const struct ata_port_info *pi_template,
+ unsigned long host_flags,
unsigned int force_port_map,
unsigned int mask_port_map);
diff --git a/include/linux/amba/xilinx_dma.h b/include/linux/amba/xilinx_dma.h
new file mode 100644
index 0000000..34b98f2
--- /dev/null
+++ b/include/linux/amba/xilinx_dma.h
@@ -0,0 +1,47 @@
+/*
+ * Xilinx DMA Engine drivers support header file
+ *
+ * Copyright (C) 2010-2014 Xilinx, Inc. All rights reserved.
+ *
+ * This is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#ifndef __DMA_XILINX_DMA_H
+#define __DMA_XILINX_DMA_H
+
+#include <linux/dma-mapping.h>
+#include <linux/dmaengine.h>
+
+/**
+ * struct xilinx_vdma_config - VDMA Configuration structure
+ * @frm_dly: Frame delay
+ * @gen_lock: Whether in gen-lock mode
+ * @master: Master that it syncs to
+ * @frm_cnt_en: Enable frame count enable
+ * @park: Whether wants to park
+ * @park_frm: Frame to park on
+ * @coalesc: Interrupt coalescing threshold
+ * @delay: Delay counter
+ * @reset: Reset Channel
+ * @ext_fsync: External Frame Sync source
+ */
+struct xilinx_vdma_config {
+ int frm_dly;
+ int gen_lock;
+ int master;
+ int frm_cnt_en;
+ int park;
+ int park_frm;
+ int coalesc;
+ int delay;
+ int reset;
+ int ext_fsync;
+};
+
+int xilinx_vdma_channel_set_config(struct dma_chan *dchan,
+ struct xilinx_vdma_config *cfg);
+
+#endif
diff --git a/include/linux/capability.h b/include/linux/capability.h
index a6ee1f9..84b13ad 100644
--- a/include/linux/capability.h
+++ b/include/linux/capability.h
@@ -210,7 +210,7 @@
struct user_namespace *ns, int cap);
extern bool capable(int cap);
extern bool ns_capable(struct user_namespace *ns, int cap);
-extern bool inode_capable(const struct inode *inode, int cap);
+extern bool capable_wrt_inode_uidgid(const struct inode *inode, int cap);
extern bool file_ns_capable(const struct file *file, struct user_namespace *ns, int cap);
/* audit system wants to get cap info from files as well */
diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h
index d60904b..8a111dd 100644
--- a/include/linux/cgroup.h
+++ b/include/linux/cgroup.h
@@ -21,6 +21,7 @@
#include <linux/percpu-refcount.h>
#include <linux/seq_file.h>
#include <linux/kernfs.h>
+#include <linux/wait.h>
#ifdef CONFIG_CGROUPS
@@ -47,21 +48,45 @@
};
#undef SUBSYS
-/* Per-subsystem/per-cgroup state maintained by the system. */
+/*
+ * Per-subsystem/per-cgroup state maintained by the system. This is the
+ * fundamental structural building block that controllers deal with.
+ *
+ * Fields marked with "PI:" are public and immutable and may be accessed
+ * directly without synchronization.
+ */
struct cgroup_subsys_state {
- /* the cgroup that this css is attached to */
+ /* PI: the cgroup that this css is attached to */
struct cgroup *cgroup;
- /* the cgroup subsystem that this css is attached to */
+ /* PI: the cgroup subsystem that this css is attached to */
struct cgroup_subsys *ss;
/* reference count - access via css_[try]get() and css_put() */
struct percpu_ref refcnt;
- /* the parent css */
+ /* PI: the parent css */
struct cgroup_subsys_state *parent;
- unsigned long flags;
+ /* siblings list anchored at the parent's ->children */
+ struct list_head sibling;
+ struct list_head children;
+
+ /*
+ * PI: Subsys-unique ID. 0 is unused and root is always 1. The
+ * matching css can be looked up using css_from_id().
+ */
+ int id;
+
+ unsigned int flags;
+
+ /*
+ * Monotonically increasing unique serial number which defines a
+ * uniform order among all csses. It's guaranteed that all
+ * ->children lists are in the ascending order of ->serial_nr and
+ * used to allow interrupting and resuming iterations.
+ */
+ u64 serial_nr;
/* percpu_ref killing and RCU release */
struct rcu_head rcu_head;
@@ -70,8 +95,9 @@
/* bits in struct cgroup_subsys_state flags field */
enum {
- CSS_ROOT = (1 << 0), /* this CSS is the root of the subsystem */
+ CSS_NO_REF = (1 << 0), /* no reference counting for this css */
CSS_ONLINE = (1 << 1), /* between ->css_online() and ->css_offline() */
+ CSS_RELEASED = (1 << 2), /* refcnt reached zero, released */
};
/**
@@ -82,8 +108,7 @@
*/
static inline void css_get(struct cgroup_subsys_state *css)
{
- /* We don't need to reference count the root state */
- if (!(css->flags & CSS_ROOT))
+ if (!(css->flags & CSS_NO_REF))
percpu_ref_get(&css->refcnt);
}
@@ -91,35 +116,51 @@
* css_tryget - try to obtain a reference on the specified css
* @css: target css
*
- * Obtain a reference on @css if it's alive. The caller naturally needs to
- * ensure that @css is accessible but doesn't have to be holding a
+ * Obtain a reference on @css unless it already has reached zero and is
+ * being released. This function doesn't care whether @css is on or
+ * offline. The caller naturally needs to ensure that @css is accessible
+ * but doesn't have to be holding a reference on it - IOW, RCU protected
+ * access is good enough for this function. Returns %true if a reference
+ * count was successfully obtained; %false otherwise.
+ */
+static inline bool css_tryget(struct cgroup_subsys_state *css)
+{
+ if (!(css->flags & CSS_NO_REF))
+ return percpu_ref_tryget(&css->refcnt);
+ return true;
+}
+
+/**
+ * css_tryget_online - try to obtain a reference on the specified css if online
+ * @css: target css
+ *
+ * Obtain a reference on @css if it's online. The caller naturally needs
+ * to ensure that @css is accessible but doesn't have to be holding a
* reference on it - IOW, RCU protected access is good enough for this
* function. Returns %true if a reference count was successfully obtained;
* %false otherwise.
*/
-static inline bool css_tryget(struct cgroup_subsys_state *css)
+static inline bool css_tryget_online(struct cgroup_subsys_state *css)
{
- if (css->flags & CSS_ROOT)
- return true;
- return percpu_ref_tryget(&css->refcnt);
+ if (!(css->flags & CSS_NO_REF))
+ return percpu_ref_tryget_live(&css->refcnt);
+ return true;
}
/**
* css_put - put a css reference
* @css: target css
*
- * Put a reference obtained via css_get() and css_tryget().
+ * Put a reference obtained via css_get() and css_tryget_online().
*/
static inline void css_put(struct cgroup_subsys_state *css)
{
- if (!(css->flags & CSS_ROOT))
+ if (!(css->flags & CSS_NO_REF))
percpu_ref_put(&css->refcnt);
}
/* bits in struct cgroup flags field */
enum {
- /* Control Group is dead */
- CGRP_DEAD,
/*
* Control Group has previously had a child cgroup or a task,
* but no longer (only if CGRP_NOTIFY_ON_RELEASE is set)
@@ -133,48 +174,37 @@
* specified at mount time and thus is implemented here.
*/
CGRP_CPUSET_CLONE_CHILDREN,
- /* see the comment above CGRP_ROOT_SANE_BEHAVIOR for details */
- CGRP_SANE_BEHAVIOR,
};
struct cgroup {
+ /* self css with NULL ->ss, points back to this cgroup */
+ struct cgroup_subsys_state self;
+
unsigned long flags; /* "unsigned long" so bitops work */
/*
* idr allocated in-hierarchy ID.
*
- * The ID of the root cgroup is always 0, and a new cgroup
- * will be assigned with a smallest available ID.
+ * ID 0 is not used, the ID of the root cgroup is always 1, and a
+ * new cgroup will be assigned with a smallest available ID.
*
* Allocating/Removing ID must be protected by cgroup_mutex.
*/
int id;
- /* the number of attached css's */
- int nr_css;
-
- atomic_t refcnt;
-
/*
- * We link our 'sibling' struct into our parent's 'children'.
- * Our children link their 'sibling' into our 'children'.
+ * If this cgroup contains any tasks, it contributes one to
+ * populated_cnt. All children with non-zero popuplated_cnt of
+ * their own contribute one. The count is zero iff there's no task
+ * in this cgroup or its subtree.
*/
- struct list_head sibling; /* my parent's children */
- struct list_head children; /* my children */
+ int populated_cnt;
- struct cgroup *parent; /* my parent */
struct kernfs_node *kn; /* cgroup kernfs entry */
+ struct kernfs_node *populated_kn; /* kn for "cgroup.subtree_populated" */
- /*
- * Monotonically increasing unique serial number which defines a
- * uniform order among all cgroups. It's guaranteed that all
- * ->children lists are in the ascending order of ->serial_nr.
- * It's used to allow interrupting and resuming iterations.
- */
- u64 serial_nr;
-
- /* The bitmask of subsystems attached to this cgroup */
- unsigned long subsys_mask;
+ /* the bitmask of subsystems enabled on the child cgroups */
+ unsigned int child_subsys_mask;
/* Private pointers for each registered subsystem */
struct cgroup_subsys_state __rcu *subsys[CGROUP_SUBSYS_COUNT];
@@ -188,6 +218,15 @@
struct list_head cset_links;
/*
+ * On the default hierarchy, a css_set for a cgroup with some
+ * susbsys disabled will point to css's which are associated with
+ * the closest ancestor which has the subsys enabled. The
+ * following lists all css_sets which point to this cgroup's css
+ * for the given subsystem.
+ */
+ struct list_head e_csets[CGROUP_SUBSYS_COUNT];
+
+ /*
* Linked list running through all cgroups that can
* potentially be reaped by the release agent. Protected by
* release_list_lock
@@ -201,12 +240,8 @@
struct list_head pidlists;
struct mutex pidlist_mutex;
- /* dummy css with NULL ->ss, points back to this cgroup */
- struct cgroup_subsys_state dummy_css;
-
- /* For css percpu_ref killing and RCU-protected deletion */
- struct rcu_head rcu_head;
- struct work_struct destroy_work;
+ /* used to wait for offlining of csses */
+ wait_queue_head_t offline_waitq;
};
#define MAX_CGROUP_ROOT_NAMELEN 64
@@ -250,6 +285,12 @@
*
* - "cgroup.clone_children" is removed.
*
+ * - "cgroup.subtree_populated" is available. Its value is 0 if
+ * the cgroup and its descendants contain no task; otherwise, 1.
+ * The file also generates kernfs notification which can be
+ * monitored through poll and [di]notify when the value of the
+ * file changes.
+ *
* - If mount is requested with sane_behavior but without any
* subsystem, the default unified hierarchy is mounted.
*
@@ -264,6 +305,8 @@
* the flag is not created.
*
* - blkcg: blk-throttle becomes properly hierarchical.
+ *
+ * - debug: disallowed on the default hierarchy.
*/
CGRP_ROOT_SANE_BEHAVIOR = (1 << 0),
@@ -282,6 +325,9 @@
struct cgroup_root {
struct kernfs_root *kf_root;
+ /* The bitmask of subsystems attached to this hierarchy */
+ unsigned int subsys_mask;
+
/* Unique id for this hierarchy. */
int hierarchy_id;
@@ -295,7 +341,7 @@
struct list_head root_list;
/* Hierarchy-specific flags */
- unsigned long flags;
+ unsigned int flags;
/* IDs for cgroups in this hierarchy */
struct idr cgroup_idr;
@@ -342,6 +388,9 @@
*/
struct list_head cgrp_links;
+ /* the default cgroup associated with this css_set */
+ struct cgroup *dfl_cgrp;
+
/*
* Set of subsystem states, one for each subsystem. This array is
* immutable after creation apart from the init_css_set during
@@ -366,6 +415,15 @@
struct cgroup *mg_src_cgrp;
struct css_set *mg_dst_cset;
+ /*
+ * On the default hierarhcy, ->subsys[ssid] may point to a css
+ * attached to an ancestor instead of the cgroup this css_set is
+ * associated with. The following node is anchored at
+ * ->subsys[ssid]->cgroup->e_csets[ssid] and provides a way to
+ * iterate through all css's attached to a given cgroup.
+ */
+ struct list_head e_cset_node[CGROUP_SUBSYS_COUNT];
+
/* For RCU-protected deletion */
struct rcu_head rcu_head;
};
@@ -405,8 +463,7 @@
/*
* The maximum length of string, excluding trailing nul, that can
- * be passed to write_string. If < PAGE_SIZE-1, PAGE_SIZE-1 is
- * assumed.
+ * be passed to write. If < PAGE_SIZE-1, PAGE_SIZE-1 is assumed.
*/
size_t max_write_len;
@@ -453,19 +510,13 @@
s64 val);
/*
- * write_string() is passed a nul-terminated kernelspace
- * buffer of maximum length determined by max_write_len.
- * Returns 0 or -ve error code.
+ * write() is the generic write callback which maps directly to
+ * kernfs write operation and overrides all other operations.
+ * Maximum write size is determined by ->max_write_len. Use
+ * of_css/cft() to access the associated css and cft.
*/
- int (*write_string)(struct cgroup_subsys_state *css, struct cftype *cft,
- char *buffer);
- /*
- * trigger() callback can be used to get some kick from the
- * userspace, when the actual string written is not important
- * at all. The private field can be used to determine the
- * kick type for multiplexing.
- */
- int (*trigger)(struct cgroup_subsys_state *css, unsigned int event);
+ ssize_t (*write)(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off);
#ifdef CONFIG_DEBUG_LOCK_ALLOC
struct lock_class_key lockdep_key;
@@ -504,14 +555,24 @@
return 0;
}
-static inline struct cftype *seq_cft(struct seq_file *seq)
+/* cft/css accessors for cftype->write() operation */
+static inline struct cftype *of_cft(struct kernfs_open_file *of)
{
- struct kernfs_open_file *of = seq->private;
-
return of->kn->priv;
}
-struct cgroup_subsys_state *seq_css(struct seq_file *seq);
+struct cgroup_subsys_state *of_css(struct kernfs_open_file *of);
+
+/* cft/css accessors for cftype->seq_*() operations */
+static inline struct cftype *seq_cft(struct seq_file *seq)
+{
+ return of_cft(seq->private);
+}
+
+static inline struct cgroup_subsys_state *seq_css(struct seq_file *seq)
+{
+ return of_css(seq->private);
+}
/*
* Name / path handling functions. All are thin wrappers around the kernfs
@@ -612,6 +673,9 @@
/* link to parent, protected by cgroup_lock() */
struct cgroup_root *root;
+ /* idr for css->id */
+ struct idr css_idr;
+
/*
* List of cftypes. Each entry is the first entry of an array
* terminated by zero length name.
@@ -627,19 +691,6 @@
#undef SUBSYS
/**
- * css_parent - find the parent css
- * @css: the target cgroup_subsys_state
- *
- * Return the parent css of @css. This function is guaranteed to return
- * non-NULL parent as long as @css isn't the root.
- */
-static inline
-struct cgroup_subsys_state *css_parent(struct cgroup_subsys_state *css)
-{
- return css->parent;
-}
-
-/**
* task_css_set_check - obtain a task's css_set with extra access conditions
* @task: the task to obtain css_set for
* @__c: extra condition expression to be passed to rcu_dereference_check()
@@ -731,14 +782,14 @@
* @pos: the css * to use as the loop cursor
* @parent: css whose children to walk
*
- * Walk @parent's children. Must be called under rcu_read_lock(). A child
- * css which hasn't finished ->css_online() or already has finished
- * ->css_offline() may show up during traversal and it's each subsystem's
- * responsibility to verify that each @pos is alive.
+ * Walk @parent's children. Must be called under rcu_read_lock().
*
- * If a subsystem synchronizes against the parent in its ->css_online() and
- * before starting iterating, a css which finished ->css_online() is
- * guaranteed to be visible in the future iterations.
+ * If a subsystem synchronizes ->css_online() and the start of iteration, a
+ * css which finished ->css_online() is guaranteed to be visible in the
+ * future iterations and will stay visible until the last reference is put.
+ * A css which hasn't finished ->css_online() or already finished
+ * ->css_offline() may show up during traversal. It's each subsystem's
+ * responsibility to synchronize against on/offlining.
*
* It is allowed to temporarily drop RCU read lock during iteration. The
* caller is responsible for ensuring that @pos remains accessible until
@@ -761,17 +812,16 @@
* @root: css whose descendants to walk
*
* Walk @root's descendants. @root is included in the iteration and the
- * first node to be visited. Must be called under rcu_read_lock(). A
- * descendant css which hasn't finished ->css_online() or already has
- * finished ->css_offline() may show up during traversal and it's each
- * subsystem's responsibility to verify that each @pos is alive.
+ * first node to be visited. Must be called under rcu_read_lock().
*
- * If a subsystem synchronizes against the parent in its ->css_online() and
- * before starting iterating, and synchronizes against @pos on each
- * iteration, any descendant css which finished ->css_online() is
- * guaranteed to be visible in the future iterations.
+ * If a subsystem synchronizes ->css_online() and the start of iteration, a
+ * css which finished ->css_online() is guaranteed to be visible in the
+ * future iterations and will stay visible until the last reference is put.
+ * A css which hasn't finished ->css_online() or already finished
+ * ->css_offline() may show up during traversal. It's each subsystem's
+ * responsibility to synchronize against on/offlining.
*
- * In other words, the following guarantees that a descendant can't escape
+ * For example, the following guarantees that a descendant can't escape
* state updates of its ancestors.
*
* my_online(@css)
@@ -827,18 +877,34 @@
*
* Similar to css_for_each_descendant_pre() but performs post-order
* traversal instead. @root is included in the iteration and the last
- * node to be visited. Note that the walk visibility guarantee described
- * in pre-order walk doesn't apply the same to post-order walks.
+ * node to be visited.
+ *
+ * If a subsystem synchronizes ->css_online() and the start of iteration, a
+ * css which finished ->css_online() is guaranteed to be visible in the
+ * future iterations and will stay visible until the last reference is put.
+ * A css which hasn't finished ->css_online() or already finished
+ * ->css_offline() may show up during traversal. It's each subsystem's
+ * responsibility to synchronize against on/offlining.
+ *
+ * Note that the walk visibility guarantee example described in pre-order
+ * walk doesn't apply the same to post-order walks.
*/
#define css_for_each_descendant_post(pos, css) \
for ((pos) = css_next_descendant_post(NULL, (css)); (pos); \
(pos) = css_next_descendant_post((pos), (css)))
+bool css_has_online_children(struct cgroup_subsys_state *css);
+
/* A css_task_iter should be treated as an opaque object */
struct css_task_iter {
- struct cgroup_subsys_state *origin_css;
- struct list_head *cset_link;
- struct list_head *task;
+ struct cgroup_subsys *ss;
+
+ struct list_head *cset_pos;
+ struct list_head *cset_head;
+
+ struct list_head *task_pos;
+ struct list_head *tasks_head;
+ struct list_head *mg_tasks_head;
};
void css_task_iter_start(struct cgroup_subsys_state *css,
@@ -849,8 +915,8 @@
int cgroup_attach_task_all(struct task_struct *from, struct task_struct *);
int cgroup_transfer_tasks(struct cgroup *to, struct cgroup *from);
-struct cgroup_subsys_state *css_tryget_from_dir(struct dentry *dentry,
- struct cgroup_subsys *ss);
+struct cgroup_subsys_state *css_tryget_online_from_dir(struct dentry *dentry,
+ struct cgroup_subsys *ss);
#else /* !CONFIG_CGROUPS */
diff --git a/include/linux/cgroup_subsys.h b/include/linux/cgroup_subsys.h
index 768fe44..98c4f9b 100644
--- a/include/linux/cgroup_subsys.h
+++ b/include/linux/cgroup_subsys.h
@@ -7,10 +7,6 @@
SUBSYS(cpuset)
#endif
-#if IS_ENABLED(CONFIG_CGROUP_DEBUG)
-SUBSYS(debug)
-#endif
-
#if IS_ENABLED(CONFIG_CGROUP_SCHED)
SUBSYS(cpu)
#endif
@@ -50,6 +46,13 @@
#if IS_ENABLED(CONFIG_CGROUP_HUGETLB)
SUBSYS(hugetlb)
#endif
+
+/*
+ * The following subsystems are not supported on the default hierarchy.
+ */
+#if IS_ENABLED(CONFIG_CGROUP_DEBUG)
+SUBSYS(debug)
+#endif
/*
* DO NOT ADD ANY SUBSYSTEM WITHOUT EXPLICIT ACKS FROM CGROUP MAINTAINERS.
*/
diff --git a/include/linux/clk-provider.h b/include/linux/clk-provider.h
index f295bab..0c287db 100644
--- a/include/linux/clk-provider.h
+++ b/include/linux/clk-provider.h
@@ -40,14 +40,14 @@
* through the clk_* api.
*
* @prepare: Prepare the clock for enabling. This must not return until
- * the clock is fully prepared, and it's safe to call clk_enable.
- * This callback is intended to allow clock implementations to
- * do any initialisation that may sleep. Called with
- * prepare_lock held.
+ * the clock is fully prepared, and it's safe to call clk_enable.
+ * This callback is intended to allow clock implementations to
+ * do any initialisation that may sleep. Called with
+ * prepare_lock held.
*
* @unprepare: Release the clock from its prepared state. This will typically
- * undo any work done in the @prepare callback. Called with
- * prepare_lock held.
+ * undo any work done in the @prepare callback. Called with
+ * prepare_lock held.
*
* @is_prepared: Queries the hardware to determine if the clock is prepared.
* This function is allowed to sleep. Optional, if this op is not
@@ -58,16 +58,16 @@
* Called with prepare mutex held. This function may sleep.
*
* @enable: Enable the clock atomically. This must not return until the
- * clock is generating a valid clock signal, usable by consumer
- * devices. Called with enable_lock held. This function must not
- * sleep.
+ * clock is generating a valid clock signal, usable by consumer
+ * devices. Called with enable_lock held. This function must not
+ * sleep.
*
* @disable: Disable the clock atomically. Called with enable_lock held.
- * This function must not sleep.
+ * This function must not sleep.
*
* @is_enabled: Queries the hardware to determine if the clock is enabled.
- * This function must not sleep. Optional, if this op is not
- * set then the enable count will be used.
+ * This function must not sleep. Optional, if this op is not
+ * set then the enable count will be used.
*
* @disable_unused: Disable the clock atomically. Only called from
* clk_disable_unused for gate clocks with special needs.
@@ -75,34 +75,35 @@
* sleep.
*
* @recalc_rate Recalculate the rate of this clock, by querying hardware. The
- * parent rate is an input parameter. It is up to the caller to
- * ensure that the prepare_mutex is held across this call.
- * Returns the calculated rate. Optional, but recommended - if
- * this op is not set then clock rate will be initialized to 0.
+ * parent rate is an input parameter. It is up to the caller to
+ * ensure that the prepare_mutex is held across this call.
+ * Returns the calculated rate. Optional, but recommended - if
+ * this op is not set then clock rate will be initialized to 0.
*
* @round_rate: Given a target rate as input, returns the closest rate actually
- * supported by the clock.
+ * supported by the clock. The parent rate is an input/output
+ * parameter.
*
* @determine_rate: Given a target rate as input, returns the closest rate
* actually supported by the clock, and optionally the parent clock
* that should be used to provide the clock rate.
*
- * @get_parent: Queries the hardware to determine the parent of a clock. The
- * return value is a u8 which specifies the index corresponding to
- * the parent clock. This index can be applied to either the
- * .parent_names or .parents arrays. In short, this function
- * translates the parent value read from hardware into an array
- * index. Currently only called when the clock is initialized by
- * __clk_init. This callback is mandatory for clocks with
- * multiple parents. It is optional (and unnecessary) for clocks
- * with 0 or 1 parents.
- *
* @set_parent: Change the input source of this clock; for clocks with multiple
- * possible parents specify a new parent by passing in the index
- * as a u8 corresponding to the parent in either the .parent_names
- * or .parents arrays. This function in affect translates an
- * array index into the value programmed into the hardware.
- * Returns 0 on success, -EERROR otherwise.
+ * possible parents specify a new parent by passing in the index
+ * as a u8 corresponding to the parent in either the .parent_names
+ * or .parents arrays. This function in affect translates an
+ * array index into the value programmed into the hardware.
+ * Returns 0 on success, -EERROR otherwise.
+ *
+ * @get_parent: Queries the hardware to determine the parent of a clock. The
+ * return value is a u8 which specifies the index corresponding to
+ * the parent clock. This index can be applied to either the
+ * .parent_names or .parents arrays. In short, this function
+ * translates the parent value read from hardware into an array
+ * index. Currently only called when the clock is initialized by
+ * __clk_init. This callback is mandatory for clocks with
+ * multiple parents. It is optional (and unnecessary) for clocks
+ * with 0 or 1 parents.
*
* @set_rate: Change the rate of this clock. The requested rate is specified
* by the second argument, which should typically be the return
@@ -110,13 +111,6 @@
* which is likely helpful for most .set_rate implementation.
* Returns 0 on success, -EERROR otherwise.
*
- * @recalc_accuracy: Recalculate the accuracy of this clock. The clock accuracy
- * is expressed in ppb (parts per billion). The parent accuracy is
- * an input parameter.
- * Returns the calculated accuracy. Optional - if this op is not
- * set then clock accuracy will be initialized to parent accuracy
- * or 0 (perfect clock) if clock has no parent.
- *
* @set_rate_and_parent: Change the rate and the parent of this clock. The
* requested rate is specified by the second argument, which
* should typically be the return of .round_rate call. The
@@ -128,6 +122,18 @@
* separately via calls to .set_parent and .set_rate.
* Returns 0 on success, -EERROR otherwise.
*
+ * @recalc_accuracy: Recalculate the accuracy of this clock. The clock accuracy
+ * is expressed in ppb (parts per billion). The parent accuracy is
+ * an input parameter.
+ * Returns the calculated accuracy. Optional - if this op is not
+ * set then clock accuracy will be initialized to parent accuracy
+ * or 0 (perfect clock) if clock has no parent.
+ *
+ * @init: Perform platform-specific initialization magic.
+ * This is not not used by any of the basic clock types.
+ * Please consider other ways of solving initialization problems
+ * before using this callback, as its use is discouraged.
+ *
* @debug_init: Set up type-specific debugfs entries for this clock. This
* is called once, after the debugfs directory entry for this
* clock has been created. The dentry pointer representing that
@@ -157,15 +163,15 @@
void (*disable_unused)(struct clk_hw *hw);
unsigned long (*recalc_rate)(struct clk_hw *hw,
unsigned long parent_rate);
- long (*round_rate)(struct clk_hw *hw, unsigned long,
- unsigned long *);
+ long (*round_rate)(struct clk_hw *hw, unsigned long rate,
+ unsigned long *parent_rate);
long (*determine_rate)(struct clk_hw *hw, unsigned long rate,
unsigned long *best_parent_rate,
struct clk **best_parent_clk);
int (*set_parent)(struct clk_hw *hw, u8 index);
u8 (*get_parent)(struct clk_hw *hw);
- int (*set_rate)(struct clk_hw *hw, unsigned long,
- unsigned long);
+ int (*set_rate)(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate);
int (*set_rate_and_parent)(struct clk_hw *hw,
unsigned long rate,
unsigned long parent_rate, u8 index);
@@ -254,12 +260,12 @@
*
* Flags:
* CLK_GATE_SET_TO_DISABLE - by default this clock sets the bit at bit_idx to
- * enable the clock. Setting this flag does the opposite: setting the bit
- * disable the clock and clearing it enables the clock
+ * enable the clock. Setting this flag does the opposite: setting the bit
+ * disable the clock and clearing it enables the clock
* CLK_GATE_HIWORD_MASK - The gate settings are only in lower 16-bit
- * of this register, and mask of gate bits are in higher 16-bit of this
- * register. While setting the gate bits, higher 16-bit should also be
- * updated to indicate changing gate bits.
+ * of this register, and mask of gate bits are in higher 16-bit of this
+ * register. While setting the gate bits, higher 16-bit should also be
+ * updated to indicate changing gate bits.
*/
struct clk_gate {
struct clk_hw hw;
@@ -298,20 +304,24 @@
*
* Flags:
* CLK_DIVIDER_ONE_BASED - by default the divisor is the value read from the
- * register plus one. If CLK_DIVIDER_ONE_BASED is set then the divider is
- * the raw value read from the register, with the value of zero considered
+ * register plus one. If CLK_DIVIDER_ONE_BASED is set then the divider is
+ * the raw value read from the register, with the value of zero considered
* invalid, unless CLK_DIVIDER_ALLOW_ZERO is set.
* CLK_DIVIDER_POWER_OF_TWO - clock divisor is 2 raised to the value read from
- * the hardware register
+ * the hardware register
* CLK_DIVIDER_ALLOW_ZERO - Allow zero divisors. For dividers which have
* CLK_DIVIDER_ONE_BASED set, it is possible to end up with a zero divisor.
* Some hardware implementations gracefully handle this case and allow a
* zero divisor by not modifying their input clock
* (divide by one / bypass).
* CLK_DIVIDER_HIWORD_MASK - The divider settings are only in lower 16-bit
- * of this register, and mask of divider bits are in higher 16-bit of this
- * register. While setting the divider bits, higher 16-bit should also be
- * updated to indicate changing divider bits.
+ * of this register, and mask of divider bits are in higher 16-bit of this
+ * register. While setting the divider bits, higher 16-bit should also be
+ * updated to indicate changing divider bits.
+ * CLK_DIVIDER_ROUND_CLOSEST - Makes the best calculated divider to be rounded
+ * to the closest integer instead of the up one.
+ * CLK_DIVIDER_READ_ONLY - The divider settings are preconfigured and should
+ * not be changed by the clock framework.
*/
struct clk_divider {
struct clk_hw hw;
@@ -327,8 +337,11 @@
#define CLK_DIVIDER_POWER_OF_TWO BIT(1)
#define CLK_DIVIDER_ALLOW_ZERO BIT(2)
#define CLK_DIVIDER_HIWORD_MASK BIT(3)
+#define CLK_DIVIDER_ROUND_CLOSEST BIT(4)
+#define CLK_DIVIDER_READ_ONLY BIT(5)
extern const struct clk_ops clk_divider_ops;
+extern const struct clk_ops clk_divider_ro_ops;
struct clk *clk_register_divider(struct device *dev, const char *name,
const char *parent_name, unsigned long flags,
void __iomem *reg, u8 shift, u8 width,
@@ -356,9 +369,9 @@
* CLK_MUX_INDEX_ONE - register index starts at 1, not 0
* CLK_MUX_INDEX_BIT - register index is a single bit (power of two)
* CLK_MUX_HIWORD_MASK - The mux settings are only in lower 16-bit of this
- * register, and mask of mux bits are in higher 16-bit of this register.
- * While setting the mux bits, higher 16-bit should also be updated to
- * indicate changing mux bits.
+ * register, and mask of mux bits are in higher 16-bit of this register.
+ * While setting the mux bits, higher 16-bit should also be updated to
+ * indicate changing mux bits.
*/
struct clk_mux {
struct clk_hw hw;
diff --git a/include/linux/clk/shmobile.h b/include/linux/clk/shmobile.h
index f9bf080..9f8a140 100644
--- a/include/linux/clk/shmobile.h
+++ b/include/linux/clk/shmobile.h
@@ -1,7 +1,9 @@
/*
* Copyright 2013 Ideas On Board SPRL
+ * Copyright 2013, 2014 Horms Solutions Ltd.
*
* Contact: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ * Contact: Simon Horman <horms@verge.net.au>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -14,6 +16,7 @@
#include <linux/types.h>
+void r8a7779_clocks_init(u32 mode);
void rcar_gen2_clocks_init(u32 mode);
#endif
diff --git a/include/linux/clk/sunxi.h b/include/linux/clk/sunxi.h
new file mode 100644
index 0000000..aed28c4
--- /dev/null
+++ b/include/linux/clk/sunxi.h
@@ -0,0 +1,22 @@
+/*
+ * Copyright 2013 - Hans de Goede <hdegoede@redhat.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __LINUX_CLK_SUNXI_H_
+#define __LINUX_CLK_SUNXI_H_
+
+#include <linux/clk.h>
+
+void clk_sunxi_mmc_phase_control(struct clk *clk, u8 sample, u8 output);
+
+#endif
diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index 8188712..95978ad 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -256,7 +256,6 @@
};
void cpu_startup_entry(enum cpuhp_state state);
-void cpu_idle(void);
void cpu_idle_poll_ctrl(bool enable);
diff --git a/include/linux/cpuidle.h b/include/linux/cpuidle.h
index c51a436..25e0df6 100644
--- a/include/linux/cpuidle.h
+++ b/include/linux/cpuidle.h
@@ -84,6 +84,7 @@
};
DECLARE_PER_CPU(struct cpuidle_device *, cpuidle_devices);
+DECLARE_PER_CPU(struct cpuidle_device, cpuidle_dev);
/**
* cpuidle_get_last_residency - retrieves the last state's residency time
diff --git a/include/linux/device.h b/include/linux/device.h
index 580e3eed..af424ac 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -692,6 +692,7 @@
* @coherent_dma_mask: Like dma_mask, but for alloc_coherent mapping as not all
* hardware supports 64-bit addresses for consistent allocations
* such descriptors.
+ * @dma_pfn_offset: offset of DMA memory range relatively of RAM
* @dma_parms: A low level driver may set these to teach IOMMU code about
* segment limitations.
* @dma_pools: Dma pools (if dma'ble device).
@@ -759,6 +760,7 @@
not all hardware supports
64 bit addresses for consistent
allocations such descriptors. */
+ unsigned long dma_pfn_offset;
struct device_dma_parameters *dma_parms;
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 0c3eab1..931b709 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -129,6 +129,13 @@
extern u64 dma_get_required_mask(struct device *dev);
+#ifndef set_arch_dma_coherent_ops
+static inline int set_arch_dma_coherent_ops(struct device *dev)
+{
+ return 0;
+}
+#endif
+
static inline unsigned int dma_get_max_seg_size(struct device *dev)
{
return dev->dma_parms ? dev->dma_parms->max_segment_size : 65536;
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index 72cb0dd..d2c5cc7 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -292,7 +292,7 @@
};
/**
- * enum dma_slave_buswidth - defines bus with of the DMA slave
+ * enum dma_slave_buswidth - defines bus width of the DMA slave
* device, source or target buses
*/
enum dma_slave_buswidth {
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 6a4d8e2..41bbf8b 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -575,6 +575,9 @@
#define EFI_FILE_SYSTEM_GUID \
EFI_GUID( 0x964e5b22, 0x6459, 0x11d2, 0x8e, 0x39, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b )
+#define DEVICE_TREE_GUID \
+ EFI_GUID( 0xb1b621d5, 0xf19c, 0x41a5, 0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0 )
+
typedef struct {
efi_guid_t guid;
u64 table;
@@ -664,6 +667,14 @@
unsigned long desc_size;
};
+struct efi_fdt_params {
+ u64 system_table;
+ u64 mmap;
+ u32 mmap_size;
+ u32 desc_size;
+ u32 desc_ver;
+};
+
typedef struct {
u32 revision;
u32 parent_handle;
@@ -861,6 +872,7 @@
extern void efi_get_time(struct timespec *now);
extern int efi_set_rtc_mmss(const struct timespec *now);
extern void efi_reserve_boot_services(void);
+extern int efi_get_fdt_params(struct efi_fdt_params *params, int verbose);
extern struct efi_memory_map memmap;
/* Iterate through an efi_memory_map */
diff --git a/include/linux/f2fs_fs.h b/include/linux/f2fs_fs.h
index df53e17..6ff0b0b 100644
--- a/include/linux/f2fs_fs.h
+++ b/include/linux/f2fs_fs.h
@@ -19,6 +19,7 @@
#define F2FS_LOG_SECTORS_PER_BLOCK 3 /* 4KB: F2FS_BLKSIZE */
#define F2FS_BLKSIZE 4096 /* support only 4KB block */
#define F2FS_MAX_EXTENSION 64 /* # of extension entries */
+#define F2FS_BLK_ALIGN(x) (((x) + F2FS_BLKSIZE - 1) / F2FS_BLKSIZE)
#define NULL_ADDR ((block_t)0) /* used as block_t addresses */
#define NEW_ADDR ((block_t)-1) /* used as block_t addresses */
@@ -75,6 +76,7 @@
__le16 volume_name[512]; /* volume name */
__le32 extension_count; /* # of extensions below */
__u8 extension_list[F2FS_MAX_EXTENSION][8]; /* extension array */
+ __le32 cp_payload;
} __packed;
/*
@@ -146,6 +148,9 @@
#define ADDRS_PER_BLOCK 1018 /* Address Pointers in a Direct Block */
#define NIDS_PER_BLOCK 1018 /* Node IDs in an Indirect Block */
+#define ADDRS_PER_PAGE(page, fi) \
+ (IS_INODE(page) ? ADDRS_PER_INODE(fi) : ADDRS_PER_BLOCK)
+
#define NODE_DIR1_BLOCK (DEF_ADDRS_PER_INODE + 1)
#define NODE_DIR2_BLOCK (DEF_ADDRS_PER_INODE + 2)
#define NODE_IND1_BLOCK (DEF_ADDRS_PER_INODE + 3)
@@ -391,6 +396,9 @@
/* MAX level for dir lookup */
#define MAX_DIR_HASH_DEPTH 63
+/* MAX buckets in one level of dir */
+#define MAX_DIR_BUCKETS (1 << ((MAX_DIR_HASH_DEPTH / 2) - 1))
+
#define SIZE_OF_DIR_ENTRY 11 /* by byte */
#define SIZE_OF_DENTRY_BITMAP ((NR_DENTRY_IN_BLOCK + BITS_PER_BYTE - 1) / \
BITS_PER_BYTE)
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index ae9504b..404a686 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -62,9 +62,6 @@
* set in the flags member.
*
* ENABLED - set/unset when ftrace_ops is registered/unregistered
- * GLOBAL - set manualy by ftrace_ops user to denote the ftrace_ops
- * is part of the global tracers sharing the same filter
- * via set_ftrace_* debugfs files.
* DYNAMIC - set when ftrace_ops is registered to denote dynamically
* allocated ftrace_ops which need special care
* CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
@@ -96,15 +93,14 @@
*/
enum {
FTRACE_OPS_FL_ENABLED = 1 << 0,
- FTRACE_OPS_FL_GLOBAL = 1 << 1,
- FTRACE_OPS_FL_DYNAMIC = 1 << 2,
- FTRACE_OPS_FL_CONTROL = 1 << 3,
- FTRACE_OPS_FL_SAVE_REGS = 1 << 4,
- FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED = 1 << 5,
- FTRACE_OPS_FL_RECURSION_SAFE = 1 << 6,
- FTRACE_OPS_FL_STUB = 1 << 7,
- FTRACE_OPS_FL_INITIALIZED = 1 << 8,
- FTRACE_OPS_FL_DELETED = 1 << 9,
+ FTRACE_OPS_FL_DYNAMIC = 1 << 1,
+ FTRACE_OPS_FL_CONTROL = 1 << 2,
+ FTRACE_OPS_FL_SAVE_REGS = 1 << 3,
+ FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED = 1 << 4,
+ FTRACE_OPS_FL_RECURSION_SAFE = 1 << 5,
+ FTRACE_OPS_FL_STUB = 1 << 6,
+ FTRACE_OPS_FL_INITIALIZED = 1 << 7,
+ FTRACE_OPS_FL_DELETED = 1 << 8,
};
/*
@@ -366,14 +362,12 @@
* IGNORE - The function is already what we want it to be
* MAKE_CALL - Start tracing the function
* MODIFY_CALL - Stop saving regs for the function
- * MODIFY_CALL_REGS - Start saving regs for the function
* MAKE_NOP - Stop tracing the function
*/
enum {
FTRACE_UPDATE_IGNORE,
FTRACE_UPDATE_MAKE_CALL,
FTRACE_UPDATE_MODIFY_CALL,
- FTRACE_UPDATE_MODIFY_CALL_REGS,
FTRACE_UPDATE_MAKE_NOP,
};
@@ -404,6 +398,8 @@
int ftrace_test_record(struct dyn_ftrace *rec, int enable);
void ftrace_run_stop_machine(int command);
unsigned long ftrace_location(unsigned long ip);
+unsigned long ftrace_get_addr_new(struct dyn_ftrace *rec);
+unsigned long ftrace_get_addr_curr(struct dyn_ftrace *rec);
extern ftrace_func_t ftrace_trace_function;
@@ -616,25 +612,27 @@
#endif
}
-#ifndef HAVE_ARCH_CALLER_ADDR
+/* All archs should have this, but we define it for consistency */
+#ifndef ftrace_return_address0
+# define ftrace_return_address0 __builtin_return_address(0)
+#endif
+
+/* Archs may use other ways for ADDR1 and beyond */
+#ifndef ftrace_return_address
# ifdef CONFIG_FRAME_POINTER
-# define CALLER_ADDR0 ((unsigned long)__builtin_return_address(0))
-# define CALLER_ADDR1 ((unsigned long)__builtin_return_address(1))
-# define CALLER_ADDR2 ((unsigned long)__builtin_return_address(2))
-# define CALLER_ADDR3 ((unsigned long)__builtin_return_address(3))
-# define CALLER_ADDR4 ((unsigned long)__builtin_return_address(4))
-# define CALLER_ADDR5 ((unsigned long)__builtin_return_address(5))
-# define CALLER_ADDR6 ((unsigned long)__builtin_return_address(6))
+# define ftrace_return_address(n) __builtin_return_address(n)
# else
-# define CALLER_ADDR0 ((unsigned long)__builtin_return_address(0))
-# define CALLER_ADDR1 0UL
-# define CALLER_ADDR2 0UL
-# define CALLER_ADDR3 0UL
-# define CALLER_ADDR4 0UL
-# define CALLER_ADDR5 0UL
-# define CALLER_ADDR6 0UL
+# define ftrace_return_address(n) 0UL
# endif
-#endif /* ifndef HAVE_ARCH_CALLER_ADDR */
+#endif
+
+#define CALLER_ADDR0 ((unsigned long)ftrace_return_address0)
+#define CALLER_ADDR1 ((unsigned long)ftrace_return_address(1))
+#define CALLER_ADDR2 ((unsigned long)ftrace_return_address(2))
+#define CALLER_ADDR3 ((unsigned long)ftrace_return_address(3))
+#define CALLER_ADDR4 ((unsigned long)ftrace_return_address(4))
+#define CALLER_ADDR5 ((unsigned long)ftrace_return_address(5))
+#define CALLER_ADDR6 ((unsigned long)ftrace_return_address(6))
#ifdef CONFIG_IRQSOFF_TRACER
extern void time_hardirqs_on(unsigned long a0, unsigned long a1);
diff --git a/include/linux/ftrace_event.h b/include/linux/ftrace_event.h
index d16da3e..cff3106 100644
--- a/include/linux/ftrace_event.h
+++ b/include/linux/ftrace_event.h
@@ -38,6 +38,9 @@
*symbol_array);
#endif
+const char *ftrace_print_bitmask_seq(struct trace_seq *p, void *bitmask_ptr,
+ unsigned int bitmask_size);
+
const char *ftrace_print_hex_seq(struct trace_seq *p,
const unsigned char *buf, int len);
diff --git a/include/linux/gpio_keys.h b/include/linux/gpio_keys.h
index a7e977f..8b62246 100644
--- a/include/linux/gpio_keys.h
+++ b/include/linux/gpio_keys.h
@@ -3,29 +3,53 @@
struct device;
+/**
+ * struct gpio_keys_button - configuration parameters
+ * @code: input event code (KEY_*, SW_*)
+ * @gpio: %-1 if this key does not support gpio
+ * @active_low: %true indicates that button is considered
+ * depressed when gpio is low
+ * @desc: label that will be attached to button's gpio
+ * @type: input event type (%EV_KEY, %EV_SW, %EV_ABS)
+ * @wakeup: configure the button as a wake-up source
+ * @debounce_interval: debounce ticks interval in msecs
+ * @can_disable: %true indicates that userspace is allowed to
+ * disable button via sysfs
+ * @value: axis value for %EV_ABS
+ * @irq: Irq number in case of interrupt keys
+ */
struct gpio_keys_button {
- /* Configuration parameters */
- unsigned int code; /* input event code (KEY_*, SW_*) */
- int gpio; /* -1 if this key does not support gpio */
+ unsigned int code;
+ int gpio;
int active_low;
const char *desc;
- unsigned int type; /* input event type (EV_KEY, EV_SW, EV_ABS) */
- int wakeup; /* configure the button as a wake-up source */
- int debounce_interval; /* debounce ticks interval in msecs */
+ unsigned int type;
+ int wakeup;
+ int debounce_interval;
bool can_disable;
- int value; /* axis value for EV_ABS */
- unsigned int irq; /* Irq number in case of interrupt keys */
+ int value;
+ unsigned int irq;
};
+/**
+ * struct gpio_keys_platform_data - platform data for gpio_keys driver
+ * @buttons: pointer to array of &gpio_keys_button structures
+ * describing buttons attached to the device
+ * @nbuttons: number of elements in @buttons array
+ * @poll_interval: polling interval in msecs - for polling driver only
+ * @rep: enable input subsystem auto repeat
+ * @enable: platform hook for enabling the device
+ * @disable: platform hook for disabling the device
+ * @name: input device name
+ */
struct gpio_keys_platform_data {
struct gpio_keys_button *buttons;
int nbuttons;
- unsigned int poll_interval; /* polling interval in msecs -
- for polling driver only */
- unsigned int rep:1; /* enable input subsystem auto repeat */
+ unsigned int poll_interval;
+ unsigned int rep:1;
int (*enable)(struct device *dev);
void (*disable)(struct device *dev);
- const char *name; /* input device name */
+ const char *name;
};
#endif
diff --git a/include/linux/i2c/atmel_mxt_ts.h b/include/linux/i2c/atmel_mxt_ts.h
index 99e379b..3891dc1 100644
--- a/include/linux/i2c/atmel_mxt_ts.h
+++ b/include/linux/i2c/atmel_mxt_ts.h
@@ -15,35 +15,14 @@
#include <linux/types.h>
-/* For key_map array */
-#define MXT_NUM_GPIO 4
-
-/* Orient */
-#define MXT_NORMAL 0x0
-#define MXT_DIAGONAL 0x1
-#define MXT_HORIZONTAL_FLIP 0x2
-#define MXT_ROTATED_90_COUNTER 0x3
-#define MXT_VERTICAL_FLIP 0x4
-#define MXT_ROTATED_90 0x5
-#define MXT_ROTATED_180 0x6
-#define MXT_DIAGONAL_COUNTER 0x7
-
/* The platform data for the Atmel maXTouch touchscreen driver */
struct mxt_platform_data {
const u8 *config;
size_t config_length;
-
- unsigned int x_line;
- unsigned int y_line;
- unsigned int x_size;
- unsigned int y_size;
- unsigned int blen;
- unsigned int threshold;
- unsigned int voltage;
- unsigned char orient;
+ u32 config_crc;
unsigned long irqflags;
- bool is_tp;
- const unsigned int key_map[MXT_NUM_GPIO];
+ u8 t19_num_keys;
+ const unsigned int *t19_keymap;
};
#endif /* __LINUX_ATMEL_MXT_TS_H */
diff --git a/include/linux/idr.h b/include/linux/idr.h
index 6af3400..013fd9b 100644
--- a/include/linux/idr.h
+++ b/include/linux/idr.h
@@ -29,21 +29,24 @@
struct idr_layer {
int prefix; /* the ID prefix of this idr_layer */
- DECLARE_BITMAP(bitmap, IDR_SIZE); /* A zero bit means "space here" */
+ int layer; /* distance from leaf */
struct idr_layer __rcu *ary[1<<IDR_BITS];
int count; /* When zero, we can release it */
- int layer; /* distance from leaf */
- struct rcu_head rcu_head;
+ union {
+ /* A zero bit means "space here" */
+ DECLARE_BITMAP(bitmap, IDR_SIZE);
+ struct rcu_head rcu_head;
+ };
};
struct idr {
struct idr_layer __rcu *hint; /* the last layer allocated from */
struct idr_layer __rcu *top;
- struct idr_layer *id_free;
int layers; /* only valid w/o concurrent changes */
- int id_free_cnt;
int cur; /* current pos for cyclic allocation */
spinlock_t lock;
+ int id_free_cnt;
+ struct idr_layer *id_free;
};
#define IDR_INIT(name) \
diff --git a/include/linux/if_team.h b/include/linux/if_team.h
index a899dc24..a6aa970 100644
--- a/include/linux/if_team.h
+++ b/include/linux/if_team.h
@@ -194,6 +194,7 @@
bool user_carrier_enabled;
bool queue_override_enabled;
struct list_head *qom_lists; /* array of queue override mapping lists */
+ bool port_mtu_change_allowed;
struct {
unsigned int count;
unsigned int interval; /* in ms */
diff --git a/include/linux/input-polldev.h b/include/linux/input-polldev.h
index ce0b724..2465182 100644
--- a/include/linux/input-polldev.h
+++ b/include/linux/input-polldev.h
@@ -48,9 +48,12 @@
/* private: */
struct delayed_work work;
+
+ bool devres_managed;
};
struct input_polled_dev *input_allocate_polled_device(void);
+struct input_polled_dev *devm_input_allocate_polled_device(struct device *dev);
void input_free_polled_device(struct input_polled_dev *dev);
int input_register_polled_device(struct input_polled_dev *dev);
void input_unregister_polled_device(struct input_polled_dev *dev);
diff --git a/include/linux/input/pixcir_ts.h b/include/linux/input/pixcir_ts.h
index 7163d91..160cf35 100644
--- a/include/linux/input/pixcir_ts.h
+++ b/include/linux/input/pixcir_ts.h
@@ -1,10 +1,52 @@
#ifndef _PIXCIR_I2C_TS_H
#define _PIXCIR_I2C_TS_H
+/*
+ * Register map
+ */
+#define PIXCIR_REG_POWER_MODE 51
+#define PIXCIR_REG_INT_MODE 52
+
+/*
+ * Power modes:
+ * active: max scan speed
+ * idle: lower scan speed with automatic transition to active on touch
+ * halt: datasheet says sleep but this is more like halt as the chip
+ * clocks are cut and it can only be brought out of this mode
+ * using the RESET pin.
+ */
+enum pixcir_power_mode {
+ PIXCIR_POWER_ACTIVE,
+ PIXCIR_POWER_IDLE,
+ PIXCIR_POWER_HALT,
+};
+
+#define PIXCIR_POWER_MODE_MASK 0x03
+#define PIXCIR_POWER_ALLOW_IDLE (1UL << 2)
+
+/*
+ * Interrupt modes:
+ * periodical: interrupt is asserted periodicaly
+ * diff coordinates: interrupt is asserted when coordinates change
+ * level on touch: interrupt level asserted during touch
+ * pulse on touch: interrupt pulse asserted druing touch
+ *
+ */
+enum pixcir_int_mode {
+ PIXCIR_INT_PERIODICAL,
+ PIXCIR_INT_DIFF_COORD,
+ PIXCIR_INT_LEVEL_TOUCH,
+ PIXCIR_INT_PULSE_TOUCH,
+};
+
+#define PIXCIR_INT_MODE_MASK 0x03
+#define PIXCIR_INT_ENABLE (1UL << 3)
+#define PIXCIR_INT_POL_HIGH (1UL << 2)
+
struct pixcir_ts_platform_data {
- int (*attb_read_val)(void);
int x_max;
int y_max;
+ int gpio_attb; /* GPIO connected to ATTB line */
};
#endif
diff --git a/include/linux/input/touchscreen.h b/include/linux/input/touchscreen.h
new file mode 100644
index 0000000..08a5ef6
--- /dev/null
+++ b/include/linux/input/touchscreen.h
@@ -0,0 +1,22 @@
+/*
+ * Copyright (c) 2014 Sebastian Reichel <sre@kernel.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ */
+
+#ifndef _TOUCHSCREEN_H
+#define _TOUCHSCREEN_H
+
+#include <linux/input.h>
+
+#ifdef CONFIG_OF
+void touchscreen_parse_of_params(struct input_dev *dev);
+#else
+static inline void touchscreen_parse_of_params(struct input_dev *dev)
+{
+}
+#endif
+
+#endif
diff --git a/include/linux/key.h b/include/linux/key.h
index 80d6774..017b082 100644
--- a/include/linux/key.h
+++ b/include/linux/key.h
@@ -309,6 +309,17 @@
extern void key_set_timeout(struct key *, unsigned);
+/*
+ * The permissions required on a key that we're looking up.
+ */
+#define KEY_NEED_VIEW 0x01 /* Require permission to view attributes */
+#define KEY_NEED_READ 0x02 /* Require permission to read content */
+#define KEY_NEED_WRITE 0x04 /* Require permission to update / modify */
+#define KEY_NEED_SEARCH 0x08 /* Require permission to search (keyring) or find (key) */
+#define KEY_NEED_LINK 0x10 /* Require permission to link */
+#define KEY_NEED_SETATTR 0x20 /* Require permission to change attributes */
+#define KEY_NEED_ALL 0x3f /* All the above permissions */
+
/**
* key_is_instantiated - Determine if a key has been positively instantiated
* @key: The key to check.
@@ -332,7 +343,7 @@
} while (0)
#ifdef CONFIG_SYSCTL
-extern ctl_table key_sysctls[];
+extern struct ctl_table key_sysctls[];
#endif
/*
* the userspace interface
diff --git a/include/linux/kmemleak.h b/include/linux/kmemleak.h
index 5bb4246..057e959 100644
--- a/include/linux/kmemleak.h
+++ b/include/linux/kmemleak.h
@@ -30,6 +30,7 @@
extern void kmemleak_free(const void *ptr) __ref;
extern void kmemleak_free_part(const void *ptr, size_t size) __ref;
extern void kmemleak_free_percpu(const void __percpu *ptr) __ref;
+extern void kmemleak_update_trace(const void *ptr) __ref;
extern void kmemleak_not_leak(const void *ptr) __ref;
extern void kmemleak_ignore(const void *ptr) __ref;
extern void kmemleak_scan_area(const void *ptr, size_t size, gfp_t gfp) __ref;
@@ -83,6 +84,9 @@
static inline void kmemleak_free_percpu(const void __percpu *ptr)
{
}
+static inline void kmemleak_update_trace(const void *ptr)
+{
+}
static inline void kmemleak_not_leak(const void *ptr)
{
}
diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h
index 925eaf2..7bd2ad0 100644
--- a/include/linux/kprobes.h
+++ b/include/linux/kprobes.h
@@ -355,7 +355,7 @@
static inline struct kprobe_ctlblk *get_kprobe_ctlblk(void)
{
- return (&__get_cpu_var(kprobe_ctlblk));
+ return this_cpu_ptr(&kprobe_ctlblk);
}
int register_kprobe(struct kprobe *p);
diff --git a/include/linux/lockd/lockd.h b/include/linux/lockd/lockd.h
index dcaad79..219d796 100644
--- a/include/linux/lockd/lockd.h
+++ b/include/linux/lockd/lockd.h
@@ -17,13 +17,13 @@
#include <linux/fs.h>
#include <linux/kref.h>
#include <linux/utsname.h>
-#include <linux/nfsd/nfsfh.h>
#include <linux/lockd/bind.h>
#include <linux/lockd/xdr.h>
#ifdef CONFIG_LOCKD_V4
#include <linux/lockd/xdr4.h>
#endif
#include <linux/lockd/debug.h>
+#include <linux/sunrpc/svc.h>
/*
* Version string
diff --git a/include/linux/mc146818rtc.h b/include/linux/mc146818rtc.h
index 2f4e957..433e0c7 100644
--- a/include/linux/mc146818rtc.h
+++ b/include/linux/mc146818rtc.h
@@ -31,6 +31,10 @@
void (*wake_on)(struct device *dev);
void (*wake_off)(struct device *dev);
+ u32 flags;
+#define CMOS_RTC_FLAGS_NOFREQ (1 << 0)
+ int address_space;
+
u8 rtc_day_alarm; /* zero, or register index */
u8 rtc_mon_alarm; /* zero, or register index */
u8 rtc_century; /* zero, or register index */
diff --git a/include/linux/mfd/abx500.h b/include/linux/mfd/abx500.h
index 3301b20..552cc1d 100644
--- a/include/linux/mfd/abx500.h
+++ b/include/linux/mfd/abx500.h
@@ -330,7 +330,6 @@
int abx500_get_chip_id(struct device *dev);
int abx500_event_registers_startup_state_get(struct device *dev, u8 *event);
int abx500_startup_irq_enabled(struct device *dev, unsigned int irq);
-void abx500_dump_all_banks(void);
struct abx500_ops {
int (*get_chip_id) (struct device *);
diff --git a/include/linux/mfd/arizona/registers.h b/include/linux/mfd/arizona/registers.h
index 7b35c21..7204d81 100644
--- a/include/linux/mfd/arizona/registers.h
+++ b/include/linux/mfd/arizona/registers.h
@@ -42,12 +42,14 @@
#define ARIZONA_SAMPLE_RATE_SEQUENCE_SELECT_2 0x62
#define ARIZONA_SAMPLE_RATE_SEQUENCE_SELECT_3 0x63
#define ARIZONA_SAMPLE_RATE_SEQUENCE_SELECT_4 0x64
-#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_1 0x68
-#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_2 0x69
-#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_3 0x6A
-#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_4 0x6B
-#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_5 0x6C
-#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_6 0x6D
+#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_1 0x66
+#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_2 0x67
+#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_3 0x68
+#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_4 0x69
+#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_5 0x6A
+#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_6 0x6B
+#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_7 0x6C
+#define ARIZONA_ALWAYS_ON_TRIGGERS_SEQUENCE_SELECT_8 0x6D
#define ARIZONA_COMFORT_NOISE_GENERATOR 0x70
#define ARIZONA_HAPTICS_CONTROL_1 0x90
#define ARIZONA_HAPTICS_CONTROL_2 0x91
diff --git a/include/linux/mfd/axp20x.h b/include/linux/mfd/axp20x.h
new file mode 100644
index 0000000..d0e31a2
--- /dev/null
+++ b/include/linux/mfd/axp20x.h
@@ -0,0 +1,180 @@
+/*
+ * Functions and registers to access AXP20X power management chip.
+ *
+ * Copyright (C) 2013, Carlo Caione <carlo@caione.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __LINUX_MFD_AXP20X_H
+#define __LINUX_MFD_AXP20X_H
+
+enum {
+ AXP202_ID = 0,
+ AXP209_ID,
+};
+
+#define AXP20X_DATACACHE(m) (0x04 + (m))
+
+/* Power supply */
+#define AXP20X_PWR_INPUT_STATUS 0x00
+#define AXP20X_PWR_OP_MODE 0x01
+#define AXP20X_USB_OTG_STATUS 0x02
+#define AXP20X_PWR_OUT_CTRL 0x12
+#define AXP20X_DCDC2_V_OUT 0x23
+#define AXP20X_DCDC2_LDO3_V_SCAL 0x25
+#define AXP20X_DCDC3_V_OUT 0x27
+#define AXP20X_LDO24_V_OUT 0x28
+#define AXP20X_LDO3_V_OUT 0x29
+#define AXP20X_VBUS_IPSOUT_MGMT 0x30
+#define AXP20X_V_OFF 0x31
+#define AXP20X_OFF_CTRL 0x32
+#define AXP20X_CHRG_CTRL1 0x33
+#define AXP20X_CHRG_CTRL2 0x34
+#define AXP20X_CHRG_BAK_CTRL 0x35
+#define AXP20X_PEK_KEY 0x36
+#define AXP20X_DCDC_FREQ 0x37
+#define AXP20X_V_LTF_CHRG 0x38
+#define AXP20X_V_HTF_CHRG 0x39
+#define AXP20X_APS_WARN_L1 0x3a
+#define AXP20X_APS_WARN_L2 0x3b
+#define AXP20X_V_LTF_DISCHRG 0x3c
+#define AXP20X_V_HTF_DISCHRG 0x3d
+
+/* Interrupt */
+#define AXP20X_IRQ1_EN 0x40
+#define AXP20X_IRQ2_EN 0x41
+#define AXP20X_IRQ3_EN 0x42
+#define AXP20X_IRQ4_EN 0x43
+#define AXP20X_IRQ5_EN 0x44
+#define AXP20X_IRQ1_STATE 0x48
+#define AXP20X_IRQ2_STATE 0x49
+#define AXP20X_IRQ3_STATE 0x4a
+#define AXP20X_IRQ4_STATE 0x4b
+#define AXP20X_IRQ5_STATE 0x4c
+
+/* ADC */
+#define AXP20X_ACIN_V_ADC_H 0x56
+#define AXP20X_ACIN_V_ADC_L 0x57
+#define AXP20X_ACIN_I_ADC_H 0x58
+#define AXP20X_ACIN_I_ADC_L 0x59
+#define AXP20X_VBUS_V_ADC_H 0x5a
+#define AXP20X_VBUS_V_ADC_L 0x5b
+#define AXP20X_VBUS_I_ADC_H 0x5c
+#define AXP20X_VBUS_I_ADC_L 0x5d
+#define AXP20X_TEMP_ADC_H 0x5e
+#define AXP20X_TEMP_ADC_L 0x5f
+#define AXP20X_TS_IN_H 0x62
+#define AXP20X_TS_IN_L 0x63
+#define AXP20X_GPIO0_V_ADC_H 0x64
+#define AXP20X_GPIO0_V_ADC_L 0x65
+#define AXP20X_GPIO1_V_ADC_H 0x66
+#define AXP20X_GPIO1_V_ADC_L 0x67
+#define AXP20X_PWR_BATT_H 0x70
+#define AXP20X_PWR_BATT_M 0x71
+#define AXP20X_PWR_BATT_L 0x72
+#define AXP20X_BATT_V_H 0x78
+#define AXP20X_BATT_V_L 0x79
+#define AXP20X_BATT_CHRG_I_H 0x7a
+#define AXP20X_BATT_CHRG_I_L 0x7b
+#define AXP20X_BATT_DISCHRG_I_H 0x7c
+#define AXP20X_BATT_DISCHRG_I_L 0x7d
+#define AXP20X_IPSOUT_V_HIGH_H 0x7e
+#define AXP20X_IPSOUT_V_HIGH_L 0x7f
+
+/* Power supply */
+#define AXP20X_DCDC_MODE 0x80
+#define AXP20X_ADC_EN1 0x82
+#define AXP20X_ADC_EN2 0x83
+#define AXP20X_ADC_RATE 0x84
+#define AXP20X_GPIO10_IN_RANGE 0x85
+#define AXP20X_GPIO1_ADC_IRQ_RIS 0x86
+#define AXP20X_GPIO1_ADC_IRQ_FAL 0x87
+#define AXP20X_TIMER_CTRL 0x8a
+#define AXP20X_VBUS_MON 0x8b
+#define AXP20X_OVER_TMP 0x8f
+
+/* GPIO */
+#define AXP20X_GPIO0_CTRL 0x90
+#define AXP20X_LDO5_V_OUT 0x91
+#define AXP20X_GPIO1_CTRL 0x92
+#define AXP20X_GPIO2_CTRL 0x93
+#define AXP20X_GPIO20_SS 0x94
+#define AXP20X_GPIO3_CTRL 0x95
+
+/* Battery */
+#define AXP20X_CHRG_CC_31_24 0xb0
+#define AXP20X_CHRG_CC_23_16 0xb1
+#define AXP20X_CHRG_CC_15_8 0xb2
+#define AXP20X_CHRG_CC_7_0 0xb3
+#define AXP20X_DISCHRG_CC_31_24 0xb4
+#define AXP20X_DISCHRG_CC_23_16 0xb5
+#define AXP20X_DISCHRG_CC_15_8 0xb6
+#define AXP20X_DISCHRG_CC_7_0 0xb7
+#define AXP20X_CC_CTRL 0xb8
+#define AXP20X_FG_RES 0xb9
+
+/* Regulators IDs */
+enum {
+ AXP20X_LDO1 = 0,
+ AXP20X_LDO2,
+ AXP20X_LDO3,
+ AXP20X_LDO4,
+ AXP20X_LDO5,
+ AXP20X_DCDC2,
+ AXP20X_DCDC3,
+ AXP20X_REG_ID_MAX,
+};
+
+/* IRQs */
+enum {
+ AXP20X_IRQ_ACIN_OVER_V = 1,
+ AXP20X_IRQ_ACIN_PLUGIN,
+ AXP20X_IRQ_ACIN_REMOVAL,
+ AXP20X_IRQ_VBUS_OVER_V,
+ AXP20X_IRQ_VBUS_PLUGIN,
+ AXP20X_IRQ_VBUS_REMOVAL,
+ AXP20X_IRQ_VBUS_V_LOW,
+ AXP20X_IRQ_BATT_PLUGIN,
+ AXP20X_IRQ_BATT_REMOVAL,
+ AXP20X_IRQ_BATT_ENT_ACT_MODE,
+ AXP20X_IRQ_BATT_EXIT_ACT_MODE,
+ AXP20X_IRQ_CHARG,
+ AXP20X_IRQ_CHARG_DONE,
+ AXP20X_IRQ_BATT_TEMP_HIGH,
+ AXP20X_IRQ_BATT_TEMP_LOW,
+ AXP20X_IRQ_DIE_TEMP_HIGH,
+ AXP20X_IRQ_CHARG_I_LOW,
+ AXP20X_IRQ_DCDC1_V_LONG,
+ AXP20X_IRQ_DCDC2_V_LONG,
+ AXP20X_IRQ_DCDC3_V_LONG,
+ AXP20X_IRQ_PEK_SHORT = 22,
+ AXP20X_IRQ_PEK_LONG,
+ AXP20X_IRQ_N_OE_PWR_ON,
+ AXP20X_IRQ_N_OE_PWR_OFF,
+ AXP20X_IRQ_VBUS_VALID,
+ AXP20X_IRQ_VBUS_NOT_VALID,
+ AXP20X_IRQ_VBUS_SESS_VALID,
+ AXP20X_IRQ_VBUS_SESS_END,
+ AXP20X_IRQ_LOW_PWR_LVL1,
+ AXP20X_IRQ_LOW_PWR_LVL2,
+ AXP20X_IRQ_TIMER,
+ AXP20X_IRQ_PEK_RIS_EDGE,
+ AXP20X_IRQ_PEK_FAL_EDGE,
+ AXP20X_IRQ_GPIO3_INPUT,
+ AXP20X_IRQ_GPIO2_INPUT,
+ AXP20X_IRQ_GPIO1_INPUT,
+ AXP20X_IRQ_GPIO0_INPUT,
+};
+
+struct axp20x_dev {
+ struct device *dev;
+ struct i2c_client *i2c_client;
+ struct regmap *regmap;
+ struct regmap_irq_chip_data *regmap_irqc;
+ long variant;
+};
+
+#endif /* __LINUX_MFD_AXP20X_H */
diff --git a/include/linux/mfd/cros_ec.h b/include/linux/mfd/cros_ec.h
index 032af7f..887ef4f 100644
--- a/include/linux/mfd/cros_ec.h
+++ b/include/linux/mfd/cros_ec.h
@@ -29,8 +29,8 @@
EC_MSG_RX_PROTO_BYTES = 3,
/* Max length of messages */
- EC_MSG_BYTES = EC_HOST_PARAM_SIZE + EC_MSG_TX_PROTO_BYTES,
-
+ EC_MSG_BYTES = EC_PROTO2_MAX_PARAM_SIZE +
+ EC_MSG_TX_PROTO_BYTES,
};
/**
diff --git a/include/linux/mfd/cros_ec_commands.h b/include/linux/mfd/cros_ec_commands.h
index 86fd069..7853a64 100644
--- a/include/linux/mfd/cros_ec_commands.h
+++ b/include/linux/mfd/cros_ec_commands.h
@@ -24,25 +24,12 @@
#define __CROS_EC_COMMANDS_H
/*
- * Protocol overview
+ * Current version of this protocol
*
- * request: CMD [ P0 P1 P2 ... Pn S ]
- * response: ERR [ P0 P1 P2 ... Pn S ]
- *
- * where the bytes are defined as follow :
- * - CMD is the command code. (defined by EC_CMD_ constants)
- * - ERR is the error code. (defined by EC_RES_ constants)
- * - Px is the optional payload.
- * it is not sent if the error code is not success.
- * (defined by ec_params_ and ec_response_ structures)
- * - S is the checksum which is the sum of all payload bytes.
- *
- * On LPC, CMD and ERR are sent/received at EC_LPC_ADDR_KERNEL|USER_CMD
- * and the payloads are sent/received at EC_LPC_ADDR_KERNEL|USER_PARAM.
- * On I2C, all bytes are sent serially in the same message.
+ * TODO(crosbug.com/p/11223): This is effectively useless; protocol is
+ * determined in other ways. Remove this once the kernel code no longer
+ * depends on it.
*/
-
-/* Current version of this protocol */
#define EC_PROTO_VERSION 0x00000002
/* Command version mask */
@@ -57,13 +44,19 @@
#define EC_LPC_ADDR_HOST_CMD 0x204
/* I/O addresses for host command args and params */
-#define EC_LPC_ADDR_HOST_ARGS 0x800
-#define EC_LPC_ADDR_HOST_PARAM 0x804
-#define EC_HOST_PARAM_SIZE 0x0fc /* Size of param area in bytes */
+/* Protocol version 2 */
+#define EC_LPC_ADDR_HOST_ARGS 0x800 /* And 0x801, 0x802, 0x803 */
+#define EC_LPC_ADDR_HOST_PARAM 0x804 /* For version 2 params; size is
+ * EC_PROTO2_MAX_PARAM_SIZE */
+/* Protocol version 3 */
+#define EC_LPC_ADDR_HOST_PACKET 0x800 /* Offset of version 3 packet */
+#define EC_LPC_HOST_PACKET_SIZE 0x100 /* Max size of version 3 packet */
-/* I/O addresses for host command params, old interface */
-#define EC_LPC_ADDR_OLD_PARAM 0x880
-#define EC_OLD_PARAM_SIZE 0x080 /* Size of param area in bytes */
+/* The actual block is 0x800-0x8ff, but some BIOSes think it's 0x880-0x8ff
+ * and they tell the kernel that so we have to think of it as two parts. */
+#define EC_HOST_CMD_REGION0 0x800
+#define EC_HOST_CMD_REGION1 0x880
+#define EC_HOST_CMD_REGION_SIZE 0x80
/* EC command register bit functions */
#define EC_LPC_CMDR_DATA (1 << 0) /* Data ready for host to read */
@@ -79,18 +72,22 @@
#define EC_MEMMAP_TEXT_MAX 8 /* Size of a string in the memory map */
/* The offset address of each type of data in mapped memory. */
-#define EC_MEMMAP_TEMP_SENSOR 0x00 /* Temp sensors */
-#define EC_MEMMAP_FAN 0x10 /* Fan speeds */
-#define EC_MEMMAP_TEMP_SENSOR_B 0x18 /* Temp sensors (second set) */
-#define EC_MEMMAP_ID 0x20 /* 'E' 'C' */
+#define EC_MEMMAP_TEMP_SENSOR 0x00 /* Temp sensors 0x00 - 0x0f */
+#define EC_MEMMAP_FAN 0x10 /* Fan speeds 0x10 - 0x17 */
+#define EC_MEMMAP_TEMP_SENSOR_B 0x18 /* More temp sensors 0x18 - 0x1f */
+#define EC_MEMMAP_ID 0x20 /* 0x20 == 'E', 0x21 == 'C' */
#define EC_MEMMAP_ID_VERSION 0x22 /* Version of data in 0x20 - 0x2f */
#define EC_MEMMAP_THERMAL_VERSION 0x23 /* Version of data in 0x00 - 0x1f */
#define EC_MEMMAP_BATTERY_VERSION 0x24 /* Version of data in 0x40 - 0x7f */
#define EC_MEMMAP_SWITCHES_VERSION 0x25 /* Version of data in 0x30 - 0x33 */
#define EC_MEMMAP_EVENTS_VERSION 0x26 /* Version of data in 0x34 - 0x3f */
-#define EC_MEMMAP_HOST_CMD_FLAGS 0x27 /* Host command interface flags */
-#define EC_MEMMAP_SWITCHES 0x30
-#define EC_MEMMAP_HOST_EVENTS 0x34
+#define EC_MEMMAP_HOST_CMD_FLAGS 0x27 /* Host cmd interface flags (8 bits) */
+/* Unused 0x28 - 0x2f */
+#define EC_MEMMAP_SWITCHES 0x30 /* 8 bits */
+/* Unused 0x31 - 0x33 */
+#define EC_MEMMAP_HOST_EVENTS 0x34 /* 32 bits */
+/* Reserve 0x38 - 0x3f for additional host event-related stuff */
+/* Battery values are all 32 bits */
#define EC_MEMMAP_BATT_VOLT 0x40 /* Battery Present Voltage */
#define EC_MEMMAP_BATT_RATE 0x44 /* Battery Present Rate */
#define EC_MEMMAP_BATT_CAP 0x48 /* Battery Remaining Capacity */
@@ -99,10 +96,24 @@
#define EC_MEMMAP_BATT_DVLT 0x54 /* Battery Design Voltage */
#define EC_MEMMAP_BATT_LFCC 0x58 /* Battery Last Full Charge Capacity */
#define EC_MEMMAP_BATT_CCNT 0x5c /* Battery Cycle Count */
+/* Strings are all 8 bytes (EC_MEMMAP_TEXT_MAX) */
#define EC_MEMMAP_BATT_MFGR 0x60 /* Battery Manufacturer String */
#define EC_MEMMAP_BATT_MODEL 0x68 /* Battery Model Number String */
#define EC_MEMMAP_BATT_SERIAL 0x70 /* Battery Serial Number String */
#define EC_MEMMAP_BATT_TYPE 0x78 /* Battery Type String */
+#define EC_MEMMAP_ALS 0x80 /* ALS readings in lux (2 X 16 bits) */
+/* Unused 0x84 - 0x8f */
+#define EC_MEMMAP_ACC_STATUS 0x90 /* Accelerometer status (8 bits )*/
+/* Unused 0x91 */
+#define EC_MEMMAP_ACC_DATA 0x92 /* Accelerometer data 0x92 - 0x9f */
+#define EC_MEMMAP_GYRO_DATA 0xa0 /* Gyroscope data 0xa0 - 0xa5 */
+/* Unused 0xa6 - 0xfe (remember, 0xff is NOT part of the memmap region) */
+
+
+/* Define the format of the accelerometer mapped memory status byte. */
+#define EC_MEMMAP_ACC_STATUS_SAMPLE_ID_MASK 0x0f
+#define EC_MEMMAP_ACC_STATUS_BUSY_BIT (1 << 4)
+#define EC_MEMMAP_ACC_STATUS_PRESENCE_BIT (1 << 7)
/* Number of temp sensors at EC_MEMMAP_TEMP_SENSOR */
#define EC_TEMP_SENSOR_ENTRIES 16
@@ -112,6 +123,8 @@
* Valid only if EC_MEMMAP_THERMAL_VERSION returns >= 2.
*/
#define EC_TEMP_SENSOR_B_ENTRIES 8
+
+/* Special values for mapped temperature sensors */
#define EC_TEMP_SENSOR_NOT_PRESENT 0xff
#define EC_TEMP_SENSOR_ERROR 0xfe
#define EC_TEMP_SENSOR_NOT_POWERED 0xfd
@@ -122,6 +135,18 @@
*/
#define EC_TEMP_SENSOR_OFFSET 200
+/*
+ * Number of ALS readings at EC_MEMMAP_ALS
+ */
+#define EC_ALS_ENTRIES 2
+
+/*
+ * The default value a temperature sensor will return when it is present but
+ * has not been read this boot. This is a reasonable number to avoid
+ * triggering alarms on the host.
+ */
+#define EC_TEMP_SENSOR_DEFAULT (296 - EC_TEMP_SENSOR_OFFSET)
+
#define EC_FAN_SPEED_ENTRIES 4 /* Number of fans at EC_MEMMAP_FAN */
#define EC_FAN_SPEED_NOT_PRESENT 0xffff /* Entry not present */
#define EC_FAN_SPEED_STALLED 0xfffe /* Fan stalled */
@@ -137,8 +162,8 @@
#define EC_SWITCH_LID_OPEN 0x01
#define EC_SWITCH_POWER_BUTTON_PRESSED 0x02
#define EC_SWITCH_WRITE_PROTECT_DISABLED 0x04
-/* Recovery requested via keyboard */
-#define EC_SWITCH_KEYBOARD_RECOVERY 0x08
+/* Was recovery requested via keyboard; now unused. */
+#define EC_SWITCH_IGNORE1 0x08
/* Recovery requested via dedicated signal (from servo board) */
#define EC_SWITCH_DEDICATED_RECOVERY 0x10
/* Was fake developer mode switch; now unused. Remove in next refactor. */
@@ -147,10 +172,15 @@
/* Host command interface flags */
/* Host command interface supports LPC args (LPC interface only) */
#define EC_HOST_CMD_FLAG_LPC_ARGS_SUPPORTED 0x01
+/* Host command interface supports version 3 protocol */
+#define EC_HOST_CMD_FLAG_VERSION_3 0x02
/* Wireless switch flags */
-#define EC_WIRELESS_SWITCH_WLAN 0x01
-#define EC_WIRELESS_SWITCH_BLUETOOTH 0x02
+#define EC_WIRELESS_SWITCH_ALL ~0x00 /* All flags */
+#define EC_WIRELESS_SWITCH_WLAN 0x01 /* WLAN radio */
+#define EC_WIRELESS_SWITCH_BLUETOOTH 0x02 /* Bluetooth radio */
+#define EC_WIRELESS_SWITCH_WWAN 0x04 /* WWAN power */
+#define EC_WIRELESS_SWITCH_WLAN_POWER 0x08 /* WLAN power */
/*
* This header file is used in coreboot both in C and ACPI code. The ACPI code
@@ -159,6 +189,14 @@
*/
#ifndef __ACPI__
+/*
+ * Define __packed if someone hasn't beat us to it. Linux kernel style
+ * checking prefers __packed over __attribute__((packed)).
+ */
+#ifndef __packed
+#define __packed __attribute__((packed))
+#endif
+
/* LPC command status byte masks */
/* EC has written a byte in the data register and host hasn't read it yet */
#define EC_LPC_STATUS_TO_HOST 0x01
@@ -198,6 +236,9 @@
EC_RES_UNAVAILABLE = 9, /* No response available */
EC_RES_TIMEOUT = 10, /* We got a timeout */
EC_RES_OVERFLOW = 11, /* Table / data overflow */
+ EC_RES_INVALID_HEADER = 12, /* Header contains invalid data */
+ EC_RES_REQUEST_TRUNCATED = 13, /* Didn't get the entire request */
+ EC_RES_RESPONSE_TOO_BIG = 14 /* Response was too big to handle */
};
/*
@@ -235,6 +276,16 @@
/* Shutdown due to battery level too low */
EC_HOST_EVENT_BATTERY_SHUTDOWN = 17,
+ /* Suggest that the AP throttle itself */
+ EC_HOST_EVENT_THROTTLE_START = 18,
+ /* Suggest that the AP resume normal speed */
+ EC_HOST_EVENT_THROTTLE_STOP = 19,
+
+ /* Hang detect logic detected a hang and host event timeout expired */
+ EC_HOST_EVENT_HANG_DETECT = 20,
+ /* Hang detect logic detected a hang and warm rebooted the AP */
+ EC_HOST_EVENT_HANG_REBOOT = 21,
+
/*
* The high bit of the event mask is not used as a host event code. If
* it reads back as set, then the entire event mask should be
@@ -279,6 +330,188 @@
*/
#define EC_HOST_ARGS_FLAG_TO_HOST 0x02
+/*****************************************************************************/
+/*
+ * Byte codes returned by EC over SPI interface.
+ *
+ * These can be used by the AP to debug the EC interface, and to determine
+ * when the EC is not in a state where it will ever get around to responding
+ * to the AP.
+ *
+ * Example of sequence of bytes read from EC for a current good transfer:
+ * 1. - - AP asserts chip select (CS#)
+ * 2. EC_SPI_OLD_READY - AP sends first byte(s) of request
+ * 3. - - EC starts handling CS# interrupt
+ * 4. EC_SPI_RECEIVING - AP sends remaining byte(s) of request
+ * 5. EC_SPI_PROCESSING - EC starts processing request; AP is clocking in
+ * bytes looking for EC_SPI_FRAME_START
+ * 6. - - EC finishes processing and sets up response
+ * 7. EC_SPI_FRAME_START - AP reads frame byte
+ * 8. (response packet) - AP reads response packet
+ * 9. EC_SPI_PAST_END - Any additional bytes read by AP
+ * 10 - - AP deasserts chip select
+ * 11 - - EC processes CS# interrupt and sets up DMA for
+ * next request
+ *
+ * If the AP is waiting for EC_SPI_FRAME_START and sees any value other than
+ * the following byte values:
+ * EC_SPI_OLD_READY
+ * EC_SPI_RX_READY
+ * EC_SPI_RECEIVING
+ * EC_SPI_PROCESSING
+ *
+ * Then the EC found an error in the request, or was not ready for the request
+ * and lost data. The AP should give up waiting for EC_SPI_FRAME_START,
+ * because the EC is unable to tell when the AP is done sending its request.
+ */
+
+/*
+ * Framing byte which precedes a response packet from the EC. After sending a
+ * request, the AP will clock in bytes until it sees the framing byte, then
+ * clock in the response packet.
+ */
+#define EC_SPI_FRAME_START 0xec
+
+/*
+ * Padding bytes which are clocked out after the end of a response packet.
+ */
+#define EC_SPI_PAST_END 0xed
+
+/*
+ * EC is ready to receive, and has ignored the byte sent by the AP. EC expects
+ * that the AP will send a valid packet header (starting with
+ * EC_COMMAND_PROTOCOL_3) in the next 32 bytes.
+ */
+#define EC_SPI_RX_READY 0xf8
+
+/*
+ * EC has started receiving the request from the AP, but hasn't started
+ * processing it yet.
+ */
+#define EC_SPI_RECEIVING 0xf9
+
+/* EC has received the entire request from the AP and is processing it. */
+#define EC_SPI_PROCESSING 0xfa
+
+/*
+ * EC received bad data from the AP, such as a packet header with an invalid
+ * length. EC will ignore all data until chip select deasserts.
+ */
+#define EC_SPI_RX_BAD_DATA 0xfb
+
+/*
+ * EC received data from the AP before it was ready. That is, the AP asserted
+ * chip select and started clocking data before the EC was ready to receive it.
+ * EC will ignore all data until chip select deasserts.
+ */
+#define EC_SPI_NOT_READY 0xfc
+
+/*
+ * EC was ready to receive a request from the AP. EC has treated the byte sent
+ * by the AP as part of a request packet, or (for old-style ECs) is processing
+ * a fully received packet but is not ready to respond yet.
+ */
+#define EC_SPI_OLD_READY 0xfd
+
+/*****************************************************************************/
+
+/*
+ * Protocol version 2 for I2C and SPI send a request this way:
+ *
+ * 0 EC_CMD_VERSION0 + (command version)
+ * 1 Command number
+ * 2 Length of params = N
+ * 3..N+2 Params, if any
+ * N+3 8-bit checksum of bytes 0..N+2
+ *
+ * The corresponding response is:
+ *
+ * 0 Result code (EC_RES_*)
+ * 1 Length of params = M
+ * 2..M+1 Params, if any
+ * M+2 8-bit checksum of bytes 0..M+1
+ */
+#define EC_PROTO2_REQUEST_HEADER_BYTES 3
+#define EC_PROTO2_REQUEST_TRAILER_BYTES 1
+#define EC_PROTO2_REQUEST_OVERHEAD (EC_PROTO2_REQUEST_HEADER_BYTES + \
+ EC_PROTO2_REQUEST_TRAILER_BYTES)
+
+#define EC_PROTO2_RESPONSE_HEADER_BYTES 2
+#define EC_PROTO2_RESPONSE_TRAILER_BYTES 1
+#define EC_PROTO2_RESPONSE_OVERHEAD (EC_PROTO2_RESPONSE_HEADER_BYTES + \
+ EC_PROTO2_RESPONSE_TRAILER_BYTES)
+
+/* Parameter length was limited by the LPC interface */
+#define EC_PROTO2_MAX_PARAM_SIZE 0xfc
+
+/* Maximum request and response packet sizes for protocol version 2 */
+#define EC_PROTO2_MAX_REQUEST_SIZE (EC_PROTO2_REQUEST_OVERHEAD + \
+ EC_PROTO2_MAX_PARAM_SIZE)
+#define EC_PROTO2_MAX_RESPONSE_SIZE (EC_PROTO2_RESPONSE_OVERHEAD + \
+ EC_PROTO2_MAX_PARAM_SIZE)
+
+/*****************************************************************************/
+
+/*
+ * Value written to legacy command port / prefix byte to indicate protocol
+ * 3+ structs are being used. Usage is bus-dependent.
+ */
+#define EC_COMMAND_PROTOCOL_3 0xda
+
+#define EC_HOST_REQUEST_VERSION 3
+
+/* Version 3 request from host */
+struct ec_host_request {
+ /* Struct version (=3)
+ *
+ * EC will return EC_RES_INVALID_HEADER if it receives a header with a
+ * version it doesn't know how to parse.
+ */
+ uint8_t struct_version;
+
+ /*
+ * Checksum of request and data; sum of all bytes including checksum
+ * should total to 0.
+ */
+ uint8_t checksum;
+
+ /* Command code */
+ uint16_t command;
+
+ /* Command version */
+ uint8_t command_version;
+
+ /* Unused byte in current protocol version; set to 0 */
+ uint8_t reserved;
+
+ /* Length of data which follows this header */
+ uint16_t data_len;
+} __packed;
+
+#define EC_HOST_RESPONSE_VERSION 3
+
+/* Version 3 response from EC */
+struct ec_host_response {
+ /* Struct version (=3) */
+ uint8_t struct_version;
+
+ /*
+ * Checksum of response and data; sum of all bytes including checksum
+ * should total to 0.
+ */
+ uint8_t checksum;
+
+ /* Result code (EC_RES_*) */
+ uint16_t result;
+
+ /* Length of data which follows this header */
+ uint16_t data_len;
+
+ /* Unused bytes in current protocol version; set to 0 */
+ uint16_t reserved;
+} __packed;
+
+/*****************************************************************************/
/*
* Notes on commands:
*
@@ -418,6 +651,68 @@
uint32_t flags; /* Mask of enum ec_comms_status */
} __packed;
+/* Fake a variety of responses, purely for testing purposes. */
+#define EC_CMD_TEST_PROTOCOL 0x0a
+
+/* Tell the EC what to send back to us. */
+struct ec_params_test_protocol {
+ uint32_t ec_result;
+ uint32_t ret_len;
+ uint8_t buf[32];
+} __packed;
+
+/* Here it comes... */
+struct ec_response_test_protocol {
+ uint8_t buf[32];
+} __packed;
+
+/* Get prococol information */
+#define EC_CMD_GET_PROTOCOL_INFO 0x0b
+
+/* Flags for ec_response_get_protocol_info.flags */
+/* EC_RES_IN_PROGRESS may be returned if a command is slow */
+#define EC_PROTOCOL_INFO_IN_PROGRESS_SUPPORTED (1 << 0)
+
+struct ec_response_get_protocol_info {
+ /* Fields which exist if at least protocol version 3 supported */
+
+ /* Bitmask of protocol versions supported (1 << n means version n)*/
+ uint32_t protocol_versions;
+
+ /* Maximum request packet size, in bytes */
+ uint16_t max_request_packet_size;
+
+ /* Maximum response packet size, in bytes */
+ uint16_t max_response_packet_size;
+
+ /* Flags; see EC_PROTOCOL_INFO_* */
+ uint32_t flags;
+} __packed;
+
+
+/*****************************************************************************/
+/* Get/Set miscellaneous values */
+
+/* The upper byte of .flags tells what to do (nothing means "get") */
+#define EC_GSV_SET 0x80000000
+
+/* The lower three bytes of .flags identifies the parameter, if that has
+ meaning for an individual command. */
+#define EC_GSV_PARAM_MASK 0x00ffffff
+
+struct ec_params_get_set_value {
+ uint32_t flags;
+ uint32_t value;
+} __packed;
+
+struct ec_response_get_set_value {
+ uint32_t flags;
+ uint32_t value;
+} __packed;
+
+/* More than one command can use these structs to get/set paramters. */
+#define EC_CMD_GSV_PAUSE_IN_S5 0x0c
+
/*****************************************************************************/
/* Flash commands */
@@ -425,6 +720,7 @@
/* Get flash info */
#define EC_CMD_FLASH_INFO 0x10
+/* Version 0 returns these fields */
struct ec_response_flash_info {
/* Usable flash size, in bytes */
uint32_t flash_size;
@@ -445,6 +741,37 @@
uint32_t protect_block_size;
} __packed;
+/* Flags for version 1+ flash info command */
+/* EC flash erases bits to 0 instead of 1 */
+#define EC_FLASH_INFO_ERASE_TO_0 (1 << 0)
+
+/*
+ * Version 1 returns the same initial fields as version 0, with additional
+ * fields following.
+ *
+ * gcc anonymous structs don't seem to get along with the __packed directive;
+ * if they did we'd define the version 0 struct as a sub-struct of this one.
+ */
+struct ec_response_flash_info_1 {
+ /* Version 0 fields; see above for description */
+ uint32_t flash_size;
+ uint32_t write_block_size;
+ uint32_t erase_block_size;
+ uint32_t protect_block_size;
+
+ /* Version 1 adds these fields: */
+ /*
+ * Ideal write size in bytes. Writes will be fastest if size is
+ * exactly this and offset is a multiple of this. For example, an EC
+ * may have a write buffer which can do half-page operations if data is
+ * aligned, and a slower word-at-a-time write mode.
+ */
+ uint32_t write_ideal_size;
+
+ /* Flags; see EC_FLASH_INFO_* */
+ uint32_t flags;
+} __packed;
+
/*
* Read flash
*
@@ -459,15 +786,15 @@
/* Write flash */
#define EC_CMD_FLASH_WRITE 0x12
+#define EC_VER_FLASH_WRITE 1
+
+/* Version 0 of the flash command supported only 64 bytes of data */
+#define EC_FLASH_WRITE_VER0_SIZE 64
struct ec_params_flash_write {
uint32_t offset; /* Byte offset to write */
uint32_t size; /* Size to write in bytes */
- /*
- * Data to write. Could really use EC_PARAM_SIZE - 8, but tidiest to
- * use a power of 2 so writes stay aligned.
- */
- uint8_t data[64];
+ /* Followed by data to write */
} __packed;
/* Erase flash */
@@ -543,7 +870,7 @@
enum ec_flash_region {
/* Region which holds read-only EC image */
- EC_FLASH_REGION_RO,
+ EC_FLASH_REGION_RO = 0,
/* Region which holds rewritable EC image */
EC_FLASH_REGION_RW,
/*
@@ -551,6 +878,8 @@
* EC_FLASH_REGION_RO)
*/
EC_FLASH_REGION_WP_RO,
+ /* Number of regions */
+ EC_FLASH_REGION_COUNT,
};
struct ec_params_flash_region_info {
@@ -639,15 +968,15 @@
*/
struct lightbar_params {
/* Timing */
- int google_ramp_up;
- int google_ramp_down;
- int s3s0_ramp_up;
- int s0_tick_delay[2]; /* AC=0/1 */
- int s0a_tick_delay[2]; /* AC=0/1 */
- int s0s3_ramp_down;
- int s3_sleep_for;
- int s3_ramp_up;
- int s3_ramp_down;
+ int32_t google_ramp_up;
+ int32_t google_ramp_down;
+ int32_t s3s0_ramp_up;
+ int32_t s0_tick_delay[2]; /* AC=0/1 */
+ int32_t s0a_tick_delay[2]; /* AC=0/1 */
+ int32_t s0s3_ramp_down;
+ int32_t s3_sleep_for;
+ int32_t s3_ramp_up;
+ int32_t s3_ramp_down;
/* Oscillation */
uint8_t new_s0;
@@ -676,7 +1005,7 @@
union {
struct {
/* no args */
- } dump, off, on, init, get_seq, get_params;
+ } dump, off, on, init, get_seq, get_params, version;
struct num {
uint8_t num;
@@ -710,6 +1039,11 @@
struct lightbar_params get_params;
+ struct version {
+ uint32_t num;
+ uint32_t flags;
+ } version;
+
struct {
/* no return params */
} off, on, init, brightness, seq, reg, rgb, demo, set_params;
@@ -730,10 +1064,62 @@
LIGHTBAR_CMD_DEMO = 9,
LIGHTBAR_CMD_GET_PARAMS = 10,
LIGHTBAR_CMD_SET_PARAMS = 11,
+ LIGHTBAR_CMD_VERSION = 12,
LIGHTBAR_NUM_CMDS
};
/*****************************************************************************/
+/* LED control commands */
+
+#define EC_CMD_LED_CONTROL 0x29
+
+enum ec_led_id {
+ /* LED to indicate battery state of charge */
+ EC_LED_ID_BATTERY_LED = 0,
+ /*
+ * LED to indicate system power state (on or in suspend).
+ * May be on power button or on C-panel.
+ */
+ EC_LED_ID_POWER_LED,
+ /* LED on power adapter or its plug */
+ EC_LED_ID_ADAPTER_LED,
+
+ EC_LED_ID_COUNT
+};
+
+/* LED control flags */
+#define EC_LED_FLAGS_QUERY (1 << 0) /* Query LED capability only */
+#define EC_LED_FLAGS_AUTO (1 << 1) /* Switch LED back to automatic control */
+
+enum ec_led_colors {
+ EC_LED_COLOR_RED = 0,
+ EC_LED_COLOR_GREEN,
+ EC_LED_COLOR_BLUE,
+ EC_LED_COLOR_YELLOW,
+ EC_LED_COLOR_WHITE,
+
+ EC_LED_COLOR_COUNT
+};
+
+struct ec_params_led_control {
+ uint8_t led_id; /* Which LED to control */
+ uint8_t flags; /* Control flags */
+
+ uint8_t brightness[EC_LED_COLOR_COUNT];
+} __packed;
+
+struct ec_response_led_control {
+ /*
+ * Available brightness value range.
+ *
+ * Range 0 means color channel not present.
+ * Range 1 means on/off control.
+ * Other values means the LED is control by PWM.
+ */
+ uint8_t brightness_range[EC_LED_COLOR_COUNT];
+} __packed;
+
+/*****************************************************************************/
/* Verified boot commands */
/*
@@ -790,6 +1176,181 @@
#define EC_VBOOT_HASH_OFFSET_RW 0xfffffffd
/*****************************************************************************/
+/*
+ * Motion sense commands. We'll make separate structs for sub-commands with
+ * different input args, so that we know how much to expect.
+ */
+#define EC_CMD_MOTION_SENSE_CMD 0x2B
+
+/* Motion sense commands */
+enum motionsense_command {
+ /*
+ * Dump command returns all motion sensor data including motion sense
+ * module flags and individual sensor flags.
+ */
+ MOTIONSENSE_CMD_DUMP = 0,
+
+ /*
+ * Info command returns data describing the details of a given sensor,
+ * including enum motionsensor_type, enum motionsensor_location, and
+ * enum motionsensor_chip.
+ */
+ MOTIONSENSE_CMD_INFO = 1,
+
+ /*
+ * EC Rate command is a setter/getter command for the EC sampling rate
+ * of all motion sensors in milliseconds.
+ */
+ MOTIONSENSE_CMD_EC_RATE = 2,
+
+ /*
+ * Sensor ODR command is a setter/getter command for the output data
+ * rate of a specific motion sensor in millihertz.
+ */
+ MOTIONSENSE_CMD_SENSOR_ODR = 3,
+
+ /*
+ * Sensor range command is a setter/getter command for the range of
+ * a specified motion sensor in +/-G's or +/- deg/s.
+ */
+ MOTIONSENSE_CMD_SENSOR_RANGE = 4,
+
+ /*
+ * Setter/getter command for the keyboard wake angle. When the lid
+ * angle is greater than this value, keyboard wake is disabled in S3,
+ * and when the lid angle goes less than this value, keyboard wake is
+ * enabled. Note, the lid angle measurement is an approximate,
+ * un-calibrated value, hence the wake angle isn't exact.
+ */
+ MOTIONSENSE_CMD_KB_WAKE_ANGLE = 5,
+
+ /* Number of motionsense sub-commands. */
+ MOTIONSENSE_NUM_CMDS
+};
+
+enum motionsensor_id {
+ EC_MOTION_SENSOR_ACCEL_BASE = 0,
+ EC_MOTION_SENSOR_ACCEL_LID = 1,
+ EC_MOTION_SENSOR_GYRO = 2,
+
+ /*
+ * Note, if more sensors are added and this count changes, the padding
+ * in ec_response_motion_sense dump command must be modified.
+ */
+ EC_MOTION_SENSOR_COUNT = 3
+};
+
+/* List of motion sensor types. */
+enum motionsensor_type {
+ MOTIONSENSE_TYPE_ACCEL = 0,
+ MOTIONSENSE_TYPE_GYRO = 1,
+};
+
+/* List of motion sensor locations. */
+enum motionsensor_location {
+ MOTIONSENSE_LOC_BASE = 0,
+ MOTIONSENSE_LOC_LID = 1,
+};
+
+/* List of motion sensor chips. */
+enum motionsensor_chip {
+ MOTIONSENSE_CHIP_KXCJ9 = 0,
+};
+
+/* Module flag masks used for the dump sub-command. */
+#define MOTIONSENSE_MODULE_FLAG_ACTIVE (1<<0)
+
+/* Sensor flag masks used for the dump sub-command. */
+#define MOTIONSENSE_SENSOR_FLAG_PRESENT (1<<0)
+
+/*
+ * Send this value for the data element to only perform a read. If you
+ * send any other value, the EC will interpret it as data to set and will
+ * return the actual value set.
+ */
+#define EC_MOTION_SENSE_NO_VALUE -1
+
+struct ec_params_motion_sense {
+ uint8_t cmd;
+ union {
+ /* Used for MOTIONSENSE_CMD_DUMP. */
+ struct {
+ /* no args */
+ } dump;
+
+ /*
+ * Used for MOTIONSENSE_CMD_EC_RATE and
+ * MOTIONSENSE_CMD_KB_WAKE_ANGLE.
+ */
+ struct {
+ /* Data to set or EC_MOTION_SENSE_NO_VALUE to read. */
+ int16_t data;
+ } ec_rate, kb_wake_angle;
+
+ /* Used for MOTIONSENSE_CMD_INFO. */
+ struct {
+ /* Should be element of enum motionsensor_id. */
+ uint8_t sensor_num;
+ } info;
+
+ /*
+ * Used for MOTIONSENSE_CMD_SENSOR_ODR and
+ * MOTIONSENSE_CMD_SENSOR_RANGE.
+ */
+ struct {
+ /* Should be element of enum motionsensor_id. */
+ uint8_t sensor_num;
+
+ /* Rounding flag, true for round-up, false for down. */
+ uint8_t roundup;
+
+ uint16_t reserved;
+
+ /* Data to set or EC_MOTION_SENSE_NO_VALUE to read. */
+ int32_t data;
+ } sensor_odr, sensor_range;
+ };
+} __packed;
+
+struct ec_response_motion_sense {
+ union {
+ /* Used for MOTIONSENSE_CMD_DUMP. */
+ struct {
+ /* Flags representing the motion sensor module. */
+ uint8_t module_flags;
+
+ /* Flags for each sensor in enum motionsensor_id. */
+ uint8_t sensor_flags[EC_MOTION_SENSOR_COUNT];
+
+ /* Array of all sensor data. Each sensor is 3-axis. */
+ int16_t data[3*EC_MOTION_SENSOR_COUNT];
+ } dump;
+
+ /* Used for MOTIONSENSE_CMD_INFO. */
+ struct {
+ /* Should be element of enum motionsensor_type. */
+ uint8_t type;
+
+ /* Should be element of enum motionsensor_location. */
+ uint8_t location;
+
+ /* Should be element of enum motionsensor_chip. */
+ uint8_t chip;
+ } info;
+
+ /*
+ * Used for MOTIONSENSE_CMD_EC_RATE, MOTIONSENSE_CMD_SENSOR_ODR,
+ * MOTIONSENSE_CMD_SENSOR_RANGE, and
+ * MOTIONSENSE_CMD_KB_WAKE_ANGLE.
+ */
+ struct {
+ /* Current value of the parameter queried. */
+ int32_t ret;
+ } ec_rate, sensor_odr, sensor_range, kb_wake_angle;
+ };
+} __packed;
+
+/*****************************************************************************/
/* USB charging control commands */
/* Set USB port charging mode */
@@ -868,20 +1429,27 @@
} __packed;
/*****************************************************************************/
-/* Thermal engine commands */
+/* Thermal engine commands. Note that there are two implementations. We'll
+ * reuse the command number, but the data and behavior is incompatible.
+ * Version 0 is what originally shipped on Link.
+ * Version 1 separates the CPU thermal limits from the fan control.
+ */
-/* Set thershold value */
#define EC_CMD_THERMAL_SET_THRESHOLD 0x50
+#define EC_CMD_THERMAL_GET_THRESHOLD 0x51
+/* The version 0 structs are opaque. You have to know what they are for
+ * the get/set commands to make any sense.
+ */
+
+/* Version 0 - set */
struct ec_params_thermal_set_threshold {
uint8_t sensor_type;
uint8_t threshold_id;
uint16_t value;
} __packed;
-/* Get threshold value */
-#define EC_CMD_THERMAL_GET_THRESHOLD 0x51
-
+/* Version 0 - get */
struct ec_params_thermal_get_threshold {
uint8_t sensor_type;
uint8_t threshold_id;
@@ -891,6 +1459,41 @@
uint16_t value;
} __packed;
+
+/* The version 1 structs are visible. */
+enum ec_temp_thresholds {
+ EC_TEMP_THRESH_WARN = 0,
+ EC_TEMP_THRESH_HIGH,
+ EC_TEMP_THRESH_HALT,
+
+ EC_TEMP_THRESH_COUNT
+};
+
+/* Thermal configuration for one temperature sensor. Temps are in degrees K.
+ * Zero values will be silently ignored by the thermal task.
+ */
+struct ec_thermal_config {
+ uint32_t temp_host[EC_TEMP_THRESH_COUNT]; /* levels of hotness */
+ uint32_t temp_fan_off; /* no active cooling needed */
+ uint32_t temp_fan_max; /* max active cooling needed */
+} __packed;
+
+/* Version 1 - get config for one sensor. */
+struct ec_params_thermal_get_threshold_v1 {
+ uint32_t sensor_num;
+} __packed;
+/* This returns a struct ec_thermal_config */
+
+/* Version 1 - set config for one sensor.
+ * Use read-modify-write for best results! */
+struct ec_params_thermal_set_threshold_v1 {
+ uint32_t sensor_num;
+ struct ec_thermal_config cfg;
+} __packed;
+/* This returns no data */
+
+/****************************************************************************/
+
/* Toggle automatic fan control */
#define EC_CMD_THERMAL_AUTO_FAN_CTRL 0x52
@@ -920,6 +1523,18 @@
float b2;
} __packed;
+/* Read raw TMP006 data */
+#define EC_CMD_TMP006_GET_RAW 0x55
+
+struct ec_params_tmp006_get_raw {
+ uint8_t index;
+} __packed;
+
+struct ec_response_tmp006_get_raw {
+ int32_t t; /* In 1/100 K */
+ int32_t v; /* In nV */
+};
+
/*****************************************************************************/
/* MKBP - Matrix KeyBoard Protocol */
@@ -1118,11 +1733,41 @@
/* Enable/disable WLAN/Bluetooth */
#define EC_CMD_SWITCH_ENABLE_WIRELESS 0x91
+#define EC_VER_SWITCH_ENABLE_WIRELESS 1
-struct ec_params_switch_enable_wireless {
+/* Version 0 params; no response */
+struct ec_params_switch_enable_wireless_v0 {
uint8_t enabled;
} __packed;
+/* Version 1 params */
+struct ec_params_switch_enable_wireless_v1 {
+ /* Flags to enable now */
+ uint8_t now_flags;
+
+ /* Which flags to copy from now_flags */
+ uint8_t now_mask;
+
+ /*
+ * Flags to leave enabled in S3, if they're on at the S0->S3
+ * transition. (Other flags will be disabled by the S0->S3
+ * transition.)
+ */
+ uint8_t suspend_flags;
+
+ /* Which flags to copy from suspend_flags */
+ uint8_t suspend_mask;
+} __packed;
+
+/* Version 1 response */
+struct ec_response_switch_enable_wireless_v1 {
+ /* Flags to enable now */
+ uint8_t now_flags;
+
+ /* Flags to leave enabled in S3 */
+ uint8_t suspend_flags;
+} __packed;
+
/*****************************************************************************/
/* GPIO commands. Only available on EC if write protect has been disabled. */
@@ -1147,11 +1792,16 @@
/*****************************************************************************/
/* I2C commands. Only available when flash write protect is unlocked. */
+/*
+ * TODO(crosbug.com/p/23570): These commands are deprecated, and will be
+ * removed soon. Use EC_CMD_I2C_XFER instead.
+ */
+
/* Read I2C bus */
#define EC_CMD_I2C_READ 0x94
struct ec_params_i2c_read {
- uint16_t addr;
+ uint16_t addr; /* 8-bit address (7-bit shifted << 1) */
uint8_t read_size; /* Either 8 or 16. */
uint8_t port;
uint8_t offset;
@@ -1165,7 +1815,7 @@
struct ec_params_i2c_write {
uint16_t data;
- uint16_t addr;
+ uint16_t addr; /* 8-bit address (7-bit shifted << 1) */
uint8_t write_size; /* Either 8 or 16. */
uint8_t port;
uint8_t offset;
@@ -1174,11 +1824,20 @@
/*****************************************************************************/
/* Charge state commands. Only available when flash write protect unlocked. */
-/* Force charge state machine to stop in idle mode */
-#define EC_CMD_CHARGE_FORCE_IDLE 0x96
+/* Force charge state machine to stop charging the battery or force it to
+ * discharge the battery.
+ */
+#define EC_CMD_CHARGE_CONTROL 0x96
+#define EC_VER_CHARGE_CONTROL 1
-struct ec_params_force_idle {
- uint8_t enabled;
+enum ec_charge_control_mode {
+ CHARGE_CONTROL_NORMAL = 0,
+ CHARGE_CONTROL_IDLE,
+ CHARGE_CONTROL_DISCHARGE,
+};
+
+struct ec_params_charge_control {
+ uint32_t mode; /* enum charge_control_mode */
} __packed;
/*****************************************************************************/
@@ -1206,14 +1865,231 @@
#define EC_CMD_BATTERY_CUT_OFF 0x99
/*****************************************************************************/
-/* Temporary debug commands. TODO: remove this crosbug.com/p/13849 */
+/* USB port mux control. */
/*
- * Dump charge state machine context.
- *
- * Response is a binary dump of charge state machine context.
+ * Switch USB mux or return to automatic switching.
*/
-#define EC_CMD_CHARGE_DUMP 0xa0
+#define EC_CMD_USB_MUX 0x9a
+
+struct ec_params_usb_mux {
+ uint8_t mux;
+} __packed;
+
+/*****************************************************************************/
+/* LDOs / FETs control. */
+
+enum ec_ldo_state {
+ EC_LDO_STATE_OFF = 0, /* the LDO / FET is shut down */
+ EC_LDO_STATE_ON = 1, /* the LDO / FET is ON / providing power */
+};
+
+/*
+ * Switch on/off a LDO.
+ */
+#define EC_CMD_LDO_SET 0x9b
+
+struct ec_params_ldo_set {
+ uint8_t index;
+ uint8_t state;
+} __packed;
+
+/*
+ * Get LDO state.
+ */
+#define EC_CMD_LDO_GET 0x9c
+
+struct ec_params_ldo_get {
+ uint8_t index;
+} __packed;
+
+struct ec_response_ldo_get {
+ uint8_t state;
+} __packed;
+
+/*****************************************************************************/
+/* Power info. */
+
+/*
+ * Get power info.
+ */
+#define EC_CMD_POWER_INFO 0x9d
+
+struct ec_response_power_info {
+ uint32_t usb_dev_type;
+ uint16_t voltage_ac;
+ uint16_t voltage_system;
+ uint16_t current_system;
+ uint16_t usb_current_limit;
+} __packed;
+
+/*****************************************************************************/
+/* I2C passthru command */
+
+#define EC_CMD_I2C_PASSTHRU 0x9e
+
+/* Slave address is 10 (not 7) bit */
+#define EC_I2C_FLAG_10BIT (1 << 16)
+
+/* Read data; if not present, message is a write */
+#define EC_I2C_FLAG_READ (1 << 15)
+
+/* Mask for address */
+#define EC_I2C_ADDR_MASK 0x3ff
+
+#define EC_I2C_STATUS_NAK (1 << 0) /* Transfer was not acknowledged */
+#define EC_I2C_STATUS_TIMEOUT (1 << 1) /* Timeout during transfer */
+
+/* Any error */
+#define EC_I2C_STATUS_ERROR (EC_I2C_STATUS_NAK | EC_I2C_STATUS_TIMEOUT)
+
+struct ec_params_i2c_passthru_msg {
+ uint16_t addr_flags; /* I2C slave address (7 or 10 bits) and flags */
+ uint16_t len; /* Number of bytes to read or write */
+} __packed;
+
+struct ec_params_i2c_passthru {
+ uint8_t port; /* I2C port number */
+ uint8_t num_msgs; /* Number of messages */
+ struct ec_params_i2c_passthru_msg msg[];
+ /* Data to write for all messages is concatenated here */
+} __packed;
+
+struct ec_response_i2c_passthru {
+ uint8_t i2c_status; /* Status flags (EC_I2C_STATUS_...) */
+ uint8_t num_msgs; /* Number of messages processed */
+ uint8_t data[]; /* Data read by messages concatenated here */
+} __packed;
+
+/*****************************************************************************/
+/* Power button hang detect */
+
+#define EC_CMD_HANG_DETECT 0x9f
+
+/* Reasons to start hang detection timer */
+/* Power button pressed */
+#define EC_HANG_START_ON_POWER_PRESS (1 << 0)
+
+/* Lid closed */
+#define EC_HANG_START_ON_LID_CLOSE (1 << 1)
+
+ /* Lid opened */
+#define EC_HANG_START_ON_LID_OPEN (1 << 2)
+
+/* Start of AP S3->S0 transition (booting or resuming from suspend) */
+#define EC_HANG_START_ON_RESUME (1 << 3)
+
+/* Reasons to cancel hang detection */
+
+/* Power button released */
+#define EC_HANG_STOP_ON_POWER_RELEASE (1 << 8)
+
+/* Any host command from AP received */
+#define EC_HANG_STOP_ON_HOST_COMMAND (1 << 9)
+
+/* Stop on end of AP S0->S3 transition (suspending or shutting down) */
+#define EC_HANG_STOP_ON_SUSPEND (1 << 10)
+
+/*
+ * If this flag is set, all the other fields are ignored, and the hang detect
+ * timer is started. This provides the AP a way to start the hang timer
+ * without reconfiguring any of the other hang detect settings. Note that
+ * you must previously have configured the timeouts.
+ */
+#define EC_HANG_START_NOW (1 << 30)
+
+/*
+ * If this flag is set, all the other fields are ignored (including
+ * EC_HANG_START_NOW). This provides the AP a way to stop the hang timer
+ * without reconfiguring any of the other hang detect settings.
+ */
+#define EC_HANG_STOP_NOW (1 << 31)
+
+struct ec_params_hang_detect {
+ /* Flags; see EC_HANG_* */
+ uint32_t flags;
+
+ /* Timeout in msec before generating host event, if enabled */
+ uint16_t host_event_timeout_msec;
+
+ /* Timeout in msec before generating warm reboot, if enabled */
+ uint16_t warm_reboot_timeout_msec;
+} __packed;
+
+/*****************************************************************************/
+/* Commands for battery charging */
+
+/*
+ * This is the single catch-all host command to exchange data regarding the
+ * charge state machine (v2 and up).
+ */
+#define EC_CMD_CHARGE_STATE 0xa0
+
+/* Subcommands for this host command */
+enum charge_state_command {
+ CHARGE_STATE_CMD_GET_STATE,
+ CHARGE_STATE_CMD_GET_PARAM,
+ CHARGE_STATE_CMD_SET_PARAM,
+ CHARGE_STATE_NUM_CMDS
+};
+
+/*
+ * Known param numbers are defined here. Ranges are reserved for board-specific
+ * params, which are handled by the particular implementations.
+ */
+enum charge_state_params {
+ CS_PARAM_CHG_VOLTAGE, /* charger voltage limit */
+ CS_PARAM_CHG_CURRENT, /* charger current limit */
+ CS_PARAM_CHG_INPUT_CURRENT, /* charger input current limit */
+ CS_PARAM_CHG_STATUS, /* charger-specific status */
+ CS_PARAM_CHG_OPTION, /* charger-specific options */
+ /* How many so far? */
+ CS_NUM_BASE_PARAMS,
+
+ /* Range for CONFIG_CHARGER_PROFILE_OVERRIDE params */
+ CS_PARAM_CUSTOM_PROFILE_MIN = 0x10000,
+ CS_PARAM_CUSTOM_PROFILE_MAX = 0x1ffff,
+
+ /* Other custom param ranges go here... */
+};
+
+struct ec_params_charge_state {
+ uint8_t cmd; /* enum charge_state_command */
+ union {
+ struct {
+ /* no args */
+ } get_state;
+
+ struct {
+ uint32_t param; /* enum charge_state_param */
+ } get_param;
+
+ struct {
+ uint32_t param; /* param to set */
+ uint32_t value; /* value to set */
+ } set_param;
+ };
+} __packed;
+
+struct ec_response_charge_state {
+ union {
+ struct {
+ int ac;
+ int chg_voltage;
+ int chg_current;
+ int chg_input_current;
+ int batt_state_of_charge;
+ } get_state;
+
+ struct {
+ uint32_t value;
+ } get_param;
+ struct {
+ /* no return values */
+ } set_param;
+ };
+} __packed;
+
/*
* Set maximum battery charging current.
@@ -1221,15 +2097,59 @@
#define EC_CMD_CHARGE_CURRENT_LIMIT 0xa1
struct ec_params_current_limit {
- uint32_t limit;
+ uint32_t limit; /* in mA */
+} __packed;
+
+/*
+ * Set maximum external power current.
+ */
+#define EC_CMD_EXT_POWER_CURRENT_LIMIT 0xa2
+
+struct ec_params_ext_power_current_limit {
+ uint32_t limit; /* in mA */
+} __packed;
+
+/*****************************************************************************/
+/* Smart battery pass-through */
+
+/* Get / Set 16-bit smart battery registers */
+#define EC_CMD_SB_READ_WORD 0xb0
+#define EC_CMD_SB_WRITE_WORD 0xb1
+
+/* Get / Set string smart battery parameters
+ * formatted as SMBUS "block".
+ */
+#define EC_CMD_SB_READ_BLOCK 0xb2
+#define EC_CMD_SB_WRITE_BLOCK 0xb3
+
+struct ec_params_sb_rd {
+ uint8_t reg;
+} __packed;
+
+struct ec_response_sb_rd_word {
+ uint16_t value;
+} __packed;
+
+struct ec_params_sb_wr_word {
+ uint8_t reg;
+ uint16_t value;
+} __packed;
+
+struct ec_response_sb_rd_block {
+ uint8_t data[32];
+} __packed;
+
+struct ec_params_sb_wr_block {
+ uint8_t reg;
+ uint16_t data[32];
} __packed;
/*****************************************************************************/
/* System commands */
/*
- * TODO: this is a confusing name, since it doesn't necessarily reboot the EC.
- * Rename to "set image" or something similar.
+ * TODO(crosbug.com/p/23747): This is a confusing name, since it doesn't
+ * necessarily reboot the EC. Rename to "image" or something similar?
*/
#define EC_CMD_REBOOT_EC 0xd2
@@ -1308,6 +2228,7 @@
#define EC_CMD_ACPI_QUERY_EVENT 0x84
/* Valid addresses in ACPI memory space, for read/write commands */
+
/* Memory space version; set to EC_ACPI_MEM_VERSION_CURRENT */
#define EC_ACPI_MEM_VERSION 0x00
/*
@@ -1317,8 +2238,60 @@
#define EC_ACPI_MEM_TEST 0x01
/* Test compliment; writes here are ignored. */
#define EC_ACPI_MEM_TEST_COMPLIMENT 0x02
+
/* Keyboard backlight brightness percent (0 - 100) */
#define EC_ACPI_MEM_KEYBOARD_BACKLIGHT 0x03
+/* DPTF Target Fan Duty (0-100, 0xff for auto/none) */
+#define EC_ACPI_MEM_FAN_DUTY 0x04
+
+/*
+ * DPTF temp thresholds. Any of the EC's temp sensors can have up to two
+ * independent thresholds attached to them. The current value of the ID
+ * register determines which sensor is affected by the THRESHOLD and COMMIT
+ * registers. The THRESHOLD register uses the same EC_TEMP_SENSOR_OFFSET scheme
+ * as the memory-mapped sensors. The COMMIT register applies those settings.
+ *
+ * The spec does not mandate any way to read back the threshold settings
+ * themselves, but when a threshold is crossed the AP needs a way to determine
+ * which sensor(s) are responsible. Each reading of the ID register clears and
+ * returns one sensor ID that has crossed one of its threshold (in either
+ * direction) since the last read. A value of 0xFF means "no new thresholds
+ * have tripped". Setting or enabling the thresholds for a sensor will clear
+ * the unread event count for that sensor.
+ */
+#define EC_ACPI_MEM_TEMP_ID 0x05
+#define EC_ACPI_MEM_TEMP_THRESHOLD 0x06
+#define EC_ACPI_MEM_TEMP_COMMIT 0x07
+/*
+ * Here are the bits for the COMMIT register:
+ * bit 0 selects the threshold index for the chosen sensor (0/1)
+ * bit 1 enables/disables the selected threshold (0 = off, 1 = on)
+ * Each write to the commit register affects one threshold.
+ */
+#define EC_ACPI_MEM_TEMP_COMMIT_SELECT_MASK (1 << 0)
+#define EC_ACPI_MEM_TEMP_COMMIT_ENABLE_MASK (1 << 1)
+/*
+ * Example:
+ *
+ * Set the thresholds for sensor 2 to 50 C and 60 C:
+ * write 2 to [0x05] -- select temp sensor 2
+ * write 0x7b to [0x06] -- C_TO_K(50) - EC_TEMP_SENSOR_OFFSET
+ * write 0x2 to [0x07] -- enable threshold 0 with this value
+ * write 0x85 to [0x06] -- C_TO_K(60) - EC_TEMP_SENSOR_OFFSET
+ * write 0x3 to [0x07] -- enable threshold 1 with this value
+ *
+ * Disable the 60 C threshold, leaving the 50 C threshold unchanged:
+ * write 2 to [0x05] -- select temp sensor 2
+ * write 0x1 to [0x07] -- disable threshold 1
+ */
+
+/* DPTF battery charging current limit */
+#define EC_ACPI_MEM_CHARGING_LIMIT 0x08
+
+/* Charging limit is specified in 64 mA steps */
+#define EC_ACPI_MEM_CHARGING_LIMIT_STEP_MA 64
+/* Value to disable DPTF battery charging limit */
+#define EC_ACPI_MEM_CHARGING_LIMIT_DISABLED 0xff
/* Current version of ACPI memory address space */
#define EC_ACPI_MEM_VERSION_CURRENT 1
@@ -1360,10 +2333,21 @@
* Header bytes greater than this indicate a later version. For example,
* EC_CMD_VERSION0 + 1 means we are using version 1.
*
- * The old EC interface must not use commands 0dc or higher.
+ * The old EC interface must not use commands 0xdc or higher.
*/
#define EC_CMD_VERSION0 0xdc
#endif /* !__ACPI__ */
+/*****************************************************************************/
+/*
+ * Deprecated constants. These constants have been renamed for clarity. The
+ * meaning and size has not changed. Programs that use the old names should
+ * switch to the new names soon, as the old names may not be carried forward
+ * forever.
+ */
+#define EC_HOST_PARAM_SIZE EC_PROTO2_MAX_PARAM_SIZE
+#define EC_LPC_ADDR_OLD_PARAM EC_HOST_CMD_REGION1
+#define EC_OLD_PARAM_SIZE EC_HOST_CMD_REGION_SIZE
+
#endif /* __CROS_EC_COMMANDS_H */
diff --git a/include/linux/mfd/ipaq-micro.h b/include/linux/mfd/ipaq-micro.h
new file mode 100644
index 0000000..5c4d29f
--- /dev/null
+++ b/include/linux/mfd/ipaq-micro.h
@@ -0,0 +1,148 @@
+/*
+ * Header file for the compaq Micro MFD
+ */
+
+#ifndef _MFD_IPAQ_MICRO_H_
+#define _MFD_IPAQ_MICRO_H_
+
+#include <linux/spinlock.h>
+#include <linux/completion.h>
+#include <linux/list.h>
+
+#define TX_BUF_SIZE 32
+#define RX_BUF_SIZE 16
+#define CHAR_SOF 0x02
+
+/*
+ * These are the different messages that can be sent to the microcontroller
+ * to control various aspects.
+ */
+#define MSG_VERSION 0x0
+#define MSG_KEYBOARD 0x2
+#define MSG_TOUCHSCREEN 0x3
+#define MSG_EEPROM_READ 0x4
+#define MSG_EEPROM_WRITE 0x5
+#define MSG_THERMAL_SENSOR 0x6
+#define MSG_NOTIFY_LED 0x8
+#define MSG_BATTERY 0x9
+#define MSG_SPI_READ 0xb
+#define MSG_SPI_WRITE 0xc
+#define MSG_BACKLIGHT 0xd /* H3600 only */
+#define MSG_CODEC_CTRL 0xe /* H3100 only */
+#define MSG_DISPLAY_CTRL 0xf /* H3100 only */
+
+/* state of receiver parser */
+enum rx_state {
+ STATE_SOF = 0, /* Next byte should be start of frame */
+ STATE_ID, /* Next byte is ID & message length */
+ STATE_DATA, /* Next byte is a data byte */
+ STATE_CHKSUM /* Next byte should be checksum */
+};
+
+/**
+ * struct ipaq_micro_txdev - TX state
+ * @len: length of message in TX buffer
+ * @index: current index into TX buffer
+ * @buf: TX buffer
+ */
+struct ipaq_micro_txdev {
+ u8 len;
+ u8 index;
+ u8 buf[TX_BUF_SIZE];
+};
+
+/**
+ * struct ipaq_micro_rxdev - RX state
+ * @state: context of RX state machine
+ * @chksum: calculated checksum
+ * @id: message ID from packet
+ * @len: RX buffer length
+ * @index: RX buffer index
+ * @buf: RX buffer
+ */
+struct ipaq_micro_rxdev {
+ enum rx_state state;
+ unsigned char chksum;
+ u8 id;
+ unsigned int len;
+ unsigned int index;
+ u8 buf[RX_BUF_SIZE];
+};
+
+/**
+ * struct ipaq_micro_msg - message to the iPAQ microcontroller
+ * @id: 4-bit ID of the message
+ * @tx_len: length of TX data
+ * @tx_data: TX data to send
+ * @rx_len: length of receieved RX data
+ * @rx_data: RX data to recieve
+ * @ack: a completion that will be completed when RX is complete
+ * @node: list node if message gets queued
+ */
+struct ipaq_micro_msg {
+ u8 id;
+ u8 tx_len;
+ u8 tx_data[TX_BUF_SIZE];
+ u8 rx_len;
+ u8 rx_data[RX_BUF_SIZE];
+ struct completion ack;
+ struct list_head node;
+};
+
+/**
+ * struct ipaq_micro - iPAQ microcontroller state
+ * @dev: corresponding platform device
+ * @base: virtual memory base for underlying serial device
+ * @sdlc: virtual memory base for Synchronous Data Link Controller
+ * @version: version string
+ * @tx: TX state
+ * @rx: RX state
+ * @lock: lock for this state container
+ * @msg: current message
+ * @queue: message queue
+ * @key: callback for asynchronous key events
+ * @key_data: data to pass along with key events
+ * @ts: callback for asynchronous touchscreen events
+ * @ts_data: data to pass along with key events
+ */
+struct ipaq_micro {
+ struct device *dev;
+ void __iomem *base;
+ void __iomem *sdlc;
+ char version[5];
+ struct ipaq_micro_txdev tx; /* transmit ISR state */
+ struct ipaq_micro_rxdev rx; /* receive ISR state */
+ spinlock_t lock;
+ struct ipaq_micro_msg *msg;
+ struct list_head queue;
+ void (*key) (void *data, int len, unsigned char *rxdata);
+ void *key_data;
+ void (*ts) (void *data, int len, unsigned char *rxdata);
+ void *ts_data;
+};
+
+extern int
+ipaq_micro_tx_msg(struct ipaq_micro *micro, struct ipaq_micro_msg *msg);
+
+static inline int
+ipaq_micro_tx_msg_sync(struct ipaq_micro *micro,
+ struct ipaq_micro_msg *msg)
+{
+ int ret;
+
+ init_completion(&msg->ack);
+ ret = ipaq_micro_tx_msg(micro, msg);
+ wait_for_completion(&msg->ack);
+
+ return ret;
+}
+
+static inline int
+ipaq_micro_tx_msg_async(struct ipaq_micro *micro,
+ struct ipaq_micro_msg *msg)
+{
+ init_completion(&msg->ack);
+ return ipaq_micro_tx_msg(micro, msg);
+}
+
+#endif /* _MFD_IPAQ_MICRO_H_ */
diff --git a/include/linux/mfd/kempld.h b/include/linux/mfd/kempld.h
index b911ef3..26e0b46 100644
--- a/include/linux/mfd/kempld.h
+++ b/include/linux/mfd/kempld.h
@@ -51,6 +51,8 @@
#define KEMPLD_TYPE_DEBUG 0x1
#define KEMPLD_TYPE_CUSTOM 0x2
+#define KEMPLD_VERSION_LEN 10
+
/**
* struct kempld_info - PLD device information structure
* @major: PLD major revision
@@ -60,6 +62,7 @@
* @type: PLD type
* @spec_major: PLD FW specification major revision
* @spec_minor: PLD FW specification minor revision
+ * @version: PLD version string
*/
struct kempld_info {
unsigned int major;
@@ -69,6 +72,7 @@
unsigned int type;
unsigned int spec_major;
unsigned int spec_minor;
+ char version[KEMPLD_VERSION_LEN];
};
/**
diff --git a/include/linux/mfd/mc13xxx.h b/include/linux/mfd/mc13xxx.h
index a326c85..d63b1d3 100644
--- a/include/linux/mfd/mc13xxx.h
+++ b/include/linux/mfd/mc13xxx.h
@@ -117,10 +117,6 @@
#define MAX_LED_CONTROL_REGS 6
-struct mc13xxx_leds_platform_data {
- struct mc13xxx_led_platform_data *led;
- int num_leds;
-
/* MC13783 LED Control 0 */
#define MC13783_LED_C0_ENABLE (1 << 0)
#define MC13783_LED_C0_TRIODE_MD (1 << 7)
@@ -169,10 +165,13 @@
/* MC34708 LED Control 0 */
#define MC34708_LED_C0_CURRENT_R(x) (((x) & 0x3) << 9)
#define MC34708_LED_C0_CURRENT_G(x) (((x) & 0x3) << 21)
+
+struct mc13xxx_leds_platform_data {
+ struct mc13xxx_led_platform_data *led;
+ int num_leds;
u32 led_control[MAX_LED_CONTROL_REGS];
};
-struct mc13xxx_buttons_platform_data {
#define MC13783_BUTTON_DBNC_0MS 0
#define MC13783_BUTTON_DBNC_30MS 1
#define MC13783_BUTTON_DBNC_150MS 2
@@ -180,6 +179,8 @@
#define MC13783_BUTTON_ENABLE (1 << 2)
#define MC13783_BUTTON_POL_INVERT (1 << 3)
#define MC13783_BUTTON_RESET_EN (1 << 4)
+
+struct mc13xxx_buttons_platform_data {
int b1on_flags;
unsigned short b1on_key;
int b2on_flags;
@@ -188,14 +189,14 @@
unsigned short b3on_key;
};
+#define MC13783_TS_ATO_FIRST false
+#define MC13783_TS_ATO_EACH true
+
struct mc13xxx_ts_platform_data {
/* Delay between Touchscreen polarization and ADC Conversion.
* Given in clock ticks of a 32 kHz clock which gives a granularity of
* about 30.5ms */
u8 ato;
-
-#define MC13783_TS_ATO_FIRST false
-#define MC13783_TS_ATO_EACH true
/* Use the ATO delay only for the first conversion or for each one */
bool atox;
};
@@ -210,11 +211,12 @@
enum mc13783_ssi_port dac_ssi_port;
};
-struct mc13xxx_platform_data {
-#define MC13XXX_USE_TOUCHSCREEN (1 << 0)
+#define MC13XXX_USE_TOUCHSCREEN (1 << 0)
#define MC13XXX_USE_CODEC (1 << 1)
#define MC13XXX_USE_ADC (1 << 2)
#define MC13XXX_USE_RTC (1 << 3)
+
+struct mc13xxx_platform_data {
unsigned int flags;
struct mc13xxx_regulator_platform_data regulators;
diff --git a/include/linux/mfd/palmas.h b/include/linux/mfd/palmas.h
index b8f87b7..3420e09e 100644
--- a/include/linux/mfd/palmas.h
+++ b/include/linux/mfd/palmas.h
@@ -482,10 +482,10 @@
/* helper macro to get correct slave number */
#define PALMAS_BASE_TO_SLAVE(x) ((x >> 8) - 1)
-#define PALMAS_BASE_TO_REG(x, y) ((x & 0xff) + y)
+#define PALMAS_BASE_TO_REG(x, y) ((x & 0xFF) + y)
/* Base addresses of IP blocks in Palmas */
-#define PALMAS_SMPS_DVS_BASE 0x20
+#define PALMAS_SMPS_DVS_BASE 0x020
#define PALMAS_RTC_BASE 0x100
#define PALMAS_VALIDITY_BASE 0x118
#define PALMAS_SMPS_BASE 0x120
@@ -504,19 +504,19 @@
#define PALMAS_TRIM_GPADC_BASE 0x3CD
/* Registers for function RTC */
-#define PALMAS_SECONDS_REG 0x0
-#define PALMAS_MINUTES_REG 0x1
-#define PALMAS_HOURS_REG 0x2
-#define PALMAS_DAYS_REG 0x3
-#define PALMAS_MONTHS_REG 0x4
-#define PALMAS_YEARS_REG 0x5
-#define PALMAS_WEEKS_REG 0x6
-#define PALMAS_ALARM_SECONDS_REG 0x8
-#define PALMAS_ALARM_MINUTES_REG 0x9
-#define PALMAS_ALARM_HOURS_REG 0xA
-#define PALMAS_ALARM_DAYS_REG 0xB
-#define PALMAS_ALARM_MONTHS_REG 0xC
-#define PALMAS_ALARM_YEARS_REG 0xD
+#define PALMAS_SECONDS_REG 0x00
+#define PALMAS_MINUTES_REG 0x01
+#define PALMAS_HOURS_REG 0x02
+#define PALMAS_DAYS_REG 0x03
+#define PALMAS_MONTHS_REG 0x04
+#define PALMAS_YEARS_REG 0x05
+#define PALMAS_WEEKS_REG 0x06
+#define PALMAS_ALARM_SECONDS_REG 0x08
+#define PALMAS_ALARM_MINUTES_REG 0x09
+#define PALMAS_ALARM_HOURS_REG 0x0A
+#define PALMAS_ALARM_DAYS_REG 0x0B
+#define PALMAS_ALARM_MONTHS_REG 0x0C
+#define PALMAS_ALARM_YEARS_REG 0x0D
#define PALMAS_RTC_CTRL_REG 0x10
#define PALMAS_RTC_STATUS_REG 0x11
#define PALMAS_RTC_INTERRUPTS_REG 0x12
@@ -527,201 +527,201 @@
/* Bit definitions for SECONDS_REG */
#define PALMAS_SECONDS_REG_SEC1_MASK 0x70
-#define PALMAS_SECONDS_REG_SEC1_SHIFT 4
-#define PALMAS_SECONDS_REG_SEC0_MASK 0x0f
-#define PALMAS_SECONDS_REG_SEC0_SHIFT 0
+#define PALMAS_SECONDS_REG_SEC1_SHIFT 0x04
+#define PALMAS_SECONDS_REG_SEC0_MASK 0x0F
+#define PALMAS_SECONDS_REG_SEC0_SHIFT 0x00
/* Bit definitions for MINUTES_REG */
#define PALMAS_MINUTES_REG_MIN1_MASK 0x70
-#define PALMAS_MINUTES_REG_MIN1_SHIFT 4
-#define PALMAS_MINUTES_REG_MIN0_MASK 0x0f
-#define PALMAS_MINUTES_REG_MIN0_SHIFT 0
+#define PALMAS_MINUTES_REG_MIN1_SHIFT 0x04
+#define PALMAS_MINUTES_REG_MIN0_MASK 0x0F
+#define PALMAS_MINUTES_REG_MIN0_SHIFT 0x00
/* Bit definitions for HOURS_REG */
#define PALMAS_HOURS_REG_PM_NAM 0x80
-#define PALMAS_HOURS_REG_PM_NAM_SHIFT 7
+#define PALMAS_HOURS_REG_PM_NAM_SHIFT 0x07
#define PALMAS_HOURS_REG_HOUR1_MASK 0x30
-#define PALMAS_HOURS_REG_HOUR1_SHIFT 4
-#define PALMAS_HOURS_REG_HOUR0_MASK 0x0f
-#define PALMAS_HOURS_REG_HOUR0_SHIFT 0
+#define PALMAS_HOURS_REG_HOUR1_SHIFT 0x04
+#define PALMAS_HOURS_REG_HOUR0_MASK 0x0F
+#define PALMAS_HOURS_REG_HOUR0_SHIFT 0x00
/* Bit definitions for DAYS_REG */
#define PALMAS_DAYS_REG_DAY1_MASK 0x30
-#define PALMAS_DAYS_REG_DAY1_SHIFT 4
-#define PALMAS_DAYS_REG_DAY0_MASK 0x0f
-#define PALMAS_DAYS_REG_DAY0_SHIFT 0
+#define PALMAS_DAYS_REG_DAY1_SHIFT 0x04
+#define PALMAS_DAYS_REG_DAY0_MASK 0x0F
+#define PALMAS_DAYS_REG_DAY0_SHIFT 0x00
/* Bit definitions for MONTHS_REG */
#define PALMAS_MONTHS_REG_MONTH1 0x10
-#define PALMAS_MONTHS_REG_MONTH1_SHIFT 4
-#define PALMAS_MONTHS_REG_MONTH0_MASK 0x0f
-#define PALMAS_MONTHS_REG_MONTH0_SHIFT 0
+#define PALMAS_MONTHS_REG_MONTH1_SHIFT 0x04
+#define PALMAS_MONTHS_REG_MONTH0_MASK 0x0F
+#define PALMAS_MONTHS_REG_MONTH0_SHIFT 0x00
/* Bit definitions for YEARS_REG */
#define PALMAS_YEARS_REG_YEAR1_MASK 0xf0
-#define PALMAS_YEARS_REG_YEAR1_SHIFT 4
-#define PALMAS_YEARS_REG_YEAR0_MASK 0x0f
-#define PALMAS_YEARS_REG_YEAR0_SHIFT 0
+#define PALMAS_YEARS_REG_YEAR1_SHIFT 0x04
+#define PALMAS_YEARS_REG_YEAR0_MASK 0x0F
+#define PALMAS_YEARS_REG_YEAR0_SHIFT 0x00
/* Bit definitions for WEEKS_REG */
#define PALMAS_WEEKS_REG_WEEK_MASK 0x07
-#define PALMAS_WEEKS_REG_WEEK_SHIFT 0
+#define PALMAS_WEEKS_REG_WEEK_SHIFT 0x00
/* Bit definitions for ALARM_SECONDS_REG */
#define PALMAS_ALARM_SECONDS_REG_ALARM_SEC1_MASK 0x70
-#define PALMAS_ALARM_SECONDS_REG_ALARM_SEC1_SHIFT 4
-#define PALMAS_ALARM_SECONDS_REG_ALARM_SEC0_MASK 0x0f
-#define PALMAS_ALARM_SECONDS_REG_ALARM_SEC0_SHIFT 0
+#define PALMAS_ALARM_SECONDS_REG_ALARM_SEC1_SHIFT 0x04
+#define PALMAS_ALARM_SECONDS_REG_ALARM_SEC0_MASK 0x0F
+#define PALMAS_ALARM_SECONDS_REG_ALARM_SEC0_SHIFT 0x00
/* Bit definitions for ALARM_MINUTES_REG */
#define PALMAS_ALARM_MINUTES_REG_ALARM_MIN1_MASK 0x70
-#define PALMAS_ALARM_MINUTES_REG_ALARM_MIN1_SHIFT 4
-#define PALMAS_ALARM_MINUTES_REG_ALARM_MIN0_MASK 0x0f
-#define PALMAS_ALARM_MINUTES_REG_ALARM_MIN0_SHIFT 0
+#define PALMAS_ALARM_MINUTES_REG_ALARM_MIN1_SHIFT 0x04
+#define PALMAS_ALARM_MINUTES_REG_ALARM_MIN0_MASK 0x0F
+#define PALMAS_ALARM_MINUTES_REG_ALARM_MIN0_SHIFT 0x00
/* Bit definitions for ALARM_HOURS_REG */
#define PALMAS_ALARM_HOURS_REG_ALARM_PM_NAM 0x80
-#define PALMAS_ALARM_HOURS_REG_ALARM_PM_NAM_SHIFT 7
+#define PALMAS_ALARM_HOURS_REG_ALARM_PM_NAM_SHIFT 0x07
#define PALMAS_ALARM_HOURS_REG_ALARM_HOUR1_MASK 0x30
-#define PALMAS_ALARM_HOURS_REG_ALARM_HOUR1_SHIFT 4
-#define PALMAS_ALARM_HOURS_REG_ALARM_HOUR0_MASK 0x0f
-#define PALMAS_ALARM_HOURS_REG_ALARM_HOUR0_SHIFT 0
+#define PALMAS_ALARM_HOURS_REG_ALARM_HOUR1_SHIFT 0x04
+#define PALMAS_ALARM_HOURS_REG_ALARM_HOUR0_MASK 0x0F
+#define PALMAS_ALARM_HOURS_REG_ALARM_HOUR0_SHIFT 0x00
/* Bit definitions for ALARM_DAYS_REG */
#define PALMAS_ALARM_DAYS_REG_ALARM_DAY1_MASK 0x30
-#define PALMAS_ALARM_DAYS_REG_ALARM_DAY1_SHIFT 4
-#define PALMAS_ALARM_DAYS_REG_ALARM_DAY0_MASK 0x0f
-#define PALMAS_ALARM_DAYS_REG_ALARM_DAY0_SHIFT 0
+#define PALMAS_ALARM_DAYS_REG_ALARM_DAY1_SHIFT 0x04
+#define PALMAS_ALARM_DAYS_REG_ALARM_DAY0_MASK 0x0F
+#define PALMAS_ALARM_DAYS_REG_ALARM_DAY0_SHIFT 0x00
/* Bit definitions for ALARM_MONTHS_REG */
#define PALMAS_ALARM_MONTHS_REG_ALARM_MONTH1 0x10
-#define PALMAS_ALARM_MONTHS_REG_ALARM_MONTH1_SHIFT 4
-#define PALMAS_ALARM_MONTHS_REG_ALARM_MONTH0_MASK 0x0f
-#define PALMAS_ALARM_MONTHS_REG_ALARM_MONTH0_SHIFT 0
+#define PALMAS_ALARM_MONTHS_REG_ALARM_MONTH1_SHIFT 0x04
+#define PALMAS_ALARM_MONTHS_REG_ALARM_MONTH0_MASK 0x0F
+#define PALMAS_ALARM_MONTHS_REG_ALARM_MONTH0_SHIFT 0x00
/* Bit definitions for ALARM_YEARS_REG */
#define PALMAS_ALARM_YEARS_REG_ALARM_YEAR1_MASK 0xf0
-#define PALMAS_ALARM_YEARS_REG_ALARM_YEAR1_SHIFT 4
-#define PALMAS_ALARM_YEARS_REG_ALARM_YEAR0_MASK 0x0f
-#define PALMAS_ALARM_YEARS_REG_ALARM_YEAR0_SHIFT 0
+#define PALMAS_ALARM_YEARS_REG_ALARM_YEAR1_SHIFT 0x04
+#define PALMAS_ALARM_YEARS_REG_ALARM_YEAR0_MASK 0x0F
+#define PALMAS_ALARM_YEARS_REG_ALARM_YEAR0_SHIFT 0x00
/* Bit definitions for RTC_CTRL_REG */
#define PALMAS_RTC_CTRL_REG_RTC_V_OPT 0x80
-#define PALMAS_RTC_CTRL_REG_RTC_V_OPT_SHIFT 7
+#define PALMAS_RTC_CTRL_REG_RTC_V_OPT_SHIFT 0x07
#define PALMAS_RTC_CTRL_REG_GET_TIME 0x40
-#define PALMAS_RTC_CTRL_REG_GET_TIME_SHIFT 6
+#define PALMAS_RTC_CTRL_REG_GET_TIME_SHIFT 0x06
#define PALMAS_RTC_CTRL_REG_SET_32_COUNTER 0x20
-#define PALMAS_RTC_CTRL_REG_SET_32_COUNTER_SHIFT 5
+#define PALMAS_RTC_CTRL_REG_SET_32_COUNTER_SHIFT 0x05
#define PALMAS_RTC_CTRL_REG_TEST_MODE 0x10
-#define PALMAS_RTC_CTRL_REG_TEST_MODE_SHIFT 4
+#define PALMAS_RTC_CTRL_REG_TEST_MODE_SHIFT 0x04
#define PALMAS_RTC_CTRL_REG_MODE_12_24 0x08
-#define PALMAS_RTC_CTRL_REG_MODE_12_24_SHIFT 3
+#define PALMAS_RTC_CTRL_REG_MODE_12_24_SHIFT 0x03
#define PALMAS_RTC_CTRL_REG_AUTO_COMP 0x04
-#define PALMAS_RTC_CTRL_REG_AUTO_COMP_SHIFT 2
+#define PALMAS_RTC_CTRL_REG_AUTO_COMP_SHIFT 0x02
#define PALMAS_RTC_CTRL_REG_ROUND_30S 0x02
-#define PALMAS_RTC_CTRL_REG_ROUND_30S_SHIFT 1
+#define PALMAS_RTC_CTRL_REG_ROUND_30S_SHIFT 0x01
#define PALMAS_RTC_CTRL_REG_STOP_RTC 0x01
-#define PALMAS_RTC_CTRL_REG_STOP_RTC_SHIFT 0
+#define PALMAS_RTC_CTRL_REG_STOP_RTC_SHIFT 0x00
/* Bit definitions for RTC_STATUS_REG */
#define PALMAS_RTC_STATUS_REG_POWER_UP 0x80
-#define PALMAS_RTC_STATUS_REG_POWER_UP_SHIFT 7
+#define PALMAS_RTC_STATUS_REG_POWER_UP_SHIFT 0x07
#define PALMAS_RTC_STATUS_REG_ALARM 0x40
-#define PALMAS_RTC_STATUS_REG_ALARM_SHIFT 6
+#define PALMAS_RTC_STATUS_REG_ALARM_SHIFT 0x06
#define PALMAS_RTC_STATUS_REG_EVENT_1D 0x20
-#define PALMAS_RTC_STATUS_REG_EVENT_1D_SHIFT 5
+#define PALMAS_RTC_STATUS_REG_EVENT_1D_SHIFT 0x05
#define PALMAS_RTC_STATUS_REG_EVENT_1H 0x10
-#define PALMAS_RTC_STATUS_REG_EVENT_1H_SHIFT 4
+#define PALMAS_RTC_STATUS_REG_EVENT_1H_SHIFT 0x04
#define PALMAS_RTC_STATUS_REG_EVENT_1M 0x08
-#define PALMAS_RTC_STATUS_REG_EVENT_1M_SHIFT 3
+#define PALMAS_RTC_STATUS_REG_EVENT_1M_SHIFT 0x03
#define PALMAS_RTC_STATUS_REG_EVENT_1S 0x04
-#define PALMAS_RTC_STATUS_REG_EVENT_1S_SHIFT 2
+#define PALMAS_RTC_STATUS_REG_EVENT_1S_SHIFT 0x02
#define PALMAS_RTC_STATUS_REG_RUN 0x02
-#define PALMAS_RTC_STATUS_REG_RUN_SHIFT 1
+#define PALMAS_RTC_STATUS_REG_RUN_SHIFT 0x01
/* Bit definitions for RTC_INTERRUPTS_REG */
#define PALMAS_RTC_INTERRUPTS_REG_IT_SLEEP_MASK_EN 0x10
-#define PALMAS_RTC_INTERRUPTS_REG_IT_SLEEP_MASK_EN_SHIFT 4
+#define PALMAS_RTC_INTERRUPTS_REG_IT_SLEEP_MASK_EN_SHIFT 0x04
#define PALMAS_RTC_INTERRUPTS_REG_IT_ALARM 0x08
-#define PALMAS_RTC_INTERRUPTS_REG_IT_ALARM_SHIFT 3
+#define PALMAS_RTC_INTERRUPTS_REG_IT_ALARM_SHIFT 0x03
#define PALMAS_RTC_INTERRUPTS_REG_IT_TIMER 0x04
-#define PALMAS_RTC_INTERRUPTS_REG_IT_TIMER_SHIFT 2
+#define PALMAS_RTC_INTERRUPTS_REG_IT_TIMER_SHIFT 0x02
#define PALMAS_RTC_INTERRUPTS_REG_EVERY_MASK 0x03
-#define PALMAS_RTC_INTERRUPTS_REG_EVERY_SHIFT 0
+#define PALMAS_RTC_INTERRUPTS_REG_EVERY_SHIFT 0x00
/* Bit definitions for RTC_COMP_LSB_REG */
-#define PALMAS_RTC_COMP_LSB_REG_RTC_COMP_LSB_MASK 0xff
-#define PALMAS_RTC_COMP_LSB_REG_RTC_COMP_LSB_SHIFT 0
+#define PALMAS_RTC_COMP_LSB_REG_RTC_COMP_LSB_MASK 0xFF
+#define PALMAS_RTC_COMP_LSB_REG_RTC_COMP_LSB_SHIFT 0x00
/* Bit definitions for RTC_COMP_MSB_REG */
-#define PALMAS_RTC_COMP_MSB_REG_RTC_COMP_MSB_MASK 0xff
-#define PALMAS_RTC_COMP_MSB_REG_RTC_COMP_MSB_SHIFT 0
+#define PALMAS_RTC_COMP_MSB_REG_RTC_COMP_MSB_MASK 0xFF
+#define PALMAS_RTC_COMP_MSB_REG_RTC_COMP_MSB_SHIFT 0x00
/* Bit definitions for RTC_RES_PROG_REG */
-#define PALMAS_RTC_RES_PROG_REG_SW_RES_PROG_MASK 0x3f
-#define PALMAS_RTC_RES_PROG_REG_SW_RES_PROG_SHIFT 0
+#define PALMAS_RTC_RES_PROG_REG_SW_RES_PROG_MASK 0x3F
+#define PALMAS_RTC_RES_PROG_REG_SW_RES_PROG_SHIFT 0x00
/* Bit definitions for RTC_RESET_STATUS_REG */
#define PALMAS_RTC_RESET_STATUS_REG_RESET_STATUS 0x01
-#define PALMAS_RTC_RESET_STATUS_REG_RESET_STATUS_SHIFT 0
+#define PALMAS_RTC_RESET_STATUS_REG_RESET_STATUS_SHIFT 0x00
/* Registers for function BACKUP */
-#define PALMAS_BACKUP0 0x0
-#define PALMAS_BACKUP1 0x1
-#define PALMAS_BACKUP2 0x2
-#define PALMAS_BACKUP3 0x3
-#define PALMAS_BACKUP4 0x4
-#define PALMAS_BACKUP5 0x5
-#define PALMAS_BACKUP6 0x6
-#define PALMAS_BACKUP7 0x7
+#define PALMAS_BACKUP0 0x00
+#define PALMAS_BACKUP1 0x01
+#define PALMAS_BACKUP2 0x02
+#define PALMAS_BACKUP3 0x03
+#define PALMAS_BACKUP4 0x04
+#define PALMAS_BACKUP5 0x05
+#define PALMAS_BACKUP6 0x06
+#define PALMAS_BACKUP7 0x07
/* Bit definitions for BACKUP0 */
-#define PALMAS_BACKUP0_BACKUP_MASK 0xff
-#define PALMAS_BACKUP0_BACKUP_SHIFT 0
+#define PALMAS_BACKUP0_BACKUP_MASK 0xFF
+#define PALMAS_BACKUP0_BACKUP_SHIFT 0x00
/* Bit definitions for BACKUP1 */
-#define PALMAS_BACKUP1_BACKUP_MASK 0xff
-#define PALMAS_BACKUP1_BACKUP_SHIFT 0
+#define PALMAS_BACKUP1_BACKUP_MASK 0xFF
+#define PALMAS_BACKUP1_BACKUP_SHIFT 0x00
/* Bit definitions for BACKUP2 */
-#define PALMAS_BACKUP2_BACKUP_MASK 0xff
-#define PALMAS_BACKUP2_BACKUP_SHIFT 0
+#define PALMAS_BACKUP2_BACKUP_MASK 0xFF
+#define PALMAS_BACKUP2_BACKUP_SHIFT 0x00
/* Bit definitions for BACKUP3 */
-#define PALMAS_BACKUP3_BACKUP_MASK 0xff
-#define PALMAS_BACKUP3_BACKUP_SHIFT 0
+#define PALMAS_BACKUP3_BACKUP_MASK 0xFF
+#define PALMAS_BACKUP3_BACKUP_SHIFT 0x00
/* Bit definitions for BACKUP4 */
-#define PALMAS_BACKUP4_BACKUP_MASK 0xff
-#define PALMAS_BACKUP4_BACKUP_SHIFT 0
+#define PALMAS_BACKUP4_BACKUP_MASK 0xFF
+#define PALMAS_BACKUP4_BACKUP_SHIFT 0x00
/* Bit definitions for BACKUP5 */
-#define PALMAS_BACKUP5_BACKUP_MASK 0xff
-#define PALMAS_BACKUP5_BACKUP_SHIFT 0
+#define PALMAS_BACKUP5_BACKUP_MASK 0xFF
+#define PALMAS_BACKUP5_BACKUP_SHIFT 0x00
/* Bit definitions for BACKUP6 */
-#define PALMAS_BACKUP6_BACKUP_MASK 0xff
-#define PALMAS_BACKUP6_BACKUP_SHIFT 0
+#define PALMAS_BACKUP6_BACKUP_MASK 0xFF
+#define PALMAS_BACKUP6_BACKUP_SHIFT 0x00
/* Bit definitions for BACKUP7 */
-#define PALMAS_BACKUP7_BACKUP_MASK 0xff
-#define PALMAS_BACKUP7_BACKUP_SHIFT 0
+#define PALMAS_BACKUP7_BACKUP_MASK 0xFF
+#define PALMAS_BACKUP7_BACKUP_SHIFT 0x00
/* Registers for function SMPS */
-#define PALMAS_SMPS12_CTRL 0x0
-#define PALMAS_SMPS12_TSTEP 0x1
-#define PALMAS_SMPS12_FORCE 0x2
-#define PALMAS_SMPS12_VOLTAGE 0x3
-#define PALMAS_SMPS3_CTRL 0x4
-#define PALMAS_SMPS3_VOLTAGE 0x7
-#define PALMAS_SMPS45_CTRL 0x8
-#define PALMAS_SMPS45_TSTEP 0x9
-#define PALMAS_SMPS45_FORCE 0xA
-#define PALMAS_SMPS45_VOLTAGE 0xB
-#define PALMAS_SMPS6_CTRL 0xC
-#define PALMAS_SMPS6_TSTEP 0xD
-#define PALMAS_SMPS6_FORCE 0xE
-#define PALMAS_SMPS6_VOLTAGE 0xF
+#define PALMAS_SMPS12_CTRL 0x00
+#define PALMAS_SMPS12_TSTEP 0x01
+#define PALMAS_SMPS12_FORCE 0x02
+#define PALMAS_SMPS12_VOLTAGE 0x03
+#define PALMAS_SMPS3_CTRL 0x04
+#define PALMAS_SMPS3_VOLTAGE 0x07
+#define PALMAS_SMPS45_CTRL 0x08
+#define PALMAS_SMPS45_TSTEP 0x09
+#define PALMAS_SMPS45_FORCE 0x0A
+#define PALMAS_SMPS45_VOLTAGE 0x0B
+#define PALMAS_SMPS6_CTRL 0x0C
+#define PALMAS_SMPS6_TSTEP 0x0D
+#define PALMAS_SMPS6_FORCE 0x0E
+#define PALMAS_SMPS6_VOLTAGE 0x0F
#define PALMAS_SMPS7_CTRL 0x10
#define PALMAS_SMPS7_VOLTAGE 0x13
#define PALMAS_SMPS8_CTRL 0x14
@@ -744,303 +744,303 @@
/* Bit definitions for SMPS12_CTRL */
#define PALMAS_SMPS12_CTRL_WR_S 0x80
-#define PALMAS_SMPS12_CTRL_WR_S_SHIFT 7
+#define PALMAS_SMPS12_CTRL_WR_S_SHIFT 0x07
#define PALMAS_SMPS12_CTRL_ROOF_FLOOR_EN 0x40
-#define PALMAS_SMPS12_CTRL_ROOF_FLOOR_EN_SHIFT 6
+#define PALMAS_SMPS12_CTRL_ROOF_FLOOR_EN_SHIFT 0x06
#define PALMAS_SMPS12_CTRL_STATUS_MASK 0x30
-#define PALMAS_SMPS12_CTRL_STATUS_SHIFT 4
+#define PALMAS_SMPS12_CTRL_STATUS_SHIFT 0x04
#define PALMAS_SMPS12_CTRL_MODE_SLEEP_MASK 0x0c
-#define PALMAS_SMPS12_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_SMPS12_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_SMPS12_CTRL_MODE_ACTIVE_MASK 0x03
-#define PALMAS_SMPS12_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_SMPS12_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for SMPS12_TSTEP */
#define PALMAS_SMPS12_TSTEP_TSTEP_MASK 0x03
-#define PALMAS_SMPS12_TSTEP_TSTEP_SHIFT 0
+#define PALMAS_SMPS12_TSTEP_TSTEP_SHIFT 0x00
/* Bit definitions for SMPS12_FORCE */
#define PALMAS_SMPS12_FORCE_CMD 0x80
-#define PALMAS_SMPS12_FORCE_CMD_SHIFT 7
-#define PALMAS_SMPS12_FORCE_VSEL_MASK 0x7f
-#define PALMAS_SMPS12_FORCE_VSEL_SHIFT 0
+#define PALMAS_SMPS12_FORCE_CMD_SHIFT 0x07
+#define PALMAS_SMPS12_FORCE_VSEL_MASK 0x7F
+#define PALMAS_SMPS12_FORCE_VSEL_SHIFT 0x00
/* Bit definitions for SMPS12_VOLTAGE */
#define PALMAS_SMPS12_VOLTAGE_RANGE 0x80
-#define PALMAS_SMPS12_VOLTAGE_RANGE_SHIFT 7
-#define PALMAS_SMPS12_VOLTAGE_VSEL_MASK 0x7f
-#define PALMAS_SMPS12_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_SMPS12_VOLTAGE_RANGE_SHIFT 0x07
+#define PALMAS_SMPS12_VOLTAGE_VSEL_MASK 0x7F
+#define PALMAS_SMPS12_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for SMPS3_CTRL */
#define PALMAS_SMPS3_CTRL_WR_S 0x80
-#define PALMAS_SMPS3_CTRL_WR_S_SHIFT 7
+#define PALMAS_SMPS3_CTRL_WR_S_SHIFT 0x07
#define PALMAS_SMPS3_CTRL_STATUS_MASK 0x30
-#define PALMAS_SMPS3_CTRL_STATUS_SHIFT 4
+#define PALMAS_SMPS3_CTRL_STATUS_SHIFT 0x04
#define PALMAS_SMPS3_CTRL_MODE_SLEEP_MASK 0x0c
-#define PALMAS_SMPS3_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_SMPS3_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_SMPS3_CTRL_MODE_ACTIVE_MASK 0x03
-#define PALMAS_SMPS3_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_SMPS3_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for SMPS3_VOLTAGE */
#define PALMAS_SMPS3_VOLTAGE_RANGE 0x80
-#define PALMAS_SMPS3_VOLTAGE_RANGE_SHIFT 7
-#define PALMAS_SMPS3_VOLTAGE_VSEL_MASK 0x7f
-#define PALMAS_SMPS3_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_SMPS3_VOLTAGE_RANGE_SHIFT 0x07
+#define PALMAS_SMPS3_VOLTAGE_VSEL_MASK 0x7F
+#define PALMAS_SMPS3_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for SMPS45_CTRL */
#define PALMAS_SMPS45_CTRL_WR_S 0x80
-#define PALMAS_SMPS45_CTRL_WR_S_SHIFT 7
+#define PALMAS_SMPS45_CTRL_WR_S_SHIFT 0x07
#define PALMAS_SMPS45_CTRL_ROOF_FLOOR_EN 0x40
-#define PALMAS_SMPS45_CTRL_ROOF_FLOOR_EN_SHIFT 6
+#define PALMAS_SMPS45_CTRL_ROOF_FLOOR_EN_SHIFT 0x06
#define PALMAS_SMPS45_CTRL_STATUS_MASK 0x30
-#define PALMAS_SMPS45_CTRL_STATUS_SHIFT 4
+#define PALMAS_SMPS45_CTRL_STATUS_SHIFT 0x04
#define PALMAS_SMPS45_CTRL_MODE_SLEEP_MASK 0x0c
-#define PALMAS_SMPS45_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_SMPS45_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_SMPS45_CTRL_MODE_ACTIVE_MASK 0x03
-#define PALMAS_SMPS45_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_SMPS45_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for SMPS45_TSTEP */
#define PALMAS_SMPS45_TSTEP_TSTEP_MASK 0x03
-#define PALMAS_SMPS45_TSTEP_TSTEP_SHIFT 0
+#define PALMAS_SMPS45_TSTEP_TSTEP_SHIFT 0x00
/* Bit definitions for SMPS45_FORCE */
#define PALMAS_SMPS45_FORCE_CMD 0x80
-#define PALMAS_SMPS45_FORCE_CMD_SHIFT 7
-#define PALMAS_SMPS45_FORCE_VSEL_MASK 0x7f
-#define PALMAS_SMPS45_FORCE_VSEL_SHIFT 0
+#define PALMAS_SMPS45_FORCE_CMD_SHIFT 0x07
+#define PALMAS_SMPS45_FORCE_VSEL_MASK 0x7F
+#define PALMAS_SMPS45_FORCE_VSEL_SHIFT 0x00
/* Bit definitions for SMPS45_VOLTAGE */
#define PALMAS_SMPS45_VOLTAGE_RANGE 0x80
-#define PALMAS_SMPS45_VOLTAGE_RANGE_SHIFT 7
-#define PALMAS_SMPS45_VOLTAGE_VSEL_MASK 0x7f
-#define PALMAS_SMPS45_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_SMPS45_VOLTAGE_RANGE_SHIFT 0x07
+#define PALMAS_SMPS45_VOLTAGE_VSEL_MASK 0x7F
+#define PALMAS_SMPS45_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for SMPS6_CTRL */
#define PALMAS_SMPS6_CTRL_WR_S 0x80
-#define PALMAS_SMPS6_CTRL_WR_S_SHIFT 7
+#define PALMAS_SMPS6_CTRL_WR_S_SHIFT 0x07
#define PALMAS_SMPS6_CTRL_ROOF_FLOOR_EN 0x40
-#define PALMAS_SMPS6_CTRL_ROOF_FLOOR_EN_SHIFT 6
+#define PALMAS_SMPS6_CTRL_ROOF_FLOOR_EN_SHIFT 0x06
#define PALMAS_SMPS6_CTRL_STATUS_MASK 0x30
-#define PALMAS_SMPS6_CTRL_STATUS_SHIFT 4
+#define PALMAS_SMPS6_CTRL_STATUS_SHIFT 0x04
#define PALMAS_SMPS6_CTRL_MODE_SLEEP_MASK 0x0c
-#define PALMAS_SMPS6_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_SMPS6_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_SMPS6_CTRL_MODE_ACTIVE_MASK 0x03
-#define PALMAS_SMPS6_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_SMPS6_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for SMPS6_TSTEP */
#define PALMAS_SMPS6_TSTEP_TSTEP_MASK 0x03
-#define PALMAS_SMPS6_TSTEP_TSTEP_SHIFT 0
+#define PALMAS_SMPS6_TSTEP_TSTEP_SHIFT 0x00
/* Bit definitions for SMPS6_FORCE */
#define PALMAS_SMPS6_FORCE_CMD 0x80
-#define PALMAS_SMPS6_FORCE_CMD_SHIFT 7
-#define PALMAS_SMPS6_FORCE_VSEL_MASK 0x7f
-#define PALMAS_SMPS6_FORCE_VSEL_SHIFT 0
+#define PALMAS_SMPS6_FORCE_CMD_SHIFT 0x07
+#define PALMAS_SMPS6_FORCE_VSEL_MASK 0x7F
+#define PALMAS_SMPS6_FORCE_VSEL_SHIFT 0x00
/* Bit definitions for SMPS6_VOLTAGE */
#define PALMAS_SMPS6_VOLTAGE_RANGE 0x80
-#define PALMAS_SMPS6_VOLTAGE_RANGE_SHIFT 7
-#define PALMAS_SMPS6_VOLTAGE_VSEL_MASK 0x7f
-#define PALMAS_SMPS6_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_SMPS6_VOLTAGE_RANGE_SHIFT 0x07
+#define PALMAS_SMPS6_VOLTAGE_VSEL_MASK 0x7F
+#define PALMAS_SMPS6_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for SMPS7_CTRL */
#define PALMAS_SMPS7_CTRL_WR_S 0x80
-#define PALMAS_SMPS7_CTRL_WR_S_SHIFT 7
+#define PALMAS_SMPS7_CTRL_WR_S_SHIFT 0x07
#define PALMAS_SMPS7_CTRL_STATUS_MASK 0x30
-#define PALMAS_SMPS7_CTRL_STATUS_SHIFT 4
+#define PALMAS_SMPS7_CTRL_STATUS_SHIFT 0x04
#define PALMAS_SMPS7_CTRL_MODE_SLEEP_MASK 0x0c
-#define PALMAS_SMPS7_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_SMPS7_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_SMPS7_CTRL_MODE_ACTIVE_MASK 0x03
-#define PALMAS_SMPS7_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_SMPS7_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for SMPS7_VOLTAGE */
#define PALMAS_SMPS7_VOLTAGE_RANGE 0x80
-#define PALMAS_SMPS7_VOLTAGE_RANGE_SHIFT 7
-#define PALMAS_SMPS7_VOLTAGE_VSEL_MASK 0x7f
-#define PALMAS_SMPS7_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_SMPS7_VOLTAGE_RANGE_SHIFT 0x07
+#define PALMAS_SMPS7_VOLTAGE_VSEL_MASK 0x7F
+#define PALMAS_SMPS7_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for SMPS8_CTRL */
#define PALMAS_SMPS8_CTRL_WR_S 0x80
-#define PALMAS_SMPS8_CTRL_WR_S_SHIFT 7
+#define PALMAS_SMPS8_CTRL_WR_S_SHIFT 0x07
#define PALMAS_SMPS8_CTRL_ROOF_FLOOR_EN 0x40
-#define PALMAS_SMPS8_CTRL_ROOF_FLOOR_EN_SHIFT 6
+#define PALMAS_SMPS8_CTRL_ROOF_FLOOR_EN_SHIFT 0x06
#define PALMAS_SMPS8_CTRL_STATUS_MASK 0x30
-#define PALMAS_SMPS8_CTRL_STATUS_SHIFT 4
+#define PALMAS_SMPS8_CTRL_STATUS_SHIFT 0x04
#define PALMAS_SMPS8_CTRL_MODE_SLEEP_MASK 0x0c
-#define PALMAS_SMPS8_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_SMPS8_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_SMPS8_CTRL_MODE_ACTIVE_MASK 0x03
-#define PALMAS_SMPS8_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_SMPS8_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for SMPS8_TSTEP */
#define PALMAS_SMPS8_TSTEP_TSTEP_MASK 0x03
-#define PALMAS_SMPS8_TSTEP_TSTEP_SHIFT 0
+#define PALMAS_SMPS8_TSTEP_TSTEP_SHIFT 0x00
/* Bit definitions for SMPS8_FORCE */
#define PALMAS_SMPS8_FORCE_CMD 0x80
-#define PALMAS_SMPS8_FORCE_CMD_SHIFT 7
-#define PALMAS_SMPS8_FORCE_VSEL_MASK 0x7f
-#define PALMAS_SMPS8_FORCE_VSEL_SHIFT 0
+#define PALMAS_SMPS8_FORCE_CMD_SHIFT 0x07
+#define PALMAS_SMPS8_FORCE_VSEL_MASK 0x7F
+#define PALMAS_SMPS8_FORCE_VSEL_SHIFT 0x00
/* Bit definitions for SMPS8_VOLTAGE */
#define PALMAS_SMPS8_VOLTAGE_RANGE 0x80
-#define PALMAS_SMPS8_VOLTAGE_RANGE_SHIFT 7
-#define PALMAS_SMPS8_VOLTAGE_VSEL_MASK 0x7f
-#define PALMAS_SMPS8_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_SMPS8_VOLTAGE_RANGE_SHIFT 0x07
+#define PALMAS_SMPS8_VOLTAGE_VSEL_MASK 0x7F
+#define PALMAS_SMPS8_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for SMPS9_CTRL */
#define PALMAS_SMPS9_CTRL_WR_S 0x80
-#define PALMAS_SMPS9_CTRL_WR_S_SHIFT 7
+#define PALMAS_SMPS9_CTRL_WR_S_SHIFT 0x07
#define PALMAS_SMPS9_CTRL_STATUS_MASK 0x30
-#define PALMAS_SMPS9_CTRL_STATUS_SHIFT 4
+#define PALMAS_SMPS9_CTRL_STATUS_SHIFT 0x04
#define PALMAS_SMPS9_CTRL_MODE_SLEEP_MASK 0x0c
-#define PALMAS_SMPS9_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_SMPS9_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_SMPS9_CTRL_MODE_ACTIVE_MASK 0x03
-#define PALMAS_SMPS9_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_SMPS9_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for SMPS9_VOLTAGE */
#define PALMAS_SMPS9_VOLTAGE_RANGE 0x80
-#define PALMAS_SMPS9_VOLTAGE_RANGE_SHIFT 7
-#define PALMAS_SMPS9_VOLTAGE_VSEL_MASK 0x7f
-#define PALMAS_SMPS9_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_SMPS9_VOLTAGE_RANGE_SHIFT 0x07
+#define PALMAS_SMPS9_VOLTAGE_VSEL_MASK 0x7F
+#define PALMAS_SMPS9_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for SMPS10_CTRL */
#define PALMAS_SMPS10_CTRL_MODE_SLEEP_MASK 0xf0
-#define PALMAS_SMPS10_CTRL_MODE_SLEEP_SHIFT 4
-#define PALMAS_SMPS10_CTRL_MODE_ACTIVE_MASK 0x0f
-#define PALMAS_SMPS10_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_SMPS10_CTRL_MODE_SLEEP_SHIFT 0x04
+#define PALMAS_SMPS10_CTRL_MODE_ACTIVE_MASK 0x0F
+#define PALMAS_SMPS10_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for SMPS10_STATUS */
-#define PALMAS_SMPS10_STATUS_STATUS_MASK 0x0f
-#define PALMAS_SMPS10_STATUS_STATUS_SHIFT 0
+#define PALMAS_SMPS10_STATUS_STATUS_MASK 0x0F
+#define PALMAS_SMPS10_STATUS_STATUS_SHIFT 0x00
/* Bit definitions for SMPS_CTRL */
#define PALMAS_SMPS_CTRL_SMPS45_SMPS457_EN 0x20
-#define PALMAS_SMPS_CTRL_SMPS45_SMPS457_EN_SHIFT 5
+#define PALMAS_SMPS_CTRL_SMPS45_SMPS457_EN_SHIFT 0x05
#define PALMAS_SMPS_CTRL_SMPS12_SMPS123_EN 0x10
-#define PALMAS_SMPS_CTRL_SMPS12_SMPS123_EN_SHIFT 4
+#define PALMAS_SMPS_CTRL_SMPS12_SMPS123_EN_SHIFT 0x04
#define PALMAS_SMPS_CTRL_SMPS45_PHASE_CTRL_MASK 0x0c
-#define PALMAS_SMPS_CTRL_SMPS45_PHASE_CTRL_SHIFT 2
+#define PALMAS_SMPS_CTRL_SMPS45_PHASE_CTRL_SHIFT 0x02
#define PALMAS_SMPS_CTRL_SMPS123_PHASE_CTRL_MASK 0x03
-#define PALMAS_SMPS_CTRL_SMPS123_PHASE_CTRL_SHIFT 0
+#define PALMAS_SMPS_CTRL_SMPS123_PHASE_CTRL_SHIFT 0x00
/* Bit definitions for SMPS_PD_CTRL */
#define PALMAS_SMPS_PD_CTRL_SMPS9 0x40
-#define PALMAS_SMPS_PD_CTRL_SMPS9_SHIFT 6
+#define PALMAS_SMPS_PD_CTRL_SMPS9_SHIFT 0x06
#define PALMAS_SMPS_PD_CTRL_SMPS8 0x20
-#define PALMAS_SMPS_PD_CTRL_SMPS8_SHIFT 5
+#define PALMAS_SMPS_PD_CTRL_SMPS8_SHIFT 0x05
#define PALMAS_SMPS_PD_CTRL_SMPS7 0x10
-#define PALMAS_SMPS_PD_CTRL_SMPS7_SHIFT 4
+#define PALMAS_SMPS_PD_CTRL_SMPS7_SHIFT 0x04
#define PALMAS_SMPS_PD_CTRL_SMPS6 0x08
-#define PALMAS_SMPS_PD_CTRL_SMPS6_SHIFT 3
+#define PALMAS_SMPS_PD_CTRL_SMPS6_SHIFT 0x03
#define PALMAS_SMPS_PD_CTRL_SMPS45 0x04
-#define PALMAS_SMPS_PD_CTRL_SMPS45_SHIFT 2
+#define PALMAS_SMPS_PD_CTRL_SMPS45_SHIFT 0x02
#define PALMAS_SMPS_PD_CTRL_SMPS3 0x02
-#define PALMAS_SMPS_PD_CTRL_SMPS3_SHIFT 1
+#define PALMAS_SMPS_PD_CTRL_SMPS3_SHIFT 0x01
#define PALMAS_SMPS_PD_CTRL_SMPS12 0x01
-#define PALMAS_SMPS_PD_CTRL_SMPS12_SHIFT 0
+#define PALMAS_SMPS_PD_CTRL_SMPS12_SHIFT 0x00
/* Bit definitions for SMPS_THERMAL_EN */
#define PALMAS_SMPS_THERMAL_EN_SMPS9 0x40
-#define PALMAS_SMPS_THERMAL_EN_SMPS9_SHIFT 6
+#define PALMAS_SMPS_THERMAL_EN_SMPS9_SHIFT 0x06
#define PALMAS_SMPS_THERMAL_EN_SMPS8 0x20
-#define PALMAS_SMPS_THERMAL_EN_SMPS8_SHIFT 5
+#define PALMAS_SMPS_THERMAL_EN_SMPS8_SHIFT 0x05
#define PALMAS_SMPS_THERMAL_EN_SMPS6 0x08
-#define PALMAS_SMPS_THERMAL_EN_SMPS6_SHIFT 3
+#define PALMAS_SMPS_THERMAL_EN_SMPS6_SHIFT 0x03
#define PALMAS_SMPS_THERMAL_EN_SMPS457 0x04
-#define PALMAS_SMPS_THERMAL_EN_SMPS457_SHIFT 2
+#define PALMAS_SMPS_THERMAL_EN_SMPS457_SHIFT 0x02
#define PALMAS_SMPS_THERMAL_EN_SMPS123 0x01
-#define PALMAS_SMPS_THERMAL_EN_SMPS123_SHIFT 0
+#define PALMAS_SMPS_THERMAL_EN_SMPS123_SHIFT 0x00
/* Bit definitions for SMPS_THERMAL_STATUS */
#define PALMAS_SMPS_THERMAL_STATUS_SMPS9 0x40
-#define PALMAS_SMPS_THERMAL_STATUS_SMPS9_SHIFT 6
+#define PALMAS_SMPS_THERMAL_STATUS_SMPS9_SHIFT 0x06
#define PALMAS_SMPS_THERMAL_STATUS_SMPS8 0x20
-#define PALMAS_SMPS_THERMAL_STATUS_SMPS8_SHIFT 5
+#define PALMAS_SMPS_THERMAL_STATUS_SMPS8_SHIFT 0x05
#define PALMAS_SMPS_THERMAL_STATUS_SMPS6 0x08
-#define PALMAS_SMPS_THERMAL_STATUS_SMPS6_SHIFT 3
+#define PALMAS_SMPS_THERMAL_STATUS_SMPS6_SHIFT 0x03
#define PALMAS_SMPS_THERMAL_STATUS_SMPS457 0x04
-#define PALMAS_SMPS_THERMAL_STATUS_SMPS457_SHIFT 2
+#define PALMAS_SMPS_THERMAL_STATUS_SMPS457_SHIFT 0x02
#define PALMAS_SMPS_THERMAL_STATUS_SMPS123 0x01
-#define PALMAS_SMPS_THERMAL_STATUS_SMPS123_SHIFT 0
+#define PALMAS_SMPS_THERMAL_STATUS_SMPS123_SHIFT 0x00
/* Bit definitions for SMPS_SHORT_STATUS */
#define PALMAS_SMPS_SHORT_STATUS_SMPS10 0x80
-#define PALMAS_SMPS_SHORT_STATUS_SMPS10_SHIFT 7
+#define PALMAS_SMPS_SHORT_STATUS_SMPS10_SHIFT 0x07
#define PALMAS_SMPS_SHORT_STATUS_SMPS9 0x40
-#define PALMAS_SMPS_SHORT_STATUS_SMPS9_SHIFT 6
+#define PALMAS_SMPS_SHORT_STATUS_SMPS9_SHIFT 0x06
#define PALMAS_SMPS_SHORT_STATUS_SMPS8 0x20
-#define PALMAS_SMPS_SHORT_STATUS_SMPS8_SHIFT 5
+#define PALMAS_SMPS_SHORT_STATUS_SMPS8_SHIFT 0x05
#define PALMAS_SMPS_SHORT_STATUS_SMPS7 0x10
-#define PALMAS_SMPS_SHORT_STATUS_SMPS7_SHIFT 4
+#define PALMAS_SMPS_SHORT_STATUS_SMPS7_SHIFT 0x04
#define PALMAS_SMPS_SHORT_STATUS_SMPS6 0x08
-#define PALMAS_SMPS_SHORT_STATUS_SMPS6_SHIFT 3
+#define PALMAS_SMPS_SHORT_STATUS_SMPS6_SHIFT 0x03
#define PALMAS_SMPS_SHORT_STATUS_SMPS45 0x04
-#define PALMAS_SMPS_SHORT_STATUS_SMPS45_SHIFT 2
+#define PALMAS_SMPS_SHORT_STATUS_SMPS45_SHIFT 0x02
#define PALMAS_SMPS_SHORT_STATUS_SMPS3 0x02
-#define PALMAS_SMPS_SHORT_STATUS_SMPS3_SHIFT 1
+#define PALMAS_SMPS_SHORT_STATUS_SMPS3_SHIFT 0x01
#define PALMAS_SMPS_SHORT_STATUS_SMPS12 0x01
-#define PALMAS_SMPS_SHORT_STATUS_SMPS12_SHIFT 0
+#define PALMAS_SMPS_SHORT_STATUS_SMPS12_SHIFT 0x00
/* Bit definitions for SMPS_NEGATIVE_CURRENT_LIMIT_EN */
#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS9 0x40
-#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS9_SHIFT 6
+#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS9_SHIFT 0x06
#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS8 0x20
-#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS8_SHIFT 5
+#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS8_SHIFT 0x05
#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS7 0x10
-#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS7_SHIFT 4
+#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS7_SHIFT 0x04
#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS6 0x08
-#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS6_SHIFT 3
+#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS6_SHIFT 0x03
#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS45 0x04
-#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS45_SHIFT 2
+#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS45_SHIFT 0x02
#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS3 0x02
-#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS3_SHIFT 1
+#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS3_SHIFT 0x01
#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS12 0x01
-#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS12_SHIFT 0
+#define PALMAS_SMPS_NEGATIVE_CURRENT_LIMIT_EN_SMPS12_SHIFT 0x00
/* Bit definitions for SMPS_POWERGOOD_MASK1 */
#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS10 0x80
-#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS10_SHIFT 7
+#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS10_SHIFT 0x07
#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS9 0x40
-#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS9_SHIFT 6
+#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS9_SHIFT 0x06
#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS8 0x20
-#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS8_SHIFT 5
+#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS8_SHIFT 0x05
#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS7 0x10
-#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS7_SHIFT 4
+#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS7_SHIFT 0x04
#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS6 0x08
-#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS6_SHIFT 3
+#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS6_SHIFT 0x03
#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS45 0x04
-#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS45_SHIFT 2
+#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS45_SHIFT 0x02
#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS3 0x02
-#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS3_SHIFT 1
+#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS3_SHIFT 0x01
#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS12 0x01
-#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS12_SHIFT 0
+#define PALMAS_SMPS_POWERGOOD_MASK1_SMPS12_SHIFT 0x00
/* Bit definitions for SMPS_POWERGOOD_MASK2 */
#define PALMAS_SMPS_POWERGOOD_MASK2_POWERGOOD_TYPE_SELECT 0x80
-#define PALMAS_SMPS_POWERGOOD_MASK2_POWERGOOD_TYPE_SELECT_SHIFT 7
+#define PALMAS_SMPS_POWERGOOD_MASK2_POWERGOOD_TYPE_SELECT_SHIFT 0x07
#define PALMAS_SMPS_POWERGOOD_MASK2_GPIO_7 0x04
-#define PALMAS_SMPS_POWERGOOD_MASK2_GPIO_7_SHIFT 2
+#define PALMAS_SMPS_POWERGOOD_MASK2_GPIO_7_SHIFT 0x02
#define PALMAS_SMPS_POWERGOOD_MASK2_VBUS 0x02
-#define PALMAS_SMPS_POWERGOOD_MASK2_VBUS_SHIFT 1
+#define PALMAS_SMPS_POWERGOOD_MASK2_VBUS_SHIFT 0x01
#define PALMAS_SMPS_POWERGOOD_MASK2_ACOK 0x01
-#define PALMAS_SMPS_POWERGOOD_MASK2_ACOK_SHIFT 0
+#define PALMAS_SMPS_POWERGOOD_MASK2_ACOK_SHIFT 0x00
/* Registers for function LDO */
-#define PALMAS_LDO1_CTRL 0x0
-#define PALMAS_LDO1_VOLTAGE 0x1
-#define PALMAS_LDO2_CTRL 0x2
-#define PALMAS_LDO2_VOLTAGE 0x3
-#define PALMAS_LDO3_CTRL 0x4
-#define PALMAS_LDO3_VOLTAGE 0x5
-#define PALMAS_LDO4_CTRL 0x6
-#define PALMAS_LDO4_VOLTAGE 0x7
-#define PALMAS_LDO5_CTRL 0x8
-#define PALMAS_LDO5_VOLTAGE 0x9
-#define PALMAS_LDO6_CTRL 0xA
-#define PALMAS_LDO6_VOLTAGE 0xB
-#define PALMAS_LDO7_CTRL 0xC
-#define PALMAS_LDO7_VOLTAGE 0xD
-#define PALMAS_LDO8_CTRL 0xE
-#define PALMAS_LDO8_VOLTAGE 0xF
+#define PALMAS_LDO1_CTRL 0x00
+#define PALMAS_LDO1_VOLTAGE 0x01
+#define PALMAS_LDO2_CTRL 0x02
+#define PALMAS_LDO2_VOLTAGE 0x03
+#define PALMAS_LDO3_CTRL 0x04
+#define PALMAS_LDO3_VOLTAGE 0x05
+#define PALMAS_LDO4_CTRL 0x06
+#define PALMAS_LDO4_VOLTAGE 0x07
+#define PALMAS_LDO5_CTRL 0x08
+#define PALMAS_LDO5_VOLTAGE 0x09
+#define PALMAS_LDO6_CTRL 0x0A
+#define PALMAS_LDO6_VOLTAGE 0x0B
+#define PALMAS_LDO7_CTRL 0x0C
+#define PALMAS_LDO7_VOLTAGE 0x0D
+#define PALMAS_LDO8_CTRL 0x0E
+#define PALMAS_LDO8_VOLTAGE 0x0F
#define PALMAS_LDO9_CTRL 0x10
#define PALMAS_LDO9_VOLTAGE 0x11
#define PALMAS_LDOLN_CTRL 0x12
@@ -1055,236 +1055,236 @@
/* Bit definitions for LDO1_CTRL */
#define PALMAS_LDO1_CTRL_WR_S 0x80
-#define PALMAS_LDO1_CTRL_WR_S_SHIFT 7
+#define PALMAS_LDO1_CTRL_WR_S_SHIFT 0x07
#define PALMAS_LDO1_CTRL_STATUS 0x10
-#define PALMAS_LDO1_CTRL_STATUS_SHIFT 4
+#define PALMAS_LDO1_CTRL_STATUS_SHIFT 0x04
#define PALMAS_LDO1_CTRL_MODE_SLEEP 0x04
-#define PALMAS_LDO1_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_LDO1_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_LDO1_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_LDO1_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_LDO1_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for LDO1_VOLTAGE */
-#define PALMAS_LDO1_VOLTAGE_VSEL_MASK 0x3f
-#define PALMAS_LDO1_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_LDO1_VOLTAGE_VSEL_MASK 0x3F
+#define PALMAS_LDO1_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for LDO2_CTRL */
#define PALMAS_LDO2_CTRL_WR_S 0x80
-#define PALMAS_LDO2_CTRL_WR_S_SHIFT 7
+#define PALMAS_LDO2_CTRL_WR_S_SHIFT 0x07
#define PALMAS_LDO2_CTRL_STATUS 0x10
-#define PALMAS_LDO2_CTRL_STATUS_SHIFT 4
+#define PALMAS_LDO2_CTRL_STATUS_SHIFT 0x04
#define PALMAS_LDO2_CTRL_MODE_SLEEP 0x04
-#define PALMAS_LDO2_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_LDO2_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_LDO2_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_LDO2_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_LDO2_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for LDO2_VOLTAGE */
-#define PALMAS_LDO2_VOLTAGE_VSEL_MASK 0x3f
-#define PALMAS_LDO2_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_LDO2_VOLTAGE_VSEL_MASK 0x3F
+#define PALMAS_LDO2_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for LDO3_CTRL */
#define PALMAS_LDO3_CTRL_WR_S 0x80
-#define PALMAS_LDO3_CTRL_WR_S_SHIFT 7
+#define PALMAS_LDO3_CTRL_WR_S_SHIFT 0x07
#define PALMAS_LDO3_CTRL_STATUS 0x10
-#define PALMAS_LDO3_CTRL_STATUS_SHIFT 4
+#define PALMAS_LDO3_CTRL_STATUS_SHIFT 0x04
#define PALMAS_LDO3_CTRL_MODE_SLEEP 0x04
-#define PALMAS_LDO3_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_LDO3_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_LDO3_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_LDO3_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_LDO3_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for LDO3_VOLTAGE */
-#define PALMAS_LDO3_VOLTAGE_VSEL_MASK 0x3f
-#define PALMAS_LDO3_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_LDO3_VOLTAGE_VSEL_MASK 0x3F
+#define PALMAS_LDO3_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for LDO4_CTRL */
#define PALMAS_LDO4_CTRL_WR_S 0x80
-#define PALMAS_LDO4_CTRL_WR_S_SHIFT 7
+#define PALMAS_LDO4_CTRL_WR_S_SHIFT 0x07
#define PALMAS_LDO4_CTRL_STATUS 0x10
-#define PALMAS_LDO4_CTRL_STATUS_SHIFT 4
+#define PALMAS_LDO4_CTRL_STATUS_SHIFT 0x04
#define PALMAS_LDO4_CTRL_MODE_SLEEP 0x04
-#define PALMAS_LDO4_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_LDO4_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_LDO4_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_LDO4_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_LDO4_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for LDO4_VOLTAGE */
-#define PALMAS_LDO4_VOLTAGE_VSEL_MASK 0x3f
-#define PALMAS_LDO4_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_LDO4_VOLTAGE_VSEL_MASK 0x3F
+#define PALMAS_LDO4_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for LDO5_CTRL */
#define PALMAS_LDO5_CTRL_WR_S 0x80
-#define PALMAS_LDO5_CTRL_WR_S_SHIFT 7
+#define PALMAS_LDO5_CTRL_WR_S_SHIFT 0x07
#define PALMAS_LDO5_CTRL_STATUS 0x10
-#define PALMAS_LDO5_CTRL_STATUS_SHIFT 4
+#define PALMAS_LDO5_CTRL_STATUS_SHIFT 0x04
#define PALMAS_LDO5_CTRL_MODE_SLEEP 0x04
-#define PALMAS_LDO5_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_LDO5_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_LDO5_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_LDO5_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_LDO5_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for LDO5_VOLTAGE */
-#define PALMAS_LDO5_VOLTAGE_VSEL_MASK 0x3f
-#define PALMAS_LDO5_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_LDO5_VOLTAGE_VSEL_MASK 0x3F
+#define PALMAS_LDO5_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for LDO6_CTRL */
#define PALMAS_LDO6_CTRL_WR_S 0x80
-#define PALMAS_LDO6_CTRL_WR_S_SHIFT 7
+#define PALMAS_LDO6_CTRL_WR_S_SHIFT 0x07
#define PALMAS_LDO6_CTRL_LDO_VIB_EN 0x40
-#define PALMAS_LDO6_CTRL_LDO_VIB_EN_SHIFT 6
+#define PALMAS_LDO6_CTRL_LDO_VIB_EN_SHIFT 0x06
#define PALMAS_LDO6_CTRL_STATUS 0x10
-#define PALMAS_LDO6_CTRL_STATUS_SHIFT 4
+#define PALMAS_LDO6_CTRL_STATUS_SHIFT 0x04
#define PALMAS_LDO6_CTRL_MODE_SLEEP 0x04
-#define PALMAS_LDO6_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_LDO6_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_LDO6_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_LDO6_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_LDO6_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for LDO6_VOLTAGE */
-#define PALMAS_LDO6_VOLTAGE_VSEL_MASK 0x3f
-#define PALMAS_LDO6_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_LDO6_VOLTAGE_VSEL_MASK 0x3F
+#define PALMAS_LDO6_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for LDO7_CTRL */
#define PALMAS_LDO7_CTRL_WR_S 0x80
-#define PALMAS_LDO7_CTRL_WR_S_SHIFT 7
+#define PALMAS_LDO7_CTRL_WR_S_SHIFT 0x07
#define PALMAS_LDO7_CTRL_STATUS 0x10
-#define PALMAS_LDO7_CTRL_STATUS_SHIFT 4
+#define PALMAS_LDO7_CTRL_STATUS_SHIFT 0x04
#define PALMAS_LDO7_CTRL_MODE_SLEEP 0x04
-#define PALMAS_LDO7_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_LDO7_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_LDO7_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_LDO7_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_LDO7_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for LDO7_VOLTAGE */
-#define PALMAS_LDO7_VOLTAGE_VSEL_MASK 0x3f
-#define PALMAS_LDO7_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_LDO7_VOLTAGE_VSEL_MASK 0x3F
+#define PALMAS_LDO7_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for LDO8_CTRL */
#define PALMAS_LDO8_CTRL_WR_S 0x80
-#define PALMAS_LDO8_CTRL_WR_S_SHIFT 7
+#define PALMAS_LDO8_CTRL_WR_S_SHIFT 0x07
#define PALMAS_LDO8_CTRL_LDO_TRACKING_EN 0x40
-#define PALMAS_LDO8_CTRL_LDO_TRACKING_EN_SHIFT 6
+#define PALMAS_LDO8_CTRL_LDO_TRACKING_EN_SHIFT 0x06
#define PALMAS_LDO8_CTRL_STATUS 0x10
-#define PALMAS_LDO8_CTRL_STATUS_SHIFT 4
+#define PALMAS_LDO8_CTRL_STATUS_SHIFT 0x04
#define PALMAS_LDO8_CTRL_MODE_SLEEP 0x04
-#define PALMAS_LDO8_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_LDO8_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_LDO8_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_LDO8_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_LDO8_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for LDO8_VOLTAGE */
-#define PALMAS_LDO8_VOLTAGE_VSEL_MASK 0x3f
-#define PALMAS_LDO8_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_LDO8_VOLTAGE_VSEL_MASK 0x3F
+#define PALMAS_LDO8_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for LDO9_CTRL */
#define PALMAS_LDO9_CTRL_WR_S 0x80
-#define PALMAS_LDO9_CTRL_WR_S_SHIFT 7
+#define PALMAS_LDO9_CTRL_WR_S_SHIFT 0x07
#define PALMAS_LDO9_CTRL_LDO_BYPASS_EN 0x40
-#define PALMAS_LDO9_CTRL_LDO_BYPASS_EN_SHIFT 6
+#define PALMAS_LDO9_CTRL_LDO_BYPASS_EN_SHIFT 0x06
#define PALMAS_LDO9_CTRL_STATUS 0x10
-#define PALMAS_LDO9_CTRL_STATUS_SHIFT 4
+#define PALMAS_LDO9_CTRL_STATUS_SHIFT 0x04
#define PALMAS_LDO9_CTRL_MODE_SLEEP 0x04
-#define PALMAS_LDO9_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_LDO9_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_LDO9_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_LDO9_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_LDO9_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for LDO9_VOLTAGE */
-#define PALMAS_LDO9_VOLTAGE_VSEL_MASK 0x3f
-#define PALMAS_LDO9_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_LDO9_VOLTAGE_VSEL_MASK 0x3F
+#define PALMAS_LDO9_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for LDOLN_CTRL */
#define PALMAS_LDOLN_CTRL_WR_S 0x80
-#define PALMAS_LDOLN_CTRL_WR_S_SHIFT 7
+#define PALMAS_LDOLN_CTRL_WR_S_SHIFT 0x07
#define PALMAS_LDOLN_CTRL_STATUS 0x10
-#define PALMAS_LDOLN_CTRL_STATUS_SHIFT 4
+#define PALMAS_LDOLN_CTRL_STATUS_SHIFT 0x04
#define PALMAS_LDOLN_CTRL_MODE_SLEEP 0x04
-#define PALMAS_LDOLN_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_LDOLN_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_LDOLN_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_LDOLN_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_LDOLN_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for LDOLN_VOLTAGE */
-#define PALMAS_LDOLN_VOLTAGE_VSEL_MASK 0x3f
-#define PALMAS_LDOLN_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_LDOLN_VOLTAGE_VSEL_MASK 0x3F
+#define PALMAS_LDOLN_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for LDOUSB_CTRL */
#define PALMAS_LDOUSB_CTRL_WR_S 0x80
-#define PALMAS_LDOUSB_CTRL_WR_S_SHIFT 7
+#define PALMAS_LDOUSB_CTRL_WR_S_SHIFT 0x07
#define PALMAS_LDOUSB_CTRL_STATUS 0x10
-#define PALMAS_LDOUSB_CTRL_STATUS_SHIFT 4
+#define PALMAS_LDOUSB_CTRL_STATUS_SHIFT 0x04
#define PALMAS_LDOUSB_CTRL_MODE_SLEEP 0x04
-#define PALMAS_LDOUSB_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_LDOUSB_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_LDOUSB_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_LDOUSB_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_LDOUSB_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for LDOUSB_VOLTAGE */
-#define PALMAS_LDOUSB_VOLTAGE_VSEL_MASK 0x3f
-#define PALMAS_LDOUSB_VOLTAGE_VSEL_SHIFT 0
+#define PALMAS_LDOUSB_VOLTAGE_VSEL_MASK 0x3F
+#define PALMAS_LDOUSB_VOLTAGE_VSEL_SHIFT 0x00
/* Bit definitions for LDO_CTRL */
#define PALMAS_LDO_CTRL_LDOUSB_ON_VBUS_VSYS 0x01
-#define PALMAS_LDO_CTRL_LDOUSB_ON_VBUS_VSYS_SHIFT 0
+#define PALMAS_LDO_CTRL_LDOUSB_ON_VBUS_VSYS_SHIFT 0x00
/* Bit definitions for LDO_PD_CTRL1 */
#define PALMAS_LDO_PD_CTRL1_LDO8 0x80
-#define PALMAS_LDO_PD_CTRL1_LDO8_SHIFT 7
+#define PALMAS_LDO_PD_CTRL1_LDO8_SHIFT 0x07
#define PALMAS_LDO_PD_CTRL1_LDO7 0x40
-#define PALMAS_LDO_PD_CTRL1_LDO7_SHIFT 6
+#define PALMAS_LDO_PD_CTRL1_LDO7_SHIFT 0x06
#define PALMAS_LDO_PD_CTRL1_LDO6 0x20
-#define PALMAS_LDO_PD_CTRL1_LDO6_SHIFT 5
+#define PALMAS_LDO_PD_CTRL1_LDO6_SHIFT 0x05
#define PALMAS_LDO_PD_CTRL1_LDO5 0x10
-#define PALMAS_LDO_PD_CTRL1_LDO5_SHIFT 4
+#define PALMAS_LDO_PD_CTRL1_LDO5_SHIFT 0x04
#define PALMAS_LDO_PD_CTRL1_LDO4 0x08
-#define PALMAS_LDO_PD_CTRL1_LDO4_SHIFT 3
+#define PALMAS_LDO_PD_CTRL1_LDO4_SHIFT 0x03
#define PALMAS_LDO_PD_CTRL1_LDO3 0x04
-#define PALMAS_LDO_PD_CTRL1_LDO3_SHIFT 2
+#define PALMAS_LDO_PD_CTRL1_LDO3_SHIFT 0x02
#define PALMAS_LDO_PD_CTRL1_LDO2 0x02
-#define PALMAS_LDO_PD_CTRL1_LDO2_SHIFT 1
+#define PALMAS_LDO_PD_CTRL1_LDO2_SHIFT 0x01
#define PALMAS_LDO_PD_CTRL1_LDO1 0x01
-#define PALMAS_LDO_PD_CTRL1_LDO1_SHIFT 0
+#define PALMAS_LDO_PD_CTRL1_LDO1_SHIFT 0x00
/* Bit definitions for LDO_PD_CTRL2 */
#define PALMAS_LDO_PD_CTRL2_LDOUSB 0x04
-#define PALMAS_LDO_PD_CTRL2_LDOUSB_SHIFT 2
+#define PALMAS_LDO_PD_CTRL2_LDOUSB_SHIFT 0x02
#define PALMAS_LDO_PD_CTRL2_LDOLN 0x02
-#define PALMAS_LDO_PD_CTRL2_LDOLN_SHIFT 1
+#define PALMAS_LDO_PD_CTRL2_LDOLN_SHIFT 0x01
#define PALMAS_LDO_PD_CTRL2_LDO9 0x01
-#define PALMAS_LDO_PD_CTRL2_LDO9_SHIFT 0
+#define PALMAS_LDO_PD_CTRL2_LDO9_SHIFT 0x00
/* Bit definitions for LDO_SHORT_STATUS1 */
#define PALMAS_LDO_SHORT_STATUS1_LDO8 0x80
-#define PALMAS_LDO_SHORT_STATUS1_LDO8_SHIFT 7
+#define PALMAS_LDO_SHORT_STATUS1_LDO8_SHIFT 0x07
#define PALMAS_LDO_SHORT_STATUS1_LDO7 0x40
-#define PALMAS_LDO_SHORT_STATUS1_LDO7_SHIFT 6
+#define PALMAS_LDO_SHORT_STATUS1_LDO7_SHIFT 0x06
#define PALMAS_LDO_SHORT_STATUS1_LDO6 0x20
-#define PALMAS_LDO_SHORT_STATUS1_LDO6_SHIFT 5
+#define PALMAS_LDO_SHORT_STATUS1_LDO6_SHIFT 0x05
#define PALMAS_LDO_SHORT_STATUS1_LDO5 0x10
-#define PALMAS_LDO_SHORT_STATUS1_LDO5_SHIFT 4
+#define PALMAS_LDO_SHORT_STATUS1_LDO5_SHIFT 0x04
#define PALMAS_LDO_SHORT_STATUS1_LDO4 0x08
-#define PALMAS_LDO_SHORT_STATUS1_LDO4_SHIFT 3
+#define PALMAS_LDO_SHORT_STATUS1_LDO4_SHIFT 0x03
#define PALMAS_LDO_SHORT_STATUS1_LDO3 0x04
-#define PALMAS_LDO_SHORT_STATUS1_LDO3_SHIFT 2
+#define PALMAS_LDO_SHORT_STATUS1_LDO3_SHIFT 0x02
#define PALMAS_LDO_SHORT_STATUS1_LDO2 0x02
-#define PALMAS_LDO_SHORT_STATUS1_LDO2_SHIFT 1
+#define PALMAS_LDO_SHORT_STATUS1_LDO2_SHIFT 0x01
#define PALMAS_LDO_SHORT_STATUS1_LDO1 0x01
-#define PALMAS_LDO_SHORT_STATUS1_LDO1_SHIFT 0
+#define PALMAS_LDO_SHORT_STATUS1_LDO1_SHIFT 0x00
/* Bit definitions for LDO_SHORT_STATUS2 */
#define PALMAS_LDO_SHORT_STATUS2_LDOVANA 0x08
-#define PALMAS_LDO_SHORT_STATUS2_LDOVANA_SHIFT 3
+#define PALMAS_LDO_SHORT_STATUS2_LDOVANA_SHIFT 0x03
#define PALMAS_LDO_SHORT_STATUS2_LDOUSB 0x04
-#define PALMAS_LDO_SHORT_STATUS2_LDOUSB_SHIFT 2
+#define PALMAS_LDO_SHORT_STATUS2_LDOUSB_SHIFT 0x02
#define PALMAS_LDO_SHORT_STATUS2_LDOLN 0x02
-#define PALMAS_LDO_SHORT_STATUS2_LDOLN_SHIFT 1
+#define PALMAS_LDO_SHORT_STATUS2_LDOLN_SHIFT 0x01
#define PALMAS_LDO_SHORT_STATUS2_LDO9 0x01
-#define PALMAS_LDO_SHORT_STATUS2_LDO9_SHIFT 0
+#define PALMAS_LDO_SHORT_STATUS2_LDO9_SHIFT 0x00
/* Registers for function PMU_CONTROL */
-#define PALMAS_DEV_CTRL 0x0
-#define PALMAS_POWER_CTRL 0x1
-#define PALMAS_VSYS_LO 0x2
-#define PALMAS_VSYS_MON 0x3
-#define PALMAS_VBAT_MON 0x4
-#define PALMAS_WATCHDOG 0x5
-#define PALMAS_BOOT_STATUS 0x6
-#define PALMAS_BATTERY_BOUNCE 0x7
-#define PALMAS_BACKUP_BATTERY_CTRL 0x8
-#define PALMAS_LONG_PRESS_KEY 0x9
-#define PALMAS_OSC_THERM_CTRL 0xA
-#define PALMAS_BATDEBOUNCING 0xB
-#define PALMAS_SWOFF_HWRST 0xF
+#define PALMAS_DEV_CTRL 0x00
+#define PALMAS_POWER_CTRL 0x01
+#define PALMAS_VSYS_LO 0x02
+#define PALMAS_VSYS_MON 0x03
+#define PALMAS_VBAT_MON 0x04
+#define PALMAS_WATCHDOG 0x05
+#define PALMAS_BOOT_STATUS 0x06
+#define PALMAS_BATTERY_BOUNCE 0x07
+#define PALMAS_BACKUP_BATTERY_CTRL 0x08
+#define PALMAS_LONG_PRESS_KEY 0x09
+#define PALMAS_OSC_THERM_CTRL 0x0A
+#define PALMAS_BATDEBOUNCING 0x0B
+#define PALMAS_SWOFF_HWRST 0x0F
#define PALMAS_SWOFF_COLDRST 0x10
#define PALMAS_SWOFF_STATUS 0x11
#define PALMAS_PMU_CONFIG 0x12
@@ -1296,668 +1296,668 @@
/* Bit definitions for DEV_CTRL */
#define PALMAS_DEV_CTRL_DEV_STATUS_MASK 0x0c
-#define PALMAS_DEV_CTRL_DEV_STATUS_SHIFT 2
+#define PALMAS_DEV_CTRL_DEV_STATUS_SHIFT 0x02
#define PALMAS_DEV_CTRL_SW_RST 0x02
-#define PALMAS_DEV_CTRL_SW_RST_SHIFT 1
+#define PALMAS_DEV_CTRL_SW_RST_SHIFT 0x01
#define PALMAS_DEV_CTRL_DEV_ON 0x01
-#define PALMAS_DEV_CTRL_DEV_ON_SHIFT 0
+#define PALMAS_DEV_CTRL_DEV_ON_SHIFT 0x00
/* Bit definitions for POWER_CTRL */
#define PALMAS_POWER_CTRL_ENABLE2_MASK 0x04
-#define PALMAS_POWER_CTRL_ENABLE2_MASK_SHIFT 2
+#define PALMAS_POWER_CTRL_ENABLE2_MASK_SHIFT 0x02
#define PALMAS_POWER_CTRL_ENABLE1_MASK 0x02
-#define PALMAS_POWER_CTRL_ENABLE1_MASK_SHIFT 1
+#define PALMAS_POWER_CTRL_ENABLE1_MASK_SHIFT 0x01
#define PALMAS_POWER_CTRL_NSLEEP_MASK 0x01
-#define PALMAS_POWER_CTRL_NSLEEP_MASK_SHIFT 0
+#define PALMAS_POWER_CTRL_NSLEEP_MASK_SHIFT 0x00
/* Bit definitions for VSYS_LO */
-#define PALMAS_VSYS_LO_THRESHOLD_MASK 0x1f
-#define PALMAS_VSYS_LO_THRESHOLD_SHIFT 0
+#define PALMAS_VSYS_LO_THRESHOLD_MASK 0x1F
+#define PALMAS_VSYS_LO_THRESHOLD_SHIFT 0x00
/* Bit definitions for VSYS_MON */
#define PALMAS_VSYS_MON_ENABLE 0x80
-#define PALMAS_VSYS_MON_ENABLE_SHIFT 7
-#define PALMAS_VSYS_MON_THRESHOLD_MASK 0x3f
-#define PALMAS_VSYS_MON_THRESHOLD_SHIFT 0
+#define PALMAS_VSYS_MON_ENABLE_SHIFT 0x07
+#define PALMAS_VSYS_MON_THRESHOLD_MASK 0x3F
+#define PALMAS_VSYS_MON_THRESHOLD_SHIFT 0x00
/* Bit definitions for VBAT_MON */
#define PALMAS_VBAT_MON_ENABLE 0x80
-#define PALMAS_VBAT_MON_ENABLE_SHIFT 7
-#define PALMAS_VBAT_MON_THRESHOLD_MASK 0x3f
-#define PALMAS_VBAT_MON_THRESHOLD_SHIFT 0
+#define PALMAS_VBAT_MON_ENABLE_SHIFT 0x07
+#define PALMAS_VBAT_MON_THRESHOLD_MASK 0x3F
+#define PALMAS_VBAT_MON_THRESHOLD_SHIFT 0x00
/* Bit definitions for WATCHDOG */
#define PALMAS_WATCHDOG_LOCK 0x20
-#define PALMAS_WATCHDOG_LOCK_SHIFT 5
+#define PALMAS_WATCHDOG_LOCK_SHIFT 0x05
#define PALMAS_WATCHDOG_ENABLE 0x10
-#define PALMAS_WATCHDOG_ENABLE_SHIFT 4
+#define PALMAS_WATCHDOG_ENABLE_SHIFT 0x04
#define PALMAS_WATCHDOG_MODE 0x08
-#define PALMAS_WATCHDOG_MODE_SHIFT 3
+#define PALMAS_WATCHDOG_MODE_SHIFT 0x03
#define PALMAS_WATCHDOG_TIMER_MASK 0x07
-#define PALMAS_WATCHDOG_TIMER_SHIFT 0
+#define PALMAS_WATCHDOG_TIMER_SHIFT 0x00
/* Bit definitions for BOOT_STATUS */
#define PALMAS_BOOT_STATUS_BOOT1 0x02
-#define PALMAS_BOOT_STATUS_BOOT1_SHIFT 1
+#define PALMAS_BOOT_STATUS_BOOT1_SHIFT 0x01
#define PALMAS_BOOT_STATUS_BOOT0 0x01
-#define PALMAS_BOOT_STATUS_BOOT0_SHIFT 0
+#define PALMAS_BOOT_STATUS_BOOT0_SHIFT 0x00
/* Bit definitions for BATTERY_BOUNCE */
-#define PALMAS_BATTERY_BOUNCE_BB_DELAY_MASK 0x3f
-#define PALMAS_BATTERY_BOUNCE_BB_DELAY_SHIFT 0
+#define PALMAS_BATTERY_BOUNCE_BB_DELAY_MASK 0x3F
+#define PALMAS_BATTERY_BOUNCE_BB_DELAY_SHIFT 0x00
/* Bit definitions for BACKUP_BATTERY_CTRL */
#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_18_15 0x80
-#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_18_15_SHIFT 7
+#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_18_15_SHIFT 0x07
#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_EN_SLP 0x40
-#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_EN_SLP_SHIFT 6
+#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_EN_SLP_SHIFT 0x06
#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_EN_OFF 0x20
-#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_EN_OFF_SHIFT 5
+#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_EN_OFF_SHIFT 0x05
#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_PWEN 0x10
-#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_PWEN_SHIFT 4
+#define PALMAS_BACKUP_BATTERY_CTRL_VRTC_PWEN_SHIFT 0x04
#define PALMAS_BACKUP_BATTERY_CTRL_BBS_BBC_LOW_ICHRG 0x08
-#define PALMAS_BACKUP_BATTERY_CTRL_BBS_BBC_LOW_ICHRG_SHIFT 3
+#define PALMAS_BACKUP_BATTERY_CTRL_BBS_BBC_LOW_ICHRG_SHIFT 0x03
#define PALMAS_BACKUP_BATTERY_CTRL_BB_SEL_MASK 0x06
-#define PALMAS_BACKUP_BATTERY_CTRL_BB_SEL_SHIFT 1
+#define PALMAS_BACKUP_BATTERY_CTRL_BB_SEL_SHIFT 0x01
#define PALMAS_BACKUP_BATTERY_CTRL_BB_CHG_EN 0x01
-#define PALMAS_BACKUP_BATTERY_CTRL_BB_CHG_EN_SHIFT 0
+#define PALMAS_BACKUP_BATTERY_CTRL_BB_CHG_EN_SHIFT 0x00
/* Bit definitions for LONG_PRESS_KEY */
#define PALMAS_LONG_PRESS_KEY_LPK_LOCK 0x80
-#define PALMAS_LONG_PRESS_KEY_LPK_LOCK_SHIFT 7
+#define PALMAS_LONG_PRESS_KEY_LPK_LOCK_SHIFT 0x07
#define PALMAS_LONG_PRESS_KEY_LPK_INT_CLR 0x10
-#define PALMAS_LONG_PRESS_KEY_LPK_INT_CLR_SHIFT 4
+#define PALMAS_LONG_PRESS_KEY_LPK_INT_CLR_SHIFT 0x04
#define PALMAS_LONG_PRESS_KEY_LPK_TIME_MASK 0x0c
-#define PALMAS_LONG_PRESS_KEY_LPK_TIME_SHIFT 2
+#define PALMAS_LONG_PRESS_KEY_LPK_TIME_SHIFT 0x02
#define PALMAS_LONG_PRESS_KEY_PWRON_DEBOUNCE_MASK 0x03
-#define PALMAS_LONG_PRESS_KEY_PWRON_DEBOUNCE_SHIFT 0
+#define PALMAS_LONG_PRESS_KEY_PWRON_DEBOUNCE_SHIFT 0x00
/* Bit definitions for OSC_THERM_CTRL */
#define PALMAS_OSC_THERM_CTRL_VANA_ON_IN_SLEEP 0x80
-#define PALMAS_OSC_THERM_CTRL_VANA_ON_IN_SLEEP_SHIFT 7
+#define PALMAS_OSC_THERM_CTRL_VANA_ON_IN_SLEEP_SHIFT 0x07
#define PALMAS_OSC_THERM_CTRL_INT_MASK_IN_SLEEP 0x40
-#define PALMAS_OSC_THERM_CTRL_INT_MASK_IN_SLEEP_SHIFT 6
+#define PALMAS_OSC_THERM_CTRL_INT_MASK_IN_SLEEP_SHIFT 0x06
#define PALMAS_OSC_THERM_CTRL_RC15MHZ_ON_IN_SLEEP 0x20
-#define PALMAS_OSC_THERM_CTRL_RC15MHZ_ON_IN_SLEEP_SHIFT 5
+#define PALMAS_OSC_THERM_CTRL_RC15MHZ_ON_IN_SLEEP_SHIFT 0x05
#define PALMAS_OSC_THERM_CTRL_THERM_OFF_IN_SLEEP 0x10
-#define PALMAS_OSC_THERM_CTRL_THERM_OFF_IN_SLEEP_SHIFT 4
+#define PALMAS_OSC_THERM_CTRL_THERM_OFF_IN_SLEEP_SHIFT 0x04
#define PALMAS_OSC_THERM_CTRL_THERM_HD_SEL_MASK 0x0c
-#define PALMAS_OSC_THERM_CTRL_THERM_HD_SEL_SHIFT 2
+#define PALMAS_OSC_THERM_CTRL_THERM_HD_SEL_SHIFT 0x02
#define PALMAS_OSC_THERM_CTRL_OSC_BYPASS 0x02
-#define PALMAS_OSC_THERM_CTRL_OSC_BYPASS_SHIFT 1
+#define PALMAS_OSC_THERM_CTRL_OSC_BYPASS_SHIFT 0x01
#define PALMAS_OSC_THERM_CTRL_OSC_HPMODE 0x01
-#define PALMAS_OSC_THERM_CTRL_OSC_HPMODE_SHIFT 0
+#define PALMAS_OSC_THERM_CTRL_OSC_HPMODE_SHIFT 0x00
/* Bit definitions for BATDEBOUNCING */
#define PALMAS_BATDEBOUNCING_BAT_DEB_BYPASS 0x80
-#define PALMAS_BATDEBOUNCING_BAT_DEB_BYPASS_SHIFT 7
+#define PALMAS_BATDEBOUNCING_BAT_DEB_BYPASS_SHIFT 0x07
#define PALMAS_BATDEBOUNCING_BINS_DEB_MASK 0x78
-#define PALMAS_BATDEBOUNCING_BINS_DEB_SHIFT 3
+#define PALMAS_BATDEBOUNCING_BINS_DEB_SHIFT 0x03
#define PALMAS_BATDEBOUNCING_BEXT_DEB_MASK 0x07
-#define PALMAS_BATDEBOUNCING_BEXT_DEB_SHIFT 0
+#define PALMAS_BATDEBOUNCING_BEXT_DEB_SHIFT 0x00
/* Bit definitions for SWOFF_HWRST */
#define PALMAS_SWOFF_HWRST_PWRON_LPK 0x80
-#define PALMAS_SWOFF_HWRST_PWRON_LPK_SHIFT 7
+#define PALMAS_SWOFF_HWRST_PWRON_LPK_SHIFT 0x07
#define PALMAS_SWOFF_HWRST_PWRDOWN 0x40
-#define PALMAS_SWOFF_HWRST_PWRDOWN_SHIFT 6
+#define PALMAS_SWOFF_HWRST_PWRDOWN_SHIFT 0x06
#define PALMAS_SWOFF_HWRST_WTD 0x20
-#define PALMAS_SWOFF_HWRST_WTD_SHIFT 5
+#define PALMAS_SWOFF_HWRST_WTD_SHIFT 0x05
#define PALMAS_SWOFF_HWRST_TSHUT 0x10
-#define PALMAS_SWOFF_HWRST_TSHUT_SHIFT 4
+#define PALMAS_SWOFF_HWRST_TSHUT_SHIFT 0x04
#define PALMAS_SWOFF_HWRST_RESET_IN 0x08
-#define PALMAS_SWOFF_HWRST_RESET_IN_SHIFT 3
+#define PALMAS_SWOFF_HWRST_RESET_IN_SHIFT 0x03
#define PALMAS_SWOFF_HWRST_SW_RST 0x04
-#define PALMAS_SWOFF_HWRST_SW_RST_SHIFT 2
+#define PALMAS_SWOFF_HWRST_SW_RST_SHIFT 0x02
#define PALMAS_SWOFF_HWRST_VSYS_LO 0x02
-#define PALMAS_SWOFF_HWRST_VSYS_LO_SHIFT 1
+#define PALMAS_SWOFF_HWRST_VSYS_LO_SHIFT 0x01
#define PALMAS_SWOFF_HWRST_GPADC_SHUTDOWN 0x01
-#define PALMAS_SWOFF_HWRST_GPADC_SHUTDOWN_SHIFT 0
+#define PALMAS_SWOFF_HWRST_GPADC_SHUTDOWN_SHIFT 0x00
/* Bit definitions for SWOFF_COLDRST */
#define PALMAS_SWOFF_COLDRST_PWRON_LPK 0x80
-#define PALMAS_SWOFF_COLDRST_PWRON_LPK_SHIFT 7
+#define PALMAS_SWOFF_COLDRST_PWRON_LPK_SHIFT 0x07
#define PALMAS_SWOFF_COLDRST_PWRDOWN 0x40
-#define PALMAS_SWOFF_COLDRST_PWRDOWN_SHIFT 6
+#define PALMAS_SWOFF_COLDRST_PWRDOWN_SHIFT 0x06
#define PALMAS_SWOFF_COLDRST_WTD 0x20
-#define PALMAS_SWOFF_COLDRST_WTD_SHIFT 5
+#define PALMAS_SWOFF_COLDRST_WTD_SHIFT 0x05
#define PALMAS_SWOFF_COLDRST_TSHUT 0x10
-#define PALMAS_SWOFF_COLDRST_TSHUT_SHIFT 4
+#define PALMAS_SWOFF_COLDRST_TSHUT_SHIFT 0x04
#define PALMAS_SWOFF_COLDRST_RESET_IN 0x08
-#define PALMAS_SWOFF_COLDRST_RESET_IN_SHIFT 3
+#define PALMAS_SWOFF_COLDRST_RESET_IN_SHIFT 0x03
#define PALMAS_SWOFF_COLDRST_SW_RST 0x04
-#define PALMAS_SWOFF_COLDRST_SW_RST_SHIFT 2
+#define PALMAS_SWOFF_COLDRST_SW_RST_SHIFT 0x02
#define PALMAS_SWOFF_COLDRST_VSYS_LO 0x02
-#define PALMAS_SWOFF_COLDRST_VSYS_LO_SHIFT 1
+#define PALMAS_SWOFF_COLDRST_VSYS_LO_SHIFT 0x01
#define PALMAS_SWOFF_COLDRST_GPADC_SHUTDOWN 0x01
-#define PALMAS_SWOFF_COLDRST_GPADC_SHUTDOWN_SHIFT 0
+#define PALMAS_SWOFF_COLDRST_GPADC_SHUTDOWN_SHIFT 0x00
/* Bit definitions for SWOFF_STATUS */
#define PALMAS_SWOFF_STATUS_PWRON_LPK 0x80
-#define PALMAS_SWOFF_STATUS_PWRON_LPK_SHIFT 7
+#define PALMAS_SWOFF_STATUS_PWRON_LPK_SHIFT 0x07
#define PALMAS_SWOFF_STATUS_PWRDOWN 0x40
-#define PALMAS_SWOFF_STATUS_PWRDOWN_SHIFT 6
+#define PALMAS_SWOFF_STATUS_PWRDOWN_SHIFT 0x06
#define PALMAS_SWOFF_STATUS_WTD 0x20
-#define PALMAS_SWOFF_STATUS_WTD_SHIFT 5
+#define PALMAS_SWOFF_STATUS_WTD_SHIFT 0x05
#define PALMAS_SWOFF_STATUS_TSHUT 0x10
-#define PALMAS_SWOFF_STATUS_TSHUT_SHIFT 4
+#define PALMAS_SWOFF_STATUS_TSHUT_SHIFT 0x04
#define PALMAS_SWOFF_STATUS_RESET_IN 0x08
-#define PALMAS_SWOFF_STATUS_RESET_IN_SHIFT 3
+#define PALMAS_SWOFF_STATUS_RESET_IN_SHIFT 0x03
#define PALMAS_SWOFF_STATUS_SW_RST 0x04
-#define PALMAS_SWOFF_STATUS_SW_RST_SHIFT 2
+#define PALMAS_SWOFF_STATUS_SW_RST_SHIFT 0x02
#define PALMAS_SWOFF_STATUS_VSYS_LO 0x02
-#define PALMAS_SWOFF_STATUS_VSYS_LO_SHIFT 1
+#define PALMAS_SWOFF_STATUS_VSYS_LO_SHIFT 0x01
#define PALMAS_SWOFF_STATUS_GPADC_SHUTDOWN 0x01
-#define PALMAS_SWOFF_STATUS_GPADC_SHUTDOWN_SHIFT 0
+#define PALMAS_SWOFF_STATUS_GPADC_SHUTDOWN_SHIFT 0x00
/* Bit definitions for PMU_CONFIG */
#define PALMAS_PMU_CONFIG_MULTI_CELL_EN 0x40
-#define PALMAS_PMU_CONFIG_MULTI_CELL_EN_SHIFT 6
+#define PALMAS_PMU_CONFIG_MULTI_CELL_EN_SHIFT 0x06
#define PALMAS_PMU_CONFIG_SPARE_MASK 0x30
-#define PALMAS_PMU_CONFIG_SPARE_SHIFT 4
+#define PALMAS_PMU_CONFIG_SPARE_SHIFT 0x04
#define PALMAS_PMU_CONFIG_SWOFF_DLY_MASK 0x0c
-#define PALMAS_PMU_CONFIG_SWOFF_DLY_SHIFT 2
+#define PALMAS_PMU_CONFIG_SWOFF_DLY_SHIFT 0x02
#define PALMAS_PMU_CONFIG_GATE_RESET_OUT 0x02
-#define PALMAS_PMU_CONFIG_GATE_RESET_OUT_SHIFT 1
+#define PALMAS_PMU_CONFIG_GATE_RESET_OUT_SHIFT 0x01
#define PALMAS_PMU_CONFIG_AUTODEVON 0x01
-#define PALMAS_PMU_CONFIG_AUTODEVON_SHIFT 0
+#define PALMAS_PMU_CONFIG_AUTODEVON_SHIFT 0x00
/* Bit definitions for SPARE */
#define PALMAS_SPARE_SPARE_MASK 0xf8
-#define PALMAS_SPARE_SPARE_SHIFT 3
+#define PALMAS_SPARE_SPARE_SHIFT 0x03
#define PALMAS_SPARE_REGEN3_OD 0x04
-#define PALMAS_SPARE_REGEN3_OD_SHIFT 2
+#define PALMAS_SPARE_REGEN3_OD_SHIFT 0x02
#define PALMAS_SPARE_REGEN2_OD 0x02
-#define PALMAS_SPARE_REGEN2_OD_SHIFT 1
+#define PALMAS_SPARE_REGEN2_OD_SHIFT 0x01
#define PALMAS_SPARE_REGEN1_OD 0x01
-#define PALMAS_SPARE_REGEN1_OD_SHIFT 0
+#define PALMAS_SPARE_REGEN1_OD_SHIFT 0x00
/* Bit definitions for PMU_SECONDARY_INT */
#define PALMAS_PMU_SECONDARY_INT_VBUS_OVV_INT_SRC 0x80
-#define PALMAS_PMU_SECONDARY_INT_VBUS_OVV_INT_SRC_SHIFT 7
+#define PALMAS_PMU_SECONDARY_INT_VBUS_OVV_INT_SRC_SHIFT 0x07
#define PALMAS_PMU_SECONDARY_INT_CHARG_DET_N_INT_SRC 0x40
-#define PALMAS_PMU_SECONDARY_INT_CHARG_DET_N_INT_SRC_SHIFT 6
+#define PALMAS_PMU_SECONDARY_INT_CHARG_DET_N_INT_SRC_SHIFT 0x06
#define PALMAS_PMU_SECONDARY_INT_BB_INT_SRC 0x20
-#define PALMAS_PMU_SECONDARY_INT_BB_INT_SRC_SHIFT 5
+#define PALMAS_PMU_SECONDARY_INT_BB_INT_SRC_SHIFT 0x05
#define PALMAS_PMU_SECONDARY_INT_FBI_INT_SRC 0x10
-#define PALMAS_PMU_SECONDARY_INT_FBI_INT_SRC_SHIFT 4
+#define PALMAS_PMU_SECONDARY_INT_FBI_INT_SRC_SHIFT 0x04
#define PALMAS_PMU_SECONDARY_INT_VBUS_OVV_MASK 0x08
-#define PALMAS_PMU_SECONDARY_INT_VBUS_OVV_MASK_SHIFT 3
+#define PALMAS_PMU_SECONDARY_INT_VBUS_OVV_MASK_SHIFT 0x03
#define PALMAS_PMU_SECONDARY_INT_CHARG_DET_N_MASK 0x04
-#define PALMAS_PMU_SECONDARY_INT_CHARG_DET_N_MASK_SHIFT 2
+#define PALMAS_PMU_SECONDARY_INT_CHARG_DET_N_MASK_SHIFT 0x02
#define PALMAS_PMU_SECONDARY_INT_BB_MASK 0x02
-#define PALMAS_PMU_SECONDARY_INT_BB_MASK_SHIFT 1
+#define PALMAS_PMU_SECONDARY_INT_BB_MASK_SHIFT 0x01
#define PALMAS_PMU_SECONDARY_INT_FBI_MASK 0x01
-#define PALMAS_PMU_SECONDARY_INT_FBI_MASK_SHIFT 0
+#define PALMAS_PMU_SECONDARY_INT_FBI_MASK_SHIFT 0x00
/* Bit definitions for SW_REVISION */
-#define PALMAS_SW_REVISION_SW_REVISION_MASK 0xff
-#define PALMAS_SW_REVISION_SW_REVISION_SHIFT 0
+#define PALMAS_SW_REVISION_SW_REVISION_MASK 0xFF
+#define PALMAS_SW_REVISION_SW_REVISION_SHIFT 0x00
/* Bit definitions for EXT_CHRG_CTRL */
#define PALMAS_EXT_CHRG_CTRL_VBUS_OVV_STATUS 0x80
-#define PALMAS_EXT_CHRG_CTRL_VBUS_OVV_STATUS_SHIFT 7
+#define PALMAS_EXT_CHRG_CTRL_VBUS_OVV_STATUS_SHIFT 0x07
#define PALMAS_EXT_CHRG_CTRL_CHARG_DET_N_STATUS 0x40
-#define PALMAS_EXT_CHRG_CTRL_CHARG_DET_N_STATUS_SHIFT 6
+#define PALMAS_EXT_CHRG_CTRL_CHARG_DET_N_STATUS_SHIFT 0x06
#define PALMAS_EXT_CHRG_CTRL_VSYS_DEBOUNCE_DELAY 0x08
-#define PALMAS_EXT_CHRG_CTRL_VSYS_DEBOUNCE_DELAY_SHIFT 3
+#define PALMAS_EXT_CHRG_CTRL_VSYS_DEBOUNCE_DELAY_SHIFT 0x03
#define PALMAS_EXT_CHRG_CTRL_CHRG_DET_N 0x04
-#define PALMAS_EXT_CHRG_CTRL_CHRG_DET_N_SHIFT 2
+#define PALMAS_EXT_CHRG_CTRL_CHRG_DET_N_SHIFT 0x02
#define PALMAS_EXT_CHRG_CTRL_AUTO_ACA_EN 0x02
-#define PALMAS_EXT_CHRG_CTRL_AUTO_ACA_EN_SHIFT 1
+#define PALMAS_EXT_CHRG_CTRL_AUTO_ACA_EN_SHIFT 0x01
#define PALMAS_EXT_CHRG_CTRL_AUTO_LDOUSB_EN 0x01
-#define PALMAS_EXT_CHRG_CTRL_AUTO_LDOUSB_EN_SHIFT 0
+#define PALMAS_EXT_CHRG_CTRL_AUTO_LDOUSB_EN_SHIFT 0x00
/* Bit definitions for PMU_SECONDARY_INT2 */
#define PALMAS_PMU_SECONDARY_INT2_DVFS2_INT_SRC 0x20
-#define PALMAS_PMU_SECONDARY_INT2_DVFS2_INT_SRC_SHIFT 5
+#define PALMAS_PMU_SECONDARY_INT2_DVFS2_INT_SRC_SHIFT 0x05
#define PALMAS_PMU_SECONDARY_INT2_DVFS1_INT_SRC 0x10
-#define PALMAS_PMU_SECONDARY_INT2_DVFS1_INT_SRC_SHIFT 4
+#define PALMAS_PMU_SECONDARY_INT2_DVFS1_INT_SRC_SHIFT 0x04
#define PALMAS_PMU_SECONDARY_INT2_DVFS2_MASK 0x02
-#define PALMAS_PMU_SECONDARY_INT2_DVFS2_MASK_SHIFT 1
+#define PALMAS_PMU_SECONDARY_INT2_DVFS2_MASK_SHIFT 0x01
#define PALMAS_PMU_SECONDARY_INT2_DVFS1_MASK 0x01
-#define PALMAS_PMU_SECONDARY_INT2_DVFS1_MASK_SHIFT 0
+#define PALMAS_PMU_SECONDARY_INT2_DVFS1_MASK_SHIFT 0x00
/* Registers for function RESOURCE */
-#define PALMAS_CLK32KG_CTRL 0x0
-#define PALMAS_CLK32KGAUDIO_CTRL 0x1
-#define PALMAS_REGEN1_CTRL 0x2
-#define PALMAS_REGEN2_CTRL 0x3
-#define PALMAS_SYSEN1_CTRL 0x4
-#define PALMAS_SYSEN2_CTRL 0x5
-#define PALMAS_NSLEEP_RES_ASSIGN 0x6
-#define PALMAS_NSLEEP_SMPS_ASSIGN 0x7
-#define PALMAS_NSLEEP_LDO_ASSIGN1 0x8
-#define PALMAS_NSLEEP_LDO_ASSIGN2 0x9
-#define PALMAS_ENABLE1_RES_ASSIGN 0xA
-#define PALMAS_ENABLE1_SMPS_ASSIGN 0xB
-#define PALMAS_ENABLE1_LDO_ASSIGN1 0xC
-#define PALMAS_ENABLE1_LDO_ASSIGN2 0xD
-#define PALMAS_ENABLE2_RES_ASSIGN 0xE
-#define PALMAS_ENABLE2_SMPS_ASSIGN 0xF
+#define PALMAS_CLK32KG_CTRL 0x00
+#define PALMAS_CLK32KGAUDIO_CTRL 0x01
+#define PALMAS_REGEN1_CTRL 0x02
+#define PALMAS_REGEN2_CTRL 0x03
+#define PALMAS_SYSEN1_CTRL 0x04
+#define PALMAS_SYSEN2_CTRL 0x05
+#define PALMAS_NSLEEP_RES_ASSIGN 0x06
+#define PALMAS_NSLEEP_SMPS_ASSIGN 0x07
+#define PALMAS_NSLEEP_LDO_ASSIGN1 0x08
+#define PALMAS_NSLEEP_LDO_ASSIGN2 0x09
+#define PALMAS_ENABLE1_RES_ASSIGN 0x0A
+#define PALMAS_ENABLE1_SMPS_ASSIGN 0x0B
+#define PALMAS_ENABLE1_LDO_ASSIGN1 0x0C
+#define PALMAS_ENABLE1_LDO_ASSIGN2 0x0D
+#define PALMAS_ENABLE2_RES_ASSIGN 0x0E
+#define PALMAS_ENABLE2_SMPS_ASSIGN 0x0F
#define PALMAS_ENABLE2_LDO_ASSIGN1 0x10
#define PALMAS_ENABLE2_LDO_ASSIGN2 0x11
#define PALMAS_REGEN3_CTRL 0x12
/* Bit definitions for CLK32KG_CTRL */
#define PALMAS_CLK32KG_CTRL_STATUS 0x10
-#define PALMAS_CLK32KG_CTRL_STATUS_SHIFT 4
+#define PALMAS_CLK32KG_CTRL_STATUS_SHIFT 0x04
#define PALMAS_CLK32KG_CTRL_MODE_SLEEP 0x04
-#define PALMAS_CLK32KG_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_CLK32KG_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_CLK32KG_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_CLK32KG_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_CLK32KG_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for CLK32KGAUDIO_CTRL */
#define PALMAS_CLK32KGAUDIO_CTRL_STATUS 0x10
-#define PALMAS_CLK32KGAUDIO_CTRL_STATUS_SHIFT 4
+#define PALMAS_CLK32KGAUDIO_CTRL_STATUS_SHIFT 0x04
#define PALMAS_CLK32KGAUDIO_CTRL_RESERVED3 0x08
-#define PALMAS_CLK32KGAUDIO_CTRL_RESERVED3_SHIFT 3
+#define PALMAS_CLK32KGAUDIO_CTRL_RESERVED3_SHIFT 0x03
#define PALMAS_CLK32KGAUDIO_CTRL_MODE_SLEEP 0x04
-#define PALMAS_CLK32KGAUDIO_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_CLK32KGAUDIO_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_CLK32KGAUDIO_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_CLK32KGAUDIO_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_CLK32KGAUDIO_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for REGEN1_CTRL */
#define PALMAS_REGEN1_CTRL_STATUS 0x10
-#define PALMAS_REGEN1_CTRL_STATUS_SHIFT 4
+#define PALMAS_REGEN1_CTRL_STATUS_SHIFT 0x04
#define PALMAS_REGEN1_CTRL_MODE_SLEEP 0x04
-#define PALMAS_REGEN1_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_REGEN1_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_REGEN1_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_REGEN1_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_REGEN1_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for REGEN2_CTRL */
#define PALMAS_REGEN2_CTRL_STATUS 0x10
-#define PALMAS_REGEN2_CTRL_STATUS_SHIFT 4
+#define PALMAS_REGEN2_CTRL_STATUS_SHIFT 0x04
#define PALMAS_REGEN2_CTRL_MODE_SLEEP 0x04
-#define PALMAS_REGEN2_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_REGEN2_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_REGEN2_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_REGEN2_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_REGEN2_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for SYSEN1_CTRL */
#define PALMAS_SYSEN1_CTRL_STATUS 0x10
-#define PALMAS_SYSEN1_CTRL_STATUS_SHIFT 4
+#define PALMAS_SYSEN1_CTRL_STATUS_SHIFT 0x04
#define PALMAS_SYSEN1_CTRL_MODE_SLEEP 0x04
-#define PALMAS_SYSEN1_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_SYSEN1_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_SYSEN1_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_SYSEN1_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_SYSEN1_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for SYSEN2_CTRL */
#define PALMAS_SYSEN2_CTRL_STATUS 0x10
-#define PALMAS_SYSEN2_CTRL_STATUS_SHIFT 4
+#define PALMAS_SYSEN2_CTRL_STATUS_SHIFT 0x04
#define PALMAS_SYSEN2_CTRL_MODE_SLEEP 0x04
-#define PALMAS_SYSEN2_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_SYSEN2_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_SYSEN2_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_SYSEN2_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_SYSEN2_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Bit definitions for NSLEEP_RES_ASSIGN */
#define PALMAS_NSLEEP_RES_ASSIGN_REGEN3 0x40
-#define PALMAS_NSLEEP_RES_ASSIGN_REGEN3_SHIFT 6
+#define PALMAS_NSLEEP_RES_ASSIGN_REGEN3_SHIFT 0x06
#define PALMAS_NSLEEP_RES_ASSIGN_CLK32KGAUDIO 0x20
-#define PALMAS_NSLEEP_RES_ASSIGN_CLK32KGAUDIO_SHIFT 5
+#define PALMAS_NSLEEP_RES_ASSIGN_CLK32KGAUDIO_SHIFT 0x05
#define PALMAS_NSLEEP_RES_ASSIGN_CLK32KG 0x10
-#define PALMAS_NSLEEP_RES_ASSIGN_CLK32KG_SHIFT 4
+#define PALMAS_NSLEEP_RES_ASSIGN_CLK32KG_SHIFT 0x04
#define PALMAS_NSLEEP_RES_ASSIGN_SYSEN2 0x08
-#define PALMAS_NSLEEP_RES_ASSIGN_SYSEN2_SHIFT 3
+#define PALMAS_NSLEEP_RES_ASSIGN_SYSEN2_SHIFT 0x03
#define PALMAS_NSLEEP_RES_ASSIGN_SYSEN1 0x04
-#define PALMAS_NSLEEP_RES_ASSIGN_SYSEN1_SHIFT 2
+#define PALMAS_NSLEEP_RES_ASSIGN_SYSEN1_SHIFT 0x02
#define PALMAS_NSLEEP_RES_ASSIGN_REGEN2 0x02
-#define PALMAS_NSLEEP_RES_ASSIGN_REGEN2_SHIFT 1
+#define PALMAS_NSLEEP_RES_ASSIGN_REGEN2_SHIFT 0x01
#define PALMAS_NSLEEP_RES_ASSIGN_REGEN1 0x01
-#define PALMAS_NSLEEP_RES_ASSIGN_REGEN1_SHIFT 0
+#define PALMAS_NSLEEP_RES_ASSIGN_REGEN1_SHIFT 0x00
/* Bit definitions for NSLEEP_SMPS_ASSIGN */
#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS10 0x80
-#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS10_SHIFT 7
+#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS10_SHIFT 0x07
#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS9 0x40
-#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS9_SHIFT 6
+#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS9_SHIFT 0x06
#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS8 0x20
-#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS8_SHIFT 5
+#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS8_SHIFT 0x05
#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS7 0x10
-#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS7_SHIFT 4
+#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS7_SHIFT 0x04
#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS6 0x08
-#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS6_SHIFT 3
+#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS6_SHIFT 0x03
#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS45 0x04
-#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS45_SHIFT 2
+#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS45_SHIFT 0x02
#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS3 0x02
-#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS3_SHIFT 1
+#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS3_SHIFT 0x01
#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS12 0x01
-#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS12_SHIFT 0
+#define PALMAS_NSLEEP_SMPS_ASSIGN_SMPS12_SHIFT 0x00
/* Bit definitions for NSLEEP_LDO_ASSIGN1 */
#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO8 0x80
-#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO8_SHIFT 7
+#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO8_SHIFT 0x07
#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO7 0x40
-#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO7_SHIFT 6
+#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO7_SHIFT 0x06
#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO6 0x20
-#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO6_SHIFT 5
+#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO6_SHIFT 0x05
#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO5 0x10
-#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO5_SHIFT 4
+#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO5_SHIFT 0x04
#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO4 0x08
-#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO4_SHIFT 3
+#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO4_SHIFT 0x03
#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO3 0x04
-#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO3_SHIFT 2
+#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO3_SHIFT 0x02
#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO2 0x02
-#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO2_SHIFT 1
+#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO2_SHIFT 0x01
#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO1 0x01
-#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO1_SHIFT 0
+#define PALMAS_NSLEEP_LDO_ASSIGN1_LDO1_SHIFT 0x00
/* Bit definitions for NSLEEP_LDO_ASSIGN2 */
#define PALMAS_NSLEEP_LDO_ASSIGN2_LDOUSB 0x04
-#define PALMAS_NSLEEP_LDO_ASSIGN2_LDOUSB_SHIFT 2
+#define PALMAS_NSLEEP_LDO_ASSIGN2_LDOUSB_SHIFT 0x02
#define PALMAS_NSLEEP_LDO_ASSIGN2_LDOLN 0x02
-#define PALMAS_NSLEEP_LDO_ASSIGN2_LDOLN_SHIFT 1
+#define PALMAS_NSLEEP_LDO_ASSIGN2_LDOLN_SHIFT 0x01
#define PALMAS_NSLEEP_LDO_ASSIGN2_LDO9 0x01
-#define PALMAS_NSLEEP_LDO_ASSIGN2_LDO9_SHIFT 0
+#define PALMAS_NSLEEP_LDO_ASSIGN2_LDO9_SHIFT 0x00
/* Bit definitions for ENABLE1_RES_ASSIGN */
#define PALMAS_ENABLE1_RES_ASSIGN_REGEN3 0x40
-#define PALMAS_ENABLE1_RES_ASSIGN_REGEN3_SHIFT 6
+#define PALMAS_ENABLE1_RES_ASSIGN_REGEN3_SHIFT 0x06
#define PALMAS_ENABLE1_RES_ASSIGN_CLK32KGAUDIO 0x20
-#define PALMAS_ENABLE1_RES_ASSIGN_CLK32KGAUDIO_SHIFT 5
+#define PALMAS_ENABLE1_RES_ASSIGN_CLK32KGAUDIO_SHIFT 0x05
#define PALMAS_ENABLE1_RES_ASSIGN_CLK32KG 0x10
-#define PALMAS_ENABLE1_RES_ASSIGN_CLK32KG_SHIFT 4
+#define PALMAS_ENABLE1_RES_ASSIGN_CLK32KG_SHIFT 0x04
#define PALMAS_ENABLE1_RES_ASSIGN_SYSEN2 0x08
-#define PALMAS_ENABLE1_RES_ASSIGN_SYSEN2_SHIFT 3
+#define PALMAS_ENABLE1_RES_ASSIGN_SYSEN2_SHIFT 0x03
#define PALMAS_ENABLE1_RES_ASSIGN_SYSEN1 0x04
-#define PALMAS_ENABLE1_RES_ASSIGN_SYSEN1_SHIFT 2
+#define PALMAS_ENABLE1_RES_ASSIGN_SYSEN1_SHIFT 0x02
#define PALMAS_ENABLE1_RES_ASSIGN_REGEN2 0x02
-#define PALMAS_ENABLE1_RES_ASSIGN_REGEN2_SHIFT 1
+#define PALMAS_ENABLE1_RES_ASSIGN_REGEN2_SHIFT 0x01
#define PALMAS_ENABLE1_RES_ASSIGN_REGEN1 0x01
-#define PALMAS_ENABLE1_RES_ASSIGN_REGEN1_SHIFT 0
+#define PALMAS_ENABLE1_RES_ASSIGN_REGEN1_SHIFT 0x00
/* Bit definitions for ENABLE1_SMPS_ASSIGN */
#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS10 0x80
-#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS10_SHIFT 7
+#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS10_SHIFT 0x07
#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS9 0x40
-#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS9_SHIFT 6
+#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS9_SHIFT 0x06
#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS8 0x20
-#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS8_SHIFT 5
+#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS8_SHIFT 0x05
#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS7 0x10
-#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS7_SHIFT 4
+#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS7_SHIFT 0x04
#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS6 0x08
-#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS6_SHIFT 3
+#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS6_SHIFT 0x03
#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS45 0x04
-#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS45_SHIFT 2
+#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS45_SHIFT 0x02
#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS3 0x02
-#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS3_SHIFT 1
+#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS3_SHIFT 0x01
#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS12 0x01
-#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS12_SHIFT 0
+#define PALMAS_ENABLE1_SMPS_ASSIGN_SMPS12_SHIFT 0x00
/* Bit definitions for ENABLE1_LDO_ASSIGN1 */
#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO8 0x80
-#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO8_SHIFT 7
+#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO8_SHIFT 0x07
#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO7 0x40
-#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO7_SHIFT 6
+#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO7_SHIFT 0x06
#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO6 0x20
-#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO6_SHIFT 5
+#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO6_SHIFT 0x05
#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO5 0x10
-#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO5_SHIFT 4
+#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO5_SHIFT 0x04
#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO4 0x08
-#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO4_SHIFT 3
+#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO4_SHIFT 0x03
#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO3 0x04
-#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO3_SHIFT 2
+#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO3_SHIFT 0x02
#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO2 0x02
-#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO2_SHIFT 1
+#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO2_SHIFT 0x01
#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO1 0x01
-#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO1_SHIFT 0
+#define PALMAS_ENABLE1_LDO_ASSIGN1_LDO1_SHIFT 0x00
/* Bit definitions for ENABLE1_LDO_ASSIGN2 */
#define PALMAS_ENABLE1_LDO_ASSIGN2_LDOUSB 0x04
-#define PALMAS_ENABLE1_LDO_ASSIGN2_LDOUSB_SHIFT 2
+#define PALMAS_ENABLE1_LDO_ASSIGN2_LDOUSB_SHIFT 0x02
#define PALMAS_ENABLE1_LDO_ASSIGN2_LDOLN 0x02
-#define PALMAS_ENABLE1_LDO_ASSIGN2_LDOLN_SHIFT 1
+#define PALMAS_ENABLE1_LDO_ASSIGN2_LDOLN_SHIFT 0x01
#define PALMAS_ENABLE1_LDO_ASSIGN2_LDO9 0x01
-#define PALMAS_ENABLE1_LDO_ASSIGN2_LDO9_SHIFT 0
+#define PALMAS_ENABLE1_LDO_ASSIGN2_LDO9_SHIFT 0x00
/* Bit definitions for ENABLE2_RES_ASSIGN */
#define PALMAS_ENABLE2_RES_ASSIGN_REGEN3 0x40
-#define PALMAS_ENABLE2_RES_ASSIGN_REGEN3_SHIFT 6
+#define PALMAS_ENABLE2_RES_ASSIGN_REGEN3_SHIFT 0x06
#define PALMAS_ENABLE2_RES_ASSIGN_CLK32KGAUDIO 0x20
-#define PALMAS_ENABLE2_RES_ASSIGN_CLK32KGAUDIO_SHIFT 5
+#define PALMAS_ENABLE2_RES_ASSIGN_CLK32KGAUDIO_SHIFT 0x05
#define PALMAS_ENABLE2_RES_ASSIGN_CLK32KG 0x10
-#define PALMAS_ENABLE2_RES_ASSIGN_CLK32KG_SHIFT 4
+#define PALMAS_ENABLE2_RES_ASSIGN_CLK32KG_SHIFT 0x04
#define PALMAS_ENABLE2_RES_ASSIGN_SYSEN2 0x08
-#define PALMAS_ENABLE2_RES_ASSIGN_SYSEN2_SHIFT 3
+#define PALMAS_ENABLE2_RES_ASSIGN_SYSEN2_SHIFT 0x03
#define PALMAS_ENABLE2_RES_ASSIGN_SYSEN1 0x04
-#define PALMAS_ENABLE2_RES_ASSIGN_SYSEN1_SHIFT 2
+#define PALMAS_ENABLE2_RES_ASSIGN_SYSEN1_SHIFT 0x02
#define PALMAS_ENABLE2_RES_ASSIGN_REGEN2 0x02
-#define PALMAS_ENABLE2_RES_ASSIGN_REGEN2_SHIFT 1
+#define PALMAS_ENABLE2_RES_ASSIGN_REGEN2_SHIFT 0x01
#define PALMAS_ENABLE2_RES_ASSIGN_REGEN1 0x01
-#define PALMAS_ENABLE2_RES_ASSIGN_REGEN1_SHIFT 0
+#define PALMAS_ENABLE2_RES_ASSIGN_REGEN1_SHIFT 0x00
/* Bit definitions for ENABLE2_SMPS_ASSIGN */
#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS10 0x80
-#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS10_SHIFT 7
+#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS10_SHIFT 0x07
#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS9 0x40
-#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS9_SHIFT 6
+#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS9_SHIFT 0x06
#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS8 0x20
-#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS8_SHIFT 5
+#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS8_SHIFT 0x05
#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS7 0x10
-#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS7_SHIFT 4
+#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS7_SHIFT 0x04
#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS6 0x08
-#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS6_SHIFT 3
+#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS6_SHIFT 0x03
#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS45 0x04
-#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS45_SHIFT 2
+#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS45_SHIFT 0x02
#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS3 0x02
-#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS3_SHIFT 1
+#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS3_SHIFT 0x01
#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS12 0x01
-#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS12_SHIFT 0
+#define PALMAS_ENABLE2_SMPS_ASSIGN_SMPS12_SHIFT 0x00
/* Bit definitions for ENABLE2_LDO_ASSIGN1 */
#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO8 0x80
-#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO8_SHIFT 7
+#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO8_SHIFT 0x07
#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO7 0x40
-#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO7_SHIFT 6
+#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO7_SHIFT 0x06
#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO6 0x20
-#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO6_SHIFT 5
+#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO6_SHIFT 0x05
#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO5 0x10
-#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO5_SHIFT 4
+#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO5_SHIFT 0x04
#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO4 0x08
-#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO4_SHIFT 3
+#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO4_SHIFT 0x03
#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO3 0x04
-#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO3_SHIFT 2
+#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO3_SHIFT 0x02
#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO2 0x02
-#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO2_SHIFT 1
+#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO2_SHIFT 0x01
#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO1 0x01
-#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO1_SHIFT 0
+#define PALMAS_ENABLE2_LDO_ASSIGN1_LDO1_SHIFT 0x00
/* Bit definitions for ENABLE2_LDO_ASSIGN2 */
#define PALMAS_ENABLE2_LDO_ASSIGN2_LDOUSB 0x04
-#define PALMAS_ENABLE2_LDO_ASSIGN2_LDOUSB_SHIFT 2
+#define PALMAS_ENABLE2_LDO_ASSIGN2_LDOUSB_SHIFT 0x02
#define PALMAS_ENABLE2_LDO_ASSIGN2_LDOLN 0x02
-#define PALMAS_ENABLE2_LDO_ASSIGN2_LDOLN_SHIFT 1
+#define PALMAS_ENABLE2_LDO_ASSIGN2_LDOLN_SHIFT 0x01
#define PALMAS_ENABLE2_LDO_ASSIGN2_LDO9 0x01
-#define PALMAS_ENABLE2_LDO_ASSIGN2_LDO9_SHIFT 0
+#define PALMAS_ENABLE2_LDO_ASSIGN2_LDO9_SHIFT 0x00
/* Bit definitions for REGEN3_CTRL */
#define PALMAS_REGEN3_CTRL_STATUS 0x10
-#define PALMAS_REGEN3_CTRL_STATUS_SHIFT 4
+#define PALMAS_REGEN3_CTRL_STATUS_SHIFT 0x04
#define PALMAS_REGEN3_CTRL_MODE_SLEEP 0x04
-#define PALMAS_REGEN3_CTRL_MODE_SLEEP_SHIFT 2
+#define PALMAS_REGEN3_CTRL_MODE_SLEEP_SHIFT 0x02
#define PALMAS_REGEN3_CTRL_MODE_ACTIVE 0x01
-#define PALMAS_REGEN3_CTRL_MODE_ACTIVE_SHIFT 0
+#define PALMAS_REGEN3_CTRL_MODE_ACTIVE_SHIFT 0x00
/* Registers for function PAD_CONTROL */
-#define PALMAS_OD_OUTPUT_CTRL2 0x2
-#define PALMAS_POLARITY_CTRL2 0x3
-#define PALMAS_PU_PD_INPUT_CTRL1 0x4
-#define PALMAS_PU_PD_INPUT_CTRL2 0x5
-#define PALMAS_PU_PD_INPUT_CTRL3 0x6
-#define PALMAS_PU_PD_INPUT_CTRL5 0x7
-#define PALMAS_OD_OUTPUT_CTRL 0x8
-#define PALMAS_POLARITY_CTRL 0x9
-#define PALMAS_PRIMARY_SECONDARY_PAD1 0xA
-#define PALMAS_PRIMARY_SECONDARY_PAD2 0xB
-#define PALMAS_I2C_SPI 0xC
-#define PALMAS_PU_PD_INPUT_CTRL4 0xD
-#define PALMAS_PRIMARY_SECONDARY_PAD3 0xE
-#define PALMAS_PRIMARY_SECONDARY_PAD4 0xF
+#define PALMAS_OD_OUTPUT_CTRL2 0x02
+#define PALMAS_POLARITY_CTRL2 0x03
+#define PALMAS_PU_PD_INPUT_CTRL1 0x04
+#define PALMAS_PU_PD_INPUT_CTRL2 0x05
+#define PALMAS_PU_PD_INPUT_CTRL3 0x06
+#define PALMAS_PU_PD_INPUT_CTRL5 0x07
+#define PALMAS_OD_OUTPUT_CTRL 0x08
+#define PALMAS_POLARITY_CTRL 0x09
+#define PALMAS_PRIMARY_SECONDARY_PAD1 0x0A
+#define PALMAS_PRIMARY_SECONDARY_PAD2 0x0B
+#define PALMAS_I2C_SPI 0x0C
+#define PALMAS_PU_PD_INPUT_CTRL4 0x0D
+#define PALMAS_PRIMARY_SECONDARY_PAD3 0x0E
+#define PALMAS_PRIMARY_SECONDARY_PAD4 0x0F
/* Bit definitions for PU_PD_INPUT_CTRL1 */
#define PALMAS_PU_PD_INPUT_CTRL1_RESET_IN_PD 0x40
-#define PALMAS_PU_PD_INPUT_CTRL1_RESET_IN_PD_SHIFT 6
+#define PALMAS_PU_PD_INPUT_CTRL1_RESET_IN_PD_SHIFT 0x06
#define PALMAS_PU_PD_INPUT_CTRL1_GPADC_START_PU 0x20
-#define PALMAS_PU_PD_INPUT_CTRL1_GPADC_START_PU_SHIFT 5
+#define PALMAS_PU_PD_INPUT_CTRL1_GPADC_START_PU_SHIFT 0x05
#define PALMAS_PU_PD_INPUT_CTRL1_GPADC_START_PD 0x10
-#define PALMAS_PU_PD_INPUT_CTRL1_GPADC_START_PD_SHIFT 4
+#define PALMAS_PU_PD_INPUT_CTRL1_GPADC_START_PD_SHIFT 0x04
#define PALMAS_PU_PD_INPUT_CTRL1_PWRDOWN_PD 0x04
-#define PALMAS_PU_PD_INPUT_CTRL1_PWRDOWN_PD_SHIFT 2
+#define PALMAS_PU_PD_INPUT_CTRL1_PWRDOWN_PD_SHIFT 0x02
#define PALMAS_PU_PD_INPUT_CTRL1_NRESWARM_PU 0x02
-#define PALMAS_PU_PD_INPUT_CTRL1_NRESWARM_PU_SHIFT 1
+#define PALMAS_PU_PD_INPUT_CTRL1_NRESWARM_PU_SHIFT 0x01
/* Bit definitions for PU_PD_INPUT_CTRL2 */
#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE2_PU 0x20
-#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE2_PU_SHIFT 5
+#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE2_PU_SHIFT 0x05
#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE2_PD 0x10
-#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE2_PD_SHIFT 4
+#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE2_PD_SHIFT 0x04
#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE1_PU 0x08
-#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE1_PU_SHIFT 3
+#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE1_PU_SHIFT 0x03
#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE1_PD 0x04
-#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE1_PD_SHIFT 2
+#define PALMAS_PU_PD_INPUT_CTRL2_ENABLE1_PD_SHIFT 0x02
#define PALMAS_PU_PD_INPUT_CTRL2_NSLEEP_PU 0x02
-#define PALMAS_PU_PD_INPUT_CTRL2_NSLEEP_PU_SHIFT 1
+#define PALMAS_PU_PD_INPUT_CTRL2_NSLEEP_PU_SHIFT 0x01
#define PALMAS_PU_PD_INPUT_CTRL2_NSLEEP_PD 0x01
-#define PALMAS_PU_PD_INPUT_CTRL2_NSLEEP_PD_SHIFT 0
+#define PALMAS_PU_PD_INPUT_CTRL2_NSLEEP_PD_SHIFT 0x00
/* Bit definitions for PU_PD_INPUT_CTRL3 */
#define PALMAS_PU_PD_INPUT_CTRL3_ACOK_PD 0x40
-#define PALMAS_PU_PD_INPUT_CTRL3_ACOK_PD_SHIFT 6
+#define PALMAS_PU_PD_INPUT_CTRL3_ACOK_PD_SHIFT 0x06
#define PALMAS_PU_PD_INPUT_CTRL3_CHRG_DET_N_PD 0x10
-#define PALMAS_PU_PD_INPUT_CTRL3_CHRG_DET_N_PD_SHIFT 4
+#define PALMAS_PU_PD_INPUT_CTRL3_CHRG_DET_N_PD_SHIFT 0x04
#define PALMAS_PU_PD_INPUT_CTRL3_POWERHOLD_PD 0x04
-#define PALMAS_PU_PD_INPUT_CTRL3_POWERHOLD_PD_SHIFT 2
+#define PALMAS_PU_PD_INPUT_CTRL3_POWERHOLD_PD_SHIFT 0x02
#define PALMAS_PU_PD_INPUT_CTRL3_MSECURE_PD 0x01
-#define PALMAS_PU_PD_INPUT_CTRL3_MSECURE_PD_SHIFT 0
+#define PALMAS_PU_PD_INPUT_CTRL3_MSECURE_PD_SHIFT 0x00
/* Bit definitions for OD_OUTPUT_CTRL */
#define PALMAS_OD_OUTPUT_CTRL_PWM_2_OD 0x80
-#define PALMAS_OD_OUTPUT_CTRL_PWM_2_OD_SHIFT 7
+#define PALMAS_OD_OUTPUT_CTRL_PWM_2_OD_SHIFT 0x07
#define PALMAS_OD_OUTPUT_CTRL_VBUSDET_OD 0x40
-#define PALMAS_OD_OUTPUT_CTRL_VBUSDET_OD_SHIFT 6
+#define PALMAS_OD_OUTPUT_CTRL_VBUSDET_OD_SHIFT 0x06
#define PALMAS_OD_OUTPUT_CTRL_PWM_1_OD 0x20
-#define PALMAS_OD_OUTPUT_CTRL_PWM_1_OD_SHIFT 5
+#define PALMAS_OD_OUTPUT_CTRL_PWM_1_OD_SHIFT 0x05
#define PALMAS_OD_OUTPUT_CTRL_INT_OD 0x08
-#define PALMAS_OD_OUTPUT_CTRL_INT_OD_SHIFT 3
+#define PALMAS_OD_OUTPUT_CTRL_INT_OD_SHIFT 0x03
/* Bit definitions for POLARITY_CTRL */
#define PALMAS_POLARITY_CTRL_INT_POLARITY 0x80
-#define PALMAS_POLARITY_CTRL_INT_POLARITY_SHIFT 7
+#define PALMAS_POLARITY_CTRL_INT_POLARITY_SHIFT 0x07
#define PALMAS_POLARITY_CTRL_ENABLE2_POLARITY 0x40
-#define PALMAS_POLARITY_CTRL_ENABLE2_POLARITY_SHIFT 6
+#define PALMAS_POLARITY_CTRL_ENABLE2_POLARITY_SHIFT 0x06
#define PALMAS_POLARITY_CTRL_ENABLE1_POLARITY 0x20
-#define PALMAS_POLARITY_CTRL_ENABLE1_POLARITY_SHIFT 5
+#define PALMAS_POLARITY_CTRL_ENABLE1_POLARITY_SHIFT 0x05
#define PALMAS_POLARITY_CTRL_NSLEEP_POLARITY 0x10
-#define PALMAS_POLARITY_CTRL_NSLEEP_POLARITY_SHIFT 4
+#define PALMAS_POLARITY_CTRL_NSLEEP_POLARITY_SHIFT 0x04
#define PALMAS_POLARITY_CTRL_RESET_IN_POLARITY 0x08
-#define PALMAS_POLARITY_CTRL_RESET_IN_POLARITY_SHIFT 3
+#define PALMAS_POLARITY_CTRL_RESET_IN_POLARITY_SHIFT 0x03
#define PALMAS_POLARITY_CTRL_GPIO_3_CHRG_DET_N_POLARITY 0x04
-#define PALMAS_POLARITY_CTRL_GPIO_3_CHRG_DET_N_POLARITY_SHIFT 2
+#define PALMAS_POLARITY_CTRL_GPIO_3_CHRG_DET_N_POLARITY_SHIFT 0x02
#define PALMAS_POLARITY_CTRL_POWERGOOD_USB_PSEL_POLARITY 0x02
-#define PALMAS_POLARITY_CTRL_POWERGOOD_USB_PSEL_POLARITY_SHIFT 1
+#define PALMAS_POLARITY_CTRL_POWERGOOD_USB_PSEL_POLARITY_SHIFT 0x01
#define PALMAS_POLARITY_CTRL_PWRDOWN_POLARITY 0x01
-#define PALMAS_POLARITY_CTRL_PWRDOWN_POLARITY_SHIFT 0
+#define PALMAS_POLARITY_CTRL_PWRDOWN_POLARITY_SHIFT 0x00
/* Bit definitions for PRIMARY_SECONDARY_PAD1 */
#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_3 0x80
-#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_3_SHIFT 7
+#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_3_SHIFT 0x07
#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_2_MASK 0x60
-#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_2_SHIFT 5
+#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_2_SHIFT 0x05
#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_1_MASK 0x18
-#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_1_SHIFT 3
+#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_1_SHIFT 0x03
#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_0 0x04
-#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_0_SHIFT 2
+#define PALMAS_PRIMARY_SECONDARY_PAD1_GPIO_0_SHIFT 0x02
#define PALMAS_PRIMARY_SECONDARY_PAD1_VAC 0x02
-#define PALMAS_PRIMARY_SECONDARY_PAD1_VAC_SHIFT 1
+#define PALMAS_PRIMARY_SECONDARY_PAD1_VAC_SHIFT 0x01
#define PALMAS_PRIMARY_SECONDARY_PAD1_POWERGOOD 0x01
-#define PALMAS_PRIMARY_SECONDARY_PAD1_POWERGOOD_SHIFT 0
+#define PALMAS_PRIMARY_SECONDARY_PAD1_POWERGOOD_SHIFT 0x00
/* Bit definitions for PRIMARY_SECONDARY_PAD2 */
#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_7_MASK 0x30
-#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_7_SHIFT 4
+#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_7_SHIFT 0x04
#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_6 0x08
-#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_6_SHIFT 3
+#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_6_SHIFT 0x03
#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_5_MASK 0x06
-#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_5_SHIFT 1
+#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_5_SHIFT 0x01
#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_4 0x01
-#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_4_SHIFT 0
+#define PALMAS_PRIMARY_SECONDARY_PAD2_GPIO_4_SHIFT 0x00
/* Bit definitions for I2C_SPI */
#define PALMAS_I2C_SPI_I2C2OTP_EN 0x80
-#define PALMAS_I2C_SPI_I2C2OTP_EN_SHIFT 7
+#define PALMAS_I2C_SPI_I2C2OTP_EN_SHIFT 0x07
#define PALMAS_I2C_SPI_I2C2OTP_PAGESEL 0x40
-#define PALMAS_I2C_SPI_I2C2OTP_PAGESEL_SHIFT 6
+#define PALMAS_I2C_SPI_I2C2OTP_PAGESEL_SHIFT 0x06
#define PALMAS_I2C_SPI_ID_I2C2 0x20
-#define PALMAS_I2C_SPI_ID_I2C2_SHIFT 5
+#define PALMAS_I2C_SPI_ID_I2C2_SHIFT 0x05
#define PALMAS_I2C_SPI_I2C_SPI 0x10
-#define PALMAS_I2C_SPI_I2C_SPI_SHIFT 4
-#define PALMAS_I2C_SPI_ID_I2C1_MASK 0x0f
-#define PALMAS_I2C_SPI_ID_I2C1_SHIFT 0
+#define PALMAS_I2C_SPI_I2C_SPI_SHIFT 0x04
+#define PALMAS_I2C_SPI_ID_I2C1_MASK 0x0F
+#define PALMAS_I2C_SPI_ID_I2C1_SHIFT 0x00
/* Bit definitions for PU_PD_INPUT_CTRL4 */
#define PALMAS_PU_PD_INPUT_CTRL4_DVFS2_DAT_PD 0x40
-#define PALMAS_PU_PD_INPUT_CTRL4_DVFS2_DAT_PD_SHIFT 6
+#define PALMAS_PU_PD_INPUT_CTRL4_DVFS2_DAT_PD_SHIFT 0x06
#define PALMAS_PU_PD_INPUT_CTRL4_DVFS2_CLK_PD 0x10
-#define PALMAS_PU_PD_INPUT_CTRL4_DVFS2_CLK_PD_SHIFT 4
+#define PALMAS_PU_PD_INPUT_CTRL4_DVFS2_CLK_PD_SHIFT 0x04
#define PALMAS_PU_PD_INPUT_CTRL4_DVFS1_DAT_PD 0x04
-#define PALMAS_PU_PD_INPUT_CTRL4_DVFS1_DAT_PD_SHIFT 2
+#define PALMAS_PU_PD_INPUT_CTRL4_DVFS1_DAT_PD_SHIFT 0x02
#define PALMAS_PU_PD_INPUT_CTRL4_DVFS1_CLK_PD 0x01
-#define PALMAS_PU_PD_INPUT_CTRL4_DVFS1_CLK_PD_SHIFT 0
+#define PALMAS_PU_PD_INPUT_CTRL4_DVFS1_CLK_PD_SHIFT 0x00
/* Bit definitions for PRIMARY_SECONDARY_PAD3 */
#define PALMAS_PRIMARY_SECONDARY_PAD3_DVFS2 0x02
-#define PALMAS_PRIMARY_SECONDARY_PAD3_DVFS2_SHIFT 1
+#define PALMAS_PRIMARY_SECONDARY_PAD3_DVFS2_SHIFT 0x01
#define PALMAS_PRIMARY_SECONDARY_PAD3_DVFS1 0x01
-#define PALMAS_PRIMARY_SECONDARY_PAD3_DVFS1_SHIFT 0
+#define PALMAS_PRIMARY_SECONDARY_PAD3_DVFS1_SHIFT 0x00
/* Registers for function LED_PWM */
-#define PALMAS_LED_PERIOD_CTRL 0x0
-#define PALMAS_LED_CTRL 0x1
-#define PALMAS_PWM_CTRL1 0x2
-#define PALMAS_PWM_CTRL2 0x3
+#define PALMAS_LED_PERIOD_CTRL 0x00
+#define PALMAS_LED_CTRL 0x01
+#define PALMAS_PWM_CTRL1 0x02
+#define PALMAS_PWM_CTRL2 0x03
/* Bit definitions for LED_PERIOD_CTRL */
#define PALMAS_LED_PERIOD_CTRL_LED_2_PERIOD_MASK 0x38
-#define PALMAS_LED_PERIOD_CTRL_LED_2_PERIOD_SHIFT 3
+#define PALMAS_LED_PERIOD_CTRL_LED_2_PERIOD_SHIFT 0x03
#define PALMAS_LED_PERIOD_CTRL_LED_1_PERIOD_MASK 0x07
-#define PALMAS_LED_PERIOD_CTRL_LED_1_PERIOD_SHIFT 0
+#define PALMAS_LED_PERIOD_CTRL_LED_1_PERIOD_SHIFT 0x00
/* Bit definitions for LED_CTRL */
#define PALMAS_LED_CTRL_LED_2_SEQ 0x20
-#define PALMAS_LED_CTRL_LED_2_SEQ_SHIFT 5
+#define PALMAS_LED_CTRL_LED_2_SEQ_SHIFT 0x05
#define PALMAS_LED_CTRL_LED_1_SEQ 0x10
-#define PALMAS_LED_CTRL_LED_1_SEQ_SHIFT 4
+#define PALMAS_LED_CTRL_LED_1_SEQ_SHIFT 0x04
#define PALMAS_LED_CTRL_LED_2_ON_TIME_MASK 0x0c
-#define PALMAS_LED_CTRL_LED_2_ON_TIME_SHIFT 2
+#define PALMAS_LED_CTRL_LED_2_ON_TIME_SHIFT 0x02
#define PALMAS_LED_CTRL_LED_1_ON_TIME_MASK 0x03
-#define PALMAS_LED_CTRL_LED_1_ON_TIME_SHIFT 0
+#define PALMAS_LED_CTRL_LED_1_ON_TIME_SHIFT 0x00
/* Bit definitions for PWM_CTRL1 */
#define PALMAS_PWM_CTRL1_PWM_FREQ_EN 0x02
-#define PALMAS_PWM_CTRL1_PWM_FREQ_EN_SHIFT 1
+#define PALMAS_PWM_CTRL1_PWM_FREQ_EN_SHIFT 0x01
#define PALMAS_PWM_CTRL1_PWM_FREQ_SEL 0x01
-#define PALMAS_PWM_CTRL1_PWM_FREQ_SEL_SHIFT 0
+#define PALMAS_PWM_CTRL1_PWM_FREQ_SEL_SHIFT 0x00
/* Bit definitions for PWM_CTRL2 */
-#define PALMAS_PWM_CTRL2_PWM_DUTY_SEL_MASK 0xff
-#define PALMAS_PWM_CTRL2_PWM_DUTY_SEL_SHIFT 0
+#define PALMAS_PWM_CTRL2_PWM_DUTY_SEL_MASK 0xFF
+#define PALMAS_PWM_CTRL2_PWM_DUTY_SEL_SHIFT 0x00
/* Registers for function INTERRUPT */
-#define PALMAS_INT1_STATUS 0x0
-#define PALMAS_INT1_MASK 0x1
-#define PALMAS_INT1_LINE_STATE 0x2
-#define PALMAS_INT1_EDGE_DETECT1_RESERVED 0x3
-#define PALMAS_INT1_EDGE_DETECT2_RESERVED 0x4
-#define PALMAS_INT2_STATUS 0x5
-#define PALMAS_INT2_MASK 0x6
-#define PALMAS_INT2_LINE_STATE 0x7
-#define PALMAS_INT2_EDGE_DETECT1_RESERVED 0x8
-#define PALMAS_INT2_EDGE_DETECT2_RESERVED 0x9
-#define PALMAS_INT3_STATUS 0xA
-#define PALMAS_INT3_MASK 0xB
-#define PALMAS_INT3_LINE_STATE 0xC
-#define PALMAS_INT3_EDGE_DETECT1_RESERVED 0xD
-#define PALMAS_INT3_EDGE_DETECT2_RESERVED 0xE
-#define PALMAS_INT4_STATUS 0xF
+#define PALMAS_INT1_STATUS 0x00
+#define PALMAS_INT1_MASK 0x01
+#define PALMAS_INT1_LINE_STATE 0x02
+#define PALMAS_INT1_EDGE_DETECT1_RESERVED 0x03
+#define PALMAS_INT1_EDGE_DETECT2_RESERVED 0x04
+#define PALMAS_INT2_STATUS 0x05
+#define PALMAS_INT2_MASK 0x06
+#define PALMAS_INT2_LINE_STATE 0x07
+#define PALMAS_INT2_EDGE_DETECT1_RESERVED 0x08
+#define PALMAS_INT2_EDGE_DETECT2_RESERVED 0x09
+#define PALMAS_INT3_STATUS 0x0A
+#define PALMAS_INT3_MASK 0x0B
+#define PALMAS_INT3_LINE_STATE 0x0C
+#define PALMAS_INT3_EDGE_DETECT1_RESERVED 0x0D
+#define PALMAS_INT3_EDGE_DETECT2_RESERVED 0x0E
+#define PALMAS_INT4_STATUS 0x0F
#define PALMAS_INT4_MASK 0x10
#define PALMAS_INT4_LINE_STATE 0x11
#define PALMAS_INT4_EDGE_DETECT1 0x12
@@ -1966,276 +1966,276 @@
/* Bit definitions for INT1_STATUS */
#define PALMAS_INT1_STATUS_VBAT_MON 0x80
-#define PALMAS_INT1_STATUS_VBAT_MON_SHIFT 7
+#define PALMAS_INT1_STATUS_VBAT_MON_SHIFT 0x07
#define PALMAS_INT1_STATUS_VSYS_MON 0x40
-#define PALMAS_INT1_STATUS_VSYS_MON_SHIFT 6
+#define PALMAS_INT1_STATUS_VSYS_MON_SHIFT 0x06
#define PALMAS_INT1_STATUS_HOTDIE 0x20
-#define PALMAS_INT1_STATUS_HOTDIE_SHIFT 5
+#define PALMAS_INT1_STATUS_HOTDIE_SHIFT 0x05
#define PALMAS_INT1_STATUS_PWRDOWN 0x10
-#define PALMAS_INT1_STATUS_PWRDOWN_SHIFT 4
+#define PALMAS_INT1_STATUS_PWRDOWN_SHIFT 0x04
#define PALMAS_INT1_STATUS_RPWRON 0x08
-#define PALMAS_INT1_STATUS_RPWRON_SHIFT 3
+#define PALMAS_INT1_STATUS_RPWRON_SHIFT 0x03
#define PALMAS_INT1_STATUS_LONG_PRESS_KEY 0x04
-#define PALMAS_INT1_STATUS_LONG_PRESS_KEY_SHIFT 2
+#define PALMAS_INT1_STATUS_LONG_PRESS_KEY_SHIFT 0x02
#define PALMAS_INT1_STATUS_PWRON 0x02
-#define PALMAS_INT1_STATUS_PWRON_SHIFT 1
+#define PALMAS_INT1_STATUS_PWRON_SHIFT 0x01
#define PALMAS_INT1_STATUS_CHARG_DET_N_VBUS_OVV 0x01
-#define PALMAS_INT1_STATUS_CHARG_DET_N_VBUS_OVV_SHIFT 0
+#define PALMAS_INT1_STATUS_CHARG_DET_N_VBUS_OVV_SHIFT 0x00
/* Bit definitions for INT1_MASK */
#define PALMAS_INT1_MASK_VBAT_MON 0x80
-#define PALMAS_INT1_MASK_VBAT_MON_SHIFT 7
+#define PALMAS_INT1_MASK_VBAT_MON_SHIFT 0x07
#define PALMAS_INT1_MASK_VSYS_MON 0x40
-#define PALMAS_INT1_MASK_VSYS_MON_SHIFT 6
+#define PALMAS_INT1_MASK_VSYS_MON_SHIFT 0x06
#define PALMAS_INT1_MASK_HOTDIE 0x20
-#define PALMAS_INT1_MASK_HOTDIE_SHIFT 5
+#define PALMAS_INT1_MASK_HOTDIE_SHIFT 0x05
#define PALMAS_INT1_MASK_PWRDOWN 0x10
-#define PALMAS_INT1_MASK_PWRDOWN_SHIFT 4
+#define PALMAS_INT1_MASK_PWRDOWN_SHIFT 0x04
#define PALMAS_INT1_MASK_RPWRON 0x08
-#define PALMAS_INT1_MASK_RPWRON_SHIFT 3
+#define PALMAS_INT1_MASK_RPWRON_SHIFT 0x03
#define PALMAS_INT1_MASK_LONG_PRESS_KEY 0x04
-#define PALMAS_INT1_MASK_LONG_PRESS_KEY_SHIFT 2
+#define PALMAS_INT1_MASK_LONG_PRESS_KEY_SHIFT 0x02
#define PALMAS_INT1_MASK_PWRON 0x02
-#define PALMAS_INT1_MASK_PWRON_SHIFT 1
+#define PALMAS_INT1_MASK_PWRON_SHIFT 0x01
#define PALMAS_INT1_MASK_CHARG_DET_N_VBUS_OVV 0x01
-#define PALMAS_INT1_MASK_CHARG_DET_N_VBUS_OVV_SHIFT 0
+#define PALMAS_INT1_MASK_CHARG_DET_N_VBUS_OVV_SHIFT 0x00
/* Bit definitions for INT1_LINE_STATE */
#define PALMAS_INT1_LINE_STATE_VBAT_MON 0x80
-#define PALMAS_INT1_LINE_STATE_VBAT_MON_SHIFT 7
+#define PALMAS_INT1_LINE_STATE_VBAT_MON_SHIFT 0x07
#define PALMAS_INT1_LINE_STATE_VSYS_MON 0x40
-#define PALMAS_INT1_LINE_STATE_VSYS_MON_SHIFT 6
+#define PALMAS_INT1_LINE_STATE_VSYS_MON_SHIFT 0x06
#define PALMAS_INT1_LINE_STATE_HOTDIE 0x20
-#define PALMAS_INT1_LINE_STATE_HOTDIE_SHIFT 5
+#define PALMAS_INT1_LINE_STATE_HOTDIE_SHIFT 0x05
#define PALMAS_INT1_LINE_STATE_PWRDOWN 0x10
-#define PALMAS_INT1_LINE_STATE_PWRDOWN_SHIFT 4
+#define PALMAS_INT1_LINE_STATE_PWRDOWN_SHIFT 0x04
#define PALMAS_INT1_LINE_STATE_RPWRON 0x08
-#define PALMAS_INT1_LINE_STATE_RPWRON_SHIFT 3
+#define PALMAS_INT1_LINE_STATE_RPWRON_SHIFT 0x03
#define PALMAS_INT1_LINE_STATE_LONG_PRESS_KEY 0x04
-#define PALMAS_INT1_LINE_STATE_LONG_PRESS_KEY_SHIFT 2
+#define PALMAS_INT1_LINE_STATE_LONG_PRESS_KEY_SHIFT 0x02
#define PALMAS_INT1_LINE_STATE_PWRON 0x02
-#define PALMAS_INT1_LINE_STATE_PWRON_SHIFT 1
+#define PALMAS_INT1_LINE_STATE_PWRON_SHIFT 0x01
#define PALMAS_INT1_LINE_STATE_CHARG_DET_N_VBUS_OVV 0x01
-#define PALMAS_INT1_LINE_STATE_CHARG_DET_N_VBUS_OVV_SHIFT 0
+#define PALMAS_INT1_LINE_STATE_CHARG_DET_N_VBUS_OVV_SHIFT 0x00
/* Bit definitions for INT2_STATUS */
#define PALMAS_INT2_STATUS_VAC_ACOK 0x80
-#define PALMAS_INT2_STATUS_VAC_ACOK_SHIFT 7
+#define PALMAS_INT2_STATUS_VAC_ACOK_SHIFT 0x07
#define PALMAS_INT2_STATUS_SHORT 0x40
-#define PALMAS_INT2_STATUS_SHORT_SHIFT 6
+#define PALMAS_INT2_STATUS_SHORT_SHIFT 0x06
#define PALMAS_INT2_STATUS_FBI_BB 0x20
-#define PALMAS_INT2_STATUS_FBI_BB_SHIFT 5
+#define PALMAS_INT2_STATUS_FBI_BB_SHIFT 0x05
#define PALMAS_INT2_STATUS_RESET_IN 0x10
-#define PALMAS_INT2_STATUS_RESET_IN_SHIFT 4
+#define PALMAS_INT2_STATUS_RESET_IN_SHIFT 0x04
#define PALMAS_INT2_STATUS_BATREMOVAL 0x08
-#define PALMAS_INT2_STATUS_BATREMOVAL_SHIFT 3
+#define PALMAS_INT2_STATUS_BATREMOVAL_SHIFT 0x03
#define PALMAS_INT2_STATUS_WDT 0x04
-#define PALMAS_INT2_STATUS_WDT_SHIFT 2
+#define PALMAS_INT2_STATUS_WDT_SHIFT 0x02
#define PALMAS_INT2_STATUS_RTC_TIMER 0x02
-#define PALMAS_INT2_STATUS_RTC_TIMER_SHIFT 1
+#define PALMAS_INT2_STATUS_RTC_TIMER_SHIFT 0x01
#define PALMAS_INT2_STATUS_RTC_ALARM 0x01
-#define PALMAS_INT2_STATUS_RTC_ALARM_SHIFT 0
+#define PALMAS_INT2_STATUS_RTC_ALARM_SHIFT 0x00
/* Bit definitions for INT2_MASK */
#define PALMAS_INT2_MASK_VAC_ACOK 0x80
-#define PALMAS_INT2_MASK_VAC_ACOK_SHIFT 7
+#define PALMAS_INT2_MASK_VAC_ACOK_SHIFT 0x07
#define PALMAS_INT2_MASK_SHORT 0x40
-#define PALMAS_INT2_MASK_SHORT_SHIFT 6
+#define PALMAS_INT2_MASK_SHORT_SHIFT 0x06
#define PALMAS_INT2_MASK_FBI_BB 0x20
-#define PALMAS_INT2_MASK_FBI_BB_SHIFT 5
+#define PALMAS_INT2_MASK_FBI_BB_SHIFT 0x05
#define PALMAS_INT2_MASK_RESET_IN 0x10
-#define PALMAS_INT2_MASK_RESET_IN_SHIFT 4
+#define PALMAS_INT2_MASK_RESET_IN_SHIFT 0x04
#define PALMAS_INT2_MASK_BATREMOVAL 0x08
-#define PALMAS_INT2_MASK_BATREMOVAL_SHIFT 3
+#define PALMAS_INT2_MASK_BATREMOVAL_SHIFT 0x03
#define PALMAS_INT2_MASK_WDT 0x04
-#define PALMAS_INT2_MASK_WDT_SHIFT 2
+#define PALMAS_INT2_MASK_WDT_SHIFT 0x02
#define PALMAS_INT2_MASK_RTC_TIMER 0x02
-#define PALMAS_INT2_MASK_RTC_TIMER_SHIFT 1
+#define PALMAS_INT2_MASK_RTC_TIMER_SHIFT 0x01
#define PALMAS_INT2_MASK_RTC_ALARM 0x01
-#define PALMAS_INT2_MASK_RTC_ALARM_SHIFT 0
+#define PALMAS_INT2_MASK_RTC_ALARM_SHIFT 0x00
/* Bit definitions for INT2_LINE_STATE */
#define PALMAS_INT2_LINE_STATE_VAC_ACOK 0x80
-#define PALMAS_INT2_LINE_STATE_VAC_ACOK_SHIFT 7
+#define PALMAS_INT2_LINE_STATE_VAC_ACOK_SHIFT 0x07
#define PALMAS_INT2_LINE_STATE_SHORT 0x40
-#define PALMAS_INT2_LINE_STATE_SHORT_SHIFT 6
+#define PALMAS_INT2_LINE_STATE_SHORT_SHIFT 0x06
#define PALMAS_INT2_LINE_STATE_FBI_BB 0x20
-#define PALMAS_INT2_LINE_STATE_FBI_BB_SHIFT 5
+#define PALMAS_INT2_LINE_STATE_FBI_BB_SHIFT 0x05
#define PALMAS_INT2_LINE_STATE_RESET_IN 0x10
-#define PALMAS_INT2_LINE_STATE_RESET_IN_SHIFT 4
+#define PALMAS_INT2_LINE_STATE_RESET_IN_SHIFT 0x04
#define PALMAS_INT2_LINE_STATE_BATREMOVAL 0x08
-#define PALMAS_INT2_LINE_STATE_BATREMOVAL_SHIFT 3
+#define PALMAS_INT2_LINE_STATE_BATREMOVAL_SHIFT 0x03
#define PALMAS_INT2_LINE_STATE_WDT 0x04
-#define PALMAS_INT2_LINE_STATE_WDT_SHIFT 2
+#define PALMAS_INT2_LINE_STATE_WDT_SHIFT 0x02
#define PALMAS_INT2_LINE_STATE_RTC_TIMER 0x02
-#define PALMAS_INT2_LINE_STATE_RTC_TIMER_SHIFT 1
+#define PALMAS_INT2_LINE_STATE_RTC_TIMER_SHIFT 0x01
#define PALMAS_INT2_LINE_STATE_RTC_ALARM 0x01
-#define PALMAS_INT2_LINE_STATE_RTC_ALARM_SHIFT 0
+#define PALMAS_INT2_LINE_STATE_RTC_ALARM_SHIFT 0x00
/* Bit definitions for INT3_STATUS */
#define PALMAS_INT3_STATUS_VBUS 0x80
-#define PALMAS_INT3_STATUS_VBUS_SHIFT 7
+#define PALMAS_INT3_STATUS_VBUS_SHIFT 0x07
#define PALMAS_INT3_STATUS_VBUS_OTG 0x40
-#define PALMAS_INT3_STATUS_VBUS_OTG_SHIFT 6
+#define PALMAS_INT3_STATUS_VBUS_OTG_SHIFT 0x06
#define PALMAS_INT3_STATUS_ID 0x20
-#define PALMAS_INT3_STATUS_ID_SHIFT 5
+#define PALMAS_INT3_STATUS_ID_SHIFT 0x05
#define PALMAS_INT3_STATUS_ID_OTG 0x10
-#define PALMAS_INT3_STATUS_ID_OTG_SHIFT 4
+#define PALMAS_INT3_STATUS_ID_OTG_SHIFT 0x04
#define PALMAS_INT3_STATUS_GPADC_EOC_RT 0x08
-#define PALMAS_INT3_STATUS_GPADC_EOC_RT_SHIFT 3
+#define PALMAS_INT3_STATUS_GPADC_EOC_RT_SHIFT 0x03
#define PALMAS_INT3_STATUS_GPADC_EOC_SW 0x04
-#define PALMAS_INT3_STATUS_GPADC_EOC_SW_SHIFT 2
+#define PALMAS_INT3_STATUS_GPADC_EOC_SW_SHIFT 0x02
#define PALMAS_INT3_STATUS_GPADC_AUTO_1 0x02
-#define PALMAS_INT3_STATUS_GPADC_AUTO_1_SHIFT 1
+#define PALMAS_INT3_STATUS_GPADC_AUTO_1_SHIFT 0x01
#define PALMAS_INT3_STATUS_GPADC_AUTO_0 0x01
-#define PALMAS_INT3_STATUS_GPADC_AUTO_0_SHIFT 0
+#define PALMAS_INT3_STATUS_GPADC_AUTO_0_SHIFT 0x00
/* Bit definitions for INT3_MASK */
#define PALMAS_INT3_MASK_VBUS 0x80
-#define PALMAS_INT3_MASK_VBUS_SHIFT 7
+#define PALMAS_INT3_MASK_VBUS_SHIFT 0x07
#define PALMAS_INT3_MASK_VBUS_OTG 0x40
-#define PALMAS_INT3_MASK_VBUS_OTG_SHIFT 6
+#define PALMAS_INT3_MASK_VBUS_OTG_SHIFT 0x06
#define PALMAS_INT3_MASK_ID 0x20
-#define PALMAS_INT3_MASK_ID_SHIFT 5
+#define PALMAS_INT3_MASK_ID_SHIFT 0x05
#define PALMAS_INT3_MASK_ID_OTG 0x10
-#define PALMAS_INT3_MASK_ID_OTG_SHIFT 4
+#define PALMAS_INT3_MASK_ID_OTG_SHIFT 0x04
#define PALMAS_INT3_MASK_GPADC_EOC_RT 0x08
-#define PALMAS_INT3_MASK_GPADC_EOC_RT_SHIFT 3
+#define PALMAS_INT3_MASK_GPADC_EOC_RT_SHIFT 0x03
#define PALMAS_INT3_MASK_GPADC_EOC_SW 0x04
-#define PALMAS_INT3_MASK_GPADC_EOC_SW_SHIFT 2
+#define PALMAS_INT3_MASK_GPADC_EOC_SW_SHIFT 0x02
#define PALMAS_INT3_MASK_GPADC_AUTO_1 0x02
-#define PALMAS_INT3_MASK_GPADC_AUTO_1_SHIFT 1
+#define PALMAS_INT3_MASK_GPADC_AUTO_1_SHIFT 0x01
#define PALMAS_INT3_MASK_GPADC_AUTO_0 0x01
-#define PALMAS_INT3_MASK_GPADC_AUTO_0_SHIFT 0
+#define PALMAS_INT3_MASK_GPADC_AUTO_0_SHIFT 0x00
/* Bit definitions for INT3_LINE_STATE */
#define PALMAS_INT3_LINE_STATE_VBUS 0x80
-#define PALMAS_INT3_LINE_STATE_VBUS_SHIFT 7
+#define PALMAS_INT3_LINE_STATE_VBUS_SHIFT 0x07
#define PALMAS_INT3_LINE_STATE_VBUS_OTG 0x40
-#define PALMAS_INT3_LINE_STATE_VBUS_OTG_SHIFT 6
+#define PALMAS_INT3_LINE_STATE_VBUS_OTG_SHIFT 0x06
#define PALMAS_INT3_LINE_STATE_ID 0x20
-#define PALMAS_INT3_LINE_STATE_ID_SHIFT 5
+#define PALMAS_INT3_LINE_STATE_ID_SHIFT 0x05
#define PALMAS_INT3_LINE_STATE_ID_OTG 0x10
-#define PALMAS_INT3_LINE_STATE_ID_OTG_SHIFT 4
+#define PALMAS_INT3_LINE_STATE_ID_OTG_SHIFT 0x04
#define PALMAS_INT3_LINE_STATE_GPADC_EOC_RT 0x08
-#define PALMAS_INT3_LINE_STATE_GPADC_EOC_RT_SHIFT 3
+#define PALMAS_INT3_LINE_STATE_GPADC_EOC_RT_SHIFT 0x03
#define PALMAS_INT3_LINE_STATE_GPADC_EOC_SW 0x04
-#define PALMAS_INT3_LINE_STATE_GPADC_EOC_SW_SHIFT 2
+#define PALMAS_INT3_LINE_STATE_GPADC_EOC_SW_SHIFT 0x02
#define PALMAS_INT3_LINE_STATE_GPADC_AUTO_1 0x02
-#define PALMAS_INT3_LINE_STATE_GPADC_AUTO_1_SHIFT 1
+#define PALMAS_INT3_LINE_STATE_GPADC_AUTO_1_SHIFT 0x01
#define PALMAS_INT3_LINE_STATE_GPADC_AUTO_0 0x01
-#define PALMAS_INT3_LINE_STATE_GPADC_AUTO_0_SHIFT 0
+#define PALMAS_INT3_LINE_STATE_GPADC_AUTO_0_SHIFT 0x00
/* Bit definitions for INT4_STATUS */
#define PALMAS_INT4_STATUS_GPIO_7 0x80
-#define PALMAS_INT4_STATUS_GPIO_7_SHIFT 7
+#define PALMAS_INT4_STATUS_GPIO_7_SHIFT 0x07
#define PALMAS_INT4_STATUS_GPIO_6 0x40
-#define PALMAS_INT4_STATUS_GPIO_6_SHIFT 6
+#define PALMAS_INT4_STATUS_GPIO_6_SHIFT 0x06
#define PALMAS_INT4_STATUS_GPIO_5 0x20
-#define PALMAS_INT4_STATUS_GPIO_5_SHIFT 5
+#define PALMAS_INT4_STATUS_GPIO_5_SHIFT 0x05
#define PALMAS_INT4_STATUS_GPIO_4 0x10
-#define PALMAS_INT4_STATUS_GPIO_4_SHIFT 4
+#define PALMAS_INT4_STATUS_GPIO_4_SHIFT 0x04
#define PALMAS_INT4_STATUS_GPIO_3 0x08
-#define PALMAS_INT4_STATUS_GPIO_3_SHIFT 3
+#define PALMAS_INT4_STATUS_GPIO_3_SHIFT 0x03
#define PALMAS_INT4_STATUS_GPIO_2 0x04
-#define PALMAS_INT4_STATUS_GPIO_2_SHIFT 2
+#define PALMAS_INT4_STATUS_GPIO_2_SHIFT 0x02
#define PALMAS_INT4_STATUS_GPIO_1 0x02
-#define PALMAS_INT4_STATUS_GPIO_1_SHIFT 1
+#define PALMAS_INT4_STATUS_GPIO_1_SHIFT 0x01
#define PALMAS_INT4_STATUS_GPIO_0 0x01
-#define PALMAS_INT4_STATUS_GPIO_0_SHIFT 0
+#define PALMAS_INT4_STATUS_GPIO_0_SHIFT 0x00
/* Bit definitions for INT4_MASK */
#define PALMAS_INT4_MASK_GPIO_7 0x80
-#define PALMAS_INT4_MASK_GPIO_7_SHIFT 7
+#define PALMAS_INT4_MASK_GPIO_7_SHIFT 0x07
#define PALMAS_INT4_MASK_GPIO_6 0x40
-#define PALMAS_INT4_MASK_GPIO_6_SHIFT 6
+#define PALMAS_INT4_MASK_GPIO_6_SHIFT 0x06
#define PALMAS_INT4_MASK_GPIO_5 0x20
-#define PALMAS_INT4_MASK_GPIO_5_SHIFT 5
+#define PALMAS_INT4_MASK_GPIO_5_SHIFT 0x05
#define PALMAS_INT4_MASK_GPIO_4 0x10
-#define PALMAS_INT4_MASK_GPIO_4_SHIFT 4
+#define PALMAS_INT4_MASK_GPIO_4_SHIFT 0x04
#define PALMAS_INT4_MASK_GPIO_3 0x08
-#define PALMAS_INT4_MASK_GPIO_3_SHIFT 3
+#define PALMAS_INT4_MASK_GPIO_3_SHIFT 0x03
#define PALMAS_INT4_MASK_GPIO_2 0x04
-#define PALMAS_INT4_MASK_GPIO_2_SHIFT 2
+#define PALMAS_INT4_MASK_GPIO_2_SHIFT 0x02
#define PALMAS_INT4_MASK_GPIO_1 0x02
-#define PALMAS_INT4_MASK_GPIO_1_SHIFT 1
+#define PALMAS_INT4_MASK_GPIO_1_SHIFT 0x01
#define PALMAS_INT4_MASK_GPIO_0 0x01
-#define PALMAS_INT4_MASK_GPIO_0_SHIFT 0
+#define PALMAS_INT4_MASK_GPIO_0_SHIFT 0x00
/* Bit definitions for INT4_LINE_STATE */
#define PALMAS_INT4_LINE_STATE_GPIO_7 0x80
-#define PALMAS_INT4_LINE_STATE_GPIO_7_SHIFT 7
+#define PALMAS_INT4_LINE_STATE_GPIO_7_SHIFT 0x07
#define PALMAS_INT4_LINE_STATE_GPIO_6 0x40
-#define PALMAS_INT4_LINE_STATE_GPIO_6_SHIFT 6
+#define PALMAS_INT4_LINE_STATE_GPIO_6_SHIFT 0x06
#define PALMAS_INT4_LINE_STATE_GPIO_5 0x20
-#define PALMAS_INT4_LINE_STATE_GPIO_5_SHIFT 5
+#define PALMAS_INT4_LINE_STATE_GPIO_5_SHIFT 0x05
#define PALMAS_INT4_LINE_STATE_GPIO_4 0x10
-#define PALMAS_INT4_LINE_STATE_GPIO_4_SHIFT 4
+#define PALMAS_INT4_LINE_STATE_GPIO_4_SHIFT 0x04
#define PALMAS_INT4_LINE_STATE_GPIO_3 0x08
-#define PALMAS_INT4_LINE_STATE_GPIO_3_SHIFT 3
+#define PALMAS_INT4_LINE_STATE_GPIO_3_SHIFT 0x03
#define PALMAS_INT4_LINE_STATE_GPIO_2 0x04
-#define PALMAS_INT4_LINE_STATE_GPIO_2_SHIFT 2
+#define PALMAS_INT4_LINE_STATE_GPIO_2_SHIFT 0x02
#define PALMAS_INT4_LINE_STATE_GPIO_1 0x02
-#define PALMAS_INT4_LINE_STATE_GPIO_1_SHIFT 1
+#define PALMAS_INT4_LINE_STATE_GPIO_1_SHIFT 0x01
#define PALMAS_INT4_LINE_STATE_GPIO_0 0x01
-#define PALMAS_INT4_LINE_STATE_GPIO_0_SHIFT 0
+#define PALMAS_INT4_LINE_STATE_GPIO_0_SHIFT 0x00
/* Bit definitions for INT4_EDGE_DETECT1 */
#define PALMAS_INT4_EDGE_DETECT1_GPIO_3_RISING 0x80
-#define PALMAS_INT4_EDGE_DETECT1_GPIO_3_RISING_SHIFT 7
+#define PALMAS_INT4_EDGE_DETECT1_GPIO_3_RISING_SHIFT 0x07
#define PALMAS_INT4_EDGE_DETECT1_GPIO_3_FALLING 0x40
-#define PALMAS_INT4_EDGE_DETECT1_GPIO_3_FALLING_SHIFT 6
+#define PALMAS_INT4_EDGE_DETECT1_GPIO_3_FALLING_SHIFT 0x06
#define PALMAS_INT4_EDGE_DETECT1_GPIO_2_RISING 0x20
-#define PALMAS_INT4_EDGE_DETECT1_GPIO_2_RISING_SHIFT 5
+#define PALMAS_INT4_EDGE_DETECT1_GPIO_2_RISING_SHIFT 0x05
#define PALMAS_INT4_EDGE_DETECT1_GPIO_2_FALLING 0x10
-#define PALMAS_INT4_EDGE_DETECT1_GPIO_2_FALLING_SHIFT 4
+#define PALMAS_INT4_EDGE_DETECT1_GPIO_2_FALLING_SHIFT 0x04
#define PALMAS_INT4_EDGE_DETECT1_GPIO_1_RISING 0x08
-#define PALMAS_INT4_EDGE_DETECT1_GPIO_1_RISING_SHIFT 3
+#define PALMAS_INT4_EDGE_DETECT1_GPIO_1_RISING_SHIFT 0x03
#define PALMAS_INT4_EDGE_DETECT1_GPIO_1_FALLING 0x04
-#define PALMAS_INT4_EDGE_DETECT1_GPIO_1_FALLING_SHIFT 2
+#define PALMAS_INT4_EDGE_DETECT1_GPIO_1_FALLING_SHIFT 0x02
#define PALMAS_INT4_EDGE_DETECT1_GPIO_0_RISING 0x02
-#define PALMAS_INT4_EDGE_DETECT1_GPIO_0_RISING_SHIFT 1
+#define PALMAS_INT4_EDGE_DETECT1_GPIO_0_RISING_SHIFT 0x01
#define PALMAS_INT4_EDGE_DETECT1_GPIO_0_FALLING 0x01
-#define PALMAS_INT4_EDGE_DETECT1_GPIO_0_FALLING_SHIFT 0
+#define PALMAS_INT4_EDGE_DETECT1_GPIO_0_FALLING_SHIFT 0x00
/* Bit definitions for INT4_EDGE_DETECT2 */
#define PALMAS_INT4_EDGE_DETECT2_GPIO_7_RISING 0x80
-#define PALMAS_INT4_EDGE_DETECT2_GPIO_7_RISING_SHIFT 7
+#define PALMAS_INT4_EDGE_DETECT2_GPIO_7_RISING_SHIFT 0x07
#define PALMAS_INT4_EDGE_DETECT2_GPIO_7_FALLING 0x40
-#define PALMAS_INT4_EDGE_DETECT2_GPIO_7_FALLING_SHIFT 6
+#define PALMAS_INT4_EDGE_DETECT2_GPIO_7_FALLING_SHIFT 0x06
#define PALMAS_INT4_EDGE_DETECT2_GPIO_6_RISING 0x20
-#define PALMAS_INT4_EDGE_DETECT2_GPIO_6_RISING_SHIFT 5
+#define PALMAS_INT4_EDGE_DETECT2_GPIO_6_RISING_SHIFT 0x05
#define PALMAS_INT4_EDGE_DETECT2_GPIO_6_FALLING 0x10
-#define PALMAS_INT4_EDGE_DETECT2_GPIO_6_FALLING_SHIFT 4
+#define PALMAS_INT4_EDGE_DETECT2_GPIO_6_FALLING_SHIFT 0x04
#define PALMAS_INT4_EDGE_DETECT2_GPIO_5_RISING 0x08
-#define PALMAS_INT4_EDGE_DETECT2_GPIO_5_RISING_SHIFT 3
+#define PALMAS_INT4_EDGE_DETECT2_GPIO_5_RISING_SHIFT 0x03
#define PALMAS_INT4_EDGE_DETECT2_GPIO_5_FALLING 0x04
-#define PALMAS_INT4_EDGE_DETECT2_GPIO_5_FALLING_SHIFT 2
+#define PALMAS_INT4_EDGE_DETECT2_GPIO_5_FALLING_SHIFT 0x02
#define PALMAS_INT4_EDGE_DETECT2_GPIO_4_RISING 0x02
-#define PALMAS_INT4_EDGE_DETECT2_GPIO_4_RISING_SHIFT 1
+#define PALMAS_INT4_EDGE_DETECT2_GPIO_4_RISING_SHIFT 0x01
#define PALMAS_INT4_EDGE_DETECT2_GPIO_4_FALLING 0x01
-#define PALMAS_INT4_EDGE_DETECT2_GPIO_4_FALLING_SHIFT 0
+#define PALMAS_INT4_EDGE_DETECT2_GPIO_4_FALLING_SHIFT 0x00
/* Bit definitions for INT_CTRL */
#define PALMAS_INT_CTRL_INT_PENDING 0x04
-#define PALMAS_INT_CTRL_INT_PENDING_SHIFT 2
+#define PALMAS_INT_CTRL_INT_PENDING_SHIFT 0x02
#define PALMAS_INT_CTRL_INT_CLEAR 0x01
-#define PALMAS_INT_CTRL_INT_CLEAR_SHIFT 0
+#define PALMAS_INT_CTRL_INT_CLEAR_SHIFT 0x00
/* Registers for function USB_OTG */
-#define PALMAS_USB_WAKEUP 0x3
-#define PALMAS_USB_VBUS_CTRL_SET 0x4
-#define PALMAS_USB_VBUS_CTRL_CLR 0x5
-#define PALMAS_USB_ID_CTRL_SET 0x6
-#define PALMAS_USB_ID_CTRL_CLEAR 0x7
-#define PALMAS_USB_VBUS_INT_SRC 0x8
-#define PALMAS_USB_VBUS_INT_LATCH_SET 0x9
-#define PALMAS_USB_VBUS_INT_LATCH_CLR 0xA
-#define PALMAS_USB_VBUS_INT_EN_LO_SET 0xB
-#define PALMAS_USB_VBUS_INT_EN_LO_CLR 0xC
-#define PALMAS_USB_VBUS_INT_EN_HI_SET 0xD
-#define PALMAS_USB_VBUS_INT_EN_HI_CLR 0xE
-#define PALMAS_USB_ID_INT_SRC 0xF
+#define PALMAS_USB_WAKEUP 0x03
+#define PALMAS_USB_VBUS_CTRL_SET 0x04
+#define PALMAS_USB_VBUS_CTRL_CLR 0x05
+#define PALMAS_USB_ID_CTRL_SET 0x06
+#define PALMAS_USB_ID_CTRL_CLEAR 0x07
+#define PALMAS_USB_VBUS_INT_SRC 0x08
+#define PALMAS_USB_VBUS_INT_LATCH_SET 0x09
+#define PALMAS_USB_VBUS_INT_LATCH_CLR 0x0A
+#define PALMAS_USB_VBUS_INT_EN_LO_SET 0x0B
+#define PALMAS_USB_VBUS_INT_EN_LO_CLR 0x0C
+#define PALMAS_USB_VBUS_INT_EN_HI_SET 0x0D
+#define PALMAS_USB_VBUS_INT_EN_HI_CLR 0x0E
+#define PALMAS_USB_ID_INT_SRC 0x0F
#define PALMAS_USB_ID_INT_LATCH_SET 0x10
#define PALMAS_USB_ID_INT_LATCH_CLR 0x11
#define PALMAS_USB_ID_INT_EN_LO_SET 0x12
@@ -2250,306 +2250,306 @@
/* Bit definitions for USB_WAKEUP */
#define PALMAS_USB_WAKEUP_ID_WK_UP_COMP 0x01
-#define PALMAS_USB_WAKEUP_ID_WK_UP_COMP_SHIFT 0
+#define PALMAS_USB_WAKEUP_ID_WK_UP_COMP_SHIFT 0x00
/* Bit definitions for USB_VBUS_CTRL_SET */
#define PALMAS_USB_VBUS_CTRL_SET_VBUS_CHRG_VSYS 0x80
-#define PALMAS_USB_VBUS_CTRL_SET_VBUS_CHRG_VSYS_SHIFT 7
+#define PALMAS_USB_VBUS_CTRL_SET_VBUS_CHRG_VSYS_SHIFT 0x07
#define PALMAS_USB_VBUS_CTRL_SET_VBUS_DISCHRG 0x20
-#define PALMAS_USB_VBUS_CTRL_SET_VBUS_DISCHRG_SHIFT 5
+#define PALMAS_USB_VBUS_CTRL_SET_VBUS_DISCHRG_SHIFT 0x05
#define PALMAS_USB_VBUS_CTRL_SET_VBUS_IADP_SRC 0x10
-#define PALMAS_USB_VBUS_CTRL_SET_VBUS_IADP_SRC_SHIFT 4
+#define PALMAS_USB_VBUS_CTRL_SET_VBUS_IADP_SRC_SHIFT 0x04
#define PALMAS_USB_VBUS_CTRL_SET_VBUS_IADP_SINK 0x08
-#define PALMAS_USB_VBUS_CTRL_SET_VBUS_IADP_SINK_SHIFT 3
+#define PALMAS_USB_VBUS_CTRL_SET_VBUS_IADP_SINK_SHIFT 0x03
#define PALMAS_USB_VBUS_CTRL_SET_VBUS_ACT_COMP 0x04
-#define PALMAS_USB_VBUS_CTRL_SET_VBUS_ACT_COMP_SHIFT 2
+#define PALMAS_USB_VBUS_CTRL_SET_VBUS_ACT_COMP_SHIFT 0x02
/* Bit definitions for USB_VBUS_CTRL_CLR */
#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_CHRG_VSYS 0x80
-#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_CHRG_VSYS_SHIFT 7
+#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_CHRG_VSYS_SHIFT 0x07
#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_DISCHRG 0x20
-#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_DISCHRG_SHIFT 5
+#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_DISCHRG_SHIFT 0x05
#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_IADP_SRC 0x10
-#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_IADP_SRC_SHIFT 4
+#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_IADP_SRC_SHIFT 0x04
#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_IADP_SINK 0x08
-#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_IADP_SINK_SHIFT 3
+#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_IADP_SINK_SHIFT 0x03
#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_ACT_COMP 0x04
-#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_ACT_COMP_SHIFT 2
+#define PALMAS_USB_VBUS_CTRL_CLR_VBUS_ACT_COMP_SHIFT 0x02
/* Bit definitions for USB_ID_CTRL_SET */
#define PALMAS_USB_ID_CTRL_SET_ID_PU_220K 0x80
-#define PALMAS_USB_ID_CTRL_SET_ID_PU_220K_SHIFT 7
+#define PALMAS_USB_ID_CTRL_SET_ID_PU_220K_SHIFT 0x07
#define PALMAS_USB_ID_CTRL_SET_ID_PU_100K 0x40
-#define PALMAS_USB_ID_CTRL_SET_ID_PU_100K_SHIFT 6
+#define PALMAS_USB_ID_CTRL_SET_ID_PU_100K_SHIFT 0x06
#define PALMAS_USB_ID_CTRL_SET_ID_GND_DRV 0x20
-#define PALMAS_USB_ID_CTRL_SET_ID_GND_DRV_SHIFT 5
+#define PALMAS_USB_ID_CTRL_SET_ID_GND_DRV_SHIFT 0x05
#define PALMAS_USB_ID_CTRL_SET_ID_SRC_16U 0x10
-#define PALMAS_USB_ID_CTRL_SET_ID_SRC_16U_SHIFT 4
+#define PALMAS_USB_ID_CTRL_SET_ID_SRC_16U_SHIFT 0x04
#define PALMAS_USB_ID_CTRL_SET_ID_SRC_5U 0x08
-#define PALMAS_USB_ID_CTRL_SET_ID_SRC_5U_SHIFT 3
+#define PALMAS_USB_ID_CTRL_SET_ID_SRC_5U_SHIFT 0x03
#define PALMAS_USB_ID_CTRL_SET_ID_ACT_COMP 0x04
-#define PALMAS_USB_ID_CTRL_SET_ID_ACT_COMP_SHIFT 2
+#define PALMAS_USB_ID_CTRL_SET_ID_ACT_COMP_SHIFT 0x02
/* Bit definitions for USB_ID_CTRL_CLEAR */
#define PALMAS_USB_ID_CTRL_CLEAR_ID_PU_220K 0x80
-#define PALMAS_USB_ID_CTRL_CLEAR_ID_PU_220K_SHIFT 7
+#define PALMAS_USB_ID_CTRL_CLEAR_ID_PU_220K_SHIFT 0x07
#define PALMAS_USB_ID_CTRL_CLEAR_ID_PU_100K 0x40
-#define PALMAS_USB_ID_CTRL_CLEAR_ID_PU_100K_SHIFT 6
+#define PALMAS_USB_ID_CTRL_CLEAR_ID_PU_100K_SHIFT 0x06
#define PALMAS_USB_ID_CTRL_CLEAR_ID_GND_DRV 0x20
-#define PALMAS_USB_ID_CTRL_CLEAR_ID_GND_DRV_SHIFT 5
+#define PALMAS_USB_ID_CTRL_CLEAR_ID_GND_DRV_SHIFT 0x05
#define PALMAS_USB_ID_CTRL_CLEAR_ID_SRC_16U 0x10
-#define PALMAS_USB_ID_CTRL_CLEAR_ID_SRC_16U_SHIFT 4
+#define PALMAS_USB_ID_CTRL_CLEAR_ID_SRC_16U_SHIFT 0x04
#define PALMAS_USB_ID_CTRL_CLEAR_ID_SRC_5U 0x08
-#define PALMAS_USB_ID_CTRL_CLEAR_ID_SRC_5U_SHIFT 3
+#define PALMAS_USB_ID_CTRL_CLEAR_ID_SRC_5U_SHIFT 0x03
#define PALMAS_USB_ID_CTRL_CLEAR_ID_ACT_COMP 0x04
-#define PALMAS_USB_ID_CTRL_CLEAR_ID_ACT_COMP_SHIFT 2
+#define PALMAS_USB_ID_CTRL_CLEAR_ID_ACT_COMP_SHIFT 0x02
/* Bit definitions for USB_VBUS_INT_SRC */
#define PALMAS_USB_VBUS_INT_SRC_VOTG_SESS_VLD 0x80
-#define PALMAS_USB_VBUS_INT_SRC_VOTG_SESS_VLD_SHIFT 7
+#define PALMAS_USB_VBUS_INT_SRC_VOTG_SESS_VLD_SHIFT 0x07
#define PALMAS_USB_VBUS_INT_SRC_VADP_PRB 0x40
-#define PALMAS_USB_VBUS_INT_SRC_VADP_PRB_SHIFT 6
+#define PALMAS_USB_VBUS_INT_SRC_VADP_PRB_SHIFT 0x06
#define PALMAS_USB_VBUS_INT_SRC_VADP_SNS 0x20
-#define PALMAS_USB_VBUS_INT_SRC_VADP_SNS_SHIFT 5
+#define PALMAS_USB_VBUS_INT_SRC_VADP_SNS_SHIFT 0x05
#define PALMAS_USB_VBUS_INT_SRC_VA_VBUS_VLD 0x08
-#define PALMAS_USB_VBUS_INT_SRC_VA_VBUS_VLD_SHIFT 3
+#define PALMAS_USB_VBUS_INT_SRC_VA_VBUS_VLD_SHIFT 0x03
#define PALMAS_USB_VBUS_INT_SRC_VA_SESS_VLD 0x04
-#define PALMAS_USB_VBUS_INT_SRC_VA_SESS_VLD_SHIFT 2
+#define PALMAS_USB_VBUS_INT_SRC_VA_SESS_VLD_SHIFT 0x02
#define PALMAS_USB_VBUS_INT_SRC_VB_SESS_VLD 0x02
-#define PALMAS_USB_VBUS_INT_SRC_VB_SESS_VLD_SHIFT 1
+#define PALMAS_USB_VBUS_INT_SRC_VB_SESS_VLD_SHIFT 0x01
#define PALMAS_USB_VBUS_INT_SRC_VB_SESS_END 0x01
-#define PALMAS_USB_VBUS_INT_SRC_VB_SESS_END_SHIFT 0
+#define PALMAS_USB_VBUS_INT_SRC_VB_SESS_END_SHIFT 0x00
/* Bit definitions for USB_VBUS_INT_LATCH_SET */
#define PALMAS_USB_VBUS_INT_LATCH_SET_VOTG_SESS_VLD 0x80
-#define PALMAS_USB_VBUS_INT_LATCH_SET_VOTG_SESS_VLD_SHIFT 7
+#define PALMAS_USB_VBUS_INT_LATCH_SET_VOTG_SESS_VLD_SHIFT 0x07
#define PALMAS_USB_VBUS_INT_LATCH_SET_VADP_PRB 0x40
-#define PALMAS_USB_VBUS_INT_LATCH_SET_VADP_PRB_SHIFT 6
+#define PALMAS_USB_VBUS_INT_LATCH_SET_VADP_PRB_SHIFT 0x06
#define PALMAS_USB_VBUS_INT_LATCH_SET_VADP_SNS 0x20
-#define PALMAS_USB_VBUS_INT_LATCH_SET_VADP_SNS_SHIFT 5
+#define PALMAS_USB_VBUS_INT_LATCH_SET_VADP_SNS_SHIFT 0x05
#define PALMAS_USB_VBUS_INT_LATCH_SET_ADP 0x10
-#define PALMAS_USB_VBUS_INT_LATCH_SET_ADP_SHIFT 4
+#define PALMAS_USB_VBUS_INT_LATCH_SET_ADP_SHIFT 0x04
#define PALMAS_USB_VBUS_INT_LATCH_SET_VA_VBUS_VLD 0x08
-#define PALMAS_USB_VBUS_INT_LATCH_SET_VA_VBUS_VLD_SHIFT 3
+#define PALMAS_USB_VBUS_INT_LATCH_SET_VA_VBUS_VLD_SHIFT 0x03
#define PALMAS_USB_VBUS_INT_LATCH_SET_VA_SESS_VLD 0x04
-#define PALMAS_USB_VBUS_INT_LATCH_SET_VA_SESS_VLD_SHIFT 2
+#define PALMAS_USB_VBUS_INT_LATCH_SET_VA_SESS_VLD_SHIFT 0x02
#define PALMAS_USB_VBUS_INT_LATCH_SET_VB_SESS_VLD 0x02
-#define PALMAS_USB_VBUS_INT_LATCH_SET_VB_SESS_VLD_SHIFT 1
+#define PALMAS_USB_VBUS_INT_LATCH_SET_VB_SESS_VLD_SHIFT 0x01
#define PALMAS_USB_VBUS_INT_LATCH_SET_VB_SESS_END 0x01
-#define PALMAS_USB_VBUS_INT_LATCH_SET_VB_SESS_END_SHIFT 0
+#define PALMAS_USB_VBUS_INT_LATCH_SET_VB_SESS_END_SHIFT 0x00
/* Bit definitions for USB_VBUS_INT_LATCH_CLR */
#define PALMAS_USB_VBUS_INT_LATCH_CLR_VOTG_SESS_VLD 0x80
-#define PALMAS_USB_VBUS_INT_LATCH_CLR_VOTG_SESS_VLD_SHIFT 7
+#define PALMAS_USB_VBUS_INT_LATCH_CLR_VOTG_SESS_VLD_SHIFT 0x07
#define PALMAS_USB_VBUS_INT_LATCH_CLR_VADP_PRB 0x40
-#define PALMAS_USB_VBUS_INT_LATCH_CLR_VADP_PRB_SHIFT 6
+#define PALMAS_USB_VBUS_INT_LATCH_CLR_VADP_PRB_SHIFT 0x06
#define PALMAS_USB_VBUS_INT_LATCH_CLR_VADP_SNS 0x20
-#define PALMAS_USB_VBUS_INT_LATCH_CLR_VADP_SNS_SHIFT 5
+#define PALMAS_USB_VBUS_INT_LATCH_CLR_VADP_SNS_SHIFT 0x05
#define PALMAS_USB_VBUS_INT_LATCH_CLR_ADP 0x10
-#define PALMAS_USB_VBUS_INT_LATCH_CLR_ADP_SHIFT 4
+#define PALMAS_USB_VBUS_INT_LATCH_CLR_ADP_SHIFT 0x04
#define PALMAS_USB_VBUS_INT_LATCH_CLR_VA_VBUS_VLD 0x08
-#define PALMAS_USB_VBUS_INT_LATCH_CLR_VA_VBUS_VLD_SHIFT 3
+#define PALMAS_USB_VBUS_INT_LATCH_CLR_VA_VBUS_VLD_SHIFT 0x03
#define PALMAS_USB_VBUS_INT_LATCH_CLR_VA_SESS_VLD 0x04
-#define PALMAS_USB_VBUS_INT_LATCH_CLR_VA_SESS_VLD_SHIFT 2
+#define PALMAS_USB_VBUS_INT_LATCH_CLR_VA_SESS_VLD_SHIFT 0x02
#define PALMAS_USB_VBUS_INT_LATCH_CLR_VB_SESS_VLD 0x02
-#define PALMAS_USB_VBUS_INT_LATCH_CLR_VB_SESS_VLD_SHIFT 1
+#define PALMAS_USB_VBUS_INT_LATCH_CLR_VB_SESS_VLD_SHIFT 0x01
#define PALMAS_USB_VBUS_INT_LATCH_CLR_VB_SESS_END 0x01
-#define PALMAS_USB_VBUS_INT_LATCH_CLR_VB_SESS_END_SHIFT 0
+#define PALMAS_USB_VBUS_INT_LATCH_CLR_VB_SESS_END_SHIFT 0x00
/* Bit definitions for USB_VBUS_INT_EN_LO_SET */
#define PALMAS_USB_VBUS_INT_EN_LO_SET_VOTG_SESS_VLD 0x80
-#define PALMAS_USB_VBUS_INT_EN_LO_SET_VOTG_SESS_VLD_SHIFT 7
+#define PALMAS_USB_VBUS_INT_EN_LO_SET_VOTG_SESS_VLD_SHIFT 0x07
#define PALMAS_USB_VBUS_INT_EN_LO_SET_VADP_PRB 0x40
-#define PALMAS_USB_VBUS_INT_EN_LO_SET_VADP_PRB_SHIFT 6
+#define PALMAS_USB_VBUS_INT_EN_LO_SET_VADP_PRB_SHIFT 0x06
#define PALMAS_USB_VBUS_INT_EN_LO_SET_VADP_SNS 0x20
-#define PALMAS_USB_VBUS_INT_EN_LO_SET_VADP_SNS_SHIFT 5
+#define PALMAS_USB_VBUS_INT_EN_LO_SET_VADP_SNS_SHIFT 0x05
#define PALMAS_USB_VBUS_INT_EN_LO_SET_VA_VBUS_VLD 0x08
-#define PALMAS_USB_VBUS_INT_EN_LO_SET_VA_VBUS_VLD_SHIFT 3
+#define PALMAS_USB_VBUS_INT_EN_LO_SET_VA_VBUS_VLD_SHIFT 0x03
#define PALMAS_USB_VBUS_INT_EN_LO_SET_VA_SESS_VLD 0x04
-#define PALMAS_USB_VBUS_INT_EN_LO_SET_VA_SESS_VLD_SHIFT 2
+#define PALMAS_USB_VBUS_INT_EN_LO_SET_VA_SESS_VLD_SHIFT 0x02
#define PALMAS_USB_VBUS_INT_EN_LO_SET_VB_SESS_VLD 0x02
-#define PALMAS_USB_VBUS_INT_EN_LO_SET_VB_SESS_VLD_SHIFT 1
+#define PALMAS_USB_VBUS_INT_EN_LO_SET_VB_SESS_VLD_SHIFT 0x01
#define PALMAS_USB_VBUS_INT_EN_LO_SET_VB_SESS_END 0x01
-#define PALMAS_USB_VBUS_INT_EN_LO_SET_VB_SESS_END_SHIFT 0
+#define PALMAS_USB_VBUS_INT_EN_LO_SET_VB_SESS_END_SHIFT 0x00
/* Bit definitions for USB_VBUS_INT_EN_LO_CLR */
#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VOTG_SESS_VLD 0x80
-#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VOTG_SESS_VLD_SHIFT 7
+#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VOTG_SESS_VLD_SHIFT 0x07
#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VADP_PRB 0x40
-#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VADP_PRB_SHIFT 6
+#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VADP_PRB_SHIFT 0x06
#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VADP_SNS 0x20
-#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VADP_SNS_SHIFT 5
+#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VADP_SNS_SHIFT 0x05
#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VA_VBUS_VLD 0x08
-#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VA_VBUS_VLD_SHIFT 3
+#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VA_VBUS_VLD_SHIFT 0x03
#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VA_SESS_VLD 0x04
-#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VA_SESS_VLD_SHIFT 2
+#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VA_SESS_VLD_SHIFT 0x02
#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VB_SESS_VLD 0x02
-#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VB_SESS_VLD_SHIFT 1
+#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VB_SESS_VLD_SHIFT 0x01
#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VB_SESS_END 0x01
-#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VB_SESS_END_SHIFT 0
+#define PALMAS_USB_VBUS_INT_EN_LO_CLR_VB_SESS_END_SHIFT 0x00
/* Bit definitions for USB_VBUS_INT_EN_HI_SET */
#define PALMAS_USB_VBUS_INT_EN_HI_SET_VOTG_SESS_VLD 0x80
-#define PALMAS_USB_VBUS_INT_EN_HI_SET_VOTG_SESS_VLD_SHIFT 7
+#define PALMAS_USB_VBUS_INT_EN_HI_SET_VOTG_SESS_VLD_SHIFT 0x07
#define PALMAS_USB_VBUS_INT_EN_HI_SET_VADP_PRB 0x40
-#define PALMAS_USB_VBUS_INT_EN_HI_SET_VADP_PRB_SHIFT 6
+#define PALMAS_USB_VBUS_INT_EN_HI_SET_VADP_PRB_SHIFT 0x06
#define PALMAS_USB_VBUS_INT_EN_HI_SET_VADP_SNS 0x20
-#define PALMAS_USB_VBUS_INT_EN_HI_SET_VADP_SNS_SHIFT 5
+#define PALMAS_USB_VBUS_INT_EN_HI_SET_VADP_SNS_SHIFT 0x05
#define PALMAS_USB_VBUS_INT_EN_HI_SET_ADP 0x10
-#define PALMAS_USB_VBUS_INT_EN_HI_SET_ADP_SHIFT 4
+#define PALMAS_USB_VBUS_INT_EN_HI_SET_ADP_SHIFT 0x04
#define PALMAS_USB_VBUS_INT_EN_HI_SET_VA_VBUS_VLD 0x08
-#define PALMAS_USB_VBUS_INT_EN_HI_SET_VA_VBUS_VLD_SHIFT 3
+#define PALMAS_USB_VBUS_INT_EN_HI_SET_VA_VBUS_VLD_SHIFT 0x03
#define PALMAS_USB_VBUS_INT_EN_HI_SET_VA_SESS_VLD 0x04
-#define PALMAS_USB_VBUS_INT_EN_HI_SET_VA_SESS_VLD_SHIFT 2
+#define PALMAS_USB_VBUS_INT_EN_HI_SET_VA_SESS_VLD_SHIFT 0x02
#define PALMAS_USB_VBUS_INT_EN_HI_SET_VB_SESS_VLD 0x02
-#define PALMAS_USB_VBUS_INT_EN_HI_SET_VB_SESS_VLD_SHIFT 1
+#define PALMAS_USB_VBUS_INT_EN_HI_SET_VB_SESS_VLD_SHIFT 0x01
#define PALMAS_USB_VBUS_INT_EN_HI_SET_VB_SESS_END 0x01
-#define PALMAS_USB_VBUS_INT_EN_HI_SET_VB_SESS_END_SHIFT 0
+#define PALMAS_USB_VBUS_INT_EN_HI_SET_VB_SESS_END_SHIFT 0x00
/* Bit definitions for USB_VBUS_INT_EN_HI_CLR */
#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VOTG_SESS_VLD 0x80
-#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VOTG_SESS_VLD_SHIFT 7
+#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VOTG_SESS_VLD_SHIFT 0x07
#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VADP_PRB 0x40
-#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VADP_PRB_SHIFT 6
+#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VADP_PRB_SHIFT 0x06
#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VADP_SNS 0x20
-#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VADP_SNS_SHIFT 5
+#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VADP_SNS_SHIFT 0x05
#define PALMAS_USB_VBUS_INT_EN_HI_CLR_ADP 0x10
-#define PALMAS_USB_VBUS_INT_EN_HI_CLR_ADP_SHIFT 4
+#define PALMAS_USB_VBUS_INT_EN_HI_CLR_ADP_SHIFT 0x04
#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VA_VBUS_VLD 0x08
-#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VA_VBUS_VLD_SHIFT 3
+#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VA_VBUS_VLD_SHIFT 0x03
#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VA_SESS_VLD 0x04
-#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VA_SESS_VLD_SHIFT 2
+#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VA_SESS_VLD_SHIFT 0x02
#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VB_SESS_VLD 0x02
-#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VB_SESS_VLD_SHIFT 1
+#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VB_SESS_VLD_SHIFT 0x01
#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VB_SESS_END 0x01
-#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VB_SESS_END_SHIFT 0
+#define PALMAS_USB_VBUS_INT_EN_HI_CLR_VB_SESS_END_SHIFT 0x00
/* Bit definitions for USB_ID_INT_SRC */
#define PALMAS_USB_ID_INT_SRC_ID_FLOAT 0x10
-#define PALMAS_USB_ID_INT_SRC_ID_FLOAT_SHIFT 4
+#define PALMAS_USB_ID_INT_SRC_ID_FLOAT_SHIFT 0x04
#define PALMAS_USB_ID_INT_SRC_ID_A 0x08
-#define PALMAS_USB_ID_INT_SRC_ID_A_SHIFT 3
+#define PALMAS_USB_ID_INT_SRC_ID_A_SHIFT 0x03
#define PALMAS_USB_ID_INT_SRC_ID_B 0x04
-#define PALMAS_USB_ID_INT_SRC_ID_B_SHIFT 2
+#define PALMAS_USB_ID_INT_SRC_ID_B_SHIFT 0x02
#define PALMAS_USB_ID_INT_SRC_ID_C 0x02
-#define PALMAS_USB_ID_INT_SRC_ID_C_SHIFT 1
+#define PALMAS_USB_ID_INT_SRC_ID_C_SHIFT 0x01
#define PALMAS_USB_ID_INT_SRC_ID_GND 0x01
-#define PALMAS_USB_ID_INT_SRC_ID_GND_SHIFT 0
+#define PALMAS_USB_ID_INT_SRC_ID_GND_SHIFT 0x00
/* Bit definitions for USB_ID_INT_LATCH_SET */
#define PALMAS_USB_ID_INT_LATCH_SET_ID_FLOAT 0x10
-#define PALMAS_USB_ID_INT_LATCH_SET_ID_FLOAT_SHIFT 4
+#define PALMAS_USB_ID_INT_LATCH_SET_ID_FLOAT_SHIFT 0x04
#define PALMAS_USB_ID_INT_LATCH_SET_ID_A 0x08
-#define PALMAS_USB_ID_INT_LATCH_SET_ID_A_SHIFT 3
+#define PALMAS_USB_ID_INT_LATCH_SET_ID_A_SHIFT 0x03
#define PALMAS_USB_ID_INT_LATCH_SET_ID_B 0x04
-#define PALMAS_USB_ID_INT_LATCH_SET_ID_B_SHIFT 2
+#define PALMAS_USB_ID_INT_LATCH_SET_ID_B_SHIFT 0x02
#define PALMAS_USB_ID_INT_LATCH_SET_ID_C 0x02
-#define PALMAS_USB_ID_INT_LATCH_SET_ID_C_SHIFT 1
+#define PALMAS_USB_ID_INT_LATCH_SET_ID_C_SHIFT 0x01
#define PALMAS_USB_ID_INT_LATCH_SET_ID_GND 0x01
-#define PALMAS_USB_ID_INT_LATCH_SET_ID_GND_SHIFT 0
+#define PALMAS_USB_ID_INT_LATCH_SET_ID_GND_SHIFT 0x00
/* Bit definitions for USB_ID_INT_LATCH_CLR */
#define PALMAS_USB_ID_INT_LATCH_CLR_ID_FLOAT 0x10
-#define PALMAS_USB_ID_INT_LATCH_CLR_ID_FLOAT_SHIFT 4
+#define PALMAS_USB_ID_INT_LATCH_CLR_ID_FLOAT_SHIFT 0x04
#define PALMAS_USB_ID_INT_LATCH_CLR_ID_A 0x08
-#define PALMAS_USB_ID_INT_LATCH_CLR_ID_A_SHIFT 3
+#define PALMAS_USB_ID_INT_LATCH_CLR_ID_A_SHIFT 0x03
#define PALMAS_USB_ID_INT_LATCH_CLR_ID_B 0x04
-#define PALMAS_USB_ID_INT_LATCH_CLR_ID_B_SHIFT 2
+#define PALMAS_USB_ID_INT_LATCH_CLR_ID_B_SHIFT 0x02
#define PALMAS_USB_ID_INT_LATCH_CLR_ID_C 0x02
-#define PALMAS_USB_ID_INT_LATCH_CLR_ID_C_SHIFT 1
+#define PALMAS_USB_ID_INT_LATCH_CLR_ID_C_SHIFT 0x01
#define PALMAS_USB_ID_INT_LATCH_CLR_ID_GND 0x01
-#define PALMAS_USB_ID_INT_LATCH_CLR_ID_GND_SHIFT 0
+#define PALMAS_USB_ID_INT_LATCH_CLR_ID_GND_SHIFT 0x00
/* Bit definitions for USB_ID_INT_EN_LO_SET */
#define PALMAS_USB_ID_INT_EN_LO_SET_ID_FLOAT 0x10
-#define PALMAS_USB_ID_INT_EN_LO_SET_ID_FLOAT_SHIFT 4
+#define PALMAS_USB_ID_INT_EN_LO_SET_ID_FLOAT_SHIFT 0x04
#define PALMAS_USB_ID_INT_EN_LO_SET_ID_A 0x08
-#define PALMAS_USB_ID_INT_EN_LO_SET_ID_A_SHIFT 3
+#define PALMAS_USB_ID_INT_EN_LO_SET_ID_A_SHIFT 0x03
#define PALMAS_USB_ID_INT_EN_LO_SET_ID_B 0x04
-#define PALMAS_USB_ID_INT_EN_LO_SET_ID_B_SHIFT 2
+#define PALMAS_USB_ID_INT_EN_LO_SET_ID_B_SHIFT 0x02
#define PALMAS_USB_ID_INT_EN_LO_SET_ID_C 0x02
-#define PALMAS_USB_ID_INT_EN_LO_SET_ID_C_SHIFT 1
+#define PALMAS_USB_ID_INT_EN_LO_SET_ID_C_SHIFT 0x01
#define PALMAS_USB_ID_INT_EN_LO_SET_ID_GND 0x01
-#define PALMAS_USB_ID_INT_EN_LO_SET_ID_GND_SHIFT 0
+#define PALMAS_USB_ID_INT_EN_LO_SET_ID_GND_SHIFT 0x00
/* Bit definitions for USB_ID_INT_EN_LO_CLR */
#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_FLOAT 0x10
-#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_FLOAT_SHIFT 4
+#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_FLOAT_SHIFT 0x04
#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_A 0x08
-#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_A_SHIFT 3
+#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_A_SHIFT 0x03
#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_B 0x04
-#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_B_SHIFT 2
+#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_B_SHIFT 0x02
#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_C 0x02
-#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_C_SHIFT 1
+#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_C_SHIFT 0x01
#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_GND 0x01
-#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_GND_SHIFT 0
+#define PALMAS_USB_ID_INT_EN_LO_CLR_ID_GND_SHIFT 0x00
/* Bit definitions for USB_ID_INT_EN_HI_SET */
#define PALMAS_USB_ID_INT_EN_HI_SET_ID_FLOAT 0x10
-#define PALMAS_USB_ID_INT_EN_HI_SET_ID_FLOAT_SHIFT 4
+#define PALMAS_USB_ID_INT_EN_HI_SET_ID_FLOAT_SHIFT 0x04
#define PALMAS_USB_ID_INT_EN_HI_SET_ID_A 0x08
-#define PALMAS_USB_ID_INT_EN_HI_SET_ID_A_SHIFT 3
+#define PALMAS_USB_ID_INT_EN_HI_SET_ID_A_SHIFT 0x03
#define PALMAS_USB_ID_INT_EN_HI_SET_ID_B 0x04
-#define PALMAS_USB_ID_INT_EN_HI_SET_ID_B_SHIFT 2
+#define PALMAS_USB_ID_INT_EN_HI_SET_ID_B_SHIFT 0x02
#define PALMAS_USB_ID_INT_EN_HI_SET_ID_C 0x02
-#define PALMAS_USB_ID_INT_EN_HI_SET_ID_C_SHIFT 1
+#define PALMAS_USB_ID_INT_EN_HI_SET_ID_C_SHIFT 0x01
#define PALMAS_USB_ID_INT_EN_HI_SET_ID_GND 0x01
-#define PALMAS_USB_ID_INT_EN_HI_SET_ID_GND_SHIFT 0
+#define PALMAS_USB_ID_INT_EN_HI_SET_ID_GND_SHIFT 0x00
/* Bit definitions for USB_ID_INT_EN_HI_CLR */
#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_FLOAT 0x10
-#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_FLOAT_SHIFT 4
+#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_FLOAT_SHIFT 0x04
#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_A 0x08
-#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_A_SHIFT 3
+#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_A_SHIFT 0x03
#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_B 0x04
-#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_B_SHIFT 2
+#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_B_SHIFT 0x02
#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_C 0x02
-#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_C_SHIFT 1
+#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_C_SHIFT 0x01
#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_GND 0x01
-#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_GND_SHIFT 0
+#define PALMAS_USB_ID_INT_EN_HI_CLR_ID_GND_SHIFT 0x00
/* Bit definitions for USB_OTG_ADP_CTRL */
#define PALMAS_USB_OTG_ADP_CTRL_ADP_EN 0x04
-#define PALMAS_USB_OTG_ADP_CTRL_ADP_EN_SHIFT 2
+#define PALMAS_USB_OTG_ADP_CTRL_ADP_EN_SHIFT 0x02
#define PALMAS_USB_OTG_ADP_CTRL_ADP_MODE_MASK 0x03
-#define PALMAS_USB_OTG_ADP_CTRL_ADP_MODE_SHIFT 0
+#define PALMAS_USB_OTG_ADP_CTRL_ADP_MODE_SHIFT 0x00
/* Bit definitions for USB_OTG_ADP_HIGH */
-#define PALMAS_USB_OTG_ADP_HIGH_T_ADP_HIGH_MASK 0xff
-#define PALMAS_USB_OTG_ADP_HIGH_T_ADP_HIGH_SHIFT 0
+#define PALMAS_USB_OTG_ADP_HIGH_T_ADP_HIGH_MASK 0xFF
+#define PALMAS_USB_OTG_ADP_HIGH_T_ADP_HIGH_SHIFT 0x00
/* Bit definitions for USB_OTG_ADP_LOW */
-#define PALMAS_USB_OTG_ADP_LOW_T_ADP_LOW_MASK 0xff
-#define PALMAS_USB_OTG_ADP_LOW_T_ADP_LOW_SHIFT 0
+#define PALMAS_USB_OTG_ADP_LOW_T_ADP_LOW_MASK 0xFF
+#define PALMAS_USB_OTG_ADP_LOW_T_ADP_LOW_SHIFT 0x00
/* Bit definitions for USB_OTG_ADP_RISE */
-#define PALMAS_USB_OTG_ADP_RISE_T_ADP_RISE_MASK 0xff
-#define PALMAS_USB_OTG_ADP_RISE_T_ADP_RISE_SHIFT 0
+#define PALMAS_USB_OTG_ADP_RISE_T_ADP_RISE_MASK 0xFF
+#define PALMAS_USB_OTG_ADP_RISE_T_ADP_RISE_SHIFT 0x00
/* Bit definitions for USB_OTG_REVISION */
#define PALMAS_USB_OTG_REVISION_OTG_REV 0x01
-#define PALMAS_USB_OTG_REVISION_OTG_REV_SHIFT 0
+#define PALMAS_USB_OTG_REVISION_OTG_REV_SHIFT 0x00
/* Registers for function VIBRATOR */
-#define PALMAS_VIBRA_CTRL 0x0
+#define PALMAS_VIBRA_CTRL 0x00
/* Bit definitions for VIBRA_CTRL */
#define PALMAS_VIBRA_CTRL_PWM_DUTY_SEL_MASK 0x06
-#define PALMAS_VIBRA_CTRL_PWM_DUTY_SEL_SHIFT 1
+#define PALMAS_VIBRA_CTRL_PWM_DUTY_SEL_SHIFT 0x01
#define PALMAS_VIBRA_CTRL_PWM_FREQ_SEL 0x01
-#define PALMAS_VIBRA_CTRL_PWM_FREQ_SEL_SHIFT 0
+#define PALMAS_VIBRA_CTRL_PWM_FREQ_SEL_SHIFT 0x00
/* Registers for function GPIO */
-#define PALMAS_GPIO_DATA_IN 0x0
-#define PALMAS_GPIO_DATA_DIR 0x1
-#define PALMAS_GPIO_DATA_OUT 0x2
-#define PALMAS_GPIO_DEBOUNCE_EN 0x3
-#define PALMAS_GPIO_CLEAR_DATA_OUT 0x4
-#define PALMAS_GPIO_SET_DATA_OUT 0x5
-#define PALMAS_PU_PD_GPIO_CTRL1 0x6
-#define PALMAS_PU_PD_GPIO_CTRL2 0x7
-#define PALMAS_OD_OUTPUT_GPIO_CTRL 0x8
-#define PALMAS_GPIO_DATA_IN2 0x9
+#define PALMAS_GPIO_DATA_IN 0x00
+#define PALMAS_GPIO_DATA_DIR 0x01
+#define PALMAS_GPIO_DATA_OUT 0x02
+#define PALMAS_GPIO_DEBOUNCE_EN 0x03
+#define PALMAS_GPIO_CLEAR_DATA_OUT 0x04
+#define PALMAS_GPIO_SET_DATA_OUT 0x05
+#define PALMAS_PU_PD_GPIO_CTRL1 0x06
+#define PALMAS_PU_PD_GPIO_CTRL2 0x07
+#define PALMAS_OD_OUTPUT_GPIO_CTRL 0x08
+#define PALMAS_GPIO_DATA_IN2 0x09
#define PALMAS_GPIO_DATA_DIR2 0x0A
#define PALMAS_GPIO_DATA_OUT2 0x0B
#define PALMAS_GPIO_DEBOUNCE_EN2 0x0C
@@ -2561,167 +2561,167 @@
/* Bit definitions for GPIO_DATA_IN */
#define PALMAS_GPIO_DATA_IN_GPIO_7_IN 0x80
-#define PALMAS_GPIO_DATA_IN_GPIO_7_IN_SHIFT 7
+#define PALMAS_GPIO_DATA_IN_GPIO_7_IN_SHIFT 0x07
#define PALMAS_GPIO_DATA_IN_GPIO_6_IN 0x40
-#define PALMAS_GPIO_DATA_IN_GPIO_6_IN_SHIFT 6
+#define PALMAS_GPIO_DATA_IN_GPIO_6_IN_SHIFT 0x06
#define PALMAS_GPIO_DATA_IN_GPIO_5_IN 0x20
-#define PALMAS_GPIO_DATA_IN_GPIO_5_IN_SHIFT 5
+#define PALMAS_GPIO_DATA_IN_GPIO_5_IN_SHIFT 0x05
#define PALMAS_GPIO_DATA_IN_GPIO_4_IN 0x10
-#define PALMAS_GPIO_DATA_IN_GPIO_4_IN_SHIFT 4
+#define PALMAS_GPIO_DATA_IN_GPIO_4_IN_SHIFT 0x04
#define PALMAS_GPIO_DATA_IN_GPIO_3_IN 0x08
-#define PALMAS_GPIO_DATA_IN_GPIO_3_IN_SHIFT 3
+#define PALMAS_GPIO_DATA_IN_GPIO_3_IN_SHIFT 0x03
#define PALMAS_GPIO_DATA_IN_GPIO_2_IN 0x04
-#define PALMAS_GPIO_DATA_IN_GPIO_2_IN_SHIFT 2
+#define PALMAS_GPIO_DATA_IN_GPIO_2_IN_SHIFT 0x02
#define PALMAS_GPIO_DATA_IN_GPIO_1_IN 0x02
-#define PALMAS_GPIO_DATA_IN_GPIO_1_IN_SHIFT 1
+#define PALMAS_GPIO_DATA_IN_GPIO_1_IN_SHIFT 0x01
#define PALMAS_GPIO_DATA_IN_GPIO_0_IN 0x01
-#define PALMAS_GPIO_DATA_IN_GPIO_0_IN_SHIFT 0
+#define PALMAS_GPIO_DATA_IN_GPIO_0_IN_SHIFT 0x00
/* Bit definitions for GPIO_DATA_DIR */
#define PALMAS_GPIO_DATA_DIR_GPIO_7_DIR 0x80
-#define PALMAS_GPIO_DATA_DIR_GPIO_7_DIR_SHIFT 7
+#define PALMAS_GPIO_DATA_DIR_GPIO_7_DIR_SHIFT 0x07
#define PALMAS_GPIO_DATA_DIR_GPIO_6_DIR 0x40
-#define PALMAS_GPIO_DATA_DIR_GPIO_6_DIR_SHIFT 6
+#define PALMAS_GPIO_DATA_DIR_GPIO_6_DIR_SHIFT 0x06
#define PALMAS_GPIO_DATA_DIR_GPIO_5_DIR 0x20
-#define PALMAS_GPIO_DATA_DIR_GPIO_5_DIR_SHIFT 5
+#define PALMAS_GPIO_DATA_DIR_GPIO_5_DIR_SHIFT 0x05
#define PALMAS_GPIO_DATA_DIR_GPIO_4_DIR 0x10
-#define PALMAS_GPIO_DATA_DIR_GPIO_4_DIR_SHIFT 4
+#define PALMAS_GPIO_DATA_DIR_GPIO_4_DIR_SHIFT 0x04
#define PALMAS_GPIO_DATA_DIR_GPIO_3_DIR 0x08
-#define PALMAS_GPIO_DATA_DIR_GPIO_3_DIR_SHIFT 3
+#define PALMAS_GPIO_DATA_DIR_GPIO_3_DIR_SHIFT 0x03
#define PALMAS_GPIO_DATA_DIR_GPIO_2_DIR 0x04
-#define PALMAS_GPIO_DATA_DIR_GPIO_2_DIR_SHIFT 2
+#define PALMAS_GPIO_DATA_DIR_GPIO_2_DIR_SHIFT 0x02
#define PALMAS_GPIO_DATA_DIR_GPIO_1_DIR 0x02
-#define PALMAS_GPIO_DATA_DIR_GPIO_1_DIR_SHIFT 1
+#define PALMAS_GPIO_DATA_DIR_GPIO_1_DIR_SHIFT 0x01
#define PALMAS_GPIO_DATA_DIR_GPIO_0_DIR 0x01
-#define PALMAS_GPIO_DATA_DIR_GPIO_0_DIR_SHIFT 0
+#define PALMAS_GPIO_DATA_DIR_GPIO_0_DIR_SHIFT 0x00
/* Bit definitions for GPIO_DATA_OUT */
#define PALMAS_GPIO_DATA_OUT_GPIO_7_OUT 0x80
-#define PALMAS_GPIO_DATA_OUT_GPIO_7_OUT_SHIFT 7
+#define PALMAS_GPIO_DATA_OUT_GPIO_7_OUT_SHIFT 0x07
#define PALMAS_GPIO_DATA_OUT_GPIO_6_OUT 0x40
-#define PALMAS_GPIO_DATA_OUT_GPIO_6_OUT_SHIFT 6
+#define PALMAS_GPIO_DATA_OUT_GPIO_6_OUT_SHIFT 0x06
#define PALMAS_GPIO_DATA_OUT_GPIO_5_OUT 0x20
-#define PALMAS_GPIO_DATA_OUT_GPIO_5_OUT_SHIFT 5
+#define PALMAS_GPIO_DATA_OUT_GPIO_5_OUT_SHIFT 0x05
#define PALMAS_GPIO_DATA_OUT_GPIO_4_OUT 0x10
-#define PALMAS_GPIO_DATA_OUT_GPIO_4_OUT_SHIFT 4
+#define PALMAS_GPIO_DATA_OUT_GPIO_4_OUT_SHIFT 0x04
#define PALMAS_GPIO_DATA_OUT_GPIO_3_OUT 0x08
-#define PALMAS_GPIO_DATA_OUT_GPIO_3_OUT_SHIFT 3
+#define PALMAS_GPIO_DATA_OUT_GPIO_3_OUT_SHIFT 0x03
#define PALMAS_GPIO_DATA_OUT_GPIO_2_OUT 0x04
-#define PALMAS_GPIO_DATA_OUT_GPIO_2_OUT_SHIFT 2
+#define PALMAS_GPIO_DATA_OUT_GPIO_2_OUT_SHIFT 0x02
#define PALMAS_GPIO_DATA_OUT_GPIO_1_OUT 0x02
-#define PALMAS_GPIO_DATA_OUT_GPIO_1_OUT_SHIFT 1
+#define PALMAS_GPIO_DATA_OUT_GPIO_1_OUT_SHIFT 0x01
#define PALMAS_GPIO_DATA_OUT_GPIO_0_OUT 0x01
-#define PALMAS_GPIO_DATA_OUT_GPIO_0_OUT_SHIFT 0
+#define PALMAS_GPIO_DATA_OUT_GPIO_0_OUT_SHIFT 0x00
/* Bit definitions for GPIO_DEBOUNCE_EN */
#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_7_DEBOUNCE_EN 0x80
-#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_7_DEBOUNCE_EN_SHIFT 7
+#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_7_DEBOUNCE_EN_SHIFT 0x07
#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_6_DEBOUNCE_EN 0x40
-#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_6_DEBOUNCE_EN_SHIFT 6
+#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_6_DEBOUNCE_EN_SHIFT 0x06
#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_5_DEBOUNCE_EN 0x20
-#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_5_DEBOUNCE_EN_SHIFT 5
+#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_5_DEBOUNCE_EN_SHIFT 0x05
#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_4_DEBOUNCE_EN 0x10
-#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_4_DEBOUNCE_EN_SHIFT 4
+#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_4_DEBOUNCE_EN_SHIFT 0x04
#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_3_DEBOUNCE_EN 0x08
-#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_3_DEBOUNCE_EN_SHIFT 3
+#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_3_DEBOUNCE_EN_SHIFT 0x03
#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_2_DEBOUNCE_EN 0x04
-#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_2_DEBOUNCE_EN_SHIFT 2
+#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_2_DEBOUNCE_EN_SHIFT 0x02
#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_1_DEBOUNCE_EN 0x02
-#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_1_DEBOUNCE_EN_SHIFT 1
+#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_1_DEBOUNCE_EN_SHIFT 0x01
#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_0_DEBOUNCE_EN 0x01
-#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_0_DEBOUNCE_EN_SHIFT 0
+#define PALMAS_GPIO_DEBOUNCE_EN_GPIO_0_DEBOUNCE_EN_SHIFT 0x00
/* Bit definitions for GPIO_CLEAR_DATA_OUT */
#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_7_CLEAR_DATA_OUT 0x80
-#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_7_CLEAR_DATA_OUT_SHIFT 7
+#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_7_CLEAR_DATA_OUT_SHIFT 0x07
#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_6_CLEAR_DATA_OUT 0x40
-#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_6_CLEAR_DATA_OUT_SHIFT 6
+#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_6_CLEAR_DATA_OUT_SHIFT 0x06
#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_5_CLEAR_DATA_OUT 0x20
-#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_5_CLEAR_DATA_OUT_SHIFT 5
+#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_5_CLEAR_DATA_OUT_SHIFT 0x05
#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_4_CLEAR_DATA_OUT 0x10
-#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_4_CLEAR_DATA_OUT_SHIFT 4
+#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_4_CLEAR_DATA_OUT_SHIFT 0x04
#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_3_CLEAR_DATA_OUT 0x08
-#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_3_CLEAR_DATA_OUT_SHIFT 3
+#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_3_CLEAR_DATA_OUT_SHIFT 0x03
#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_2_CLEAR_DATA_OUT 0x04
-#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_2_CLEAR_DATA_OUT_SHIFT 2
+#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_2_CLEAR_DATA_OUT_SHIFT 0x02
#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_1_CLEAR_DATA_OUT 0x02
-#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_1_CLEAR_DATA_OUT_SHIFT 1
+#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_1_CLEAR_DATA_OUT_SHIFT 0x01
#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_0_CLEAR_DATA_OUT 0x01
-#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_0_CLEAR_DATA_OUT_SHIFT 0
+#define PALMAS_GPIO_CLEAR_DATA_OUT_GPIO_0_CLEAR_DATA_OUT_SHIFT 0x00
/* Bit definitions for GPIO_SET_DATA_OUT */
#define PALMAS_GPIO_SET_DATA_OUT_GPIO_7_SET_DATA_OUT 0x80
-#define PALMAS_GPIO_SET_DATA_OUT_GPIO_7_SET_DATA_OUT_SHIFT 7
+#define PALMAS_GPIO_SET_DATA_OUT_GPIO_7_SET_DATA_OUT_SHIFT 0x07
#define PALMAS_GPIO_SET_DATA_OUT_GPIO_6_SET_DATA_OUT 0x40
-#define PALMAS_GPIO_SET_DATA_OUT_GPIO_6_SET_DATA_OUT_SHIFT 6
+#define PALMAS_GPIO_SET_DATA_OUT_GPIO_6_SET_DATA_OUT_SHIFT 0x06
#define PALMAS_GPIO_SET_DATA_OUT_GPIO_5_SET_DATA_OUT 0x20
-#define PALMAS_GPIO_SET_DATA_OUT_GPIO_5_SET_DATA_OUT_SHIFT 5
+#define PALMAS_GPIO_SET_DATA_OUT_GPIO_5_SET_DATA_OUT_SHIFT 0x05
#define PALMAS_GPIO_SET_DATA_OUT_GPIO_4_SET_DATA_OUT 0x10
-#define PALMAS_GPIO_SET_DATA_OUT_GPIO_4_SET_DATA_OUT_SHIFT 4
+#define PALMAS_GPIO_SET_DATA_OUT_GPIO_4_SET_DATA_OUT_SHIFT 0x04
#define PALMAS_GPIO_SET_DATA_OUT_GPIO_3_SET_DATA_OUT 0x08
-#define PALMAS_GPIO_SET_DATA_OUT_GPIO_3_SET_DATA_OUT_SHIFT 3
+#define PALMAS_GPIO_SET_DATA_OUT_GPIO_3_SET_DATA_OUT_SHIFT 0x03
#define PALMAS_GPIO_SET_DATA_OUT_GPIO_2_SET_DATA_OUT 0x04
-#define PALMAS_GPIO_SET_DATA_OUT_GPIO_2_SET_DATA_OUT_SHIFT 2
+#define PALMAS_GPIO_SET_DATA_OUT_GPIO_2_SET_DATA_OUT_SHIFT 0x02
#define PALMAS_GPIO_SET_DATA_OUT_GPIO_1_SET_DATA_OUT 0x02
-#define PALMAS_GPIO_SET_DATA_OUT_GPIO_1_SET_DATA_OUT_SHIFT 1
+#define PALMAS_GPIO_SET_DATA_OUT_GPIO_1_SET_DATA_OUT_SHIFT 0x01
#define PALMAS_GPIO_SET_DATA_OUT_GPIO_0_SET_DATA_OUT 0x01
-#define PALMAS_GPIO_SET_DATA_OUT_GPIO_0_SET_DATA_OUT_SHIFT 0
+#define PALMAS_GPIO_SET_DATA_OUT_GPIO_0_SET_DATA_OUT_SHIFT 0x00
/* Bit definitions for PU_PD_GPIO_CTRL1 */
#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_3_PD 0x40
-#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_3_PD_SHIFT 6
+#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_3_PD_SHIFT 0x06
#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_2_PU 0x20
-#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_2_PU_SHIFT 5
+#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_2_PU_SHIFT 0x05
#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_2_PD 0x10
-#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_2_PD_SHIFT 4
+#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_2_PD_SHIFT 0x04
#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_1_PU 0x08
-#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_1_PU_SHIFT 3
+#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_1_PU_SHIFT 0x03
#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_1_PD 0x04
-#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_1_PD_SHIFT 2
+#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_1_PD_SHIFT 0x02
#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_0_PD 0x01
-#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_0_PD_SHIFT 0
+#define PALMAS_PU_PD_GPIO_CTRL1_GPIO_0_PD_SHIFT 0x00
/* Bit definitions for PU_PD_GPIO_CTRL2 */
#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_7_PD 0x40
-#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_7_PD_SHIFT 6
+#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_7_PD_SHIFT 0x06
#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_6_PU 0x20
-#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_6_PU_SHIFT 5
+#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_6_PU_SHIFT 0x05
#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_6_PD 0x10
-#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_6_PD_SHIFT 4
+#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_6_PD_SHIFT 0x04
#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_5_PU 0x08
-#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_5_PU_SHIFT 3
+#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_5_PU_SHIFT 0x03
#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_5_PD 0x04
-#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_5_PD_SHIFT 2
+#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_5_PD_SHIFT 0x02
#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_4_PU 0x02
-#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_4_PU_SHIFT 1
+#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_4_PU_SHIFT 0x01
#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_4_PD 0x01
-#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_4_PD_SHIFT 0
+#define PALMAS_PU_PD_GPIO_CTRL2_GPIO_4_PD_SHIFT 0x00
/* Bit definitions for OD_OUTPUT_GPIO_CTRL */
#define PALMAS_OD_OUTPUT_GPIO_CTRL_GPIO_5_OD 0x20
-#define PALMAS_OD_OUTPUT_GPIO_CTRL_GPIO_5_OD_SHIFT 5
+#define PALMAS_OD_OUTPUT_GPIO_CTRL_GPIO_5_OD_SHIFT 0x05
#define PALMAS_OD_OUTPUT_GPIO_CTRL_GPIO_2_OD 0x04
-#define PALMAS_OD_OUTPUT_GPIO_CTRL_GPIO_2_OD_SHIFT 2
+#define PALMAS_OD_OUTPUT_GPIO_CTRL_GPIO_2_OD_SHIFT 0x02
#define PALMAS_OD_OUTPUT_GPIO_CTRL_GPIO_1_OD 0x02
-#define PALMAS_OD_OUTPUT_GPIO_CTRL_GPIO_1_OD_SHIFT 1
+#define PALMAS_OD_OUTPUT_GPIO_CTRL_GPIO_1_OD_SHIFT 0x01
/* Registers for function GPADC */
-#define PALMAS_GPADC_CTRL1 0x0
-#define PALMAS_GPADC_CTRL2 0x1
-#define PALMAS_GPADC_RT_CTRL 0x2
-#define PALMAS_GPADC_AUTO_CTRL 0x3
-#define PALMAS_GPADC_STATUS 0x4
-#define PALMAS_GPADC_RT_SELECT 0x5
-#define PALMAS_GPADC_RT_CONV0_LSB 0x6
-#define PALMAS_GPADC_RT_CONV0_MSB 0x7
-#define PALMAS_GPADC_AUTO_SELECT 0x8
-#define PALMAS_GPADC_AUTO_CONV0_LSB 0x9
-#define PALMAS_GPADC_AUTO_CONV0_MSB 0xA
-#define PALMAS_GPADC_AUTO_CONV1_LSB 0xB
-#define PALMAS_GPADC_AUTO_CONV1_MSB 0xC
-#define PALMAS_GPADC_SW_SELECT 0xD
-#define PALMAS_GPADC_SW_CONV0_LSB 0xE
-#define PALMAS_GPADC_SW_CONV0_MSB 0xF
+#define PALMAS_GPADC_CTRL1 0x00
+#define PALMAS_GPADC_CTRL2 0x01
+#define PALMAS_GPADC_RT_CTRL 0x02
+#define PALMAS_GPADC_AUTO_CTRL 0x03
+#define PALMAS_GPADC_STATUS 0x04
+#define PALMAS_GPADC_RT_SELECT 0x05
+#define PALMAS_GPADC_RT_CONV0_LSB 0x06
+#define PALMAS_GPADC_RT_CONV0_MSB 0x07
+#define PALMAS_GPADC_AUTO_SELECT 0x08
+#define PALMAS_GPADC_AUTO_CONV0_LSB 0x09
+#define PALMAS_GPADC_AUTO_CONV0_MSB 0x0A
+#define PALMAS_GPADC_AUTO_CONV1_LSB 0x0B
+#define PALMAS_GPADC_AUTO_CONV1_MSB 0x0C
+#define PALMAS_GPADC_SW_SELECT 0x0D
+#define PALMAS_GPADC_SW_CONV0_LSB 0x0E
+#define PALMAS_GPADC_SW_CONV0_MSB 0x0F
#define PALMAS_GPADC_THRES_CONV0_LSB 0x10
#define PALMAS_GPADC_THRES_CONV0_MSB 0x11
#define PALMAS_GPADC_THRES_CONV1_LSB 0x12
@@ -2731,150 +2731,150 @@
/* Bit definitions for GPADC_CTRL1 */
#define PALMAS_GPADC_CTRL1_RESERVED_MASK 0xc0
-#define PALMAS_GPADC_CTRL1_RESERVED_SHIFT 6
+#define PALMAS_GPADC_CTRL1_RESERVED_SHIFT 0x06
#define PALMAS_GPADC_CTRL1_CURRENT_SRC_CH3_MASK 0x30
-#define PALMAS_GPADC_CTRL1_CURRENT_SRC_CH3_SHIFT 4
+#define PALMAS_GPADC_CTRL1_CURRENT_SRC_CH3_SHIFT 0x04
#define PALMAS_GPADC_CTRL1_CURRENT_SRC_CH0_MASK 0x0c
-#define PALMAS_GPADC_CTRL1_CURRENT_SRC_CH0_SHIFT 2
+#define PALMAS_GPADC_CTRL1_CURRENT_SRC_CH0_SHIFT 0x02
#define PALMAS_GPADC_CTRL1_BAT_REMOVAL_DET 0x02
-#define PALMAS_GPADC_CTRL1_BAT_REMOVAL_DET_SHIFT 1
+#define PALMAS_GPADC_CTRL1_BAT_REMOVAL_DET_SHIFT 0x01
#define PALMAS_GPADC_CTRL1_GPADC_FORCE 0x01
-#define PALMAS_GPADC_CTRL1_GPADC_FORCE_SHIFT 0
+#define PALMAS_GPADC_CTRL1_GPADC_FORCE_SHIFT 0x00
/* Bit definitions for GPADC_CTRL2 */
#define PALMAS_GPADC_CTRL2_RESERVED_MASK 0x06
-#define PALMAS_GPADC_CTRL2_RESERVED_SHIFT 1
+#define PALMAS_GPADC_CTRL2_RESERVED_SHIFT 0x01
/* Bit definitions for GPADC_RT_CTRL */
#define PALMAS_GPADC_RT_CTRL_EXTEND_DELAY 0x02
-#define PALMAS_GPADC_RT_CTRL_EXTEND_DELAY_SHIFT 1
+#define PALMAS_GPADC_RT_CTRL_EXTEND_DELAY_SHIFT 0x01
#define PALMAS_GPADC_RT_CTRL_START_POLARITY 0x01
-#define PALMAS_GPADC_RT_CTRL_START_POLARITY_SHIFT 0
+#define PALMAS_GPADC_RT_CTRL_START_POLARITY_SHIFT 0x00
/* Bit definitions for GPADC_AUTO_CTRL */
#define PALMAS_GPADC_AUTO_CTRL_SHUTDOWN_CONV1 0x80
-#define PALMAS_GPADC_AUTO_CTRL_SHUTDOWN_CONV1_SHIFT 7
+#define PALMAS_GPADC_AUTO_CTRL_SHUTDOWN_CONV1_SHIFT 0x07
#define PALMAS_GPADC_AUTO_CTRL_SHUTDOWN_CONV0 0x40
-#define PALMAS_GPADC_AUTO_CTRL_SHUTDOWN_CONV0_SHIFT 6
+#define PALMAS_GPADC_AUTO_CTRL_SHUTDOWN_CONV0_SHIFT 0x06
#define PALMAS_GPADC_AUTO_CTRL_AUTO_CONV1_EN 0x20
-#define PALMAS_GPADC_AUTO_CTRL_AUTO_CONV1_EN_SHIFT 5
+#define PALMAS_GPADC_AUTO_CTRL_AUTO_CONV1_EN_SHIFT 0x05
#define PALMAS_GPADC_AUTO_CTRL_AUTO_CONV0_EN 0x10
-#define PALMAS_GPADC_AUTO_CTRL_AUTO_CONV0_EN_SHIFT 4
-#define PALMAS_GPADC_AUTO_CTRL_COUNTER_CONV_MASK 0x0f
-#define PALMAS_GPADC_AUTO_CTRL_COUNTER_CONV_SHIFT 0
+#define PALMAS_GPADC_AUTO_CTRL_AUTO_CONV0_EN_SHIFT 0x04
+#define PALMAS_GPADC_AUTO_CTRL_COUNTER_CONV_MASK 0x0F
+#define PALMAS_GPADC_AUTO_CTRL_COUNTER_CONV_SHIFT 0x00
/* Bit definitions for GPADC_STATUS */
#define PALMAS_GPADC_STATUS_GPADC_AVAILABLE 0x10
-#define PALMAS_GPADC_STATUS_GPADC_AVAILABLE_SHIFT 4
+#define PALMAS_GPADC_STATUS_GPADC_AVAILABLE_SHIFT 0x04
/* Bit definitions for GPADC_RT_SELECT */
#define PALMAS_GPADC_RT_SELECT_RT_CONV_EN 0x80
-#define PALMAS_GPADC_RT_SELECT_RT_CONV_EN_SHIFT 7
-#define PALMAS_GPADC_RT_SELECT_RT_CONV0_SEL_MASK 0x0f
-#define PALMAS_GPADC_RT_SELECT_RT_CONV0_SEL_SHIFT 0
+#define PALMAS_GPADC_RT_SELECT_RT_CONV_EN_SHIFT 0x07
+#define PALMAS_GPADC_RT_SELECT_RT_CONV0_SEL_MASK 0x0F
+#define PALMAS_GPADC_RT_SELECT_RT_CONV0_SEL_SHIFT 0x00
/* Bit definitions for GPADC_RT_CONV0_LSB */
-#define PALMAS_GPADC_RT_CONV0_LSB_RT_CONV0_LSB_MASK 0xff
-#define PALMAS_GPADC_RT_CONV0_LSB_RT_CONV0_LSB_SHIFT 0
+#define PALMAS_GPADC_RT_CONV0_LSB_RT_CONV0_LSB_MASK 0xFF
+#define PALMAS_GPADC_RT_CONV0_LSB_RT_CONV0_LSB_SHIFT 0x00
/* Bit definitions for GPADC_RT_CONV0_MSB */
-#define PALMAS_GPADC_RT_CONV0_MSB_RT_CONV0_MSB_MASK 0x0f
-#define PALMAS_GPADC_RT_CONV0_MSB_RT_CONV0_MSB_SHIFT 0
+#define PALMAS_GPADC_RT_CONV0_MSB_RT_CONV0_MSB_MASK 0x0F
+#define PALMAS_GPADC_RT_CONV0_MSB_RT_CONV0_MSB_SHIFT 0x00
/* Bit definitions for GPADC_AUTO_SELECT */
-#define PALMAS_GPADC_AUTO_SELECT_AUTO_CONV1_SEL_MASK 0xf0
-#define PALMAS_GPADC_AUTO_SELECT_AUTO_CONV1_SEL_SHIFT 4
-#define PALMAS_GPADC_AUTO_SELECT_AUTO_CONV0_SEL_MASK 0x0f
-#define PALMAS_GPADC_AUTO_SELECT_AUTO_CONV0_SEL_SHIFT 0
+#define PALMAS_GPADC_AUTO_SELECT_AUTO_CONV1_SEL_MASK 0xF0
+#define PALMAS_GPADC_AUTO_SELECT_AUTO_CONV1_SEL_SHIFT 0x04
+#define PALMAS_GPADC_AUTO_SELECT_AUTO_CONV0_SEL_MASK 0x0F
+#define PALMAS_GPADC_AUTO_SELECT_AUTO_CONV0_SEL_SHIFT 0x00
/* Bit definitions for GPADC_AUTO_CONV0_LSB */
-#define PALMAS_GPADC_AUTO_CONV0_LSB_AUTO_CONV0_LSB_MASK 0xff
-#define PALMAS_GPADC_AUTO_CONV0_LSB_AUTO_CONV0_LSB_SHIFT 0
+#define PALMAS_GPADC_AUTO_CONV0_LSB_AUTO_CONV0_LSB_MASK 0xFF
+#define PALMAS_GPADC_AUTO_CONV0_LSB_AUTO_CONV0_LSB_SHIFT 0x00
/* Bit definitions for GPADC_AUTO_CONV0_MSB */
-#define PALMAS_GPADC_AUTO_CONV0_MSB_AUTO_CONV0_MSB_MASK 0x0f
-#define PALMAS_GPADC_AUTO_CONV0_MSB_AUTO_CONV0_MSB_SHIFT 0
+#define PALMAS_GPADC_AUTO_CONV0_MSB_AUTO_CONV0_MSB_MASK 0x0F
+#define PALMAS_GPADC_AUTO_CONV0_MSB_AUTO_CONV0_MSB_SHIFT 0x00
/* Bit definitions for GPADC_AUTO_CONV1_LSB */
-#define PALMAS_GPADC_AUTO_CONV1_LSB_AUTO_CONV1_LSB_MASK 0xff
-#define PALMAS_GPADC_AUTO_CONV1_LSB_AUTO_CONV1_LSB_SHIFT 0
+#define PALMAS_GPADC_AUTO_CONV1_LSB_AUTO_CONV1_LSB_MASK 0xFF
+#define PALMAS_GPADC_AUTO_CONV1_LSB_AUTO_CONV1_LSB_SHIFT 0x00
/* Bit definitions for GPADC_AUTO_CONV1_MSB */
-#define PALMAS_GPADC_AUTO_CONV1_MSB_AUTO_CONV1_MSB_MASK 0x0f
-#define PALMAS_GPADC_AUTO_CONV1_MSB_AUTO_CONV1_MSB_SHIFT 0
+#define PALMAS_GPADC_AUTO_CONV1_MSB_AUTO_CONV1_MSB_MASK 0x0F
+#define PALMAS_GPADC_AUTO_CONV1_MSB_AUTO_CONV1_MSB_SHIFT 0x00
/* Bit definitions for GPADC_SW_SELECT */
#define PALMAS_GPADC_SW_SELECT_SW_CONV_EN 0x80
-#define PALMAS_GPADC_SW_SELECT_SW_CONV_EN_SHIFT 7
+#define PALMAS_GPADC_SW_SELECT_SW_CONV_EN_SHIFT 0x07
#define PALMAS_GPADC_SW_SELECT_SW_START_CONV0 0x10
-#define PALMAS_GPADC_SW_SELECT_SW_START_CONV0_SHIFT 4
-#define PALMAS_GPADC_SW_SELECT_SW_CONV0_SEL_MASK 0x0f
-#define PALMAS_GPADC_SW_SELECT_SW_CONV0_SEL_SHIFT 0
+#define PALMAS_GPADC_SW_SELECT_SW_START_CONV0_SHIFT 0x04
+#define PALMAS_GPADC_SW_SELECT_SW_CONV0_SEL_MASK 0x0F
+#define PALMAS_GPADC_SW_SELECT_SW_CONV0_SEL_SHIFT 0x00
/* Bit definitions for GPADC_SW_CONV0_LSB */
-#define PALMAS_GPADC_SW_CONV0_LSB_SW_CONV0_LSB_MASK 0xff
-#define PALMAS_GPADC_SW_CONV0_LSB_SW_CONV0_LSB_SHIFT 0
+#define PALMAS_GPADC_SW_CONV0_LSB_SW_CONV0_LSB_MASK 0xFF
+#define PALMAS_GPADC_SW_CONV0_LSB_SW_CONV0_LSB_SHIFT 0x00
/* Bit definitions for GPADC_SW_CONV0_MSB */
-#define PALMAS_GPADC_SW_CONV0_MSB_SW_CONV0_MSB_MASK 0x0f
-#define PALMAS_GPADC_SW_CONV0_MSB_SW_CONV0_MSB_SHIFT 0
+#define PALMAS_GPADC_SW_CONV0_MSB_SW_CONV0_MSB_MASK 0x0F
+#define PALMAS_GPADC_SW_CONV0_MSB_SW_CONV0_MSB_SHIFT 0x00
/* Bit definitions for GPADC_THRES_CONV0_LSB */
-#define PALMAS_GPADC_THRES_CONV0_LSB_THRES_CONV0_LSB_MASK 0xff
-#define PALMAS_GPADC_THRES_CONV0_LSB_THRES_CONV0_LSB_SHIFT 0
+#define PALMAS_GPADC_THRES_CONV0_LSB_THRES_CONV0_LSB_MASK 0xFF
+#define PALMAS_GPADC_THRES_CONV0_LSB_THRES_CONV0_LSB_SHIFT 0x00
/* Bit definitions for GPADC_THRES_CONV0_MSB */
#define PALMAS_GPADC_THRES_CONV0_MSB_THRES_CONV0_POL 0x80
-#define PALMAS_GPADC_THRES_CONV0_MSB_THRES_CONV0_POL_SHIFT 7
-#define PALMAS_GPADC_THRES_CONV0_MSB_THRES_CONV0_MSB_MASK 0x0f
-#define PALMAS_GPADC_THRES_CONV0_MSB_THRES_CONV0_MSB_SHIFT 0
+#define PALMAS_GPADC_THRES_CONV0_MSB_THRES_CONV0_POL_SHIFT 0x07
+#define PALMAS_GPADC_THRES_CONV0_MSB_THRES_CONV0_MSB_MASK 0x0F
+#define PALMAS_GPADC_THRES_CONV0_MSB_THRES_CONV0_MSB_SHIFT 0x00
/* Bit definitions for GPADC_THRES_CONV1_LSB */
-#define PALMAS_GPADC_THRES_CONV1_LSB_THRES_CONV1_LSB_MASK 0xff
-#define PALMAS_GPADC_THRES_CONV1_LSB_THRES_CONV1_LSB_SHIFT 0
+#define PALMAS_GPADC_THRES_CONV1_LSB_THRES_CONV1_LSB_MASK 0xFF
+#define PALMAS_GPADC_THRES_CONV1_LSB_THRES_CONV1_LSB_SHIFT 0x00
/* Bit definitions for GPADC_THRES_CONV1_MSB */
#define PALMAS_GPADC_THRES_CONV1_MSB_THRES_CONV1_POL 0x80
-#define PALMAS_GPADC_THRES_CONV1_MSB_THRES_CONV1_POL_SHIFT 7
-#define PALMAS_GPADC_THRES_CONV1_MSB_THRES_CONV1_MSB_MASK 0x0f
-#define PALMAS_GPADC_THRES_CONV1_MSB_THRES_CONV1_MSB_SHIFT 0
+#define PALMAS_GPADC_THRES_CONV1_MSB_THRES_CONV1_POL_SHIFT 0x07
+#define PALMAS_GPADC_THRES_CONV1_MSB_THRES_CONV1_MSB_MASK 0x0F
+#define PALMAS_GPADC_THRES_CONV1_MSB_THRES_CONV1_MSB_SHIFT 0x00
/* Bit definitions for GPADC_SMPS_ILMONITOR_EN */
#define PALMAS_GPADC_SMPS_ILMONITOR_EN_SMPS_ILMON_EN 0x20
-#define PALMAS_GPADC_SMPS_ILMONITOR_EN_SMPS_ILMON_EN_SHIFT 5
+#define PALMAS_GPADC_SMPS_ILMONITOR_EN_SMPS_ILMON_EN_SHIFT 0x05
#define PALMAS_GPADC_SMPS_ILMONITOR_EN_SMPS_ILMON_REXT 0x10
-#define PALMAS_GPADC_SMPS_ILMONITOR_EN_SMPS_ILMON_REXT_SHIFT 4
-#define PALMAS_GPADC_SMPS_ILMONITOR_EN_SMPS_ILMON_SEL_MASK 0x0f
-#define PALMAS_GPADC_SMPS_ILMONITOR_EN_SMPS_ILMON_SEL_SHIFT 0
+#define PALMAS_GPADC_SMPS_ILMONITOR_EN_SMPS_ILMON_REXT_SHIFT 0x04
+#define PALMAS_GPADC_SMPS_ILMONITOR_EN_SMPS_ILMON_SEL_MASK 0x0F
+#define PALMAS_GPADC_SMPS_ILMONITOR_EN_SMPS_ILMON_SEL_SHIFT 0x00
/* Bit definitions for GPADC_SMPS_VSEL_MONITORING */
#define PALMAS_GPADC_SMPS_VSEL_MONITORING_ACTIVE_PHASE 0x80
-#define PALMAS_GPADC_SMPS_VSEL_MONITORING_ACTIVE_PHASE_SHIFT 7
-#define PALMAS_GPADC_SMPS_VSEL_MONITORING_SMPS_VSEL_MONITORING_MASK 0x7f
-#define PALMAS_GPADC_SMPS_VSEL_MONITORING_SMPS_VSEL_MONITORING_SHIFT 0
+#define PALMAS_GPADC_SMPS_VSEL_MONITORING_ACTIVE_PHASE_SHIFT 0x07
+#define PALMAS_GPADC_SMPS_VSEL_MONITORING_SMPS_VSEL_MONITORING_MASK 0x7F
+#define PALMAS_GPADC_SMPS_VSEL_MONITORING_SMPS_VSEL_MONITORING_SHIFT 0x00
/* Registers for function GPADC */
-#define PALMAS_GPADC_TRIM1 0x0
-#define PALMAS_GPADC_TRIM2 0x1
-#define PALMAS_GPADC_TRIM3 0x2
-#define PALMAS_GPADC_TRIM4 0x3
-#define PALMAS_GPADC_TRIM5 0x4
-#define PALMAS_GPADC_TRIM6 0x5
-#define PALMAS_GPADC_TRIM7 0x6
-#define PALMAS_GPADC_TRIM8 0x7
-#define PALMAS_GPADC_TRIM9 0x8
-#define PALMAS_GPADC_TRIM10 0x9
-#define PALMAS_GPADC_TRIM11 0xA
-#define PALMAS_GPADC_TRIM12 0xB
-#define PALMAS_GPADC_TRIM13 0xC
-#define PALMAS_GPADC_TRIM14 0xD
-#define PALMAS_GPADC_TRIM15 0xE
-#define PALMAS_GPADC_TRIM16 0xF
+#define PALMAS_GPADC_TRIM1 0x00
+#define PALMAS_GPADC_TRIM2 0x01
+#define PALMAS_GPADC_TRIM3 0x02
+#define PALMAS_GPADC_TRIM4 0x03
+#define PALMAS_GPADC_TRIM5 0x04
+#define PALMAS_GPADC_TRIM6 0x05
+#define PALMAS_GPADC_TRIM7 0x06
+#define PALMAS_GPADC_TRIM8 0x07
+#define PALMAS_GPADC_TRIM9 0x08
+#define PALMAS_GPADC_TRIM10 0x09
+#define PALMAS_GPADC_TRIM11 0x0A
+#define PALMAS_GPADC_TRIM12 0x0B
+#define PALMAS_GPADC_TRIM13 0x0C
+#define PALMAS_GPADC_TRIM14 0x0D
+#define PALMAS_GPADC_TRIM15 0x0E
+#define PALMAS_GPADC_TRIM16 0x0F
static inline int palmas_read(struct palmas *palmas, unsigned int base,
unsigned int reg, unsigned int *val)
{
- unsigned int addr = PALMAS_BASE_TO_REG(base, reg);
+ unsigned int addr = PALMAS_BASE_TO_REG(base, reg);
int slave_id = PALMAS_BASE_TO_SLAVE(base);
return regmap_read(palmas->regmap[slave_id], addr, val);
diff --git a/include/linux/mfd/pm8xxx/core.h b/include/linux/mfd/pm8xxx/core.h
deleted file mode 100644
index bd2f4f6..0000000
--- a/include/linux/mfd/pm8xxx/core.h
+++ /dev/null
@@ -1,81 +0,0 @@
-/*
- * Copyright (c) 2011, Code Aurora Forum. All rights reserved.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 and
- * only version 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- */
-/*
- * Qualcomm PMIC 8xxx driver header file
- *
- */
-
-#ifndef __MFD_PM8XXX_CORE_H
-#define __MFD_PM8XXX_CORE_H
-
-#include <linux/mfd/core.h>
-
-struct pm8xxx_drvdata {
- int (*pmic_readb) (const struct device *dev, u16 addr, u8 *val);
- int (*pmic_writeb) (const struct device *dev, u16 addr, u8 val);
- int (*pmic_read_buf) (const struct device *dev, u16 addr, u8 *buf,
- int n);
- int (*pmic_write_buf) (const struct device *dev, u16 addr, u8 *buf,
- int n);
- int (*pmic_read_irq_stat) (const struct device *dev, int irq);
- void *pm_chip_data;
-};
-
-static inline int pm8xxx_readb(const struct device *dev, u16 addr, u8 *val)
-{
- struct pm8xxx_drvdata *dd = dev_get_drvdata(dev);
-
- if (!dd)
- return -EINVAL;
- return dd->pmic_readb(dev, addr, val);
-}
-
-static inline int pm8xxx_writeb(const struct device *dev, u16 addr, u8 val)
-{
- struct pm8xxx_drvdata *dd = dev_get_drvdata(dev);
-
- if (!dd)
- return -EINVAL;
- return dd->pmic_writeb(dev, addr, val);
-}
-
-static inline int pm8xxx_read_buf(const struct device *dev, u16 addr, u8 *buf,
- int n)
-{
- struct pm8xxx_drvdata *dd = dev_get_drvdata(dev);
-
- if (!dd)
- return -EINVAL;
- return dd->pmic_read_buf(dev, addr, buf, n);
-}
-
-static inline int pm8xxx_write_buf(const struct device *dev, u16 addr, u8 *buf,
- int n)
-{
- struct pm8xxx_drvdata *dd = dev_get_drvdata(dev);
-
- if (!dd)
- return -EINVAL;
- return dd->pmic_write_buf(dev, addr, buf, n);
-}
-
-static inline int pm8xxx_read_irq_stat(const struct device *dev, int irq)
-{
- struct pm8xxx_drvdata *dd = dev_get_drvdata(dev);
-
- if (!dd)
- return -EINVAL;
- return dd->pmic_read_irq_stat(dev, irq);
-}
-
-#endif
diff --git a/include/linux/mfd/rdc321x.h b/include/linux/mfd/rdc321x.h
index 4bdf19c..442743a 100644
--- a/include/linux/mfd/rdc321x.h
+++ b/include/linux/mfd/rdc321x.h
@@ -12,7 +12,7 @@
#define RDC321X_GPIO_CTRL_REG2 0x84
#define RDC321X_GPIO_DATA_REG2 0x88
-#define RDC321X_MAX_GPIO 58
+#define RDC321X_NUM_GPIO 59
struct rdc321x_gpio_pdata {
struct pci_dev *sb_pdev;
diff --git a/include/linux/mfd/samsung/core.h b/include/linux/mfd/samsung/core.h
index 157e32b..47d8424 100644
--- a/include/linux/mfd/samsung/core.h
+++ b/include/linux/mfd/samsung/core.h
@@ -24,35 +24,36 @@
};
/**
- * struct sec_pmic_dev - s5m87xx master device for sub-drivers
- * @dev: master device of the chip (can be used to access platform data)
- * @pdata: pointer to private data used to pass platform data to child
- * @i2c: i2c client private data for regulator
- * @rtc: i2c client private data for rtc
- * @iolock: mutex for serializing io access
- * @irqlock: mutex for buslock
- * @irq_base: base IRQ number for sec-pmic, required for IRQs
- * @irq: generic IRQ number for s5m87xx
- * @ono: power onoff IRQ number for s5m87xx
- * @irq_masks_cur: currently active value
- * @irq_masks_cache: cached hardware value
- * @type: indicate which s5m87xx "variant" is used
+ * struct sec_pmic_dev - s2m/s5m master device for sub-drivers
+ * @dev: Master device of the chip
+ * @pdata: Platform data populated with data from DTS
+ * or board files
+ * @regmap_pmic: Regmap associated with PMIC's I2C address
+ * @i2c: I2C client of the main driver
+ * @device_type: Type of device, matches enum sec_device_type
+ * @irq_base: Base IRQ number for device, required for IRQs
+ * @irq: Generic IRQ number for device
+ * @irq_data: Runtime data structure for IRQ controller
+ * @ono: Power onoff IRQ number for s5m87xx
+ * @wakeup: Whether or not this is a wakeup device
+ * @wtsr_smpl: Whether or not to enable in RTC driver the Watchdog
+ * Timer Software Reset (registers set to default value
+ * after PWRHOLD falling) and Sudden Momentary Power Loss
+ * (PMIC will enter power on sequence after short drop in
+ * VBATT voltage).
*/
struct sec_pmic_dev {
struct device *dev;
struct sec_platform_data *pdata;
struct regmap *regmap_pmic;
- struct regmap *regmap_rtc;
struct i2c_client *i2c;
- struct i2c_client *rtc;
- int device_type;
+ unsigned long device_type;
int irq_base;
int irq;
struct regmap_irq_chip_data *irq_data;
int ono;
- unsigned long type;
bool wakeup;
bool wtsr_smpl;
};
diff --git a/include/linux/mfd/samsung/rtc.h b/include/linux/mfd/samsung/rtc.h
index 3e02b76..b6401e7 100644
--- a/include/linux/mfd/samsung/rtc.h
+++ b/include/linux/mfd/samsung/rtc.h
@@ -18,38 +18,38 @@
#ifndef __LINUX_MFD_SEC_RTC_H
#define __LINUX_MFD_SEC_RTC_H
-enum sec_rtc_reg {
- SEC_RTC_SEC,
- SEC_RTC_MIN,
- SEC_RTC_HOUR,
- SEC_RTC_WEEKDAY,
- SEC_RTC_DATE,
- SEC_RTC_MONTH,
- SEC_RTC_YEAR1,
- SEC_RTC_YEAR2,
- SEC_ALARM0_SEC,
- SEC_ALARM0_MIN,
- SEC_ALARM0_HOUR,
- SEC_ALARM0_WEEKDAY,
- SEC_ALARM0_DATE,
- SEC_ALARM0_MONTH,
- SEC_ALARM0_YEAR1,
- SEC_ALARM0_YEAR2,
- SEC_ALARM1_SEC,
- SEC_ALARM1_MIN,
- SEC_ALARM1_HOUR,
- SEC_ALARM1_WEEKDAY,
- SEC_ALARM1_DATE,
- SEC_ALARM1_MONTH,
- SEC_ALARM1_YEAR1,
- SEC_ALARM1_YEAR2,
- SEC_ALARM0_CONF,
- SEC_ALARM1_CONF,
- SEC_RTC_STATUS,
- SEC_WTSR_SMPL_CNTL,
- SEC_RTC_UDR_CON,
+enum s5m_rtc_reg {
+ S5M_RTC_SEC,
+ S5M_RTC_MIN,
+ S5M_RTC_HOUR,
+ S5M_RTC_WEEKDAY,
+ S5M_RTC_DATE,
+ S5M_RTC_MONTH,
+ S5M_RTC_YEAR1,
+ S5M_RTC_YEAR2,
+ S5M_ALARM0_SEC,
+ S5M_ALARM0_MIN,
+ S5M_ALARM0_HOUR,
+ S5M_ALARM0_WEEKDAY,
+ S5M_ALARM0_DATE,
+ S5M_ALARM0_MONTH,
+ S5M_ALARM0_YEAR1,
+ S5M_ALARM0_YEAR2,
+ S5M_ALARM1_SEC,
+ S5M_ALARM1_MIN,
+ S5M_ALARM1_HOUR,
+ S5M_ALARM1_WEEKDAY,
+ S5M_ALARM1_DATE,
+ S5M_ALARM1_MONTH,
+ S5M_ALARM1_YEAR1,
+ S5M_ALARM1_YEAR2,
+ S5M_ALARM0_CONF,
+ S5M_ALARM1_CONF,
+ S5M_RTC_STATUS,
+ S5M_WTSR_SMPL_CNTL,
+ S5M_RTC_UDR_CON,
- SEC_RTC_REG_MAX,
+ S5M_RTC_REG_MAX,
};
enum s2mps_rtc_reg {
@@ -88,9 +88,9 @@
#define HOUR_12 (1 << 7)
#define HOUR_AMPM (1 << 6)
#define HOUR_PM (1 << 5)
-#define ALARM0_STATUS (1 << 1)
-#define ALARM1_STATUS (1 << 2)
-#define UPDATE_AD (1 << 0)
+#define S5M_ALARM0_STATUS (1 << 1)
+#define S5M_ALARM1_STATUS (1 << 2)
+#define S5M_UPDATE_AD (1 << 0)
#define S2MPS_ALARM0_STATUS (1 << 2)
#define S2MPS_ALARM1_STATUS (1 << 1)
@@ -101,16 +101,26 @@
#define MODEL24_SHIFT 1
#define MODEL24_MASK (1 << MODEL24_SHIFT)
/* RTC Update Register1 */
-#define RTC_UDR_SHIFT 0
-#define RTC_UDR_MASK (1 << RTC_UDR_SHIFT)
+#define S5M_RTC_UDR_SHIFT 0
+#define S5M_RTC_UDR_MASK (1 << S5M_RTC_UDR_SHIFT)
#define S2MPS_RTC_WUDR_SHIFT 4
#define S2MPS_RTC_WUDR_MASK (1 << S2MPS_RTC_WUDR_SHIFT)
#define S2MPS_RTC_RUDR_SHIFT 0
#define S2MPS_RTC_RUDR_MASK (1 << S2MPS_RTC_RUDR_SHIFT)
#define RTC_TCON_SHIFT 1
#define RTC_TCON_MASK (1 << RTC_TCON_SHIFT)
-#define RTC_TIME_EN_SHIFT 3
-#define RTC_TIME_EN_MASK (1 << RTC_TIME_EN_SHIFT)
+#define S5M_RTC_TIME_EN_SHIFT 3
+#define S5M_RTC_TIME_EN_MASK (1 << S5M_RTC_TIME_EN_SHIFT)
+/*
+ * UDR_T field in S5M_RTC_UDR_CON register determines the time needed
+ * for updating alarm and time registers. Default is 7.32 ms.
+ */
+#define S5M_RTC_UDR_T_SHIFT 6
+#define S5M_RTC_UDR_T_MASK (0x3 << S5M_RTC_UDR_T_SHIFT)
+#define S5M_RTC_UDR_T_7320_US (0x0 << S5M_RTC_UDR_T_SHIFT)
+#define S5M_RTC_UDR_T_1830_US (0x1 << S5M_RTC_UDR_T_SHIFT)
+#define S5M_RTC_UDR_T_3660_US (0x2 << S5M_RTC_UDR_T_SHIFT)
+#define S5M_RTC_UDR_T_450_US (0x3 << S5M_RTC_UDR_T_SHIFT)
/* RTC Hour register */
#define HOUR_PM_SHIFT 6
diff --git a/include/linux/mfd/stmpe.h b/include/linux/mfd/stmpe.h
index 48395a6..575a86c 100644
--- a/include/linux/mfd/stmpe.h
+++ b/include/linux/mfd/stmpe.h
@@ -11,6 +11,7 @@
#include <linux/mutex.h>
struct device;
+struct regulator;
enum stmpe_block {
STMPE_BLOCK_GPIO = 1 << 0,
@@ -62,6 +63,8 @@
/**
* struct stmpe - STMPE MFD structure
+ * @vcc: optional VCC regulator
+ * @vio: optional VIO regulator
* @lock: lock protecting I/O operations
* @irq_lock: IRQ bus lock
* @dev: device, mostly for dev_dbg()
@@ -73,13 +76,14 @@
* @regs: list of addresses of registers which are at different addresses on
* different variants. Indexed by one of STMPE_IDX_*.
* @irq: irq number for stmpe
- * @irq_base: starting IRQ number for internal IRQs
* @num_gpios: number of gpios, differs for variants
* @ier: cache of IER registers for bus_lock
* @oldier: cache of IER registers for bus_lock
* @pdata: platform data
*/
struct stmpe {
+ struct regulator *vcc;
+ struct regulator *vio;
struct mutex lock;
struct mutex irq_lock;
struct device *dev;
@@ -91,7 +95,6 @@
const u8 *regs;
int irq;
- int irq_base;
int num_gpios;
u8 ier[2];
u8 oldier[2];
@@ -132,8 +135,6 @@
/**
* struct stmpe_gpio_platform_data - STMPE GPIO platform data
- * @gpio_base: first gpio number assigned. A maximum of
- * %STMPE_NR_GPIOS GPIOs will be allocated.
* @norequest_mask: bitmask specifying which GPIOs should _not_ be
* requestable due to different usage (e.g. touch, keypad)
* STMPE_GPIO_NOREQ_* macros can be used here.
@@ -141,7 +142,6 @@
* @remove: board specific remove callback
*/
struct stmpe_gpio_platform_data {
- int gpio_base;
unsigned norequest_mask;
void (*setup)(struct stmpe *stmpe, unsigned gpio_base);
void (*remove)(struct stmpe *stmpe, unsigned gpio_base);
@@ -195,8 +195,6 @@
* @irq_trigger: IRQ trigger to use for the interrupt to the host
* @autosleep: bool to enable/disable stmpe autosleep
* @autosleep_timeout: inactivity timeout in milliseconds for autosleep
- * @irq_base: base IRQ number. %STMPE_NR_IRQS irqs will be used, or
- * %STMPE_NR_INTERNAL_IRQS if the GPIO driver is not used.
* @irq_over_gpio: true if gpio is used to get irq
* @irq_gpio: gpio number over which irq will be requested (significant only if
* irq_over_gpio is true)
@@ -207,7 +205,6 @@
struct stmpe_platform_data {
int id;
unsigned int blocks;
- int irq_base;
unsigned int irq_trigger;
bool autosleep;
bool irq_over_gpio;
@@ -219,10 +216,4 @@
struct stmpe_ts_platform_data *ts;
};
-#define STMPE_NR_INTERNAL_IRQS 9
-#define STMPE_INT_GPIO(x) (STMPE_NR_INTERNAL_IRQS + (x))
-
-#define STMPE_NR_GPIOS 24
-#define STMPE_NR_IRQS STMPE_INT_GPIO(STMPE_NR_GPIOS)
-
#endif
diff --git a/include/linux/mfd/syscon.h b/include/linux/mfd/syscon.h
index 8789fa3c..75e543b 100644
--- a/include/linux/mfd/syscon.h
+++ b/include/linux/mfd/syscon.h
@@ -15,6 +15,8 @@
#ifndef __LINUX_MFD_SYSCON_H__
#define __LINUX_MFD_SYSCON_H__
+#include <linux/err.h>
+
struct device_node;
#ifdef CONFIG_MFD_SYSCON
diff --git a/include/linux/mfd/tps65218.h b/include/linux/mfd/tps65218.h
index d2e357d..2f9b593 100644
--- a/include/linux/mfd/tps65218.h
+++ b/include/linux/mfd/tps65218.h
@@ -267,7 +267,6 @@
u32 irq_mask;
struct regmap_irq_chip_data *irq_data;
struct regulator_desc desc[TPS65218_NUM_REGULATOR];
- struct regulator_dev *rdev[TPS65218_NUM_REGULATOR];
struct tps_info *info[TPS65218_NUM_REGULATOR];
struct regmap *regmap;
};
diff --git a/include/linux/mfd/twl6040.h b/include/linux/mfd/twl6040.h
index 81f639b..8f9fc3d 100644
--- a/include/linux/mfd/twl6040.h
+++ b/include/linux/mfd/twl6040.h
@@ -28,6 +28,7 @@
#include <linux/interrupt.h>
#include <linux/mfd/core.h>
#include <linux/regulator/consumer.h>
+#include <linux/clk.h>
#define TWL6040_REG_ASICID 0x01
#define TWL6040_REG_ASICREV 0x02
@@ -157,6 +158,7 @@
#define TWL6040_I2CSEL 0x01
#define TWL6040_RESETSPLIT 0x04
#define TWL6040_INTCLRMODE 0x08
+#define TWL6040_I2CMODE(x) ((x & 0x3) << 4)
/* STATUS (0x2E) fields */
@@ -222,6 +224,7 @@
struct regmap *regmap;
struct regmap_irq_chip_data *irq_data;
struct regulator_bulk_data supplies[2]; /* supplies for vio, v2v1 */
+ struct clk *clk32k;
struct mutex mutex;
struct mutex irq_mutex;
struct mfd_cell cells[TWL6040_CELLS];
diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h
index ba87bd2..3447bea 100644
--- a/include/linux/mlx4/device.h
+++ b/include/linux/mlx4/device.h
@@ -401,6 +401,7 @@
int max_rq_desc_sz;
int max_qp_init_rdma;
int max_qp_dest_rdma;
+ u32 *qp0_qkey;
u32 *qp0_proxy;
u32 *qp1_proxy;
u32 *qp0_tunnel;
@@ -837,7 +838,7 @@
}
int mlx4_buf_alloc(struct mlx4_dev *dev, int size, int max_direct,
- struct mlx4_buf *buf);
+ struct mlx4_buf *buf, gfp_t gfp);
void mlx4_buf_free(struct mlx4_dev *dev, int size, struct mlx4_buf *buf);
static inline void *mlx4_buf_offset(struct mlx4_buf *buf, int offset)
{
@@ -874,9 +875,10 @@
int mlx4_write_mtt(struct mlx4_dev *dev, struct mlx4_mtt *mtt,
int start_index, int npages, u64 *page_list);
int mlx4_buf_write_mtt(struct mlx4_dev *dev, struct mlx4_mtt *mtt,
- struct mlx4_buf *buf);
+ struct mlx4_buf *buf, gfp_t gfp);
-int mlx4_db_alloc(struct mlx4_dev *dev, struct mlx4_db *db, int order);
+int mlx4_db_alloc(struct mlx4_dev *dev, struct mlx4_db *db, int order,
+ gfp_t gfp);
void mlx4_db_free(struct mlx4_dev *dev, struct mlx4_db *db);
int mlx4_alloc_hwq_res(struct mlx4_dev *dev, struct mlx4_hwq_resources *wqres,
@@ -892,7 +894,8 @@
int mlx4_qp_reserve_range(struct mlx4_dev *dev, int cnt, int align, int *base);
void mlx4_qp_release_range(struct mlx4_dev *dev, int base_qpn, int cnt);
-int mlx4_qp_alloc(struct mlx4_dev *dev, int qpn, struct mlx4_qp *qp);
+int mlx4_qp_alloc(struct mlx4_dev *dev, int qpn, struct mlx4_qp *qp,
+ gfp_t gfp);
void mlx4_qp_free(struct mlx4_dev *dev, struct mlx4_qp *qp);
int mlx4_srq_alloc(struct mlx4_dev *dev, u32 pdn, u32 cqn, u16 xrcdn,
@@ -1234,4 +1237,8 @@
int mlx4_get_base_gid_ix(struct mlx4_dev *dev, int slave, int port);
int mlx4_config_vxlan_port(struct mlx4_dev *dev, __be16 udp_port);
+int mlx4_vf_smi_enabled(struct mlx4_dev *dev, int slave, int port);
+int mlx4_vf_get_enable_smi_admin(struct mlx4_dev *dev, int slave, int port);
+int mlx4_vf_set_enable_smi_admin(struct mlx4_dev *dev, int slave, int port,
+ int enable);
#endif /* MLX4_DEVICE_H */
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index 93cef63..2bce4aa 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -427,7 +427,6 @@
u64 size;
u32 key;
u32 pd;
- u32 access;
};
struct mlx5_core_srq {
diff --git a/include/linux/mmc/card.h b/include/linux/mmc/card.h
index b730272..d424b9d 100644
--- a/include/linux/mmc/card.h
+++ b/include/linux/mmc/card.h
@@ -63,12 +63,12 @@
unsigned int power_off_longtime; /* Units: ms */
u8 power_off_notification; /* state */
unsigned int hs_max_dtr;
+ unsigned int hs200_max_dtr;
#define MMC_HIGH_26_MAX_DTR 26000000
#define MMC_HIGH_52_MAX_DTR 52000000
#define MMC_HIGH_DDR_MAX_DTR 52000000
#define MMC_HS200_MAX_DTR 200000000
unsigned int sectors;
- unsigned int card_type;
unsigned int hc_erase_size; /* In sectors */
unsigned int hc_erase_timeout; /* In milliseconds */
unsigned int sec_trim_mult; /* Secure trim multiplier */
@@ -110,6 +110,7 @@
u8 raw_pwr_cl_200_360; /* 237 */
u8 raw_pwr_cl_ddr_52_195; /* 238 */
u8 raw_pwr_cl_ddr_52_360; /* 239 */
+ u8 raw_pwr_cl_ddr_200_360; /* 253 */
u8 raw_bkops_status; /* 246 */
u8 raw_sectors[4]; /* 212 - 4 bytes */
@@ -194,6 +195,7 @@
};
struct mmc_host;
+struct mmc_ios;
struct sdio_func;
struct sdio_func_tuple;
@@ -250,15 +252,11 @@
unsigned int state; /* (our) card state */
#define MMC_STATE_PRESENT (1<<0) /* present in sysfs */
#define MMC_STATE_READONLY (1<<1) /* card is read-only */
-#define MMC_STATE_HIGHSPEED (1<<2) /* card is in high speed mode */
-#define MMC_STATE_BLOCKADDR (1<<3) /* card uses block-addressing */
-#define MMC_STATE_HIGHSPEED_DDR (1<<4) /* card is in high speed mode */
-#define MMC_STATE_ULTRAHIGHSPEED (1<<5) /* card is in ultra high speed mode */
-#define MMC_CARD_SDXC (1<<6) /* card is SDXC */
-#define MMC_CARD_REMOVED (1<<7) /* card has been removed */
-#define MMC_STATE_HIGHSPEED_200 (1<<8) /* card is in HS200 mode */
-#define MMC_STATE_DOING_BKOPS (1<<10) /* card is doing BKOPS */
-#define MMC_STATE_SUSPENDED (1<<11) /* card is suspended */
+#define MMC_STATE_BLOCKADDR (1<<2) /* card uses block-addressing */
+#define MMC_CARD_SDXC (1<<3) /* card is SDXC */
+#define MMC_CARD_REMOVED (1<<4) /* card has been removed */
+#define MMC_STATE_DOING_BKOPS (1<<5) /* card is doing BKOPS */
+#define MMC_STATE_SUSPENDED (1<<6) /* card is suspended */
unsigned int quirks; /* card quirks */
#define MMC_QUIRK_LENIENT_FN0 (1<<0) /* allow SDIO FN0 writes outside of the VS CCCR range */
#define MMC_QUIRK_BLKSZ_FOR_BYTE_MODE (1<<1) /* use func->cur_blksize */
@@ -301,6 +299,7 @@
struct sdio_func_tuple *tuples; /* unknown common tuples */
unsigned int sd_bus_speed; /* Bus Speed Mode set for the card */
+ unsigned int mmc_avail_type; /* supported device type by both host and card */
struct dentry *debugfs_root;
struct mmc_part part[MMC_NUM_PHY_PARTITION]; /* physical partitions */
@@ -353,7 +352,7 @@
#define CID_OEMID_ANY ((unsigned short) -1)
#define CID_NAME_ANY (NULL)
-#define END_FIXUP { 0 }
+#define END_FIXUP { NULL }
#define _FIXUP_EXT(_name, _manfid, _oemid, _rev_start, _rev_end, \
_cis_vendor, _cis_device, \
@@ -418,11 +417,7 @@
#define mmc_card_present(c) ((c)->state & MMC_STATE_PRESENT)
#define mmc_card_readonly(c) ((c)->state & MMC_STATE_READONLY)
-#define mmc_card_highspeed(c) ((c)->state & MMC_STATE_HIGHSPEED)
-#define mmc_card_hs200(c) ((c)->state & MMC_STATE_HIGHSPEED_200)
#define mmc_card_blockaddr(c) ((c)->state & MMC_STATE_BLOCKADDR)
-#define mmc_card_ddr_mode(c) ((c)->state & MMC_STATE_HIGHSPEED_DDR)
-#define mmc_card_uhs(c) ((c)->state & MMC_STATE_ULTRAHIGHSPEED)
#define mmc_card_ext_capacity(c) ((c)->state & MMC_CARD_SDXC)
#define mmc_card_removed(c) ((c) && ((c)->state & MMC_CARD_REMOVED))
#define mmc_card_doing_bkops(c) ((c)->state & MMC_STATE_DOING_BKOPS)
@@ -430,11 +425,7 @@
#define mmc_card_set_present(c) ((c)->state |= MMC_STATE_PRESENT)
#define mmc_card_set_readonly(c) ((c)->state |= MMC_STATE_READONLY)
-#define mmc_card_set_highspeed(c) ((c)->state |= MMC_STATE_HIGHSPEED)
-#define mmc_card_set_hs200(c) ((c)->state |= MMC_STATE_HIGHSPEED_200)
#define mmc_card_set_blockaddr(c) ((c)->state |= MMC_STATE_BLOCKADDR)
-#define mmc_card_set_ddr_mode(c) ((c)->state |= MMC_STATE_HIGHSPEED_DDR)
-#define mmc_card_set_uhs(c) ((c)->state |= MMC_STATE_ULTRAHIGHSPEED)
#define mmc_card_set_ext_capacity(c) ((c)->state |= MMC_CARD_SDXC)
#define mmc_card_set_removed(c) ((c)->state |= MMC_CARD_REMOVED)
#define mmc_card_set_doing_bkops(c) ((c)->state |= MMC_STATE_DOING_BKOPS)
diff --git a/include/linux/mmc/dw_mmc.h b/include/linux/mmc/dw_mmc.h
index 6ce7d2c..babaea9 100644
--- a/include/linux/mmc/dw_mmc.h
+++ b/include/linux/mmc/dw_mmc.h
@@ -248,20 +248,6 @@
/* delay in mS before detecting cards after interrupt */
u32 detect_delay_ms;
- int (*init)(u32 slot_id, irq_handler_t , void *);
- int (*get_ro)(u32 slot_id);
- int (*get_cd)(u32 slot_id);
- int (*get_ocr)(u32 slot_id);
- int (*get_bus_wd)(u32 slot_id);
- /*
- * Enable power to selected slot and set voltage to desired level.
- * Voltage levels are specified using MMC_VDD_xxx defines defined
- * in linux/mmc/host.h file.
- */
- void (*setpower)(u32 slot_id, u32 volt);
- void (*exit)(u32 slot_id);
- void (*select_slot)(u32 slot_id);
-
struct dw_mci_dma_ops *dma_ops;
struct dma_pdata *data;
struct block_settings *blk_settings;
diff --git a/include/linux/mmc/host.h b/include/linux/mmc/host.h
index cb61ea4..7960424 100644
--- a/include/linux/mmc/host.h
+++ b/include/linux/mmc/host.h
@@ -17,6 +17,7 @@
#include <linux/fault-inject.h>
#include <linux/mmc/core.h>
+#include <linux/mmc/card.h>
#include <linux/mmc/pm.h>
struct mmc_ios {
@@ -58,13 +59,9 @@
#define MMC_TIMING_UHS_SDR50 5
#define MMC_TIMING_UHS_SDR104 6
#define MMC_TIMING_UHS_DDR50 7
-#define MMC_TIMING_MMC_HS200 8
-
-#define MMC_SDR_MODE 0
-#define MMC_1_2V_DDR_MODE 1
-#define MMC_1_8V_DDR_MODE 2
-#define MMC_1_2V_SDR_MODE 3
-#define MMC_1_8V_SDR_MODE 4
+#define MMC_TIMING_MMC_DDR52 8
+#define MMC_TIMING_MMC_HS200 9
+#define MMC_TIMING_MMC_HS400 10
unsigned char signal_voltage; /* signalling voltage (1.8V or 3.3V) */
@@ -136,6 +133,9 @@
/* The tuning command opcode value is different for SD and eMMC cards */
int (*execute_tuning)(struct mmc_host *host, u32 opcode);
+
+ /* Prepare HS400 target operating frequency depending host driver */
+ int (*prepare_hs400_tuning)(struct mmc_host *host, struct mmc_ios *ios);
int (*select_drive_strength)(unsigned int max_dtr, int host_drv, int card_drv);
void (*hw_reset)(struct mmc_host *host);
void (*card_event)(struct mmc_host *host);
@@ -278,6 +278,11 @@
#define MMC_CAP2_PACKED_CMD (MMC_CAP2_PACKED_RD | \
MMC_CAP2_PACKED_WR)
#define MMC_CAP2_NO_PRESCAN_POWERUP (1 << 14) /* Don't power up before scan */
+#define MMC_CAP2_HS400_1_8V (1 << 15) /* Can support HS400 1.8V */
+#define MMC_CAP2_HS400_1_2V (1 << 16) /* Can support HS400 1.2V */
+#define MMC_CAP2_HS400 (MMC_CAP2_HS400_1_8V | \
+ MMC_CAP2_HS400_1_2V)
+#define MMC_CAP2_SDIO_IRQ_NOTHREAD (1 << 17)
mmc_pm_flag_t pm_caps; /* supported pm features */
@@ -318,6 +323,8 @@
int rescan_disable; /* disable card detection */
int rescan_entered; /* used with nonremovable devices */
+ bool trigger_card_event; /* card_event necessary */
+
struct mmc_card *card; /* device attached to this host */
wait_queue_head_t wq;
@@ -391,12 +398,13 @@
wake_up_process(host->sdio_irq_thread);
}
+void sdio_run_irqs(struct mmc_host *host);
+
#ifdef CONFIG_REGULATOR
int mmc_regulator_get_ocrmask(struct regulator *supply);
int mmc_regulator_set_ocr(struct mmc_host *mmc,
struct regulator *supply,
unsigned short vdd_bit);
-int mmc_regulator_get_supply(struct mmc_host *mmc);
#else
static inline int mmc_regulator_get_ocrmask(struct regulator *supply)
{
@@ -409,13 +417,10 @@
{
return 0;
}
-
-static inline int mmc_regulator_get_supply(struct mmc_host *mmc)
-{
- return 0;
-}
#endif
+int mmc_regulator_get_supply(struct mmc_host *mmc);
+
int mmc_pm_notify(struct notifier_block *notify_block, unsigned long, void *);
static inline int mmc_card_is_removable(struct mmc_host *host)
@@ -475,4 +480,32 @@
return host->ios.clock;
}
#endif
+
+static inline int mmc_card_hs(struct mmc_card *card)
+{
+ return card->host->ios.timing == MMC_TIMING_SD_HS ||
+ card->host->ios.timing == MMC_TIMING_MMC_HS;
+}
+
+static inline int mmc_card_uhs(struct mmc_card *card)
+{
+ return card->host->ios.timing >= MMC_TIMING_UHS_SDR12 &&
+ card->host->ios.timing <= MMC_TIMING_UHS_DDR50;
+}
+
+static inline bool mmc_card_hs200(struct mmc_card *card)
+{
+ return card->host->ios.timing == MMC_TIMING_MMC_HS200;
+}
+
+static inline bool mmc_card_ddr52(struct mmc_card *card)
+{
+ return card->host->ios.timing == MMC_TIMING_MMC_DDR52;
+}
+
+static inline bool mmc_card_hs400(struct mmc_card *card)
+{
+ return card->host->ios.timing == MMC_TIMING_MMC_HS400;
+}
+
#endif /* LINUX_MMC_HOST_H */
diff --git a/include/linux/mmc/mmc.h b/include/linux/mmc/mmc.h
index 50bcde3..64ec963 100644
--- a/include/linux/mmc/mmc.h
+++ b/include/linux/mmc/mmc.h
@@ -325,6 +325,7 @@
#define EXT_CSD_POWER_OFF_LONG_TIME 247 /* RO */
#define EXT_CSD_GENERIC_CMD6_TIME 248 /* RO */
#define EXT_CSD_CACHE_SIZE 249 /* RO, 4 bytes */
+#define EXT_CSD_PWR_CL_DDR_200_360 253 /* RO */
#define EXT_CSD_TAG_UNIT_SIZE 498 /* RO */
#define EXT_CSD_DATA_TAG_SUPPORT 499 /* RO */
#define EXT_CSD_MAX_PACKED_WRITES 500 /* RO */
@@ -354,18 +355,25 @@
#define EXT_CSD_CMD_SET_SECURE (1<<1)
#define EXT_CSD_CMD_SET_CPSECURE (1<<2)
-#define EXT_CSD_CARD_TYPE_26 (1<<0) /* Card can run at 26MHz */
-#define EXT_CSD_CARD_TYPE_52 (1<<1) /* Card can run at 52MHz */
-#define EXT_CSD_CARD_TYPE_MASK 0x3F /* Mask out reserved bits */
+#define EXT_CSD_CARD_TYPE_HS_26 (1<<0) /* Card can run at 26MHz */
+#define EXT_CSD_CARD_TYPE_HS_52 (1<<1) /* Card can run at 52MHz */
+#define EXT_CSD_CARD_TYPE_HS (EXT_CSD_CARD_TYPE_HS_26 | \
+ EXT_CSD_CARD_TYPE_HS_52)
#define EXT_CSD_CARD_TYPE_DDR_1_8V (1<<2) /* Card can run at 52MHz */
/* DDR mode @1.8V or 3V I/O */
#define EXT_CSD_CARD_TYPE_DDR_1_2V (1<<3) /* Card can run at 52MHz */
/* DDR mode @1.2V I/O */
#define EXT_CSD_CARD_TYPE_DDR_52 (EXT_CSD_CARD_TYPE_DDR_1_8V \
| EXT_CSD_CARD_TYPE_DDR_1_2V)
-#define EXT_CSD_CARD_TYPE_SDR_1_8V (1<<4) /* Card can run at 200MHz */
-#define EXT_CSD_CARD_TYPE_SDR_1_2V (1<<5) /* Card can run at 200MHz */
+#define EXT_CSD_CARD_TYPE_HS200_1_8V (1<<4) /* Card can run at 200MHz */
+#define EXT_CSD_CARD_TYPE_HS200_1_2V (1<<5) /* Card can run at 200MHz */
/* SDR mode @1.2V I/O */
+#define EXT_CSD_CARD_TYPE_HS200 (EXT_CSD_CARD_TYPE_HS200_1_8V | \
+ EXT_CSD_CARD_TYPE_HS200_1_2V)
+#define EXT_CSD_CARD_TYPE_HS400_1_8V (1<<6) /* Card can run at 200MHz DDR, 1.8V */
+#define EXT_CSD_CARD_TYPE_HS400_1_2V (1<<7) /* Card can run at 200MHz DDR, 1.2V */
+#define EXT_CSD_CARD_TYPE_HS400 (EXT_CSD_CARD_TYPE_HS400_1_8V | \
+ EXT_CSD_CARD_TYPE_HS400_1_2V)
#define EXT_CSD_BUS_WIDTH_1 0 /* Card is in 1 bit mode */
#define EXT_CSD_BUS_WIDTH_4 1 /* Card is in 4 bit mode */
@@ -373,6 +381,11 @@
#define EXT_CSD_DDR_BUS_WIDTH_4 5 /* Card is in 4 bit DDR mode */
#define EXT_CSD_DDR_BUS_WIDTH_8 6 /* Card is in 8 bit DDR mode */
+#define EXT_CSD_TIMING_BC 0 /* Backwards compatility */
+#define EXT_CSD_TIMING_HS 1 /* High speed */
+#define EXT_CSD_TIMING_HS200 2 /* HS200 */
+#define EXT_CSD_TIMING_HS400 3 /* HS400 */
+
#define EXT_CSD_SEC_ER_EN BIT(0)
#define EXT_CSD_SEC_BD_BLK_EN BIT(2)
#define EXT_CSD_SEC_GB_CL_EN BIT(4)
diff --git a/include/linux/mmc/sdhci.h b/include/linux/mmc/sdhci.h
index 7be12b88..08abe99 100644
--- a/include/linux/mmc/sdhci.h
+++ b/include/linux/mmc/sdhci.h
@@ -57,12 +57,8 @@
#define SDHCI_QUIRK_BROKEN_CARD_DETECTION (1<<15)
/* Controller reports inverted write-protect state */
#define SDHCI_QUIRK_INVERTED_WRITE_PROTECT (1<<16)
-/* Controller has nonstandard clock management */
-#define SDHCI_QUIRK_NONSTANDARD_CLOCK (1<<17)
/* Controller does not like fast PIO transfers */
#define SDHCI_QUIRK_PIO_NEEDS_DELAY (1<<18)
-/* Controller losing signal/interrupt enable states after reset */
-#define SDHCI_QUIRK_RESTORE_IRQS_AFTER_RESET (1<<19)
/* Controller has to be forced to use block size of 2048 bytes */
#define SDHCI_QUIRK_FORCE_BLK_SZ_2048 (1<<20)
/* Controller cannot do multi-block transfers */
@@ -147,6 +143,7 @@
bool runtime_suspended; /* Host is runtime suspended */
bool bus_on; /* Bus power prevents runtime suspend */
+ bool preset_enabled; /* Preset is enabled */
struct mmc_request *mrq; /* Current request */
struct mmc_command *cmd; /* Current command */
@@ -164,8 +161,7 @@
dma_addr_t adma_addr; /* Mapped ADMA descr. table */
dma_addr_t align_addr; /* Mapped bounce buffer */
- struct tasklet_struct card_tasklet; /* Tasklet structures */
- struct tasklet_struct finish_tasklet;
+ struct tasklet_struct finish_tasklet; /* Tasklet structures */
struct timer_list timer; /* Timer for timeouts */
@@ -177,6 +173,13 @@
unsigned int ocr_avail_mmc;
u32 ocr_mask; /* available voltages */
+ unsigned timing; /* Current timing */
+
+ u32 thread_isr;
+
+ /* cached registers */
+ u32 ier;
+
wait_queue_head_t buf_ready_int; /* Waitqueue for Buffer Read Ready interrupt */
unsigned int tuning_done; /* Condition flag set when CMD19 succeeds */
diff --git a/include/linux/mtd/nand.h b/include/linux/mtd/nand.h
index 450d61e..2f0af28 100644
--- a/include/linux/mtd/nand.h
+++ b/include/linux/mtd/nand.h
@@ -176,6 +176,11 @@
/* Chip may not exist, so silence any errors in scan */
#define NAND_SCAN_SILENT_NODEV 0x00040000
/*
+ * This option could be defined by controller drivers to protect against
+ * kmap'ed, vmalloc'ed highmem buffers being passed from upper layers
+ */
+#define NAND_USE_BOUNCE_BUFFER 0x00080000
+/*
* Autodetect nand buswidth with readid/onfi.
* This suppose the driver will configure the hardware in 8 bits mode
* when calling nand_scan_ident, and update its configuration
@@ -552,8 +557,7 @@
* @ecc: [BOARDSPECIFIC] ECC control structure
* @buffers: buffer structure for read/write
* @hwcontrol: platform-specific hardware control structure
- * @erase_cmd: [INTERN] erase command write function, selectable due
- * to AND support.
+ * @erase: [REPLACEABLE] erase function
* @scan_bbt: [REPLACEABLE] function to scan bad block table
* @chip_delay: [BOARDSPECIFIC] chip dependent delay for transferring
* data from array to read regs (tR).
@@ -637,7 +641,7 @@
void (*cmdfunc)(struct mtd_info *mtd, unsigned command, int column,
int page_addr);
int(*waitfunc)(struct mtd_info *mtd, struct nand_chip *this);
- void (*erase_cmd)(struct mtd_info *mtd, int page);
+ int (*erase)(struct mtd_info *mtd, int page);
int (*scan_bbt)(struct mtd_info *mtd);
int (*errstat)(struct mtd_info *mtd, struct nand_chip *this, int state,
int status, int page);
diff --git a/include/linux/mtd/pfow.h b/include/linux/mtd/pfow.h
index b730d4f..42ff7ff0 100644
--- a/include/linux/mtd/pfow.h
+++ b/include/linux/mtd/pfow.h
@@ -101,9 +101,6 @@
unsigned long len, map_word *datum)
{
int bits_per_chip = map_bankwidth(map) * 8;
- int chipnum;
- struct lpddr_private *lpddr = map->fldrv_priv;
- chipnum = adr >> lpddr->chipshift;
map_write(map, CMD(cmd_code), map->pfow_base + PFOW_COMMAND_CODE);
map_write(map, CMD(adr & ((1<<bits_per_chip) - 1)),
diff --git a/include/linux/mtd/spi-nor.h b/include/linux/mtd/spi-nor.h
new file mode 100644
index 0000000..5324184
--- /dev/null
+++ b/include/linux/mtd/spi-nor.h
@@ -0,0 +1,214 @@
+/*
+ * Copyright (C) 2014 Freescale Semiconductor, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#ifndef __LINUX_MTD_SPI_NOR_H
+#define __LINUX_MTD_SPI_NOR_H
+
+/*
+ * Note on opcode nomenclature: some opcodes have a format like
+ * SPINOR_OP_FUNCTION{4,}_x_y_z. The numbers x, y, and z stand for the number
+ * of I/O lines used for the opcode, address, and data (respectively). The
+ * FUNCTION has an optional suffix of '4', to represent an opcode which
+ * requires a 4-byte (32-bit) address.
+ */
+
+/* Flash opcodes. */
+#define SPINOR_OP_WREN 0x06 /* Write enable */
+#define SPINOR_OP_RDSR 0x05 /* Read status register */
+#define SPINOR_OP_WRSR 0x01 /* Write status register 1 byte */
+#define SPINOR_OP_READ 0x03 /* Read data bytes (low frequency) */
+#define SPINOR_OP_READ_FAST 0x0b /* Read data bytes (high frequency) */
+#define SPINOR_OP_READ_1_1_2 0x3b /* Read data bytes (Dual SPI) */
+#define SPINOR_OP_READ_1_1_4 0x6b /* Read data bytes (Quad SPI) */
+#define SPINOR_OP_PP 0x02 /* Page program (up to 256 bytes) */
+#define SPINOR_OP_BE_4K 0x20 /* Erase 4KiB block */
+#define SPINOR_OP_BE_4K_PMC 0xd7 /* Erase 4KiB block on PMC chips */
+#define SPINOR_OP_BE_32K 0x52 /* Erase 32KiB block */
+#define SPINOR_OP_CHIP_ERASE 0xc7 /* Erase whole flash chip */
+#define SPINOR_OP_SE 0xd8 /* Sector erase (usually 64KiB) */
+#define SPINOR_OP_RDID 0x9f /* Read JEDEC ID */
+#define SPINOR_OP_RDCR 0x35 /* Read configuration register */
+
+/* 4-byte address opcodes - used on Spansion and some Macronix flashes. */
+#define SPINOR_OP_READ4 0x13 /* Read data bytes (low frequency) */
+#define SPINOR_OP_READ4_FAST 0x0c /* Read data bytes (high frequency) */
+#define SPINOR_OP_READ4_1_1_2 0x3c /* Read data bytes (Dual SPI) */
+#define SPINOR_OP_READ4_1_1_4 0x6c /* Read data bytes (Quad SPI) */
+#define SPINOR_OP_PP_4B 0x12 /* Page program (up to 256 bytes) */
+#define SPINOR_OP_SE_4B 0xdc /* Sector erase (usually 64KiB) */
+
+/* Used for SST flashes only. */
+#define SPINOR_OP_BP 0x02 /* Byte program */
+#define SPINOR_OP_WRDI 0x04 /* Write disable */
+#define SPINOR_OP_AAI_WP 0xad /* Auto address increment word program */
+
+/* Used for Macronix and Winbond flashes. */
+#define SPINOR_OP_EN4B 0xb7 /* Enter 4-byte mode */
+#define SPINOR_OP_EX4B 0xe9 /* Exit 4-byte mode */
+
+/* Used for Spansion flashes only. */
+#define SPINOR_OP_BRWR 0x17 /* Bank register write */
+
+/* Status Register bits. */
+#define SR_WIP 1 /* Write in progress */
+#define SR_WEL 2 /* Write enable latch */
+/* meaning of other SR_* bits may differ between vendors */
+#define SR_BP0 4 /* Block protect 0 */
+#define SR_BP1 8 /* Block protect 1 */
+#define SR_BP2 0x10 /* Block protect 2 */
+#define SR_SRWD 0x80 /* SR write protect */
+
+#define SR_QUAD_EN_MX 0x40 /* Macronix Quad I/O */
+
+/* Configuration Register bits. */
+#define CR_QUAD_EN_SPAN 0x2 /* Spansion Quad I/O */
+
+enum read_mode {
+ SPI_NOR_NORMAL = 0,
+ SPI_NOR_FAST,
+ SPI_NOR_DUAL,
+ SPI_NOR_QUAD,
+};
+
+/**
+ * struct spi_nor_xfer_cfg - Structure for defining a Serial Flash transfer
+ * @wren: command for "Write Enable", or 0x00 for not required
+ * @cmd: command for operation
+ * @cmd_pins: number of pins to send @cmd (1, 2, 4)
+ * @addr: address for operation
+ * @addr_pins: number of pins to send @addr (1, 2, 4)
+ * @addr_width: number of address bytes
+ * (3,4, or 0 for address not required)
+ * @mode: mode data
+ * @mode_pins: number of pins to send @mode (1, 2, 4)
+ * @mode_cycles: number of mode cycles (0 for mode not required)
+ * @dummy_cycles: number of dummy cycles (0 for dummy not required)
+ */
+struct spi_nor_xfer_cfg {
+ u8 wren;
+ u8 cmd;
+ u8 cmd_pins;
+ u32 addr;
+ u8 addr_pins;
+ u8 addr_width;
+ u8 mode;
+ u8 mode_pins;
+ u8 mode_cycles;
+ u8 dummy_cycles;
+};
+
+#define SPI_NOR_MAX_CMD_SIZE 8
+enum spi_nor_ops {
+ SPI_NOR_OPS_READ = 0,
+ SPI_NOR_OPS_WRITE,
+ SPI_NOR_OPS_ERASE,
+ SPI_NOR_OPS_LOCK,
+ SPI_NOR_OPS_UNLOCK,
+};
+
+/**
+ * struct spi_nor - Structure for defining a the SPI NOR layer
+ * @mtd: point to a mtd_info structure
+ * @lock: the lock for the read/write/erase/lock/unlock operations
+ * @dev: point to a spi device, or a spi nor controller device.
+ * @page_size: the page size of the SPI NOR
+ * @addr_width: number of address bytes
+ * @erase_opcode: the opcode for erasing a sector
+ * @read_opcode: the read opcode
+ * @read_dummy: the dummy needed by the read operation
+ * @program_opcode: the program opcode
+ * @flash_read: the mode of the read
+ * @sst_write_second: used by the SST write operation
+ * @cfg: used by the read_xfer/write_xfer
+ * @cmd_buf: used by the write_reg
+ * @prepare: [OPTIONAL] do some preparations for the
+ * read/write/erase/lock/unlock operations
+ * @unprepare: [OPTIONAL] do some post work after the
+ * read/write/erase/lock/unlock operations
+ * @read_xfer: [OPTIONAL] the read fundamental primitive
+ * @write_xfer: [OPTIONAL] the writefundamental primitive
+ * @read_reg: [DRIVER-SPECIFIC] read out the register
+ * @write_reg: [DRIVER-SPECIFIC] write data to the register
+ * @read_id: [REPLACEABLE] read out the ID data, and find
+ * the proper spi_device_id
+ * @wait_till_ready: [REPLACEABLE] wait till the NOR becomes ready
+ * @read: [DRIVER-SPECIFIC] read data from the SPI NOR
+ * @write: [DRIVER-SPECIFIC] write data to the SPI NOR
+ * @erase: [DRIVER-SPECIFIC] erase a sector of the SPI NOR
+ * at the offset @offs
+ * @priv: the private data
+ */
+struct spi_nor {
+ struct mtd_info *mtd;
+ struct mutex lock;
+ struct device *dev;
+ u32 page_size;
+ u8 addr_width;
+ u8 erase_opcode;
+ u8 read_opcode;
+ u8 read_dummy;
+ u8 program_opcode;
+ enum read_mode flash_read;
+ bool sst_write_second;
+ struct spi_nor_xfer_cfg cfg;
+ u8 cmd_buf[SPI_NOR_MAX_CMD_SIZE];
+
+ int (*prepare)(struct spi_nor *nor, enum spi_nor_ops ops);
+ void (*unprepare)(struct spi_nor *nor, enum spi_nor_ops ops);
+ int (*read_xfer)(struct spi_nor *nor, struct spi_nor_xfer_cfg *cfg,
+ u8 *buf, size_t len);
+ int (*write_xfer)(struct spi_nor *nor, struct spi_nor_xfer_cfg *cfg,
+ u8 *buf, size_t len);
+ int (*read_reg)(struct spi_nor *nor, u8 opcode, u8 *buf, int len);
+ int (*write_reg)(struct spi_nor *nor, u8 opcode, u8 *buf, int len,
+ int write_enable);
+ const struct spi_device_id *(*read_id)(struct spi_nor *nor);
+ int (*wait_till_ready)(struct spi_nor *nor);
+
+ int (*read)(struct spi_nor *nor, loff_t from,
+ size_t len, size_t *retlen, u_char *read_buf);
+ void (*write)(struct spi_nor *nor, loff_t to,
+ size_t len, size_t *retlen, const u_char *write_buf);
+ int (*erase)(struct spi_nor *nor, loff_t offs);
+
+ void *priv;
+};
+
+/**
+ * spi_nor_scan() - scan the SPI NOR
+ * @nor: the spi_nor structure
+ * @id: the spi_device_id provided by the driver
+ * @mode: the read mode supported by the driver
+ *
+ * The drivers can use this fuction to scan the SPI NOR.
+ * In the scanning, it will try to get all the necessary information to
+ * fill the mtd_info{} and the spi_nor{}.
+ *
+ * The board may assigns a spi_device_id with @id which be used to compared with
+ * the spi_device_id detected by the scanning.
+ *
+ * Return: 0 for success, others for failure.
+ */
+int spi_nor_scan(struct spi_nor *nor, const struct spi_device_id *id,
+ enum read_mode mode);
+extern const struct spi_device_id spi_nor_ids[];
+
+/**
+ * spi_nor_match_id() - find the spi_device_id by the name
+ * @name: the name of the spi_device_id
+ *
+ * The drivers use this function to find the spi_device_id
+ * specified by the @name.
+ *
+ * Return: returns the right spi_device_id pointer on success,
+ * and returns NULL on failure.
+ */
+const struct spi_device_id *spi_nor_match_id(char *name);
+
+#endif
diff --git a/include/linux/netlink.h b/include/linux/netlink.h
index f64b017..034cda7 100644
--- a/include/linux/netlink.h
+++ b/include/linux/netlink.h
@@ -16,9 +16,10 @@
}
enum netlink_skb_flags {
- NETLINK_SKB_MMAPED = 0x1, /* Packet data is mmaped */
- NETLINK_SKB_TX = 0x2, /* Packet was sent by userspace */
- NETLINK_SKB_DELIVERED = 0x4, /* Packet was delivered */
+ NETLINK_SKB_MMAPED = 0x1, /* Packet data is mmaped */
+ NETLINK_SKB_TX = 0x2, /* Packet was sent by userspace */
+ NETLINK_SKB_DELIVERED = 0x4, /* Packet was delivered */
+ NETLINK_SKB_DST = 0x8, /* Dst set in sendto or sendmsg */
};
struct netlink_skb_parms {
diff --git a/include/linux/nfs.h b/include/linux/nfs.h
index 3e794c1..610af51 100644
--- a/include/linux/nfs.h
+++ b/include/linux/nfs.h
@@ -46,6 +46,9 @@
enum nfs3_stable_how {
NFS_UNSTABLE = 0,
NFS_DATA_SYNC = 1,
- NFS_FILE_SYNC = 2
+ NFS_FILE_SYNC = 2,
+
+ /* used by direct.c to mark verf as invalid */
+ NFS_INVALID_STABLE_HOW = -1
};
#endif /* _LINUX_NFS_H */
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 12c2cb9..a1e3064 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -399,8 +399,6 @@
#define FATTR4_WORD2_LAYOUT_BLKSIZE (1UL << 1)
#define FATTR4_WORD2_MDSTHRESHOLD (1UL << 4)
#define FATTR4_WORD2_SECURITY_LABEL (1UL << 16)
-#define FATTR4_WORD2_CHANGE_SECURITY_LABEL \
- (1UL << 17)
/* MDS threshold bitmap bits */
#define THRESHOLD_RD (1UL << 0)
diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h
index fa6918b..919576b8 100644
--- a/include/linux/nfs_fs.h
+++ b/include/linux/nfs_fs.h
@@ -520,7 +520,6 @@
extern int nfs_writepages(struct address_space *, struct writeback_control *);
extern int nfs_flush_incompatible(struct file *file, struct page *page);
extern int nfs_updatepage(struct file *, struct page *, unsigned int, unsigned int);
-extern void nfs_writeback_done(struct rpc_task *, struct nfs_write_data *);
/*
* Try to write back everything synchronously (but check the
@@ -553,7 +552,6 @@
extern int nfs_readpage(struct file *, struct page *);
extern int nfs_readpages(struct file *, struct address_space *,
struct list_head *, unsigned);
-extern int nfs_readpage_result(struct rpc_task *, struct nfs_read_data *);
extern int nfs_readpage_async(struct nfs_open_context *, struct inode *,
struct page *);
diff --git a/include/linux/nfs_page.h b/include/linux/nfs_page.h
index 92ce578..7d9096d 100644
--- a/include/linux/nfs_page.h
+++ b/include/linux/nfs_page.h
@@ -22,12 +22,17 @@
* Valid flags for a dirty buffer
*/
enum {
- PG_BUSY = 0,
- PG_MAPPED,
- PG_CLEAN,
- PG_NEED_COMMIT,
- PG_NEED_RESCHED,
- PG_COMMIT_TO_DS,
+ PG_BUSY = 0, /* nfs_{un}lock_request */
+ PG_MAPPED, /* page private set for buffered io */
+ PG_CLEAN, /* write succeeded */
+ PG_COMMIT_TO_DS, /* used by pnfs layouts */
+ PG_INODE_REF, /* extra ref held by inode (head req only) */
+ PG_HEADLOCK, /* page group lock of wb_head */
+ PG_TEARDOWN, /* page group sync for destroy */
+ PG_UNLOCKPAGE, /* page group sync bit in read path */
+ PG_UPTODATE, /* page group sync bit in read path */
+ PG_WB_END, /* page group sync bit in write path */
+ PG_REMOVE, /* page group sync bit in write path */
};
struct nfs_inode;
@@ -43,15 +48,29 @@
struct kref wb_kref; /* reference count */
unsigned long wb_flags;
struct nfs_write_verifier wb_verf; /* Commit cookie */
+ struct nfs_page *wb_this_page; /* list of reqs for this page */
+ struct nfs_page *wb_head; /* head pointer for req list */
};
struct nfs_pageio_descriptor;
struct nfs_pageio_ops {
void (*pg_init)(struct nfs_pageio_descriptor *, struct nfs_page *);
- bool (*pg_test)(struct nfs_pageio_descriptor *, struct nfs_page *, struct nfs_page *);
+ size_t (*pg_test)(struct nfs_pageio_descriptor *, struct nfs_page *,
+ struct nfs_page *);
int (*pg_doio)(struct nfs_pageio_descriptor *);
};
+struct nfs_rw_ops {
+ const fmode_t rw_mode;
+ struct nfs_rw_header *(*rw_alloc_header)(void);
+ void (*rw_free_header)(struct nfs_rw_header *);
+ void (*rw_release)(struct nfs_pgio_data *);
+ int (*rw_done)(struct rpc_task *, struct nfs_pgio_data *, struct inode *);
+ void (*rw_result)(struct rpc_task *, struct nfs_pgio_data *);
+ void (*rw_initiate)(struct nfs_pgio_data *, struct rpc_message *,
+ struct rpc_task_setup *, int);
+};
+
struct nfs_pageio_descriptor {
struct list_head pg_list;
unsigned long pg_bytes_written;
@@ -63,6 +82,7 @@
struct inode *pg_inode;
const struct nfs_pageio_ops *pg_ops;
+ const struct nfs_rw_ops *pg_rw_ops;
int pg_ioflags;
int pg_error;
const struct rpc_call_ops *pg_rpc_callops;
@@ -75,29 +95,33 @@
#define NFS_WBACK_BUSY(req) (test_bit(PG_BUSY,&(req)->wb_flags))
extern struct nfs_page *nfs_create_request(struct nfs_open_context *ctx,
- struct inode *inode,
struct page *page,
+ struct nfs_page *last,
unsigned int offset,
unsigned int count);
-extern void nfs_release_request(struct nfs_page *req);
+extern void nfs_release_request(struct nfs_page *);
extern void nfs_pageio_init(struct nfs_pageio_descriptor *desc,
struct inode *inode,
const struct nfs_pageio_ops *pg_ops,
const struct nfs_pgio_completion_ops *compl_ops,
+ const struct nfs_rw_ops *rw_ops,
size_t bsize,
int how);
extern int nfs_pageio_add_request(struct nfs_pageio_descriptor *,
struct nfs_page *);
extern void nfs_pageio_complete(struct nfs_pageio_descriptor *desc);
extern void nfs_pageio_cond_complete(struct nfs_pageio_descriptor *, pgoff_t);
-extern bool nfs_generic_pg_test(struct nfs_pageio_descriptor *desc,
+extern size_t nfs_generic_pg_test(struct nfs_pageio_descriptor *desc,
struct nfs_page *prev,
struct nfs_page *req);
extern int nfs_wait_on_request(struct nfs_page *);
extern void nfs_unlock_request(struct nfs_page *req);
-extern void nfs_unlock_and_release_request(struct nfs_page *req);
+extern void nfs_unlock_and_release_request(struct nfs_page *);
+extern void nfs_page_group_lock(struct nfs_page *);
+extern void nfs_page_group_unlock(struct nfs_page *);
+extern bool nfs_page_group_sync_on_bit(struct nfs_page *, unsigned int);
/*
* Lock the page of an asynchronous request
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 6fb5b23..9a1396e 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -489,44 +489,8 @@
};
/*
- * Arguments to the read call.
- */
-struct nfs_readargs {
- struct nfs4_sequence_args seq_args;
- struct nfs_fh * fh;
- struct nfs_open_context *context;
- struct nfs_lock_context *lock_context;
- nfs4_stateid stateid;
- __u64 offset;
- __u32 count;
- unsigned int pgbase;
- struct page ** pages;
-};
-
-struct nfs_readres {
- struct nfs4_sequence_res seq_res;
- struct nfs_fattr * fattr;
- __u32 count;
- int eof;
-};
-
-/*
* Arguments to the write call.
*/
-struct nfs_writeargs {
- struct nfs4_sequence_args seq_args;
- struct nfs_fh * fh;
- struct nfs_open_context *context;
- struct nfs_lock_context *lock_context;
- nfs4_stateid stateid;
- __u64 offset;
- __u32 count;
- enum nfs3_stable_how stable;
- unsigned int pgbase;
- struct page ** pages;
- const u32 * bitmask;
-};
-
struct nfs_write_verifier {
char data[8];
};
@@ -536,12 +500,31 @@
enum nfs3_stable_how committed;
};
-struct nfs_writeres {
+/*
+ * Arguments shared by the read and write call.
+ */
+struct nfs_pgio_args {
+ struct nfs4_sequence_args seq_args;
+ struct nfs_fh * fh;
+ struct nfs_open_context *context;
+ struct nfs_lock_context *lock_context;
+ nfs4_stateid stateid;
+ __u64 offset;
+ __u32 count;
+ unsigned int pgbase;
+ struct page ** pages;
+ const u32 * bitmask; /* used by write */
+ enum nfs3_stable_how stable; /* used by write */
+};
+
+struct nfs_pgio_res {
struct nfs4_sequence_res seq_res;
struct nfs_fattr * fattr;
- struct nfs_writeverf * verf;
__u32 count;
- const struct nfs_server *server;
+ int eof; /* used by read */
+ struct nfs_writeverf * verf; /* used by write */
+ const struct nfs_server *server; /* used by write */
+
};
/*
@@ -1129,6 +1112,7 @@
struct list_head committing;
struct pnfs_layout_segment *wlseg;
struct pnfs_layout_segment *clseg;
+ struct nfs_writeverf direct_verf;
};
struct pnfs_ds_commit_info {
@@ -1264,20 +1248,6 @@
struct page *page_array[NFS_PAGEVEC_SIZE];
};
-struct nfs_read_data {
- struct nfs_pgio_header *header;
- struct list_head list;
- struct rpc_task task;
- struct nfs_fattr fattr; /* fattr storage */
- struct nfs_readargs args;
- struct nfs_readres res;
- unsigned long timestamp; /* For lease renewal */
- int (*read_done_cb) (struct rpc_task *task, struct nfs_read_data *data);
- __u64 mds_offset;
- struct nfs_page_array pages;
- struct nfs_client *ds_clp; /* pNFS data server */
-};
-
/* used as flag bits in nfs_pgio_header */
enum {
NFS_IOHDR_ERROR = 0,
@@ -1287,19 +1257,22 @@
NFS_IOHDR_NEED_RESCHED,
};
+struct nfs_pgio_data;
+
struct nfs_pgio_header {
struct inode *inode;
struct rpc_cred *cred;
struct list_head pages;
- struct list_head rpc_list;
+ struct nfs_pgio_data *data;
atomic_t refcnt;
struct nfs_page *req;
- struct nfs_writeverf *verf;
+ struct nfs_writeverf verf; /* Used for writes */
struct pnfs_layout_segment *lseg;
loff_t io_start;
const struct rpc_call_ops *mds_ops;
void (*release) (struct nfs_pgio_header *hdr);
const struct nfs_pgio_completion_ops *completion_ops;
+ const struct nfs_rw_ops *rw_ops;
struct nfs_direct_req *dreq;
void *layout_private;
spinlock_t lock;
@@ -1310,30 +1283,24 @@
unsigned long flags;
};
-struct nfs_read_header {
- struct nfs_pgio_header header;
- struct nfs_read_data rpc_data;
-};
-
-struct nfs_write_data {
+struct nfs_pgio_data {
struct nfs_pgio_header *header;
- struct list_head list;
struct rpc_task task;
struct nfs_fattr fattr;
- struct nfs_writeverf verf;
- struct nfs_writeargs args; /* argument struct */
- struct nfs_writeres res; /* result struct */
+ struct nfs_writeverf verf; /* Used for writes */
+ struct nfs_pgio_args args; /* argument struct */
+ struct nfs_pgio_res res; /* result struct */
unsigned long timestamp; /* For lease renewal */
- int (*write_done_cb) (struct rpc_task *task, struct nfs_write_data *data);
+ int (*pgio_done_cb) (struct rpc_task *task, struct nfs_pgio_data *data);
__u64 mds_offset; /* Filelayout dense stripe */
struct nfs_page_array pages;
struct nfs_client *ds_clp; /* pNFS data server */
+ int ds_idx; /* ds index if ds_clp is set */
};
-struct nfs_write_header {
+struct nfs_rw_header {
struct nfs_pgio_header header;
- struct nfs_write_data rpc_data;
- struct nfs_writeverf verf;
+ struct nfs_pgio_data rpc_data;
};
struct nfs_mds_commit_info {
@@ -1465,16 +1432,11 @@
struct nfs_pathconf *);
int (*set_capabilities)(struct nfs_server *, struct nfs_fh *);
int (*decode_dirent)(struct xdr_stream *, struct nfs_entry *, int);
- void (*read_setup) (struct nfs_read_data *, struct rpc_message *);
- void (*read_pageio_init)(struct nfs_pageio_descriptor *, struct inode *,
- const struct nfs_pgio_completion_ops *);
- int (*read_rpc_prepare)(struct rpc_task *, struct nfs_read_data *);
- int (*read_done) (struct rpc_task *, struct nfs_read_data *);
- void (*write_setup) (struct nfs_write_data *, struct rpc_message *);
- void (*write_pageio_init)(struct nfs_pageio_descriptor *, struct inode *, int,
- const struct nfs_pgio_completion_ops *);
- int (*write_rpc_prepare)(struct rpc_task *, struct nfs_write_data *);
- int (*write_done) (struct rpc_task *, struct nfs_write_data *);
+ int (*pgio_rpc_prepare)(struct rpc_task *, struct nfs_pgio_data *);
+ void (*read_setup) (struct nfs_pgio_data *, struct rpc_message *);
+ int (*read_done) (struct rpc_task *, struct nfs_pgio_data *);
+ void (*write_setup) (struct nfs_pgio_data *, struct rpc_message *);
+ int (*write_done) (struct rpc_task *, struct nfs_pgio_data *);
void (*commit_setup) (struct nfs_commit_data *, struct rpc_message *);
void (*commit_rpc_prepare)(struct rpc_task *, struct nfs_commit_data *);
int (*commit_done) (struct rpc_task *, struct nfs_commit_data *);
diff --git a/include/linux/nfsd/debug.h b/include/linux/nfsd/debug.h
deleted file mode 100644
index 19ef837..0000000
--- a/include/linux/nfsd/debug.h
+++ /dev/null
@@ -1,19 +0,0 @@
-/*
- * linux/include/linux/nfsd/debug.h
- *
- * Debugging-related stuff for nfsd
- *
- * Copyright (C) 1995 Olaf Kirch <okir@monad.swb.de>
- */
-#ifndef LINUX_NFSD_DEBUG_H
-#define LINUX_NFSD_DEBUG_H
-
-#include <uapi/linux/nfsd/debug.h>
-
-# undef ifdebug
-# ifdef NFSD_DEBUG
-# define ifdebug(flag) if (nfsd_debug & NFSDDBG_##flag)
-# else
-# define ifdebug(flag) if (0)
-# endif
-#endif /* LINUX_NFSD_DEBUG_H */
diff --git a/include/linux/nfsd/nfsfh.h b/include/linux/nfsd/nfsfh.h
deleted file mode 100644
index a93593f1..0000000
--- a/include/linux/nfsd/nfsfh.h
+++ /dev/null
@@ -1,63 +0,0 @@
-/*
- * include/linux/nfsd/nfsfh.h
- *
- * This file describes the layout of the file handles as passed
- * over the wire.
- *
- * Earlier versions of knfsd used to sign file handles using keyed MD5
- * or SHA. I've removed this code, because it doesn't give you more
- * security than blocking external access to port 2049 on your firewall.
- *
- * Copyright (C) 1995, 1996, 1997 Olaf Kirch <okir@monad.swb.de>
- */
-#ifndef _LINUX_NFSD_FH_H
-#define _LINUX_NFSD_FH_H
-
-# include <linux/sunrpc/svc.h>
-#include <uapi/linux/nfsd/nfsfh.h>
-
-static inline __u32 ino_t_to_u32(ino_t ino)
-{
- return (__u32) ino;
-}
-
-static inline ino_t u32_to_ino_t(__u32 uino)
-{
- return (ino_t) uino;
-}
-
-/*
- * This is the internal representation of an NFS handle used in knfsd.
- * pre_mtime/post_version will be used to support wcc_attr's in NFSv3.
- */
-typedef struct svc_fh {
- struct knfsd_fh fh_handle; /* FH data */
- struct dentry * fh_dentry; /* validated dentry */
- struct svc_export * fh_export; /* export pointer */
- int fh_maxsize; /* max size for fh_handle */
-
- unsigned char fh_locked; /* inode locked by us */
- unsigned char fh_want_write; /* remount protection taken */
-
-#ifdef CONFIG_NFSD_V3
- unsigned char fh_post_saved; /* post-op attrs saved */
- unsigned char fh_pre_saved; /* pre-op attrs saved */
-
- /* Pre-op attributes saved during fh_lock */
- __u64 fh_pre_size; /* size before operation */
- struct timespec fh_pre_mtime; /* mtime before oper */
- struct timespec fh_pre_ctime; /* ctime before oper */
- /*
- * pre-op nfsv4 change attr: note must check IS_I_VERSION(inode)
- * to find out if it is valid.
- */
- u64 fh_pre_change;
-
- /* Post-op attributes saved in fh_unlock */
- struct kstat fh_post_attr; /* full attrs after operation */
- u64 fh_post_change; /* nfsv4 change; see above */
-#endif /* CONFIG_NFSD_V3 */
-
-} svc_fh;
-
-#endif /* _LINUX_NFSD_FH_H */
diff --git a/include/linux/of_address.h b/include/linux/of_address.h
index 906ca76..c13b878 100644
--- a/include/linux/of_address.h
+++ b/include/linux/of_address.h
@@ -62,6 +62,9 @@
extern struct of_pci_range *of_pci_range_parser_one(
struct of_pci_range_parser *parser,
struct of_pci_range *range);
+extern int of_dma_get_range(struct device_node *np, u64 *dma_addr,
+ u64 *paddr, u64 *size);
+extern bool of_dma_is_coherent(struct device_node *np);
#else /* CONFIG_OF_ADDRESS */
static inline struct device_node *of_find_matching_node_by_address(
struct device_node *from,
@@ -89,6 +92,17 @@
{
return NULL;
}
+
+static inline int of_dma_get_range(struct device_node *np, u64 *dma_addr,
+ u64 *paddr, u64 *size)
+{
+ return -ENODEV;
+}
+
+static inline bool of_dma_is_coherent(struct device_node *np)
+{
+ return false;
+}
#endif /* CONFIG_OF_ADDRESS */
#ifdef CONFIG_OF
diff --git a/include/linux/omap-dma.h b/include/linux/omap-dma.h
index c29a6de..88e6ea4 100644
--- a/include/linux/omap-dma.h
+++ b/include/linux/omap-dma.h
@@ -1,23 +1,6 @@
-/*
- * OMAP DMA Engine support
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
#ifndef __LINUX_OMAP_DMA_H
#define __LINUX_OMAP_DMA_H
-
-struct dma_chan;
-
-#if defined(CONFIG_DMA_OMAP) || (defined(CONFIG_DMA_OMAP_MODULE) && defined(MODULE))
-bool omap_dma_filter_fn(struct dma_chan *, void *);
-#else
-static inline bool omap_dma_filter_fn(struct dma_chan *c, void *d)
-{
- return false;
-}
-#endif
+#include <linux/omap-dmaengine.h>
/*
* Legacy OMAP DMA handling defines and functions
diff --git a/include/linux/omap-dmaengine.h b/include/linux/omap-dmaengine.h
new file mode 100644
index 0000000..8e6906c
--- /dev/null
+++ b/include/linux/omap-dmaengine.h
@@ -0,0 +1,21 @@
+/*
+ * OMAP DMA Engine support
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#ifndef __LINUX_OMAP_DMAENGINE_H
+#define __LINUX_OMAP_DMAENGINE_H
+
+struct dma_chan;
+
+#if defined(CONFIG_DMA_OMAP) || (defined(CONFIG_DMA_OMAP_MODULE) && defined(MODULE))
+bool omap_dma_filter_fn(struct dma_chan *, void *);
+#else
+static inline bool omap_dma_filter_fn(struct dma_chan *c, void *d)
+{
+ return false;
+}
+#endif
+#endif /* __LINUX_OMAP_DMAENGINE_H */
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 2093eb7..3c545b4 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -319,13 +319,23 @@
extern void cancel_dirty_page(struct page *page, unsigned int account_size);
int test_clear_page_writeback(struct page *page);
-int test_set_page_writeback(struct page *page);
+int __test_set_page_writeback(struct page *page, bool keep_write);
+
+#define test_set_page_writeback(page) \
+ __test_set_page_writeback(page, false)
+#define test_set_page_writeback_keepwrite(page) \
+ __test_set_page_writeback(page, true)
static inline void set_page_writeback(struct page *page)
{
test_set_page_writeback(page);
}
+static inline void set_page_writeback_keepwrite(struct page *page)
+{
+ test_set_page_writeback_keepwrite(page);
+}
+
#ifdef CONFIG_PAGEFLAGS_EXTENDED
/*
* System with lots of page flags available. This allows separate
diff --git a/include/linux/percpu-refcount.h b/include/linux/percpu-refcount.h
index 95961f0..5d8920e 100644
--- a/include/linux/percpu-refcount.h
+++ b/include/linux/percpu-refcount.h
@@ -110,7 +110,7 @@
pcpu_count = ACCESS_ONCE(ref->pcpu_count);
if (likely(REF_STATUS(pcpu_count) == PCPU_REF_PTR))
- __this_cpu_inc(*pcpu_count);
+ this_cpu_inc(*pcpu_count);
else
atomic_inc(&ref->count);
@@ -121,13 +121,10 @@
* percpu_ref_tryget - try to increment a percpu refcount
* @ref: percpu_ref to try-get
*
- * Increment a percpu refcount unless it has already been killed. Returns
- * %true on success; %false on failure.
+ * Increment a percpu refcount unless its count already reached zero.
+ * Returns %true on success; %false on failure.
*
- * Completion of percpu_ref_kill() in itself doesn't guarantee that tryget
- * will fail. For such guarantee, percpu_ref_kill_and_confirm() should be
- * used. After the confirm_kill callback is invoked, it's guaranteed that
- * no new reference will be given out by percpu_ref_tryget().
+ * The caller is responsible for ensuring that @ref stays accessible.
*/
static inline bool percpu_ref_tryget(struct percpu_ref *ref)
{
@@ -139,7 +136,42 @@
pcpu_count = ACCESS_ONCE(ref->pcpu_count);
if (likely(REF_STATUS(pcpu_count) == PCPU_REF_PTR)) {
- __this_cpu_inc(*pcpu_count);
+ this_cpu_inc(*pcpu_count);
+ ret = true;
+ } else {
+ ret = atomic_inc_not_zero(&ref->count);
+ }
+
+ rcu_read_unlock_sched();
+
+ return ret;
+}
+
+/**
+ * percpu_ref_tryget_live - try to increment a live percpu refcount
+ * @ref: percpu_ref to try-get
+ *
+ * Increment a percpu refcount unless it has already been killed. Returns
+ * %true on success; %false on failure.
+ *
+ * Completion of percpu_ref_kill() in itself doesn't guarantee that tryget
+ * will fail. For such guarantee, percpu_ref_kill_and_confirm() should be
+ * used. After the confirm_kill callback is invoked, it's guaranteed that
+ * no new reference will be given out by percpu_ref_tryget().
+ *
+ * The caller is responsible for ensuring that @ref stays accessible.
+ */
+static inline bool percpu_ref_tryget_live(struct percpu_ref *ref)
+{
+ unsigned __percpu *pcpu_count;
+ int ret = false;
+
+ rcu_read_lock_sched();
+
+ pcpu_count = ACCESS_ONCE(ref->pcpu_count);
+
+ if (likely(REF_STATUS(pcpu_count) == PCPU_REF_PTR)) {
+ this_cpu_inc(*pcpu_count);
ret = true;
}
@@ -164,7 +196,7 @@
pcpu_count = ACCESS_ONCE(ref->pcpu_count);
if (likely(REF_STATUS(pcpu_count) == PCPU_REF_PTR))
- __this_cpu_dec(*pcpu_count);
+ this_cpu_dec(*pcpu_count);
else if (unlikely(atomic_dec_and_test(&ref->count)))
ref->release(ref);
diff --git a/include/linux/percpu.h b/include/linux/percpu.h
index 495c654..8419053 100644
--- a/include/linux/percpu.h
+++ b/include/linux/percpu.h
@@ -29,7 +29,7 @@
*/
#define get_cpu_var(var) (*({ \
preempt_disable(); \
- &__get_cpu_var(var); }))
+ this_cpu_ptr(&var); }))
/*
* The weird & is necessary because sparse considers (void)(var) to be
diff --git a/include/linux/platform_data/elm.h b/include/linux/platform_data/elm.h
index 4edb406..780d1e9 100644
--- a/include/linux/platform_data/elm.h
+++ b/include/linux/platform_data/elm.h
@@ -21,6 +21,7 @@
enum bch_ecc {
BCH4_ECC = 0,
BCH8_ECC,
+ BCH16_ECC,
};
/* ELM support 8 error syndrome process */
@@ -38,7 +39,7 @@
bool error_reported;
bool error_uncorrectable;
int error_count;
- int error_loc[ERROR_VECTOR_MAX];
+ int error_loc[16];
};
void elm_decode_bch_error_page(struct device *dev, u8 *ecc_calc,
diff --git a/include/linux/platform_data/intel-mid_wdt.h b/include/linux/platform_data/intel-mid_wdt.h
new file mode 100644
index 0000000..b982534
--- /dev/null
+++ b/include/linux/platform_data/intel-mid_wdt.h
@@ -0,0 +1,22 @@
+/*
+ * intel-mid_wdt: generic Intel MID SCU watchdog driver
+ *
+ * Copyright (C) 2014 Intel Corporation. All rights reserved.
+ * Contact: David Cohen <david.a.cohen@linux.intel.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of version 2 of the GNU General
+ * Public License as published by the Free Software Foundation.
+ */
+
+#ifndef __INTEL_MID_WDT_H__
+#define __INTEL_MID_WDT_H__
+
+#include <linux/platform_device.h>
+
+struct intel_mid_wdt_pdata {
+ int irq;
+ int (*probe)(struct platform_device *pdev);
+};
+
+#endif /*__INTEL_MID_WDT_H__*/
diff --git a/include/linux/platform_data/ipmmu-vmsa.h b/include/linux/platform_data/ipmmu-vmsa.h
new file mode 100644
index 0000000..5275b3a
--- /dev/null
+++ b/include/linux/platform_data/ipmmu-vmsa.h
@@ -0,0 +1,24 @@
+/*
+ * IPMMU VMSA Platform Data
+ *
+ * Copyright (C) 2014 Renesas Electronics Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ */
+
+#ifndef __IPMMU_VMSA_H__
+#define __IPMMU_VMSA_H__
+
+struct ipmmu_vmsa_master {
+ const char *name;
+ unsigned int utlb;
+};
+
+struct ipmmu_vmsa_platform_data {
+ const struct ipmmu_vmsa_master *masters;
+ unsigned int num_masters;
+};
+
+#endif /* __IPMMU_VMSA_H__ */
diff --git a/include/linux/platform_data/mtd-nand-omap2.h b/include/linux/platform_data/mtd-nand-omap2.h
index 3e9dd66..660c029 100644
--- a/include/linux/platform_data/mtd-nand-omap2.h
+++ b/include/linux/platform_data/mtd-nand-omap2.h
@@ -31,6 +31,8 @@
OMAP_ECC_BCH8_CODE_HW_DETECTION_SW,
/* 8-bit ECC calculation by GPMC, Error detection by ELM */
OMAP_ECC_BCH8_CODE_HW,
+ /* 16-bit ECC calculation by GPMC, Error detection by ELM */
+ OMAP_ECC_BCH16_CODE_HW,
};
struct gpmc_nand_regs {
@@ -50,6 +52,9 @@
void __iomem *gpmc_bch_result1[GPMC_BCH_NUM_REMAINDER];
void __iomem *gpmc_bch_result2[GPMC_BCH_NUM_REMAINDER];
void __iomem *gpmc_bch_result3[GPMC_BCH_NUM_REMAINDER];
+ void __iomem *gpmc_bch_result4[GPMC_BCH_NUM_REMAINDER];
+ void __iomem *gpmc_bch_result5[GPMC_BCH_NUM_REMAINDER];
+ void __iomem *gpmc_bch_result6[GPMC_BCH_NUM_REMAINDER];
};
struct omap_nand_platform_data {
diff --git a/include/linux/platform_data/mtd-nand-pxa3xx.h b/include/linux/platform_data/mtd-nand-pxa3xx.h
index a941471..ac4ea2e 100644
--- a/include/linux/platform_data/mtd-nand-pxa3xx.h
+++ b/include/linux/platform_data/mtd-nand-pxa3xx.h
@@ -58,6 +58,9 @@
/* use an flash-based bad block table */
bool flash_bbt;
+ /* requested ECC strength and ECC step size */
+ int ecc_strength, ecc_step_size;
+
const struct mtd_partition *parts[NUM_CHIP_SELECT];
unsigned int nr_parts[NUM_CHIP_SELECT];
diff --git a/include/linux/platform_data/omap4-keypad.h b/include/linux/platform_data/omap4-keypad.h
deleted file mode 100644
index 4eef5fb..0000000
--- a/include/linux/platform_data/omap4-keypad.h
+++ /dev/null
@@ -1,13 +0,0 @@
-#ifndef __LINUX_INPUT_OMAP4_KEYPAD_H
-#define __LINUX_INPUT_OMAP4_KEYPAD_H
-
-#include <linux/input/matrix_keypad.h>
-
-struct omap4_keypad_platform_data {
- const struct matrix_keymap_data *keymap_data;
-
- u8 rows;
- u8 cols;
-};
-
-#endif /* __LINUX_INPUT_OMAP4_KEYPAD_H */
diff --git a/include/linux/ptrace.h b/include/linux/ptrace.h
index 07d0df6..077904c 100644
--- a/include/linux/ptrace.h
+++ b/include/linux/ptrace.h
@@ -5,6 +5,7 @@
#include <linux/sched.h> /* For struct task_struct. */
#include <linux/err.h> /* for IS_ERR_VALUE */
#include <linux/bug.h> /* For BUG_ON. */
+#include <linux/pid_namespace.h> /* For task_active_pid_ns. */
#include <uapi/linux/ptrace.h>
/*
@@ -129,6 +130,37 @@
}
/**
+ * ptrace_event_pid - possibly stop for a ptrace event notification
+ * @event: %PTRACE_EVENT_* value to report
+ * @pid: process identifier for %PTRACE_GETEVENTMSG to return
+ *
+ * Check whether @event is enabled and, if so, report @event and @pid
+ * to the ptrace parent. @pid is reported as the pid_t seen from the
+ * the ptrace parent's pid namespace.
+ *
+ * Called without locks.
+ */
+static inline void ptrace_event_pid(int event, struct pid *pid)
+{
+ /*
+ * FIXME: There's a potential race if a ptracer in a different pid
+ * namespace than parent attaches between computing message below and
+ * when we acquire tasklist_lock in ptrace_stop(). If this happens,
+ * the ptracer will get a bogus pid from PTRACE_GETEVENTMSG.
+ */
+ unsigned long message = 0;
+ struct pid_namespace *ns;
+
+ rcu_read_lock();
+ ns = task_active_pid_ns(rcu_dereference(current->parent));
+ if (ns)
+ message = pid_nr_ns(pid, ns);
+ rcu_read_unlock();
+
+ ptrace_event(event, message);
+}
+
+/**
* ptrace_init_task - initialize ptrace state for a new child
* @child: new child task
* @ptrace: true if child should be ptrace'd by parent's tracer
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 8fcd0e6..ea74596 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -2414,9 +2414,6 @@
extern void do_group_exit(int);
-extern int allow_signal(int);
-extern int disallow_signal(int);
-
extern int do_execve(struct filename *,
const char __user * const __user *,
const char __user * const __user *);
diff --git a/include/linux/security.h b/include/linux/security.h
index 6478ce3..9c6b972 100644
--- a/include/linux/security.h
+++ b/include/linux/security.h
@@ -1708,7 +1708,7 @@
void (*key_free) (struct key *key);
int (*key_permission) (key_ref_t key_ref,
const struct cred *cred,
- key_perm_t perm);
+ unsigned perm);
int (*key_getsecurity)(struct key *key, char **_buffer);
#endif /* CONFIG_KEYS */
@@ -3034,7 +3034,7 @@
int security_key_alloc(struct key *key, const struct cred *cred, unsigned long flags);
void security_key_free(struct key *key);
int security_key_permission(key_ref_t key_ref,
- const struct cred *cred, key_perm_t perm);
+ const struct cred *cred, unsigned perm);
int security_key_getsecurity(struct key *key, char **_buffer);
#else
@@ -3052,7 +3052,7 @@
static inline int security_key_permission(key_ref_t key_ref,
const struct cred *cred,
- key_perm_t perm)
+ unsigned perm)
{
return 0;
}
diff --git a/include/linux/shdma-base.h b/include/linux/shdma-base.h
index f92c0a4..abdf1f2 100644
--- a/include/linux/shdma-base.h
+++ b/include/linux/shdma-base.h
@@ -54,6 +54,7 @@
dma_cookie_t cookie;
int chunks;
int mark;
+ bool cyclic; /* used as cyclic transfer */
};
struct shdma_chan {
diff --git a/include/linux/shm.h b/include/linux/shm.h
index 1e2cd2e..57d7770 100644
--- a/include/linux/shm.h
+++ b/include/linux/shm.h
@@ -3,9 +3,8 @@
#include <asm/page.h>
#include <uapi/linux/shm.h>
-
-#define SHMALL (SHMMAX/PAGE_SIZE*(SHMMNI/16)) /* max shm system wide (pages) */
#include <asm/shmparam.h>
+
struct shmid_kernel /* private to the kernel */
{
struct kern_ipc_perm shm_perm;
diff --git a/include/linux/signal.h b/include/linux/signal.h
index 2ac423b..c9e6536 100644
--- a/include/linux/signal.h
+++ b/include/linux/signal.h
@@ -63,11 +63,6 @@
return 1 & (set->sig[sig / _NSIG_BPW] >> (sig % _NSIG_BPW));
}
-static inline int sigfindinword(unsigned long word)
-{
- return ffz(~word);
-}
-
#endif /* __HAVE_ARCH_SIG_BITOPS */
static inline int sigisemptyset(sigset_t *set)
@@ -289,6 +284,22 @@
extern void signal_setup_done(int failed, struct ksignal *ksig, int stepping);
extern void signal_delivered(int sig, siginfo_t *info, struct k_sigaction *ka, struct pt_regs *regs, int stepping);
extern void exit_signals(struct task_struct *tsk);
+extern void kernel_sigaction(int, __sighandler_t);
+
+static inline void allow_signal(int sig)
+{
+ /*
+ * Kernel threads handle their own signals. Let the signal code
+ * know it'll be handled, so that they don't get converted to
+ * SIGKILL or just silently dropped.
+ */
+ kernel_sigaction(sig, (__force __sighandler_t)2);
+}
+
+static inline void disallow_signal(int sig)
+{
+ kernel_sigaction(sig, SIG_IGN);
+}
/*
* Eventually that'll replace get_signal_to_deliver(); macro for now,
diff --git a/include/linux/smp.h b/include/linux/smp.h
index 633f5ed..34347f2 100644
--- a/include/linux/smp.h
+++ b/include/linux/smp.h
@@ -13,8 +13,6 @@
#include <linux/init.h>
#include <linux/llist.h>
-extern void cpu_idle(void);
-
typedef void (*smp_call_func_t)(void *info);
struct call_single_data {
struct llist_node llist;
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index 04e7632..1bc7cd0 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -244,6 +244,7 @@
struct page * rq_pages[RPCSVC_MAXPAGES];
struct page * *rq_respages; /* points into rq_pages */
struct page * *rq_next_page; /* next reply page to use */
+ struct page * *rq_page_end; /* one past the last page */
struct kvec rq_vec[RPCSVC_MAXPAGES]; /* generally useful.. */
@@ -254,11 +255,15 @@
u32 rq_prot; /* IP protocol */
unsigned short
rq_secure : 1; /* secure port */
+ unsigned short rq_local : 1; /* local request */
void * rq_argp; /* decoded arguments */
void * rq_resp; /* xdr'd results */
void * rq_auth_data; /* flavor-specific data */
-
+ int rq_auth_slack; /* extra space xdr code
+ * should leave in head
+ * for krb5i, krb5p.
+ */
int rq_reserved; /* space on socket outq
* reserved for this request
*/
@@ -454,11 +459,7 @@
*/
static inline void svc_reserve_auth(struct svc_rqst *rqstp, int space)
{
- int added_space = 0;
-
- if (rqstp->rq_authop->flavour)
- added_space = RPC_MAX_AUTH_SIZE;
- svc_reserve(rqstp, space + added_space);
+ svc_reserve(rqstp, space + rqstp->rq_auth_slack);
}
#endif /* SUNRPC_SVC_H */
diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h
index 0b8e3e6..5cf99a01 100644
--- a/include/linux/sunrpc/svc_rdma.h
+++ b/include/linux/sunrpc/svc_rdma.h
@@ -115,14 +115,13 @@
struct list_head frmr_list;
};
struct svc_rdma_req_map {
- struct svc_rdma_fastreg_mr *frmr;
unsigned long count;
union {
struct kvec sge[RPCSVC_MAXPAGES];
struct svc_rdma_chunk_sge ch[RPCSVC_MAXPAGES];
+ unsigned long lkey[RPCSVC_MAXPAGES];
};
};
-#define RDMACTXT_F_FAST_UNREG 1
#define RDMACTXT_F_LAST_CTXT 2
#define SVCRDMA_DEVCAP_FAST_REG 1 /* fast mr registration */
diff --git a/include/linux/sunrpc/svc_xprt.h b/include/linux/sunrpc/svc_xprt.h
index b05963f..7235040 100644
--- a/include/linux/sunrpc/svc_xprt.h
+++ b/include/linux/sunrpc/svc_xprt.h
@@ -24,6 +24,7 @@
void (*xpo_release_rqst)(struct svc_rqst *);
void (*xpo_detach)(struct svc_xprt *);
void (*xpo_free)(struct svc_xprt *);
+ int (*xpo_secure_port)(struct svc_rqst *);
};
struct svc_xprt_class {
@@ -63,6 +64,7 @@
#define XPT_DETACHED 10 /* detached from tempsocks list */
#define XPT_LISTENER 11 /* listening endpoint */
#define XPT_CACHE_AUTH 12 /* cache auth info */
+#define XPT_LOCAL 13 /* connection from loopback interface */
struct svc_serv *xpt_server; /* service for transport */
atomic_t xpt_reserved; /* space on outq that is rsvd */
diff --git a/include/linux/sunrpc/xdr.h b/include/linux/sunrpc/xdr.h
index 15f9204..70c6b92 100644
--- a/include/linux/sunrpc/xdr.h
+++ b/include/linux/sunrpc/xdr.h
@@ -215,6 +215,9 @@
extern void xdr_init_encode(struct xdr_stream *xdr, struct xdr_buf *buf, __be32 *p);
extern __be32 *xdr_reserve_space(struct xdr_stream *xdr, size_t nbytes);
+extern void xdr_commit_encode(struct xdr_stream *xdr);
+extern void xdr_truncate_encode(struct xdr_stream *xdr, size_t len);
+extern int xdr_restrict_buflen(struct xdr_stream *xdr, int newbuflen);
extern void xdr_write_pages(struct xdr_stream *xdr, struct page **pages,
unsigned int base, unsigned int len);
extern unsigned int xdr_stream_pos(const struct xdr_stream *xdr);
diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h
index 3876f0f..fcbfe87 100644
--- a/include/linux/sunrpc/xprt.h
+++ b/include/linux/sunrpc/xprt.h
@@ -24,6 +24,12 @@
#define RPC_MAX_SLOT_TABLE_LIMIT (65536U)
#define RPC_MAX_SLOT_TABLE RPC_MAX_SLOT_TABLE_LIMIT
+#define RPC_CWNDSHIFT (8U)
+#define RPC_CWNDSCALE (1U << RPC_CWNDSHIFT)
+#define RPC_INITCWND RPC_CWNDSCALE
+#define RPC_MAXCWND(xprt) ((xprt)->max_reqs << RPC_CWNDSHIFT)
+#define RPCXPRT_CONGESTED(xprt) ((xprt)->cong >= (xprt)->cwnd)
+
/*
* This describes a timeout strategy
*/
diff --git a/include/linux/suspend.h b/include/linux/suspend.h
index 91d66fd..f76994b 100644
--- a/include/linux/suspend.h
+++ b/include/linux/suspend.h
@@ -327,6 +327,8 @@
extern void hibernation_set_ops(const struct platform_hibernation_ops *ops);
extern int hibernate(void);
extern bool system_entering_hibernation(void);
+asmlinkage int swsusp_save(void);
+extern struct pbe *restore_pblist;
#else /* CONFIG_HIBERNATION */
static inline void register_nosave_region(unsigned long b, unsigned long e) {}
static inline void register_nosave_region_late(unsigned long b, unsigned long e) {}
diff --git a/include/linux/trace_seq.h b/include/linux/trace_seq.h
index a32d86e..1361169 100644
--- a/include/linux/trace_seq.h
+++ b/include/linux/trace_seq.h
@@ -46,6 +46,9 @@
extern void *trace_seq_reserve(struct trace_seq *s, size_t len);
extern int trace_seq_path(struct trace_seq *s, const struct path *path);
+extern int trace_seq_bitmask(struct trace_seq *s, const unsigned long *maskp,
+ int nmaskbits);
+
#else /* CONFIG_TRACING */
static inline int trace_seq_printf(struct trace_seq *s, const char *fmt, ...)
{
@@ -57,6 +60,13 @@
return 0;
}
+static inline int
+trace_seq_bitmask(struct trace_seq *s, const unsigned long *maskp,
+ int nmaskbits)
+{
+ return 0;
+}
+
static inline int trace_print_seq(struct seq_file *m, struct trace_seq *s)
{
return 0;
diff --git a/include/linux/tracepoint.h b/include/linux/tracepoint.h
index 9d30ee4..2e2a5f7 100644
--- a/include/linux/tracepoint.h
+++ b/include/linux/tracepoint.h
@@ -185,6 +185,11 @@
static inline void \
check_trace_callback_type_##name(void (*cb)(data_proto)) \
{ \
+ } \
+ static inline bool \
+ trace_##name##_enabled(void) \
+ { \
+ return static_key_false(&__tracepoint_##name.key); \
}
/*
@@ -230,6 +235,11 @@
} \
static inline void check_trace_callback_type_##name(void (*cb)(data_proto)) \
{ \
+ } \
+ static inline bool \
+ trace_##name##_enabled(void) \
+ { \
+ return false; \
}
#define DEFINE_TRACE_FN(name, reg, unreg)
diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h
index edff2b9..c52f827 100644
--- a/include/linux/uprobes.h
+++ b/include/linux/uprobes.h
@@ -32,6 +32,7 @@
struct mm_struct;
struct inode;
struct notifier_block;
+struct page;
#define UPROBE_HANDLER_REMOVE 1
#define UPROBE_HANDLER_MASK 1
@@ -127,6 +128,8 @@
extern void arch_uprobe_abort_xol(struct arch_uprobe *aup, struct pt_regs *regs);
extern unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs);
extern bool __weak arch_uprobe_ignore(struct arch_uprobe *aup, struct pt_regs *regs);
+extern void __weak arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
+ void *src, unsigned long len);
#else /* !CONFIG_UPROBES */
struct uprobes_state {
};
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index 81022a52..8ec980b 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -86,9 +86,8 @@
* from user space. This allows us to easily determine if the provided
* structure is sized to include various fields.
*/
-#define offsetofend(TYPE, MEMBER) ({ \
- TYPE tmp; \
- offsetof(TYPE, MEMBER) + sizeof(tmp.MEMBER); }) \
+#define offsetofend(TYPE, MEMBER) \
+ (offsetof(TYPE, MEMBER) + sizeof(((TYPE *)0)->MEMBER))
/*
* External user API
diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h
index 1b22c42..a0cc2e9 100644
--- a/include/linux/workqueue.h
+++ b/include/linux/workqueue.h
@@ -56,9 +56,8 @@
WORK_NR_COLORS = (1 << WORK_STRUCT_COLOR_BITS) - 1,
WORK_NO_COLOR = WORK_NR_COLORS,
- /* special cpu IDs */
+ /* not bound to any CPU, prefer the local CPU */
WORK_CPU_UNBOUND = NR_CPUS,
- WORK_CPU_END = NR_CPUS + 1,
/*
* Reserve 7 bits off of pwq pointer w/ debugobjects turned off.
@@ -274,13 +273,6 @@
#define delayed_work_pending(w) \
work_pending(&(w)->work)
-/**
- * work_clear_pending - for internal use only, mark a work item as not pending
- * @work: The work item in question
- */
-#define work_clear_pending(work) \
- clear_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work))
-
/*
* Workqueue flags and constants. For details, please refer to
* Documentation/workqueue.txt.
@@ -340,6 +332,9 @@
* short queue flush time. Don't queue works which can run for too
* long.
*
+ * system_highpri_wq is similar to system_wq but for work items which
+ * require WQ_HIGHPRI.
+ *
* system_long_wq is similar to system_wq but may host long running
* works. Queue flushing might take relatively long.
*
@@ -358,26 +353,13 @@
* 'wq_power_efficient' is disabled. See WQ_POWER_EFFICIENT for more info.
*/
extern struct workqueue_struct *system_wq;
+extern struct workqueue_struct *system_highpri_wq;
extern struct workqueue_struct *system_long_wq;
extern struct workqueue_struct *system_unbound_wq;
extern struct workqueue_struct *system_freezable_wq;
extern struct workqueue_struct *system_power_efficient_wq;
extern struct workqueue_struct *system_freezable_power_efficient_wq;
-static inline struct workqueue_struct * __deprecated __system_nrt_wq(void)
-{
- return system_wq;
-}
-
-static inline struct workqueue_struct * __deprecated __system_nrt_freezable_wq(void)
-{
- return system_freezable_wq;
-}
-
-/* equivlalent to system_wq and system_freezable_wq, deprecated */
-#define system_nrt_wq __system_nrt_wq()
-#define system_nrt_freezable_wq __system_nrt_freezable_wq()
-
extern struct workqueue_struct *
__alloc_workqueue_key(const char *fmt, unsigned int flags, int max_active,
struct lock_class_key *key, const char *lock_name, ...) __printf(1, 6);
@@ -587,18 +569,6 @@
return system_wq != NULL;
}
-/* used to be different but now identical to flush_work(), deprecated */
-static inline bool __deprecated flush_work_sync(struct work_struct *work)
-{
- return flush_work(work);
-}
-
-/* used to be different but now identical to flush_delayed_work(), deprecated */
-static inline bool __deprecated flush_delayed_work_sync(struct delayed_work *dwork)
-{
- return flush_delayed_work(dwork);
-}
-
#ifndef CONFIG_SMP
static inline long work_on_cpu(int cpu, long (*fn)(void *), void *arg)
{
diff --git a/include/media/adv7604.h b/include/media/adv7604.h
index c6b3937..aa1c447 100644
--- a/include/media/adv7604.h
+++ b/include/media/adv7604.h
@@ -32,14 +32,18 @@
ADV7604_AIN9_4_5_6_SYNC_2_1 = 4,
};
-/* Bus rotation and reordering (IO register 0x04, [7:5]) */
-enum adv7604_op_ch_sel {
- ADV7604_OP_CH_SEL_GBR = 0,
- ADV7604_OP_CH_SEL_GRB = 1,
- ADV7604_OP_CH_SEL_BGR = 2,
- ADV7604_OP_CH_SEL_RGB = 3,
- ADV7604_OP_CH_SEL_BRG = 4,
- ADV7604_OP_CH_SEL_RBG = 5,
+/*
+ * Bus rotation and reordering. This is used to specify component reordering on
+ * the board and describes the components order on the bus when the ADV7604
+ * outputs RGB.
+ */
+enum adv7604_bus_order {
+ ADV7604_BUS_ORDER_RGB, /* No operation */
+ ADV7604_BUS_ORDER_GRB, /* Swap 1-2 */
+ ADV7604_BUS_ORDER_RBG, /* Swap 2-3 */
+ ADV7604_BUS_ORDER_BGR, /* Swap 1-3 */
+ ADV7604_BUS_ORDER_BRG, /* Rotate right */
+ ADV7604_BUS_ORDER_GBR, /* Rotate left */
};
/* Input Color Space (IO register 0x02, [7:4]) */
@@ -55,29 +59,11 @@
ADV7604_INP_COLOR_SPACE_AUTO = 0xf,
};
-/* Select output format (IO register 0x03, [7:0]) */
-enum adv7604_op_format_sel {
- ADV7604_OP_FORMAT_SEL_SDR_ITU656_8 = 0x00,
- ADV7604_OP_FORMAT_SEL_SDR_ITU656_10 = 0x01,
- ADV7604_OP_FORMAT_SEL_SDR_ITU656_12_MODE0 = 0x02,
- ADV7604_OP_FORMAT_SEL_SDR_ITU656_12_MODE1 = 0x06,
- ADV7604_OP_FORMAT_SEL_SDR_ITU656_12_MODE2 = 0x0a,
- ADV7604_OP_FORMAT_SEL_DDR_422_8 = 0x20,
- ADV7604_OP_FORMAT_SEL_DDR_422_10 = 0x21,
- ADV7604_OP_FORMAT_SEL_DDR_422_12_MODE0 = 0x22,
- ADV7604_OP_FORMAT_SEL_DDR_422_12_MODE1 = 0x23,
- ADV7604_OP_FORMAT_SEL_DDR_422_12_MODE2 = 0x24,
- ADV7604_OP_FORMAT_SEL_SDR_444_24 = 0x40,
- ADV7604_OP_FORMAT_SEL_SDR_444_30 = 0x41,
- ADV7604_OP_FORMAT_SEL_SDR_444_36_MODE0 = 0x42,
- ADV7604_OP_FORMAT_SEL_DDR_444_24 = 0x60,
- ADV7604_OP_FORMAT_SEL_DDR_444_30 = 0x61,
- ADV7604_OP_FORMAT_SEL_DDR_444_36 = 0x62,
- ADV7604_OP_FORMAT_SEL_SDR_ITU656_16 = 0x80,
- ADV7604_OP_FORMAT_SEL_SDR_ITU656_20 = 0x81,
- ADV7604_OP_FORMAT_SEL_SDR_ITU656_24_MODE0 = 0x82,
- ADV7604_OP_FORMAT_SEL_SDR_ITU656_24_MODE1 = 0x86,
- ADV7604_OP_FORMAT_SEL_SDR_ITU656_24_MODE2 = 0x8a,
+/* Select output format (IO register 0x03, [4:2]) */
+enum adv7604_op_format_mode_sel {
+ ADV7604_OP_FORMAT_MODE0 = 0x00,
+ ADV7604_OP_FORMAT_MODE1 = 0x04,
+ ADV7604_OP_FORMAT_MODE2 = 0x08,
};
enum adv7604_drive_strength {
@@ -86,6 +72,30 @@
ADV7604_DR_STR_HIGH = 3,
};
+enum adv7604_int1_config {
+ ADV7604_INT1_CONFIG_OPEN_DRAIN,
+ ADV7604_INT1_CONFIG_ACTIVE_LOW,
+ ADV7604_INT1_CONFIG_ACTIVE_HIGH,
+ ADV7604_INT1_CONFIG_DISABLED,
+};
+
+enum adv7604_page {
+ ADV7604_PAGE_IO,
+ ADV7604_PAGE_AVLINK,
+ ADV7604_PAGE_CEC,
+ ADV7604_PAGE_INFOFRAME,
+ ADV7604_PAGE_ESDP,
+ ADV7604_PAGE_DPP,
+ ADV7604_PAGE_AFE,
+ ADV7604_PAGE_REP,
+ ADV7604_PAGE_EDID,
+ ADV7604_PAGE_HDMI,
+ ADV7604_PAGE_TEST,
+ ADV7604_PAGE_CP,
+ ADV7604_PAGE_VDP,
+ ADV7604_PAGE_MAX,
+};
+
/* Platform dependent definition */
struct adv7604_platform_data {
/* DIS_PWRDNB: 1 if the PWRDNB pin is unused and unconnected */
@@ -94,30 +104,34 @@
/* DIS_CABLE_DET_RST: 1 if the 5V pins are unused and unconnected */
unsigned disable_cable_det_rst:1;
+ int default_input;
+
/* Analog input muxing mode */
enum adv7604_ain_sel ain_sel;
/* Bus rotation and reordering */
- enum adv7604_op_ch_sel op_ch_sel;
+ enum adv7604_bus_order bus_order;
- /* Select output format */
- enum adv7604_op_format_sel op_format_sel;
+ /* Select output format mode */
+ enum adv7604_op_format_mode_sel op_format_mode_sel;
+
+ /* Configuration of the INT1 pin */
+ enum adv7604_int1_config int1_config;
/* IO register 0x02 */
unsigned alt_gamma:1;
unsigned op_656_range:1;
- unsigned rgb_out:1;
unsigned alt_data_sat:1;
/* IO register 0x05 */
unsigned blank_data:1;
unsigned insert_av_codes:1;
unsigned replicate_av_codes:1;
- unsigned invert_cbcr:1;
/* IO register 0x06 */
unsigned inv_vs_pol:1;
unsigned inv_hs_pol:1;
+ unsigned inv_llc_pol:1;
/* IO register 0x14 */
enum adv7604_drive_strength dr_str_data;
@@ -131,34 +145,22 @@
unsigned hdmi_free_run_mode;
/* i2c addresses: 0 == use default */
- u8 i2c_avlink;
- u8 i2c_cec;
- u8 i2c_infoframe;
- u8 i2c_esdp;
- u8 i2c_dpp;
- u8 i2c_afe;
- u8 i2c_repeater;
- u8 i2c_edid;
- u8 i2c_hdmi;
- u8 i2c_test;
- u8 i2c_cp;
- u8 i2c_vdp;
+ u8 i2c_addresses[ADV7604_PAGE_MAX];
};
-enum adv7604_input_port {
- ADV7604_INPUT_HDMI_PORT_A,
- ADV7604_INPUT_HDMI_PORT_B,
- ADV7604_INPUT_HDMI_PORT_C,
- ADV7604_INPUT_HDMI_PORT_D,
- ADV7604_INPUT_VGA_RGB,
- ADV7604_INPUT_VGA_COMP,
+enum adv7604_pad {
+ ADV7604_PAD_HDMI_PORT_A = 0,
+ ADV7604_PAD_HDMI_PORT_B = 1,
+ ADV7604_PAD_HDMI_PORT_C = 2,
+ ADV7604_PAD_HDMI_PORT_D = 3,
+ ADV7604_PAD_VGA_RGB = 4,
+ ADV7604_PAD_VGA_COMP = 5,
+ /* The source pad is either 1 (ADV7611) or 6 (ADV7604) */
+ ADV7604_PAD_SOURCE = 6,
+ ADV7611_PAD_SOURCE = 1,
+ ADV7604_PAD_MAX = 7,
};
-#define ADV7604_EDID_PORT_A 0
-#define ADV7604_EDID_PORT_B 1
-#define ADV7604_EDID_PORT_C 2
-#define ADV7604_EDID_PORT_D 3
-
#define V4L2_CID_ADV_RX_ANALOG_SAMPLING_PHASE (V4L2_CID_DV_CLASS_BASE + 0x1000)
#define V4L2_CID_ADV_RX_FREE_RUN_COLOR_MANUAL (V4L2_CID_DV_CLASS_BASE + 0x1001)
#define V4L2_CID_ADV_RX_FREE_RUN_COLOR (V4L2_CID_DV_CLASS_BASE + 0x1002)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 9fab013..d746572 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -338,12 +338,8 @@
struct v4l2_dv_timings *timings);
int (*g_dv_timings)(struct v4l2_subdev *sd,
struct v4l2_dv_timings *timings);
- int (*enum_dv_timings)(struct v4l2_subdev *sd,
- struct v4l2_enum_dv_timings *timings);
int (*query_dv_timings)(struct v4l2_subdev *sd,
struct v4l2_dv_timings *timings);
- int (*dv_timings_cap)(struct v4l2_subdev *sd,
- struct v4l2_dv_timings_cap *cap);
int (*enum_mbus_fmt)(struct v4l2_subdev *sd, unsigned int index,
enum v4l2_mbus_pixelcode *code);
int (*enum_mbus_fsizes)(struct v4l2_subdev *sd,
diff --git a/include/net/inetpeer.h b/include/net/inetpeer.h
index 6efe73c..058271b 100644
--- a/include/net/inetpeer.h
+++ b/include/net/inetpeer.h
@@ -177,16 +177,9 @@
/* can be called with or without local BH being disabled */
static inline int inet_getid(struct inet_peer *p, int more)
{
- int old, new;
more++;
inet_peer_refcheck(p);
- do {
- old = atomic_read(&p->ip_id_count);
- new = old + more;
- if (!new)
- new = 1;
- } while (atomic_cmpxchg(&p->ip_id_count, old, new) != old);
- return new;
+ return atomic_add_return(more, &p->ip_id_count) - more;
}
#endif /* _NET_INETPEER_H */
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index acd8251..7ccef34 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -80,8 +80,8 @@
RDMA_TRANSPORT_USNIC_UDP
};
-enum rdma_transport_type
-rdma_node_get_transport(enum rdma_node_type node_type) __attribute_const__;
+__attribute_const__ enum rdma_transport_type
+rdma_node_get_transport(enum rdma_node_type node_type);
enum rdma_link_layer {
IB_LINK_LAYER_UNSPECIFIED,
@@ -466,14 +466,14 @@
* converted to 2, since 5 Gbit/sec is 2 * 2.5 Gbit/sec.
* @rate: rate to convert.
*/
-int ib_rate_to_mult(enum ib_rate rate) __attribute_const__;
+__attribute_const__ int ib_rate_to_mult(enum ib_rate rate);
/**
* ib_rate_to_mbps - Convert the IB rate enum to Mbps.
* For example, IB_RATE_2_5_GBPS will be converted to 2500.
* @rate: rate to convert.
*/
-int ib_rate_to_mbps(enum ib_rate rate) __attribute_const__;
+__attribute_const__ int ib_rate_to_mbps(enum ib_rate rate);
enum ib_mr_create_flags {
IB_MR_SIGNATURE_EN = 1,
@@ -604,7 +604,7 @@
* enum.
* @mult: multiple to convert.
*/
-enum ib_rate mult_to_ib_rate(int mult) __attribute_const__;
+__attribute_const__ enum ib_rate mult_to_ib_rate(int mult);
struct ib_ah_attr {
struct ib_global_route grh;
@@ -783,6 +783,7 @@
IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK = 1 << 1,
IB_QP_CREATE_NETIF_QP = 1 << 5,
IB_QP_CREATE_SIGNATURE_EN = 1 << 6,
+ IB_QP_CREATE_USE_GFP_NOIO = 1 << 7,
/* reserve bits 26-31 for low level drivers' internal use */
IB_QP_CREATE_RESERVED_START = 1 << 26,
IB_QP_CREATE_RESERVED_END = 1 << 31,
diff --git a/include/rdma/iw_portmap.h b/include/rdma/iw_portmap.h
new file mode 100644
index 0000000..928b277
--- /dev/null
+++ b/include/rdma/iw_portmap.h
@@ -0,0 +1,199 @@
+/*
+ * Copyright (c) 2014 Intel Corporation. All rights reserved.
+ * Copyright (c) 2014 Chelsio, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+#ifndef _IW_PORTMAP_H
+#define _IW_PORTMAP_H
+
+#define IWPM_ULIBNAME_SIZE 32
+#define IWPM_DEVNAME_SIZE 32
+#define IWPM_IFNAME_SIZE 16
+#define IWPM_IPADDR_SIZE 16
+
+enum {
+ IWPM_INVALID_NLMSG_ERR = 10,
+ IWPM_CREATE_MAPPING_ERR,
+ IWPM_DUPLICATE_MAPPING_ERR,
+ IWPM_UNKNOWN_MAPPING_ERR,
+ IWPM_CLIENT_DEV_INFO_ERR,
+ IWPM_USER_LIB_INFO_ERR,
+ IWPM_REMOTE_QUERY_REJECT
+};
+
+struct iwpm_dev_data {
+ char dev_name[IWPM_DEVNAME_SIZE];
+ char if_name[IWPM_IFNAME_SIZE];
+};
+
+struct iwpm_sa_data {
+ struct sockaddr_storage loc_addr;
+ struct sockaddr_storage mapped_loc_addr;
+ struct sockaddr_storage rem_addr;
+ struct sockaddr_storage mapped_rem_addr;
+};
+
+/**
+ * iwpm_init - Allocate resources for the iwarp port mapper
+ *
+ * Should be called when network interface goes up.
+ */
+int iwpm_init(u8);
+
+/**
+ * iwpm_exit - Deallocate resources for the iwarp port mapper
+ *
+ * Should be called when network interface goes down.
+ */
+int iwpm_exit(u8);
+
+/**
+ * iwpm_valid_pid - Check if the userspace iwarp port mapper pid is valid
+ *
+ * Returns true if the pid is greater than zero, otherwise returns false
+ */
+int iwpm_valid_pid(void);
+
+/**
+ * iwpm_register_pid - Send a netlink query to userspace
+ * to get the iwarp port mapper pid
+ * @pm_msg: Contains driver info to send to the userspace port mapper
+ * @nl_client: The index of the netlink client
+ */
+int iwpm_register_pid(struct iwpm_dev_data *pm_msg, u8 nl_client);
+
+/**
+ * iwpm_add_mapping - Send a netlink add mapping request to
+ * the userspace port mapper
+ * @pm_msg: Contains the local ip/tcp address info to send
+ * @nl_client: The index of the netlink client
+ *
+ * If the request is successful, the pm_msg stores
+ * the port mapper response (mapped address info)
+ */
+int iwpm_add_mapping(struct iwpm_sa_data *pm_msg, u8 nl_client);
+
+/**
+ * iwpm_add_and_query_mapping - Send a netlink add and query mapping request
+ * to the userspace port mapper
+ * @pm_msg: Contains the local and remote ip/tcp address info to send
+ * @nl_client: The index of the netlink client
+ *
+ * If the request is successful, the pm_msg stores the
+ * port mapper response (mapped local and remote address info)
+ */
+int iwpm_add_and_query_mapping(struct iwpm_sa_data *pm_msg, u8 nl_client);
+
+/**
+ * iwpm_remove_mapping - Send a netlink remove mapping request
+ * to the userspace port mapper
+ *
+ * @local_addr: Local ip/tcp address to remove
+ * @nl_client: The index of the netlink client
+ */
+int iwpm_remove_mapping(struct sockaddr_storage *local_addr, u8 nl_client);
+
+/**
+ * iwpm_register_pid_cb - Process the port mapper response to
+ * iwpm_register_pid query
+ * @skb:
+ * @cb: Contains the received message (payload and netlink header)
+ *
+ * If successful, the function receives the userspace port mapper pid
+ * which is used in future communication with the port mapper
+ */
+int iwpm_register_pid_cb(struct sk_buff *, struct netlink_callback *);
+
+/**
+ * iwpm_add_mapping_cb - Process the port mapper response to
+ * iwpm_add_mapping request
+ * @skb:
+ * @cb: Contains the received message (payload and netlink header)
+ */
+int iwpm_add_mapping_cb(struct sk_buff *, struct netlink_callback *);
+
+/**
+ * iwpm_add_and_query_mapping_cb - Process the port mapper response to
+ * iwpm_add_and_query_mapping request
+ * @skb:
+ * @cb: Contains the received message (payload and netlink header)
+ */
+int iwpm_add_and_query_mapping_cb(struct sk_buff *, struct netlink_callback *);
+
+/**
+ * iwpm_mapping_error_cb - Process port mapper notification for error
+ *
+ * @skb:
+ * @cb: Contains the received message (payload and netlink header)
+ */
+int iwpm_mapping_error_cb(struct sk_buff *, struct netlink_callback *);
+
+/**
+ * iwpm_mapping_info_cb - Process a notification that the userspace
+ * port mapper daemon is started
+ * @skb:
+ * @cb: Contains the received message (payload and netlink header)
+ *
+ * Using the received port mapper pid, send all the local mapping
+ * info records to the userspace port mapper
+ */
+int iwpm_mapping_info_cb(struct sk_buff *, struct netlink_callback *);
+
+/**
+ * iwpm_ack_mapping_info_cb - Process the port mapper ack for
+ * the provided local mapping info records
+ * @skb:
+ * @cb: Contains the received message (payload and netlink header)
+ */
+int iwpm_ack_mapping_info_cb(struct sk_buff *, struct netlink_callback *);
+
+/**
+ * iwpm_create_mapinfo - Store local and mapped IPv4/IPv6 address
+ * info in a hash table
+ * @local_addr: Local ip/tcp address
+ * @mapped_addr: Mapped local ip/tcp address
+ * @nl_client: The index of the netlink client
+ */
+int iwpm_create_mapinfo(struct sockaddr_storage *local_addr,
+ struct sockaddr_storage *mapped_addr, u8 nl_client);
+
+/**
+ * iwpm_remove_mapinfo - Remove local and mapped IPv4/IPv6 address
+ * info from the hash table
+ * @local_addr: Local ip/tcp address
+ * @mapped_addr: Mapped local ip/tcp address
+ *
+ * Returns err code if mapping info is not found in the hash table,
+ * otherwise returns 0
+ */
+int iwpm_remove_mapinfo(struct sockaddr_storage *local_addr,
+ struct sockaddr_storage *mapped_addr);
+
+#endif /* _IW_PORTMAP_H */
diff --git a/include/rdma/rdma_netlink.h b/include/rdma/rdma_netlink.h
index e38de79..0790882 100644
--- a/include/rdma/rdma_netlink.h
+++ b/include/rdma/rdma_netlink.h
@@ -43,7 +43,7 @@
* Returns the allocated buffer on success and NULL on failure.
*/
void *ibnl_put_msg(struct sk_buff *skb, struct nlmsghdr **nlh, int seq,
- int len, int client, int op);
+ int len, int client, int op, int flags);
/**
* Put a new attribute in a supplied skb.
* @skb: The netlink skb.
@@ -56,4 +56,25 @@
int ibnl_put_attr(struct sk_buff *skb, struct nlmsghdr *nlh,
int len, void *data, int type);
+/**
+ * Send the supplied skb to a specific userspace PID.
+ * @skb: The netlink skb
+ * @nlh: Header of the netlink message to send
+ * @pid: Userspace netlink process ID
+ * Returns 0 on success or a negative error code.
+ */
+int ibnl_unicast(struct sk_buff *skb, struct nlmsghdr *nlh,
+ __u32 pid);
+
+/**
+ * Send the supplied skb to a netlink group.
+ * @skb: The netlink skb
+ * @nlh: Header of the netlink message to send
+ * @group: Netlink group ID
+ * @flags: allocation flags
+ * Returns 0 on success or a negative error code.
+ */
+int ibnl_multicast(struct sk_buff *skb, struct nlmsghdr *nlh,
+ unsigned int group, gfp_t flags);
+
#endif /* _RDMA_NETLINK_H */
diff --git a/include/scsi/osd_protocol.h b/include/scsi/osd_protocol.h
index 25ac628..a2594af 100644
--- a/include/scsi/osd_protocol.h
+++ b/include/scsi/osd_protocol.h
@@ -263,16 +263,16 @@
* Ex name = FORMAT_OSD we have OSD_ACT_FORMAT_OSD && OSDv1_ACT_FORMAT_OSD
*/
#define OSD_ACT___(Name, Num) \
- OSD_ACT_##Name = __constant_cpu_to_be16(0x8880 + Num), \
- OSDv1_ACT_##Name = __constant_cpu_to_be16(0x8800 + Num),
+ OSD_ACT_##Name = cpu_to_be16(0x8880 + Num), \
+ OSDv1_ACT_##Name = cpu_to_be16(0x8800 + Num),
/* V2 only actions */
#define OSD_ACT_V2(Name, Num) \
- OSD_ACT_##Name = __constant_cpu_to_be16(0x8880 + Num),
+ OSD_ACT_##Name = cpu_to_be16(0x8880 + Num),
#define OSD_ACT_V1_V2(Name, Num1, Num2) \
- OSD_ACT_##Name = __constant_cpu_to_be16(Num2), \
- OSDv1_ACT_##Name = __constant_cpu_to_be16(Num1),
+ OSD_ACT_##Name = cpu_to_be16(Num2), \
+ OSDv1_ACT_##Name = cpu_to_be16(Num1),
enum osd_service_actions {
OSD_ACT_V2(OBJECT_STRUCTURE_CHECK, 0x00)
diff --git a/include/scsi/scsi_cmnd.h b/include/scsi/scsi_cmnd.h
index dd7c998..e016e2a 100644
--- a/include/scsi/scsi_cmnd.h
+++ b/include/scsi/scsi_cmnd.h
@@ -133,6 +133,15 @@
unsigned char tag; /* SCSI-II queued command tag */
};
+/*
+ * Return the driver private allocation behind the command.
+ * Only works if cmd_size is set in the host template.
+ */
+static inline void *scsi_cmd_priv(struct scsi_cmnd *cmd)
+{
+ return cmd + 1;
+}
+
/* make sure not to use it with REQ_TYPE_BLOCK_PC commands */
static inline struct scsi_driver *scsi_cmd_to_driver(struct scsi_cmnd *cmd)
{
diff --git a/include/scsi/scsi_driver.h b/include/scsi/scsi_driver.h
index 20fdfc2..36c4114 100644
--- a/include/scsi/scsi_driver.h
+++ b/include/scsi/scsi_driver.h
@@ -4,17 +4,17 @@
#include <linux/device.h>
struct module;
+struct request;
struct scsi_cmnd;
struct scsi_device;
-struct request;
-struct request_queue;
-
struct scsi_driver {
struct module *owner;
struct device_driver gendrv;
void (*rescan)(struct device *);
+ int (*init_command)(struct scsi_cmnd *);
+ void (*uninit_command)(struct scsi_cmnd *);
int (*done)(struct scsi_cmnd *);
int (*eh_action)(struct scsi_cmnd *, int);
};
@@ -31,8 +31,5 @@
int scsi_setup_blk_pc_cmnd(struct scsi_device *sdev, struct request *req);
int scsi_setup_fs_cmnd(struct scsi_device *sdev, struct request *req);
-int scsi_prep_state_check(struct scsi_device *sdev, struct request *req);
-int scsi_prep_return(struct request_queue *q, struct request *req, int ret);
-int scsi_prep_fn(struct request_queue *, struct request *);
#endif /* _SCSI_SCSI_DRIVER_H */
diff --git a/include/trace/events/ext4.h b/include/trace/events/ext4.h
index 6a1a024..d4f70a7 100644
--- a/include/trace/events/ext4.h
+++ b/include/trace/events/ext4.h
@@ -36,7 +36,7 @@
#define show_map_flags(flags) __print_flags(flags, "|", \
{ EXT4_GET_BLOCKS_CREATE, "CREATE" }, \
- { EXT4_GET_BLOCKS_UNINIT_EXT, "UNINIT" }, \
+ { EXT4_GET_BLOCKS_UNWRIT_EXT, "UNWRIT" }, \
{ EXT4_GET_BLOCKS_DELALLOC_RESERVE, "DELALLOC" }, \
{ EXT4_GET_BLOCKS_PRE_IO, "PRE_IO" }, \
{ EXT4_GET_BLOCKS_CONVERT, "CONVERT" }, \
@@ -51,7 +51,6 @@
{ EXT4_MAP_MAPPED, "M" }, \
{ EXT4_MAP_UNWRITTEN, "U" }, \
{ EXT4_MAP_BOUNDARY, "B" }, \
- { EXT4_MAP_UNINIT, "u" }, \
{ EXT4_MAP_FROM_CLUSTER, "C" })
#define show_free_flags(flags) __print_flags(flags, "|", \
@@ -1497,7 +1496,7 @@
TP_ARGS(inode)
);
-/* 'ux' is the uninitialized extent. */
+/* 'ux' is the unwritten extent. */
TRACE_EVENT(ext4_ext_convert_to_initialized_enter,
TP_PROTO(struct inode *inode, struct ext4_map_blocks *map,
struct ext4_extent *ux),
@@ -1533,7 +1532,7 @@
);
/*
- * 'ux' is the uninitialized extent.
+ * 'ux' is the unwritten extent.
* 'ix' is the initialized extent to which blocks are transferred.
*/
TRACE_EVENT(ext4_ext_convert_to_initialized_fastpath,
@@ -1811,7 +1810,7 @@
TP_ARGS(sb, group, start, len)
);
-TRACE_EVENT(ext4_ext_handle_uninitialized_extents,
+TRACE_EVENT(ext4_ext_handle_unwritten_extents,
TP_PROTO(struct inode *inode, struct ext4_map_blocks *map, int flags,
unsigned int allocated, ext4_fsblk_t newblock),
diff --git a/include/trace/events/f2fs.h b/include/trace/events/f2fs.h
index 67f38fa..b983990 100644
--- a/include/trace/events/f2fs.h
+++ b/include/trace/events/f2fs.h
@@ -659,6 +659,66 @@
TP_CONDITION(bio)
);
+TRACE_EVENT(f2fs_write_begin,
+
+ TP_PROTO(struct inode *inode, loff_t pos, unsigned int len,
+ unsigned int flags),
+
+ TP_ARGS(inode, pos, len, flags),
+
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(ino_t, ino)
+ __field(loff_t, pos)
+ __field(unsigned int, len)
+ __field(unsigned int, flags)
+ ),
+
+ TP_fast_assign(
+ __entry->dev = inode->i_sb->s_dev;
+ __entry->ino = inode->i_ino;
+ __entry->pos = pos;
+ __entry->len = len;
+ __entry->flags = flags;
+ ),
+
+ TP_printk("dev = (%d,%d), ino = %lu, pos = %llu, len = %u, flags = %u",
+ show_dev_ino(__entry),
+ (unsigned long long)__entry->pos,
+ __entry->len,
+ __entry->flags)
+);
+
+TRACE_EVENT(f2fs_write_end,
+
+ TP_PROTO(struct inode *inode, loff_t pos, unsigned int len,
+ unsigned int copied),
+
+ TP_ARGS(inode, pos, len, copied),
+
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(ino_t, ino)
+ __field(loff_t, pos)
+ __field(unsigned int, len)
+ __field(unsigned int, copied)
+ ),
+
+ TP_fast_assign(
+ __entry->dev = inode->i_sb->s_dev;
+ __entry->ino = inode->i_ino;
+ __entry->pos = pos;
+ __entry->len = len;
+ __entry->copied = copied;
+ ),
+
+ TP_printk("dev = (%d,%d), ino = %lu, pos = %llu, len = %u, copied = %u",
+ show_dev_ino(__entry),
+ (unsigned long long)__entry->pos,
+ __entry->len,
+ __entry->copied)
+);
+
DECLARE_EVENT_CLASS(f2fs__page,
TP_PROTO(struct page *page, int type),
@@ -672,6 +732,7 @@
__field(int, dir)
__field(pgoff_t, index)
__field(int, dirty)
+ __field(int, uptodate)
),
TP_fast_assign(
@@ -681,14 +742,31 @@
__entry->dir = S_ISDIR(page->mapping->host->i_mode);
__entry->index = page->index;
__entry->dirty = PageDirty(page);
+ __entry->uptodate = PageUptodate(page);
),
- TP_printk("dev = (%d,%d), ino = %lu, %s, %s, index = %lu, dirty = %d",
+ TP_printk("dev = (%d,%d), ino = %lu, %s, %s, index = %lu, "
+ "dirty = %d, uptodate = %d",
show_dev_ino(__entry),
show_block_type(__entry->type),
show_file_type(__entry->dir),
(unsigned long)__entry->index,
- __entry->dirty)
+ __entry->dirty,
+ __entry->uptodate)
+);
+
+DEFINE_EVENT(f2fs__page, f2fs_writepage,
+
+ TP_PROTO(struct page *page, int type),
+
+ TP_ARGS(page, type)
+);
+
+DEFINE_EVENT(f2fs__page, f2fs_readpage,
+
+ TP_PROTO(struct page *page, int type),
+
+ TP_ARGS(page, type)
);
DEFINE_EVENT(f2fs__page, f2fs_set_page_dirty,
@@ -705,6 +783,70 @@
TP_ARGS(page, type)
);
+TRACE_EVENT(f2fs_writepages,
+
+ TP_PROTO(struct inode *inode, struct writeback_control *wbc, int type),
+
+ TP_ARGS(inode, wbc, type),
+
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(ino_t, ino)
+ __field(int, type)
+ __field(int, dir)
+ __field(long, nr_to_write)
+ __field(long, pages_skipped)
+ __field(loff_t, range_start)
+ __field(loff_t, range_end)
+ __field(pgoff_t, writeback_index)
+ __field(int, sync_mode)
+ __field(char, for_kupdate)
+ __field(char, for_background)
+ __field(char, tagged_writepages)
+ __field(char, for_reclaim)
+ __field(char, range_cyclic)
+ __field(char, for_sync)
+ ),
+
+ TP_fast_assign(
+ __entry->dev = inode->i_sb->s_dev;
+ __entry->ino = inode->i_ino;
+ __entry->type = type;
+ __entry->dir = S_ISDIR(inode->i_mode);
+ __entry->nr_to_write = wbc->nr_to_write;
+ __entry->pages_skipped = wbc->pages_skipped;
+ __entry->range_start = wbc->range_start;
+ __entry->range_end = wbc->range_end;
+ __entry->writeback_index = inode->i_mapping->writeback_index;
+ __entry->sync_mode = wbc->sync_mode;
+ __entry->for_kupdate = wbc->for_kupdate;
+ __entry->for_background = wbc->for_background;
+ __entry->tagged_writepages = wbc->tagged_writepages;
+ __entry->for_reclaim = wbc->for_reclaim;
+ __entry->range_cyclic = wbc->range_cyclic;
+ __entry->for_sync = wbc->for_sync;
+ ),
+
+ TP_printk("dev = (%d,%d), ino = %lu, %s, %s, nr_to_write %ld, "
+ "skipped %ld, start %lld, end %lld, wb_idx %lu, sync_mode %d, "
+ "kupdate %u background %u tagged %u reclaim %u cyclic %u sync %u",
+ show_dev_ino(__entry),
+ show_block_type(__entry->type),
+ show_file_type(__entry->dir),
+ __entry->nr_to_write,
+ __entry->pages_skipped,
+ __entry->range_start,
+ __entry->range_end,
+ (unsigned long)__entry->writeback_index,
+ __entry->sync_mode,
+ __entry->for_kupdate,
+ __entry->for_background,
+ __entry->tagged_writepages,
+ __entry->for_reclaim,
+ __entry->range_cyclic,
+ __entry->for_sync)
+);
+
TRACE_EVENT(f2fs_submit_page_mbio,
TP_PROTO(struct page *page, int rw, int type, block_t blk_addr),
diff --git a/include/trace/ftrace.h b/include/trace/ftrace.h
index 0a1a4f7..0fd06fe 100644
--- a/include/trace/ftrace.h
+++ b/include/trace/ftrace.h
@@ -53,6 +53,9 @@
#undef __string
#define __string(item, src) __dynamic_array(char, item, -1)
+#undef __bitmask
+#define __bitmask(item, nr_bits) __dynamic_array(char, item, -1)
+
#undef TP_STRUCT__entry
#define TP_STRUCT__entry(args...) args
@@ -128,6 +131,9 @@
#undef __string
#define __string(item, src) __dynamic_array(char, item, -1)
+#undef __bitmask
+#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
+
#undef DECLARE_EVENT_CLASS
#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print) \
struct ftrace_data_offsets_##call { \
@@ -197,9 +203,22 @@
#define __get_dynamic_array(field) \
((void *)__entry + (__entry->__data_loc_##field & 0xffff))
+#undef __get_dynamic_array_len
+#define __get_dynamic_array_len(field) \
+ ((__entry->__data_loc_##field >> 16) & 0xffff)
+
#undef __get_str
#define __get_str(field) (char *)__get_dynamic_array(field)
+#undef __get_bitmask
+#define __get_bitmask(field) \
+ ({ \
+ void *__bitmask = __get_dynamic_array(field); \
+ unsigned int __bitmask_size; \
+ __bitmask_size = __get_dynamic_array_len(field); \
+ ftrace_print_bitmask_seq(p, __bitmask, __bitmask_size); \
+ })
+
#undef __print_flags
#define __print_flags(flag, delim, flag_array...) \
({ \
@@ -322,6 +341,9 @@
#undef __string
#define __string(item, src) __dynamic_array(char, item, -1)
+#undef __bitmask
+#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
+
#undef DECLARE_EVENT_CLASS
#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, func, print) \
static int notrace __init \
@@ -372,6 +394,29 @@
#define __string(item, src) __dynamic_array(char, item, \
strlen((src) ? (const char *)(src) : "(null)") + 1)
+/*
+ * __bitmask_size_in_bytes_raw is the number of bytes needed to hold
+ * num_possible_cpus().
+ */
+#define __bitmask_size_in_bytes_raw(nr_bits) \
+ (((nr_bits) + 7) / 8)
+
+#define __bitmask_size_in_longs(nr_bits) \
+ ((__bitmask_size_in_bytes_raw(nr_bits) + \
+ ((BITS_PER_LONG / 8) - 1)) / (BITS_PER_LONG / 8))
+
+/*
+ * __bitmask_size_in_bytes is the number of bytes needed to hold
+ * num_possible_cpus() padded out to the nearest long. This is what
+ * is saved in the buffer, just to be consistent.
+ */
+#define __bitmask_size_in_bytes(nr_bits) \
+ (__bitmask_size_in_longs(nr_bits) * (BITS_PER_LONG / 8))
+
+#undef __bitmask
+#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, \
+ __bitmask_size_in_longs(nr_bits))
+
#undef DECLARE_EVENT_CLASS
#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print) \
static inline notrace int ftrace_get_offsets_##call( \
@@ -513,12 +558,22 @@
__entry->__data_loc_##item = __data_offsets.item;
#undef __string
-#define __string(item, src) __dynamic_array(char, item, -1) \
+#define __string(item, src) __dynamic_array(char, item, -1)
#undef __assign_str
#define __assign_str(dst, src) \
strcpy(__get_str(dst), (src) ? (const char *)(src) : "(null)");
+#undef __bitmask
+#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
+
+#undef __get_bitmask
+#define __get_bitmask(field) (char *)__get_dynamic_array(field)
+
+#undef __assign_bitmask
+#define __assign_bitmask(dst, src, nr_bits) \
+ memcpy(__get_bitmask(dst), (src), __bitmask_size_in_bytes(nr_bits))
+
#undef TP_fast_assign
#define TP_fast_assign(args...) args
@@ -585,7 +640,9 @@
#undef __print_symbolic
#undef __print_hex
#undef __get_dynamic_array
+#undef __get_dynamic_array_len
#undef __get_str
+#undef __get_bitmask
#undef TP_printk
#define TP_printk(fmt, args...) "\"" fmt "\", " __stringify(args)
@@ -648,9 +705,16 @@
#define __get_dynamic_array(field) \
((void *)__entry + (__entry->__data_loc_##field & 0xffff))
+#undef __get_dynamic_array_len
+#define __get_dynamic_array_len(field) \
+ ((__entry->__data_loc_##field >> 16) & 0xffff)
+
#undef __get_str
#define __get_str(field) (char *)__get_dynamic_array(field)
+#undef __get_bitmask
+#define __get_bitmask(field) (char *)__get_dynamic_array(field)
+
#undef __perf_addr
#define __perf_addr(a) (__addr = (a))
diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h
index 2841f86..bf6cd7d 100644
--- a/include/uapi/linux/kvm_para.h
+++ b/include/uapi/linux/kvm_para.h
@@ -20,6 +20,9 @@
#define KVM_HC_FEATURES 3
#define KVM_HC_PPC_MAP_MAGIC_PAGE 4
#define KVM_HC_KICK_CPU 5
+#define KVM_HC_MIPS_GET_CLOCK_FREQ 6
+#define KVM_HC_MIPS_EXIT_VM 7
+#define KVM_HC_MIPS_CONSOLE_OUTPUT 8
/*
* hypercalls use architecture specific
diff --git a/include/uapi/linux/nfsd/nfsfh.h b/include/uapi/linux/nfsd/nfsfh.h
index 616e3b3..2039123 100644
--- a/include/uapi/linux/nfsd/nfsfh.h
+++ b/include/uapi/linux/nfsd/nfsfh.h
@@ -1,13 +1,7 @@
/*
- * include/linux/nfsd/nfsfh.h
- *
* This file describes the layout of the file handles as passed
* over the wire.
*
- * Earlier versions of knfsd used to sign file handles using keyed MD5
- * or SHA. I've removed this code, because it doesn't give you more
- * security than blocking external access to port 2049 on your firewall.
- *
* Copyright (C) 1995, 1996, 1997 Olaf Kirch <okir@monad.swb.de>
*/
@@ -37,7 +31,7 @@
};
/*
- * This is the new flexible, extensible style NFSv2/v3 file handle.
+ * This is the new flexible, extensible style NFSv2/v3/v4 file handle.
* by Neil Brown <neilb@cse.unsw.edu.au> - March 2000
*
* The file handle starts with a sequence of four-byte words.
@@ -47,14 +41,7 @@
*
* All four-byte values are in host-byte-order.
*
- * The auth_type field specifies how the filehandle can be authenticated
- * This might allow a file to be confirmed to be in a writable part of a
- * filetree without checking the path from it up to the root.
- * Current values:
- * 0 - No authentication. fb_auth is 0 bytes long
- * Possible future values:
- * 1 - 4 bytes taken from MD5 hash of the remainer of the file handle
- * prefixed by a secret and with the important export flags.
+ * The auth_type field is deprecated and must be set to 0.
*
* The fsid_type identifies how the filesystem (or export point) is
* encoded.
@@ -71,14 +58,9 @@
* 7 - 8 byte inode number and 16 byte uuid
*
* The fileid_type identified how the file within the filesystem is encoded.
- * This is (will be) passed to, and set by, the underlying filesystem if it supports
- * filehandle operations. The filesystem must not use the value '0' or '0xff' and may
- * only use the values 1 and 2 as defined below:
- * Current values:
- * 0 - The root, or export point, of the filesystem. fb_fileid is 0 bytes.
- * 1 - 32bit inode number, 32 bit generation number.
- * 2 - 32bit inode number, 32 bit generation number, 32 bit parent directory inode number.
- *
+ * The values for this field are filesystem specific, exccept that
+ * filesystems must not use the values '0' or '0xff'. 'See enum fid_type'
+ * in include/linux/exportfs.h for currently registered values.
*/
struct nfs_fhbase_new {
__u8 fb_version; /* == 1, even => nfs_fhbase_old */
@@ -114,9 +96,9 @@
#define fh_fsid_type fh_base.fh_new.fb_fsid_type
#define fh_auth_type fh_base.fh_new.fb_auth_type
#define fh_fileid_type fh_base.fh_new.fb_fileid_type
-#define fh_auth fh_base.fh_new.fb_auth
#define fh_fsid fh_base.fh_new.fb_auth
-
+/* Do not use, provided for userspace compatiblity. */
+#define fh_auth fh_base.fh_new.fb_auth
#endif /* _UAPI_LINUX_NFSD_FH_H */
diff --git a/include/uapi/linux/shm.h b/include/uapi/linux/shm.h
index 78b6941..1fbf24e 100644
--- a/include/uapi/linux/shm.h
+++ b/include/uapi/linux/shm.h
@@ -8,19 +8,20 @@
#endif
/*
- * SHMMAX, SHMMNI and SHMALL are upper limits are defaults which can
- * be increased by sysctl
+ * SHMMNI, SHMMAX and SHMALL are default upper limits which can be
+ * modified by sysctl. The SHMMAX and SHMALL values have been chosen to
+ * be as large possible without facilitating scenarios where userspace
+ * causes overflows when adjusting the limits via operations of the form
+ * "retrieve current limit; add X; update limit". It is therefore not
+ * advised to make SHMMAX and SHMALL any larger. These limits are
+ * suitable for both 32 and 64-bit systems.
*/
-
-#define SHMMAX 0x2000000 /* max shared seg size (bytes) */
#define SHMMIN 1 /* min shared seg size (bytes) */
#define SHMMNI 4096 /* max num of segs system wide */
-#ifndef __KERNEL__
-#define SHMALL (SHMMAX/getpagesize()*(SHMMNI/16))
-#endif
+#define SHMMAX (ULONG_MAX - (1UL << 24)) /* max shared seg size (bytes) */
+#define SHMALL (ULONG_MAX - (1UL << 24)) /* max shm system wide (pages) */
#define SHMSEG SHMMNI /* max shared segs per process */
-
/* Obsolete, used only for backwards compatibility and libc5 compiles */
struct shmid_ds {
struct ipc_perm shm_perm; /* operation perms */
diff --git a/include/uapi/linux/usb/Kbuild b/include/uapi/linux/usb/Kbuild
index 6cb4ea8..4cc4d6e 100644
--- a/include/uapi/linux/usb/Kbuild
+++ b/include/uapi/linux/usb/Kbuild
@@ -1,6 +1,7 @@
# UAPI Header export list
header-y += audio.h
header-y += cdc.h
+header-y += cdc-wdm.h
header-y += ch11.h
header-y += ch9.h
header-y += functionfs.h
diff --git a/include/uapi/linux/usb/cdc-wdm.h b/include/uapi/linux/usb/cdc-wdm.h
index f03134f..0dc132e 100644
--- a/include/uapi/linux/usb/cdc-wdm.h
+++ b/include/uapi/linux/usb/cdc-wdm.h
@@ -9,6 +9,8 @@
#ifndef _UAPI__LINUX_USB_CDC_WDM_H
#define _UAPI__LINUX_USB_CDC_WDM_H
+#include <linux/types.h>
+
/*
* This IOCTL is used to retrieve the wMaxCommand for the device,
* defining the message limit for both reading and writing.
diff --git a/include/uapi/mtd/mtd-abi.h b/include/uapi/mtd/mtd-abi.h
index e272ea0..763bb69 100644
--- a/include/uapi/mtd/mtd-abi.h
+++ b/include/uapi/mtd/mtd-abi.h
@@ -109,6 +109,7 @@
#define MTD_CAP_RAM (MTD_WRITEABLE | MTD_BIT_WRITEABLE | MTD_NO_ERASE)
#define MTD_CAP_NORFLASH (MTD_WRITEABLE | MTD_BIT_WRITEABLE)
#define MTD_CAP_NANDFLASH (MTD_WRITEABLE)
+#define MTD_CAP_NVRAM (MTD_WRITEABLE | MTD_BIT_WRITEABLE | MTD_NO_ERASE)
/* Obsolete ECC byte placement modes (used with obsolete MEMGETOOBSEL) */
#define MTD_NANDECC_OFF 0 // Switch off ECC (Not recommended)
diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h
index 8297285..de69170 100644
--- a/include/uapi/rdma/rdma_netlink.h
+++ b/include/uapi/rdma/rdma_netlink.h
@@ -4,7 +4,16 @@
#include <linux/types.h>
enum {
- RDMA_NL_RDMA_CM = 1
+ RDMA_NL_RDMA_CM = 1,
+ RDMA_NL_NES,
+ RDMA_NL_C4IW,
+ RDMA_NL_NUM_CLIENTS
+};
+
+enum {
+ RDMA_NL_GROUP_CM = 1,
+ RDMA_NL_GROUP_IWPM,
+ RDMA_NL_NUM_GROUPS
};
#define RDMA_NL_GET_CLIENT(type) ((type & (((1 << 6) - 1) << 10)) >> 10)
@@ -22,6 +31,18 @@
RDMA_NL_RDMA_CM_NUM_ATTR,
};
+/* iwarp port mapper op-codes */
+enum {
+ RDMA_NL_IWPM_REG_PID = 0,
+ RDMA_NL_IWPM_ADD_MAPPING,
+ RDMA_NL_IWPM_QUERY_MAPPING,
+ RDMA_NL_IWPM_REMOVE_MAPPING,
+ RDMA_NL_IWPM_HANDLE_ERR,
+ RDMA_NL_IWPM_MAPINFO,
+ RDMA_NL_IWPM_MAPINFO_NUM,
+ RDMA_NL_IWPM_NUM_OPS
+};
+
struct rdma_cm_id_stats {
__u32 qp_num;
__u32 bound_dev_if;
@@ -33,5 +54,78 @@
__u8 qp_type;
};
+enum {
+ IWPM_NLA_REG_PID_UNSPEC = 0,
+ IWPM_NLA_REG_PID_SEQ,
+ IWPM_NLA_REG_IF_NAME,
+ IWPM_NLA_REG_IBDEV_NAME,
+ IWPM_NLA_REG_ULIB_NAME,
+ IWPM_NLA_REG_PID_MAX
+};
+
+enum {
+ IWPM_NLA_RREG_PID_UNSPEC = 0,
+ IWPM_NLA_RREG_PID_SEQ,
+ IWPM_NLA_RREG_IBDEV_NAME,
+ IWPM_NLA_RREG_ULIB_NAME,
+ IWPM_NLA_RREG_ULIB_VER,
+ IWPM_NLA_RREG_PID_ERR,
+ IWPM_NLA_RREG_PID_MAX
+
+};
+
+enum {
+ IWPM_NLA_MANAGE_MAPPING_UNSPEC = 0,
+ IWPM_NLA_MANAGE_MAPPING_SEQ,
+ IWPM_NLA_MANAGE_ADDR,
+ IWPM_NLA_MANAGE_MAPPED_LOC_ADDR,
+ IWPM_NLA_RMANAGE_MAPPING_ERR,
+ IWPM_NLA_RMANAGE_MAPPING_MAX
+};
+
+#define IWPM_NLA_MANAGE_MAPPING_MAX 3
+#define IWPM_NLA_QUERY_MAPPING_MAX 4
+#define IWPM_NLA_MAPINFO_SEND_MAX 3
+
+enum {
+ IWPM_NLA_QUERY_MAPPING_UNSPEC = 0,
+ IWPM_NLA_QUERY_MAPPING_SEQ,
+ IWPM_NLA_QUERY_LOCAL_ADDR,
+ IWPM_NLA_QUERY_REMOTE_ADDR,
+ IWPM_NLA_RQUERY_MAPPED_LOC_ADDR,
+ IWPM_NLA_RQUERY_MAPPED_REM_ADDR,
+ IWPM_NLA_RQUERY_MAPPING_ERR,
+ IWPM_NLA_RQUERY_MAPPING_MAX
+};
+
+enum {
+ IWPM_NLA_MAPINFO_REQ_UNSPEC = 0,
+ IWPM_NLA_MAPINFO_ULIB_NAME,
+ IWPM_NLA_MAPINFO_ULIB_VER,
+ IWPM_NLA_MAPINFO_REQ_MAX
+};
+
+enum {
+ IWPM_NLA_MAPINFO_UNSPEC = 0,
+ IWPM_NLA_MAPINFO_LOCAL_ADDR,
+ IWPM_NLA_MAPINFO_MAPPED_ADDR,
+ IWPM_NLA_MAPINFO_MAX
+};
+
+enum {
+ IWPM_NLA_MAPINFO_NUM_UNSPEC = 0,
+ IWPM_NLA_MAPINFO_SEQ,
+ IWPM_NLA_MAPINFO_SEND_NUM,
+ IWPM_NLA_MAPINFO_ACK_NUM,
+ IWPM_NLA_MAPINFO_NUM_MAX
+};
+
+enum {
+ IWPM_NLA_ERR_UNSPEC = 0,
+ IWPM_NLA_ERR_SEQ,
+ IWPM_NLA_ERR_CODE,
+ IWPM_NLA_ERR_MAX
+};
+
#endif /* _UAPI_RDMA_NETLINK_H */
diff --git a/ipc/compat.c b/ipc/compat.c
index 45d035d..b5ef4f7 100644
--- a/ipc/compat.c
+++ b/ipc/compat.c
@@ -30,7 +30,7 @@
#include <linux/ptrace.h>
#include <linux/mutex.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
#include "util.h"
diff --git a/ipc/compat_mq.c b/ipc/compat_mq.c
index 90d29f5..ef6f91c 100644
--- a/ipc/compat_mq.c
+++ b/ipc/compat_mq.c
@@ -12,7 +12,7 @@
#include <linux/mqueue.h>
#include <linux/syscalls.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
struct compat_mq_attr {
compat_long_t mq_flags; /* message queue flags */
diff --git a/ipc/ipc_sysctl.c b/ipc/ipc_sysctl.c
index 998d31b..c3f0326 100644
--- a/ipc/ipc_sysctl.c
+++ b/ipc/ipc_sysctl.c
@@ -18,7 +18,7 @@
#include <linux/msg.h>
#include "util.h"
-static void *get_ipc(ctl_table *table)
+static void *get_ipc(struct ctl_table *table)
{
char *which = table->data;
struct ipc_namespace *ipc_ns = current->nsproxy->ipc_ns;
@@ -27,7 +27,7 @@
}
#ifdef CONFIG_PROC_SYSCTL
-static int proc_ipc_dointvec(ctl_table *table, int write,
+static int proc_ipc_dointvec(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
struct ctl_table ipc_table;
@@ -38,7 +38,7 @@
return proc_dointvec(&ipc_table, write, buffer, lenp, ppos);
}
-static int proc_ipc_dointvec_minmax(ctl_table *table, int write,
+static int proc_ipc_dointvec_minmax(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
struct ctl_table ipc_table;
@@ -49,7 +49,7 @@
return proc_dointvec_minmax(&ipc_table, write, buffer, lenp, ppos);
}
-static int proc_ipc_dointvec_minmax_orphans(ctl_table *table, int write,
+static int proc_ipc_dointvec_minmax_orphans(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
struct ipc_namespace *ns = current->nsproxy->ipc_ns;
@@ -62,7 +62,7 @@
return err;
}
-static int proc_ipc_callback_dointvec_minmax(ctl_table *table, int write,
+static int proc_ipc_callback_dointvec_minmax(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
struct ctl_table ipc_table;
@@ -85,7 +85,7 @@
return rc;
}
-static int proc_ipc_doulongvec_minmax(ctl_table *table, int write,
+static int proc_ipc_doulongvec_minmax(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
struct ctl_table ipc_table;
@@ -119,7 +119,7 @@
}
}
-static int proc_ipcauto_dointvec_minmax(ctl_table *table, int write,
+static int proc_ipcauto_dointvec_minmax(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
struct ctl_table ipc_table;
diff --git a/ipc/mq_sysctl.c b/ipc/mq_sysctl.c
index 5bb8bfe6..68d4e95 100644
--- a/ipc/mq_sysctl.c
+++ b/ipc/mq_sysctl.c
@@ -14,7 +14,7 @@
#include <linux/sysctl.h>
#ifdef CONFIG_PROC_SYSCTL
-static void *get_mq(ctl_table *table)
+static void *get_mq(struct ctl_table *table)
{
char *which = table->data;
struct ipc_namespace *ipc_ns = current->nsproxy->ipc_ns;
@@ -22,7 +22,7 @@
return which;
}
-static int proc_mq_dointvec(ctl_table *table, int write,
+static int proc_mq_dointvec(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
struct ctl_table mq_table;
@@ -32,7 +32,7 @@
return proc_dointvec(&mq_table, write, buffer, lenp, ppos);
}
-static int proc_mq_dointvec_minmax(ctl_table *table, int write,
+static int proc_mq_dointvec_minmax(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
struct ctl_table mq_table;
@@ -53,7 +53,7 @@
static int msg_maxsize_limit_min = MIN_MSGSIZEMAX;
static int msg_maxsize_limit_max = HARD_MSGSIZEMAX;
-static ctl_table mq_sysctls[] = {
+static struct ctl_table mq_sysctls[] = {
{
.procname = "queues_max",
.data = &init_ipc_ns.mq_queues_max,
@@ -100,7 +100,7 @@
{}
};
-static ctl_table mq_sysctl_dir[] = {
+static struct ctl_table mq_sysctl_dir[] = {
{
.procname = "mqueue",
.mode = 0555,
@@ -109,7 +109,7 @@
{}
};
-static ctl_table mq_sysctl_root[] = {
+static struct ctl_table mq_sysctl_root[] = {
{
.procname = "fs",
.mode = 0555,
diff --git a/ipc/msg.c b/ipc/msg.c
index 6498531..c5d8e37 100644
--- a/ipc/msg.c
+++ b/ipc/msg.c
@@ -39,12 +39,10 @@
#include <linux/ipc_namespace.h>
#include <asm/current.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
#include "util.h"
-/*
- * one msg_receiver structure for each sleeping receiver:
- */
+/* one msg_receiver structure for each sleeping receiver */
struct msg_receiver {
struct list_head r_list;
struct task_struct *r_tsk;
@@ -53,6 +51,12 @@
long r_msgtype;
long r_maxsize;
+ /*
+ * Mark r_msg volatile so that the compiler
+ * does not try to get smart and optimize
+ * it. We rely on this for the lockless
+ * receive algorithm.
+ */
struct msg_msg *volatile r_msg;
};
@@ -70,75 +74,6 @@
#define msg_ids(ns) ((ns)->ids[IPC_MSG_IDS])
-static void freeque(struct ipc_namespace *, struct kern_ipc_perm *);
-static int newque(struct ipc_namespace *, struct ipc_params *);
-#ifdef CONFIG_PROC_FS
-static int sysvipc_msg_proc_show(struct seq_file *s, void *it);
-#endif
-
-/*
- * Scale msgmni with the available lowmem size: the memory dedicated to msg
- * queues should occupy at most 1/MSG_MEM_SCALE of lowmem.
- * Also take into account the number of nsproxies created so far.
- * This should be done staying within the (MSGMNI , IPCMNI/nr_ipc_ns) range.
- */
-void recompute_msgmni(struct ipc_namespace *ns)
-{
- struct sysinfo i;
- unsigned long allowed;
- int nb_ns;
-
- si_meminfo(&i);
- allowed = (((i.totalram - i.totalhigh) / MSG_MEM_SCALE) * i.mem_unit)
- / MSGMNB;
- nb_ns = atomic_read(&nr_ipc_ns);
- allowed /= nb_ns;
-
- if (allowed < MSGMNI) {
- ns->msg_ctlmni = MSGMNI;
- return;
- }
-
- if (allowed > IPCMNI / nb_ns) {
- ns->msg_ctlmni = IPCMNI / nb_ns;
- return;
- }
-
- ns->msg_ctlmni = allowed;
-}
-
-void msg_init_ns(struct ipc_namespace *ns)
-{
- ns->msg_ctlmax = MSGMAX;
- ns->msg_ctlmnb = MSGMNB;
-
- recompute_msgmni(ns);
-
- atomic_set(&ns->msg_bytes, 0);
- atomic_set(&ns->msg_hdrs, 0);
- ipc_init_ids(&ns->ids[IPC_MSG_IDS]);
-}
-
-#ifdef CONFIG_IPC_NS
-void msg_exit_ns(struct ipc_namespace *ns)
-{
- free_ipcs(ns, &msg_ids(ns), freeque);
- idr_destroy(&ns->ids[IPC_MSG_IDS].ipcs_idr);
-}
-#endif
-
-void __init msg_init(void)
-{
- msg_init_ns(&init_ipc_ns);
-
- printk(KERN_INFO "msgmni has been set to %d\n",
- init_ipc_ns.msg_ctlmni);
-
- ipc_init_proc_interface("sysvipc/msg",
- " key msqid perms cbytes qnum lspid lrpid uid gid cuid cgid stime rtime ctime\n",
- IPC_MSG_IDS, sysvipc_msg_proc_show);
-}
-
static inline struct msg_queue *msq_obtain_object(struct ipc_namespace *ns, int id)
{
struct kern_ipc_perm *ipcp = ipc_obtain_object(&msg_ids(ns), id);
@@ -227,7 +162,7 @@
static inline void ss_add(struct msg_queue *msq, struct msg_sender *mss)
{
mss->tsk = current;
- current->state = TASK_INTERRUPTIBLE;
+ __set_current_state(TASK_INTERRUPTIBLE);
list_add_tail(&mss->list, &msq->q_senders);
}
@@ -306,15 +241,14 @@
SYSCALL_DEFINE2(msgget, key_t, key, int, msgflg)
{
struct ipc_namespace *ns;
- struct ipc_ops msg_ops;
+ static const struct ipc_ops msg_ops = {
+ .getnew = newque,
+ .associate = msg_security,
+ };
struct ipc_params msg_params;
ns = current->nsproxy->ipc_ns;
- msg_ops.getnew = newque;
- msg_ops.associate = msg_security;
- msg_ops.more_checks = NULL;
-
msg_params.key = key;
msg_params.flg = msgflg;
@@ -612,23 +546,22 @@
static int testmsg(struct msg_msg *msg, long type, int mode)
{
- switch (mode)
- {
- case SEARCH_ANY:
- case SEARCH_NUMBER:
+ switch (mode) {
+ case SEARCH_ANY:
+ case SEARCH_NUMBER:
+ return 1;
+ case SEARCH_LESSEQUAL:
+ if (msg->m_type <= type)
return 1;
- case SEARCH_LESSEQUAL:
- if (msg->m_type <= type)
- return 1;
- break;
- case SEARCH_EQUAL:
- if (msg->m_type == type)
- return 1;
- break;
- case SEARCH_NOTEQUAL:
- if (msg->m_type != type)
- return 1;
- break;
+ break;
+ case SEARCH_EQUAL:
+ if (msg->m_type == type)
+ return 1;
+ break;
+ case SEARCH_NOTEQUAL:
+ if (msg->m_type != type)
+ return 1;
+ break;
}
return 0;
}
@@ -978,7 +911,7 @@
else
msr_d.r_maxsize = bufsz;
msr_d.r_msg = ERR_PTR(-EAGAIN);
- current->state = TASK_INTERRUPTIBLE;
+ __set_current_state(TASK_INTERRUPTIBLE);
ipc_unlock_object(&msq->q_perm);
rcu_read_unlock();
@@ -1056,6 +989,57 @@
return do_msgrcv(msqid, msgp, msgsz, msgtyp, msgflg, do_msg_fill);
}
+/*
+ * Scale msgmni with the available lowmem size: the memory dedicated to msg
+ * queues should occupy at most 1/MSG_MEM_SCALE of lowmem.
+ * Also take into account the number of nsproxies created so far.
+ * This should be done staying within the (MSGMNI , IPCMNI/nr_ipc_ns) range.
+ */
+void recompute_msgmni(struct ipc_namespace *ns)
+{
+ struct sysinfo i;
+ unsigned long allowed;
+ int nb_ns;
+
+ si_meminfo(&i);
+ allowed = (((i.totalram - i.totalhigh) / MSG_MEM_SCALE) * i.mem_unit)
+ / MSGMNB;
+ nb_ns = atomic_read(&nr_ipc_ns);
+ allowed /= nb_ns;
+
+ if (allowed < MSGMNI) {
+ ns->msg_ctlmni = MSGMNI;
+ return;
+ }
+
+ if (allowed > IPCMNI / nb_ns) {
+ ns->msg_ctlmni = IPCMNI / nb_ns;
+ return;
+ }
+
+ ns->msg_ctlmni = allowed;
+}
+
+void msg_init_ns(struct ipc_namespace *ns)
+{
+ ns->msg_ctlmax = MSGMAX;
+ ns->msg_ctlmnb = MSGMNB;
+
+ recompute_msgmni(ns);
+
+ atomic_set(&ns->msg_bytes, 0);
+ atomic_set(&ns->msg_hdrs, 0);
+ ipc_init_ids(&ns->ids[IPC_MSG_IDS]);
+}
+
+#ifdef CONFIG_IPC_NS
+void msg_exit_ns(struct ipc_namespace *ns)
+{
+ free_ipcs(ns, &msg_ids(ns), freeque);
+ idr_destroy(&ns->ids[IPC_MSG_IDS].ipcs_idr);
+}
+#endif
+
#ifdef CONFIG_PROC_FS
static int sysvipc_msg_proc_show(struct seq_file *s, void *it)
{
@@ -1080,3 +1064,15 @@
msq->q_ctime);
}
#endif
+
+void __init msg_init(void)
+{
+ msg_init_ns(&init_ipc_ns);
+
+ printk(KERN_INFO "msgmni has been set to %d\n",
+ init_ipc_ns.msg_ctlmni);
+
+ ipc_init_proc_interface("sysvipc/msg",
+ " key msqid perms cbytes qnum lspid lrpid uid gid cuid cgid stime rtime ctime\n",
+ IPC_MSG_IDS, sysvipc_msg_proc_show);
+}
diff --git a/ipc/sem.c b/ipc/sem.c
index bee5554..454f6c6 100644
--- a/ipc/sem.c
+++ b/ipc/sem.c
@@ -47,8 +47,7 @@
* Thus: Perfect SMP scaling between independent semaphore arrays.
* If multiple semaphores in one array are used, then cache line
* trashing on the semaphore array spinlock will limit the scaling.
- * - semncnt and semzcnt are calculated on demand in count_semncnt() and
- * count_semzcnt()
+ * - semncnt and semzcnt are calculated on demand in count_semcnt()
* - the task that performs a successful semop() scans the list of all
* sleeping tasks and completes any pending operations that can be fulfilled.
* Semaphores are actively given to waiting tasks (necessary for FIFO).
@@ -87,7 +86,7 @@
#include <linux/nsproxy.h>
#include <linux/ipc_namespace.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
#include "util.h"
/* One semaphore structure for each semaphore in the system. */
@@ -110,6 +109,7 @@
int pid; /* process id of requesting process */
int status; /* completion status of operation */
struct sembuf *sops; /* array of pending operations */
+ struct sembuf *blocking; /* the operation that blocked */
int nsops; /* number of operations */
int alter; /* does *sops alter the array? */
};
@@ -160,7 +160,7 @@
* sem_array.pending{_alter,_cont},
* sem_array.sem_undo: global sem_lock() for read/write
* sem_undo.proc_next: only "current" is allowed to read/write that field.
- *
+ *
* sem_array.sem_base[i].pending_{const,alter}:
* global or semaphore sem_lock() for read/write
*/
@@ -564,7 +564,11 @@
SYSCALL_DEFINE3(semget, key_t, key, int, nsems, int, semflg)
{
struct ipc_namespace *ns;
- struct ipc_ops sem_ops;
+ static const struct ipc_ops sem_ops = {
+ .getnew = newary,
+ .associate = sem_security,
+ .more_checks = sem_more_checks,
+ };
struct ipc_params sem_params;
ns = current->nsproxy->ipc_ns;
@@ -572,10 +576,6 @@
if (nsems < 0 || nsems > ns->sc_semmsl)
return -EINVAL;
- sem_ops.getnew = newary;
- sem_ops.associate = sem_security;
- sem_ops.more_checks = sem_more_checks;
-
sem_params.key = key;
sem_params.flg = semflg;
sem_params.u.nsems = nsems;
@@ -586,21 +586,23 @@
/**
* perform_atomic_semop - Perform (if possible) a semaphore operation
* @sma: semaphore array
- * @sops: array with operations that should be checked
- * @nsops: number of operations
- * @un: undo array
- * @pid: pid that did the change
+ * @q: struct sem_queue that describes the operation
*
* Returns 0 if the operation was possible.
* Returns 1 if the operation is impossible, the caller must sleep.
* Negative values are error codes.
*/
-static int perform_atomic_semop(struct sem_array *sma, struct sembuf *sops,
- int nsops, struct sem_undo *un, int pid)
+static int perform_atomic_semop(struct sem_array *sma, struct sem_queue *q)
{
- int result, sem_op;
+ int result, sem_op, nsops, pid;
struct sembuf *sop;
struct sem *curr;
+ struct sembuf *sops;
+ struct sem_undo *un;
+
+ sops = q->sops;
+ nsops = q->nsops;
+ un = q->undo;
for (sop = sops; sop < sops + nsops; sop++) {
curr = sma->sem_base + sop->sem_num;
@@ -628,6 +630,7 @@
}
sop--;
+ pid = q->pid;
while (sop >= sops) {
sma->sem_base[sop->sem_num].sempid = pid;
sop--;
@@ -640,6 +643,8 @@
goto undo;
would_block:
+ q->blocking = sop;
+
if (sop->sem_flg & IPC_NOWAIT)
result = -EAGAIN;
else
@@ -780,8 +785,7 @@
q = container_of(walk, struct sem_queue, list);
walk = walk->next;
- error = perform_atomic_semop(sma, q->sops, q->nsops,
- q->undo, q->pid);
+ error = perform_atomic_semop(sma, q);
if (error <= 0) {
/* operation completed, remove from queue & wakeup */
@@ -893,8 +897,7 @@
if (semnum != -1 && sma->sem_base[semnum].semval == 0)
break;
- error = perform_atomic_semop(sma, q->sops, q->nsops,
- q->undo, q->pid);
+ error = perform_atomic_semop(sma, q);
/* Does q->sleeper still need to sleep? */
if (error > 0)
@@ -989,65 +992,74 @@
set_semotime(sma, sops);
}
+/*
+ * check_qop: Test if a queued operation sleeps on the semaphore semnum
+ */
+static int check_qop(struct sem_array *sma, int semnum, struct sem_queue *q,
+ bool count_zero)
+{
+ struct sembuf *sop = q->blocking;
+
+ /*
+ * Linux always (since 0.99.10) reported a task as sleeping on all
+ * semaphores. This violates SUS, therefore it was changed to the
+ * standard compliant behavior.
+ * Give the administrators a chance to notice that an application
+ * might misbehave because it relies on the Linux behavior.
+ */
+ pr_info_once("semctl(GETNCNT/GETZCNT) is since 3.16 Single Unix Specification compliant.\n"
+ "The task %s (%d) triggered the difference, watch for misbehavior.\n",
+ current->comm, task_pid_nr(current));
+
+ if (sop->sem_num != semnum)
+ return 0;
+
+ if (count_zero && sop->sem_op == 0)
+ return 1;
+ if (!count_zero && sop->sem_op < 0)
+ return 1;
+
+ return 0;
+}
+
/* The following counts are associated to each semaphore:
* semncnt number of tasks waiting on semval being nonzero
* semzcnt number of tasks waiting on semval being zero
- * This model assumes that a task waits on exactly one semaphore.
- * Since semaphore operations are to be performed atomically, tasks actually
- * wait on a whole sequence of semaphores simultaneously.
- * The counts we return here are a rough approximation, but still
- * warrant that semncnt+semzcnt>0 if the task is on the pending queue.
+ *
+ * Per definition, a task waits only on the semaphore of the first semop
+ * that cannot proceed, even if additional operation would block, too.
*/
-static int count_semncnt(struct sem_array *sma, ushort semnum)
+static int count_semcnt(struct sem_array *sma, ushort semnum,
+ bool count_zero)
{
- int semncnt;
+ struct list_head *l;
struct sem_queue *q;
+ int semcnt;
- semncnt = 0;
- list_for_each_entry(q, &sma->sem_base[semnum].pending_alter, list) {
- struct sembuf *sops = q->sops;
- BUG_ON(sops->sem_num != semnum);
- if ((sops->sem_op < 0) && !(sops->sem_flg & IPC_NOWAIT))
- semncnt++;
+ semcnt = 0;
+ /* First: check the simple operations. They are easy to evaluate */
+ if (count_zero)
+ l = &sma->sem_base[semnum].pending_const;
+ else
+ l = &sma->sem_base[semnum].pending_alter;
+
+ list_for_each_entry(q, l, list) {
+ /* all task on a per-semaphore list sleep on exactly
+ * that semaphore
+ */
+ semcnt++;
}
+ /* Then: check the complex operations. */
list_for_each_entry(q, &sma->pending_alter, list) {
- struct sembuf *sops = q->sops;
- int nsops = q->nsops;
- int i;
- for (i = 0; i < nsops; i++)
- if (sops[i].sem_num == semnum
- && (sops[i].sem_op < 0)
- && !(sops[i].sem_flg & IPC_NOWAIT))
- semncnt++;
+ semcnt += check_qop(sma, semnum, q, count_zero);
}
- return semncnt;
-}
-
-static int count_semzcnt(struct sem_array *sma, ushort semnum)
-{
- int semzcnt;
- struct sem_queue *q;
-
- semzcnt = 0;
- list_for_each_entry(q, &sma->sem_base[semnum].pending_const, list) {
- struct sembuf *sops = q->sops;
- BUG_ON(sops->sem_num != semnum);
- if ((sops->sem_op == 0) && !(sops->sem_flg & IPC_NOWAIT))
- semzcnt++;
+ if (count_zero) {
+ list_for_each_entry(q, &sma->pending_const, list) {
+ semcnt += check_qop(sma, semnum, q, count_zero);
+ }
}
-
- list_for_each_entry(q, &sma->pending_const, list) {
- struct sembuf *sops = q->sops;
- int nsops = q->nsops;
- int i;
- for (i = 0; i < nsops; i++)
- if (sops[i].sem_num == semnum
- && (sops[i].sem_op == 0)
- && !(sops[i].sem_flg & IPC_NOWAIT))
- semzcnt++;
- }
- return semzcnt;
+ return semcnt;
}
/* Free a semaphore set. freeary() is called with sem_ids.rwsem locked
@@ -1161,7 +1173,7 @@
err = security_sem_semctl(NULL, cmd);
if (err)
return err;
-
+
memset(&seminfo, 0, sizeof(seminfo));
seminfo.semmni = ns->sc_semmni;
seminfo.semmns = ns->sc_semmns;
@@ -1181,7 +1193,7 @@
}
max_id = ipc_get_maxid(&sem_ids(ns));
up_read(&sem_ids(ns).rwsem);
- if (copy_to_user(p, &seminfo, sizeof(struct seminfo)))
+ if (copy_to_user(p, &seminfo, sizeof(struct seminfo)))
return -EFAULT;
return (max_id < 0) ? 0 : max_id;
}
@@ -1449,10 +1461,10 @@
err = curr->sempid;
goto out_unlock;
case GETNCNT:
- err = count_semncnt(sma, semnum);
+ err = count_semcnt(sma, semnum, 0);
goto out_unlock;
case GETZCNT:
- err = count_semzcnt(sma, semnum);
+ err = count_semcnt(sma, semnum, 1);
goto out_unlock;
}
@@ -1866,8 +1878,13 @@
if (un && un->semid == -1)
goto out_unlock_free;
- error = perform_atomic_semop(sma, sops, nsops, un,
- task_tgid_vnr(current));
+ queue.sops = sops;
+ queue.nsops = nsops;
+ queue.undo = un;
+ queue.pid = task_tgid_vnr(current);
+ queue.alter = alter;
+
+ error = perform_atomic_semop(sma, &queue);
if (error == 0) {
/* If the operation was successful, then do
* the required updates.
@@ -1883,12 +1900,6 @@
/* We need to sleep on this operation, so we put the current
* task into the pending queue and go to sleep.
*/
-
- queue.sops = sops;
- queue.nsops = nsops;
- queue.undo = un;
- queue.pid = task_tgid_vnr(current);
- queue.alter = alter;
if (nsops == 1) {
struct sem *curr;
@@ -2016,7 +2027,7 @@
return error;
atomic_inc(&undo_list->refcnt);
tsk->sysvsem.undo_list = undo_list;
- } else
+ } else
tsk->sysvsem.undo_list = NULL;
return 0;
diff --git a/ipc/shm.c b/ipc/shm.c
index 7645961..89fc354 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -43,7 +43,7 @@
#include <linux/mount.h>
#include <linux/ipc_namespace.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
#include "util.h"
@@ -493,7 +493,11 @@
if (size < SHMMIN || size > ns->shm_ctlmax)
return -EINVAL;
- if (ns->shm_tot + numpages > ns->shm_ctlall)
+ if (numpages << PAGE_SHIFT < size)
+ return -ENOSPC;
+
+ if (ns->shm_tot + numpages < ns->shm_tot ||
+ ns->shm_tot + numpages > ns->shm_ctlall)
return -ENOSPC;
shp = ipc_rcu_alloc(sizeof(*shp));
@@ -609,15 +613,15 @@
SYSCALL_DEFINE3(shmget, key_t, key, size_t, size, int, shmflg)
{
struct ipc_namespace *ns;
- struct ipc_ops shm_ops;
+ static const struct ipc_ops shm_ops = {
+ .getnew = newseg,
+ .associate = shm_security,
+ .more_checks = shm_more_checks,
+ };
struct ipc_params shm_params;
ns = current->nsproxy->ipc_ns;
- shm_ops.getnew = newseg;
- shm_ops.associate = shm_security;
- shm_ops.more_checks = shm_more_checks;
-
shm_params.key = key;
shm_params.flg = shmflg;
shm_params.u.size = size;
@@ -694,7 +698,7 @@
out.shmmin = in->shmmin;
out.shmmni = in->shmmni;
out.shmseg = in->shmseg;
- out.shmall = in->shmall;
+ out.shmall = in->shmall;
return copy_to_user(buf, &out, sizeof(out));
}
@@ -1160,6 +1164,9 @@
down_write(¤t->mm->mmap_sem);
if (addr && !(shmflg & SHM_REMAP)) {
err = -EINVAL;
+ if (addr + size < addr)
+ goto invalid;
+
if (find_vma_intersection(current->mm, addr, addr + size))
goto invalid;
/*
diff --git a/ipc/util.c b/ipc/util.c
index 2eb0d1e..27d74e6 100644
--- a/ipc/util.c
+++ b/ipc/util.c
@@ -183,7 +183,7 @@
* ipc_findkey - find a key in an ipc identifier set
* @ids: ipc identifier set
* @key: key to find
- *
+ *
* Returns the locked pointer to the ipc structure if found or NULL
* otherwise. If key is found ipc points to the owning ipc structure
*
@@ -317,7 +317,7 @@
* when the key is IPC_PRIVATE.
*/
static int ipcget_new(struct ipc_namespace *ns, struct ipc_ids *ids,
- struct ipc_ops *ops, struct ipc_params *params)
+ const struct ipc_ops *ops, struct ipc_params *params)
{
int err;
@@ -344,7 +344,7 @@
*/
static int ipc_check_perms(struct ipc_namespace *ns,
struct kern_ipc_perm *ipcp,
- struct ipc_ops *ops,
+ const struct ipc_ops *ops,
struct ipc_params *params)
{
int err;
@@ -375,7 +375,7 @@
* On success, the ipc id is returned.
*/
static int ipcget_public(struct ipc_namespace *ns, struct ipc_ids *ids,
- struct ipc_ops *ops, struct ipc_params *params)
+ const struct ipc_ops *ops, struct ipc_params *params)
{
struct kern_ipc_perm *ipcp;
int flg = params->flg;
@@ -538,7 +538,7 @@
else if (in_group_p(ipcp->cgid) || in_group_p(ipcp->gid))
granted_mode >>= 3;
/* is there some bit set in requested_mode but not in granted_mode? */
- if ((requested_mode & ~granted_mode & 0007) &&
+ if ((requested_mode & ~granted_mode & 0007) &&
!ns_capable(ns->user_ns, CAP_IPC_OWNER))
return -1;
@@ -678,7 +678,7 @@
* Common routine called by sys_msgget(), sys_semget() and sys_shmget().
*/
int ipcget(struct ipc_namespace *ns, struct ipc_ids *ids,
- struct ipc_ops *ops, struct ipc_params *params)
+ const struct ipc_ops *ops, struct ipc_params *params)
{
if (params->key == IPC_PRIVATE)
return ipcget_new(ns, ids, ops, params);
diff --git a/ipc/util.h b/ipc/util.h
index 9c47d6f..1a5a0fc 100644
--- a/ipc/util.h
+++ b/ipc/util.h
@@ -78,9 +78,9 @@
* . routine to call for an extra check if needed
*/
struct ipc_ops {
- int (*getnew) (struct ipc_namespace *, struct ipc_params *);
- int (*associate) (struct kern_ipc_perm *, int);
- int (*more_checks) (struct kern_ipc_perm *, struct ipc_params *);
+ int (*getnew)(struct ipc_namespace *, struct ipc_params *);
+ int (*associate)(struct kern_ipc_perm *, int);
+ int (*more_checks)(struct kern_ipc_perm *, struct ipc_params *);
};
struct seq_file;
@@ -142,7 +142,7 @@
struct ipc64_perm *perm, int extra_perm);
#ifndef CONFIG_ARCH_WANT_IPC_PARSE_VERSION
- /* On IA-64, we always use the "64-bit version" of the IPC structures. */
+/* On IA-64, we always use the "64-bit version" of the IPC structures. */
# define ipc_parse_version(cmd) IPC_64
#else
int ipc_parse_version(int *cmd);
@@ -201,7 +201,7 @@
struct kern_ipc_perm *ipc_obtain_object_check(struct ipc_ids *ids, int id);
int ipcget(struct ipc_namespace *ns, struct ipc_ids *ids,
- struct ipc_ops *ops, struct ipc_params *params);
+ const struct ipc_ops *ops, struct ipc_params *params);
void free_ipcs(struct ipc_namespace *ns, struct ipc_ids *ids,
void (*free)(struct ipc_namespace *, struct kern_ipc_perm *));
#endif
diff --git a/kernel/acct.c b/kernel/acct.c
index 8d6e145..808a86f 100644
--- a/kernel/acct.c
+++ b/kernel/acct.c
@@ -55,7 +55,7 @@
#include <linux/times.h>
#include <linux/syscalls.h>
#include <linux/mount.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
#include <asm/div64.h>
#include <linux/blkdev.h> /* sector_div */
#include <linux/pid_namespace.h>
@@ -134,7 +134,7 @@
spin_lock(&acct_lock);
if (file != acct->file) {
if (act)
- res = act>0;
+ res = act > 0;
goto out;
}
@@ -262,7 +262,7 @@
if (name) {
struct filename *tmp = getname(name);
if (IS_ERR(tmp))
- return (PTR_ERR(tmp));
+ return PTR_ERR(tmp);
error = acct_on(tmp);
putname(tmp);
} else {
diff --git a/kernel/audit.c b/kernel/audit.c
index 47845c5..f301064 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -44,7 +44,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/init.h>
-#include <asm/types.h>
+#include <linux/types.h>
#include <linux/atomic.h>
#include <linux/mm.h>
#include <linux/export.h>
diff --git a/kernel/auditsc.c b/kernel/auditsc.c
index f251a5e..21eae3c 100644
--- a/kernel/auditsc.c
+++ b/kernel/auditsc.c
@@ -728,6 +728,22 @@
return AUDIT_BUILD_CONTEXT;
}
+static int audit_in_mask(const struct audit_krule *rule, unsigned long val)
+{
+ int word, bit;
+
+ if (val > 0xffffffff)
+ return false;
+
+ word = AUDIT_WORD(val);
+ if (word >= AUDIT_BITMASK_SIZE)
+ return false;
+
+ bit = AUDIT_BIT(val);
+
+ return rule->mask[word] & bit;
+}
+
/* At syscall entry and exit time, this filter is called if the
* audit_state is not low enough that auditing cannot take place, but is
* also not high enough that we already know we have to write an audit
@@ -745,11 +761,8 @@
rcu_read_lock();
if (!list_empty(list)) {
- int word = AUDIT_WORD(ctx->major);
- int bit = AUDIT_BIT(ctx->major);
-
list_for_each_entry_rcu(e, list, list) {
- if ((e->rule.mask[word] & bit) == bit &&
+ if (audit_in_mask(&e->rule, ctx->major) &&
audit_filter_rules(tsk, &e->rule, ctx, NULL,
&state, false)) {
rcu_read_unlock();
@@ -769,20 +782,16 @@
static int audit_filter_inode_name(struct task_struct *tsk,
struct audit_names *n,
struct audit_context *ctx) {
- int word, bit;
int h = audit_hash_ino((u32)n->ino);
struct list_head *list = &audit_inode_hash[h];
struct audit_entry *e;
enum audit_state state;
- word = AUDIT_WORD(ctx->major);
- bit = AUDIT_BIT(ctx->major);
-
if (list_empty(list))
return 0;
list_for_each_entry_rcu(e, list, list) {
- if ((e->rule.mask[word] & bit) == bit &&
+ if (audit_in_mask(&e->rule, ctx->major) &&
audit_filter_rules(tsk, &e->rule, ctx, n, &state, false)) {
ctx->current_state = state;
return 1;
diff --git a/kernel/capability.c b/kernel/capability.c
index 84b2bbf..a5cf13c 100644
--- a/kernel/capability.c
+++ b/kernel/capability.c
@@ -424,23 +424,19 @@
EXPORT_SYMBOL(capable);
/**
- * inode_capable - Check superior capability over inode
+ * capable_wrt_inode_uidgid - Check nsown_capable and uid and gid mapped
* @inode: The inode in question
* @cap: The capability in question
*
- * Return true if the current task has the given superior capability
- * targeted at it's own user namespace and that the given inode is owned
- * by the current user namespace or a child namespace.
- *
- * Currently we check to see if an inode is owned by the current
- * user namespace by seeing if the inode's owner maps into the
- * current user namespace.
- *
+ * Return true if the current task has the given capability targeted at
+ * its own user namespace and that the given inode's uid and gid are
+ * mapped into the current user namespace.
*/
-bool inode_capable(const struct inode *inode, int cap)
+bool capable_wrt_inode_uidgid(const struct inode *inode, int cap)
{
struct user_namespace *ns = current_user_ns();
- return ns_capable(ns, cap) && kuid_has_mapping(ns, inode->i_uid);
+ return ns_capable(ns, cap) && kuid_has_mapping(ns, inode->i_uid) &&
+ kgid_has_mapping(ns, inode->i_gid);
}
-EXPORT_SYMBOL(inode_capable);
+EXPORT_SYMBOL(capable_wrt_inode_uidgid);
diff --git a/kernel/cgroup.c b/kernel/cgroup.c
index ceee0c5..7868fc3 100644
--- a/kernel/cgroup.c
+++ b/kernel/cgroup.c
@@ -26,6 +26,8 @@
* distribution for more details.
*/
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/cgroup.h>
#include <linux/cred.h>
#include <linux/ctype.h>
@@ -70,15 +72,6 @@
MAX_CFTYPE_NAME + 2)
/*
- * cgroup_tree_mutex nests above cgroup_mutex and protects cftypes, file
- * creation/removal and hierarchy changing operations including cgroup
- * creation, removal, css association and controller rebinding. This outer
- * lock is needed mainly to resolve the circular dependency between kernfs
- * active ref and cgroup_mutex. cgroup_tree_mutex nests above both.
- */
-static DEFINE_MUTEX(cgroup_tree_mutex);
-
-/*
* cgroup_mutex is the master lock. Any modification to cgroup or its
* hierarchy must be performed while holding it.
*
@@ -99,16 +92,21 @@
#endif
/*
+ * Protects cgroup_idr and css_idr so that IDs can be released without
+ * grabbing cgroup_mutex.
+ */
+static DEFINE_SPINLOCK(cgroup_idr_lock);
+
+/*
* Protects cgroup_subsys->release_agent_path. Modifying it also requires
* cgroup_mutex. Reading requires either cgroup_mutex or this spinlock.
*/
static DEFINE_SPINLOCK(release_agent_path_lock);
-#define cgroup_assert_mutexes_or_rcu_locked() \
+#define cgroup_assert_mutex_or_rcu_locked() \
rcu_lockdep_assert(rcu_read_lock_held() || \
- lockdep_is_held(&cgroup_tree_mutex) || \
lockdep_is_held(&cgroup_mutex), \
- "cgroup_[tree_]mutex or RCU read lock required");
+ "cgroup_mutex or RCU read lock required");
/*
* cgroup destruction makes heavy use of work items and there can be a lot
@@ -151,6 +149,13 @@
*/
static bool cgrp_dfl_root_visible;
+/* some controllers are not supported in the default hierarchy */
+static const unsigned int cgrp_dfl_root_inhibit_ss_mask = 0
+#ifdef CONFIG_CGROUP_DEBUG
+ | (1 << debug_cgrp_id)
+#endif
+ ;
+
/* The list of hierarchy roots */
static LIST_HEAD(cgroup_roots);
@@ -160,14 +165,13 @@
static DEFINE_IDR(cgroup_hierarchy_idr);
/*
- * Assign a monotonically increasing serial number to cgroups. It
- * guarantees cgroups with bigger numbers are newer than those with smaller
- * numbers. Also, as cgroups are always appended to the parent's
- * ->children list, it guarantees that sibling cgroups are always sorted in
- * the ascending serial number order on the list. Protected by
- * cgroup_mutex.
+ * Assign a monotonically increasing serial number to csses. It guarantees
+ * cgroups with bigger numbers are newer than those with smaller numbers.
+ * Also, as csses are always appended to the parent's ->children list, it
+ * guarantees that sibling csses are always sorted in the ascending serial
+ * number order on the list. Protected by cgroup_mutex.
*/
-static u64 cgroup_serial_nr_next = 1;
+static u64 css_serial_nr_next = 1;
/* This flag indicates whether tasks in the fork and exit paths should
* check for fork/exit handlers to call. This avoids us having to do
@@ -180,17 +184,59 @@
static void cgroup_put(struct cgroup *cgrp);
static int rebind_subsystems(struct cgroup_root *dst_root,
- unsigned long ss_mask);
-static void cgroup_destroy_css_killed(struct cgroup *cgrp);
+ unsigned int ss_mask);
static int cgroup_destroy_locked(struct cgroup *cgrp);
+static int create_css(struct cgroup *cgrp, struct cgroup_subsys *ss);
+static void css_release(struct percpu_ref *ref);
+static void kill_css(struct cgroup_subsys_state *css);
static int cgroup_addrm_files(struct cgroup *cgrp, struct cftype cfts[],
bool is_add);
static void cgroup_pidlist_destroy_all(struct cgroup *cgrp);
+/* IDR wrappers which synchronize using cgroup_idr_lock */
+static int cgroup_idr_alloc(struct idr *idr, void *ptr, int start, int end,
+ gfp_t gfp_mask)
+{
+ int ret;
+
+ idr_preload(gfp_mask);
+ spin_lock_bh(&cgroup_idr_lock);
+ ret = idr_alloc(idr, ptr, start, end, gfp_mask);
+ spin_unlock_bh(&cgroup_idr_lock);
+ idr_preload_end();
+ return ret;
+}
+
+static void *cgroup_idr_replace(struct idr *idr, void *ptr, int id)
+{
+ void *ret;
+
+ spin_lock_bh(&cgroup_idr_lock);
+ ret = idr_replace(idr, ptr, id);
+ spin_unlock_bh(&cgroup_idr_lock);
+ return ret;
+}
+
+static void cgroup_idr_remove(struct idr *idr, int id)
+{
+ spin_lock_bh(&cgroup_idr_lock);
+ idr_remove(idr, id);
+ spin_unlock_bh(&cgroup_idr_lock);
+}
+
+static struct cgroup *cgroup_parent(struct cgroup *cgrp)
+{
+ struct cgroup_subsys_state *parent_css = cgrp->self.parent;
+
+ if (parent_css)
+ return container_of(parent_css, struct cgroup, self);
+ return NULL;
+}
+
/**
* cgroup_css - obtain a cgroup's css for the specified subsystem
* @cgrp: the cgroup of interest
- * @ss: the subsystem of interest (%NULL returns the dummy_css)
+ * @ss: the subsystem of interest (%NULL returns @cgrp->self)
*
* Return @cgrp's css (cgroup_subsys_state) associated with @ss. This
* function must be called either under cgroup_mutex or rcu_read_lock() and
@@ -203,23 +249,49 @@
{
if (ss)
return rcu_dereference_check(cgrp->subsys[ss->id],
- lockdep_is_held(&cgroup_tree_mutex) ||
lockdep_is_held(&cgroup_mutex));
else
- return &cgrp->dummy_css;
+ return &cgrp->self;
+}
+
+/**
+ * cgroup_e_css - obtain a cgroup's effective css for the specified subsystem
+ * @cgrp: the cgroup of interest
+ * @ss: the subsystem of interest (%NULL returns @cgrp->self)
+ *
+ * Similar to cgroup_css() but returns the effctive css, which is defined
+ * as the matching css of the nearest ancestor including self which has @ss
+ * enabled. If @ss is associated with the hierarchy @cgrp is on, this
+ * function is guaranteed to return non-NULL css.
+ */
+static struct cgroup_subsys_state *cgroup_e_css(struct cgroup *cgrp,
+ struct cgroup_subsys *ss)
+{
+ lockdep_assert_held(&cgroup_mutex);
+
+ if (!ss)
+ return &cgrp->self;
+
+ if (!(cgrp->root->subsys_mask & (1 << ss->id)))
+ return NULL;
+
+ while (cgroup_parent(cgrp) &&
+ !(cgroup_parent(cgrp)->child_subsys_mask & (1 << ss->id)))
+ cgrp = cgroup_parent(cgrp);
+
+ return cgroup_css(cgrp, ss);
}
/* convenient tests for these bits */
static inline bool cgroup_is_dead(const struct cgroup *cgrp)
{
- return test_bit(CGRP_DEAD, &cgrp->flags);
+ return !(cgrp->self.flags & CSS_ONLINE);
}
-struct cgroup_subsys_state *seq_css(struct seq_file *seq)
+struct cgroup_subsys_state *of_css(struct kernfs_open_file *of)
{
- struct kernfs_open_file *of = seq->private;
struct cgroup *cgrp = of->kn->parent->priv;
- struct cftype *cft = seq_cft(seq);
+ struct cftype *cft = of_cft(of);
/*
* This is open and unprotected implementation of cgroup_css().
@@ -232,9 +304,9 @@
if (cft->ss)
return rcu_dereference_raw(cgrp->subsys[cft->ss->id]);
else
- return &cgrp->dummy_css;
+ return &cgrp->self;
}
-EXPORT_SYMBOL_GPL(seq_css);
+EXPORT_SYMBOL_GPL(of_css);
/**
* cgroup_is_descendant - test ancestry
@@ -250,7 +322,7 @@
while (cgrp) {
if (cgrp == ancestor)
return true;
- cgrp = cgrp->parent;
+ cgrp = cgroup_parent(cgrp);
}
return false;
}
@@ -274,17 +346,30 @@
* @ssid: the index of the subsystem, CGROUP_SUBSYS_COUNT after reaching the end
* @cgrp: the target cgroup to iterate css's of
*
- * Should be called under cgroup_mutex.
+ * Should be called under cgroup_[tree_]mutex.
*/
#define for_each_css(css, ssid, cgrp) \
for ((ssid) = 0; (ssid) < CGROUP_SUBSYS_COUNT; (ssid)++) \
if (!((css) = rcu_dereference_check( \
(cgrp)->subsys[(ssid)], \
- lockdep_is_held(&cgroup_tree_mutex) || \
lockdep_is_held(&cgroup_mutex)))) { } \
else
/**
+ * for_each_e_css - iterate all effective css's of a cgroup
+ * @css: the iteration cursor
+ * @ssid: the index of the subsystem, CGROUP_SUBSYS_COUNT after reaching the end
+ * @cgrp: the target cgroup to iterate css's of
+ *
+ * Should be called under cgroup_[tree_]mutex.
+ */
+#define for_each_e_css(css, ssid, cgrp) \
+ for ((ssid) = 0; (ssid) < CGROUP_SUBSYS_COUNT; (ssid)++) \
+ if (!((css) = cgroup_e_css(cgrp, cgroup_subsys[(ssid)]))) \
+ ; \
+ else
+
+/**
* for_each_subsys - iterate all enabled cgroup subsystems
* @ss: the iteration cursor
* @ssid: the index of @ss, CGROUP_SUBSYS_COUNT after reaching the end
@@ -297,22 +382,13 @@
#define for_each_root(root) \
list_for_each_entry((root), &cgroup_roots, root_list)
-/**
- * cgroup_lock_live_group - take cgroup_mutex and check that cgrp is alive.
- * @cgrp: the cgroup to be checked for liveness
- *
- * On success, returns true; the mutex should be later unlocked. On
- * failure returns false with no lock held.
- */
-static bool cgroup_lock_live_group(struct cgroup *cgrp)
-{
- mutex_lock(&cgroup_mutex);
- if (cgroup_is_dead(cgrp)) {
- mutex_unlock(&cgroup_mutex);
- return false;
- }
- return true;
-}
+/* iterate over child cgrps, lock should be held throughout iteration */
+#define cgroup_for_each_live_child(child, cgrp) \
+ list_for_each_entry((child), &(cgrp)->self.children, self.sibling) \
+ if (({ lockdep_assert_held(&cgroup_mutex); \
+ cgroup_is_dead(child); })) \
+ ; \
+ else
/* the list of cgroups eligible for automatic release. Protected by
* release_list_lock */
@@ -360,6 +436,43 @@
static int css_set_count = 1; /* 1 for init_css_set */
+/**
+ * cgroup_update_populated - updated populated count of a cgroup
+ * @cgrp: the target cgroup
+ * @populated: inc or dec populated count
+ *
+ * @cgrp is either getting the first task (css_set) or losing the last.
+ * Update @cgrp->populated_cnt accordingly. The count is propagated
+ * towards root so that a given cgroup's populated_cnt is zero iff the
+ * cgroup and all its descendants are empty.
+ *
+ * @cgrp's interface file "cgroup.populated" is zero if
+ * @cgrp->populated_cnt is zero and 1 otherwise. When @cgrp->populated_cnt
+ * changes from or to zero, userland is notified that the content of the
+ * interface file has changed. This can be used to detect when @cgrp and
+ * its descendants become populated or empty.
+ */
+static void cgroup_update_populated(struct cgroup *cgrp, bool populated)
+{
+ lockdep_assert_held(&css_set_rwsem);
+
+ do {
+ bool trigger;
+
+ if (populated)
+ trigger = !cgrp->populated_cnt++;
+ else
+ trigger = !--cgrp->populated_cnt;
+
+ if (!trigger)
+ break;
+
+ if (cgrp->populated_kn)
+ kernfs_notify(cgrp->populated_kn);
+ cgrp = cgroup_parent(cgrp);
+ } while (cgrp);
+}
+
/*
* hash table for cgroup groups. This improves the performance to find
* an existing css_set. This hash doesn't (currently) take into
@@ -384,6 +497,8 @@
static void put_css_set_locked(struct css_set *cset, bool taskexit)
{
struct cgrp_cset_link *link, *tmp_link;
+ struct cgroup_subsys *ss;
+ int ssid;
lockdep_assert_held(&css_set_rwsem);
@@ -391,6 +506,8 @@
return;
/* This css_set is dead. unlink it and release cgroup refcounts */
+ for_each_subsys(ss, ssid)
+ list_del(&cset->e_cset_node[ssid]);
hash_del(&cset->hlist);
css_set_count--;
@@ -401,10 +518,13 @@
list_del(&link->cgrp_link);
/* @cgrp can't go away while we're holding css_set_rwsem */
- if (list_empty(&cgrp->cset_links) && notify_on_release(cgrp)) {
- if (taskexit)
- set_bit(CGRP_RELEASABLE, &cgrp->flags);
- check_for_release(cgrp);
+ if (list_empty(&cgrp->cset_links)) {
+ cgroup_update_populated(cgrp, false);
+ if (notify_on_release(cgrp)) {
+ if (taskexit)
+ set_bit(CGRP_RELEASABLE, &cgrp->flags);
+ check_for_release(cgrp);
+ }
}
kfree(link);
@@ -453,20 +573,20 @@
{
struct list_head *l1, *l2;
- if (memcmp(template, cset->subsys, sizeof(cset->subsys))) {
- /* Not all subsystems matched */
+ /*
+ * On the default hierarchy, there can be csets which are
+ * associated with the same set of cgroups but different csses.
+ * Let's first ensure that csses match.
+ */
+ if (memcmp(template, cset->subsys, sizeof(cset->subsys)))
return false;
- }
/*
* Compare cgroup pointers in order to distinguish between
- * different cgroups in heirarchies with no subsystems. We
- * could get by with just this check alone (and skip the
- * memcmp above) but on most setups the memcmp check will
- * avoid the need for this more expensive check on almost all
- * candidates.
+ * different cgroups in hierarchies. As different cgroups may
+ * share the same effective css, this comparison is always
+ * necessary.
*/
-
l1 = &cset->cgrp_links;
l2 = &old_cset->cgrp_links;
while (1) {
@@ -530,14 +650,17 @@
* won't change, so no need for locking.
*/
for_each_subsys(ss, i) {
- if (root->cgrp.subsys_mask & (1UL << i)) {
- /* Subsystem is in this hierarchy. So we want
- * the subsystem state from the new
- * cgroup */
- template[i] = cgroup_css(cgrp, ss);
+ if (root->subsys_mask & (1UL << i)) {
+ /*
+ * @ss is in this hierarchy, so we want the
+ * effective css from @cgrp.
+ */
+ template[i] = cgroup_e_css(cgrp, ss);
} else {
- /* Subsystem is not in this hierarchy, so we
- * don't want to change the subsystem state */
+ /*
+ * @ss is not in this hierarchy, so we don't want
+ * to change the css.
+ */
template[i] = old_cset->subsys[i];
}
}
@@ -603,10 +726,18 @@
struct cgrp_cset_link *link;
BUG_ON(list_empty(tmp_links));
+
+ if (cgroup_on_dfl(cgrp))
+ cset->dfl_cgrp = cgrp;
+
link = list_first_entry(tmp_links, struct cgrp_cset_link, cset_link);
link->cset = cset;
link->cgrp = cgrp;
+
+ if (list_empty(&cgrp->cset_links))
+ cgroup_update_populated(cgrp, true);
list_move(&link->cset_link, &cgrp->cset_links);
+
/*
* Always add links to the tail of the list so that the list
* is sorted by order of hierarchy creation
@@ -629,7 +760,9 @@
struct css_set *cset;
struct list_head tmp_links;
struct cgrp_cset_link *link;
+ struct cgroup_subsys *ss;
unsigned long key;
+ int ssid;
lockdep_assert_held(&cgroup_mutex);
@@ -680,10 +813,14 @@
css_set_count++;
- /* Add this cgroup group to the hash table */
+ /* Add @cset to the hash table */
key = css_set_hash(cset->subsys);
hash_add(css_set_table, &cset->hlist, key);
+ for_each_subsys(ss, ssid)
+ list_add_tail(&cset->e_cset_node[ssid],
+ &cset->subsys[ssid]->cgroup->e_csets[ssid]);
+
up_write(&css_set_rwsem);
return cset;
@@ -736,14 +873,13 @@
struct cgroup *cgrp = &root->cgrp;
struct cgrp_cset_link *link, *tmp_link;
- mutex_lock(&cgroup_tree_mutex);
mutex_lock(&cgroup_mutex);
BUG_ON(atomic_read(&root->nr_cgrps));
- BUG_ON(!list_empty(&cgrp->children));
+ BUG_ON(!list_empty(&cgrp->self.children));
/* Rebind all subsystems back to the default hierarchy */
- rebind_subsystems(&cgrp_dfl_root, cgrp->subsys_mask);
+ rebind_subsystems(&cgrp_dfl_root, root->subsys_mask);
/*
* Release all the links from cset_links to this hierarchy's
@@ -766,7 +902,6 @@
cgroup_exit_root_id(root);
mutex_unlock(&cgroup_mutex);
- mutex_unlock(&cgroup_tree_mutex);
kernfs_destroy_root(root->kf_root);
cgroup_free_root(root);
@@ -849,7 +984,7 @@
* update of a tasks cgroup pointer by cgroup_attach_task()
*/
-static int cgroup_populate_dir(struct cgroup *cgrp, unsigned long subsys_mask);
+static int cgroup_populate_dir(struct cgroup *cgrp, unsigned int subsys_mask);
static struct kernfs_syscall_ops cgroup_kf_syscall_ops;
static const struct file_operations proc_cgroupstats_operations;
@@ -884,79 +1019,95 @@
if (cft->read_u64 || cft->read_s64 || cft->seq_show)
mode |= S_IRUGO;
- if (cft->write_u64 || cft->write_s64 || cft->write_string ||
- cft->trigger)
+ if (cft->write_u64 || cft->write_s64 || cft->write)
mode |= S_IWUSR;
return mode;
}
-static void cgroup_free_fn(struct work_struct *work)
-{
- struct cgroup *cgrp = container_of(work, struct cgroup, destroy_work);
-
- atomic_dec(&cgrp->root->nr_cgrps);
- cgroup_pidlist_destroy_all(cgrp);
-
- if (cgrp->parent) {
- /*
- * We get a ref to the parent, and put the ref when this
- * cgroup is being freed, so it's guaranteed that the
- * parent won't be destroyed before its children.
- */
- cgroup_put(cgrp->parent);
- kernfs_put(cgrp->kn);
- kfree(cgrp);
- } else {
- /*
- * This is root cgroup's refcnt reaching zero, which
- * indicates that the root should be released.
- */
- cgroup_destroy_root(cgrp->root);
- }
-}
-
-static void cgroup_free_rcu(struct rcu_head *head)
-{
- struct cgroup *cgrp = container_of(head, struct cgroup, rcu_head);
-
- INIT_WORK(&cgrp->destroy_work, cgroup_free_fn);
- queue_work(cgroup_destroy_wq, &cgrp->destroy_work);
-}
-
static void cgroup_get(struct cgroup *cgrp)
{
WARN_ON_ONCE(cgroup_is_dead(cgrp));
- WARN_ON_ONCE(atomic_read(&cgrp->refcnt) <= 0);
- atomic_inc(&cgrp->refcnt);
+ css_get(&cgrp->self);
}
static void cgroup_put(struct cgroup *cgrp)
{
- if (!atomic_dec_and_test(&cgrp->refcnt))
- return;
- if (WARN_ON_ONCE(cgrp->parent && !cgroup_is_dead(cgrp)))
- return;
+ css_put(&cgrp->self);
+}
+
+/**
+ * cgroup_kn_unlock - unlocking helper for cgroup kernfs methods
+ * @kn: the kernfs_node being serviced
+ *
+ * This helper undoes cgroup_kn_lock_live() and should be invoked before
+ * the method finishes if locking succeeded. Note that once this function
+ * returns the cgroup returned by cgroup_kn_lock_live() may become
+ * inaccessible any time. If the caller intends to continue to access the
+ * cgroup, it should pin it before invoking this function.
+ */
+static void cgroup_kn_unlock(struct kernfs_node *kn)
+{
+ struct cgroup *cgrp;
+
+ if (kernfs_type(kn) == KERNFS_DIR)
+ cgrp = kn->priv;
+ else
+ cgrp = kn->parent->priv;
+
+ mutex_unlock(&cgroup_mutex);
+
+ kernfs_unbreak_active_protection(kn);
+ cgroup_put(cgrp);
+}
+
+/**
+ * cgroup_kn_lock_live - locking helper for cgroup kernfs methods
+ * @kn: the kernfs_node being serviced
+ *
+ * This helper is to be used by a cgroup kernfs method currently servicing
+ * @kn. It breaks the active protection, performs cgroup locking and
+ * verifies that the associated cgroup is alive. Returns the cgroup if
+ * alive; otherwise, %NULL. A successful return should be undone by a
+ * matching cgroup_kn_unlock() invocation.
+ *
+ * Any cgroup kernfs method implementation which requires locking the
+ * associated cgroup should use this helper. It avoids nesting cgroup
+ * locking under kernfs active protection and allows all kernfs operations
+ * including self-removal.
+ */
+static struct cgroup *cgroup_kn_lock_live(struct kernfs_node *kn)
+{
+ struct cgroup *cgrp;
+
+ if (kernfs_type(kn) == KERNFS_DIR)
+ cgrp = kn->priv;
+ else
+ cgrp = kn->parent->priv;
/*
- * XXX: cgrp->id is only used to look up css's. As cgroup and
- * css's lifetimes will be decoupled, it should be made
- * per-subsystem and moved to css->id so that lookups are
- * successful until the target css is released.
+ * We're gonna grab cgroup_mutex which nests outside kernfs
+ * active_ref. cgroup liveliness check alone provides enough
+ * protection against removal. Ensure @cgrp stays accessible and
+ * break the active_ref protection.
*/
- mutex_lock(&cgroup_mutex);
- idr_remove(&cgrp->root->cgroup_idr, cgrp->id);
- mutex_unlock(&cgroup_mutex);
- cgrp->id = -1;
+ cgroup_get(cgrp);
+ kernfs_break_active_protection(kn);
- call_rcu(&cgrp->rcu_head, cgroup_free_rcu);
+ mutex_lock(&cgroup_mutex);
+
+ if (!cgroup_is_dead(cgrp))
+ return cgrp;
+
+ cgroup_kn_unlock(kn);
+ return NULL;
}
static void cgroup_rm_file(struct cgroup *cgrp, const struct cftype *cft)
{
char name[CGROUP_FILE_NAME_MAX];
- lockdep_assert_held(&cgroup_tree_mutex);
+ lockdep_assert_held(&cgroup_mutex);
kernfs_remove_by_name(cgrp->kn, cgroup_file_name(cgrp, cft, name));
}
@@ -965,7 +1116,7 @@
* @cgrp: target cgroup
* @subsys_mask: mask of the subsystem ids whose files should be removed
*/
-static void cgroup_clear_dir(struct cgroup *cgrp, unsigned long subsys_mask)
+static void cgroup_clear_dir(struct cgroup *cgrp, unsigned int subsys_mask)
{
struct cgroup_subsys *ss;
int i;
@@ -973,40 +1124,40 @@
for_each_subsys(ss, i) {
struct cftype *cfts;
- if (!test_bit(i, &subsys_mask))
+ if (!(subsys_mask & (1 << i)))
continue;
list_for_each_entry(cfts, &ss->cfts, node)
cgroup_addrm_files(cgrp, cfts, false);
}
}
-static int rebind_subsystems(struct cgroup_root *dst_root,
- unsigned long ss_mask)
+static int rebind_subsystems(struct cgroup_root *dst_root, unsigned int ss_mask)
{
struct cgroup_subsys *ss;
- int ssid, ret;
+ unsigned int tmp_ss_mask;
+ int ssid, i, ret;
- lockdep_assert_held(&cgroup_tree_mutex);
lockdep_assert_held(&cgroup_mutex);
for_each_subsys(ss, ssid) {
if (!(ss_mask & (1 << ssid)))
continue;
- /* if @ss is on the dummy_root, we can always move it */
- if (ss->root == &cgrp_dfl_root)
- continue;
-
- /* if @ss has non-root cgroups attached to it, can't move */
- if (!list_empty(&ss->root->cgrp.children))
+ /* if @ss has non-root csses attached to it, can't move */
+ if (css_next_child(NULL, cgroup_css(&ss->root->cgrp, ss)))
return -EBUSY;
/* can't move between two non-dummy roots either */
- if (dst_root != &cgrp_dfl_root)
+ if (ss->root != &cgrp_dfl_root && dst_root != &cgrp_dfl_root)
return -EBUSY;
}
- ret = cgroup_populate_dir(&dst_root->cgrp, ss_mask);
+ /* skip creating root files on dfl_root for inhibited subsystems */
+ tmp_ss_mask = ss_mask;
+ if (dst_root == &cgrp_dfl_root)
+ tmp_ss_mask &= ~cgrp_dfl_root_inhibit_ss_mask;
+
+ ret = cgroup_populate_dir(&dst_root->cgrp, tmp_ss_mask);
if (ret) {
if (dst_root != &cgrp_dfl_root)
return ret;
@@ -1018,9 +1169,9 @@
* Just warn about it and continue.
*/
if (cgrp_dfl_root_visible) {
- pr_warning("cgroup: failed to create files (%d) while rebinding 0x%lx to default root\n",
- ret, ss_mask);
- pr_warning("cgroup: you may retry by moving them to a different hierarchy and unbinding\n");
+ pr_warn("failed to create files (%d) while rebinding 0x%x to default root\n",
+ ret, ss_mask);
+ pr_warn("you may retry by moving them to a different hierarchy and unbinding\n");
}
}
@@ -1028,15 +1179,14 @@
* Nothing can fail from this point on. Remove files for the
* removed subsystems and rebind each subsystem.
*/
- mutex_unlock(&cgroup_mutex);
for_each_subsys(ss, ssid)
if (ss_mask & (1 << ssid))
cgroup_clear_dir(&ss->root->cgrp, 1 << ssid);
- mutex_lock(&cgroup_mutex);
for_each_subsys(ss, ssid) {
struct cgroup_root *src_root;
struct cgroup_subsys_state *css;
+ struct css_set *cset;
if (!(ss_mask & (1 << ssid)))
continue;
@@ -1051,8 +1201,19 @@
ss->root = dst_root;
css->cgroup = &dst_root->cgrp;
- src_root->cgrp.subsys_mask &= ~(1 << ssid);
- dst_root->cgrp.subsys_mask |= 1 << ssid;
+ down_write(&css_set_rwsem);
+ hash_for_each(css_set_table, i, cset, hlist)
+ list_move_tail(&cset->e_cset_node[ss->id],
+ &dst_root->cgrp.e_csets[ss->id]);
+ up_write(&css_set_rwsem);
+
+ src_root->subsys_mask &= ~(1 << ssid);
+ src_root->cgrp.child_subsys_mask &= ~(1 << ssid);
+
+ /* default hierarchy doesn't enable controllers by default */
+ dst_root->subsys_mask |= 1 << ssid;
+ if (dst_root != &cgrp_dfl_root)
+ dst_root->cgrp.child_subsys_mask |= 1 << ssid;
if (ss->bind)
ss->bind(css);
@@ -1070,7 +1231,7 @@
int ssid;
for_each_subsys(ss, ssid)
- if (root->cgrp.subsys_mask & (1 << ssid))
+ if (root->subsys_mask & (1 << ssid))
seq_printf(seq, ",%s", ss->name);
if (root->flags & CGRP_ROOT_SANE_BEHAVIOR)
seq_puts(seq, ",sane_behavior");
@@ -1092,8 +1253,8 @@
}
struct cgroup_sb_opts {
- unsigned long subsys_mask;
- unsigned long flags;
+ unsigned int subsys_mask;
+ unsigned int flags;
char *release_agent;
bool cpuset_clone_children;
char *name;
@@ -1101,24 +1262,16 @@
bool none;
};
-/*
- * Convert a hierarchy specifier into a bitmask of subsystems and
- * flags. Call with cgroup_mutex held to protect the cgroup_subsys[]
- * array. This function takes refcounts on subsystems to be used, unless it
- * returns error, in which case no refcounts are taken.
- */
static int parse_cgroupfs_options(char *data, struct cgroup_sb_opts *opts)
{
char *token, *o = data;
bool all_ss = false, one_ss = false;
- unsigned long mask = (unsigned long)-1;
+ unsigned int mask = -1U;
struct cgroup_subsys *ss;
int i;
- BUG_ON(!mutex_is_locked(&cgroup_mutex));
-
#ifdef CONFIG_CPUSETS
- mask = ~(1UL << cpuset_cgrp_id);
+ mask = ~(1U << cpuset_cgrp_id);
#endif
memset(opts, 0, sizeof(*opts));
@@ -1199,7 +1352,7 @@
/* Mutually exclusive option 'all' + subsystem name */
if (all_ss)
return -EINVAL;
- set_bit(i, &opts->subsys_mask);
+ opts->subsys_mask |= (1 << i);
one_ss = true;
break;
@@ -1211,12 +1364,12 @@
/* Consistency checks */
if (opts->flags & CGRP_ROOT_SANE_BEHAVIOR) {
- pr_warning("cgroup: sane_behavior: this is still under development and its behaviors will change, proceed at your own risk\n");
+ pr_warn("sane_behavior: this is still under development and its behaviors will change, proceed at your own risk\n");
if ((opts->flags & (CGRP_ROOT_NOPREFIX | CGRP_ROOT_XATTR)) ||
opts->cpuset_clone_children || opts->release_agent ||
opts->name) {
- pr_err("cgroup: sane_behavior: noprefix, xattr, clone_children, release_agent and name are not allowed\n");
+ pr_err("sane_behavior: noprefix, xattr, clone_children, release_agent and name are not allowed\n");
return -EINVAL;
}
} else {
@@ -1228,7 +1381,7 @@
if (all_ss || (!one_ss && !opts->none && !opts->name))
for_each_subsys(ss, i)
if (!ss->disabled)
- set_bit(i, &opts->subsys_mask);
+ opts->subsys_mask |= (1 << i);
/*
* We either have to specify by name or by subsystems. (So
@@ -1259,14 +1412,13 @@
int ret = 0;
struct cgroup_root *root = cgroup_root_from_kf(kf_root);
struct cgroup_sb_opts opts;
- unsigned long added_mask, removed_mask;
+ unsigned int added_mask, removed_mask;
if (root->flags & CGRP_ROOT_SANE_BEHAVIOR) {
- pr_err("cgroup: sane_behavior: remount is not allowed\n");
+ pr_err("sane_behavior: remount is not allowed\n");
return -EINVAL;
}
- mutex_lock(&cgroup_tree_mutex);
mutex_lock(&cgroup_mutex);
/* See what subsystems are wanted */
@@ -1274,17 +1426,17 @@
if (ret)
goto out_unlock;
- if (opts.subsys_mask != root->cgrp.subsys_mask || opts.release_agent)
- pr_warning("cgroup: option changes via remount are deprecated (pid=%d comm=%s)\n",
- task_tgid_nr(current), current->comm);
+ if (opts.subsys_mask != root->subsys_mask || opts.release_agent)
+ pr_warn("option changes via remount are deprecated (pid=%d comm=%s)\n",
+ task_tgid_nr(current), current->comm);
- added_mask = opts.subsys_mask & ~root->cgrp.subsys_mask;
- removed_mask = root->cgrp.subsys_mask & ~opts.subsys_mask;
+ added_mask = opts.subsys_mask & ~root->subsys_mask;
+ removed_mask = root->subsys_mask & ~opts.subsys_mask;
/* Don't allow flags or name to change at remount */
if (((opts.flags ^ root->flags) & CGRP_ROOT_OPTION_MASK) ||
(opts.name && strcmp(opts.name, root->name))) {
- pr_err("cgroup: option or name mismatch, new: 0x%lx \"%s\", old: 0x%lx \"%s\"\n",
+ pr_err("option or name mismatch, new: 0x%x \"%s\", old: 0x%x \"%s\"\n",
opts.flags & CGRP_ROOT_OPTION_MASK, opts.name ?: "",
root->flags & CGRP_ROOT_OPTION_MASK, root->name);
ret = -EINVAL;
@@ -1292,7 +1444,7 @@
}
/* remounting is not allowed for populated hierarchies */
- if (!list_empty(&root->cgrp.children)) {
+ if (!list_empty(&root->cgrp.self.children)) {
ret = -EBUSY;
goto out_unlock;
}
@@ -1312,7 +1464,6 @@
kfree(opts.release_agent);
kfree(opts.name);
mutex_unlock(&cgroup_mutex);
- mutex_unlock(&cgroup_tree_mutex);
return ret;
}
@@ -1370,14 +1521,22 @@
static void init_cgroup_housekeeping(struct cgroup *cgrp)
{
- atomic_set(&cgrp->refcnt, 1);
- INIT_LIST_HEAD(&cgrp->sibling);
- INIT_LIST_HEAD(&cgrp->children);
+ struct cgroup_subsys *ss;
+ int ssid;
+
+ INIT_LIST_HEAD(&cgrp->self.sibling);
+ INIT_LIST_HEAD(&cgrp->self.children);
INIT_LIST_HEAD(&cgrp->cset_links);
INIT_LIST_HEAD(&cgrp->release_list);
INIT_LIST_HEAD(&cgrp->pidlists);
mutex_init(&cgrp->pidlist_mutex);
- cgrp->dummy_css.cgroup = cgrp;
+ cgrp->self.cgroup = cgrp;
+ cgrp->self.flags |= CSS_ONLINE;
+
+ for_each_subsys(ss, ssid)
+ INIT_LIST_HEAD(&cgrp->e_csets[ssid]);
+
+ init_waitqueue_head(&cgrp->offline_waitq);
}
static void init_cgroup_root(struct cgroup_root *root,
@@ -1400,21 +1559,24 @@
set_bit(CGRP_CPUSET_CLONE_CHILDREN, &root->cgrp.flags);
}
-static int cgroup_setup_root(struct cgroup_root *root, unsigned long ss_mask)
+static int cgroup_setup_root(struct cgroup_root *root, unsigned int ss_mask)
{
LIST_HEAD(tmp_links);
struct cgroup *root_cgrp = &root->cgrp;
struct css_set *cset;
int i, ret;
- lockdep_assert_held(&cgroup_tree_mutex);
lockdep_assert_held(&cgroup_mutex);
- ret = idr_alloc(&root->cgroup_idr, root_cgrp, 0, 1, GFP_KERNEL);
+ ret = cgroup_idr_alloc(&root->cgroup_idr, root_cgrp, 1, 2, GFP_NOWAIT);
if (ret < 0)
goto out;
root_cgrp->id = ret;
+ ret = percpu_ref_init(&root_cgrp->self.refcnt, css_release);
+ if (ret)
+ goto out;
+
/*
* We're accessing css_set_count without locking css_set_rwsem here,
* but that's OK - it can only be increased by someone holding
@@ -1423,11 +1585,11 @@
*/
ret = allocate_cgrp_cset_links(css_set_count, &tmp_links);
if (ret)
- goto out;
+ goto cancel_ref;
ret = cgroup_init_root_id(root);
if (ret)
- goto out;
+ goto cancel_ref;
root->kf_root = kernfs_create_root(&cgroup_kf_syscall_ops,
KERNFS_ROOT_CREATE_DEACTIVATED,
@@ -1463,7 +1625,7 @@
link_css_set(&tmp_links, cset, root_cgrp);
up_write(&css_set_rwsem);
- BUG_ON(!list_empty(&root_cgrp->children));
+ BUG_ON(!list_empty(&root_cgrp->self.children));
BUG_ON(atomic_read(&root->nr_cgrps) != 1);
kernfs_activate(root_cgrp->kn);
@@ -1475,6 +1637,8 @@
root->kf_root = NULL;
exit_root_id:
cgroup_exit_root_id(root);
+cancel_ref:
+ percpu_ref_cancel_init(&root_cgrp->self.refcnt);
out:
free_cgrp_cset_links(&tmp_links);
return ret;
@@ -1497,14 +1661,13 @@
if (!use_task_css_set_links)
cgroup_enable_task_cg_lists();
- mutex_lock(&cgroup_tree_mutex);
mutex_lock(&cgroup_mutex);
/* First find the desired set of subsystems */
ret = parse_cgroupfs_options(data, &opts);
if (ret)
goto out_unlock;
-retry:
+
/* look for a matching existing root */
if (!opts.subsys_mask && !opts.none && !opts.name) {
cgrp_dfl_root_visible = true;
@@ -1536,7 +1699,7 @@
* subsystems) then they must match.
*/
if ((opts.subsys_mask || opts.none) &&
- (opts.subsys_mask != root->cgrp.subsys_mask)) {
+ (opts.subsys_mask != root->subsys_mask)) {
if (!name_match)
continue;
ret = -EBUSY;
@@ -1545,28 +1708,27 @@
if ((root->flags ^ opts.flags) & CGRP_ROOT_OPTION_MASK) {
if ((root->flags | opts.flags) & CGRP_ROOT_SANE_BEHAVIOR) {
- pr_err("cgroup: sane_behavior: new mount options should match the existing superblock\n");
+ pr_err("sane_behavior: new mount options should match the existing superblock\n");
ret = -EINVAL;
goto out_unlock;
} else {
- pr_warning("cgroup: new mount options do not match the existing superblock, will be ignored\n");
+ pr_warn("new mount options do not match the existing superblock, will be ignored\n");
}
}
/*
- * A root's lifetime is governed by its root cgroup. Zero
- * ref indicate that the root is being destroyed. Wait for
- * destruction to complete so that the subsystems are free.
- * We can use wait_queue for the wait but this path is
- * super cold. Let's just sleep for a bit and retry.
+ * A root's lifetime is governed by its root cgroup.
+ * tryget_live failure indicate that the root is being
+ * destroyed. Wait for destruction to complete so that the
+ * subsystems are free. We can use wait_queue for the wait
+ * but this path is super cold. Let's just sleep for a bit
+ * and retry.
*/
- if (!atomic_inc_not_zero(&root->cgrp.refcnt)) {
+ if (!percpu_ref_tryget_live(&root->cgrp.self.refcnt)) {
mutex_unlock(&cgroup_mutex);
- mutex_unlock(&cgroup_tree_mutex);
msleep(10);
- mutex_lock(&cgroup_tree_mutex);
- mutex_lock(&cgroup_mutex);
- goto retry;
+ ret = restart_syscall();
+ goto out_free;
}
ret = 0;
@@ -1597,8 +1759,7 @@
out_unlock:
mutex_unlock(&cgroup_mutex);
- mutex_unlock(&cgroup_tree_mutex);
-
+out_free:
kfree(opts.release_agent);
kfree(opts.name);
@@ -1617,7 +1778,19 @@
struct kernfs_root *kf_root = kernfs_root_from_sb(sb);
struct cgroup_root *root = cgroup_root_from_kf(kf_root);
- cgroup_put(&root->cgrp);
+ /*
+ * If @root doesn't have any mounts or children, start killing it.
+ * This prevents new mounts by disabling percpu_ref_tryget_live().
+ * cgroup_mount() may wait for @root's release.
+ *
+ * And don't kill the default root.
+ */
+ if (css_has_online_children(&root->cgrp.self) ||
+ root == &cgrp_dfl_root)
+ cgroup_put(&root->cgrp);
+ else
+ percpu_ref_kill(&root->cgrp.self.refcnt);
+
kernfs_kill_sb(sb);
}
@@ -1739,7 +1912,7 @@
/**
* cgroup_task_migrate - move a task from one cgroup to another.
- * @old_cgrp; the cgroup @tsk is being migrated from
+ * @old_cgrp: the cgroup @tsk is being migrated from
* @tsk: the task being migrated
* @new_cset: the new css_set @tsk is being attached to
*
@@ -1831,10 +2004,6 @@
src_cgrp = cset_cgroup_from_root(src_cset, dst_cgrp->root);
- /* nothing to do if this cset already belongs to the cgroup */
- if (src_cgrp == dst_cgrp)
- return;
-
if (!list_empty(&src_cset->mg_preload_node))
return;
@@ -1849,13 +2018,14 @@
/**
* cgroup_migrate_prepare_dst - prepare destination css_sets for migration
- * @dst_cgrp: the destination cgroup
+ * @dst_cgrp: the destination cgroup (may be %NULL)
* @preloaded_csets: list of preloaded source css_sets
*
* Tasks are about to be moved to @dst_cgrp and all the source css_sets
* have been preloaded to @preloaded_csets. This function looks up and
- * pins all destination css_sets, links each to its source, and put them on
- * @preloaded_csets.
+ * pins all destination css_sets, links each to its source, and append them
+ * to @preloaded_csets. If @dst_cgrp is %NULL, the destination of each
+ * source css_set is assumed to be its cgroup on the default hierarchy.
*
* This function must be called after cgroup_migrate_add_src() has been
* called on each migration source css_set. After migration is performed
@@ -1866,19 +2036,42 @@
struct list_head *preloaded_csets)
{
LIST_HEAD(csets);
- struct css_set *src_cset;
+ struct css_set *src_cset, *tmp_cset;
lockdep_assert_held(&cgroup_mutex);
+ /*
+ * Except for the root, child_subsys_mask must be zero for a cgroup
+ * with tasks so that child cgroups don't compete against tasks.
+ */
+ if (dst_cgrp && cgroup_on_dfl(dst_cgrp) && cgroup_parent(dst_cgrp) &&
+ dst_cgrp->child_subsys_mask)
+ return -EBUSY;
+
/* look up the dst cset for each src cset and link it to src */
- list_for_each_entry(src_cset, preloaded_csets, mg_preload_node) {
+ list_for_each_entry_safe(src_cset, tmp_cset, preloaded_csets, mg_preload_node) {
struct css_set *dst_cset;
- dst_cset = find_css_set(src_cset, dst_cgrp);
+ dst_cset = find_css_set(src_cset,
+ dst_cgrp ?: src_cset->dfl_cgrp);
if (!dst_cset)
goto err;
WARN_ON_ONCE(src_cset->mg_dst_cset || dst_cset->mg_dst_cset);
+
+ /*
+ * If src cset equals dst, it's noop. Drop the src.
+ * cgroup_migrate() will skip the cset too. Note that we
+ * can't handle src == dst as some nodes are used by both.
+ */
+ if (src_cset == dst_cset) {
+ src_cset->mg_src_cgrp = NULL;
+ list_del_init(&src_cset->mg_preload_node);
+ put_css_set(src_cset, false);
+ put_css_set(dst_cset, false);
+ continue;
+ }
+
src_cset->mg_dst_cset = dst_cset;
if (list_empty(&dst_cset->mg_preload_node))
@@ -1887,7 +2080,7 @@
put_css_set(dst_cset, false);
}
- list_splice(&csets, preloaded_csets);
+ list_splice_tail(&csets, preloaded_csets);
return 0;
err:
cgroup_migrate_finish(&csets);
@@ -1968,7 +2161,7 @@
return 0;
/* check that we can legitimately attach to the cgroup */
- for_each_css(css, i, cgrp) {
+ for_each_e_css(css, i, cgrp) {
if (css->ss->can_attach) {
ret = css->ss->can_attach(css, &tset);
if (ret) {
@@ -1998,7 +2191,7 @@
*/
tset.csets = &tset.dst_csets;
- for_each_css(css, i, cgrp)
+ for_each_e_css(css, i, cgrp)
if (css->ss->attach)
css->ss->attach(css, &tset);
@@ -2006,7 +2199,7 @@
goto out_release_tset;
out_cancel_attach:
- for_each_css(css, i, cgrp) {
+ for_each_e_css(css, i, cgrp) {
if (css == failed_css)
break;
if (css->ss->cancel_attach)
@@ -2065,13 +2258,20 @@
* function to attach either it or all tasks in its threadgroup. Will lock
* cgroup_mutex and threadgroup.
*/
-static int attach_task_by_pid(struct cgroup *cgrp, u64 pid, bool threadgroup)
+static ssize_t __cgroup_procs_write(struct kernfs_open_file *of, char *buf,
+ size_t nbytes, loff_t off, bool threadgroup)
{
struct task_struct *tsk;
const struct cred *cred = current_cred(), *tcred;
+ struct cgroup *cgrp;
+ pid_t pid;
int ret;
- if (!cgroup_lock_live_group(cgrp))
+ if (kstrtoint(strstrip(buf), 0, &pid) || pid < 0)
+ return -EINVAL;
+
+ cgrp = cgroup_kn_lock_live(of->kn);
+ if (!cgrp)
return -ENODEV;
retry_find_task:
@@ -2137,8 +2337,8 @@
put_task_struct(tsk);
out_unlock_cgroup:
- mutex_unlock(&cgroup_mutex);
- return ret;
+ cgroup_kn_unlock(of->kn);
+ return ret ?: nbytes;
}
/**
@@ -2172,43 +2372,44 @@
}
EXPORT_SYMBOL_GPL(cgroup_attach_task_all);
-static int cgroup_tasks_write(struct cgroup_subsys_state *css,
- struct cftype *cft, u64 pid)
+static ssize_t cgroup_tasks_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
- return attach_task_by_pid(css->cgroup, pid, false);
+ return __cgroup_procs_write(of, buf, nbytes, off, false);
}
-static int cgroup_procs_write(struct cgroup_subsys_state *css,
- struct cftype *cft, u64 tgid)
+static ssize_t cgroup_procs_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
- return attach_task_by_pid(css->cgroup, tgid, true);
+ return __cgroup_procs_write(of, buf, nbytes, off, true);
}
-static int cgroup_release_agent_write(struct cgroup_subsys_state *css,
- struct cftype *cft, char *buffer)
+static ssize_t cgroup_release_agent_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
- struct cgroup_root *root = css->cgroup->root;
+ struct cgroup *cgrp;
- BUILD_BUG_ON(sizeof(root->release_agent_path) < PATH_MAX);
- if (!cgroup_lock_live_group(css->cgroup))
+ BUILD_BUG_ON(sizeof(cgrp->root->release_agent_path) < PATH_MAX);
+
+ cgrp = cgroup_kn_lock_live(of->kn);
+ if (!cgrp)
return -ENODEV;
spin_lock(&release_agent_path_lock);
- strlcpy(root->release_agent_path, buffer,
- sizeof(root->release_agent_path));
+ strlcpy(cgrp->root->release_agent_path, strstrip(buf),
+ sizeof(cgrp->root->release_agent_path));
spin_unlock(&release_agent_path_lock);
- mutex_unlock(&cgroup_mutex);
- return 0;
+ cgroup_kn_unlock(of->kn);
+ return nbytes;
}
static int cgroup_release_agent_show(struct seq_file *seq, void *v)
{
struct cgroup *cgrp = seq_css(seq)->cgroup;
- if (!cgroup_lock_live_group(cgrp))
- return -ENODEV;
+ spin_lock(&release_agent_path_lock);
seq_puts(seq, cgrp->root->release_agent_path);
+ spin_unlock(&release_agent_path_lock);
seq_putc(seq, '\n');
- mutex_unlock(&cgroup_mutex);
return 0;
}
@@ -2220,6 +2421,320 @@
return 0;
}
+static void cgroup_print_ss_mask(struct seq_file *seq, unsigned int ss_mask)
+{
+ struct cgroup_subsys *ss;
+ bool printed = false;
+ int ssid;
+
+ for_each_subsys(ss, ssid) {
+ if (ss_mask & (1 << ssid)) {
+ if (printed)
+ seq_putc(seq, ' ');
+ seq_printf(seq, "%s", ss->name);
+ printed = true;
+ }
+ }
+ if (printed)
+ seq_putc(seq, '\n');
+}
+
+/* show controllers which are currently attached to the default hierarchy */
+static int cgroup_root_controllers_show(struct seq_file *seq, void *v)
+{
+ struct cgroup *cgrp = seq_css(seq)->cgroup;
+
+ cgroup_print_ss_mask(seq, cgrp->root->subsys_mask &
+ ~cgrp_dfl_root_inhibit_ss_mask);
+ return 0;
+}
+
+/* show controllers which are enabled from the parent */
+static int cgroup_controllers_show(struct seq_file *seq, void *v)
+{
+ struct cgroup *cgrp = seq_css(seq)->cgroup;
+
+ cgroup_print_ss_mask(seq, cgroup_parent(cgrp)->child_subsys_mask);
+ return 0;
+}
+
+/* show controllers which are enabled for a given cgroup's children */
+static int cgroup_subtree_control_show(struct seq_file *seq, void *v)
+{
+ struct cgroup *cgrp = seq_css(seq)->cgroup;
+
+ cgroup_print_ss_mask(seq, cgrp->child_subsys_mask);
+ return 0;
+}
+
+/**
+ * cgroup_update_dfl_csses - update css assoc of a subtree in default hierarchy
+ * @cgrp: root of the subtree to update csses for
+ *
+ * @cgrp's child_subsys_mask has changed and its subtree's (self excluded)
+ * css associations need to be updated accordingly. This function looks up
+ * all css_sets which are attached to the subtree, creates the matching
+ * updated css_sets and migrates the tasks to the new ones.
+ */
+static int cgroup_update_dfl_csses(struct cgroup *cgrp)
+{
+ LIST_HEAD(preloaded_csets);
+ struct cgroup_subsys_state *css;
+ struct css_set *src_cset;
+ int ret;
+
+ lockdep_assert_held(&cgroup_mutex);
+
+ /* look up all csses currently attached to @cgrp's subtree */
+ down_read(&css_set_rwsem);
+ css_for_each_descendant_pre(css, cgroup_css(cgrp, NULL)) {
+ struct cgrp_cset_link *link;
+
+ /* self is not affected by child_subsys_mask change */
+ if (css->cgroup == cgrp)
+ continue;
+
+ list_for_each_entry(link, &css->cgroup->cset_links, cset_link)
+ cgroup_migrate_add_src(link->cset, cgrp,
+ &preloaded_csets);
+ }
+ up_read(&css_set_rwsem);
+
+ /* NULL dst indicates self on default hierarchy */
+ ret = cgroup_migrate_prepare_dst(NULL, &preloaded_csets);
+ if (ret)
+ goto out_finish;
+
+ list_for_each_entry(src_cset, &preloaded_csets, mg_preload_node) {
+ struct task_struct *last_task = NULL, *task;
+
+ /* src_csets precede dst_csets, break on the first dst_cset */
+ if (!src_cset->mg_src_cgrp)
+ break;
+
+ /*
+ * All tasks in src_cset need to be migrated to the
+ * matching dst_cset. Empty it process by process. We
+ * walk tasks but migrate processes. The leader might even
+ * belong to a different cset but such src_cset would also
+ * be among the target src_csets because the default
+ * hierarchy enforces per-process membership.
+ */
+ while (true) {
+ down_read(&css_set_rwsem);
+ task = list_first_entry_or_null(&src_cset->tasks,
+ struct task_struct, cg_list);
+ if (task) {
+ task = task->group_leader;
+ WARN_ON_ONCE(!task_css_set(task)->mg_src_cgrp);
+ get_task_struct(task);
+ }
+ up_read(&css_set_rwsem);
+
+ if (!task)
+ break;
+
+ /* guard against possible infinite loop */
+ if (WARN(last_task == task,
+ "cgroup: update_dfl_csses failed to make progress, aborting in inconsistent state\n"))
+ goto out_finish;
+ last_task = task;
+
+ threadgroup_lock(task);
+ /* raced against de_thread() from another thread? */
+ if (!thread_group_leader(task)) {
+ threadgroup_unlock(task);
+ put_task_struct(task);
+ continue;
+ }
+
+ ret = cgroup_migrate(src_cset->dfl_cgrp, task, true);
+
+ threadgroup_unlock(task);
+ put_task_struct(task);
+
+ if (WARN(ret, "cgroup: failed to update controllers for the default hierarchy (%d), further operations may crash or hang\n", ret))
+ goto out_finish;
+ }
+ }
+
+out_finish:
+ cgroup_migrate_finish(&preloaded_csets);
+ return ret;
+}
+
+/* change the enabled child controllers for a cgroup in the default hierarchy */
+static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes,
+ loff_t off)
+{
+ unsigned int enable = 0, disable = 0;
+ struct cgroup *cgrp, *child;
+ struct cgroup_subsys *ss;
+ char *tok;
+ int ssid, ret;
+
+ /*
+ * Parse input - space separated list of subsystem names prefixed
+ * with either + or -.
+ */
+ buf = strstrip(buf);
+ while ((tok = strsep(&buf, " "))) {
+ if (tok[0] == '\0')
+ continue;
+ for_each_subsys(ss, ssid) {
+ if (ss->disabled || strcmp(tok + 1, ss->name) ||
+ ((1 << ss->id) & cgrp_dfl_root_inhibit_ss_mask))
+ continue;
+
+ if (*tok == '+') {
+ enable |= 1 << ssid;
+ disable &= ~(1 << ssid);
+ } else if (*tok == '-') {
+ disable |= 1 << ssid;
+ enable &= ~(1 << ssid);
+ } else {
+ return -EINVAL;
+ }
+ break;
+ }
+ if (ssid == CGROUP_SUBSYS_COUNT)
+ return -EINVAL;
+ }
+
+ cgrp = cgroup_kn_lock_live(of->kn);
+ if (!cgrp)
+ return -ENODEV;
+
+ for_each_subsys(ss, ssid) {
+ if (enable & (1 << ssid)) {
+ if (cgrp->child_subsys_mask & (1 << ssid)) {
+ enable &= ~(1 << ssid);
+ continue;
+ }
+
+ /*
+ * Because css offlining is asynchronous, userland
+ * might try to re-enable the same controller while
+ * the previous instance is still around. In such
+ * cases, wait till it's gone using offline_waitq.
+ */
+ cgroup_for_each_live_child(child, cgrp) {
+ DEFINE_WAIT(wait);
+
+ if (!cgroup_css(child, ss))
+ continue;
+
+ cgroup_get(child);
+ prepare_to_wait(&child->offline_waitq, &wait,
+ TASK_UNINTERRUPTIBLE);
+ cgroup_kn_unlock(of->kn);
+ schedule();
+ finish_wait(&child->offline_waitq, &wait);
+ cgroup_put(child);
+
+ return restart_syscall();
+ }
+
+ /* unavailable or not enabled on the parent? */
+ if (!(cgrp_dfl_root.subsys_mask & (1 << ssid)) ||
+ (cgroup_parent(cgrp) &&
+ !(cgroup_parent(cgrp)->child_subsys_mask & (1 << ssid)))) {
+ ret = -ENOENT;
+ goto out_unlock;
+ }
+ } else if (disable & (1 << ssid)) {
+ if (!(cgrp->child_subsys_mask & (1 << ssid))) {
+ disable &= ~(1 << ssid);
+ continue;
+ }
+
+ /* a child has it enabled? */
+ cgroup_for_each_live_child(child, cgrp) {
+ if (child->child_subsys_mask & (1 << ssid)) {
+ ret = -EBUSY;
+ goto out_unlock;
+ }
+ }
+ }
+ }
+
+ if (!enable && !disable) {
+ ret = 0;
+ goto out_unlock;
+ }
+
+ /*
+ * Except for the root, child_subsys_mask must be zero for a cgroup
+ * with tasks so that child cgroups don't compete against tasks.
+ */
+ if (enable && cgroup_parent(cgrp) && !list_empty(&cgrp->cset_links)) {
+ ret = -EBUSY;
+ goto out_unlock;
+ }
+
+ /*
+ * Create csses for enables and update child_subsys_mask. This
+ * changes cgroup_e_css() results which in turn makes the
+ * subsequent cgroup_update_dfl_csses() associate all tasks in the
+ * subtree to the updated csses.
+ */
+ for_each_subsys(ss, ssid) {
+ if (!(enable & (1 << ssid)))
+ continue;
+
+ cgroup_for_each_live_child(child, cgrp) {
+ ret = create_css(child, ss);
+ if (ret)
+ goto err_undo_css;
+ }
+ }
+
+ cgrp->child_subsys_mask |= enable;
+ cgrp->child_subsys_mask &= ~disable;
+
+ ret = cgroup_update_dfl_csses(cgrp);
+ if (ret)
+ goto err_undo_css;
+
+ /* all tasks are now migrated away from the old csses, kill them */
+ for_each_subsys(ss, ssid) {
+ if (!(disable & (1 << ssid)))
+ continue;
+
+ cgroup_for_each_live_child(child, cgrp)
+ kill_css(cgroup_css(child, ss));
+ }
+
+ kernfs_activate(cgrp->kn);
+ ret = 0;
+out_unlock:
+ cgroup_kn_unlock(of->kn);
+ return ret ?: nbytes;
+
+err_undo_css:
+ cgrp->child_subsys_mask &= ~enable;
+ cgrp->child_subsys_mask |= disable;
+
+ for_each_subsys(ss, ssid) {
+ if (!(enable & (1 << ssid)))
+ continue;
+
+ cgroup_for_each_live_child(child, cgrp) {
+ struct cgroup_subsys_state *css = cgroup_css(child, ss);
+ if (css)
+ kill_css(css);
+ }
+ }
+ goto out_unlock;
+}
+
+static int cgroup_populated_show(struct seq_file *seq, void *v)
+{
+ seq_printf(seq, "%d\n", (bool)seq_css(seq)->cgroup->populated_cnt);
+ return 0;
+}
+
static ssize_t cgroup_file_write(struct kernfs_open_file *of, char *buf,
size_t nbytes, loff_t off)
{
@@ -2228,6 +2743,9 @@
struct cgroup_subsys_state *css;
int ret;
+ if (cft->write)
+ return cft->write(of, buf, nbytes, off);
+
/*
* kernfs guarantees that a file isn't deleted with operations in
* flight, which means that the matching css is and stays alive and
@@ -2238,9 +2756,7 @@
css = cgroup_css(cgrp, cft->ss);
rcu_read_unlock();
- if (cft->write_string) {
- ret = cft->write_string(css, cft, strstrip(buf));
- } else if (cft->write_u64) {
+ if (cft->write_u64) {
unsigned long long v;
ret = kstrtoull(buf, 0, &v);
if (!ret)
@@ -2250,8 +2766,6 @@
ret = kstrtoll(buf, 0, &v);
if (!ret)
ret = cft->write_s64(css, cft, v);
- } else if (cft->trigger) {
- ret = cft->trigger(css, (unsigned int)cft->private);
} else {
ret = -EINVAL;
}
@@ -2328,20 +2842,18 @@
return -EPERM;
/*
- * We're gonna grab cgroup_tree_mutex which nests outside kernfs
+ * We're gonna grab cgroup_mutex which nests outside kernfs
* active_ref. kernfs_rename() doesn't require active_ref
- * protection. Break them before grabbing cgroup_tree_mutex.
+ * protection. Break them before grabbing cgroup_mutex.
*/
kernfs_break_active_protection(new_parent);
kernfs_break_active_protection(kn);
- mutex_lock(&cgroup_tree_mutex);
mutex_lock(&cgroup_mutex);
ret = kernfs_rename(kn, new_parent, new_name_str);
mutex_unlock(&cgroup_mutex);
- mutex_unlock(&cgroup_tree_mutex);
kernfs_unbreak_active_protection(kn);
kernfs_unbreak_active_protection(new_parent);
@@ -2379,9 +2891,14 @@
return PTR_ERR(kn);
ret = cgroup_kn_set_ugid(kn);
- if (ret)
+ if (ret) {
kernfs_remove(kn);
- return ret;
+ return ret;
+ }
+
+ if (cft->seq_show == cgroup_populated_show)
+ cgrp->populated_kn = kn;
+ return 0;
}
/**
@@ -2401,7 +2918,7 @@
struct cftype *cft;
int ret;
- lockdep_assert_held(&cgroup_tree_mutex);
+ lockdep_assert_held(&cgroup_mutex);
for (cft = cfts; cft->name[0] != '\0'; cft++) {
/* does cft->flags tell us to skip this file on @cgrp? */
@@ -2409,16 +2926,16 @@
continue;
if ((cft->flags & CFTYPE_INSANE) && cgroup_sane_behavior(cgrp))
continue;
- if ((cft->flags & CFTYPE_NOT_ON_ROOT) && !cgrp->parent)
+ if ((cft->flags & CFTYPE_NOT_ON_ROOT) && !cgroup_parent(cgrp))
continue;
- if ((cft->flags & CFTYPE_ONLY_ON_ROOT) && cgrp->parent)
+ if ((cft->flags & CFTYPE_ONLY_ON_ROOT) && cgroup_parent(cgrp))
continue;
if (is_add) {
ret = cgroup_add_file(cgrp, cft);
if (ret) {
- pr_warn("cgroup_addrm_files: failed to add %s, err=%d\n",
- cft->name, ret);
+ pr_warn("%s: failed to add %s, err=%d\n",
+ __func__, cft->name, ret);
return ret;
}
} else {
@@ -2436,11 +2953,7 @@
struct cgroup_subsys_state *css;
int ret = 0;
- lockdep_assert_held(&cgroup_tree_mutex);
-
- /* don't bother if @ss isn't attached */
- if (ss->root == &cgrp_dfl_root)
- return 0;
+ lockdep_assert_held(&cgroup_mutex);
/* add/rm files for all cgroups created before */
css_for_each_descendant_pre(css, cgroup_css(root, ss)) {
@@ -2508,7 +3021,7 @@
static int cgroup_rm_cftypes_locked(struct cftype *cfts)
{
- lockdep_assert_held(&cgroup_tree_mutex);
+ lockdep_assert_held(&cgroup_mutex);
if (!cfts || !cfts[0].ss)
return -ENOENT;
@@ -2534,9 +3047,9 @@
{
int ret;
- mutex_lock(&cgroup_tree_mutex);
+ mutex_lock(&cgroup_mutex);
ret = cgroup_rm_cftypes_locked(cfts);
- mutex_unlock(&cgroup_tree_mutex);
+ mutex_unlock(&cgroup_mutex);
return ret;
}
@@ -2558,6 +3071,9 @@
{
int ret;
+ if (ss->disabled)
+ return 0;
+
if (!cfts || cfts[0].name[0] == '\0')
return 0;
@@ -2565,14 +3081,14 @@
if (ret)
return ret;
- mutex_lock(&cgroup_tree_mutex);
+ mutex_lock(&cgroup_mutex);
list_add_tail(&cfts->node, &ss->cfts);
ret = cgroup_apply_cftypes(cfts, true);
if (ret)
cgroup_rm_cftypes_locked(cfts);
- mutex_unlock(&cgroup_tree_mutex);
+ mutex_unlock(&cgroup_mutex);
return ret;
}
@@ -2596,57 +3112,65 @@
/**
* css_next_child - find the next child of a given css
- * @pos_css: the current position (%NULL to initiate traversal)
- * @parent_css: css whose children to walk
+ * @pos: the current position (%NULL to initiate traversal)
+ * @parent: css whose children to walk
*
- * This function returns the next child of @parent_css and should be called
+ * This function returns the next child of @parent and should be called
* under either cgroup_mutex or RCU read lock. The only requirement is
- * that @parent_css and @pos_css are accessible. The next sibling is
- * guaranteed to be returned regardless of their states.
+ * that @parent and @pos are accessible. The next sibling is guaranteed to
+ * be returned regardless of their states.
+ *
+ * If a subsystem synchronizes ->css_online() and the start of iteration, a
+ * css which finished ->css_online() is guaranteed to be visible in the
+ * future iterations and will stay visible until the last reference is put.
+ * A css which hasn't finished ->css_online() or already finished
+ * ->css_offline() may show up during traversal. It's each subsystem's
+ * responsibility to synchronize against on/offlining.
*/
-struct cgroup_subsys_state *
-css_next_child(struct cgroup_subsys_state *pos_css,
- struct cgroup_subsys_state *parent_css)
+struct cgroup_subsys_state *css_next_child(struct cgroup_subsys_state *pos,
+ struct cgroup_subsys_state *parent)
{
- struct cgroup *pos = pos_css ? pos_css->cgroup : NULL;
- struct cgroup *cgrp = parent_css->cgroup;
- struct cgroup *next;
+ struct cgroup_subsys_state *next;
- cgroup_assert_mutexes_or_rcu_locked();
+ cgroup_assert_mutex_or_rcu_locked();
/*
- * @pos could already have been removed. Once a cgroup is removed,
- * its ->sibling.next is no longer updated when its next sibling
- * changes. As CGRP_DEAD assertion is serialized and happens
- * before the cgroup is taken off the ->sibling list, if we see it
- * unasserted, it's guaranteed that the next sibling hasn't
- * finished its grace period even if it's already removed, and thus
- * safe to dereference from this RCU critical section. If
- * ->sibling.next is inaccessible, cgroup_is_dead() is guaranteed
- * to be visible as %true here.
+ * @pos could already have been unlinked from the sibling list.
+ * Once a cgroup is removed, its ->sibling.next is no longer
+ * updated when its next sibling changes. CSS_RELEASED is set when
+ * @pos is taken off list, at which time its next pointer is valid,
+ * and, as releases are serialized, the one pointed to by the next
+ * pointer is guaranteed to not have started release yet. This
+ * implies that if we observe !CSS_RELEASED on @pos in this RCU
+ * critical section, the one pointed to by its next pointer is
+ * guaranteed to not have finished its RCU grace period even if we
+ * have dropped rcu_read_lock() inbetween iterations.
*
- * If @pos is dead, its next pointer can't be dereferenced;
- * however, as each cgroup is given a monotonically increasing
- * unique serial number and always appended to the sibling list,
- * the next one can be found by walking the parent's children until
- * we see a cgroup with higher serial number than @pos's. While
- * this path can be slower, it's taken only when either the current
- * cgroup is removed or iteration and removal race.
+ * If @pos has CSS_RELEASED set, its next pointer can't be
+ * dereferenced; however, as each css is given a monotonically
+ * increasing unique serial number and always appended to the
+ * sibling list, the next one can be found by walking the parent's
+ * children until the first css with higher serial number than
+ * @pos's. While this path can be slower, it happens iff iteration
+ * races against release and the race window is very small.
*/
if (!pos) {
- next = list_entry_rcu(cgrp->children.next, struct cgroup, sibling);
- } else if (likely(!cgroup_is_dead(pos))) {
- next = list_entry_rcu(pos->sibling.next, struct cgroup, sibling);
+ next = list_entry_rcu(parent->children.next, struct cgroup_subsys_state, sibling);
+ } else if (likely(!(pos->flags & CSS_RELEASED))) {
+ next = list_entry_rcu(pos->sibling.next, struct cgroup_subsys_state, sibling);
} else {
- list_for_each_entry_rcu(next, &cgrp->children, sibling)
+ list_for_each_entry_rcu(next, &parent->children, sibling)
if (next->serial_nr > pos->serial_nr)
break;
}
- if (&next->sibling == &cgrp->children)
- return NULL;
-
- return cgroup_css(next, parent_css->ss);
+ /*
+ * @next, if not pointing to the head, can be dereferenced and is
+ * the next sibling.
+ */
+ if (&next->sibling != &parent->children)
+ return next;
+ return NULL;
}
/**
@@ -2662,6 +3186,13 @@
* doesn't require the whole traversal to be contained in a single critical
* section. This function will return the correct next descendant as long
* as both @pos and @root are accessible and @pos is a descendant of @root.
+ *
+ * If a subsystem synchronizes ->css_online() and the start of iteration, a
+ * css which finished ->css_online() is guaranteed to be visible in the
+ * future iterations and will stay visible until the last reference is put.
+ * A css which hasn't finished ->css_online() or already finished
+ * ->css_offline() may show up during traversal. It's each subsystem's
+ * responsibility to synchronize against on/offlining.
*/
struct cgroup_subsys_state *
css_next_descendant_pre(struct cgroup_subsys_state *pos,
@@ -2669,7 +3200,7 @@
{
struct cgroup_subsys_state *next;
- cgroup_assert_mutexes_or_rcu_locked();
+ cgroup_assert_mutex_or_rcu_locked();
/* if first iteration, visit @root */
if (!pos)
@@ -2682,10 +3213,10 @@
/* no child, visit my or the closest ancestor's next sibling */
while (pos != root) {
- next = css_next_child(pos, css_parent(pos));
+ next = css_next_child(pos, pos->parent);
if (next)
return next;
- pos = css_parent(pos);
+ pos = pos->parent;
}
return NULL;
@@ -2709,7 +3240,7 @@
{
struct cgroup_subsys_state *last, *tmp;
- cgroup_assert_mutexes_or_rcu_locked();
+ cgroup_assert_mutex_or_rcu_locked();
do {
last = pos;
@@ -2749,6 +3280,13 @@
* section. This function will return the correct next descendant as long
* as both @pos and @cgroup are accessible and @pos is a descendant of
* @cgroup.
+ *
+ * If a subsystem synchronizes ->css_online() and the start of iteration, a
+ * css which finished ->css_online() is guaranteed to be visible in the
+ * future iterations and will stay visible until the last reference is put.
+ * A css which hasn't finished ->css_online() or already finished
+ * ->css_offline() may show up during traversal. It's each subsystem's
+ * responsibility to synchronize against on/offlining.
*/
struct cgroup_subsys_state *
css_next_descendant_post(struct cgroup_subsys_state *pos,
@@ -2756,7 +3294,7 @@
{
struct cgroup_subsys_state *next;
- cgroup_assert_mutexes_or_rcu_locked();
+ cgroup_assert_mutex_or_rcu_locked();
/* if first iteration, visit leftmost descendant which may be @root */
if (!pos)
@@ -2767,12 +3305,36 @@
return NULL;
/* if there's an unvisited sibling, visit its leftmost descendant */
- next = css_next_child(pos, css_parent(pos));
+ next = css_next_child(pos, pos->parent);
if (next)
return css_leftmost_descendant(next);
/* no sibling left, visit parent */
- return css_parent(pos);
+ return pos->parent;
+}
+
+/**
+ * css_has_online_children - does a css have online children
+ * @css: the target css
+ *
+ * Returns %true if @css has any online children; otherwise, %false. This
+ * function can be called from any context but the caller is responsible
+ * for synchronizing against on/offlining as necessary.
+ */
+bool css_has_online_children(struct cgroup_subsys_state *css)
+{
+ struct cgroup_subsys_state *child;
+ bool ret = false;
+
+ rcu_read_lock();
+ css_for_each_child(child, css) {
+ if (css->flags & CSS_ONLINE) {
+ ret = true;
+ break;
+ }
+ }
+ rcu_read_unlock();
+ return ret;
}
/**
@@ -2783,27 +3345,36 @@
*/
static void css_advance_task_iter(struct css_task_iter *it)
{
- struct list_head *l = it->cset_link;
+ struct list_head *l = it->cset_pos;
struct cgrp_cset_link *link;
struct css_set *cset;
/* Advance to the next non-empty css_set */
do {
l = l->next;
- if (l == &it->origin_css->cgroup->cset_links) {
- it->cset_link = NULL;
+ if (l == it->cset_head) {
+ it->cset_pos = NULL;
return;
}
- link = list_entry(l, struct cgrp_cset_link, cset_link);
- cset = link->cset;
+
+ if (it->ss) {
+ cset = container_of(l, struct css_set,
+ e_cset_node[it->ss->id]);
+ } else {
+ link = list_entry(l, struct cgrp_cset_link, cset_link);
+ cset = link->cset;
+ }
} while (list_empty(&cset->tasks) && list_empty(&cset->mg_tasks));
- it->cset_link = l;
+ it->cset_pos = l;
if (!list_empty(&cset->tasks))
- it->task = cset->tasks.next;
+ it->task_pos = cset->tasks.next;
else
- it->task = cset->mg_tasks.next;
+ it->task_pos = cset->mg_tasks.next;
+
+ it->tasks_head = &cset->tasks;
+ it->mg_tasks_head = &cset->mg_tasks;
}
/**
@@ -2829,8 +3400,14 @@
down_read(&css_set_rwsem);
- it->origin_css = css;
- it->cset_link = &css->cgroup->cset_links;
+ it->ss = css->ss;
+
+ if (it->ss)
+ it->cset_pos = &css->cgroup->e_csets[css->ss->id];
+ else
+ it->cset_pos = &css->cgroup->cset_links;
+
+ it->cset_head = it->cset_pos;
css_advance_task_iter(it);
}
@@ -2846,12 +3423,10 @@
struct task_struct *css_task_iter_next(struct css_task_iter *it)
{
struct task_struct *res;
- struct list_head *l = it->task;
- struct cgrp_cset_link *link = list_entry(it->cset_link,
- struct cgrp_cset_link, cset_link);
+ struct list_head *l = it->task_pos;
/* If the iterator cg is NULL, we have no tasks */
- if (!it->cset_link)
+ if (!it->cset_pos)
return NULL;
res = list_entry(l, struct task_struct, cg_list);
@@ -2862,13 +3437,13 @@
*/
l = l->next;
- if (l == &link->cset->tasks)
- l = link->cset->mg_tasks.next;
+ if (l == it->tasks_head)
+ l = it->mg_tasks_head->next;
- if (l == &link->cset->mg_tasks)
+ if (l == it->mg_tasks_head)
css_advance_task_iter(it);
else
- it->task = l;
+ it->task_pos = l;
return res;
}
@@ -2921,7 +3496,7 @@
* ->can_attach() fails.
*/
do {
- css_task_iter_start(&from->dummy_css, &it);
+ css_task_iter_start(&from->self, &it);
task = css_task_iter_next(&it);
if (task)
get_task_struct(task);
@@ -3186,7 +3761,7 @@
if (!array)
return -ENOMEM;
/* now, populate the array */
- css_task_iter_start(&cgrp->dummy_css, &it);
+ css_task_iter_start(&cgrp->self, &it);
while ((tsk = css_task_iter_next(&it))) {
if (unlikely(n == length))
break;
@@ -3248,7 +3823,7 @@
/*
* We aren't being called from kernfs and there's no guarantee on
- * @kn->priv's validity. For this and css_tryget_from_dir(),
+ * @kn->priv's validity. For this and css_tryget_online_from_dir(),
* @kn->priv is RCU safe. Let's do the RCU dancing.
*/
rcu_read_lock();
@@ -3260,7 +3835,7 @@
}
rcu_read_unlock();
- css_task_iter_start(&cgrp->dummy_css, &it);
+ css_task_iter_start(&cgrp->self, &it);
while ((tsk = css_task_iter_next(&it))) {
switch (tsk->state) {
case TASK_RUNNING:
@@ -3390,17 +3965,6 @@
return seq_printf(s, "%d\n", *(int *)v);
}
-/*
- * seq_operations functions for iterating on pidlists through seq_file -
- * independent of whether it's tasks or procs
- */
-static const struct seq_operations cgroup_pidlist_seq_operations = {
- .start = cgroup_pidlist_start,
- .stop = cgroup_pidlist_stop,
- .next = cgroup_pidlist_next,
- .show = cgroup_pidlist_show,
-};
-
static u64 cgroup_read_notify_on_release(struct cgroup_subsys_state *css,
struct cftype *cft)
{
@@ -3442,7 +4006,7 @@
.seq_stop = cgroup_pidlist_stop,
.seq_show = cgroup_pidlist_show,
.private = CGROUP_FILE_PROCS,
- .write_u64 = cgroup_procs_write,
+ .write = cgroup_procs_write,
.mode = S_IRUGO | S_IWUSR,
},
{
@@ -3456,6 +4020,27 @@
.flags = CFTYPE_ONLY_ON_ROOT,
.seq_show = cgroup_sane_behavior_show,
},
+ {
+ .name = "cgroup.controllers",
+ .flags = CFTYPE_ONLY_ON_DFL | CFTYPE_ONLY_ON_ROOT,
+ .seq_show = cgroup_root_controllers_show,
+ },
+ {
+ .name = "cgroup.controllers",
+ .flags = CFTYPE_ONLY_ON_DFL | CFTYPE_NOT_ON_ROOT,
+ .seq_show = cgroup_controllers_show,
+ },
+ {
+ .name = "cgroup.subtree_control",
+ .flags = CFTYPE_ONLY_ON_DFL,
+ .seq_show = cgroup_subtree_control_show,
+ .write = cgroup_subtree_control_write,
+ },
+ {
+ .name = "cgroup.populated",
+ .flags = CFTYPE_ONLY_ON_DFL | CFTYPE_NOT_ON_ROOT,
+ .seq_show = cgroup_populated_show,
+ },
/*
* Historical crazy stuff. These don't have "cgroup." prefix and
@@ -3470,7 +4055,7 @@
.seq_stop = cgroup_pidlist_stop,
.seq_show = cgroup_pidlist_show,
.private = CGROUP_FILE_TASKS,
- .write_u64 = cgroup_tasks_write,
+ .write = cgroup_tasks_write,
.mode = S_IRUGO | S_IWUSR,
},
{
@@ -3483,7 +4068,7 @@
.name = "release_agent",
.flags = CFTYPE_INSANE | CFTYPE_ONLY_ON_ROOT,
.seq_show = cgroup_release_agent_show,
- .write_string = cgroup_release_agent_write,
+ .write = cgroup_release_agent_write,
.max_write_len = PATH_MAX - 1,
},
{ } /* terminate */
@@ -3496,7 +4081,7 @@
*
* On failure, no file is added.
*/
-static int cgroup_populate_dir(struct cgroup *cgrp, unsigned long subsys_mask)
+static int cgroup_populate_dir(struct cgroup *cgrp, unsigned int subsys_mask)
{
struct cgroup_subsys *ss;
int i, ret = 0;
@@ -3505,7 +4090,7 @@
for_each_subsys(ss, i) {
struct cftype *cfts;
- if (!test_bit(i, &subsys_mask))
+ if (!(subsys_mask & (1 << i)))
continue;
list_for_each_entry(cfts, &ss->cfts, node) {
@@ -3527,9 +4112,9 @@
* Implemented in kill_css().
*
* 2. When the percpu_ref is confirmed to be visible as killed on all CPUs
- * and thus css_tryget() is guaranteed to fail, the css can be offlined
- * by invoking offline_css(). After offlining, the base ref is put.
- * Implemented in css_killed_work_fn().
+ * and thus css_tryget_online() is guaranteed to fail, the css can be
+ * offlined by invoking offline_css(). After offlining, the base ref is
+ * put. Implemented in css_killed_work_fn().
*
* 3. When the percpu_ref reaches zero, the only possible remaining
* accessors are inside RCU read sections. css_release() schedules the
@@ -3548,11 +4133,37 @@
container_of(work, struct cgroup_subsys_state, destroy_work);
struct cgroup *cgrp = css->cgroup;
- if (css->parent)
- css_put(css->parent);
+ if (css->ss) {
+ /* css free path */
+ if (css->parent)
+ css_put(css->parent);
- css->ss->css_free(css);
- cgroup_put(cgrp);
+ css->ss->css_free(css);
+ cgroup_put(cgrp);
+ } else {
+ /* cgroup free path */
+ atomic_dec(&cgrp->root->nr_cgrps);
+ cgroup_pidlist_destroy_all(cgrp);
+
+ if (cgroup_parent(cgrp)) {
+ /*
+ * We get a ref to the parent, and put the ref when
+ * this cgroup is being freed, so it's guaranteed
+ * that the parent won't be destroyed before its
+ * children.
+ */
+ cgroup_put(cgroup_parent(cgrp));
+ kernfs_put(cgrp->kn);
+ kfree(cgrp);
+ } else {
+ /*
+ * This is root cgroup's refcnt reaching zero,
+ * which indicates that the root should be
+ * released.
+ */
+ cgroup_destroy_root(cgrp->root);
+ }
+ }
}
static void css_free_rcu_fn(struct rcu_head *rcu_head)
@@ -3564,26 +4175,59 @@
queue_work(cgroup_destroy_wq, &css->destroy_work);
}
+static void css_release_work_fn(struct work_struct *work)
+{
+ struct cgroup_subsys_state *css =
+ container_of(work, struct cgroup_subsys_state, destroy_work);
+ struct cgroup_subsys *ss = css->ss;
+ struct cgroup *cgrp = css->cgroup;
+
+ mutex_lock(&cgroup_mutex);
+
+ css->flags |= CSS_RELEASED;
+ list_del_rcu(&css->sibling);
+
+ if (ss) {
+ /* css release path */
+ cgroup_idr_remove(&ss->css_idr, css->id);
+ } else {
+ /* cgroup release path */
+ cgroup_idr_remove(&cgrp->root->cgroup_idr, cgrp->id);
+ cgrp->id = -1;
+ }
+
+ mutex_unlock(&cgroup_mutex);
+
+ call_rcu(&css->rcu_head, css_free_rcu_fn);
+}
+
static void css_release(struct percpu_ref *ref)
{
struct cgroup_subsys_state *css =
container_of(ref, struct cgroup_subsys_state, refcnt);
- RCU_INIT_POINTER(css->cgroup->subsys[css->ss->id], NULL);
- call_rcu(&css->rcu_head, css_free_rcu_fn);
+ INIT_WORK(&css->destroy_work, css_release_work_fn);
+ queue_work(cgroup_destroy_wq, &css->destroy_work);
}
-static void init_css(struct cgroup_subsys_state *css, struct cgroup_subsys *ss,
- struct cgroup *cgrp)
+static void init_and_link_css(struct cgroup_subsys_state *css,
+ struct cgroup_subsys *ss, struct cgroup *cgrp)
{
+ lockdep_assert_held(&cgroup_mutex);
+
+ cgroup_get(cgrp);
+
+ memset(css, 0, sizeof(*css));
css->cgroup = cgrp;
css->ss = ss;
- css->flags = 0;
+ INIT_LIST_HEAD(&css->sibling);
+ INIT_LIST_HEAD(&css->children);
+ css->serial_nr = css_serial_nr_next++;
- if (cgrp->parent)
- css->parent = cgroup_css(cgrp->parent, ss);
- else
- css->flags |= CSS_ROOT;
+ if (cgroup_parent(cgrp)) {
+ css->parent = cgroup_css(cgroup_parent(cgrp), ss);
+ css_get(css->parent);
+ }
BUG_ON(cgroup_css(cgrp, ss));
}
@@ -3594,14 +4238,12 @@
struct cgroup_subsys *ss = css->ss;
int ret = 0;
- lockdep_assert_held(&cgroup_tree_mutex);
lockdep_assert_held(&cgroup_mutex);
if (ss->css_online)
ret = ss->css_online(css);
if (!ret) {
css->flags |= CSS_ONLINE;
- css->cgroup->nr_css++;
rcu_assign_pointer(css->cgroup->subsys[ss->id], css);
}
return ret;
@@ -3612,7 +4254,6 @@
{
struct cgroup_subsys *ss = css->ss;
- lockdep_assert_held(&cgroup_tree_mutex);
lockdep_assert_held(&cgroup_mutex);
if (!(css->flags & CSS_ONLINE))
@@ -3622,8 +4263,9 @@
ss->css_offline(css);
css->flags &= ~CSS_ONLINE;
- css->cgroup->nr_css--;
- RCU_INIT_POINTER(css->cgroup->subsys[ss->id], css);
+ RCU_INIT_POINTER(css->cgroup->subsys[ss->id], NULL);
+
+ wake_up_all(&css->cgroup->offline_waitq);
}
/**
@@ -3637,111 +4279,102 @@
*/
static int create_css(struct cgroup *cgrp, struct cgroup_subsys *ss)
{
- struct cgroup *parent = cgrp->parent;
+ struct cgroup *parent = cgroup_parent(cgrp);
+ struct cgroup_subsys_state *parent_css = cgroup_css(parent, ss);
struct cgroup_subsys_state *css;
int err;
lockdep_assert_held(&cgroup_mutex);
- css = ss->css_alloc(cgroup_css(parent, ss));
+ css = ss->css_alloc(parent_css);
if (IS_ERR(css))
return PTR_ERR(css);
+ init_and_link_css(css, ss, cgrp);
+
err = percpu_ref_init(&css->refcnt, css_release);
if (err)
goto err_free_css;
- init_css(css, ss, cgrp);
+ err = cgroup_idr_alloc(&ss->css_idr, NULL, 2, 0, GFP_NOWAIT);
+ if (err < 0)
+ goto err_free_percpu_ref;
+ css->id = err;
err = cgroup_populate_dir(cgrp, 1 << ss->id);
if (err)
- goto err_free_percpu_ref;
+ goto err_free_id;
+
+ /* @css is ready to be brought online now, make it visible */
+ list_add_tail_rcu(&css->sibling, &parent_css->children);
+ cgroup_idr_replace(&ss->css_idr, css, css->id);
err = online_css(css);
if (err)
- goto err_clear_dir;
-
- cgroup_get(cgrp);
- css_get(css->parent);
-
- cgrp->subsys_mask |= 1 << ss->id;
+ goto err_list_del;
if (ss->broken_hierarchy && !ss->warned_broken_hierarchy &&
- parent->parent) {
- pr_warning("cgroup: %s (%d) created nested cgroup for controller \"%s\" which has incomplete hierarchy support. Nested cgroups may change behavior in the future.\n",
- current->comm, current->pid, ss->name);
+ cgroup_parent(parent)) {
+ pr_warn("%s (%d) created nested cgroup for controller \"%s\" which has incomplete hierarchy support. Nested cgroups may change behavior in the future.\n",
+ current->comm, current->pid, ss->name);
if (!strcmp(ss->name, "memory"))
- pr_warning("cgroup: \"memory\" requires setting use_hierarchy to 1 on the root.\n");
+ pr_warn("\"memory\" requires setting use_hierarchy to 1 on the root\n");
ss->warned_broken_hierarchy = true;
}
return 0;
-err_clear_dir:
+err_list_del:
+ list_del_rcu(&css->sibling);
cgroup_clear_dir(css->cgroup, 1 << css->ss->id);
+err_free_id:
+ cgroup_idr_remove(&ss->css_idr, css->id);
err_free_percpu_ref:
percpu_ref_cancel_init(&css->refcnt);
err_free_css:
- ss->css_free(css);
+ call_rcu(&css->rcu_head, css_free_rcu_fn);
return err;
}
-/**
- * cgroup_create - create a cgroup
- * @parent: cgroup that will be parent of the new cgroup
- * @name: name of the new cgroup
- * @mode: mode to set on new cgroup
- */
-static long cgroup_create(struct cgroup *parent, const char *name,
- umode_t mode)
+static int cgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
+ umode_t mode)
{
- struct cgroup *cgrp;
- struct cgroup_root *root = parent->root;
- int ssid, err;
+ struct cgroup *parent, *cgrp;
+ struct cgroup_root *root;
struct cgroup_subsys *ss;
struct kernfs_node *kn;
+ int ssid, ret;
- /*
- * XXX: The default hierarchy isn't fully implemented yet. Block
- * !root cgroup creation on it for now.
- */
- if (root == &cgrp_dfl_root)
- return -EINVAL;
+ parent = cgroup_kn_lock_live(parent_kn);
+ if (!parent)
+ return -ENODEV;
+ root = parent->root;
/* allocate the cgroup and its ID, 0 is reserved for the root */
cgrp = kzalloc(sizeof(*cgrp), GFP_KERNEL);
- if (!cgrp)
- return -ENOMEM;
-
- mutex_lock(&cgroup_tree_mutex);
-
- /*
- * Only live parents can have children. Note that the liveliness
- * check isn't strictly necessary because cgroup_mkdir() and
- * cgroup_rmdir() are fully synchronized by i_mutex; however, do it
- * anyway so that locking is contained inside cgroup proper and we
- * don't get nasty surprises if we ever grow another caller.
- */
- if (!cgroup_lock_live_group(parent)) {
- err = -ENODEV;
- goto err_unlock_tree;
+ if (!cgrp) {
+ ret = -ENOMEM;
+ goto out_unlock;
}
+ ret = percpu_ref_init(&cgrp->self.refcnt, css_release);
+ if (ret)
+ goto out_free_cgrp;
+
/*
* Temporarily set the pointer to NULL, so idr_find() won't return
* a half-baked cgroup.
*/
- cgrp->id = idr_alloc(&root->cgroup_idr, NULL, 1, 0, GFP_KERNEL);
+ cgrp->id = cgroup_idr_alloc(&root->cgroup_idr, NULL, 2, 0, GFP_NOWAIT);
if (cgrp->id < 0) {
- err = -ENOMEM;
- goto err_unlock;
+ ret = -ENOMEM;
+ goto out_cancel_ref;
}
init_cgroup_housekeeping(cgrp);
- cgrp->parent = parent;
- cgrp->dummy_css.parent = &parent->dummy_css;
- cgrp->root = parent->root;
+ cgrp->self.parent = &parent->self;
+ cgrp->root = root;
if (notify_on_release(parent))
set_bit(CGRP_NOTIFY_ON_RELEASE, &cgrp->flags);
@@ -3752,8 +4385,8 @@
/* create the directory */
kn = kernfs_create_dir(parent->kn, name, mode, cgrp);
if (IS_ERR(kn)) {
- err = PTR_ERR(kn);
- goto err_free_id;
+ ret = PTR_ERR(kn);
+ goto out_free_id;
}
cgrp->kn = kn;
@@ -3763,10 +4396,10 @@
*/
kernfs_get(kn);
- cgrp->serial_nr = cgroup_serial_nr_next++;
+ cgrp->self.serial_nr = css_serial_nr_next++;
/* allocation complete, commit to creation */
- list_add_tail_rcu(&cgrp->sibling, &cgrp->parent->children);
+ list_add_tail_rcu(&cgrp->self.sibling, &cgroup_parent(cgrp)->self.children);
atomic_inc(&root->nr_cgrps);
cgroup_get(parent);
@@ -3774,107 +4407,66 @@
* @cgrp is now fully operational. If something fails after this
* point, it'll be released via the normal destruction path.
*/
- idr_replace(&root->cgroup_idr, cgrp, cgrp->id);
+ cgroup_idr_replace(&root->cgroup_idr, cgrp, cgrp->id);
- err = cgroup_kn_set_ugid(kn);
- if (err)
- goto err_destroy;
+ ret = cgroup_kn_set_ugid(kn);
+ if (ret)
+ goto out_destroy;
- err = cgroup_addrm_files(cgrp, cgroup_base_files, true);
- if (err)
- goto err_destroy;
+ ret = cgroup_addrm_files(cgrp, cgroup_base_files, true);
+ if (ret)
+ goto out_destroy;
/* let's create and online css's */
for_each_subsys(ss, ssid) {
- if (root->cgrp.subsys_mask & (1 << ssid)) {
- err = create_css(cgrp, ss);
- if (err)
- goto err_destroy;
+ if (parent->child_subsys_mask & (1 << ssid)) {
+ ret = create_css(cgrp, ss);
+ if (ret)
+ goto out_destroy;
}
}
+ /*
+ * On the default hierarchy, a child doesn't automatically inherit
+ * child_subsys_mask from the parent. Each is configured manually.
+ */
+ if (!cgroup_on_dfl(cgrp))
+ cgrp->child_subsys_mask = parent->child_subsys_mask;
+
kernfs_activate(kn);
- mutex_unlock(&cgroup_mutex);
- mutex_unlock(&cgroup_tree_mutex);
+ ret = 0;
+ goto out_unlock;
- return 0;
-
-err_free_id:
- idr_remove(&root->cgroup_idr, cgrp->id);
-err_unlock:
- mutex_unlock(&cgroup_mutex);
-err_unlock_tree:
- mutex_unlock(&cgroup_tree_mutex);
+out_free_id:
+ cgroup_idr_remove(&root->cgroup_idr, cgrp->id);
+out_cancel_ref:
+ percpu_ref_cancel_init(&cgrp->self.refcnt);
+out_free_cgrp:
kfree(cgrp);
- return err;
-
-err_destroy:
- cgroup_destroy_locked(cgrp);
- mutex_unlock(&cgroup_mutex);
- mutex_unlock(&cgroup_tree_mutex);
- return err;
-}
-
-static int cgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
- umode_t mode)
-{
- struct cgroup *parent = parent_kn->priv;
- int ret;
-
- /*
- * cgroup_create() grabs cgroup_tree_mutex which nests outside
- * kernfs active_ref and cgroup_create() already synchronizes
- * properly against removal through cgroup_lock_live_group().
- * Break it before calling cgroup_create().
- */
- cgroup_get(parent);
- kernfs_break_active_protection(parent_kn);
-
- ret = cgroup_create(parent, name, mode);
-
- kernfs_unbreak_active_protection(parent_kn);
- cgroup_put(parent);
+out_unlock:
+ cgroup_kn_unlock(parent_kn);
return ret;
+
+out_destroy:
+ cgroup_destroy_locked(cgrp);
+ goto out_unlock;
}
/*
* This is called when the refcnt of a css is confirmed to be killed.
- * css_tryget() is now guaranteed to fail.
+ * css_tryget_online() is now guaranteed to fail. Tell the subsystem to
+ * initate destruction and put the css ref from kill_css().
*/
static void css_killed_work_fn(struct work_struct *work)
{
struct cgroup_subsys_state *css =
container_of(work, struct cgroup_subsys_state, destroy_work);
- struct cgroup *cgrp = css->cgroup;
- mutex_lock(&cgroup_tree_mutex);
mutex_lock(&cgroup_mutex);
-
- /*
- * css_tryget() is guaranteed to fail now. Tell subsystems to
- * initate destruction.
- */
offline_css(css);
-
- /*
- * If @cgrp is marked dead, it's waiting for refs of all css's to
- * be disabled before proceeding to the second phase of cgroup
- * destruction. If we are the last one, kick it off.
- */
- if (!cgrp->nr_css && cgroup_is_dead(cgrp))
- cgroup_destroy_css_killed(cgrp);
-
mutex_unlock(&cgroup_mutex);
- mutex_unlock(&cgroup_tree_mutex);
- /*
- * Put the css refs from kill_css(). Each css holds an extra
- * reference to the cgroup's dentry and cgroup removal proceeds
- * regardless of css refs. On the last put of each css, whenever
- * that may be, the extra dentry ref is put so that dentry
- * destruction happens only after all css's are released.
- */
css_put(css);
}
@@ -3888,9 +4480,18 @@
queue_work(cgroup_destroy_wq, &css->destroy_work);
}
-static void __kill_css(struct cgroup_subsys_state *css)
+/**
+ * kill_css - destroy a css
+ * @css: css to destroy
+ *
+ * This function initiates destruction of @css by removing cgroup interface
+ * files and putting its base reference. ->css_offline() will be invoked
+ * asynchronously once css_tryget_online() is guaranteed to fail and when
+ * the reference count reaches zero, @css will be released.
+ */
+static void kill_css(struct cgroup_subsys_state *css)
{
- lockdep_assert_held(&cgroup_tree_mutex);
+ lockdep_assert_held(&cgroup_mutex);
/*
* This must happen before css is disassociated with its cgroup.
@@ -3907,7 +4508,7 @@
/*
* cgroup core guarantees that, by the time ->css_offline() is
* invoked, no new css reference will be given out via
- * css_tryget(). We can't simply call percpu_ref_kill() and
+ * css_tryget_online(). We can't simply call percpu_ref_kill() and
* proceed to offlining css's because percpu_ref_kill() doesn't
* guarantee that the ref is seen as killed on all CPUs on return.
*
@@ -3918,36 +4519,14 @@
}
/**
- * kill_css - destroy a css
- * @css: css to destroy
- *
- * This function initiates destruction of @css by removing cgroup interface
- * files and putting its base reference. ->css_offline() will be invoked
- * asynchronously once css_tryget() is guaranteed to fail and when the
- * reference count reaches zero, @css will be released.
- */
-static void kill_css(struct cgroup_subsys_state *css)
-{
- struct cgroup *cgrp = css->cgroup;
-
- lockdep_assert_held(&cgroup_tree_mutex);
-
- /* if already killed, noop */
- if (cgrp->subsys_mask & (1 << css->ss->id)) {
- cgrp->subsys_mask &= ~(1 << css->ss->id);
- __kill_css(css);
- }
-}
-
-/**
* cgroup_destroy_locked - the first stage of cgroup destruction
* @cgrp: cgroup to be destroyed
*
* css's make use of percpu refcnts whose killing latency shouldn't be
* exposed to userland and are RCU protected. Also, cgroup core needs to
- * guarantee that css_tryget() won't succeed by the time ->css_offline() is
- * invoked. To satisfy all the requirements, destruction is implemented in
- * the following two steps.
+ * guarantee that css_tryget_online() won't succeed by the time
+ * ->css_offline() is invoked. To satisfy all the requirements,
+ * destruction is implemented in the following two steps.
*
* s1. Verify @cgrp can be destroyed and mark it dying. Remove all
* userland visible parts and start killing the percpu refcnts of
@@ -3966,12 +4545,10 @@
static int cgroup_destroy_locked(struct cgroup *cgrp)
__releases(&cgroup_mutex) __acquires(&cgroup_mutex)
{
- struct cgroup *child;
struct cgroup_subsys_state *css;
bool empty;
int ssid;
- lockdep_assert_held(&cgroup_tree_mutex);
lockdep_assert_held(&cgroup_mutex);
/*
@@ -3985,127 +4562,68 @@
return -EBUSY;
/*
- * Make sure there's no live children. We can't test ->children
- * emptiness as dead children linger on it while being destroyed;
- * otherwise, "rmdir parent/child parent" may fail with -EBUSY.
+ * Make sure there's no live children. We can't test emptiness of
+ * ->self.children as dead children linger on it while being
+ * drained; otherwise, "rmdir parent/child parent" may fail.
*/
- empty = true;
- rcu_read_lock();
- list_for_each_entry_rcu(child, &cgrp->children, sibling) {
- empty = cgroup_is_dead(child);
- if (!empty)
- break;
- }
- rcu_read_unlock();
- if (!empty)
+ if (css_has_online_children(&cgrp->self))
return -EBUSY;
/*
* Mark @cgrp dead. This prevents further task migration and child
- * creation by disabling cgroup_lock_live_group(). Note that
- * CGRP_DEAD assertion is depended upon by css_next_child() to
- * resume iteration after dropping RCU read lock. See
- * css_next_child() for details.
+ * creation by disabling cgroup_lock_live_group().
*/
- set_bit(CGRP_DEAD, &cgrp->flags);
+ cgrp->self.flags &= ~CSS_ONLINE;
- /*
- * Initiate massacre of all css's. cgroup_destroy_css_killed()
- * will be invoked to perform the rest of destruction once the
- * percpu refs of all css's are confirmed to be killed. This
- * involves removing the subsystem's files, drop cgroup_mutex.
- */
- mutex_unlock(&cgroup_mutex);
+ /* initiate massacre of all css's */
for_each_css(css, ssid, cgrp)
kill_css(css);
- mutex_lock(&cgroup_mutex);
- /* CGRP_DEAD is set, remove from ->release_list for the last time */
+ /* CSS_ONLINE is clear, remove from ->release_list for the last time */
raw_spin_lock(&release_list_lock);
if (!list_empty(&cgrp->release_list))
list_del_init(&cgrp->release_list);
raw_spin_unlock(&release_list_lock);
/*
- * If @cgrp has css's attached, the second stage of cgroup
- * destruction is kicked off from css_killed_work_fn() after the
- * refs of all attached css's are killed. If @cgrp doesn't have
- * any css, we kick it off here.
+ * Remove @cgrp directory along with the base files. @cgrp has an
+ * extra ref on its kn.
*/
- if (!cgrp->nr_css)
- cgroup_destroy_css_killed(cgrp);
+ kernfs_remove(cgrp->kn);
- /* remove @cgrp directory along with the base files */
- mutex_unlock(&cgroup_mutex);
+ set_bit(CGRP_RELEASABLE, &cgroup_parent(cgrp)->flags);
+ check_for_release(cgroup_parent(cgrp));
- /*
- * There are two control paths which try to determine cgroup from
- * dentry without going through kernfs - cgroupstats_build() and
- * css_tryget_from_dir(). Those are supported by RCU protecting
- * clearing of cgrp->kn->priv backpointer, which should happen
- * after all files under it have been removed.
- */
- kernfs_remove(cgrp->kn); /* @cgrp has an extra ref on its kn */
- RCU_INIT_POINTER(*(void __rcu __force **)&cgrp->kn->priv, NULL);
-
- mutex_lock(&cgroup_mutex);
+ /* put the base reference */
+ percpu_ref_kill(&cgrp->self.refcnt);
return 0;
};
-/**
- * cgroup_destroy_css_killed - the second step of cgroup destruction
- * @work: cgroup->destroy_free_work
- *
- * This function is invoked from a work item for a cgroup which is being
- * destroyed after all css's are offlined and performs the rest of
- * destruction. This is the second step of destruction described in the
- * comment above cgroup_destroy_locked().
- */
-static void cgroup_destroy_css_killed(struct cgroup *cgrp)
-{
- struct cgroup *parent = cgrp->parent;
-
- lockdep_assert_held(&cgroup_tree_mutex);
- lockdep_assert_held(&cgroup_mutex);
-
- /* delete this cgroup from parent->children */
- list_del_rcu(&cgrp->sibling);
-
- cgroup_put(cgrp);
-
- set_bit(CGRP_RELEASABLE, &parent->flags);
- check_for_release(parent);
-}
-
static int cgroup_rmdir(struct kernfs_node *kn)
{
- struct cgroup *cgrp = kn->priv;
+ struct cgroup *cgrp;
int ret = 0;
- /*
- * This is self-destruction but @kn can't be removed while this
- * callback is in progress. Let's break active protection. Once
- * the protection is broken, @cgrp can be destroyed at any point.
- * Pin it so that it stays accessible.
- */
- cgroup_get(cgrp);
- kernfs_break_active_protection(kn);
+ cgrp = cgroup_kn_lock_live(kn);
+ if (!cgrp)
+ return 0;
+ cgroup_get(cgrp); /* for @kn->priv clearing */
- mutex_lock(&cgroup_tree_mutex);
- mutex_lock(&cgroup_mutex);
+ ret = cgroup_destroy_locked(cgrp);
+
+ cgroup_kn_unlock(kn);
/*
- * @cgrp might already have been destroyed while we're trying to
- * grab the mutexes.
+ * There are two control paths which try to determine cgroup from
+ * dentry without going through kernfs - cgroupstats_build() and
+ * css_tryget_online_from_dir(). Those are supported by RCU
+ * protecting clearing of cgrp->kn->priv backpointer, which should
+ * happen after all files under it have been removed.
*/
- if (!cgroup_is_dead(cgrp))
- ret = cgroup_destroy_locked(cgrp);
+ if (!ret)
+ RCU_INIT_POINTER(*(void __rcu __force **)&kn->priv, NULL);
- mutex_unlock(&cgroup_mutex);
- mutex_unlock(&cgroup_tree_mutex);
-
- kernfs_unbreak_active_protection(kn);
cgroup_put(cgrp);
return ret;
}
@@ -4118,15 +4636,15 @@
.rename = cgroup_rename,
};
-static void __init cgroup_init_subsys(struct cgroup_subsys *ss)
+static void __init cgroup_init_subsys(struct cgroup_subsys *ss, bool early)
{
struct cgroup_subsys_state *css;
printk(KERN_INFO "Initializing cgroup subsys %s\n", ss->name);
- mutex_lock(&cgroup_tree_mutex);
mutex_lock(&cgroup_mutex);
+ idr_init(&ss->css_idr);
INIT_LIST_HEAD(&ss->cfts);
/* Create the root cgroup state for this subsystem */
@@ -4134,7 +4652,21 @@
css = ss->css_alloc(cgroup_css(&cgrp_dfl_root.cgrp, ss));
/* We don't handle early failures gracefully */
BUG_ON(IS_ERR(css));
- init_css(css, ss, &cgrp_dfl_root.cgrp);
+ init_and_link_css(css, ss, &cgrp_dfl_root.cgrp);
+
+ /*
+ * Root csses are never destroyed and we can't initialize
+ * percpu_ref during early init. Disable refcnting.
+ */
+ css->flags |= CSS_NO_REF;
+
+ if (early) {
+ /* allocation can't be done safely during early init */
+ css->id = 1;
+ } else {
+ css->id = cgroup_idr_alloc(&ss->css_idr, css, 1, 2, GFP_KERNEL);
+ BUG_ON(css->id < 0);
+ }
/* Update the init_css_set to contain a subsys
* pointer to this state - since the subsystem is
@@ -4151,10 +4683,7 @@
BUG_ON(online_css(css));
- cgrp_dfl_root.cgrp.subsys_mask |= 1 << ss->id;
-
mutex_unlock(&cgroup_mutex);
- mutex_unlock(&cgroup_tree_mutex);
}
/**
@@ -4171,6 +4700,8 @@
int i;
init_cgroup_root(&cgrp_dfl_root, &opts);
+ cgrp_dfl_root.cgrp.self.flags |= CSS_NO_REF;
+
RCU_INIT_POINTER(init_task.cgroups, &init_css_set);
for_each_subsys(ss, i) {
@@ -4185,7 +4716,7 @@
ss->name = cgroup_subsys_name[i];
if (ss->early_init)
- cgroup_init_subsys(ss);
+ cgroup_init_subsys(ss, true);
}
return 0;
}
@@ -4204,7 +4735,6 @@
BUG_ON(cgroup_init_cftypes(NULL, cgroup_base_files));
- mutex_lock(&cgroup_tree_mutex);
mutex_lock(&cgroup_mutex);
/* Add init_css_set to the hash table */
@@ -4214,18 +4744,31 @@
BUG_ON(cgroup_setup_root(&cgrp_dfl_root, 0));
mutex_unlock(&cgroup_mutex);
- mutex_unlock(&cgroup_tree_mutex);
for_each_subsys(ss, ssid) {
- if (!ss->early_init)
- cgroup_init_subsys(ss);
+ if (ss->early_init) {
+ struct cgroup_subsys_state *css =
+ init_css_set.subsys[ss->id];
+
+ css->id = cgroup_idr_alloc(&ss->css_idr, css, 1, 2,
+ GFP_KERNEL);
+ BUG_ON(css->id < 0);
+ } else {
+ cgroup_init_subsys(ss, false);
+ }
+
+ list_add_tail(&init_css_set.e_cset_node[ssid],
+ &cgrp_dfl_root.cgrp.e_csets[ssid]);
/*
- * cftype registration needs kmalloc and can't be done
- * during early_init. Register base cftypes separately.
+ * Setting dfl_root subsys_mask needs to consider the
+ * disabled flag and cftype registration needs kmalloc,
+ * both of which aren't available during early_init.
*/
- if (ss->base_cftypes)
+ if (!ss->disabled) {
+ cgrp_dfl_root.subsys_mask |= 1 << ss->id;
WARN_ON(cgroup_add_cftypes(ss, ss->base_cftypes));
+ }
}
cgroup_kobj = kobject_create_and_add("cgroup", fs_kobj);
@@ -4308,7 +4851,7 @@
seq_printf(m, "%d:", root->hierarchy_id);
for_each_subsys(ss, ssid)
- if (root->cgrp.subsys_mask & (1 << ssid))
+ if (root->subsys_mask & (1 << ssid))
seq_printf(m, "%s%s", count++ ? "," : "", ss->name);
if (strlen(root->name))
seq_printf(m, "%sname=%s", count ? "," : "",
@@ -4503,8 +5046,8 @@
static void check_for_release(struct cgroup *cgrp)
{
- if (cgroup_is_releasable(cgrp) &&
- list_empty(&cgrp->cset_links) && list_empty(&cgrp->children)) {
+ if (cgroup_is_releasable(cgrp) && list_empty(&cgrp->cset_links) &&
+ !css_has_online_children(&cgrp->self)) {
/*
* Control Group is currently removeable. If it's not
* already queued for a userspace notification, queue
@@ -4621,7 +5164,7 @@
__setup("cgroup_disable=", cgroup_disable);
/**
- * css_tryget_from_dir - get corresponding css from the dentry of a cgroup dir
+ * css_tryget_online_from_dir - get corresponding css from a cgroup dentry
* @dentry: directory dentry of interest
* @ss: subsystem of interest
*
@@ -4629,8 +5172,8 @@
* to get the corresponding css and return it. If such css doesn't exist
* or can't be pinned, an ERR_PTR value is returned.
*/
-struct cgroup_subsys_state *css_tryget_from_dir(struct dentry *dentry,
- struct cgroup_subsys *ss)
+struct cgroup_subsys_state *css_tryget_online_from_dir(struct dentry *dentry,
+ struct cgroup_subsys *ss)
{
struct kernfs_node *kn = kernfs_node_from_dentry(dentry);
struct cgroup_subsys_state *css = NULL;
@@ -4646,13 +5189,13 @@
/*
* This path doesn't originate from kernfs and @kn could already
* have been or be removed at any point. @kn->priv is RCU
- * protected for this access. See destroy_locked() for details.
+ * protected for this access. See cgroup_rmdir() for details.
*/
cgrp = rcu_dereference(kn->priv);
if (cgrp)
css = cgroup_css(cgrp, ss);
- if (!css || !css_tryget(css))
+ if (!css || !css_tryget_online(css))
css = ERR_PTR(-ENOENT);
rcu_read_unlock();
@@ -4669,14 +5212,8 @@
*/
struct cgroup_subsys_state *css_from_id(int id, struct cgroup_subsys *ss)
{
- struct cgroup *cgrp;
-
- cgroup_assert_mutexes_or_rcu_locked();
-
- cgrp = idr_find(&ss->root->cgroup_idr, id);
- if (cgrp)
- return cgroup_css(cgrp, ss);
- return NULL;
+ WARN_ON_ONCE(!rcu_read_lock_held());
+ return idr_find(&ss->css_idr, id);
}
#ifdef CONFIG_CGROUP_DEBUG
diff --git a/kernel/cgroup_freezer.c b/kernel/cgroup_freezer.c
index 345628c..a79e40f 100644
--- a/kernel/cgroup_freezer.c
+++ b/kernel/cgroup_freezer.c
@@ -59,7 +59,7 @@
static struct freezer *parent_freezer(struct freezer *freezer)
{
- return css_freezer(css_parent(&freezer->css));
+ return css_freezer(freezer->css.parent);
}
bool cgroup_freezing(struct task_struct *task)
@@ -73,10 +73,6 @@
return ret;
}
-/*
- * cgroups_write_string() limits the size of freezer state strings to
- * CGROUP_LOCAL_BUFFER_SIZE
- */
static const char *freezer_state_strs(unsigned int state)
{
if (state & CGROUP_FROZEN)
@@ -304,7 +300,7 @@
/* update states bottom-up */
css_for_each_descendant_post(pos, css) {
- if (!css_tryget(pos))
+ if (!css_tryget_online(pos))
continue;
rcu_read_unlock();
@@ -404,7 +400,7 @@
struct freezer *pos_f = css_freezer(pos);
struct freezer *parent = parent_freezer(pos_f);
- if (!css_tryget(pos))
+ if (!css_tryget_online(pos))
continue;
rcu_read_unlock();
@@ -423,20 +419,22 @@
mutex_unlock(&freezer_mutex);
}
-static int freezer_write(struct cgroup_subsys_state *css, struct cftype *cft,
- char *buffer)
+static ssize_t freezer_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
bool freeze;
- if (strcmp(buffer, freezer_state_strs(0)) == 0)
+ buf = strstrip(buf);
+
+ if (strcmp(buf, freezer_state_strs(0)) == 0)
freeze = false;
- else if (strcmp(buffer, freezer_state_strs(CGROUP_FROZEN)) == 0)
+ else if (strcmp(buf, freezer_state_strs(CGROUP_FROZEN)) == 0)
freeze = true;
else
return -EINVAL;
- freezer_change_state(css_freezer(css), freeze);
- return 0;
+ freezer_change_state(css_freezer(of_css(of)), freeze);
+ return nbytes;
}
static u64 freezer_self_freezing_read(struct cgroup_subsys_state *css,
@@ -460,7 +458,7 @@
.name = "state",
.flags = CFTYPE_NOT_ON_ROOT,
.seq_show = freezer_read,
- .write_string = freezer_write,
+ .write = freezer_write,
},
{
.name = "self_freezing",
diff --git a/kernel/cpuset.c b/kernel/cpuset.c
index 1300178..f6b33c69 100644
--- a/kernel/cpuset.c
+++ b/kernel/cpuset.c
@@ -119,7 +119,7 @@
static inline struct cpuset *parent_cs(struct cpuset *cs)
{
- return css_cs(css_parent(&cs->css));
+ return css_cs(cs->css.parent);
}
#ifdef CONFIG_NUMA
@@ -691,11 +691,8 @@
if (nslot == ndoms) {
static int warnings = 10;
if (warnings) {
- printk(KERN_WARNING
- "rebuild_sched_domains confused:"
- " nslot %d, ndoms %d, csn %d, i %d,"
- " apn %d\n",
- nslot, ndoms, csn, i, apn);
+ pr_warn("rebuild_sched_domains confused: nslot %d, ndoms %d, csn %d, i %d, apn %d\n",
+ nslot, ndoms, csn, i, apn);
warnings--;
}
continue;
@@ -870,7 +867,7 @@
continue;
}
}
- if (!css_tryget(&cp->css))
+ if (!css_tryget_online(&cp->css))
continue;
rcu_read_unlock();
@@ -885,6 +882,7 @@
/**
* update_cpumask - update the cpus_allowed mask of a cpuset and all tasks in it
* @cs: the cpuset to consider
+ * @trialcs: trial cpuset
* @buf: buffer of cpu numbers written to this cpuset
*/
static int update_cpumask(struct cpuset *cs, struct cpuset *trialcs,
@@ -1105,7 +1103,7 @@
continue;
}
}
- if (!css_tryget(&cp->css))
+ if (!css_tryget_online(&cp->css))
continue;
rcu_read_unlock();
@@ -1600,13 +1598,15 @@
/*
* Common handling for a write to a "cpus" or "mems" file.
*/
-static int cpuset_write_resmask(struct cgroup_subsys_state *css,
- struct cftype *cft, char *buf)
+static ssize_t cpuset_write_resmask(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
- struct cpuset *cs = css_cs(css);
+ struct cpuset *cs = css_cs(of_css(of));
struct cpuset *trialcs;
int retval = -ENODEV;
+ buf = strstrip(buf);
+
/*
* CPU or memory hotunplug may leave @cs w/o any execution
* resources, in which case the hotplug code asynchronously updates
@@ -1630,7 +1630,7 @@
goto out_unlock;
}
- switch (cft->private) {
+ switch (of_cft(of)->private) {
case FILE_CPULIST:
retval = update_cpumask(cs, trialcs, buf);
break;
@@ -1645,7 +1645,7 @@
free_trial_cpuset(trialcs);
out_unlock:
mutex_unlock(&cpuset_mutex);
- return retval;
+ return retval ?: nbytes;
}
/*
@@ -1747,7 +1747,7 @@
{
.name = "cpus",
.seq_show = cpuset_common_seq_show,
- .write_string = cpuset_write_resmask,
+ .write = cpuset_write_resmask,
.max_write_len = (100U + 6 * NR_CPUS),
.private = FILE_CPULIST,
},
@@ -1755,7 +1755,7 @@
{
.name = "mems",
.seq_show = cpuset_common_seq_show,
- .write_string = cpuset_write_resmask,
+ .write = cpuset_write_resmask,
.max_write_len = (100U + 6 * MAX_NUMNODES),
.private = FILE_MEMLIST,
},
@@ -2011,7 +2011,7 @@
parent = parent_cs(parent);
if (cgroup_transfer_tasks(parent->css.cgroup, cs->css.cgroup)) {
- printk(KERN_ERR "cpuset: failed to transfer tasks out of empty cpuset ");
+ pr_err("cpuset: failed to transfer tasks out of empty cpuset ");
pr_cont_cgroup_name(cs->css.cgroup);
pr_cont("\n");
}
@@ -2149,7 +2149,7 @@
rcu_read_lock();
cpuset_for_each_descendant_pre(cs, pos_css, &top_cpuset) {
- if (cs == &top_cpuset || !css_tryget(&cs->css))
+ if (cs == &top_cpuset || !css_tryget_online(&cs->css))
continue;
rcu_read_unlock();
@@ -2530,7 +2530,7 @@
/**
* cpuset_print_task_mems_allowed - prints task's cpuset and mems_allowed
- * @task: pointer to task_struct of some task.
+ * @tsk: pointer to task_struct of some task.
*
* Description: Prints @task's name, cpuset name, and cached copy of its
* mems_allowed to the kernel log.
@@ -2548,7 +2548,7 @@
cgrp = task_cs(tsk)->css.cgroup;
nodelist_scnprintf(cpuset_nodelist, CPUSET_NODELIST_LEN,
tsk->mems_allowed);
- printk(KERN_INFO "%s cpuset=", tsk->comm);
+ pr_info("%s cpuset=", tsk->comm);
pr_cont_cgroup_name(cgrp);
pr_cont(" mems_allowed=%s\n", cpuset_nodelist);
@@ -2640,10 +2640,10 @@
/* Display task mems_allowed in /proc/<pid>/status file. */
void cpuset_task_status_allowed(struct seq_file *m, struct task_struct *task)
{
- seq_printf(m, "Mems_allowed:\t");
+ seq_puts(m, "Mems_allowed:\t");
seq_nodemask(m, &task->mems_allowed);
- seq_printf(m, "\n");
- seq_printf(m, "Mems_allowed_list:\t");
+ seq_puts(m, "\n");
+ seq_puts(m, "Mems_allowed_list:\t");
seq_nodemask_list(m, &task->mems_allowed);
- seq_printf(m, "\n");
+ seq_puts(m, "\n");
}
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 689237a0..24d35cc 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -608,7 +608,8 @@
if (!f.file)
return -EBADF;
- css = css_tryget_from_dir(f.file->f_dentry, &perf_event_cgrp_subsys);
+ css = css_tryget_online_from_dir(f.file->f_dentry,
+ &perf_event_cgrp_subsys);
if (IS_ERR(css)) {
ret = PTR_ERR(css);
goto out;
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index d1edc5e..adcd76a 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -1291,14 +1291,8 @@
if (unlikely(!xol_vaddr))
return 0;
- /* Initialize the slot */
- copy_to_page(area->page, xol_vaddr,
- &uprobe->arch.ixol, sizeof(uprobe->arch.ixol));
- /*
- * We probably need flush_icache_user_range() but it needs vma.
- * This should work on supported architectures too.
- */
- flush_dcache_page(area->page);
+ arch_uprobe_copy_ixol(area->page, xol_vaddr,
+ &uprobe->arch.ixol, sizeof(uprobe->arch.ixol));
return xol_vaddr;
}
@@ -1341,6 +1335,21 @@
}
}
+void __weak arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
+ void *src, unsigned long len)
+{
+ /* Initialize the slot */
+ copy_to_page(page, vaddr, src, len);
+
+ /*
+ * We probably need flush_icache_user_range() but it needs vma.
+ * This should work on most of architectures by default. If
+ * architecture needs to do something different it can define
+ * its own version of the function.
+ */
+ flush_dcache_page(page);
+}
+
/**
* uprobe_get_swbp_addr - compute address of swbp given post-swbp regs
* @regs: Reflects the saved state of the task after it has hit a breakpoint
diff --git a/kernel/exit.c b/kernel/exit.c
index 750c2e5..e5c4668 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -313,45 +313,6 @@
}
}
-/*
- * Let kernel threads use this to say that they allow a certain signal.
- * Must not be used if kthread was cloned with CLONE_SIGHAND.
- */
-int allow_signal(int sig)
-{
- if (!valid_signal(sig) || sig < 1)
- return -EINVAL;
-
- spin_lock_irq(¤t->sighand->siglock);
- /* This is only needed for daemonize()'ed kthreads */
- sigdelset(¤t->blocked, sig);
- /*
- * Kernel threads handle their own signals. Let the signal code
- * know it'll be handled, so that they don't get converted to
- * SIGKILL or just silently dropped.
- */
- current->sighand->action[(sig)-1].sa.sa_handler = (void __user *)2;
- recalc_sigpending();
- spin_unlock_irq(¤t->sighand->siglock);
- return 0;
-}
-
-EXPORT_SYMBOL(allow_signal);
-
-int disallow_signal(int sig)
-{
- if (!valid_signal(sig) || sig < 1)
- return -EINVAL;
-
- spin_lock_irq(¤t->sighand->siglock);
- current->sighand->action[(sig)-1].sa.sa_handler = SIG_IGN;
- recalc_sigpending();
- spin_unlock_irq(¤t->sighand->siglock);
- return 0;
-}
-
-EXPORT_SYMBOL(disallow_signal);
-
#ifdef CONFIG_MEMCG
/*
* A task is exiting. If it owned this mm, find a new owner for the mm.
diff --git a/kernel/fork.c b/kernel/fork.c
index 0d53eb0..d2799d1 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1606,10 +1606,12 @@
*/
if (!IS_ERR(p)) {
struct completion vfork;
+ struct pid *pid;
trace_sched_process_fork(current, p);
- nr = task_pid_vnr(p);
+ pid = get_task_pid(p, PIDTYPE_PID);
+ nr = pid_vnr(pid);
if (clone_flags & CLONE_PARENT_SETTID)
put_user(nr, parent_tidptr);
@@ -1624,12 +1626,14 @@
/* forking complete and child started to run, tell ptracer */
if (unlikely(trace))
- ptrace_event(trace, nr);
+ ptrace_event_pid(trace, pid);
if (clone_flags & CLONE_VFORK) {
if (!wait_for_vfork_done(p, &vfork))
- ptrace_event(PTRACE_EVENT_VFORK_DONE, nr);
+ ptrace_event_pid(PTRACE_EVENT_VFORK_DONE, pid);
}
+
+ put_pid(pid);
} else {
nr = PTR_ERR(p);
}
diff --git a/kernel/futex.c b/kernel/futex.c
index 89bc9d5..b632b5f 100644
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -743,10 +743,58 @@
raw_spin_unlock_irq(&curr->pi_lock);
}
+/*
+ * We need to check the following states:
+ *
+ * Waiter | pi_state | pi->owner | uTID | uODIED | ?
+ *
+ * [1] NULL | --- | --- | 0 | 0/1 | Valid
+ * [2] NULL | --- | --- | >0 | 0/1 | Valid
+ *
+ * [3] Found | NULL | -- | Any | 0/1 | Invalid
+ *
+ * [4] Found | Found | NULL | 0 | 1 | Valid
+ * [5] Found | Found | NULL | >0 | 1 | Invalid
+ *
+ * [6] Found | Found | task | 0 | 1 | Valid
+ *
+ * [7] Found | Found | NULL | Any | 0 | Invalid
+ *
+ * [8] Found | Found | task | ==taskTID | 0/1 | Valid
+ * [9] Found | Found | task | 0 | 0 | Invalid
+ * [10] Found | Found | task | !=taskTID | 0/1 | Invalid
+ *
+ * [1] Indicates that the kernel can acquire the futex atomically. We
+ * came came here due to a stale FUTEX_WAITERS/FUTEX_OWNER_DIED bit.
+ *
+ * [2] Valid, if TID does not belong to a kernel thread. If no matching
+ * thread is found then it indicates that the owner TID has died.
+ *
+ * [3] Invalid. The waiter is queued on a non PI futex
+ *
+ * [4] Valid state after exit_robust_list(), which sets the user space
+ * value to FUTEX_WAITERS | FUTEX_OWNER_DIED.
+ *
+ * [5] The user space value got manipulated between exit_robust_list()
+ * and exit_pi_state_list()
+ *
+ * [6] Valid state after exit_pi_state_list() which sets the new owner in
+ * the pi_state but cannot access the user space value.
+ *
+ * [7] pi_state->owner can only be NULL when the OWNER_DIED bit is set.
+ *
+ * [8] Owner and user space value match
+ *
+ * [9] There is no transient state which sets the user space TID to 0
+ * except exit_robust_list(), but this is indicated by the
+ * FUTEX_OWNER_DIED bit. See [4]
+ *
+ * [10] There is no transient state which leaves owner and user space
+ * TID out of sync.
+ */
static int
lookup_pi_state(u32 uval, struct futex_hash_bucket *hb,
- union futex_key *key, struct futex_pi_state **ps,
- struct task_struct *task)
+ union futex_key *key, struct futex_pi_state **ps)
{
struct futex_pi_state *pi_state = NULL;
struct futex_q *this, *next;
@@ -756,12 +804,13 @@
plist_for_each_entry_safe(this, next, &hb->chain, list) {
if (match_futex(&this->key, key)) {
/*
- * Another waiter already exists - bump up
- * the refcount and return its pi_state:
+ * Sanity check the waiter before increasing
+ * the refcount and attaching to it.
*/
pi_state = this->pi_state;
/*
- * Userspace might have messed up non-PI and PI futexes
+ * Userspace might have messed up non-PI and
+ * PI futexes [3]
*/
if (unlikely(!pi_state))
return -EINVAL;
@@ -769,44 +818,70 @@
WARN_ON(!atomic_read(&pi_state->refcount));
/*
- * When pi_state->owner is NULL then the owner died
- * and another waiter is on the fly. pi_state->owner
- * is fixed up by the task which acquires
- * pi_state->rt_mutex.
- *
- * We do not check for pid == 0 which can happen when
- * the owner died and robust_list_exit() cleared the
- * TID.
+ * Handle the owner died case:
*/
- if (pid && pi_state->owner) {
+ if (uval & FUTEX_OWNER_DIED) {
/*
- * Bail out if user space manipulated the
- * futex value.
+ * exit_pi_state_list sets owner to NULL and
+ * wakes the topmost waiter. The task which
+ * acquires the pi_state->rt_mutex will fixup
+ * owner.
*/
- if (pid != task_pid_vnr(pi_state->owner))
+ if (!pi_state->owner) {
+ /*
+ * No pi state owner, but the user
+ * space TID is not 0. Inconsistent
+ * state. [5]
+ */
+ if (pid)
+ return -EINVAL;
+ /*
+ * Take a ref on the state and
+ * return. [4]
+ */
+ goto out_state;
+ }
+
+ /*
+ * If TID is 0, then either the dying owner
+ * has not yet executed exit_pi_state_list()
+ * or some waiter acquired the rtmutex in the
+ * pi state, but did not yet fixup the TID in
+ * user space.
+ *
+ * Take a ref on the state and return. [6]
+ */
+ if (!pid)
+ goto out_state;
+ } else {
+ /*
+ * If the owner died bit is not set,
+ * then the pi_state must have an
+ * owner. [7]
+ */
+ if (!pi_state->owner)
return -EINVAL;
}
/*
- * Protect against a corrupted uval. If uval
- * is 0x80000000 then pid is 0 and the waiter
- * bit is set. So the deadlock check in the
- * calling code has failed and we did not fall
- * into the check above due to !pid.
+ * Bail out if user space manipulated the
+ * futex value. If pi state exists then the
+ * owner TID must be the same as the user
+ * space TID. [9/10]
*/
- if (task && pi_state->owner == task)
- return -EDEADLK;
+ if (pid != task_pid_vnr(pi_state->owner))
+ return -EINVAL;
+ out_state:
atomic_inc(&pi_state->refcount);
*ps = pi_state;
-
return 0;
}
}
/*
* We are the first waiter - try to look up the real owner and attach
- * the new pi_state to it, but bail out when TID = 0
+ * the new pi_state to it, but bail out when TID = 0 [1]
*/
if (!pid)
return -ESRCH;
@@ -839,6 +914,9 @@
return ret;
}
+ /*
+ * No existing pi state. First waiter. [2]
+ */
pi_state = alloc_pi_state();
/*
@@ -910,10 +988,18 @@
return -EDEADLK;
/*
- * Surprise - we got the lock. Just return to userspace:
+ * Surprise - we got the lock, but we do not trust user space at all.
*/
- if (unlikely(!curval))
- return 1;
+ if (unlikely(!curval)) {
+ /*
+ * We verify whether there is kernel state for this
+ * futex. If not, we can safely assume, that the 0 ->
+ * TID transition is correct. If state exists, we do
+ * not bother to fixup the user space state as it was
+ * corrupted already.
+ */
+ return futex_top_waiter(hb, key) ? -EINVAL : 1;
+ }
uval = curval;
@@ -951,7 +1037,7 @@
* We dont have the lock. Look up the PI state (or create it if
* we are the first waiter):
*/
- ret = lookup_pi_state(uval, hb, key, ps, task);
+ ret = lookup_pi_state(uval, hb, key, ps);
if (unlikely(ret)) {
switch (ret) {
@@ -1044,6 +1130,7 @@
struct task_struct *new_owner;
struct futex_pi_state *pi_state = this->pi_state;
u32 uninitialized_var(curval), newval;
+ int ret = 0;
if (!pi_state)
return -EINVAL;
@@ -1067,23 +1154,19 @@
new_owner = this->task;
/*
- * We pass it to the next owner. (The WAITERS bit is always
- * kept enabled while there is PI state around. We must also
- * preserve the owner died bit.)
+ * We pass it to the next owner. The WAITERS bit is always
+ * kept enabled while there is PI state around. We cleanup the
+ * owner died bit, because we are the owner.
*/
- if (!(uval & FUTEX_OWNER_DIED)) {
- int ret = 0;
+ newval = FUTEX_WAITERS | task_pid_vnr(new_owner);
- newval = FUTEX_WAITERS | task_pid_vnr(new_owner);
-
- if (cmpxchg_futex_value_locked(&curval, uaddr, uval, newval))
- ret = -EFAULT;
- else if (curval != uval)
- ret = -EINVAL;
- if (ret) {
- raw_spin_unlock(&pi_state->pi_mutex.wait_lock);
- return ret;
- }
+ if (cmpxchg_futex_value_locked(&curval, uaddr, uval, newval))
+ ret = -EFAULT;
+ else if (curval != uval)
+ ret = -EINVAL;
+ if (ret) {
+ raw_spin_unlock(&pi_state->pi_mutex.wait_lock);
+ return ret;
}
raw_spin_lock_irq(&pi_state->owner->pi_lock);
@@ -1442,6 +1525,13 @@
if (requeue_pi) {
/*
+ * Requeue PI only works on two distinct uaddrs. This
+ * check is only valid for private futexes. See below.
+ */
+ if (uaddr1 == uaddr2)
+ return -EINVAL;
+
+ /*
* requeue_pi requires a pi_state, try to allocate it now
* without any locks in case it fails.
*/
@@ -1479,6 +1569,15 @@
if (unlikely(ret != 0))
goto out_put_key1;
+ /*
+ * The check above which compares uaddrs is not sufficient for
+ * shared futexes. We need to compare the keys:
+ */
+ if (requeue_pi && match_futex(&key1, &key2)) {
+ ret = -EINVAL;
+ goto out_put_keys;
+ }
+
hb1 = hash_futex(&key1);
hb2 = hash_futex(&key2);
@@ -1544,7 +1643,7 @@
* rereading and handing potential crap to
* lookup_pi_state.
*/
- ret = lookup_pi_state(ret, hb2, &key2, &pi_state, NULL);
+ ret = lookup_pi_state(ret, hb2, &key2, &pi_state);
}
switch (ret) {
@@ -2327,9 +2426,10 @@
/*
* To avoid races, try to do the TID -> 0 atomic transition
* again. If it succeeds then we can return without waking
- * anyone else up:
+ * anyone else up. We only try this if neither the waiters nor
+ * the owner died bit are set.
*/
- if (!(uval & FUTEX_OWNER_DIED) &&
+ if (!(uval & ~FUTEX_TID_MASK) &&
cmpxchg_futex_value_locked(&uval, uaddr, vpid, 0))
goto pi_faulted;
/*
@@ -2359,11 +2459,9 @@
/*
* No waiters - kernel unlocks the futex:
*/
- if (!(uval & FUTEX_OWNER_DIED)) {
- ret = unlock_futex_pi(uaddr, uval);
- if (ret == -EFAULT)
- goto pi_faulted;
- }
+ ret = unlock_futex_pi(uaddr, uval);
+ if (ret == -EFAULT)
+ goto pi_faulted;
out_unlock:
spin_unlock(&hb->lock);
@@ -2525,6 +2623,15 @@
if (ret)
goto out_key2;
+ /*
+ * The check above which compares uaddrs is not sufficient for
+ * shared futexes. We need to compare the keys:
+ */
+ if (match_futex(&q.key, &key2)) {
+ ret = -EINVAL;
+ goto out_put_keys;
+ }
+
/* Queue the futex_q, drop the hb lock, wait for wakeup. */
futex_wait_queue_me(hb, &q, to);
diff --git a/kernel/gcov/base.c b/kernel/gcov/base.c
index f45b75b..b358a80 100644
--- a/kernel/gcov/base.c
+++ b/kernel/gcov/base.c
@@ -85,6 +85,12 @@
}
EXPORT_SYMBOL(__gcov_merge_ior);
+void __gcov_merge_time_profile(gcov_type *counters, unsigned int n_counters)
+{
+ /* Unused. */
+}
+EXPORT_SYMBOL(__gcov_merge_time_profile);
+
/**
* gcov_enable_events - enable event reporting through gcov_event()
*
diff --git a/kernel/gcov/gcc_4_7.c b/kernel/gcov/gcc_4_7.c
index 2c6e463..826ba9f 100644
--- a/kernel/gcov/gcc_4_7.c
+++ b/kernel/gcov/gcc_4_7.c
@@ -18,7 +18,12 @@
#include <linux/vmalloc.h>
#include "gcov.h"
+#if __GNUC__ == 4 && __GNUC_MINOR__ >= 9
+#define GCOV_COUNTERS 9
+#else
#define GCOV_COUNTERS 8
+#endif
+
#define GCOV_TAG_FUNCTION_LENGTH 3
static struct gcov_info *gcov_info_head;
diff --git a/kernel/kexec.c b/kernel/kexec.c
index 28c5706..6748688 100644
--- a/kernel/kexec.c
+++ b/kernel/kexec.c
@@ -125,8 +125,8 @@
unsigned long dest);
static int do_kimage_alloc(struct kimage **rimage, unsigned long entry,
- unsigned long nr_segments,
- struct kexec_segment __user *segments)
+ unsigned long nr_segments,
+ struct kexec_segment __user *segments)
{
size_t segment_bytes;
struct kimage *image;
@@ -257,13 +257,13 @@
image->control_code_page = kimage_alloc_control_pages(image,
get_order(KEXEC_CONTROL_PAGE_SIZE));
if (!image->control_code_page) {
- printk(KERN_ERR "Could not allocate control_code_buffer\n");
+ pr_err("Could not allocate control_code_buffer\n");
goto out_free;
}
image->swap_page = kimage_alloc_control_pages(image, 0);
if (!image->swap_page) {
- printk(KERN_ERR "Could not allocate swap buffer\n");
+ pr_err("Could not allocate swap buffer\n");
goto out_free;
}
@@ -332,7 +332,7 @@
image->control_code_page = kimage_alloc_control_pages(image,
get_order(KEXEC_CONTROL_PAGE_SIZE));
if (!image->control_code_page) {
- printk(KERN_ERR "Could not allocate control_code_buffer\n");
+ pr_err("Could not allocate control_code_buffer\n");
goto out_free;
}
@@ -621,8 +621,8 @@
#define for_each_kimage_entry(image, ptr, entry) \
for (ptr = &image->head; (entry = *ptr) && !(entry & IND_DONE); \
- ptr = (entry & IND_INDIRECTION)? \
- phys_to_virt((entry & PAGE_MASK)): ptr +1)
+ ptr = (entry & IND_INDIRECTION) ? \
+ phys_to_virt((entry & PAGE_MASK)) : ptr + 1)
static void kimage_free_entry(kimage_entry_t entry)
{
@@ -650,8 +650,7 @@
* done with it.
*/
ind = entry;
- }
- else if (entry & IND_SOURCE)
+ } else if (entry & IND_SOURCE)
kimage_free_entry(entry);
}
/* Free the final indirection page */
@@ -774,8 +773,7 @@
addr = old_addr;
page = old_page;
break;
- }
- else {
+ } else {
/* Place the page on the destination list I
* will use it later.
*/
@@ -1059,7 +1057,7 @@
return -EINVAL;
ksegments = compat_alloc_user_space(nr_segments * sizeof(out));
- for (i=0; i < nr_segments; i++) {
+ for (i = 0; i < nr_segments; i++) {
result = copy_from_user(&in, &segments[i], sizeof(in));
if (result)
return -EFAULT;
@@ -1214,14 +1212,14 @@
* squirrelled away. ELF notes happen to provide
* all of that, so there is no need to invent something new.
*/
- buf = (u32*)per_cpu_ptr(crash_notes, cpu);
+ buf = (u32 *)per_cpu_ptr(crash_notes, cpu);
if (!buf)
return;
memset(&prstatus, 0, sizeof(prstatus));
prstatus.pr_pid = current->pid;
elf_core_copy_kernel_regs(&prstatus.pr_reg, regs);
buf = append_elf_note(buf, KEXEC_CORE_NOTE_NAME, NT_PRSTATUS,
- &prstatus, sizeof(prstatus));
+ &prstatus, sizeof(prstatus));
final_note(buf);
}
@@ -1230,8 +1228,7 @@
/* Allocate memory for saving cpu registers. */
crash_notes = alloc_percpu(note_buf_t);
if (!crash_notes) {
- printk("Kexec: Memory allocation for saving cpu register"
- " states failed\n");
+ pr_warn("Kexec: Memory allocation for saving cpu register states failed\n");
return -ENOMEM;
}
return 0;
@@ -1253,10 +1250,10 @@
*
* The function returns 0 on success and -EINVAL on failure.
*/
-static int __init parse_crashkernel_mem(char *cmdline,
- unsigned long long system_ram,
- unsigned long long *crash_size,
- unsigned long long *crash_base)
+static int __init parse_crashkernel_mem(char *cmdline,
+ unsigned long long system_ram,
+ unsigned long long *crash_size,
+ unsigned long long *crash_base)
{
char *cur = cmdline, *tmp;
@@ -1267,12 +1264,12 @@
/* get the start of the range */
start = memparse(cur, &tmp);
if (cur == tmp) {
- pr_warning("crashkernel: Memory value expected\n");
+ pr_warn("crashkernel: Memory value expected\n");
return -EINVAL;
}
cur = tmp;
if (*cur != '-') {
- pr_warning("crashkernel: '-' expected\n");
+ pr_warn("crashkernel: '-' expected\n");
return -EINVAL;
}
cur++;
@@ -1281,31 +1278,30 @@
if (*cur != ':') {
end = memparse(cur, &tmp);
if (cur == tmp) {
- pr_warning("crashkernel: Memory "
- "value expected\n");
+ pr_warn("crashkernel: Memory value expected\n");
return -EINVAL;
}
cur = tmp;
if (end <= start) {
- pr_warning("crashkernel: end <= start\n");
+ pr_warn("crashkernel: end <= start\n");
return -EINVAL;
}
}
if (*cur != ':') {
- pr_warning("crashkernel: ':' expected\n");
+ pr_warn("crashkernel: ':' expected\n");
return -EINVAL;
}
cur++;
size = memparse(cur, &tmp);
if (cur == tmp) {
- pr_warning("Memory value expected\n");
+ pr_warn("Memory value expected\n");
return -EINVAL;
}
cur = tmp;
if (size >= system_ram) {
- pr_warning("crashkernel: invalid size\n");
+ pr_warn("crashkernel: invalid size\n");
return -EINVAL;
}
@@ -1323,8 +1319,7 @@
cur++;
*crash_base = memparse(cur, &tmp);
if (cur == tmp) {
- pr_warning("Memory value expected "
- "after '@'\n");
+ pr_warn("Memory value expected after '@'\n");
return -EINVAL;
}
}
@@ -1336,26 +1331,26 @@
/*
* That function parses "simple" (old) crashkernel command lines like
*
- * crashkernel=size[@offset]
+ * crashkernel=size[@offset]
*
* It returns 0 on success and -EINVAL on failure.
*/
-static int __init parse_crashkernel_simple(char *cmdline,
- unsigned long long *crash_size,
- unsigned long long *crash_base)
+static int __init parse_crashkernel_simple(char *cmdline,
+ unsigned long long *crash_size,
+ unsigned long long *crash_base)
{
char *cur = cmdline;
*crash_size = memparse(cmdline, &cur);
if (cmdline == cur) {
- pr_warning("crashkernel: memory value expected\n");
+ pr_warn("crashkernel: memory value expected\n");
return -EINVAL;
}
if (*cur == '@')
*crash_base = memparse(cur+1, &cur);
else if (*cur != ' ' && *cur != '\0') {
- pr_warning("crashkernel: unrecognized char\n");
+ pr_warn("crashkernel: unrecognized char\n");
return -EINVAL;
}
@@ -1691,7 +1686,7 @@
* CPU hotplug again; so re-enable it here.
*/
cpu_hotplug_enable();
- printk(KERN_EMERG "Starting new kernel\n");
+ pr_emerg("Starting new kernel\n");
machine_shutdown();
}
diff --git a/kernel/kmod.c b/kernel/kmod.c
index 0ac67a5..8637e04 100644
--- a/kernel/kmod.c
+++ b/kernel/kmod.c
@@ -285,10 +285,7 @@
pid_t pid;
/* If SIGCLD is ignored sys_wait4 won't populate the status. */
- spin_lock_irq(¤t->sighand->siglock);
- current->sighand->action[SIGCHLD-1].sa.sa_handler = SIG_DFL;
- spin_unlock_irq(¤t->sighand->siglock);
-
+ kernel_sigaction(SIGCHLD, SIG_DFL);
pid = kernel_thread(____call_usermodehelper, sub_info, SIGCHLD);
if (pid < 0) {
sub_info->retval = pid;
diff --git a/kernel/panic.c b/kernel/panic.c
index d02fa9f..62e16ce 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -32,6 +32,7 @@
static int pause_on_oops;
static int pause_on_oops_flag;
static DEFINE_SPINLOCK(pause_on_oops_lock);
+static bool crash_kexec_post_notifiers;
int panic_timeout = CONFIG_PANIC_TIMEOUT;
EXPORT_SYMBOL_GPL(panic_timeout);
@@ -112,9 +113,11 @@
/*
* If we have crashed and we have a crash kernel loaded let it handle
* everything else.
- * Do we want to call this before we try to display a message?
+ * If we want to run this after calling panic_notifiers, pass
+ * the "crash_kexec_post_notifiers" option to the kernel.
*/
- crash_kexec(NULL);
+ if (!crash_kexec_post_notifiers)
+ crash_kexec(NULL);
/*
* Note smp_send_stop is the usual smp shutdown function, which
@@ -131,6 +134,15 @@
kmsg_dump(KMSG_DUMP_PANIC);
+ /*
+ * If you doubt kdump always works fine in any situation,
+ * "crash_kexec_post_notifiers" offers you a chance to run
+ * panic_notifiers and dumping kmsg before kdump.
+ * Note: since some panic_notifiers can make crashed kernel
+ * more unstable, it can increase risks of the kdump failure too.
+ */
+ crash_kexec(NULL);
+
bust_spinlocks(0);
if (!panic_blink)
@@ -472,6 +484,13 @@
core_param(panic, panic_timeout, int, 0644);
core_param(pause_on_oops, pause_on_oops, int, 0644);
+static int __init setup_crash_kexec_post_notifiers(char *s)
+{
+ crash_kexec_post_notifiers = true;
+ return 0;
+}
+early_param("crash_kexec_post_notifiers", setup_crash_kexec_post_notifiers);
+
static int __init oops_setup(char *s)
{
if (!s)
diff --git a/kernel/profile.c b/kernel/profile.c
index cb980f0..54bf5ba 100644
--- a/kernel/profile.c
+++ b/kernel/profile.c
@@ -52,9 +52,9 @@
int profile_setup(char *str)
{
- static char schedstr[] = "schedule";
- static char sleepstr[] = "sleep";
- static char kvmstr[] = "kvm";
+ static const char schedstr[] = "schedule";
+ static const char sleepstr[] = "sleep";
+ static const char kvmstr[] = "kvm";
int par;
if (!strncmp(str, sleepstr, strlen(sleepstr))) {
@@ -64,12 +64,10 @@
str += strlen(sleepstr) + 1;
if (get_option(&str, &par))
prof_shift = par;
- printk(KERN_INFO
- "kernel sleep profiling enabled (shift: %ld)\n",
+ pr_info("kernel sleep profiling enabled (shift: %ld)\n",
prof_shift);
#else
- printk(KERN_WARNING
- "kernel sleep profiling requires CONFIG_SCHEDSTATS\n");
+ pr_warn("kernel sleep profiling requires CONFIG_SCHEDSTATS\n");
#endif /* CONFIG_SCHEDSTATS */
} else if (!strncmp(str, schedstr, strlen(schedstr))) {
prof_on = SCHED_PROFILING;
@@ -77,8 +75,7 @@
str += strlen(schedstr) + 1;
if (get_option(&str, &par))
prof_shift = par;
- printk(KERN_INFO
- "kernel schedule profiling enabled (shift: %ld)\n",
+ pr_info("kernel schedule profiling enabled (shift: %ld)\n",
prof_shift);
} else if (!strncmp(str, kvmstr, strlen(kvmstr))) {
prof_on = KVM_PROFILING;
@@ -86,13 +83,12 @@
str += strlen(kvmstr) + 1;
if (get_option(&str, &par))
prof_shift = par;
- printk(KERN_INFO
- "kernel KVM profiling enabled (shift: %ld)\n",
+ pr_info("kernel KVM profiling enabled (shift: %ld)\n",
prof_shift);
} else if (get_option(&str, &par)) {
prof_shift = par;
prof_on = CPU_PROFILING;
- printk(KERN_INFO "kernel profiling enabled (shift: %ld)\n",
+ pr_info("kernel profiling enabled (shift: %ld)\n",
prof_shift);
}
return 1;
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index caf03e8..c6b9879 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3723,7 +3723,7 @@
if (retval)
return retval;
- if (attr.sched_policy < 0)
+ if ((int)attr.sched_policy < 0)
return -EINVAL;
rcu_read_lock();
@@ -7669,7 +7669,7 @@
static int cpu_cgroup_css_online(struct cgroup_subsys_state *css)
{
struct task_group *tg = css_tg(css);
- struct task_group *parent = css_tg(css_parent(css));
+ struct task_group *parent = css_tg(css->parent);
if (parent)
sched_online_group(tg, parent);
@@ -7800,8 +7800,7 @@
/* restart the period timer (if active) to handle new period expiry */
if (runtime_enabled && cfs_b->timer_active) {
/* force a reprogram */
- cfs_b->timer_active = 0;
- __start_cfs_bandwidth(cfs_b);
+ __start_cfs_bandwidth(cfs_b, true);
}
raw_spin_unlock_irq(&cfs_b->lock);
diff --git a/kernel/sched/cpuacct.c b/kernel/sched/cpuacct.c
index c143ee3..9cf350c 100644
--- a/kernel/sched/cpuacct.c
+++ b/kernel/sched/cpuacct.c
@@ -46,7 +46,7 @@
static inline struct cpuacct *parent_ca(struct cpuacct *ca)
{
- return css_ca(css_parent(&ca->css));
+ return css_ca(ca->css.parent);
}
static DEFINE_PER_CPU(u64, root_cpuacct_cpuusage);
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index e1574fc..2b8cbf0 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -508,9 +508,17 @@
struct sched_dl_entity,
dl_timer);
struct task_struct *p = dl_task_of(dl_se);
- struct rq *rq = task_rq(p);
+ struct rq *rq;
+again:
+ rq = task_rq(p);
raw_spin_lock(&rq->lock);
+ if (rq != task_rq(p)) {
+ /* Task was moved, retrying. */
+ raw_spin_unlock(&rq->lock);
+ goto again;
+ }
+
/*
* We need to take care of a possible races here. In fact, the
* task might have changed its scheduling policy to something
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index c9617b7..9855e87d 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -1120,7 +1120,7 @@
old_imb = orig_dst_load * 100 - orig_src_load * env->imbalance_pct;
/* Would this change make things worse? */
- return (old_imb > imb);
+ return (imb > old_imb);
}
/*
@@ -1745,18 +1745,19 @@
void task_numa_free(struct task_struct *p)
{
struct numa_group *grp = p->numa_group;
- int i;
void *numa_faults = p->numa_faults_memory;
+ unsigned long flags;
+ int i;
if (grp) {
- spin_lock_irq(&grp->lock);
+ spin_lock_irqsave(&grp->lock, flags);
for (i = 0; i < NR_NUMA_HINT_FAULT_STATS * nr_node_ids; i++)
grp->faults[i] -= p->numa_faults_memory[i];
grp->total_faults -= p->total_numa_faults;
list_del(&p->numa_entry);
grp->nr_tasks--;
- spin_unlock_irq(&grp->lock);
+ spin_unlock_irqrestore(&grp->lock, flags);
rcu_assign_pointer(p->numa_group, NULL);
put_numa_group(grp);
}
@@ -3179,7 +3180,7 @@
*/
if (!cfs_b->timer_active) {
__refill_cfs_bandwidth_runtime(cfs_b);
- __start_cfs_bandwidth(cfs_b);
+ __start_cfs_bandwidth(cfs_b, false);
}
if (cfs_b->runtime > 0) {
@@ -3358,7 +3359,7 @@
raw_spin_lock(&cfs_b->lock);
list_add_tail_rcu(&cfs_rq->throttled_list, &cfs_b->throttled_cfs_rq);
if (!cfs_b->timer_active)
- __start_cfs_bandwidth(cfs_b);
+ __start_cfs_bandwidth(cfs_b, false);
raw_spin_unlock(&cfs_b->lock);
}
@@ -3740,7 +3741,7 @@
}
/* requires cfs_b->lock, may release to reprogram timer */
-void __start_cfs_bandwidth(struct cfs_bandwidth *cfs_b)
+void __start_cfs_bandwidth(struct cfs_bandwidth *cfs_b, bool force)
{
/*
* The timer may be active because we're trying to set a new bandwidth
@@ -3755,7 +3756,7 @@
cpu_relax();
raw_spin_lock(&cfs_b->lock);
/* if someone else restarted the timer then we're done */
- if (cfs_b->timer_active)
+ if (!force && cfs_b->timer_active)
return;
}
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 600e229..e47679b 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -278,7 +278,7 @@
extern int sched_group_set_shares(struct task_group *tg, unsigned long shares);
extern void __refill_cfs_bandwidth_runtime(struct cfs_bandwidth *cfs_b);
-extern void __start_cfs_bandwidth(struct cfs_bandwidth *cfs_b);
+extern void __start_cfs_bandwidth(struct cfs_bandwidth *cfs_b, bool force);
extern void unthrottle_cfs_rq(struct cfs_rq *cfs_rq);
extern void free_rt_sched_group(struct task_group *tg);
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index b35c215..f6d76be 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -39,7 +39,7 @@
* is only needed for handling filters shared across tasks.
* @prev: points to a previously installed, or inherited, filter
* @len: the number of instructions in the program
- * @insns: the BPF program instructions to evaluate
+ * @insnsi: the BPF program instructions to evaluate
*
* seccomp_filter objects are organized in a tree linked via the @prev
* pointer. For any task, it appears to be a singly-linked list starting
@@ -220,7 +220,7 @@
return -ENOMEM;
/*
- * Installing a seccomp filter requires that the task have
+ * Installing a seccomp filter requires that the task has
* CAP_SYS_ADMIN in its namespace or be running with no_new_privs.
* This avoids scenarios where unprivileged tasks can affect the
* behavior of privileged children.
diff --git a/kernel/signal.c b/kernel/signal.c
index 6e600aaa..a4077e9 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -277,6 +277,7 @@
{
if (unlikely(task->jobctl & JOBCTL_TRAPPING)) {
task->jobctl &= ~JOBCTL_TRAPPING;
+ smp_mb(); /* advised by wake_up_bit() */
wake_up_bit(&task->jobctl, JOBCTL_TRAPPING_BIT);
}
}
@@ -705,11 +706,8 @@
* Returns 1 if any signals were found.
*
* All callers must be holding the siglock.
- *
- * This version takes a sigset mask and looks at all signals,
- * not just those in the first mask word.
*/
-static int rm_from_queue_full(sigset_t *mask, struct sigpending *s)
+static int flush_sigqueue_mask(sigset_t *mask, struct sigpending *s)
{
struct sigqueue *q, *n;
sigset_t m;
@@ -727,29 +725,6 @@
}
return 1;
}
-/*
- * Remove signals in mask from the pending set and queue.
- * Returns 1 if any signals were found.
- *
- * All callers must be holding the siglock.
- */
-static int rm_from_queue(unsigned long mask, struct sigpending *s)
-{
- struct sigqueue *q, *n;
-
- if (!sigtestsetmask(&s->signal, mask))
- return 0;
-
- sigdelsetmask(&s->signal, mask);
- list_for_each_entry_safe(q, n, &s->list, list) {
- if (q->info.si_signo < SIGRTMIN &&
- (mask & sigmask(q->info.si_signo))) {
- list_del_init(&q->list);
- __sigqueue_free(q);
- }
- }
- return 1;
-}
static inline int is_si_special(const struct siginfo *info)
{
@@ -861,6 +836,7 @@
{
struct signal_struct *signal = p->signal;
struct task_struct *t;
+ sigset_t flush;
if (signal->flags & (SIGNAL_GROUP_EXIT | SIGNAL_GROUP_COREDUMP)) {
if (signal->flags & SIGNAL_GROUP_COREDUMP)
@@ -872,26 +848,25 @@
/*
* This is a stop signal. Remove SIGCONT from all queues.
*/
- rm_from_queue(sigmask(SIGCONT), &signal->shared_pending);
- t = p;
- do {
- rm_from_queue(sigmask(SIGCONT), &t->pending);
- } while_each_thread(p, t);
+ siginitset(&flush, sigmask(SIGCONT));
+ flush_sigqueue_mask(&flush, &signal->shared_pending);
+ for_each_thread(p, t)
+ flush_sigqueue_mask(&flush, &t->pending);
} else if (sig == SIGCONT) {
unsigned int why;
/*
* Remove all stop signals from all queues, wake all threads.
*/
- rm_from_queue(SIG_KERNEL_STOP_MASK, &signal->shared_pending);
- t = p;
- do {
+ siginitset(&flush, SIG_KERNEL_STOP_MASK);
+ flush_sigqueue_mask(&flush, &signal->shared_pending);
+ for_each_thread(p, t) {
+ flush_sigqueue_mask(&flush, &t->pending);
task_clear_jobctl_pending(t, JOBCTL_STOP_PENDING);
- rm_from_queue(SIG_KERNEL_STOP_MASK, &t->pending);
if (likely(!(t->ptrace & PT_SEIZED)))
wake_up_state(t, __TASK_STOPPED);
else
ptrace_trap_notify(t);
- } while_each_thread(p, t);
+ }
/*
* Notify the parent with CLD_CONTINUED if we were stopped.
@@ -2854,7 +2829,7 @@
spin_lock_irq(&tsk->sighand->siglock);
__set_task_blocked(tsk, &tsk->real_blocked);
- siginitset(&tsk->real_blocked, 0);
+ sigemptyset(&tsk->real_blocked);
sig = dequeue_signal(tsk, &mask, info);
}
spin_unlock_irq(&tsk->sighand->siglock);
@@ -3091,18 +3066,39 @@
}
#endif
+/*
+ * For kthreads only, must not be used if cloned with CLONE_SIGHAND
+ */
+void kernel_sigaction(int sig, __sighandler_t action)
+{
+ spin_lock_irq(¤t->sighand->siglock);
+ current->sighand->action[sig - 1].sa.sa_handler = action;
+ if (action == SIG_IGN) {
+ sigset_t mask;
+
+ sigemptyset(&mask);
+ sigaddset(&mask, sig);
+
+ flush_sigqueue_mask(&mask, ¤t->signal->shared_pending);
+ flush_sigqueue_mask(&mask, ¤t->pending);
+ recalc_sigpending();
+ }
+ spin_unlock_irq(¤t->sighand->siglock);
+}
+EXPORT_SYMBOL(kernel_sigaction);
+
int do_sigaction(int sig, struct k_sigaction *act, struct k_sigaction *oact)
{
- struct task_struct *t = current;
+ struct task_struct *p = current, *t;
struct k_sigaction *k;
sigset_t mask;
if (!valid_signal(sig) || sig < 1 || (act && sig_kernel_only(sig)))
return -EINVAL;
- k = &t->sighand->action[sig-1];
+ k = &p->sighand->action[sig-1];
- spin_lock_irq(¤t->sighand->siglock);
+ spin_lock_irq(&p->sighand->siglock);
if (oact)
*oact = *k;
@@ -3121,21 +3117,20 @@
* (for example, SIGCHLD), shall cause the pending signal to
* be discarded, whether or not it is blocked"
*/
- if (sig_handler_ignored(sig_handler(t, sig), sig)) {
+ if (sig_handler_ignored(sig_handler(p, sig), sig)) {
sigemptyset(&mask);
sigaddset(&mask, sig);
- rm_from_queue_full(&mask, &t->signal->shared_pending);
- do {
- rm_from_queue_full(&mask, &t->pending);
- } while_each_thread(current, t);
+ flush_sigqueue_mask(&mask, &p->signal->shared_pending);
+ for_each_thread(p, t)
+ flush_sigqueue_mask(&mask, &t->pending);
}
}
- spin_unlock_irq(¤t->sighand->siglock);
+ spin_unlock_irq(&p->sighand->siglock);
return 0;
}
-static int
+static int
do_sigaltstack (const stack_t __user *uss, stack_t __user *uoss, unsigned long sp)
{
stack_t oss;
diff --git a/kernel/smp.c b/kernel/smp.c
index 06d574e..306f818 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -185,14 +185,26 @@
{
struct llist_node *entry;
struct call_single_data *csd, *csd_next;
+ static bool warned;
+
+ entry = llist_del_all(&__get_cpu_var(call_single_queue));
+ entry = llist_reverse_order(entry);
/*
* Shouldn't receive this interrupt on a cpu that is not yet online.
*/
- WARN_ON_ONCE(!cpu_online(smp_processor_id()));
+ if (unlikely(!cpu_online(smp_processor_id()) && !warned)) {
+ warned = true;
+ WARN(1, "IPI on offline CPU %d\n", smp_processor_id());
- entry = llist_del_all(&__get_cpu_var(call_single_queue));
- entry = llist_reverse_order(entry);
+ /*
+ * We don't have to use the _safe() variant here
+ * because we are not invoking the IPI handlers yet.
+ */
+ llist_for_each_entry(csd, entry, llist)
+ pr_warn("IPI callback %pS sent to offline CPU\n",
+ csd->func);
+ }
llist_for_each_entry_safe(csd, csd_next, entry, llist) {
csd->func(csd->info);
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 40ce2d9..db19e3e 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -173,6 +173,13 @@
#endif
#ifdef CONFIG_PROC_SYSCTL
+
+#define SYSCTL_WRITES_LEGACY -1
+#define SYSCTL_WRITES_WARN 0
+#define SYSCTL_WRITES_STRICT 1
+
+static int sysctl_writes_strict = SYSCTL_WRITES_WARN;
+
static int proc_do_cad_pid(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos);
static int proc_taint(struct ctl_table *table, int write,
@@ -195,7 +202,7 @@
/* Note: sysrq code uses it's own private copy */
static int __sysrq_enabled = CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE;
-static int sysrq_sysctl_handler(ctl_table *table, int write,
+static int sysrq_sysctl_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp,
loff_t *ppos)
{
@@ -495,6 +502,15 @@
.mode = 0644,
.proc_handler = proc_taint,
},
+ {
+ .procname = "sysctl_writes_strict",
+ .data = &sysctl_writes_strict,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &neg_one,
+ .extra2 = &one,
+ },
#endif
#ifdef CONFIG_LATENCYTOP
{
@@ -1703,8 +1719,8 @@
#ifdef CONFIG_PROC_SYSCTL
-static int _proc_do_string(void* data, int maxlen, int write,
- void __user *buffer,
+static int _proc_do_string(char *data, int maxlen, int write,
+ char __user *buffer,
size_t *lenp, loff_t *ppos)
{
size_t len;
@@ -1717,21 +1733,30 @@
}
if (write) {
- len = 0;
+ if (sysctl_writes_strict == SYSCTL_WRITES_STRICT) {
+ /* Only continue writes not past the end of buffer. */
+ len = strlen(data);
+ if (len > maxlen - 1)
+ len = maxlen - 1;
+
+ if (*ppos > len)
+ return 0;
+ len = *ppos;
+ } else {
+ /* Start writing from beginning of buffer. */
+ len = 0;
+ }
+
+ *ppos += *lenp;
p = buffer;
- while (len < *lenp) {
+ while ((p - buffer) < *lenp && len < maxlen - 1) {
if (get_user(c, p++))
return -EFAULT;
if (c == 0 || c == '\n')
break;
- len++;
+ data[len++] = c;
}
- if (len >= maxlen)
- len = maxlen-1;
- if(copy_from_user(data, buffer, len))
- return -EFAULT;
- ((char *) data)[len] = 0;
- *ppos += *lenp;
+ data[len] = 0;
} else {
len = strlen(data);
if (len > maxlen)
@@ -1748,10 +1773,10 @@
if (len > *lenp)
len = *lenp;
if (len)
- if(copy_to_user(buffer, data, len))
+ if (copy_to_user(buffer, data, len))
return -EFAULT;
if (len < *lenp) {
- if(put_user('\n', ((char __user *) buffer) + len))
+ if (put_user('\n', buffer + len))
return -EFAULT;
len++;
}
@@ -1761,6 +1786,14 @@
return 0;
}
+static void warn_sysctl_write(struct ctl_table *table)
+{
+ pr_warn_once("%s wrote to %s when file position was not 0!\n"
+ "This will not be supported in the future. To silence this\n"
+ "warning, set kernel.sysctl_writes_strict = -1\n",
+ current->comm, table->procname);
+}
+
/**
* proc_dostring - read a string sysctl
* @table: the sysctl table
@@ -1781,8 +1814,11 @@
int proc_dostring(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
- return _proc_do_string(table->data, table->maxlen, write,
- buffer, lenp, ppos);
+ if (write && *ppos && sysctl_writes_strict == SYSCTL_WRITES_WARN)
+ warn_sysctl_write(table);
+
+ return _proc_do_string((char *)(table->data), table->maxlen, write,
+ (char __user *)buffer, lenp, ppos);
}
static size_t proc_skip_spaces(char **buf)
@@ -1956,6 +1992,18 @@
conv = do_proc_dointvec_conv;
if (write) {
+ if (*ppos) {
+ switch (sysctl_writes_strict) {
+ case SYSCTL_WRITES_STRICT:
+ goto out;
+ case SYSCTL_WRITES_WARN:
+ warn_sysctl_write(table);
+ break;
+ default:
+ break;
+ }
+ }
+
if (left > PAGE_SIZE - 1)
left = PAGE_SIZE - 1;
page = __get_free_page(GFP_TEMPORARY);
@@ -2013,6 +2061,7 @@
return err ? : -EINVAL;
}
*lenp -= left;
+out:
*ppos += *lenp;
return err;
}
@@ -2205,6 +2254,18 @@
left = *lenp;
if (write) {
+ if (*ppos) {
+ switch (sysctl_writes_strict) {
+ case SYSCTL_WRITES_STRICT:
+ goto out;
+ case SYSCTL_WRITES_WARN:
+ warn_sysctl_write(table);
+ break;
+ default:
+ break;
+ }
+ }
+
if (left > PAGE_SIZE - 1)
left = PAGE_SIZE - 1;
page = __get_free_page(GFP_TEMPORARY);
@@ -2260,6 +2321,7 @@
return err ? : -EINVAL;
}
*lenp -= left;
+out:
*ppos += *lenp;
return err;
}
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 8639819..d440935 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -535,6 +535,36 @@
Say N, unless you absolutely know what you are doing.
+config TRACEPOINT_BENCHMARK
+ bool "Add tracepoint that benchmarks tracepoints"
+ help
+ This option creates the tracepoint "benchmark:benchmark_event".
+ When the tracepoint is enabled, it kicks off a kernel thread that
+ goes into an infinite loop (calling cond_sched() to let other tasks
+ run), and calls the tracepoint. Each iteration will record the time
+ it took to write to the tracepoint and the next iteration that
+ data will be passed to the tracepoint itself. That is, the tracepoint
+ will report the time it took to do the previous tracepoint.
+ The string written to the tracepoint is a static string of 128 bytes
+ to keep the time the same. The initial string is simply a write of
+ "START". The second string records the cold cache time of the first
+ write which is not added to the rest of the calculations.
+
+ As it is a tight loop, it benchmarks as hot cache. That's fine because
+ we care most about hot paths that are probably in cache already.
+
+ An example of the output:
+
+ START
+ first=3672 [COLD CACHED]
+ last=632 first=3672 max=632 min=632 avg=316 std=446 std^2=199712
+ last=278 first=3672 max=632 min=278 avg=303 std=316 std^2=100337
+ last=277 first=3672 max=632 min=277 avg=296 std=258 std^2=67064
+ last=273 first=3672 max=632 min=273 avg=292 std=224 std^2=50411
+ last=273 first=3672 max=632 min=273 avg=288 std=200 std^2=40389
+ last=281 first=3672 max=632 min=273 avg=287 std=183 std^2=33666
+
+
config RING_BUFFER_BENCHMARK
tristate "Ring buffer benchmark stress tester"
depends on RING_BUFFER
diff --git a/kernel/trace/Makefile b/kernel/trace/Makefile
index 1378e84..2611613 100644
--- a/kernel/trace/Makefile
+++ b/kernel/trace/Makefile
@@ -17,6 +17,7 @@
KBUILD_CFLAGS += -DDISABLE_BRANCH_PROFILING
endif
+CFLAGS_trace_benchmark.o := -I$(src)
CFLAGS_trace_events_filter.o := -I$(src)
obj-$(CONFIG_TRACE_CLOCK) += trace_clock.o
@@ -62,4 +63,6 @@
obj-$(CONFIG_PROBE_EVENTS) += trace_probe.o
obj-$(CONFIG_UPROBE_EVENT) += trace_uprobe.o
+obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
+
libftrace-y := ftrace.o
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index 4a54a25..5b372e3 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -62,7 +62,7 @@
#define FTRACE_HASH_DEFAULT_BITS 10
#define FTRACE_HASH_MAX_BITS 12
-#define FL_GLOBAL_CONTROL_MASK (FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_CONTROL)
+#define FL_GLOBAL_CONTROL_MASK (FTRACE_OPS_FL_CONTROL)
#ifdef CONFIG_DYNAMIC_FTRACE
#define INIT_REGEX_LOCK(opsname) \
@@ -103,7 +103,6 @@
static DEFINE_MUTEX(ftrace_lock);
-static struct ftrace_ops *ftrace_global_list __read_mostly = &ftrace_list_end;
static struct ftrace_ops *ftrace_control_list __read_mostly = &ftrace_list_end;
static struct ftrace_ops *ftrace_ops_list __read_mostly = &ftrace_list_end;
ftrace_func_t ftrace_trace_function __read_mostly = ftrace_stub;
@@ -171,23 +170,6 @@
return cnt;
}
-static void
-ftrace_global_list_func(unsigned long ip, unsigned long parent_ip,
- struct ftrace_ops *op, struct pt_regs *regs)
-{
- int bit;
-
- bit = trace_test_and_set_recursion(TRACE_GLOBAL_START, TRACE_GLOBAL_MAX);
- if (bit < 0)
- return;
-
- do_for_each_ftrace_op(op, ftrace_global_list) {
- op->func(ip, parent_ip, op, regs);
- } while_for_each_ftrace_op(op);
-
- trace_clear_recursion(bit);
-}
-
static void ftrace_pid_func(unsigned long ip, unsigned long parent_ip,
struct ftrace_ops *op, struct pt_regs *regs)
{
@@ -237,43 +219,6 @@
return 0;
}
-static void update_global_ops(void)
-{
- ftrace_func_t func = ftrace_global_list_func;
- void *private = NULL;
-
- /* The list has its own recursion protection. */
- global_ops.flags |= FTRACE_OPS_FL_RECURSION_SAFE;
-
- /*
- * If there's only one function registered, then call that
- * function directly. Otherwise, we need to iterate over the
- * registered callers.
- */
- if (ftrace_global_list == &ftrace_list_end ||
- ftrace_global_list->next == &ftrace_list_end) {
- func = ftrace_global_list->func;
- private = ftrace_global_list->private;
- /*
- * As we are calling the function directly.
- * If it does not have recursion protection,
- * the function_trace_op needs to be updated
- * accordingly.
- */
- if (!(ftrace_global_list->flags & FTRACE_OPS_FL_RECURSION_SAFE))
- global_ops.flags &= ~FTRACE_OPS_FL_RECURSION_SAFE;
- }
-
- /* If we filter on pids, update to use the pid function */
- if (!list_empty(&ftrace_pids)) {
- set_ftrace_pid_function(func);
- func = ftrace_pid_func;
- }
-
- global_ops.func = func;
- global_ops.private = private;
-}
-
static void ftrace_sync(struct work_struct *work)
{
/*
@@ -301,8 +246,6 @@
{
ftrace_func_t func;
- update_global_ops();
-
/*
* If we are at the end of the list and this ops is
* recursion safe and not dynamic and the arch supports passing ops,
@@ -314,10 +257,7 @@
(ftrace_ops_list->flags & FTRACE_OPS_FL_RECURSION_SAFE) &&
!FTRACE_FORCE_LIST_FUNC)) {
/* Set the ftrace_ops that the arch callback uses */
- if (ftrace_ops_list == &global_ops)
- set_function_trace_op = ftrace_global_list;
- else
- set_function_trace_op = ftrace_ops_list;
+ set_function_trace_op = ftrace_ops_list;
func = ftrace_ops_list->func;
} else {
/* Just use the default ftrace_ops */
@@ -373,6 +313,11 @@
ftrace_trace_function = func;
}
+int using_ftrace_ops_list_func(void)
+{
+ return ftrace_trace_function == ftrace_ops_list_func;
+}
+
static void add_ftrace_ops(struct ftrace_ops **list, struct ftrace_ops *ops)
{
ops->next = *list;
@@ -434,16 +379,9 @@
if (ops->flags & FTRACE_OPS_FL_DELETED)
return -EINVAL;
- if (FTRACE_WARN_ON(ops == &global_ops))
- return -EINVAL;
-
if (WARN_ON(ops->flags & FTRACE_OPS_FL_ENABLED))
return -EBUSY;
- /* We don't support both control and global flags set. */
- if ((ops->flags & FL_GLOBAL_CONTROL_MASK) == FL_GLOBAL_CONTROL_MASK)
- return -EINVAL;
-
#ifndef CONFIG_DYNAMIC_FTRACE_WITH_REGS
/*
* If the ftrace_ops specifies SAVE_REGS, then it only can be used
@@ -461,10 +399,7 @@
if (!core_kernel_data((unsigned long)ops))
ops->flags |= FTRACE_OPS_FL_DYNAMIC;
- if (ops->flags & FTRACE_OPS_FL_GLOBAL) {
- add_ftrace_list_ops(&ftrace_global_list, &global_ops, ops);
- ops->flags |= FTRACE_OPS_FL_ENABLED;
- } else if (ops->flags & FTRACE_OPS_FL_CONTROL) {
+ if (ops->flags & FTRACE_OPS_FL_CONTROL) {
if (control_ops_alloc(ops))
return -ENOMEM;
add_ftrace_list_ops(&ftrace_control_list, &control_ops, ops);
@@ -484,15 +419,7 @@
if (WARN_ON(!(ops->flags & FTRACE_OPS_FL_ENABLED)))
return -EBUSY;
- if (FTRACE_WARN_ON(ops == &global_ops))
- return -EINVAL;
-
- if (ops->flags & FTRACE_OPS_FL_GLOBAL) {
- ret = remove_ftrace_list_ops(&ftrace_global_list,
- &global_ops, ops);
- if (!ret)
- ops->flags &= ~FTRACE_OPS_FL_ENABLED;
- } else if (ops->flags & FTRACE_OPS_FL_CONTROL) {
+ if (ops->flags & FTRACE_OPS_FL_CONTROL) {
ret = remove_ftrace_list_ops(&ftrace_control_list,
&control_ops, ops);
} else
@@ -895,7 +822,7 @@
local_irq_save(flags);
- stat = &__get_cpu_var(ftrace_profile_stats);
+ stat = this_cpu_ptr(&ftrace_profile_stats);
if (!stat->hash || !ftrace_profile_enabled)
goto out;
@@ -926,7 +853,7 @@
unsigned long flags;
local_irq_save(flags);
- stat = &__get_cpu_var(ftrace_profile_stats);
+ stat = this_cpu_ptr(&ftrace_profile_stats);
if (!stat->hash || !ftrace_profile_enabled)
goto out;
@@ -1178,7 +1105,7 @@
static struct ftrace_page *ftrace_pages_start;
static struct ftrace_page *ftrace_pages;
-static bool ftrace_hash_empty(struct ftrace_hash *hash)
+static bool __always_inline ftrace_hash_empty(struct ftrace_hash *hash)
{
return !hash || !hash->count;
}
@@ -1625,7 +1552,14 @@
in_other_hash = !!ftrace_lookup_ip(other_hash, rec->ip);
/*
+ * If filter_hash is set, we want to match all functions
+ * that are in the hash but not in the other hash.
*
+ * If filter_hash is not set, then we are decrementing.
+ * That means we match anything that is in the hash
+ * and also in the other_hash. That is, we need to turn
+ * off functions in the other hash because they are disabled
+ * by this hash.
*/
if (filter_hash && in_hash && !in_other_hash)
match = 1;
@@ -1767,19 +1701,15 @@
/*
* If this record is being updated from a nop, then
* return UPDATE_MAKE_CALL.
- * Otherwise, if the EN flag is set, then return
- * UPDATE_MODIFY_CALL_REGS to tell the caller to convert
- * from the non-save regs, to a save regs function.
* Otherwise,
* return UPDATE_MODIFY_CALL to tell the caller to convert
- * from the save regs, to a non-save regs function.
+ * from the save regs, to a non-save regs function or
+ * vice versa.
*/
if (flag & FTRACE_FL_ENABLED)
return FTRACE_UPDATE_MAKE_CALL;
- else if (rec->flags & FTRACE_FL_REGS_EN)
- return FTRACE_UPDATE_MODIFY_CALL_REGS;
- else
- return FTRACE_UPDATE_MODIFY_CALL;
+
+ return FTRACE_UPDATE_MODIFY_CALL;
}
if (update) {
@@ -1821,6 +1751,42 @@
return ftrace_check_record(rec, enable, 0);
}
+/**
+ * ftrace_get_addr_new - Get the call address to set to
+ * @rec: The ftrace record descriptor
+ *
+ * If the record has the FTRACE_FL_REGS set, that means that it
+ * wants to convert to a callback that saves all regs. If FTRACE_FL_REGS
+ * is not not set, then it wants to convert to the normal callback.
+ *
+ * Returns the address of the trampoline to set to
+ */
+unsigned long ftrace_get_addr_new(struct dyn_ftrace *rec)
+{
+ if (rec->flags & FTRACE_FL_REGS)
+ return (unsigned long)FTRACE_REGS_ADDR;
+ else
+ return (unsigned long)FTRACE_ADDR;
+}
+
+/**
+ * ftrace_get_addr_curr - Get the call address that is already there
+ * @rec: The ftrace record descriptor
+ *
+ * The FTRACE_FL_REGS_EN is set when the record already points to
+ * a function that saves all the regs. Basically the '_EN' version
+ * represents the current state of the function.
+ *
+ * Returns the address of the trampoline that is currently being called
+ */
+unsigned long ftrace_get_addr_curr(struct dyn_ftrace *rec)
+{
+ if (rec->flags & FTRACE_FL_REGS_EN)
+ return (unsigned long)FTRACE_REGS_ADDR;
+ else
+ return (unsigned long)FTRACE_ADDR;
+}
+
static int
__ftrace_replace_code(struct dyn_ftrace *rec, int enable)
{
@@ -1828,12 +1794,12 @@
unsigned long ftrace_addr;
int ret;
- ret = ftrace_update_record(rec, enable);
+ ftrace_addr = ftrace_get_addr_new(rec);
- if (rec->flags & FTRACE_FL_REGS)
- ftrace_addr = (unsigned long)FTRACE_REGS_ADDR;
- else
- ftrace_addr = (unsigned long)FTRACE_ADDR;
+ /* This needs to be done before we call ftrace_update_record */
+ ftrace_old_addr = ftrace_get_addr_curr(rec);
+
+ ret = ftrace_update_record(rec, enable);
switch (ret) {
case FTRACE_UPDATE_IGNORE:
@@ -1845,13 +1811,7 @@
case FTRACE_UPDATE_MAKE_NOP:
return ftrace_make_nop(NULL, rec, ftrace_addr);
- case FTRACE_UPDATE_MODIFY_CALL_REGS:
case FTRACE_UPDATE_MODIFY_CALL:
- if (rec->flags & FTRACE_FL_REGS)
- ftrace_old_addr = (unsigned long)FTRACE_ADDR;
- else
- ftrace_old_addr = (unsigned long)FTRACE_REGS_ADDR;
-
return ftrace_modify_call(rec, ftrace_old_addr, ftrace_addr);
}
@@ -2115,7 +2075,6 @@
static int ftrace_startup(struct ftrace_ops *ops, int command)
{
- bool hash_enable = true;
int ret;
if (unlikely(ftrace_disabled))
@@ -2128,18 +2087,9 @@
ftrace_start_up++;
command |= FTRACE_UPDATE_CALLS;
- /* ops marked global share the filter hashes */
- if (ops->flags & FTRACE_OPS_FL_GLOBAL) {
- ops = &global_ops;
- /* Don't update hash if global is already set */
- if (global_start_up)
- hash_enable = false;
- global_start_up++;
- }
-
ops->flags |= FTRACE_OPS_FL_ENABLED;
- if (hash_enable)
- ftrace_hash_rec_enable(ops, 1);
+
+ ftrace_hash_rec_enable(ops, 1);
ftrace_startup_enable(command);
@@ -2148,7 +2098,6 @@
static int ftrace_shutdown(struct ftrace_ops *ops, int command)
{
- bool hash_disable = true;
int ret;
if (unlikely(ftrace_disabled))
@@ -2166,21 +2115,9 @@
*/
WARN_ON_ONCE(ftrace_start_up < 0);
- if (ops->flags & FTRACE_OPS_FL_GLOBAL) {
- ops = &global_ops;
- global_start_up--;
- WARN_ON_ONCE(global_start_up < 0);
- /* Don't update hash if global still has users */
- if (global_start_up) {
- WARN_ON_ONCE(!ftrace_start_up);
- hash_disable = false;
- }
- }
+ ftrace_hash_rec_disable(ops, 1);
- if (hash_disable)
- ftrace_hash_rec_disable(ops, 1);
-
- if (ops != &global_ops || !global_start_up)
+ if (!global_start_up)
ops->flags &= ~FTRACE_OPS_FL_ENABLED;
command |= FTRACE_UPDATE_CALLS;
@@ -3524,10 +3461,6 @@
struct ftrace_hash *hash;
int ret;
- /* All global ops uses the global ops filters */
- if (ops->flags & FTRACE_OPS_FL_GLOBAL)
- ops = &global_ops;
-
if (unlikely(ftrace_disabled))
return -ENODEV;
@@ -3639,8 +3572,7 @@
}
EXPORT_SYMBOL_GPL(ftrace_set_notrace);
/**
- * ftrace_set_filter - set a function to filter on in ftrace
- * @ops - the ops to set the filter with
+ * ftrace_set_global_filter - set a function to filter on with global tracers
* @buf - the string that holds the function filter text.
* @len - the length of the string.
* @reset - non zero to reset all filters before applying this filter.
@@ -3655,8 +3587,7 @@
EXPORT_SYMBOL_GPL(ftrace_set_global_filter);
/**
- * ftrace_set_notrace - set a function to not trace in ftrace
- * @ops - the ops to set the notrace filter with
+ * ftrace_set_global_notrace - set a function to not trace with global tracers
* @buf - the string that holds the function notrace text.
* @len - the length of the string.
* @reset - non zero to reset all filters before applying this filter.
@@ -4443,6 +4374,34 @@
#endif /* CONFIG_DYNAMIC_FTRACE */
+__init void ftrace_init_global_array_ops(struct trace_array *tr)
+{
+ tr->ops = &global_ops;
+ tr->ops->private = tr;
+}
+
+void ftrace_init_array_ops(struct trace_array *tr, ftrace_func_t func)
+{
+ /* If we filter on pids, update to use the pid function */
+ if (tr->flags & TRACE_ARRAY_FL_GLOBAL) {
+ if (WARN_ON(tr->ops->func != ftrace_stub))
+ printk("ftrace ops had %pS for function\n",
+ tr->ops->func);
+ /* Only the top level instance does pid tracing */
+ if (!list_empty(&ftrace_pids)) {
+ set_ftrace_pid_function(func);
+ func = ftrace_pid_func;
+ }
+ }
+ tr->ops->func = func;
+ tr->ops->private = tr;
+}
+
+void ftrace_reset_array_ops(struct trace_array *tr)
+{
+ tr->ops->func = ftrace_stub;
+}
+
static void
ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip,
struct ftrace_ops *op, struct pt_regs *regs)
@@ -4501,9 +4460,16 @@
*/
preempt_disable_notrace();
do_for_each_ftrace_op(op, ftrace_ops_list) {
- if (ftrace_ops_test(op, ip, regs))
+ if (ftrace_ops_test(op, ip, regs)) {
+ if (WARN_ON(!op->func)) {
+ function_trace_stop = 1;
+ printk("op=%p %pS\n", op, op);
+ goto out;
+ }
op->func(ip, parent_ip, op, regs);
+ }
} while_for_each_ftrace_op(op);
+out:
preempt_enable_notrace();
trace_clear_recursion(bit);
}
@@ -4908,7 +4874,6 @@
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
static int ftrace_graph_active;
-static struct notifier_block ftrace_suspend_notifier;
int ftrace_graph_entry_stub(struct ftrace_graph_ent *trace)
{
@@ -5054,13 +5019,6 @@
return NOTIFY_DONE;
}
-/* Just a place holder for function graph */
-static struct ftrace_ops fgraph_ops __read_mostly = {
- .func = ftrace_stub,
- .flags = FTRACE_OPS_FL_STUB | FTRACE_OPS_FL_GLOBAL |
- FTRACE_OPS_FL_RECURSION_SAFE,
-};
-
static int ftrace_graph_entry_test(struct ftrace_graph_ent *trace)
{
if (!ftrace_ops_test(&global_ops, trace->func, NULL))
@@ -5085,6 +5043,10 @@
ftrace_graph_entry = ftrace_graph_entry_test;
}
+static struct notifier_block ftrace_suspend_notifier = {
+ .notifier_call = ftrace_suspend_notifier_call,
+};
+
int register_ftrace_graph(trace_func_graph_ret_t retfunc,
trace_func_graph_ent_t entryfunc)
{
@@ -5098,7 +5060,6 @@
goto out;
}
- ftrace_suspend_notifier.notifier_call = ftrace_suspend_notifier_call;
register_pm_notifier(&ftrace_suspend_notifier);
ftrace_graph_active++;
@@ -5120,7 +5081,10 @@
ftrace_graph_entry = ftrace_graph_entry_test;
update_function_graph_func();
- ret = ftrace_startup(&fgraph_ops, FTRACE_START_FUNC_RET);
+ /* Function graph doesn't use the .func field of global_ops */
+ global_ops.flags |= FTRACE_OPS_FL_STUB;
+
+ ret = ftrace_startup(&global_ops, FTRACE_START_FUNC_RET);
out:
mutex_unlock(&ftrace_lock);
@@ -5138,7 +5102,8 @@
ftrace_graph_return = (trace_func_graph_ret_t)ftrace_stub;
ftrace_graph_entry = ftrace_graph_entry_stub;
__ftrace_graph_entry = ftrace_graph_entry_stub;
- ftrace_shutdown(&fgraph_ops, FTRACE_STOP_FUNC_RET);
+ ftrace_shutdown(&global_ops, FTRACE_STOP_FUNC_RET);
+ global_ops.flags &= ~FTRACE_OPS_FL_STUB;
unregister_pm_notifier(&ftrace_suspend_notifier);
unregister_trace_sched_switch(ftrace_graph_probe_sched_switch, NULL);
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 737b0efa..16f7038 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -275,7 +275,7 @@
}
EXPORT_SYMBOL_GPL(call_filter_check_discard);
-cycle_t buffer_ftrace_now(struct trace_buffer *buf, int cpu)
+static cycle_t buffer_ftrace_now(struct trace_buffer *buf, int cpu)
{
u64 ts;
@@ -599,7 +599,7 @@
return 0;
}
-void free_snapshot(struct trace_array *tr)
+static void free_snapshot(struct trace_array *tr)
{
/*
* We don't free the ring buffer. instead, resize it because
@@ -963,27 +963,9 @@
return cnt;
}
-/*
- * ftrace_max_lock is used to protect the swapping of buffers
- * when taking a max snapshot. The buffers themselves are
- * protected by per_cpu spinlocks. But the action of the swap
- * needs its own lock.
- *
- * This is defined as a arch_spinlock_t in order to help
- * with performance when lockdep debugging is enabled.
- *
- * It is also used in other places outside the update_max_tr
- * so it needs to be defined outside of the
- * CONFIG_TRACER_MAX_TRACE.
- */
-static arch_spinlock_t ftrace_max_lock =
- (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED;
-
unsigned long __read_mostly tracing_thresh;
#ifdef CONFIG_TRACER_MAX_TRACE
-unsigned long __read_mostly tracing_max_latency;
-
/*
* Copy the new maximum trace into the separate maximum-trace
* structure. (this way the maximum trace is permanently saved,
@@ -1000,7 +982,7 @@
max_buf->cpu = cpu;
max_buf->time_start = data->preempt_timestamp;
- max_data->saved_latency = tracing_max_latency;
+ max_data->saved_latency = tr->max_latency;
max_data->critical_start = data->critical_start;
max_data->critical_end = data->critical_end;
@@ -1048,14 +1030,14 @@
return;
}
- arch_spin_lock(&ftrace_max_lock);
+ arch_spin_lock(&tr->max_lock);
buf = tr->trace_buffer.buffer;
tr->trace_buffer.buffer = tr->max_buffer.buffer;
tr->max_buffer.buffer = buf;
__update_max_tr(tr, tsk, cpu);
- arch_spin_unlock(&ftrace_max_lock);
+ arch_spin_unlock(&tr->max_lock);
}
/**
@@ -1081,7 +1063,7 @@
return;
}
- arch_spin_lock(&ftrace_max_lock);
+ arch_spin_lock(&tr->max_lock);
ret = ring_buffer_swap_cpu(tr->max_buffer.buffer, tr->trace_buffer.buffer, cpu);
@@ -1099,11 +1081,11 @@
WARN_ON_ONCE(ret && ret != -EAGAIN && ret != -EBUSY);
__update_max_tr(tr, tsk, cpu);
- arch_spin_unlock(&ftrace_max_lock);
+ arch_spin_unlock(&tr->max_lock);
}
#endif /* CONFIG_TRACER_MAX_TRACE */
-static void default_wait_pipe(struct trace_iterator *iter)
+static void wait_on_pipe(struct trace_iterator *iter)
{
/* Iterators are static, they should be filled or empty */
if (trace_buffer_iter(iter, iter->cpu_file))
@@ -1220,8 +1202,6 @@
else
if (!type->flags->opts)
type->flags->opts = dummy_tracer_opt;
- if (!type->wait_pipe)
- type->wait_pipe = default_wait_pipe;
ret = run_tracer_selftest(type);
if (ret < 0)
@@ -1305,22 +1285,71 @@
}
}
-#define SAVED_CMDLINES 128
+#define SAVED_CMDLINES_DEFAULT 128
#define NO_CMDLINE_MAP UINT_MAX
-static unsigned map_pid_to_cmdline[PID_MAX_DEFAULT+1];
-static unsigned map_cmdline_to_pid[SAVED_CMDLINES];
-static char saved_cmdlines[SAVED_CMDLINES][TASK_COMM_LEN];
-static int cmdline_idx;
static arch_spinlock_t trace_cmdline_lock = __ARCH_SPIN_LOCK_UNLOCKED;
+struct saved_cmdlines_buffer {
+ unsigned map_pid_to_cmdline[PID_MAX_DEFAULT+1];
+ unsigned *map_cmdline_to_pid;
+ unsigned cmdline_num;
+ int cmdline_idx;
+ char *saved_cmdlines;
+};
+static struct saved_cmdlines_buffer *savedcmd;
/* temporary disable recording */
static atomic_t trace_record_cmdline_disabled __read_mostly;
-static void trace_init_cmdlines(void)
+static inline char *get_saved_cmdlines(int idx)
{
- memset(&map_pid_to_cmdline, NO_CMDLINE_MAP, sizeof(map_pid_to_cmdline));
- memset(&map_cmdline_to_pid, NO_CMDLINE_MAP, sizeof(map_cmdline_to_pid));
- cmdline_idx = 0;
+ return &savedcmd->saved_cmdlines[idx * TASK_COMM_LEN];
+}
+
+static inline void set_cmdline(int idx, const char *cmdline)
+{
+ memcpy(get_saved_cmdlines(idx), cmdline, TASK_COMM_LEN);
+}
+
+static int allocate_cmdlines_buffer(unsigned int val,
+ struct saved_cmdlines_buffer *s)
+{
+ s->map_cmdline_to_pid = kmalloc(val * sizeof(*s->map_cmdline_to_pid),
+ GFP_KERNEL);
+ if (!s->map_cmdline_to_pid)
+ return -ENOMEM;
+
+ s->saved_cmdlines = kmalloc(val * TASK_COMM_LEN, GFP_KERNEL);
+ if (!s->saved_cmdlines) {
+ kfree(s->map_cmdline_to_pid);
+ return -ENOMEM;
+ }
+
+ s->cmdline_idx = 0;
+ s->cmdline_num = val;
+ memset(&s->map_pid_to_cmdline, NO_CMDLINE_MAP,
+ sizeof(s->map_pid_to_cmdline));
+ memset(s->map_cmdline_to_pid, NO_CMDLINE_MAP,
+ val * sizeof(*s->map_cmdline_to_pid));
+
+ return 0;
+}
+
+static int trace_create_savedcmd(void)
+{
+ int ret;
+
+ savedcmd = kmalloc(sizeof(struct saved_cmdlines_buffer), GFP_KERNEL);
+ if (!savedcmd)
+ return -ENOMEM;
+
+ ret = allocate_cmdlines_buffer(SAVED_CMDLINES_DEFAULT, savedcmd);
+ if (ret < 0) {
+ kfree(savedcmd);
+ savedcmd = NULL;
+ return -ENOMEM;
+ }
+
+ return 0;
}
int is_tracing_stopped(void)
@@ -1353,7 +1382,7 @@
}
/* Prevent the buffers from switching */
- arch_spin_lock(&ftrace_max_lock);
+ arch_spin_lock(&global_trace.max_lock);
buffer = global_trace.trace_buffer.buffer;
if (buffer)
@@ -1365,7 +1394,7 @@
ring_buffer_record_enable(buffer);
#endif
- arch_spin_unlock(&ftrace_max_lock);
+ arch_spin_unlock(&global_trace.max_lock);
ftrace_start();
out:
@@ -1420,7 +1449,7 @@
goto out;
/* Prevent the buffers from switching */
- arch_spin_lock(&ftrace_max_lock);
+ arch_spin_lock(&global_trace.max_lock);
buffer = global_trace.trace_buffer.buffer;
if (buffer)
@@ -1432,7 +1461,7 @@
ring_buffer_record_disable(buffer);
#endif
- arch_spin_unlock(&ftrace_max_lock);
+ arch_spin_unlock(&global_trace.max_lock);
out:
raw_spin_unlock_irqrestore(&global_trace.start_lock, flags);
@@ -1461,12 +1490,12 @@
void trace_stop_cmdline_recording(void);
-static void trace_save_cmdline(struct task_struct *tsk)
+static int trace_save_cmdline(struct task_struct *tsk)
{
unsigned pid, idx;
if (!tsk->pid || unlikely(tsk->pid > PID_MAX_DEFAULT))
- return;
+ return 0;
/*
* It's not the end of the world if we don't get
@@ -1475,11 +1504,11 @@
* so if we miss here, then better luck next time.
*/
if (!arch_spin_trylock(&trace_cmdline_lock))
- return;
+ return 0;
- idx = map_pid_to_cmdline[tsk->pid];
+ idx = savedcmd->map_pid_to_cmdline[tsk->pid];
if (idx == NO_CMDLINE_MAP) {
- idx = (cmdline_idx + 1) % SAVED_CMDLINES;
+ idx = (savedcmd->cmdline_idx + 1) % savedcmd->cmdline_num;
/*
* Check whether the cmdline buffer at idx has a pid
@@ -1487,22 +1516,24 @@
* need to clear the map_pid_to_cmdline. Otherwise we
* would read the new comm for the old pid.
*/
- pid = map_cmdline_to_pid[idx];
+ pid = savedcmd->map_cmdline_to_pid[idx];
if (pid != NO_CMDLINE_MAP)
- map_pid_to_cmdline[pid] = NO_CMDLINE_MAP;
+ savedcmd->map_pid_to_cmdline[pid] = NO_CMDLINE_MAP;
- map_cmdline_to_pid[idx] = tsk->pid;
- map_pid_to_cmdline[tsk->pid] = idx;
+ savedcmd->map_cmdline_to_pid[idx] = tsk->pid;
+ savedcmd->map_pid_to_cmdline[tsk->pid] = idx;
- cmdline_idx = idx;
+ savedcmd->cmdline_idx = idx;
}
- memcpy(&saved_cmdlines[idx], tsk->comm, TASK_COMM_LEN);
+ set_cmdline(idx, tsk->comm);
arch_spin_unlock(&trace_cmdline_lock);
+
+ return 1;
}
-void trace_find_cmdline(int pid, char comm[])
+static void __trace_find_cmdline(int pid, char comm[])
{
unsigned map;
@@ -1521,13 +1552,19 @@
return;
}
- preempt_disable();
- arch_spin_lock(&trace_cmdline_lock);
- map = map_pid_to_cmdline[pid];
+ map = savedcmd->map_pid_to_cmdline[pid];
if (map != NO_CMDLINE_MAP)
- strcpy(comm, saved_cmdlines[map]);
+ strcpy(comm, get_saved_cmdlines(map));
else
strcpy(comm, "<...>");
+}
+
+void trace_find_cmdline(int pid, char comm[])
+{
+ preempt_disable();
+ arch_spin_lock(&trace_cmdline_lock);
+
+ __trace_find_cmdline(pid, comm);
arch_spin_unlock(&trace_cmdline_lock);
preempt_enable();
@@ -1541,9 +1578,8 @@
if (!__this_cpu_read(trace_cmdline_save))
return;
- __this_cpu_write(trace_cmdline_save, false);
-
- trace_save_cmdline(tsk);
+ if (trace_save_cmdline(tsk))
+ __this_cpu_write(trace_cmdline_save, false);
}
void
@@ -1746,7 +1782,7 @@
*/
barrier();
if (use_stack == 1) {
- trace.entries = &__get_cpu_var(ftrace_stack).calls[0];
+ trace.entries = this_cpu_ptr(ftrace_stack.calls);
trace.max_entries = FTRACE_STACK_MAX_ENTRIES;
if (regs)
@@ -1995,7 +2031,21 @@
if (alloc_percpu_trace_buffer())
return;
- pr_info("ftrace: Allocated trace_printk buffers\n");
+ /* trace_printk() is for debug use only. Don't use it in production. */
+
+ pr_warning("\n**********************************************************\n");
+ pr_warning("** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE **\n");
+ pr_warning("** **\n");
+ pr_warning("** trace_printk() being used. Allocating extra memory. **\n");
+ pr_warning("** **\n");
+ pr_warning("** This means that this is a DEBUG kernel and it is **\n");
+ pr_warning("** unsafe for produciton use. **\n");
+ pr_warning("** **\n");
+ pr_warning("** If you see this message and you are not debugging **\n");
+ pr_warning("** the kernel, report this immediately to your vendor! **\n");
+ pr_warning("** **\n");
+ pr_warning("** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE **\n");
+ pr_warning("**********************************************************\n");
/* Expand the buffers to set size */
tracing_update_buffers();
@@ -3333,7 +3383,7 @@
mutex_lock(&tracing_cpumask_update_lock);
local_irq_disable();
- arch_spin_lock(&ftrace_max_lock);
+ arch_spin_lock(&tr->max_lock);
for_each_tracing_cpu(cpu) {
/*
* Increase/decrease the disabled counter if we are
@@ -3350,7 +3400,7 @@
ring_buffer_record_enable_cpu(tr->trace_buffer.buffer, cpu);
}
}
- arch_spin_unlock(&ftrace_max_lock);
+ arch_spin_unlock(&tr->max_lock);
local_irq_enable();
cpumask_copy(tr->tracing_cpumask, tracing_cpumask_new);
@@ -3592,6 +3642,7 @@
" trace_options\t\t- Set format or modify how tracing happens\n"
"\t\t\t Disable an option by adding a suffix 'no' to the\n"
"\t\t\t option name\n"
+ " saved_cmdlines_size\t- echo command number in here to store comm-pid list\n"
#ifdef CONFIG_DYNAMIC_FTRACE
"\n available_filter_functions - list of functions that can be filtered on\n"
" set_ftrace_filter\t- echo function name in here to only trace these\n"
@@ -3705,55 +3756,153 @@
.llseek = generic_file_llseek,
};
-static ssize_t
-tracing_saved_cmdlines_read(struct file *file, char __user *ubuf,
- size_t cnt, loff_t *ppos)
+static void *saved_cmdlines_next(struct seq_file *m, void *v, loff_t *pos)
{
- char *buf_comm;
- char *file_buf;
- char *buf;
- int len = 0;
- int pid;
- int i;
+ unsigned int *ptr = v;
- file_buf = kmalloc(SAVED_CMDLINES*(16+TASK_COMM_LEN), GFP_KERNEL);
- if (!file_buf)
- return -ENOMEM;
+ if (*pos || m->count)
+ ptr++;
- buf_comm = kmalloc(TASK_COMM_LEN, GFP_KERNEL);
- if (!buf_comm) {
- kfree(file_buf);
- return -ENOMEM;
- }
+ (*pos)++;
- buf = file_buf;
-
- for (i = 0; i < SAVED_CMDLINES; i++) {
- int r;
-
- pid = map_cmdline_to_pid[i];
- if (pid == -1 || pid == NO_CMDLINE_MAP)
+ for (; ptr < &savedcmd->map_cmdline_to_pid[savedcmd->cmdline_num];
+ ptr++) {
+ if (*ptr == -1 || *ptr == NO_CMDLINE_MAP)
continue;
- trace_find_cmdline(pid, buf_comm);
- r = sprintf(buf, "%d %s\n", pid, buf_comm);
- buf += r;
- len += r;
+ return ptr;
}
- len = simple_read_from_buffer(ubuf, cnt, ppos,
- file_buf, len);
+ return NULL;
+}
- kfree(file_buf);
- kfree(buf_comm);
+static void *saved_cmdlines_start(struct seq_file *m, loff_t *pos)
+{
+ void *v;
+ loff_t l = 0;
- return len;
+ preempt_disable();
+ arch_spin_lock(&trace_cmdline_lock);
+
+ v = &savedcmd->map_cmdline_to_pid[0];
+ while (l <= *pos) {
+ v = saved_cmdlines_next(m, v, &l);
+ if (!v)
+ return NULL;
+ }
+
+ return v;
+}
+
+static void saved_cmdlines_stop(struct seq_file *m, void *v)
+{
+ arch_spin_unlock(&trace_cmdline_lock);
+ preempt_enable();
+}
+
+static int saved_cmdlines_show(struct seq_file *m, void *v)
+{
+ char buf[TASK_COMM_LEN];
+ unsigned int *pid = v;
+
+ __trace_find_cmdline(*pid, buf);
+ seq_printf(m, "%d %s\n", *pid, buf);
+ return 0;
+}
+
+static const struct seq_operations tracing_saved_cmdlines_seq_ops = {
+ .start = saved_cmdlines_start,
+ .next = saved_cmdlines_next,
+ .stop = saved_cmdlines_stop,
+ .show = saved_cmdlines_show,
+};
+
+static int tracing_saved_cmdlines_open(struct inode *inode, struct file *filp)
+{
+ if (tracing_disabled)
+ return -ENODEV;
+
+ return seq_open(filp, &tracing_saved_cmdlines_seq_ops);
}
static const struct file_operations tracing_saved_cmdlines_fops = {
- .open = tracing_open_generic,
- .read = tracing_saved_cmdlines_read,
- .llseek = generic_file_llseek,
+ .open = tracing_saved_cmdlines_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = seq_release,
+};
+
+static ssize_t
+tracing_saved_cmdlines_size_read(struct file *filp, char __user *ubuf,
+ size_t cnt, loff_t *ppos)
+{
+ char buf[64];
+ int r;
+
+ arch_spin_lock(&trace_cmdline_lock);
+ r = sprintf(buf, "%u\n", savedcmd->cmdline_num);
+ arch_spin_unlock(&trace_cmdline_lock);
+
+ return simple_read_from_buffer(ubuf, cnt, ppos, buf, r);
+}
+
+static void free_saved_cmdlines_buffer(struct saved_cmdlines_buffer *s)
+{
+ kfree(s->saved_cmdlines);
+ kfree(s->map_cmdline_to_pid);
+ kfree(s);
+}
+
+static int tracing_resize_saved_cmdlines(unsigned int val)
+{
+ struct saved_cmdlines_buffer *s, *savedcmd_temp;
+
+ s = kmalloc(sizeof(struct saved_cmdlines_buffer), GFP_KERNEL);
+ if (!s)
+ return -ENOMEM;
+
+ if (allocate_cmdlines_buffer(val, s) < 0) {
+ kfree(s);
+ return -ENOMEM;
+ }
+
+ arch_spin_lock(&trace_cmdline_lock);
+ savedcmd_temp = savedcmd;
+ savedcmd = s;
+ arch_spin_unlock(&trace_cmdline_lock);
+ free_saved_cmdlines_buffer(savedcmd_temp);
+
+ return 0;
+}
+
+static ssize_t
+tracing_saved_cmdlines_size_write(struct file *filp, const char __user *ubuf,
+ size_t cnt, loff_t *ppos)
+{
+ unsigned long val;
+ int ret;
+
+ ret = kstrtoul_from_user(ubuf, cnt, 10, &val);
+ if (ret)
+ return ret;
+
+ /* must have at least 1 entry or less than PID_MAX_DEFAULT */
+ if (!val || val > PID_MAX_DEFAULT)
+ return -EINVAL;
+
+ ret = tracing_resize_saved_cmdlines((unsigned int)val);
+ if (ret < 0)
+ return ret;
+
+ *ppos += cnt;
+
+ return cnt;
+}
+
+static const struct file_operations tracing_saved_cmdlines_size_fops = {
+ .open = tracing_open_generic,
+ .read = tracing_saved_cmdlines_size_read,
+ .write = tracing_saved_cmdlines_size_write,
};
static ssize_t
@@ -4225,25 +4374,6 @@
return trace_poll(iter, filp, poll_table);
}
-/*
- * This is a make-shift waitqueue.
- * A tracer might use this callback on some rare cases:
- *
- * 1) the current tracer might hold the runqueue lock when it wakes up
- * a reader, hence a deadlock (sched, function, and function graph tracers)
- * 2) the function tracers, trace all functions, we don't want
- * the overhead of calling wake_up and friends
- * (and tracing them too)
- *
- * Anyway, this is really very primitive wakeup.
- */
-void poll_wait_pipe(struct trace_iterator *iter)
-{
- set_current_state(TASK_INTERRUPTIBLE);
- /* sleep for 100 msecs, and try again. */
- schedule_timeout(HZ / 10);
-}
-
/* Must be called with trace_types_lock mutex held. */
static int tracing_wait_pipe(struct file *filp)
{
@@ -4255,15 +4385,6 @@
return -EAGAIN;
}
- mutex_unlock(&iter->mutex);
-
- iter->trace->wait_pipe(iter);
-
- mutex_lock(&iter->mutex);
-
- if (signal_pending(current))
- return -EINTR;
-
/*
* We block until we read something and tracing is disabled.
* We still block if tracing is disabled, but we have never
@@ -4275,6 +4396,15 @@
*/
if (!tracing_is_on() && iter->pos)
break;
+
+ mutex_unlock(&iter->mutex);
+
+ wait_on_pipe(iter);
+
+ mutex_lock(&iter->mutex);
+
+ if (signal_pending(current))
+ return -EINTR;
}
return 1;
@@ -5197,7 +5327,7 @@
goto out_unlock;
}
mutex_unlock(&trace_types_lock);
- iter->trace->wait_pipe(iter);
+ wait_on_pipe(iter);
mutex_lock(&trace_types_lock);
if (signal_pending(current)) {
size = -EINTR;
@@ -5408,7 +5538,7 @@
goto out;
}
mutex_unlock(&trace_types_lock);
- iter->trace->wait_pipe(iter);
+ wait_on_pipe(iter);
mutex_lock(&trace_types_lock);
if (signal_pending(current)) {
ret = -EINTR;
@@ -6102,6 +6232,25 @@
return 0;
}
+static void free_trace_buffers(struct trace_array *tr)
+{
+ if (!tr)
+ return;
+
+ if (tr->trace_buffer.buffer) {
+ ring_buffer_free(tr->trace_buffer.buffer);
+ tr->trace_buffer.buffer = NULL;
+ free_percpu(tr->trace_buffer.data);
+ }
+
+#ifdef CONFIG_TRACER_MAX_TRACE
+ if (tr->max_buffer.buffer) {
+ ring_buffer_free(tr->max_buffer.buffer);
+ tr->max_buffer.buffer = NULL;
+ }
+#endif
+}
+
static int new_instance_create(const char *name)
{
struct trace_array *tr;
@@ -6131,6 +6280,8 @@
raw_spin_lock_init(&tr->start_lock);
+ tr->max_lock = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED;
+
tr->current_trace = &nop_trace;
INIT_LIST_HEAD(&tr->systems);
@@ -6158,8 +6309,7 @@
return 0;
out_free_tr:
- if (tr->trace_buffer.buffer)
- ring_buffer_free(tr->trace_buffer.buffer);
+ free_trace_buffers(tr);
free_cpumask_var(tr->tracing_cpumask);
kfree(tr->name);
kfree(tr);
@@ -6199,8 +6349,7 @@
event_trace_del_tracer(tr);
ftrace_destroy_function_files(tr);
debugfs_remove_recursive(tr->dir);
- free_percpu(tr->trace_buffer.data);
- ring_buffer_free(tr->trace_buffer.buffer);
+ free_trace_buffers(tr);
kfree(tr->name);
kfree(tr);
@@ -6328,6 +6477,11 @@
trace_create_file("tracing_on", 0644, d_tracer,
tr, &rb_simple_fops);
+#ifdef CONFIG_TRACER_MAX_TRACE
+ trace_create_file("tracing_max_latency", 0644, d_tracer,
+ &tr->max_latency, &tracing_max_lat_fops);
+#endif
+
if (ftrace_create_function_files(tr, d_tracer))
WARN(1, "Could not allocate function filter files");
@@ -6353,11 +6507,6 @@
init_tracer_debugfs(&global_trace, d_tracer);
-#ifdef CONFIG_TRACER_MAX_TRACE
- trace_create_file("tracing_max_latency", 0644, d_tracer,
- &tracing_max_latency, &tracing_max_lat_fops);
-#endif
-
trace_create_file("tracing_thresh", 0644, d_tracer,
&tracing_thresh, &tracing_max_lat_fops);
@@ -6367,6 +6516,9 @@
trace_create_file("saved_cmdlines", 0444, d_tracer,
NULL, &tracing_saved_cmdlines_fops);
+ trace_create_file("saved_cmdlines_size", 0644, d_tracer,
+ NULL, &tracing_saved_cmdlines_size_fops);
+
#ifdef CONFIG_DYNAMIC_FTRACE
trace_create_file("dyn_ftrace_total_info", 0444, d_tracer,
&ftrace_update_tot_cnt, &tracing_dyn_info_fops);
@@ -6603,18 +6755,19 @@
if (!temp_buffer)
goto out_free_cpumask;
+ if (trace_create_savedcmd() < 0)
+ goto out_free_temp_buffer;
+
/* TODO: make the number of buffers hot pluggable with CPUS */
if (allocate_trace_buffers(&global_trace, ring_buf_size) < 0) {
printk(KERN_ERR "tracer: failed to allocate ring buffer!\n");
WARN_ON(1);
- goto out_free_temp_buffer;
+ goto out_free_savedcmd;
}
if (global_trace.buffer_disabled)
tracing_off();
- trace_init_cmdlines();
-
if (trace_boot_clock) {
ret = tracing_set_clock(&global_trace, trace_boot_clock);
if (ret < 0)
@@ -6629,6 +6782,10 @@
*/
global_trace.current_trace = &nop_trace;
+ global_trace.max_lock = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED;
+
+ ftrace_init_global_array_ops(&global_trace);
+
register_tracer(&nop_trace);
/* All seems OK, enable tracing */
@@ -6656,13 +6813,11 @@
return 0;
+out_free_savedcmd:
+ free_saved_cmdlines_buffer(savedcmd);
out_free_temp_buffer:
ring_buffer_free(temp_buffer);
out_free_cpumask:
- free_percpu(global_trace.trace_buffer.data);
-#ifdef CONFIG_TRACER_MAX_TRACE
- free_percpu(global_trace.max_buffer.data);
-#endif
free_cpumask_var(global_trace.tracing_cpumask);
out_free_buffer_mask:
free_cpumask_var(tracing_buffer_mask);
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 2e29d7b..9e82551 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -190,7 +190,22 @@
*/
struct trace_buffer max_buffer;
bool allocated_snapshot;
+ unsigned long max_latency;
#endif
+ /*
+ * max_lock is used to protect the swapping of buffers
+ * when taking a max snapshot. The buffers themselves are
+ * protected by per_cpu spinlocks. But the action of the swap
+ * needs its own lock.
+ *
+ * This is defined as a arch_spinlock_t in order to help
+ * with performance when lockdep debugging is enabled.
+ *
+ * It is also used in other places outside the update_max_tr
+ * so it needs to be defined outside of the
+ * CONFIG_TRACER_MAX_TRACE.
+ */
+ arch_spinlock_t max_lock;
int buffer_disabled;
#ifdef CONFIG_FTRACE_SYSCALLS
int sys_refcount_enter;
@@ -237,6 +252,9 @@
{
struct trace_array *tr;
+ if (list_empty(ftrace_trace_arrays.prev))
+ return NULL;
+
tr = list_entry(ftrace_trace_arrays.prev,
typeof(*tr), list);
WARN_ON(!(tr->flags & TRACE_ARRAY_FL_GLOBAL));
@@ -323,7 +341,6 @@
* @stop: called when tracing is paused (echo 0 > tracing_enabled)
* @open: called when the trace file is opened
* @pipe_open: called when the trace_pipe file is opened
- * @wait_pipe: override how the user waits for traces on trace_pipe
* @close: called when the trace file is released
* @pipe_close: called when the trace_pipe file is released
* @read: override the default read callback on trace_pipe
@@ -342,7 +359,6 @@
void (*stop)(struct trace_array *tr);
void (*open)(struct trace_iterator *iter);
void (*pipe_open)(struct trace_iterator *iter);
- void (*wait_pipe)(struct trace_iterator *iter);
void (*close)(struct trace_iterator *iter);
void (*pipe_close)(struct trace_iterator *iter);
ssize_t (*read)(struct trace_iterator *iter,
@@ -416,13 +432,7 @@
TRACE_FTRACE_IRQ_BIT,
TRACE_FTRACE_SIRQ_BIT,
- /* GLOBAL_BITs must be greater than FTRACE_BITs */
- TRACE_GLOBAL_BIT,
- TRACE_GLOBAL_NMI_BIT,
- TRACE_GLOBAL_IRQ_BIT,
- TRACE_GLOBAL_SIRQ_BIT,
-
- /* INTERNAL_BITs must be greater than GLOBAL_BITs */
+ /* INTERNAL_BITs must be greater than FTRACE_BITs */
TRACE_INTERNAL_BIT,
TRACE_INTERNAL_NMI_BIT,
TRACE_INTERNAL_IRQ_BIT,
@@ -449,9 +459,6 @@
#define TRACE_FTRACE_START TRACE_FTRACE_BIT
#define TRACE_FTRACE_MAX ((1 << (TRACE_FTRACE_START + TRACE_CONTEXT_BITS)) - 1)
-#define TRACE_GLOBAL_START TRACE_GLOBAL_BIT
-#define TRACE_GLOBAL_MAX ((1 << (TRACE_GLOBAL_START + TRACE_CONTEXT_BITS)) - 1)
-
#define TRACE_LIST_START TRACE_INTERNAL_BIT
#define TRACE_LIST_MAX ((1 << (TRACE_LIST_START + TRACE_CONTEXT_BITS)) - 1)
@@ -560,8 +567,6 @@
void tracing_iter_reset(struct trace_iterator *iter, int cpu);
-void poll_wait_pipe(struct trace_iterator *iter);
-
void tracing_sched_switch_trace(struct trace_array *tr,
struct task_struct *prev,
struct task_struct *next,
@@ -608,8 +613,6 @@
extern unsigned long tracing_thresh;
#ifdef CONFIG_TRACER_MAX_TRACE
-extern unsigned long tracing_max_latency;
-
void update_max_tr(struct trace_array *tr, struct task_struct *tsk, int cpu);
void update_max_tr_single(struct trace_array *tr,
struct task_struct *tsk, int cpu);
@@ -724,6 +727,8 @@
#define TRACE_GRAPH_PRINT_PROC 0x8
#define TRACE_GRAPH_PRINT_DURATION 0x10
#define TRACE_GRAPH_PRINT_ABS_TIME 0x20
+#define TRACE_GRAPH_PRINT_IRQS 0x40
+#define TRACE_GRAPH_PRINT_TAIL 0x80
#define TRACE_GRAPH_PRINT_FILL_SHIFT 28
#define TRACE_GRAPH_PRINT_FILL_MASK (0x3 << TRACE_GRAPH_PRINT_FILL_SHIFT)
@@ -823,6 +828,10 @@
int ftrace_create_function_files(struct trace_array *tr,
struct dentry *parent);
void ftrace_destroy_function_files(struct trace_array *tr);
+void ftrace_init_global_array_ops(struct trace_array *tr);
+void ftrace_init_array_ops(struct trace_array *tr, ftrace_func_t func);
+void ftrace_reset_array_ops(struct trace_array *tr);
+int using_ftrace_ops_list_func(void);
#else
static inline int ftrace_trace_task(struct task_struct *task)
{
@@ -836,6 +845,11 @@
return 0;
}
static inline void ftrace_destroy_function_files(struct trace_array *tr) { }
+static inline __init void
+ftrace_init_global_array_ops(struct trace_array *tr) { }
+static inline void ftrace_reset_array_ops(struct trace_array *tr) { }
+/* ftace_func_t type is not defined, use macro instead of static inline */
+#define ftrace_init_array_ops(tr, func) do { } while (0)
#endif /* CONFIG_FUNCTION_TRACER */
#if defined(CONFIG_FUNCTION_TRACER) && defined(CONFIG_DYNAMIC_FTRACE)
diff --git a/kernel/trace/trace_benchmark.c b/kernel/trace/trace_benchmark.c
new file mode 100644
index 0000000..40a14cb
--- /dev/null
+++ b/kernel/trace/trace_benchmark.c
@@ -0,0 +1,198 @@
+#include <linux/delay.h>
+#include <linux/module.h>
+#include <linux/kthread.h>
+#include <linux/trace_clock.h>
+
+#define CREATE_TRACE_POINTS
+#include "trace_benchmark.h"
+
+static struct task_struct *bm_event_thread;
+
+static char bm_str[BENCHMARK_EVENT_STRLEN] = "START";
+
+static u64 bm_total;
+static u64 bm_totalsq;
+static u64 bm_last;
+static u64 bm_max;
+static u64 bm_min;
+static u64 bm_first;
+static u64 bm_cnt;
+static u64 bm_stddev;
+static unsigned int bm_avg;
+static unsigned int bm_std;
+
+/*
+ * This gets called in a loop recording the time it took to write
+ * the tracepoint. What it writes is the time statistics of the last
+ * tracepoint write. As there is nothing to write the first time
+ * it simply writes "START". As the first write is cold cache and
+ * the rest is hot, we save off that time in bm_first and it is
+ * reported as "first", which is shown in the second write to the
+ * tracepoint. The "first" field is writen within the statics from
+ * then on but never changes.
+ */
+static void trace_do_benchmark(void)
+{
+ u64 start;
+ u64 stop;
+ u64 delta;
+ u64 stddev;
+ u64 seed;
+ u64 last_seed;
+ unsigned int avg;
+ unsigned int std = 0;
+
+ /* Only run if the tracepoint is actually active */
+ if (!trace_benchmark_event_enabled())
+ return;
+
+ local_irq_disable();
+ start = trace_clock_local();
+ trace_benchmark_event(bm_str);
+ stop = trace_clock_local();
+ local_irq_enable();
+
+ bm_cnt++;
+
+ delta = stop - start;
+
+ /*
+ * The first read is cold cached, keep it separate from the
+ * other calculations.
+ */
+ if (bm_cnt == 1) {
+ bm_first = delta;
+ scnprintf(bm_str, BENCHMARK_EVENT_STRLEN,
+ "first=%llu [COLD CACHED]", bm_first);
+ return;
+ }
+
+ bm_last = delta;
+
+ if (delta > bm_max)
+ bm_max = delta;
+ if (!bm_min || delta < bm_min)
+ bm_min = delta;
+
+ /*
+ * When bm_cnt is greater than UINT_MAX, it breaks the statistics
+ * accounting. Freeze the statistics when that happens.
+ * We should have enough data for the avg and stddev anyway.
+ */
+ if (bm_cnt > UINT_MAX) {
+ scnprintf(bm_str, BENCHMARK_EVENT_STRLEN,
+ "last=%llu first=%llu max=%llu min=%llu ** avg=%u std=%d std^2=%lld",
+ bm_last, bm_first, bm_max, bm_min, bm_avg, bm_std, bm_stddev);
+ return;
+ }
+
+ bm_total += delta;
+ bm_totalsq += delta * delta;
+
+
+ if (bm_cnt > 1) {
+ /*
+ * Apply Welford's method to calculate standard deviation:
+ * s^2 = 1 / (n * (n-1)) * (n * \Sum (x_i)^2 - (\Sum x_i)^2)
+ */
+ stddev = (u64)bm_cnt * bm_totalsq - bm_total * bm_total;
+ do_div(stddev, (u32)bm_cnt);
+ do_div(stddev, (u32)bm_cnt - 1);
+ } else
+ stddev = 0;
+
+ delta = bm_total;
+ do_div(delta, bm_cnt);
+ avg = delta;
+
+ if (stddev > 0) {
+ int i = 0;
+ /*
+ * stddev is the square of standard deviation but
+ * we want the actualy number. Use the average
+ * as our seed to find the std.
+ *
+ * The next try is:
+ * x = (x + N/x) / 2
+ *
+ * Where N is the squared number to find the square
+ * root of.
+ */
+ seed = avg;
+ do {
+ last_seed = seed;
+ seed = stddev;
+ if (!last_seed)
+ break;
+ do_div(seed, last_seed);
+ seed += last_seed;
+ do_div(seed, 2);
+ } while (i++ < 10 && last_seed != seed);
+
+ std = seed;
+ }
+
+ scnprintf(bm_str, BENCHMARK_EVENT_STRLEN,
+ "last=%llu first=%llu max=%llu min=%llu avg=%u std=%d std^2=%lld",
+ bm_last, bm_first, bm_max, bm_min, avg, std, stddev);
+
+ bm_std = std;
+ bm_avg = avg;
+ bm_stddev = stddev;
+}
+
+static int benchmark_event_kthread(void *arg)
+{
+ /* sleep a bit to make sure the tracepoint gets activated */
+ msleep(100);
+
+ while (!kthread_should_stop()) {
+
+ trace_do_benchmark();
+
+ /*
+ * We don't go to sleep, but let others
+ * run as well.
+ */
+ cond_resched();
+ }
+
+ return 0;
+}
+
+/*
+ * When the benchmark tracepoint is enabled, it calls this
+ * function and the thread that calls the tracepoint is created.
+ */
+void trace_benchmark_reg(void)
+{
+ bm_event_thread = kthread_run(benchmark_event_kthread,
+ NULL, "event_benchmark");
+ WARN_ON(!bm_event_thread);
+}
+
+/*
+ * When the benchmark tracepoint is disabled, it calls this
+ * function and the thread that calls the tracepoint is deleted
+ * and all the numbers are reset.
+ */
+void trace_benchmark_unreg(void)
+{
+ if (!bm_event_thread)
+ return;
+
+ kthread_stop(bm_event_thread);
+
+ strcpy(bm_str, "START");
+ bm_total = 0;
+ bm_totalsq = 0;
+ bm_last = 0;
+ bm_max = 0;
+ bm_min = 0;
+ bm_cnt = 0;
+ /* These don't need to be reset but reset them anyway */
+ bm_first = 0;
+ bm_std = 0;
+ bm_avg = 0;
+ bm_stddev = 0;
+}
diff --git a/kernel/trace/trace_benchmark.h b/kernel/trace/trace_benchmark.h
new file mode 100644
index 0000000..3c1df1df
--- /dev/null
+++ b/kernel/trace/trace_benchmark.h
@@ -0,0 +1,41 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM benchmark
+
+#if !defined(_TRACE_BENCHMARK_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_BENCHMARK_H
+
+#include <linux/tracepoint.h>
+
+extern void trace_benchmark_reg(void);
+extern void trace_benchmark_unreg(void);
+
+#define BENCHMARK_EVENT_STRLEN 128
+
+TRACE_EVENT_FN(benchmark_event,
+
+ TP_PROTO(const char *str),
+
+ TP_ARGS(str),
+
+ TP_STRUCT__entry(
+ __array( char, str, BENCHMARK_EVENT_STRLEN )
+ ),
+
+ TP_fast_assign(
+ memcpy(__entry->str, str, BENCHMARK_EVENT_STRLEN);
+ ),
+
+ TP_printk("%s", __entry->str),
+
+ trace_benchmark_reg, trace_benchmark_unreg
+);
+
+#endif /* _TRACE_BENCHMARK_H */
+
+#undef TRACE_INCLUDE_FILE
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#define TRACE_INCLUDE_FILE trace_benchmark
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index 3ddfd8f..f99e0b3 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -574,6 +574,9 @@
{
struct trace_array *tr = top_trace_array();
+ if (!tr)
+ return -ENODEV;
+
return __ftrace_set_clr_event(tr, NULL, system, event, set);
}
EXPORT_SYMBOL_GPL(trace_set_clr_event);
@@ -2065,6 +2068,9 @@
bool enable;
int ret;
+ if (!tr)
+ return -ENODEV;
+
/* hash funcs only work with set_ftrace_filter */
if (!enabled || !param)
return -EINVAL;
@@ -2396,6 +2402,9 @@
char *token;
int ret;
+ if (!tr)
+ return -ENODEV;
+
for_each_event(iter, __start_ftrace_events, __stop_ftrace_events) {
call = *iter;
@@ -2442,6 +2451,8 @@
int ret;
tr = top_trace_array();
+ if (!tr)
+ return -ENODEV;
d_tracer = tracing_init_dentry();
if (!d_tracer)
@@ -2535,6 +2546,8 @@
int ret;
tr = top_trace_array();
+ if (!tr)
+ return;
pr_info("Running tests on trace events:\n");
diff --git a/kernel/trace/trace_functions.c b/kernel/trace/trace_functions.c
index ffd5635..57f0ec9 100644
--- a/kernel/trace/trace_functions.c
+++ b/kernel/trace/trace_functions.c
@@ -26,8 +26,6 @@
static void
function_stack_trace_call(unsigned long ip, unsigned long parent_ip,
struct ftrace_ops *op, struct pt_regs *pt_regs);
-static struct ftrace_ops trace_ops;
-static struct ftrace_ops trace_stack_ops;
static struct tracer_flags func_flags;
/* Our option */
@@ -83,28 +81,24 @@
static int function_trace_init(struct trace_array *tr)
{
- struct ftrace_ops *ops;
+ ftrace_func_t func;
- if (tr->flags & TRACE_ARRAY_FL_GLOBAL) {
- /* There's only one global tr */
- if (!trace_ops.private) {
- trace_ops.private = tr;
- trace_stack_ops.private = tr;
- }
-
- if (func_flags.val & TRACE_FUNC_OPT_STACK)
- ops = &trace_stack_ops;
- else
- ops = &trace_ops;
- tr->ops = ops;
- } else if (!tr->ops) {
- /*
- * Instance trace_arrays get their ops allocated
- * at instance creation. Unless it failed
- * the allocation.
- */
+ /*
+ * Instance trace_arrays get their ops allocated
+ * at instance creation. Unless it failed
+ * the allocation.
+ */
+ if (!tr->ops)
return -ENOMEM;
- }
+
+ /* Currently only the global instance can do stack tracing */
+ if (tr->flags & TRACE_ARRAY_FL_GLOBAL &&
+ func_flags.val & TRACE_FUNC_OPT_STACK)
+ func = function_stack_trace_call;
+ else
+ func = function_trace_call;
+
+ ftrace_init_array_ops(tr, func);
tr->trace_buffer.cpu = get_cpu();
put_cpu();
@@ -118,6 +112,7 @@
{
tracing_stop_function_trace(tr);
tracing_stop_cmdline_record();
+ ftrace_reset_array_ops(tr);
}
static void function_trace_start(struct trace_array *tr)
@@ -199,18 +194,6 @@
local_irq_restore(flags);
}
-static struct ftrace_ops trace_ops __read_mostly =
-{
- .func = function_trace_call,
- .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
-};
-
-static struct ftrace_ops trace_stack_ops __read_mostly =
-{
- .func = function_stack_trace_call,
- .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
-};
-
static struct tracer_opt func_opts[] = {
#ifdef CONFIG_STACKTRACE
{ TRACER_OPT(func_stack_trace, TRACE_FUNC_OPT_STACK) },
@@ -248,10 +231,10 @@
unregister_ftrace_function(tr->ops);
if (set) {
- tr->ops = &trace_stack_ops;
+ tr->ops->func = function_stack_trace_call;
register_ftrace_function(tr->ops);
} else {
- tr->ops = &trace_ops;
+ tr->ops->func = function_trace_call;
register_ftrace_function(tr->ops);
}
@@ -269,7 +252,6 @@
.init = function_trace_init,
.reset = function_trace_reset,
.start = function_trace_start,
- .wait_pipe = poll_wait_pipe,
.flags = &func_flags,
.set_flag = func_set_flag,
.allow_instances = true,
diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c
index deff112..4de3e57 100644
--- a/kernel/trace/trace_functions_graph.c
+++ b/kernel/trace/trace_functions_graph.c
@@ -38,15 +38,6 @@
#define TRACE_GRAPH_INDENT 2
-/* Flag options */
-#define TRACE_GRAPH_PRINT_OVERRUN 0x1
-#define TRACE_GRAPH_PRINT_CPU 0x2
-#define TRACE_GRAPH_PRINT_OVERHEAD 0x4
-#define TRACE_GRAPH_PRINT_PROC 0x8
-#define TRACE_GRAPH_PRINT_DURATION 0x10
-#define TRACE_GRAPH_PRINT_ABS_TIME 0x20
-#define TRACE_GRAPH_PRINT_IRQS 0x40
-
static unsigned int max_depth;
static struct tracer_opt trace_opts[] = {
@@ -64,11 +55,13 @@
{ TRACER_OPT(funcgraph-abstime, TRACE_GRAPH_PRINT_ABS_TIME) },
/* Display interrupts */
{ TRACER_OPT(funcgraph-irqs, TRACE_GRAPH_PRINT_IRQS) },
+ /* Display function name after trailing } */
+ { TRACER_OPT(funcgraph-tail, TRACE_GRAPH_PRINT_TAIL) },
{ } /* Empty entry */
};
static struct tracer_flags tracer_flags = {
- /* Don't display overruns and proc by default */
+ /* Don't display overruns, proc, or tail by default */
.val = TRACE_GRAPH_PRINT_CPU | TRACE_GRAPH_PRINT_OVERHEAD |
TRACE_GRAPH_PRINT_DURATION | TRACE_GRAPH_PRINT_IRQS,
.opts = trace_opts
@@ -1176,9 +1169,10 @@
* If the return function does not have a matching entry,
* then the entry was lost. Instead of just printing
* the '}' and letting the user guess what function this
- * belongs to, write out the function name.
+ * belongs to, write out the function name. Always do
+ * that if the funcgraph-tail option is enabled.
*/
- if (func_match) {
+ if (func_match && !(flags & TRACE_GRAPH_PRINT_TAIL)) {
ret = trace_seq_puts(s, "}\n");
if (!ret)
return TRACE_TYPE_PARTIAL_LINE;
@@ -1505,7 +1499,6 @@
.pipe_open = graph_trace_open,
.close = graph_trace_close,
.pipe_close = graph_trace_close,
- .wait_pipe = poll_wait_pipe,
.init = graph_trace_init,
.reset = graph_trace_reset,
.print_line = print_graph_function,
diff --git a/kernel/trace/trace_irqsoff.c b/kernel/trace/trace_irqsoff.c
index 8ff02cb..9bb104f7 100644
--- a/kernel/trace/trace_irqsoff.c
+++ b/kernel/trace/trace_irqsoff.c
@@ -151,12 +151,6 @@
atomic_dec(&data->disabled);
}
-
-static struct ftrace_ops trace_ops __read_mostly =
-{
- .func = irqsoff_tracer_call,
- .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
-};
#endif /* CONFIG_FUNCTION_TRACER */
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
@@ -176,7 +170,7 @@
for_each_possible_cpu(cpu)
per_cpu(tracing_cpu, cpu) = 0;
- tracing_max_latency = 0;
+ tr->max_latency = 0;
tracing_reset_online_cpus(&irqsoff_trace->trace_buffer);
return start_irqsoff_tracer(irqsoff_trace, set);
@@ -303,13 +297,13 @@
/*
* Should this new latency be reported/recorded?
*/
-static int report_latency(cycle_t delta)
+static int report_latency(struct trace_array *tr, cycle_t delta)
{
if (tracing_thresh) {
if (delta < tracing_thresh)
return 0;
} else {
- if (delta <= tracing_max_latency)
+ if (delta <= tr->max_latency)
return 0;
}
return 1;
@@ -333,13 +327,13 @@
pc = preempt_count();
- if (!report_latency(delta))
+ if (!report_latency(tr, delta))
goto out;
raw_spin_lock_irqsave(&max_trace_lock, flags);
/* check if we are still the max latency */
- if (!report_latency(delta))
+ if (!report_latency(tr, delta))
goto out_unlock;
__trace_function(tr, CALLER_ADDR0, parent_ip, flags, pc);
@@ -352,7 +346,7 @@
data->critical_end = parent_ip;
if (likely(!is_tracing_stopped())) {
- tracing_max_latency = delta;
+ tr->max_latency = delta;
update_max_tr_single(tr, current, cpu);
}
@@ -531,7 +525,7 @@
}
#endif /* CONFIG_PREEMPT_TRACER */
-static int register_irqsoff_function(int graph, int set)
+static int register_irqsoff_function(struct trace_array *tr, int graph, int set)
{
int ret;
@@ -543,7 +537,7 @@
ret = register_ftrace_graph(&irqsoff_graph_return,
&irqsoff_graph_entry);
else
- ret = register_ftrace_function(&trace_ops);
+ ret = register_ftrace_function(tr->ops);
if (!ret)
function_enabled = true;
@@ -551,7 +545,7 @@
return ret;
}
-static void unregister_irqsoff_function(int graph)
+static void unregister_irqsoff_function(struct trace_array *tr, int graph)
{
if (!function_enabled)
return;
@@ -559,17 +553,17 @@
if (graph)
unregister_ftrace_graph();
else
- unregister_ftrace_function(&trace_ops);
+ unregister_ftrace_function(tr->ops);
function_enabled = false;
}
-static void irqsoff_function_set(int set)
+static void irqsoff_function_set(struct trace_array *tr, int set)
{
if (set)
- register_irqsoff_function(is_graph(), 1);
+ register_irqsoff_function(tr, is_graph(), 1);
else
- unregister_irqsoff_function(is_graph());
+ unregister_irqsoff_function(tr, is_graph());
}
static int irqsoff_flag_changed(struct trace_array *tr, u32 mask, int set)
@@ -577,7 +571,7 @@
struct tracer *tracer = tr->current_trace;
if (mask & TRACE_ITER_FUNCTION)
- irqsoff_function_set(set);
+ irqsoff_function_set(tr, set);
return trace_keep_overwrite(tracer, mask, set);
}
@@ -586,7 +580,7 @@
{
int ret;
- ret = register_irqsoff_function(graph, 0);
+ ret = register_irqsoff_function(tr, graph, 0);
if (!ret && tracing_is_enabled())
tracer_enabled = 1;
@@ -600,25 +594,37 @@
{
tracer_enabled = 0;
- unregister_irqsoff_function(graph);
+ unregister_irqsoff_function(tr, graph);
}
-static void __irqsoff_tracer_init(struct trace_array *tr)
+static bool irqsoff_busy;
+
+static int __irqsoff_tracer_init(struct trace_array *tr)
{
+ if (irqsoff_busy)
+ return -EBUSY;
+
save_flags = trace_flags;
/* non overwrite screws up the latency tracers */
set_tracer_flag(tr, TRACE_ITER_OVERWRITE, 1);
set_tracer_flag(tr, TRACE_ITER_LATENCY_FMT, 1);
- tracing_max_latency = 0;
+ tr->max_latency = 0;
irqsoff_trace = tr;
/* make sure that the tracer is visible */
smp_wmb();
tracing_reset_online_cpus(&tr->trace_buffer);
- if (start_irqsoff_tracer(tr, is_graph()))
+ ftrace_init_array_ops(tr, irqsoff_tracer_call);
+
+ /* Only toplevel instance supports graph tracing */
+ if (start_irqsoff_tracer(tr, (tr->flags & TRACE_ARRAY_FL_GLOBAL &&
+ is_graph())))
printk(KERN_ERR "failed to start irqsoff tracer\n");
+
+ irqsoff_busy = true;
+ return 0;
}
static void irqsoff_tracer_reset(struct trace_array *tr)
@@ -630,6 +636,9 @@
set_tracer_flag(tr, TRACE_ITER_LATENCY_FMT, lat_flag);
set_tracer_flag(tr, TRACE_ITER_OVERWRITE, overwrite_flag);
+ ftrace_reset_array_ops(tr);
+
+ irqsoff_busy = false;
}
static void irqsoff_tracer_start(struct trace_array *tr)
@@ -647,8 +656,7 @@
{
trace_type = TRACER_IRQS_OFF;
- __irqsoff_tracer_init(tr);
- return 0;
+ return __irqsoff_tracer_init(tr);
}
static struct tracer irqsoff_tracer __read_mostly =
{
@@ -668,6 +676,7 @@
#endif
.open = irqsoff_trace_open,
.close = irqsoff_trace_close,
+ .allow_instances = true,
.use_max_tr = true,
};
# define register_irqsoff(trace) register_tracer(&trace)
@@ -680,8 +689,7 @@
{
trace_type = TRACER_PREEMPT_OFF;
- __irqsoff_tracer_init(tr);
- return 0;
+ return __irqsoff_tracer_init(tr);
}
static struct tracer preemptoff_tracer __read_mostly =
@@ -702,6 +710,7 @@
#endif
.open = irqsoff_trace_open,
.close = irqsoff_trace_close,
+ .allow_instances = true,
.use_max_tr = true,
};
# define register_preemptoff(trace) register_tracer(&trace)
@@ -716,8 +725,7 @@
{
trace_type = TRACER_IRQS_OFF | TRACER_PREEMPT_OFF;
- __irqsoff_tracer_init(tr);
- return 0;
+ return __irqsoff_tracer_init(tr);
}
static struct tracer preemptirqsoff_tracer __read_mostly =
@@ -738,6 +746,7 @@
#endif
.open = irqsoff_trace_open,
.close = irqsoff_trace_close,
+ .allow_instances = true,
.use_max_tr = true,
};
diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index 903ae28..ef2fba1 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -1377,6 +1377,9 @@
struct trace_kprobe *tk;
struct ftrace_event_file *file;
+ if (tracing_is_disabled())
+ return -ENODEV;
+
target = kprobe_trace_selftest_target;
pr_info("Testing kprobe tracing: ");
diff --git a/kernel/trace/trace_nop.c b/kernel/trace/trace_nop.c
index 69a5cc9..fcf0a9e 100644
--- a/kernel/trace/trace_nop.c
+++ b/kernel/trace/trace_nop.c
@@ -91,7 +91,6 @@
.name = "nop",
.init = nop_trace_init,
.reset = nop_trace_reset,
- .wait_pipe = poll_wait_pipe,
#ifdef CONFIG_FTRACE_SELFTEST
.selftest = trace_selftest_startup_nop,
#endif
diff --git a/kernel/trace/trace_output.c b/kernel/trace/trace_output.c
index a436de1..f3dad80 100644
--- a/kernel/trace/trace_output.c
+++ b/kernel/trace/trace_output.c
@@ -126,6 +126,34 @@
EXPORT_SYMBOL_GPL(trace_seq_printf);
/**
+ * trace_seq_bitmask - put a list of longs as a bitmask print output
+ * @s: trace sequence descriptor
+ * @maskp: points to an array of unsigned longs that represent a bitmask
+ * @nmaskbits: The number of bits that are valid in @maskp
+ *
+ * It returns 0 if the trace oversizes the buffer's free
+ * space, 1 otherwise.
+ *
+ * Writes a ASCII representation of a bitmask string into @s.
+ */
+int
+trace_seq_bitmask(struct trace_seq *s, const unsigned long *maskp,
+ int nmaskbits)
+{
+ int len = (PAGE_SIZE - 1) - s->len;
+ int ret;
+
+ if (s->full || !len)
+ return 0;
+
+ ret = bitmap_scnprintf(s->buffer, len, maskp, nmaskbits);
+ s->len += ret;
+
+ return 1;
+}
+EXPORT_SYMBOL_GPL(trace_seq_bitmask);
+
+/**
* trace_seq_vprintf - sequence printing of trace information
* @s: trace sequence descriptor
* @fmt: printf format string
@@ -399,6 +427,19 @@
#endif
const char *
+ftrace_print_bitmask_seq(struct trace_seq *p, void *bitmask_ptr,
+ unsigned int bitmask_size)
+{
+ const char *ret = p->buffer + p->len;
+
+ trace_seq_bitmask(p, bitmask_ptr, bitmask_size * 8);
+ trace_seq_putc(p, 0);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(ftrace_print_bitmask_seq);
+
+const char *
ftrace_print_hex_seq(struct trace_seq *p, const unsigned char *buf, int buf_len)
{
int i;
diff --git a/kernel/trace/trace_sched_wakeup.c b/kernel/trace/trace_sched_wakeup.c
index e14da5e9..19bd892 100644
--- a/kernel/trace/trace_sched_wakeup.c
+++ b/kernel/trace/trace_sched_wakeup.c
@@ -130,15 +130,9 @@
atomic_dec(&data->disabled);
preempt_enable_notrace();
}
-
-static struct ftrace_ops trace_ops __read_mostly =
-{
- .func = wakeup_tracer_call,
- .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
-};
#endif /* CONFIG_FUNCTION_TRACER */
-static int register_wakeup_function(int graph, int set)
+static int register_wakeup_function(struct trace_array *tr, int graph, int set)
{
int ret;
@@ -150,7 +144,7 @@
ret = register_ftrace_graph(&wakeup_graph_return,
&wakeup_graph_entry);
else
- ret = register_ftrace_function(&trace_ops);
+ ret = register_ftrace_function(tr->ops);
if (!ret)
function_enabled = true;
@@ -158,7 +152,7 @@
return ret;
}
-static void unregister_wakeup_function(int graph)
+static void unregister_wakeup_function(struct trace_array *tr, int graph)
{
if (!function_enabled)
return;
@@ -166,17 +160,17 @@
if (graph)
unregister_ftrace_graph();
else
- unregister_ftrace_function(&trace_ops);
+ unregister_ftrace_function(tr->ops);
function_enabled = false;
}
-static void wakeup_function_set(int set)
+static void wakeup_function_set(struct trace_array *tr, int set)
{
if (set)
- register_wakeup_function(is_graph(), 1);
+ register_wakeup_function(tr, is_graph(), 1);
else
- unregister_wakeup_function(is_graph());
+ unregister_wakeup_function(tr, is_graph());
}
static int wakeup_flag_changed(struct trace_array *tr, u32 mask, int set)
@@ -184,16 +178,16 @@
struct tracer *tracer = tr->current_trace;
if (mask & TRACE_ITER_FUNCTION)
- wakeup_function_set(set);
+ wakeup_function_set(tr, set);
return trace_keep_overwrite(tracer, mask, set);
}
-static int start_func_tracer(int graph)
+static int start_func_tracer(struct trace_array *tr, int graph)
{
int ret;
- ret = register_wakeup_function(graph, 0);
+ ret = register_wakeup_function(tr, graph, 0);
if (!ret && tracing_is_enabled())
tracer_enabled = 1;
@@ -203,11 +197,11 @@
return ret;
}
-static void stop_func_tracer(int graph)
+static void stop_func_tracer(struct trace_array *tr, int graph)
{
tracer_enabled = 0;
- unregister_wakeup_function(graph);
+ unregister_wakeup_function(tr, graph);
}
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
@@ -221,12 +215,12 @@
if (!(is_graph() ^ set))
return 0;
- stop_func_tracer(!set);
+ stop_func_tracer(tr, !set);
wakeup_reset(wakeup_trace);
- tracing_max_latency = 0;
+ tr->max_latency = 0;
- return start_func_tracer(set);
+ return start_func_tracer(tr, set);
}
static int wakeup_graph_entry(struct ftrace_graph_ent *trace)
@@ -350,13 +344,13 @@
/*
* Should this new latency be reported/recorded?
*/
-static int report_latency(cycle_t delta)
+static int report_latency(struct trace_array *tr, cycle_t delta)
{
if (tracing_thresh) {
if (delta < tracing_thresh)
return 0;
} else {
- if (delta <= tracing_max_latency)
+ if (delta <= tr->max_latency)
return 0;
}
return 1;
@@ -424,11 +418,11 @@
T1 = ftrace_now(cpu);
delta = T1-T0;
- if (!report_latency(delta))
+ if (!report_latency(wakeup_trace, delta))
goto out_unlock;
if (likely(!is_tracing_stopped())) {
- tracing_max_latency = delta;
+ wakeup_trace->max_latency = delta;
update_max_tr(wakeup_trace, wakeup_task, wakeup_cpu);
}
@@ -587,7 +581,7 @@
*/
smp_wmb();
- if (start_func_tracer(is_graph()))
+ if (start_func_tracer(tr, is_graph()))
printk(KERN_ERR "failed to start wakeup tracer\n");
return;
@@ -600,13 +594,15 @@
static void stop_wakeup_tracer(struct trace_array *tr)
{
tracer_enabled = 0;
- stop_func_tracer(is_graph());
+ stop_func_tracer(tr, is_graph());
unregister_trace_sched_switch(probe_wakeup_sched_switch, NULL);
unregister_trace_sched_wakeup_new(probe_wakeup, NULL);
unregister_trace_sched_wakeup(probe_wakeup, NULL);
unregister_trace_sched_migrate_task(probe_wakeup_migrate_task, NULL);
}
+static bool wakeup_busy;
+
static int __wakeup_tracer_init(struct trace_array *tr)
{
save_flags = trace_flags;
@@ -615,14 +611,20 @@
set_tracer_flag(tr, TRACE_ITER_OVERWRITE, 1);
set_tracer_flag(tr, TRACE_ITER_LATENCY_FMT, 1);
- tracing_max_latency = 0;
+ tr->max_latency = 0;
wakeup_trace = tr;
+ ftrace_init_array_ops(tr, wakeup_tracer_call);
start_wakeup_tracer(tr);
+
+ wakeup_busy = true;
return 0;
}
static int wakeup_tracer_init(struct trace_array *tr)
{
+ if (wakeup_busy)
+ return -EBUSY;
+
wakeup_dl = 0;
wakeup_rt = 0;
return __wakeup_tracer_init(tr);
@@ -630,6 +632,9 @@
static int wakeup_rt_tracer_init(struct trace_array *tr)
{
+ if (wakeup_busy)
+ return -EBUSY;
+
wakeup_dl = 0;
wakeup_rt = 1;
return __wakeup_tracer_init(tr);
@@ -637,6 +642,9 @@
static int wakeup_dl_tracer_init(struct trace_array *tr)
{
+ if (wakeup_busy)
+ return -EBUSY;
+
wakeup_dl = 1;
wakeup_rt = 0;
return __wakeup_tracer_init(tr);
@@ -653,6 +661,8 @@
set_tracer_flag(tr, TRACE_ITER_LATENCY_FMT, lat_flag);
set_tracer_flag(tr, TRACE_ITER_OVERWRITE, overwrite_flag);
+ ftrace_reset_array_ops(tr);
+ wakeup_busy = false;
}
static void wakeup_tracer_start(struct trace_array *tr)
@@ -684,6 +694,7 @@
#endif
.open = wakeup_trace_open,
.close = wakeup_trace_close,
+ .allow_instances = true,
.use_max_tr = true,
};
@@ -694,7 +705,6 @@
.reset = wakeup_tracer_reset,
.start = wakeup_tracer_start,
.stop = wakeup_tracer_stop,
- .wait_pipe = poll_wait_pipe,
.print_max = true,
.print_header = wakeup_print_header,
.print_line = wakeup_print_line,
@@ -706,6 +716,7 @@
#endif
.open = wakeup_trace_open,
.close = wakeup_trace_close,
+ .allow_instances = true,
.use_max_tr = true,
};
@@ -716,7 +727,6 @@
.reset = wakeup_tracer_reset,
.start = wakeup_tracer_start,
.stop = wakeup_tracer_stop,
- .wait_pipe = poll_wait_pipe,
.print_max = true,
.print_header = wakeup_print_header,
.print_line = wakeup_print_line,
diff --git a/kernel/trace/trace_selftest.c b/kernel/trace/trace_selftest.c
index e98fca6..5ef6049 100644
--- a/kernel/trace/trace_selftest.c
+++ b/kernel/trace/trace_selftest.c
@@ -65,7 +65,7 @@
/* Don't allow flipping of max traces now */
local_irq_save(flags);
- arch_spin_lock(&ftrace_max_lock);
+ arch_spin_lock(&buf->tr->max_lock);
cnt = ring_buffer_entries(buf->buffer);
@@ -83,7 +83,7 @@
break;
}
tracing_on();
- arch_spin_unlock(&ftrace_max_lock);
+ arch_spin_unlock(&buf->tr->max_lock);
local_irq_restore(flags);
if (count)
@@ -161,11 +161,6 @@
.flags = FTRACE_OPS_FL_RECURSION_SAFE,
};
-static struct ftrace_ops test_global = {
- .func = trace_selftest_test_global_func,
- .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
-};
-
static void print_counts(void)
{
printk("(%d %d %d %d %d) ",
@@ -185,7 +180,7 @@
trace_selftest_test_dyn_cnt = 0;
}
-static int trace_selftest_ops(int cnt)
+static int trace_selftest_ops(struct trace_array *tr, int cnt)
{
int save_ftrace_enabled = ftrace_enabled;
struct ftrace_ops *dyn_ops;
@@ -220,7 +215,11 @@
register_ftrace_function(&test_probe1);
register_ftrace_function(&test_probe2);
register_ftrace_function(&test_probe3);
- register_ftrace_function(&test_global);
+ /* First time we are running with main function */
+ if (cnt > 1) {
+ ftrace_init_array_ops(tr, trace_selftest_test_global_func);
+ register_ftrace_function(tr->ops);
+ }
DYN_FTRACE_TEST_NAME();
@@ -232,8 +231,10 @@
goto out;
if (trace_selftest_test_probe3_cnt != 1)
goto out;
- if (trace_selftest_test_global_cnt == 0)
- goto out;
+ if (cnt > 1) {
+ if (trace_selftest_test_global_cnt == 0)
+ goto out;
+ }
DYN_FTRACE_TEST_NAME2();
@@ -269,8 +270,10 @@
goto out_free;
if (trace_selftest_test_probe3_cnt != 3)
goto out_free;
- if (trace_selftest_test_global_cnt == 0)
- goto out;
+ if (cnt > 1) {
+ if (trace_selftest_test_global_cnt == 0)
+ goto out;
+ }
if (trace_selftest_test_dyn_cnt == 0)
goto out_free;
@@ -295,7 +298,9 @@
unregister_ftrace_function(&test_probe1);
unregister_ftrace_function(&test_probe2);
unregister_ftrace_function(&test_probe3);
- unregister_ftrace_function(&test_global);
+ if (cnt > 1)
+ unregister_ftrace_function(tr->ops);
+ ftrace_reset_array_ops(tr);
/* Make sure everything is off */
reset_counts();
@@ -315,9 +320,9 @@
}
/* Test dynamic code modification and ftrace filters */
-int trace_selftest_startup_dynamic_tracing(struct tracer *trace,
- struct trace_array *tr,
- int (*func)(void))
+static int trace_selftest_startup_dynamic_tracing(struct tracer *trace,
+ struct trace_array *tr,
+ int (*func)(void))
{
int save_ftrace_enabled = ftrace_enabled;
unsigned long count;
@@ -388,7 +393,7 @@
}
/* Test the ops with global tracing running */
- ret = trace_selftest_ops(1);
+ ret = trace_selftest_ops(tr, 1);
trace->reset(tr);
out:
@@ -399,7 +404,7 @@
/* Test the ops with global tracing off */
if (!ret)
- ret = trace_selftest_ops(2);
+ ret = trace_selftest_ops(tr, 2);
return ret;
}
@@ -802,7 +807,7 @@
int
trace_selftest_startup_irqsoff(struct tracer *trace, struct trace_array *tr)
{
- unsigned long save_max = tracing_max_latency;
+ unsigned long save_max = tr->max_latency;
unsigned long count;
int ret;
@@ -814,7 +819,7 @@
}
/* reset the max latency */
- tracing_max_latency = 0;
+ tr->max_latency = 0;
/* disable interrupts for a bit */
local_irq_disable();
udelay(100);
@@ -841,7 +846,7 @@
ret = -1;
}
- tracing_max_latency = save_max;
+ tr->max_latency = save_max;
return ret;
}
@@ -851,7 +856,7 @@
int
trace_selftest_startup_preemptoff(struct tracer *trace, struct trace_array *tr)
{
- unsigned long save_max = tracing_max_latency;
+ unsigned long save_max = tr->max_latency;
unsigned long count;
int ret;
@@ -876,7 +881,7 @@
}
/* reset the max latency */
- tracing_max_latency = 0;
+ tr->max_latency = 0;
/* disable preemption for a bit */
preempt_disable();
udelay(100);
@@ -903,7 +908,7 @@
ret = -1;
}
- tracing_max_latency = save_max;
+ tr->max_latency = save_max;
return ret;
}
@@ -913,7 +918,7 @@
int
trace_selftest_startup_preemptirqsoff(struct tracer *trace, struct trace_array *tr)
{
- unsigned long save_max = tracing_max_latency;
+ unsigned long save_max = tr->max_latency;
unsigned long count;
int ret;
@@ -938,7 +943,7 @@
}
/* reset the max latency */
- tracing_max_latency = 0;
+ tr->max_latency = 0;
/* disable preemption and interrupts for a bit */
preempt_disable();
@@ -973,7 +978,7 @@
}
/* do the test by disabling interrupts first this time */
- tracing_max_latency = 0;
+ tr->max_latency = 0;
tracing_start();
trace->start(tr);
@@ -1004,7 +1009,7 @@
tracing_start();
out_no_start:
trace->reset(tr);
- tracing_max_latency = save_max;
+ tr->max_latency = save_max;
return ret;
}
@@ -1057,7 +1062,7 @@
int
trace_selftest_startup_wakeup(struct tracer *trace, struct trace_array *tr)
{
- unsigned long save_max = tracing_max_latency;
+ unsigned long save_max = tr->max_latency;
struct task_struct *p;
struct completion is_ready;
unsigned long count;
@@ -1083,7 +1088,7 @@
}
/* reset the max latency */
- tracing_max_latency = 0;
+ tr->max_latency = 0;
while (p->on_rq) {
/*
@@ -1113,7 +1118,7 @@
trace->reset(tr);
tracing_start();
- tracing_max_latency = save_max;
+ tr->max_latency = save_max;
/* kill the thread */
kthread_stop(p);
diff --git a/kernel/trace/trace_stack.c b/kernel/trace/trace_stack.c
index 21b320e..8a4e5cb 100644
--- a/kernel/trace/trace_stack.c
+++ b/kernel/trace/trace_stack.c
@@ -51,11 +51,33 @@
int stack_tracer_enabled;
static int last_stack_tracer_enabled;
+static inline void print_max_stack(void)
+{
+ long i;
+ int size;
+
+ pr_emerg(" Depth Size Location (%d entries)\n"
+ " ----- ---- --------\n",
+ max_stack_trace.nr_entries - 1);
+
+ for (i = 0; i < max_stack_trace.nr_entries; i++) {
+ if (stack_dump_trace[i] == ULONG_MAX)
+ break;
+ if (i+1 == max_stack_trace.nr_entries ||
+ stack_dump_trace[i+1] == ULONG_MAX)
+ size = stack_dump_index[i];
+ else
+ size = stack_dump_index[i] - stack_dump_index[i+1];
+
+ pr_emerg("%3ld) %8d %5d %pS\n", i, stack_dump_index[i],
+ size, (void *)stack_dump_trace[i]);
+ }
+}
+
static inline void
check_stack(unsigned long ip, unsigned long *stack)
{
- unsigned long this_size, flags;
- unsigned long *p, *top, *start;
+ unsigned long this_size, flags; unsigned long *p, *top, *start;
static int tracer_frame;
int frame_size = ACCESS_ONCE(tracer_frame);
int i;
@@ -85,8 +107,12 @@
max_stack_size = this_size;
- max_stack_trace.nr_entries = 0;
- max_stack_trace.skip = 3;
+ max_stack_trace.nr_entries = 0;
+
+ if (using_ftrace_ops_list_func())
+ max_stack_trace.skip = 4;
+ else
+ max_stack_trace.skip = 3;
save_stack_trace(&max_stack_trace);
@@ -145,8 +171,12 @@
i++;
}
- BUG_ON(current != &init_task &&
- *(end_of_stack(current)) != STACK_END_MAGIC);
+ if ((current != &init_task &&
+ *(end_of_stack(current)) != STACK_END_MAGIC)) {
+ print_max_stack();
+ BUG();
+ }
+
out:
arch_spin_unlock(&max_stack_lock);
local_irq_restore(flags);
diff --git a/kernel/user_namespace.c b/kernel/user_namespace.c
index bf71b4b..fcc0256 100644
--- a/kernel/user_namespace.c
+++ b/kernel/user_namespace.c
@@ -286,7 +286,7 @@
/**
* make_kgid - Map a user-namespace gid pair into a kgid.
* @ns: User namespace that the gid is in
- * @uid: group identifier
+ * @gid: group identifier
*
* Maps a user-namespace gid pair into a kernel internal kgid,
* and returns that kgid.
@@ -482,7 +482,8 @@
return 0;
}
-static void *m_start(struct seq_file *seq, loff_t *ppos, struct uid_gid_map *map)
+static void *m_start(struct seq_file *seq, loff_t *ppos,
+ struct uid_gid_map *map)
{
struct uid_gid_extent *extent = NULL;
loff_t pos = *ppos;
@@ -546,7 +547,8 @@
.show = projid_m_show,
};
-static bool mappings_overlap(struct uid_gid_map *new_map, struct uid_gid_extent *extent)
+static bool mappings_overlap(struct uid_gid_map *new_map,
+ struct uid_gid_extent *extent)
{
u32 upper_first, lower_first, upper_last, lower_last;
unsigned idx;
@@ -653,7 +655,7 @@
ret = -EINVAL;
pos = kbuf;
new_map.nr_extents = 0;
- for (;pos; pos = next_line) {
+ for (; pos; pos = next_line) {
extent = &new_map.extent[new_map.nr_extents];
/* Find the end of line and ensure I don't look past it */
@@ -687,13 +689,16 @@
/* Verify we have been given valid starting values */
if ((extent->first == (u32) -1) ||
- (extent->lower_first == (u32) -1 ))
+ (extent->lower_first == (u32) -1))
goto out;
- /* Verify count is not zero and does not cause the extent to wrap */
+ /* Verify count is not zero and does not cause the
+ * extent to wrap
+ */
if ((extent->first + extent->count) <= extent->first)
goto out;
- if ((extent->lower_first + extent->count) <= extent->lower_first)
+ if ((extent->lower_first + extent->count) <=
+ extent->lower_first)
goto out;
/* Do the ranges in extent overlap any previous extents? */
@@ -751,7 +756,8 @@
return ret;
}
-ssize_t proc_uid_map_write(struct file *file, const char __user *buf, size_t size, loff_t *ppos)
+ssize_t proc_uid_map_write(struct file *file, const char __user *buf,
+ size_t size, loff_t *ppos)
{
struct seq_file *seq = file->private_data;
struct user_namespace *ns = seq->private;
@@ -767,7 +773,8 @@
&ns->uid_map, &ns->parent->uid_map);
}
-ssize_t proc_gid_map_write(struct file *file, const char __user *buf, size_t size, loff_t *ppos)
+ssize_t proc_gid_map_write(struct file *file, const char __user *buf,
+ size_t size, loff_t *ppos)
{
struct seq_file *seq = file->private_data;
struct user_namespace *ns = seq->private;
@@ -783,7 +790,8 @@
&ns->gid_map, &ns->parent->gid_map);
}
-ssize_t proc_projid_map_write(struct file *file, const char __user *buf, size_t size, loff_t *ppos)
+ssize_t proc_projid_map_write(struct file *file, const char __user *buf,
+ size_t size, loff_t *ppos)
{
struct seq_file *seq = file->private_data;
struct user_namespace *ns = seq->private;
@@ -800,7 +808,7 @@
&ns->projid_map, &ns->parent->projid_map);
}
-static bool new_idmap_permitted(const struct file *file,
+static bool new_idmap_permitted(const struct file *file,
struct user_namespace *ns, int cap_setid,
struct uid_gid_map *new_map)
{
@@ -811,8 +819,7 @@
kuid_t uid = make_kuid(ns->parent, id);
if (uid_eq(uid, file->f_cred->fsuid))
return true;
- }
- else if (cap_setid == CAP_SETGID) {
+ } else if (cap_setid == CAP_SETGID) {
kgid_t gid = make_kgid(ns->parent, id);
if (gid_eq(gid, file->f_cred->fsgid))
return true;
diff --git a/kernel/utsname_sysctl.c b/kernel/utsname_sysctl.c
index 6fbe811..c8eac43 100644
--- a/kernel/utsname_sysctl.c
+++ b/kernel/utsname_sysctl.c
@@ -17,7 +17,7 @@
#ifdef CONFIG_PROC_SYSCTL
-static void *get_uts(ctl_table *table, int write)
+static void *get_uts(struct ctl_table *table, int write)
{
char *which = table->data;
struct uts_namespace *uts_ns;
@@ -32,7 +32,7 @@
return which;
}
-static void put_uts(ctl_table *table, int write, void *which)
+static void put_uts(struct ctl_table *table, int write, void *which)
{
if (!write)
up_read(&uts_sem);
@@ -44,7 +44,7 @@
* Special case of dostring for the UTS structure. This has locks
* to observe. Should this be in kernel/sys.c ????
*/
-static int proc_do_uts_string(ctl_table *table, int write,
+static int proc_do_uts_string(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
struct ctl_table uts_table;
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index a4bab46..6203d29 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -65,15 +65,12 @@
* be executing on any CPU. The pool behaves as an unbound one.
*
* Note that DISASSOCIATED should be flipped only while holding
- * manager_mutex to avoid changing binding state while
- * create_worker() is in progress.
+ * attach_mutex to avoid changing binding state while
+ * worker_attach_to_pool() is in progress.
*/
- POOL_MANAGE_WORKERS = 1 << 0, /* need to manage workers */
POOL_DISASSOCIATED = 1 << 2, /* cpu can't serve workers */
- POOL_FREEZING = 1 << 3, /* freeze in progress */
/* worker flags */
- WORKER_STARTED = 1 << 0, /* started */
WORKER_DIE = 1 << 1, /* die die die */
WORKER_IDLE = 1 << 2, /* is idle */
WORKER_PREP = 1 << 3, /* preparing to run works */
@@ -124,8 +121,7 @@
* cpu or grabbing pool->lock is enough for read access. If
* POOL_DISASSOCIATED is set, it's identical to L.
*
- * MG: pool->manager_mutex and pool->lock protected. Writes require both
- * locks. Reads can happen under either lock.
+ * A: pool->attach_mutex protected.
*
* PL: wq_pool_mutex protected.
*
@@ -163,8 +159,11 @@
/* see manage_workers() for details on the two manager mutexes */
struct mutex manager_arb; /* manager arbitration */
- struct mutex manager_mutex; /* manager exclusion */
- struct idr worker_idr; /* MG: worker IDs and iteration */
+ struct mutex attach_mutex; /* attach/detach exclusion */
+ struct list_head workers; /* A: attached workers */
+ struct completion *detach_completion; /* all workers detached */
+
+ struct ida worker_ida; /* worker IDs for task name */
struct workqueue_attrs *attrs; /* I: worker attributes */
struct hlist_node hash_node; /* PL: unbound_pool_hash node */
@@ -340,16 +339,6 @@
lockdep_is_held(&wq->mutex), \
"sched RCU or wq->mutex should be held")
-#ifdef CONFIG_LOCKDEP
-#define assert_manager_or_pool_lock(pool) \
- WARN_ONCE(debug_locks && \
- !lockdep_is_held(&(pool)->manager_mutex) && \
- !lockdep_is_held(&(pool)->lock), \
- "pool->manager_mutex or ->lock should be held")
-#else
-#define assert_manager_or_pool_lock(pool) do { } while (0)
-#endif
-
#define for_each_cpu_worker_pool(pool, cpu) \
for ((pool) = &per_cpu(cpu_worker_pools, cpu)[0]; \
(pool) < &per_cpu(cpu_worker_pools, cpu)[NR_STD_WORKER_POOLS]; \
@@ -375,17 +364,16 @@
/**
* for_each_pool_worker - iterate through all workers of a worker_pool
* @worker: iteration cursor
- * @wi: integer used for iteration
* @pool: worker_pool to iterate workers of
*
- * This must be called with either @pool->manager_mutex or ->lock held.
+ * This must be called with @pool->attach_mutex.
*
* The if/else clause exists only for the lockdep assertion and can be
* ignored.
*/
-#define for_each_pool_worker(worker, wi, pool) \
- idr_for_each_entry(&(pool)->worker_idr, (worker), (wi)) \
- if (({ assert_manager_or_pool_lock((pool)); false; })) { } \
+#define for_each_pool_worker(worker, pool) \
+ list_for_each_entry((worker), &(pool)->workers, node) \
+ if (({ lockdep_assert_held(&pool->attach_mutex); false; })) { } \
else
/**
@@ -763,13 +751,6 @@
return need_more_worker(pool) && !may_start_working(pool);
}
-/* Do I need to be the manager? */
-static bool need_to_manage_workers(struct worker_pool *pool)
-{
- return need_to_create_worker(pool) ||
- (pool->flags & POOL_MANAGE_WORKERS);
-}
-
/* Do we have too many workers and should some go away? */
static bool too_many_workers(struct worker_pool *pool)
{
@@ -791,8 +772,8 @@
* Wake up functions.
*/
-/* Return the first worker. Safe with preemption disabled */
-static struct worker *first_worker(struct worker_pool *pool)
+/* Return the first idle worker. Safe with preemption disabled */
+static struct worker *first_idle_worker(struct worker_pool *pool)
{
if (unlikely(list_empty(&pool->idle_list)))
return NULL;
@@ -811,7 +792,7 @@
*/
static void wake_up_worker(struct worker_pool *pool)
{
- struct worker *worker = first_worker(pool);
+ struct worker *worker = first_idle_worker(pool);
if (likely(worker))
wake_up_process(worker->task);
@@ -885,7 +866,7 @@
*/
if (atomic_dec_and_test(&pool->nr_running) &&
!list_empty(&pool->worklist))
- to_wakeup = first_worker(pool);
+ to_wakeup = first_idle_worker(pool);
return to_wakeup ? to_wakeup->task : NULL;
}
@@ -1621,70 +1602,6 @@
list_del_init(&worker->entry);
}
-/**
- * worker_maybe_bind_and_lock - try to bind %current to worker_pool and lock it
- * @pool: target worker_pool
- *
- * Bind %current to the cpu of @pool if it is associated and lock @pool.
- *
- * Works which are scheduled while the cpu is online must at least be
- * scheduled to a worker which is bound to the cpu so that if they are
- * flushed from cpu callbacks while cpu is going down, they are
- * guaranteed to execute on the cpu.
- *
- * This function is to be used by unbound workers and rescuers to bind
- * themselves to the target cpu and may race with cpu going down or
- * coming online. kthread_bind() can't be used because it may put the
- * worker to already dead cpu and set_cpus_allowed_ptr() can't be used
- * verbatim as it's best effort and blocking and pool may be
- * [dis]associated in the meantime.
- *
- * This function tries set_cpus_allowed() and locks pool and verifies the
- * binding against %POOL_DISASSOCIATED which is set during
- * %CPU_DOWN_PREPARE and cleared during %CPU_ONLINE, so if the worker
- * enters idle state or fetches works without dropping lock, it can
- * guarantee the scheduling requirement described in the first paragraph.
- *
- * CONTEXT:
- * Might sleep. Called without any lock but returns with pool->lock
- * held.
- *
- * Return:
- * %true if the associated pool is online (@worker is successfully
- * bound), %false if offline.
- */
-static bool worker_maybe_bind_and_lock(struct worker_pool *pool)
-__acquires(&pool->lock)
-{
- while (true) {
- /*
- * The following call may fail, succeed or succeed
- * without actually migrating the task to the cpu if
- * it races with cpu hotunplug operation. Verify
- * against POOL_DISASSOCIATED.
- */
- if (!(pool->flags & POOL_DISASSOCIATED))
- set_cpus_allowed_ptr(current, pool->attrs->cpumask);
-
- spin_lock_irq(&pool->lock);
- if (pool->flags & POOL_DISASSOCIATED)
- return false;
- if (task_cpu(current) == pool->cpu &&
- cpumask_equal(¤t->cpus_allowed, pool->attrs->cpumask))
- return true;
- spin_unlock_irq(&pool->lock);
-
- /*
- * We've raced with CPU hot[un]plug. Give it a breather
- * and retry migration. cond_resched() is required here;
- * otherwise, we might deadlock against cpu_stop trying to
- * bring down the CPU on non-preemptive kernel.
- */
- cpu_relax();
- cond_resched();
- }
-}
-
static struct worker *alloc_worker(void)
{
struct worker *worker;
@@ -1693,6 +1610,7 @@
if (worker) {
INIT_LIST_HEAD(&worker->entry);
INIT_LIST_HEAD(&worker->scheduled);
+ INIT_LIST_HEAD(&worker->node);
/* on creation a worker is in !idle && prep state */
worker->flags = WORKER_PREP;
}
@@ -1700,12 +1618,68 @@
}
/**
+ * worker_attach_to_pool() - attach a worker to a pool
+ * @worker: worker to be attached
+ * @pool: the target pool
+ *
+ * Attach @worker to @pool. Once attached, the %WORKER_UNBOUND flag and
+ * cpu-binding of @worker are kept coordinated with the pool across
+ * cpu-[un]hotplugs.
+ */
+static void worker_attach_to_pool(struct worker *worker,
+ struct worker_pool *pool)
+{
+ mutex_lock(&pool->attach_mutex);
+
+ /*
+ * set_cpus_allowed_ptr() will fail if the cpumask doesn't have any
+ * online CPUs. It'll be re-applied when any of the CPUs come up.
+ */
+ set_cpus_allowed_ptr(worker->task, pool->attrs->cpumask);
+
+ /*
+ * The pool->attach_mutex ensures %POOL_DISASSOCIATED remains
+ * stable across this function. See the comments above the
+ * flag definition for details.
+ */
+ if (pool->flags & POOL_DISASSOCIATED)
+ worker->flags |= WORKER_UNBOUND;
+
+ list_add_tail(&worker->node, &pool->workers);
+
+ mutex_unlock(&pool->attach_mutex);
+}
+
+/**
+ * worker_detach_from_pool() - detach a worker from its pool
+ * @worker: worker which is attached to its pool
+ * @pool: the pool @worker is attached to
+ *
+ * Undo the attaching which had been done in worker_attach_to_pool(). The
+ * caller worker shouldn't access to the pool after detached except it has
+ * other reference to the pool.
+ */
+static void worker_detach_from_pool(struct worker *worker,
+ struct worker_pool *pool)
+{
+ struct completion *detach_completion = NULL;
+
+ mutex_lock(&pool->attach_mutex);
+ list_del(&worker->node);
+ if (list_empty(&pool->workers))
+ detach_completion = pool->detach_completion;
+ mutex_unlock(&pool->attach_mutex);
+
+ if (detach_completion)
+ complete(detach_completion);
+}
+
+/**
* create_worker - create a new workqueue worker
* @pool: pool the new worker will belong to
*
- * Create a new worker which is bound to @pool. The returned worker
- * can be started by calling start_worker() or destroyed using
- * destroy_worker().
+ * Create a new worker which is attached to @pool. The new worker must be
+ * started by start_worker().
*
* CONTEXT:
* Might sleep. Does GFP_KERNEL allocations.
@@ -1719,19 +1693,8 @@
int id = -1;
char id_buf[16];
- lockdep_assert_held(&pool->manager_mutex);
-
- /*
- * ID is needed to determine kthread name. Allocate ID first
- * without installing the pointer.
- */
- idr_preload(GFP_KERNEL);
- spin_lock_irq(&pool->lock);
-
- id = idr_alloc(&pool->worker_idr, NULL, 0, 0, GFP_NOWAIT);
-
- spin_unlock_irq(&pool->lock);
- idr_preload_end();
+ /* ID is needed to determine kthread name */
+ id = ida_simple_get(&pool->worker_ida, 0, 0, GFP_KERNEL);
if (id < 0)
goto fail;
@@ -1758,33 +1721,14 @@
/* prevent userland from meddling with cpumask of workqueue workers */
worker->task->flags |= PF_NO_SETAFFINITY;
- /*
- * set_cpus_allowed_ptr() will fail if the cpumask doesn't have any
- * online CPUs. It'll be re-applied when any of the CPUs come up.
- */
- set_cpus_allowed_ptr(worker->task, pool->attrs->cpumask);
-
- /*
- * The caller is responsible for ensuring %POOL_DISASSOCIATED
- * remains stable across this function. See the comments above the
- * flag definition for details.
- */
- if (pool->flags & POOL_DISASSOCIATED)
- worker->flags |= WORKER_UNBOUND;
-
- /* successful, commit the pointer to idr */
- spin_lock_irq(&pool->lock);
- idr_replace(&pool->worker_idr, worker, worker->id);
- spin_unlock_irq(&pool->lock);
+ /* successful, attach the worker to the pool */
+ worker_attach_to_pool(worker, pool);
return worker;
fail:
- if (id >= 0) {
- spin_lock_irq(&pool->lock);
- idr_remove(&pool->worker_idr, id);
- spin_unlock_irq(&pool->lock);
- }
+ if (id >= 0)
+ ida_simple_remove(&pool->worker_ida, id);
kfree(worker);
return NULL;
}
@@ -1800,7 +1744,6 @@
*/
static void start_worker(struct worker *worker)
{
- worker->flags |= WORKER_STARTED;
worker->pool->nr_workers++;
worker_enter_idle(worker);
wake_up_process(worker->task);
@@ -1818,8 +1761,6 @@
{
struct worker *worker;
- mutex_lock(&pool->manager_mutex);
-
worker = create_worker(pool);
if (worker) {
spin_lock_irq(&pool->lock);
@@ -1827,8 +1768,6 @@
spin_unlock_irq(&pool->lock);
}
- mutex_unlock(&pool->manager_mutex);
-
return worker ? 0 : -ENOMEM;
}
@@ -1836,46 +1775,30 @@
* destroy_worker - destroy a workqueue worker
* @worker: worker to be destroyed
*
- * Destroy @worker and adjust @pool stats accordingly.
+ * Destroy @worker and adjust @pool stats accordingly. The worker should
+ * be idle.
*
* CONTEXT:
- * spin_lock_irq(pool->lock) which is released and regrabbed.
+ * spin_lock_irq(pool->lock).
*/
static void destroy_worker(struct worker *worker)
{
struct worker_pool *pool = worker->pool;
- lockdep_assert_held(&pool->manager_mutex);
lockdep_assert_held(&pool->lock);
/* sanity check frenzy */
if (WARN_ON(worker->current_work) ||
- WARN_ON(!list_empty(&worker->scheduled)))
+ WARN_ON(!list_empty(&worker->scheduled)) ||
+ WARN_ON(!(worker->flags & WORKER_IDLE)))
return;
- if (worker->flags & WORKER_STARTED)
- pool->nr_workers--;
- if (worker->flags & WORKER_IDLE)
- pool->nr_idle--;
-
- /*
- * Once WORKER_DIE is set, the kworker may destroy itself at any
- * point. Pin to ensure the task stays until we're done with it.
- */
- get_task_struct(worker->task);
+ pool->nr_workers--;
+ pool->nr_idle--;
list_del_init(&worker->entry);
worker->flags |= WORKER_DIE;
-
- idr_remove(&pool->worker_idr, worker->id);
-
- spin_unlock_irq(&pool->lock);
-
- kthread_stop(worker->task);
- put_task_struct(worker->task);
- kfree(worker);
-
- spin_lock_irq(&pool->lock);
+ wake_up_process(worker->task);
}
static void idle_worker_timeout(unsigned long __pool)
@@ -1884,7 +1807,7 @@
spin_lock_irq(&pool->lock);
- if (too_many_workers(pool)) {
+ while (too_many_workers(pool)) {
struct worker *worker;
unsigned long expires;
@@ -1892,13 +1815,12 @@
worker = list_entry(pool->idle_list.prev, struct worker, entry);
expires = worker->last_active + IDLE_WORKER_TIMEOUT;
- if (time_before(jiffies, expires))
+ if (time_before(jiffies, expires)) {
mod_timer(&pool->idle_timer, expires);
- else {
- /* it's been idle for too long, wake up manager */
- pool->flags |= POOL_MANAGE_WORKERS;
- wake_up_worker(pool);
+ break;
}
+
+ destroy_worker(worker);
}
spin_unlock_irq(&pool->lock);
@@ -2017,44 +1939,6 @@
}
/**
- * maybe_destroy_worker - destroy workers which have been idle for a while
- * @pool: pool to destroy workers for
- *
- * Destroy @pool workers which have been idle for longer than
- * IDLE_WORKER_TIMEOUT.
- *
- * LOCKING:
- * spin_lock_irq(pool->lock) which may be released and regrabbed
- * multiple times. Called only from manager.
- *
- * Return:
- * %false if no action was taken and pool->lock stayed locked, %true
- * otherwise.
- */
-static bool maybe_destroy_workers(struct worker_pool *pool)
-{
- bool ret = false;
-
- while (too_many_workers(pool)) {
- struct worker *worker;
- unsigned long expires;
-
- worker = list_entry(pool->idle_list.prev, struct worker, entry);
- expires = worker->last_active + IDLE_WORKER_TIMEOUT;
-
- if (time_before(jiffies, expires)) {
- mod_timer(&pool->idle_timer, expires);
- break;
- }
-
- destroy_worker(worker);
- ret = true;
- }
-
- return ret;
-}
-
-/**
* manage_workers - manage worker pool
* @worker: self
*
@@ -2083,8 +1967,6 @@
bool ret = false;
/*
- * Managership is governed by two mutexes - manager_arb and
- * manager_mutex. manager_arb handles arbitration of manager role.
* Anyone who successfully grabs manager_arb wins the arbitration
* and becomes the manager. mutex_trylock() on pool->manager_arb
* failure while holding pool->lock reliably indicates that someone
@@ -2093,40 +1975,12 @@
* grabbing manager_arb is responsible for actually performing
* manager duties. If manager_arb is grabbed and released without
* actual management, the pool may stall indefinitely.
- *
- * manager_mutex is used for exclusion of actual management
- * operations. The holder of manager_mutex can be sure that none
- * of management operations, including creation and destruction of
- * workers, won't take place until the mutex is released. Because
- * manager_mutex doesn't interfere with manager role arbitration,
- * it is guaranteed that the pool's management, while may be
- * delayed, won't be disturbed by someone else grabbing
- * manager_mutex.
*/
if (!mutex_trylock(&pool->manager_arb))
return ret;
- /*
- * With manager arbitration won, manager_mutex would be free in
- * most cases. trylock first without dropping @pool->lock.
- */
- if (unlikely(!mutex_trylock(&pool->manager_mutex))) {
- spin_unlock_irq(&pool->lock);
- mutex_lock(&pool->manager_mutex);
- spin_lock_irq(&pool->lock);
- ret = true;
- }
-
- pool->flags &= ~POOL_MANAGE_WORKERS;
-
- /*
- * Destroy and then create so that may_start_working() is true
- * on return.
- */
- ret |= maybe_destroy_workers(pool);
ret |= maybe_create_worker(pool);
- mutex_unlock(&pool->manager_mutex);
mutex_unlock(&pool->manager_arb);
return ret;
}
@@ -2314,6 +2168,11 @@
spin_unlock_irq(&pool->lock);
WARN_ON_ONCE(!list_empty(&worker->entry));
worker->task->flags &= ~PF_WQ_WORKER;
+
+ set_task_comm(worker->task, "kworker/dying");
+ ida_simple_remove(&pool->worker_ida, worker->id);
+ worker_detach_from_pool(worker, pool);
+ kfree(worker);
return 0;
}
@@ -2361,9 +2220,6 @@
worker_set_flags(worker, WORKER_PREP, false);
sleep:
- if (unlikely(need_to_manage_workers(pool)) && manage_workers(worker))
- goto recheck;
-
/*
* pool->lock is held and there's no work to process and no need to
* manage, sleep. Workers are woken up only while holding
@@ -2440,8 +2296,9 @@
spin_unlock_irq(&wq_mayday_lock);
- /* migrate to the target cpu if possible */
- worker_maybe_bind_and_lock(pool);
+ worker_attach_to_pool(rescuer, pool);
+
+ spin_lock_irq(&pool->lock);
rescuer->pool = pool;
/*
@@ -2454,6 +2311,11 @@
move_linked_works(work, scheduled, &n);
process_scheduled_works(rescuer);
+ spin_unlock_irq(&pool->lock);
+
+ worker_detach_from_pool(rescuer, pool);
+
+ spin_lock_irq(&pool->lock);
/*
* Put the reference grabbed by send_mayday(). @pool won't
@@ -3550,9 +3412,10 @@
(unsigned long)pool);
mutex_init(&pool->manager_arb);
- mutex_init(&pool->manager_mutex);
- idr_init(&pool->worker_idr);
+ mutex_init(&pool->attach_mutex);
+ INIT_LIST_HEAD(&pool->workers);
+ ida_init(&pool->worker_ida);
INIT_HLIST_NODE(&pool->hash_node);
pool->refcnt = 1;
@@ -3567,7 +3430,7 @@
{
struct worker_pool *pool = container_of(rcu, struct worker_pool, rcu);
- idr_destroy(&pool->worker_idr);
+ ida_destroy(&pool->worker_ida);
free_workqueue_attrs(pool->attrs);
kfree(pool);
}
@@ -3585,6 +3448,7 @@
*/
static void put_unbound_pool(struct worker_pool *pool)
{
+ DECLARE_COMPLETION_ONSTACK(detach_completion);
struct worker *worker;
lockdep_assert_held(&wq_pool_mutex);
@@ -3605,18 +3469,24 @@
/*
* Become the manager and destroy all workers. Grabbing
* manager_arb prevents @pool's workers from blocking on
- * manager_mutex.
+ * attach_mutex.
*/
mutex_lock(&pool->manager_arb);
- mutex_lock(&pool->manager_mutex);
- spin_lock_irq(&pool->lock);
- while ((worker = first_worker(pool)))
+ spin_lock_irq(&pool->lock);
+ while ((worker = first_idle_worker(pool)))
destroy_worker(worker);
WARN_ON(pool->nr_workers || pool->nr_idle);
-
spin_unlock_irq(&pool->lock);
- mutex_unlock(&pool->manager_mutex);
+
+ mutex_lock(&pool->attach_mutex);
+ if (!list_empty(&pool->workers))
+ pool->detach_completion = &detach_completion;
+ mutex_unlock(&pool->attach_mutex);
+
+ if (pool->detach_completion)
+ wait_for_completion(pool->detach_completion);
+
mutex_unlock(&pool->manager_arb);
/* shut down the timers */
@@ -3662,9 +3532,6 @@
if (!pool || init_worker_pool(pool) < 0)
goto fail;
- if (workqueue_freezing)
- pool->flags |= POOL_FREEZING;
-
lockdep_set_subclass(&pool->lock, 1); /* see put_pwq() */
copy_workqueue_attrs(pool->attrs, attrs);
@@ -3771,7 +3638,12 @@
spin_lock_irq(&pwq->pool->lock);
- if (!freezable || !(pwq->pool->flags & POOL_FREEZING)) {
+ /*
+ * During [un]freezing, the caller is responsible for ensuring that
+ * this function is called at least once after @workqueue_freezing
+ * is updated and visible.
+ */
+ if (!freezable || !workqueue_freezing) {
pwq->max_active = wq->saved_max_active;
while (!list_empty(&pwq->delayed_works) &&
@@ -4103,17 +3975,13 @@
* Let's determine what needs to be done. If the target cpumask is
* different from wq's, we need to compare it to @pwq's and create
* a new one if they don't match. If the target cpumask equals
- * wq's, the default pwq should be used. If @pwq is already the
- * default one, nothing to do; otherwise, install the default one.
+ * wq's, the default pwq should be used.
*/
if (wq_calc_node_cpumask(wq->unbound_attrs, node, cpu_off, cpumask)) {
if (cpumask_equal(cpumask, pwq->pool->attrs->cpumask))
goto out_unlock;
} else {
- if (pwq == wq->dfl_pwq)
- goto out_unlock;
- else
- goto use_dfl_pwq;
+ goto use_dfl_pwq;
}
mutex_unlock(&wq->mutex);
@@ -4121,8 +3989,8 @@
/* create a new pwq */
pwq = alloc_unbound_pwq(wq, target_attrs);
if (!pwq) {
- pr_warning("workqueue: allocation failed while updating NUMA affinity of \"%s\"\n",
- wq->name);
+ pr_warn("workqueue: allocation failed while updating NUMA affinity of \"%s\"\n",
+ wq->name);
mutex_lock(&wq->mutex);
goto use_dfl_pwq;
}
@@ -4599,28 +4467,27 @@
int cpu = smp_processor_id();
struct worker_pool *pool;
struct worker *worker;
- int wi;
for_each_cpu_worker_pool(pool, cpu) {
WARN_ON_ONCE(cpu != smp_processor_id());
- mutex_lock(&pool->manager_mutex);
+ mutex_lock(&pool->attach_mutex);
spin_lock_irq(&pool->lock);
/*
- * We've blocked all manager operations. Make all workers
+ * We've blocked all attach/detach operations. Make all workers
* unbound and set DISASSOCIATED. Before this, all workers
* except for the ones which are still executing works from
* before the last CPU down must be on the cpu. After
* this, they may become diasporas.
*/
- for_each_pool_worker(worker, wi, pool)
+ for_each_pool_worker(worker, pool)
worker->flags |= WORKER_UNBOUND;
pool->flags |= POOL_DISASSOCIATED;
spin_unlock_irq(&pool->lock);
- mutex_unlock(&pool->manager_mutex);
+ mutex_unlock(&pool->attach_mutex);
/*
* Call schedule() so that we cross rq->lock and thus can
@@ -4660,9 +4527,8 @@
static void rebind_workers(struct worker_pool *pool)
{
struct worker *worker;
- int wi;
- lockdep_assert_held(&pool->manager_mutex);
+ lockdep_assert_held(&pool->attach_mutex);
/*
* Restore CPU affinity of all workers. As all idle workers should
@@ -4671,13 +4537,13 @@
* of all workers first and then clear UNBOUND. As we're called
* from CPU_ONLINE, the following shouldn't fail.
*/
- for_each_pool_worker(worker, wi, pool)
+ for_each_pool_worker(worker, pool)
WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task,
pool->attrs->cpumask) < 0);
spin_lock_irq(&pool->lock);
- for_each_pool_worker(worker, wi, pool) {
+ for_each_pool_worker(worker, pool) {
unsigned int worker_flags = worker->flags;
/*
@@ -4729,9 +4595,8 @@
{
static cpumask_t cpumask;
struct worker *worker;
- int wi;
- lockdep_assert_held(&pool->manager_mutex);
+ lockdep_assert_held(&pool->attach_mutex);
/* is @cpu allowed for @pool? */
if (!cpumask_test_cpu(cpu, pool->attrs->cpumask))
@@ -4743,7 +4608,7 @@
return;
/* as we're called from CPU_ONLINE, the following shouldn't fail */
- for_each_pool_worker(worker, wi, pool)
+ for_each_pool_worker(worker, pool)
WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task,
pool->attrs->cpumask) < 0);
}
@@ -4776,7 +4641,7 @@
mutex_lock(&wq_pool_mutex);
for_each_pool(pool, pi) {
- mutex_lock(&pool->manager_mutex);
+ mutex_lock(&pool->attach_mutex);
if (pool->cpu == cpu) {
spin_lock_irq(&pool->lock);
@@ -4788,7 +4653,7 @@
restore_unbound_workers_cpumask(pool, cpu);
}
- mutex_unlock(&pool->manager_mutex);
+ mutex_unlock(&pool->attach_mutex);
}
/* update NUMA affinity of unbound workqueues */
@@ -4887,24 +4752,14 @@
*/
void freeze_workqueues_begin(void)
{
- struct worker_pool *pool;
struct workqueue_struct *wq;
struct pool_workqueue *pwq;
- int pi;
mutex_lock(&wq_pool_mutex);
WARN_ON_ONCE(workqueue_freezing);
workqueue_freezing = true;
- /* set FREEZING */
- for_each_pool(pool, pi) {
- spin_lock_irq(&pool->lock);
- WARN_ON_ONCE(pool->flags & POOL_FREEZING);
- pool->flags |= POOL_FREEZING;
- spin_unlock_irq(&pool->lock);
- }
-
list_for_each_entry(wq, &workqueues, list) {
mutex_lock(&wq->mutex);
for_each_pwq(pwq, wq)
@@ -4974,21 +4829,13 @@
{
struct workqueue_struct *wq;
struct pool_workqueue *pwq;
- struct worker_pool *pool;
- int pi;
mutex_lock(&wq_pool_mutex);
if (!workqueue_freezing)
goto out_unlock;
- /* clear FREEZING */
- for_each_pool(pool, pi) {
- spin_lock_irq(&pool->lock);
- WARN_ON_ONCE(!(pool->flags & POOL_FREEZING));
- pool->flags &= ~POOL_FREEZING;
- spin_unlock_irq(&pool->lock);
- }
+ workqueue_freezing = false;
/* restore max_active and repopulate worklist */
list_for_each_entry(wq, &workqueues, list) {
@@ -4998,7 +4845,6 @@
mutex_unlock(&wq->mutex);
}
- workqueue_freezing = false;
out_unlock:
mutex_unlock(&wq_pool_mutex);
}
diff --git a/kernel/workqueue_internal.h b/kernel/workqueue_internal.h
index 7e2204d..4521587 100644
--- a/kernel/workqueue_internal.h
+++ b/kernel/workqueue_internal.h
@@ -37,6 +37,8 @@
struct task_struct *task; /* I: worker task */
struct worker_pool *pool; /* I: the associated pool */
/* L: for rescuers */
+ struct list_head node; /* A: anchored at pool->workers */
+ /* A: runs through worker->node */
unsigned long last_active; /* L: last active timestamp */
unsigned int flags; /* X: flags */
diff --git a/lib/Makefile b/lib/Makefile
index 0cd7b68..74a32dc 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -148,7 +148,8 @@
obj-$(CONFIG_STMP_DEVICE) += stmp_device.o
-libfdt_files = fdt.o fdt_ro.o fdt_wip.o fdt_rw.o fdt_sw.o fdt_strerror.o
+libfdt_files = fdt.o fdt_ro.o fdt_wip.o fdt_rw.o fdt_sw.o fdt_strerror.o \
+ fdt_empty_tree.o
$(foreach file, $(libfdt_files), \
$(eval CFLAGS_$(file) = -I$(src)/../scripts/dtc/libfdt))
lib-$(CONFIG_LIBFDT) += $(libfdt_files)
diff --git a/lib/fdt_empty_tree.c b/lib/fdt_empty_tree.c
new file mode 100644
index 0000000..5d30c58
--- /dev/null
+++ b/lib/fdt_empty_tree.c
@@ -0,0 +1,2 @@
+#include <linux/libfdt_env.h>
+#include "../scripts/dtc/libfdt/fdt_empty_tree.c"
diff --git a/lib/idr.c b/lib/idr.c
index 2642fa8..39158ab 100644
--- a/lib/idr.c
+++ b/lib/idr.c
@@ -18,12 +18,6 @@
* pointer or what ever, we treat it as a (void *). You can pass this
* id to a user for him to pass back at a later time. You then pass
* that id to this code and it returns your pointer.
-
- * You can release ids at any time. When all ids are released, most of
- * the memory is returned (we keep MAX_IDR_FREE) in a local pool so we
- * don't need to go to the memory "store" during an id allocate, just
- * so you don't need to be too concerned about locking and conflicts
- * with the slab allocator.
*/
#ifndef TEST // to test in user space...
@@ -151,7 +145,7 @@
static inline void free_layer(struct idr *idr, struct idr_layer *p)
{
- if (idr->hint && idr->hint == p)
+ if (idr->hint == p)
RCU_INIT_POINTER(idr->hint, NULL);
call_rcu(&p->rcu_head, idr_layer_rcu_free);
}
@@ -249,7 +243,7 @@
id = (id | ((1 << (IDR_BITS * l)) - 1)) + 1;
/* if already at the top layer, we need to grow */
- if (id >= 1 << (idp->layers * IDR_BITS)) {
+ if (id > idr_max(idp->layers)) {
*starting_id = id;
return -EAGAIN;
}
@@ -562,6 +556,11 @@
if (id < 0)
return;
+ if (id > idr_max(idp->layers)) {
+ idr_remove_warning(id);
+ return;
+ }
+
sub_remove(idp, (idp->layers - 1) * IDR_BITS, id);
if (idp->top && idp->top->count == 1 && (idp->layers > 1) &&
idp->top->ary[0]) {
@@ -579,16 +578,6 @@
bitmap_clear(to_free->bitmap, 0, IDR_SIZE);
free_layer(idp, to_free);
}
- while (idp->id_free_cnt >= MAX_IDR_FREE) {
- p = get_from_free_list(idp);
- /*
- * Note: we don't call the rcu callback here, since the only
- * layers that fall into the freelist are those that have been
- * preallocated.
- */
- kmem_cache_free(idr_layer_cache, p);
- }
- return;
}
EXPORT_SYMBOL(idr_remove);
@@ -809,14 +798,12 @@
p = idp->top;
if (!p)
- return ERR_PTR(-EINVAL);
+ return ERR_PTR(-ENOENT);
- n = (p->layer+1) * IDR_BITS;
+ if (id > idr_max(p->layer + 1))
+ return ERR_PTR(-ENOENT);
- if (id >= (1 << n))
- return ERR_PTR(-EINVAL);
-
- n -= IDR_BITS;
+ n = p->layer * IDR_BITS;
while ((n > 0) && p) {
p = p->ary[(id >> n) & IDR_MASK];
n -= IDR_BITS;
@@ -1027,6 +1014,9 @@
int n;
struct ida_bitmap *bitmap;
+ if (idr_id > idr_max(ida->idr.layers))
+ goto err;
+
/* clear full bits while looking up the leaf idr_layer */
while ((shift > 0) && p) {
n = (idr_id >> shift) & IDR_MASK;
@@ -1042,7 +1032,7 @@
__clear_bit(n, p->bitmap);
bitmap = (void *)p->ary[n];
- if (!test_bit(offset, bitmap->bitmap))
+ if (!bitmap || !test_bit(offset, bitmap->bitmap))
goto err;
/* update bitmap and remove it if empty */
diff --git a/lib/nlattr.c b/lib/nlattr.c
index 0c57787..9c3e85f 100644
--- a/lib/nlattr.c
+++ b/lib/nlattr.c
@@ -203,8 +203,8 @@
}
if (unlikely(rem > 0))
- printk(KERN_WARNING "netlink: %d bytes leftover after parsing "
- "attributes.\n", rem);
+ pr_warn_ratelimited("netlink: %d bytes leftover after parsing attributes in process `%s'.\n",
+ rem, current->comm);
err = 0;
errout:
diff --git a/lib/radix-tree.c b/lib/radix-tree.c
index d648156..3291a8e 100644
--- a/lib/radix-tree.c
+++ b/lib/radix-tree.c
@@ -27,6 +27,7 @@
#include <linux/radix-tree.h>
#include <linux/percpu.h>
#include <linux/slab.h>
+#include <linux/kmemleak.h>
#include <linux/notifier.h>
#include <linux/cpu.h>
#include <linux/string.h>
@@ -200,6 +201,11 @@
rtp->nodes[rtp->nr - 1] = NULL;
rtp->nr--;
}
+ /*
+ * Update the allocation stack trace as this is more useful
+ * for debugging.
+ */
+ kmemleak_update_trace(ret);
}
if (ret == NULL)
ret = kmem_cache_alloc(radix_tree_node_cachep, gfp_mask);
diff --git a/mm/fremap.c b/mm/fremap.c
index 2c5646f..72b8fa3 100644
--- a/mm/fremap.c
+++ b/mm/fremap.c
@@ -149,6 +149,10 @@
int has_write_lock = 0;
vm_flags_t vm_flags = 0;
+ pr_warn_once("%s (%d) uses deprecated remap_file_pages() syscall. "
+ "See Documentation/vm/remap_file_pages.txt.\n",
+ current->comm, current->pid);
+
if (prot)
return err;
/*
diff --git a/mm/hugetlb_cgroup.c b/mm/hugetlb_cgroup.c
index 595d7fd..493f7584 100644
--- a/mm/hugetlb_cgroup.c
+++ b/mm/hugetlb_cgroup.c
@@ -52,7 +52,7 @@
static inline struct hugetlb_cgroup *
parent_hugetlb_cgroup(struct hugetlb_cgroup *h_cg)
{
- return hugetlb_cgroup_from_css(css_parent(&h_cg->css));
+ return hugetlb_cgroup_from_css(h_cg->css.parent);
}
static inline bool hugetlb_cgroup_have_usage(struct hugetlb_cgroup *h_cg)
@@ -181,7 +181,7 @@
again:
rcu_read_lock();
h_cg = hugetlb_cgroup_from_task(current);
- if (!css_tryget(&h_cg->css)) {
+ if (!css_tryget_online(&h_cg->css)) {
rcu_read_unlock();
goto again;
}
@@ -253,15 +253,16 @@
return res_counter_read_u64(&h_cg->hugepage[idx], name);
}
-static int hugetlb_cgroup_write(struct cgroup_subsys_state *css,
- struct cftype *cft, char *buffer)
+static ssize_t hugetlb_cgroup_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
int idx, name, ret;
unsigned long long val;
- struct hugetlb_cgroup *h_cg = hugetlb_cgroup_from_css(css);
+ struct hugetlb_cgroup *h_cg = hugetlb_cgroup_from_css(of_css(of));
- idx = MEMFILE_IDX(cft->private);
- name = MEMFILE_ATTR(cft->private);
+ buf = strstrip(buf);
+ idx = MEMFILE_IDX(of_cft(of)->private);
+ name = MEMFILE_ATTR(of_cft(of)->private);
switch (name) {
case RES_LIMIT:
@@ -271,7 +272,7 @@
break;
}
/* This function does all necessary parse...reuse it */
- ret = res_counter_memparse_write_strategy(buffer, &val);
+ ret = res_counter_memparse_write_strategy(buf, &val);
if (ret)
break;
ret = res_counter_set_limit(&h_cg->hugepage[idx], val);
@@ -280,17 +281,17 @@
ret = -EINVAL;
break;
}
- return ret;
+ return ret ?: nbytes;
}
-static int hugetlb_cgroup_reset(struct cgroup_subsys_state *css,
- unsigned int event)
+static ssize_t hugetlb_cgroup_reset(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
int idx, name, ret = 0;
- struct hugetlb_cgroup *h_cg = hugetlb_cgroup_from_css(css);
+ struct hugetlb_cgroup *h_cg = hugetlb_cgroup_from_css(of_css(of));
- idx = MEMFILE_IDX(event);
- name = MEMFILE_ATTR(event);
+ idx = MEMFILE_IDX(of_cft(of)->private);
+ name = MEMFILE_ATTR(of_cft(of)->private);
switch (name) {
case RES_MAX_USAGE:
@@ -303,7 +304,7 @@
ret = -EINVAL;
break;
}
- return ret;
+ return ret ?: nbytes;
}
static char *mem_fmt(char *buf, int size, unsigned long hsize)
@@ -331,7 +332,7 @@
snprintf(cft->name, MAX_CFTYPE_NAME, "%s.limit_in_bytes", buf);
cft->private = MEMFILE_PRIVATE(idx, RES_LIMIT);
cft->read_u64 = hugetlb_cgroup_read_u64;
- cft->write_string = hugetlb_cgroup_write;
+ cft->write = hugetlb_cgroup_write;
/* Add the usage file */
cft = &h->cgroup_files[1];
@@ -343,14 +344,14 @@
cft = &h->cgroup_files[2];
snprintf(cft->name, MAX_CFTYPE_NAME, "%s.max_usage_in_bytes", buf);
cft->private = MEMFILE_PRIVATE(idx, RES_MAX_USAGE);
- cft->trigger = hugetlb_cgroup_reset;
+ cft->write = hugetlb_cgroup_reset;
cft->read_u64 = hugetlb_cgroup_read_u64;
/* Add the failcntfile */
cft = &h->cgroup_files[3];
snprintf(cft->name, MAX_CFTYPE_NAME, "%s.failcnt", buf);
cft->private = MEMFILE_PRIVATE(idx, RES_FAILCNT);
- cft->trigger = hugetlb_cgroup_reset;
+ cft->write = hugetlb_cgroup_reset;
cft->read_u64 = hugetlb_cgroup_read_u64;
/* NULL terminate the last cft */
diff --git a/mm/kmemleak-test.c b/mm/kmemleak-test.c
index ff0d977..dcdcadb 100644
--- a/mm/kmemleak-test.c
+++ b/mm/kmemleak-test.c
@@ -18,6 +18,8 @@
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
+#define pr_fmt(fmt) "kmemleak: " fmt
+
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
@@ -50,25 +52,25 @@
printk(KERN_INFO "Kmemleak testing\n");
/* make some orphan objects */
- pr_info("kmemleak: kmalloc(32) = %p\n", kmalloc(32, GFP_KERNEL));
- pr_info("kmemleak: kmalloc(32) = %p\n", kmalloc(32, GFP_KERNEL));
- pr_info("kmemleak: kmalloc(1024) = %p\n", kmalloc(1024, GFP_KERNEL));
- pr_info("kmemleak: kmalloc(1024) = %p\n", kmalloc(1024, GFP_KERNEL));
- pr_info("kmemleak: kmalloc(2048) = %p\n", kmalloc(2048, GFP_KERNEL));
- pr_info("kmemleak: kmalloc(2048) = %p\n", kmalloc(2048, GFP_KERNEL));
- pr_info("kmemleak: kmalloc(4096) = %p\n", kmalloc(4096, GFP_KERNEL));
- pr_info("kmemleak: kmalloc(4096) = %p\n", kmalloc(4096, GFP_KERNEL));
+ pr_info("kmalloc(32) = %p\n", kmalloc(32, GFP_KERNEL));
+ pr_info("kmalloc(32) = %p\n", kmalloc(32, GFP_KERNEL));
+ pr_info("kmalloc(1024) = %p\n", kmalloc(1024, GFP_KERNEL));
+ pr_info("kmalloc(1024) = %p\n", kmalloc(1024, GFP_KERNEL));
+ pr_info("kmalloc(2048) = %p\n", kmalloc(2048, GFP_KERNEL));
+ pr_info("kmalloc(2048) = %p\n", kmalloc(2048, GFP_KERNEL));
+ pr_info("kmalloc(4096) = %p\n", kmalloc(4096, GFP_KERNEL));
+ pr_info("kmalloc(4096) = %p\n", kmalloc(4096, GFP_KERNEL));
#ifndef CONFIG_MODULES
- pr_info("kmemleak: kmem_cache_alloc(files_cachep) = %p\n",
+ pr_info("kmem_cache_alloc(files_cachep) = %p\n",
kmem_cache_alloc(files_cachep, GFP_KERNEL));
- pr_info("kmemleak: kmem_cache_alloc(files_cachep) = %p\n",
+ pr_info("kmem_cache_alloc(files_cachep) = %p\n",
kmem_cache_alloc(files_cachep, GFP_KERNEL));
#endif
- pr_info("kmemleak: vmalloc(64) = %p\n", vmalloc(64));
- pr_info("kmemleak: vmalloc(64) = %p\n", vmalloc(64));
- pr_info("kmemleak: vmalloc(64) = %p\n", vmalloc(64));
- pr_info("kmemleak: vmalloc(64) = %p\n", vmalloc(64));
- pr_info("kmemleak: vmalloc(64) = %p\n", vmalloc(64));
+ pr_info("vmalloc(64) = %p\n", vmalloc(64));
+ pr_info("vmalloc(64) = %p\n", vmalloc(64));
+ pr_info("vmalloc(64) = %p\n", vmalloc(64));
+ pr_info("vmalloc(64) = %p\n", vmalloc(64));
+ pr_info("vmalloc(64) = %p\n", vmalloc(64));
/*
* Add elements to a list. They should only appear as orphan
@@ -76,7 +78,7 @@
*/
for (i = 0; i < 10; i++) {
elem = kzalloc(sizeof(*elem), GFP_KERNEL);
- pr_info("kmemleak: kzalloc(sizeof(*elem)) = %p\n", elem);
+ pr_info("kzalloc(sizeof(*elem)) = %p\n", elem);
if (!elem)
return -ENOMEM;
INIT_LIST_HEAD(&elem->list);
@@ -85,7 +87,7 @@
for_each_possible_cpu(i) {
per_cpu(kmemleak_test_pointer, i) = kmalloc(129, GFP_KERNEL);
- pr_info("kmemleak: kmalloc(129) = %p\n",
+ pr_info("kmalloc(129) = %p\n",
per_cpu(kmemleak_test_pointer, i));
}
diff --git a/mm/kmemleak.c b/mm/kmemleak.c
index 736ade3..3cda50c 100644
--- a/mm/kmemleak.c
+++ b/mm/kmemleak.c
@@ -387,7 +387,7 @@
pr_notice(" min_count = %d\n", object->min_count);
pr_notice(" count = %d\n", object->count);
pr_notice(" flags = 0x%lx\n", object->flags);
- pr_notice(" checksum = %d\n", object->checksum);
+ pr_notice(" checksum = %u\n", object->checksum);
pr_notice(" backtrace:\n");
print_stack_trace(&trace, 4);
}
@@ -990,6 +990,40 @@
EXPORT_SYMBOL_GPL(kmemleak_free_percpu);
/**
+ * kmemleak_update_trace - update object allocation stack trace
+ * @ptr: pointer to beginning of the object
+ *
+ * Override the object allocation stack trace for cases where the actual
+ * allocation place is not always useful.
+ */
+void __ref kmemleak_update_trace(const void *ptr)
+{
+ struct kmemleak_object *object;
+ unsigned long flags;
+
+ pr_debug("%s(0x%p)\n", __func__, ptr);
+
+ if (!kmemleak_enabled || IS_ERR_OR_NULL(ptr))
+ return;
+
+ object = find_and_get_object((unsigned long)ptr, 1);
+ if (!object) {
+#ifdef DEBUG
+ kmemleak_warn("Updating stack trace for unknown object at %p\n",
+ ptr);
+#endif
+ return;
+ }
+
+ spin_lock_irqsave(&object->lock, flags);
+ object->trace_len = __save_stack_trace(object->trace);
+ spin_unlock_irqrestore(&object->lock, flags);
+
+ put_object(object);
+}
+EXPORT_SYMBOL(kmemleak_update_trace);
+
+/**
* kmemleak_not_leak - mark an allocated object as false positive
* @ptr: pointer to beginning of the object
*
diff --git a/mm/memblock.c b/mm/memblock.c
index 0aa0d2b..6d2f219 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -691,6 +691,7 @@
(unsigned long long)base + size - 1,
(void *)_RET_IP_);
+ kmemleak_free_part(__va(base), size);
return memblock_remove_range(&memblock.reserved, base, size);
}
@@ -1043,9 +1044,14 @@
align = SMP_CACHE_BYTES;
found = memblock_find_in_range_node(size, align, start, end, nid);
- if (found && !memblock_reserve(found, size))
+ if (found && !memblock_reserve(found, size)) {
+ /*
+ * The min_count is set to 0 so that memblock allocations are
+ * never reported as leaks.
+ */
+ kmemleak_alloc(__va(found), size, 0, 0);
return found;
-
+ }
return 0;
}
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index a500cb0..a2c7bcb 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -526,18 +526,14 @@
static inline unsigned short mem_cgroup_id(struct mem_cgroup *memcg)
{
- /*
- * The ID of the root cgroup is 0, but memcg treat 0 as an
- * invalid ID, so we return (cgroup_id + 1).
- */
- return memcg->css.cgroup->id + 1;
+ return memcg->css.id;
}
static inline struct mem_cgroup *mem_cgroup_from_id(unsigned short id)
{
struct cgroup_subsys_state *css;
- css = css_from_id(id - 1, &memory_cgrp_subsys);
+ css = css_from_id(id, &memory_cgrp_subsys);
return mem_cgroup_from_css(css);
}
@@ -570,7 +566,8 @@
memcg = mem_cgroup_from_task(current);
cg_proto = sk->sk_prot->proto_cgroup(memcg);
if (!mem_cgroup_is_root(memcg) &&
- memcg_proto_active(cg_proto) && css_tryget(&memcg->css)) {
+ memcg_proto_active(cg_proto) &&
+ css_tryget_online(&memcg->css)) {
sk->sk_cgrp = cg_proto;
}
rcu_read_unlock();
@@ -676,9 +673,11 @@
static void drain_all_stock_async(struct mem_cgroup *memcg);
static struct mem_cgroup_per_zone *
-mem_cgroup_zoneinfo(struct mem_cgroup *memcg, int nid, int zid)
+mem_cgroup_zone_zoneinfo(struct mem_cgroup *memcg, struct zone *zone)
{
- VM_BUG_ON((unsigned)nid >= nr_node_ids);
+ int nid = zone_to_nid(zone);
+ int zid = zone_idx(zone);
+
return &memcg->nodeinfo[nid]->zoneinfo[zid];
}
@@ -688,12 +687,12 @@
}
static struct mem_cgroup_per_zone *
-page_cgroup_zoneinfo(struct mem_cgroup *memcg, struct page *page)
+mem_cgroup_page_zoneinfo(struct mem_cgroup *memcg, struct page *page)
{
int nid = page_to_nid(page);
int zid = page_zonenum(page);
- return mem_cgroup_zoneinfo(memcg, nid, zid);
+ return &memcg->nodeinfo[nid]->zoneinfo[zid];
}
static struct mem_cgroup_tree_per_zone *
@@ -711,11 +710,9 @@
return &soft_limit_tree.rb_tree_per_node[nid]->rb_tree_per_zone[zid];
}
-static void
-__mem_cgroup_insert_exceeded(struct mem_cgroup *memcg,
- struct mem_cgroup_per_zone *mz,
- struct mem_cgroup_tree_per_zone *mctz,
- unsigned long long new_usage_in_excess)
+static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_zone *mz,
+ struct mem_cgroup_tree_per_zone *mctz,
+ unsigned long long new_usage_in_excess)
{
struct rb_node **p = &mctz->rb_root.rb_node;
struct rb_node *parent = NULL;
@@ -745,10 +742,8 @@
mz->on_tree = true;
}
-static void
-__mem_cgroup_remove_exceeded(struct mem_cgroup *memcg,
- struct mem_cgroup_per_zone *mz,
- struct mem_cgroup_tree_per_zone *mctz)
+static void __mem_cgroup_remove_exceeded(struct mem_cgroup_per_zone *mz,
+ struct mem_cgroup_tree_per_zone *mctz)
{
if (!mz->on_tree)
return;
@@ -756,13 +751,11 @@
mz->on_tree = false;
}
-static void
-mem_cgroup_remove_exceeded(struct mem_cgroup *memcg,
- struct mem_cgroup_per_zone *mz,
- struct mem_cgroup_tree_per_zone *mctz)
+static void mem_cgroup_remove_exceeded(struct mem_cgroup_per_zone *mz,
+ struct mem_cgroup_tree_per_zone *mctz)
{
spin_lock(&mctz->lock);
- __mem_cgroup_remove_exceeded(memcg, mz, mctz);
+ __mem_cgroup_remove_exceeded(mz, mctz);
spin_unlock(&mctz->lock);
}
@@ -772,16 +765,14 @@
unsigned long long excess;
struct mem_cgroup_per_zone *mz;
struct mem_cgroup_tree_per_zone *mctz;
- int nid = page_to_nid(page);
- int zid = page_zonenum(page);
- mctz = soft_limit_tree_from_page(page);
+ mctz = soft_limit_tree_from_page(page);
/*
* Necessary to update all ancestors when hierarchy is used.
* because their event counter is not touched.
*/
for (; memcg; memcg = parent_mem_cgroup(memcg)) {
- mz = mem_cgroup_zoneinfo(memcg, nid, zid);
+ mz = mem_cgroup_page_zoneinfo(memcg, page);
excess = res_counter_soft_limit_excess(&memcg->res);
/*
* We have to update the tree if mz is on RB-tree or
@@ -791,12 +782,12 @@
spin_lock(&mctz->lock);
/* if on-tree, remove it */
if (mz->on_tree)
- __mem_cgroup_remove_exceeded(memcg, mz, mctz);
+ __mem_cgroup_remove_exceeded(mz, mctz);
/*
* Insert again. mz->usage_in_excess will be updated.
* If excess is 0, no tree ops.
*/
- __mem_cgroup_insert_exceeded(memcg, mz, mctz, excess);
+ __mem_cgroup_insert_exceeded(mz, mctz, excess);
spin_unlock(&mctz->lock);
}
}
@@ -804,15 +795,15 @@
static void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg)
{
- int node, zone;
- struct mem_cgroup_per_zone *mz;
struct mem_cgroup_tree_per_zone *mctz;
+ struct mem_cgroup_per_zone *mz;
+ int nid, zid;
- for_each_node(node) {
- for (zone = 0; zone < MAX_NR_ZONES; zone++) {
- mz = mem_cgroup_zoneinfo(memcg, node, zone);
- mctz = soft_limit_tree_node_zone(node, zone);
- mem_cgroup_remove_exceeded(memcg, mz, mctz);
+ for_each_node(nid) {
+ for (zid = 0; zid < MAX_NR_ZONES; zid++) {
+ mz = &memcg->nodeinfo[nid]->zoneinfo[zid];
+ mctz = soft_limit_tree_node_zone(nid, zid);
+ mem_cgroup_remove_exceeded(mz, mctz);
}
}
}
@@ -835,9 +826,9 @@
* we will to add it back at the end of reclaim to its correct
* position in the tree.
*/
- __mem_cgroup_remove_exceeded(mz->memcg, mz, mctz);
+ __mem_cgroup_remove_exceeded(mz, mctz);
if (!res_counter_soft_limit_excess(&mz->memcg->res) ||
- !css_tryget(&mz->memcg->css))
+ !css_tryget_online(&mz->memcg->css))
goto retry;
done:
return mz;
@@ -946,8 +937,7 @@
__this_cpu_add(memcg->stat->nr_page_events, nr_pages);
}
-unsigned long
-mem_cgroup_get_lru_size(struct lruvec *lruvec, enum lru_list lru)
+unsigned long mem_cgroup_get_lru_size(struct lruvec *lruvec, enum lru_list lru)
{
struct mem_cgroup_per_zone *mz;
@@ -955,46 +945,38 @@
return mz->lru_size[lru];
}
-static unsigned long
-mem_cgroup_zone_nr_lru_pages(struct mem_cgroup *memcg, int nid, int zid,
- unsigned int lru_mask)
+static unsigned long mem_cgroup_node_nr_lru_pages(struct mem_cgroup *memcg,
+ int nid,
+ unsigned int lru_mask)
{
- struct mem_cgroup_per_zone *mz;
- enum lru_list lru;
- unsigned long ret = 0;
-
- mz = mem_cgroup_zoneinfo(memcg, nid, zid);
-
- for_each_lru(lru) {
- if (BIT(lru) & lru_mask)
- ret += mz->lru_size[lru];
- }
- return ret;
-}
-
-static unsigned long
-mem_cgroup_node_nr_lru_pages(struct mem_cgroup *memcg,
- int nid, unsigned int lru_mask)
-{
- u64 total = 0;
+ unsigned long nr = 0;
int zid;
- for (zid = 0; zid < MAX_NR_ZONES; zid++)
- total += mem_cgroup_zone_nr_lru_pages(memcg,
- nid, zid, lru_mask);
+ VM_BUG_ON((unsigned)nid >= nr_node_ids);
- return total;
+ for (zid = 0; zid < MAX_NR_ZONES; zid++) {
+ struct mem_cgroup_per_zone *mz;
+ enum lru_list lru;
+
+ for_each_lru(lru) {
+ if (!(BIT(lru) & lru_mask))
+ continue;
+ mz = &memcg->nodeinfo[nid]->zoneinfo[zid];
+ nr += mz->lru_size[lru];
+ }
+ }
+ return nr;
}
static unsigned long mem_cgroup_nr_lru_pages(struct mem_cgroup *memcg,
unsigned int lru_mask)
{
+ unsigned long nr = 0;
int nid;
- u64 total = 0;
for_each_node_state(nid, N_MEMORY)
- total += mem_cgroup_node_nr_lru_pages(memcg, nid, lru_mask);
- return total;
+ nr += mem_cgroup_node_nr_lru_pages(memcg, nid, lru_mask);
+ return nr;
}
static bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg,
@@ -1088,7 +1070,7 @@
if (unlikely(!memcg))
memcg = root_mem_cgroup;
}
- } while (!css_tryget(&memcg->css));
+ } while (!css_tryget_online(&memcg->css));
rcu_read_unlock();
return memcg;
}
@@ -1125,7 +1107,8 @@
*/
if (next_css) {
if ((next_css == &root->css) ||
- ((next_css->flags & CSS_ONLINE) && css_tryget(next_css)))
+ ((next_css->flags & CSS_ONLINE) &&
+ css_tryget_online(next_css)))
return mem_cgroup_from_css(next_css);
prev_css = next_css;
@@ -1171,7 +1154,7 @@
* would be returned all the time.
*/
if (position && position != root &&
- !css_tryget(&position->css))
+ !css_tryget_online(&position->css))
position = NULL;
}
return position;
@@ -1242,11 +1225,9 @@
int uninitialized_var(seq);
if (reclaim) {
- int nid = zone_to_nid(reclaim->zone);
- int zid = zone_idx(reclaim->zone);
struct mem_cgroup_per_zone *mz;
- mz = mem_cgroup_zoneinfo(root, nid, zid);
+ mz = mem_cgroup_zone_zoneinfo(root, reclaim->zone);
iter = &mz->reclaim_iter[reclaim->priority];
if (prev && reclaim->generation != iter->generation) {
iter->last_visited = NULL;
@@ -1353,7 +1334,7 @@
goto out;
}
- mz = mem_cgroup_zoneinfo(memcg, zone_to_nid(zone), zone_idx(zone));
+ mz = mem_cgroup_zone_zoneinfo(memcg, zone);
lruvec = &mz->lruvec;
out:
/*
@@ -1412,7 +1393,7 @@
if (!PageLRU(page) && !PageCgroupUsed(pc) && memcg != root_mem_cgroup)
pc->mem_cgroup = memcg = root_mem_cgroup;
- mz = page_cgroup_zoneinfo(memcg, page);
+ mz = mem_cgroup_page_zoneinfo(memcg, page);
lruvec = &mz->lruvec;
out:
/*
@@ -1550,7 +1531,7 @@
int mem_cgroup_swappiness(struct mem_cgroup *memcg)
{
/* root ? */
- if (!css_parent(&memcg->css))
+ if (mem_cgroup_disabled() || !memcg->css.parent)
return vm_swappiness;
return memcg->swappiness;
@@ -2786,9 +2767,9 @@
/*
* A helper function to get mem_cgroup from ID. must be called under
- * rcu_read_lock(). The caller is responsible for calling css_tryget if
- * the mem_cgroup is used for charging. (dropping refcnt from swap can be
- * called against removed memcg.)
+ * rcu_read_lock(). The caller is responsible for calling
+ * css_tryget_online() if the mem_cgroup is used for charging. (dropping
+ * refcnt from swap can be called against removed memcg.)
*/
static struct mem_cgroup *mem_cgroup_lookup(unsigned short id)
{
@@ -2811,14 +2792,14 @@
lock_page_cgroup(pc);
if (PageCgroupUsed(pc)) {
memcg = pc->mem_cgroup;
- if (memcg && !css_tryget(&memcg->css))
+ if (memcg && !css_tryget_online(&memcg->css))
memcg = NULL;
} else if (PageSwapCache(page)) {
ent.val = page_private(page);
id = lookup_swap_cgroup_id(ent);
rcu_read_lock();
memcg = mem_cgroup_lookup(id);
- if (memcg && !css_tryget(&memcg->css))
+ if (memcg && !css_tryget_online(&memcg->css))
memcg = NULL;
rcu_read_unlock();
}
@@ -3382,7 +3363,7 @@
}
/* The corresponding put will be done in the workqueue. */
- if (!css_tryget(&memcg->css))
+ if (!css_tryget_online(&memcg->css))
goto out;
rcu_read_unlock();
@@ -4142,8 +4123,8 @@
memcg = mem_cgroup_lookup(id);
if (memcg) {
/*
- * We uncharge this because swap is freed.
- * This memcg can be obsolete one. We avoid calling css_tryget
+ * We uncharge this because swap is freed. This memcg can
+ * be obsolete one. We avoid calling css_tryget_online().
*/
if (!mem_cgroup_is_root(memcg))
res_counter_uncharge(&memcg->memsw, PAGE_SIZE);
@@ -4597,7 +4578,7 @@
break;
} while (1);
}
- __mem_cgroup_remove_exceeded(mz->memcg, mz, mctz);
+ __mem_cgroup_remove_exceeded(mz, mctz);
excess = res_counter_soft_limit_excess(&mz->memcg->res);
/*
* One school of thought says that we should not add
@@ -4608,7 +4589,7 @@
* term TODO.
*/
/* If excess == 0, no tree ops */
- __mem_cgroup_insert_exceeded(mz->memcg, mz, mctz, excess);
+ __mem_cgroup_insert_exceeded(mz, mctz, excess);
spin_unlock(&mctz->lock);
css_put(&mz->memcg->css);
loop++;
@@ -4728,18 +4709,28 @@
} while (usage > 0);
}
+/*
+ * Test whether @memcg has children, dead or alive. Note that this
+ * function doesn't care whether @memcg has use_hierarchy enabled and
+ * returns %true if there are child csses according to the cgroup
+ * hierarchy. Testing use_hierarchy is the caller's responsiblity.
+ */
static inline bool memcg_has_children(struct mem_cgroup *memcg)
{
- lockdep_assert_held(&memcg_create_mutex);
+ bool ret;
+
/*
- * The lock does not prevent addition or deletion to the list
- * of children, but it prevents a new child from being
- * initialized based on this parent in css_online(), so it's
- * enough to decide whether hierarchically inherited
- * attributes can still be changed or not.
+ * The lock does not prevent addition or deletion of children, but
+ * it prevents a new child from being initialized based on this
+ * parent in css_online(), so it's enough to decide whether
+ * hierarchically inherited attributes can still be changed or not.
*/
- return memcg->use_hierarchy &&
- !list_empty(&memcg->css.cgroup->children);
+ lockdep_assert_held(&memcg_create_mutex);
+
+ rcu_read_lock();
+ ret = css_next_child(NULL, &memcg->css);
+ rcu_read_unlock();
+ return ret;
}
/*
@@ -4751,11 +4742,6 @@
static int mem_cgroup_force_empty(struct mem_cgroup *memcg)
{
int nr_retries = MEM_CGROUP_RECLAIM_RETRIES;
- struct cgroup *cgrp = memcg->css.cgroup;
-
- /* returns EBUSY if there is a task or if we come here twice. */
- if (cgroup_has_tasks(cgrp) || !list_empty(&cgrp->children))
- return -EBUSY;
/* we call try-to-free pages for make this cgroup empty */
lru_add_drain_all();
@@ -4775,20 +4761,19 @@
}
}
- lru_add_drain();
- mem_cgroup_reparent_charges(memcg);
return 0;
}
-static int mem_cgroup_force_empty_write(struct cgroup_subsys_state *css,
- unsigned int event)
+static ssize_t mem_cgroup_force_empty_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes,
+ loff_t off)
{
- struct mem_cgroup *memcg = mem_cgroup_from_css(css);
+ struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of));
if (mem_cgroup_is_root(memcg))
return -EINVAL;
- return mem_cgroup_force_empty(memcg);
+ return mem_cgroup_force_empty(memcg) ?: nbytes;
}
static u64 mem_cgroup_hierarchy_read(struct cgroup_subsys_state *css,
@@ -4802,7 +4787,7 @@
{
int retval = 0;
struct mem_cgroup *memcg = mem_cgroup_from_css(css);
- struct mem_cgroup *parent_memcg = mem_cgroup_from_css(css_parent(&memcg->css));
+ struct mem_cgroup *parent_memcg = mem_cgroup_from_css(memcg->css.parent);
mutex_lock(&memcg_create_mutex);
@@ -4819,7 +4804,7 @@
*/
if ((!parent_memcg || !parent_memcg->use_hierarchy) &&
(val == 1 || val == 0)) {
- if (list_empty(&memcg->css.cgroup->children))
+ if (!memcg_has_children(memcg))
memcg->use_hierarchy = val;
else
retval = -EBUSY;
@@ -4936,7 +4921,8 @@
* of course permitted.
*/
mutex_lock(&memcg_create_mutex);
- if (cgroup_has_tasks(memcg->css.cgroup) || memcg_has_children(memcg))
+ if (cgroup_has_tasks(memcg->css.cgroup) ||
+ (memcg->use_hierarchy && memcg_has_children(memcg)))
err = -EBUSY;
mutex_unlock(&memcg_create_mutex);
if (err)
@@ -5038,17 +5024,18 @@
* The user of this function is...
* RES_LIMIT.
*/
-static int mem_cgroup_write(struct cgroup_subsys_state *css, struct cftype *cft,
- char *buffer)
+static ssize_t mem_cgroup_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
- struct mem_cgroup *memcg = mem_cgroup_from_css(css);
+ struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of));
enum res_type type;
int name;
unsigned long long val;
int ret;
- type = MEMFILE_TYPE(cft->private);
- name = MEMFILE_ATTR(cft->private);
+ buf = strstrip(buf);
+ type = MEMFILE_TYPE(of_cft(of)->private);
+ name = MEMFILE_ATTR(of_cft(of)->private);
switch (name) {
case RES_LIMIT:
@@ -5057,7 +5044,7 @@
break;
}
/* This function does all necessary parse...reuse it */
- ret = res_counter_memparse_write_strategy(buffer, &val);
+ ret = res_counter_memparse_write_strategy(buf, &val);
if (ret)
break;
if (type == _MEM)
@@ -5070,7 +5057,7 @@
return -EINVAL;
break;
case RES_SOFT_LIMIT:
- ret = res_counter_memparse_write_strategy(buffer, &val);
+ ret = res_counter_memparse_write_strategy(buf, &val);
if (ret)
break;
/*
@@ -5087,7 +5074,7 @@
ret = -EINVAL; /* should be BUG() ? */
break;
}
- return ret;
+ return ret ?: nbytes;
}
static void memcg_get_hierarchical_limit(struct mem_cgroup *memcg,
@@ -5100,8 +5087,8 @@
if (!memcg->use_hierarchy)
goto out;
- while (css_parent(&memcg->css)) {
- memcg = mem_cgroup_from_css(css_parent(&memcg->css));
+ while (memcg->css.parent) {
+ memcg = mem_cgroup_from_css(memcg->css.parent);
if (!memcg->use_hierarchy)
break;
tmp = res_counter_read_u64(&memcg->res, RES_LIMIT);
@@ -5114,14 +5101,15 @@
*memsw_limit = min_memsw_limit;
}
-static int mem_cgroup_reset(struct cgroup_subsys_state *css, unsigned int event)
+static ssize_t mem_cgroup_reset(struct kernfs_open_file *of, char *buf,
+ size_t nbytes, loff_t off)
{
- struct mem_cgroup *memcg = mem_cgroup_from_css(css);
+ struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of));
int name;
enum res_type type;
- type = MEMFILE_TYPE(event);
- name = MEMFILE_ATTR(event);
+ type = MEMFILE_TYPE(of_cft(of)->private);
+ name = MEMFILE_ATTR(of_cft(of)->private);
switch (name) {
case RES_MAX_USAGE:
@@ -5146,7 +5134,7 @@
break;
}
- return 0;
+ return nbytes;
}
static u64 mem_cgroup_move_charge_read(struct cgroup_subsys_state *css,
@@ -5305,7 +5293,7 @@
for_each_online_node(nid)
for (zid = 0; zid < MAX_NR_ZONES; zid++) {
- mz = mem_cgroup_zoneinfo(memcg, nid, zid);
+ mz = &memcg->nodeinfo[nid]->zoneinfo[zid];
rstat = &mz->lruvec.reclaim_stat;
recent_rotated[0] += rstat->recent_rotated[0];
@@ -5339,7 +5327,7 @@
if (val > 100)
return -EINVAL;
- if (css_parent(css))
+ if (css->parent)
memcg->swappiness = val;
else
vm_swappiness = val;
@@ -5676,7 +5664,7 @@
struct mem_cgroup *memcg = mem_cgroup_from_css(css);
/* cannot set to root cgroup and only 0 and 1 are allowed */
- if (!css_parent(css) || !((val == 0) || (val == 1)))
+ if (!css->parent || !((val == 0) || (val == 1)))
return -EINVAL;
memcg->oom_kill_disable = val;
@@ -5722,10 +5710,10 @@
* which is then paired with css_put during uncharge resp. here.
*
* Although this might sound strange as this path is called from
- * css_offline() when the referencemight have dropped down to 0
- * and shouldn't be incremented anymore (css_tryget would fail)
- * we do not have other options because of the kmem allocations
- * lifetime.
+ * css_offline() when the referencemight have dropped down to 0 and
+ * shouldn't be incremented anymore (css_tryget_online() would
+ * fail) we do not have other options because of the kmem
+ * allocations lifetime.
*/
css_get(&memcg->css);
@@ -5844,9 +5832,10 @@
* Input must be in format '<event_fd> <control_fd> <args>'.
* Interpretation of args is defined by control file implementation.
*/
-static int memcg_write_event_control(struct cgroup_subsys_state *css,
- struct cftype *cft, char *buffer)
+static ssize_t memcg_write_event_control(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
+ struct cgroup_subsys_state *css = of_css(of);
struct mem_cgroup *memcg = mem_cgroup_from_css(css);
struct mem_cgroup_event *event;
struct cgroup_subsys_state *cfile_css;
@@ -5857,15 +5846,17 @@
char *endp;
int ret;
- efd = simple_strtoul(buffer, &endp, 10);
+ buf = strstrip(buf);
+
+ efd = simple_strtoul(buf, &endp, 10);
if (*endp != ' ')
return -EINVAL;
- buffer = endp + 1;
+ buf = endp + 1;
- cfd = simple_strtoul(buffer, &endp, 10);
+ cfd = simple_strtoul(buf, &endp, 10);
if ((*endp != ' ') && (*endp != '\0'))
return -EINVAL;
- buffer = endp + 1;
+ buf = endp + 1;
event = kzalloc(sizeof(*event), GFP_KERNEL);
if (!event)
@@ -5933,8 +5924,8 @@
* automatically removed on cgroup destruction but the removal is
* asynchronous, so take an extra ref on @css.
*/
- cfile_css = css_tryget_from_dir(cfile.file->f_dentry->d_parent,
- &memory_cgrp_subsys);
+ cfile_css = css_tryget_online_from_dir(cfile.file->f_dentry->d_parent,
+ &memory_cgrp_subsys);
ret = -EINVAL;
if (IS_ERR(cfile_css))
goto out_put_cfile;
@@ -5943,7 +5934,7 @@
goto out_put_cfile;
}
- ret = event->register_event(memcg, event->eventfd, buffer);
+ ret = event->register_event(memcg, event->eventfd, buf);
if (ret)
goto out_put_css;
@@ -5956,7 +5947,7 @@
fdput(cfile);
fdput(efile);
- return 0;
+ return nbytes;
out_put_css:
css_put(css);
@@ -5981,25 +5972,25 @@
{
.name = "max_usage_in_bytes",
.private = MEMFILE_PRIVATE(_MEM, RES_MAX_USAGE),
- .trigger = mem_cgroup_reset,
+ .write = mem_cgroup_reset,
.read_u64 = mem_cgroup_read_u64,
},
{
.name = "limit_in_bytes",
.private = MEMFILE_PRIVATE(_MEM, RES_LIMIT),
- .write_string = mem_cgroup_write,
+ .write = mem_cgroup_write,
.read_u64 = mem_cgroup_read_u64,
},
{
.name = "soft_limit_in_bytes",
.private = MEMFILE_PRIVATE(_MEM, RES_SOFT_LIMIT),
- .write_string = mem_cgroup_write,
+ .write = mem_cgroup_write,
.read_u64 = mem_cgroup_read_u64,
},
{
.name = "failcnt",
.private = MEMFILE_PRIVATE(_MEM, RES_FAILCNT),
- .trigger = mem_cgroup_reset,
+ .write = mem_cgroup_reset,
.read_u64 = mem_cgroup_read_u64,
},
{
@@ -6008,7 +5999,7 @@
},
{
.name = "force_empty",
- .trigger = mem_cgroup_force_empty_write,
+ .write = mem_cgroup_force_empty_write,
},
{
.name = "use_hierarchy",
@@ -6018,7 +6009,7 @@
},
{
.name = "cgroup.event_control", /* XXX: for compat */
- .write_string = memcg_write_event_control,
+ .write = memcg_write_event_control,
.flags = CFTYPE_NO_PREFIX,
.mode = S_IWUGO,
},
@@ -6051,7 +6042,7 @@
{
.name = "kmem.limit_in_bytes",
.private = MEMFILE_PRIVATE(_KMEM, RES_LIMIT),
- .write_string = mem_cgroup_write,
+ .write = mem_cgroup_write,
.read_u64 = mem_cgroup_read_u64,
},
{
@@ -6062,13 +6053,13 @@
{
.name = "kmem.failcnt",
.private = MEMFILE_PRIVATE(_KMEM, RES_FAILCNT),
- .trigger = mem_cgroup_reset,
+ .write = mem_cgroup_reset,
.read_u64 = mem_cgroup_read_u64,
},
{
.name = "kmem.max_usage_in_bytes",
.private = MEMFILE_PRIVATE(_KMEM, RES_MAX_USAGE),
- .trigger = mem_cgroup_reset,
+ .write = mem_cgroup_reset,
.read_u64 = mem_cgroup_read_u64,
},
#ifdef CONFIG_SLABINFO
@@ -6091,19 +6082,19 @@
{
.name = "memsw.max_usage_in_bytes",
.private = MEMFILE_PRIVATE(_MEMSWAP, RES_MAX_USAGE),
- .trigger = mem_cgroup_reset,
+ .write = mem_cgroup_reset,
.read_u64 = mem_cgroup_read_u64,
},
{
.name = "memsw.limit_in_bytes",
.private = MEMFILE_PRIVATE(_MEMSWAP, RES_LIMIT),
- .write_string = mem_cgroup_write,
+ .write = mem_cgroup_write,
.read_u64 = mem_cgroup_read_u64,
},
{
.name = "memsw.failcnt",
.private = MEMFILE_PRIVATE(_MEMSWAP, RES_FAILCNT),
- .trigger = mem_cgroup_reset,
+ .write = mem_cgroup_reset,
.read_u64 = mem_cgroup_read_u64,
},
{ }, /* terminate */
@@ -6281,9 +6272,9 @@
mem_cgroup_css_online(struct cgroup_subsys_state *css)
{
struct mem_cgroup *memcg = mem_cgroup_from_css(css);
- struct mem_cgroup *parent = mem_cgroup_from_css(css_parent(css));
+ struct mem_cgroup *parent = mem_cgroup_from_css(css->parent);
- if (css->cgroup->id > MEM_CGROUP_ID_MAX)
+ if (css->id > MEM_CGROUP_ID_MAX)
return -ENOSPC;
if (!parent)
@@ -6378,7 +6369,7 @@
/*
* XXX: css_offline() would be where we should reparent all
* memory to prepare the cgroup for destruction. However,
- * memcg does not do css_tryget() and res_counter charging
+ * memcg does not do css_tryget_online() and res_counter charging
* under the same RCU lock region, which means that charging
* could race with offlining. Offlining only happens to
* cgroups with no tasks in them but charges can show up
@@ -6392,9 +6383,9 @@
* lookup_swap_cgroup_id()
* rcu_read_lock()
* mem_cgroup_lookup()
- * css_tryget()
+ * css_tryget_online()
* rcu_read_unlock()
- * disable css_tryget()
+ * disable css_tryget_online()
* call_rcu()
* offline_css()
* reparent_charges()
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 16bc9fa..2849742 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -65,6 +65,8 @@
kernel is not always grateful with that.
*/
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/mempolicy.h>
#include <linux/mm.h>
#include <linux/highmem.h>
@@ -91,6 +93,7 @@
#include <linux/ctype.h>
#include <linux/mm_inline.h>
#include <linux/mmu_notifier.h>
+#include <linux/printk.h>
#include <asm/tlbflush.h>
#include <asm/uaccess.h>
@@ -526,9 +529,13 @@
int nid;
struct page *page;
spinlock_t *ptl;
+ pte_t entry;
ptl = huge_pte_lock(hstate_vma(vma), vma->vm_mm, (pte_t *)pmd);
- page = pte_page(huge_ptep_get((pte_t *)pmd));
+ entry = huge_ptep_get((pte_t *)pmd);
+ if (!pte_present(entry))
+ goto unlock;
+ page = pte_page(entry);
nid = page_to_nid(page);
if (node_isset(nid, *nodes) == !!(flags & MPOL_MF_INVERT))
goto unlock;
@@ -2645,7 +2652,7 @@
node_set(prefer, interleave_nodes);
if (do_set_mempolicy(MPOL_INTERLEAVE, 0, &interleave_nodes))
- printk("numa_policy_init: interleaving failed\n");
+ pr_err("%s: interleaving failed\n", __func__);
check_numabalancing_enable();
}
diff --git a/mm/mempool.c b/mm/mempool.c
index 455d468..e209c98 100644
--- a/mm/mempool.c
+++ b/mm/mempool.c
@@ -10,6 +10,7 @@
#include <linux/mm.h>
#include <linux/slab.h>
+#include <linux/kmemleak.h>
#include <linux/export.h>
#include <linux/mempool.h>
#include <linux/blkdev.h>
@@ -222,6 +223,11 @@
spin_unlock_irqrestore(&pool->lock, flags);
/* paired with rmb in mempool_free(), read comment there */
smp_wmb();
+ /*
+ * Update the allocation stack trace as this is more useful
+ * for debugging.
+ */
+ kmemleak_update_trace(element);
return element;
}
diff --git a/mm/mmap.c b/mm/mmap.c
index ced5efc..129b847 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -6,6 +6,8 @@
* Address space accounting code <alan@lxorguk.ukuu.org.uk>
*/
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/kernel.h>
#include <linux/slab.h>
#include <linux/backing-dev.h>
@@ -37,6 +39,7 @@
#include <linux/sched/sysctl.h>
#include <linux/notifier.h>
#include <linux/memory.h>
+#include <linux/printk.h>
#include <asm/uaccess.h>
#include <asm/cacheflush.h>
@@ -361,20 +364,20 @@
struct vm_area_struct *vma;
vma = rb_entry(nd, struct vm_area_struct, vm_rb);
if (vma->vm_start < prev) {
- printk("vm_start %lx prev %lx\n", vma->vm_start, prev);
+ pr_info("vm_start %lx prev %lx\n", vma->vm_start, prev);
bug = 1;
}
if (vma->vm_start < pend) {
- printk("vm_start %lx pend %lx\n", vma->vm_start, pend);
+ pr_info("vm_start %lx pend %lx\n", vma->vm_start, pend);
bug = 1;
}
if (vma->vm_start > vma->vm_end) {
- printk("vm_end %lx < vm_start %lx\n",
+ pr_info("vm_end %lx < vm_start %lx\n",
vma->vm_end, vma->vm_start);
bug = 1;
}
if (vma->rb_subtree_gap != vma_compute_subtree_gap(vma)) {
- printk("free gap %lx, correct %lx\n",
+ pr_info("free gap %lx, correct %lx\n",
vma->rb_subtree_gap,
vma_compute_subtree_gap(vma));
bug = 1;
@@ -388,7 +391,7 @@
for (nd = pn; nd; nd = rb_prev(nd))
j++;
if (i != j) {
- printk("backwards %d, forwards %d\n", j, i);
+ pr_info("backwards %d, forwards %d\n", j, i);
bug = 1;
}
return bug ? -1 : i;
@@ -423,17 +426,17 @@
i++;
}
if (i != mm->map_count) {
- printk("map_count %d vm_next %d\n", mm->map_count, i);
+ pr_info("map_count %d vm_next %d\n", mm->map_count, i);
bug = 1;
}
if (highest_address != mm->highest_vm_end) {
- printk("mm->highest_vm_end %lx, found %lx\n",
+ pr_info("mm->highest_vm_end %lx, found %lx\n",
mm->highest_vm_end, highest_address);
bug = 1;
}
i = browse_rb(&mm->mm_rb);
if (i != mm->map_count) {
- printk("map_count %d rb %d\n", mm->map_count, i);
+ pr_info("map_count %d rb %d\n", mm->map_count, i);
bug = 1;
}
BUG_ON(bug);
@@ -3280,7 +3283,7 @@
static int __meminit init_reserve_notifier(void)
{
if (register_hotmemory_notifier(&reserve_mem_nb))
- printk("Failed registering memory add/remove notifier for admin reserve");
+ pr_err("Failed registering memory add/remove notifier for admin reserve\n");
return 0;
}
diff --git a/mm/nobootmem.c b/mm/nobootmem.c
index 04a9d94..7ed5860 100644
--- a/mm/nobootmem.c
+++ b/mm/nobootmem.c
@@ -197,7 +197,6 @@
void __init free_bootmem_node(pg_data_t *pgdat, unsigned long physaddr,
unsigned long size)
{
- kmemleak_free_part(__va(physaddr), size);
memblock_free(physaddr, size);
}
@@ -212,7 +211,6 @@
*/
void __init free_bootmem(unsigned long addr, unsigned long size)
{
- kmemleak_free_part(__va(addr), size);
memblock_free(addr, size);
}
diff --git a/mm/nommu.c b/mm/nommu.c
index 85f8d66..b78e3a8 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -13,6 +13,8 @@
* Copyright (c) 2007-2010 Paul Mundt <lethal@linux-sh.org>
*/
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/export.h>
#include <linux/mm.h>
#include <linux/vmacache.h>
@@ -32,6 +34,7 @@
#include <linux/syscalls.h>
#include <linux/audit.h>
#include <linux/sched/sysctl.h>
+#include <linux/printk.h>
#include <asm/uaccess.h>
#include <asm/tlb.h>
@@ -1246,7 +1249,7 @@
return ret;
enomem:
- printk("Allocation of length %lu from process %d (%s) failed\n",
+ pr_err("Allocation of length %lu from process %d (%s) failed\n",
len, current->pid, current->comm);
show_free_areas(0);
return -ENOMEM;
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 533fa60..518e2c3 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -1664,7 +1664,7 @@
/*
* sysctl handler for /proc/sys/vm/dirty_writeback_centisecs
*/
-int dirty_writeback_centisecs_handler(ctl_table *table, int write,
+int dirty_writeback_centisecs_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *length, loff_t *ppos)
{
proc_dointvec(table, write, buffer, length, ppos);
@@ -2380,7 +2380,7 @@
return ret;
}
-int test_set_page_writeback(struct page *page)
+int __test_set_page_writeback(struct page *page, bool keep_write)
{
struct address_space *mapping = page_mapping(page);
int ret;
@@ -2405,9 +2405,10 @@
radix_tree_tag_clear(&mapping->page_tree,
page_index(page),
PAGECACHE_TAG_DIRTY);
- radix_tree_tag_clear(&mapping->page_tree,
- page_index(page),
- PAGECACHE_TAG_TOWRITE);
+ if (!keep_write)
+ radix_tree_tag_clear(&mapping->page_tree,
+ page_index(page),
+ PAGECACHE_TAG_TOWRITE);
spin_unlock_irqrestore(&mapping->tree_lock, flags);
} else {
ret = TestSetPageWriteback(page);
@@ -2418,7 +2419,7 @@
return ret;
}
-EXPORT_SYMBOL(test_set_page_writeback);
+EXPORT_SYMBOL(__test_set_page_writeback);
/*
* Return true if any of the pages in the mapping are marked with the
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a59bdb6..4f59fa2 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3389,7 +3389,7 @@
/*
* sysctl handler for numa_zonelist_order
*/
-int numa_zonelist_order_handler(ctl_table *table, int write,
+int numa_zonelist_order_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *length,
loff_t *ppos)
{
@@ -5805,7 +5805,7 @@
* that we can call two helper functions whenever min_free_kbytes
* changes.
*/
-int min_free_kbytes_sysctl_handler(ctl_table *table, int write,
+int min_free_kbytes_sysctl_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *length, loff_t *ppos)
{
int rc;
@@ -5822,7 +5822,7 @@
}
#ifdef CONFIG_NUMA
-int sysctl_min_unmapped_ratio_sysctl_handler(ctl_table *table, int write,
+int sysctl_min_unmapped_ratio_sysctl_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *length, loff_t *ppos)
{
struct zone *zone;
@@ -5838,7 +5838,7 @@
return 0;
}
-int sysctl_min_slab_ratio_sysctl_handler(ctl_table *table, int write,
+int sysctl_min_slab_ratio_sysctl_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *length, loff_t *ppos)
{
struct zone *zone;
@@ -5864,7 +5864,7 @@
* minimum watermarks. The lowmem reserve ratio can only make sense
* if in function of the boot time zone sizes.
*/
-int lowmem_reserve_ratio_sysctl_handler(ctl_table *table, int write,
+int lowmem_reserve_ratio_sysctl_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *length, loff_t *ppos)
{
proc_dointvec_minmax(table, write, buffer, length, ppos);
@@ -5877,7 +5877,7 @@
* cpu. It is the fraction of total pages in each zone that a hot per cpu
* pagelist can have before it gets flushed back to buddy allocator.
*/
-int percpu_pagelist_fraction_sysctl_handler(ctl_table *table, int write,
+int percpu_pagelist_fraction_sysctl_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *length, loff_t *ppos)
{
struct zone *zone;
diff --git a/mm/rmap.c b/mm/rmap.c
index ea8e20d..bf05fc8 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -1591,10 +1591,9 @@
{
struct anon_vma *root = anon_vma->root;
+ anon_vma_free(anon_vma);
if (root != anon_vma && atomic_dec_and_test(&root->refcount))
anon_vma_free(root);
-
- anon_vma_free(anon_vma);
}
static struct anon_vma *rmap_walk_anon_lock(struct page *page,
diff --git a/mm/slub.c b/mm/slub.c
index fdf0fe4..b2b0473 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1726,7 +1726,7 @@
struct kmem_cache_cpu *c)
{
void *object;
- int searchnode = (node == NUMA_NO_NODE) ? numa_node_id() : node;
+ int searchnode = (node == NUMA_NO_NODE) ? numa_mem_id() : node;
object = get_partial_node(s, get_node(s, searchnode), c, flags);
if (object || node != NUMA_NO_NODE)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 9149444..e01ded3 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -11,6 +11,8 @@
* Multiqueue VM started 5.8.00, Rik van Riel.
*/
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/gfp.h>
@@ -43,6 +45,7 @@
#include <linux/sysctl.h>
#include <linux/oom.h>
#include <linux/prefetch.h>
+#include <linux/printk.h>
#include <asm/tlbflush.h>
#include <asm/div64.h>
@@ -83,6 +86,9 @@
/* Scan (total_size >> priority) pages at once */
int priority;
+ /* anon vs. file LRUs scanning "ratio" */
+ int swappiness;
+
/*
* The memory cgroup that hit its limit and as a result is the
* primary target of this reclaim invocation.
@@ -477,7 +483,7 @@
if (page_has_private(page)) {
if (try_to_free_buffers(page)) {
ClearPageDirty(page);
- printk("%s: orphaned page\n", __func__);
+ pr_info("%s: orphaned page\n", __func__);
return PAGE_CLEAN;
}
}
@@ -1567,20 +1573,18 @@
* If dirty pages are scanned that are not queued for IO, it
* implies that flushers are not keeping up. In this case, flag
* the zone ZONE_TAIL_LRU_DIRTY and kswapd will start writing
- * pages from reclaim context. It will forcibly stall in the
- * next check.
+ * pages from reclaim context.
*/
if (nr_unqueued_dirty == nr_taken)
zone_set_flag(zone, ZONE_TAIL_LRU_DIRTY);
/*
- * In addition, if kswapd scans pages marked marked for
- * immediate reclaim and under writeback (nr_immediate), it
- * implies that pages are cycling through the LRU faster than
+ * If kswapd scans pages marked marked for immediate
+ * reclaim and under writeback (nr_immediate), it implies
+ * that pages are cycling through the LRU faster than
* they are written so also forcibly stall.
*/
- if ((nr_unqueued_dirty == nr_taken || nr_immediate) &&
- current_may_throttle())
+ if (nr_immediate && current_may_throttle())
congestion_wait(BLK_RW_ASYNC, HZ/10);
}
@@ -1845,13 +1849,6 @@
return shrink_inactive_list(nr_to_scan, lruvec, sc, lru);
}
-static int vmscan_swappiness(struct scan_control *sc)
-{
- if (global_reclaim(sc))
- return vm_swappiness;
- return mem_cgroup_swappiness(sc->target_mem_cgroup);
-}
-
enum scan_balance {
SCAN_EQUAL,
SCAN_FRACT,
@@ -1912,7 +1909,7 @@
* using the memory controller's swap limit feature would be
* too expensive.
*/
- if (!global_reclaim(sc) && !vmscan_swappiness(sc)) {
+ if (!global_reclaim(sc) && !sc->swappiness) {
scan_balance = SCAN_FILE;
goto out;
}
@@ -1922,7 +1919,7 @@
* system is close to OOM, scan both anon and file equally
* (unless the swappiness setting disagrees with swapping).
*/
- if (!sc->priority && vmscan_swappiness(sc)) {
+ if (!sc->priority && sc->swappiness) {
scan_balance = SCAN_EQUAL;
goto out;
}
@@ -1965,7 +1962,7 @@
* With swappiness at 100, anonymous and file have the same priority.
* This scanning priority is essentially the inverse of IO cost.
*/
- anon_prio = vmscan_swappiness(sc);
+ anon_prio = sc->swappiness;
file_prio = 200 - anon_prio;
/*
@@ -2265,6 +2262,7 @@
lruvec = mem_cgroup_zone_lruvec(zone, memcg);
+ sc->swappiness = mem_cgroup_swappiness(memcg);
shrink_lruvec(lruvec, sc);
/*
@@ -2731,6 +2729,7 @@
.may_swap = !noswap,
.order = 0,
.priority = 0,
+ .swappiness = mem_cgroup_swappiness(memcg),
.target_mem_cgroup = memcg,
};
struct lruvec *lruvec = mem_cgroup_zone_lruvec(zone, memcg);
@@ -3372,7 +3371,10 @@
}
}
+ tsk->flags &= ~(PF_MEMALLOC | PF_SWAPWRITE | PF_KSWAPD);
current->reclaim_state = NULL;
+ lockdep_clear_current_reclaim_state();
+
return 0;
}
diff --git a/net/batman-adv/multicast.c b/net/batman-adv/multicast.c
index 8c7ca81..96b66fd 100644
--- a/net/batman-adv/multicast.c
+++ b/net/batman-adv/multicast.c
@@ -415,7 +415,7 @@
hlist_for_each_entry_rcu(tmp_orig_node,
&bat_priv->mcast.want_all_ipv4_list,
mcast_want_all_ipv4_node) {
- if (!atomic_inc_not_zero(&orig_node->refcount))
+ if (!atomic_inc_not_zero(&tmp_orig_node->refcount))
continue;
orig_node = tmp_orig_node;
@@ -442,7 +442,7 @@
hlist_for_each_entry_rcu(tmp_orig_node,
&bat_priv->mcast.want_all_ipv6_list,
mcast_want_all_ipv6_node) {
- if (!atomic_inc_not_zero(&orig_node->refcount))
+ if (!atomic_inc_not_zero(&tmp_orig_node->refcount))
continue;
orig_node = tmp_orig_node;
@@ -493,7 +493,7 @@
hlist_for_each_entry_rcu(tmp_orig_node,
&bat_priv->mcast.want_all_unsnoopables_list,
mcast_want_all_unsnoopables_node) {
- if (!atomic_inc_not_zero(&orig_node->refcount))
+ if (!atomic_inc_not_zero(&tmp_orig_node->refcount))
continue;
orig_node = tmp_orig_node;
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index a1e5bb7d..dc4d301 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -7519,9 +7519,9 @@
l2cap_debugfs = debugfs_create_file("l2cap", 0444, bt_debugfs,
NULL, &l2cap_debugfs_fops);
- debugfs_create_u16("l2cap_le_max_credits", 0466, bt_debugfs,
+ debugfs_create_u16("l2cap_le_max_credits", 0644, bt_debugfs,
&le_max_credits);
- debugfs_create_u16("l2cap_le_default_mps", 0466, bt_debugfs,
+ debugfs_create_u16("l2cap_le_default_mps", 0644, bt_debugfs,
&le_default_mps);
bt_6lowpan_init();
diff --git a/net/bridge/br_fdb.c b/net/bridge/br_fdb.c
index 9203d5a..474d36f 100644
--- a/net/bridge/br_fdb.c
+++ b/net/bridge/br_fdb.c
@@ -487,6 +487,7 @@
{
struct hlist_head *head = &br->hash[br_mac_hash(addr, vid)];
struct net_bridge_fdb_entry *fdb;
+ bool fdb_modified = false;
/* some users want to always flood. */
if (hold_time(br) == 0)
@@ -507,10 +508,15 @@
source->dev->name);
} else {
/* fastpath: update of existing entry */
- fdb->dst = source;
+ if (unlikely(source != fdb->dst)) {
+ fdb->dst = source;
+ fdb_modified = true;
+ }
fdb->updated = jiffies;
if (unlikely(added_by_user))
fdb->added_by_user = 1;
+ if (unlikely(fdb_modified))
+ fdb_notify(br, fdb, RTM_NEWNEIGH);
}
} else {
spin_lock(&br->hash_lock);
diff --git a/net/bridge/br_input.c b/net/bridge/br_input.c
index 7985dea..04d6348 100644
--- a/net/bridge/br_input.c
+++ b/net/bridge/br_input.c
@@ -147,8 +147,8 @@
struct net_bridge_port *p = br_port_get_rcu(skb->dev);
u16 vid = 0;
- br_vlan_get_tag(skb, &vid);
- if (p->flags & BR_LEARNING)
+ /* check if vlan is allowed, to avoid spoofing */
+ if (p->flags & BR_LEARNING && br_should_learn(p, skb, &vid))
br_fdb_update(p->br, p, eth_hdr(skb)->h_source, vid, false);
return 0; /* process further */
}
diff --git a/net/bridge/br_private.h b/net/bridge/br_private.h
index 06811d7..59d3a85 100644
--- a/net/bridge/br_private.h
+++ b/net/bridge/br_private.h
@@ -581,6 +581,7 @@
struct sk_buff *skb, u16 *vid);
bool br_allowed_egress(struct net_bridge *br, const struct net_port_vlans *v,
const struct sk_buff *skb);
+bool br_should_learn(struct net_bridge_port *p, struct sk_buff *skb, u16 *vid);
struct sk_buff *br_handle_vlan(struct net_bridge *br,
const struct net_port_vlans *v,
struct sk_buff *skb);
@@ -648,6 +649,12 @@
return true;
}
+static inline bool br_should_learn(struct net_bridge_port *p,
+ struct sk_buff *skb, u16 *vid)
+{
+ return true;
+}
+
static inline struct sk_buff *br_handle_vlan(struct net_bridge *br,
const struct net_port_vlans *v,
struct sk_buff *skb)
diff --git a/net/bridge/br_vlan.c b/net/bridge/br_vlan.c
index 4a37161..5fee2fe 100644
--- a/net/bridge/br_vlan.c
+++ b/net/bridge/br_vlan.c
@@ -241,6 +241,34 @@
return false;
}
+/* Called under RCU */
+bool br_should_learn(struct net_bridge_port *p, struct sk_buff *skb, u16 *vid)
+{
+ struct net_bridge *br = p->br;
+ struct net_port_vlans *v;
+
+ if (!br->vlan_enabled)
+ return true;
+
+ v = rcu_dereference(p->vlan_info);
+ if (!v)
+ return false;
+
+ br_vlan_get_tag(skb, vid);
+ if (!*vid) {
+ *vid = br_get_pvid(v);
+ if (*vid == VLAN_N_VID)
+ return false;
+
+ return true;
+ }
+
+ if (test_bit(*vid, v->vlan_bitmap))
+ return true;
+
+ return false;
+}
+
/* Must be protected by RTNL.
* Must be called with vid in range from 1 to 4094 inclusive.
*/
diff --git a/net/core/dev.c b/net/core/dev.c
index 8b07db3..8908a68 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2283,8 +2283,8 @@
__be16 skb_network_protocol(struct sk_buff *skb, int *depth)
{
+ unsigned int vlan_depth = skb->mac_len;
__be16 type = skb->protocol;
- int vlan_depth = skb->mac_len;
/* Tunnel gso handlers can set protocol to ethernet. */
if (type == htons(ETH_P_TEB)) {
@@ -2297,15 +2297,30 @@
type = eth->h_proto;
}
- while (type == htons(ETH_P_8021Q) || type == htons(ETH_P_8021AD)) {
- struct vlan_hdr *vh;
+ /* if skb->protocol is 802.1Q/AD then the header should already be
+ * present at mac_len - VLAN_HLEN (if mac_len > 0), or at
+ * ETH_HLEN otherwise
+ */
+ if (type == htons(ETH_P_8021Q) || type == htons(ETH_P_8021AD)) {
+ if (vlan_depth) {
+ if (unlikely(WARN_ON(vlan_depth < VLAN_HLEN)))
+ return 0;
+ vlan_depth -= VLAN_HLEN;
+ } else {
+ vlan_depth = ETH_HLEN;
+ }
+ do {
+ struct vlan_hdr *vh;
- if (unlikely(!pskb_may_pull(skb, vlan_depth + VLAN_HLEN)))
- return 0;
+ if (unlikely(!pskb_may_pull(skb,
+ vlan_depth + VLAN_HLEN)))
+ return 0;
- vh = (struct vlan_hdr *)(skb->data + vlan_depth);
- type = vh->h_vlan_encapsulated_proto;
- vlan_depth += VLAN_HLEN;
+ vh = (struct vlan_hdr *)(skb->data + vlan_depth);
+ type = vh->h_vlan_encapsulated_proto;
+ vlan_depth += VLAN_HLEN;
+ } while (type == htons(ETH_P_8021Q) ||
+ type == htons(ETH_P_8021AD));
}
*depth = vlan_depth;
diff --git a/net/core/filter.c b/net/core/filter.c
index 9d79ca0..4aec7b9 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -1559,8 +1559,13 @@
fp->jited = 0;
err = sk_chk_filter(fp->insns, fp->len);
- if (err)
+ if (err) {
+ if (sk != NULL)
+ sk_filter_uncharge(sk, fp);
+ else
+ kfree(fp);
return ERR_PTR(err);
+ }
/* Probe if we can JIT compile the filter and if so, do
* the compilation of the filter.
diff --git a/net/core/netclassid_cgroup.c b/net/core/netclassid_cgroup.c
index 22931e1..30d903b 100644
--- a/net/core/netclassid_cgroup.c
+++ b/net/core/netclassid_cgroup.c
@@ -42,7 +42,7 @@
static int cgrp_css_online(struct cgroup_subsys_state *css)
{
struct cgroup_cls_state *cs = css_cls_state(css);
- struct cgroup_cls_state *parent = css_cls_state(css_parent(css));
+ struct cgroup_cls_state *parent = css_cls_state(css->parent);
if (parent)
cs->classid = parent->classid;
diff --git a/net/core/netprio_cgroup.c b/net/core/netprio_cgroup.c
index 3825f66..2f385b9 100644
--- a/net/core/netprio_cgroup.c
+++ b/net/core/netprio_cgroup.c
@@ -140,7 +140,7 @@
static int cgrp_css_online(struct cgroup_subsys_state *css)
{
- struct cgroup_subsys_state *parent_css = css_parent(css);
+ struct cgroup_subsys_state *parent_css = css->parent;
struct net_device *dev;
int ret = 0;
@@ -185,15 +185,15 @@
return 0;
}
-static int write_priomap(struct cgroup_subsys_state *css, struct cftype *cft,
- char *buffer)
+static ssize_t write_priomap(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
char devname[IFNAMSIZ + 1];
struct net_device *dev;
u32 prio;
int ret;
- if (sscanf(buffer, "%"__stringify(IFNAMSIZ)"s %u", devname, &prio) != 2)
+ if (sscanf(buf, "%"__stringify(IFNAMSIZ)"s %u", devname, &prio) != 2)
return -EINVAL;
dev = dev_get_by_name(&init_net, devname);
@@ -202,11 +202,11 @@
rtnl_lock();
- ret = netprio_set_prio(css, dev, prio);
+ ret = netprio_set_prio(of_css(of), dev, prio);
rtnl_unlock();
dev_put(dev);
- return ret;
+ return ret ?: nbytes;
}
static int update_netprio(const void *v, struct file *file, unsigned n)
@@ -239,7 +239,7 @@
{
.name = "ifpriomap",
.seq_show = read_priomap,
- .write_string = write_priomap,
+ .write = write_priomap,
},
{ } /* terminate */
};
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index d6b46eb..3a26b3b 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -2684,13 +2684,12 @@
bool recovered = !before(tp->snd_una, tp->high_seq);
if (tp->frto) { /* F-RTO RFC5682 sec 3.1 (sack enhanced version). */
- if (flag & FLAG_ORIG_SACK_ACKED) {
- /* Step 3.b. A timeout is spurious if not all data are
- * lost, i.e., never-retransmitted data are (s)acked.
- */
- tcp_try_undo_loss(sk, true);
+ /* Step 3.b. A timeout is spurious if not all data are
+ * lost, i.e., never-retransmitted data are (s)acked.
+ */
+ if (tcp_try_undo_loss(sk, flag & FLAG_ORIG_SACK_ACKED))
return;
- }
+
if (after(tp->snd_nxt, tp->high_seq) &&
(flag & FLAG_DATA_SACKED || is_dupack)) {
tp->frto = 0; /* Loss was real: 2nd part of step 3.a */
diff --git a/net/ipv4/tcp_memcontrol.c b/net/ipv4/tcp_memcontrol.c
index d4f015ad6..f7a2ec3 100644
--- a/net/ipv4/tcp_memcontrol.c
+++ b/net/ipv4/tcp_memcontrol.c
@@ -102,17 +102,19 @@
return 0;
}
-static int tcp_cgroup_write(struct cgroup_subsys_state *css, struct cftype *cft,
- char *buffer)
+static ssize_t tcp_cgroup_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
- struct mem_cgroup *memcg = mem_cgroup_from_css(css);
+ struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of));
unsigned long long val;
int ret = 0;
- switch (cft->private) {
+ buf = strstrip(buf);
+
+ switch (of_cft(of)->private) {
case RES_LIMIT:
/* see memcontrol.c */
- ret = res_counter_memparse_write_strategy(buffer, &val);
+ ret = res_counter_memparse_write_strategy(buf, &val);
if (ret)
break;
ret = tcp_update_limit(memcg, val);
@@ -121,7 +123,7 @@
ret = -EINVAL;
break;
}
- return ret;
+ return ret ?: nbytes;
}
static u64 tcp_read_stat(struct mem_cgroup *memcg, int type, u64 default_val)
@@ -168,17 +170,18 @@
return val;
}
-static int tcp_cgroup_reset(struct cgroup_subsys_state *css, unsigned int event)
+static ssize_t tcp_cgroup_reset(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
struct mem_cgroup *memcg;
struct cg_proto *cg_proto;
- memcg = mem_cgroup_from_css(css);
+ memcg = mem_cgroup_from_css(of_css(of));
cg_proto = tcp_prot.proto_cgroup(memcg);
if (!cg_proto)
- return 0;
+ return nbytes;
- switch (event) {
+ switch (of_cft(of)->private) {
case RES_MAX_USAGE:
res_counter_reset_max(&cg_proto->memory_allocated);
break;
@@ -187,13 +190,13 @@
break;
}
- return 0;
+ return nbytes;
}
static struct cftype tcp_files[] = {
{
.name = "kmem.tcp.limit_in_bytes",
- .write_string = tcp_cgroup_write,
+ .write = tcp_cgroup_write,
.read_u64 = tcp_cgroup_read,
.private = RES_LIMIT,
},
@@ -205,13 +208,13 @@
{
.name = "kmem.tcp.failcnt",
.private = RES_FAILCNT,
- .trigger = tcp_cgroup_reset,
+ .write = tcp_cgroup_reset,
.read_u64 = tcp_cgroup_read,
},
{
.name = "kmem.tcp.max_usage_in_bytes",
.private = RES_MAX_USAGE,
- .trigger = tcp_cgroup_reset,
+ .write = tcp_cgroup_reset,
.read_u64 = tcp_cgroup_read,
},
{ } /* terminate */
diff --git a/net/ipv6/output_core.c b/net/ipv6/output_core.c
index 6313abd..56596ce 100644
--- a/net/ipv6/output_core.c
+++ b/net/ipv6/output_core.c
@@ -12,7 +12,7 @@
{
static atomic_t ipv6_fragmentation_id;
struct in6_addr addr;
- int old, new;
+ int ident;
#if IS_ENABLED(CONFIG_IPV6)
struct inet_peer *peer;
@@ -26,15 +26,10 @@
return;
}
#endif
- do {
- old = atomic_read(&ipv6_fragmentation_id);
- new = old + 1;
- if (!new)
- new = 1;
- } while (atomic_cmpxchg(&ipv6_fragmentation_id, old, new) != old);
+ ident = atomic_inc_return(&ipv6_fragmentation_id);
addr = rt->rt6i_dst.addr;
- addr.s6_addr32[0] ^= (__force __be32)new;
+ addr.s6_addr32[0] ^= (__force __be32)ident;
fhdr->identification = htonl(secure_ipv6_id(addr.s6_addr32));
}
EXPORT_SYMBOL(ipv6_select_ident);
diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c
index 4f26ee4..3d2d2c8 100644
--- a/net/netfilter/ipvs/ip_vs_core.c
+++ b/net/netfilter/ipvs/ip_vs_core.c
@@ -1392,15 +1392,19 @@
if (ipip) {
__be32 info = ic->un.gateway;
+ __u8 type = ic->type;
+ __u8 code = ic->code;
/* Update the MTU */
if (ic->type == ICMP_DEST_UNREACH &&
ic->code == ICMP_FRAG_NEEDED) {
struct ip_vs_dest *dest = cp->dest;
u32 mtu = ntohs(ic->un.frag.mtu);
+ __be16 frag_off = cih->frag_off;
/* Strip outer IP and ICMP, go to IPIP header */
- __skb_pull(skb, ihl + sizeof(_icmph));
+ if (pskb_pull(skb, ihl + sizeof(_icmph)) == NULL)
+ goto ignore_ipip;
offset2 -= ihl + sizeof(_icmph);
skb_reset_network_header(skb);
IP_VS_DBG(12, "ICMP for IPIP %pI4->%pI4: mtu=%u\n",
@@ -1408,7 +1412,7 @@
ipv4_update_pmtu(skb, dev_net(skb->dev),
mtu, 0, 0, 0, 0);
/* Client uses PMTUD? */
- if (!(cih->frag_off & htons(IP_DF)))
+ if (!(frag_off & htons(IP_DF)))
goto ignore_ipip;
/* Prefer the resulting PMTU */
if (dest) {
@@ -1427,12 +1431,13 @@
/* Strip outer IP, ICMP and IPIP, go to IP header of
* original request.
*/
- __skb_pull(skb, offset2);
+ if (pskb_pull(skb, offset2) == NULL)
+ goto ignore_ipip;
skb_reset_network_header(skb);
IP_VS_DBG(12, "Sending ICMP for %pI4->%pI4: t=%u, c=%u, i=%u\n",
&ip_hdr(skb)->saddr, &ip_hdr(skb)->daddr,
- ic->type, ic->code, ntohl(info));
- icmp_send(skb, ic->type, ic->code, info);
+ type, code, ntohl(info));
+ icmp_send(skb, type, code, info);
/* ICMP can be shorter but anyways, account it */
ip_vs_out_stats(cp, skb);
diff --git a/net/netfilter/xt_repldata.h b/net/netfilter/xt_repldata.h
index 6efe4e5..8fd3241 100644
--- a/net/netfilter/xt_repldata.h
+++ b/net/netfilter/xt_repldata.h
@@ -5,23 +5,35 @@
* they serve as the hanging-off data accessed through repl.data[].
*/
+/* tbl has the following structure equivalent, but is C99 compliant:
+ * struct {
+ * struct type##_replace repl;
+ * struct type##_standard entries[nhooks];
+ * struct type##_error term;
+ * } *tbl;
+ */
+
#define xt_alloc_initial_table(type, typ2) ({ \
unsigned int hook_mask = info->valid_hooks; \
unsigned int nhooks = hweight32(hook_mask); \
unsigned int bytes = 0, hooknum = 0, i = 0; \
struct { \
struct type##_replace repl; \
- struct type##_standard entries[nhooks]; \
- struct type##_error term; \
- } *tbl = kzalloc(sizeof(*tbl), GFP_KERNEL); \
+ struct type##_standard entries[]; \
+ } *tbl; \
+ struct type##_error *term; \
+ size_t term_offset = (offsetof(typeof(*tbl), entries[nhooks]) + \
+ __alignof__(*term) - 1) & ~(__alignof__(*term) - 1); \
+ tbl = kzalloc(term_offset + sizeof(*term), GFP_KERNEL); \
if (tbl == NULL) \
return NULL; \
+ term = (struct type##_error *)&(((char *)tbl)[term_offset]); \
strncpy(tbl->repl.name, info->name, sizeof(tbl->repl.name)); \
- tbl->term = (struct type##_error)typ2##_ERROR_INIT; \
+ *term = (struct type##_error)typ2##_ERROR_INIT; \
tbl->repl.valid_hooks = hook_mask; \
tbl->repl.num_entries = nhooks + 1; \
tbl->repl.size = nhooks * sizeof(struct type##_standard) + \
- sizeof(struct type##_error); \
+ sizeof(struct type##_error); \
for (; hook_mask != 0; hook_mask >>= 1, ++hooknum) { \
if (!(hook_mask & 1)) \
continue; \
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index 81dca96..f22757a 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -1373,7 +1373,9 @@
bool __netlink_ns_capable(const struct netlink_skb_parms *nsp,
struct user_namespace *user_ns, int cap)
{
- return sk_ns_capable(nsp->sk, user_ns, cap);
+ return ((nsp->flags & NETLINK_SKB_DST) ||
+ file_ns_capable(nsp->sk->sk_socket->file, user_ns, cap)) &&
+ ns_capable(user_ns, cap);
}
EXPORT_SYMBOL(__netlink_ns_capable);
@@ -2293,6 +2295,7 @@
struct sk_buff *skb;
int err;
struct scm_cookie scm;
+ u32 netlink_skb_flags = 0;
if (msg->msg_flags&MSG_OOB)
return -EOPNOTSUPP;
@@ -2314,6 +2317,7 @@
if ((dst_group || dst_portid) &&
!netlink_allowed(sock, NL_CFG_F_NONROOT_SEND))
goto out;
+ netlink_skb_flags |= NETLINK_SKB_DST;
} else {
dst_portid = nlk->dst_portid;
dst_group = nlk->dst_group;
@@ -2343,6 +2347,7 @@
NETLINK_CB(skb).portid = nlk->portid;
NETLINK_CB(skb).dst_group = dst_group;
NETLINK_CB(skb).creds = siocb->scm->creds;
+ NETLINK_CB(skb).flags = netlink_skb_flags;
err = -EFAULT;
if (memcpy_fromiovec(skb_put(skb, len), msg->msg_iov, len)) {
diff --git a/net/sunrpc/auth_gss/gss_mech_switch.c b/net/sunrpc/auth_gss/gss_mech_switch.c
index 27ce2624..92d5ab9 100644
--- a/net/sunrpc/auth_gss/gss_mech_switch.c
+++ b/net/sunrpc/auth_gss/gss_mech_switch.c
@@ -218,10 +218,8 @@
spin_lock(®istered_mechs_lock);
list_for_each_entry(pos, ®istered_mechs, gm_list) {
- if (!mech_supports_pseudoflavor(pos, pseudoflavor)) {
- module_put(pos->gm_owner);
+ if (!mech_supports_pseudoflavor(pos, pseudoflavor))
continue;
- }
if (try_module_get(pos->gm_owner))
gm = pos;
break;
diff --git a/net/sunrpc/auth_gss/svcauth_gss.c b/net/sunrpc/auth_gss/svcauth_gss.c
index 0f73f45..4ce5ecce 100644
--- a/net/sunrpc/auth_gss/svcauth_gss.c
+++ b/net/sunrpc/auth_gss/svcauth_gss.c
@@ -1503,6 +1503,7 @@
if (unwrap_integ_data(rqstp, &rqstp->rq_arg,
gc->gc_seq, rsci->mechctx))
goto garbage_args;
+ rqstp->rq_auth_slack = RPC_MAX_AUTH_SIZE;
break;
case RPC_GSS_SVC_PRIVACY:
/* placeholders for length and seq. number: */
@@ -1511,6 +1512,7 @@
if (unwrap_priv_data(rqstp, &rqstp->rq_arg,
gc->gc_seq, rsci->mechctx))
goto garbage_args;
+ rqstp->rq_auth_slack = RPC_MAX_AUTH_SIZE * 2;
break;
default:
goto auth_err;
diff --git a/net/sunrpc/cache.c b/net/sunrpc/cache.c
index ae333c1..0663621 100644
--- a/net/sunrpc/cache.c
+++ b/net/sunrpc/cache.c
@@ -374,7 +374,7 @@
}
return;
out:
- printk(KERN_ERR "nfsd: failed to unregister %s cache\n", cd->name);
+ printk(KERN_ERR "RPC: failed to unregister %s cache\n", cd->name);
}
EXPORT_SYMBOL_GPL(sunrpc_destroy_cache_detail);
diff --git a/net/sunrpc/sched.c b/net/sunrpc/sched.c
index 25578af..c0365c1 100644
--- a/net/sunrpc/sched.c
+++ b/net/sunrpc/sched.c
@@ -832,7 +832,8 @@
* @size: requested byte size
*
* To prevent rpciod from hanging, this allocator never sleeps,
- * returning NULL if the request cannot be serviced immediately.
+ * returning NULL and suppressing warning if the request cannot be serviced
+ * immediately.
* The caller can arrange to sleep in a way that is safe for rpciod.
*
* Most requests are 'small' (under 2KiB) and can be serviced from a
@@ -845,7 +846,7 @@
void *rpc_malloc(struct rpc_task *task, size_t size)
{
struct rpc_buffer *buf;
- gfp_t gfp = GFP_NOWAIT;
+ gfp_t gfp = GFP_NOWAIT | __GFP_NOWARN;
if (RPC_IS_SWAPPER(task))
gfp |= __GFP_MEMALLOC;
diff --git a/net/sunrpc/sunrpc.h b/net/sunrpc/sunrpc.h
index 14c9f6d1..f2b7cb5 100644
--- a/net/sunrpc/sunrpc.h
+++ b/net/sunrpc/sunrpc.h
@@ -43,6 +43,19 @@
(task->tk_msg.rpc_proc->p_decode != NULL);
}
+static inline int sock_is_loopback(struct sock *sk)
+{
+ struct dst_entry *dst;
+ int loopback = 0;
+ rcu_read_lock();
+ dst = rcu_dereference(sk->sk_dst_cache);
+ if (dst && dst->dev &&
+ (dst->dev->features & NETIF_F_LOOPBACK))
+ loopback = 1;
+ rcu_read_unlock();
+ return loopback;
+}
+
int svc_send_common(struct socket *sock, struct xdr_buf *xdr,
struct page *headpage, unsigned long headoffset,
struct page *tailpage, unsigned long tailoffset);
diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c
index 06c6ff0..b4737fb 100644
--- a/net/sunrpc/svc_xprt.c
+++ b/net/sunrpc/svc_xprt.c
@@ -597,6 +597,7 @@
}
rqstp->rq_pages[i] = p;
}
+ rqstp->rq_page_end = &rqstp->rq_pages[i];
rqstp->rq_pages[i++] = NULL; /* this might be seen in nfs_read_actor */
/* Make arg->head point to first page and arg->pages point to rest */
@@ -730,6 +731,8 @@
newxpt = xprt->xpt_ops->xpo_accept(xprt);
if (newxpt)
svc_add_new_temp_xprt(serv, newxpt);
+ else
+ module_put(xprt->xpt_class->xcl_owner);
} else if (xprt->xpt_ops->xpo_has_wspace(xprt)) {
/* XPT_DATA|XPT_DEFERRED case: */
dprintk("svc: server %p, pool %u, transport %p, inuse=%d\n",
@@ -793,7 +796,7 @@
clear_bit(XPT_OLD, &xprt->xpt_flags);
- rqstp->rq_secure = svc_port_is_privileged(svc_addr(rqstp));
+ rqstp->rq_secure = xprt->xpt_ops->xpo_secure_port(rqstp);
rqstp->rq_chandle.defer = svc_defer;
if (serv->sv_stats)
diff --git a/net/sunrpc/svcauth.c b/net/sunrpc/svcauth.c
index 2af7b0c..79c0f34 100644
--- a/net/sunrpc/svcauth.c
+++ b/net/sunrpc/svcauth.c
@@ -54,6 +54,8 @@
}
spin_unlock(&authtab_lock);
+ rqstp->rq_auth_slack = 0;
+
rqstp->rq_authop = aops;
return aops->accept(rqstp, authp);
}
diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
index 43bcb46..b507cd3 100644
--- a/net/sunrpc/svcsock.c
+++ b/net/sunrpc/svcsock.c
@@ -400,6 +400,12 @@
release_sock(sock->sk);
#endif
}
+
+static int svc_sock_secure_port(struct svc_rqst *rqstp)
+{
+ return svc_port_is_privileged(svc_addr(rqstp));
+}
+
/*
* INET callback when data has been received on the socket.
*/
@@ -678,6 +684,7 @@
.xpo_prep_reply_hdr = svc_udp_prep_reply_hdr,
.xpo_has_wspace = svc_udp_has_wspace,
.xpo_accept = svc_udp_accept,
+ .xpo_secure_port = svc_sock_secure_port,
};
static struct svc_xprt_class svc_udp_class = {
@@ -842,8 +849,7 @@
* tell us anything. For now just warn about unpriv connections.
*/
if (!svc_port_is_privileged(sin)) {
- dprintk(KERN_WARNING
- "%s: connect from unprivileged port: %s\n",
+ dprintk("%s: connect from unprivileged port: %s\n",
serv->sv_name,
__svc_print_addr(sin, buf, sizeof(buf)));
}
@@ -867,6 +873,10 @@
}
svc_xprt_set_local(&newsvsk->sk_xprt, sin, slen);
+ if (sock_is_loopback(newsock->sk))
+ set_bit(XPT_LOCAL, &newsvsk->sk_xprt.xpt_flags);
+ else
+ clear_bit(XPT_LOCAL, &newsvsk->sk_xprt.xpt_flags);
if (serv->sv_stats)
serv->sv_stats->nettcpconn++;
@@ -1112,6 +1122,7 @@
rqstp->rq_xprt_ctxt = NULL;
rqstp->rq_prot = IPPROTO_TCP;
+ rqstp->rq_local = !!test_bit(XPT_LOCAL, &svsk->sk_xprt.xpt_flags);
p = (__be32 *)rqstp->rq_arg.head[0].iov_base;
calldir = p[1];
@@ -1234,6 +1245,7 @@
.xpo_detach = svc_bc_tcp_sock_detach,
.xpo_free = svc_bc_sock_free,
.xpo_prep_reply_hdr = svc_tcp_prep_reply_hdr,
+ .xpo_secure_port = svc_sock_secure_port,
};
static struct svc_xprt_class svc_tcp_bc_class = {
@@ -1272,6 +1284,7 @@
.xpo_prep_reply_hdr = svc_tcp_prep_reply_hdr,
.xpo_has_wspace = svc_tcp_has_wspace,
.xpo_accept = svc_tcp_accept,
+ .xpo_secure_port = svc_sock_secure_port,
};
static struct svc_xprt_class svc_tcp_class = {
diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c
index dd97ba3..23fb4e7 100644
--- a/net/sunrpc/xdr.c
+++ b/net/sunrpc/xdr.c
@@ -462,6 +462,7 @@
struct kvec *iov = buf->head;
int scratch_len = buf->buflen - buf->page_len - buf->tail[0].iov_len;
+ xdr_set_scratch_buffer(xdr, NULL, 0);
BUG_ON(scratch_len < 0);
xdr->buf = buf;
xdr->iov = iov;
@@ -482,6 +483,73 @@
EXPORT_SYMBOL_GPL(xdr_init_encode);
/**
+ * xdr_commit_encode - Ensure all data is written to buffer
+ * @xdr: pointer to xdr_stream
+ *
+ * We handle encoding across page boundaries by giving the caller a
+ * temporary location to write to, then later copying the data into
+ * place; xdr_commit_encode does that copying.
+ *
+ * Normally the caller doesn't need to call this directly, as the
+ * following xdr_reserve_space will do it. But an explicit call may be
+ * required at the end of encoding, or any other time when the xdr_buf
+ * data might be read.
+ */
+void xdr_commit_encode(struct xdr_stream *xdr)
+{
+ int shift = xdr->scratch.iov_len;
+ void *page;
+
+ if (shift == 0)
+ return;
+ page = page_address(*xdr->page_ptr);
+ memcpy(xdr->scratch.iov_base, page, shift);
+ memmove(page, page + shift, (void *)xdr->p - page);
+ xdr->scratch.iov_len = 0;
+}
+EXPORT_SYMBOL_GPL(xdr_commit_encode);
+
+__be32 *xdr_get_next_encode_buffer(struct xdr_stream *xdr, size_t nbytes)
+{
+ static __be32 *p;
+ int space_left;
+ int frag1bytes, frag2bytes;
+
+ if (nbytes > PAGE_SIZE)
+ return NULL; /* Bigger buffers require special handling */
+ if (xdr->buf->len + nbytes > xdr->buf->buflen)
+ return NULL; /* Sorry, we're totally out of space */
+ frag1bytes = (xdr->end - xdr->p) << 2;
+ frag2bytes = nbytes - frag1bytes;
+ if (xdr->iov)
+ xdr->iov->iov_len += frag1bytes;
+ else
+ xdr->buf->page_len += frag1bytes;
+ xdr->page_ptr++;
+ xdr->iov = NULL;
+ /*
+ * If the last encode didn't end exactly on a page boundary, the
+ * next one will straddle boundaries. Encode into the next
+ * page, then copy it back later in xdr_commit_encode. We use
+ * the "scratch" iov to track any temporarily unused fragment of
+ * space at the end of the previous buffer:
+ */
+ xdr->scratch.iov_base = xdr->p;
+ xdr->scratch.iov_len = frag1bytes;
+ p = page_address(*xdr->page_ptr);
+ /*
+ * Note this is where the next encode will start after we've
+ * shifted this one back:
+ */
+ xdr->p = (void *)p + frag2bytes;
+ space_left = xdr->buf->buflen - xdr->buf->len;
+ xdr->end = (void *)p + min_t(int, space_left, PAGE_SIZE);
+ xdr->buf->page_len += frag2bytes;
+ xdr->buf->len += nbytes;
+ return p;
+}
+
+/**
* xdr_reserve_space - Reserve buffer space for sending
* @xdr: pointer to xdr_stream
* @nbytes: number of bytes to reserve
@@ -495,20 +563,122 @@
__be32 *p = xdr->p;
__be32 *q;
+ xdr_commit_encode(xdr);
/* align nbytes on the next 32-bit boundary */
nbytes += 3;
nbytes &= ~3;
q = p + (nbytes >> 2);
if (unlikely(q > xdr->end || q < p))
- return NULL;
+ return xdr_get_next_encode_buffer(xdr, nbytes);
xdr->p = q;
- xdr->iov->iov_len += nbytes;
+ if (xdr->iov)
+ xdr->iov->iov_len += nbytes;
+ else
+ xdr->buf->page_len += nbytes;
xdr->buf->len += nbytes;
return p;
}
EXPORT_SYMBOL_GPL(xdr_reserve_space);
/**
+ * xdr_truncate_encode - truncate an encode buffer
+ * @xdr: pointer to xdr_stream
+ * @len: new length of buffer
+ *
+ * Truncates the xdr stream, so that xdr->buf->len == len,
+ * and xdr->p points at offset len from the start of the buffer, and
+ * head, tail, and page lengths are adjusted to correspond.
+ *
+ * If this means moving xdr->p to a different buffer, we assume that
+ * that the end pointer should be set to the end of the current page,
+ * except in the case of the head buffer when we assume the head
+ * buffer's current length represents the end of the available buffer.
+ *
+ * This is *not* safe to use on a buffer that already has inlined page
+ * cache pages (as in a zero-copy server read reply), except for the
+ * simple case of truncating from one position in the tail to another.
+ *
+ */
+void xdr_truncate_encode(struct xdr_stream *xdr, size_t len)
+{
+ struct xdr_buf *buf = xdr->buf;
+ struct kvec *head = buf->head;
+ struct kvec *tail = buf->tail;
+ int fraglen;
+ int new, old;
+
+ if (len > buf->len) {
+ WARN_ON_ONCE(1);
+ return;
+ }
+ xdr_commit_encode(xdr);
+
+ fraglen = min_t(int, buf->len - len, tail->iov_len);
+ tail->iov_len -= fraglen;
+ buf->len -= fraglen;
+ if (tail->iov_len && buf->len == len) {
+ xdr->p = tail->iov_base + tail->iov_len;
+ /* xdr->end, xdr->iov should be set already */
+ return;
+ }
+ WARN_ON_ONCE(fraglen);
+ fraglen = min_t(int, buf->len - len, buf->page_len);
+ buf->page_len -= fraglen;
+ buf->len -= fraglen;
+
+ new = buf->page_base + buf->page_len;
+ old = new + fraglen;
+ xdr->page_ptr -= (old >> PAGE_SHIFT) - (new >> PAGE_SHIFT);
+
+ if (buf->page_len && buf->len == len) {
+ xdr->p = page_address(*xdr->page_ptr);
+ xdr->end = (void *)xdr->p + PAGE_SIZE;
+ xdr->p = (void *)xdr->p + (new % PAGE_SIZE);
+ /* xdr->iov should already be NULL */
+ return;
+ }
+ if (fraglen) {
+ xdr->end = head->iov_base + head->iov_len;
+ xdr->page_ptr--;
+ }
+ /* (otherwise assume xdr->end is already set) */
+ head->iov_len = len;
+ buf->len = len;
+ xdr->p = head->iov_base + head->iov_len;
+ xdr->iov = buf->head;
+}
+EXPORT_SYMBOL(xdr_truncate_encode);
+
+/**
+ * xdr_restrict_buflen - decrease available buffer space
+ * @xdr: pointer to xdr_stream
+ * @newbuflen: new maximum number of bytes available
+ *
+ * Adjust our idea of how much space is available in the buffer.
+ * If we've already used too much space in the buffer, returns -1.
+ * If the available space is already smaller than newbuflen, returns 0
+ * and does nothing. Otherwise, adjusts xdr->buf->buflen to newbuflen
+ * and ensures xdr->end is set at most offset newbuflen from the start
+ * of the buffer.
+ */
+int xdr_restrict_buflen(struct xdr_stream *xdr, int newbuflen)
+{
+ struct xdr_buf *buf = xdr->buf;
+ int left_in_this_buf = (void *)xdr->end - (void *)xdr->p;
+ int end_offset = buf->len + left_in_this_buf;
+
+ if (newbuflen < 0 || newbuflen < buf->len)
+ return -1;
+ if (newbuflen > buf->buflen)
+ return 0;
+ if (newbuflen < end_offset)
+ xdr->end = (void *)xdr->end + newbuflen - end_offset;
+ buf->buflen = newbuflen;
+ return 0;
+}
+EXPORT_SYMBOL(xdr_restrict_buflen);
+
+/**
* xdr_write_pages - Insert a list of pages into an XDR buffer for sending
* @xdr: pointer to xdr_stream
* @pages: list of pages
diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c
index 89d051d..c3b2b33 100644
--- a/net/sunrpc/xprt.c
+++ b/net/sunrpc/xprt.c
@@ -71,24 +71,6 @@
static DEFINE_SPINLOCK(xprt_list_lock);
static LIST_HEAD(xprt_list);
-/*
- * The transport code maintains an estimate on the maximum number of out-
- * standing RPC requests, using a smoothed version of the congestion
- * avoidance implemented in 44BSD. This is basically the Van Jacobson
- * congestion algorithm: If a retransmit occurs, the congestion window is
- * halved; otherwise, it is incremented by 1/cwnd when
- *
- * - a reply is received and
- * - a full number of requests are outstanding and
- * - the congestion window hasn't been updated recently.
- */
-#define RPC_CWNDSHIFT (8U)
-#define RPC_CWNDSCALE (1U << RPC_CWNDSHIFT)
-#define RPC_INITCWND RPC_CWNDSCALE
-#define RPC_MAXCWND(xprt) ((xprt)->max_reqs << RPC_CWNDSHIFT)
-
-#define RPCXPRT_CONGESTED(xprt) ((xprt)->cong >= (xprt)->cwnd)
-
/**
* xprt_register_transport - register a transport implementation
* @transport: transport to register
@@ -446,7 +428,15 @@
* @task: recently completed RPC request used to adjust window
* @result: result code of completed RPC request
*
- * We use a time-smoothed congestion estimator to avoid heavy oscillation.
+ * The transport code maintains an estimate on the maximum number of out-
+ * standing RPC requests, using a smoothed version of the congestion
+ * avoidance implemented in 44BSD. This is basically the Van Jacobson
+ * congestion algorithm: If a retransmit occurs, the congestion window is
+ * halved; otherwise, it is incremented by 1/cwnd when
+ *
+ * - a reply is received and
+ * - a full number of requests are outstanding and
+ * - the congestion window hasn't been updated recently.
*/
void xprt_adjust_cwnd(struct rpc_xprt *xprt, struct rpc_task *task, int result)
{
diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c
index 96ead52..693966d 100644
--- a/net/sunrpc/xprtrdma/rpc_rdma.c
+++ b/net/sunrpc/xprtrdma/rpc_rdma.c
@@ -78,8 +78,7 @@
* elements. Segments are then coalesced when registered, if possible
* within the selected memreg mode.
*
- * Note, this routine is never called if the connection's memory
- * registration strategy is 0 (bounce buffers).
+ * Returns positive number of segments converted, or a negative errno.
*/
static int
@@ -102,10 +101,17 @@
page_base = xdrbuf->page_base & ~PAGE_MASK;
p = 0;
while (len && n < nsegs) {
+ if (!ppages[p]) {
+ /* alloc the pagelist for receiving buffer */
+ ppages[p] = alloc_page(GFP_ATOMIC);
+ if (!ppages[p])
+ return -ENOMEM;
+ }
seg[n].mr_page = ppages[p];
seg[n].mr_offset = (void *)(unsigned long) page_base;
seg[n].mr_len = min_t(u32, PAGE_SIZE - page_base, len);
- BUG_ON(seg[n].mr_len > PAGE_SIZE);
+ if (seg[n].mr_len > PAGE_SIZE)
+ return -EIO;
len -= seg[n].mr_len;
++n;
++p;
@@ -114,7 +120,7 @@
/* Message overflows the seg array */
if (len && n == nsegs)
- return 0;
+ return -EIO;
if (xdrbuf->tail[0].iov_len) {
/* the rpcrdma protocol allows us to omit any trailing
@@ -123,7 +129,7 @@
return n;
if (n == nsegs)
/* Tail remains, but we're out of segments */
- return 0;
+ return -EIO;
seg[n].mr_page = NULL;
seg[n].mr_offset = xdrbuf->tail[0].iov_base;
seg[n].mr_len = xdrbuf->tail[0].iov_len;
@@ -164,15 +170,17 @@
* Reply chunk (a counted array):
* N elements:
* 1 - N - HLOO - HLOO - ... - HLOO
+ *
+ * Returns positive RPC/RDMA header size, or negative errno.
*/
-static unsigned int
+static ssize_t
rpcrdma_create_chunks(struct rpc_rqst *rqst, struct xdr_buf *target,
struct rpcrdma_msg *headerp, enum rpcrdma_chunktype type)
{
struct rpcrdma_req *req = rpcr_to_rdmar(rqst);
struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(rqst->rq_xprt);
- int nsegs, nchunks = 0;
+ int n, nsegs, nchunks = 0;
unsigned int pos;
struct rpcrdma_mr_seg *seg = req->rl_segments;
struct rpcrdma_read_chunk *cur_rchunk = NULL;
@@ -198,12 +206,11 @@
pos = target->head[0].iov_len;
nsegs = rpcrdma_convert_iovs(target, pos, type, seg, RPCRDMA_MAX_SEGS);
- if (nsegs == 0)
- return 0;
+ if (nsegs < 0)
+ return nsegs;
do {
- /* bind/register the memory, then build chunk from result. */
- int n = rpcrdma_register_external(seg, nsegs,
+ n = rpcrdma_register_external(seg, nsegs,
cur_wchunk != NULL, r_xprt);
if (n <= 0)
goto out;
@@ -248,10 +255,6 @@
/* success. all failures return above */
req->rl_nchunks = nchunks;
- BUG_ON(nchunks == 0);
- BUG_ON((r_xprt->rx_ia.ri_memreg_strategy == RPCRDMA_FRMR)
- && (nchunks > 3));
-
/*
* finish off header. If write, marshal discrim and nchunks.
*/
@@ -278,8 +281,8 @@
out:
for (pos = 0; nchunks--;)
pos += rpcrdma_deregister_external(
- &req->rl_segments[pos], r_xprt, NULL);
- return 0;
+ &req->rl_segments[pos], r_xprt);
+ return n;
}
/*
@@ -361,6 +364,8 @@
* [1] -- the RPC header/data, marshaled by RPC and the NFS protocol.
* [2] -- optional padding.
* [3] -- if padded, header only in [1] and data here.
+ *
+ * Returns zero on success, otherwise a negative errno.
*/
int
@@ -370,7 +375,8 @@
struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt);
struct rpcrdma_req *req = rpcr_to_rdmar(rqst);
char *base;
- size_t hdrlen, rpclen, padlen;
+ size_t rpclen, padlen;
+ ssize_t hdrlen;
enum rpcrdma_chunktype rtype, wtype;
struct rpcrdma_msg *headerp;
@@ -441,14 +447,10 @@
/* The following simplification is not true forever */
if (rtype != rpcrdma_noch && wtype == rpcrdma_replych)
wtype = rpcrdma_noch;
- BUG_ON(rtype != rpcrdma_noch && wtype != rpcrdma_noch);
-
- if (r_xprt->rx_ia.ri_memreg_strategy == RPCRDMA_BOUNCEBUFFERS &&
- (rtype != rpcrdma_noch || wtype != rpcrdma_noch)) {
- /* forced to "pure inline"? */
- dprintk("RPC: %s: too much data (%d/%d) for inline\n",
- __func__, rqst->rq_rcv_buf.len, rqst->rq_snd_buf.len);
- return -1;
+ if (rtype != rpcrdma_noch && wtype != rpcrdma_noch) {
+ dprintk("RPC: %s: cannot marshal multiple chunk lists\n",
+ __func__);
+ return -EIO;
}
hdrlen = 28; /*sizeof *headerp;*/
@@ -474,8 +476,11 @@
headerp->rm_body.rm_padded.rm_pempty[1] = xdr_zero;
headerp->rm_body.rm_padded.rm_pempty[2] = xdr_zero;
hdrlen += 2 * sizeof(u32); /* extra words in padhdr */
- BUG_ON(wtype != rpcrdma_noch);
-
+ if (wtype != rpcrdma_noch) {
+ dprintk("RPC: %s: invalid chunk list\n",
+ __func__);
+ return -EIO;
+ }
} else {
headerp->rm_body.rm_nochunks.rm_empty[0] = xdr_zero;
headerp->rm_body.rm_nochunks.rm_empty[1] = xdr_zero;
@@ -492,8 +497,7 @@
* on receive. Therefore, we request a reply chunk
* for non-writes wherever feasible and efficient.
*/
- if (wtype == rpcrdma_noch &&
- r_xprt->rx_ia.ri_memreg_strategy > RPCRDMA_REGISTER)
+ if (wtype == rpcrdma_noch)
wtype = rpcrdma_replych;
}
}
@@ -511,9 +515,8 @@
hdrlen = rpcrdma_create_chunks(rqst,
&rqst->rq_rcv_buf, headerp, wtype);
}
-
- if (hdrlen == 0)
- return -1;
+ if (hdrlen < 0)
+ return hdrlen;
dprintk("RPC: %s: %s: hdrlen %zd rpclen %zd padlen %zd"
" headerp 0x%p base 0x%p lkey 0x%x\n",
@@ -680,15 +683,11 @@
rqst->rq_private_buf = rqst->rq_rcv_buf;
}
-/*
- * This function is called when an async event is posted to
- * the connection which changes the connection state. All it
- * does at this point is mark the connection up/down, the rpc
- * timers do the rest.
- */
void
-rpcrdma_conn_func(struct rpcrdma_ep *ep)
+rpcrdma_connect_worker(struct work_struct *work)
{
+ struct rpcrdma_ep *ep =
+ container_of(work, struct rpcrdma_ep, rep_connect_worker.work);
struct rpc_xprt *xprt = ep->rep_xprt;
spin_lock_bh(&xprt->transport_lock);
@@ -705,13 +704,15 @@
}
/*
- * This function is called when memory window unbind which we are waiting
- * for completes. Just use rr_func (zeroed by upcall) to signal completion.
+ * This function is called when an async event is posted to
+ * the connection which changes the connection state. All it
+ * does at this point is mark the connection up/down, the rpc
+ * timers do the rest.
*/
-static void
-rpcrdma_unbind_func(struct rpcrdma_rep *rep)
+void
+rpcrdma_conn_func(struct rpcrdma_ep *ep)
{
- wake_up(&rep->rr_unbind);
+ schedule_delayed_work(&ep->rep_connect_worker, 0);
}
/*
@@ -728,7 +729,8 @@
struct rpc_xprt *xprt = rep->rr_xprt;
struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt);
__be32 *iptr;
- int i, rdmalen, status;
+ int rdmalen, status;
+ unsigned long cwnd;
/* Check status. If bad, signal disconnect and return rep to pool */
if (rep->rr_len == ~0U) {
@@ -783,6 +785,7 @@
/* from here on, the reply is no longer an orphan */
req->rl_reply = rep;
+ xprt->reestablish_timeout = 0;
/* check for expected message types */
/* The order of some of these tests is important. */
@@ -857,26 +860,10 @@
break;
}
- /* If using mw bind, start the deregister process now. */
- /* (Note: if mr_free(), cannot perform it here, in tasklet context) */
- if (req->rl_nchunks) switch (r_xprt->rx_ia.ri_memreg_strategy) {
- case RPCRDMA_MEMWINDOWS:
- for (i = 0; req->rl_nchunks-- > 1;)
- i += rpcrdma_deregister_external(
- &req->rl_segments[i], r_xprt, NULL);
- /* Optionally wait (not here) for unbinds to complete */
- rep->rr_func = rpcrdma_unbind_func;
- (void) rpcrdma_deregister_external(&req->rl_segments[i],
- r_xprt, rep);
- break;
- case RPCRDMA_MEMWINDOWS_ASYNC:
- for (i = 0; req->rl_nchunks--;)
- i += rpcrdma_deregister_external(&req->rl_segments[i],
- r_xprt, NULL);
- break;
- default:
- break;
- }
+ cwnd = xprt->cwnd;
+ xprt->cwnd = atomic_read(&r_xprt->rx_buf.rb_credits) << RPC_CWNDSHIFT;
+ if (xprt->cwnd > cwnd)
+ xprt_release_rqst_cong(rqst->rq_task);
dprintk("RPC: %s: xprt_complete_rqst(0x%p, 0x%p, %d)\n",
__func__, xprt, rqst, status);
diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
index 8d904e4e..8f92a61 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
@@ -1,4 +1,5 @@
/*
+ * Copyright (c) 2014 Open Grid Computing, Inc. All rights reserved.
* Copyright (c) 2005-2006 Network Appliance, Inc. All rights reserved.
*
* This software is available to you under a choice of one of two
@@ -69,7 +70,8 @@
/* Set up the XDR head */
rqstp->rq_arg.head[0].iov_base = page_address(page);
- rqstp->rq_arg.head[0].iov_len = min(byte_count, ctxt->sge[0].length);
+ rqstp->rq_arg.head[0].iov_len =
+ min_t(size_t, byte_count, ctxt->sge[0].length);
rqstp->rq_arg.len = byte_count;
rqstp->rq_arg.buflen = byte_count;
@@ -85,7 +87,7 @@
page = ctxt->pages[sge_no];
put_page(rqstp->rq_pages[sge_no]);
rqstp->rq_pages[sge_no] = page;
- bc -= min(bc, ctxt->sge[sge_no].length);
+ bc -= min_t(u32, bc, ctxt->sge[sge_no].length);
rqstp->rq_arg.buflen += ctxt->sge[sge_no].length;
sge_no++;
}
@@ -113,291 +115,265 @@
rqstp->rq_arg.tail[0].iov_len = 0;
}
-/* Encode a read-chunk-list as an array of IB SGE
- *
- * Assumptions:
- * - chunk[0]->position points to pages[0] at an offset of 0
- * - pages[] is not physically or virtually contiguous and consists of
- * PAGE_SIZE elements.
- *
- * Output:
- * - sge array pointing into pages[] array.
- * - chunk_sge array specifying sge index and count for each
- * chunk in the read list
- *
- */
-static int map_read_chunks(struct svcxprt_rdma *xprt,
- struct svc_rqst *rqstp,
- struct svc_rdma_op_ctxt *head,
- struct rpcrdma_msg *rmsgp,
- struct svc_rdma_req_map *rpl_map,
- struct svc_rdma_req_map *chl_map,
- int ch_count,
- int byte_count)
-{
- int sge_no;
- int sge_bytes;
- int page_off;
- int page_no;
- int ch_bytes;
- int ch_no;
- struct rpcrdma_read_chunk *ch;
-
- sge_no = 0;
- page_no = 0;
- page_off = 0;
- ch = (struct rpcrdma_read_chunk *)&rmsgp->rm_body.rm_chunks[0];
- ch_no = 0;
- ch_bytes = ntohl(ch->rc_target.rs_length);
- head->arg.head[0] = rqstp->rq_arg.head[0];
- head->arg.tail[0] = rqstp->rq_arg.tail[0];
- head->arg.pages = &head->pages[head->count];
- head->hdr_count = head->count; /* save count of hdr pages */
- head->arg.page_base = 0;
- head->arg.page_len = ch_bytes;
- head->arg.len = rqstp->rq_arg.len + ch_bytes;
- head->arg.buflen = rqstp->rq_arg.buflen + ch_bytes;
- head->count++;
- chl_map->ch[0].start = 0;
- while (byte_count) {
- rpl_map->sge[sge_no].iov_base =
- page_address(rqstp->rq_arg.pages[page_no]) + page_off;
- sge_bytes = min_t(int, PAGE_SIZE-page_off, ch_bytes);
- rpl_map->sge[sge_no].iov_len = sge_bytes;
- /*
- * Don't bump head->count here because the same page
- * may be used by multiple SGE.
- */
- head->arg.pages[page_no] = rqstp->rq_arg.pages[page_no];
- rqstp->rq_respages = &rqstp->rq_arg.pages[page_no+1];
- rqstp->rq_next_page = rqstp->rq_respages + 1;
-
- byte_count -= sge_bytes;
- ch_bytes -= sge_bytes;
- sge_no++;
- /*
- * If all bytes for this chunk have been mapped to an
- * SGE, move to the next SGE
- */
- if (ch_bytes == 0) {
- chl_map->ch[ch_no].count =
- sge_no - chl_map->ch[ch_no].start;
- ch_no++;
- ch++;
- chl_map->ch[ch_no].start = sge_no;
- ch_bytes = ntohl(ch->rc_target.rs_length);
- /* If bytes remaining account for next chunk */
- if (byte_count) {
- head->arg.page_len += ch_bytes;
- head->arg.len += ch_bytes;
- head->arg.buflen += ch_bytes;
- }
- }
- /*
- * If this SGE consumed all of the page, move to the
- * next page
- */
- if ((sge_bytes + page_off) == PAGE_SIZE) {
- page_no++;
- page_off = 0;
- /*
- * If there are still bytes left to map, bump
- * the page count
- */
- if (byte_count)
- head->count++;
- } else
- page_off += sge_bytes;
- }
- BUG_ON(byte_count != 0);
- return sge_no;
-}
-
-/* Map a read-chunk-list to an XDR and fast register the page-list.
- *
- * Assumptions:
- * - chunk[0] position points to pages[0] at an offset of 0
- * - pages[] will be made physically contiguous by creating a one-off memory
- * region using the fastreg verb.
- * - byte_count is # of bytes in read-chunk-list
- * - ch_count is # of chunks in read-chunk-list
- *
- * Output:
- * - sge array pointing into pages[] array.
- * - chunk_sge array specifying sge index and count for each
- * chunk in the read list
- */
-static int fast_reg_read_chunks(struct svcxprt_rdma *xprt,
- struct svc_rqst *rqstp,
- struct svc_rdma_op_ctxt *head,
- struct rpcrdma_msg *rmsgp,
- struct svc_rdma_req_map *rpl_map,
- struct svc_rdma_req_map *chl_map,
- int ch_count,
- int byte_count)
-{
- int page_no;
- int ch_no;
- u32 offset;
- struct rpcrdma_read_chunk *ch;
- struct svc_rdma_fastreg_mr *frmr;
- int ret = 0;
-
- frmr = svc_rdma_get_frmr(xprt);
- if (IS_ERR(frmr))
- return -ENOMEM;
-
- head->frmr = frmr;
- head->arg.head[0] = rqstp->rq_arg.head[0];
- head->arg.tail[0] = rqstp->rq_arg.tail[0];
- head->arg.pages = &head->pages[head->count];
- head->hdr_count = head->count; /* save count of hdr pages */
- head->arg.page_base = 0;
- head->arg.page_len = byte_count;
- head->arg.len = rqstp->rq_arg.len + byte_count;
- head->arg.buflen = rqstp->rq_arg.buflen + byte_count;
-
- /* Fast register the page list */
- frmr->kva = page_address(rqstp->rq_arg.pages[0]);
- frmr->direction = DMA_FROM_DEVICE;
- frmr->access_flags = (IB_ACCESS_LOCAL_WRITE|IB_ACCESS_REMOTE_WRITE);
- frmr->map_len = byte_count;
- frmr->page_list_len = PAGE_ALIGN(byte_count) >> PAGE_SHIFT;
- for (page_no = 0; page_no < frmr->page_list_len; page_no++) {
- frmr->page_list->page_list[page_no] =
- ib_dma_map_page(xprt->sc_cm_id->device,
- rqstp->rq_arg.pages[page_no], 0,
- PAGE_SIZE, DMA_FROM_DEVICE);
- if (ib_dma_mapping_error(xprt->sc_cm_id->device,
- frmr->page_list->page_list[page_no]))
- goto fatal_err;
- atomic_inc(&xprt->sc_dma_used);
- head->arg.pages[page_no] = rqstp->rq_arg.pages[page_no];
- }
- head->count += page_no;
-
- /* rq_respages points one past arg pages */
- rqstp->rq_respages = &rqstp->rq_arg.pages[page_no];
- rqstp->rq_next_page = rqstp->rq_respages + 1;
-
- /* Create the reply and chunk maps */
- offset = 0;
- ch = (struct rpcrdma_read_chunk *)&rmsgp->rm_body.rm_chunks[0];
- for (ch_no = 0; ch_no < ch_count; ch_no++) {
- int len = ntohl(ch->rc_target.rs_length);
- rpl_map->sge[ch_no].iov_base = frmr->kva + offset;
- rpl_map->sge[ch_no].iov_len = len;
- chl_map->ch[ch_no].count = 1;
- chl_map->ch[ch_no].start = ch_no;
- offset += len;
- ch++;
- }
-
- ret = svc_rdma_fastreg(xprt, frmr);
- if (ret)
- goto fatal_err;
-
- return ch_no;
-
- fatal_err:
- printk("svcrdma: error fast registering xdr for xprt %p", xprt);
- svc_rdma_put_frmr(xprt, frmr);
- return -EIO;
-}
-
-static int rdma_set_ctxt_sge(struct svcxprt_rdma *xprt,
- struct svc_rdma_op_ctxt *ctxt,
- struct svc_rdma_fastreg_mr *frmr,
- struct kvec *vec,
- u64 *sgl_offset,
- int count)
-{
- int i;
- unsigned long off;
-
- ctxt->count = count;
- ctxt->direction = DMA_FROM_DEVICE;
- for (i = 0; i < count; i++) {
- ctxt->sge[i].length = 0; /* in case map fails */
- if (!frmr) {
- BUG_ON(!virt_to_page(vec[i].iov_base));
- off = (unsigned long)vec[i].iov_base & ~PAGE_MASK;
- ctxt->sge[i].addr =
- ib_dma_map_page(xprt->sc_cm_id->device,
- virt_to_page(vec[i].iov_base),
- off,
- vec[i].iov_len,
- DMA_FROM_DEVICE);
- if (ib_dma_mapping_error(xprt->sc_cm_id->device,
- ctxt->sge[i].addr))
- return -EINVAL;
- ctxt->sge[i].lkey = xprt->sc_dma_lkey;
- atomic_inc(&xprt->sc_dma_used);
- } else {
- ctxt->sge[i].addr = (unsigned long)vec[i].iov_base;
- ctxt->sge[i].lkey = frmr->mr->lkey;
- }
- ctxt->sge[i].length = vec[i].iov_len;
- *sgl_offset = *sgl_offset + vec[i].iov_len;
- }
- return 0;
-}
-
static int rdma_read_max_sge(struct svcxprt_rdma *xprt, int sge_count)
{
- if ((rdma_node_get_transport(xprt->sc_cm_id->device->node_type) ==
- RDMA_TRANSPORT_IWARP) &&
- sge_count > 1)
+ if (rdma_node_get_transport(xprt->sc_cm_id->device->node_type) ==
+ RDMA_TRANSPORT_IWARP)
return 1;
else
return min_t(int, sge_count, xprt->sc_max_sge);
}
-/*
- * Use RDMA_READ to read data from the advertised client buffer into the
- * XDR stream starting at rq_arg.head[0].iov_base.
- * Each chunk in the array
- * contains the following fields:
- * discrim - '1', This isn't used for data placement
- * position - The xdr stream offset (the same for every chunk)
- * handle - RMR for client memory region
- * length - data transfer length
- * offset - 64 bit tagged offset in remote memory region
- *
- * On our side, we need to read into a pagelist. The first page immediately
- * follows the RPC header.
- *
- * This function returns:
- * 0 - No error and no read-list found.
- *
- * 1 - Successful read-list processing. The data is not yet in
- * the pagelist and therefore the RPC request must be deferred. The
- * I/O completion will enqueue the transport again and
- * svc_rdma_recvfrom will complete the request.
- *
- * <0 - Error processing/posting read-list.
- *
- * NOTE: The ctxt must not be touched after the last WR has been posted
- * because the I/O completion processing may occur on another
- * processor and free / modify the context. Ne touche pas!
- */
-static int rdma_read_xdr(struct svcxprt_rdma *xprt,
- struct rpcrdma_msg *rmsgp,
- struct svc_rqst *rqstp,
- struct svc_rdma_op_ctxt *hdr_ctxt)
+typedef int (*rdma_reader_fn)(struct svcxprt_rdma *xprt,
+ struct svc_rqst *rqstp,
+ struct svc_rdma_op_ctxt *head,
+ int *page_no,
+ u32 *page_offset,
+ u32 rs_handle,
+ u32 rs_length,
+ u64 rs_offset,
+ int last);
+
+/* Issue an RDMA_READ using the local lkey to map the data sink */
+static int rdma_read_chunk_lcl(struct svcxprt_rdma *xprt,
+ struct svc_rqst *rqstp,
+ struct svc_rdma_op_ctxt *head,
+ int *page_no,
+ u32 *page_offset,
+ u32 rs_handle,
+ u32 rs_length,
+ u64 rs_offset,
+ int last)
+{
+ struct ib_send_wr read_wr;
+ int pages_needed = PAGE_ALIGN(*page_offset + rs_length) >> PAGE_SHIFT;
+ struct svc_rdma_op_ctxt *ctxt = svc_rdma_get_context(xprt);
+ int ret, read, pno;
+ u32 pg_off = *page_offset;
+ u32 pg_no = *page_no;
+
+ ctxt->direction = DMA_FROM_DEVICE;
+ ctxt->read_hdr = head;
+ pages_needed =
+ min_t(int, pages_needed, rdma_read_max_sge(xprt, pages_needed));
+ read = min_t(int, pages_needed << PAGE_SHIFT, rs_length);
+
+ for (pno = 0; pno < pages_needed; pno++) {
+ int len = min_t(int, rs_length, PAGE_SIZE - pg_off);
+
+ head->arg.pages[pg_no] = rqstp->rq_arg.pages[pg_no];
+ head->arg.page_len += len;
+ head->arg.len += len;
+ if (!pg_off)
+ head->count++;
+ rqstp->rq_respages = &rqstp->rq_arg.pages[pg_no+1];
+ rqstp->rq_next_page = rqstp->rq_respages + 1;
+ ctxt->sge[pno].addr =
+ ib_dma_map_page(xprt->sc_cm_id->device,
+ head->arg.pages[pg_no], pg_off,
+ PAGE_SIZE - pg_off,
+ DMA_FROM_DEVICE);
+ ret = ib_dma_mapping_error(xprt->sc_cm_id->device,
+ ctxt->sge[pno].addr);
+ if (ret)
+ goto err;
+ atomic_inc(&xprt->sc_dma_used);
+
+ /* The lkey here is either a local dma lkey or a dma_mr lkey */
+ ctxt->sge[pno].lkey = xprt->sc_dma_lkey;
+ ctxt->sge[pno].length = len;
+ ctxt->count++;
+
+ /* adjust offset and wrap to next page if needed */
+ pg_off += len;
+ if (pg_off == PAGE_SIZE) {
+ pg_off = 0;
+ pg_no++;
+ }
+ rs_length -= len;
+ }
+
+ if (last && rs_length == 0)
+ set_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags);
+ else
+ clear_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags);
+
+ memset(&read_wr, 0, sizeof(read_wr));
+ read_wr.wr_id = (unsigned long)ctxt;
+ read_wr.opcode = IB_WR_RDMA_READ;
+ ctxt->wr_op = read_wr.opcode;
+ read_wr.send_flags = IB_SEND_SIGNALED;
+ read_wr.wr.rdma.rkey = rs_handle;
+ read_wr.wr.rdma.remote_addr = rs_offset;
+ read_wr.sg_list = ctxt->sge;
+ read_wr.num_sge = pages_needed;
+
+ ret = svc_rdma_send(xprt, &read_wr);
+ if (ret) {
+ pr_err("svcrdma: Error %d posting RDMA_READ\n", ret);
+ set_bit(XPT_CLOSE, &xprt->sc_xprt.xpt_flags);
+ goto err;
+ }
+
+ /* return current location in page array */
+ *page_no = pg_no;
+ *page_offset = pg_off;
+ ret = read;
+ atomic_inc(&rdma_stat_read);
+ return ret;
+ err:
+ svc_rdma_unmap_dma(ctxt);
+ svc_rdma_put_context(ctxt, 0);
+ return ret;
+}
+
+/* Issue an RDMA_READ using an FRMR to map the data sink */
+static int rdma_read_chunk_frmr(struct svcxprt_rdma *xprt,
+ struct svc_rqst *rqstp,
+ struct svc_rdma_op_ctxt *head,
+ int *page_no,
+ u32 *page_offset,
+ u32 rs_handle,
+ u32 rs_length,
+ u64 rs_offset,
+ int last)
{
struct ib_send_wr read_wr;
struct ib_send_wr inv_wr;
- int err = 0;
- int ch_no;
- int ch_count;
- int byte_count;
- int sge_count;
- u64 sgl_offset;
+ struct ib_send_wr fastreg_wr;
+ u8 key;
+ int pages_needed = PAGE_ALIGN(*page_offset + rs_length) >> PAGE_SHIFT;
+ struct svc_rdma_op_ctxt *ctxt = svc_rdma_get_context(xprt);
+ struct svc_rdma_fastreg_mr *frmr = svc_rdma_get_frmr(xprt);
+ int ret, read, pno;
+ u32 pg_off = *page_offset;
+ u32 pg_no = *page_no;
+
+ if (IS_ERR(frmr))
+ return -ENOMEM;
+
+ ctxt->direction = DMA_FROM_DEVICE;
+ ctxt->frmr = frmr;
+ pages_needed = min_t(int, pages_needed, xprt->sc_frmr_pg_list_len);
+ read = min_t(int, pages_needed << PAGE_SHIFT, rs_length);
+
+ frmr->kva = page_address(rqstp->rq_arg.pages[pg_no]);
+ frmr->direction = DMA_FROM_DEVICE;
+ frmr->access_flags = (IB_ACCESS_LOCAL_WRITE|IB_ACCESS_REMOTE_WRITE);
+ frmr->map_len = pages_needed << PAGE_SHIFT;
+ frmr->page_list_len = pages_needed;
+
+ for (pno = 0; pno < pages_needed; pno++) {
+ int len = min_t(int, rs_length, PAGE_SIZE - pg_off);
+
+ head->arg.pages[pg_no] = rqstp->rq_arg.pages[pg_no];
+ head->arg.page_len += len;
+ head->arg.len += len;
+ if (!pg_off)
+ head->count++;
+ rqstp->rq_respages = &rqstp->rq_arg.pages[pg_no+1];
+ rqstp->rq_next_page = rqstp->rq_respages + 1;
+ frmr->page_list->page_list[pno] =
+ ib_dma_map_page(xprt->sc_cm_id->device,
+ head->arg.pages[pg_no], 0,
+ PAGE_SIZE, DMA_FROM_DEVICE);
+ ret = ib_dma_mapping_error(xprt->sc_cm_id->device,
+ frmr->page_list->page_list[pno]);
+ if (ret)
+ goto err;
+ atomic_inc(&xprt->sc_dma_used);
+
+ /* adjust offset and wrap to next page if needed */
+ pg_off += len;
+ if (pg_off == PAGE_SIZE) {
+ pg_off = 0;
+ pg_no++;
+ }
+ rs_length -= len;
+ }
+
+ if (last && rs_length == 0)
+ set_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags);
+ else
+ clear_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags);
+
+ /* Bump the key */
+ key = (u8)(frmr->mr->lkey & 0x000000FF);
+ ib_update_fast_reg_key(frmr->mr, ++key);
+
+ ctxt->sge[0].addr = (unsigned long)frmr->kva + *page_offset;
+ ctxt->sge[0].lkey = frmr->mr->lkey;
+ ctxt->sge[0].length = read;
+ ctxt->count = 1;
+ ctxt->read_hdr = head;
+
+ /* Prepare FASTREG WR */
+ memset(&fastreg_wr, 0, sizeof(fastreg_wr));
+ fastreg_wr.opcode = IB_WR_FAST_REG_MR;
+ fastreg_wr.send_flags = IB_SEND_SIGNALED;
+ fastreg_wr.wr.fast_reg.iova_start = (unsigned long)frmr->kva;
+ fastreg_wr.wr.fast_reg.page_list = frmr->page_list;
+ fastreg_wr.wr.fast_reg.page_list_len = frmr->page_list_len;
+ fastreg_wr.wr.fast_reg.page_shift = PAGE_SHIFT;
+ fastreg_wr.wr.fast_reg.length = frmr->map_len;
+ fastreg_wr.wr.fast_reg.access_flags = frmr->access_flags;
+ fastreg_wr.wr.fast_reg.rkey = frmr->mr->lkey;
+ fastreg_wr.next = &read_wr;
+
+ /* Prepare RDMA_READ */
+ memset(&read_wr, 0, sizeof(read_wr));
+ read_wr.send_flags = IB_SEND_SIGNALED;
+ read_wr.wr.rdma.rkey = rs_handle;
+ read_wr.wr.rdma.remote_addr = rs_offset;
+ read_wr.sg_list = ctxt->sge;
+ read_wr.num_sge = 1;
+ if (xprt->sc_dev_caps & SVCRDMA_DEVCAP_READ_W_INV) {
+ read_wr.opcode = IB_WR_RDMA_READ_WITH_INV;
+ read_wr.wr_id = (unsigned long)ctxt;
+ read_wr.ex.invalidate_rkey = ctxt->frmr->mr->lkey;
+ } else {
+ read_wr.opcode = IB_WR_RDMA_READ;
+ read_wr.next = &inv_wr;
+ /* Prepare invalidate */
+ memset(&inv_wr, 0, sizeof(inv_wr));
+ inv_wr.wr_id = (unsigned long)ctxt;
+ inv_wr.opcode = IB_WR_LOCAL_INV;
+ inv_wr.send_flags = IB_SEND_SIGNALED | IB_SEND_FENCE;
+ inv_wr.ex.invalidate_rkey = frmr->mr->lkey;
+ }
+ ctxt->wr_op = read_wr.opcode;
+
+ /* Post the chain */
+ ret = svc_rdma_send(xprt, &fastreg_wr);
+ if (ret) {
+ pr_err("svcrdma: Error %d posting RDMA_READ\n", ret);
+ set_bit(XPT_CLOSE, &xprt->sc_xprt.xpt_flags);
+ goto err;
+ }
+
+ /* return current location in page array */
+ *page_no = pg_no;
+ *page_offset = pg_off;
+ ret = read;
+ atomic_inc(&rdma_stat_read);
+ return ret;
+ err:
+ svc_rdma_unmap_dma(ctxt);
+ svc_rdma_put_context(ctxt, 0);
+ svc_rdma_put_frmr(xprt, frmr);
+ return ret;
+}
+
+static int rdma_read_chunks(struct svcxprt_rdma *xprt,
+ struct rpcrdma_msg *rmsgp,
+ struct svc_rqst *rqstp,
+ struct svc_rdma_op_ctxt *head)
+{
+ int page_no, ch_count, ret;
struct rpcrdma_read_chunk *ch;
- struct svc_rdma_op_ctxt *ctxt = NULL;
- struct svc_rdma_req_map *rpl_map;
- struct svc_rdma_req_map *chl_map;
+ u32 page_offset, byte_count;
+ u64 rs_offset;
+ rdma_reader_fn reader;
/* If no read list is present, return 0 */
ch = svc_rdma_get_read_chunk(rmsgp);
@@ -408,122 +384,55 @@
if (ch_count > RPCSVC_MAXPAGES)
return -EINVAL;
- /* Allocate temporary reply and chunk maps */
- rpl_map = svc_rdma_get_req_map();
- chl_map = svc_rdma_get_req_map();
+ /* The request is completed when the RDMA_READs complete. The
+ * head context keeps all the pages that comprise the
+ * request.
+ */
+ head->arg.head[0] = rqstp->rq_arg.head[0];
+ head->arg.tail[0] = rqstp->rq_arg.tail[0];
+ head->arg.pages = &head->pages[head->count];
+ head->hdr_count = head->count;
+ head->arg.page_base = 0;
+ head->arg.page_len = 0;
+ head->arg.len = rqstp->rq_arg.len;
+ head->arg.buflen = rqstp->rq_arg.buflen;
- if (!xprt->sc_frmr_pg_list_len)
- sge_count = map_read_chunks(xprt, rqstp, hdr_ctxt, rmsgp,
- rpl_map, chl_map, ch_count,
- byte_count);
+ /* Use FRMR if supported */
+ if (xprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG)
+ reader = rdma_read_chunk_frmr;
else
- sge_count = fast_reg_read_chunks(xprt, rqstp, hdr_ctxt, rmsgp,
- rpl_map, chl_map, ch_count,
- byte_count);
- if (sge_count < 0) {
- err = -EIO;
- goto out;
- }
+ reader = rdma_read_chunk_lcl;
- sgl_offset = 0;
- ch_no = 0;
-
+ page_no = 0; page_offset = 0;
for (ch = (struct rpcrdma_read_chunk *)&rmsgp->rm_body.rm_chunks[0];
- ch->rc_discrim != 0; ch++, ch_no++) {
- u64 rs_offset;
-next_sge:
- ctxt = svc_rdma_get_context(xprt);
- ctxt->direction = DMA_FROM_DEVICE;
- ctxt->frmr = hdr_ctxt->frmr;
- ctxt->read_hdr = NULL;
- clear_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags);
- clear_bit(RDMACTXT_F_FAST_UNREG, &ctxt->flags);
+ ch->rc_discrim != 0; ch++) {
- /* Prepare READ WR */
- memset(&read_wr, 0, sizeof read_wr);
- read_wr.wr_id = (unsigned long)ctxt;
- read_wr.opcode = IB_WR_RDMA_READ;
- ctxt->wr_op = read_wr.opcode;
- read_wr.send_flags = IB_SEND_SIGNALED;
- read_wr.wr.rdma.rkey = ntohl(ch->rc_target.rs_handle);
xdr_decode_hyper((__be32 *)&ch->rc_target.rs_offset,
&rs_offset);
- read_wr.wr.rdma.remote_addr = rs_offset + sgl_offset;
- read_wr.sg_list = ctxt->sge;
- read_wr.num_sge =
- rdma_read_max_sge(xprt, chl_map->ch[ch_no].count);
- err = rdma_set_ctxt_sge(xprt, ctxt, hdr_ctxt->frmr,
- &rpl_map->sge[chl_map->ch[ch_no].start],
- &sgl_offset,
- read_wr.num_sge);
- if (err) {
- svc_rdma_unmap_dma(ctxt);
- svc_rdma_put_context(ctxt, 0);
- goto out;
- }
- if (((ch+1)->rc_discrim == 0) &&
- (read_wr.num_sge == chl_map->ch[ch_no].count)) {
- /*
- * Mark the last RDMA_READ with a bit to
- * indicate all RPC data has been fetched from
- * the client and the RPC needs to be enqueued.
- */
- set_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags);
- if (hdr_ctxt->frmr) {
- set_bit(RDMACTXT_F_FAST_UNREG, &ctxt->flags);
- /*
- * Invalidate the local MR used to map the data
- * sink.
- */
- if (xprt->sc_dev_caps &
- SVCRDMA_DEVCAP_READ_W_INV) {
- read_wr.opcode =
- IB_WR_RDMA_READ_WITH_INV;
- ctxt->wr_op = read_wr.opcode;
- read_wr.ex.invalidate_rkey =
- ctxt->frmr->mr->lkey;
- } else {
- /* Prepare INVALIDATE WR */
- memset(&inv_wr, 0, sizeof inv_wr);
- inv_wr.opcode = IB_WR_LOCAL_INV;
- inv_wr.send_flags = IB_SEND_SIGNALED;
- inv_wr.ex.invalidate_rkey =
- hdr_ctxt->frmr->mr->lkey;
- read_wr.next = &inv_wr;
- }
- }
- ctxt->read_hdr = hdr_ctxt;
- }
- /* Post the read */
- err = svc_rdma_send(xprt, &read_wr);
- if (err) {
- printk(KERN_ERR "svcrdma: Error %d posting RDMA_READ\n",
- err);
- set_bit(XPT_CLOSE, &xprt->sc_xprt.xpt_flags);
- svc_rdma_unmap_dma(ctxt);
- svc_rdma_put_context(ctxt, 0);
- goto out;
- }
- atomic_inc(&rdma_stat_read);
+ byte_count = ntohl(ch->rc_target.rs_length);
- if (read_wr.num_sge < chl_map->ch[ch_no].count) {
- chl_map->ch[ch_no].count -= read_wr.num_sge;
- chl_map->ch[ch_no].start += read_wr.num_sge;
- goto next_sge;
+ while (byte_count > 0) {
+ ret = reader(xprt, rqstp, head,
+ &page_no, &page_offset,
+ ntohl(ch->rc_target.rs_handle),
+ byte_count, rs_offset,
+ ((ch+1)->rc_discrim == 0) /* last */
+ );
+ if (ret < 0)
+ goto err;
+ byte_count -= ret;
+ rs_offset += ret;
+ head->arg.buflen += ret;
}
- sgl_offset = 0;
- err = 1;
}
-
- out:
- svc_rdma_put_req_map(rpl_map);
- svc_rdma_put_req_map(chl_map);
-
+ ret = 1;
+ err:
/* Detach arg pages. svc_recv will replenish them */
- for (ch_no = 0; &rqstp->rq_pages[ch_no] < rqstp->rq_respages; ch_no++)
- rqstp->rq_pages[ch_no] = NULL;
+ for (page_no = 0;
+ &rqstp->rq_pages[page_no] < rqstp->rq_respages; page_no++)
+ rqstp->rq_pages[page_no] = NULL;
- return err;
+ return ret;
}
static int rdma_read_complete(struct svc_rqst *rqstp,
@@ -595,13 +504,9 @@
struct svc_rdma_op_ctxt,
dto_q);
list_del_init(&ctxt->dto_q);
- }
- if (ctxt) {
spin_unlock_bh(&rdma_xprt->sc_rq_dto_lock);
return rdma_read_complete(rqstp, ctxt);
- }
-
- if (!list_empty(&rdma_xprt->sc_rq_dto_q)) {
+ } else if (!list_empty(&rdma_xprt->sc_rq_dto_q)) {
ctxt = list_entry(rdma_xprt->sc_rq_dto_q.next,
struct svc_rdma_op_ctxt,
dto_q);
@@ -621,7 +526,6 @@
if (test_bit(XPT_CLOSE, &xprt->xpt_flags))
goto close_out;
- BUG_ON(ret);
goto out;
}
dprintk("svcrdma: processing ctxt=%p on xprt=%p, rqstp=%p, status=%d\n",
@@ -644,12 +548,11 @@
}
/* Read read-list data. */
- ret = rdma_read_xdr(rdma_xprt, rmsgp, rqstp, ctxt);
+ ret = rdma_read_chunks(rdma_xprt, rmsgp, rqstp, ctxt);
if (ret > 0) {
/* read-list posted, defer until data received from client. */
goto defer;
- }
- if (ret < 0) {
+ } else if (ret < 0) {
/* Post of read-list failed, free context. */
svc_rdma_put_context(ctxt, 1);
return 0;
diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
index 7e024a5..49fd21a 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
@@ -1,4 +1,5 @@
/*
+ * Copyright (c) 2014 Open Grid Computing, Inc. All rights reserved.
* Copyright (c) 2005-2006 Network Appliance, Inc. All rights reserved.
*
* This software is available to you under a choice of one of two
@@ -49,152 +50,6 @@
#define RPCDBG_FACILITY RPCDBG_SVCXPRT
-/* Encode an XDR as an array of IB SGE
- *
- * Assumptions:
- * - head[0] is physically contiguous.
- * - tail[0] is physically contiguous.
- * - pages[] is not physically or virtually contiguous and consists of
- * PAGE_SIZE elements.
- *
- * Output:
- * SGE[0] reserved for RCPRDMA header
- * SGE[1] data from xdr->head[]
- * SGE[2..sge_count-2] data from xdr->pages[]
- * SGE[sge_count-1] data from xdr->tail.
- *
- * The max SGE we need is the length of the XDR / pagesize + one for
- * head + one for tail + one for RPCRDMA header. Since RPCSVC_MAXPAGES
- * reserves a page for both the request and the reply header, and this
- * array is only concerned with the reply we are assured that we have
- * on extra page for the RPCRMDA header.
- */
-static int fast_reg_xdr(struct svcxprt_rdma *xprt,
- struct xdr_buf *xdr,
- struct svc_rdma_req_map *vec)
-{
- int sge_no;
- u32 sge_bytes;
- u32 page_bytes;
- u32 page_off;
- int page_no = 0;
- u8 *frva;
- struct svc_rdma_fastreg_mr *frmr;
-
- frmr = svc_rdma_get_frmr(xprt);
- if (IS_ERR(frmr))
- return -ENOMEM;
- vec->frmr = frmr;
-
- /* Skip the RPCRDMA header */
- sge_no = 1;
-
- /* Map the head. */
- frva = (void *)((unsigned long)(xdr->head[0].iov_base) & PAGE_MASK);
- vec->sge[sge_no].iov_base = xdr->head[0].iov_base;
- vec->sge[sge_no].iov_len = xdr->head[0].iov_len;
- vec->count = 2;
- sge_no++;
-
- /* Map the XDR head */
- frmr->kva = frva;
- frmr->direction = DMA_TO_DEVICE;
- frmr->access_flags = 0;
- frmr->map_len = PAGE_SIZE;
- frmr->page_list_len = 1;
- page_off = (unsigned long)xdr->head[0].iov_base & ~PAGE_MASK;
- frmr->page_list->page_list[page_no] =
- ib_dma_map_page(xprt->sc_cm_id->device,
- virt_to_page(xdr->head[0].iov_base),
- page_off,
- PAGE_SIZE - page_off,
- DMA_TO_DEVICE);
- if (ib_dma_mapping_error(xprt->sc_cm_id->device,
- frmr->page_list->page_list[page_no]))
- goto fatal_err;
- atomic_inc(&xprt->sc_dma_used);
-
- /* Map the XDR page list */
- page_off = xdr->page_base;
- page_bytes = xdr->page_len + page_off;
- if (!page_bytes)
- goto encode_tail;
-
- /* Map the pages */
- vec->sge[sge_no].iov_base = frva + frmr->map_len + page_off;
- vec->sge[sge_no].iov_len = page_bytes;
- sge_no++;
- while (page_bytes) {
- struct page *page;
-
- page = xdr->pages[page_no++];
- sge_bytes = min_t(u32, page_bytes, (PAGE_SIZE - page_off));
- page_bytes -= sge_bytes;
-
- frmr->page_list->page_list[page_no] =
- ib_dma_map_page(xprt->sc_cm_id->device,
- page, page_off,
- sge_bytes, DMA_TO_DEVICE);
- if (ib_dma_mapping_error(xprt->sc_cm_id->device,
- frmr->page_list->page_list[page_no]))
- goto fatal_err;
-
- atomic_inc(&xprt->sc_dma_used);
- page_off = 0; /* reset for next time through loop */
- frmr->map_len += PAGE_SIZE;
- frmr->page_list_len++;
- }
- vec->count++;
-
- encode_tail:
- /* Map tail */
- if (0 == xdr->tail[0].iov_len)
- goto done;
-
- vec->count++;
- vec->sge[sge_no].iov_len = xdr->tail[0].iov_len;
-
- if (((unsigned long)xdr->tail[0].iov_base & PAGE_MASK) ==
- ((unsigned long)xdr->head[0].iov_base & PAGE_MASK)) {
- /*
- * If head and tail use the same page, we don't need
- * to map it again.
- */
- vec->sge[sge_no].iov_base = xdr->tail[0].iov_base;
- } else {
- void *va;
-
- /* Map another page for the tail */
- page_off = (unsigned long)xdr->tail[0].iov_base & ~PAGE_MASK;
- va = (void *)((unsigned long)xdr->tail[0].iov_base & PAGE_MASK);
- vec->sge[sge_no].iov_base = frva + frmr->map_len + page_off;
-
- frmr->page_list->page_list[page_no] =
- ib_dma_map_page(xprt->sc_cm_id->device, virt_to_page(va),
- page_off,
- PAGE_SIZE,
- DMA_TO_DEVICE);
- if (ib_dma_mapping_error(xprt->sc_cm_id->device,
- frmr->page_list->page_list[page_no]))
- goto fatal_err;
- atomic_inc(&xprt->sc_dma_used);
- frmr->map_len += PAGE_SIZE;
- frmr->page_list_len++;
- }
-
- done:
- if (svc_rdma_fastreg(xprt, frmr))
- goto fatal_err;
-
- return 0;
-
- fatal_err:
- printk("svcrdma: Error fast registering memory for xprt %p\n", xprt);
- vec->frmr = NULL;
- svc_rdma_put_frmr(xprt, frmr);
- return -EIO;
-}
-
static int map_xdr(struct svcxprt_rdma *xprt,
struct xdr_buf *xdr,
struct svc_rdma_req_map *vec)
@@ -208,9 +63,6 @@
BUG_ON(xdr->len !=
(xdr->head[0].iov_len + xdr->page_len + xdr->tail[0].iov_len));
- if (xprt->sc_frmr_pg_list_len)
- return fast_reg_xdr(xprt, xdr, vec);
-
/* Skip the first sge, this is for the RPCRDMA header */
sge_no = 1;
@@ -282,8 +134,6 @@
}
/* Assumptions:
- * - We are using FRMR
- * - or -
* - The specified write_len can be represented in sc_max_sge * PAGE_SIZE
*/
static int send_write(struct svcxprt_rdma *xprt, struct svc_rqst *rqstp,
@@ -327,23 +177,16 @@
sge_bytes = min_t(size_t,
bc, vec->sge[xdr_sge_no].iov_len-sge_off);
sge[sge_no].length = sge_bytes;
- if (!vec->frmr) {
- sge[sge_no].addr =
- dma_map_xdr(xprt, &rqstp->rq_res, xdr_off,
- sge_bytes, DMA_TO_DEVICE);
- xdr_off += sge_bytes;
- if (ib_dma_mapping_error(xprt->sc_cm_id->device,
- sge[sge_no].addr))
- goto err;
- atomic_inc(&xprt->sc_dma_used);
- sge[sge_no].lkey = xprt->sc_dma_lkey;
- } else {
- sge[sge_no].addr = (unsigned long)
- vec->sge[xdr_sge_no].iov_base + sge_off;
- sge[sge_no].lkey = vec->frmr->mr->lkey;
- }
+ sge[sge_no].addr =
+ dma_map_xdr(xprt, &rqstp->rq_res, xdr_off,
+ sge_bytes, DMA_TO_DEVICE);
+ xdr_off += sge_bytes;
+ if (ib_dma_mapping_error(xprt->sc_cm_id->device,
+ sge[sge_no].addr))
+ goto err;
+ atomic_inc(&xprt->sc_dma_used);
+ sge[sge_no].lkey = xprt->sc_dma_lkey;
ctxt->count++;
- ctxt->frmr = vec->frmr;
sge_off = 0;
sge_no++;
xdr_sge_no++;
@@ -369,7 +212,6 @@
return 0;
err:
svc_rdma_unmap_dma(ctxt);
- svc_rdma_put_frmr(xprt, vec->frmr);
svc_rdma_put_context(ctxt, 0);
/* Fatal error, close transport */
return -EIO;
@@ -397,10 +239,7 @@
res_ary = (struct rpcrdma_write_array *)
&rdma_resp->rm_body.rm_chunks[1];
- if (vec->frmr)
- max_write = vec->frmr->map_len;
- else
- max_write = xprt->sc_max_sge * PAGE_SIZE;
+ max_write = xprt->sc_max_sge * PAGE_SIZE;
/* Write chunks start at the pagelist */
for (xdr_off = rqstp->rq_res.head[0].iov_len, chunk_no = 0;
@@ -472,10 +311,7 @@
res_ary = (struct rpcrdma_write_array *)
&rdma_resp->rm_body.rm_chunks[2];
- if (vec->frmr)
- max_write = vec->frmr->map_len;
- else
- max_write = xprt->sc_max_sge * PAGE_SIZE;
+ max_write = xprt->sc_max_sge * PAGE_SIZE;
/* xdr offset starts at RPC message */
nchunks = ntohl(arg_ary->wc_nchunks);
@@ -545,7 +381,6 @@
int byte_count)
{
struct ib_send_wr send_wr;
- struct ib_send_wr inv_wr;
int sge_no;
int sge_bytes;
int page_no;
@@ -559,7 +394,6 @@
"svcrdma: could not post a receive buffer, err=%d."
"Closing transport %p.\n", ret, rdma);
set_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags);
- svc_rdma_put_frmr(rdma, vec->frmr);
svc_rdma_put_context(ctxt, 0);
return -ENOTCONN;
}
@@ -567,11 +401,6 @@
/* Prepare the context */
ctxt->pages[0] = page;
ctxt->count = 1;
- ctxt->frmr = vec->frmr;
- if (vec->frmr)
- set_bit(RDMACTXT_F_FAST_UNREG, &ctxt->flags);
- else
- clear_bit(RDMACTXT_F_FAST_UNREG, &ctxt->flags);
/* Prepare the SGE for the RPCRDMA Header */
ctxt->sge[0].lkey = rdma->sc_dma_lkey;
@@ -590,21 +419,15 @@
int xdr_off = 0;
sge_bytes = min_t(size_t, vec->sge[sge_no].iov_len, byte_count);
byte_count -= sge_bytes;
- if (!vec->frmr) {
- ctxt->sge[sge_no].addr =
- dma_map_xdr(rdma, &rqstp->rq_res, xdr_off,
- sge_bytes, DMA_TO_DEVICE);
- xdr_off += sge_bytes;
- if (ib_dma_mapping_error(rdma->sc_cm_id->device,
- ctxt->sge[sge_no].addr))
- goto err;
- atomic_inc(&rdma->sc_dma_used);
- ctxt->sge[sge_no].lkey = rdma->sc_dma_lkey;
- } else {
- ctxt->sge[sge_no].addr = (unsigned long)
- vec->sge[sge_no].iov_base;
- ctxt->sge[sge_no].lkey = vec->frmr->mr->lkey;
- }
+ ctxt->sge[sge_no].addr =
+ dma_map_xdr(rdma, &rqstp->rq_res, xdr_off,
+ sge_bytes, DMA_TO_DEVICE);
+ xdr_off += sge_bytes;
+ if (ib_dma_mapping_error(rdma->sc_cm_id->device,
+ ctxt->sge[sge_no].addr))
+ goto err;
+ atomic_inc(&rdma->sc_dma_used);
+ ctxt->sge[sge_no].lkey = rdma->sc_dma_lkey;
ctxt->sge[sge_no].length = sge_bytes;
}
BUG_ON(byte_count != 0);
@@ -627,6 +450,7 @@
ctxt->sge[page_no+1].length = 0;
}
rqstp->rq_next_page = rqstp->rq_respages + 1;
+
BUG_ON(sge_no > rdma->sc_max_sge);
memset(&send_wr, 0, sizeof send_wr);
ctxt->wr_op = IB_WR_SEND;
@@ -635,15 +459,6 @@
send_wr.num_sge = sge_no;
send_wr.opcode = IB_WR_SEND;
send_wr.send_flags = IB_SEND_SIGNALED;
- if (vec->frmr) {
- /* Prepare INVALIDATE WR */
- memset(&inv_wr, 0, sizeof inv_wr);
- inv_wr.opcode = IB_WR_LOCAL_INV;
- inv_wr.send_flags = IB_SEND_SIGNALED;
- inv_wr.ex.invalidate_rkey =
- vec->frmr->mr->lkey;
- send_wr.next = &inv_wr;
- }
ret = svc_rdma_send(rdma, &send_wr);
if (ret)
@@ -653,7 +468,6 @@
err:
svc_rdma_unmap_dma(ctxt);
- svc_rdma_put_frmr(rdma, vec->frmr);
svc_rdma_put_context(ctxt, 1);
return -EIO;
}
diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c
index 25688fa..e7323fb 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_transport.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c
@@ -1,4 +1,5 @@
/*
+ * Copyright (c) 2014 Open Grid Computing, Inc. All rights reserved.
* Copyright (c) 2005-2007 Network Appliance, Inc. All rights reserved.
*
* This software is available to you under a choice of one of two
@@ -65,6 +66,7 @@
static void svc_rdma_detach(struct svc_xprt *xprt);
static void svc_rdma_free(struct svc_xprt *xprt);
static int svc_rdma_has_wspace(struct svc_xprt *xprt);
+static int svc_rdma_secure_port(struct svc_rqst *);
static void rq_cq_reap(struct svcxprt_rdma *xprt);
static void sq_cq_reap(struct svcxprt_rdma *xprt);
@@ -82,6 +84,7 @@
.xpo_prep_reply_hdr = svc_rdma_prep_reply_hdr,
.xpo_has_wspace = svc_rdma_has_wspace,
.xpo_accept = svc_rdma_accept,
+ .xpo_secure_port = svc_rdma_secure_port,
};
struct svc_xprt_class svc_rdma_class = {
@@ -160,7 +163,6 @@
schedule_timeout_uninterruptible(msecs_to_jiffies(500));
}
map->count = 0;
- map->frmr = NULL;
return map;
}
@@ -336,22 +338,21 @@
switch (ctxt->wr_op) {
case IB_WR_SEND:
- if (test_bit(RDMACTXT_F_FAST_UNREG, &ctxt->flags))
- svc_rdma_put_frmr(xprt, ctxt->frmr);
+ BUG_ON(ctxt->frmr);
svc_rdma_put_context(ctxt, 1);
break;
case IB_WR_RDMA_WRITE:
+ BUG_ON(ctxt->frmr);
svc_rdma_put_context(ctxt, 0);
break;
case IB_WR_RDMA_READ:
case IB_WR_RDMA_READ_WITH_INV:
+ svc_rdma_put_frmr(xprt, ctxt->frmr);
if (test_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags)) {
struct svc_rdma_op_ctxt *read_hdr = ctxt->read_hdr;
BUG_ON(!read_hdr);
- if (test_bit(RDMACTXT_F_FAST_UNREG, &ctxt->flags))
- svc_rdma_put_frmr(xprt, ctxt->frmr);
spin_lock_bh(&xprt->sc_rq_dto_lock);
set_bit(XPT_DATA, &xprt->sc_xprt.xpt_flags);
list_add_tail(&read_hdr->dto_q,
@@ -363,6 +364,7 @@
break;
default:
+ BUG_ON(1);
printk(KERN_ERR "svcrdma: unexpected completion type, "
"opcode=%d\n",
ctxt->wr_op);
@@ -378,29 +380,42 @@
static void sq_cq_reap(struct svcxprt_rdma *xprt)
{
struct svc_rdma_op_ctxt *ctxt = NULL;
- struct ib_wc wc;
+ struct ib_wc wc_a[6];
+ struct ib_wc *wc;
struct ib_cq *cq = xprt->sc_sq_cq;
int ret;
+ memset(wc_a, 0, sizeof(wc_a));
+
if (!test_and_clear_bit(RDMAXPRT_SQ_PENDING, &xprt->sc_flags))
return;
ib_req_notify_cq(xprt->sc_sq_cq, IB_CQ_NEXT_COMP);
atomic_inc(&rdma_stat_sq_poll);
- while ((ret = ib_poll_cq(cq, 1, &wc)) > 0) {
- if (wc.status != IB_WC_SUCCESS)
- /* Close the transport */
- set_bit(XPT_CLOSE, &xprt->sc_xprt.xpt_flags);
+ while ((ret = ib_poll_cq(cq, ARRAY_SIZE(wc_a), wc_a)) > 0) {
+ int i;
- /* Decrement used SQ WR count */
- atomic_dec(&xprt->sc_sq_count);
- wake_up(&xprt->sc_send_wait);
+ for (i = 0; i < ret; i++) {
+ wc = &wc_a[i];
+ if (wc->status != IB_WC_SUCCESS) {
+ dprintk("svcrdma: sq wc err status %d\n",
+ wc->status);
- ctxt = (struct svc_rdma_op_ctxt *)(unsigned long)wc.wr_id;
- if (ctxt)
- process_context(xprt, ctxt);
+ /* Close the transport */
+ set_bit(XPT_CLOSE, &xprt->sc_xprt.xpt_flags);
+ }
- svc_xprt_put(&xprt->sc_xprt);
+ /* Decrement used SQ WR count */
+ atomic_dec(&xprt->sc_sq_count);
+ wake_up(&xprt->sc_send_wait);
+
+ ctxt = (struct svc_rdma_op_ctxt *)
+ (unsigned long)wc->wr_id;
+ if (ctxt)
+ process_context(xprt, ctxt);
+
+ svc_xprt_put(&xprt->sc_xprt);
+ }
}
if (ctxt)
@@ -993,7 +1008,11 @@
need_dma_mr = 0;
break;
case RDMA_TRANSPORT_IB:
- if (!(devattr.device_cap_flags & IB_DEVICE_LOCAL_DMA_LKEY)) {
+ if (!(newxprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG)) {
+ need_dma_mr = 1;
+ dma_mr_acc = IB_ACCESS_LOCAL_WRITE;
+ } else if (!(devattr.device_cap_flags &
+ IB_DEVICE_LOCAL_DMA_LKEY)) {
need_dma_mr = 1;
dma_mr_acc = IB_ACCESS_LOCAL_WRITE;
} else
@@ -1190,14 +1209,7 @@
container_of(xprt, struct svcxprt_rdma, sc_xprt);
/*
- * If there are fewer SQ WR available than required to send a
- * simple response, return false.
- */
- if ((rdma->sc_sq_depth - atomic_read(&rdma->sc_sq_count) < 3))
- return 0;
-
- /*
- * ...or there are already waiters on the SQ,
+ * If there are already waiters on the SQ,
* return false.
*/
if (waitqueue_active(&rdma->sc_send_wait))
@@ -1207,6 +1219,11 @@
return 1;
}
+static int svc_rdma_secure_port(struct svc_rqst *rqstp)
+{
+ return 1;
+}
+
/*
* Attempt to register the kvec representing the RPC memory with the
* device.
diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c
index 1eb9c468..66f91f0 100644
--- a/net/sunrpc/xprtrdma/transport.c
+++ b/net/sunrpc/xprtrdma/transport.c
@@ -149,6 +149,11 @@
#endif
+#define RPCRDMA_BIND_TO (60U * HZ)
+#define RPCRDMA_INIT_REEST_TO (5U * HZ)
+#define RPCRDMA_MAX_REEST_TO (30U * HZ)
+#define RPCRDMA_IDLE_DISC_TO (5U * 60 * HZ)
+
static struct rpc_xprt_ops xprt_rdma_procs; /* forward reference */
static void
@@ -229,7 +234,6 @@
xprt_rdma_destroy(struct rpc_xprt *xprt)
{
struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt);
- int rc;
dprintk("RPC: %s: called\n", __func__);
@@ -238,10 +242,7 @@
xprt_clear_connected(xprt);
rpcrdma_buffer_destroy(&r_xprt->rx_buf);
- rc = rpcrdma_ep_destroy(&r_xprt->rx_ep, &r_xprt->rx_ia);
- if (rc)
- dprintk("RPC: %s: rpcrdma_ep_destroy returned %i\n",
- __func__, rc);
+ rpcrdma_ep_destroy(&r_xprt->rx_ep, &r_xprt->rx_ia);
rpcrdma_ia_close(&r_xprt->rx_ia);
xprt_rdma_free_addresses(xprt);
@@ -289,9 +290,9 @@
/* 60 second timeout, no retries */
xprt->timeout = &xprt_rdma_default_timeout;
- xprt->bind_timeout = (60U * HZ);
- xprt->reestablish_timeout = (5U * HZ);
- xprt->idle_timeout = (5U * 60 * HZ);
+ xprt->bind_timeout = RPCRDMA_BIND_TO;
+ xprt->reestablish_timeout = RPCRDMA_INIT_REEST_TO;
+ xprt->idle_timeout = RPCRDMA_IDLE_DISC_TO;
xprt->resvport = 0; /* privileged port not needed */
xprt->tsh_size = 0; /* RPC-RDMA handles framing */
@@ -391,7 +392,7 @@
xprt_rdma_free_addresses(xprt);
rc = -EINVAL;
out3:
- (void) rpcrdma_ep_destroy(new_ep, &new_xprt->rx_ia);
+ rpcrdma_ep_destroy(new_ep, &new_xprt->rx_ia);
out2:
rpcrdma_ia_close(&new_xprt->rx_ia);
out1:
@@ -436,10 +437,10 @@
schedule_delayed_work(&r_xprt->rdma_connect,
xprt->reestablish_timeout);
xprt->reestablish_timeout <<= 1;
- if (xprt->reestablish_timeout > (30 * HZ))
- xprt->reestablish_timeout = (30 * HZ);
- else if (xprt->reestablish_timeout < (5 * HZ))
- xprt->reestablish_timeout = (5 * HZ);
+ if (xprt->reestablish_timeout > RPCRDMA_MAX_REEST_TO)
+ xprt->reestablish_timeout = RPCRDMA_MAX_REEST_TO;
+ else if (xprt->reestablish_timeout < RPCRDMA_INIT_REEST_TO)
+ xprt->reestablish_timeout = RPCRDMA_INIT_REEST_TO;
} else {
schedule_delayed_work(&r_xprt->rdma_connect, 0);
if (!RPC_IS_ASYNC(task))
@@ -447,23 +448,6 @@
}
}
-static int
-xprt_rdma_reserve_xprt(struct rpc_xprt *xprt, struct rpc_task *task)
-{
- struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt);
- int credits = atomic_read(&r_xprt->rx_buf.rb_credits);
-
- /* == RPC_CWNDSCALE @ init, but *after* setup */
- if (r_xprt->rx_buf.rb_cwndscale == 0UL) {
- r_xprt->rx_buf.rb_cwndscale = xprt->cwnd;
- dprintk("RPC: %s: cwndscale %lu\n", __func__,
- r_xprt->rx_buf.rb_cwndscale);
- BUG_ON(r_xprt->rx_buf.rb_cwndscale <= 0);
- }
- xprt->cwnd = credits * r_xprt->rx_buf.rb_cwndscale;
- return xprt_reserve_xprt_cong(xprt, task);
-}
-
/*
* The RDMA allocate/free functions need the task structure as a place
* to hide the struct rpcrdma_req, which is necessary for the actual send/recv
@@ -479,7 +463,8 @@
struct rpcrdma_req *req, *nreq;
req = rpcrdma_buffer_get(&rpcx_to_rdmax(xprt)->rx_buf);
- BUG_ON(NULL == req);
+ if (req == NULL)
+ return NULL;
if (size > req->rl_size) {
dprintk("RPC: %s: size %zd too large for buffer[%zd]: "
@@ -503,18 +488,6 @@
* If the allocation or registration fails, the RPC framework
* will (doggedly) retry.
*/
- if (rpcx_to_rdmax(xprt)->rx_ia.ri_memreg_strategy ==
- RPCRDMA_BOUNCEBUFFERS) {
- /* forced to "pure inline" */
- dprintk("RPC: %s: too much data (%zd) for inline "
- "(r/w max %d/%d)\n", __func__, size,
- rpcx_to_rdmad(xprt).inline_rsize,
- rpcx_to_rdmad(xprt).inline_wsize);
- size = req->rl_size;
- rpc_exit(task, -EIO); /* fail the operation */
- rpcx_to_rdmax(xprt)->rx_stats.failed_marshal_count++;
- goto out;
- }
if (task->tk_flags & RPC_TASK_SWAPPER)
nreq = kmalloc(sizeof *req + size, GFP_ATOMIC);
else
@@ -543,7 +516,6 @@
req = nreq;
}
dprintk("RPC: %s: size %zd, request 0x%p\n", __func__, size, req);
-out:
req->rl_connect_cookie = 0; /* our reserved value */
return req->rl_xdr_buf;
@@ -579,9 +551,7 @@
__func__, rep, (rep && rep->rr_func) ? " (with waiter)" : "");
/*
- * Finish the deregistration. When using mw bind, this was
- * begun in rpcrdma_reply_handler(). In all other modes, we
- * do it here, in thread context. The process is considered
+ * Finish the deregistration. The process is considered
* complete when the rr_func vector becomes NULL - this
* was put in place during rpcrdma_reply_handler() - the wait
* call below will not block if the dereg is "done". If
@@ -590,12 +560,7 @@
for (i = 0; req->rl_nchunks;) {
--req->rl_nchunks;
i += rpcrdma_deregister_external(
- &req->rl_segments[i], r_xprt, NULL);
- }
-
- if (rep && wait_event_interruptible(rep->rr_unbind, !rep->rr_func)) {
- rep->rr_func = NULL; /* abandon the callback */
- req->rl_reply = NULL;
+ &req->rl_segments[i], r_xprt);
}
if (req->rl_iov.length == 0) { /* see allocate above */
@@ -630,13 +595,12 @@
struct rpc_xprt *xprt = rqst->rq_xprt;
struct rpcrdma_req *req = rpcr_to_rdmar(rqst);
struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt);
+ int rc;
- /* marshal the send itself */
- if (req->rl_niovs == 0 && rpcrdma_marshal_req(rqst) != 0) {
- r_xprt->rx_stats.failed_marshal_count++;
- dprintk("RPC: %s: rpcrdma_marshal_req failed\n",
- __func__);
- return -EIO;
+ if (req->rl_niovs == 0) {
+ rc = rpcrdma_marshal_req(rqst);
+ if (rc < 0)
+ goto failed_marshal;
}
if (req->rl_reply == NULL) /* e.g. reconnection */
@@ -660,6 +624,12 @@
rqst->rq_bytes_sent = 0;
return 0;
+failed_marshal:
+ r_xprt->rx_stats.failed_marshal_count++;
+ dprintk("RPC: %s: rpcrdma_marshal_req failed, status %i\n",
+ __func__, rc);
+ if (rc == -EIO)
+ return -EIO;
drop_connection:
xprt_disconnect_done(xprt);
return -ENOTCONN; /* implies disconnect */
@@ -705,7 +675,7 @@
*/
static struct rpc_xprt_ops xprt_rdma_procs = {
- .reserve_xprt = xprt_rdma_reserve_xprt,
+ .reserve_xprt = xprt_reserve_xprt_cong,
.release_xprt = xprt_release_xprt_cong, /* sunrpc/xprt.c */
.alloc_slot = xprt_alloc_slot,
.release_request = xprt_release_rqst_cong, /* ditto */
diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c
index 93726560..13dbd1c 100644
--- a/net/sunrpc/xprtrdma/verbs.c
+++ b/net/sunrpc/xprtrdma/verbs.c
@@ -48,8 +48,8 @@
*/
#include <linux/interrupt.h>
-#include <linux/pci.h> /* for Tavor hack below */
#include <linux/slab.h>
+#include <asm/bitops.h>
#include "xprt_rdma.h"
@@ -142,98 +142,139 @@
}
}
-static inline
-void rpcrdma_event_process(struct ib_wc *wc)
+static void
+rpcrdma_sendcq_process_wc(struct ib_wc *wc)
{
- struct rpcrdma_mw *frmr;
- struct rpcrdma_rep *rep =
- (struct rpcrdma_rep *)(unsigned long) wc->wr_id;
+ struct rpcrdma_mw *frmr = (struct rpcrdma_mw *)(unsigned long)wc->wr_id;
- dprintk("RPC: %s: event rep %p status %X opcode %X length %u\n",
- __func__, rep, wc->status, wc->opcode, wc->byte_len);
+ dprintk("RPC: %s: frmr %p status %X opcode %d\n",
+ __func__, frmr, wc->status, wc->opcode);
- if (!rep) /* send or bind completion that we don't care about */
+ if (wc->wr_id == 0ULL)
+ return;
+ if (wc->status != IB_WC_SUCCESS)
return;
- if (IB_WC_SUCCESS != wc->status) {
- dprintk("RPC: %s: WC opcode %d status %X, connection lost\n",
- __func__, wc->opcode, wc->status);
- rep->rr_len = ~0U;
- if (wc->opcode != IB_WC_FAST_REG_MR && wc->opcode != IB_WC_LOCAL_INV)
- rpcrdma_schedule_tasklet(rep);
- return;
- }
-
- switch (wc->opcode) {
- case IB_WC_FAST_REG_MR:
- frmr = (struct rpcrdma_mw *)(unsigned long)wc->wr_id;
+ if (wc->opcode == IB_WC_FAST_REG_MR)
frmr->r.frmr.state = FRMR_IS_VALID;
- break;
- case IB_WC_LOCAL_INV:
- frmr = (struct rpcrdma_mw *)(unsigned long)wc->wr_id;
+ else if (wc->opcode == IB_WC_LOCAL_INV)
frmr->r.frmr.state = FRMR_IS_INVALID;
- break;
- case IB_WC_RECV:
- rep->rr_len = wc->byte_len;
- ib_dma_sync_single_for_cpu(
- rdmab_to_ia(rep->rr_buffer)->ri_id->device,
- rep->rr_iov.addr, rep->rr_len, DMA_FROM_DEVICE);
- /* Keep (only) the most recent credits, after check validity */
- if (rep->rr_len >= 16) {
- struct rpcrdma_msg *p =
- (struct rpcrdma_msg *) rep->rr_base;
- unsigned int credits = ntohl(p->rm_credit);
- if (credits == 0) {
- dprintk("RPC: %s: server"
- " dropped credits to 0!\n", __func__);
- /* don't deadlock */
- credits = 1;
- } else if (credits > rep->rr_buffer->rb_max_requests) {
- dprintk("RPC: %s: server"
- " over-crediting: %d (%d)\n",
- __func__, credits,
- rep->rr_buffer->rb_max_requests);
- credits = rep->rr_buffer->rb_max_requests;
- }
- atomic_set(&rep->rr_buffer->rb_credits, credits);
- }
- /* fall through */
- case IB_WC_BIND_MW:
- rpcrdma_schedule_tasklet(rep);
- break;
- default:
- dprintk("RPC: %s: unexpected WC event %X\n",
- __func__, wc->opcode);
- break;
- }
}
-static inline int
-rpcrdma_cq_poll(struct ib_cq *cq)
+static int
+rpcrdma_sendcq_poll(struct ib_cq *cq, struct rpcrdma_ep *ep)
{
- struct ib_wc wc;
- int rc;
+ struct ib_wc *wcs;
+ int budget, count, rc;
- for (;;) {
- rc = ib_poll_cq(cq, 1, &wc);
- if (rc < 0) {
- dprintk("RPC: %s: ib_poll_cq failed %i\n",
- __func__, rc);
+ budget = RPCRDMA_WC_BUDGET / RPCRDMA_POLLSIZE;
+ do {
+ wcs = ep->rep_send_wcs;
+
+ rc = ib_poll_cq(cq, RPCRDMA_POLLSIZE, wcs);
+ if (rc <= 0)
return rc;
- }
- if (rc == 0)
- break;
- rpcrdma_event_process(&wc);
- }
-
+ count = rc;
+ while (count-- > 0)
+ rpcrdma_sendcq_process_wc(wcs++);
+ } while (rc == RPCRDMA_POLLSIZE && --budget);
return 0;
}
/*
- * rpcrdma_cq_event_upcall
+ * Handle send, fast_reg_mr, and local_inv completions.
*
- * This upcall handles recv, send, bind and unbind events.
+ * Send events are typically suppressed and thus do not result
+ * in an upcall. Occasionally one is signaled, however. This
+ * prevents the provider's completion queue from wrapping and
+ * losing a completion.
+ */
+static void
+rpcrdma_sendcq_upcall(struct ib_cq *cq, void *cq_context)
+{
+ struct rpcrdma_ep *ep = (struct rpcrdma_ep *)cq_context;
+ int rc;
+
+ rc = rpcrdma_sendcq_poll(cq, ep);
+ if (rc) {
+ dprintk("RPC: %s: ib_poll_cq failed: %i\n",
+ __func__, rc);
+ return;
+ }
+
+ rc = ib_req_notify_cq(cq,
+ IB_CQ_NEXT_COMP | IB_CQ_REPORT_MISSED_EVENTS);
+ if (rc == 0)
+ return;
+ if (rc < 0) {
+ dprintk("RPC: %s: ib_req_notify_cq failed: %i\n",
+ __func__, rc);
+ return;
+ }
+
+ rpcrdma_sendcq_poll(cq, ep);
+}
+
+static void
+rpcrdma_recvcq_process_wc(struct ib_wc *wc)
+{
+ struct rpcrdma_rep *rep =
+ (struct rpcrdma_rep *)(unsigned long)wc->wr_id;
+
+ dprintk("RPC: %s: rep %p status %X opcode %X length %u\n",
+ __func__, rep, wc->status, wc->opcode, wc->byte_len);
+
+ if (wc->status != IB_WC_SUCCESS) {
+ rep->rr_len = ~0U;
+ goto out_schedule;
+ }
+ if (wc->opcode != IB_WC_RECV)
+ return;
+
+ rep->rr_len = wc->byte_len;
+ ib_dma_sync_single_for_cpu(rdmab_to_ia(rep->rr_buffer)->ri_id->device,
+ rep->rr_iov.addr, rep->rr_len, DMA_FROM_DEVICE);
+
+ if (rep->rr_len >= 16) {
+ struct rpcrdma_msg *p = (struct rpcrdma_msg *)rep->rr_base;
+ unsigned int credits = ntohl(p->rm_credit);
+
+ if (credits == 0)
+ credits = 1; /* don't deadlock */
+ else if (credits > rep->rr_buffer->rb_max_requests)
+ credits = rep->rr_buffer->rb_max_requests;
+ atomic_set(&rep->rr_buffer->rb_credits, credits);
+ }
+
+out_schedule:
+ rpcrdma_schedule_tasklet(rep);
+}
+
+static int
+rpcrdma_recvcq_poll(struct ib_cq *cq, struct rpcrdma_ep *ep)
+{
+ struct ib_wc *wcs;
+ int budget, count, rc;
+
+ budget = RPCRDMA_WC_BUDGET / RPCRDMA_POLLSIZE;
+ do {
+ wcs = ep->rep_recv_wcs;
+
+ rc = ib_poll_cq(cq, RPCRDMA_POLLSIZE, wcs);
+ if (rc <= 0)
+ return rc;
+
+ count = rc;
+ while (count-- > 0)
+ rpcrdma_recvcq_process_wc(wcs++);
+ } while (rc == RPCRDMA_POLLSIZE && --budget);
+ return 0;
+}
+
+/*
+ * Handle receive completions.
+ *
* It is reentrant but processes single events in order to maintain
* ordering of receives to keep server credits.
*
@@ -242,26 +283,31 @@
* connection shutdown. That is, the structures required for
* the completion of the reply handler must remain intact until
* all memory has been reclaimed.
- *
- * Note that send events are suppressed and do not result in an upcall.
*/
static void
-rpcrdma_cq_event_upcall(struct ib_cq *cq, void *context)
+rpcrdma_recvcq_upcall(struct ib_cq *cq, void *cq_context)
{
+ struct rpcrdma_ep *ep = (struct rpcrdma_ep *)cq_context;
int rc;
- rc = rpcrdma_cq_poll(cq);
- if (rc)
- return;
-
- rc = ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
+ rc = rpcrdma_recvcq_poll(cq, ep);
if (rc) {
- dprintk("RPC: %s: ib_req_notify_cq failed %i\n",
+ dprintk("RPC: %s: ib_poll_cq failed: %i\n",
__func__, rc);
return;
}
- rpcrdma_cq_poll(cq);
+ rc = ib_req_notify_cq(cq,
+ IB_CQ_NEXT_COMP | IB_CQ_REPORT_MISSED_EVENTS);
+ if (rc == 0)
+ return;
+ if (rc < 0) {
+ dprintk("RPC: %s: ib_req_notify_cq failed: %i\n",
+ __func__, rc);
+ return;
+ }
+
+ rpcrdma_recvcq_poll(cq, ep);
}
#ifdef RPC_DEBUG
@@ -493,54 +539,32 @@
ia->ri_dma_lkey = ia->ri_id->device->local_dma_lkey;
}
- switch (memreg) {
- case RPCRDMA_MEMWINDOWS:
- case RPCRDMA_MEMWINDOWS_ASYNC:
- if (!(devattr.device_cap_flags & IB_DEVICE_MEM_WINDOW)) {
- dprintk("RPC: %s: MEMWINDOWS registration "
- "specified but not supported by adapter, "
- "using slower RPCRDMA_REGISTER\n",
- __func__);
- memreg = RPCRDMA_REGISTER;
- }
- break;
- case RPCRDMA_MTHCAFMR:
- if (!ia->ri_id->device->alloc_fmr) {
-#if RPCRDMA_PERSISTENT_REGISTRATION
- dprintk("RPC: %s: MTHCAFMR registration "
- "specified but not supported by adapter, "
- "using riskier RPCRDMA_ALLPHYSICAL\n",
- __func__);
- memreg = RPCRDMA_ALLPHYSICAL;
-#else
- dprintk("RPC: %s: MTHCAFMR registration "
- "specified but not supported by adapter, "
- "using slower RPCRDMA_REGISTER\n",
- __func__);
- memreg = RPCRDMA_REGISTER;
-#endif
- }
- break;
- case RPCRDMA_FRMR:
+ if (memreg == RPCRDMA_FRMR) {
/* Requires both frmr reg and local dma lkey */
if ((devattr.device_cap_flags &
(IB_DEVICE_MEM_MGT_EXTENSIONS|IB_DEVICE_LOCAL_DMA_LKEY)) !=
(IB_DEVICE_MEM_MGT_EXTENSIONS|IB_DEVICE_LOCAL_DMA_LKEY)) {
-#if RPCRDMA_PERSISTENT_REGISTRATION
dprintk("RPC: %s: FRMR registration "
- "specified but not supported by adapter, "
- "using riskier RPCRDMA_ALLPHYSICAL\n",
- __func__);
+ "not supported by HCA\n", __func__);
+ memreg = RPCRDMA_MTHCAFMR;
+ } else {
+ /* Mind the ia limit on FRMR page list depth */
+ ia->ri_max_frmr_depth = min_t(unsigned int,
+ RPCRDMA_MAX_DATA_SEGS,
+ devattr.max_fast_reg_page_list_len);
+ }
+ }
+ if (memreg == RPCRDMA_MTHCAFMR) {
+ if (!ia->ri_id->device->alloc_fmr) {
+ dprintk("RPC: %s: MTHCAFMR registration "
+ "not supported by HCA\n", __func__);
+#if RPCRDMA_PERSISTENT_REGISTRATION
memreg = RPCRDMA_ALLPHYSICAL;
#else
- dprintk("RPC: %s: FRMR registration "
- "specified but not supported by adapter, "
- "using slower RPCRDMA_REGISTER\n",
- __func__);
- memreg = RPCRDMA_REGISTER;
+ rc = -ENOMEM;
+ goto out2;
#endif
}
- break;
}
/*
@@ -552,8 +576,6 @@
* adapter.
*/
switch (memreg) {
- case RPCRDMA_BOUNCEBUFFERS:
- case RPCRDMA_REGISTER:
case RPCRDMA_FRMR:
break;
#if RPCRDMA_PERSISTENT_REGISTRATION
@@ -563,30 +585,26 @@
IB_ACCESS_REMOTE_READ;
goto register_setup;
#endif
- case RPCRDMA_MEMWINDOWS_ASYNC:
- case RPCRDMA_MEMWINDOWS:
- mem_priv = IB_ACCESS_LOCAL_WRITE |
- IB_ACCESS_MW_BIND;
- goto register_setup;
case RPCRDMA_MTHCAFMR:
if (ia->ri_have_dma_lkey)
break;
mem_priv = IB_ACCESS_LOCAL_WRITE;
+#if RPCRDMA_PERSISTENT_REGISTRATION
register_setup:
+#endif
ia->ri_bind_mem = ib_get_dma_mr(ia->ri_pd, mem_priv);
if (IS_ERR(ia->ri_bind_mem)) {
printk(KERN_ALERT "%s: ib_get_dma_mr for "
- "phys register failed with %lX\n\t"
- "Will continue with degraded performance\n",
+ "phys register failed with %lX\n",
__func__, PTR_ERR(ia->ri_bind_mem));
- memreg = RPCRDMA_REGISTER;
- ia->ri_bind_mem = NULL;
+ rc = -ENOMEM;
+ goto out2;
}
break;
default:
- printk(KERN_ERR "%s: invalid memory registration mode %d\n",
- __func__, memreg);
- rc = -EINVAL;
+ printk(KERN_ERR "RPC: Unsupported memory "
+ "registration mode: %d\n", memreg);
+ rc = -ENOMEM;
goto out2;
}
dprintk("RPC: %s: memory registration strategy is %d\n",
@@ -640,6 +658,7 @@
struct rpcrdma_create_data_internal *cdata)
{
struct ib_device_attr devattr;
+ struct ib_cq *sendcq, *recvcq;
int rc, err;
rc = ib_query_device(ia->ri_id->device, &devattr);
@@ -659,32 +678,42 @@
ep->rep_attr.srq = NULL;
ep->rep_attr.cap.max_send_wr = cdata->max_requests;
switch (ia->ri_memreg_strategy) {
- case RPCRDMA_FRMR:
+ case RPCRDMA_FRMR: {
+ int depth = 7;
+
/* Add room for frmr register and invalidate WRs.
* 1. FRMR reg WR for head
* 2. FRMR invalidate WR for head
- * 3. FRMR reg WR for pagelist
- * 4. FRMR invalidate WR for pagelist
+ * 3. N FRMR reg WRs for pagelist
+ * 4. N FRMR invalidate WRs for pagelist
* 5. FRMR reg WR for tail
* 6. FRMR invalidate WR for tail
* 7. The RDMA_SEND WR
*/
- ep->rep_attr.cap.max_send_wr *= 7;
+
+ /* Calculate N if the device max FRMR depth is smaller than
+ * RPCRDMA_MAX_DATA_SEGS.
+ */
+ if (ia->ri_max_frmr_depth < RPCRDMA_MAX_DATA_SEGS) {
+ int delta = RPCRDMA_MAX_DATA_SEGS -
+ ia->ri_max_frmr_depth;
+
+ do {
+ depth += 2; /* FRMR reg + invalidate */
+ delta -= ia->ri_max_frmr_depth;
+ } while (delta > 0);
+
+ }
+ ep->rep_attr.cap.max_send_wr *= depth;
if (ep->rep_attr.cap.max_send_wr > devattr.max_qp_wr) {
- cdata->max_requests = devattr.max_qp_wr / 7;
+ cdata->max_requests = devattr.max_qp_wr / depth;
if (!cdata->max_requests)
return -EINVAL;
- ep->rep_attr.cap.max_send_wr = cdata->max_requests * 7;
+ ep->rep_attr.cap.max_send_wr = cdata->max_requests *
+ depth;
}
break;
- case RPCRDMA_MEMWINDOWS_ASYNC:
- case RPCRDMA_MEMWINDOWS:
- /* Add room for mw_binds+unbinds - overkill! */
- ep->rep_attr.cap.max_send_wr++;
- ep->rep_attr.cap.max_send_wr *= (2 * RPCRDMA_MAX_SEGS);
- if (ep->rep_attr.cap.max_send_wr > devattr.max_qp_wr)
- return -EINVAL;
- break;
+ }
default:
break;
}
@@ -705,46 +734,51 @@
ep->rep_attr.cap.max_recv_sge);
/* set trigger for requesting send completion */
- ep->rep_cqinit = ep->rep_attr.cap.max_send_wr/2 /* - 1*/;
- switch (ia->ri_memreg_strategy) {
- case RPCRDMA_MEMWINDOWS_ASYNC:
- case RPCRDMA_MEMWINDOWS:
- ep->rep_cqinit -= RPCRDMA_MAX_SEGS;
- break;
- default:
- break;
- }
+ ep->rep_cqinit = ep->rep_attr.cap.max_send_wr/2 - 1;
if (ep->rep_cqinit <= 2)
ep->rep_cqinit = 0;
INIT_CQCOUNT(ep);
ep->rep_ia = ia;
init_waitqueue_head(&ep->rep_connect_wait);
+ INIT_DELAYED_WORK(&ep->rep_connect_worker, rpcrdma_connect_worker);
- /*
- * Create a single cq for receive dto and mw_bind (only ever
- * care about unbind, really). Send completions are suppressed.
- * Use single threaded tasklet upcalls to maintain ordering.
- */
- ep->rep_cq = ib_create_cq(ia->ri_id->device, rpcrdma_cq_event_upcall,
- rpcrdma_cq_async_error_upcall, NULL,
- ep->rep_attr.cap.max_recv_wr +
+ sendcq = ib_create_cq(ia->ri_id->device, rpcrdma_sendcq_upcall,
+ rpcrdma_cq_async_error_upcall, ep,
ep->rep_attr.cap.max_send_wr + 1, 0);
- if (IS_ERR(ep->rep_cq)) {
- rc = PTR_ERR(ep->rep_cq);
- dprintk("RPC: %s: ib_create_cq failed: %i\n",
+ if (IS_ERR(sendcq)) {
+ rc = PTR_ERR(sendcq);
+ dprintk("RPC: %s: failed to create send CQ: %i\n",
__func__, rc);
goto out1;
}
- rc = ib_req_notify_cq(ep->rep_cq, IB_CQ_NEXT_COMP);
+ rc = ib_req_notify_cq(sendcq, IB_CQ_NEXT_COMP);
if (rc) {
dprintk("RPC: %s: ib_req_notify_cq failed: %i\n",
__func__, rc);
goto out2;
}
- ep->rep_attr.send_cq = ep->rep_cq;
- ep->rep_attr.recv_cq = ep->rep_cq;
+ recvcq = ib_create_cq(ia->ri_id->device, rpcrdma_recvcq_upcall,
+ rpcrdma_cq_async_error_upcall, ep,
+ ep->rep_attr.cap.max_recv_wr + 1, 0);
+ if (IS_ERR(recvcq)) {
+ rc = PTR_ERR(recvcq);
+ dprintk("RPC: %s: failed to create recv CQ: %i\n",
+ __func__, rc);
+ goto out2;
+ }
+
+ rc = ib_req_notify_cq(recvcq, IB_CQ_NEXT_COMP);
+ if (rc) {
+ dprintk("RPC: %s: ib_req_notify_cq failed: %i\n",
+ __func__, rc);
+ ib_destroy_cq(recvcq);
+ goto out2;
+ }
+
+ ep->rep_attr.send_cq = sendcq;
+ ep->rep_attr.recv_cq = recvcq;
/* Initialize cma parameters */
@@ -754,9 +788,7 @@
/* Client offers RDMA Read but does not initiate */
ep->rep_remote_cma.initiator_depth = 0;
- if (ia->ri_memreg_strategy == RPCRDMA_BOUNCEBUFFERS)
- ep->rep_remote_cma.responder_resources = 0;
- else if (devattr.max_qp_rd_atom > 32) /* arbitrary but <= 255 */
+ if (devattr.max_qp_rd_atom > 32) /* arbitrary but <= 255 */
ep->rep_remote_cma.responder_resources = 32;
else
ep->rep_remote_cma.responder_resources = devattr.max_qp_rd_atom;
@@ -768,7 +800,7 @@
return 0;
out2:
- err = ib_destroy_cq(ep->rep_cq);
+ err = ib_destroy_cq(sendcq);
if (err)
dprintk("RPC: %s: ib_destroy_cq returned %i\n",
__func__, err);
@@ -782,11 +814,8 @@
* Disconnect and destroy endpoint. After this, the only
* valid operations on the ep are to free it (if dynamically
* allocated) or re-create it.
- *
- * The caller's error handling must be sure to not leak the endpoint
- * if this function fails.
*/
-int
+void
rpcrdma_ep_destroy(struct rpcrdma_ep *ep, struct rpcrdma_ia *ia)
{
int rc;
@@ -794,6 +823,8 @@
dprintk("RPC: %s: entering, connected is %d\n",
__func__, ep->rep_connected);
+ cancel_delayed_work_sync(&ep->rep_connect_worker);
+
if (ia->ri_id->qp) {
rc = rpcrdma_ep_disconnect(ep, ia);
if (rc)
@@ -809,13 +840,17 @@
ep->rep_pad_mr = NULL;
}
- rpcrdma_clean_cq(ep->rep_cq);
- rc = ib_destroy_cq(ep->rep_cq);
+ rpcrdma_clean_cq(ep->rep_attr.recv_cq);
+ rc = ib_destroy_cq(ep->rep_attr.recv_cq);
if (rc)
dprintk("RPC: %s: ib_destroy_cq returned %i\n",
__func__, rc);
- return rc;
+ rpcrdma_clean_cq(ep->rep_attr.send_cq);
+ rc = ib_destroy_cq(ep->rep_attr.send_cq);
+ if (rc)
+ dprintk("RPC: %s: ib_destroy_cq returned %i\n",
+ __func__, rc);
}
/*
@@ -831,17 +866,20 @@
if (ep->rep_connected != 0) {
struct rpcrdma_xprt *xprt;
retry:
+ dprintk("RPC: %s: reconnecting...\n", __func__);
rc = rpcrdma_ep_disconnect(ep, ia);
if (rc && rc != -ENOTCONN)
dprintk("RPC: %s: rpcrdma_ep_disconnect"
" status %i\n", __func__, rc);
- rpcrdma_clean_cq(ep->rep_cq);
+
+ rpcrdma_clean_cq(ep->rep_attr.recv_cq);
+ rpcrdma_clean_cq(ep->rep_attr.send_cq);
xprt = container_of(ia, struct rpcrdma_xprt, rx_ia);
id = rpcrdma_create_id(xprt, ia,
(struct sockaddr *)&xprt->rx_data.addr);
if (IS_ERR(id)) {
- rc = PTR_ERR(id);
+ rc = -EHOSTUNREACH;
goto out;
}
/* TEMP TEMP TEMP - fail if new device:
@@ -855,35 +893,32 @@
printk("RPC: %s: can't reconnect on "
"different device!\n", __func__);
rdma_destroy_id(id);
- rc = -ENETDOWN;
+ rc = -ENETUNREACH;
goto out;
}
/* END TEMP */
+ rc = rdma_create_qp(id, ia->ri_pd, &ep->rep_attr);
+ if (rc) {
+ dprintk("RPC: %s: rdma_create_qp failed %i\n",
+ __func__, rc);
+ rdma_destroy_id(id);
+ rc = -ENETUNREACH;
+ goto out;
+ }
rdma_destroy_qp(ia->ri_id);
rdma_destroy_id(ia->ri_id);
ia->ri_id = id;
+ } else {
+ dprintk("RPC: %s: connecting...\n", __func__);
+ rc = rdma_create_qp(ia->ri_id, ia->ri_pd, &ep->rep_attr);
+ if (rc) {
+ dprintk("RPC: %s: rdma_create_qp failed %i\n",
+ __func__, rc);
+ /* do not update ep->rep_connected */
+ return -ENETUNREACH;
+ }
}
- rc = rdma_create_qp(ia->ri_id, ia->ri_pd, &ep->rep_attr);
- if (rc) {
- dprintk("RPC: %s: rdma_create_qp failed %i\n",
- __func__, rc);
- goto out;
- }
-
-/* XXX Tavor device performs badly with 2K MTU! */
-if (strnicmp(ia->ri_id->device->dma_device->bus->name, "pci", 3) == 0) {
- struct pci_dev *pcid = to_pci_dev(ia->ri_id->device->dma_device);
- if (pcid->device == PCI_DEVICE_ID_MELLANOX_TAVOR &&
- (pcid->vendor == PCI_VENDOR_ID_MELLANOX ||
- pcid->vendor == PCI_VENDOR_ID_TOPSPIN)) {
- struct ib_qp_attr attr = {
- .path_mtu = IB_MTU_1024
- };
- rc = ib_modify_qp(ia->ri_id->qp, &attr, IB_QP_PATH_MTU);
- }
-}
-
ep->rep_connected = 0;
rc = rdma_connect(ia->ri_id, &ep->rep_remote_cma);
@@ -944,7 +979,8 @@
{
int rc;
- rpcrdma_clean_cq(ep->rep_cq);
+ rpcrdma_clean_cq(ep->rep_attr.recv_cq);
+ rpcrdma_clean_cq(ep->rep_attr.send_cq);
rc = rdma_disconnect(ia->ri_id);
if (!rc) {
/* returns without wait if not connected */
@@ -967,7 +1003,7 @@
struct rpcrdma_ia *ia, struct rpcrdma_create_data_internal *cdata)
{
char *p;
- size_t len;
+ size_t len, rlen, wlen;
int i, rc;
struct rpcrdma_mw *r;
@@ -997,11 +1033,6 @@
len += (buf->rb_max_requests + 1) * RPCRDMA_MAX_SEGS *
sizeof(struct rpcrdma_mw);
break;
- case RPCRDMA_MEMWINDOWS_ASYNC:
- case RPCRDMA_MEMWINDOWS:
- len += (buf->rb_max_requests + 1) * RPCRDMA_MAX_SEGS *
- sizeof(struct rpcrdma_mw);
- break;
default:
break;
}
@@ -1032,32 +1063,29 @@
}
p += cdata->padding;
- /*
- * Allocate the fmr's, or mw's for mw_bind chunk registration.
- * We "cycle" the mw's in order to minimize rkey reuse,
- * and also reduce unbind-to-bind collision.
- */
INIT_LIST_HEAD(&buf->rb_mws);
r = (struct rpcrdma_mw *)p;
switch (ia->ri_memreg_strategy) {
case RPCRDMA_FRMR:
for (i = buf->rb_max_requests * RPCRDMA_MAX_SEGS; i; i--) {
r->r.frmr.fr_mr = ib_alloc_fast_reg_mr(ia->ri_pd,
- RPCRDMA_MAX_SEGS);
+ ia->ri_max_frmr_depth);
if (IS_ERR(r->r.frmr.fr_mr)) {
rc = PTR_ERR(r->r.frmr.fr_mr);
dprintk("RPC: %s: ib_alloc_fast_reg_mr"
" failed %i\n", __func__, rc);
goto out;
}
- r->r.frmr.fr_pgl =
- ib_alloc_fast_reg_page_list(ia->ri_id->device,
- RPCRDMA_MAX_SEGS);
+ r->r.frmr.fr_pgl = ib_alloc_fast_reg_page_list(
+ ia->ri_id->device,
+ ia->ri_max_frmr_depth);
if (IS_ERR(r->r.frmr.fr_pgl)) {
rc = PTR_ERR(r->r.frmr.fr_pgl);
dprintk("RPC: %s: "
"ib_alloc_fast_reg_page_list "
"failed %i\n", __func__, rc);
+
+ ib_dereg_mr(r->r.frmr.fr_mr);
goto out;
}
list_add(&r->mw_list, &buf->rb_mws);
@@ -1082,21 +1110,6 @@
++r;
}
break;
- case RPCRDMA_MEMWINDOWS_ASYNC:
- case RPCRDMA_MEMWINDOWS:
- /* Allocate one extra request's worth, for full cycling */
- for (i = (buf->rb_max_requests+1) * RPCRDMA_MAX_SEGS; i; i--) {
- r->r.mw = ib_alloc_mw(ia->ri_pd, IB_MW_TYPE_1);
- if (IS_ERR(r->r.mw)) {
- rc = PTR_ERR(r->r.mw);
- dprintk("RPC: %s: ib_alloc_mw"
- " failed %i\n", __func__, rc);
- goto out;
- }
- list_add(&r->mw_list, &buf->rb_mws);
- ++r;
- }
- break;
default:
break;
}
@@ -1105,16 +1118,16 @@
* Allocate/init the request/reply buffers. Doing this
* using kmalloc for now -- one for each buf.
*/
+ wlen = 1 << fls(cdata->inline_wsize + sizeof(struct rpcrdma_req));
+ rlen = 1 << fls(cdata->inline_rsize + sizeof(struct rpcrdma_rep));
+ dprintk("RPC: %s: wlen = %zu, rlen = %zu\n",
+ __func__, wlen, rlen);
+
for (i = 0; i < buf->rb_max_requests; i++) {
struct rpcrdma_req *req;
struct rpcrdma_rep *rep;
- len = cdata->inline_wsize + sizeof(struct rpcrdma_req);
- /* RPC layer requests *double* size + 1K RPC_SLACK_SPACE! */
- /* Typical ~2400b, so rounding up saves work later */
- if (len < 4096)
- len = 4096;
- req = kmalloc(len, GFP_KERNEL);
+ req = kmalloc(wlen, GFP_KERNEL);
if (req == NULL) {
dprintk("RPC: %s: request buffer %d alloc"
" failed\n", __func__, i);
@@ -1126,16 +1139,16 @@
buf->rb_send_bufs[i]->rl_buffer = buf;
rc = rpcrdma_register_internal(ia, req->rl_base,
- len - offsetof(struct rpcrdma_req, rl_base),
+ wlen - offsetof(struct rpcrdma_req, rl_base),
&buf->rb_send_bufs[i]->rl_handle,
&buf->rb_send_bufs[i]->rl_iov);
if (rc)
goto out;
- buf->rb_send_bufs[i]->rl_size = len-sizeof(struct rpcrdma_req);
+ buf->rb_send_bufs[i]->rl_size = wlen -
+ sizeof(struct rpcrdma_req);
- len = cdata->inline_rsize + sizeof(struct rpcrdma_rep);
- rep = kmalloc(len, GFP_KERNEL);
+ rep = kmalloc(rlen, GFP_KERNEL);
if (rep == NULL) {
dprintk("RPC: %s: reply buffer %d alloc failed\n",
__func__, i);
@@ -1145,10 +1158,9 @@
memset(rep, 0, sizeof(struct rpcrdma_rep));
buf->rb_recv_bufs[i] = rep;
buf->rb_recv_bufs[i]->rr_buffer = buf;
- init_waitqueue_head(&rep->rr_unbind);
rc = rpcrdma_register_internal(ia, rep->rr_base,
- len - offsetof(struct rpcrdma_rep, rr_base),
+ rlen - offsetof(struct rpcrdma_rep, rr_base),
&buf->rb_recv_bufs[i]->rr_handle,
&buf->rb_recv_bufs[i]->rr_iov);
if (rc)
@@ -1179,7 +1191,6 @@
/* clean up in reverse order from create
* 1. recv mr memory (mr free, then kfree)
- * 1a. bind mw memory
* 2. send mr memory (mr free, then kfree)
* 3. padding (if any) [moved to rpcrdma_ep_destroy]
* 4. arrays
@@ -1194,41 +1205,6 @@
kfree(buf->rb_recv_bufs[i]);
}
if (buf->rb_send_bufs && buf->rb_send_bufs[i]) {
- while (!list_empty(&buf->rb_mws)) {
- r = list_entry(buf->rb_mws.next,
- struct rpcrdma_mw, mw_list);
- list_del(&r->mw_list);
- switch (ia->ri_memreg_strategy) {
- case RPCRDMA_FRMR:
- rc = ib_dereg_mr(r->r.frmr.fr_mr);
- if (rc)
- dprintk("RPC: %s:"
- " ib_dereg_mr"
- " failed %i\n",
- __func__, rc);
- ib_free_fast_reg_page_list(r->r.frmr.fr_pgl);
- break;
- case RPCRDMA_MTHCAFMR:
- rc = ib_dealloc_fmr(r->r.fmr);
- if (rc)
- dprintk("RPC: %s:"
- " ib_dealloc_fmr"
- " failed %i\n",
- __func__, rc);
- break;
- case RPCRDMA_MEMWINDOWS_ASYNC:
- case RPCRDMA_MEMWINDOWS:
- rc = ib_dealloc_mw(r->r.mw);
- if (rc)
- dprintk("RPC: %s:"
- " ib_dealloc_mw"
- " failed %i\n",
- __func__, rc);
- break;
- default:
- break;
- }
- }
rpcrdma_deregister_internal(ia,
buf->rb_send_bufs[i]->rl_handle,
&buf->rb_send_bufs[i]->rl_iov);
@@ -1236,6 +1212,33 @@
}
}
+ while (!list_empty(&buf->rb_mws)) {
+ r = list_entry(buf->rb_mws.next,
+ struct rpcrdma_mw, mw_list);
+ list_del(&r->mw_list);
+ switch (ia->ri_memreg_strategy) {
+ case RPCRDMA_FRMR:
+ rc = ib_dereg_mr(r->r.frmr.fr_mr);
+ if (rc)
+ dprintk("RPC: %s:"
+ " ib_dereg_mr"
+ " failed %i\n",
+ __func__, rc);
+ ib_free_fast_reg_page_list(r->r.frmr.fr_pgl);
+ break;
+ case RPCRDMA_MTHCAFMR:
+ rc = ib_dealloc_fmr(r->r.fmr);
+ if (rc)
+ dprintk("RPC: %s:"
+ " ib_dealloc_fmr"
+ " failed %i\n",
+ __func__, rc);
+ break;
+ default:
+ break;
+ }
+ }
+
kfree(buf->rb_pool);
}
@@ -1299,21 +1302,17 @@
int i;
unsigned long flags;
- BUG_ON(req->rl_nchunks != 0);
spin_lock_irqsave(&buffers->rb_lock, flags);
buffers->rb_send_bufs[--buffers->rb_send_index] = req;
req->rl_niovs = 0;
if (req->rl_reply) {
buffers->rb_recv_bufs[--buffers->rb_recv_index] = req->rl_reply;
- init_waitqueue_head(&req->rl_reply->rr_unbind);
req->rl_reply->rr_func = NULL;
req->rl_reply = NULL;
}
switch (ia->ri_memreg_strategy) {
case RPCRDMA_FRMR:
case RPCRDMA_MTHCAFMR:
- case RPCRDMA_MEMWINDOWS_ASYNC:
- case RPCRDMA_MEMWINDOWS:
/*
* Cycle mw's back in reverse order, and "spin" them.
* This delays and scrambles reuse as much as possible.
@@ -1358,8 +1357,7 @@
/*
* Put reply buffers back into pool when not attached to
- * request. This happens in error conditions, and when
- * aborting unbinds. Pre-decrement counter/array index.
+ * request. This happens in error conditions.
*/
void
rpcrdma_recv_buffer_put(struct rpcrdma_rep *rep)
@@ -1498,8 +1496,8 @@
seg1->mr_offset -= pageoff; /* start of page */
seg1->mr_len += pageoff;
len = -pageoff;
- if (*nsegs > RPCRDMA_MAX_DATA_SEGS)
- *nsegs = RPCRDMA_MAX_DATA_SEGS;
+ if (*nsegs > ia->ri_max_frmr_depth)
+ *nsegs = ia->ri_max_frmr_depth;
for (page_no = i = 0; i < *nsegs;) {
rpcrdma_map_one(ia, seg, writing);
pa = seg->mr_dma;
@@ -1536,10 +1534,6 @@
} else
post_wr = &frmr_wr;
- /* Bump the key */
- key = (u8)(seg1->mr_chunk.rl_mw->r.frmr.fr_mr->rkey & 0x000000FF);
- ib_update_fast_reg_key(seg1->mr_chunk.rl_mw->r.frmr.fr_mr, ++key);
-
/* Prepare FRMR WR */
memset(&frmr_wr, 0, sizeof frmr_wr);
frmr_wr.wr_id = (unsigned long)(void *)seg1->mr_chunk.rl_mw;
@@ -1550,7 +1544,16 @@
frmr_wr.wr.fast_reg.page_list_len = page_no;
frmr_wr.wr.fast_reg.page_shift = PAGE_SHIFT;
frmr_wr.wr.fast_reg.length = page_no << PAGE_SHIFT;
- BUG_ON(frmr_wr.wr.fast_reg.length < len);
+ if (frmr_wr.wr.fast_reg.length < len) {
+ while (seg1->mr_nsegs--)
+ rpcrdma_unmap_one(ia, seg++);
+ return -EIO;
+ }
+
+ /* Bump the key */
+ key = (u8)(seg1->mr_chunk.rl_mw->r.frmr.fr_mr->rkey & 0x000000FF);
+ ib_update_fast_reg_key(seg1->mr_chunk.rl_mw->r.frmr.fr_mr, ++key);
+
frmr_wr.wr.fast_reg.access_flags = (writing ?
IB_ACCESS_REMOTE_WRITE | IB_ACCESS_LOCAL_WRITE :
IB_ACCESS_REMOTE_READ);
@@ -1661,135 +1664,6 @@
return rc;
}
-static int
-rpcrdma_register_memwin_external(struct rpcrdma_mr_seg *seg,
- int *nsegs, int writing, struct rpcrdma_ia *ia,
- struct rpcrdma_xprt *r_xprt)
-{
- int mem_priv = (writing ? IB_ACCESS_REMOTE_WRITE :
- IB_ACCESS_REMOTE_READ);
- struct ib_mw_bind param;
- int rc;
-
- *nsegs = 1;
- rpcrdma_map_one(ia, seg, writing);
- param.bind_info.mr = ia->ri_bind_mem;
- param.wr_id = 0ULL; /* no send cookie */
- param.bind_info.addr = seg->mr_dma;
- param.bind_info.length = seg->mr_len;
- param.send_flags = 0;
- param.bind_info.mw_access_flags = mem_priv;
-
- DECR_CQCOUNT(&r_xprt->rx_ep);
- rc = ib_bind_mw(ia->ri_id->qp, seg->mr_chunk.rl_mw->r.mw, ¶m);
- if (rc) {
- dprintk("RPC: %s: failed ib_bind_mw "
- "%u@0x%llx status %i\n",
- __func__, seg->mr_len,
- (unsigned long long)seg->mr_dma, rc);
- rpcrdma_unmap_one(ia, seg);
- } else {
- seg->mr_rkey = seg->mr_chunk.rl_mw->r.mw->rkey;
- seg->mr_base = param.bind_info.addr;
- seg->mr_nsegs = 1;
- }
- return rc;
-}
-
-static int
-rpcrdma_deregister_memwin_external(struct rpcrdma_mr_seg *seg,
- struct rpcrdma_ia *ia,
- struct rpcrdma_xprt *r_xprt, void **r)
-{
- struct ib_mw_bind param;
- LIST_HEAD(l);
- int rc;
-
- BUG_ON(seg->mr_nsegs != 1);
- param.bind_info.mr = ia->ri_bind_mem;
- param.bind_info.addr = 0ULL; /* unbind */
- param.bind_info.length = 0;
- param.bind_info.mw_access_flags = 0;
- if (*r) {
- param.wr_id = (u64) (unsigned long) *r;
- param.send_flags = IB_SEND_SIGNALED;
- INIT_CQCOUNT(&r_xprt->rx_ep);
- } else {
- param.wr_id = 0ULL;
- param.send_flags = 0;
- DECR_CQCOUNT(&r_xprt->rx_ep);
- }
- rc = ib_bind_mw(ia->ri_id->qp, seg->mr_chunk.rl_mw->r.mw, ¶m);
- rpcrdma_unmap_one(ia, seg);
- if (rc)
- dprintk("RPC: %s: failed ib_(un)bind_mw,"
- " status %i\n", __func__, rc);
- else
- *r = NULL; /* will upcall on completion */
- return rc;
-}
-
-static int
-rpcrdma_register_default_external(struct rpcrdma_mr_seg *seg,
- int *nsegs, int writing, struct rpcrdma_ia *ia)
-{
- int mem_priv = (writing ? IB_ACCESS_REMOTE_WRITE :
- IB_ACCESS_REMOTE_READ);
- struct rpcrdma_mr_seg *seg1 = seg;
- struct ib_phys_buf ipb[RPCRDMA_MAX_DATA_SEGS];
- int len, i, rc = 0;
-
- if (*nsegs > RPCRDMA_MAX_DATA_SEGS)
- *nsegs = RPCRDMA_MAX_DATA_SEGS;
- for (len = 0, i = 0; i < *nsegs;) {
- rpcrdma_map_one(ia, seg, writing);
- ipb[i].addr = seg->mr_dma;
- ipb[i].size = seg->mr_len;
- len += seg->mr_len;
- ++seg;
- ++i;
- /* Check for holes */
- if ((i < *nsegs && offset_in_page(seg->mr_offset)) ||
- offset_in_page((seg-1)->mr_offset+(seg-1)->mr_len))
- break;
- }
- seg1->mr_base = seg1->mr_dma;
- seg1->mr_chunk.rl_mr = ib_reg_phys_mr(ia->ri_pd,
- ipb, i, mem_priv, &seg1->mr_base);
- if (IS_ERR(seg1->mr_chunk.rl_mr)) {
- rc = PTR_ERR(seg1->mr_chunk.rl_mr);
- dprintk("RPC: %s: failed ib_reg_phys_mr "
- "%u@0x%llx (%d)... status %i\n",
- __func__, len,
- (unsigned long long)seg1->mr_dma, i, rc);
- while (i--)
- rpcrdma_unmap_one(ia, --seg);
- } else {
- seg1->mr_rkey = seg1->mr_chunk.rl_mr->rkey;
- seg1->mr_nsegs = i;
- seg1->mr_len = len;
- }
- *nsegs = i;
- return rc;
-}
-
-static int
-rpcrdma_deregister_default_external(struct rpcrdma_mr_seg *seg,
- struct rpcrdma_ia *ia)
-{
- struct rpcrdma_mr_seg *seg1 = seg;
- int rc;
-
- rc = ib_dereg_mr(seg1->mr_chunk.rl_mr);
- seg1->mr_chunk.rl_mr = NULL;
- while (seg1->mr_nsegs--)
- rpcrdma_unmap_one(ia, seg++);
- if (rc)
- dprintk("RPC: %s: failed ib_dereg_mr,"
- " status %i\n", __func__, rc);
- return rc;
-}
-
int
rpcrdma_register_external(struct rpcrdma_mr_seg *seg,
int nsegs, int writing, struct rpcrdma_xprt *r_xprt)
@@ -1819,16 +1693,8 @@
rc = rpcrdma_register_fmr_external(seg, &nsegs, writing, ia);
break;
- /* Registration using memory windows */
- case RPCRDMA_MEMWINDOWS_ASYNC:
- case RPCRDMA_MEMWINDOWS:
- rc = rpcrdma_register_memwin_external(seg, &nsegs, writing, ia, r_xprt);
- break;
-
- /* Default registration each time */
default:
- rc = rpcrdma_register_default_external(seg, &nsegs, writing, ia);
- break;
+ return -1;
}
if (rc)
return -1;
@@ -1838,7 +1704,7 @@
int
rpcrdma_deregister_external(struct rpcrdma_mr_seg *seg,
- struct rpcrdma_xprt *r_xprt, void *r)
+ struct rpcrdma_xprt *r_xprt)
{
struct rpcrdma_ia *ia = &r_xprt->rx_ia;
int nsegs = seg->mr_nsegs, rc;
@@ -1847,9 +1713,7 @@
#if RPCRDMA_PERSISTENT_REGISTRATION
case RPCRDMA_ALLPHYSICAL:
- BUG_ON(nsegs != 1);
rpcrdma_unmap_one(ia, seg);
- rc = 0;
break;
#endif
@@ -1861,21 +1725,9 @@
rc = rpcrdma_deregister_fmr_external(seg, ia);
break;
- case RPCRDMA_MEMWINDOWS_ASYNC:
- case RPCRDMA_MEMWINDOWS:
- rc = rpcrdma_deregister_memwin_external(seg, ia, r_xprt, &r);
- break;
-
default:
- rc = rpcrdma_deregister_default_external(seg, ia);
break;
}
- if (r) {
- struct rpcrdma_rep *rep = r;
- void (*func)(struct rpcrdma_rep *) = rep->rr_func;
- rep->rr_func = NULL;
- func(rep); /* dereg done, callback now */
- }
return nsegs;
}
@@ -1950,7 +1802,6 @@
ib_dma_sync_single_for_cpu(ia->ri_id->device,
rep->rr_iov.addr, rep->rr_iov.length, DMA_BIDIRECTIONAL);
- DECR_CQCOUNT(ep);
rc = ib_post_recv(ia->ri_id->qp, &recv_wr, &recv_wr_fail);
if (rc)
diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
index cc1445d..89e7cd4 100644
--- a/net/sunrpc/xprtrdma/xprt_rdma.h
+++ b/net/sunrpc/xprtrdma/xprt_rdma.h
@@ -43,6 +43,7 @@
#include <linux/wait.h> /* wait_queue_head_t, etc */
#include <linux/spinlock.h> /* spinlock_t, etc */
#include <linux/atomic.h> /* atomic_t, etc */
+#include <linux/workqueue.h> /* struct work_struct */
#include <rdma/rdma_cm.h> /* RDMA connection api */
#include <rdma/ib_verbs.h> /* RDMA verbs api */
@@ -66,18 +67,21 @@
struct completion ri_done;
int ri_async_rc;
enum rpcrdma_memreg ri_memreg_strategy;
+ unsigned int ri_max_frmr_depth;
};
/*
* RDMA Endpoint -- one per transport instance
*/
+#define RPCRDMA_WC_BUDGET (128)
+#define RPCRDMA_POLLSIZE (16)
+
struct rpcrdma_ep {
atomic_t rep_cqcount;
int rep_cqinit;
int rep_connected;
struct rpcrdma_ia *rep_ia;
- struct ib_cq *rep_cq;
struct ib_qp_init_attr rep_attr;
wait_queue_head_t rep_connect_wait;
struct ib_sge rep_pad; /* holds zeroed pad */
@@ -86,6 +90,9 @@
struct rpc_xprt *rep_xprt; /* for rep_func */
struct rdma_conn_param rep_remote_cma;
struct sockaddr_storage rep_remote_addr;
+ struct delayed_work rep_connect_worker;
+ struct ib_wc rep_send_wcs[RPCRDMA_POLLSIZE];
+ struct ib_wc rep_recv_wcs[RPCRDMA_POLLSIZE];
};
#define INIT_CQCOUNT(ep) atomic_set(&(ep)->rep_cqcount, (ep)->rep_cqinit)
@@ -124,7 +131,6 @@
struct rpc_xprt *rr_xprt; /* needed for request/reply matching */
void (*rr_func)(struct rpcrdma_rep *);/* called by tasklet in softint */
struct list_head rr_list; /* tasklet list */
- wait_queue_head_t rr_unbind; /* optional unbind wait */
struct ib_sge rr_iov; /* for posting */
struct ib_mr *rr_handle; /* handle for mem in rr_iov */
char rr_base[MAX_RPCRDMAHDR]; /* minimal inline receive buffer */
@@ -159,7 +165,6 @@
struct ib_mr *rl_mr; /* if registered directly */
struct rpcrdma_mw { /* if registered from region */
union {
- struct ib_mw *mw;
struct ib_fmr *fmr;
struct {
struct ib_fast_reg_page_list *fr_pgl;
@@ -207,7 +212,6 @@
struct rpcrdma_buffer {
spinlock_t rb_lock; /* protects indexes */
atomic_t rb_credits; /* most recent server credits */
- unsigned long rb_cwndscale; /* cached framework rpc_cwndscale */
int rb_max_requests;/* client max requests */
struct list_head rb_mws; /* optional memory windows/fmrs/frmrs */
int rb_send_index;
@@ -300,7 +304,7 @@
*/
int rpcrdma_ep_create(struct rpcrdma_ep *, struct rpcrdma_ia *,
struct rpcrdma_create_data_internal *);
-int rpcrdma_ep_destroy(struct rpcrdma_ep *, struct rpcrdma_ia *);
+void rpcrdma_ep_destroy(struct rpcrdma_ep *, struct rpcrdma_ia *);
int rpcrdma_ep_connect(struct rpcrdma_ep *, struct rpcrdma_ia *);
int rpcrdma_ep_disconnect(struct rpcrdma_ep *, struct rpcrdma_ia *);
@@ -330,11 +334,12 @@
int rpcrdma_register_external(struct rpcrdma_mr_seg *,
int, int, struct rpcrdma_xprt *);
int rpcrdma_deregister_external(struct rpcrdma_mr_seg *,
- struct rpcrdma_xprt *, void *);
+ struct rpcrdma_xprt *);
/*
* RPC/RDMA connection management calls - xprtrdma/rpc_rdma.c
*/
+void rpcrdma_connect_worker(struct work_struct *);
void rpcrdma_conn_func(struct rpcrdma_ep *);
void rpcrdma_reply_handler(struct rpcrdma_rep *);
diff --git a/scripts/decode_stacktrace.sh b/scripts/decode_stacktrace.sh
new file mode 100755
index 0000000..515c4c0
--- /dev/null
+++ b/scripts/decode_stacktrace.sh
@@ -0,0 +1,126 @@
+#!/bin/bash
+# (c) 2014, Sasha Levin <sasha.levin@oracle.com>
+#set -x
+
+if [[ $# != 2 ]]; then
+ echo "Usage:"
+ echo " $0 [vmlinux] [base path]"
+ exit 1
+fi
+
+vmlinux=$1
+basepath=$2
+declare -A cache
+
+parse_symbol() {
+ # The structure of symbol at this point is:
+ # [name]+[offset]/[total length]
+ #
+ # For example:
+ # do_basic_setup+0x9c/0xbf
+
+
+ # Strip the symbol name so that we could look it up
+ local name=${symbol%+*}
+
+ # Use 'nm vmlinux' to figure out the base address of said symbol.
+ # It's actually faster to call it every time than to load it
+ # all into bash.
+ if [[ "${cache[$name]+isset}" == "isset" ]]; then
+ local base_addr=${cache[$name]}
+ else
+ local base_addr=$(nm "$vmlinux" | grep -i ' t ' | awk "/ $name\$/ {print \$1}" | head -n1)
+ cache["$name"]="$base_addr"
+ fi
+ # Let's start doing the math to get the exact address into the
+ # symbol. First, strip out the symbol total length.
+ local expr=${symbol%/*}
+
+ # Now, replace the symbol name with the base address we found
+ # before.
+ expr=${expr/$name/0x$base_addr}
+
+ # Evaluate it to find the actual address
+ expr=$((expr))
+ local address=$(printf "%x\n" "$expr")
+
+ # Pass it to addr2line to get filename and line number
+ # Could get more than one result
+ if [[ "${cache[$address]+isset}" == "isset" ]]; then
+ local code=${cache[$address]}
+ else
+ local code=$(addr2line -i -e "$vmlinux" "$address")
+ cache[$address]=$code
+ fi
+
+ # addr2line doesn't return a proper error code if it fails, so
+ # we detect it using the value it prints so that we could preserve
+ # the offset/size into the function and bail out
+ if [[ $code == "??:0" ]]; then
+ return
+ fi
+
+ # Strip out the base of the path
+ code=${code//$basepath/""}
+
+ # In the case of inlines, move everything to same line
+ code=${code//$'\n'/' '}
+
+ # Replace old address with pretty line numbers
+ symbol="$name ($code)"
+}
+
+decode_code() {
+ local scripts=`dirname "${BASH_SOURCE[0]}"`
+
+ echo "$1" | $scripts/decodecode
+}
+
+handle_line() {
+ local words
+
+ # Tokenize
+ read -a words <<<"$1"
+
+ # Remove hex numbers. Do it ourselves until it happens in the
+ # kernel
+
+ # We need to know the index of the last element before we
+ # remove elements because arrays are sparse
+ local last=$(( ${#words[@]} - 1 ))
+
+ for i in "${!words[@]}"; do
+ # Remove the address
+ if [[ ${words[$i]} =~ \[\<([^]]+)\>\] ]]; then
+ unset words[$i]
+ fi
+
+ # Format timestamps with tabs
+ if [[ ${words[$i]} == \[ && ${words[$i+1]} == *\] ]]; then
+ unset words[$i]
+ words[$i+1]=$(printf "[%13s\n" "${words[$i+1]}")
+ fi
+ done
+
+ # The symbol is the last element, process it
+ symbol=${words[$last]}
+ unset words[$last]
+ parse_symbol # modifies $symbol
+
+ # Add up the line number to the symbol
+ echo "${words[@]}" "$symbol"
+}
+
+while read line; do
+ # Let's see if we have an address in the line
+ if [[ $line =~ \[\<([^]]+)\>\] ]]; then
+ # Translate address to line numbers
+ handle_line "$line"
+ # Is it a code line?
+ elif [[ $line == *Code:* ]]; then
+ decode_code "$line"
+ else
+ # Nothing special in this line, show it as is
+ echo "$line"
+ fi
+done
diff --git a/scripts/recordmcount.c b/scripts/recordmcount.c
index 9c22317..e11aa4a 100644
--- a/scripts/recordmcount.c
+++ b/scripts/recordmcount.c
@@ -40,6 +40,11 @@
#define R_METAG_NONE 3
#endif
+#ifndef EM_AARCH64
+#define EM_AARCH64 183
+#define R_AARCH64_ABS64 257
+#endif
+
static int fd_map; /* File descriptor for file being modified. */
static int mmap_failed; /* Boolean flag. */
static void *ehdr_curr; /* current ElfXX_Ehdr * for resource cleanup */
@@ -347,6 +352,8 @@
case EM_ARM: reltype = R_ARM_ABS32;
altmcount = "__gnu_mcount_nc";
break;
+ case EM_AARCH64:
+ reltype = R_AARCH64_ABS64; gpfx = '_'; break;
case EM_IA_64: reltype = R_IA64_IMM64; gpfx = '_'; break;
case EM_METAG: reltype = R_METAG_ADDR32;
altmcount = "_mcount_wrapper";
diff --git a/scripts/recordmcount.pl b/scripts/recordmcount.pl
index 91280b8..397b6b8 100755
--- a/scripts/recordmcount.pl
+++ b/scripts/recordmcount.pl
@@ -279,6 +279,11 @@
$mcount_regex = "^\\s*([0-9a-fA-F]+):\\s*R_ARM_(CALL|PC24|THM_CALL)" .
"\\s+(__gnu_mcount_nc|mcount)\$";
+} elsif ($arch eq "arm64") {
+ $alignment = 3;
+ $section_type = '%progbits';
+ $mcount_regex = "^\\s*([0-9a-fA-F]+):\\s*R_AARCH64_CALL26\\s+_mcount\$";
+ $type = ".quad";
} elsif ($arch eq "ia64") {
$mcount_regex = "^\\s*([0-9a-fA-F]+):.*\\s_mcount\$";
$type = "data8";
diff --git a/security/capability.c b/security/capability.c
index ad0d4de..e76373d 100644
--- a/security/capability.c
+++ b/security/capability.c
@@ -879,7 +879,7 @@
}
static int cap_key_permission(key_ref_t key_ref, const struct cred *cred,
- key_perm_t perm)
+ unsigned perm)
{
return 0;
}
diff --git a/security/device_cgroup.c b/security/device_cgroup.c
index 9134dbf..d9d69e6 100644
--- a/security/device_cgroup.c
+++ b/security/device_cgroup.c
@@ -182,7 +182,7 @@
static int devcgroup_online(struct cgroup_subsys_state *css)
{
struct dev_cgroup *dev_cgroup = css_to_devcgroup(css);
- struct dev_cgroup *parent_dev_cgroup = css_to_devcgroup(css_parent(css));
+ struct dev_cgroup *parent_dev_cgroup = css_to_devcgroup(css->parent);
int ret = 0;
mutex_lock(&devcgroup_mutex);
@@ -455,7 +455,7 @@
static int parent_has_perm(struct dev_cgroup *childcg,
struct dev_exception_item *ex)
{
- struct dev_cgroup *parent = css_to_devcgroup(css_parent(&childcg->css));
+ struct dev_cgroup *parent = css_to_devcgroup(childcg->css.parent);
if (!parent)
return 1;
@@ -476,7 +476,7 @@
static bool parent_allows_removal(struct dev_cgroup *childcg,
struct dev_exception_item *ex)
{
- struct dev_cgroup *parent = css_to_devcgroup(css_parent(&childcg->css));
+ struct dev_cgroup *parent = css_to_devcgroup(childcg->css.parent);
if (!parent)
return true;
@@ -587,13 +587,6 @@
return rc;
}
-static inline bool has_children(struct dev_cgroup *devcgroup)
-{
- struct cgroup *cgrp = devcgroup->css.cgroup;
-
- return !list_empty(&cgrp->children);
-}
-
/*
* Modify the exception list using allow/deny rules.
* CAP_SYS_ADMIN is needed for this. It's at least separate from CAP_MKNOD
@@ -614,7 +607,7 @@
char temp[12]; /* 11 + 1 characters needed for a u32 */
int count, rc = 0;
struct dev_exception_item ex;
- struct dev_cgroup *parent = css_to_devcgroup(css_parent(&devcgroup->css));
+ struct dev_cgroup *parent = css_to_devcgroup(devcgroup->css.parent);
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
@@ -626,7 +619,7 @@
case 'a':
switch (filetype) {
case DEVCG_ALLOW:
- if (has_children(devcgroup))
+ if (css_has_online_children(&devcgroup->css))
return -EINVAL;
if (!may_allow_all(parent))
@@ -642,7 +635,7 @@
return rc;
break;
case DEVCG_DENY:
- if (has_children(devcgroup))
+ if (css_has_online_children(&devcgroup->css))
return -EINVAL;
dev_exception_clean(devcgroup);
@@ -767,27 +760,27 @@
return rc;
}
-static int devcgroup_access_write(struct cgroup_subsys_state *css,
- struct cftype *cft, char *buffer)
+static ssize_t devcgroup_access_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
{
int retval;
mutex_lock(&devcgroup_mutex);
- retval = devcgroup_update_access(css_to_devcgroup(css),
- cft->private, buffer);
+ retval = devcgroup_update_access(css_to_devcgroup(of_css(of)),
+ of_cft(of)->private, strstrip(buf));
mutex_unlock(&devcgroup_mutex);
- return retval;
+ return retval ?: nbytes;
}
static struct cftype dev_cgroup_files[] = {
{
.name = "allow",
- .write_string = devcgroup_access_write,
+ .write = devcgroup_access_write,
.private = DEVCG_ALLOW,
},
{
.name = "deny",
- .write_string = devcgroup_access_write,
+ .write = devcgroup_access_write,
.private = DEVCG_DENY,
},
{
diff --git a/security/integrity/ima/ima_api.c b/security/integrity/ima/ima_api.c
index ba9e4d7..d9cd5ce 100644
--- a/security/integrity/ima/ima_api.c
+++ b/security/integrity/ima/ima_api.c
@@ -199,6 +199,7 @@
struct evm_ima_xattr_data **xattr_value,
int *xattr_len)
{
+ const char *audit_cause = "failed";
struct inode *inode = file_inode(file);
const char *filename = file->f_dentry->d_name.name;
int result = 0;
@@ -213,6 +214,12 @@
if (!(iint->flags & IMA_COLLECTED)) {
u64 i_version = file_inode(file)->i_version;
+ if (file->f_flags & O_DIRECT) {
+ audit_cause = "failed(directio)";
+ result = -EACCES;
+ goto out;
+ }
+
/* use default hash algorithm */
hash.hdr.algo = ima_hash_algo;
@@ -233,9 +240,10 @@
result = -ENOMEM;
}
}
+out:
if (result)
integrity_audit_msg(AUDIT_INTEGRITY_DATA, inode,
- filename, "collect_data", "failed",
+ filename, "collect_data", audit_cause,
result, 0);
return result;
}
diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c
index 52ac6cf..dcc98cf 100644
--- a/security/integrity/ima/ima_main.c
+++ b/security/integrity/ima/ima_main.c
@@ -214,8 +214,11 @@
xattr_ptr = &xattr_value;
rc = ima_collect_measurement(iint, file, xattr_ptr, &xattr_len);
- if (rc != 0)
+ if (rc != 0) {
+ if (file->f_flags & O_DIRECT)
+ rc = (iint->flags & IMA_PERMIT_DIRECTIO) ? 0 : -EACCES;
goto out_digsig;
+ }
pathname = filename ?: ima_d_path(&file->f_path, &pathbuf);
diff --git a/security/integrity/ima/ima_policy.c b/security/integrity/ima/ima_policy.c
index 93873a4..40a7488 100644
--- a/security/integrity/ima/ima_policy.c
+++ b/security/integrity/ima/ima_policy.c
@@ -353,7 +353,7 @@
Opt_obj_user, Opt_obj_role, Opt_obj_type,
Opt_subj_user, Opt_subj_role, Opt_subj_type,
Opt_func, Opt_mask, Opt_fsmagic, Opt_uid, Opt_fowner,
- Opt_appraise_type, Opt_fsuuid
+ Opt_appraise_type, Opt_fsuuid, Opt_permit_directio
};
static match_table_t policy_tokens = {
@@ -375,6 +375,7 @@
{Opt_uid, "uid=%s"},
{Opt_fowner, "fowner=%s"},
{Opt_appraise_type, "appraise_type=%s"},
+ {Opt_permit_directio, "permit_directio"},
{Opt_err, NULL}
};
@@ -622,6 +623,9 @@
else
result = -EINVAL;
break;
+ case Opt_permit_directio:
+ entry->flags |= IMA_PERMIT_DIRECTIO;
+ break;
case Opt_err:
ima_log_string(ab, "UNKNOWN", p);
result = -EINVAL;
diff --git a/security/integrity/integrity.h b/security/integrity/integrity.h
index 2fb5e53..33c0a70 100644
--- a/security/integrity/integrity.h
+++ b/security/integrity/integrity.h
@@ -30,6 +30,7 @@
#define IMA_ACTION_FLAGS 0xff000000
#define IMA_DIGSIG 0x01000000
#define IMA_DIGSIG_REQUIRED 0x02000000
+#define IMA_PERMIT_DIRECTIO 0x04000000
#define IMA_DO_MASK (IMA_MEASURE | IMA_APPRAISE | IMA_AUDIT | \
IMA_APPRAISE_SUBMASK)
diff --git a/security/keys/internal.h b/security/keys/internal.h
index 80b2aac..5f20da01 100644
--- a/security/keys/internal.h
+++ b/security/keys/internal.h
@@ -176,20 +176,11 @@
/*
* Check to see whether permission is granted to use a key in the desired way.
*/
-static inline int key_permission(const key_ref_t key_ref, key_perm_t perm)
+static inline int key_permission(const key_ref_t key_ref, unsigned perm)
{
return key_task_permission(key_ref, current_cred(), perm);
}
-/* required permissions */
-#define KEY_VIEW 0x01 /* require permission to view attributes */
-#define KEY_READ 0x02 /* require permission to read content */
-#define KEY_WRITE 0x04 /* require permission to update / modify */
-#define KEY_SEARCH 0x08 /* require permission to search (keyring) or find (key) */
-#define KEY_LINK 0x10 /* require permission to link */
-#define KEY_SETATTR 0x20 /* require permission to change attributes */
-#define KEY_ALL 0x3f /* all the above permissions */
-
/*
* Authorisation record for request_key().
*/
diff --git a/security/keys/key.c b/security/keys/key.c
index 6e21c11..2048a11 100644
--- a/security/keys/key.c
+++ b/security/keys/key.c
@@ -714,7 +714,7 @@
int ret;
/* need write permission on the key to update it */
- ret = key_permission(key_ref, KEY_WRITE);
+ ret = key_permission(key_ref, KEY_NEED_WRITE);
if (ret < 0)
goto error;
@@ -838,7 +838,7 @@
/* if we're going to allocate a new key, we're going to have
* to modify the keyring */
- ret = key_permission(keyring_ref, KEY_WRITE);
+ ret = key_permission(keyring_ref, KEY_NEED_WRITE);
if (ret < 0) {
key_ref = ERR_PTR(ret);
goto error_link_end;
@@ -928,7 +928,7 @@
key_check(key);
/* the key must be writable */
- ret = key_permission(key_ref, KEY_WRITE);
+ ret = key_permission(key_ref, KEY_NEED_WRITE);
if (ret < 0)
goto error;
diff --git a/security/keys/keyctl.c b/security/keys/keyctl.c
index cee72ce..cd5bd0c 100644
--- a/security/keys/keyctl.c
+++ b/security/keys/keyctl.c
@@ -111,7 +111,7 @@
}
/* find the target keyring (which must be writable) */
- keyring_ref = lookup_user_key(ringid, KEY_LOOKUP_CREATE, KEY_WRITE);
+ keyring_ref = lookup_user_key(ringid, KEY_LOOKUP_CREATE, KEY_NEED_WRITE);
if (IS_ERR(keyring_ref)) {
ret = PTR_ERR(keyring_ref);
goto error3;
@@ -195,7 +195,7 @@
dest_ref = NULL;
if (destringid) {
dest_ref = lookup_user_key(destringid, KEY_LOOKUP_CREATE,
- KEY_WRITE);
+ KEY_NEED_WRITE);
if (IS_ERR(dest_ref)) {
ret = PTR_ERR(dest_ref);
goto error3;
@@ -253,7 +253,7 @@
long ret;
lflags = create ? KEY_LOOKUP_CREATE : 0;
- key_ref = lookup_user_key(id, lflags, KEY_SEARCH);
+ key_ref = lookup_user_key(id, lflags, KEY_NEED_SEARCH);
if (IS_ERR(key_ref)) {
ret = PTR_ERR(key_ref);
goto error;
@@ -334,7 +334,7 @@
}
/* find the target key (which must be writable) */
- key_ref = lookup_user_key(id, 0, KEY_WRITE);
+ key_ref = lookup_user_key(id, 0, KEY_NEED_WRITE);
if (IS_ERR(key_ref)) {
ret = PTR_ERR(key_ref);
goto error2;
@@ -365,12 +365,12 @@
key_ref_t key_ref;
long ret;
- key_ref = lookup_user_key(id, 0, KEY_WRITE);
+ key_ref = lookup_user_key(id, 0, KEY_NEED_WRITE);
if (IS_ERR(key_ref)) {
ret = PTR_ERR(key_ref);
if (ret != -EACCES)
goto error;
- key_ref = lookup_user_key(id, 0, KEY_SETATTR);
+ key_ref = lookup_user_key(id, 0, KEY_NEED_SETATTR);
if (IS_ERR(key_ref)) {
ret = PTR_ERR(key_ref);
goto error;
@@ -401,7 +401,7 @@
kenter("%d", id);
- key_ref = lookup_user_key(id, 0, KEY_SEARCH);
+ key_ref = lookup_user_key(id, 0, KEY_NEED_SEARCH);
if (IS_ERR(key_ref)) {
ret = PTR_ERR(key_ref);
goto error;
@@ -428,7 +428,7 @@
key_ref_t keyring_ref;
long ret;
- keyring_ref = lookup_user_key(ringid, KEY_LOOKUP_CREATE, KEY_WRITE);
+ keyring_ref = lookup_user_key(ringid, KEY_LOOKUP_CREATE, KEY_NEED_WRITE);
if (IS_ERR(keyring_ref)) {
ret = PTR_ERR(keyring_ref);
@@ -470,13 +470,13 @@
key_ref_t keyring_ref, key_ref;
long ret;
- keyring_ref = lookup_user_key(ringid, KEY_LOOKUP_CREATE, KEY_WRITE);
+ keyring_ref = lookup_user_key(ringid, KEY_LOOKUP_CREATE, KEY_NEED_WRITE);
if (IS_ERR(keyring_ref)) {
ret = PTR_ERR(keyring_ref);
goto error;
}
- key_ref = lookup_user_key(id, KEY_LOOKUP_CREATE, KEY_LINK);
+ key_ref = lookup_user_key(id, KEY_LOOKUP_CREATE, KEY_NEED_LINK);
if (IS_ERR(key_ref)) {
ret = PTR_ERR(key_ref);
goto error2;
@@ -505,7 +505,7 @@
key_ref_t keyring_ref, key_ref;
long ret;
- keyring_ref = lookup_user_key(ringid, 0, KEY_WRITE);
+ keyring_ref = lookup_user_key(ringid, 0, KEY_NEED_WRITE);
if (IS_ERR(keyring_ref)) {
ret = PTR_ERR(keyring_ref);
goto error;
@@ -548,7 +548,7 @@
char *tmpbuf;
long ret;
- key_ref = lookup_user_key(keyid, KEY_LOOKUP_PARTIAL, KEY_VIEW);
+ key_ref = lookup_user_key(keyid, KEY_LOOKUP_PARTIAL, KEY_NEED_VIEW);
if (IS_ERR(key_ref)) {
/* viewing a key under construction is permitted if we have the
* authorisation token handy */
@@ -639,7 +639,7 @@
}
/* get the keyring at which to begin the search */
- keyring_ref = lookup_user_key(ringid, 0, KEY_SEARCH);
+ keyring_ref = lookup_user_key(ringid, 0, KEY_NEED_SEARCH);
if (IS_ERR(keyring_ref)) {
ret = PTR_ERR(keyring_ref);
goto error2;
@@ -649,7 +649,7 @@
dest_ref = NULL;
if (destringid) {
dest_ref = lookup_user_key(destringid, KEY_LOOKUP_CREATE,
- KEY_WRITE);
+ KEY_NEED_WRITE);
if (IS_ERR(dest_ref)) {
ret = PTR_ERR(dest_ref);
goto error3;
@@ -676,7 +676,7 @@
/* link the resulting key to the destination keyring if we can */
if (dest_ref) {
- ret = key_permission(key_ref, KEY_LINK);
+ ret = key_permission(key_ref, KEY_NEED_LINK);
if (ret < 0)
goto error6;
@@ -727,7 +727,7 @@
key = key_ref_to_ptr(key_ref);
/* see if we can read it directly */
- ret = key_permission(key_ref, KEY_READ);
+ ret = key_permission(key_ref, KEY_NEED_READ);
if (ret == 0)
goto can_read_key;
if (ret != -EACCES)
@@ -799,7 +799,7 @@
goto error;
key_ref = lookup_user_key(id, KEY_LOOKUP_CREATE | KEY_LOOKUP_PARTIAL,
- KEY_SETATTR);
+ KEY_NEED_SETATTR);
if (IS_ERR(key_ref)) {
ret = PTR_ERR(key_ref);
goto error;
@@ -905,7 +905,7 @@
goto error;
key_ref = lookup_user_key(id, KEY_LOOKUP_CREATE | KEY_LOOKUP_PARTIAL,
- KEY_SETATTR);
+ KEY_NEED_SETATTR);
if (IS_ERR(key_ref)) {
ret = PTR_ERR(key_ref);
goto error;
@@ -947,7 +947,7 @@
/* if a specific keyring is nominated by ID, then use that */
if (ringid > 0) {
- dkref = lookup_user_key(ringid, KEY_LOOKUP_CREATE, KEY_WRITE);
+ dkref = lookup_user_key(ringid, KEY_LOOKUP_CREATE, KEY_NEED_WRITE);
if (IS_ERR(dkref))
return PTR_ERR(dkref);
*_dest_keyring = key_ref_to_ptr(dkref);
@@ -1315,7 +1315,7 @@
long ret;
key_ref = lookup_user_key(id, KEY_LOOKUP_CREATE | KEY_LOOKUP_PARTIAL,
- KEY_SETATTR);
+ KEY_NEED_SETATTR);
if (IS_ERR(key_ref)) {
/* setting the timeout on a key under construction is permitted
* if we have the authorisation token handy */
@@ -1418,7 +1418,7 @@
char *context;
long ret;
- key_ref = lookup_user_key(keyid, KEY_LOOKUP_PARTIAL, KEY_VIEW);
+ key_ref = lookup_user_key(keyid, KEY_LOOKUP_PARTIAL, KEY_NEED_VIEW);
if (IS_ERR(key_ref)) {
if (PTR_ERR(key_ref) != -EACCES)
return PTR_ERR(key_ref);
@@ -1482,7 +1482,7 @@
struct cred *cred;
int ret;
- keyring_r = lookup_user_key(KEY_SPEC_SESSION_KEYRING, 0, KEY_LINK);
+ keyring_r = lookup_user_key(KEY_SPEC_SESSION_KEYRING, 0, KEY_NEED_LINK);
if (IS_ERR(keyring_r))
return PTR_ERR(keyring_r);
diff --git a/security/keys/keyring.c b/security/keys/keyring.c
index 2fb2576..9cf2575 100644
--- a/security/keys/keyring.c
+++ b/security/keys/keyring.c
@@ -541,7 +541,7 @@
/* key must have search permissions */
if (!(ctx->flags & KEYRING_SEARCH_NO_CHECK_PERM) &&
key_task_permission(make_key_ref(key, ctx->possessed),
- ctx->cred, KEY_SEARCH) < 0) {
+ ctx->cred, KEY_NEED_SEARCH) < 0) {
ctx->result = ERR_PTR(-EACCES);
kleave(" = %d [!perm]", ctx->skipped_ret);
goto skipped;
@@ -721,7 +721,7 @@
/* Search a nested keyring */
if (!(ctx->flags & KEYRING_SEARCH_NO_CHECK_PERM) &&
key_task_permission(make_key_ref(key, ctx->possessed),
- ctx->cred, KEY_SEARCH) < 0)
+ ctx->cred, KEY_NEED_SEARCH) < 0)
continue;
/* stack the current position */
@@ -843,7 +843,7 @@
return ERR_PTR(-ENOTDIR);
if (!(ctx->flags & KEYRING_SEARCH_NO_CHECK_PERM)) {
- err = key_task_permission(keyring_ref, ctx->cred, KEY_SEARCH);
+ err = key_task_permission(keyring_ref, ctx->cred, KEY_NEED_SEARCH);
if (err < 0)
return ERR_PTR(err);
}
@@ -973,7 +973,7 @@
if (!skip_perm_check &&
key_permission(make_key_ref(keyring, 0),
- KEY_SEARCH) < 0)
+ KEY_NEED_SEARCH) < 0)
continue;
/* we've got a match but we might end up racing with
diff --git a/security/keys/permission.c b/security/keys/permission.c
index efcc0c8..732cc0b 100644
--- a/security/keys/permission.c
+++ b/security/keys/permission.c
@@ -28,7 +28,7 @@
* permissions bits or the LSM check.
*/
int key_task_permission(const key_ref_t key_ref, const struct cred *cred,
- key_perm_t perm)
+ unsigned perm)
{
struct key *key;
key_perm_t kperm;
@@ -68,7 +68,7 @@
if (is_key_possessed(key_ref))
kperm |= key->perm >> 24;
- kperm = kperm & perm & KEY_ALL;
+ kperm = kperm & perm & KEY_NEED_ALL;
if (kperm != perm)
return -EACCES;
diff --git a/security/keys/persistent.c b/security/keys/persistent.c
index 0ad3ee2..c9fae5e 100644
--- a/security/keys/persistent.c
+++ b/security/keys/persistent.c
@@ -108,7 +108,7 @@
return PTR_ERR(persistent_ref);
found:
- ret = key_task_permission(persistent_ref, current_cred(), KEY_LINK);
+ ret = key_task_permission(persistent_ref, current_cred(), KEY_NEED_LINK);
if (ret == 0) {
persistent = key_ref_to_ptr(persistent_ref);
ret = key_link(key_ref_to_ptr(dest_ref), persistent);
@@ -151,7 +151,7 @@
}
/* There must be a destination keyring */
- dest_ref = lookup_user_key(destid, KEY_LOOKUP_CREATE, KEY_WRITE);
+ dest_ref = lookup_user_key(destid, KEY_LOOKUP_CREATE, KEY_NEED_WRITE);
if (IS_ERR(dest_ref))
return PTR_ERR(dest_ref);
if (key_ref_to_ptr(dest_ref)->type != &key_type_keyring) {
diff --git a/security/keys/proc.c b/security/keys/proc.c
index 88e9a46..d3f6f2f 100644
--- a/security/keys/proc.c
+++ b/security/keys/proc.c
@@ -218,7 +218,7 @@
* - the caller holds a spinlock, and thus the RCU read lock, making our
* access to __current_cred() safe
*/
- rc = key_task_permission(key_ref, ctx.cred, KEY_VIEW);
+ rc = key_task_permission(key_ref, ctx.cred, KEY_NEED_VIEW);
if (rc < 0)
return 0;
diff --git a/security/keys/sysctl.c b/security/keys/sysctl.c
index 8c0af08..b68faa1 100644
--- a/security/keys/sysctl.c
+++ b/security/keys/sysctl.c
@@ -15,7 +15,7 @@
static const int zero, one = 1, max = INT_MAX;
-ctl_table key_sysctls[] = {
+struct ctl_table key_sysctls[] = {
{
.procname = "maxkeys",
.data = &key_quota_maxkeys,
diff --git a/security/security.c b/security/security.c
index 8b774f3..31614e9 100644
--- a/security/security.c
+++ b/security/security.c
@@ -1425,7 +1425,7 @@
}
int security_key_permission(key_ref_t key_ref,
- const struct cred *cred, key_perm_t perm)
+ const struct cred *cred, unsigned perm)
{
return security_ops->key_permission(key_ref, cred, perm);
}
diff --git a/security/selinux/avc.c b/security/selinux/avc.c
index fc3e662..a18f1fa 100644
--- a/security/selinux/avc.c
+++ b/security/selinux/avc.c
@@ -444,11 +444,15 @@
avc_dump_query(ab, ad->selinux_audit_data->ssid,
ad->selinux_audit_data->tsid,
ad->selinux_audit_data->tclass);
+ if (ad->selinux_audit_data->denied) {
+ audit_log_format(ab, " permissive=%u",
+ ad->selinux_audit_data->result ? 0 : 1);
+ }
}
/* This is the slow part of avc audit with big stack footprint */
noinline int slow_avc_audit(u32 ssid, u32 tsid, u16 tclass,
- u32 requested, u32 audited, u32 denied,
+ u32 requested, u32 audited, u32 denied, int result,
struct common_audit_data *a,
unsigned flags)
{
@@ -477,6 +481,7 @@
sad.tsid = tsid;
sad.audited = audited;
sad.denied = denied;
+ sad.result = result;
a->selinux_audit_data = &sad;
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c
index 2c7341d..83d06db 100644
--- a/security/selinux/hooks.c
+++ b/security/selinux/hooks.c
@@ -2123,11 +2123,13 @@
new_tsec->exec_sid = 0;
/*
- * Minimize confusion: if no_new_privs and a transition is
- * explicitly requested, then fail the exec.
+ * Minimize confusion: if no_new_privs or nosuid and a
+ * transition is explicitly requested, then fail the exec.
*/
if (bprm->unsafe & LSM_UNSAFE_NO_NEW_PRIVS)
return -EPERM;
+ if (bprm->file->f_path.mnt->mnt_flags & MNT_NOSUID)
+ return -EACCES;
} else {
/* Check for a default transition on this program. */
rc = security_transition_sid(old_tsec->sid, isec->sid,
@@ -2770,6 +2772,7 @@
static noinline int audit_inode_permission(struct inode *inode,
u32 perms, u32 audited, u32 denied,
+ int result,
unsigned flags)
{
struct common_audit_data ad;
@@ -2780,7 +2783,7 @@
ad.u.inode = inode;
rc = slow_avc_audit(current_sid(), isec->sid, isec->sclass, perms,
- audited, denied, &ad, flags);
+ audited, denied, result, &ad, flags);
if (rc)
return rc;
return 0;
@@ -2822,7 +2825,7 @@
if (likely(!audited))
return rc;
- rc2 = audit_inode_permission(inode, perms, audited, denied, flags);
+ rc2 = audit_inode_permission(inode, perms, audited, denied, rc, flags);
if (rc2)
return rc2;
return rc;
@@ -5722,7 +5725,7 @@
static int selinux_key_permission(key_ref_t key_ref,
const struct cred *cred,
- key_perm_t perm)
+ unsigned perm)
{
struct key *key;
struct key_security_struct *ksec;
diff --git a/security/selinux/include/avc.h b/security/selinux/include/avc.h
index f53ee3c..ddf8eec 100644
--- a/security/selinux/include/avc.h
+++ b/security/selinux/include/avc.h
@@ -102,7 +102,7 @@
}
int slow_avc_audit(u32 ssid, u32 tsid, u16 tclass,
- u32 requested, u32 audited, u32 denied,
+ u32 requested, u32 audited, u32 denied, int result,
struct common_audit_data *a,
unsigned flags);
@@ -137,7 +137,7 @@
if (likely(!audited))
return 0;
return slow_avc_audit(ssid, tsid, tclass,
- requested, audited, denied,
+ requested, audited, denied, result,
a, 0);
}
diff --git a/security/selinux/ss/hashtab.c b/security/selinux/ss/hashtab.c
index 933e735..2cc4961 100644
--- a/security/selinux/ss/hashtab.c
+++ b/security/selinux/ss/hashtab.c
@@ -6,6 +6,7 @@
#include <linux/kernel.h>
#include <linux/slab.h>
#include <linux/errno.h>
+#include <linux/sched.h>
#include "hashtab.h"
struct hashtab *hashtab_create(u32 (*hash_value)(struct hashtab *h, const void *key),
@@ -40,6 +41,8 @@
u32 hvalue;
struct hashtab_node *prev, *cur, *newnode;
+ cond_resched();
+
if (!h || h->nel == HASHTAB_MAX_NODES)
return -EINVAL;
diff --git a/security/selinux/ss/mls.c b/security/selinux/ss/mls.c
index c85bc1e..d307b37 100644
--- a/security/selinux/ss/mls.c
+++ b/security/selinux/ss/mls.c
@@ -492,6 +492,8 @@
rc = ebitmap_set_bit(&bitmap, catdatum->value - 1, 1);
if (rc)
return rc;
+
+ cond_resched();
}
ebitmap_destroy(&c->range.level[l].cat);
c->range.level[l].cat = bitmap;
diff --git a/security/smack/smack.h b/security/smack/smack.h
index d072fd3..020307e 100644
--- a/security/smack/smack.h
+++ b/security/smack/smack.h
@@ -80,8 +80,8 @@
struct socket_smack {
struct smack_known *smk_out; /* outbound label */
- char *smk_in; /* inbound label */
- char *smk_packet; /* TCP peer label */
+ struct smack_known *smk_in; /* inbound label */
+ struct smack_known *smk_packet; /* TCP peer label */
};
/*
@@ -133,7 +133,7 @@
struct list_head list;
struct sock *smk_sock; /* socket initialized on */
unsigned short smk_port; /* the port number */
- char *smk_in; /* incoming label */
+ struct smack_known *smk_in; /* inbound label */
struct smack_known *smk_out; /* outgoing label */
};
@@ -177,6 +177,14 @@
#define SMACK_CIPSO_MAXCATNUM 184 /* 23 * 8 */
/*
+ * Ptrace rules
+ */
+#define SMACK_PTRACE_DEFAULT 0
+#define SMACK_PTRACE_EXACT 1
+#define SMACK_PTRACE_DRACONIAN 2
+#define SMACK_PTRACE_MAX SMACK_PTRACE_DRACONIAN
+
+/*
* Flags for untraditional access modes.
* It shouldn't be necessary to avoid conflicts with definitions
* in fs.h, but do so anyway.
@@ -225,6 +233,7 @@
*/
int smk_access_entry(char *, char *, struct list_head *);
int smk_access(struct smack_known *, char *, int, struct smk_audit_info *);
+int smk_tskacc(struct task_smack *, char *, u32, struct smk_audit_info *);
int smk_curacc(char *, u32, struct smk_audit_info *);
struct smack_known *smack_from_secid(const u32);
char *smk_parse_smack(const char *string, int len);
@@ -244,6 +253,7 @@
extern struct smack_known *smack_onlycap;
extern struct smack_known *smack_syslog_label;
extern const char *smack_cipso_option;
+extern int smack_ptrace_rule;
extern struct smack_known smack_known_floor;
extern struct smack_known smack_known_hat;
diff --git a/security/smack/smack_access.c b/security/smack/smack_access.c
index 14293cd..c062e94 100644
--- a/security/smack/smack_access.c
+++ b/security/smack/smack_access.c
@@ -192,20 +192,21 @@
}
/**
- * smk_curacc - determine if current has a specific access to an object
+ * smk_tskacc - determine if a task has a specific access to an object
+ * @tsp: a pointer to the subject task
* @obj_label: a pointer to the object's Smack label
* @mode: the access requested, in "MAY" format
* @a : common audit data
*
- * This function checks the current subject label/object label pair
+ * This function checks the subject task's label/object label pair
* in the access rule list and returns 0 if the access is permitted,
- * non zero otherwise. It allows that current may have the capability
+ * non zero otherwise. It allows that the task may have the capability
* to override the rules.
*/
-int smk_curacc(char *obj_label, u32 mode, struct smk_audit_info *a)
+int smk_tskacc(struct task_smack *subject, char *obj_label,
+ u32 mode, struct smk_audit_info *a)
{
- struct task_smack *tsp = current_security();
- struct smack_known *skp = smk_of_task(tsp);
+ struct smack_known *skp = smk_of_task(subject);
int may;
int rc;
@@ -219,7 +220,7 @@
* it can further restrict access.
*/
may = smk_access_entry(skp->smk_known, obj_label,
- &tsp->smk_rules);
+ &subject->smk_rules);
if (may < 0)
goto out_audit;
if ((mode & may) == mode)
@@ -241,6 +242,24 @@
return rc;
}
+/**
+ * smk_curacc - determine if current has a specific access to an object
+ * @obj_label: a pointer to the object's Smack label
+ * @mode: the access requested, in "MAY" format
+ * @a : common audit data
+ *
+ * This function checks the current subject label/object label pair
+ * in the access rule list and returns 0 if the access is permitted,
+ * non zero otherwise. It allows that current may have the capability
+ * to override the rules.
+ */
+int smk_curacc(char *obj_label, u32 mode, struct smk_audit_info *a)
+{
+ struct task_smack *tsp = current_security();
+
+ return smk_tskacc(tsp, obj_label, mode, a);
+}
+
#ifdef CONFIG_AUDIT
/**
* smack_str_from_perm : helper to transalate an int to a
@@ -285,7 +304,10 @@
audit_log_untrustedstring(ab, sad->subject);
audit_log_format(ab, " object=");
audit_log_untrustedstring(ab, sad->object);
- audit_log_format(ab, " requested=%s", sad->request);
+ if (sad->request[0] == '\0')
+ audit_log_format(ab, " labels_differ");
+ else
+ audit_log_format(ab, " requested=%s", sad->request);
}
/**
diff --git a/security/smack/smack_lsm.c b/security/smack/smack_lsm.c
index 14f52be..f2c3080 100644
--- a/security/smack/smack_lsm.c
+++ b/security/smack/smack_lsm.c
@@ -157,6 +157,74 @@
return rc;
}
+/**
+ * smk_ptrace_mode - helper function for converting PTRACE_MODE_* into MAY_*
+ * @mode - input mode in form of PTRACE_MODE_*
+ *
+ * Returns a converted MAY_* mode usable by smack rules
+ */
+static inline unsigned int smk_ptrace_mode(unsigned int mode)
+{
+ switch (mode) {
+ case PTRACE_MODE_READ:
+ return MAY_READ;
+ case PTRACE_MODE_ATTACH:
+ return MAY_READWRITE;
+ }
+
+ return 0;
+}
+
+/**
+ * smk_ptrace_rule_check - helper for ptrace access
+ * @tracer: tracer process
+ * @tracee_label: label of the process that's about to be traced,
+ * the pointer must originate from smack structures
+ * @mode: ptrace attachment mode (PTRACE_MODE_*)
+ * @func: name of the function that called us, used for audit
+ *
+ * Returns 0 on access granted, -error on error
+ */
+static int smk_ptrace_rule_check(struct task_struct *tracer, char *tracee_label,
+ unsigned int mode, const char *func)
+{
+ int rc;
+ struct smk_audit_info ad, *saip = NULL;
+ struct task_smack *tsp;
+ struct smack_known *skp;
+
+ if ((mode & PTRACE_MODE_NOAUDIT) == 0) {
+ smk_ad_init(&ad, func, LSM_AUDIT_DATA_TASK);
+ smk_ad_setfield_u_tsk(&ad, tracer);
+ saip = &ad;
+ }
+
+ tsp = task_security(tracer);
+ skp = smk_of_task(tsp);
+
+ if ((mode & PTRACE_MODE_ATTACH) &&
+ (smack_ptrace_rule == SMACK_PTRACE_EXACT ||
+ smack_ptrace_rule == SMACK_PTRACE_DRACONIAN)) {
+ if (skp->smk_known == tracee_label)
+ rc = 0;
+ else if (smack_ptrace_rule == SMACK_PTRACE_DRACONIAN)
+ rc = -EACCES;
+ else if (capable(CAP_SYS_PTRACE))
+ rc = 0;
+ else
+ rc = -EACCES;
+
+ if (saip)
+ smack_log(skp->smk_known, tracee_label, 0, rc, saip);
+
+ return rc;
+ }
+
+ /* In case of rule==SMACK_PTRACE_DEFAULT or mode==PTRACE_MODE_READ */
+ rc = smk_tskacc(tsp, tracee_label, smk_ptrace_mode(mode), saip);
+ return rc;
+}
+
/*
* LSM hooks.
* We he, that is fun!
@@ -165,16 +233,15 @@
/**
* smack_ptrace_access_check - Smack approval on PTRACE_ATTACH
* @ctp: child task pointer
- * @mode: ptrace attachment mode
+ * @mode: ptrace attachment mode (PTRACE_MODE_*)
*
* Returns 0 if access is OK, an error code otherwise
*
- * Do the capability checks, and require read and write.
+ * Do the capability checks.
*/
static int smack_ptrace_access_check(struct task_struct *ctp, unsigned int mode)
{
int rc;
- struct smk_audit_info ad;
struct smack_known *skp;
rc = cap_ptrace_access_check(ctp, mode);
@@ -182,10 +249,8 @@
return rc;
skp = smk_of_task(task_security(ctp));
- smk_ad_init(&ad, __func__, LSM_AUDIT_DATA_TASK);
- smk_ad_setfield_u_tsk(&ad, ctp);
- rc = smk_curacc(skp->smk_known, mode, &ad);
+ rc = smk_ptrace_rule_check(current, skp->smk_known, mode, __func__);
return rc;
}
@@ -195,23 +260,21 @@
*
* Returns 0 if access is OK, an error code otherwise
*
- * Do the capability checks, and require read and write.
+ * Do the capability checks, and require PTRACE_MODE_ATTACH.
*/
static int smack_ptrace_traceme(struct task_struct *ptp)
{
int rc;
- struct smk_audit_info ad;
struct smack_known *skp;
rc = cap_ptrace_traceme(ptp);
if (rc != 0)
return rc;
- skp = smk_of_task(task_security(ptp));
- smk_ad_init(&ad, __func__, LSM_AUDIT_DATA_TASK);
- smk_ad_setfield_u_tsk(&ad, ptp);
+ skp = smk_of_task(current_security());
- rc = smk_curacc(skp->smk_known, MAY_READWRITE, &ad);
+ rc = smk_ptrace_rule_check(ptp, skp->smk_known,
+ PTRACE_MODE_ATTACH, __func__);
return rc;
}
@@ -413,9 +476,11 @@
* Initialize the root inode.
*/
isp = inode->i_security;
- if (inode->i_security == NULL) {
- inode->i_security = new_inode_smack(sp->smk_root);
- isp = inode->i_security;
+ if (isp == NULL) {
+ isp = new_inode_smack(sp->smk_root);
+ if (isp == NULL)
+ return -ENOMEM;
+ inode->i_security = isp;
} else
isp->smk_inode = sp->smk_root;
@@ -453,7 +518,7 @@
* smack_bprm_set_creds - set creds for exec
* @bprm: the exec information
*
- * Returns 0 if it gets a blob, -ENOMEM otherwise
+ * Returns 0 if it gets a blob, -EPERM if exec forbidden and -ENOMEM otherwise
*/
static int smack_bprm_set_creds(struct linux_binprm *bprm)
{
@@ -473,7 +538,22 @@
if (isp->smk_task == NULL || isp->smk_task == bsp->smk_task)
return 0;
- if (bprm->unsafe)
+ if (bprm->unsafe & (LSM_UNSAFE_PTRACE | LSM_UNSAFE_PTRACE_CAP)) {
+ struct task_struct *tracer;
+ rc = 0;
+
+ rcu_read_lock();
+ tracer = ptrace_parent(current);
+ if (likely(tracer != NULL))
+ rc = smk_ptrace_rule_check(tracer,
+ isp->smk_task->smk_known,
+ PTRACE_MODE_ATTACH,
+ __func__);
+ rcu_read_unlock();
+
+ if (rc != 0)
+ return rc;
+ } else if (bprm->unsafe)
return -EPERM;
bsp->smk_task = isp->smk_task;
@@ -880,18 +960,20 @@
return;
}
- skp = smk_import_entry(value, size);
if (strcmp(name, XATTR_NAME_SMACK) == 0) {
+ skp = smk_import_entry(value, size);
if (skp != NULL)
isp->smk_inode = skp->smk_known;
else
isp->smk_inode = smack_known_invalid.smk_known;
} else if (strcmp(name, XATTR_NAME_SMACKEXEC) == 0) {
+ skp = smk_import_entry(value, size);
if (skp != NULL)
isp->smk_task = skp;
else
isp->smk_task = &smack_known_invalid;
} else if (strcmp(name, XATTR_NAME_SMACKMMAP) == 0) {
+ skp = smk_import_entry(value, size);
if (skp != NULL)
isp->smk_mmap = skp;
else
@@ -938,24 +1020,37 @@
strcmp(name, XATTR_NAME_SMACKIPOUT) == 0 ||
strcmp(name, XATTR_NAME_SMACKEXEC) == 0 ||
strcmp(name, XATTR_NAME_SMACKTRANSMUTE) == 0 ||
- strcmp(name, XATTR_NAME_SMACKMMAP)) {
+ strcmp(name, XATTR_NAME_SMACKMMAP) == 0) {
if (!smack_privileged(CAP_MAC_ADMIN))
rc = -EPERM;
} else
rc = cap_inode_removexattr(dentry, name);
+ if (rc != 0)
+ return rc;
+
smk_ad_init(&ad, __func__, LSM_AUDIT_DATA_DENTRY);
smk_ad_setfield_u_fs_path_dentry(&ad, dentry);
- if (rc == 0)
- rc = smk_curacc(smk_of_inode(dentry->d_inode), MAY_WRITE, &ad);
- if (rc == 0) {
- isp = dentry->d_inode->i_security;
+ rc = smk_curacc(smk_of_inode(dentry->d_inode), MAY_WRITE, &ad);
+ if (rc != 0)
+ return rc;
+
+ isp = dentry->d_inode->i_security;
+ /*
+ * Don't do anything special for these.
+ * XATTR_NAME_SMACKIPIN
+ * XATTR_NAME_SMACKIPOUT
+ * XATTR_NAME_SMACKEXEC
+ */
+ if (strcmp(name, XATTR_NAME_SMACK) == 0)
isp->smk_task = NULL;
+ else if (strcmp(name, XATTR_NAME_SMACKMMAP) == 0)
isp->smk_mmap = NULL;
- }
+ else if (strcmp(name, XATTR_NAME_SMACKTRANSMUTE) == 0)
+ isp->smk_flags &= ~SMK_INODE_TRANSMUTE;
- return rc;
+ return 0;
}
/**
@@ -1000,7 +1095,7 @@
ssp = sock->sk->sk_security;
if (strcmp(name, XATTR_SMACK_IPIN) == 0)
- isp = ssp->smk_in;
+ isp = ssp->smk_in->smk_known;
else if (strcmp(name, XATTR_SMACK_IPOUT) == 0)
isp = ssp->smk_out->smk_known;
else
@@ -1367,19 +1462,32 @@
/**
* smack_file_open - Smack dentry open processing
* @file: the object
- * @cred: unused
+ * @cred: task credential
*
* Set the security blob in the file structure.
+ * Allow the open only if the task has read access. There are
+ * many read operations (e.g. fstat) that you can do with an
+ * fd even if you have the file open write-only.
*
* Returns 0
*/
static int smack_file_open(struct file *file, const struct cred *cred)
{
+ struct task_smack *tsp = cred->security;
struct inode_smack *isp = file_inode(file)->i_security;
+ struct smk_audit_info ad;
+ int rc;
- file->f_security = isp->smk_inode;
+ if (smack_privileged(CAP_MAC_OVERRIDE))
+ return 0;
- return 0;
+ smk_ad_init(&ad, __func__, LSM_AUDIT_DATA_PATH);
+ smk_ad_setfield_u_fs_path(&ad, file->f_path);
+ rc = smk_access(tsp->smk_task, isp->smk_inode, MAY_READ, &ad);
+ if (rc == 0)
+ file->f_security = isp->smk_inode;
+
+ return rc;
}
/*
@@ -1764,7 +1872,7 @@
if (ssp == NULL)
return -ENOMEM;
- ssp->smk_in = skp->smk_known;
+ ssp->smk_in = skp;
ssp->smk_out = skp;
ssp->smk_packet = NULL;
@@ -2004,7 +2112,7 @@
if (act == SMK_RECEIVING) {
skp = smack_net_ambient;
- object = ssp->smk_in;
+ object = ssp->smk_in->smk_known;
} else {
skp = ssp->smk_out;
object = smack_net_ambient->smk_known;
@@ -2034,9 +2142,9 @@
list_for_each_entry(spp, &smk_ipv6_port_list, list) {
if (spp->smk_port != port)
continue;
- object = spp->smk_in;
+ object = spp->smk_in->smk_known;
if (act == SMK_CONNECTING)
- ssp->smk_packet = spp->smk_out->smk_known;
+ ssp->smk_packet = spp->smk_out;
break;
}
@@ -2076,7 +2184,7 @@
int rc = 0;
if (value == NULL || size > SMK_LONGLABEL || size == 0)
- return -EACCES;
+ return -EINVAL;
skp = smk_import_entry(value, size);
if (skp == NULL)
@@ -2100,7 +2208,7 @@
ssp = sock->sk->sk_security;
if (strcmp(name, XATTR_SMACK_IPIN) == 0)
- ssp->smk_in = skp->smk_known;
+ ssp->smk_in = skp;
else if (strcmp(name, XATTR_SMACK_IPOUT) == 0) {
ssp->smk_out = skp;
if (sock->sk->sk_family == PF_INET) {
@@ -2713,6 +2821,15 @@
* of the superblock.
*/
if (opt_dentry->d_parent == opt_dentry) {
+ if (sbp->s_magic == CGROUP_SUPER_MAGIC) {
+ /*
+ * The cgroup filesystem is never mounted,
+ * so there's no opportunity to set the mount
+ * options.
+ */
+ sbsp->smk_root = smack_known_star.smk_known;
+ sbsp->smk_default = smack_known_star.smk_known;
+ }
isp->smk_inode = sbsp->smk_root;
isp->smk_flags |= SMK_INODE_INSTANT;
goto unlockandout;
@@ -2726,16 +2843,20 @@
*/
switch (sbp->s_magic) {
case SMACK_MAGIC:
+ case PIPEFS_MAGIC:
+ case SOCKFS_MAGIC:
+ case CGROUP_SUPER_MAGIC:
/*
* Casey says that it's a little embarrassing
* that the smack file system doesn't do
* extended attributes.
- */
- final = smack_known_star.smk_known;
- break;
- case PIPEFS_MAGIC:
- /*
+ *
* Casey says pipes are easy (?)
+ *
+ * Socket access is controlled by the socket
+ * structures associated with the task involved.
+ *
+ * Cgroupfs is special
*/
final = smack_known_star.smk_known;
break;
@@ -2747,13 +2868,6 @@
*/
final = ckp->smk_known;
break;
- case SOCKFS_MAGIC:
- /*
- * Socket access is controlled by the socket
- * structures associated with the task involved.
- */
- final = smack_known_star.smk_known;
- break;
case PROC_SUPER_MAGIC:
/*
* Casey says procfs appears not to care.
@@ -2959,30 +3073,34 @@
struct sock *other, struct sock *newsk)
{
struct smack_known *skp;
+ struct smack_known *okp;
struct socket_smack *ssp = sock->sk_security;
struct socket_smack *osp = other->sk_security;
struct socket_smack *nsp = newsk->sk_security;
struct smk_audit_info ad;
int rc = 0;
-
#ifdef CONFIG_AUDIT
struct lsm_network_audit net;
-
- smk_ad_init_net(&ad, __func__, LSM_AUDIT_DATA_NET, &net);
- smk_ad_setfield_u_net_sk(&ad, other);
#endif
if (!smack_privileged(CAP_MAC_OVERRIDE)) {
skp = ssp->smk_out;
- rc = smk_access(skp, osp->smk_in, MAY_WRITE, &ad);
+ okp = osp->smk_out;
+#ifdef CONFIG_AUDIT
+ smk_ad_init_net(&ad, __func__, LSM_AUDIT_DATA_NET, &net);
+ smk_ad_setfield_u_net_sk(&ad, other);
+#endif
+ rc = smk_access(skp, okp->smk_known, MAY_WRITE, &ad);
+ if (rc == 0)
+ rc = smk_access(okp, okp->smk_known, MAY_WRITE, NULL);
}
/*
* Cross reference the peer labels for SO_PEERSEC.
*/
if (rc == 0) {
- nsp->smk_packet = ssp->smk_out->smk_known;
- ssp->smk_packet = osp->smk_out->smk_known;
+ nsp->smk_packet = ssp->smk_out;
+ ssp->smk_packet = osp->smk_out;
}
return rc;
@@ -3014,7 +3132,7 @@
return 0;
skp = ssp->smk_out;
- return smk_access(skp, osp->smk_in, MAY_WRITE, &ad);
+ return smk_access(skp, osp->smk_in->smk_known, MAY_WRITE, &ad);
}
/**
@@ -3109,7 +3227,7 @@
if (found)
return skp;
- if (ssp != NULL && ssp->smk_in == smack_known_star.smk_known)
+ if (ssp != NULL && ssp->smk_in == &smack_known_star)
return &smack_known_web;
return &smack_known_star;
}
@@ -3228,7 +3346,7 @@
* This is the simplist possible security model
* for networking.
*/
- rc = smk_access(skp, ssp->smk_in, MAY_WRITE, &ad);
+ rc = smk_access(skp, ssp->smk_in->smk_known, MAY_WRITE, &ad);
if (rc != 0)
netlbl_skbuff_err(skb, rc, 0);
break;
@@ -3263,7 +3381,7 @@
ssp = sock->sk->sk_security;
if (ssp->smk_packet != NULL) {
- rcp = ssp->smk_packet;
+ rcp = ssp->smk_packet->smk_known;
slen = strlen(rcp) + 1;
}
@@ -3348,7 +3466,7 @@
return;
ssp = sk->sk_security;
- ssp->smk_in = skp->smk_known;
+ ssp->smk_in = skp;
ssp->smk_out = skp;
/* cssp->smk_packet is already set in smack_inet_csk_clone() */
}
@@ -3408,7 +3526,7 @@
* Receiving a packet requires that the other end be able to write
* here. Read access is not required.
*/
- rc = smk_access(skp, ssp->smk_in, MAY_WRITE, &ad);
+ rc = smk_access(skp, ssp->smk_in->smk_known, MAY_WRITE, &ad);
if (rc != 0)
return rc;
@@ -3452,7 +3570,7 @@
if (req->peer_secid != 0) {
skp = smack_from_secid(req->peer_secid);
- ssp->smk_packet = skp->smk_known;
+ ssp->smk_packet = skp;
} else
ssp->smk_packet = NULL;
}
@@ -3506,11 +3624,12 @@
* an error code otherwise
*/
static int smack_key_permission(key_ref_t key_ref,
- const struct cred *cred, key_perm_t perm)
+ const struct cred *cred, unsigned perm)
{
struct key *keyp;
struct smk_audit_info ad;
struct smack_known *tkp = smk_of_task(cred->security);
+ int request = 0;
keyp = key_ref_to_ptr(key_ref);
if (keyp == NULL)
@@ -3531,7 +3650,11 @@
ad.a.u.key_struct.key = keyp->serial;
ad.a.u.key_struct.key_desc = keyp->description;
#endif
- return smk_access(tkp, keyp->security, MAY_READWRITE, &ad);
+ if (perm & KEY_NEED_READ)
+ request = MAY_READ;
+ if (perm & (KEY_NEED_WRITE | KEY_NEED_LINK | KEY_NEED_SETATTR))
+ request = MAY_WRITE;
+ return smk_access(tkp, keyp->security, request, &ad);
}
#endif /* CONFIG_KEYS */
diff --git a/security/smack/smackfs.c b/security/smack/smackfs.c
index 3198cfe..32b2488 100644
--- a/security/smack/smackfs.c
+++ b/security/smack/smackfs.c
@@ -53,6 +53,7 @@
SMK_REVOKE_SUBJ = 18, /* set rules with subject label to '-' */
SMK_CHANGE_RULE = 19, /* change or add rules (long labels) */
SMK_SYSLOG = 20, /* change syslog label) */
+ SMK_PTRACE = 21, /* set ptrace rule */
};
/*
@@ -101,6 +102,15 @@
struct smack_known *smack_syslog_label;
/*
+ * Ptrace current rule
+ * SMACK_PTRACE_DEFAULT regular smack ptrace rules (/proc based)
+ * SMACK_PTRACE_EXACT labels must match, but can be overriden with
+ * CAP_SYS_PTRACE
+ * SMACK_PTRACE_DRACONIAN lables must match, CAP_SYS_PTRACE has no effect
+ */
+int smack_ptrace_rule = SMACK_PTRACE_DEFAULT;
+
+/*
* Certain IP addresses may be designated as single label hosts.
* Packets are sent there unlabeled, but only from tasks that
* can write to the specified label.
@@ -1183,7 +1193,7 @@
data[count] = '\0';
- rc = sscanf(data, "%hhd.%hhd.%hhd.%hhd/%d %s",
+ rc = sscanf(data, "%hhd.%hhd.%hhd.%hhd/%u %s",
&host[0], &host[1], &host[2], &host[3], &m, smack);
if (rc != 6) {
rc = sscanf(data, "%hhd.%hhd.%hhd.%hhd %s",
@@ -2244,6 +2254,68 @@
/**
+ * smk_read_ptrace - read() for /smack/ptrace
+ * @filp: file pointer, not actually used
+ * @buf: where to put the result
+ * @count: maximum to send along
+ * @ppos: where to start
+ *
+ * Returns number of bytes read or error code, as appropriate
+ */
+static ssize_t smk_read_ptrace(struct file *filp, char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ char temp[32];
+ ssize_t rc;
+
+ if (*ppos != 0)
+ return 0;
+
+ sprintf(temp, "%d\n", smack_ptrace_rule);
+ rc = simple_read_from_buffer(buf, count, ppos, temp, strlen(temp));
+ return rc;
+}
+
+/**
+ * smk_write_ptrace - write() for /smack/ptrace
+ * @file: file pointer
+ * @buf: data from user space
+ * @count: bytes sent
+ * @ppos: where to start - must be 0
+ */
+static ssize_t smk_write_ptrace(struct file *file, const char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ char temp[32];
+ int i;
+
+ if (!smack_privileged(CAP_MAC_ADMIN))
+ return -EPERM;
+
+ if (*ppos != 0 || count >= sizeof(temp) || count == 0)
+ return -EINVAL;
+
+ if (copy_from_user(temp, buf, count) != 0)
+ return -EFAULT;
+
+ temp[count] = '\0';
+
+ if (sscanf(temp, "%d", &i) != 1)
+ return -EINVAL;
+ if (i < SMACK_PTRACE_DEFAULT || i > SMACK_PTRACE_MAX)
+ return -EINVAL;
+ smack_ptrace_rule = i;
+
+ return count;
+}
+
+static const struct file_operations smk_ptrace_ops = {
+ .write = smk_write_ptrace,
+ .read = smk_read_ptrace,
+ .llseek = default_llseek,
+};
+
+/**
* smk_fill_super - fill the smackfs superblock
* @sb: the empty superblock
* @data: unused
@@ -2296,6 +2368,8 @@
"change-rule", &smk_change_rule_ops, S_IRUGO|S_IWUSR},
[SMK_SYSLOG] = {
"syslog", &smk_syslog_ops, S_IRUGO|S_IWUSR},
+ [SMK_PTRACE] = {
+ "ptrace", &smk_ptrace_ops, S_IRUGO|S_IWUSR},
/* last one */
{""}
};
diff --git a/tools/perf/util/dwarf-aux.c b/tools/perf/util/dwarf-aux.c
index 7defd77..cc66c40 100644
--- a/tools/perf/util/dwarf-aux.c
+++ b/tools/perf/util/dwarf-aux.c
@@ -747,14 +747,17 @@
static int __die_find_variable_cb(Dwarf_Die *die_mem, void *data)
{
struct __find_variable_param *fvp = data;
+ Dwarf_Attribute attr;
int tag;
tag = dwarf_tag(die_mem);
if ((tag == DW_TAG_formal_parameter ||
tag == DW_TAG_variable) &&
- die_compare_name(die_mem, fvp->name))
+ die_compare_name(die_mem, fvp->name) &&
+ /* Does the DIE have location information or external instance? */
+ (dwarf_attr(die_mem, DW_AT_external, &attr) ||
+ dwarf_attr(die_mem, DW_AT_location, &attr)))
return DIE_FIND_CB_END;
-
if (dwarf_haspc(die_mem, fvp->addr))
return DIE_FIND_CB_CONTINUE;
else
diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
index 5627621..9d8eb26 100644
--- a/tools/perf/util/probe-finder.c
+++ b/tools/perf/util/probe-finder.c
@@ -511,12 +511,12 @@
ret = convert_variable_location(vr_die, pf->addr, pf->fb_ops,
&pf->sp_die, pf->tvar);
- if (ret == -ENOENT)
+ if (ret == -ENOENT || ret == -EINVAL)
pr_err("Failed to find the location of %s at this address.\n"
" Perhaps, it has been optimized out.\n", pf->pvar->var);
else if (ret == -ENOTSUP)
pr_err("Sorry, we don't support this variable location yet.\n");
- else if (pf->pvar->field) {
+ else if (ret == 0 && pf->pvar->field) {
ret = convert_variable_fields(vr_die, pf->pvar->var,
pf->pvar->field, &pf->tvar->ref,
&die_mem);
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 32487ed..e66e710 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -10,6 +10,7 @@
TARGETS += vm
TARGETS += powerpc
TARGETS += user
+TARGETS += sysctl
all:
for TARGET in $(TARGETS); do \
diff --git a/tools/testing/selftests/powerpc/Makefile b/tools/testing/selftests/powerpc/Makefile
index 316194f..b3dbe9e 100644
--- a/tools/testing/selftests/powerpc/Makefile
+++ b/tools/testing/selftests/powerpc/Makefile
@@ -13,7 +13,7 @@
export CC CFLAGS
-TARGETS = pmu copyloops
+TARGETS = pmu copyloops mm
endif
diff --git a/tools/testing/selftests/powerpc/copyloops/asm/ppc_asm.h b/tools/testing/selftests/powerpc/copyloops/asm/ppc_asm.h
index ccd9c84..d1dc374 100644
--- a/tools/testing/selftests/powerpc/copyloops/asm/ppc_asm.h
+++ b/tools/testing/selftests/powerpc/copyloops/asm/ppc_asm.h
@@ -46,12 +46,15 @@
#define R20 r20
#define R21 r21
#define R22 r22
+#define R29 r29
+#define R30 r30
+#define R31 r31
#define STACKFRAMESIZE 256
-#define STK_PARAM(i) (48 + ((i)-3)*8)
#define STK_REG(i) (112 + ((i)-14)*8)
#define _GLOBAL(A) FUNC_START(test_ ## A)
+#define _GLOBAL_TOC(A) _GLOBAL(A)
#define PPC_MTOCRF(A, B) mtocrf A, B
diff --git a/tools/testing/selftests/powerpc/mm/Makefile b/tools/testing/selftests/powerpc/mm/Makefile
new file mode 100644
index 0000000..357ccbd
--- /dev/null
+++ b/tools/testing/selftests/powerpc/mm/Makefile
@@ -0,0 +1,18 @@
+noarg:
+ $(MAKE) -C ../
+
+PROGS := hugetlb_vs_thp_test
+
+all: $(PROGS)
+
+$(PROGS): ../harness.c
+
+run_tests: all
+ @-for PROG in $(PROGS); do \
+ ./$$PROG; \
+ done;
+
+clean:
+ rm -f $(PROGS)
+
+.PHONY: all run_tests clean
diff --git a/tools/testing/selftests/powerpc/mm/hugetlb_vs_thp_test.c b/tools/testing/selftests/powerpc/mm/hugetlb_vs_thp_test.c
new file mode 100644
index 0000000..3d8e5b0
--- /dev/null
+++ b/tools/testing/selftests/powerpc/mm/hugetlb_vs_thp_test.c
@@ -0,0 +1,72 @@
+#include <stdio.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+#include "utils.h"
+
+/* This must match the huge page & THP size */
+#define SIZE (16 * 1024 * 1024)
+
+static int test_body(void)
+{
+ void *addr;
+ char *p;
+
+ addr = (void *)0xa0000000;
+
+ p = mmap(addr, SIZE, PROT_READ | PROT_WRITE,
+ MAP_HUGETLB | MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+ if (p != MAP_FAILED) {
+ /*
+ * Typically the mmap will fail because no huge pages are
+ * allocated on the system. But if there are huge pages
+ * allocated the mmap will succeed. That's fine too, we just
+ * munmap here before continuing.
+ */
+ munmap(addr, SIZE);
+ }
+
+ p = mmap(addr, SIZE, PROT_READ | PROT_WRITE,
+ MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+ if (p == MAP_FAILED) {
+ printf("Mapping failed @ %p\n", addr);
+ perror("mmap");
+ return 1;
+ }
+
+ /*
+ * Either a user or kernel access is sufficient to trigger the bug.
+ * A kernel access is easier to spot & debug, as it will trigger the
+ * softlockup or RCU stall detectors, and when the system is kicked
+ * into xmon we get a backtrace in the kernel.
+ *
+ * A good option is:
+ * getcwd(p, SIZE);
+ *
+ * For the purposes of this testcase it's preferable to spin in
+ * userspace, so the harness can kill us if we get stuck. That way we
+ * see a test failure rather than a dead system.
+ */
+ *p = 0xf;
+
+ munmap(addr, SIZE);
+
+ return 0;
+}
+
+static int test_main(void)
+{
+ int i;
+
+ /* 10,000 because it's a "bunch", and completes reasonably quickly */
+ for (i = 0; i < 10000; i++)
+ if (test_body())
+ return 1;
+
+ return 0;
+}
+
+int main(void)
+{
+ return test_harness(test_main, "hugetlb_vs_thp");
+}
diff --git a/tools/testing/selftests/sysctl/Makefile b/tools/testing/selftests/sysctl/Makefile
new file mode 100644
index 0000000..0a92ada
--- /dev/null
+++ b/tools/testing/selftests/sysctl/Makefile
@@ -0,0 +1,19 @@
+# Makefile for sysctl selftests.
+# Expects kernel.sysctl_writes_strict=1.
+
+# No binaries, but make sure arg-less "make" doesn't trigger "run_tests".
+all:
+
+# Allow specific tests to be selected.
+test_num:
+ @/bin/sh ./run_numerictests
+
+test_string:
+ @/bin/sh ./run_stringtests
+
+run_tests: all test_num test_string
+
+# Nothing to clean up.
+clean:
+
+.PHONY: all run_tests clean test_num test_string
diff --git a/tools/testing/selftests/sysctl/common_tests b/tools/testing/selftests/sysctl/common_tests
new file mode 100644
index 0000000..17d534b
--- /dev/null
+++ b/tools/testing/selftests/sysctl/common_tests
@@ -0,0 +1,109 @@
+#!/bin/sh
+
+TEST_FILE=$(mktemp)
+
+echo "== Testing sysctl behavior against ${TARGET} =="
+
+set_orig()
+{
+ echo "${ORIG}" > "${TARGET}"
+}
+
+set_test()
+{
+ echo "${TEST_STR}" > "${TARGET}"
+}
+
+verify()
+{
+ local seen
+ seen=$(cat "$1")
+ if [ "${seen}" != "${TEST_STR}" ]; then
+ return 1
+ fi
+ return 0
+}
+
+trap 'set_orig; rm -f "${TEST_FILE}"' EXIT
+
+rc=0
+
+echo -n "Writing test file ... "
+echo "${TEST_STR}" > "${TEST_FILE}"
+if ! verify "${TEST_FILE}"; then
+ echo "FAIL" >&2
+ exit 1
+else
+ echo "ok"
+fi
+
+echo -n "Checking sysctl is not set to test value ... "
+if verify "${TARGET}"; then
+ echo "FAIL" >&2
+ exit 1
+else
+ echo "ok"
+fi
+
+echo -n "Writing sysctl from shell ... "
+set_test
+if ! verify "${TARGET}"; then
+ echo "FAIL" >&2
+ exit 1
+else
+ echo "ok"
+fi
+
+echo -n "Resetting sysctl to original value ... "
+set_orig
+if verify "${TARGET}"; then
+ echo "FAIL" >&2
+ exit 1
+else
+ echo "ok"
+fi
+
+# Now that we've validated the sanity of "set_test" and "set_orig",
+# we can use those functions to set starting states before running
+# specific behavioral tests.
+
+echo -n "Writing entire sysctl in single write ... "
+set_orig
+dd if="${TEST_FILE}" of="${TARGET}" bs=4096 2>/dev/null
+if ! verify "${TARGET}"; then
+ echo "FAIL" >&2
+ rc=1
+else
+ echo "ok"
+fi
+
+echo -n "Writing middle of sysctl after synchronized seek ... "
+set_test
+dd if="${TEST_FILE}" of="${TARGET}" bs=1 seek=1 skip=1 2>/dev/null
+if ! verify "${TARGET}"; then
+ echo "FAIL" >&2
+ rc=1
+else
+ echo "ok"
+fi
+
+echo -n "Writing beyond end of sysctl ... "
+set_orig
+dd if="${TEST_FILE}" of="${TARGET}" bs=20 seek=2 2>/dev/null
+if verify "${TARGET}"; then
+ echo "FAIL" >&2
+ rc=1
+else
+ echo "ok"
+fi
+
+echo -n "Writing sysctl with multiple long writes ... "
+set_orig
+(perl -e 'print "A" x 50;'; echo "${TEST_STR}") | \
+ dd of="${TARGET}" bs=50 2>/dev/null
+if verify "${TARGET}"; then
+ echo "FAIL" >&2
+ rc=1
+else
+ echo "ok"
+fi
diff --git a/tools/testing/selftests/sysctl/run_numerictests b/tools/testing/selftests/sysctl/run_numerictests
new file mode 100644
index 0000000..8510f93
--- /dev/null
+++ b/tools/testing/selftests/sysctl/run_numerictests
@@ -0,0 +1,10 @@
+#!/bin/sh
+
+SYSCTL="/proc/sys"
+TARGET="${SYSCTL}/vm/swappiness"
+ORIG=$(cat "${TARGET}")
+TEST_STR=$(( $ORIG + 1 ))
+
+. ./common_tests
+
+exit $rc
diff --git a/tools/testing/selftests/sysctl/run_stringtests b/tools/testing/selftests/sysctl/run_stringtests
new file mode 100644
index 0000000..90a9293
--- /dev/null
+++ b/tools/testing/selftests/sysctl/run_stringtests
@@ -0,0 +1,77 @@
+#!/bin/sh
+
+SYSCTL="/proc/sys"
+TARGET="${SYSCTL}/kernel/domainname"
+ORIG=$(cat "${TARGET}")
+TEST_STR="Testing sysctl"
+
+. ./common_tests
+
+# Only string sysctls support seeking/appending.
+MAXLEN=65
+
+echo -n "Writing entire sysctl in short writes ... "
+set_orig
+dd if="${TEST_FILE}" of="${TARGET}" bs=1 2>/dev/null
+if ! verify "${TARGET}"; then
+ echo "FAIL" >&2
+ rc=1
+else
+ echo "ok"
+fi
+
+echo -n "Writing middle of sysctl after unsynchronized seek ... "
+set_test
+dd if="${TEST_FILE}" of="${TARGET}" bs=1 seek=1 2>/dev/null
+if verify "${TARGET}"; then
+ echo "FAIL" >&2
+ rc=1
+else
+ echo "ok"
+fi
+
+echo -n "Checking sysctl maxlen is at least $MAXLEN ... "
+set_orig
+perl -e 'print "A" x ('"${MAXLEN}"'-2), "B";' | \
+ dd of="${TARGET}" bs="${MAXLEN}" 2>/dev/null
+if ! grep -q B "${TARGET}"; then
+ echo "FAIL" >&2
+ rc=1
+else
+ echo "ok"
+fi
+
+echo -n "Checking sysctl keeps original string on overflow append ... "
+set_orig
+perl -e 'print "A" x ('"${MAXLEN}"'-1), "B";' | \
+ dd of="${TARGET}" bs=$(( MAXLEN - 1 )) 2>/dev/null
+if grep -q B "${TARGET}"; then
+ echo "FAIL" >&2
+ rc=1
+else
+ echo "ok"
+fi
+
+echo -n "Checking sysctl stays NULL terminated on write ... "
+set_orig
+perl -e 'print "A" x ('"${MAXLEN}"'-1), "B";' | \
+ dd of="${TARGET}" bs="${MAXLEN}" 2>/dev/null
+if grep -q B "${TARGET}"; then
+ echo "FAIL" >&2
+ rc=1
+else
+ echo "ok"
+fi
+
+echo -n "Checking sysctl stays NULL terminated on overwrite ... "
+set_orig
+perl -e 'print "A" x ('"${MAXLEN}"'-1), "BB";' | \
+ dd of="${TARGET}" bs=$(( $MAXLEN + 1 )) 2>/dev/null
+if grep -q B "${TARGET}"; then
+ echo "FAIL" >&2
+ rc=1
+else
+ echo "ok"
+fi
+
+exit $rc
diff --git a/usr/Kconfig b/usr/Kconfig
index 642f503..2d4c77e 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -98,80 +98,3 @@
help
Support loading of a LZ4 encoded initial ramdisk or cpio buffer
If unsure, say N.
-
-choice
- prompt "Built-in initramfs compression mode" if INITRAMFS_SOURCE!=""
- help
- This option decides by which algorithm the builtin initramfs
- will be compressed. Several compression algorithms are
- available, which differ in efficiency, compression and
- decompression speed. Compression speed is only relevant
- when building a kernel. Decompression speed is relevant at
- each boot.
-
- If you have any problems with bzip2 or LZMA compressed
- initramfs, mail me (Alain Knaff) <alain@knaff.lu>.
-
- High compression options are mostly useful for users who are
- low on RAM, since it reduces the memory consumption during
- boot.
-
- If in doubt, select 'gzip'
-
-config INITRAMFS_COMPRESSION_NONE
- bool "None"
- help
- Do not compress the built-in initramfs at all. This may
- sound wasteful in space, but, you should be aware that the
- built-in initramfs will be compressed at a later stage
- anyways along with the rest of the kernel, on those
- architectures that support this.
- However, not compressing the initramfs may lead to slightly
- higher memory consumption during a short time at boot, while
- both the cpio image and the unpacked filesystem image will
- be present in memory simultaneously
-
-config INITRAMFS_COMPRESSION_GZIP
- bool "Gzip"
- depends on RD_GZIP
- help
- The old and tried gzip compression. It provides a good balance
- between compression ratio and decompression speed.
-
-config INITRAMFS_COMPRESSION_BZIP2
- bool "Bzip2"
- depends on RD_BZIP2
- help
- Its compression ratio and speed is intermediate.
- Decompression speed is slowest among the choices. The initramfs
- size is about 10% smaller with bzip2, in comparison to gzip.
- Bzip2 uses a large amount of memory. For modern kernels you
- will need at least 8MB RAM or more for booting.
-
-config INITRAMFS_COMPRESSION_LZMA
- bool "LZMA"
- depends on RD_LZMA
- help
- This algorithm's compression ratio is best.
- Decompression speed is between the other choices.
- Compression is slowest. The initramfs size is about 33%
- smaller with LZMA in comparison to gzip.
-
-config INITRAMFS_COMPRESSION_XZ
- bool "XZ"
- depends on RD_XZ
- help
- XZ uses the LZMA2 algorithm. The initramfs size is about 30%
- smaller with XZ in comparison to gzip. Decompression speed
- is better than that of bzip2 but worse than gzip and LZO.
- Compression is slow.
-
-config INITRAMFS_COMPRESSION_LZO
- bool "LZO"
- depends on RD_LZO
- help
- Its compression ratio is the poorest among the choices. The kernel
- size is about 10% bigger than gzip; however its speed
- (both compression and decompression) is the fastest.
-
-endchoice
