[kernel] bug#43: elroy cfg access should use PDC_PAT_IO

X-PA-RISC Linux-PR-Message: report 43
X-PA-RISC Linux-PR-Package: kernel
X-Loop: daniel_frazier@hp.com
Received: via spool by bugs@bugs.parisc-linux.org id=B.98358058921606
          (code B ref -1); Sat, 03 Mar 2001 01:03:02 GMT
Date: Fri, 2 Mar 2001 16:52:54 -0800 (PST)
From: Grant Grundler <grundler@cup.hp.com>
Message-Id: <200103030052.QAA00515@milano.cup.hp.com>
To: submit@bugs.parisc-linux.org
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit


Package: kernel
Version: SMP/PAT
Severity: wishlist

The appended mail describes a PAT PDC syncronization issue we are
exposed to on any SMP PAT box.

Severity (wishlist) is driven by the fact that HPUX is exposed to this
same problem and has *not* fixed it in over two years. So either they
aren't recognizing the problem when it occurs (random HPMC or garbage data
from PDC call) or they aren't seeing it.  Until parisc-linux makes lots
of PAT PDC/IODC calls (eg PCI OLAR) during runtime *and* accesses PCI
config space at the same time, I see no reason to change current behavior.

I'm ok with either solution (proposed below) but prefer the OS grabbing
the pdc_lock (easiest and most reliable).  Performance is not an issue
with PCI config space access.

grant



>From: "LANE,DUNCAN (HP-Roseville,ex1)" <duncan_lane@hp.com>
To: "'Grant Grundler'" <grundler@cup.hp.com>
Cc: "'terryl@rose.hp.com'" <terryl@rose.hp.com>,
        "ORTIZ,RICHARD (HP-Roseville,ex1)" <richard_ortiz@hp.com>,
        "'palinux_rd@ldl.fc.hp.com'" <palinux_rd@ldl.fc.hp.com>,
        "'gkuntz@rose.hp.com'" <gkuntz@rose.hp.com>
Subject: RE: Use PDC_PAT_IO for CONFIG?
Date: Mon, 26 Feb 2001 18:55:00 -0800

Hi Grant,

The atomicity issue that I was refering to is if two independent
code threads on separate CPUs both try an access the same config
space at the same time.  Config space accesses are a two step
process.  First, write the address then read or write the data.
This leads to ordering issues unless the accesses are somehow
semaphored.

Using PDC_PAT_IO takes care of the ordering issues.

Using direct kernel access (with correct internal semaphoring)
also takes care of the ordering issues.

However, sometimes using PDC_PAT_IO and sometimes using direct
kernel access opens up the ordering issue since the access
semaphoring will be circumvented.  I believe that this is why 
Terry was concerned when he found out that PDC_PAT_IO was not
always being used for config space accesses.

Duncan

-----Original Message-----
>From: Grant Grundler [mailto:grundler@cup.hp.com]
Sent: Monday, February 26, 2001 5:40 PM
To: LANE,DUNCAN (HP-Roseville,ex1)
Cc: 'terryl@rose.hp.com'; ORTIZ,RICHARD (HP-Roseville,ex1);
'palinux_rd@ldl.fc.hp.com'; 'gkuntz@rose.hp.com'
Subject: Re: Use PDC_PAT_IO for CONFIG?


"LANE,DUNCAN (HP-Roseville,ex1)" wrote:
> Hi Grant et al,
> 
> Terry Lee alerted me to this issue earlier today but I am still
> trying to come up to speed on it.  As I understand it, the issue
> with the kernel not using PDC_PAT_IO to do config space accesses
> is that the we lose the guarantee of atomicity on config space.
> 
> I assume that the kernel correctly handles atomicity of config
> space accesses within its own environment and that we are only
> talking about a clash between a kernel access and an access via
> PDC_PAT_IO?

If there are issues with atomicity, I'm not aware of them since
I haven't been involved with PAT PDC development or elroy for the
past two years about.

> So is the real issue that some programs use PDC_PAT_IO and the
> kernel does not? 

I don't know what the issue is. Originally we proposed PDC_PAT_IO/CONFIG
calls so the OS wouldn't have to worry about which bloody version
of Elroy was installed in the box. And also how bugs in the various
Elroy versions would impact PCI OLAR support. It was mostly a hedge
against bugs in future versions of elroy or replacement of elroy with
a totally different chip.


> The e-mail thread proposes changing the kernel to use PDC_PAT_IO.
> Would a better solution be for the kernel to simply grab its
> PDC call semaphore (or however PDC call access is managed) and 
> then do its own direct access to config space?

That would be kind of ugly but it could. yes.

> This would
> guarantee that PDC_PAT_IO could not be called at the same time
> as the kernel doing the config space access and would avoid
> the speed penalty of making the PDC call.

Configuration space accesses don't have to worry about performance.
As long as the execution time is measured in milli or microseconds that is.

> The other benefit
> would be that you do not need to certify that PDC_PAT_IO
> config space accesses work on all supported platforms.

True. That's a good observation.

> The above idea would have problems on HD if you are allowing
> multi-threaded PDC calls.
> 
> Before I get too deep into thinking up solutions, can someone
> confirm that I have correctly stated the problem?
> 
> Also, I am not clear on how frequently the kernel accesses PCI
> config space.  I assume it is only done during device init or
> error handling.  Is this true or is it done in any code that
> would impact driver performance?

Right now, definitely during initial PCI bus walk, loading a
new dynamically loadable driver (device initialization),
and in the future it would also be during PCI OLAR operations.
I think PCI config space is accessible via /proc/pci as well.
So user programs like "lspci" can at least read PCI config space.
Not sure if they can write it too.

thanks,
grant

Grant Grundler
parisc-linux {PCI|IOMMU|SMP} hacker
+1.408.447.7253