[parisc-linux] Re: syncdma question (back to ccio drivers)

[Joel Soete]

> On Sun, Nov 05, 2006 at 11:50:45AM +0000, Joel Soete wrote:
> ..
> > > What makes you think this is a problem with IOMMU coherency?
> > >
> > Remember: 53c700 driver on b180 (don't use any ccio) works fine but the 
> > same 53c700 driver on c110/d380 failed sadely:
> >
<http://lists.parisc-linux.org/pipermail/parisc-linux/2006-September/030202.html>
> > 
> > and according to James' comment:
> >
<http://lists.parisc-linux.org/pipermail/parisc-linux/2006-September/030204.html>
> > 
> > this should be a pb in sg list management; not in 53c700 (because works 
> > fine without ccio) but well in ccio to which this 53c700 driver has to 
> > address its io request, right?
> 
> Ah ok. It doesn't have to be the SG list handling.
> 
That's the only stuff I tried to backport as far as I can (I would mainly get
rid of some macro, anoying to me)
If that doesn't fix the pb, it doesn't do thing worse (system still booting
and survive outside perf test ;-))

> So what is wrong? I have no clue.
> 
> 
> This might also be a write-posting problem with MMIO register writes.
> The CCIO chip might be introducing enough delay to expose the problem.
> 
Next investigation so ;-)

> My second guess is a coherency problem with "consistent" data.
> Ie control data that is allocated with pci_alloc_consistent().
> I find this unlikely but it's possible.
> 
mmm, till now I didn't think to do read and write tests separately:
  - write test failed quickly
  - read test failed need more time but finaly failed with same errors

> > In a first step, I so manage to backport all your sba job since the time 
> > those drivers looks like brotherhood:
> > around
> >
<http://cvs.parisc-linux.org/linux-2.4/arch/parisc/kernel/sba_iommu.c?rev=1.26&view=markup>
> 
> Those two driver do have alot in common. But the TLB replacement algorithms
> are NOT the same. The IO Pdir has different coherency rules as well.
> Unfortunately, I don't remember all the details.
> 
As far as I can understand I try to do my best to not touch it (even
thought would require an expert review)

> > This seems to help to improve ncr53c720 driver (not absolutely sure: run 
> > untar/rm loop only 1h while it failed after few min before, but not yet 
> > enough for me and more over it seems to break dino on the d380 additional 
> > nic, though) but if didn't seem to degrade 53c700 driver, it didn't improve 
> > it at all.
> > 
> > In a second step, I suspected specific stuff to ccio and specialy what 
> > doesn't seems to exist here in ccio:
> > the sba
> >     /* flush purges */
> >     READ_REG32(ioc->ioc_hpa+IOC_PCOM);
> > 
> > but without doc (not yet publicaly available) I couldn't go further in this 
> > investigation.
> 
> Yes, that's another difference. IIRC, SBA can flush a _range_ of TLB entries
> and CCIO (Uturn/U2) can not.
> 
btw in pdc20-v1.1-ch5-iodc.pdf I found some interesting stuff in paragraph
"5.5.6 Bus Converter Port Specific IODC":
The definition of the IODC_SVERSION[opt] byte is:
R D S R
24 26 2728 29 31
D Indicates that the Bus converter is D-coherent. If the D-bit is 1, then
  purges and flushes do not need to be done after I/O operations. If the bit
  is set to 0, then purges and flushes of the memory range DMAed to must be
  done after an I/O operation.
S Indicates that the Bus converter does Synchronzation on I/O. If the S-bit 
  is 1, then no SYNCDMA instructions must be executed in conjunction with a
  DMA operation. If the S-bit is 0, then each I/O operation requires SYNCDMA
  instructions.

unfortunately this is not specified like this in pdc11-v0.96-ch3-iodc.pdf :_(
(I trust the pdc spec of my c110 for sure and d380 even though born with pa8000)

> > 
> > Let so assume that's ok.
> > 
> > Anyway something else could show a pb of synchronization: the driver perf 
> > which can be a bit improved by disabling CCIO_SEARCH_TIME as this comment 
> > said:
> > /*
> >  * CCIO_SEARCH_TIME can help measure how fast the bitmap search is.
> >  * impacts performance though - ditch it if you don't use it.
> >  */
> > 
> > If that make top stat completely false
> 
> Sorry -ENOPARSE.
> 
No worry just a note.

> > that mainly made 53c700 behaviour 
> > even worse: (with the same driver code and same up config) even only one 
> > untar/rm didn't reach to complete (iirc it didn't even finished untar) 
> > while it could at least complete 2 or 3 loop before.
> 
> Sounds like there is a race condition between asking for a mapping
> and it's use. Enableing CCIO_SEARCH_TIME will just make that longer.
> Maybe experiement with adding udelay(10) or udelay(100) in the
> same code path to see what happens.
> 
No it doesn't help, seems definitely a pb as hazardious as was smp kernel
failures on n4k (I mean it's easy to reproduce but need more or less time to
occure)

> > This latest test make me though it could also be a pb of synchronization 
> > somewhere between ccio and 53c700 and may be a pb of cache?
>  
> Maybe.
> 
> hth,
> grant
> 
> 
Yes, thanks a lot. I mainly think that I have to definitely wait UTurn/U2 doc ;-)

Joel


----------
Club Scarlet : Tout le monde gagne! Si vous devenez aujourd'hui Scarlet One grace a un client existant de Scarlet, vous recevez tous les deux un cadeau d'une valeur de 50 euros! Surfez vite sur http://www.clubscarlet.be