VoIP Mailing List Archives
Mailing list archives for the VoIP community |
|
| View previous topic :: View next topic |
| Author |
Message |
bwentdg at pipeline.com
Guest
|
 Posted: Tue Apr 15, 2008 2:37 pm Post subject: [asterisk-users] zaptel 1.4.10 regression with TE220B on Pro |
 |
|
exvito - I know it is a pain in the cahoonkus - but would you consider
sharing the OTHER Digium board issues you are having , the recommended
steps you were given by Digium to troubleshoot them, and the results ?
I think this "real-wold" experience wold be invaluable to the list.
THX in Advance for sharing !
Ex Vito wrote:
| Quote: | | Quote: | Your stack trace appears to possibly be stack corruption.
Could you try either this branch:
http://svn.digium.com/view/zaptel/team/mattf/zaptel-1.4-stackcleanup/
Or with a kernel that does not have 4K stacks enabled? You can check if your installed kernel does with the following command.
$ cat /boot/config-`uname -r` | grep 4K
# CONFIG_4KSTACKS is not set
|
...thanks for your feedback Shaun.
I am currently nearing other troubleshooting issues regarding
a TC400B (which will probably lead me to get in touch with
Digium install support).
So I have no schedule today to test your suggestions; maybe
tomorrow / thursday.
They are noted, however. 
Cheers,
--
exvito
_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
|
|
|
| Back to top |
|
 |
creslin at digium.com
Guest
|
| Posted: Wed Apr 16, 2008 10:20 am Post subject: [asterisk-users] zaptel 1.4.10 regression with TE220B on Pro |
|
|
Ex Vito wrote:
| Quote: | On Wed, Apr 16, 2008 at 3:26 PM, Matthew Fredrickson <creslin at digium.com> wrote:
| Quote: | The softlockup indicator should be benign. It gets called when loaded
the firmware for the part since the firmware image is so large and it
takes a long time to load. However, I might have a fix for you.
Can you try my stack reduction branch at:
https://origsvn.digium.com/svn/zaptel/team/mattf/zaptel-1.4-stackcleanup
If that does not work, please contact me directly and I will work with
you to get a resolution.
|
Matt,
Thanks for your feedback. We've already tested the following
branch as per Shaun's suggestion, without getting a different
behaviour (see today's earlier email to the list):
http://svn.digium.com/view/zaptel/team/mattf/zaptel-1.4-stackcleanup/
Question:
- The url you suggest is very similar, are we talking about
a different "stackcleanup" branch ?
We are now in the middle of rebuilding a non 4K stack page
kernel so as to give it a try with 1.4.10, the branch Shaun
suggested, 1.4.9.2 and the branch you mention, if it is in fact
different from Shaun's.
We wait your confirmation and will post non 4K stack kernel
results later today.
|
One thing also I would like to see is your kernel .config file. Another
thing that would for sure remove that warning is to disable the kernel
softlockup detector which is giving a false lockup warning in this case.
I belive it's under the "KERNEL HACKING" configuration menu if you are
using menuconfig.
--
Matthew Fredrickson
Software/Firmware Engineer
Digium, Inc. |
|
| Back to top |
|
|
creslin at digium.com
Guest
|
| Posted: Wed Apr 16, 2008 10:46 am Post subject: [asterisk-users] zaptel 1.4.10 regression with TE220B on Pro |
|
|
Ex Vito wrote:
| Quote: | On Wed, Apr 16, 2008 at 4:20 PM, Matthew Fredrickson <creslin at digium.com> wrote:
| Quote: |
One thing also I would like to see is your kernel .config file. Another
thing that would for sure remove that warning is to disable the kernel
softlockup detector which is giving a false lockup warning in this case.
I belive it's under the "KERNEL HACKING" configuration menu if you are
using menuconfig.
|
Up till now we're running stock CentOS kernel: 2.6.18-53.1.14.el5
The .config is publicly available but we can fwd it to you should you
prefer.
The kernel we're now building (it is taking quite a while... but it also
has been quite a few years since we've built custom kernels... since
the 2.0.3x days ?) is based on the stock CentOS kernel with only
the 4K stacks option disabled.
Please confirm if the SVN branch you suggested is the same or
different from the one Shaun suggested yesterday which we already
tested.
|
It's the same. Sorry, I sent you that email before I saw his message.
I just got an idea for a clever way to make the softlockup detector not
complain. I'll let you know when I have a patch to try.
--
Matthew Fredrickson
Software/Firmware Engineer
Digium, Inc. |
|
| Back to top |
|
|
creslin at digium.com
Guest
|
| Posted: Wed Apr 16, 2008 12:51 pm Post subject: [asterisk-users] zaptel 1.4.10 regression with TE220B on Pro |
|
|
Ex Vito wrote:
| Quote: | update with no 4K stack kernel:
- The kernel was build from stock centos 5 kernel 2.6.18-53.1.14.el5
- The only .config change was to disable the CONFIG_4KSTACKS
Tested zaptel-1.4.10, 1.4.9.2 and the stackcleanup svn branch as
suggested by Shaun and Mathew.
Short: Results are about the same (stack traces are different).
1.4.10 and the stackcleanup lead to soft hangups, 1.4.9.2
does not.
1.4.10 dmesg snippet:
|
One thing you can also do is pass the "nosoftlockup" kernel parameter
into the kernel from the bootloader. That should disable the softlockup
detector.
Matthew Fredrickson
| Quote: |
Zapata Telephony Interface Registered on major 196
Zaptel Version: 1.4.10
Zaptel Echo Canceller: MG2
ACPI: PCI Interrupt 0000:12:01.0[A] -> GSI 25 (level, low) -> IRQ 154
wcte12xp: Setting up global serial parameters for T1
wcte12xp: Found a Wildcard TE122
ACPI: PCI Interrupt 0000:18:08.0[A] -> GSI 19 (level, low) -> IRQ 162
Found TE2XXP at base address fdff0000, remapped to f893e000
TE2XXP version c01a016a, burst ON
Octasic optimized!
FALC version: 00000005, Board ID: 00
Reg 0: 0x3613a400
Reg 1: 0x3613a000
Reg 2: 0xffffffff
Reg 3: 0x00000000
Reg 4: 0x00003101
Reg 5: 0x00000000
Reg 6: 0xc01a016a
Reg 7: 0x00001300
Reg 8: 0x00000000
Reg 9: 0x00ff0031
Reg 10: 0x0000004a
TE2XXP: Launching card: 0
TE2XXP: Setting up global serial parameters
Found a Wildcard: Wildcard TE220 (4th Gen)
About to enter spanconfig!
Done with spanconfig!
About to enter spanconfig!
Done with spanconfig!
Registered tone zone 25 (Portugal)
wcte12xp: Span configured for ESF/B8ZS
About to enter startup!
TE2XXP: Span 1 configured for CCS/HDB3/CRC4
timing source auto card 0!
wct2xxp: Setting yellow alarm on span 1
timing source auto card 0!
SPAN 2: Primary Sync Source
VPM400: Not Present
wcte12xp: Setting yellow alarm
VPM450: echo cancellation for 64 channels
wcte12xp: Clearing yellow alarm
BUG: soft lockup detected on CPU#1!
[<c044d480>] softlockup_tick+0x96/0xa4
[<c042de00>] update_process_times+0x39/0x5c
[<c04196ef>] smp_apic_timer_interrupt+0x5b/0x6c
[<c04059bf>] apic_timer_interrupt+0x1f/0x24
[<c0605c30>] _spin_unlock_irqrestore+0x8/0x9
[<f8e82d57>] Oct6100UserDriverWriteBurstApi+0x1d/0x27 [wct4xxp]
[<f8e95de0>] Oct6100ApiLoadImage+0x1b5/0x289 [wct4xxp]
[<f8e9afc4>] Oct6100ChipOpen+0x166/0x25e [wct4xxp]
[<f8e83050>] init_vpm450m+0x196/0x306 [wct4xxp]
[<f8e6ab11>] t4_vpm450_init+0x18ce/0x198c [wct4xxp]
[<f8e6eee4>] t4_startup+0x4315/0x43c7 [wct4xxp]
[<c042624e>] release_console_sem+0x1b0/0x1b8
[<c042680e>] printk+0x18/0x8e
[<f8af6fe4>] t1_configure_t1+0xc10/0xc18 [wcte12xp]
[<f8ac65ef>] zt_rbs_sethook+0x102/0x13b [zaptel]
[<f8acdf6a>] zt_ioctl+0x273/0x144f [zaptel]
[<f885626f>] __journal_file_buffer+0x10e/0x1e3 [jbd]
[<f885626f>] __journal_file_buffer+0x10e/0x1e3 [jbd]
[<c0483cb3>] __d_lookup+0x98/0xdb
[<c047b32c>] do_lookup+0x53/0x166
[<c047d9ec>] do_path_lookup+0x20e/0x25e
[<c0471053>] get_empty_filp+0x99/0x15e
[<c047b5a5>] permission+0xa2/0xb5
[<c04e1a36>] kobject_get+0xf/0x13
[<c046ea1e>] __dentry_open+0xea/0x1ab
[<c046eb43>] nameidata_to_filp+0x19/0x28
[<c046eb7d>] do_filp_open+0x2b/0x31
[<c047f4a7>] do_ioctl+0x47/0x5d
[<c047f707>] vfs_ioctl+0x24a/0x25c
[<c0470de6>] __fput+0x13f/0x167
[<c047f761>] sys_ioctl+0x48/0x5f
[<c0404eff>] syscall_call+0x7/0xb
=======================
VPM450: hardware DTMF disabled.
VPM450: Present and operational servicing 2 span(s)
Completed startup!
About to enter startup!
TE2XXP: Span 2 configured for CCS/HDB3/CRC4
wct2xxp: Setting yellow alarm on span 2
timing source auto card 0!
SPAN 3: Secondary Sync Source
Completed startup!
1.4.9.2 dmesg snippet:
Zapata Telephony Interface Registered on major 196
Zaptel Version: 1.4.9.2
Zaptel Echo Canceller: MG2
PCI: Enabling device 0000:12:01.0 (0150 -> 0153)
ACPI: PCI Interrupt 0000:12:01.0[A] -> GSI 25 (level, low) -> IRQ 154
wcte12x[p]: Setting up global serial parameters for T1
wcte12x[p]: Found a Wildcard TE122
Found TE2XXP at base address fdff0000, remapped to f893e000
TE2XXP version c01a016a, burst ON
Octasic optimized!
FALC version: 00000005, Board ID: 00
Reg 0: 0x3571b400
Reg 1: 0x3571b000
Reg 2: 0xffffffff
Reg 3: 0x00000000
Reg 4: 0x00000101
Reg 5: 0x00000000
Reg 6: 0xc01a016a
Reg 7: 0x00001300
Reg 8: 0x010200ff
Reg 9: 0x00fd0001
Reg 10: 0x0000004a
TE2XXP: Launching card: 0
TE2XXP: Setting up global serial parameters
Found a Wildcard: Wildcard TE220 (4th Gen)
About to enter spanconfig!
Done with spanconfig!
About to enter spanconfig!
Done with spanconfig!
Registered tone zone 25 (Portugal)
wcte12x[p]: Span configured for ESF/B8ZS
About to enter startup!
TE2XXP: Span 1 configured for CCS/HDB3/CRC4
timing source auto card 0!
wct2xxp: Setting yellow alarm on span 1
SPAN 2: Primary Sync Source
timing source auto card 0!
VPM400: Not Present
VPM450: echo cancellation for 64 channels
VPM450: hardware DTMF disabled.
VPM450: Present and operational servicing 2 span(s)
Completed startup!
About to enter startup!
TE2XXP: Span 2 configured for CCS/HDB3/CRC4
wct2xxp: Setting yellow alarm on span 2
SPAN 3: Secondary Sync Source
Completed startup!
timing source auto card 0!
1.4-stackcleanup-r4163 dmesg snippet:
Zapata Telephony Interface Registered on major 196
Zaptel Version: SVN-mattf-zaptel-1.4-stackcleanup-r4163
Zaptel Echo Canceller: MG2
wcte12xp: Setting up global serial parameters for T1
wcte12xp: Found a Wildcard TE122
Found TE2XXP at base address fdff0000, remapped to f893e000
TE2XXP version c01a016a, burst ON
Octasic optimized!
FALC version: 00000005, Board ID: 00
Reg 0: 0x3491a400
Reg 1: 0x3491a000
Reg 2: 0xffffffff
Reg 3: 0x00000000
Reg 4: 0x00000101
Reg 5: 0x00000000
Reg 6: 0xc01a016a
Reg 7: 0x00001300
Reg 8: 0x010200ff
Reg 9: 0x00fd0001
Reg 10: 0x0000004a
TE2XXP: Launching card: 0
TE2XXP: Setting up global serial parameters
Found a Wildcard: Wildcard TE220 (4th Gen)
About to enter spanconfig!
Done with spanconfig!
About to enter spanconfig!
Done with spanconfig!
Registered tone zone 25 (Portugal)
wcte12xp: Span configured for ESF/B8ZS
About to enter startup!
TE2XXP: Span 1 configured for CCS/HDB3/CRC4
timing source auto card 0!
wct2xxp: Setting yellow alarm on span 1
SPAN 2: Primary Sync Source
timing source auto card 0!
VPM400: Not Present
VPM450: echo cancellation for 64 channels
BUG: soft lockup detected on CPU#1!
[<c044d480>] softlockup_tick+0x96/0xa4
[<c042de00>] update_process_times+0x39/0x5c
[<c04196ef>] smp_apic_timer_interrupt+0x5b/0x6c
[<c04059bf>] apic_timer_interrupt+0x1f/0x24
[<c0605c30>] _spin_unlock_irqrestore+0x8/0x9
[<f8e82d57>] Oct6100UserDriverWriteBurstApi+0x1d/0x27 [wct4xxp]
[<f8e95de0>] Oct6100ApiLoadImage+0x1b5/0x289 [wct4xxp]
[<f8e9afc4>] Oct6100ChipOpen+0x166/0x25e [wct4xxp]
[<f8e83050>] init_vpm450m+0x196/0x306 [wct4xxp]
[<f8e6ab11>] t4_vpm450_init+0x18ce/0x198c [wct4xxp]
[<f8e6eee4>] t4_startup+0x4315/0x43c7 [wct4xxp]
[<c042624e>] release_console_sem+0x1b0/0x1b8
[<c042680e>] printk+0x18/0x8e
[<f8af6fe4>] t1_configure_t1+0xc10/0xc18 [wcte12xp]
[<f8ac65ef>] zt_rbs_sethook+0x102/0x13b [zaptel]
[<f8acdf39>] zt_ioctl+0x273/0x14be [zaptel]
[<c04777b1>] chrdev_open+0x11e/0x132
[<c0477693>] chrdev_open+0x0/0x132
[<c046ea1e>] __dentry_open+0xea/0x1ab
[<c047f4a7>] do_ioctl+0x47/0x5d
[<c047f707>] vfs_ioctl+0x24a/0x25c
[<c0470de6>] __fput+0x13f/0x167
[<c047f761>] sys_ioctl+0x48/0x5f
[<c0404eff>] syscall_call+0x7/0xb
=======================
wcte12xp0: Missed interrupt. Increasing latency to 4 ms in order to compensate.
VPM450: hardware DTMF disabled.
VPM450: Present and operational servicing 2 span(s)
Completed startup!
About to enter startup!
TE2XXP: Span 2 configured for CCS/HDB3/CRC4
wct2xxp: Setting yellow alarm on span 2
timing source auto card 0!
SPAN 3: Secondary Sync Source
Completed startup!
More context info regarding this system:
- The objective was to go live in about a week
- Since we are RMAing a TC400B which we had trouble
getting to work (according to Digium install support, it is
probably faulty), we will probably have to delay the
go live date -- our supplier won't be as fast as we wished
Question:
- If we were to go live now, should we go with 1.4.9.2 or
1.4.10 and disregard the soft hangup since it is related
to FW update ?...
- Could this prevent future FW upgrades on the card ?
Regards,
--
exvito
_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
|
--
Matthew Fredrickson
Software/Firmware Engineer
Digium, Inc. |
|
| Back to top |
|
|
creslin at digium.com
Guest
|
| Posted: Fri Apr 18, 2008 10:15 am Post subject: [asterisk-users] zaptel 1.4.10 regression with TE220B on Pro |
|
|
Ex Vito wrote:
| Quote: | On Wed, Apr 16, 2008 at 7:18 PM, Matthew Fredrickson <creslin at digium.com> wrote:
| Quote: | Ex Vito wrote:
| Quote: | Tested with no 4K stack kernel and stackcleanup svn branch
zaptel version. Correct, the kernel no longer "complains" about
the soft hangup.
However the system still hangs (console inoperative, etc) while
ztcfg'ing...
|
That is normal while the firmware is loading. It should go away after the
firmware has loaded.
|
Ok. So here is our reasoning according to collected info. Please
correct us where appropriate:
1. The system is supposed to hang while the firmware loads into
the DSPs under any zaptel version
2. zaptel 1.4.10 leads to a soft hangup detected, zaptel 1.4.9.2
does not (assuming softhangup detection active in kernel)
3. zaptel 1.4.10 takes much longer ztcfg'ing than 1.4.9.2, that's
why the soft hangup is detected under zaptel 1.4.10
(difficult to time, but let's say 1.4.10 takes 10s, 1.4.9.2
takes 3s)
Now, back to the original question:
- Should this be considered a regression ?
- Next steps:
a) file a bug and move this analysis to the bug tracker
b) don't file bug and move analysis to the dev list
c) don't file bug, keep on working on the users list
| Quote: | I recommend 1.4.10 by default. However, from what you said it would appear
that you are having problems with 1.4.10 so you might stay with 1.4.10 if
you are not having any issues with it.
|
|
I just realized where this is coming from. I was attempting to patch
this from a different angle, but as soon as you mentioned the drastic
difference in load time I realized what had happened. I'm going to make
another update to my stack reduction branch to see if I can fix this.
I'll let you know when it's done.
--
Matthew Fredrickson
Software/Firmware Engineer
Digium, Inc. |
|
| Back to top |
|
|
creslin at digium.com
Guest
|
| Posted: Fri Apr 18, 2008 2:20 pm Post subject: [asterisk-users] zaptel 1.4.10 regression with TE220B on Pro |
|
|
Ex Vito wrote:
| Quote: | On Fri, Apr 18, 2008 at 4:15 PM, Matthew Fredrickson <creslin at digium.com> wrote:
| Quote: |
I just realized where this is coming from. I was attempting to patch
this from a different angle, but as soon as you mentioned the drastic
difference in load time I realized what had happened. I'm going to make
another update to my stack reduction branch to see if I can fix this.
I'll let you know when it's done.
|
Great. We'll be right here... Since the bug has been closed, we post the
timing results we did within this context.
|
I just updated the branch. Wait about 5-10 minutes in case for the
changes to get mirrored, and then try updating and doing it again.
--
Matthew Fredrickson
Software/Firmware Engineer
Digium, Inc. |
|
| Back to top |
|
|
creslin at digium.com
Guest
|
| Posted: Fri Apr 18, 2008 4:12 pm Post subject: [asterisk-users] zaptel 1.4.10 regression with TE220B on Pro |
|
|
Ex Vito wrote:
| Quote: | On Fri, Apr 18, 2008 at 9:36 PM, Ex Vito <ex.vitorino at gmail.com> wrote:
| Quote: | On Fri, Apr 18, 2008 at 8:20 PM, Matthew Fredrickson <creslin at digium.com> wrote:
| Quote: |
I just updated the branch. Wait about 5-10 minutes in case for the
changes to get mirrored, and then try updating and doing it again.
|
Looks better, no more soft lockup and ztcfg time is comparable to
1.4.9.2's:
|
Matthew,
...is there any specific test you'd like us to perform on this revision ?
(considering that currently we have no PSTN line to attach to... we
can cross-connect the spans and generate traffic or, cross-connect
with another lab system)
|
Not really from me specifically. You already tested what I wanted to be
tested, and that was to see if I could fix the load time issue and
softlockup warning.
--
Matthew Fredrickson
Software/Firmware Engineer
Digium, Inc. |
|
| Back to top |
|
|
creslin at digium.com
Guest
|
| Posted: Mon Apr 21, 2008 10:38 am Post subject: [asterisk-users] zaptel 1.4.10 regression with TE220B on Pro |
|
|
Ex Vito wrote:
| Quote: | On Fri, Apr 18, 2008 at 10:12 PM, Matthew Fredrickson
<creslin at digium.com> wrote:
| Quote: | Ex Vito wrote:
| Quote: |
Matthew,
...is there any specific test you'd like us to perform on this revision ?
(considering that currently we have no PSTN line to attach to... we
can cross-connect the spans and generate traffic or, cross-connect
with another lab system)
|
Not really from me specifically. You already tested what I wanted to be
tested, and that was to see if I could fix the load time issue and
softlockup warning.
|
Ok. So, since the bug we logged was closed and these tests weren't
registered along with it, when can one expect to have your new code
available in a zaptel release ?
In the next one or maybe later because the branch you're working on
has lots of different things to merge ?
|
It should be in the next release.
--
Matthew Fredrickson
Software/Firmware Engineer
Digium, Inc. |
|
| Back to top |
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
Search
